ShotView transforms the golf fans' experience through Digital Twin technology
As part of a range of articles that explore the breadth of NTT DATA's activities, the head of IOWN Innovation at NTT DATA sat down with Laurence Norman, the chief technologist for entertainment and sports. In a fascinating conversation, they discussed the real-life use case of digital twin for sports technology and how it has changed the game for viewers the world's biggest golf championship. Through the utilization of this new class of technology, fans can now virtually follow along with their favorite players and how they are playing across a variety of sports.
Moderator Profile: Eiji Yoshida is the head of the IOWN Innovation Office, NTT DATA. The IOWN (Innovative Optical & Wireless Network) is an initiative for a wide range of R&D projects, from digital twin computing to data utilization and using connectivity for greater advantages.
Speaker Profile: Laurence Norman is the chief technologist for entertainment and sports at NTT DATA. He was involved in the activation of the digital twin technologies at the 150th Open Championship (The Open). He seeks to leverage innovative technologies to elevate the sports experience for viewers, whether they are watching in person or across the world.
Through the application of the digital twin at The Open, Eiji and Laurence discuss the future use of the digital twin from an entertainment and sports technology standpoint.
Introducing ShotView for the 150th British Open Golf Championship in 2022
NTT DATA has been official patron of The Open since the 142nd event in 2013. For the 150th Open, NTT DATA had taken on the challenge to deliver an exceptionally accurate and engaging experience for the spectators.
The 2023 championship was held at the Old Course in Saint Andrews, the home of golf. To provide a second-to-none viewing experience at this historic site for golfers, NTT DATA created ShotView, an exact virtual copy of the Old Course, created through one of the many applications of the digital twin. With ShotView, ball positioning data of all 156 players throughout the 4-day competition was plotted on the virtual model in real time, along with the latest information and the statistics for each hole.
Laurence discussed with Eiji on how there were three main aspects at play to provide an experience to engage more fans through ShotView.
Precise Mapping
A precise digital model of the Old Course was needed to create the basis for the experience. Without this, the ball and player information would feel disconnected. For the fans to have a never before felt experience, the course was recreated with a 2-centimeter accuracy, which was made possible through drones and LIDAR mapping. These were combined with the cooperation of the major course events team, which were already conducting mapping for staging purposes.
Creating a digital twin is labor and cost-intensive; it takes a considerable operation involving scanning the entire area, which is both technically and physically demanding. The sheer size of a golf course amplifies both this difficulty and cost. Still, by taking advantage of the data that was already available and finetuning it to a higher level of precision, NTT DATA was able to make ShotView a reality. As The Open cycles through 9 different venues, future challenges involve recreating digital twins for these courses more efficiently and increasing the level of stable, high speed and scalable connectivity infrastructure to support these new levels of fan experience.
Accurate Data for Players and Stats
The next key aspect was collecting accurate data of where the individual players are and where the ball has landed, in real-time. These data allow the audience to follow along with the tournament closer and faster than ever before. More than 240 volunteers were positioned across the fairways and greens of all 18 holes, and using a combination of augmented GPS receivers, laser rangefinders and machine vision cameras - were able to send ball position data to a 2cm accuracy, within 20 seconds of the ball being hit by the player. This was done for all 156 players, for 10 hours a day over the 4 days of the championship. A total of 32,225 shots were recorded to this level of precision and latency.
Improving Fan Engagement
The third and final aspect of the solution, is the end use case. For The Open, this was firmly positioned as a fan engagement and entertainment challenge. Large scale golf competitions are challenging to understand – with, at some point, 3-4 players on each of the 18 holes playing simultaneously across more than 7305 yards on The Old Course; it's impossible to keep up to date on everything that's happening. Until now.
After gathering the player and ball information, it was applied for both in-person and online use. At the Spectator Village in the venue itself, the information was used for the NTT DATA wall, a 20-meter screen that provided an in-person shared digital experience for the 300,000 fans who were lucky enough to attend in person. Using prioritized algorithms and real-time rendered graphics to ensure everyone was kept up to date with the most important events happening across the course.
. For a personalized digital experience, viewers could go online to select specific players, track their performance, and compare it with either the player's past performance or even with other players' performances. ShotView allowed golf fans to track players of interest in real-time, select and dive into the statistics, rather than rely on the general broadcast.
Results of utilizing Digital Twin technology
Through the innovative use of this Digital Twin technology, fans can now experience the oldest golf tournament in a brand-new way. Laurence stated that this resulted in the Data Wall being the centerpiece of the event for on-site visitors, with roughly 150,000 to 200,000 people actively engaging. Online golf fans also engaged three times longer with the ShotView webpage in 2022, compared to the engagement in 2021 when the webpage did not have ShotView. Laurence found an incredibly positive response towards ShotView, as survey results indicated that 89% of users were more immersed in the tournament thanks to ShotView. The application of digital twin in sports technology to create an interactive experience through ShotView drove increasing engagement levels on the webpage, with users spending an impressive 37 minutes on average using it.
Laurence was also surprised about the amount of positive feedback he had received from sports influencers on site. Through ShotView on the webpage, they were able to access data that had previously been inaccessible to them while out on the course. Commentators and journalists were able to tell a more informative story in real time about the player's strategies to liven up the tournament for golf fans.
Other use cases Laurence saw on-site were fans using the application to navigate to their favorite players. However, Laurence stated it had become increasingly clear that providing strong mobile connectivity throughout 7305 yards of the golf course was a challenge. Bringing in more than 300,000 fans, transport, hospitality, broadcast and other aspects puts a Significant strain on WiFi and mobile connectivity - and technology like ShotView demands high performance to deliver the perfect experience.
Future potential Digital Twin Technology for the Open
Both Laurence and Eiji agreed that 5G seemed to be the solution to improve connectivity, with private 5G and 6G research and development being a concept that IOWN has been working on.
When inquired by Eiji about further improvements that could be made, Laurence indicated his interest in transforming the physical environment based on the digital twin and modeling the ball's actual trajectory.
Laurence's unique view of using the digital twin as a feedback loop to influence the physical world can be beneficial in golf championships as it can be used to redesign courses to be more challenging for the professionals while improving sustainability and limiting environmental impact With the single largest contribution of Co2e for an event like The Open related to fan journeys to and from the venue, digital twin is uniquely positioned to understand, optimize and reduce this.
Digital Twin Computing (DTC) initiatives at IOWN can create digital twins for advanced simulations and apply the changes in cyberspace, to make sure that they can deliver the desired results in the actual physical 3D space.
In terms of modeling the ball's flight trajectory, Laurence sees the opportunity to make full use of the digital twin by tracking both actual and predicted trajectories. However, strict regulations around golf equipment prevent them from embedding sensors to acquire actual flight trajectory data, while the use of passive sensors including launch radar and advanced machine vision techniques are increasingly being used to provide these additional data.
Exploring the Opportunities with the Digital Twin
The use of digital twin technology extends beyond golf in sports technology. Eiji foresees the possibility of sports technology influencing the rules and equipment for sports. As an example, the World Cup uses balls with sensors that enable them to make more accurate real-time calls to officiate the sport. There are still some improvements to be made, as accurate decisions come at the cost of a slight offside delay with sensor and tracking camera data taking time to process.
Laurence has seen a trend in sensors as they have become smaller, tougher, and more reliable, which has spurred the evolution of sports. Laurence elaborated on the use of data to make use of the predictive technologies supported by the digital twin at the Indianapolis 500, the longest continuously running motor sport event in the world. In motor racing, cars already have many sensors embedded for many different purposes. The NTT Team had repurposed the collected data to input into the digital twin to engage fans better by predicting race strategies.
In terms of golf, Laurence observed that precise data collection is starting to take off due to fan engagement and the data demands for gaming organizations. Data acquisition and analysis are costly, and in some golf tournaments, the data feed is sponsored by gambling organizations. However, these data are true primary resources which can be used for many and varied purposes.
Eiji expanded on the opportunities available to monetize the data by repurposing it for applications beyond the sporting viewership. DTC, in the IOWN initiative, also explores ways to make use of the data such as building out the course in the metaverse or further monetizing by selling data rich NFTs.
Data around sports have become a new focus of rights management, opening doors to new business models of selling data. Once the data is created, it is there forever and can be used to build out new models of data utilization whether it be for officiating the sport, virtual games or other not imagined yet opportunities.
Laurence foresees the use of the digital twin to improve the skills of the players as well. Through the digital twin, players will be able to incorporate events that can occur in the 3D environment in cyberspace, modify strategies and repeat the simulation and derive actions that could lead them to victory.
IOWN's Digital Twin Computing Initiative
Digital Twin Computing is one of the main key areas in NTT's IOWN initiative, which is an infrastructure that will be supporting smart societies beyond what the current internet is capable of handling. Within this initiative, "Digital Twin Computing (DTC)" multiplies digital twins to perform various simulations and to make predictions.
Eiji stated that the use of sports data and the digital twin at The Open is a flagship initiative to provide a real-life use case to support the value that the digital twin will be able to create. He hopes to explore whether even more value can be created by incorporating IOWN's DTC.
He stated that even if the data may not have much impact as a single unit, by using the DTC to multiply the data through various simulations made on the digital twin, they were able to create an advanced experience for spectators as they fused the outcomes together.
Furthermore, the IOWN initiative explores ways in which data that had been collected can be reused for other purposes. For example, the Urban Development DTC combined data that had been collected for Internet of Things initiatives, weather information, and human flow statistics from city blocks to support the creation of services relating to food loss, green air conditioning management, and health behavior.
The NTT Group currently sponsors NTT INDYCAR SERIES, Tour de France and The Open Championship which are all involved in the collection of data and improving connectivity to enhance the spectators' experiences at the events. The common challenge across the three events is fan uptake. Looking ahead to overcome this challenge, Laurence seeks to deepen the collaboration with IOWN to make use of the data to further expand on the opportunities in improving the spectator experience through digital twin technology.