Viewers can control their viewpoints while experiencing live music and sports!
Canon U.S.A., Inc., a leader in digital imaging solutions, announced that its parent company, Canon Inc., has successfully tested live-streamed volumetric video, including data capture, 3D data generation, and distribution to viewers in real-time*. Live streaming of 3D data allows viewers to enjoy live XR content (VR/AR/MR) from any location they choose, in real-time, and on their preferred compatible device, including tablets, phones, and headsets.
Volumetric video technology captures an entire space using numerous cameras and renders a virtual 3D space from 2D data, making it possible to produce images that can be viewed from virtually any angle.
In recent years, there has been increasing demand for 3D content for viewing on platforms such as XR headsets. Accordingly, there has been growing interest in using this exciting new technology in many markets as a means of offering an entirely new viewing experience.
Canon’s volumetric video system is currently being utilized in a wide range of fields, mainly for the delivery of 2D images with perspectives chosen by the system operator, such as the production of replays for sports broadcasts and the live streaming of music captured at Canon’s Volumetric Video Studio Kawasaki in Japan. While these projects have been well received and regarded as high-impact experiences that complement traditional broadcast productions, to further enhance the value of volumetric video technology, the distribution of 3D content that allows viewers to select and control their viewpoints in real-time on their own devices will be possible.
Now, Canon successfully tested a live streaming system for XR content that processes volumetric data at high speed. Storage is highly scalable according to the number of viewers, and Canon utilizes cloud services that enable efficient data transmission that optimizes the linkage between the systems in each process, such as capture, 3D data generation, and XR content generation. This makes it possible to create a real-time data capture to delivery workflow that leverages volumetric video content to provide rich, high-quality XR content that viewers can enjoy from their own compatible device, and from their own individual viewing perspective!
To deliver the content live, 3D point cloud data is generated in real-time from video captured by Canon’s volumetric video system and transmitted to a cloud server. The XR content is then generated on the cloud server and delivered to the viewer’s compatible device, such as a smartphone or head-mounted XR display. In doing so, the system continues to generate and distribute images for each individual viewer in the cloud server in response to multiple viewer viewpoint operations, allowing users to experience their own personalized, immersive viewing experience. For example, in the case of live music in a metaverse, viewers can watch the show anywhere they want while moving around freely. And in the case of watching sports, users can experience the decisive moment in real-time from the same perspective as the actual players on the field, making it possible to provide new viewing perspectives like never before.
Canon is strongly committed to the development and expansion of volumetric video technology and is actively exploring strategic partnerships with other companies such as telecom, networking, and cloud service providers, with the goal of providing comprehensive and flexible end-to-end workflow solutions. We will continue to test and optimize this service with the aim of developing highly interactive viewing services for live music, sports, and other productions.
Live streaming of XR content using volumetric data (Canon Official) - YouTube
Canon Press Release Source: Canon USA