How the transformation of Wireless Edge will help in the development of photorealistic mobile augmented reality systems.
(Extended Reality, XR) is already giving users a revolutionary experience, but to achieve even greater realism and a new level of immersion, given the limitations associated with the performance and cooling of thin portable gadgets, is not a trivial task.
Looking to the Future: Slim and Stylish Augmented Reality Glasses
With the transformation of Wireless Edge systems (wireless systems operating at the junction of a network and a gadget), a new era of distributed computing will begin, in which 5G technologies, information processing on the devices themselves and edge cloud computing will be actively used. And it is this transformation that should help find an optimized solution.
The Best of Both Worlds
What if we could take all the advantages of mobile XR devices and combine them with the performance of PC-based XR systems? Augmented reality mobile gadgets are the future of XR because they can be used anywhere, anytime, without any preparation and without wires. The PC-based implementation of XR, while not the future of augmented reality, has the important advantage of being unlimited in terms of power consumption and cooling efficiency, which in turn allows for more extensive computing. With 5G networks with lower latency and higher bandwidth, we plan to take the best of both worlds. Sharing computing tasks with 5G technologies will offer the best of both worlds – a limitless mobile XR experience and photorealistic graphics in thin and affordable XR headsets. As a result, users will have “limitless” possibilities in every sense, because they will be able to connect to augmented reality wherever they want, and the degree of immersion in XR applications will become even higher.
Infinite augmented reality technologies offer the best of mobile XR and PC connected devices
Improving the efficiency of augmented reality data processing on the device
Working with graphics in augmented reality systems requires large computing power and is sensitive to response time. A systematic approach is required for the correct separation of calculations. Let's take a look at how using edge cloud computing can help complement on-device processing more effectively, creating borderless augmented reality systems with photorealistic graphics (for more information, see our ).
When an XR system user turns their head, on-device processing detects head position and sends that data to the edge computing cloud over a 5G link with minimal latency and high quality of service. This system uses the received head position data to partially render the next frame of the image, encode the data and send it back to the XR headset. The headset then decodes the last received packet and, using regularly updated data on changes in head position, continues to render and adjust the image to minimize motion-to-photon latency (the delay between changing the position of the user's head and changing the image in the headset). Recall that in accordance with this indicator, all processing must be completed in a time not exceeding 20 milliseconds. Exceeding this threshold leads to unpleasant sensations for the user and a decrease in the level of immersion in augmented reality.
On-device computing is augmented with edge cloud computing and a low latency 5G link.
As you can see, XR needs a system solution with low response time and high reliability to achieve a high-quality immersive effect, so 5G networks with their low latency, high bandwidth are a critical element of XR. As 5G networks improve and coverage grows, users will be able to enjoy photo-realistic graphics in XR apps in more locations and will be confident that the premium offline XR experience will continue to be available through efficient on-device computing.
And this is a key point worth emphasizing again: the processing of data on the device remains a critical factor in any scenario. In standalone mode, the "on-board" computing on the device is responsible for generally all calculations related to the XR. When paired with an edge cloud computing system, on-board processing will provide the XR headset with low-latency, low-latency, high-performance rendering and tracking capabilities.
Creating "limitless" augmented reality
Qualcomm Technologies is already building high performance autonomous mobile XR solutions and remains a leader in in the world. But we can't turn our ideas for the "limitless" XR into reality on our own. We are actively collaborating with the largest representatives of the XR and 5G ecosystems, including OEMs and content creators, service providers and infrastructure providers, because the split rendering architecture is a system solution.
Members of the XR and 5G Ecosystems Must Work Together to Make “Endless” XR Technologies a Reality
As a result of synergy, all participants in the XR ecosystem will receive a greater overall benefit from its active development, and the name of this benefit is “increased consumer acceptance”. For example, carriers will benefit from the transformation of Wireless Edge in general, but let's look at the benefits of the development of the borderless XR. First, with the advent of 5G networks, improved broadband access will increase capacity, reduce response time, and provide guaranteed class of service, enabling richer and more interactive XR applications. Second, as operators increase edge cloud computing capacity, they will be able to offer completely new services to the mass user, such as XR applications with photorealistic graphics.
We believe that a new revolutionary user experience will be a big advantage, including real-time interactive collaboration, multiplayer games with photorealistic graphics, a new generation of videos with support for six degrees of freedom, immersive educational applications and personalized shopping like never before. These perspectives are inspiring, so we look forward to fruitful collaboration with other participants in the ecosystem - this work will help turn our vision of XR into reality.
Source: habr.com