Navigating the Virtual Realm: Unveiling the Crucial Link Between Network Conditions and XR User Experience

5 mins read

The adoption of XR technologies is continually expanding as businesses recognize the potential for innovation, improved efficiency, and enhanced user experiences.  Whether it is for military training simulations, interactive gaming or virtual field trips, the common enabler is the XR technology and networks required to deliver different levels of immersion, or blending, of the real world and virtual elements. 

AR – Augmented reality is a view of the real world with computer generated content overlayed into the physical space.

VR – Virtual Reality creates new computer-generated worlds that fully immerse a user.

MR – Mixed Reality. Is a lesser used term but is commonly used to describe physical objects in the real world enhanced by computer generated content and data.

 

Poor user experience – the barrier to success

Despite the door to a virtual realm, and its potential being truly open, user adoption of the technology is low and one reason for this is the quality of user experience.  Headsets can be invasive, socially awkward and in some cases induce motion sickness all leading to a poor experience and deterring users.

The simple way to address this for hardware manufacturers may be to make the headsets smaller and less intrusive but in doing so this also reduces the processing power of the device. This consequently then requires the processing to be offloaded as much as possible to the network edge in order to still be as close to the user as possible.

However as soon as networks become part of the solution they also become part of the problem.

 

 

Network Got you Down? The challenge of motion to photon latency.

Real world networks are full of imperfections and the XR headsets ability to tolerate, perform and recover from these imperfections, in the real world, is not yet fully understood.

Take cloud gaming as an example.  Latency and jitter on a network can result in a momentary blip in video, a buffer window or laggy performance for online and cloud gamers, which in the worst case scenario is frustrating.  When XR headsets become part of the game experience however the impact becomes a lot worse and possibly long-lasting.

As the cloud deals with potentially more of the processing, the impact of latency and jitter is experienced through the rendering of the virtual world on the XR headsets.  This is described as Motion to Photon latency and represents the time it takes for user input, such as turning your head, to be rendered on the users XR device.  If this exceeds 20ms, which can include any network latency and processing time of the action either in the cloud or local on the headset, this can induce motion sickness for the user.  In this circumstance for any users who may require prolonged use of the headset, this can actually lead to long-term health effects. 

 

 

Getting the offload balance right

The solution to this is to get an understanding of the optimal balance offload for any particular user requirement.

Today’s headsets are mainly low-mid offload with local high offload headsets allowing users to stream a game from their home PC.

 


– Low offload headsets reduce the network requirements of the device, performing similar to a mobile phone. User inputs are rendered on screen via local processing and web-based updates are received. Low offload devices currently consume the most power, have highest hardware cost and are the bulkiest for the user to wear.

-Mid offload devices offload some processing to the network edge. This reduces the size and cost of the headset but increases the effects that the adverse network conditions have. For users to maintain immersion and avoid motion sickness, the device must be able to handle a wide variety of network conditions.

-High offload devices off load nearly all processing to the network edge. These devices rely solely on the cloud to display content to users. High offload devices will be the smallest of headsets, look similar to a normal pair of glasses and have some of the greatest potential to bring XR to the masses.  But these devices are highly sensitive to turbulent network conditions and the most likely to cause the user to feel the effects of motion sickness.

With every network connection and individual user scenario going to be different, being able to gain insight into the ideal performance characteristics for that is critical to ensure that the right off-load set-up is adopted. 

 

 

Offload balance – who is responsible, the provider or the user?

In short, everyone.  If the success of the XR world is partly dependent on user experience, a collaborative approach is then required to deliver a good one.  Adverse network conditions will always occur, especially for immersive technology such as XR and devices need to be able to always render and display content to retain user immersion. The only way to confidently deliver this is to test it out and gain confidence in performance before you deploy.

For those manufacturing hardware, understanding the scope and variety of user needs the associated offload balance required to meet these needs, will help inform design considerations and enable effective decision making that delivers commercial success.

For the network providers, understanding the optimal design which can place processing as close to the end user as possible whilst integrating and managing edge resources efficiently, will prepare them to deliver solutions fit to meet fluctuating end user demands at scale. 

And for those relying on both the infrastructure and network to effectively deliver their application and content, confidence that performance expectations will be met, will reduce the risk of common network issues shattering that desirable exceptional user experience.