AIOs and XR Streaming — What’s The Next Inflection Point in XR

In the past couple years, All-In-One (AIO) headsets have created an inflection point leading to significant growth of the VR user base. Some estimates have VR HMD annual sales doubling to nearly 20 million by 2023. 

It appears the ease of simply putting on the HMD and finding yourself immersed in VR right away —  without a PC or lighthouses — is one of the missing pieces of the puzzle of a vibrant VR market. This is huge news for consumers where VR games can be pared down both in memory needs and graphics quality to fit on mobile SoCs.

Yet, the number one HMD used in PCVR (SteamVR) in February 2022 was the Quest 2! In fact, almost 51 percent of PCVR users ran their experiences on an AIO HMD. This means users of AIOs are still immersing themselves in VR experiences that require a full-fledged gaming rig or graphics workstation.

This is especially true for users in the enterprise space, where the need for high-fidelity datasets and photorealistic graphics are mandatory for design, design review, and even realistic training applications. 

Coupling AIO ease of use and a PC or higher level of computing power is where we at NVIDIA see the next XR inflection point.  With this future in our sights we have developed our “Four Pillars of XR” strategy: 

1. Photorealism
2. Collaboration
3. Artificial Intelligence
4. Streaming

These four pillars will be important over the next few years as we enter a new era of XR streaming.  (The Ghost Howls site has a more in-depth interview on NVIDIA's Pillars)

Photorealism is a pretty straightforward idea and the heart and soul of NVIDIA graphics from day one.  Whether for design and design review or for optimal knowledge transfer from highly realistic training experiences, photoreal and fidelity are incredibly important in XR. 

Collaboration is an XR superpower and a huge value-add of XR systems; we found that out with our technology demo of the Holodeck software so naturally it’s a pillar of NVIDIA’s XR strategy

The next generation of XR will benefit from significant additions of Artificial Intelligence from the obvious of replacing data-occluding GUIs with conversational AIs for C2 of applications to AI agents looking at our real environment via sensors in AR glasses and providing us with analytic context - AIs will change the experience in XR.

How do we get to the compute for Photorealism, eventually ray-tracing and AI agents, not to mention sharing scenes and meta data for collaboration - via the fourth Pillar - Streaming.  And, that is why I will spend the rest of this post talking about XR streaming and pointing toward a few excellent #NVIDIAGTC streaming XR sessions!

 The Streaming XR Journey

Our journey of streaming XR maps alongside others in the quest, including groups like Meta, Vive, Arvizio, QuarkXR, and Holo-Light. At first, remote XR seemed relatively impossible with the need for a motion-to-photon latency of <20ms. But that was two years ago, and as a community we’ve moved past the question of can we stream XR. Now, the questions are:  Where do we stream from? And, more specifically, what network do we stream XR on, and How do we scale?

Over the last two years, NVIDIA and it’s partners have shown XR streaming from the cloud, data center, and edge over high-end WiFi and 5G networks. These technologies are showing that advanced XR streaming is not only feasible, but can create scalable, multi-user experiences.

Essentially, it means that anyone with an HMD, phone, or upcoming AR/MR device can be an XR user with access to the highest fidelity data and the highest quality graphics possible.  The convergence of AIO ease of use and mobile devices with PCVR-level compute power on the network will power the next XR inflection point.

At NVIDIA GTC, which starts on March 21, we’re showcasing what’s possible with the latest advanced technologies in XR streaming. Though we are going virtual again, and I’ll miss seeing everyone in person, it’s still going to be an exciting event — we have great content on tap that shows fantastic examples of streaming XR for enterprise use cases. 

These sessions really show what can be achieved with wireless VR and collaboration: 

• The Journey to Collaborative Virtual Workspaces: PopUp-XR at Volvo Group [S41539] - Steven Horton, Group Manager Vehicle Strategy, Volvo Group Trucks Technology and Jan Wurster, Solution and Technology Expert, ESI Group
• Creating "Phygital" Collaborative XR Spaces to Optimize the Design of 3D models [S41235] - Alexis Vartanian, CTO, TechViz

The next couple sessions highlight what streaming XR means in the AEC industry. This is where I think some of the largest impacts of cloud XR streaming are waiting for architects, their partners and customers: 

• Next-Gen CloudXR: Bring your engineering and construction projects to life [S42027] - William Cannady, Technical Product Manager, NVIDIA and Greg Demchak, Director of Digital Innovation Lab, Bentley Systems
• Removing Barriers to VR in Architecture Using Streaming VR [S42260] - Dan Stine, Director of Design Technology, Lake Flato Architects and Petr Mitev, Vice President, Visualization Product Group, Enscape

Learn more about the importance of 5G and XR streaming. The Green Planet experience is one of my favorite examples of new types of content hitting mobile devices — really exciting:

• Bring the Green Planet into Your Hands with 5G, AR, and Edge Compute [S42074] - Stephen Stewart, CTO, Factory 42
• Build Up a 5G Cloud Platform to Provide An Open Mixed-reality Space - Tzu-Hsien Yang, Project Leader Engineer, Compal

And, finally, there are a number of sessions where the XR Ecosystem is building tools for scale and utility from VMware’s Workspace One HubXR to NVIDIA’s own Project Aurora:

• How to Deliver High Fidelity Virtual Worlds [S42029] - Matt Coppinger, Director, Product Management, VMware
• XR Hand Tracking: Streaming Your Hands Across the Network [S42072]  - Matt Tullis, Director of Spatial Computing, Ultraleap
• High-resolution Neuroimaging Visualization with 3D Slicer and Cloud XR (Presented by Amazon Web Services) [S42527]  - Steve Pieper, 3D Slicer Chief Architect, CEO, Isomics, Inc.
• Server Platform for Deploying XR Applications in Private and Public Networks [S42075] - Rene Peters, Product Manager - Augmented & Virtual Reality, NVIDIA

There is more in XR to explore at GTC, check out the XR session catalog and make sure and listen in on NVIDIA CEO Jensen Huang’s keynote!

Registration is free — come join us and hear from leading companies and partners across industries to see how they're enhancing XR experiences with NVIDIA technology.