Devices and Form FactorsTS 26.928

Extended reality devices are of different form factors as shown below. These form factors may differ in processing capabilities, communication types and power consumption.

The majority of Virtual Reality (VR) and Augmented Reality (AR) devices are head-mounted displays (HMDs). In AR, the display is usually transparent and digital information is superimposed onto real life objects. In VR, the display is not transparent and only virtual information and images are displayed in front of wearer's eyes. Head-mounted display ( HMD) is a device worn over the head. It features a display in front of one or both the eyes. The display streams data, images and other information in front of the wearer's eye(s). Certain HMDs have displays over both of their users' eyes, others only have a display over one of the users' eyes.

Typical components of HMDs are listed as follows:

• Optical systems: Display and lenses
• Tracking sensors and possibly additional sensors
• Cameras
• XR related processing (summarized as XR engine) including GPUs, CPUs, ASICs (e.g. dedicated media encoding and decoding), etc.
• Communication functionalities, as for example provided by a 5G System

smartphone (defined as XR5G-P1 device type) may be used both for AR as well as for VR (together with a card-board). In both cases, typically an XR engine/runtime is available to support processing of sensor data, viewport rendering as well as SLAM processing. In this case, the 5G modem and all media/XR processing is integrated in the device. Power consumption of such devices is important, but not ultimately critical due to the onboard battery.

VR HMDs typically provide the a large field of view, stereoscopic 3D imagery as well as rotational, translational and positional tracking for full 6DoF experiences. For VR, the following device types are identified:

• XR5G-V1 - Simple VR Display wired: Such device types are commonly available in 2019. They only include sensors for tracking as well as a display. The remaining processing is done on a remote device, e.g. a puck or a smartphone. XR/Media Processing, connectivity and power supply are provided through wired tethering.
• XR5G-V2 - Simple VR Display wireless: Such device types are not yet available in 2019. They include sensors for tracking, a display, a wireless connection (which could be WiFi based or 5G Sidelink based), and a power supply. The remaining processing is done on a remote device, e.g. a puck or a smartphone.
• XR5G-V3 - Smart VR Viewer wireless tethering: Such device types are not yet available in 2019. They include sensors for tracking, a display, a wireless connection (which could be WiFi based or 5G Sidelink based), at least some XR processing, as well as a power supply. The remaining processing is done on a remote device, e.g. a puck or a smartphone.
• XR5G-V4 - VR HMD standalone: Such device types are commonly available in 2019, except 5G connectivity. For such devices, the 5G modem, power supply as well as all media/XR processing is expected to be integrated into a single device.

An optical head-mounted display is a type of head-mounted display that projects images and allows the user to see through its display and is used in augmented reality (AR). Unlike Virtual Reality HMDs that obscures the vision of the real world, AR devices allow to see the surroundings while streaming data and image overlays in front of the eyes. Optical head-mounted displays may cover only 1 eye or both eyes. Wearers can interact with the projected digital content through input methods such as voice commands, gestures and controllers.

For AR Glasses, design constraints are significantly more important. In particular, design constraints apply in terms of sleekness, weight and power. The processing power is expected to be low to avoid battery consumption and thermal dissipation. Wireless AR glasses are commercially compelling. AR is typically associated to using glasses, also referred to as optical see-through. However, AR experiences may be achieved using HMDs with video-see through functionalities. The following device types are identified:

• XR5G-A1 - Simple AR Wearable Glass wired: Such device types are available in 2019. They include a minimum number of sensors, possibly cameras for AR localization, as well as a display. Power, XR processing and connectivity is supplied from an external source.
• XR5G-A2 - Simple AR Wearable Glass wireless: Such device types are not available in 2019 and are still far out. They would include a minimum number of sensors, possibly cameras for AR localization, power supply, as well as a wireless modem for connectivity. XR processing, AR localization as well as network connectivity is provided by external means, e.g. a puck or a smartphone.
• XR5G-A3 - Smart AR HMD video see-through: Such device types are an initial entry for AR applications. There are much closer to XR5G-V1 type of devices, but by having cameras projecting the real world on the screen, the VR device can operate as an AR device.
• XR5G-A4 - AR Wearable Glass standalone: Such device types are not available in 2019 but are under consideration. For such devices, the 5G modem, power supply, as well as all media/XR processing is expected to be integrated in a single device.
• XR5G-A5 - Smart AR Wearable Glass wireless: Such device types are not available in 2019 and are still far out. In addition to an XR5G-A2 device, such a device would include at least a certain amount of XR/Media processing capabilities such as encoders/decoders and XRprocessing.

One of the most important issues when considering form factors and processing is the ability of the device to dissipate power, especially when no external cooling is available. Figure below shows the temperature rise depending on the surface power density. As example, two points on the figure can be considered:

• At 5C rise over ambient, power density that can be dissipated is 0.023 W/square inch. A smart phone whould have a surface area from 20 to 30 square inch, i.e. the power that can be dissipated is 0.5 to 0.75 Watt.
• AT 25C rise over ambient, power density that can be dissipated is 0.18 W/square inch. For a smartphone this would allow around 4 to 5 W continuous power dissipation. However, for an AR glass, the surface area is much smaller and so much less power can be dissipated, somewhere in the range of 1W. As an example, in a 25C room, the device enclosure surface temp would be 50C, which is already on the higher end of what is generally considered acceptable.

A summary of the different device types is provided below along with tethering examples, placement of 5G Uu modem, XR engine and localization support, power supply and typical maximum available power. In all device types, the sensors are on the device. The table also addresses the options applicable for tethering between the device carrying the 5G Uu Modem, and the XR device, if applicable. The table also addresses options for the XR engine that includes scene recognition and viewport rendering. The following definitions for the XR engine are used:

• External: the device only supports display and receives a fully rendered viewport data that can be displayed directly. Any scene recognition, if applicable, is not on the device.
• Split: the external device does a pre-rendering of the viewport based on sensor information and the device does the final rendering considering the latest sensor information. Different degrees of split exist, as discussed before. Similalrly, scene recognition can be subject to split computation.
• XR device: that device does the full rendering of the viewport in the device, sensor information is only processed locally. Any scene recognition, if applicable, is on the device.

Power Consumption

When designing media processing, XR functionality and 5G connectivity, it is important to understand the power consumption of different components that are possibly integrated in XR devices. The following should be considered:

• Tracking and Sensing
  • 3DoF tracking may be done with low power consumption, e.g., below 1 Watt
  • 6DoF tracking involving for example, capturing cameras, LEDs for eye and hand tracking, etc. are more power-consumption intense
• Display
• Display power consumption is critical and depends on the device.
• Display power consumption can be in the range of 0.3W up to 1W
• Render (GPU)
  • The power consumption of the GPU depends on frame rates, resolution, display technology
  • The power consumption can be from several mWatt to several Watt depending on the use case
• Compute and Media Processing (CPU)
  • Similar observation as for the GPU
  • If encoding is involved, power consumption is typically higher.
• Connectivity
  • The power consumption of wireless contection such as 5G depends on several factors including bitrates, distance from radio access network, channel conditions, frequeny range, etc.
  • The power consumption can be from several mWatt to several Watt depending on the use case.

It is expected that each of the components will undergo improvements to address power savings. It is important that in the development of technical specifications of XR devices, the power consumption of each component is considered.