Virtual reality technologies
A comparison of the hardware, tracking systems and software behind virtual reality headsets, summarising the available literature.
Elliot Négrel-Jerzy, ENSEEIHT - Computer Science Dept.
Abstract
Virtual reality, a dream become reality, a software marvel combined with decades of experience in computer hardware, sensors, but also, most importantly a tool, now used in a large variety of domains. Although at first sight, the concept might seem easy to get a grasp of, the technology behind it is considerably more complex than one might think at first sight. The available literature is extremely detailed and comprehensive; therefore, this article’s aim is to compare and summarize in a clear and concise manner the many approaches chosen to address the subject. We’ll be exploring the existing hardware, as well as the most commonly used tracking systems and the software bringing this whole machinery to life.
VR display hardware solutions
The state-of-the-art virtual reality headsets available on the market have led major companies such as Meta or Google to seriously consider investing in this domain, despite the initial drawbacks that the new technology came with. Their interest is, to this day, one of the main reasons why these devices are being mass produced, and are as a result, substantially cheap.
After taking a look at the actual design of one of these headsets (Parkin, 2014), we can identify three “main” components, the first being the circuit board. It features a highly powerful graphical processing unit, due to the very high requirements in terms of display latency: extended looking at a slow and unresponsive video feed has proven to induce nausea and headaches.
Figure 1: Inside view of an Oculus Rift VR headset (Parkin, 2014)
This board is then connected to a high-definition display module, whose technology can vary depending on the budget (LCD or OLED solutions are often preferred). However, the most commonly encountered design uses a backlit liquid crystal display, i.e., a non-emissive, translucent layer of so-called liquid crystals (extensively small cells whose color can be dynamically changed by applying a specific voltage for each color), coupled with a monochrome light-emitting backlight unit (BLU), which ensured the visibility of the content displayed on the aforementioned layer, by shining light through it.
Figure 2: Schematic diagram of an LCD (Liquid crystal display and organic light-emitting diode display: present status and future perspectives, 2017)
Finally, two lenses are positioned in front of both eyes, ensuring that the perceived image is positioned “at infinity”, so as to reduce eye strain. This setting essentially means that the light coming from the objects as seen through the lens will appear to be coming from a very long distance, thus allowing the user’s eyes to effortlessly focus on them.
At this point, we do have a fairly accurate view of how an average entry-level VR headset’s works; therefore, we can dive into the existing hardware solutions used in movement and position tracking.
Tracking systems
Presently, virtual reality is being used in a plethora of domains, most of which require an interaction between the user and the virtual environment in which he is immersed, hence the necessity of tracking movement and position in space, sometimes even to the accuracy level of finger movement.
Multiple solutions do exist, and yet it would be a lie to say that these are in competition on the market, as each and every single one of them is more suitable for certain specific use cases. Open environments with great freedom of movement, such as virtual reality battle arenas (e.g., EVA) or military combat training infrastructure (Lele, 2011), will require positional tracking of limbs and weapons, which best works using optical tracking methods. The monitored zone cannot be infinite, and must be surrounded with cameras, coupled with highly powerful light projectors (preferably infrared, due to its imperceptibility to the human eye). The user should be equipped with judiciously placed retro-reflective markers, so as to minimize the positional estimation error; obviously, these markers are in general specifically designed to reflect extremely well the light frequency emitted by the aforementioned projectors. In the end, what the cameras see is an extremely dark image, covered with blindingly white dots, which correspond to the limbs of the user, and whose coordinates can later be determined, in order to programmatically calculate his position. Variations of this method may involve laser emitters instead of markers (Simulation of a virtual reality tracking system, 2011), whose beams will project dots of light on the walls of the monitored room. However, both techniques make use of cameras when it comes to locating them.
Figure 3: Visualization of an optical tracker's detection system (RetroTracker: Upgrading Existing Virtual Reality Tracking Systems, 2019)
High precision motion capture is usually sought after in VR apps which call for (included but not limited to) hand-gesture detection and close contact interactions with the environment. Again, several solutions are available to the public: optical detection in conjunction with AI algorithms, or gloves/suits equipped with structural deformation sensors.
Figure 4: Motion capture using an Optitrack marker suit and IMU-based gloves (Parger, et al., 2020)
Yet, these systems could never be this efficient if it wasn’t for the software that analyses the generated data.
Software appliances
The main challenge that programmers have to face when designing so-called “VR-ready” software, is the need for two points of view, or cameras, one for each eye. This is due to the fact that our perception of depth is highly dependent on our ability to merge two slightly different images, allowing us to see both sides of an object at the same time when it is close to us.
The drastically different interface, when comparing a headset to a common computer monitor also requires many modifications in terms of user interface, but also of sound quality and graphics. Rendering techniques such as caustic shading (Négrel-Jerzy, 2022) or ray tracing may be implemented with the aim to improve the realism of an environment in which the user is supposed to feel immersed.
Conclusion
The emergence of fully virtual worlds such as the metaverse by Meta (formerly Facebook) shows the increasing importance of VR in our lives, sometimes without the need for special gear: augmented reality is slowly making its way into our daily routines, mostly through some of our smartphone applications. This technology is constantly evolving, breakthroughs are being made every day, improving both its looks and its performance. And yet, this persistent seeking of innovation and progress has been proven to be our society’s main vulnerability. With climate change on our doorstep, should we really keep on following such an ethically unacceptable industrial model? It is to be hopped that we will eventually learn from our mistakes, and save ourselves from the consequences of our own choices.
Bibliography
- 2013. John Carmack Talks Virtual Reality Latency Mitigation Strategies. TOVR. [Online] February 24, 2013. [Cited: May 14, 2022.] https://www.roadtovr.com/john-carmack-talks-virtual-reality-latency-mitigation-strategies/.
- Lele, Ajey. 2011. Virtual reality and its military utility. 2011.
- Liquid crystal display and organic light-emitting diode display: present status and future perspectives. Chen, Hai-Wei, et al. 2017. 2017.
- Négrel-Jerzy, Elliot. 2022. Rendering techniques applied to Caustics. 2022.
- Parger, Mathias, et al. 2020. UNOC: Understanding Occlusion for Embodied Presence in Virtual Reality. 2020.
- Parkin, Simon. 2014. Oculus Rift. MIT technology review. [Online] April 23, 2014. [Cited: May 14, 2022.] https://www.technologyreview.com/technology/oculus-rift/.
- RetroTracker: Upgrading Existing Virtual Reality Tracking Systems. Krzanich, Kylee, et al. 2019. 2019.
- Simulation of a virtual reality tracking system. Patel, Kapil and Stuerzlinger, Wolfgang. 2011. 2011.
© 2022 Elliot Négrel-Jerzy. All rights reserved. Please do not reproduce this article without permission; short quotations with attribution are welcome.