Tech Explained: Mixed Reality

Find out how mixed reality merges the real and virtual worlds to produce new environments and visualisations in our latest Tech Explained feature.

By now, most people are familiar with virtual reality and augmented reality, but fewer are familiar with the concept of mixed reality. The term mixed reality (MR) was introduced in a 1994 paper by computer engineers Paul Milgram and Fumio Kishino. The paper introduced the concept of a virtuality continuum, a scale ranging between the completely real and the completely virtual. So, what exactly is mixed reality and what can it be used for?

Firstly, it’s important to understand the difference between VR, AR and MR. Virtual reality is the creation of a completely artificial environment. With VR, users experience artificial sounds and sights and feel as if they’re in a digital world. Virtual reality has been applied to uses as diverse as therapy and helping learner drivers. In augmented reality, virtual objects are overlaid onto a real-world environment. In addition to popular games such as Pokemon Go, this can be seen in applications that bring newspapers to life and create virtual business cards. Mixed reality, in contrast, combines the real world with a virtual environment, allowing users to interact with both the real world and the virtual world.

Unlike, AR, which is viewed through a flat screen, such as a smartphone or tablet, MR is headset-based. This is similar to VR in that users wear a headset, watch the screen in front of their eyes and have their movements tracked so they can move around in the virtual world. Where MR differs from VR is in using a system of front-mounted cameras that can add real-world objects to the virtual world, or virtual objects to the real world. Mixed Reality is designed to make users feel like they are a participant in the action, rather than a spectator. In the hybrid environment of MR, interactive virtual objects can be mapped to the physical environment, seamlessly blending the real and the virtual.

There are two main forms of MR. In one, the virtual content is not only overlaid on the real environment (as in AR) but is anchored to and interacts with that environment. Users can see virtual objects just like in AR, but these objects can also interact with the real world. This type of MR is like a more immersive and interactive version of AR. Headsets for this type of MR have translucent glasses that allow users to see their surroundings, and the virtual experiences are created with the help of holograms. This is how Microsoft HoloLens works.

Another form of MR allows users to see and interact with a completely virtual environment, overlaid on the real world around them. This kind of MR uses a headset to track the real world and adjust the virtual environment accordingly. Windows mixed-reality headsets, such as those made by Acer and HP, work this way.

So, what is the potential for MR? Many industries could benefit from MR. In communications, MR could enable people to collaborate more efficiently. One existing example of this is Skype for Microsoft HoloLens. Mixed reality can also boost education by providing immersive learning experiences. Similarly, MR can allow workers to see holographic instructions overlaid onto machinery. Some manufacturing companies have already started using this MR technology. For example, Renault Trucks uses MR to control quality at one of its factories.