Virtual vs. Mixed vs. Augmented Reality
There seems to be a lot confusion about reality lately. As we talk to our clients about use cases for the rapidly emerging technologies of Virtual Realty, Augmented Reality, and Mixed Reality, it helps to start by defining the terms. There are a few different definitions depending where you look on the internet, so here are the ones we use and why we use them when discussing projects. One key factor that will become apparent as you read the definitions below is our differentiation between the concepts of “context” and “virtuality” (Yes, we just made up the word “virtuality,” which we define as the state of being “virtual” – not physically existing as such but made by software to appear to do so.)
Virtual Reality
Virtual reality, put simply, is defined by a fully immersive experience other than the surrounding physical reality. If the viewer has no direct view of the physical world around them, then they are in a completely virtual environment. Some examples of Virtual Reality technology would include the Oculus Rift, the HTC Vive, and Google Daydream technology on mobile devices. The question that naturally follows from our rather simple definition is whether or not 360° Video and 360° Photography can constitute virtual environments. We would argue that they can, because they can by themselves be fully immersive. Whether the viewer is surrounded by a computer generated environment or a photographic (or video) representation of a real physical environment, they are still immersed in an environment other than the physical one where they are present. We reserve the one instance of what would otherwise be considered Virtual Reality as a separate context; this instance being a live, fully immersive video representation of a viewers current physical environment. It would be hard to consider such an environment as “virtual” if the viewer is physically there and looking at the environment in the present time.
Mixed Reality
Mixed Reality, in contrast, is a form of Augmented Reality where the viewer is not totally immersed. We’ll go into more detail about what Augmented Reality is below, but felt that we needed to first make the distinction between Mixed Reality and Virtual Reality as contexts. We’ve seen definitions by other parties which argue that Mixed Reality requires augmented virtual objects to be able to be anchored to or interactive with the physical environment. We would argue that this type of interaction is more a characteristic of Augmented Reality, and will go into detail about that below. Examples of Mixed Reality technologies include Microsoft Hololens, Avegant Light Field, and Meta 2. In each of these cases, the viewer can see the physical environment surrounding them through the hardware, and the hardware is simply projecting virtual objects into their field of view. We differentiate the ability to see the actual physical environment from a video representation of the viewers physical environment for technical reasons, namely because full immersion is what defines Virtual Reality. Without differentiating, it becomes difficult to maintain the separation between the contexts.
Augmented Reality
Augmented Reality, by our definition, is the mixing of virtual objects within the context of a physical environment. This means that Augmented Reality can exist within the context of Virtual Reality, Mixed Reality, or in its own context. The key factor is the mixing of something that is physically present with something that is not. One example of Augmented Reality would be a Snapchat Filter, where something virtual is being added to the real physical context of a photograph. Another example would be the eXpoReality mobile application, which augments virtual objects into a mobile device’s live camera view of the physical environment at a trade show. A final example, demonstrating how Augmented Reality can exist within Virtual Reality context, is present in a Georgia Pacific Excel mobile application, where a Virtual Reality 360° video includes an augmented whiteboard video which was added in after the video of the physical space was recorded. By this definition, augmented reality is a much larger area of technology than the contextual “environments” of Virtual Reality and Mixed Reality.
In fact, using this definition, Augmented Reality is not a new technology at all. It would include a number of other technologies as well, such as chroma key green screens in TV news rooms and Pepper’s Ghost in theater environments. This is why we feel that there needs to a be a clear differentiation between context and “virtuality”, and therefore between Augmented Reality as a technology concept and Augmented Reality as a context for viewing virtual objects.
Augmented Reality Context
You may notice, based on our definitions of Virtual Reality and Mixed Reality, that there is a gap which needs to be filled. This gap is why we define the third context of Augmented Reality. Where Virtual Reality requires that the user be fully immersed in an environment other than the real time physical one they are in, and Mixed Reality requires the ability to see the physical environment surrounding the viewer, Augmented Reality context is the grey area between the other two. This means that an Augment Reality context is one where a user is viewing a video representation of the environment where they are physically present at the current moment, but one which also contains virtual objects. As stated above, and with a clarifying example below, this third context is necessary to provide the separation between Virtual Reality and Mixed Reality.
Interaction between Physical and Virtual
If you continue to read and learn about the concepts surrounding Augmented Reality technology, you will eventually come across the term SLAM (Simultaneous Localization And Mapping). SLAM refers to the ability to, in real time, accurately map the location of virtual objects into physical space. What makes SLAM important is that it allows shared experience of Mixed, Augmented and Virtual Reality between multiple viewers. As an example, SLAM can theoretically be demonstrated by considering the possibility of three people playing catch using a virtual ball, with one player each in the contexts of Mixed Reality, Augmented Reality, and Virtual Reality. The augmented virtual object of the ball would need to exist, equally and simultaneously within the context of each viewer’s experience of the physical environment, and possibly even subject to the physical environment’s laws of physics (such as gravity, or the inability of the ball to pass through a physical solid object). This is why we we have trouble differentiating Augmented Reality from Mixed Reality based a virtual object’s ability to anchor to or interact with physical space, and also why we need to maintain the separation between Virtual and Mixed Reality. Mapping the location of a virtual object is easy to do in a completely virtual 3D environment. It is much more difficult to map virtual to physical when the computer technology that is rendering the virtual object does not have a complete definition of the space where the virtual object is located – both because each viewer has a different perspective of it and because the computer does not have control of the physical space. The physical environment changes, and the virtual environment needs to be able to respond to the physical changes.
Why Does Any of this Matter?
As technologies evolve that allow computers to more accurately identify, map, and render physical environments (such as Google Tango and the Structure Sensor), it will become more important to define the environments where we experience various combinations of physical and virtual objects. As people begin to derive value from virtual assets and identities, and legal questions begin to take shape regarding ownership of these assets, it will be necessary to draw the lines and have clear definitions. This is why we believe the definitions above will stand the test of time, even if we are only using them now in regards to our current client relationships.