-> Part of the [[Mixed Reality]] spectrum. When I use the term “Augmented Reality” I mean that a user is provided computer generated information alongside their perception of reality, thus combining the digital and physical reality into one coherent perception. Example: Seeing directions integrated into the view of physical reality. As humans experience the world via their senses and in principle every one of those can be digitally augmented: Screens & glasses augment the visual sense, headphones the audible sense, gloves the haptic sense and so on. Devices can have very different approaches to showing data to a user. These can be roughly aggregated into two categories, or modes: Annotation, unification and replacement. Currently these approaches manifest themselves as device classes, as they have to deal with different technical challenges. But really these are operational modes that achieve different things in different situations and scenarios. So it’s important to note that no device class / mode is strictly better than the other, they just have different optimal use cases. I believe that in the future these device classes will converge and one device will be able to utilize every mode, based on which provides the best user experience in a given situation. However until the technology has caught up, they will remain separated. ## Mode 1: Annotated Reality The simplest mode of AR is adding information alongside physical reality. A simple way to think about this is a smartphone screen permanently located in the viewport of the user. While the augmented output is separated from the physical environment, the key is some form of contextual awareness. A system used for navigation might utilize GPS and a compass to know where it is and provide the relevant information about the physical environment on the screen (“You are here and this is what’s around you”). That way, although the screen is separated from the environment, there is a contextual connection between both. This is also the default mode for "hearables" - headphones that play context-aware information. Example: [Get Directions through Google Glass](https://www.youtube.com/watch?v=V8ofTlynWPo) by [[Google]] * Comparable to: Smartphone in your vision ("Second Screens"), HUD in a car or an airplane, guided audio tour. * Example visual devices: Google Glass, Epson MOVERIO, Vusix Blade & M * Example audio devices: Bragi dash, Apple Airpods ## Mode 2: Extended Reality (XR) and Mixed Reality (MR) This integrates the digital aspects seamlessly into the physical environment. Instead of “screens in the air” the digital elements are in and around physical objects. When done well the user is not put off by the integration of digital and physical elements and both feel equally natural and intuitive. Such a system is also context aware, but beyond that it also has spatial alignment. For example a navigation system would use the environmental understanding to highlight the actual lane you need to take and possibly other relevant information. For audio this means that sounds are woven into actual sounds to change or extend them. An example would be playing augmented engine noises inside a car to change the perceived driving experience for the driver and passengers. Example: [AR hub, Ceres Holographics](https://www.ceresholographics.com/ar-hud.html * Comparable to: Iron Man design scenes * Example devices: Microsoft HoloLens, Magic Leap One ## Mode 3: Augmented Virtuality This seems similar to [[#Mode 2 Extended Reality XR and Mixed Reality MR]], however it works the other way around: It integrates the physical world seamlessly into an artificial digital environment. In other words, the user is not able to see the outside world, however the physical environment is recreated in a context specific way, matching the physical reality if required. Arguably active noise cancelling headphones are also such a use-case since they take the current audio around a user and process it to create a replaced experience. Example: [holoride](https://www.holoride.com/) by Audi * Comparable to: Audi VR experience, Valerian “Big Market” scene * Example devices: Varjo XR-1, HTV Vive Pro ## Future visions [Land Rover Reveals Transparent Bonnet Concept](https://www.youtube.com/watch?v=1OlqditIsoM), April 2014 Land Rover reveals world's first Transparent Bonnet Concept allowing a new level of driver awareness with a 'see-through' augmented reality view of the terrain ahead, making the front of the car 'virtually' invisible from inside the cabin. [Google Glass "One Day"](https://www.youtube.com/watch?v=9c6W4CCU9M4), April 2012 *"This is an early concept video that was made when Project Glass was just getting started. While a lot has changed since then, our motivation to get technology out of the way remains the same."* [Nokia future vision](https://www.youtube.com/watch?v=A4pDf7m2UPE), ~2009 From the Nokia Research Center, now https://www.nokia.com/innovation/