Augmented reality has intrigued me since learning about it, but the chance to dig in first-hand didn’t arise until this year’s SXSW Gaming Expo. I met Aaron, and asked if I could throw some questions his way. Aaron is a storyteller and teacher of interactive narratives, who has been working with augmented reality since 2011. His project “what if im the bad guy”, an interactive documentary about soldiers accused of war crimes in Afghanistan, was profiled in a special AR issue of Leonardo Electronic Almanac, and he has written extensively about the design of his most recent AR project, Ice-Bound.
Travis Blair: How would you explain what augmented reality is, to someone who is not familiar with the term?
Aaron Reed: Most people know what virtual reality (VR) is, where you get completely immersed in a computer-mediated world via eyewear and sometimes other tech (headphones, gloves, etc). With augmented reality (AR) you’re still seeing the regular world, with virtual elements overlaid on top of it. So you might see a character who isn’t really there, or hear sounds coming from a particular real-world point as you move around. It can also be done through glasses (although here with pass-through lenses), or through a device like a phone or tablet that you hold up in front of you, as if it’s a magic window you’re looking through to reveal hidden parts of the world.
Travis: What technological advancement brought about AR?
Aaron: Though it’s been technically possible for a while now and there’s AR projects dating back for decades, it’s only been with the rise of smartphones (and especially tablets with bigger screens) with GPS, camera, compass, and accelerometer capabilities that AR has really taken off. My first AR project came out in 2011, near the launch of the iPad 2 (the first major tablet to have a rear-facing camera), and it’s really only been in the few years since then that AR has been available as a relatively mainstream platform.
Travis: You created Ice-Bound, a game that uses AR technology. What component must someone consider, when creating anything that uses augmented reality?
Aaron: AR is so new that the rules for what works and what doesn’t are still being discovered and invented. For instance, in Ice-Bound, we knew we wanted to use AR to make the pages of a printed book seem to change when you looked at them through your device, but we had no idea at first what kinds of changes would be interesting, effective, or practical. As with any new medium, what seems obvious in hindsight takes lots of experimentation and pioneering to initially discover.
So I think it’s still too early to answer this definitively, but one pattern I’ve seen so far is that creators can never forget about the practicalities of the real world with AR. In my first AR project, an art installation, participants were asked to follow a line of virtual people, from the check-out counter across a real-world field to the GPS points where the piece was positioned. At first we didn’t account for the fact that the field had gopher holes and if people were looking up at the screen they were liable to trip and hurt themselves! We had to redesign this so the piece had them visually identify their destination without moving, then turned off the screen until they arrived at it, to force them to watch their footing rather than the screen. These kinds of considerations are always coming up with work in AR, which is an interesting gotcha if you’re used to working on purely digital projects.
Travis: Are there any new or upcoming applications of the technology that you think will make it more widely used?
Aaron: The new VR craze has an AR component too, and a lot of the newly announced headset devices are partially or fully designed with AR in mind (like the Microsoft Hololens). I think there’s definitely a lot of “bubble” around this tech right now: once some of the big players actually make it to market and release products, we’ll see how many people are interested in buying them. Time will tell, but I actually think smartphone/tablet devices may remain the platforms of choice in the medium-term, since people are already used to carrying these around and making them part of their everyday life. Longterm, if something like Google Glass becomes ubiquitous, we might see new genres of AR games emerge that are continuously integrated into your everyday life: but I think that’s still years off, at least.
Travis: What will be the next advancement that will improve augmented reality?
Aaron: Honestly I think it’ll be a design advancement. The latest smartphones have more than enough technical capability to deliver amazing AR experiences: it’s just that designers are still trying to figure out what AR is good for. There’s been a lot of business interest in monetizing AR tech: showing you the way to the nearest Starbucks, for example, or putting floating email notifications in your field of view. It’s also still incredibly difficult and expensive to develop for AR, with many of the best libraries being too expensive or restrictive for everyday folks to use.
I think as more open source tools start coming out, and a wider and wider community of people start experimenting with how augmented reality can legitimately enhance the human experience, not just be a showy gimmick or advertising tool, we’ll start seeing the real advances in AR as a genuinely new and exciting platform for storytelling and expression.