Designing an Augmented Reality User Experience for an Exact Location
Overview
Location-based augmented reality is not a new challenge. There are many mobile apps that use this technology successfully by accessing the user’s mobile device’s GPS location and superimposing virtual elements to your real-world surroundings. The user is left with an experience of a little yellow character popping up on their street or a T-Rex rambling about in their living room. But how do you design an experience where the user’s physical location and spatial orientation are just as important as the augmentation?
We are currently working on an augmented reality application for a client that depicts historical events, battles and famous figures within the Gettysburg National Military Park. We are creating an engaging user experience in a historical setting with the goal of educating users as to exactly what happened here, and why it’s important to remember it. For this reason, the exact location of the augmentation is crucial to narrating the complete and accurate story.
As we began this project, we experimented using a Scene Recognition method that uses image targets or visual cues that can be detected by the mobile device and application as a way to launch the user’s experience. But as our team made our way through the fields and forests of the park on a cold February day, we began to realize the full challenges of the project and the limitations of this type of technology. In an outdoor environment, with sun and shadows, vegetation and deciduous trees, the scene is constantly changing and would not remain a reliable trigger. In addition, many of our scenes take place over a 20+ feet area and most image target technology removes the display of the 3D object once the image target is lost.
Given the limitation of Scene Detection to meet our requirements, we looked at a promising alternative, Location-Based AR technologies. A quick search listed 3 basic types of Location-Based AR technology that are widely used today:
Marker-less or location-based AR using GPS
- Projection-based AR, using images and text that are projected onto physical objects
- Superimposition-based AR replacing the original appearance of an element
But what do you do if there are no street signs to guide a user to a specific location? No markers to trigger an AR app? No physical structures to project or superimpose upon? And while Geolocation gets the user close to the position, the margin of error with GPS and heading detection is too great. Plus, GPS coordinates lack directional information, so perhaps the user navigates to the correct spot but is facing in the wrong direction. We would still need to explain to the user which way to face in order to place the scene accurately. Because it is crucial to our client that the user sees the Confederate army charge up the exact side of the hill of Little Round Top, or the actual stone wall that soldiers jumped during the battle of Pickett’s Charge, we feel that none of these solutions alone will give us the best user experience nor the accuracy that’s required.
For this application, we are focusing on three User Experience goals.
- Be authentic. The experience has to feel real and meaningful. The subject matter should be presented in a way that is accurate, reverent and educational.
- Be simplistic. Any user should be able to pick up this application and use it immediately with little or no onboarding.
- Be scalable. The user experience should be the same at all 11 park scenes. There should be predictability and continuity of scene placement and functionality.
So now we are looking at a combination of strategies, both technical and non-technical, to assist the user in the exact placement of the AR scenes. In combination with Geo-location technology, we are exploring giving the user physical maps and walking directions to the sites. We are experimenting with limiting the user’s plane detection (used to find the ground on which to place the scene) and hopefully limiting the margin of placement error. Lastly, we are kicking around the idea of a kind of scene “targeting” whereas the user must line up the AR characters in their physical environment to match those of the example screen in front of them, thus providing a type of jigsaw clue to the user for exact placement.
Where this User Experience ends up is yet to be determined. While our development team is exploring the functionality of the technology, our creative team will be conducting user testing of these theories to understand if it meets our outlined goals above. Whatever the final outcome, this application aspires to bring people to the Gettysburg National Military Park to explore and remember this nation’s rich history in a new and dramatic way.
Stay tuned!