Get Ready for AR

With the major releases of Apple’s ARKit and Google’s ARCore, platforms for serving augmented reality (AR) apps, the AR market is expected to heat up. SuperData Research forecast augmented and mixed reality revenue will more than double to $3.2 billion this year with the help of Pokémon GO and the upcoming Harry Potter: Wizards Unite and will surpass virtual reality revenue in 2021. Digi-Capital estimates mobile AR could reach an installed base of 900 million by the end of 2018 and near 3.5 billion by 2022.

DDM attended Apple’s “Introduction to Apple’s ARKit: Best Practices and Recent Updates” session at this year’s Game Developers Conference in San Francisco. There is a ton of technology packed into Apple devices starting with the iPhone 6 and iPad Pro, all of which will ease the adoption by app developers and consumers. The ARKit works with Unreal, Unity and custom rendering engines and enables three primary features for AR:

• Tracking: a combination of the built-in camera, gyroscope and front-facing camera to determine how the camera is moving or being used in relation to the real world, and includes face and expression tracking as well as world tracking which enables virtual objects to be rooted in the real world, all without the need for external sensors or cameras and in real-time
• Scene Understanding: analysis of the scene to detect vertical and horizontal planes like floors and walls, lighting in the real-world which is then applied to virtual objects, image detection to check for a pre-loaded 2D image in the background that triggers an event and hit-testing to determine if there is a correlation between the screen taps to the virtual objects in the real-world 3D setting
• Rendering: delivers the AR view that is a combination of real-world and digital content at 60fps for the phone orientation and aspect ratio

As many consumers will be encountering AR for the first time through a mobile device, Apple gave some brief best practices:

• Prompt the user on what will happen such as starting the camera
• Indicate the readiness of tracking such as a focus box that locks onto objects
• Size objects realistically and provide depth clues like drop shadows
• Use large tap zones and gesture thresholds that prevent accidental movement of objects
• Give feedback if encountering problems such as low lighting
• After an interruption, like a phone call, attempt relocalization to restore virtual objects in their original real-world positions, and allow the user to reset the experience

Examples of well-implemented AR included JigSpace, The Virtual Reality Company’s Follow Me Dragon, Ketchapp’s Stack AR, Limbic Software’s Zombie Gunship, Directive Games’ The Machines, and Touch Press/StoryToys Entertainment’s My Very Hungry Caterpillar AR.

More best practices for starting the AR experience, rendering content, placing/interacting with objects, image detection and handling interruptions can be found online in Apple’s Human Interface Guidelines.

Like ARKit, Google’s ARCore has a suite of technology that covers the foundation for motion tracking, environmental understanding and light estimation; it is compatible with a number of Android devices for a current total 100 million. Google also recently opened up its Maps API on Unity, allowing content to be tied to specific locations (think Pokémon GO). Look for the recorded conference session in the GDC Vault.