Apple patents augmented reality mapping system for iPhone

Apple on Tuesday was granted a patent detailing an augmented reality mapping system that harnesses iPhone hardware to overlay visual enhancements onto live video, lending credence to recent rumors suggesting the company plans to implement an iOS-based AR strategy in the near future.

As published by the U.S. Patent and Trademark Office, Apple's U.S. Patent No. 9,488,488 for "Augmented reality maps" describes a mapping app capable of tapping into iPhone's advanced sensor suite to present users with real-time augmented views of their surrounding environment.

In some embodiments, digital information like street names, points of interest and more are overlaid on top of live video pulled from a device's camera. Other variations include complex functions like GPS-based navigation.

For example, a user might open activate iPhone's rear-facing camera by navigating to an AR feature within the Maps app. The system automatically correlates data from onboard sensors, including gyroscope, digital compass and accelerometer components, with GPS data to pinpoint iPhone's position in space, tilt and alignment.

Using the newly gleaned information, the app is able to intelligently determine an approximation of the displayed live video scene, including field of view. From there, the captured image, or live video feed, is visually augmented with vital mapping information downloaded from an offsite server.

Certain embodiments offer simple POI identification, for example the name of buildings and monuments, though the system allows for more complex computations. In some examples, Apple's invention generates interactive route guidance complete with onscreen path indicators overlaid onto the live streamed video. The method can also pull data from the cloud to alert users that they are heading down a one-way street, for example.

Depending on device orientation, the system is capable of switching display formats from live video to a bird's-eye view, making transitioning into and out of AR mode a seamless experience.

Today's USPTO award comes five years to the day since Apple first applied for the patent grant in 2011. Only minor changes were made to the document to clarify and enhance claims.

Apple is known to be working on augmented reality solutions, though exactly what form that product will take is unclear. CEO Tim Cook in an quarterly investors conference call in July signaled Apple is particularly interested in AR, a sentiment he reiterated multiple times since then.

As tech industry competitors like Google, Microsoft and Facebook field their own AR/VR systems, pundits believe Apple plans to dip its toe into the AR/VR waters with an iPhone-based solution. Bolstering those claims are a multitude of AR patents, the latest covering device localization technology assigned to Apple from recent acquisition Flyby Media.

While an iPhone AR mapping app is intriguing, integrating the same technology into an in-car heads-up display would prove revolutionary. Analyst Ming-Chi Kuo recently hinted that Apple might use AR technology developed for mobile to break into other areas of interest like autonomous driving systems pulled from the ashes of Project Titan.

Apple's augmented map patent was first filed for in February 2010 and credits Jaron Waldman as its inventor.