Apple's future AR smart glasses could help you find things in your house

Apple is working on an AR mapping app for an iPhone or smart glasses, with identification of not just distant locations, but also on a smaller scale, like a car dashboard or items in your house.

Mapping and navigation systems are handy for many users in telling people where to go and how to get to their desired destination. While useful for those who can adequately read maps or those who have a general idea of their current location's geography, sometimes users want to be able to have a more visual guide to where things are in relation to their current position.

Granted by the U.S. Patent and Trademark Office on Tuesday, Apple's patent for a "Method for representing points of interest in a view of a real environment on a mobile device and mobile device therefor" effectively describes how an augmented reality system could point out important areas in a landscape for a user.

The patent is in fact one that surfaced in 2017, revealed by the USPTO in July of that year after being applied for by Apple in April.

An illustration of an AR overlay on an iPhone

According to the patent, the mobile device would capture an image of a view, likely one from a live camera feed, and overlay an indicator on the screen in the relative position of where a known point of interest is based. In a live video feed, the position of the indicator will change in relation to alterations in the view, so the indicator will continue to show the correct position.

With each indicator, extra supplemental information could also be offered in a separate and uncluttered part of the display, such as its name and distance. By interacting with the indicator, this could trigger functions such as navigating the user to it or offering more information or content, to learn more about the place without going there.

Illustrating how an iPad could be used to show features of a car in AR

While the concept of an AR view for maps isn't an entirely new concept, there are some elements of the patent that differ from the usual idea.

First, it is suggested that the system could be used for small environments as well as larger spaces, such as things in a room. In one example image, an iPad-like tablet is used to show off key features of a car's interior, identifying and highlighting important items and potentially giving the user more opportunities to learn about each item. The text discusses location of items in a house, like pinpointing where your iPhone is in your house, instead of using "Find my iPhone" to make it play a sound.

The patent also references the use of a "semi-transparent display," in that rather than using a live camera feed, it could be graphics overlaying a real-world view seen through glass. Some of the example illustrations depict a head-mounted display view that would be expected from the usage of smart glassesor a VR headset.

Illustrating how the system could work in hypothetical smart glasses

As usual for patents and applications, Apple files many documents with the USPTO each week, and while they can be used as an indicator of Apple's fields of interest, they are not a guarantee that Apple will be working in that area in the future.

Apple has a number of patents and applications already under the field of Augmented Reality, to go with its existing and popular ARKit platform. In January 2017 it received two patents for a mobile AR system that could detect its surroundings and provide information to users in real-time, similar to the in-car function from the latest filing.

Apple also received a patent for "Augmented reality maps" in 2016, which laid out road names and routes on the streets and roads of the camera's view, as well as highlighting points of interest. More recently, a December 2018 patent application showed how AR could help drivers and passengers of a ride-hailing service find each other in crowded or busy locations.

The company has been rumored to be working on its own smart glasses and AR headsets for some time, with the suggestion WiGig and 8K-resolution eyepieces could be used in such headwear. Such a product is unlikely to ship anytime soon, as analysts believe the earliest a head-mounted AR device from Apple could ship is 2021.