Apple Maps may add AR view highlighting the route ahead while driving

A future version of the Maps app could provide more intuitive navigational instructions to drivers in the future, by using augmented reality to superimpose the route to take on top of a live view of the road ahead, indicating which lanes a driver should take.

Navigation systems in vehicles have become quite sophisticated over the years, providing routes to desired locations while also recalculating the optimal pathway if the driver goes another direction, as well as taking into account data on traffic flows, detours, and accidents.

The systems and mobile apps are also able to provide assistance for multi-lane highways and complex junction systems, advising of which lane the driver should be in for the journey, saving road users from one more headache. However, the information provided is usually offered as either verbal instructions that could be misheard or not heard at all, or as symbols on a screen that may take effort to understand, which isn't that useful in such stressful situations.

In a patent application titled "Navigation using Augmented Reality" published by the US Patent and Trademark Office on Thursday, Apple believes it can provide better visual information to the driver, by taking advantage of AR.

In Apple's solution, a device with a camera and a display is mounted to the dash of a vehicle, with the camera positioned for an optimal view of the road. The device could easily be an iPhone, as it is one that also needs to communicate with an online server to determine the route the driver needs to take.

Apple's proposed solution resembles an iPhone mounted to the front of the vehicle cabin

The system uses the camera to take in a series images of the road ahead for analysis, which involves determining the horizon line as a reference point for the camera angle, as well as elements of the road. At a minimum, it has to determine where the road is, or in a multi-lane situation, which lane the vehicle is in, when compared with the devices' position in the route data, as well as previously-established road layout information.

Once the position on the road and the camera angle are determined, the system then prepares the live video feed for the driver to see. Along with the usual imagery for navigation superimposed on the image, the system can also darken junction turnings and lanes that are not for use by the driver, effectively highlighting just the route the driver should take.

Guidance arrows could also be placed on top of the highlighted road view to advise the driver of upcoming turns, or that they should switch lanes in a multi-lane roadway. These arrows could be animated or blink to attract the driver's vision, becoming more aggressive as the turn or required lane change gets nearer.

Examples of lane-change and turning views with the path ahead highlighted

Apple files numerous patent applications with the USPTO on a weekly basis, but while the existence of the filing is an indicator of where Apple's research and development efforts have been in the past, it isn't a guarantee that the concept will appear in a future product or service.

Given the patent application could effectively consist of an iPhone mounted to a vehicle in view of a driver and already has the technology and the software capabilities to perform such a task, as well as Apple's existing Maps app and navigational expertise, it is entirely plausible Maps could gain an AR view in the future.

This is not the only navigational use Apple has for AR systems.

In August 2016, Apple was granted a patent for "visual-based inertial navigation," a system that used an onboard camera and sensor data to determine the device's motion, similar to how ARKit performs its own tracking. Apple's patent lends itself heavily to indoor navigation systems, such as for finding items within a store, that cannot easily rely on GPS.

A February 2019 patent suggested the use of AR for identifying locations of points of interest on a mobile device or smart glasses, which would overlay indicators of a live view relative to where the wanted location is in the camera's vision.

The system doesn't even have to be presented via an iPhone. An April 2018 patent application for an "Adaptive vehicle augmented reality display using stereographic imagery" envisioned a navigation system that appeared in a car's windscreen.

The idea of AR-based navigation has also been picked up by Apple's competitors. In May 2018, Google revealed a version of its Maps app which used an AR view to superimpose arrows of where to go. There was also the suggestion of animated characters, such as a fox, following the intended route so as to eliminate doubt of exactly where to go while on foot.