Apple Car self-driving system could tell iPhones in the car what it is about to do

The Apple Car self-driving system could pre-alert drivers and passengers of maneuvers it will be performing, by posting notifications on the screens of iPhones and iPads in the vehicle.

The development of self-driving systems such as those as part of the "Apple Car" project, could lead to an easier time on the road for everyone in a vehicle, including the driver. Depending on how far along autonomous control progresses, the designated driver of a vehicle may not have much incentive to actively watch the road to make sure everything is going smoothly.

It is already the case for some drivers that minimal attention is being paid to the road, even in situations where self-driving systems aren't actually being used. The reports of people texting and using smartphones while behind the wheel, as well as growing criticism, prompted Apple to introduce "Do Not Disturb While Driving," though the feature can only do so much.

In a self-driving car, there could be an even greater desire by drivers to pass the time using their mobile devices instead of looking at the road or instrument panels. Since a self-driving vehicle has the potential need to hand over control to a driver at any time, this is not a safe situation to create.

An example of a notice where the car updates the driver on a light change.

A patent awarded to Apple by the US Patent and Trademark Office on Tuesday titled "Device, method, and graphical user interface for presenting vehicular notifications" aims to solve the problem by bringing warnings and alerts about a vehicle to the display of a device being used by the supposed driver.

By pushing the alerts to an iPhone or iPad screen, Apple hopes to give drivers every opportunity to assume control if something is going wrong, or at the very least to be aware of what the car is about to do. Apple reckons that a driver operating a handheld device may be hard to grab the attention of away from the device, so a display on the device itself that cannot be avoided would be appropriate.

If the car has to stop, go at traffic lights, or needs to hand over control for safety or other reasons, an alert is passed over, and can be displayed in a variety of ways. A general status indicator could provide basic updates of transit, while more general notifications in a portion of a display or animations in the middle of the screen will do more to grab the drivers attention.

A car may prompt an iPad-using driver to pay more attention to the road.

These indicators could also alert the driver to interact with the vehicle if a predetermined time since the last interaction has passed, with notifications accompanied by haptic feedback or an audio cue. The display itself could be obscured, video or audio content paused, or even the remainder of the screen rendered unresponsive in some situations.

Conversely, if the system determines the driver's attention is no longer needed, it can withdraw the notifications, as well as to make unresponsive screen areas responsive once again, and to recommence playback of media.

The number of notifications and what a driver can do with their mobile devices could change depending on the level of attention is required, which in turn can be dictated by the level of semi-autonomous or autonomous control is in play. It is suggested the levels would be those determined by the Society of Automotive Engineers, with semi-autonomous corresponding to levels 1 to 4, while autonomous control is level 5.

Notifications may not be central on the screen, as they may be smaller or take up only a portion of the display.

The patent lists its inventors as Andre Boule, Stephen Chick, Thomas R. Powell, and Emily Schubert.

Powell is a Car Services Engineer who is also named on the "Instrument cluster metadata to support second screen" patent, which could lead to CarPlay displaying more information in a car's regular dashboard controls. Schubert is named on a patent for a smart bike, as well as one that allows car control from an iPhone alongside Stephen Chick. Boule has a large number of interface-related patents to his name, including some dealing with graphical user interfaces spanning multiple devices.

Apple files many patent applications on a weekly basis, but while the existence of a patent application indicates areas of interest for Apple's research and development teams, it isn't a guarantee that it will appear in a future product or service.

Apple Car

The Apple Car, believed to be under "Project Titan," is rumored to be an Apple-designed vehicle, one which includes many novel elements of car design as well as a self-driving vehicle system. Apple has invested heavily in self-driving technology, including acquiring self-driving startup Drive.ai in 2019, among other purchases and strategic hires.

It also operates its own fleet of test vehicles, and at one point had 72 cars and 144 drivers on roads in California.

Naturally, it has secured many patents and filed applications in the field, including one from 2016 covering a path-finding method for "collision avoidance of arbitrary polygonal obstacles." It has also developed a "confidence" system to make driving decisions from just-enough data without consuming too many resources, a system that changes its behavior depending on the stress level of vehicle occupants, and inter-car communications for enhanced situational awareness.