Apple is considering ways to make drivers more aware of potential road hazards, including headlights that can single out road elements that the driver needs to be aware of, and a heads-up display that can highlight pedestrians and other items of interest on a car's windscreen.
Published on Thursday by the United States Patent and Trademark Office, the patent application for "System and Method for Light and Image Projection" primarily deals with assisting the driver of a vehicle to better understand the road ahead of them. While existing warning systems use sensors to detect dangers, such as drifting between lanes or proximity to other road users, and providing a basic alarm as a warning, Apple's systems instead aim to highlight things in front of the vehicle with a purely visual system.
All of the implementations start with the vehicle detecting what is ahead of it in the road, and to the sides, using an array of sensors. The data from the sensors are fed into a system that can analyze the street scene, including obstacles, and determine how the vehicle should ideally move as well as the velocity of any moving hazards.
In one implementation of the proposed patent, the system would determine items of interest and then change how they are lit by the vehicle's headlamps. This could include adding an "illuminated indicator" at the point where the object in question is headed, advising to the driver what the item is and where it is going.
Another implementation is to control the contrast of an illuminated scene, namely making sure the driver's view is lit up properly so that everything is visible. Apple argues that some road objects may reflect more than others, or are poorly lit due to being in a rural location without street lighting, or the presence of rain or fog could affect how everything is lit by the headlamps.
Using the sensors again, the headlights can use multiple levels of illumination to more evenly light the scene, with areas that reflect light more easily receiving less light, and areas that are relatively dark receiving more.
Apple also suggests that, while the light level can be adjusted on the road to increase visibility, notifications warning the driver of road elements may not be projected onto the road in public view, but instead as a heads-up display for the driver. While helmets are suggested, it is posited that the notifications could instead be placed onto the windshield, directly in the driver's view.
This could enable more advanced notifications for the driver, such as showing the next step they should take in a set of navigational instructions. In extremely low-light situations, and in cases where the headlamps are not providing enough illumination, the system could highlight road elements that are visible to the sensors, but not to the driver.
One example involves pedestrians, with one image in the filing suggesting they could appear in the display as a form of a skeleton, with dots appearing at the major joints of the person's body to illustrate their movements.
Apple files numerous patent applications on a weekly basis, and there is no guarantee that the concepts described will ever make it into a consumer product.
This system sounds similar to what is being used in Apple's trial of self-driving vehicles, which requires an array of sensors to need to read the road and its potential hazards, in order to continue on a journey. A refocus of "Project Titan" from a rumored car to self-driving systems, the project now involves a fleet of vehicles running on public roads, with Apple also keen to use them in projects such as PAIL, which would ferry employees between offices using autonomous vans.
As the sensors in the patent application and the self-driving cars share the same core features of monitoring the road, it is feasible for such an illumination and warning system to be used in a self-driving car at times when the user wants to drive for themselves. It would potentially offer the benefits of an AI-based driving system in terms of road safety, while still allowing people to drive their cars how they want.
This is not the only patent application relating to Apple's vehicular ambitions. Previous filings include a self-driving system altering its driving style based on the observed stress of passengers, gesture-based controls both within the car and externally, and a system for a self-driving car to notify other drivers of its intentions to perform specific maneuvers.