Apple Car passengers could use AR and VR headsets without feeling sick

Passengers of the Apple Car may be able to use AR or VR without worrying about a car's movements causing nausea, while infrared sensors inside the vehicle may be hidden from view by light-reflective panels.

Apple has been working on a variety of concepts for its Apple Car — also known as "Project Titan" — for a number of years, and the constant supply of patents around the project indicate Apple has grand ideas both in the form and functionality of the vehicles.

In-car AR and VR inertial measurement

The first of two patents, the possibly misspelled "Relative inertial measurement system," explains how the inertia sensed in a device on a user's moving body could be measured, and compared against a larger inertial force. Smaller devices that perform motion sensing or gesture recognition rely on measuring inertia, but a larger external movement force could provide a considerable amount of inertia that could be misinterpreted by the device.

In Apple's case suggested in the patent, it proposes a system where the inertial measurements of a user's head monitored by a virtual reality or augmented reality headset's sensors can work properly by taking into account the measured inertia from a vehicle changing direction or adjusting their speed. In effect, this could allow a VR or AR system to only interpret the true inertial movement of the user, namely eliminating instances where a car's movements could be mistaken for head movements.

Apple claims the discrepancies between the user's relative motion within a vehicle and motion observed in a virtual scene could lead to nausea or sickness.

For Apple's system, it suggests the use of a device coupled with a body part that moves with the user, one that includes an inertial measurement device. The sensors, which can include accelerometers, gyroscopes, and magnetometers, is used to measure the combination of inertia from the user's movements and inertia caused by the vehicle.

A simplified example of a VR or AR headset and inertial measuring system within a vehicle

A second inertial measurement device is included to give an alternate point of reference, but one that is just for inertia generated by the vehicle, not the user.

By combining the two measurements together, a system could compare values and determine just the inertia generated by the user's movements. This can then be fed back into the system used to produce the VR or AR experience.

The patent's inventor, Fletcher R Rothkopf, is a senior director of product design in the Technology Development Group within Apple, and has worked for the company for over 13 years, including working on the Apple Watch. Aside from numerous patents relating to the Apple Watch and connected areas, he also worked on a "Foveated Display," which could reduce the data bandwidth and power needs for displays used in head-mounted systems.

Apple has been working on ways to incorporate AR technology in vehicles, largely in the form of in-windscreen systems overlaying the real-world view, but the use of headsets could be more for passenger use, as a form of entertainment.

Hiding infrared components

The second patent, for "Systems with colored infrared-transparent layers," is less about the user experience and more about the aesthetics of vehicular design.

By adding user-sensing equipment and systems for automated driving, among other features, this adds a number of cameras and other items into the mix. However, this increase could also affect how the car appears, with too many sensor apertures and lenses on show potentially making the inside of a car or van an eyesore for the driver.

In short, Apple proposes the use of a variety of different coatings to make panels and covers for infrared optical equipment that is capable of allowing infrared light to pass through, while reflecting visible light, effectively hiding the sensors from view. As infrared light could pass through, the same surface techniques could be used for the transmission of light out into the vehicle, as well as for detection purposes.

Apple's proposal involves the use of an infrared-transparent substrate, or another support structure capable of transmitting infrared light, onto which a polymer layer can be applied. Said polymer layer can contain plasmonic nanoparticles that can reflect visible white light, while the addition of colored dye or pigment to the polymer layer will help give the panel the designer's desired appearance.

Examples of layers with nanoparticles used to reflect visible light, while letting through infrared light to sensors

The plasmonic nanoparticles are said to include solid metal nanoparticles and "metal-coated-dielectric-core" nanoparticles, with the dimensions of each particle varying to adjust their absorption and scattering spectrums. Using multiple different types of nanoparticles could be used in a layer to reflect white light, allowing a non-black colorant to be used to color the infrared-light-transmitting layer.

The patent credits Malcolm J. Northcott, Arthur Y. Zhang, Bradford J. Zercoe, Matthew E. Last, Jack E. Graves, and Irene Perali as inventors.

Northcott has been a sensor systems engineer at Apple for over 5 years, including the use of optics and image processing, which would pertain to a self-driving vehicle's systems and in other areas. He has been identified as being an inventor for the similar-in-concept "Systems with matte infrared-transparent layers," along with others relating to image capture and "interactive scene projection" within a vehicle.

Zhang a display expert for Apple, linked to patents involving screens of varying kinds, such as storing the long-term history of pixel luminance values to compensate for the effects of burn-in, ambient light color compensation, and lighting in augmented reality environments. He was previously linked to a patent for alternate wing mirror systems for an Apple car.

Zercoe works as a Product Design Engineer in Apple's Special Projects Group and has filed a wide variety of patents with Apple, covering key mechanisms, the matte infrared-transparent layers patent, and keyboard assemblies.

Formerly an Apple engineering manager linked to the Special Projects Group but currently employed at Waymo, Last is, as expected, connected with a lot of car patents. These range from the lighting systems for vehicle seats to "systems with matte infrared-transparent layers," autonomous navigation systems, securable storage compartments, and heads-up displays.

Few details surface on searches for Graves within Apple's patents.

Perali was formerly an Apple engineer, working on sensing systems for autonomous systems for four years until April 2019. She also appears on the aforementioned "matte infrared-transparent layers" patent.