Inside

Apple AR

Apple AR

Last updated: 3 weeks ago

Apple has been rumored to be working on a few alternative-reality devices, including an AR headset, AR glasses, and a colloquially known "iRing." While there is little concrete evidence of any devices being created, the newly introduced ARKit 3, as well as several patents released in 2019, it's likely that Apple will step into the AR hardware arena by as early as 2022.

● Rumored 2022 release
● People occlusion
● Holographic elements to correct for field of view
● May utilize iPhone or iPhone-like screen
● iRing may provide movement information to device


What is augmented reality?

Augmented reality — often stylized as AR — is the act of superimposing a computer-generated image into the "real world," usually via a smartphone camera and the screen. People often compare AR to VR — or virtual reality. Virtual reality, however, is the act of creating a fully simulated environment that a user can "step into."

AR bowling exampleApple AR experiences would be greatly improved with a hands-free headset

Current examples of augmented reality are games like Pokemon Go, which features an AR mode that allows users to hunt, photograph, and catch Pokemon via their smartphone's camera. Social media platforms such as Instagram and Snapchat use augmented reality filters to encourage users to share pictures and videos.

Apps like Measure are also considered AR based Apps.

Some retailers allow users to preview items in their homes via augmented reality as well.

ARKit

Glasses or a headset might come in the future Imagine viewing games through something over your eyes rather than through a tablet in your hands.

Introduced in WWDC 2017, ARKit is Apple's software development kit that enables app developers to incorporate augmented reality into their apps. ARKit handles many of the tough tasks associated with AR, including detecting movement and the local environment, simplifying the process for developers to place virtual objects in an everyday scene.

In iOS 13, ARKit 3 was released. ARKit 3 received many notable upgrades over its predecessor. New features included the introduction of people occlusion, or the ability to place objects behind or in front of users as needed.

It also enabled more face track multiple faces, and the ability to place digital objects into complex environments.

A new iOS 14 code leak suggests that there will be a dedicated AR app, codenamed "Gobi." The app would use specialized Apple QR codes for displaying content while out shopping. The leak also had details about Find My getting AR functionality for finding lost objects.

Apple's possible AR headset or glasses

Patent mockup A patent mockup of Apple AR glasses

Apple has been rumored to be working on an augmented reality headset or smart glasses for some time and has used considerable resources in the field of AR and VR. While Apple has so far declined to publicly offer details of when to expect AR-equipped head-mounted hardware in the future if at all, it is allegedly offering up timelines to employees.

Patents have surfaced as well, suggesting that Apple may be in the early stages of planning how such devices are going to work.

In US patent application number 20190349662, Apple describes an AR headset having a "device-mount audio mode."

The design suggests a headset with another device like an iPhone mounted in front of the user's eyes. Similar headsets have existed for some time, with Google Cardboard being a notable example.

"[The headset can be] configured to: determine whether the mobile device is mounted on a head-mounted display (HMD) mount," says the patent, "[then] transmit an audio signal from the device processor to a wireless headphone; and change an audio mode of one or more of the mobile device or the wireless headphone."

So when the iPhone is mounted in the headset, the audio can be automatically routed to that headset's speakers. When it isn't, the iPhone's speakers are used instead.

Another patent application, 20190285897, suggests that Apple may be working on a higher-end AR headset that features holographic elements. The patent describes a complex system for displaying an image for the user in an AR headset. 

Rather than using a display showing a composite view of the environment and virtual content, Apple proposes the use of a "reflective holographic combiner" to serve the same purpose, reflecting light for the AR elements while at the same time passing through environmental light.

The benefits of a holographic system are many, but the most significant improvement would likely be the reduction of unpleasant side effects. Both VR and AR have been shown to cause eye strain, headaches, and occasionally nausea. 

The holographic elements would allow Apple to correct for depth, seamlessly integrating the projections into a user's field of view. By providing integration, it would help to prevent the usual level of discomfort wearers feel when using similar headsets. 

A patent suggests that instead of using a controller or gloves, users could pan around their AR environment using a trackpad.

Apple is investigating the use of multiple holographic elements projected onto the lens for displaying information to the wearer. This system would allow for much smaller and lighter glasses and keep the resolution of the images seen as high as possible.

The AR glasses may not launch until sometime in 2022, at the earliest. At least that is according to long time analyst Ming-Chi Kuo. A contradicting report from Jon Prosser claims that Kuo is wrong, stating that the glasses would ship in mid-2021, and that he's seen them.

Prosser continued leaking information about the upcoming wearable, citing their name as "Apple Glass" with a late 2021 launch for $499. The glasses would take prescription lenses and have no cameras attached, just LiDAR.

Apple's iRing

Patent mockup of a smart ring with camera Patent mockup of a smart ring with a camera

Patent application number 20190346938 covers a "finger-mounted device with fabric."

Previous patents have shown Apple describing various systems using gloves or gloves with rings on several fingers to detect motion; this filing proposes that one smart ring could provide the necessary motion information.

"The use of wearable devices to gather input for controlling electronic equipment can pose challenges," says this patent. "If care is not taken, a device such as a glove may affect the ability of a user to feel objects in the user's surroundings, may be uncomfortable to use, or may not gather suitable input from the user."

The ring could contain "components such as force sensors, accelerometers, and other sensors and for haptic output devices."

"During operation, a user may wear the finger mounted units on the tips of the user's fingers while interacting with external objects," it continues.

AR is becoming more ubiquitous with the inclusion of a LiDAR system on the new iPad Pro, and even audio AR with AirPods Pro noise cancellation and transparency mode. Expect more devices to bring better-augmented reality experiences soon.

 

Apple AR Related Stories