Last updated: 1 month ago
Apple is rumored to be working on a few alternative-reality devices, including an AR headset, AR glasses, and a colloquially known "iRing." While there is little concrete evidence of any specific devices being created, the newly introduced ARKit 3, as well as several patents released in 2019, suggest Apple will likely step into the AR hardware arena before long -- perhaps as early as 2022.
● Rumored 2022 release
● People occlusion
● Holographic elements to correct for field of view
● May utilize iPhone or iPhone-like screen
● iRing may provide movement information to device
What is augmented reality?
Augmented reality – often stylized as AR – is the superimposing of a computer-generated image into the "real world," usually via a smartphone camera and the screen. People often compare AR to VR, or virtual reality. Virtual reality, however, creates a fully simulated environment that a user can "step into."
Current examples of augmented reality are games like Pokemon Go, which includes an AR mode that allows users to hunt, photograph, and catch Pokemon via their iPhone camera. Social media platforms such as Instagram and Snapchat use augmented reality filters to encourage users to share pictures and videos. Apps like Measure are also AR based apps, and some retailers' apps allow users to preview AR items in their homes.
Currently available AR hardware includes Magic Leap One and Microsoft's HoloLens. These headsets, while fairly large and conspicuous, convincingly place rendered 3D objects and animations in the user's environment. The user sees their real environment through a transparent visor, which also houses a stereoscopic display that creates the illusions. Apart from the form factor, this approach is most similar to Apple's rumored glasses or headset.
Introduced at WWDC 2017, ARKit is Apple's software development kit that enables app developers to incorporate augmented reality into their apps. ARKit handles many of the tough tasks associated with AR, including detecting movement and the local environment, simplifying the process for developers to place virtual objects in an everyday scene.
In iOS 13, ARKit 3 was released. ARKit 3 received many notable upgrades over its predecessor. New features included the introduction of people occlusion, or the ability to place objects behind or in front of users as needed. It also enabled more face track multiple faces, and the ability to place digital objects into complex environments.
Apple's possible AR headset or glasses
Apple is reportedly working on an augmented reality headset or smart glasses. The rumors, which have persisted for years, suggest Apple is sinking considerable resources into the fields of AR and VR. While Apple hasn't revealed any public details of when to expect AR-equipped head-mounted hardware, it is allegedly offering up timelines to employees.
One patent application, 20190285897, suggests that Apple may be working on a higher-end AR headset with holographic elements. The patent describes a complex system for displaying an image for the user in an AR headset.
Rather than using a display showing a composite view of the environment and virtual content, Apple proposes the use of a "reflective holographic combiner" to serve the same purpose, reflecting light for the AR elements while at the same time passing through environmental light.
The benefits of a holographic system are many, but the most significant improvement would likely be the reduction of unpleasant side effects. Both VR and AR have been shown to cause eye strain, headaches, and occasionally nausea.
The holographic elements would allow Apple to correct for depth, seamlessly integrating the projections into a user's field of view. By providing integration, it would help to prevent the usual level of discomfort wearers feel when using similar headsets.
A patent suggests that instead of using a controller or gloves, users could pan around their AR environment using a trackpad.
Apple is investigating the use of multiple holographic elements projected onto the lens for displaying information to the wearer. This system would allow for much smaller and lighter glasses and keep the resolution of the images seen as high as possible.
The AR glasses may not launch until sometime in 2022, at the earliest. At least that is according to long time analyst Ming-Chi Kuo. A contradicting report from Jon Prosser claims that Kuo is wrong, stating that the glasses would ship in mid-2021, and that he's seen them.
Other Apple patents have surfaced as well, including one describing an AR headset having a "device-mount audio mode." The design suggests a headset with another device like an iPhone mounted in front of the user's eyes. Similar VR headsets have existed for some time, with Samsung Gear and Google Cardboard being notable examples.
"[The headset can be] configured to: determine whether the mobile device is mounted on a head-mounted display (HMD) mount," says the patent, "[then] transmit an audio signal from the device processor to a wireless headphone; and change an audio mode of one or more of the mobile device or the wireless headphone."
So, when the iPhone is mounted in the headset, the audio can be automatically routed to that headset's speakers. When it isn't, the iPhone's speakers are used instead.
Prosser continued leaking information about an upcoming Apple AR wearable, citing the name as "Apple Glass" with a late 2021 launch for $499. The glasses would take prescription lenses and have no cameras attached, just LiDAR.
Apple may be seeking Sony out as a supplier of micro-OLED displays for use in a heads-up-display. The 1280 x 960 half-inch displays would be perfect for a VR or mixed reality headset. Some suggest they may be used for "Apple Glass," though they would need transparent glass displays.
Apple is also rumored to be working on a smart ring of some sort that can serve as an input device for an AR headset.
The company's patent application number 20190346938 covers a "finger-mounted device with fabric." Previous patents have shown Apple describing various systems using gloves or gloves with rings on several fingers to detect motion; this filing proposes that one smart ring could provide the necessary motion information.
"The use of wearable devices to gather input for controlling electronic equipment can pose challenges," says this patent. "If care is not taken, a device such as a glove may affect the ability of a user to feel objects in the user's surroundings, may be uncomfortable to use, or may not gather suitable input from the user."
The ring could contain "components such as force sensors, accelerometers, and other sensors and for haptic output devices."
"During operation, a user may wear the finger mounted units on the tips of the user's fingers while interacting with external objects," it continues.
AR is becoming more ubiquitous with the inclusion of a LiDAR system on the new iPad Pro, iPhone 12 Pro, and iPhone 12 Pro Max. You could even consider AirPods Pro's noise cancellation, transparency mode, and spatial audio to be audio AR. Expect more devices to bring better-augmented reality experiences soon.