Apple's AR headset: Rounding up the rumors about Apple's smart glasses

Earlier today, Bloomberg claimed Apple was still working on an augmented reality headset, codenamed "T288," which would be a standalone unit with its own display and processor, rather that relying on a smartphone or other device for processing and powering its main features.

Said to run on its own "rOS" platform, it is thought Apple is pursuing a "very aggressive" timeline that would develop the required technology by 2019 ready for a market release in 2020. Looking back over earlier reports, it seems Apple has already put in a considerable amount of work into the project already, with some aspects dating back to 2010.

Patents and acquisitions

A patent granted to Apple in 2014 for a "Transparent Electronic Device" effectively describes a display with one or more screens that could be used to overlay data within a visible window. The combination of virtual imagery and a real-world view offers the opportunity to provide an AR experience, with data overlaying objects in a user's vision, such as information about an exhibit at a museum.

Though the patent, which was initially submitted to the USPTO on January 2010, could be intended for glasses, Apple also gave the example of using the technology on the windows of a tour bus. This could allow passengers on a tour to get more information about locations passing it by.

Apple has also acquired and applied for various patents that would also apply to any Apple AR projects, including a number relating to "Augmented Reality Maps," as well as one for a "Method and device for illustrating a virtual object in a real environment. While most of the patents aim more towards AR from a mobile device, like an iPhone or iPad, it is equally likely that they could be implemented in a headset-based system.

Acquisitions in the virtual and augmented reality field have also helped Apple make its ARKit an extremely powerful platform, notably including Emotient, Faceshift, and PrimeSense relating to the iPhone X's front-facing camera, Face ID, and its face-based AR features. The German AR firm Metaio was picked up in 2015, with its technology at the time somewhat similar to what ARKit currently offers its developers.

Focusing on headsets specifically, the acquisition of note has to be SensoMotoric Instruments, another German hardware producer involved in AR bought this year, but one aimed more at tracking eye movements. At the time of purchase, the firm's technology was used for a variety of medical applications as well as AR, including the early detection of autism in children, brain mapping, and neurology.

The key component of interest to Apple is the eye tracking technology, which can monitor the wearer's gaze 120 times per second in real-time. Aside from giving applications an accurate and up-to-date measurement of the user's gaze, the system has the potential to reduce input lag, in turn minimizing the effects of motion sickness caused through any AR imagery failing to match up to shifts in the user's perspective.

On a head-mounted system where the screen is much closer to the eyes than current generation VR headsets, the need to match up to the user's perspective is even more vital.

Analyst predictions and media reports

Outside of acquisitions and patents, there have been some third-party reports making various claims about Apple's AR headset progress in recent years, albeit with varying levels of believability.

In March 2015, Piper Jaffray analyst Gene Munster published a report suggesting Apple had fielded a small team tasked with experimenting in the AR space. Suggesting consumer-oriented AR systems were ten years away, Munster was still bullish on Apple's work in the field, suggesting it could be as transformative to technology as the introduction of the smartphone.

Even this early, Munster mused that Apple's "evolving fashion advantage" would give it an edge over other AR systems, by creating a device that consumers would want to wear. While Google Glass is a good example of how data could be provided to the user in an AR-style way, the device was panned by critics due to how they look when worn, a mistake Apple is likely keen to avoid.

AR and VR evangelist Robert Scoble made claims in January 2017 that Apple could debut mixed reality glasses in 2018, and is partnering with optics producer Carl Zeiss to create lightweight glasses. Scoble cited Zeiss' lack of AR and VR products at its CES 2017 booth as evidence, despite Zeiss' work on the VR One Plus headset, suggesting Apple had muzzled the firm until the project ends or related hardware is released.

Bloomberg's March report suggests Apple's AR team totals "hundreds" of employees, including engineers from Dolby, Meta, Oculus, and Microsoft, though only part is working on the glasses project. While some are said to be specialists in cameras and optical lenses, some of the team are tasked with sourcing materials for the glasses.

A similar report from the Financial Times at the same time made the claim AR had overtaken the "Project Titan" self-driving car initiative as Apple's next big priority, aside from the iPhone. Citing sources familiar with Apple's plans, the report mused that a retail launch was at least a year away, but that Apple was allocating more resources to "step up its efforts" in AR headset development.

A Reddit poster by the name of "Foxconninsider" made a number of claims about Apple's upcoming hardware in June this year, albeit with some legitimacy issues over some of the presented information. Among the user's claims was for smart glasses called "Project Mirrorshades," which was said to be "very likely to be cancelled" with a 65 percent probability of failure.

Aside from issues relating to an earlier Reddit post suggesting the Sonder keyboard was an Apple prototype, it is worth taking the claims of "Foxconninsider" with a pinch of salt. The other pre-WWDC claims made about the rest of Apple's product line were found to be largely inaccurate by the time Apple's developer event rolled around.

Said to cost $150 in materials to construct and supposedly expected to be sold for four times the construction price, the glasses are claimed to consist of a acetate frame mounting prescription or polarized lenses with Zeiss smart optics. Audio was said to be delivered by bone conduction, complete with noise cancellation modules, with the list of onboard sensors said to include a light sensor, an accelerometer, and a magnetometer for navigation.

In August, a second Financial Times claimed AR glasses had become "a particular area of experimentation" within the company, with various styles and types of glasses being worked on. A team of engineers had apparently advocated for a pair of glasses with 3D cameras but no screens, relying on an iPhone as the primary display, a somewhat similar system to Samsung's Gear VR headset by description, while others opted for designs where screens are integrated into the glasses.

Work-related AR injuries

While speculation certainly suggests there to be work being carried out at Apple relating to AR hardware, Apple has officially stayed as quiet as usual about its future endeavors. There has however been one leak indicating that work on a headset-style product is still taking place.

An incident report from an Apple Environment Health and Safety contractor was reportedly leaked to hundreds of Apple staff earlier this year, describing an event on Feb. 21 at Apple's De Anza, Cupertino office requiring "medical treatment beyond first aid." Reports about the leak mentions how a user of a device "experienced discomfort in her eye" and was able to see a "laser flash" several times during a study, before being referred to an optometrist.

A later incident at Apple's Vallco Parkway office in Cupertino on March 2 may be for a similar piece of hardware, with the employee reporting "eye pain after working with new prototype, though it may be associated with use," the document states.

Though it is not mentioned if the hardware in question relates to AR or VR, it does at least confirm some work in the field is happening behind Apple's closed doors.

The official line

Up until the announcement of ARKit, Apple had kept relatively quiet about its AR projects, but now with developers using Apple's framework in iOS apps, the company has opened up about the potential of AR itself. As usual, Apple's policy of refusing to talk about future product plans means it is still keeping quiet about all of its AR hardware efforts.

When asked about the possibility of Apple-branded AR glasses or goggles in October, CEO Tim Cook kept to the company line about any possible hardware development, but did suggest such technology "doesn't exist to do that in a quality way."

"The display technology required, as well as putting enough stuff around your face - there's huge challenges with that," Cook said in an interview. Technical challenges, such as optics providing an adequate field of view and "the display itself," hold the concept back from being an available consumer product.

"We don' give a rats' about being first, we want to be the best, and give people a great experience. But now anything you would see on the market any time soon would not be something any of us would be satisfied with," said Cook. "Nor do I think the vast majority of people would be satisfied.