Apple Vision Pro

Apple has been rumored to be working on some kind of Apple VR headset for years, and it's finally here. Though the Apple Vision Pro isn't quite VR, instead, it is mixed reality with AR and VR applications.

The headset and its operating system visionOS are referred to by Apple as spatial computing. They blend software with the world around the user via video passthrough.

Apple Vision Pro is a headset that obscures the user's vision, not a pair of transparent goggles or glasses. But high-quality cameras pass 3D video of the environment to the user through a pair of pixel-dense displays.

It is seen as the first step into an entirely new kind of computing that could lead to the long-rumored Apple Glass — AR glasses. Even this headset, with its "pro" moniker, is likely just a stepping stone to a more budget-friendly Apple Vision model down the line.

Rumors are already building about future models. Apple could be preparing four new models with some focused on lowering the price, but may not release until 2026.

Apple Vision Pro design

The ski goggle-like design that's been rumored since early 2021 ended up being the final product. A curved piece of glass acts as a lens for the multiple cameras, while the exterior case is aluminum.

The Solo Knit Band is 3D-knitted as a single piece, easily adjusted with a fit dial, and attached using a pull-tab for easy removal. It provides a snug fit, but leaves most of the headset weight on the wearer's face.

The headset is so form-fitting that users can't wear glasses, unlike PlayStation VR 2. Apple partnered with Zeiss to offer prescription or reader lenses that are available as a separate $150 or $100 purchase.

The sides of Apple Vision Pro house small speakers called Audio Pods that point spatial audio into the user's ears. The top has a button for controlling certain features like the 3D camera, and the Digital Crown on the right acts as a Home Button and controls immersion.

A power cable attaches to the side via a proprietary mechanism. It is attached to an external battery pack that enables around 2 hours of portable use, but users can attach the battery to power if needed.

If the battery dies, the entire headset shuts down, as no backup power is available. When Apple Vision Pro is shut down, window placement is forgotten and has to be reset.

The front glass acts as an external display for a feature called eyeSight. It enables users to indicate if they are paying attention or not via an admittedly odd effect.

When the user is fully immersed and can't see the outside world, a colorful waveform appears. If passthrough is active, the user's eyes are visible through a lenticular display. Nothing is shown on the external display if a person isn't looking at it.

eyeSight was designed to ensure that users could feel confident in using the headset in any environment. However, it seems some social hurdles will have to be overcome before such technology is commonplace during social interactions.

Since eyeSight relies on a user's Persona, it has the same uncanny valley effect. Personas are still in beta, so odd bugs appear that make user's eyes look shut or distorted.

Apple Vision Pro technology

Apple's headset is a standalone device running on an M2 processor and R1 co-processor. The dual-chip design enables spatial experiences — M2 processing visionOS and graphics, while R1 is dedicated to processing camera input, sensors, and microphones.

A reference found in Xcode suggests the Apple Vision Pro has 16GB of RAM, which is the same amount in developer kits. Customers can configure their devices with 256GB, 512GB, or 1TB of storage.

There is a 12-millisecond lag from camera to display, which users shouldn't be able to notice. The passthrough works best in bright environments, but it is done well enough that depth perception isn't an issue.

An eye-tracking system of LEDs and infrared cameras enables precise controls just by looking. Users can glance at a text input field and then speak to fill it in.

There are 12 cameras, five sensors, and six microphones in the aluminum and glass enclosure. This all adds up to a heavier-than-average product when compared to competing VR headsets.

Two high-resolution cameras transmit over one billion pixels per second to the displays. Each display has a 3,660 pixels by 3,200 pixels resolution, or about 3,380 pixels per inch.

The display system uses a micro-OLED backplane with pixels seven and a half microns wide, allowing for it to fit about 50 pixels in the same space as a single iPhone pixel. A custom three-element lens was designed to magnify the screen and make it wrap around as wide as possible for the user's vision.

LiDAR and a TrueDepth camera work together to create a fused 3D map of the environment. Infrared flood illuminators work with other sensors to enhance hand tracking in low light.

There is a fan keeping the processors cool and vent holes in the top and bottom of the headset. While certain spots on the Apple Vision Pro exterior might feel warm during use, it's never hot on the user's face thanks to separation by the Light Seal.

Since Apple Vision Pro has multiple cameras, it can capture "spatial" 3D content. Recorded spatial video can be viewed in the Photos app on an Apple Vision Pro headset. Over 150 3D movies were available within the Apple TV app at launch.

Apple added basic 3D video recording to the iPhone 15 Pro with iOS 17. It can capture 1080p 30Hz spatial video that can be played back within the headset natively.

Spatial video captured on Apple Vision Pro has more depth, but spatial video captured on iPhone is more crisp. Either way, future hardware will make today's videos look terrible by comparison.

Optic ID, privacy, and security

For security, Apple introduces Optic ID as an iris-scanning system. Much like Face ID or Touch ID, it will be used to authenticate the user, enable Apple Pay purchases, and other security-related elements.

Apple explains that Optic ID uses invisible LED light sources to analyze the iris, which is then compared to the enrolled data stored on the Secure Enclave, which is similar to the process used for Face ID and Touch ID. The Optic ID data is fully encrypted, not provided to apps, and never leaves the device itself.

Continuing the privacy theme, the areas where the user looks on the display and eye tracking data aren't shared with Apple or websites. Instead, data from sensors and cameras are processed at a system level so that apps don't necessarily need to "see" the user or their surroundings to provide spatial experiences.

Apple VR, AR, or MR

Apple avoided calling any experience offered by Apple Vision Pro a conventional name like "Apple VR." Instead, it stuck to its own marketing terms like Spatial Computing.

There has been some contention on what to call this headset. Some refer to it as Apple VR since users put on a headset and are shown content. Others call it Apple AR with options for full immersion thanks to the Digital Crown.

Even the term Mixed Reality (MR) doesn't quite fit. Even if it is just marketing, Spatial Computing describes the experience well.

Think of it this way — Oculus and PlayStation VR are easily classified as VR headsets since virtual reality experiences are their primary function. For Apple Vision Pro, VR is just a feature, not the entire purpose.

During the keynote and developer sessions, Apple used in-house marketing terms to refer to each experience. It calls 2D apps "windows," 3D objects "volumes," and fully immersive VR "spaces."

visionOS

The software is run via visionOS with apps from iPhone and iPad able to operate with little to no developer intervention. Compatible apps shown in the 3D space are still 2D windows that constrains content within that space.

Native apps take on 3D effects and developers can add 3D objects around a window called ornaments that enhance interactions. Menus and controls appear to pop in front of or beside the app.

Objects created in the USDZ format can be pulled into a space, but it seems developers will need to tool unique experiences for visionOS using specific frameworks. Apps previously built with ARKit will show up as a 2D window.

Full Apple VR experiences weren't the focal point of the keynote, but they do exist. Apple plans to film multiple 360-degree videos for users to become immersed in, not to mention third-party VR apps.

The visionOS SDK was provided to developers in late June, 2023. It included a simulator to interact with software in a testing environment, but full-on-device testing wasn't possible until hardware kits and sessions arrived in July.

Many developers opted out of including their existing iPad app in the visionOS app store at launch. The lack of access to hardware to test the apps on made it difficult to know if the apps would be usable or lead to customer complaints.

Only about 600 native applications were available at launch on February 2, 2024. That number is increasing rapidly as developers finally get their hands on the hardware.

Spatial Computing competition

Apple may have only just announced its Spatial Computing platform, but that doesn't mean it's the only one on the market. Competitors have been exploring AR and other extended-reality solutions for over a decade.

Google has abandoned Glass, and Microsoft has moved on from Hololens, but other options still exist. Xreal Air are among the few products attempting to provide AR today, despite the limited technology in play at the consumer level.

Using Apple Vision Pro (pre-release)

Apple is providing developer labs and kits to help with the pre-release development of apps for the Spatial Computing platform. This has allowed some first-hand experience beyond Apple's initial WWDC announcement to make it out to the public.

Overall, those who have interacted with the headset come away with similar ideas and feelings. This is a first-generation product that is entering a crowded space but also stands apart as a mix of AR and VR with high-end technology making things feel effortless.

Users familiar with the competitor's headsets say that Apple Vision Pro has a wider and taller field of view. Certain lighting environments, like super bright rooms or dim areas, show the limitations of the video feed, but it isn't distracting.

Eye tracking and gesture controls seem to be excellent, even in pre-release demos. These systems feel well thought out and executed and are missing some of the usual growing pains new interaction paradigms tend to have.

Dedicated apps seem to work well and blend in with the environment, giving enterprise developers an excellent option for job or maintenance training. And while Apple says iPad apps can be ported with little effort, developers will need to give some attention to their apps to ensure legibility and controls translate to the platform.

Weight wasn't an issue, though after over an hour of constant use, fatigue did start to set in. Eye strain didn't seem to be an issue, given the quality of the displays.

Apple Vision Pro release date and price

Apple tends to announce products shortly before they are released, but since Apple Vision Pro is a new platform, it gave itself plenty of space. Developers were given about a year to work on their visionOS experiences, and developer kits are available with stringent secrecy protocols via a request application.

Estimates place the first year of shipments between 400,000 and 1 million units. The high price may prove prohibitive, but ambitious supply chain numbers indicate 10 million shipped in three years.

Apple Vision Pro was available to pre-order on January 19 starting at $3,499 with 256GB of storage and shipped February 2, 2024. It is bundled with two head straps, light shields, and the battery. The optical inserts are available online-only at $99 for reader-strength lenses and $149 for prescription-strength lenses.