As Apple Vision Pro approaches its second birthday, there's still not much Apple Immersive Content available for the platform. That's set to change as the production pipeline has finally solidified for creators.
Apple Immersive Video is a new format used for those 8K short films Apple Vision Pro users can watch in the Apple TV app. They're usually between 10 and 20 minutes in length and a quite stunning showcase of the technology in use.
However, until now, there's been little word of how anyone besides Apple can actually produce such media. A Medium post from Anthony Maes shared the details of a 10-hour Create Immersive event held by Apple that was also shared to the Apple Developer YouTube channel (in two parts).
The event covered a lot of detail around what Apple Vision Pro's technology stack provided to users from a developer standpoint. It explained, again, what Windows, Volumes, and Spaces are, as well as details on immersive audio and Personas.
While the sessions were primarily focused on apps for the first half of the presentation, it also spent a lot of time on implementing support for new video types supported by visionOS, like 180-degree video. The second half focused on developing immersive video for the platform and the pipeline that now exists to do so.
There's a lot of inside baseball on formats and implementation I won't dive into here. The Medium post covers plenty, so I'll focus on what this event means for Apple Vision Pro users.
Third-party immersive video is finally coming
There has always been plenty of 3D video and legacy VR content to view on Apple Vision Pro, especially in streaming apps like Apple TV and Disney+. However, the new format, Apple Immersive Video, was a format that was seemingly exclusive to Apple from its inception to today.
Apple Immersive Video has only been made by Apple so far
What the Create Immersive event revealed is that this wasn't an intentional limitation — there was just a lack of a pipeline for capturing, editing, and distributing such content. So, unless you were Apple working with Blackmagic to create Apple Immersive Video, you weren't making it, period.
Apple was capturing Immersive Video using pre-release Blackmagic cameras or retrofit rigs alongside custom, sometimes still-in-beta software to get everything working. The end result was a beautiful video users could view on Apple Vision Pro, but the process was held together by duct tape and not ready for wider distribution.
That changes now, as the Blackmagic cameras are now widely available and are being shipped to studios. Of course, these are $33,000 cameras, so the average Joe won't be making Apple Immersive Video in their backyard.
The hardware is only half the story, of course. Apple also announced that a software production and distribution pipeline was also forming. It discussed software like DaVinci Resolve Studio, Colorfront, SpatialGen, and more getting support for editing and viewing the immersive video.
Apple also shared how hosting and streaming Immersive Video is possible. It also shared that Vimeo was bringing the ability to upload Apple Immersive Video as well.
Some technical details
I mentioned in my Apple Vision Pro with M5 review that Apple Immersive Video content, 3D video, and other media wouldn't appear any sharper with the new hardware. Apple was already performing tricks to ensure users got the most of the media, and the details were revealed during the Create Immersive event.
Apple built the Immersive Video format to mimic human eyesight
Apple Immersive Video uses something called Static Foveation, which takes what the camera captures, 8160 pixels by 7200 pixels per eye, and converts it to a streamable 4320 pixels by 4320 pixels per eye. The fisheye projection is stretched across virtual rounded displays to preserve the pixels per degree of the original video, ensuring no video is warped, even at the edges.
Producing a film in this format isn't as simple as getting the expensive camera and editing software. Details were shared about how some of the existing Immersive Video was made, and it required extensive planning and analysis of scenes.
Actors had to understand everything about what was expected of them, backgrounds had to be polished knowing that users could glance over at them during a scene, and audio had to be built to ensure immersion was kept throughout.
Consumer-grade tech will eventually emerge that will let users create similar experiences, but it could take years before anything of the sort emerges. In the meantime, we have to trust that the professionals and indie studios can make interesting content for us to watch.
A flood of content is coming
The 10 hours of video and extensive analysis in the Medium post show Apple is trying to create an incredible format. While it only has a home on an expensive and niche headset, it is clear Apple wants content made regardless.
3D movies are great on Apple Vision Pro, but Apple Immersive Video is incredible
While it isn't backing up truckloads of cash to convince content creators to flock to the platform, it doesn't really need to. Apple Immersive Video is, at its most basic, interesting as a format, and that'll drive a lot of content on its own.
The lack of cameras to rent, software to process recorded video, and lack of distribution options highly limited what was possible before. Now, we're seeing the signs of what it means when all of this is out in the wild.
Apple is already making moves in the space thanks to the availability of cameras — NBA immersive streams of select Lakers games. There's a sure chance we'll see Immersive Video of MLB recaps, F1 races, and MLS matches in the future as well.
We're still in the experimental phase of Apple Immersive Video. The short film Submerged was a showcase of the challenges of creating a narrative in an entirely new format where the camera was the viewer's head.
Apple Immersive Video is still only available on the niche Apple Vision Pro
While independent studios and companies like Disney work on Immersive Video experiences, we can only hope Apple's production of in-house content accelerates. We've yet to see anything beyond short films, so a step into some kind of scripted show or feature-length film has to be the next step for the format.
Sure, the viewer might need to take a break after viewing a 30-minute TV show or pause intermittently during a 2-hour film, but these kinds of media should be the goal. We can already watch standard 2D film and TV in Apple Vision Pro, so it is only a matter of time before the Apple Immersive Video version of that content arrives.
Apple has shown that it has put a significant effort into developing the Apple Immersive Video pipeline from production to distribution. It has shown similar resolve in developing apps and games for visionOS from a technical perspective — now if only they'd bother following through with more native apps and games.
In the end, Apple still needs to do better on the Apple Vision Pro evangelism side of things. It can build pipelines that result in beautiful films, apps, and games, but if no one bothers building them, then there's no point.
