What's Apple's Vision Pro killer app?

When Apple pulled back the curtains at WWDC to show off its new Vision Pro hardware, the responses were so predictable that a middling AI bot could have written them out in advance.

From the Ballmeresque, pearl clutching of "but it's so expensive" to the defeatist handwringing of analysts who shared their concerns of how everyone's already tried everything in the VR product space and failed, through "everyone knows Apple can't innovate, and this other company already has something we should be talking about instead," the tech media's collective manufactured sausage-grinding has offered little more than a reheated response of we were last fed at the debut of iPod, iPhone, iPad, Apple Watch, and so on.

So perhaps, if these concerns have all ready been voiced to oblivion, they've also already been answered in a way that can offer us some insight into how this particular wrinkle of consumer tech will unfold for Apple, beginning in the upcoming year.

This all happened before

The core tenants of how to blow out a blockbuster consumer tech product have been on public display dating back to the early year's of Apple's founding. As I detailed in the previous article, looking at how Apple has so consistently hit bangers out of the ballpark in a way that's changed how the world works, this firstly involves studying the status quo market and building a product that delivers a complete, even if flawed, solution to a need; then effectively communicating this value proposition to a large enough audience, while working with partners to deliver even wider options through software; and then operationally delivering the solution at scale with enough profitability to maintain the momentum.

In fewer words: "delivering a cohesive, valuable experience that Apple's customers will want to buy, and that Apple's partners will want to add value to."

Thus the question is set up. What specific sort of valuable experience can Vision Pro deliver that will bedazzle us to the point where enough of us not only spend serious coin to get it, but also that we collectively as a society will decide that this is something we want to be paying for on an ongoing basis, to enable an entirely new ecosystem of immersive applications that all need to be developed and maintained?

This is a preposterously high bar.

Apple isn't just an artist that cranks out monumental, standalone works that attempt to "express an idea." For an Apple product to "work" and be considered minimally successful, it has to actually deliver a life changing level of value across a very large audience willing to pay actual money for it, and it has to pique the curiosity of third party thinkers to influence them to pour their lives into building new forms of interconnected utility or convenience or entertainment to add value to what's already there.

As it did with the original Apple II, the Mac, iPod, iPhone, iPad, Apple Watch and so on, Apple is launching not just a new hardware product for sale, but is also putting together an ecosystem of added value. This distinguishes Apple from other makers of products that largely provide a self-contained unit of value.

If you buy a fancy refrigerator, or a pair of shoes, or a car, or a potato peeler, much simpler laws of supply and demand determine if you get a good deal at an acceptable price and if the maker can continue their production goals and if the product is a hit or a flop.

With Vision Pro, Apple is launching an entire platform based on the concept that if there's software, the real end value will be astronomically higher than if its just a one trick pony that does a particular job, such as playing immersive games. The fun thing about Apple is that the company is remarkably good at successfully launching platforms, in a world where potential platforms of any kind are complicated, tricky, and littered with the bodies of failures.

Apple is good at making difficult look easy.

Bigger than big

As is generally the case, Apple is not the first big mover in this emerging conceptual space of "immersive computing." Other vast tech titans have already spent enormous amounts of money to develop something seemingly comparable.

Notably, almost seven years ago Microsoft floated its HoloLens helmet, which on the surface seems to share some things in common with Vision Pro, including its $3,000 starting price tier. Yet since HoloLens first appeared as a "Development Edition" back in early 2016, and despite being subsidized by American government contractors as interesting technology it hasn't changed how the world works in any appreciable significant way. Certainly not for consumers.

Three incremental generations in and billions of dollars under the bridge, HoloLens is an Intel-based Windows PC with custom graphics hardware to overlay projected images on top of what you're looking at. It's best shot at producing a "killer app" to drive sales was immersive learning, with the U.S. military making a large purchase for use in training as a quite literal killer app.

That idea was so disturbing to the Holo development team that a swath of the people who worked on it made a public protest that they were upset to see their efforts being used for such an application of "war and oppression."

Microsoft has a long and storied history of finding ways to repose its Windows PC to sit as a placeholder in various product categories, from a phone to a music player to a tablet to a watch. Yet all of these efforts ended up being busywork that didn't really change the world or even sell in commercially significant or successful quantities.

As the "opposite of Apple" in the consumer space, Microsoft can only shuffle around and recommend its one PC hit as a solution for new markets, but can't quite manage to actually pull this off. Certainly not as successfully as Apple, despite having blown so much money so many times in various attempts aimed at the same market.

HoloLens does however demonstrate that even a company with a vast ecosystem of Windows developers and gads of money can't figure out how to position an "immersive computing" product in a way that would entice anyone to buy it, even when defining the target audience as being limited to enterprise users.

But Microsoft was also a delusional mess in smartphones, in tablets, music players, watches, and in other areas that both preceded Apple's debut in those markets — and in some cases attempted to turn around and copy Apple's delivered success — all without ever really achieving any of its own. So the flop of HoloLens can't really provide any evidence that the public isn't potentially interested in immersive computing.

It only proves that big PC licensing companies with a lot of money can't figure out how to do the things that Apple can, with the seemingly effortless panache of a $3 trillion dollar, scrappy startup of 40 years.

More than money

Another bag of money, this time from surveillance advertising, threw an outlandish amount of possibility at the goal of beating Apple into the world of immersive computing. Well, two. First was Google with its characteristic approach of half-assedly floating out some partially done work with the expectation that third parties would jump in and do all the heavy lifting needed to deliver a successful product, and that doe-eyed fanboys would go out and evangelize the public in why anyone wanted to buy this, largely based on the idea that it has a low price subsidized by all the surveillance advertising and spyware that covers some of the bill while converting the user into a product to be shaken down for their behavioral data.

Remember Cardboard? Google's VR goggle efforts were largely oriented around attaching an Android phone to your face. Importantly, neither Google nor its partners nor its fans really put enough effort into doing their jobs, so nothing ever materialized apart from a lot of people wasting some their time to look like clowns.

But that also doesn't prove there isn't a killer app in immersive computing. It just means that a major new category shift isn't easy to pull off in your spare time without much effort.

Google's repeated attempts at doing a lot — but not enough — of the critical work needed to build out a complete thought have all contributed to a graveyard of failed alphas and betas.

Facebook, the other surveillance advertising bag of money, took the most reasonable shortcut in acquiring Oculus' existing work rather than building its own VR helmet from scratch. Yet despite taking somebody else's already finished product and throwing huge sums at building a "meta world" as a differentiating killer app to drive adoption, Facebook is struggling to see real enthusiasm from consumer audiences.

This parallels Facebook's similar efforts in delivering a Facebook phone, the thing that was supposed to derail iPhone at multiple times.

Nothing could be more "meta" than trying to use marketing to concoct a gigantic, desperate platform of "metaverse" to add nuance to something where the only real value was what you know about something. Yet nothing of Facebook's Meta really delivers an attractive, sustainable, competitive framework of value that enough consumers will continue buy in a cycle.

Meta is effectively trying to sell Quest VR as a technology, a fun quirk of a thing that could be used to do something fun, but if only it were better and only if it could do something justifiably useful.

Not just a Game

Like Meta's Quest, Sony's PlayStation VR, the HTC Vibe and others, much of the existing Virtual Reality headset space has centered on VR gaming as its killer app. And certainly, people who want to play games have represented the majority of interest in the immersion space. What else of commercial value could one accomplish, beyond training soldiers or technicians remotely, other than offer some sort of surround gaming experience?

Conversely, in the space of gaming based VR, goggle prices begin around $350-500, with Meta's higher end Pro model asking around $1,200. These prices can deliver a fun gaming experience, but are selling to a niche market: single digit millions. The lower prices are trying to reach a larger market, but that also prevents them from delivering core utility that can do much beyond basic gaming. How would someone get their company to buy a VR gaming toy for them?

Today's VR helmets are a bit like 3DTV from a few years ago. They are fun to try for about 15 minutes, and appeal to certain small groups of technology enthusiasts. No amount of "journalistic" handwaving or other public relations activity has managed to get consumers to really enjoy 3DTV enough to drive sales of 3DTV sets in any sort of commercially sustainable fashion.

I once bought a 3DTV and some pairs of active LCD glasses just to frivolously experience the future promised by the big companies who were assuring us that we really wanted this latest fad and how this new technology would change how we experience things. I have to say — it was no Macintosh.

Even with some concerted efforts at trying to enjoy and appreciate all the work that went into delivering it, 3DTV ended up a passing fancy. Video games, movies, even realtime simulation of existing content in 3D never really found its footing as on experience that anyone decided they should pay for with any longevity. I struggled to ever watch more than half of a 3DTV movie before reverting to 2D.

After a fiercely relentless push, TV makers eventually threw in the towel on 3DTV, shortly after 3D effects had made their valiant but fleeting efforts on smartphones and handheld video game devices.

Like 3DTV — and the early smartphones of the early 2000s — today's VR helmets are sharply constrained in utility by the limits of their low prices, while at the same time, devices asking for more money can't woo enough market share to develop a higher quality tier of functionality to establish a sustainable market that could support dynamic ongoing development. There's simply not enough critical mass to ignite an ecosystem to support life.

Following the model of big box retailers, big box device makers threw out their best efforts and then effectively responded to disinterest shown by the market. Just as the paper thin 3DTV experience flopped out of the gate, they gave up. The mirage of dimension layered onto the conventional TV wasn't going to be the new thing that sold another generation of TVs.

Core utility in a vision

At the same time however, there is clearly some core utility in replicating a first hand visual experience. One of the most powerful, in person experiences to be created by civilization is the performance of live music and theater. Over the past century, the inherent entertainment value in live performances spawned the development of technology needed to record and deliver a reproduction of these experiences, both for the convenience and flexibility of delivering these to new and larger audiences, in other places and at other times.

Live music acts, theater, and vaudeville were first captured in audio recordings and rebroadcast as radio just over a century ago. The entire business of audio recording has been a rapidly evolving phenomenon up to today's reality of audio streaming, live FaceTime communications, and even Apple Watch's wearable Walkie-Talkie.

Apple only recently launched its immersive, surround sound spatial audio format, which quietly made a debut at WWDC before anyone seemed to anticipate the step arriving. For some time now, we've been able to deliver immersive, stereo audio to headphones in a way that makes you feel convincingly connected to a remote event, such as listening to a studio recording of a song. The evolution of teleporting a visual experience has been different and more complicated.

Apple uniquely saw the value in delivering a marketplace for recorded immersive spatial audio that can respond to head positioning, something that no other consumer tech makers were really focused on. Delivering a similarly immersive visual experience is more complicated.

Yet that's the core utility premise of Vision Pro: being able to experience immersive visuals. And not only see things, but be able to interact with what you're seeing in a human user interface presentation intuitively familiar to Mac, iPhone and iPad users.

We have two ears and two eyes, but the way our brains perceive audio and visual reality are different enough that it's quite a lot easier to deliver an immersive audio experience with headphones. Negotiating a way to not only present immersive video information to our eyes, but also sync that with head movement to create a convincing feeling of reality requires a significant level of infrastructure!

Apple didn't just focus on the concept of a wide, immersive field of view to put you close up and inside a video world, which is the minimum needed to deliver VR gaming. Vision Pro goes beyond that to deliver the kind of product Apple has become very successful in consistently producing: a bit mapped display tied to a consistent human user interface where targets are selected and manipulated as directly as possible.

That began with the Mac's mouse and later trackpads, then iPad's touchweel, iPhone's direct capacitive touch, and Apple Watch's Digital Crown and hand gestures. With Vision Pro, this interface expands to include eye tracking, allowing the user to interact with what they are looking at as effortlessly and intuitively as possible.

Headed to Spatial

Apple also focused Vision Pro on the shoulders of giant technologies it had uniquely brought to the mass market, including spatial audio and the work it did to deliver augmented reality on the iPhone screen: effectively linking a moving screen and camera inputs to a synthetically created computing interface. These deliver the Mac-like human interface tied to a virtual spot in front of the user, allowing it to float in tandem with their head movements.

It took Apple several years to develop AR features and the developer platform tools to support an AR ecosystem. It first positioned these as a cool feature on iPhones and iPad, but the real target was its eventual deployment on Vision Pro in an immersion environment.

Notably different from existing, purely VR helmets, Apple's AR approach allows the user to wear the device while not being isolated in their own darkened world of graphics. Rather than just looking at an eye level, immersive display, Vision Pro superimposes its graphical interface on top of what the user would be seeing without the helmet installed: a video feed of the outside world.

This dramatically changes the VR experience into one where the user isn't merely dropped into a solitary, enveloping world of fantasy, but is instead able to call up applications in front of them, and enjoy immersive audio and video experiences that they can dial themselves in and out of as they desire. Apple's recognition that users don't feel comfortable for extended sessions in a purely isolated VR world seems pretty essential to delivering a product one could wear in the workplace, or even at home, without fear of falling down or without needing full isolation from the inherent hazards of wearing a VR headset.

Further, the EyeSight feature Apple demonstrated— which displays an image of the wearer's eyes on the front of the headset— serves another important goal: it further integrates the wearer into their surroundings to other people in the room. Instead of being obscured and isolated inside of a VR realm, wearers are integrated into their surroundings, enabling the wearer to interact with others in the room and not trip over an end table or a child.

These important minimum capabilities of Vision Pro greatly expand its cost, making it significantly more expensive than gaming-centric VR rigs. But this core value and functionality contributes to making a system users can spend enough time using— and functionally using— greatly increasing its value proposition to the most valuable markets available.

Thus I think the killer app for Vision Pro will be a combination of its features: enabling virtual, facial appearance based communications rather than the cartoon avatars dreamed up in most VR, immersive enjoyment of recorded entertainment including virtually putting the user in an event, meeting or concert audience where they can actually participate in the experience both through AR inputs, and appearing to outsiders as an actual person, and an immersive desktop of familiar apps.

Skating to where the puck will be

If these things sound familiar, perhaps you are recalling Steve Jobs' 2007 introduction of the iPhone, which he unveiled as an "iPod with touch controls, a phone, and a breakthrough internet communications device," which we could paraphrase as "media, communications and networked apps." It wasn't just three devices, it was three killer apps, all in one device.

At the time, Jobs likely didn't fully appreciate exactly how users would embrace iPod features from iTunes; Apple Music hadn't yet made the huge shift toward streaming (popularized by Spotify) as market demand shifted away from individual song purchases. Apple hadn't even lined up movie rentals in iTunes yet, which Jobs would later debut in 2008. Today's media experience on iOS has dramatically changed from its humble origins.

Apple also didn't quite know how Messages would ultimately work on iPhone. At the time, Jobs was promoting the utility of "desktop-class" emails over phone based SMS. The first iPhone couldn't even send MMS images.

Apple didn't debut FaceTime until iOS 4, and at its introduction, Jobs anticipated that the other phone makers would openly license the technology cross platform. Instead, its competitors of 2010 all rapidly died off, leaving only Google and its Android partners with nothing but contempt for Apple and its FaceTime. Google expected it would introduce its own FaceTime killer and leave iPhones out in the cold.

Instead, Apple today effectively owns secure phone messaging in the U.S. It brought Messages cross platform to its non-mobile IP devices (Mac, iPad and Apple Watch) beyond the iPhone's first origins as a way to interact with other phones. Messages is a primary reason why people migrate to iPhone. Who'd have thought at the release of iPhone this would happen?

Further, the apps platform Apple was working to deliver for iPhone in 2007 was limited to the first party tiles Apple could get out the door on the first generation. It had to scramble over the next year to put together third party access to outside developers.

Many thought Apple had simply failed to grasp the potential of third party apps for iPhone, but I had the opportunity to ask Jobs about Apple's plans for apps before iPhone's public release. He very clearly explained, in front of a meeting full of shareholders, that Apple knew this would be essential to deliver but that there were complex problems to solve first, including security and privacy issues.

At the time, pundits were adamant that Apple needed to deliver third party phone apps using Java and Flash. This was clearly not going to happen to anyone who had seen what Apple was working on at WWDC. I was able to write "Thoughts on Flash" before Jobs published his own version. Apple already had a development platform, and iPhone was effectively just a portable Mac. Apple didn't need Flash, it already had Xcode.

Killer apps in reverse

Way back in 2007, Apple had outlined what killer apps would power iPhone. It wouldn't be one specific program the way VisiCalc had driven Apple II sales back in the late 70s, or the way PageMaker's desktop publishing had initially justified the purchase of high end Macs in the 1980s.

In the 1990s, Apple's platforms were driven by multimedia, but the increasing absence of key apps— largely Microsoft's aging Office titles, as well as a variety of other key apps that were tied to Windows or prioritized their development to the PC, including many video games— were killer apps in reverse: the essential software that kept people from buying Macs.

As the 2000s were about to begin, Microsoft and Sony DRM also threatened to cut off unobstructed access to commercial music, TV and movies to Apple's Mac platform, resulting in Apple's huge existential push to establish iTunes as a functional market for music, video purchases and eventually movie rentals and media streaming. That helped save the Mac from the killer app threat posed by DRM withholding media access. That effort also powered the rise of iPod and Apple's other mass-market, mobile devices.

By 2010, Apple was establishing iOS, and then iPadOS, as the most powerful and valuable mobile app markets, a huge shift from being the Mac underdog. Into the current 2020s, Apple has refined and homogenized its app development tools and deployment to facilitate seamless, portable app development across its platforms. This has set the stage for a powerful immersion app platform using a familiar interface, powered by the advanced new eye sight-based navigation of Vision Pro.

Three killers in one

Like the original iPhone, the killer app for Vision Pro will be a similar, familiar combination of media, messaging, and apps that will evolve in concert with the direction of society and consumer demands. The first big features will be Apple's immersive version of its own apps, including support for viewing spatial images, panoramas and commercial movies projected across the user's view; other apps Apple demonstrated, including games; and AR-powered messaging that allows remote users to speak in a group as if they were actually in the same room.

Some aspects of these features already exist for VR helmets, the same way smartphones in 2006 could handle some form of media, communications and networked apps. Java and Flash applets weren't enough to keep enough excited buyers from opting to try iPhone, despite its cost premium over basic smartphones.

As I documented at the time, there were lots of expensive smartphones that cost more than an iPhone when it first debuted. But they were hard to use, offered unfamiliar interfaces, didn't achieve their own installed base with a critical mass to compete for app attention that could justify paying $900 or more for a cell phone to very many people. Today millions of Apple's customers willingly pay more than $1,000 for the latest iPhone every year.

Killer app evolution

As Vision Pro begins to be more widely adopted, expect Apple's trio of killer app features to expand into roles uniquely exploited by its immersive hardware. One potential example in media is the experience of being at a concert or in front of a DJ or at a music festival or right on the field of a sporting event.

Perhaps you couldn't attend because it was far away, or months ago, or you had some disability that prevented you from being there in person. With Vision Pro, you can experience such commercial events either live remotely, or recorded as a memory you can virtually be inside. Many live experiences don't fully translate into a televised broadcast, which you notice when you have the opportunity to right in the action.

Some of today's iOS or iPad apps won't make an effective transition to Vision Pro. Clearly games will, as well as titles such as dating apps or virtual shopping apps that take you right to a merchant or inside of a property for sale or rent. You could visit a dealership and virtually kick the tires of a car that isn't for sale near you.

It's noteworthy that Apple pioneered early efforts to bring objects and 3D scenes to the Mac desktop with Quicktime VR back in the early 1990s. At the time, there wasn't today's App Store infrastructure or ubiquitous network connectivity to commercially exploit this capability. There sure is today.

In the area of communications, with Vision Pro you can speak to far away family members as if they are right there in the room. You can share the feeling of three dimensional memories as if you were watching from an angel's distance. Join a professional meeting and rather than being partitioned into Zoom squares, you're right there discussing subjects and seeing important facial cues and able to share your view of VR objects and scenes.

The most killer of apps? I think the core reason for many early adopters to spring for the initial model will ostensibly be a combination of collaborative work documents and AR FaceTime on expense reports, but will really be the immersive nature of relaxing to watch a spatial movie experience or to virtually arrive at a rave to other cultural experiences when you're too sick or perhaps too old to join in reality.

For these users, $3,500 will be money easily invested in a new platform from the established leader in mobile apps, media and communications, particularly one that has taken a strong public stance on protecting the privacy and security of its users. Google and Facebook tried to listen to your phone's microphone to harvest data on what you're doing. Would you trust ad firms to know what you're looking at? They're user exploitation firms, not secure device sellers.

The next question: how long can Apple deliver an unassailable lead in immersive experience headgear before the low price hardware makers and surveillance advertising firms rush in to try to compete on price with cheaper, less functional hardware and bundled spyware and ads designed to subsidize their discount? I'll take a look in an article next week, but share your ideas in the comments below.