WWDC 2018: Apple, Siri and the future of mobile voice automation in iOS 12

Apple's Worldwide Developer Conference is just over three months away, but we already have some idea of what the company will likely be showing off in iOS 12 to drive the next generation of our mobile devices. Here's a look at Apple's Siri voice assistant and automation.

An inflated tale of constant political battles among incompetents

According to an emerging media narrative, Apple fumbled its lead in voice assistants with Siri, and has been relegated into a distant third behind the new leaders in talking to cloud-connected microphones: Amazon's Alexa-powered WiFi mics and its ecosystem of connected devices and a platform of third-party Alexa Skills; and Google's me-too efforts tied to its Assistant running on Android, iOS and some Google-branded appliances.

A recent piece by The Information described the entire history of Siri at Apple as being non-stop dramatic infighting over a series of foundational mistakes and fragile code all falling apart while new acquisitions were thrown on top.

The profile sounded a lot like the same site's earlier depictions of infighting between Android frenemies Google and Samsung related to everything from Tizen smartwatches to enterprise goals, or the bumbling changes in strategic direction within Google and its infighting hardware fiefdoms of Android, Chrome and acquisitions including Motorola Mobility and Nest.

Like its previous juicy tell-all depictions of drama in Silicon Valley, The Information largely cited its sources among disgruntled former employees whose stories were contradicted by others. But the main difference between the three portrayals is in final outcomes.

Google and Samsung actually failed to sell smartwatches, whether Android Wear or Tizen, and fumbled their enterprise pitch, while in parallel Google's Android, Chrome, Motorola and Next hardware have also all been huge expensive catastrophes without sales of any real commercial significance. The last several years of Apple's Siri look like a genius home run next to all of that.

Siri is failing all the way to the bank

People love to hate on Siri, but the worst thing Siri ever did was sell lots of iPhone 4s models, draw attention to other Apple introductions (from Apple Watch to CarPlay to Apple TV 4 to HomePod) and, of course, fail to understand a lot of questions from frustrated users. Ask your HomePod how to make a Negroni and it doesn't help at all. But Apple doesn't sell Siri as a voice first platform.Apple doesn't sell Siri as a voice first platform

Apple doesn't sell Siri as a product at all, so the only way to brand Siri as a "failure" would be to establish that rivals were somehow undercutting Apple's business or future potential with their own voice services. Yet even The Information admitted that Apple remains by far the largest supplier of voice requests, and the only one to support more than 20 languages (Alexa supports three). Siri is at worst assisting in profitable sales of Apple hardware.

If Android gets a leadership success trophy just for voluntarily participating in the volume sales of low-end phones that don't make money despite its variety of serious architectural problems, performance issues and security flaws, surely we can throw Siri a bone for riding along as a supporting actor in Apple's massive, global cash machine, even if it can't tell you what's in a drink on demand faster than you can just look it up on your iPhone.

Hey Siri, where's Alexa?

Nobody is passing over Apple Watch (the most popular smartwatch by a vast margin) to get Alexa-connected smartwatches. Buyers are not shrugging off CarPlay— which is so popular it has forced every major carmaker to offer support, even those who once actively resisted it such as Toyota— to instead get car-mounted Alexa hardware. HomePod actually outsold comparably-priced "smart speakers" and much cheaper Amazon Echo devices in its debut, and even Apple TV has no competitors asking a similar price for hardware just because they're powered by Alexa.

For being such a celebrated business, Alexa doesn't seem to exercise much commercial muscle.

Alexa and the 25,000 Skills

The real power of Amazon's Alexa is supposedly the third-party support for its voice action extensions, called "Skills." These allow app developers to expand its voice capabilities to support controlling devices and tasks such as ordering a favorite pizza from Dominos. If you review any listing of "the best 50 Alexa Skills," you mostly get a junkyard of fluff

The Information didn't dwell on the value of these skills. Yet if you review any listing of "the best 50 Alexa Skills," you mostly get a junkyard of fluff. For example, TomsGuide listed a series of 50 Skills that included lots of things Siri already does, such as music playback; dictating messages; asking about stocks prices; asking for news reports from NPR, CNN or the BBC; controlling home lighting automation; listening to podcasts; asking how to spell a word; or requesting a car from Uber or Lyft.

Other "Skills" were odd novelties, such as playing cat noises or thunderstorm sound effects, asking for a word definition of the day or an inspiring quote. Others Skills offered voice assistance for things Siri can look up on your phone, such as how to make a drink or get first aid advice. Other Skills were audio-only versions of apps. The most frequently cited among "top Skills" is a banking app for Capital One that accesses your bank balance and can make a payment. To use it, you have to speak out a security PIN code. That just seems like a bad idea. Alexa integration sometimes uses the same type of "security" to unlock your doors.

Other Skills involved devices, such as telling your (high-end model) Roomba to begin vacuuming or to adjust the temperature on a digital Sous Vide. You can also start your car remotely (if you have one of a few new vehicles) or control future home appliances from Whirlpool. It's almost as hard to see the valuable utility here as it is to discern any proprietary advantage Amazon Skills can wield in the long term.

Apple, Apps, Siri and Automation at WWDC 18

Virtually all of these Skills are tasks you can do from a smartphone app, just triggered by your voice via a speaker instead. How will Apple ever catch up? The answer may lie with Workflow, an app the company acquired nearly a year ago.

Hey Siri: Workflow makes complex tasks into a triggerable event

Workflow is the graphical (rather than voice) version of what a lot of people seem to imagine as the ideal goal of Siri: it lets you define a set of actions and then trigger them with a touch, from the Home page, from within apps (on the Share Sheet), from the Today widgets page or even from your Apple Watch. Given how turns a trigger into complex action, a Workflow Intent for Siri could enable third parties to craft any sort of skill for users to launch with their voice, or graphically from any iOS device.

The insurmountable Skills advantage of Alexa largely goes away with a way for iOS app developers to add Siri automation via Workflow. The pizza order that Alexa lets you repeat would now be something that any app developer could expose as a voice trigger, much the same way that HomeKit allows devices to respond to a set of controls via Siri, or from direct control in apps or widgets.

Note that Alexa doesn't have to go away for Siri to work well or improve. Users might choose to have both, using an Alexa mic to order products while using HomePod to fill their living room with great sound and provide access to their personal information. However, the narrative that Apple is headed toward oblivion because Amazon has sold tens of millions of Alexa mics over the past few years is really not true by any stretch.

A surprising deficit of the Information

Reinforcing the huge gap in logic that accompanies much of the recent reporting related to Siri, The Information appeared to leave out material facts in order to contrive a tale of "everyone else in the voice universe" muscling into or otherwise disrupting Siri. The piece noted that Apple banned its former Siri employees (who left to launch Viv Lab) from playing basketball on its campus among concerns they'd recruit away the Siri talent Apple supposedly doesn't have.

There was no mention of the fact that those Viv employees (Dag Kittlaus, Adam Cheyer and Chris Brigham) not only launched a project to compete against Siri (which they publicly demonstrated in early 2016) but also sold themselves to Samsung toward the end of that year to work on what was intended to become Bixby.

Viv Labs was in the news as the Siri-killer until it failed to kill Siri | Source: The Washington Post

The Viv team promoted their product as enabling more conversational, complex voice interactions than Apple's Siri could. But they couldn't ultimately get it to work. According to the Wall Street Journal, Samsung was forced to revert to the existing code of its languishing internal S Voice project.

Last year Samsung had to delay its Bixby voice service from the very flagship that it was supposed to differentiate. And across the next year reviews continued to describe it as "unforgivable," or "the kindest way I can describe it right now is utterly unimpressive."

Rather than shipping Viv's "better than Siri" on an imagined 400 million devices, Bixby ended up as an anti-feature dogging Samsung's flagship Galaxy S8 and doing no favors for the latest S9 this year. "Bixby is, by far, the most frustrating part of the S9," wrote Brian Heater for TechCrunch last week.

Isn't the massive failure of Viv to deliver a "better Siri" across the last three years— despite all the market power afforded by the world's largest shipper of smartphones— more important to understanding the voice assistant market than an anecdote about Apple not wanting Vivi to poach their employees on its own campus?

Selective reporting that portrays Alexa's largely frivolous Skills as mountains of pure gold while brushing off the fact that selling voice appliance devices to the world would require support for more than three languages and at least some presence in both of the world's largest two markets isn't really good information.

Real people on the utility of voice

Responding to dramatic portrayal of Siri woes at Apple, one reader of The Information wrote, "I wish Aaron and Kevin had also looked outside the Bay Bubble to see what motivates buyers of smartphones and 'Smart Speakers' - and find out what they really want out of their Digital Assistants.

"My own home is littered with two Echo 'First Gen' tubes, two Echo Dots, a Google Home and a HomePod— and they're used almost entirely for weather (all roughly equal), timers (advantage HomePod, because it's audible a room away), spelling (advantage Echo dot, because it's cheap and can be placed beside a computer) and music (massive advantage to HomePod, which gives the only acceptable music stream.

"We have an iPhone 7, iPhone X and Samsung s8+, upon which the digital assistants are used to make phone calls, deliver weather forecasts, set timers, set alarms and give driving directions. I'd rank Siri and Google Assistant about equal at these tasks - and Siri is actually a lot more useful to me because she's on Apple Watch, which I use far more often than any of the speaking tubes. Google Maps are probably more reliable in tests than Apple Maps, but I find the latter good enough in Car Play, which I find myself using far more often than Android Auto.

"You'll note I haven't once mentioned IoT. We do have a home theater and thermostats that are Alexa-enabled. I could enable the skills and memorize the commands, buteh. The Android and iOS Apps are simpler, and a Control4 remote appeared spontaneously on my Apple Watch. Both functions are simpler to control WITHOUT voice."

HomePod is "audible a room away"

Siri expectations for WWDC 18

So while you can expect Apple to expand Siri with some new Intents and to potentially add third-party task automation support in Workflow, don't expect the company to devote major competitive efforts toward replicating the extremely low-value Amazon Skills ecosystem that offers users little more than a more clumsy way to use apps without looking at them (the way Google has blindly, desperately tried to copy Alexa).

On the other hand, Apple has been enhancing how different Siri devices work together on the same network, an issue that neither Amazon nor Google really have. This is still a work in progress, but HomePod already does a good job of silencing your iPhone when both are listening for "Hey Siri."

What could be improved is Continuity handoff between devices, allowing HomePod to delegate questions to your iPhone or Apple Watch when it can't answer them, potentially even working together to deliver a UI on your phone or watch while listening to you from HomePod.

Similar HomePod integration with Apple TV, enabling you to ask it to play a specific show, also seems likely, enabling third-party tvOS developers to build new types of apps and games that can listen for feedback and commands using HomePod's fancy array of microphones.

HomePod will also likely expand its ability to support new types of requests (Hey Siri, where's my iPhone?), and expand what apps and data it can access under a Family Sharing account. You might also expect Apple to expand upon Type to Siri as a way to discreetly invoke Siri-type searches without using your voice at all.

What do you anticipate WWDC 18 will introduce for Siri, Workflow, HomePod and Continuity? Add your comments below.