Roger Fingas
While the Apple HomePod is the "best sounding" smartspeaker and has a "measurably better" user experience in many areas, its underlying AI assistant — Siri — failed dramatically in a query test versus Amazon Alexa, Google Assistant, and Microsoft Cortana, according to Loup Ventures.
Three HomePods were subjected to 782 queries by the firm, said analyst Gene Munster. While Siri understood 99.4 percent of them, it was only able to answer 52.3 percent of them correctly. The latter figure compares with rates of 81 percent for Google, 64 percent for Alexa, and 57 percent for Cortana.
Siri did beat out Alexa and Cortana in test categories like "Local" (e.g. "Where can I find a good cup of coffee around here?") and "Commerce" ("Help me buy some new shoes"), but still failed in part because a number of functions — such as email, calling, calendars, and navigation — are simply inaccessible on a HomePod. Likewise, where Siri would display a list of Google search results on an iPhone, it outright fails to provide anything on a HomePod.
Removing navigation, calling, email, and calendar-related categories improves Siri's correct answer rate to 67 percent, and Munster argues that Apple will probably improve the HomePod's options over time, making it "vastly more useful and integrated with your other Apple devices."
The Loup Ventures analysis lines up with early HomePod reviews, which have typically pegged Siri as the weakest link. Apple has directed its marketing focus to sound quality, emphasizing technologies like beamforming and the use of seven separate tweeters.
Loup Ventures is forecasting sales of 7 million HomePods in 2018, boosting Apple's revenue and earnings by 1 percent. Sales are expected to hit 10.9 million in 2019, and grow 40 to 45 percent annually during the next three years.
Charles Martin
Christine McKee
Wesley Hilliard
Malcolm Owen
Andrew Orr
William Gallagher
Sponsored Content
113 Comments
Disclaimer: I didn’t read the article. I’m just commenting on the title.
I would personally much rather be taking things slow with all of this new artificial intelligence than blindly following the race to the first without ever thinking about the potential consequences.
That aside... whatever we we may say about who is first and who is best. Majority of people still don’t even talk to their assistants.
Considering Apple is marketing it as a high end speaker, it’s not surprising Siri is lacking. But it’ll improve with updates. You can’t improve hardware once the device is sold.
I hope more people doing a test like this. If reports like this won’t push Apple to be serious about SIRI I don’t know what will.
There’re so many fundamental things SIRI could be improved. If Apple won’t do it I hope they get called out like this until they understand it will hurt their brand.
(I’m talking about SIRI in general, like on iPhone, not specifically on HomePod.)
HomePod is a quality speaker with a poor voice assistant added on. It’s sold as a speaker.
The Echo is a poor speaker with a quality voice assistant added on. It’s sold as a voice assistant.
However, software updates won’t improve the sound quality of the Echo but a software update can improve Siri.
Hopefully all of the reviews will encourage Apple to improve Siri.
Then hopefully that improvement will force Amazon to improve Alexa. Ideally to understand more than 3 languages.
That being said, who really ever uses a voice assistant? I set alarms for cooking and that’s it.