AirPlay may gain intelligent routing to predict where you want to watch

Apple is investigating multiple ways to simplify how users can select what they want to watch and then screen it on the device they want.

Right now, Apple may want all you watch to go through the Apple TV app — and it will fail until Netflix joins in — but in future, it's betting it can win by drastically simplifying video. Apple aims to replace our having to remember which app to launch on which device, then choose what show to watch, and then set up which other device to stream the video to.

Apple has already taken steps toward this. If you have an Apple TV, you can ask Siri to show you "films starring Reese Witherspoon," for instance. It will then show you a banner with selected movies. Click on any of them, and it will display a page with details, including where to watch.

That will typically always include the ability to buy the film or show from iTunes, but it will also search across services such as Disney+ if you have a subscription.

Three separate but related new patent applications want to take this much further, though. "Predictive Media Routing," US Patent Application No 20200221180, bemoans the admittedly first-world problem of starting a movie.

"For example, the user will typically open a software application," starts the patent application, "select a media content item, select a play button to begin playing the selected content item locally on the user device, select another button to indicate that the user wishes to stream or send the media content to a playback device, navigate through a list of available playback devices, and select a playback device to cause the user device to begin streaming or sending the selected content item to the selected remote playback device."

It reads as if this is really bugging the 11 credited inventors, who quite unusually include Craig Federighi. "This multi-input process for routing media to a playback device can be cumbersome, time consuming, and frustrating," says the application, as it calms down.

Detail from one of the patents showing how devices could make decisions to save user's time

The similar "Predictive Routing Based on Microlocation," US Patent Application No 20200221366, also wants to start by making it easier to find what to watch from the many applications a user may have.

"[An] application may be designed to play media (e.g., music, video, etc.), control the operation of a remote device (such as a television or speaker), manage connections with other users, etc," it says. "The number of applications that are stored and operate on mobile devices is increasing. For example, it may not be uncommon for mobile devices to store hundreds of applications ready for operation."

"While storing numerous applications may allow a mobile device to be particularly useful to the user," it continues, "it may be difficult and time consuming for the user to identify and execute a particular intended application or function among all of the available applications stored on the mobile device."

This particular patent application describes a way of making your devices do the figuring out of which app you want to stream to which screen. Using Machine Learning, Apple devices could predict what you want in a similar way to how all your Apple gear calculates which one you are talking to when you say, "Hey, Siri."

Perhaps you always watch "Star Trek: Discovery" through the CBS app on your iPhone, and every week you stream it to the Apple TV in your den. Apple devices could save you the trouble of launching that app or working through AirPlay to stream it.

"In some instances, the media may be automatically routed to the predicated device (as soon as the user requests playback of the media)," says the patent application.

A third patent application, "Content Playback on Multiple Devices," US Patent Application No 20200221155, also looks at this whole issue of how many buttons a user has to press to start watching something. But it suggests skipping the buttons entirely and just using voice.

That's more complicated than it seems, though. It's not just a question of getting Siri to recognize what screen you want to use when you ask for Monday Night Football.

Detail from one of the patent applications showing familiar playback controls plus an indication of where the video is being streamed

"For example, a user may wish to transfer playback of currently playing media from a first device to a second device, such as from mobile phone to a television," says this application. "Additionally, the user may ambiguously refer to content and devices using references such as "this" or "that," resulting in failed attempts to play the desired content."

This and the other two related patent applications are, as ever in this form of document, long on covering every possible permutation of the ideas, but short on specific detail. Across the three, though, there are 23 different credited inventors, so this is a long way from being a hobby at Apple.