Apple investigating advanced AirPlay system with device-specific UIs

Source: USPTO

An Apple invention filed with the U.S. Patent and Trademark Office describes a method in which a single app provides multiple user interfaces that can be modified according to the type of device being used as a display.

The patent application, titled "Application interaction via multiple user interfaces," outlines a system, or piece of software, that can present multiple two-way UI versions for different display devices. For example, instead of merely mirroring a slideshow from the iOS Photos app on a television via Apple TV, the software would present a fully interactive interface that is able to control the slideshow and dynamically make changes to the data.

Some embodiments of the invention call for "concurrently presenting multiple, distinct user interfaces for a single software application" on a variety of display devices, each of which can be manipulated by remote, keyboard or other input method. Any changes made to the data is reflected across all devices, or can be selectively displayed on a subset of devices.

Emodiment of patent with intermediate device.

When the data is modified, an action which can range from scrubbing through a movie to more complex scenarios like photo manipulation, the changes are reflected across all UIs, with each data set being formatted specifically for a device's display.

In essence, the invention can be seen as a more robust implementation of the current AirPlay protocol, with most of the heavy lifting to be done by a single app. Shades of the invention are seen in existing technology, which allows users to navigate movies and music from a host device through an Apple TV via a remote control, though more advanced functions are not currently supported.

While very similar to AirPlay, the patent application importantly notes a wide variety of app support, such as a productivity application, a video game, a web browser, or "any other type of software application that can be operated via a user interface." Some of those media types are already supported by AirPlay to some extent, but the invention calls for deeper interaction than is currently offered.

Illustration of user interface as seen on a computer monitor.

One example describes a digital photo application that can not only present slideshows from a laptop or iOS device, but allow for user interaction from a TV via a stripped-down, customized UI. Touching up images, file name changes, metadata modification and more can be accomplished through a simple interface which is distinct from that of the host device.

In some cases, there can be two distinct apps running; one on the host device and another on an intermediate device like the Apple TV. In this case, the intermediate app can interoperate with a number of different host apps, including commands, and present that information on a display device.

Illustration of user interface as seen on a connected TV.

Also mentioned is interoperability between two devices of the same type. For example, both host and intermediate devices can be iPads.

There are many possible applications for such technology, such as creating a portable remote A/V setup with an iPad outputting to an intermediate iPhone which is in turn connected to a TV.

Apple's UI patent application was first filed for in 2011 and credits Nikhil M. Bhatt as its inventor.