What's next for Apple in 2016: New features & apps for Apple TV

From hobby to hero

It was back in 2005 that Steve Jobs first unveiled Front Row, an app for the G5 iMac that presented a novel interface for photos, music and movie playback, suitable for presenting on a television screen. The next year, Apple uncharacteristically demonstrated an yet-unreleased new standalone product, then referred to as iTV, sporting the same Front Row interface.

That product, released the next spring in early 2007 under the name Apple TV, was essentially a stripped down Mac capable of only running Front Row, navigated by a simple IR Remote and designed exclusively to connect to a television for output. This essentially made Apple TV a Mac that worked like an iPod attached to a TV; its playback content was typically synced over the network to the device from a Mac running iTunes.

When the original iPhone began shipping in the summer of 2007, Apple TV gained the ability to stream YouTube videos, thanks to a partnership with Google that transcoded YouTube's vast catalogs of Flash videos into H.264 video playable on iPhones (and the new Apple TV). This was one of the first and most important steps in Apple's strategy to break the web free from its dependance upon Flash.

The next year, Apple announced a "Take Two" software update for Apple TV that focused more on streaming content from local Macs or PCs running iTunes, or content from the iTunes Store (including new movie downloads and rentals), Apple's .Mac cloud service, or partners like YouTube and new support for Flickr photos. This made Apple TV less dependent upon its hard drive and more of a network steaming device.

After the Take Two release, Steve Jobs told USA Today in an interview, "we have the Mac business, which is a $10 billion business, and music — our iPod and iTunes business — which is $10 billion. We hope the iPhone is the third leg on our chair, and maybe one day, Apple TV will be the fourth leg."

At the time, a variety of pundits insisted that Apple should add a Tivo-like DVR, an optical disc reader (perhaps Blu-ray), or build Apple TV into an actual HDTV set. While those suggestions are now laughable in hindsight, we suggested that Apple should add support for Netflix and Hulu, add an iTunes Store with an SDK for developing interactive content (including games) and add support for streaming Internet radio and directly delivering video feeds from broadcasters, offering new sources of alternative content. That's exactly what Apple did, eventually."We hope the iPhone is the third leg on our chair, and maybe one day, Apple TV will be the fourth leg" - Steve Jobs

In the summer of 2008, Apple again cross pollinated Apple TV with iPhone by releasing an iOS app that let users navigate Apple TV using their iPhone or iPod touch.

A the end of 2009, Apple released Apple TV 3.0, with support for iTunes Extras and iTunes LP, movies and albums with interactive content, as well as Internet streaming radio.

Rather then making Apple TV the App Store-powered console we imagined, Apple instead focused on the release of iPad for the beginning of 2010. That makes sense, because tablets subsequently became a $20-30 billion business for Apple immediately after the release of iPad. The more limited market potential of a TV box would need to wait.

However, in late 2010 Apple did release a new, smaller and much less expensive "second generation" Apple TV based on iOS and running on the same A4 chip used in that year's iPad and iPhone 4— rather than being a scaled-down Mac running OS X on a low-end Intel Pentium M processor.

This new device had no internal disk for downloads, and instead was exclusively designed for streaming. It also supported AirPlay for wireless streaming of content directly from Macs or iOS devices. In 2012, Apple released an enhanced version running an A5 chip capable of supporting 1080p content from iTunes and Netflix, a new partner that broadened the appeal of the device beyond just being a way to deliver the users personal photos and videos— and Apple's own iTunes content— to a TV.

While still only half way to what we had predicted at the beginning of 2009, even in its basic role the new, smaller Apple TV became a $1 billion business by the end of 2013, becoming the company's fastest-growing hardware product and showing real potential for iOS in the living room.

Across three years, the new iOS-powered Apple TV gradually gained new channels from partners in batches, starting with Hulu, Netflix, Major League Baseball, the National Basketball Association, the National Hockey League, Vimeo, Flickr, and The Wall Street Journal, then adding HBO Go, WatchESPN, Sky News, CrunchyRoll and Qello in June of 2013, followed by Vevo, Disney Channel, Disney XD, Weather Channel and the Smithsonian Channel in August.

In 2014, Apple added its own iTunes Festival channel for streaming the massive series of live concerts it organized in London, as well as an Apple Events channel for streaming its new product introductions and the WWDC keynote.

Apple also made Apple TV a remote access hub for HomeKit commands in 2014, enabling users to relay control of their home automation devices from the outside Internet.

Apple TV 4 & tvOS

Three years of incremental enhancements to the iOS-based Apple TV passed before Apple launched the current fourth generation Apple TV, powered by a much faster A8 chip and driven by an App Store of third party software, notably including games, shopping sites and other apps that no previous Apple TV had supported.

Over the past decade, Apple's vision for Apple TV has evolved from being a simple way to export local and iTunes content to a television, into being a TV-optimized iOS computer, navigated by Siri and a touchpad remote, with an icon-centric interface, App Store and app-switching behaviors that are already familiar to iPad and iPhone users. Apple's vision for Apple TV has evolved from being a simple way to export local and iTunes content to a television, into being a TV-optimized iOS computer

Many of the initial titles for Apple TV are, in effect, iPad apps, with their touch interface transposed to the touchpad remote. This has required the development of a new "targeting" user interface, which combines the touch controls of iOS with the "selected item" highlighting more commonly associated with mouse-driven PCs.

This subtlety different yet transformative change in the user interface is distinguished and highlighted by Apple in calling the Apple TV's platform tvOS, despite most of its underpinning technology being based on the same iOS powering iPads and iPhones.

Apple's tvOS platform built upon iOS, iCloud, HLS

Apple's first apps for tvOS focus on its most valuable iTunes and iCloud services: iTunes movies and TV shows (both of which support movie downloads and rentals), iCloud Photos, Apple Music (with support for streaming songs and music videos and playing Beats 1 radio and playlists), and the App Store, which opens up the field to third party developers.

The company has also put a lot of effort into the tvOS platform itself, creating a general purpose Siri Remote with motion controls and a touch pad suitable for playing basic games in addition to navigating screens, and improving upon the tools content creators can use to develop not only standard apps, but also easy to deploy video streaming services.

A variety of the technologies behind tvOS have been in development at Apple for years. It first launched Siri in 2011, shortly before it released its last pre-tvOS Apple TV model. The touchpad of the remote has a long navigation legacy on Macs. The software frameworks that support gaming, including Game Center, Metal for graphics, Scene Kit and Sprite Kit, ReplayKit and GameplayKit, were all developed with Apple TV in mind. The hardware for supporting MFi gaming controllers was also clearly a building block toward living room games.

Apple TV's streaming video services are built in JavaScript and XML-based "TVML" templates, and make use of HLS, the open HTTP Live Streaming protocol that Apple developed to make it easy to stream video content over the web with minimal hassle from firewalls.

HLS, first introduced by Apple back in 2009's iOS 3 as a challenge to Adobe Flash's dominance over video streaming (as well as an alternative to Microsoft's proprietary Silverlight-based Smooth Streaming concept), uses a standard web server to send segments of H.264 video to clients. It can dynamically scale the quality of each segment sent to the client to accommodate changes in the available bandwidth, and supports DRM for protecting streams from piracy as well as subtitles and multiple audio channel support.

These technologies all make tvOS very easy to develop custom apps for, in addition to making it very easy to stream videos on demand to Apple TV users. Already, Apple TV sports a variety of typical TV channel feeds, ranging from news to sports to general programming, from a wide variety of free international broadcasters ranging from Al Jazeera to the Canadian Broadcasting Company to American PBS to Japan's NHK to the UK's Sky News, as well as premium subscription channels such as HBO, Showtime, Netflix and Hulu.

Apple's missing tvOS features looking for a 2016 fix

In the first few weeks after its launch, Apple enhanced its existing Remote app for iOS (below) to work with the new tvOS platform, enabling users to bypass the tedious text entry steps that are painful to perform using the bundled Siri Remote. There's still some basic improvements that could be made, however.

For starters, Siri doesn't support dictation, another solution to text entry that would often be very useful within apps. Also, Apple TV doesn't ever remember common account information such as your name or email address, exaggerating the inconvenience of its awkward text entry. Where is iCloud-synced Keychain for remembering your online passwords and syncing account information?

Even more importantly, Apple needs to automate app authentication itself. Currently, a variety of TV broadcasters are contractually tied to cable operators, and can only make their content available on Apple TV after a cable subscriber enters their cable provider account and password. Other channels (such as PBS) want to link their Apple TV viewers to a location so they can better understand their audience and customize their local content.

Apple should provide a central mechanism that lets users register their cable subscription and profile data so that they can activate channels without jumping through hoops on a computer. Both Twitter and Facebook have volunteered to solve this issue using their social network profiles, but Apple should at least offer an option to link things together without forcing users to divulge a vast history of personal information linked to their social network accounts.

Reportedly, Apple also wanted to bundle a cable-alternative package that let Apple TV users subscribe to package of paid content for a monthly fee. However, Apple's plan was blocked by broadcasters who want to perpetuate the status quo of cable, where users are forced to pay a high price for a huge bundle of channels, most of which are garbage. Apple's plan was blocked by broadcasters who want to perpetuate the status quo of cable, where users are forced to pay a high price for a huge bundle of channels, most of which are garbage

In the early 2000s, Apple's iTunes previously reached agreements (after long negotiations) with music labels to similarly offer users just the content they wanted at a reasonable price, resulting in 99 cent song downloads rather than $20 albums with often only a few desirable tracks (then the status quo for CDs).

This changed the music industry; today broadcasters are concerned that a similar subscription deal for Apple TV would force them to sell their TV content at a reasonable price, compared to the $100+ monthly subscriptions cable operators charge for hundreds of channels, most of which are unwatchable.

While it works to negotiate a real challenge to the current mess of cable programming that many users are increasingly dissatisfied with, Apple should also be working to develop and enhance additional content for Apple TV users, starting with some apps of its own.

Apple's missing tvOS apps that should appear in 2016

For starters, that includes making it easier to share and stream Photos and iMovies with other users. Currently, Photos on Apple TV just shows a static listing of your own iCloud Photos and shared streams, but all configuration must be done on a Mac or iOS device. There's also no way for users to tap into publicly shared streams of others. Currently, the easiest way to share content on Apple TV is AirPlay.

Using AirPlay Mirroring, iOS users can wirelessly steam virtually anything on their screen to Apple TV. However, the TV display remains a copy of what's on the small screen, so the UI elements can be out of scale. This works for Maps, for example, although an iPhone's UI chrome takes up too much of the screen when displayed on TV.

Apple should either create a new AirPlay Mirroring option that automatically scales down the UI for display on a TV, or launch some of its best iOS apps for Apple TV directly, a step that might best be paired with the touch navigation on an iPhone or iPad.

A Maps app for Apple TV, for example, could present Flyover graphics, street or transit maps users can navigate from their phone screen. In a previous article, I wished for Maps Flyovers that users could view from Apple TV. Imagine a screen saver created dynamically from Apple's Maps, such as flying along the beaches of a select coastline, or gliding between city landmarks.

Along the same lines, while "web browsers on TV" have always remained a idea that never really panned out due to the difficulty of navigating the web using a pointer or joystick-type device, a similar iOS/Apple TV tandem experience would enable users to browse pages in Safari using direct touch on their iPhone or iPad screen. This would be of particular value for educators and business presentations using Apple TV in a conference room.

Safari is another example of an app that could be delivered via AirPlay, particularly if iOS were given the option of optimizing for display on a TV.

Other apps from Apple that should make an appearance on Apple TV in 2016 include iTunes U, a vast trove of educational resources that are perfect for viewing on the big screen. Another example is Apple's own WWDC videos, which would be simple for Apple to offer on Apple TV for free. And of course, Podcasts, which would provide existing feeds a whole new type of audience. Currently, Apple TV only supports podcast playback via AirPlay, iTunes Home Sharing or via a third party app.

Another obvious example is Apple News, which should be easy to present and navigate on Apple TV, even if confined to the basic controls of the included Siri Remote. Imagine being able to simply ask for news about a given subject as easy as finding TV shows or music. Also, Apple should bring iBooks to Apple TV, paired with text-to-speech capabilities.

Another app that Apple TV sort of begs for is an iOS app that manages Apple TV itself, allowing users to reorganize their Apple TV's app icons by touch, rather than using the rather tedious drag and drop procedure with the Siri Remote. Along those same lines, it will quickly be necessary to add support for organizing folders or some similar mechanism, in order to make it easier to select apps once dozens have been downloaded.

Other tvOS features on the horizon

I previously asked for a Home app for Apple TV as a central way to organize and visualize your entire HomeKit configuration. With Bluetooth, Apple TV should also function as an iBeacon, letting you map out inside navigation similar to GPS outside. This could enable smart controls, such as turning on your TV when you enter the room and beginning to play the news, or a given playlist, or a favorite app depending upon the time of day and whether its you walking in alone after work or your child waking up to play video games on a Saturday morning. Apple TV should also function as an iBeacon, letting you map out inside navigation similar to GPS outside. This could enable smart controls, such as turning on your TV when you enter the room and beginning to play the news

Another pioneering feature Apple has developed is Continuity. Currently, there's no apparent way for developers to advertise their app on your iPhone so that Apple TV can be offered as a Handoff, such as walking into the room playing music or a video, and wanting to swipe up to immediately transfer playback to your Apple TV.

Paired with a standalone iSight camera (which Apple doesn't even make anymore), Apple TV could also work as a big FaceTime screen. There's really a ton of innovative uses for a truly smart Apple TV, leveraging HomeKit, Continuity and iBeacons, not just in the home but also in retail and educational settings.

Another capability of Apple TV that seems to be waiting on just a tvOS update is picture-in-picture support, so you could watch video in one app while switching to another. It should also be possible to begin playing music, then switch to playing a game while the music plays in the background.

It also seems like a matter of time before Apple adds old-school iTunes-style music visualizers back to Apple Music on Apple TV, given that the box lights up your TV but typically has nothing on the screen but small album art when playing music. Also, you may not be aware of it, as Apple seems to make no mention, but thanks to HDMI 4.1, Apple TV apps can already support 3D gaming if you use it with a TV that supports 3D, as we anticipated it might.

That leaves Apple TV and its new tvOS looking like a much larger opportunity that many seem to realize, particularly given the almost surprising popularity of previous Apple TV models, which had far less potential as a more basic appliance with a static feature set. Things should get really interesting for Apple TV in 2016.