What's coming soon to tvOS 18, HomeKit, and AirPods

The bento graphic is an Apple tradition now. It's the company's quick and flashy way to summarize all of the key features of products and services after it finishes presenting them.

The Audio & Home bento actually combines features from a variety of hardware and operating systems. It includes elements from tvOS 18, AirPods, and HomeKit, all in a single graphic.

Redesigned Apple Fitness+

The Apple Fitness+ app for tvOS 18 is being redesigned. The idea is that it will help users check out more of the robust workout library, and attempt to keep them motivated about their fitness routine.

The changes include a new personalized For You space, sections for Explore and Library, search features, and enhancements to awards.

Hands-free unlock with UWB locks

As part of HomeKit's security features, locks with Ultra Wideband communications can be set to automatically unlock for a user. This can happen when the user's iPhone is up to six feet away from the lock itself.

Unlocking a door will make it easier for owners to enter their home with their hands full, since they won't need to fumble around to unlock the door at all.

FaceTime Live Captions

Already a feature of FaceTime on Apple's main platforms, the closed captioning feature will be available in visionOS 2. Apple initially said it was on the way as part of a pre-WWDC accessibility announcement.

For users, this will mean that contacts they FaceTime while wearing the Apple Vision Pro can be subtitled, allowing the hard of hearing to follow along with the conversation.

InSight for Apple Originals

The Apple TV app in tvOS 18 will gain a new feature called InSight, which will apply to all Apple Originals. When enabled it will offer up information about what's happening in real time.

This information includes actors, characters, and music, with further information also available. Users can select an actor to see their background and filmography page, while songs can be added to an Apple Music playlist.

This information appears on-screen on the connected TV, but if you're using an iPhone as an Apple TV remote, you'll see the data there.

InSight seems to be Apple's take on Amazon's X-Ray, which it uses for its own Fire TV device range.

On-device Siri on Apple TV

Instead of relying on servers to process queries, Apple will be bringing more Siri processing to the Apple TV itself.

To users, this could result in quicker responses for basic queries. It also helps improve privacy and security, simply by keeping some queries locally processed instead of traversing the Internet to a server and back.

Subtitles on mute

Subtitles in tvOS 18 will be smarter, displaying automatically if the user mutes the currently played show or movie so they can continue to follow along.

They will also appear at appropriate moments in tvOS 18, such as when the spoken language doesn't match the set device language. Subtitles can also appear if a user skips back while watching something, in case the reason is because they couldn't hear what was being said.

Enhance Dialogue on TVs, receivers, and Bluetooth speakers

Enhance Dialogue will be improved in tvOS with machine learning. This will enhance the vocal clarity of actors over the background noise for TV shows and films.

In short, it will become a lot easier to understand the dialogue of a film or TV show when there's a lot of background audio occurring. For example, up-tempo music, engine noise, and crowd scenes.

It will work with a TV speakers, as well as HDMI-connected speakers, AirPods, and Bluetooth-connected devices, or when playing supported content on iPhone and iPad.

Personalized Spatial Audio API

A new Personalized Spatial Audio with dynamic head tracking feature includes support for games played on AirPods and an iPhone. AirPods Pro users will also be able to use 16-bit 48kHz audio for in-game chat.

As part of this, Apple is providing game developers with an API so they can add support to their in-development titles.

Voice Isolation on AirPods Pro

Helping users hear others, Voice Isolation is being brought to the AirPods Pro, running on the H2 chip. The feature, using machine learning, will enhance voice quality while removing significant background noise, for the benefit of the listener.

New Apple TV screen savers

The Apple TV will gain a number of new screen savers with tvOS 18. The list includes Portraits, TV and Movies, and an animation of Snoopy and Woodstock playing and resting in various locations.

Guest access

The Home app will include guest access, which will provide a guest with control of various systems in a house. For example, a home owner may allow a guest's iPhone to unlock the front door's smart lock, but not others.

The access can be limited to a schedule or for a limited amount of time.

Siri interactions on AirPods Pro

A high point of the keynote for AirPods Pro users, they will soon be able to deal with some Siri prompts without using their hands or voice. Instead, they can respond by nodding or shaking their head.

Apple proposes that this can be handy when a user is in a situation where speaking isn't advisable, and if their hands are occupied. This was demonstrated by a man in a crowded lift declining an inbound call.

SharePlay with HomePod

SharePlay will be added to HomePod and HomePod mini, so many people can add songs control playback from a shared playlist. Users can join by bringing their iPhone close to another iPhone already in the group, or scan a QR code in the Apple Music SharePlay window on an iPhone.

Users can add to the SharePlay list without needing an Apple Music subscription.

Home electricity

Home owners will be able to view their electricity usage in the Home app's Energy category. It will help home owners determine how best to improve their energy usage during peak and off-peak hours.

This requires a connection to their utility account to function. At first, Pacific Gas & Electric in California will support the feature, with other partnerships planned.

Robot vacuums in Home app

Robot vacuum cleaners will be able to work with the Home app directly. This can be useful for home owners who want to incorporate the cleaners as part of automations or scenes, managed from the Home app instead of using the cleaner's companion app.

This will also enable voice-based requests to clean through Siri.

21:9 format for projectors

Projectors with support for 21:9 video will be able to play content in that aspect ratio in tvOS 18. Many films are produced in such an aspect ratio, so it should give a more film-like experience to users.