In the future, you may be able to salute your HomePod to turn up the volume, make a sweeping gesture to close your smart curtains with HomeKit, or just hold up your hand to pause everything.
There's already an optional gesture for answering a phone call on Apple Watch, but Apple wants you to be able to control everything, everywhere, all at once
There's a lot to be said for being deep in Apple's ecosystem, but it's very easy to end up surrounded by Macs, iPhones, iPads, Apple TV 4K, and more. It's not likely that you'll be playing audio or video on all of them simultaneously, but practically every device is listening out for you to say "Hey, Siri."
A newly-revealed patent application shows that Apple is thinking about its whole ecosystem, and how you can more easily use it. "Multi-Device Gesture Control," describes a situation where, yes, you can wave an arm to silence everything.
Really, though, it describes your doing this by waving an arm that has an Apple Watch on it.
Detail from the patent application showing the interconnectedness of all things
"The electronic device can efficiently provide gesture control for multiple other devices by mapping a finite set of user gestures to a specific set of gesture-control elements for each of the multiple other devices," says the patent application. "In this way a single gesture can be detected for potentially controlling various different functions of various different devices."
It sounds like it might take a time to set this up, but you could have it that when you raise your hand, every device reacts differently. Walk into a room, raise your hand, and the smart blinds roll down, the smart lights switch to a romantic scene, and Barry White starts playing on the HomePod.
You've made one gesture but, separately, the blinds can be set to interpret it as an instruction to open or close. The lights switch to a particular setting.
And perhaps a device runs a Shortcut to select a particular Apple Music playlist.
Apple's aim is not just "for gesture control," but specifically for "a finite set of gestures detectable at a first electronic device," which can trigger whatever a user wants.
That "first electronic device" is key. It is not that every Apple device has to have a camera so it can recognize gestures, it's that at least one device must detect them -- and pass them on correctly.
As ever, Apple patent applications are concerned with how something could be achieved, rather than specifically what purpose it can then be put to.
So the majority of this patent application concerns just how that "first device" -- "such as a smart watch or other wearable device, a smartphone, or the like" -- can detect a gesture.
Presumably you could turn your hand upside down and pretend you're Spider-Man flinging a web at these devices. Presumably.
The answer is by using "an ultra-wide band (UWB) sensor [or] a Bluetooth Low Energy (BLE) sensor." It could be that one of your devices is looking at you, so the system could use "an image sensor coupled with... another device."
There's also an issue that maybe the user wants more fine control, so the Apple Watch or other wearable device has to surface some extra options. But the main thrust of the patent application is gesture detection, which Apple says can also be via "inertial measurement unit (IMU) sensors... an accelerometer, a gyroscope, and a magnetometer."
Apple also ropes in "additional sensors such as electromyography (EMG) sensors)" in order to make this patent application as broad as possible.
The patent application is credited to four inventors, including Ali Moin, many of whom previously worked on machine learning technology for classifying biosignals.