Future Siri may adjust speech based on environmental conditions, user input

Siri could one day adjust its volume and pitch based on environmental conditions

Apple is researching how to detect environmental noise levels and user voice patterns so Siri can respond with a yell or a whisper as needed.

Siri has been widely criticized for its lack of feature parity with rivals. The discontinuation of HomePod has again placed Siri in the spotlight as the smart assistant approaches its tenth birthday.

A patent application published on Thursday shows one area Apple seeks to improve with its assistant — environmental awareness. Currently, Siri will respond with the same voice, volume, and inflection regardless of how noisy a room is or how the command is given.

Other voice assistants like Alexa can respond in a lower volume and a "whisper" voice when a command is given quietly. Siri lacks this feature and can respond to commands at sometimes jarring volumes.

Siri could one day consider effects like room noise, device location, and distance from the user when considering response volume. Apple proposes that on-device processors can listen to the user's tone, inflection, and volume of the command. Other data points would be used, too, like time of day or previously set volumes.

With these considerations in mind, a user could whisper a command to Siri late at night and get a quiet response in turn. Loud environments would produce a slower but louder response from Siri to ensure it was heard and understood.

This patent is credited to Narimene Lezzoum, Sylvain J. Choisel, Richard Powell, Ashrith Deshpande, and Ameya Joshi. Richard Powell was previously credited with audio-based research applications.

Siri produces its voice using synthesized audio obtained from voice actors. A Neural Text To Speech engine converts a processor's command into audible speech. As of iOS 14.5, Siri will have two additional English voices with more humanlike speech patterns.

Stay on top of all Apple news right from your HomePod. Say, "Hey, Siri, play AppleInsider," and you'll get the latest AppleInsider Podcast. Or ask your HomePod mini for "AppleInsider Daily" instead and you'll hear a fast update direct from our news team. And, if you're interested in Apple-centric home automation, say "Hey, Siri, play HomeKit Insider," and you'll be listening to our newest specialized podcast in moments.

Follow AppleInsider on Google News

7 Comments

bsimpsen

said about 3 years ago

Siri's volume control is 100% broken in our house...

Me: Hey Siri, what is your volume?
Siri (whispering): My volume is 19%.
Me: Hey Siri, set your volume to 100%"
Siri (whispering): That's very loud, are you sure?"

Me: Yes
Siri (yelling): Okay this is as loud as I get.
Me: Hey Siri, what is your volume.
Siri (whispering): My volume is 19%.

We have 14 HomePods and Minis sprinkled around the house, they all do this. In addition it's quite common for a HomePod halfway across the house to intercept a command issued to one two feet away. We've reverted to using our oven timer because Siri is so vexing.

Me (speaking to a HomePod mini next to our stove): Hey Siri, set a timer for 10 minutes.
Siri (whispering from our family room, 30 feet away): Okay, ten minutes, counting down.
Ten minutes later, the timer goes off, but we don't hear it until the muffins are burnt.
Me (yelling from the kitchen when I finally hear the alarm): Hey Siri, stop.
No response. I must walk into the family room and yell at the HomePod to shut it up.

It's hard to imagine a system more broken than this.

pjohnt

said about 3 years ago

bsimpsen said:

Siri's volume control is 100% broken in our house...

Me: Hey Siri, what is your volume?
Siri (whispering): My volume is 19%.
Me: Hey Siri, set your volume to 100%"
Siri (whispering): That's very loud, are you sure?"
Me: Yes
Siri (yelling): Okay this is as loud as I get.
Me: Hey Siri, what is your volume.
Siri (whispering): My volume is 19%.

We have 14 HomePods and Minis sprinkled around the house, they all do this. In addition it's quite common for a HomePod halfway across the house to intercept a command issued to one two feet away. We've reverted to using our oven timer because Siri is so vexing.

Me (speaking to a HomePod mini next to our stove): Hey Siri, set a timer for 10 minutes.
Siri (whispering from our family room, 30 feet away): Okay, ten minutes, counting down.
Ten minutes later, the timer goes off, but we don't hear it until the muffins are burnt.
Me (yelling from the kitchen when I finally hear the alarm): Hey Siri, stop.
No response. I must walk into the family room and yell at the HomePod to shut it up.

It's hard to imagine a system more broken than this.

Same issues. Just last night I had music playing on my phone laying on the other side of the couch. I told Siri to stop the music. The HomePod down the hall in another room about 30 feet away responded that nothing was playing. I did this twice. The third time I was specific and told her to turn off the music on the phone.

When I stand right in front of my bedroom HomePod and ask it to play music and my phone is in the same room, about half the time it will start playing music on my phone. Brilliant.

And as for the timer thing, I activate Siri manually on my phone for that exact same reason.

Anilu_777

said about 3 years ago

For the two points above, it needs to be pointed out that Siri will respond on the last device used or on the most relevant device. For example, as I’m playing music on my HomePod Minis, I’m working on my iPad, my iPhone beside me on the table and my OG HomePod behind me next to the TV. If I say, “hey Siri” the Minis react with the Siri light but the iPad will also pause playback of an app’s music. But if I direct her to “stop the music”, the Minis will react and the music will stop. The iPad app’s music will resume. That’s how it’s supposed to work - between all the devices they figure out which is being addressed and which has to respond.

What I’m hoping Siri will do (as implied in this article) is what my BlackBerry used to do years ago - raise and lower the music volume in my car as the car/traffic noise increases or decreases.

dewme

said about 3 years ago

I’m always struck by patent applications that seemingly recast existing inventions from other problem domains into a new one. Nothing really wrong with harvesting existing inventions and using them in new ways - that’s innovation, but not invention. What is described above is automatic gain control (AGC) or automatic volume control (AVC). AGC has been a feature on pretty much every audio producing device that has to adapt to changing ambient and environmental conditions using a feedback control loop. The feedback loop can include actively measuring reverberation or distortion, sampling ambient noise levels, using input variables such as vehicle speed, or in this case sampling the sound intensity of a voice command. My old 2006 Mazda had this AGC tied to vehicle speed and wind noise on the audio system. Commercial radio receivers from the early1930s had AGC/AVC features.

That said, the Amazon Echo “whisper mode” feature and its implementation is totally awesome. I wish Siri was even half as smart as Alexa, but every AI child is unique and has their own path to follow.

nicholfd

said about 3 years ago

bsimpsen said:

Siri's volume control is 100% broken in our house...

Me: Hey Siri, what is your volume?
Siri (whispering): My volume is 19%.
Me: Hey Siri, set your volume to 100%"
Siri (whispering): That's very loud, are you sure?"
Me: Yes
Siri (yelling): Okay this is as loud as I get.
Me: Hey Siri, what is your volume.
Siri (whispering): My volume is 19%.

We have 14 HomePods and Minis sprinkled around the house, they all do this. In addition it's quite common for a HomePod halfway across the house to intercept a command issued to one two feet away. We've reverted to using our oven timer because Siri is so vexing.

Me (speaking to a HomePod mini next to our stove): Hey Siri, set a timer for 10 minutes.
Siri (whispering from our family room, 30 feet away): Okay, ten minutes, counting down.
Ten minutes later, the timer goes off, but we don't hear it until the muffins are burnt.
Me (yelling from the kitchen when I finally hear the alarm): Hey Siri, stop.
No response. I must walk into the family room and yell at the HomePod to shut it up.

It's hard to imagine a system more broken than this.

Sounds like you have some 2.4GHz and/or 5GHz interference. The HomePods have to be able to communicate via WiFi or BT to tell the others who is handling the command.

Also, if you have things sitting close/around a HomePod, this can make the volume sound lower to that device.

So might not be the HomePods, just your home environment.

We have 9 HomePods and have never had an issue with the wrong one responding. But I know I have a good infrastructure (network/radio engineer) and my HomePods have at least 6 inches open all the way around them.