Google Assistant response speed getting improved by on-device processing

The promise of on-device processing of digital assistants offer the possibility of faster response times for queries based on locally-held data, without necessarily requiring an internet connection. It also has the prospect of fending off any potential privacy issues caused through dispatching requests to the cloud, where the data may be analyzed and used for other purposes.

At Google I/O, the search company suggested it was getting closer to the idea of on-device virtual assistants, with the next version capable of processing requests up to ten times faster.

An on-stage demonstration showed someone rattling through multiple queries in quick succession, with a small corner transcript of what was being heard revealing a lot of the processing was being carried out at a very high speed. Much faster than what would normally be expected by offloading processing to a separate server.

According to Google, it has reduced the size of the artificial intelligence models used for speech recognition and language understanding down from hundreds of gigabytes in size to less than half a gigabyte, making it more manageable to store on a smartphone. This enables the next generation Assistant to perform near real-time transcription without an Internet connection.

The Assistant also includes query features like Continued Conversation, where multiple queries could be made in a row without requiring the use of "Hey Google" every time. The examples also showed how a user could write a message out verbally and insert an image from a separate app, all without requiring any finger presses or vocal prompts to change task.

Google anticipates the next-generation Assistant will be arriving initially on its new Pixel smartphones later this year. It is unclear when it will roll out to other Android device, or even to the iOS app.

Google is not the only company grappling with the idea of on-device processing for virtual assistants.

Siri, Apple's digital assistant that is on a par with Google Assistant in terms of worldwide usage, could have an offline mode if a November patent application comes to fruition. Titled "Offline personal assistant," the Apple filing suggests the use of speech-to-text processing and validation to occur on the device, and if a connection was available, could perform both local and remote processing for two possible interpretations of a query, which could be used to further confirm a transcription as correct.