Vein-scanning could help future Apple Watch detect no-touch gestures

The Apple Watch may be able to detect hand gestures without relying on touch in the future, by monitoring the veins within the wrist of the user to determine how the hand itself is moving.

The Apple Watch offers a number of simple gestures that users can make to perform functions with the wearable device, though so far the range has been quite limited. In terms of no-touch gestures, most users are familiar with raising the Apple Watch to wake it, or to trigger Siri by raising it to their mouth, but there is certainly more space to add gestures to the range.

One problem with detecting movements as a way to analyze gestures is that it could result in unwanted recognition, with unconnected movements possibly being interpreted as a command.

In a patent granted by the US Patent and Trademark Office on Tuesday titled "Vein scanning device for automatic gesture and finger recognition," Apple proposes taking a look at the veins within a user's wrist as a way to determine what actions are being performed on the same hand.

Hands have veins that flow into and around joints.

According to the patent, the system would capture images of the user's wrist veins as it goes through various poses. It doesn't need to capture images for all veins, only a small set correlating to those that flow to joints, and a very small snapshot of the veins at that.

What the system does require is the capturing of vein positions, relative spacing, shape, displacement, and blood flow. By examining how the veins are in specific hand positions, it is possible for the system to determine how the user moves their hand in general, and infer what kind of gesture they are performing.

The system could use a predetermined model to assist with its detection, along with a prediction system to refine the model and improve detection in general, based on a confidence value. If a sequence of poses is determined with a high enough level of confidence, the detection system could confirm a gesture has been carried out based on the vein movements, which is passed on to the rest of the computing system.

Changing a hand gesture can alter the vein positions in a wrist.

As well as a predetermined model, there can be a library of files outlining a catalog of gestures and expected vein movements and positions, which can be compared against and updated as time rolls on.

While it is feasible for Apple to rely on a similar setup to its existing Apple Watch sensor to perform such vein detection, it is also suggested more sensors could be at play, including adding sensors to areas of the watch strap that lie on the opposite side of the wrist to the Apple Watch body itself, in order to gain access to veins closer to the surface.

It is suggested the filing could apply to more than gestures on an Apple Watch, as it could assist with other activities. It is suggested the system could be used for input commands in AR or VR applications, a concept that would eliminate the need to wear gloves, and could provide more control over a device like Apple Glass without adding much extra hardware.

The patent lists a single inventor: Michael R. Brennan, who has previously appeared on an Apple patent application for a "System for iteratively training an artificial intelligence using cloud-based metrics."

An Apple Watch with extra sensors in the band.

Apple files numerous patent applications on a weekly basis, but while the existence of patent filings indicate areas of interest for the company's research and development efforts, they do not guarantee the concepts will appear in a future product or service.

Apple has examined the possibility of using veins in the past, but more for security than for activity. In 2019, it gained a patent for using an Apple Watch to authenticate a user by scanning elements under the skin on their wrist, while another from 2018 explored the possibility of using vein scanning on a user's face.

There have also been attempts to look into no-touch gesture recognition techniques that employ the Apple Watch band. A patent application from 2016 used a band containing force sensors to see if the user is making a fist or other motions.

Another from June 2020 employed sensors in the watch band to detect electrical signals generated by the wearer's muscles as a way to determine individual gestures and intentions, potentially without the user needing to carry out a full gesture.