'Apple Glass' may create AR keyboards, help poor typists

Users wearing "Apple Glass," or other Apple AR headsets, may turn any surface into a keyboard — and also display typing over the keys to help hunt-and-peck typists.

Short of buying a new Mac, the single quickest way to speed up your work is to learn how to touch type. That ability does literally speed up how you press the keys, but it also means being able to look at the screen so you see mistakes as you make them.

In future, you may be able to see those mistakes, and all of your correct typing, displayed right in front of your eyes as you hunt around the keyboard.

"Keyboard operation with head-mounted device," is a newly-granted patent that is concerned with helping you type faster, and even do so without an actual keyboard.

"The operation of a keyboard, another input device, and/or any surface can be enhanced by features of a head-mounted device, which can display feedback, outputs, or other features based on the user's actions," says the patent.

Detail from the patent showing how a user could be shown their typing directly over the keyboard

"For example, the head-mounted device can display text generated by the user's operation of the keyboard," it continues. "The text can be displayed in a manner that allows a user to readily see the keyboard, the user's hands, and the text that is generated by operation of the keyboard."

So without even glancing up occasionally to see that you're typing what you think you are, you would be able to see exactly what you're doing.

"The head-mounted device can further display features that facilitate the user's operation of the keyboard," says Apple. "For example, suggested text, keystrokes, or other features correlated with keys of the keyboard can be displayed for selection by a user."

In other words, autocorrect is coming to Apple AR headsets. But the key point there may be that this is an AR idea. The text and the suggested corrections could be displayed within the AR headset, but for that matter, so could the entire keyboard.

"By further example," says the patent, "the keyboard can be displayed in a position and orientation that conforms to an arrangement of the user's hands within a field of view of the head-mounted device."

That isn't quite what it sounds like. If you have a real keyboard in front of you and for some reason are more comfortable with it turned a little to the left, you have to move your hands to suit.

However, if the surface in front of you is not a real keyboard, you — and the AR headset — can do whatever you like. There are already virtual keyboards that work by projecting laser images onto a surface but they leave you typing on, say, a table, and getting no feedback beyond mounting blisters.

That might be the case with this kind of AR system, too, but it could be a lot better.

"Other outputs provided by the head-mounted device can include audio output and/or haptic feedback," says Apple.

The system knows when you've "pressed a key," and so can do something, can react. Depending on the surface, it's hard to imagine any actual haptic feedback, but there could be a clicking sound.

That would be like the clicks you get by default when you type on an iPhone keyboard and haven't yet found where to turn that off.

The patent doesn't cover those cases where you are typing, but you haven't, say, clicked in the text field you thought you had. So it might be that you can see every letter you type, yet none of it is going where you thought.

There are limits, though. Apple can only help so much.

This invention is credited to three inventors, Keith J. Hendren, Alex J. Lehmann, and the extremely prolific Paul X. Wang.

Wang and Lehmann's previous related work includes a patent for creating haptic feedback in a touch-based input device, although that was expected to be for the Apple Pencil. Hendren is named on a patent application for a glass keyboard.