Apple is investigating methods of incorporating camera hardware into its Apple Watch product lineup, focusing specifically on systems that avoid the inherent pitfalls of embedding image capturing equipment in a device not well-suited to the task.
Source: USPTO
According to a patent granted to Apple by the U.S. Patent and Trademark Office on Tuesday, the tech giant is looking to broaden the feature set of its wearable by incorporating a novel camera system capable of automatically cropping in on subject matter, tracking objects like a user's face and generating angle-adjusted avatars for FaceTime calls.
Apple's U.S. Patent No. 10,129,503 for an "Image-capturing watch" details a hardware and software solution that makes a camera-toting Apple Watch not only feasible, but useful.
As noted in the property's language, such designs would allow users to be less reliant on their iPhones, perhaps ditching the handset altogether in certain situations. With a camera-enabled Watch, users would be able to leave bulky handheld equipment behind when playing sports, working out or performing other strenuous activities.
A workable smartwatch camera solution is difficult to execute. Simply graphing a traditional handheld image capture system, like an iPhone camera, onto Apple Watch would be a user experience nightmare. Owners would be hard pressed to aim the camera at a subject without moving the entire platform -- their wrist -- to check framing, while at the same time dealing with a host of other concerns, from video stability to ergonomics.
To address the litany of anticipated issues, Apple proposes a system in which a wide-angle lens is fitted to a sensor built into a watch band. Alternatively, multiple cameras might simultaneously capture image data that is seamlessly stitched together to create one continuous image with a field of view that is much wider than any single-lens camera can provide.
The resulting wide-angle image reduces the need for precise aiming. Instead of framing subject matter beforehand, the final photo or video can be cropped after the fact to home in on a target or point of interest. This process can be conducted algorithmically or manually by the user.
Apple notes the technique not only works well for initial image acquisition -- users need only point the lens in a target's general direction -- but also adds a new level of convenience not usually available to portable camera users. Crop-after-capture can assist in recording spur of the moment events, for example.
Illustration of teleconference call technique.
Beyond basic photography, the patent delves into teleconferencing. Here, too, Apple provides thoughtful context to the proposed technology.
In some embodiments, the watch processes a captured image and applies facial recognition software to recognize a user's face. The device dynamically outputs a cropped image centered around the identified visage, making continuous on-the-fly adjustments to keep the face in frame even if the watch is in motion.
Perhaps most interesting is a contingency for rectifying unflattering "up nose" angles. If a user were to employ Watch in a FaceTime call, and do so without holding the device directly in front of their face as promised by Apple's auto-cropping tech, the resulting image would at times be captured from a low vantage point.
To negate "up nose" shots, Apple proposes outputting a representation of a user's face that appears to be taken from a straight-on perspective.
As noted in the patent, Watch's onboard processor might generate an angle-corrected image or representation of a user's face by compiling stored facial data, perhaps information gathered during biometric registration. Motion data captured by the camera and processed by the watch is then mapped onto the computer generated representation, which mimics a user's facial expressions and movements in real time.
Alternatively, source motion data can be used to inform the movements of a non-human avatar like Apple's Animoji and Memoji.
Illustration of automated cropping (top) and "up nose" angle correction.
Whether Apple intends to bring its Apple Watch camera band technology remains unknown. The company was in 2015 rumored to integrate a FaceTime camera into a future Apple Watch model, though the rumblings failed to bear fruit.
Apple's camera watch band patent was first filed for in September 2016 and credits Seung Wook Kim and Megan A. McClain as its inventors.