The U.S. Patent and Trademark Office on Tuesday awarded Apple a patent for a camera system that uses a microlens adaptor to enable refocusing of an image after the initial shot is taken, much like the Lytro light-field camera.
Source: USPTO
Apple's U.S. Patent No. 8,593,564 for a "Digital camera including refocusable imaging mode adaptor," describes a method in which a user can take a picture at a certain resolution and refocus the resulting image after the fact. Also noted in the patent is the inclusion of such a system in portable devices, like an iPhone.
The patent describes a plenoptic camera, otherwise known as a "light-field camera," that uses a microlens array in the form of an adaptor rather than a fixed and integrated component as seen on what is arguably the most famous consumer light field camera, the Lytro. Founded by Stanford University computer science graduate student Ren Ng, Lytro Inc. has drawn funding from heavy-hitting investors despite its limited hardware lineup. Late Apple cofounder Steve Jobs famously showed interest in the tech and reportedly invited Ng to his Palo Alto, Calif., home for a live demonstration of the camera before it went public.
In fact, Apple's patent cites Ng's invention as prior art, but notes improvements can be made in regard to picture quality and resolution.
Light field technology uses so-called microlenses disposed in front of a CMOS or CCD sensor, which together form a "ray sensor" capable of capturing intensity of light as a function of position and angle. The microlens array directs light in a specific and predictable manner based on the lens aperture. In the case of Apple's patent, each microlens corresponds to a fixed set of imaging pixels. After the light information has been recorded, software can be used to focus and refocus the corresponding image.
Unlike Lytro, Apple's design employs a movable adaptor situated between the lens element and the imaging sensor. The adaptor holds the microlens array, meaning the camera can operate in two separate modes: high-resolution non-refocusable and low-resolution refocuasable. Lytro's microlens component can also be moved closer and farther from the sensor, but cannot be completely removed from the light path.
Apple's patent describes the adaptor as a switchable optical module mounted to a bracket, on which a glass plate is also positioned for normal high-resolution photos.
Low-resolution refocusable mode with microlens array active.
When in low-resolution refocusing mode, as seen above, the optical module is positioned between the lens and the sensor, thus creating a ray sensor. The patent notes the sensor records light data from the microlens (positioning) and angle of incidence (direction). Spatial resolution is limited to that of the microlens array.
In high-resolution non-refocusing mode, seen below, the switchable optical module moves to a different position in which the glass plate is situated between the lens and imaging sensor. Light is redirected by the glass pane onto the sensor to create an image of the scene as would a normal camera. The glass has a thickness and refraction index that compensates for the focal plane adjustment made to accommodate the microlens array. This mode allows for the sensor's full resolution to be used.
High-resolution non-refocusable mode with microlens array inactive.
In some embodiments, the hardware includes smartphone-like features, like the ability to place phone calls and connect to a wireless network. Other forms of the patent include a system that can be used as an add-on accessory for any existing camera, an interesting thought given the standalone Lytro starts at $299.
It is unknown if Apple will employ the technology in a later iPhone model, though the solution would be a welcome addition for many users who have trouble focusing on small screens. Perhaps most interesting is the possibility of a light-field video camera. Refocusable movies would be a great addition to the iPhone, which lacks tactile controls for focusing on the fly.
Apple's plenoptic adaptor patent was first filed for in 2011 and credits John Norvold Border and Richard D. Young as its inventors.