Future Apple Pencil could detect colors and textures

Apple could be planning a real-world equivalent of the familiar eye-dropper tool in image apps, where a color could be set by an Apple Pencil tapping against it in an existing image.

Back in 2019, Apple applied for a patent concerning an Apple Pencil that could include "an on-board display" that showed different colors. In that case, the idea was that a user would see what color pen or brush stroke was about to be applied.

Then in 2023, Apple applied for a patent that in hindsight seems like an obvious extension of this. According to the proposals in "Electronic Device With Optical Sensor For Sampling Surfaces," the Pencil tip could sense colors.

Now that patent has been granted. Much of it does effectively repeat some of the earlier patent about changing or selecting colors, but now it's more specific.

The key new thing is that pressing the tip of the stylus against a color to identify and copy it — regardless of where that color is.

"The color sensor may be used to sample the color of the surface of an external object," says Apple. So this is not solely about tapping the Pencil against, say, an image on an iPad screen, it could be touching the tip against real-world objects.

Detail from the patent showing sensors in an Apple Pencil tip

"The color sensor may have a color sensing light detector having a plurality of photodetectors each of which measures light for a different respective color channel," continues Apple. "The color sensor may also have a light emitter... [and the] light spectrum may be adjusted during color sensing measurements using information such as ambient light color measurements."

So it's intended to be used on physical objects, but once you start doing that, there may be more that some future Apple Pencil — or other device — could do.

"Texture measurements and/or other measurements on the appearance of the object may also be made," explains Apple. "Texture may be measured using light that strikes the surface of the external object at glancing angles."

The majority of the 12,000-word patent application concentrates on defining the device that samples — a "cylindrical housing... [or] pencil-shaped" — and how the sampled data is then used.

It's important to note that while this patent has now been granted, that alone is not proof that such an Apple Pencil will be launched. Apple applies for very many hundreds of patents every year, and even when they are granted, it is not a guarantee that it will lead to a shipping product.