Affiliate Disclosure
If you buy through our links, we may get a commission. Read our ethics policy.

Inside iOS 10: New Photos app uses face, object & scene recognition to forge links

One of the main enhancements in the iOS 10 Photos app is face, object, and scene recognition powered by advanced machine learning technology and applied to improve photo search and automated image management features.

The app employs something Apple refers to as advanced computer vision, which analyzes the contents of photos using local processor power and various "deep learning" techniques. iOS devices are in fact said to perform over 11 billion computations per photo, picking out not just people but things like horses, mountains, and bodies of water.

Although Apple hasn't gone into extraordinary detail, the result is more narrowly-targeted search and browsing which helps discover images that might otherwise get buried. A "People" section under the Albums tab, for instance, lets users tap on a name to see every photo with that person detected.

The technology truly comes into play with Memories, a new tab filled with albums pre-populated and named based on their connections. Though some of these may not depend on computer vision analysis — an album like "Lake Tahoe" might just group together images shot in the same place and time — others will be based on who and what's in them, such as "On the Mountain."

The default view for each Memories album is also an auto-generated set of highlights, rather than complete contents. Users have to hit a "Show All" button to browse everything. Other album sections identify the people and places involved, and suggest related albums, again based on Apple's computer vision tech.

Events further include their own slideshow movies, which are compiled from highlights and can be tweaked for length and a choice of stock soundtracks.

iOS 10 is already available in a developer beta, which will be followed by a public beta in July. A final release is expected this fall, presumably in September alongside new iPhones.