This third-year WWDC scholarship winner built an ML model to recognize beer in one day

Every year Apple invites student developers to its Worldwide Developer Conference. This year there were 350 scholarship winners in attendance. This afternoon I met up with Collin DeWaters of Fredericksburg, Virginia, who already has four titles in the App Store and put together a new 3D racing game for his Swift Playground application in three days. He's 21.

Student prodigies are all over the place at WWDC. I ran into two at this morning's Nike Run Club. But even by lofty WWDC scholarship winner standards, DeWaters seems extraordinarily diverse in his interests and his familiarity with Apple's APIs.

Despite being toured around as a guest of Apple during the event— which includes a trip to Apple Park's Steve Jobs Theater to meet with Tim Cook— DeWaters found time to build prototype code making use of the newest APIs to be released. He also managed a stop at Apple's Infinite Loop store to track down the just announced Pride Month band for Apple Watch. And during WWDC week, he made an appearance at AltConf (his interview there is on YouTube).

A Swift learner

DeWaters got interested in computers initially working with PCs, but then got a Mac in high school and spent a couple years figuring out Apple's Objective-C development language.

In 2014 Apple released Swift, and DeWaters quickly became a fan of the new language. He noted that it was easy to understand, easier to learn and easy to read and collaborate on code, particularly in comparison to ObjC. In 2015 he launched his first App Store title, "Avoid," a game where players dodge and defend against floating blips to stay alive as long as possible.

That year he also put in an application for a WWDC scholarship but wasn't accepted. That didn't stop him though. He created a social network for music and won a scholarship to attend in 2016. The next year, he worked with SpriteKit and GamePlay Kit to build the retro 2D game Bit Hockey, which he submitted as a Swift Playground and was again invited to WWDC.

Bit Hockey

This year, he applied with his latest project, a 3D racing game— with the intent of adapting it to work with ARKit. He's actually written the game twice: first as a conventional racing game for Mac that steers a car around a racetrack (there are billboards in the game cleverly advertising his other software titles), and again as an iOS game, with plans to turn it into an Augmented Reality title where players control the turning speed of Hot Wheels-sized cars that race around a track that can be virtually positioned on any ARKit-recognized horizontal surface.

DeWaters noted that most of the GameKit and SceneKit logic of his racing game was easy to port between Macs and iOS. The two platforms require more custom work in building their platform-native user interface, however. He stated that UIKit on iOS made it relatively simple to build the user interface elements, and in comparison to the Mac's AppKit API was a lot easier to understand.

One of the new features Apple demonstrated this year was an internal effort to host its own UIKit-based apps on the Mac in macOS Mojave, including News, Stocks, Home and Voice Memos. Each of the apps has a simple UI that's easy to adapt to the Mac's mouse and windowing UI without needing to rewriting it to macOS AppKit APIs.

Next year, after Apple polishes its implementation of UIKit on macOS, it expects to make this work public to third-party developers. DeWaters noted that this could facilitate moving his own iOS games (and other apps with a simple UI) to the Mac.

Music Memories

Of all of the projects he's worked on, DeWaters said he's most proud of Music Memories. That original app lets users select photos taken on specific dates, pick a calendar event or a selected date range and it then suggests (using MusicKit) the songs the user was listening to (using machine learning and some original algorithms), creating a zeitgeist playlist that can be accessed everywhere Apple Music works.

Music Memories

Once a Music Memory is created, the song playlist appears on iOS devices, Apple Watch, your Mac and can even be requested by Siri to play on HomePod.

The creative title (a free app) was recently featured by Apple in the App Store, jumping its user base to 1,000 daily users. It's already available globally, but currently limited to English. DeWaters said he's attending sessions at WWDC with an interest in localizing it for other speakers and locations.

He also has plans for a 2.0 version that makes use of a subscription model, featuring both a Mac edition of the app and the ability to save memories in the cloud. Its App Store reviews are glowing, with one user writing, "I love using this app especially with the 'dynamic memories'! I was able to listen to a playlist of music from 3 years ago in college! The fact that all of this works with the built-in Music app makes this app a real gem."

Mojave MacBook Pro, with Touch Bar

DeWaters was already using the new macOS Mojave on his MacBook Pro. He assured me he's dual booting it, but also said the new developer release is already very usable. I asked what he thought about the new Dark IU. "I'm never going to turn it off. I love it!" he said.

I asked him about the Touch Bar on his MacBook Pro. Did he find it useful? Turns out he's already built apps that use it. His iOS Bit Hockey game started out on the Mac, where he used the Touch Bar APIs to provide menu shortcuts and to pause the game.

He also said he finds his Touch Bar handy in other apps, including of course Apple's Xcode development tools, where with a single tap of the Touch Bar he can comment out a selection of code, isolating lines that might contain a problem.

What else does he anticipate Apple working on? Well, the rumors are swirling about eyewear featuring an Augmented Reality experience for AR without needing to point a phone. Maybe by 2020.

ML Beer

DeWaters already has plenty to think about right now. He somehow found time to work with Apple's newly released Create ML to build a machine learning model.

Yesterday (!) while at AltConf (a meeting held around the corner from WWDC) he whipped up a machine learning model to recognize drinks, and tell if a photo or camera image was water, wine, beer or some other drink.

He took 400 photos of peoples' drinks and created a model in ten minutes that he demonstrated for me by looking at image search photos.

"That's wine," he pointed out. "And here's a beer," he said, as his demo app labeled drinks in images as he pointed his camera at them, first with a confidence predictor and then with a pinned, 3D label floating in space. Seemed to be some gratuitous ARKit on display.

And speaking of beers: this year is DeWaters' third WWDC but tomorrow will be his first WWDC Bash where he's old enough to get an armband for adult drinks. Beers all around.