Apple is cutting down Google Gemini's massive models into smaller and more secure parts through distillation, to create elements that are more suited to on-device Apple Intelligence processing.
The deal Apple made with Google allows the iPhone maker to use Google's Gemini AI models as a basis for its own updated AI. It now appears that the deal lets Apple use some techniques to make models that will work better for on-device processing than the mainly server-based Gemini itself.
It was previously known that Apple could adjust the Google Gemini model to respond to queries in specific ways. However, according to The Information on Wednesday, Apple has a lot more leeway in what it can do and access within the model.
This includes complete access to the model within its own data centers, which lets its engineers have the ability to closely examine Gemini and how it functions.
A key element to the report is that Apple can perform distillation, a technique that can transfer knowledge from a larger model into a smaller one. The idea is for some of the smaller models within Gemini to teach a smaller Apple model, gradually segmenting and separating out the model's knowledge base.
To do this, Apple's external model acts as a student, learning how Gemini's internal computations work for a function and mimicking the processing. Apple can ask for high-quality results and the Gemini model's "chain of thought," which can be used to train its own small model.
The end result is the creation of a smaller model that performs a specific function just like the larger model. That smaller model would run at about the same speed and accuracy as the original Gemini model for that function.
The benefit is that the smaller model is cheaper to run, and could also use less powerful hardware too. That means models that could be created to work on devices like an iPhone, instead of requiring a server.
Small models, but big are questionable
The distillation effort is one of the tasks being carried out by Apple Foundational Models, the team working on Apple's AI projects. The ultimate scope of the AFM's remit, however, is still unknown.
To report sources, the creation of mini models splintered off from Gemini is clearly for on-device processing purposes. But questions remain about making a bigger version.
One unnamed source doubted that the distilled models would be used as a base for creating new larger models. They don't believe the team is actually trying to make a direct competitor to Gemini at all at this time.
For the moment, Gemini will continue to provide responses for Siri answers. With expectations of Apple finally pulling off its big Siri refresh in June, Google's model may have more work to do in its undistilled form.
It is evident that Apple clearly believes in on-device processing being the path ahead for its AI success. It's in the process of acquiring model pieces to continue down that road, but a bigger comprehensive Siri model is off the table for the moment at least.
