Affiliate Disclosure
If you buy through our links, we may get a commission. Read our ethics policy.

Apple talks about the origins of Math Notes in iPadOS 18

In a new interview, Apple's Jenny Chen and Ty Jordan have revealed how both the new Math Notes and Smart Script for iPadOS 18 have their roots in years of Apple technology.

Math Notes is the new feature where iPadOS 18 will not only recognize a handwritten sum or equation, but it will solve it — and solve it in the user's own handwriting. Then with Smart Script, your actual handwriting gets smoothed out intelligently, so that it preserves what makes your writing yours, but also makes it more legible.

In a new YouTube interview, engineering manager Jenny Chan, and product manager Ty Jordan, say that these features are the culmination of what else Apple has been working on for a long time.

"So this actually builds on a lot of the technology that we've been building over many years," said Chen. "So with Scribble [handwriting recognition], we spent a lot of time actually working on recognizing [text]... but, basically, we were able to recognize your handwriting, turn it into typed text."

"And so I feel like [Math Notes is] just taking the technology from Scribble and taking it to the next level of not only recognizing, but being able to also synthesize text that's actually similar to your handwriting," she continued.

Ty Jordan said that the ability to recognize text and then emulate it also gave Apple the ability to manipulate handwriting and give it some of the features of typewritten text.

"So if you tap and hold your pencil, it can just move words around and you'll see that reflows in your paragraph so you can get some more space," he said. "You can even just paste typed text as your own handwritten text right in the middle of a paragraph."

"So a lot of the things that we take for granted with typed text," he continued, "you can now do with hand text, which is really cool."

Math Notes and Smart Script are part of iPadOS 18, which will be officially launched in the fall. A public beta has just been released, and Apple is now several rounds into beta testing with developers.



7 Comments

🍪
briano 3 Years · 1 comment

Interesting Video - but just how many times can you say 'Awesome' and 'literally'
I guess none of these people were around when the Apple Newton was under development in 1987...
:-)

❄️
Stabitha_Christie 3 Years · 582 comments

briano said:
Interesting Video - but just how many times can you say 'Awesome' and 'literally'
I guess none of these people were around when the Apple Newton was under development in 1987...
:-)

Obviously not around when the Newton was in development. The key difference is that their stuff actually works as intended. The Newton handwriting recognition sucked diddly ucked until it was repurposed as Inkwell in 2002. RIP Inkwell. 

🍪
nubus 8 Years · 629 comments

briano said:
Interesting Video - but just how many times can you say 'Awesome' and 'literally'
I guess none of these people were around when the Apple Newton was under development in 1987...
:-)
Obviously not around when the Newton was in development. The key difference is that their stuff actually works as intended. The Newton handwriting recognition sucked diddly ucked until it was repurposed as Inkwell in 2002. RIP Inkwell. 

The data soup of NewtOS was interesting but impossible to sync to other devices, paying extra for TCP/IP-support was mad, but people wanted to touch and try it.

I really like what Apple is doing here. Not sure any other company would be both interested in and able to.

❄️
Stabitha_Christie 3 Years · 582 comments

nubus said:
briano said:
Interesting Video - but just how many times can you say 'Awesome' and 'literally'
I guess none of these people were around when the Apple Newton was under development in 1987...
:-)
Obviously not around when the Newton was in development. The key difference is that their stuff actually works as intended. The Newton handwriting recognition sucked diddly ucked until it was repurposed as Inkwell in 2002. RIP Inkwell. 
The data soup of NewtOS was interesting but impossible to sync to other devices, paying extra for TCP/IP-support was mad, but people wanted to touch and try it.

I really like what Apple is doing here. Not sure any other company would be both interested in and able to.

I had a Newton 120 and it was fun to play with but I never found a "must have" use for it. It was ahead of its time and probably shouldn't have been released in the state that it was in. My understanding was that the MP2000 and MP2100 were both far better. 

🎁
tht 23 Years · 5658 comments

nubus said:
briano said:
Interesting Video - but just how many times can you say 'Awesome' and 'literally'
I guess none of these people were around when the Apple Newton was under development in 1987...
:-)
Obviously not around when the Newton was in development. The key difference is that their stuff actually works as intended. The Newton handwriting recognition sucked diddly ucked until it was repurposed as Inkwell in 2002. RIP Inkwell. 
The data soup of NewtOS was interesting but impossible to sync to other devices, paying extra for TCP/IP-support was mad, but people wanted to touch and try it.

I really like what Apple is doing here. Not sure any other company would be both interested in and able to.
I had a Newton 120 and it was fun to play with but I never found a "must have" use for it. It was ahead of its time and probably shouldn't have been released in the state that it was in. My understanding was that the MP2000 and MP2100 were both far better. 

It was impossible for Apple achieve the proper form factor with the Newton. Display too small, device too thick. Palm solved for half the problem by simplifying the UI and making it a pocketable handheld, but they gave up the intriguing part of a digital notebook where you can handwrite on it. Just took another 20 years to get there with iOS 18 and machine learning models.

Yes, like the Apple engineers in the video, I would highly encourage handwritten notes in school. Typing notes just isn't the same thing. The act of writing down equations really helps you understand the math. Doing this in Latex is cumbersome and I'm not sure there is retention with it.