Scribble, a new feature in iPadOS 14, means users can handwrite into almost any field and the device will convert that into typed text. Speaking to Popular Mechanics, Craig Federighi, senior vice president of software engineering at Apple, explained how the company created the tool. It takes an approach that is far more complex than simply scanning and analyzing existing handwriting.
Scribble for iPad – Different Strokes From Different Folks
The first element is understanding the strokes people make when they write. “When it comes to understanding [handwriting] strokes, we do data-gathering. We find people all over the world, and have them write things,” explained Mr. Federighi. “We give them a Pencil, and we have them write fast, we have them write slow, write at a tilt. All of this variation.” This then goes further. According to the Apple exec, the company wanted the device to learn the strokes that formed each letter. “If you understand the strokes and how the strokes went down, that can be used to disambiguate what was being written,” he said.
The statistical calculations that make the Scribble feature work happen on-device, not at a data center. “It’s gotta be happening in real-time, right now, on the device that you’re holding. Which means that the computational power of the device has to be such that it can do that level of processing locally,” said Mr. Fedirighi.