Not many people may care about this one but the Google Handwriting Input has allowed mobile users to handwrite text on their phones and tablets. Aside from the standard keyboard input, Android users can “write” on the screen. This has become beneficial especially for those non-English speakers. This feature now supports about 82 languages as Google keeps on adding more languages. This handwriting recognition has been added to Gboard. Over 100 languages are ready so everyone in the world can use Gboard.
Google keeps on making improvements and the most notable move was machine learning. From hand-designed heuristics, the Handwriting Input now relies on newer architectures and training methodologies.
Google wants to make sure error rates are reduced by using a single machine learning model, thanks to AI. It has worked on new models and even published “Fast Multi-language LSTM-based Online Handwriting Recognition”. It’s a paper explaining the research.
What Google has discovered is that touch points are the starting point for any online handwriting recognizer. It comes with a timestamp and strokes and their sequences are saved. Google then aims to normalize those touch-point coordinates and then captures the shape accurately. The sequence of points is then converted into a sequence of cubic Bézier curves as described.
The use of a sequence of cubic Bézier curves is preferred because they allow a consistent representation of the input. The sequence is then translated to actual written characters. This process of character decoding results in multiple types of RNNs being experimented on.
The RNNs become QRNNs (quasi-recurrent neural networks). They provide better predictive performance as made possible by the alternating convolutional and recurrent layers.
Google then made all the processes and sequences work on-device. This will be the ultimate user-experience as they are expected to be fast. The Gboard has achieved the lowest latency by converting the recognition models to TensorFlow Lite models.
Kudos to Google’s Handwriting Team for all the great work. Read more about the Handwriting Recognition in Gboard HERE.
SOURCE: Google AI Blog
