On-device AI on Pixel isn’t new. The Pixel 4 has it. The old Pixel 3 also came with it bringing new features. The idea of on-device AI is to help the device owner with many things by using the Google Assistant among others. Multi-tasking is only one function. Other helpful actions include capturing photos, translating foreign text, or even identifying songs that are playing. It makes the Pixel phone “listen” and “do things” for you. Making it perform faster is the on-phone AI capability as it processes data and information on the device.
The idea of on-device AI also means the phone is more private and secure. It doesn’t have to go online or access the cloud to get information as the important ones are stored on the phone. This way, the Pixel 4 gets the right data more quickly.
Some new features available on the Pixel 4 include the Lens Suggestions, Recorder, and Now Playing. Lens Suggestions are provided by Google Lens. With your Pixel, you can get suggestions by simply pointing your camera at an email address, URL, or a phone number. The phone “sees” and then gives suggestions on what you can do like maybe visit the website, call a number, or save the email address to your phonebook. Other Lens Suggestions include scanning a document or copying text. The phone can recognize if you point your cam to a page or document.
Translate is also a suggestion and will be particularly useful if you’re traveling to a different country. When the Pixel 4 sees a language it knows you cannot understand, it will suggest to Translate to English. It currently works on English, Japanese, Hindi, German, and Spanish. We can expect the feature will be available in other languages someday.
The Pixel 4 comes with a Now Playing feature. This one can identify the songs playing in the background. It listens and can identify the song. It can also play the song again for you by streaming the song from YouTube Music or Spotify.
Federated Analytics helps in identifying the songs as it knows what popular songs are being played on Pixel phone in your part of the world. Now Playing becomes more accurate as it frequently gets updates from a database.
Another feature that takes advantage of on-device AI is the Recorder. The app is new on the Pixel 4. It’s useful as it can record and transcribe audio. You can also search for clips for future reference.
With Recorder, the Pixel 4 can transcribe speech directly. It can even identify other sounds like whistling or applause. With the text transcribed, this means you can search within an audio recording. Things are recorded but data will remain private.
