The competition to become the best digital voice assistant has been heating up as more people are now telling their phones to do stuff for them or asking questions from their smart speakers. There are different research companies that will now do annual tests from amongst the biggest ones in order to give us “informed” decisions as to which one is the most accurate and therefore best to use. LoupVenture’s Annual Digital Assistant IQ Test shows Google Assistant as the best in terms of accuracy.
This round of tests focused on smartphone-based digital assistants since their behavior sometimes differed from those asked from smart speakers. They tested Google Assistant, Apple’s Siri, Microsoft’s Cortana, and they now also added Amazon’s Alexa. They asked the same 800 questions from their previous test and it was divided into five categories: Local (like nearest places), Commerce (where to order things), Navigation (how to get to places), Information (what and who mostly), and Command (self-explanatory). The two main metrics they used were: Did it understand the question/command and did it give a correct answer or response. Some of the questions were modified due to changes and updates to digital assistants.
The results showed the following:
* Google Assistant answered 86% correctly and understood 100% of the questions
* Siri answered 79% correctly and misunderstood 11 questions
* Alexa answered 61% correctly and misunderstood 13 questions
* Cortana answered 52% correctly and misunderstood 19 questions
Some of the general observations were that almost all of the misunderstood questions involved a proper noun, like the name of a local place or restaurant and that all (except Alexa, which doesn’t have historical data) showed significant improvement since both the voice recognition and the natural language processing has grown by leaps and bounds across all these digital assistants. We’ve come to a point where they should all be able to understand everything that users say, unlike before where we have so many fail stories about misheard questions and commands.
In terms of the answers per category, Google Assistant dominated almost all except for the Command part where Siri was better in terms of controlling your phone, smart home, music. In the Information category, it wasn’t really surprising that Google dominated it since after all, the company was founded on giving us the information that we want and need. You would expect Alexa to be top with the Commerce category, but since most of the questions involved researching about products and not buying products themselves, it did not do that well. LoupVenture said that their questions reflected how actual users are using these digital assistants anyway.
Google Assistant and Siri have an advantage over the others since they are baked into the smartphone while the other two are just third-party apps in the eco-system. But over-all, the fact that they have all improved is a good sign for the future of digital assistants. Have you tried using more than one on your device? What has been your experience if you compare the two or more that you’ve already used?
SOURCE: LoupVentures
