While Alexa, Google Assistant, and Siri are still not perfect, it looks like these digital assistants are getting better every year. At least that’s what the annual IQ test of Loup Ventures is showing. Google Assistant is still the best among the pack but Amazon’s Alex is catching up and has shown big signs of improvement. This year is also the first time that Microsoft’s Cortana isn’t part of the test and that’s only fitting since they are looking at a different direction for their digital assistant anyway.
The test once again asked the same questions for all of the digital assistants involved and this time around they focused on smartphone-based digital assistants. There have also been studies for smart speakers but this one focused solely on the Google Assistant on Pixel XL running Android Pie, Alexa through the iOS app, and Siri through iOS 12.4. They are graded on two metrics: first, did they understand what was being asked and second, did they give a correct response.
On these metrics, Google Assistant was once again the best as it understood all of the 800 questions posed and had a 93% accuracy when it comes to correctly answering those queries. While Alexa was only 3rd, it was also the most improved. It had a 99.9% accuracy in understanding the question and then correctly answered them 79.8% of the time. This was a huge improvement in its 61.4% from last year and is the biggest jump since they started recording results.
The 800 questions asked were divided into five categories: local, commerce, navigation, information, and command, all of which are pretty self-explanatory. Four out of those five, Google Assistant was the runaway winner. But when it comes to commands, once again, Siri beat them to the punch. Alexa is understandably lagging behind the two when it comes to commands since it’s considered a third-party app.
Over-all, Loup Ventures says the performance of these three digital assistants have been steadily improving. Google Assistant improved by 7 percentage points while Alexa had a whooping 18 point improvement. And since the scores are almost at 100% already it may seem like they will be able to soon answer perfectly but of course what they’re able to understand is “within reason” but not always generally intelligent just yet.
