Loup Ventures likes to put digital assistants and smart speakers to the test in order to annually assess how Artificial Intelligence and these supposedly smart assistants actually work in the real world. This year’s test focused on smart speakers Amazon Echo (Alexa), Google Home (Google Assistant), HomePod (Siri), and Invoke (Cortana). Unsurprisingly, Google Assistant is still at the top of the heap but Siri made had the highest points improvement over the 12-month period. Alexa was third and Cortana 4th.
The methodology of their test remains almost the same. They ask the same 800 questions to all of the smart speakers and they are graded if they understood it and if they provided the correct response. The questions are divided into 5 categories: local, commerce, navigation, information, and command. The question sets are modified every year since we all know that AI assistants’ capabilities are changing (hopefully improving) every few months or so.
This year’s results showed that the Google Home was able to understand all queries perfectly with a 100% score. When it comes to responding, it’s slightly lower at 88% but still better than all of the others. It also improved by 7 points as compared to last year’s test. It was also top with four out of five categories with HomePod beating it out in the Command category, mostly because music-related queries are the specialty of Siri.
While Google Home was the sort-of runaway winner, the HomePod just barely beat Amazon Echo by a couple of points. But over-all, all the smart speakers improved, with Google Home by 7 points, Amazon Echo by 9 points, Invoke by 7 points, and HomePod by 22 points, although the last one is just in a 9-month period given it was released just last February.
Scores are nearing 80-90% for almost all the digital assistants and it’s a sign that soon we might actually get almost everything that we ask from them. Loup Ventures says that the improvement with these AI assistants will come from allowing more functions to be controlled by your voice, which some might find a bit too scary of course.
SOURCE: Loup Ventures
