Amazon Alexa ‘wins’ CES, but how well does the virtual assistant really perform?

I finally gave Alexa a try, and I’m a convert—to a point. I think there are many more things it can do, but I don’t know how to ask her to do them.

Amazon Alexa ‘wins’ CES, but how well does the voice assistant really perform?
Amazon

According to multiple reports, Amazon’s Alexa voice assistant dominated last week’s Consumer Electronics Show (CES) in Las Vegas, despite the fact the company had little official presence. Spreading its wings far beyond Amazon’s own Echo, Dot and Tap devices, Alexa popped up on wide variety of new devices demo’d at the show—from refrigerators to air purifiers, baby monitors, headphones and even cars. 

But Alexa’s rapid spread begs the question: What is using the voice-controlled system really like? What can it actually do, and how well does it do it in real-world kitchens and bedrooms—not to mention cars? 

Alexa moves in! 

Like a lot of other people, I acquired an Alexa-powered device from Amazon—the Amazon Echo Dot—over the holiday season. I know, I’m a bit late to the party, but frankly, I was not convinced any existing voice assistant system was really ready for prime time. 

After interacting with Alexa for a couple weeks, though, I’m now a convert—up to a point, anyway. 

Ironically, though, the very things I like best about Alexa don’t give me much confidence in how the system will perform once she’s showing up pretty much everywhere. 

Less is more, more or less 

Here’s the thing about my experience with Alexa. She (I know giving an AI chatbot a gender is ridiculous, but…) does great with simple requests with limited options: 

  • Ask her the time. She almost always understands and responds appropriately. 
  • Want a weather report? No problem, as long as you’ve properly set up the device’s location. 
  • Want to listen to the radio? Done, as long as the station is available on Tune In or you’re willing to set up I Heart Radio. 
  • Want to hear a joke? Fine, as long you don’t care whether it’s funny. 
  • Need to create a shopping or to-do list? Easy peasy, but good luck exporting it to another application. 

Overall, in my experience, Alexa does far better than Apple’s Siri in correctly figuring out and responding to the simple, basic requests she knows. It’s important to phrase things the right way, though. Saying “Alexa, stop” will pause the music, for example, but “Alexa, off” doesn’t seem to have any effect. 

Significantly, Alexa seems far more consistent dealing with simple tasks than Siri, who all too often gets an instruction right the first three times and then misses the mark the next three attempts. When it comes to keeping frustration levels in check and promoting usage, consistency may be more important than breadth of possibilities. 

Actions vs. words 

One big difference is that Alexa excels at actually doing things. It’s easy to turn on the lights (if you’ve connected them), pair with an external Bluetooth speaker (if it’s in pairing mode), order more paper towels (sometimes she seems a little too eager to go shopping). 

Answering questions, however, often flummox her outside of a few simple categories. Siri, on the other hand, at least tries to find the answer to complex questins or gives you links to related websites. She may get it wrong, but at least she doesn’t give up. Part of the difference, of course, is that Siri lives on your phone and can show you things on the screen. Alexa is a voice-only chatbot, which fundamentally limits her range of responses. 

What does Alexa know, and when did she know it?

The biggest issue I have with Alexa is an ongoing uncertainty over what she can and can’t do, and how to teach her to do more. The process of discovery for new Alexa skills is awkward when talking to the Echo, and it’s still confusing when looking online or on the Alexa app. I am pretty sure Alexa could do a lot more than what I’ve tried so far, but I either don’t know about the skill or haven’t figured out how to set it up or connect to the relevant service. 

I’m OK with that when I’m talking with Alexa in my kitchen. But I might get really frustrated when trying to get it to start my car or do whatever Alexa on a baby monitor is supposed to do. So, while I am happy enough working with Alexa as is, every time I talk to her, I get a severe case of FOMO (Fear Of Missing Out) on all the stuff I just know she could help me with if only I knew how to ask.

Maybe that’s a deliberate design choice by Amazon, or maybe it’s an artifact of the current state of chatbot/voice assistant development. But it’s really annoying, and it makes me think that voice assistants like Alexa still aren’t quite ready for prime time.

Join the Network World communities on Facebook and LinkedIn to comment on topics that are top of mind.

Copyright © 2017 IDG Communications, Inc.

IT Salary Survey: The results are in