Playing Alexa at Chess

Chess mate devices

Over the last few years there’s been a big push in voice-driven technologies, and with products such as Amazon Echo and Google Home being launched to the masses.  They have seemingly endless capabilities if the marketing hype is to be believed, but we wanted to see how much they can do from a technical development point of view.

Ever keen to find new ways to drive growth in our client’s businesses, we’ve constantly got our eye on the ball when it comes to new technologies that are hitting the market, as every new technology is a possible new way for businesses to improve efficiency, cut costs and reach new customers. So when we noticed that voice interfaces were gaining real momentum and with Amazon Echo sales exploding we just had to get our hands on one and have a play.

Alexa is Amazon’s intelligent personal assistant app that is capable of voice interaction, amongst other things, and Amazon Echo is a device that ships with Alexa installed. This essentially provides a medium by which Alexa can be integrated into your home.

Building an Alexa Application

We knew very little about how Alexa’s voice interaction worked other than you can speak to it and write a program to respond to that speech. We had some assumptions about what it could do, the biggest of those being that, to some extent, you *could* have a conversation with Alexa. We assumed the conversation that follows would be achievable:

User: Alexa, how are you doing today.
Alexa: I’m very well today, how are you?
User: I’m very good thanks, did you have a good day?
Alexa: It’s always a good day if I can assist you.

Now that’s a silly example as far as conversations go but it demonstrates the simple back and forth type of flow that we thought possible.

So with that assumption in mind, we excitedly starting sifting through what seemed like an infinite number of ideas for apps that we could build, eventually settling on a chess game.

They say assumptions are the mother of all f*ckups and oh how this proved correct. The first thing we learned when we began coding our app: Alexa doesn’t do conversations. The way in which Alexa has been coded means all it can reliably do is take a single command and issue a response to it. For example:

User: Alexa, will it rain today?
Alexa: No it won’t rain today (but I’m going to rain on your development parade.)

Changing multiple commands together to form a conversation is not possible. This is exacerbated by the fact that it’s next to impossible to get Alexa to keep an app open for more than 16 seconds. This means that unless the user can decide on their chess move in under 16 seconds they’d have to open the app up every time they want to issue a command.

We’d hoped to achieve something that felt organic as you interacted with Alexa. A simple game might go as follows:

User: Alexa, start Chess Mate.
Alexa: Welcome, it’s your move.
User: Move A2 to A4
Alexa: My move is F7 to F5
User: Move A1 to A3

That was a far cry from what we ended up with:
User: Alexa, start Chess Mate.
Alexa: Welcome, it’s your move.
User: Move A2 to A4
Alexa: My move is F7 to F5
… App closes

User: Alexa, start Chess Mate.
Alexa: Welcome, it’s your move.
User: Move A1 to A3
Alexa: My move is D7 to D5
… App closes.

This threw a real spanner in the works as it meant all of the nice conversation-based interactions that we’d planned and hoped would make the game a seamless experience, just wasn’t possible. This instantly took us back to the drawing board and we had to completely re-plan how the user would interact with the game.

It wasn’t too long until we ran into the next big issue: Alexa doesn’t truly understand you. Instead, it kind of best matches what the user says to one of the commands, called “intents” in the Alexa ecosystem, that has been defined in the app.

This is best explained with an example. Let’s say that we are creating a simple weather app and we define 3 intents that we want the user to be able to say and get a response to in turn.

  1. Will to rain today?
  2. What’s the average temperature today?
  3. Will it be windy today?

You would hope that if you open this hypothetical app and said “chicken nuggets taste great”, that Alexa would respond along the lines of “Sorry I didn’t understand you”. That’s not the case. Instead, Alexa will try to work out what intent it thought you meant and then do that. The problem is that it will always match it to something. This means that once you’ve opened the app you can literally say anything and Alexa will ALWAYS do something. If I was to open our hypothetical app and say “I’m more of dog person than a cat person”, Alexa would respond with “no it will not rain today”.

On some level, a little flexibility is great as it means the user doesn’t have to say the commands verbatim but in practice, it wreaks havoc when you’ve got two commands that are similar or someone says something unrelated to the game.

During the course of testing our game, Alexa would constantly match up what we said to the wrong commands, triggering it to do something incredibly annoying such as restart a game when we were right in the middle of it.

Again we found ourselves accepting defeat and trying our best to work around this limitation by building the game in such a way as to mitigate, as much as possible, the chance of it issuing the wrong command.

During the course of this project, we found ourselves having to work around a multitude of different problems that are inherent in the way Alexa has been built; the two mentioned above are the worst. One major sore point for us that was present the entire time was

Alexa’s very poor ability to understand what we were saying. It constantly misunderstands moves and gives the wrong input, resulting in the wrong moves being made or commands being issued.

What We Learned

When we started on our Alexa journey we had high expectations about what we could achieve. These were quickly dashed upon the rocks as we realised what, at least as we see them, gigantic, inherent flaws Alexa has. The game we ended up producing is a shadow of what we had originally planned (it involved robotic arms) and it was entirely limited by the way Alexa has been built.

About 60% of the time we spent working on this project was spent coming up with workarounds to limitations that we feel should simply not be there. Many of the features present on the web interface for the game are attempts that try to address these limitations.

Hypothetically you can make Alexa do anything that you can program a computer to do, however, in practice, this is far from the case. The way Alexa has been built to interact with a user makes it next to impossible to deliver a good user experience in all but a few use cases.
At this current time, the technology seems to be mostly gimmicky with its key selling point being the novelty of being able to speak to it and get a response back. That novelty wears off quickly though and it becomes an incredibly frustrating experience to work with and use.

Looking through the app store many of the negative reviews are because of Alexa’s poor ability to understand what you say, and the fact is that most users will put that blame for that on the developer of the app, not realising it has nothing to do with them but is entirely down to Alexa simply not being able to understand you.

I would be genuinely concerned about launching an app for a business on Alexa as users are sure to put the blame for any negative experiences on you, and that negative experience of your product will stay with them for a long time.

To be fair the technology is still in its infancy and it will surely get better as time goes on, but at this time, for all but a few very niche use cases, we would advise against building applications for Alexa, at least until the technology matures.

For all its flaws and limitations we have made the BEST Chess game on Alex called Chess Mate, if you have an Alexa you can download it here.

If you have an idea for a Voice App you’d like to talk to us about then do get in touch as we’d love to talk about it with you.

Got an idea you want to talk to us about?