How Private Is Amazon Echo?

Put a microphone in your product, and someone is going to assume you're listening to them. That's one of the challenges Amazon Echo – the online retailer's "Siri in a totem pole" – faces, with suspicion about just how much Jeff Bezos & Co. (or his algorithms, at least) are actually eavesdropping on. Given the power of Amazon's recommendation engines and the amount of data it gathers just from casual browsing, you can certainly see where some of the paranoia might come from, too. A microphone-mute button takes pride of place on top of Echo, but will it be enough to persuade potential users that the virtual assistant is working for them and not for Amazon itself? I went hunting for some answers on just what Echo shares and how you can tame it.

Amazon bills Echo as your family's friend: answering the kid's questions while they do their homework, and keeping a running shopping list for the parents without making them reach for a pen in the kitchen. The concern, though, is just how much listening Echo is doing.

As soon as the product was revealed, in a quiet pre-registration launch on Thursday this week, critics were already making semi-serious jokes about how sensible it would be to pay Amazon for the privilege of bringing Echo into your home.

Echo's technology aims to look like magic to users, but it's actually built on top of several acquisitions Amazon has made over the past few years. Back in late 2011, for instance, the retailer bought Yap, a voice-to-text and speech recognition company that at the time was offering services like voicemail transcription.

In 2013, meanwhile, Amazon bought both Evi – a Siri-style app which had already been pulled from the iPhone's App Store for apparently being too close to Apple's own functionality, though was later restored – and IVONA Software, a text-to-speech specialist.

IVONA was already providing the tech which powered the Kindle Fire's ability to read out text on-screen, part of its "Explore by Touch" feature among others.

Some of those talents have already been implemented in Amazon Dash, a barcode-scanning wand that can also be instructed by voice as to what you want to buy through the Amazon Fresh grocery delivery service. Dash's voice recognition is triggered by a button, rather than the always-listening feature on Echo.

Exactly what Echo is listening to, though, has led to some confusion. Just as with the Moto X, "always on" actually means "always listening for a specific command." In Echo's case, that's either "Alexa" or "Amazon", either of which will wake the column up and start it monitoring for more complex instructions.

Over time, Amazon tells me, Echo will support more wake-commands, though it's uncertain whether there'll be support for user-customizable commands as Motorola added in the 2014 version of its phone.

What you also get is a physical microphone mute button on the top. Pressing that shuts off even the trigger word system and, in theory, stops Echo from listening to you at all.

Of course, whether people will actually walk all the way over to wherever Echo is plugged in and press that button is questionable. The device's seven microphone array and beam-forming technology is designed to listen to you wherever you are in the room, which suggests Echo can hear clearly from a lot further away than the length of most peoples' arms.

A spoken command to mute the microphone – which would require physically re-enabling it afterwards, though even when muted you can still trigger with the voice button on the bundled Echo remote, Amazon pointed out to me. - could be a useful addition. Amazon will be able to remotely upgrade Echo with new features, as well as add voice-controlled services in the cloud.

As with other voice recognition systems, Amazon relies on recordings to improve the quality of its algorithms. While there's an optional Voice Training system in the Amazon Echo App, the wireless speaker will do run its own learning process in the background, too.

"When you use the wake word to talk to Amazon Echo, the audio stream includes a few seconds of audio before the wake word, and closes once Amazon Echo has processed your question or request," Amazon told me. There's also an optional "wake up sound," the company said, effectively a short tone that indicates Echo is streaming audio to the cloud.

A similar "end of request" tone can be activated, too, to flag when Echo has stopped streaming.

There'll also be access to all of those recordings if you log into Amazon's browser-based dashboard. Interactions are grouped by question or request, Amazon says, and you can not only listen to the clip but send feedback about how well it was processed.

Individual clips can be deleted, or the entire history wiped out, though Echo's accuracy will take a dive back to out-of-the-box condition if you do so.

Nonetheless, Amazon is still doing a little extra homework in the background as a shortcut to knowing you better. Things like your music playlists are automatically processed by Echo's voice services, "to improve response time and accuracy"; similarly, Amazon reserves the right to share your music titles, radio stations, and zip code with "third party services", though doesn't confirm who they might be.

In the end, how much you trust Echo comes down to a more wide-ranging question about how you trust the cloud. On the one hand, there have been enough data leaks and privacy goofs over the past twelve months to make you understandably wary of what's going on with the servers of other; on the other, if you want the convenience features, cloud processing is still the most efficient and flexible way of doing it.

A minor software tweak could certainly improve Echo's usability in getting a temporary respite from its eavesdropping, and it's not something that would be too tricky for Amazon to implement. Still, for the truly privacy-concerned, any sort of microphone – no matter how many buttons to turn them off they have – is going to be a no-go area on a device.