
Amazon Sends 1,700 Alexa Voice Recordings to a Random Person - 0xmohit
https://threatpost.com/amazon-1700-alexa-voice-recordings/140201/
======
madrox
I've never seen a demographic breakdown of HN, but I'm 37 years old. I'm
definitely not the oldest guy here, but I see history repeat itself regularly.
I remember when people used location tracking as a reason to not own a cell
phone. I'm talking about flip phones, mind you, not even iPhones with GPS
receivers. I recall the bulk of complaints in that vein died down around the
time Google Maps for phones came out.

If this happened today but with Verizon and someone's location via cell tower
records, no one would use that as evidence that no one should own a cell
phone. That probably tells you what you need to know about the trajectory of
voice assistants.

~~~
lisper
There's a big difference: having a GPS in my phone means I never have to worry
about buying or carrying a map with me ever again, at the cost of potentially
revealing my location to someone who I'd rather not know it.

Having a voice assistant means I don't have to spend a few seconds typing, at
the cost of having everything I say potentially overheard by someone who I'd
rather didn't hear it.

By my quality metric, the first is a big win at a small cost while the second
is a tiny win at a huge cost. YMMV.

~~~
jancsika
> Having a voice assistant means I don't have to spend a few seconds typing,
> at the cost of having everything I say potentially overheard by someone who
> I'd rather didn't hear it.

You're underplaying it in order to artificially inflate your mileage.

Having a voice assistant means not having to _learn to change application
contexts inside an operating system_ follwed by a few seconds typing. You are
ignoring the hundreds/thousands of hours you've spent learning which software
is appropriate for which contexts. Not to mention the latency of instantiating
them, or possibly even installing them, migrating data between them, etc.

Voice assistants have contexts but they map fairly closely to natural
language. And non-technical users can teach other non-technical users what
those contexts are. "Here's how you set an alarm." I've never used Alexa but
I'd bet it's something like, "Set an alarm for blah," or even, "Wake me up at
blah." And I bet if I asked an Alexa user how to do that I'll remember because
it's all natural language. That's a huge win for usability.

Now try teaching a non-technical user about cron, its flags, and its
relationship to other shell commands, using natural language. In a decade that
bullshit will sound more antiquated than the amplitude modulation on the voice
of a Dalek.

~~~
iforgotpassword
> Having a voice assistant means not having to learn to change application
> contexts inside an operating system follwed by a few seconds typing. You are
> ignoring the hundreds/thousands of hours you've spent learning which
> software is appropriate for which contexts. Not to mention the latency of
> instantiating them, or possibly even installing them, migrating data between
> them, etc.

I can type everything I'd issue as a voice command into my home screen. I have
to install and learn just as few/many things and commands as I'd have to using
voice recognition. Op might have underplayed something, you just made things
up.

~~~
poooogles
>I can type everything I'd issue as a voice command into my home screen. I
have to install and learn just as few/many things and commands as I'd have to
using voice recognition. Op might have underplayed something, you just made
things up.

You are not the market that these are targeted at then. The normal person
cannot do those things.

~~~
iforgotpassword
A normal person cannot type an English sentence? Wow, I really seem to he
disconnected from normal people.

------
accnumnplus1
This is just facebook all over again. Why can't people see trouble-tech? I
never used facebook and won't ever use an alexa, nor whatever google's thing
is.

~~~
matte_black
The thing is that for most people avoiding this kind of tech and platforms is
unlikely to have any kind of payoff for them. Life is too short.

Most peoples lives are as simple as going to school and work, raising a
family, taking vacations, paying bills, and then passing away.

There will never be a moment where they can say “ _Yes! My avoidance of
Facebook and Alexa for all my life has finally paid off! Hahaha!_ ”

The only people who can say that are people who have good reasons to hide:
drug dealers with multiple lines of business, professional black hat hackers,
home grown terrorists, life insurance fraudsters, etc.

The average person has nothing of interest to offer except more information
about them in order to better perceive trends and distribute ads to them so
that maybe they buy something.

~~~
bzbarsky
Your list is missing "pro-democracy advocates", "Muslims", "Christians",
"Jews", "atheists", "women who want an abortion", "people trying to escape an
abusive relationship", "people with interesting medical conditions", etc,
depending on location.

Please don't pretend that both "only criminals have something to hide" and
"criminals are by definition bad people" are true statements. Neither is true.

~~~
matte_black
If you live under a government where those examples are a reason to hide, then
in the eyes of that government you are technically a criminal. Doesn’t
describe _my_ government.

Even under the most oppressive governments though, there will still be a
sizable majority that doesn’t have anything to hide and can feel secure using
most technology, and they’ll be fine.

~~~
bzbarsky
First, having talked a lot to people who actually lived under a somewhat
oppressive government (USSR, and by the 80s it wasn't even that oppressive),
most people _did_ in fact have something to hide.

Second, just because it doesn't describe your government now doesn't mean it
won't in the future, unfortunately. Consider examples of a 1920s German, or a
1960s Iranian (the old regime was oppressive too, but in different ways), or a
2000s citizen of Poland.

Third, there are certainly places where the danger is not from the government
but from your neighbors, including to some of the groups I listed. So thinking
about this only from the perspective of governments and criminality is off.
Hence my claim that "only criminals have something to hide" is a false
statement.

Fourth, I suspect that perception of how much one has to hide is highly age-
dependent. As people get older they discover more and more things that they
either want to hide or wish they could have hidden. Maybe I'm wrong in my
guess as to your age, of course, but if I am not, I strongly urge talking
about this sort of thing with people a few decades older. It can be _very_
eye-opening. Certainly was for me.

~~~
matte_black
> Second, just because it doesn't describe your government now doesn't mean it
> won't in the future, unfortunately. Consider examples of a 1920s German, or
> a 1960s Iranian (the old regime was oppressive too, but in different ways),
> or a 2000s citizen of Poland.

This is the risk averse argument against certain technologies. If you believe
your government won’t change and are willing to take that bet, you can lean
into technologies very hard and reap the benefits.

Of course, if you think there may be a day where your government will change
and use technologies against you, you could live out in the boonies and never
even touch a computer, and thus get no benefits from emerging tech.

It’s a trade off between being at the bleeding edge and being as secure and
private as possible. Personally, I have made my decision. Could it be my
undoing one day if government decides I’m guilty of everything? Sure, but I
doubt it will happen. I’ll take the bet.

~~~
bzbarsky
As I said above everyone can make this decision for themselves, and should.
It's just important to me to not paint any of the decisions here as "only
criminals would do that"! Past that, I figure adults can and should decide
these things for themselves.

That said, I also think we should work toward changing the nature of the
tradeoff, e.g. by changing how voice assistants work to minimize the privacy
issues. I suspect there's a lot of work we could do on that front while not
compromising the functionality of the voice assistants.

------
amai
There is an alternative to the spying devices of Amazon, Apple, Google and
Microsoft:

[https://snips.ai/technology/](https://snips.ai/technology/)

"Our technology runs on-device, works offline and guarantees Privacy by
Design"

~~~
flukus
I can't believe that so many people, especially the ones on tech sites like
this, bought into the idea that speech recognition is such a hard job that it
needs to be run on the supercomputers of tech giants. We had somewhat decent
voice dictation software on desktops 20 years ago, when 100Mhz processors and
32MB of RAM were top of the line, yet now it's impossible with an order of
magnitude more resources.

~~~
Analog24
20 years ago you spoke directly into a microphone and could only use an
extremely limited set of supported languages and locales/accents.

You're also missing the point. I don't think anyone has ever claimed that
speech recognition can only be done on supercomputers. Your laptop can surely
run one of these models (though it would take a long time to train one). But
there's a reason why an Echo Dot cost $20 and not $1000.

~~~
pseudalopex
A Raspberry Pi can run Snips. I don't know how long it takes to train or how
well it works.

------
rdtsc
I am baffled by how many people have no problems adding always listening and
recording Google and Facebook devices to their living rooms.

Some are considered tech people who should know stuff like this could happen,
but they just don't care or don't think it will happen to them.

Somewhere there is an ex-Stasi officer who is thinking "I wish we were that
good and had people fooled to voluntarily install listening devices and didn't
have to tap phone lines and crawl around basements and rooftops".

~~~
stevecalifornia
1) It's always listening for a keyword, and then it starts recording. It's a
voice interface for a computer.

2) Your phone is doing the same thing except for it follows you around
everywhere you go and records your map location, what apps you use, who you
are talking to and what your web searches are. Your phone isn't recording just
what you say...its recording what you do and where you do it and when you do
it. It's overwhelmingly more pervasive and invasive if you value privacy.

3) The utility of being able to interact with a computer via my voice wildly
surpasses the risk. Even if it had been my voice sent to this random other
person all they would get is 3000 recordings of me saying "Alexa, what is the
weight of an elephant?" "Alexa, play Magic Sword." "Alexa, whats the weather?"
My father-in-law thought these things were the devil until he realized he
could ask it to play ANY SONG HE WANTED and it would do it. Now he is always
unplugging ours and plugging it back in on the porch so he can use it out
there.

Literally every article about home assistants brings out a number of people
who continue to say how they can't understand how people could have this in
their homes. Great, you can't understand it. Meanwhile, an enormous number of
people find immense utility in home assistants otherwise they wouldn't be
proliferating so wildly for this long.

~~~
LinuxBender
1) Nobody can prove that. It's always listening and you have to assume always
recording and transmitting.

2) My phone is a $12 throw away that speaks voice and sms (text).

3) Loose lips sink ships.

~~~
Klathmon
If we are playing the "you can't prove that" game, then you also can't prove
that your $12 throw away phone isn't always recording, or that you new kitchen
countertop doesn't have a recording device embedded in it, or that the maker
of whatever device you are using to type that comment isn't breaking a
plethora of laws to record everything you type or say or do around that device
at any time.

~~~
LinuxBender
When my throw away phone is transmitting, it gets warm. I also set it next to
an amplifier and I can actually hear it bleed into the amp when it chats with
the cell site. I can even tell a couple of seconds ahead of time when I am
getting a text or a phone call.

~~~
Klathmon
And all a hypothetical "bug" needs to do is listen for some key phrases, and
send a single byte or a few bytes of data across the wire, even through side
channels.

I can imagine a system that throws an extra byte or 2 on those "chats" with
the cell tower, or even abuses the timing that it decides to reach out and
check the tower as the method of communication.

The actual voice recording and processing tech is small enough that it could
be thrown right alongside the actual phone, for pretty cheap, and could run
entirely separate from the phone itself, maybe even having its own battery!

------
danso
I don’t understand how this problem of “human error” happened at a company
like Amazon/AWS. Perhaps requests for Alexa info are rare enough that the
internal interface for servicing such requests isn’t fully automated. But I’d
be shocked if the process involved someone (low-level data entry person, or
engineer) manually typing in a customer ID number.

~~~
testplzignore
There is obviously more to this story than Amazon is telling.
[https://www.heise.de/downloads/18/2/5/6/5/3/9/6/ct.0119.016-...](https://www.heise.de/downloads/18/2/5/6/5/3/9/6/ct.0119.016-018_engl.pdf)
says that this was a "one-time error" and that "Amazon also claimed that they
had discovered the error themselves". It is highly unlikely that either of
these are true. The one time they made this error just happens to involve
someone who is savvy enough to contact the right journalists to investigate
it? And Amazon coincidently discovered the error themselves after being
contacted by c't? Amazon is flat out lying.

If there is ever a case for the max GDPR fine to be imposed, this is it.

------
closetohome
I'm having trouble getting worked up about this one. Yeah a privacy breach
happened, but it was only one person's data exposed, and only to one other
person.

The only reason it made the news is because people are already paranoid about
voice assistants.

~~~
tgsovlerkhgsel
Was the data supposed to be stored in the first place according to the privacy
policy/user consent? If not, it'd mean that Amazon stored highly sensitive
data (audio recordings from people's bedrooms) illegally and in breach of
user's trust.

~~~
DaniloDias
if you enable Alexa to tune to your voice, it retains recordings. You can
listen to the recordings in you Alexa mobile app- so it shouldn’t be that much
of a surprise.

It shouldn’t be a shocker if you understand ML and infer what “tuning to your
voice” implies. Most people aren’t sophisticated enough to infer that,
however.

To me- this is a big demo of unexpected consequences of GDPR. Sounds like a
great law, but forcing companies to share everything they know about you
increases the risk that a nice package of everything is shared with the wrong
party.

------
randyrand
Is there a good non-internet connected device for turning on lights with your
voice?

~~~
isostatic
Children

~~~
rpeden
I've heard they're expensive to maintain and eventually stop responding to
commands.

------
pasbesoin
Among other things, this makes me suspect that there is no formal internal
audit record/trail with regard to accessing these records.

We are left trusting individual Amazon employees to do the right thing.

Most will, anyway. But hand in hand with "privacy" comes "accountability", on
the part of records holders.

------
miles
Not the first time:

Amazon Echo records private conversation and sends to co-worker
[https://www.youtube.com/watch?v=IQrv5GbOKhc](https://www.youtube.com/watch?v=IQrv5GbOKhc)

~~~
samplesize
User who doesn't understand how device works accidentally sends voice message
to coworker.

~~~
ndnxhs
The original article stated that they do know how it works but the device
malfunctioned and detected a command where there was none without the owner
even being aware that it was activated.

------
SlowRobotAhead
> _" Using these files, it was fairly easy to identify the person involved and
> his female companion; weather queries, first names, and even someone’s last
> name enabled us to quickly zero in on his circle of friends,” according to
> the report. “Public data from Facebook and Twitter rounded out the
> picture.”_

Please keep in mind the next time you or a friend says "well, what do I care
if Amazon knows when I ask to turn the heat up".

I read about a cool Echo Dot hardware hack where it was made into an old phone
with mute on the speaker and microphone until picked up. As a tech people it
boggles my mind that other tech people accept an always on microphone in their
homes. I truly don't get it!

~~~
sciurus
Speaking of phones, aren't most people's smartphones already "an always on
microphone in their homes"?

~~~
tjoff
On the vast majority of devices it is not always on.

It is not a nitpick. Accidentally recording or even intentionally recording is
many orders of magnitude simpler when the device is supposed to always be
listening.

The fact that a single employee even could make the mistake mentioned in the
articles is one example of why. Eventhough it also really does hint that the
backend isn't up to par at amazon.

~~~
dlubarov
> On the vast majority of devices it is not always on.

Would you consider the device "listening" if a DSP is listening for a
particular hotword, but no audio data is being uploaded?

I'm pretty sure that's what happens in either case, whether you're using
Alexa, Siri, or Google Assistant, and whether it's a "home" device or a phone
with hotword detection. I used to work at Google on some related hardware
projects.

If you're suspicious because the home devices seem much more accurate, that's
because they have microphone arrays which can facilitate beamforming.

~~~
tjoff
Yes.

There have also been issues of google devices accidentally recording
constantly. And then "accidentally" stream it to the cloud.

[https://www.gearbrain.com/google-home-mini-recorded-
everythi...](https://www.gearbrain.com/google-home-mini-recorded-
everything-2495445625.html)

The level of safe guards google et al put in to deal with stuff like this are
incomprehensible poor.

The accuracy is irrelevant.

------
withinrafael
I'm a generally happy Echo owner but am getting increasingly worried these
mistakes are only going to ramp up, especially with the new Drop In feature,
allowing users to enable the microphone on any Echo -- not limited to their
own -- given the appropriate permissions are in place. (It's not clear if
these are enforced at all on-device or simply cloud-based permissions.)

[1]
[https://www.amazon.com/gp/help/customer/display.html?nodeId=...](https://www.amazon.com/gp/help/customer/display.html?nodeId=202153130)

------
vufufuffu
I'm just waiting until people become accustomed to these devices and the tech
giants reveal their new real estate play with these devices always on and
built into the walls of homes.

EDIT: I am definitely feeling what Yanis Varoufakis was talking about when he
said that if something isn't done soon, The Matrix will be a documentary.

------
dixie_land
Does Amazon keep the raw recordings for GDPR compliance reasons?

It seems to me keeping them just creates liability? (Yes data analysis for
targeting and such would be valuable, but those can be done with, say
transcripts instead of the raw recordings)

~~~
samangan
The actual recordings are the most valuable data for modeling purposes so I
actually assume they try to store it to improve the system

~~~
jonas21
I'd imagine that a competent company would keep access to user audio
recordings tightly locked down, either because they actually care about
privacy or because they care about the bad PR of recordings getting leaked.

But if the GDPR requires them to send the recordings to users upon request,
they need to have a way to do that -- either by exposing a service on the web,
or by providing access to customer service employees to handle the request.
Not sure how I feel about that.

~~~
roywiggins
The Alexa app lets you play back your own recordings.

------
zmix
Didn't Amazon tell it's customers, that Alexa constantly records locally, but
only sends the words after the "Alexa" magic, including 6 seconds before and 6
seconds after it, to the servers?

~~~
archgoon
There is nothing in this article that disproves that; and it likely would be
featuring prominently if it were more.

Also, it seems quite silly for Amazon to lie about this if it could be
disproved with a GDPR request.

The mistake that was made here is that the wrong customer's data was sent in a
GDPR request.

~~~
zmix
You are right. But I remember, that I had read a similar story, a week, or
two, ago and the topic was, that the police found out about the killer in a
murder case, by listening to Echo recordings from the kitchen. I doubt, that
this would be possible, if there is no constant stream of data on Amazon's
servers.

That gave me the impression, that they store it all. At least for a while.

------
onetimemanytime
I feel for average, clueless Joe that has limited knowledge, but anyone that
has average and above average knowledge of tech, deserves what he /she gets in
these cases. How stupid or tech ignorant would you have to be to add a device
that records you in _your_ home?

Imagine, fights /sex with wife, lover, arguments with kids, talk about drug
deals, tax evasion, leaving job, stealing neighbors home, cheating on husband
/wife...WTF would you want to take the chance that a recording device is even
within 10 000 feet of your home?

~~~
Kiro
Still worth it for me. And if they actually recorded all that and used it
against me it would be like living in a real dystopian science-fiction movie,
which would be an awesome experience in itself. Similar like how you want to
experience a zombie apocalypse. Yes, I think it's that unlikely.

------
noobiemcfoob
If civilization keeps increasing data production around ambivalent use cases,
perhaps we can successfully hide >99% of the lethal data siting around out
there until its utility expires.

------
gigatexal
I paid full retail for my HomePod (BestBuy in the states has them for 150 usd
off) and love it. But I love it more because of the company behind it: Apple
is not in the business to sell my data to the highest bidder. And their stance
on security and customer privacy allowed me to put a talking tube in my home.
There was no way in hell I was going to add a listening device from Google or
Amazon.

------
tjpnz
It wasn't long ago that people were up in arms about the Xbox One sending
audio back to Microsoft in order to implement voice commands. In the time
since this kind of thing seems to have become acceptable or even mundane. What
is it that changed? It's not as if big tech has become more trustworthy in
recent years.

------
_Codemonkeyism
After two years of Alexa it will move out 2019.

Experience was underwhelming, especially play lists from Spotify were hit and
miss, one day no problem, the next day sometimes an excuse, sometimes just a
beep, sometimes silence. I would need to turn on the light by myself in the
future though.

------
bduerst
Does this qualify as a GDPR violation?

~~~
whoisjuan
Very unlikely. In fact, the error came from an actual GDPR Right of Access
request. If they inform the affected person and take measures to prevent it
from happening again, then they're in compliance of GDPR.

GDPR was created to prevent organizations from hiding data breaches, trading
or transfering customer information, avoiding privacy compliance, etc.

This is making the news because is Amazon, but you will be surprised how many
of these breaches happen in a daily basis in all types of organizations.

------
burtonator
When I was in uni root send me the unshadowed /etc/passwd file... no idea what
that was about. Just logged in one day with an email of 'passwd' and
everyone's passwords were there salted... almost seemed like they were daring
me!

------
dbg31415
What happened to all the layers of security Alexa is supposed to have to
prevent data from being sent upstream, isn't it supposed to process everything
locally? Eww. Come on guys, Rule 1: Don't make creepy tech.

------
Kiro
I'm all against unwarranted surveillance but for voice assistants you
basically sell the risk of privacy intrusion for utility. You're making an
active choice. I see nothing wrong with that.

------
purplezooey
Glad we have a GDPR-like thing in California and wish some of the lesser
states would get their house in order. :)

------
bitrrrate
There’s just no end to the abuse of our data is there? Sorry, no Alexa, Siri,
Google Assistant for me.

------
ivolimmen
He probably asked for it.... xD

