
Amazon device recorded private conversation, sent it out to random contact - spking
https://www.kiro7.com/www.kiro7.com/news/local/woman-says-her-amazon-device-recorded-private-conversation-sent-it-out-to-random-contact/755507974
======
yason
I am oversimplifying here but:

* install device that is designed to listen to speech in the house

* the device is connected to internet

* the device is capable of contacting other internet peers/services/hosts

* the device knows a bit about its owner's internet presence such as contacts

* the device is equipped with simple conversational user interface based on fuzzy human speech-command detection

These basically bound the failure modes of the complete system. I am not
surprised that a glitch like that happened. I can certainly attest to how
shocking it might be to discover first hand that it really did happen, but
given the context I can't say it's hard to not foresee something like this.

~~~
duxup
I remember the first voice command stuff on google. Buddy of mine was trying
it out and I couldn't help myself but shout:

"NEW TEXT MESSAGE TO JENNY! FUCK YOU! SEND!"

Fortunately the phone was too slow to take it all (even if it had Jenny would
have found it amusing, she was a cube or two away). But yeah first thing I
thought of was that if it's listening how easy is it for it to mistake intent
or someone else or etc.

Humans talk funny, computers don't get it, they're gonna mistake commands all
the time.

As it is I hate how I can't talk ABOUT "ok google" around my phone... because
it always goes off.

~~~
monk_e_boy
My friend installed IBM OS/2 WARP beta when it came out, it too was voice
controlled. We would shout into his room, "select all, delete" or "format c"
or other childish fun commands.

~~~
duxup
Humans are terrible. When the computers take over.... I won't be happy about
it, but I'll kinda understand.

~~~
eXpl0it3r
Just shout "self-destruct" all the time and we'll be fine.

~~~
vidarh
Alexa has 4-5 joke responses to self destruct. E.g. "I'll start the countdown
but only on the understanding you'll dramatically cancel it at the last
second", and "3-2-1 Boom. Hmm that didn't quite work did it?" or something
like that, and a few others.

~~~
teddyh
Like the countdown at the end of _Monsters vs. Aliens_ :

[https://www.youtube.com/watch?v=FJ5DA22tBOY#t=4m27s](https://www.youtube.com/watch?v=FJ5DA22tBOY#t=4m27s)

“8… 7… 6… 5… 4… 3… 2… 1…

…

Hmm, nothing happened, maybe my count w… [ＢＯＯＭ]”

------
dustinmoorenet
I have this idea of a system I would like to have in my house. It contains
cameras in every room that are constantly watching where people are and
relaying the coordinates to a central server. That server makes decisions on
if lights should be on or if A/C should be running in that room. But I would
never buy this system. I would have to make it myself. I am hopeful that open
source software and hardware can produce individual components that I can
trust to piece together.

~~~
kinsomo
> I have this idea of a system I would like to have in my house. It contains
> cameras in every room that are constantly watching where people are and
> relaying the coordinates to a central server. That server makes decisions on
> if lights should be on or if A/C should be running in that room. But I would
> never buy this system. I would have to make it myself. I am hopeful that
> open source software and hardware can produce individual components that I
> can trust to piece together.

You don't need cameras for that, just motion sensors.

> That server makes decisions on if lights should be on...in that room.

You don't need a server for that, just a motion-sensing switch. They can be
totally offline. My office has them to _shut off_ lights automatically when a
conference room becomes unoccupied.

[https://www.homedepot.com/b/Electrical-Wiring-Devices-
Light-...](https://www.homedepot.com/b/Electrical-Wiring-Devices-Light-
Controls-Motion-Sensors/N-5yc1vZc32r)

It can get annoying if you're not moving much, but the key point is: no camera
or server required.

> That server makes decisions on...if A/C should be running in that room.

Ecobee thermostats can already do that using motion/temperature sensors. They
run fine without an internet connection.

[https://www.ecobee.com/room-sensors/](https://www.ecobee.com/room-sensors/)

~~~
carbocation
Might be cool to add thermal sensors to that, so the lights don't shut off
because you're not moving enough.

~~~
Cthulhu_
That's the main problem with motion activated office lights tbf, especially
when you're doing software development for example - not enough motion to keep
them on.

~~~
eric_h
My problem with the motion activated office lights (at least the ones at the
wework my company is located) is that you can't turn them off. The button
appears to turn the lights off and disable the motion sensor for some (short)
period of time, then movement turns the lights back on again. Very annoying.

~~~
hug
My office has a piece of paper taped over the motion sensor on the roof for
that reason.

------
gnicholas
Amazon's response, from Ars' article:

> _Echo woke up due to a word in background conversation sounding like
> "Alexa." Then, the subsequent conversation was heard as a "send message"
> request. At which point, Alexa said out loud "To whom?" At which point, the
> background conversation was interpreted as a name in the customers contact
> list. Alexa then asked out loud, "[contact name], right?" Alexa then
> interpreted background conversation as "right." As unlikely as this string
> of events is, we are evaluating options to make this case even less likely._

[https://arstechnica.com/gadgets/2018/05/amazon-confirms-
that...](https://arstechnica.com/gadgets/2018/05/amazon-confirms-that-echo-
device-secretly-shared-users-private-audio/)

~~~
mindfulhack
I want some YouTube tech channel to test this out and see if this story by
Amazon is bullshit or not. Have a private-sounding conversation and
intersperse it with a sudden nonchalant, 'hey, alexa visited me today and...'
and then keep talking and then 'I really want to send a message to these
students, you know and...' and so on and so forth.

~~~
tetrep
I think it's decently plausible if your Alexa hears you, but you don't hear
your Alexa. So you continue on your conversation, and you say enough to keep
progressing the flow.

I would assume any failed speech recognition attpempts are recorded so Amazon
can have a human look at and classify them, but at the very least they
probably keep logs, so they should have this information when debugging. Maybe
we'll get a more detailed postmortem later.

------
michael_j_ward
I received one of these messages just a week ago. Alexa sent me a message of
my friend and his girlfriend having a private conversation. I immediately
texted him to ask if he intended to do that and he did not- so weird.

~~~
nicky0
Can you explain a bit more. What was the context of the message? was it an
email? What was the subject line and the text? Was the audio an attachment or
a link? A link to where...?

"Amazon sent me a message" is so frustratingly vague...

~~~
amhokies
You're able to send voice messages to other people that have an Alexa.
Basically like voicemail.

~~~
p49k
That means that, unlike butt-dialing, you're more likely to accidentally send
a message to someone you're actually talking about since the person's name is
part of the command. Yikes!

------
asn0
Summary of vague technical details (which may be all we hear about this):

> an Alexa engineer investigated ... they said 'our engineers went through
> your logs, and they saw exactly what you told us, they saw exactly what you
> said happened, and we're sorry.' He apologized like 15 times in a matter of
> 30 minutes and he said we really appreciate you bringing this to our
> attention, this is something we need to fix!"

> the engineer did not provide specifics about why it happened, or if it's a
> widespread issue.

> "He told us that the device just guessed what we were saying" The device did
> not audibly advise that it was preparing to send the recording, something
> it’s programmed to do.

~~~
DoofusOfDeath
> The device did not audibly advise that it was preparing to send the
> recording, something it’s programmed to do.

Apparently it's _not_ programmed to do that. Unless this was a hardware glitch
or cosmic-ray event.

~~~
cortesoft
As developers, we use 'programmed' in that manner all the time, to mean what
we were trying to get it to do rather than what it actually does.

~~~
DoofusOfDeath
I don't. And in this particular scenario, it's an important distinction to
make because there's a risk of Amazon using equivocation to deflect
responsibility.

~~~
cortesoft
If this was intentional, then yes they shouldn't be able to deflect
responsibility. However, if it was just a bug, I think it is a bit unfair to
vilify either the company or the developers. Bugs happen, and hopefully they
can learn what caused this and prevent this class of bugs in the future.

~~~
toss1
And this is where "software" diverges from "engineering".

Bugs happen in architecture, aircraft, etc. too. the difference is that the
actual engineers are paid to have a precautionary approach and spend
significant resources to actively prevent bugs from making it into the final
product.

In contrast, software is often written to "ship first", be "agile", and "move
fast and break things". Yet when it causes problems, they just say "bugs
happen", and "it is unfair to vilify them".

Features are not better than reliable security.

And yes, negligence is less bad than malice, but it is still damaging and
developers and managers need to be held to account.

If your manager is pressing you to do unsafe crap in too big a hurry, it's
your responsibility to push back, and if unsuccessful, leave for saner
pastures and make it more difficult for that management to proceed.

~~~
cortesoft
> Features are not better than reliable security.

Well, yes and no. Bugs, security, reliability, etc are all important things to
consider, but they can't be the only focus. Security doesn't matter if the
thing you are creating has no features; it would be the same as if it didn't
exist at all.

Instead, we must manage risk; the risk of bugs, the risk of security
vulnerabilities, etc. Nothing we do is risk free. Even walking across the
bathroom floor has SOME risk; we might slip and fall. Does that mean we should
just stay in bed all day to avoid any risk?

No, we measure risk by factoring the chance of the bad thing happening and the
consequence of the bad thing happening. We then determine how much effort we
should spend on that risk, since there are infinite risks and only a finite
amount of effort we can expend.

If the consequence of a risk is death, then we should absolutely put a lot of
effort into minimizing that risk. If the consequence of a risk is that a
private conversation is sent to a contact, we should definitely put a lot of
effort into minimizing that risk, but probably not quite as much as you would
into something that has the consequence of death.

Even when the risk is death, however, we don't put infinite effort into
avoiding it. We choose to cross the street, even when we know there is a risk
of death when we do it. We drive cars that have chances of mechanical failures
that could cause our death, but we don't bring the car to the shop every day
to check for failures.

Things are not so black and white as to say "security is always the most
important thing"

~~~
toss1
I did not say >"security is always the most important thing". I did not
suggest that anyone should develop such that >" the thing you are creating has
no features". I certainly never suggested that no one get out of bed because
they might slip in the bath. These are strawman arguments.

You do not need to lecture me about risk. I've had a career in international
downhill ski racing, have won auto racing championships, and enjoyed lots of
technical rock climbing, all of which require a high degree of risk
assessment, both in extended preparation phases and at split-second time
scales. I've also run risk analysis for UAV flight systems

I understand well the difference between smart-crazy and dumb-crazy, and where
the pseudo-mathematical risk models like yours break down.

Your 'analysis' to "...measure risk by factoring the chance..." would have fit
right in at the meetings where Ford decided to just go ahead with the design
of the Pinto/Bobcat because the lawsuits would cost less than the fix -- they
wound up killing dozens of people.

Your 'analysis' would have fit right in where the trading algorithms were
being designed, which worked fantastically profitably, until they didn't and
ended up crashing the global economy in 2007-8.

You cannot simply multiply the cost of the consequences by the expected
probability and get an allocation of resources. That is what you do to see if
the lottery jackpot is big enough for you to want to buy a $2 ticket this
week.

You must instead 1) fully examine the system for potential critical failure
points/modes and then 2) allocate WHATEVER resources are necessary to account
for preventing those critical failures, then implement those remedies along
with the features.

These preventative measures may involve installing redundant systems around
the critical points, redesigning the points so they fail in a safe mode (e.g.,
fail to send the data vs sending it off, shut down vs, explode, etc.), adding
check procedures around the potential critical failure, etc.

Note that NONE of these measures involve not implementing the feature. They
involve 1) checking for critical failure modes, 2) allocating R&D to develop
preventative & fail-safe measures, 3) implementing the measures, 4) testing,
and 5) field monitoring.

This is what you do if you are serious about risk.

~~~
cortesoft
Ok, I never said that the 'analysis' should be purely based on dollar value. I
never even said it should be a mathematical model. You accuse me of a strawman
and then turn around and do the same to me.

I was simply pointing out that we can never get to zero risk, and since we
can't, we have to weigh risks based on consequences and probability.

> 1) fully examine the system for potential critical failure points/modes

Sure, to the best of your ability. How can you know for certain you have found
all potential critical failure points? You can get pretty sure, but never
fully sure. We still have industrial accidents, in every single industry in
the world.

You also have to define what a 'critical risk' is. I don't think it is an a
priori fact that accidentally sending a recording of a conversation to a
contact is a 'critical risk'.

~~~
toss1
So your model isn't even mathematical, it's what, just a SWAG of the combined
hazards and odds? That works for linear, small risks.

It absolutely does NOT work for serious risks, e.g., of death, serious injury,
massive privacy violation, and other potential life-changing events.

The concept you are clearly avoiding or missing non-linear risk.

You (and amzn_engineer1) are advocating for simply subsuming risk assessment
into the ordinary development cycle, and calling it "taking it seriously".

That is fooling yourself.

Taking it seriously is actually making full and serious effort OUT OF THE
NORMAL DEVELOPMENT CYCLE for no other purpose than to SEEK and identify
potential critical risks.

It is then engineering a variety of in-depth solutions to prevent those
critical failure points from ever seeing the light of day. And implementing
them. and testing them. And monitoring them.

>> I don't think it is an a priori fact that accidentally sending a recording
of a conversation to a contact is a 'critical risk'.

This is an exact example of this sort of failure: 'it's not a priori bad'...,
minimize it and streamline it into dev.

I really want to know in what world any sane person would say that it's OK to
randomly divulging an intimate conversation to a contact or random recipient
-- seriously, who would say that?

I mean sure, most conversations are benign, but some could be utterly life-
changing if revealed. and that's OK with you?

------
gnicholas
> _But Danielle is hoping Amazon gives her a refund for her devices, which she
> said their representatives have been unwilling to do._

Based on Amazon's admission and apologies, I'd think they'd be willing to give
a lot more than just a refund. Seems like a huge liability for them.

~~~
awat
Unless it’s expected behavior that malfunctioned. Antecdotally I’ve found a
lot of companies support have almost no leeway for appeasement if the
triggering behavior is considered “expected behavior” or a bug based off of
it.

~~~
chaostheory
But this is Amazon. I guess things have changed for the worse on the customer
service front though. This happened not too long ago (2-3 years) - probably
around the time they posted a net profit

In terms of the big picture, this was dumb PR wise and this story should have
never happened. For less than a $1000, Amazon could have stopped the story if
the PR dept were more in sync with customer service

~~~
awat
I don’t disagree at all on the PR side. Spent sometime in the contact center
tools space and my general experience is if a company allows it exempt higher
level employees avoid engaging with customer service directly at pretty much
all costs.

Not saying it happened here but thats how you get $10/hour vendors making
judgement calls on customer situations that are front page news.

~~~
chaostheory
Yes, I also used to work in that space, our software / platform was used by
Fortune 500 companies.

My point still stands. This is Amazon and not random company X's customer
service. There are very few companies like this that people can name (e.g.
Nordstrom, Patagonia, ...) It this used to not happen at Amazon. Customer
service used to be empowered to make customers happy, which was what made
Amazon legendary. Things have changed. Customer service is now crippled like
almost every other mediocre company. This is just one of the symptoms. Maybe
Amazon feels that it has enough market share now, so it doesn't matter as
much?

~~~
awat
I see what you are saying it definitely could be the market share or maybe
even just the overall volume (QA at scale).

I guess I just don’t have a long enough track record with them to see them in
that grouping of great customer service. They’ve pretty much fought me tooth
and nail over price matching and prime shipping that doesnt meet the two
guarantee over the last couple of years which is probably biasing my view.

------
djsumdog
I've told my roommate I'm moving out if he ever buys an Alex/Google/Apple
assistant device.

I have a microphone and I've been intending to get one of the open source
solutions working and just tie it in to mpd, weather and a few other things.
But all the processing should really be done on your own device, by hardware
you own, software that's open and that you configure, and not send up to
someone else's computer (aka "the cloud").

Of course non-tech people probably wouldn't bother because that's a steep
curve and you're talking about more expensive devices to do on-board
processing. There are some companies that are trying to make this more
accessible to regular consumers. I hope we see a move in this direction.

~~~
evv
> I've told my roommate I'm moving out if he ever buys an Alex/Google/Apple
> assistant device.

Do you have an Apple or Google device near you right now? Does it have a
microphone, battery power, and a connection to the internet? What about your
roommate? Do you ever have sensitive conversations near these devices?

Of course I'm not suggesting you should abandon such technology. I'm just
wondering why you draw the line at "Alex/Google/Apple assistant device"

~~~
confounded
> _I 'm just wondering why you draw the line at "Alex/Google/Apple assistant
> device"_

There’s a big difference between:

\- Devices which are by design always actively listening and sending real-time
audio to computers you can’t control or even really trust, with unknown
security properties, and consumer-appliance security life-cycles/support

\- Devices that _can_ be configured to do that, but which you have some
control over (phones, laptops, etc.) and generally won’t, without some form of
consent (even if via a dark pattern, e.g. LinkedIn on Android).

~~~
givinguflac
To be fair, Apple’s homepod doesn’t send any data until it’s activated with
the wake word. Amazon and google products send a ton of data back constantly.

~~~
wilsonnb
Where have you read that Amazon and Google send a ton of data back? I've heard
that none of these devices send anything back until they're activated by a
specific phrase.

~~~
givinguflac
I've seen and blocked it on my own network.

------
amzn_engineer1
What actually happened: Alexa misinterpreted some voice commands and activated
a "call" skill. The people involved and local news got very excited and
escalated this into a conspiracy story.

Amazon takes customer privacy EXTREMELY seriously. There's no way a team would
get the "ok" to build a skill that randomly records private conversations then
sends them to a random contact. It also doesn't make any logical sense to
build such a skill.

Yes, I might sound biased because I am an engineer at Amazon. This statement
is my own and unrelated to Amazon's opinion.

~~~
snarf21
I get what you are saying but I would say that Amazon does _NOT_ take privacy
extremely seriously or this couldn't have happened. Let me be clear that I'm
not saying they don't care at all or they are conspiring with the NSA.

What I mean by the above is that the "call" skill is _much_ different than the
"weather" skill. All Alexa has to do is have a confirmation prompt in the
"call" skill and this wouldn't have happened. That is what extremely serious
looks like. This is exactly the same as the phantom laughter incident from a
few months ago. Alexa "heard" someone say 'Alexa laugh' and laughed, but that
wasn't the user's intent. It was fixed by moving to 'Sure, I can laugh,'
followed by laughter.

Voice UI is very hard and still in its infancy but ability for personal harm
(physical or emotional) must be considered in these interfaces. Turning off
the lights may not need confirmation but unlocking the doors or turning off
the alarm probably should. Sending recordings or answering calls or even
calling people should require more hoops or at least allow the user to control
the risk/reward.

~~~
cwkoss
> they are conspiring with the NSA

There is absolutely no reason to believe they are _not_ conspiring with the
NSA. They have a huge deal with the CIA, plausible some of these funds are for
surveillance capabilities. They would not (and likely would be legally
prohibited from) disclosing any relationship they have.

[https://venturebeat.com/2014/03/18/snowden-slams-amazon-
for-...](https://venturebeat.com/2014/03/18/snowden-slams-amazon-for-leaking-
customer-data-to-the-nsa/)

[https://www.theatlantic.com/technology/archive/2014/07/the-d...](https://www.theatlantic.com/technology/archive/2014/07/the-
details-about-the-cias-deal-with-amazon/374632/)

~~~
smsm42
> There is absolutely no reason to believe they are not conspiring with the
> NSA.

Or, more likely, being made an offer they can not refuse by the NSA. And the
rank-in-file engineers may not even know it's happening, all it takes is
inserting some diverting code into the pipeline and calling it "QA monitoring"
or something. Couple of people in the whole org would know that somebody from
some IP connects and downloads these "QA" data periodically, all the rest
would be completely ignorant and indignant at the thought. Don't see anything
preventing this from happening at Amazon - or anywhere else.

------
gricardo99
The most shocking thing about this story is how a customer was able to get
qualified engineering support for an unusual support request, from a large
company.

Or... what's missing is the hours spent going through support tiers 1 through
N, and somehow not getting dropped/lost along the way.

------
myself248
Back in the early 90s, I remember seeing a cartoon in one of the tech
magazines, possibly BYTE or PC/Computing. It featured an elevator opening onto
a floor of office cubicles.

The guy in the elevator screams "COMPUTER! FORMAT SEE COLON SLASH WHY ENTER"
across the whole cube farm. The doors close, and the elevator is gone.

Caption was something along the lines of "Speaker-independent voice
recognition might be a bit tricky."

------
jadedhacker
I feel creeped out by these home listening devices and I don't own one, but
don't our phones already have this capability? You can turn "Ok Google" on on
an android phone. I sometimes record audio, and the mic is incredibly good. Is
there a substantial difference between our phones and these devices?

EDIT: Just realized the substantial difference is that Google and Amazon own
all of these things. They don't control all makes and models of phones.

~~~
mns
I got an Echo Dot a month ago, because there was a special offer on Amazon for
the Philips Hue system, and it included one free. After reading this and
having a discussion with my girlfriend last night about how she needs a backup
of something, but we don't have a DVD writer at home (she's not tech savvy and
thinks people still back-up stuff on DVDs), and now, the second day my Amazon
daily offers is filled with portable DVD writers, it starts to creep me out
and the first thing that I want to do when I go home is to unplug it. I know
it can be a crazy coincidence and I was never the kind of guy that believed
this Facebook is listening to you, but still, what if they actually just
listen for some keywords?

~~~
titzer
It might just be a coincidence and a psychological bias that you noticed it,
but given that it would have a very, very clear economic benefit to Amazon if
it were true, we're stuck with these facts:

1.) They have an economic incentive to do so

2.) Only they know how their systems work

3.) Their network traffic is encrypted

4.) They face legal risks and user backlash if caught doing so

Given 2+3, you can't be entirely sure that they aren't doing it. If they deny
it, your only recourse is to hope that their cost/benefit calculus considers
4.) to be more costly in terms of dollars.

------
simion314
This devices should by law always have a big LED on when recording, you would
have the option to set it's brightness though.

Also why could it send data out without any confirmation from the user.

~~~
kinsomo
> This devices should by law always have a big LED on when recording

And the LED should be forbidden by law from being software-controlled.

They should also have a physical inline toggle switch to disconnect the
microphone(s) entirely, whose state is easy to visually confirm.

~~~
imrehg
I wonder how would you even do that? It's not a tape recorder, where someone
has to physically press a button, there are no moving parts, thus it must be
software controlled.

~~~
rolha-capoeira
I assume they mean that the light should not be accessible to software, just
wired so that "if camera is on, light is on"

~~~
dpark
Which means the light is always on. The device _must_ be listening or the wake
word won't work.

~~~
squeaky-clean
There are usually 2 stages, a local processor that can only detect the wake
word, and then once that hears the wakeword it begins streaming data to the
internet.

~~~
dpark
I’m aware. The microphone is always on in order for stage 1 to work.

~~~
squeaky-clean
It doesn't need to be based on the microphone, it can be based on that second
processor. It can be based on network activity (my USB Wifi adapter blinks
whenever data is sent/received, even though it's connected 24/7, I don't see
why anything else couldn't do that).

We're also discussing theoretical hardware changes, there's no reason it can't
have 2 sets of microphones where one is hardwired only to the wake-word
processor which has no direct connection to the main processor except some
one-way signalling, and the LED is hardwired to the second set.

~~~
dpark
Your WiFi led is probably blinking pretty constantly, which would freak people
out if they were told that blinking means active listening (as opposed to
checking it updates or any of dozens of other things that might use the
network). It’s probably also software controlled anyway.

The idea that there would somehow be a dedicated wake word microphone is a
little ridiculous. Firstly, no one would trust this supposed 1-way connection.
Second, it would require a dedicated processor to make the wake word even
work, driving up costs. Third, the echo uses an array of microphones so your
wake word would either be unreliable or drive costs up further as you
duplicate the entire array. Hardly a net win.

The reality is that if you don’t trust amazon to do the right thing you
shouldn’t install their listening device in your home. (Likewise for your
phone.)

------
bootsz
Every day I become more convinced that we've really let tech get a little too
out of control, and could probably benefit from putting the brakes on a bit to
get some very critical stuff under control first (e.g. security/privacy).

Don't get me wrong, technology is amazing and the cutting edge stuff going on
today is super exciting... But it seems the capabilities of software now are
far, far outpacing our abilities to ensure an adequate level of security and
personal privacy. We've been blinded by flashy tech, too busy being amazed at
everything all the time ("look at what AI can do!") and not stopping to
consider the huge hidden price we're paying.

The reasons this has happened are kind of obvious... I just fear that it's
going to take things getting to a point where we have a really major
catastrophe on our hands for anyone to be willing to really do anything about
it. A lot of people can and will get seriously hurt. I can imagine such an
event resulting in a great "cooling" period in tech where advances are slowed
while bigger investments are made into security, cryptography, solving the
"identity problem", etc. But it would be nice if we could just have the
foresight to fix things before it gets to that point...

------
newman8r
The article doesn't explain much, but it sounds to me like the device
mistakenly interpreted a voice command to record and send a message.

~~~
untog
Sure, but isn't that still terrifying? It brings me back to something I wonder
about with these voice activated devices: you have no idea what it can do.
Voice UIs are so utterly opaque, and Amazon(/Google/whoever) pushes new
software updates without informing you. So it'll add new commands all the
time, and you won't know about it until you accidentally trigger it, making it
do... _whatever_.

But either way, they should surely be confirming this before send, the same
way Siri does when I ask it to send a text.

~~~
newman8r
yeah it's still off-putting.

I haven't used a smartphone in years. Looking back on the days when I did use
a smartphone, there was always this tiny fear in the back of my head that
maybe I'd pocket-dialed someone and they'd be able to hear my conversation. It
actually had a subtle chilling effect.

The Hawthorne effect (which discusses how people react to being observed)
seems to be especially relevant nowadays as there's a real possibility that
we're being recorded at any time.
[https://en.wikipedia.org/wiki/Hawthorne_effect](https://en.wikipedia.org/wiki/Hawthorne_effect)

It's actually a big theme in _1984_ where the mere possibility that you were
being listened to via telescreen was enough to keep you in line and influence
your behavior.

------
jefe_
I imagine there are a number of normal conversations that can trigger
unexpected behaviors from these smart devices, especially when you consider
the thousands of different regional accents, along with the thousands of ways
of speaking the same words with various speeds, pitches, and annunciations.
The article mentioned they were talking about wood floors:

...Oak in the annex and for the record...

 _recording begins_

...it's just the annex, stop fretting...

 _recording stops_

just send [flooring person] a quick message with samples you like.

 _sends recording to someone with same name as flooring person_

It sounds like the problem is that it was overly confident it understood the
user's intentions, and did all of this silently.

------
leroy_masochist
Tangential point, but you would think that Amazon would have immediately given
this woman the refund she asked for, rather than instead offering to take her
through a technically complex de-provisioning process. It just makes them look
even worse during the inevitable bad publicity that they had to know they were
going to get from this.

------
thefifthsetpin
Google let me listen to my own queries following, "okay, google." Mostly they
were reasonable, but sometimes it heard "okay google" in what sounded to me
like regular driving noise. It's a good reminder that even though we've
trained computers to recognize sounds and images, we didn't train them to
listen & see the way that we do. The mistakes that these algorithms make need
not be anything like the mistakes that a person would make.

------
andreyk
The explanation for this is fairly unremarkable
([https://techcrunch.com/2018/05/24/family-claims-their-
echo-s...](https://techcrunch.com/2018/05/24/family-claims-their-echo-sent-a-
private-conversation-to-a-random-contact/)):

Echo woke up due to a word in background conversation sounding like “Alexa.”
Then, the subsequent conversation was heard as a “send message” request. At
which point, Alexa said out loud “To whom?” At which point, the background
conversation was interpreted as a name in the customers contact list. Alexa
then asked out loud, “[contact name], right?” Alexa then interpreted
background conversation as “right”. As unlikely as this string of events is,
we are evaluating options to make this case even less likely.

~~~
titzer
Actually it's _very_ remarkable on how bad the voice recognition is, and how
Amazon can knowingly ship this crap to people, putting it in their homes,
recording their _private_ interactions. Ya'll recorded people having sex yet?

It sounds like the machine learning models are tuned for specific phrases and
have terrible, terrible false positive rates. "Evaluating options to make this
less likely"? Now I don't even know what _mental_ model you guys are using (if
you are from Amazon). This shouldn't be a matter of "likelihood". But OK,
fine, let's use math. AFAICT there is about a 0% chance that a human would
make such a string of errors interpreting human speech, but let's call it 1 in
a trillion. If you guys aren't doing better than about 1 in a trillion for
this string of at least 4 interactions, then one of those terms is stupidly,
stupidly high.[1] It actually interpreted random conversation as the name of
someone on a contact list? Horrible.

trust--

[1] Not even to mention, a human has context, understands boundaries,
preferences, and has a ML voice recognition model developed and tuned over
_decades_ of _interacting_ with _real people_. A human would also be smart
enough to _understand_ the other human's situation, context, state of mind,
and realize that even just the cadence of the conversation not changing in
response to queries was indicative of the humans not acknowledging the query.
Machines are f'in stupid.

~~~
andreyk
It's a 1-in-a-million edge case.... I would hope people on HN at least would
understand this stuff happens in software.

~~~
titzer
1 in a million happens about 4000 times a second on a single CPU running at
4ghz...

Did you really want to talk concrete numbers? Because if so, I am wondering
about the probability that Amazon's voice recognition mistakes random
conversation as a valid entry in someone's contact list, as well as the other
terms in this equation.

I don't know if you work for Amazon, and if so I don't want to single you out
specifically, but this is a pretty bad screwup, and it does not inspire
confidence. Please don't brush it away with "oh it's a 1-in-a-million edge
case". That attitude is even worse, and if it is indeed that attitude inside
of Amazon, then I am even more strongly against this and I hope that further,
deeper scrutiny is applied here, because this screwup is actually _illegal_.

~~~
andreyk
I do not work in Amazon. I side with them on this because look at this
protocol:

1 Wake up w “Alexa.” 2 respond to “send message” 3 respond to “To whom?” 4
respond to “[contact name], right?”

As an engineer (well, an AI researcher who used to engineer), that looks to me
like they were not negligent and it was hard to predict background
conversation would produce this unlikely set of inputs - it would be nice to
see stats but this is the first time it is covered in the media to my
knowledge. And as with the laughter story, they will now change the inputs to
make it likely 1-in-a-trillion this will happen.

However, it does seem like Alexa etc. will have to be better about recognizing
audio from TVs/conversations and stuff directed at it - and I am sure they are
working on it.

PS comparison to CPU is not great obvs since it's about number of instances
(how many times does "Alexa" get woken up by background audio - not 4ghz)

~~~
titzer
They would like to put this in a billion homes no doubt, and suppose that
people are home, talking, a couple hours a day. Now we are talking real
numbers. A few billion conversation-hours per day, 365 days in a year--
suddenly one in a trillion is starting to look like it's gonna happen a couple
times a year. Now if Amazon knows these probabilities--which they don't,
because they clearly have not done due diligence in understanding their
rapidly evolving, inscrutable voice models--they are now knowingly violating
eavesdropping laws, probabilistically.

This is part of the problem. People want to handwave away small probabilities
when they should be busting their asses to make probabilities _actually_ 0--
solutions like not having this crap in their house at all.

------
Bhilai
I am seeing comments saying, Alexa misinterpreted some voice commands and
activated a "call" skill. This sounds like a good excuse but does absolve
Amazon of responsibility. If their voice and command recognition is broken
they should add second step verifications like "Are you sure you want to call
X" or "Okay, I am calling 'X' in 10s" or something similar. Crazy to imagine
Amazon is shipping this device with capability of video calling people and
buying stuff online without having fully testing and thinking about all
scenarios.

If there is even a slightest possibility that Alexa/Siri/Google/Cortana is
going to misinterpret commands (with privacy implications) then they should do
two step verification of some sort.

------
rectang
"Always listening" is a fundamentally unsafe design.

Once recordings of private conversations leave the local environment and make
it to the cloud, eventually they will leak. It's akin to data collection by
law enforcement: once the data exists, eventually it will be abused.

~~~
wilsonnb
So you're saying that all data that has made it to the cloud will eventually
leak? That seems like a strange assumption to me.

~~~
rectang
Yes. Assume that all data uploaded to the cloud will eventually be
compromised. That is the only safe assumption.

Not even the most responsible companies (e.g. Google) can hold out 100% of the
time in the face of determined assault by government. Some of that data is
going to leak to three-letter agencies, or similar.

Somewhat less responsible companies (e.g. Amazon) will leak data more often,
to a wider range of threat agents.

So then we have to consider how valuable this data is. Random sampling of
private conversations within the home? Sometimes innocuous -- but if the wrong
moment gets leaked, the consequences are potentially life-shattering.

~~~
wilsonnb
That seems like an extreme perspective. Kind of like the tech version of
abstinence only sex ed. Sure, it's the only 100% safe way, but it's not useful
for most people. It doesn't weigh the benefits against the potential cons, or
even take into consideration the actually likelihood of data being leaked.

~~~
rectang
The alternative perspective you're presenting sounds to me utterly cavalier
about the prospect of ruining people's lives. It's like Equifax's attitude
towards identity theft: it doesn't affect their profitability, so why care?

It's because such blithe dismissal of the damage caused by data gathering is
so prevalent in the industry that the likelihood of devastating compromise is
so high and the costs borne by the populace are spiraling upwards.

Some data should never be collected. Some data should never even be uploaded.

~~~
wilsonnb
The problem I have is that you are conflating _potential_ damage and _actual_
damage as the same thing, which is not how you accurately measure risk.

I am honestly confused as to how you interpreted my last comment as "utterly
cavalier about the prospect of ruining people's lives", when all I said was
that your assumption doesn't take into account the _actual_ probability of
data being leaked and it doesn't weigh any of the benefits of data collection
against that risk.

~~~
rectang
Small-likelihood times many-chances times grave-consequences equals a finite
but significant number of lives wrecked. A gamble you deem acceptable.

I can only hope that karma visits those who arrogate to themselves the
decision to sacrifice a few of their fellow human beings: may they and their
loved ones become the sacrifices.

------
_jal
If you bug your own home and consent to third-party, opaque management of the
surveillance, things like this will happen[1].

I expect worse to come.

[1] Am not saying anyone deserves this, or that AZON is either evil or
beneficent, or anything else other than precisely what was stated.

~~~
fastball
Amazon's ticker is AMZN.

~~~
_jal
Oops, no idea what my fingers were doing there.

------
titzer
These devices are all driven by machine learning voice recognition. I have no
idea how I am supposed to derive trust in the software running on them when
even the engineers themselves are arms-distance away from their machine
models, and the machine models are tuned to terabytes of input data and are
trained with reinforcement learning. I don't want to disparage engineers
working on these products, but hell, we don't even know how big software
systems really work anymore, and we've got neural networks thrown in the mix.
We're all just plugging shit together until it limps these days.

I vote no on telescreens.

------
falcolas
Well, when you consider that Amazon's devices once interpreted random noises
as a "<trigger>, laugh" command (as per their explanation), that a voice
command to record and send a message could be created from random conversation
is not in the least bit surprising.

Inappropriate, worrying, but not surprising.

------
mundo
I wonder how similar this is to [https://www.cs.cmu.edu/~sbhagava/papers/face-
rec-ccs16.pdf](https://www.cs.cmu.edu/~sbhagava/papers/face-rec-ccs16.pdf) ?

That's the "Facial recognition fooled by funny-colored glasses" study from
Carngie Mellon, where researchers were able to make machine learning
algorithms fail disastrously (e.g. mistake a man for Milla Jovovich) with a
pair of glasses printed with what looks like a random assortment of colorful
pixels, but is in fact a targeted attack specifically designed to trick the
algorithm.

This Alexa failure is obviously not a targeted attack; but when your system is
exposed to enough data, eventually you will stumble over some input that
happens to resemble a targeted attack by pure chance, right? It's equivalent
to saying that if you aimed the facial recognition algos in that paper at
millions of faces wearing randomly-colored glasses, eventually some of the
glasses would be close enough to the targeted-attack glasses to produce the
same effect.

Obviously I'm theorizing on almost no data here - I don't know anything about
Alexa's voice recognition, maybe it contains no ML at all. But it seems
plausible that this might be what happened here - not a bug per se, but the
natural and totally expected result of giving an opaque, machine-generated
system with a very low failure rate so much input data that the failure rate
is significant.

~~~
wilburTheDog
I recently stayed at a house which had an alexa device. In a conversation
where I said the words light switch several times Alexa beeped and responded
to me each time. I think most of us just believe Amazon when they say "Alexa
responds to its name" and don't stop to consider the possible failure modes.
We want to believe it can understand our words, when it's really just
guessing. Over long enough time it's inevitable that it will misunderstand you
and do something you don't want.

Still, I wonder how it heard "record this conversation and send it to someone
on my contact list".

------
IkmoIkmo
I recall a few weeks ago how some researchers figured out that the voice-
commands could be triggered with subliminal messages, too. i.e., you wouldn't
have to go 'Ok Google', but rather just play a fingerprint of that sound that
makes the algorithm think you said that, without it being audible or
understandable to human beings. And you could hide it in other audio like
music.

That feels pretty scary particularly because these devices have such a high
mandate. Right now you can shop for things online by voice command. Record and
send messages. What's next, sending money, sending a data dump of sensitive
data like emails, passwords, contact lists?

We've already heard of some reports where you'd have a smart home device
listen to a commercial on television where they were demo'ing a purchase of
some product via such a device, and interpret it as a command from the owner.
Amazon took steps to avoid that with their superbowl commercial, but it seems
to have done so by changing the commercial, not the product itself.

Just another attack vector to worry about. I'd happily buy these home speakers
if I could limit them to just downloading information, and only uploading
limited pieces of information, e.g. a music playlist. I'd want to be able to
shut off any of the commercial/financial or social capabilities. I just don't
care about them and they're risky.

~~~
kevingrahl
Do you happen to have a link to said research? I’d be curious to fiddle around
with that.. for educational purposes of course..

------
amingilani
I turned on recording indication sounds on my Google home in accessibility
options.

Now I hear it beep and randomly start listening when my wife and I talk.
Sometimes it does it randomly without me making a sound.

I ask it what's up and it ignores me and goes to sleep.

Smart speakers are so dumb I'm hardly surprised by this.

~~~
dmix
Yeah Amazon said there should have been an audible warning it was recording
and sending the message. It's apparently just a 'rare' glitch.

Based on Amazon's response I assume the device _didn 't_ notify the user at
all and it wasn't just a matter of not hearing it.

------
awat
My concern is that this almost sounds like something not customer facing was
accidentally made customer facing imho.

------
jschwartzi
This is the story I'm going to link to when my tech geek friend asks me why I
don't have an Alexa like he does.

------
iooi
How soon until Amazon starts sending the private conversations of people
discussing illegal things directly to law enforcement?

It will be pretty easy to justify by bringing up bomb threats, but it would
mostly be people buying/using drugs.

~~~
asn0
Amazon could add this to the suite of law enforcement products they've already
been selling[1]

1\. [https://www.nytimes.com/2018/05/22/technology/amazon-
facial-...](https://www.nytimes.com/2018/05/22/technology/amazon-facial-
recognition.html)

------
patsplat
Voice is not for personal computing. It is best for social computing. This is
a completely new metaphor for interactivity, and problems are to be expected.

This incident speaks more to Amazon's speech recognition rather than a
weakness of voice. Am not an expert in the topic, but it certainly seems like
Alexa tries to hear patterns rather than translating speech to text then text
to commands. Google Home's recognition is much better.

------
2T1Qka0rEiPr
Question - how can these potentially nefarious acts be mitigated, where I plug
a device into my home network which listens/watches "periodically".

I guess naively I'm thinking some sort of light/display which says "Amazon is
sending/retrieving data" \- it would need to be from a third party to have
integrity, and I suppose in order to actually allay the fears of the masses it
would need to be plug-and-playable.

Does such a thing actually exist?

~~~
faitswulff
You could always set up a dedicated router for your smart home devices if you
really wanted.

------
rm_-rf_slash
This is why I unplug my Echo whenever I’m not using it.

Side note: why is it that after years and years of privacy breaches, there has
been almost no support to use hardware switches for cameras and microphones?

I would feel 100000% safer using my internet-connected cameras and microphones
knowing that I can turn off those devices independently of their host devices
and that no hacker can monitor my cameras and microphones, even if they’ve
rootkitted every device I own.

------
TheSpiceIsLife
_" He told us that the device just guessed what we were saying," she said.
Danielle said the device did not audibly advise her it was preparing to send
the recording, something it’s programmed to do._

Is it possible there's an exploit out in the wild for these devices that
allows an attacker to control the device remotely.

Is it possible an attacker told the device not to make an audible
announcement?

~~~
sleepychu
Yes, but it's improbable. You can communicate with Alexa at human inaudible
frequency and instruct it to set it's volume to 1/10(0/10?) but of course you
need to be reasonably local and targeted.

Is it possible that there's a remote exploit? Yeah of course but if it's in
the wild enough that this particular individual was struck by a seemingly
random (and valueless) attack we'd be seeing a lot more of these in the wild
than we are.

------
parliament32
I feel that while these devices are great in principal, there still needs to
be work done on whether the "trigger word" activation is the best way to
enable them. Maybe we need longer / more complex trigger words? Maybe there's
a different way we can activate them alltogether, avoiding the "always
listening" problem?

------
8bitsrule
Really, folks... we can't even get the net to work the way we want, but then
we heap these enormously sophisticated, delicate tasks onto relatively-
untested, non-deterministic warez, and pray all goes well.

Much like running into a human on a bike, but less deadly. (Unless there's a
psycho in my contact list.)

------
albi_lander
This makes private-by-design alternatives like Snips
([https://snips.ai/](https://snips.ai/)) even more legit. When everything is
processed locally, the user does not have to trust anyone/anything.

------
rhacker
Same thing with my Android phone. When I am in an area without internet
access, but phone-calling signal (at least) the dumb ass phone defaults to
using an internal low-quality OK-Google speech recognition. So instead of
looking up hours to a nearby restaurant I look at my phone and it is suddenly
calling someone in my contact list (that I haven't really talked to in
years)... and of course since the speech recognition blasted the CPU's
horsepower, nothing on the screen is responsive and my thumbs are constantly
slamming the screen with no feedback - the calls goes through for about a
second or ringing before my phone decide to hang up fully.

------
softwarefounder
Pays Amazon for a listening device and installs in home...gets upset when it
listens.

~~~
DoofusOfDeath
That seems overly simplistic to me.

It's like saying that someone buys a car that's powered by an explosive
liquid, and then complains when all the gas in the tank explodes at the same
time.

~~~
chapium
Both a car and an amazon echo have a switch that turns that functionality off.

~~~
peeters
I feel like you're being purposefully reductive here. You know they're
different right? The entire value proposition (and marketing strategy) of
Alexa and related devices is that they are voice activated, so you don't have
to physically interact with the device to make a request.

Given that, it would be ridiculous to expect paying customers to switch it off
when not in use. The onus on Amazon is to protect their customers' privacy
while the device is being used as intended.

If you really want to pick apart the car analogy, maybe this is closer to
having automatic start and driving capability on a car, but the car sometimes
randomly starts and drives away. Are you really going to blame the customer
for that, even though they could disable the feature every time they exit the
car?

~~~
chapium
The car analogies are usually quite poor. However, it depends on your use
case. Echo Dot's have a mute button which prevents monitoring, or at least it
seems so. I'm not an echo power user, so my needs are definitely different
than for intended uses like home automation.

I think for some, at least users like me would like a feature similar to "su".
I'd like to confirm that Alexa should start doing stuff if it has not been
confirmed for x number of minutes. I think the real problem with these home
assistant devices is they are not designed with a confirmation message. This
would be annoying in some cases, but surely a balance can be struck.

I'm not trying to be reductive at all about the car analogy. When you turn on
a car, you acknowledge that the engine is running. You can also mute an echo
if you have the foreknowledge that you will not be doing activities that
require voice activation.

------
DiabloD3
So when will such devices just be made illegal entirely? It doesn't seem they
can be made to be safe for average people if stuff like this keeps happening
and companies are unwilling to compensate people.

What if this was a lawyer talking about clients or a doctor talking about
patients? Is Amazon (or Google, or Microsoft) willing to deal with such legal
liabilities? Is this what move fast and break things looks like?

Edit: To those trying to downvote, remember, the Silicon Valley lives in a
bubble, and for the safety of people inside and outside of the bubble, the
bubble must be poked every so often. This is one of those times.

~~~
function_seven
You ever butt dial someone? Should all cell phones be outlawed?

I'd say the Echo needs to have a more difficult activation routine available.
Maybe have the option of setting the wake word to "Alexa, can you please" or
something similarly long and unlikely to appear in normal conversation.

Also, I wonder if they had their wake word set to "Computer". I did that once,
and quickly reverted after so many false activations.

------
madrox
This is something I don’t believe we’ve really figured out in terms of QAing
stochastic processes. Doing QA on normal code is binary...did it work or not?
Doing a QA on a model is a lot more difficult. You need to think about Type I
and Type II error. If I were a product manager thinking about wake words, I’d
want my Type I error to be as small as possible...to the point where I’d
accept a pretty high Type II error.

Lately I’ve been seeing Alexa wake on accident more often. It’s like someone
isn’t thinking critically about what kind of Type I error they’re willing to
accept.

------
jonjojr
"My husband and I would joke and say I'd bet these devices are listening to
what we're saying,"

Think about this statement for a sec. Of course is listening to you. It has to
listen to you because it needs to be able to respond to "Alexa". So yes, this
is always listening.

Now, once you come to terms with that, do you feel comfortable having a
technology with access to the internet, location, habits, account, contacts,
email, phone, CC information etc etc, to be actively listening to your most
intimate private conversations?

It does not take a 5 year old two seconds to figure this out.

~~~
wilsonnb
The problem with this logic is that it applies to phones, tablets, and laptops
as well.

There isn't much additional risk from having an Echo or a Google Home if you
already keep a smartphone within 10 feet of you at all times, which most of
the people who are buying these devices do.

~~~
pessimizer
Why is that a problem?

------
mdeeks
My Echo somehow woke up when we were in another room, then heard itself
talking to itself, and called my friend in the middle of the night. You can
see the log of what it said and how it heard the last word as the command:
[https://www.facebook.com/mike.deeks/posts/10215464075417775](https://www.facebook.com/mike.deeks/posts/10215464075417775)

It was both hilarious and infuriating. I immediately turned the calling
feature off (you have to contact support btw) and later we switched to Google
Home.

------
fusl
I don't see where Amazon would be at fault here. Remember back in the old days
of T9 keyboard phones, remember that one time where you forgot to lock your
keyboard when you put it into your pocket and it called a random person from
your contact list? You didn't immediately call the press to say it was Nokias,
Ericssons, Siemens' (or whatever your phone manufacturer is) fault you left
your phone unlocked and it called a random person because of the hardware keys
being pressed within your pocket.

------
mabbo
I was chatting with my wife about something the other day and my android
phone, from my pocket, randomly added to the conversation "That's good to
know". O_O

------
merinowool
Wouldn't that be a crime to record someone without consent and share that to a
3rd party? I would have called the police or is Amazon too big for the law to
grasp?

~~~
macintux
That's an interesting point. In California and some other states, having a
device actively recording any conversation without consent from everyone
involved is illegal, much less sharing it.

------
dep_b
My experience is totally anecdotal and I'm not even an Alexa owner, but I had
a Skype conversation with a client that had one them set up in his office and
it was constantly triggering, even when the word wasn't used.

So if it though it heard "Alexa, send this message to John: {conversation of
ten minutes}" it just did what it thought it was supposed to do. But it's
weird they didn't hear any audible confirmation or anything.

------
tinycakes
It seems like an easy safeguard to implement to prevent sending unwanted
emails/texts would be for Alexa to generate a unique confirmation phrase that
the user has to repeat back in order to actually execute the send command.
Something like Alexa: "Say 'purple people eater' to send email." Real Person:
"Purple people eater." Alexa: "Email sent."

------
m3kw9
Hmm the AI model which is widely considered a undebugable black box glitches
on a percentage of users. What else could happen next?

------
gesman
>>>But Danielle is hoping Amazon gives her a refund for her devices, which she
said their representatives have been unwilling to do. She says she’s curious
to find out if anyone else has experienced the same issue.

If that's true - that dumb greedy move is going to cost AMZN way more than
simply giving customer refund.

------
aphextron
Is this a surprise to _anyone_? That was literally my first thought the moment
they announced these things.

------
IronWolve
And Bank of America is now pushing its Digital Assistant, Erica. Just what I
need, another robot to talk too...

[https://promo.bankofamerica.com/Erica/she-is-
here/](https://promo.bankofamerica.com/Erica/she-is-here/)

------
ComodoHacker
>"The person on the other line said, 'unplug your Alexa devices right now,'"
she said. "'You're being hacked.'"

Perhaps they were indeed hacked. Got into test group for some black hats
project.

------
alistproducer2
I've said it before, but this solidifies it for me - I'm never installing a
internet enabled hot mic in my home. I'd rather spend the 2 seconds finding
the song on my phone.

------
stevenhubertron
This just seems so unlikely. How was it sent? Was a MP3 file generated and
attached to an email? Was it a voice call? I don't really understand a
circumstance that this could have happened.

------
perseusprime11
I am trying to think what kind of architecture, code design could have led to
such a defect if indeed it's a defect. My guess this is a feature that got
triggered by an easter egg.

------
tschellenbach
I have 3 google home devices and they work very well. It also doesn't feel
like a bigger privacy threat than my laptop or my phone. Also, we all want
JARVIS!

------
ddtaylor
It's okay because they take privacy seriously.

------
mihaela
I do see a dveice like that useful in some scenarios. Like a lonely elderly
person, living alone.. Talking to Alexa might be nice.

------
mandeepj
Please help me in understanding this - are these features built on purpose? If
no then what type of testing has their team done?

------
blunte
The chances of this happening all the time are unknown but believable.

We as tech users are constantly trying to balance convenience with safety

------
gshakir
Even though I like Apple stance on privacy, I always turn off microphone
access to apps like 'WhatsApp' and others.

------
trumped
How did they receive the conversation? email? was the receiving address
similar to some other address (the real recipient)?

------
tambourine_man
People are volunteerelly paying for their houses to be tapped for the
convenience of being able to shout: play me some song.

I’m so out of touch with this world it’s scary.

~~~
joshstrange
It's a little more than just that. Every light in my house is controlled with
Alexa which is a very nice and hard to explain until you have tried it. Also
my home theater system is all controllable via Alexa as well. This kind of
thinking makes no sense to me.

> People are voluntarily paying for their houses to be tapped for the
> convenience of being able to shout: play me some song. > I’m so out of touch
> with this world it’s scary.

I could just as easily say "People are voluntarily paying for their locations
to be tracked for the convenience of being able to get live maps on the go"
(ie. cell phones, which also are listening to everything you say unless you
disable it)

~~~
qyv
There is a massive difference between walking a few steps to turn on a light
switch (which is only useful when you are in close proximity) vs having a
accurate real-time mapping functionality when on the go.

Similarly, there is a massive difference in the privacy implications of
listening to every conversation _everyone_ in your home (including guests) is
having vs. having your current location known. Wiretapping laws exist for a
reason.

~~~
ben336
Turning _off_ all the light switches that have been left on in a big house
though is a meaningful convenience. I think the privacy point is strong enough
without having to trash people's use cases.

~~~
qyv
No trashing it at all, just contrasting it. I do see the convenience (luxury)
of home automation, but when you compare that against accurate mobile mapping,
location finding and navigation, the difference in usefulness is orders of
magnitude. One can literally save your life.

------
randyrand
How does alexa get your contacts? I don't think I've given it mine.

------
ReedJessen
I am very skeptical of this person's claim.

------
NVRM
But seriously what do you more or less expect from this. Anyone littlely
involved in tech will tell to throw this shit out the window.

~~~
ryanwaggoner
I'm "involved with tech" (developer for 15+ years) and I have two of them. You
might also be horrified to learn that my house contains multiple smartphones,
computers, routers, and other devices, _all of which are connected to the
internet!_

I enjoy living dangerously.

~~~
NVRM
I am also a developer,living in countless universes, each enveloped in its own
shell, compelled by the wheel of time, like particles of dust blowing about in
the sky, and I don't understand what's wrong with you.

------
yani
I am so happy with GDPR right now

------
PeterStuer
Obligatory
[https://www.youtube.com/watch?v=sAz_UvnUeuU](https://www.youtube.com/watch?v=sAz_UvnUeuU)

------
jacksmith21006
Not sure why people would be buying the Echo over the Google Home any longer?
The GH offers a much better UX and now is well ahead of Alexa in sales last
quarter.

"Google takes top spot in global smart speaker market, HomePod nowhere to be
found"

[https://appleinsider.com/articles/18/05/23/google-takes-
top-...](https://appleinsider.com/articles/18/05/23/google-takes-top-spot-in-
global-smart-speaker-market-homepod-nowhere-to-be-found)

Came to the market 2 years later and already passed. This is before we even
get Duplex.

[https://www.youtube.com/watch?v=bd1mEm2Fy08](https://www.youtube.com/watch?v=bd1mEm2Fy08)

Google goes in the lead on something they tend to not give it up.

~~~
ocdtrekkie
Uh, dude, I know being a fan of Google is _your thing_ here, but this was less
than a year ago: [http://money.cnn.com/2017/10/11/technology/google-home-
mini-...](http://money.cnn.com/2017/10/11/technology/google-home-mini-
security-flaw/index.html)

And it's a flaw so bad they had to outright remove the button entirely,
because they couldn't fix it without replacing the hardware.

It's a little disingenuous to insinuate a bug with recording audio is a reason
people shouldn't be buying company X's product over company Y's, which also
has had a bug with recording audio.

~~~
jacksmith21006
Google quickly fixed the issue caused by the person putting it in the bathroom
and causing the button to stick.

The person that it happened to got a pre-release version. Google even went to
the persons house on a Friday evening to resolve. The person found the problem
on the same day. None of the units sold to the general public had the issue.
Realize Google literally dropped everything to take care of the issue. Now
that is what you want to see.

The reason you buy a Google Home is because they provide a much better user
experience and why Google has already passed Amazon even though they did not
get to the market until 2 years later.

"Google takes top spot in global smart speaker market, HomePod nowhere to be
found"

[https://appleinsider.com/articles/18/05/23/google-takes-
top-...](https://appleinsider.com/articles/18/05/23/google-takes-top-spot-in-
global-smart-speaker-market-homepod-nowhere-to-be-found)

We started with an Echo and then purchased the Google Home and then several
more Google Homes.

My son had graduation earlier this week at an arena that has a name that is
difficult to pronounce. It is named after someone that gave a lot of money to
the University.

There was also a MLB game that night that started 15 minutes later than the
graduation.

Walking through the kitchen asked drive time thinking no problem as people are
leaving the city usually not going to the city.

Google got it the first try and gave drive time which was 2x more than normal.

Google saved my butt as quickly got ready and left and would not have
otherwise. Asking our Echo and Siri the same question and neither could
handle.

That is why you buy the Google Home. Well also because of new things like
Duplex coming.

[https://www.youtube.com/watch?v=bd1mEm2Fy08](https://www.youtube.com/watch?v=bd1mEm2Fy08)

Google is just so far ahead in every layer of the AI stack and just going to
be able to provide a far superior product.

~~~
ocdtrekkie
Google took the top spot because they're giving them away for free. Sales
numbers are easy to pad, see the New York Times Bestseller lists woes in that
regard.

~~~
jacksmith21006
Wish they were giving away for free. Really wish they would just discount the
Max a little. Have one but want a second.

Echo was out 2 years longer and Google already passed as they are just a lot
smarter and easier to use. It is that Google AI is just superior in every
layer of the stack.

Can't wait for Duplex.

[https://www.youtube.com/watch?v=bd1mEm2Fy08](https://www.youtube.com/watch?v=bd1mEm2Fy08)

Once Google leads in an area they do not give it up. Something I just love
about Google is they never rest and will just keep innovating.

The new voices are incredible. You can not tell the difference between the
machine and a human any longer.

We started with the Echo but then purchased a Google Home and more Google
Homes. Looks like made the right decision on which to standardize on.

------
ataturk
All devices with mics and cameras should come with physical switches to
disable them. Even a couple of DIP switches on the back of the device would be
fine.

------
tapatio
People who use these "microphones" are idiots. They are right on up there with
the people that give away their DNA to web companies.

~~~
mikeash
I’m one of those “idiots.” Can you explain how it’s any worse than carrying a
cell phone?

~~~
fouric
Android has per-app microphone permissions that you can disable, if you want.
At least in theory, Google apps honor these permissions, and on LineageOS, you
can verify that this is the case - while with the Alexa, not only is there no
way to audit any of the code running on the device, but it's explicitly
marketed as being always listening.

~~~
mikeash
The baseband is a black box and has direct access to the microphone.

~~~
fouric
Oh, this is true - although the likelihood and ease of it being exploited as a
listening device are much lower than that of a device explicitly designed to
do so.

Edit: Not to mention that the entire Alexa is a black box, too.

Second Edit: Not trying to claim that you're an idiot, just provide some
perspective.

~~~
mikeash
I suspect that the capability is already there, to be turned on at the request
of law enforcement.

------
bmans94
It's absurd to me that someone would wire up "every room in their house" with
internet connected microphones (in a technology that is still in it's
infancy!) and then claim you felt like your privacy was "invaded". At best,
it's a Trojan Horse, but the naivety here is astounding. Of course your Echo
shouldn't send out private conversations, and it looks like a legitimate bug
occurred. But man, the buy-now think-later mindset some people have is
worrying.

~~~
saas_sam
You literally carry an audio & video recording device with a GPS chip and
multiple wireless attack vectors on you for nearly every waking hour of your
life.

~~~
bmans94
Of course I do. But that's not my point. They wallpapered their house with
Echos! And now they are 'shocked'. When I get home, my phone usually gets
tossed on a charger in my room, and forgotten about (while checking for
messages every once in a while). And my phones sole purpose, while capable of
it, is not to listen to everything I say and interpret voice commands. I never
use Siri, and I would be 'shocked' at this point if it sent out a message,
that I didn't intend, without alerting me. It's a older, tested technology (in
my opinion).

I'm not surprised that my GPS capable phone knows where I am at all times. I
wouldn't be surprised if my Echo recorded my voice and sent a message to
someone. I wouldn't be surprised if there were some major bugs yet to be
addressed in the software for the Echo. I guess this is a little harsh, but my
sympathies are somewhat lost on these people. Not their fault for what
happened, but they did take a risk.

"A husband and wife in the _privacy of their home_ have conversations that
they're not expecting to be sent to someone (in) their address book,"

It just doesn't sit well with me.

~~~
saas_sam
> I would be 'shocked' at this point if it sent out a message, that I didn't
> intend, without alerting me.

Yep. So was couple. About Alexa. Same thing.

~~~
bmans94
But I don't use that function on my phone. And it's not the main purpose of
the device. And it's been around longer than Alexa. That's my point. Not the
same thing.

