Hacker News new | past | comments | ask | show | jobs | submit login
Is Anyone Listening to You on Alexa? A Global Team Reviews Audio (bloomberg.com)
333 points by minimaxir 14 days ago | hide | past | web | favorite | 205 comments




I don't work for Amazon or Google, but I agree with them in this case.

I read through the full text of the bill (http://www.ilga.gov/legislation/fulltext.asp?DocName=&Sessio...) and it seems like it sounds like the companies can be sued if one user agrees to a written policy, but then is used by another user (e.g. a spouse or sibling or friend), which makes smart speakers basically impossible to exist. (User identification isn't good enough, and even if it were, mistakes can happen)

> No private entity may turn on or enable, cause to be turned on or enabled, or otherwise use a digital device's microphone to listen for or collect information, including spoken words or other audible or inaudible sounds, unless a user first agrees to a written policy informing the user[..]


> which makes smart speakers basically impossible to exist.

Then perhaps in their current form, they shouldn't exist. It's already illegal to record people without their consent in many states. We shouldn't give up this right to privacy for a bit of convenience.


I think all it means is everyone with a smart speaker should have a sign on their front door informing all who enter that “conversations may be monitored”.

Realistically though don’t a lot of stores have cameras and mics lawfully recording for security purposes? Are they all legally obligated to post warnings?


> Realistically though don’t a lot of stores have cameras and mics lawfully recording for security purposes? Are they all legally obligated to post warnings?

Yes, at least here in the UK.


In the US there is the concept of "Expectation of Privacy" [0] so it depends.

[0] https://en.wikipedia.org/wiki/Expectation_of_privacy


The link elaborates that this concept is enshrined in the US Constitution Bill of Rights (4th amendment), so it covers government surveillance. It is not directly applicable to private entities using recording devices.


By that logic, any device with a microphone and potential to record should not exist.

Smart speakers are not telephone calls, and even if they were, federal law only requires consent of one party to record (11 states require all-party consent).

In public spaces, you generally lose any right to consent to recording. If you're in someone else's home and they record you with their smart speaker device, then your beef is with that person whose private property you're being recorded on. Same as if they were recording you with their phone's camera in their private residence, vs recording you in a public space.


I don’t agree. In your example, being in someone else’s home doesn’t put you in a public space for purposes of recording consent.

A dumb recording device can’t do something illegal without its owner’s/user’s affirmative action. If the homeowner uses the recording thing illegally they’re liable for that use.

In contrast, Alexa and similar devices operate on rules built-in by their builders. They do what their builders intend, not what applicable law or their owners require. Liability should rest with the builders.


Right - being in someone else's home is their private residence. If they break the law using devices there then they are liable, not the device manufacturer. Maybe I didn't specify that enough.

> By that logic, any device with a microphone and potential to record should not exist.

Not at all, it's how you use it. Cars can run over people but cars can exist; driving over someone is a crime committed by the operator. A machine with a microphone need not be, of itself, illegal.

There is an issue of a secret microphone, as with Google's camera that secretly included microphones. That seems fraudulent because the user is not able to knowledgable decide if she can trust the device.


I understand the distinction you're drawing with regards to the company secretly recording without anyone's consent, but in the imagined scenario discussed in this thread where a spouse or whoever gets recorded after a different user agrees to the terms and conditions wouldn't your reasoning place the blame on the original user who misused the tool, not on the company who supplied the tool? There are scenarios where a user might want to have an always-on microphone in a certain section of their house without violating anybody else's privacy.

Certainly, as long as the device does not trivialize the violation of those rights. If a car designed for road use by a human driver did not have a windshield to see out of, I'd say it is designed in a way to violate the rights of others.

Are Alexa and others designed to reasonably ensure only consenting users are being recorded?


I have practically no doubt that Alexa and others are malevolently spying on their users, I hate to argue their case but I would like to be able to buy one if I wanted to.

Should I be allowed to buy a vehicle without a clear view of my surroundings if I only intend to use it on private land? Should I be allowed to buy a recording device with no privacy features if I intend to use it only in private? Does the potential harm produced by the availability of tools without bells and whistles for public safety outweigh the desires of users who would gladly pay for the bare-bones version and use it private?


I don't believe Alexa and others are malevolently spying, but if the data exists, you should expect nothing less than it to be public at some point, or at least be accessed by someone you'd not expect to share your life with.

You _should_ absolutely be allowed to buy a non-street legal vehicle to operate on private land. You _should_ be able to buy a recording device that has no privacy features. You should _not_ be able to sell a recording device that by default is streaming all audio all the time for common consumers without a big label saying "everything this device is within hearing distance of is now public data and you agree you are in violation of the law, where applicable, for using it".

We are no longer in the days where consumers can be expected to know what their devices are even capable of, let alone what rights are being trampled.


I mean, that is technically impossible? You have to record a voice to be able to analyze it to determine if the voice is of a person that consented.

This basically would outlaw ANY device that is voice activated, because it has to constantly be 'recording' to be able to be activated.


No, it's not. We can use facial recognition, we can use voice signatures. It doesn't even have to be recorded, as the device setup can record those signatures of consenting users and refuse to transmit or save any data that does not match those signatures.

Tell me, which of these do Alexa and the like perform?

Would it be more difficult? Yes! This is the reality for any company where rights and laws actually matter. If you can't abide, you can't release. Very simple.


Ok, you are using a different definition of recording than me. I would say your definition is "recorded and stored"

I was thinking that in order to fingerprint a voice, you have to "record" it to a digital format that you can process to determine if it is the user who has consented. Even if it is deleted after processing, it was still "recorded" for a few milliseconds at least.

Does the law as written make that distinction? Would recording locally to check a fingerprint not still break the law?


the law on "copying" was abused by the music industry to try to get royalties on the buffering in dismal music devices and computers. The law eventually caught up and now there is no copyright violation in transient cacheing and buffering. I imagine the same will eventually happen for de minimums "recording" used to figure out if you may record or not (e.g. face recognition, voice printing and the like).

You don't need to record it, or at least you only need to record locally the part being spoken while you're doing the analysis; if they've consented you upload the recorded section and then realtime info; if they aren't one that consented you discard the cached snippet and stop recording that person.

Right, but that is still recording, even if it is only temporary and local.

No its the responsibility of the device creator. This is more like an analogy where a self driving car killed someone as the company made the device activate under certain commands with no failsafe to prevent users who didn't agree to terms. Company was greedy to get the recordings.

The telephone is only relevant in some jurisdictions. For instance in Illlinois (an all-party state), you cannot record a private conversation. There's no mention of a phone or telecoms, and you should expect to get sued if you're in a conference room recording folks without their consent.


Precisely - the person who owns the conference room and placed the device there should expect to get sued. The person who manufactured the device shouldn't since they didn't use the device to break the law. The owner of the private property did.

It's the same with phone and security cameras - the manufacturers are not the ones at fault when the devices are incorrectly used by owners in states with specific legal laws regarding recording.


Imagine how long the devices would stay on the market if owners started being held liable for what the device does.

I'm not suggesting the device be made illegal, but I do think a ton of common applications should be... and without those applications, the device loses viability.


I imagine they will face the same market consequences that smart phones, security cameras, and other devices that have the potential to be used to record people in private places currently face. Customers with a low tolerance for being recorded simply just won't buy.


Security cameras often explicitly lack audio because of these laws. The other devices you mentioned all have primary functionality that doesn't depend of recording nearby audio arbitrarily. The possible exception is Siri and Google's assistant, which fall into the category of services/devices being questioned in this instance.


Tape recorders are still on the market.


All of those toget devices you mention are activated manually by the user. Smart speakers activate automatically.


They activate on very specific commands. It's no different then adding a trigger phrase to start recording with those devices. Whether it's a VUI, physical button UI, or even motion-sensor triggered is splitting hairs, since the device is still owned and operated by the person who put it there.


It's very different when the trigger is locally a not-so-great guess followed by shipping the audio to a remote service that can make a slightly more informed guess.

> The person who manufactured the device shouldn't since they didn't use the device to break the law.

The issue at hand is that if the manufacturer is the one who turns on the device, then yes, it is 100% them breaking the law.

If my phone is hacked and the microphone is remotely enabled to record a conversation and ends up violating the law, I'm not at fault, the hacker is.

So how is it different if the "hacker" is just the manufacturer using a private channel that they built for themselves?


Ultimately what that boils down to is that if I don't want to be recorded, I should stay in my own home and not even visit anywhere, because only at home I can expect privacy.

That's not privacy.


Minor correction, you said:

> private property

When you actually meant:

> personal property


Wait, you can just not give this up by not buying one and not hanging out with your friends who have these.

Why do you want to impose this on the rest of us? Why can't I just let people give me this useful service in my own home?


So? Just because a billionaire tech company is inconvenienced by privacy laws doesn't mean they should be able to ignore them.


The Google speech API supports Speaker Diarization and Android supports voice unlock so they are getting to a point where they are able to ID voices and should be able to tell which voice has consented to being recorded.


But doesn't it have to record the voice to be able to identify it? Once the sound waves are converted to a format that the machine can process, it has been recorded.

To my knowledge, we've generally recognized a difference between recording and immediate processing. Does a voice changer "record" you, if it adjusts the audio on the fly and doesn't store anything? VoIP phones arguably "record" the audio from your microphone and transfer it over the line to output to another party, but we don't call that recording either.

I think the biggest issue here is how smart speakers work: They record your audio and then send it to be permanently stored on Google, Amazon, or Apple's servers, rather than being processed locally and discarded, which we have the technology to do just fine today.

The only reason we're retaining voice recordings is to provide valuable data to the companies in question.


Recording implies storage. It's possible to do voice ID on device and discard any non-matched data.

You can't discard data that is not stored.

Are you really trying to argue over the definition of the term?

Indeed you can discard unstored data.

I write data pipelines for a living. We often use "discard" as a term for parts of data on which no further processing is performed and are not stored.

It's an extremely common usage of the term. See [1] for example.

[1] https://bash.cyberciti.biz/guide//dev/null_discards_unwanted...


>which makes smart speakers basically impossible to exist

Aha! Exactly. However, you could make a smart speaker that doesn't transmit the audio to a server.


> inaudible sounds

What?


TV ads emit inaudible (to humans) sounds to track which ads are being viewed.

https://arstechnica.com/tech-policy/2015/11/beware-of-ads-th...


Can average TV speakers emit sounds outside the frequency of human hearing? I didn't think they could.

Yes, most speakers are 20Hz-20kHz or better. Here's a pair of $10 headphones that do just that[0]. Most humans have range inside that band. Those both are really the limits of human hearing and as you get older it dramatically decreases. I remember there were those ringtones (IIRC called cricket?) that used high frequencies so teachers wouldn't hear it go off. Some malls also used high frequency sounds so teens wouldn't loiter outside their stores. Most of these were even below 18kHz.

tldr: yes

[0]https://smile.amazon.com/Earphones-Earbuds-Headphones-Premiu...


Humans have a fairly limited frequency range in which they can hear, typically said to be 20 Hz to 20 kHz, but usually worse than that for most people. There's all kinds of sound in the world that is inaudible to humans.


A few choice excerpts:

"Sometimes they hear recordings they find upsetting, or possibly criminal. Two of the workers said they picked up what they believe was a sexual assault. When something like that happens, they may share the experience in the internal chat room as a way of relieving stress. Amazon says it has procedures in place for workers to follow when they hear something distressing, but two Romania-based employees said that, after requesting guidance for such cases, they were told it wasn’t Amazon’s job to interfere."

"Amazon, in its marketing and privacy policy materials, doesn’t explicitly say humans are listening to recordings of some conversations picked up by Alexa. “We use your requests to Alexa to train our speech recognition and natural language understanding systems,” the company says in a list of frequently asked questions."


Whatever the Amazon company policy is, Amazon can't legally demand workers not to report crimes to authorities. No amount of NDA or legalese makes that possible. If the workers suspect that there is violent crime going on, especially if children are harmed, they should report it to authorities, no matter what the corporate policy is.

It seems that workers are willing to sing-off their basic humanity and dignity to corporate authorities just like they did in Milgram experiments.

If Amazon does not want to get involved, they should not get involved by listening.


People sometimes say alarming things as part of sex play.


People may say alarming things while practicing their lines for a play, too. You can probably think of other plausible reasons. But Amazon workers can't know the context.

If Amazon workers hear something that sounds like a situation the cops should intervene in, they're in a tough situation. Not calling the cops could mean letting someone die. Calling the cops could embarrass someone or, somewhat less likely but possible, get someone killed because the situation unintentionally escalates.

Calling the cops definitely draws attention to the fact that Amazon was listening in the first place, which I would argue is a good thing and which Amazon would really like you to forget.

My feeling is that for the sake of the workers' humanity, they should be allowed to call the cops if they are concerned about the user's safety; otherwise you're asking them to ignore their conscience, which is dehumanizing and likely to haunt them.

And if users plan to say alarming things, they should turn off their AI microphones first. Having them know to do this brings healthy attention to the true nature of the devices.


And if I hear some of those things through an apartment wall, without understanding that it's consensual, I'll probably call err on the side of caution and call the cops.

Sometimes they don't.

Sure.

And if there's any question, that's why we agree on safe words.


Sounds like the setup for a Sorry, Wrong Number [0] of the modern age.

> https://en.m.wikipedia.org/wiki/Sorry,_Wrong_Number


This could work as a short film as most of these human labor arbitrage opportunities exist in emerging economies where they're considered "good" jobs. Such as the Philippines, or Eastern Europe.

I hope they provide counseling and adequate protection for staff, eg frequent polling and 1:1s to early find signs of when it becomes a problem.

Wifes sister is an EMT and she's really a special kind of person. Whether natural for her or a consequence of her profession, things just tend to slip off her like water on a goose. One has to wonder though how much sticks on a deeper level, perhaps surfacing during personal distress years later.


I mean, is this really news? We know that Amazon records the clips of what you say to Alexa for the purposes of improving the recognition. Everything after you say "Alexa..."

How would they tell whether their models are right or wrong without listening and having someone compare?

I see nothing in this article to suggest the clips they're listening to are related to an always-on microphone.


Everything after you say "Alexa..."

It's everything after the product thinks you said "Alexa".


That's probably one of the only competitive advantages of Bixby. That's likely not going to get misheard very often. Though I guess "OK Google" is probably hard to mishear, too.


The first thing I do when I set up a smart speaker is turn on the accessibility option that chimes when it's listening so I can tell when the mic is active.

Coincidentally, this is how my wife discovered and fixed a filler word habit of "Ok, good, well..."


As someone who works in digital marketing and SEO, I can't tell you how many times my google assistant thinks I'm talking to it during the course of my day. I really wish they would let me change the wake word.


Not so sure about that. Google has now become a verb, not hard to imagine false positives.


True, I imagine its false positive rate is higher than Bixby's. But I think the specific combination of "o-kay-goo-gull" probably doesn't clash with too many things; even when talking about Google. But my Alexa seems to pick up all sorts of words vaguely sounding like Alexa when I play videos.


So I’m sitting in my office, talking with a colleague…

“OK, Google patent 9,876,543 and see if we’re infringing.”

or if my colleague is named Alexa…

“Alexa, let’s hammer those idiots.”

I think the first would wake Google Home. I guess Alexa lets you change the wake word to ‘echo’ or ‘computer’. It would be better if it let you use something arbitrary like ‘Rumplestiltskin’

IP leakage or potential misunderstandings don’t seem so improbable to me. Especially if the listeners in (from OP Bloomberg article) in Costa Rica, India, or Romania aren’t au courant with “hammer those idiots” as English idiom in context.


"OK, Google it!"

I use the word google a lot in other contexts simply because they are a big company with lots of services, and they happen to make the (tech) news quite often. So only the word OK would have to be misheard. Bixby seems safer in this regard.


They should not save recordings by default. They only should save recordings of people who opted-in to be a beta tester or people who spotted a bug and want to send a bug report. But companies instead make every user a guinea pig by default and try not to disclose it clearly hoping the user won't realise it.

It is interesting that while there are several manufacturers, all of them opt-in users as testers by default, no matter what product you choose. So maybe such market needs a little stricter regulation.


One major issue that you're glossing over is that Alexa is easily triggered, even when no one explicitly says "Alexa". So audio clips of private conversations are being sent to Amazon.

For an example of accidental triggering, look up news on "Alexa creepy laughter".


> How would they tell whether their models are right or wrong without listening and having someone compare?

If they can't do it, maybe their product shouldn't exist yet.


yes and google does it too, which is very creepy and not useful to the user at all.

unlike the location history where I understand the use case and maybe search history. but all those should be disabled by default.


Hotels in Seattle used for flyins by $bigtechco have Alexas.

I unplugged it for one flyin.

The housekeeper plugged it back in while I was out.

I unplugged it again.

Why in the world would I want any of these smart speakers?


There are "smart-apartments" going up in my area that have Alexas and whatnot preinstalled. They're some of the nicer apartments in the area and I'd love to live there. However, it's against the leasing agreement (which is pretty firm) to remove / disable them. Insane.


> However, it's against the leasing agreement (which is pretty firm) to remove / disable them.

I find that hard to believe. In fact, I can't imagine how it's not illegal. Or if it's not, how it can remain legal.


IIRC they are doing some kind of tech study which got them a grant to build the place. There's an entire suite of things from "smart-solar" to the Alexas. The thing about the lease is not fact, it's just what the leasing office told me when I asked about it, I did not read an actual legal document.

If that's true then the hype about tech companies being literally Hitler is pretty much right. That's fucking insane.

Find a decorative copper mesh basket to place over it in order to disguise it and hold a nice vase on top.

Agreed. It's almost spooky how friends, family, hotels, and other places that used to be considered "private/semi-private" now have microphones listening all the time, in plain sight. Most people don't even notice or seem to care.


Isn’t it crazy how worked up people get about Alexa, etc., conviently forgetting about the always on microphone they’re carrying around with them every day.

I know people that have said "I don't care about my Alexa because I have my phone all the time anyways", but I think this is really the wrong way to approach privacy. They're just accepting their fate without any type of recourse.

I am a fan of Android and it's what I use, but I don't like the privacy concerns. Maybe I'll switch to iOS soon. I'm not patient enough to try to use a device like the Librem phone either.


Or people /don't/ forget, and work hard to turn off that functionality on their mobile devices too, and hate that it is there in the first place (and hope to god turning it off actually disables it).

I get that you needed to fulfill some need for a pithy comeback though. Well played.


Deliberate obfuscation is the only way to have privacy...

They probably trust that the company doesn't use it for bad stuff, combined with the fact that they likely have pretty "normal lives" I guess.

Facebook is using smartphones to listen to what people say: https://www.independent.co.uk/life-style/gadgets-and-tech/ne...

A lot of people said Facebook ads suggest them products or services debated on phone calls.

If you have Echo or Kindle Fire TV, guess it can be easily reproduced when Amazon really listening and analyze your voice.


"A lot of people said" is not enough for a proof. What might be happening, is that Alice and Bob are friends. Bob spends some time researching a topic, say looking for a new car of brand X. He calls Alice and tells her about him wanting to buy a car of brand X. Some time later, Alice sees ads for car of brand X. Now, it could have been FB listening on the phone call. But there's another possibility - FB knows that Alice and Bob are friends, and it knows Bob likes brand X, so it assumes that a friend of Bob will be interested in brand X.

> "A lot of people said" is not enough for a proof.

Agree. Here's something like video proof from BBC: https://www.bbc.com/news/technology-35639549


This sounds like a technological proof of concept that someone _could_ build such a thing, and a few anecdotes.

I could see it working like that. However, the best way to figure out is to just test it. Talk about something random and see if those ads popup on your feed. I did it on my own and well, the results came out to be true or a really big coincidence.

I think people underestimate the number of data points fb/google has access to. Many of these examples can be explained by other things.

Receiving baby items ads while no one knows your wife is pregnant ? Well maybe she googled a brand of baby strollers and fb used its graph + the fact that you're at the same location everyday after work to determine you're her husband.

Receiving ads for a small music band you only heard at that open mic night you randomly joined 3 days ago ? There probably was an event on fb with the list of bands that played that night + logged in the bar wifi.

With the number of people in the "hacker" community who want to prove fb and google are bad we'd already have real technical proof of it happening. Either they send the data back to a server and process it there, or process it on the device directly, both of which should be fairly easy to detect.


This is the magical thinking explanation. What is much more plausible is that they have such enormous datasets that they are able to train machine learning models to predict interests of people on the basis of any number of online behaviors that they record.

The behavior doesn't even need to "make sense" - it could be that people who log on at these times, live here, travel by train, like dog photos and belong to certain groups are highly to be interested in a particular product. It doesn't matter, the system will learn these relations anyway. It might seem spooky and eavesdropping-y from a naive user's perspective, but the simple fact is that when you have that much data you don't even need to eavesdrop.


Nearly all cases I've seen of this are better explained by other information that Facebook has about you.

People in the sixties: "I better not say that or the government will wiretap my house."

Today: "Hey, wiretap, do you have a recipe for pancakes?"

https://twitter.com/andreacoravos/status/999761670540025856?...


The joke doesn’t really make any sense though. How will they know to wiretap your house because of what you’re saying unless it’s wiretapped already?

That is pretty much the way it works now using parallel construction to make up a reason to make the wiretap official.

I understood this statement (“whoops, probably on a list now”) to mean that the government is always listening for key words, and using one would assign someone to listen to every word.

Informers

Hm two comments and both are saying this is ridiculous when in reality - this is reality.

Guys I think we lost to the tech overlords.


Seems like a false equivalence. First, a corporation is not the same as the government. Second, the government wiretapping your house is a violation of your constitutional rights, whereas you freely choosing to add a smart device into your home is not.


A corporation can give or sell your data to the government.

Although Motel 6 just got in trouble for doing that

https://abcnews.go.com/US/motel-pay-12m-settle-lawsuit-shari...


Interesting case. What surprises me most is that the lawsuit was filed by Washington's Attorney General which is technically also a part of the government. Probably that is because he was elected rather than appointed.

Something to keep in mind is that the government is not a whole. It's lost of disparate entities all working towards their own interests. This [1] is their AG and his own personal ideological perspective is what you'd expect for the person that filed the lawsuit. And, you'll find this to invariably be the case on government (and private) actions. This is another example of why real diversity, ideological diversity, is absolutely critical. It keeps everybody honest. If their AG was ideologically in support of the consequences of the actions of the Motel, it's much less likely that this case would have been filed.

This reality is something that should never be ignored as it goes all the way to the top. For instance the Supreme Court is the highest voice in our nation on legal matters and they generally only hear cases where the outcome should be complex and difficult to perceive. Yet on most matters you could predict with a pretty good degree of certainty exactly how most the court would vote, on most issues, based on little more than knowledge of their own individual ideologies. This is why I'd never want a 9-0 supreme court (or even a 7-2), even if the majority aligned strongly with my views and values. We're all subject to bias and other issues. And the only really good way to make sure we keep ourselves based and honest is to ensure the presence of dissenting voices.

[1] - https://en.wikipedia.org/wiki/Bob_Ferguson_(politician)


Also state vs federal. Very different.

And this is exactly the loophole that the government relies on.

Most of all major US based tech corporations today are part of various secretive government programs including PRISM [1]. Internal NSA slides, never meant to be publicly revealed, state that PRISM facilitates "extensive, in-depth surveillance on live communications and stored information." Examples offered included surveillance on email, video and voice chat, videos, photos, voice-over-IP chats (such as Skype), file transfers, and social networking details. Snowden in describing the system stated that, "in general, the reality is this: if an NSA, FBI, CIA, DIA, etc. analyst has access to query raw SIGINT [signals intelligence] databases, they can enter and get results for anything they want."

This is really where the constitution is starting to fail us. The government can increasingly circumvent the constitution by simply pressuring, or enticing, monopolizing companies to cooperate with them meaning they need not pass any particular law, which in turns means that the constitution is increasingly powerless to constrain the behavior of the government. As one obvious example, imagine the government wanted to stop videos from being published on some topic. In times past this would have been a huge deal and they would have had to try to pass a law, which would have directly run into the first amendment. Today, all they need to do is pressure, or incentivize, a tiny handful of companies, Google and Facebook in particular, to cooperate and they can censor whatever they like without passing a single law.

I wonder if this is analogous to how the interweaving of church and state felt in times past, before we started to pass laws requiring the separation of church and state. Both entities are able to covertly pursue their own ends with mutual plausible deniability. 'We're not being anti-competitive. We're behaving within the bounds of the law set by the government.' 'We're not censoring anything. Corporations have every right to set their own rules and policies.' Whatever the case I've no doubt that the next great nation that is started from scratch, as the US was, will undoubtedly make some effort to isolate government from business, and business from government.

[1] - https://en.wikipedia.org/wiki/PRISM_(surveillance_program)


Shouldn't people be having fun with or trolling these always-listening systems by speaking gibberish, in tongues or reciting custom sea shanties that reenact purely fictional accounts of high crimes on the open seas?

It's usually good for a company to listen to what their customers are saying. This time, not so much.


It’s called “supervised” machine learning for a reason...

The important part is that tens of millions of people used Alexa everyday and the utterances are anonymized before being used as training data, so you don’t know who said what, just that somewhere someone said “blah”.


I'm sure their anonymization policy is un-misleading as their terms around whether employees listen to Alexa commands


How have we got to the point where someone listening via a microphone inside my house is fine as long as it's not real-time?


Everyone who owns one of these devices has given express permission and must know. Its fine since they have told Amazon its fine

It's antagonistic and obviously misleading to state that a person has given explicit permission to a company without any further explanation when the reality you're basing that on is that the permission was in the wording of a contract that person clicked "agree" on.

You may insist all day that this person was irresponsible in examining the contract, which I will also disagree with, but claiming that they have given explicit permission is a lie on your part, and I would have a hard time believing that you don't know that.


> utterances are anonymized

if only we had some technology that could identify a speaker based on their voice. oh, wait a second. but, lol, no, they aren't anonymized ... "recordings sent to the Alexa auditors don’t provide a user’s full name and address but are associated with an account number, as well as the user’s first name and the device’s serial number"


Apologies to the poor folks reviewing audio, but the only room my Echo is allowed in is the bathroom.

No joke, part of me believes that bathrooms (and kitchens, to a lesser degree) are the first places voice-controlled smart home tech will really take off. My place is about as smart as it can be today but it's still mostly a novelty.

Once people can perform actually useful tasks with their voice - "hey siri, turn the shower on to 40 degrees" or "alexa, preheat the oven to 300" - while their hands are full doing something else it'll kickstart the whole field.


This sounds dangerous.

How so? The device could always require confirmation for hazardous commands - "are you sure you want me to turn the oven on?" Most already do this for e.g. deleting calendar appointments anyway.

This could be hacked and burn you.

I mean....there will be a future in which voice data is processed locally rather than in the cloud. Nobody said this stuff has to be connected to the internet.

Snips' home assistant does local processing

https://www.home-assistant.io/components/snips/


Oooh, thanks for the heads up - I've been playing around with HA but hadn't seen this component. I'll check it out.

Right, I don't know if NSA and other agencies will allow this kind of future but who knows, you might be right after all.

There's an interesting disconnect from pieces like this and those demanding that tech companies monitor every piece of content on their networks.

Is there? Seems like a massive difference between recordings in the privacy of one’s home and, say, tweets intentionally published to the world.

There's no disconnect when those are entirely different categories of event

As a user its not just I don't mind but I expect them to do so, at least to improve the product.


As I bystander, I expect them to never record my voice since it's against my express wishes. Only one of us ever gets what he wants. Doesn't seem fair.


This is the case all the time, though. You get recorded by CCTV cameras everywhere you go, even if you don't want them to.

In a public place I expect some level of privacy intrusions. CCTV cameras aren't so bad in public as long as the recordings are only stored for a few weeks and the data is not analyised unless there is reason to believe it contains evidence.

For example its acceptable to access the video to prove someone stole from your store but it is not acceptable to use facial recognition on your cctv to see who your best customers are.


It is in my opinion absolutely unacceptable that this submission was originally the article title and that was subsequently changed, presumably to defuse it. Someone needs to clarify when the norm of using the submission title applies, when it’s thrown out, why it’s thrown out, and who gets to decide.

In my opinion, the only sensible approach is for the title policy to be unilaterally enforced. Any departure from it will invariably involve someone’s subjective ‘political’ stance on a matter.

As it stands, it looks as if someone at Amazon applied pressure to have this changed. I really hope that isn’t the case because it’s almost too shady to be believed.


I agree with you to certain extent. I think the problem with titles directly out of the article is that many times they are bait.

The writers of those articles are purposefully leaving key information out, to grab people's attention and likely force them to read the entire article to find the missing information or fulfill the title entice.

In this case I think the change of title is definitely defusing the article, but it's also giving the key information that was left out in the bait title.

I agree with you because changing the original title feels like a disruption of the discourse and opens a bias door for whoever changes the title. But, I also believe that bait titles erode the quality of the content and make harder to consume and evaluate information. It's a hard problem.


I disagree almost entirely on the basis of that being a ludicrous sort of infantilization, as if this community can’t be trusted to sort fact from fiction and must instead be coddled towards the “correct” interpretation.

nvm

I think this sort of process, assuming the privacy and security teams have their say, is an absolutely legitimate part of product improvement.

Don't the people developing these speech/AI methods generally have advanced degrees?

In that case, shouldn't they be aware, from the same grad school training that prepared them for this work, that a (genuine) human subjects board would require informed consent for this, at the least?

Do they have informed consent?


I am both an Ex-Googler and a PhD researcher (working with human data). You don't require informed consent in this case, it's not human subjects research, and the terms of service cover this already. What is being done is completely legal and if people don't like it, they're free not to use the service.

How is this not human subjects research?

And is modern consumer "terms of service" the human subjects ethical standard for corporate researchers?


because humans aren't the subject of research here, instead they are data providers. The alexa product is the subject of research here (and it's not research, it's product improvement).

I don't understand your question about terms of service and ethics here. I work with real human subjects research data and it's a completely different field that does not apply to this situation.


Is this supposed to be sinister? They need to QA the product they are building so it's a better experience for customers.

I accept that Amazon, Google, etc. are likely listening to me with computer AI, but the fact the employees could pass these around causes some worry. It's like when I worked my first job at AOL and people would read Rosie O'Donnell's mail and chats. The human element is more troubling than the AI.

The Bloomberg cookie banner is such a dark pattern (literally) it’s laughable. Can’t tell yes from no with the buttons.


I was reading a while ago about how many nukes we set off in the 50s and 60s and thought: what are we doing today that future generations will think "what were they thinking?"

Surveillance capitalism was the first thing to come to mind.


For better or worse, I expect the opposite will happen. Future generations simply will not have the same expectations and fears around privacy.

The modern Western conceptions of privacy are relatively anomalous from a historical perspective. There's nothing to stop them from changing again.


What's happening now is totally different from the lack of privacy experienced in a tribe or a close knit village. This is top down panopticon surveillance where an opaque eye spies on everyone on behalf of a complex of corporations, advertisers, criminals, and governments. The lack of privacy in a village was reciprocal. This is completely asymmetrical and exploitative.

You might find this video interesting:

https://youtu.be/LLCF7vPanrY

Shows every nuke set off between 1945 to 1998.


Why did you add capitalism to surveillance?

I was using it as a technical term more than a political one. Capital is a rough synonym for assets, but "surveillance asset accumulation" is clunky. The capital being amassed here is information about people. That capital can be leveraged to control and manipulate for both economic and political reasons and can be used by both (politically) capitalist and less capitalist authoritarian regimes.

Amazon products are the worst if you are concerned about privacy (their tablets are awful)... Google probably comes at a close 2nd and Huawei maybe 20th? (that's a wild guess but Amazon is hard to beat)

Wow - really? I'd say that Facebook is far, far, far worse than Amazon. And Google is certainly worse as well. Look at the data collection on Android phones, for example.

At least Amazon, from day 1 of the Echo/Alexa launch, noted clearly (not in small print, but right there in pictures on the product detail page) that the user's voice was going up to the cloud. And provided a microphone-off switch right on top of the device. Where's the mic-off button on your (typical) Android or iOS phone?


Presumably Apple are the best in your totally impartial and unbiased opinion, right?

Apple is better at PR for sure... but we have seen in the past that they lie about sharing your information with the government...

Ai is so pnambic right now.¹

We are flooded with hype about AI, but they seldom mention the armies of people it takes to get it working and to keep it working.

Even members f the public are recruited to train AI system without their knowledge, i.e.: captcha google translate suggestions.

Sounds more like a smokescreen for normal human intelligence to me.

1: http://www.jargon.net/jargonfile/p/pnambic.html


Have you thought how funny it is that Amazon is using people in order to train AI to get a competitive advantage?

No, I haven't, because that's how all ML models learn. It's called supervised learning and supervision comes in the form of labels produced by human taggers. It's not something special that only Amazon Alexa does. Tagging is as necessary in ML as data entry in accounting.

How else would they train AI?

Another reason to support on-device audio processing coughApplecough

I know someone who does this. Some people use Alexa to text so they get a bunch of sexts. Many, many people just like to say the worst things they can think of to try and get Alexa to say something anti Semitic or what not.

I don't mind not buying a Google Home it whatever, but is Google listening on my "Ok Google" enabled Android phone as well?


I can't recall the google settings screen but you can review everything that google has picked up. I reviewed mine and while most were ok there were a good number of private conversations with my wife that google had randomly picked up. Most of them sounded like stuff I was saying while driving. Which means that it was purely based on what android thought it had heard vs any manual input. I can only assume google home will have similar moments.


Go to https://myaccount.google.com/data-and-personalization

then click on 'Audio and Voice Activity'


How else would it catch the words "Ok, Google" in the first place?


All these things have dedicated low-power hardware that just listens for the trigger phrase and has no network access. That's why you can't choose your trigger phrase.

offline (on device) model for that only instead of cloud based. I'm a deep learning researcher, and that can be done for sure

Just don't enable "OK Google", it's really not that useful anyway.

You should be able to see the services (daemons) in Developer mode.


Idk about everyone else here but for a while now and forever more when I see people in the comments justifying tech companies immoral behavior I assume its shills. Whether directly paid or whether the companies have propaganda campaigns that just get idle commenter on their side.


Man, that is pretty reductive. There is a lot of nuance here, and to act like this is somehow a priori "immoral" is not a reasonable conclusion. This behavior is clearly related to trying to provide the service that people bought. We can argue over it being immoral or not, but this isn't as cut and dry as you are making it out to be.

Of course, you already said I am a shill, so I don't know why I am trying to respond.


It's not that nuanced in this case. These things are literally wire-taps. You can have a voice assistant that does just about as well as Alexa/Echo/etc, without having to sacrifice your data privacy.

Are you sure about that? Every offline voice assistant I have tried does a lot worse than Alexa. You have any examples of ones that are as good?

What's your metric? And are these offline assistants smartphone apps? NLP is expensive computing, so it'd make sense for those to do poorly. And this doesn't even have to strictly be offline; You can have your data processed online in a privacy-respecting way, so long as you actually know what's going on under the hood and have control over the data.

My shill is MyCroft[0]. They're partnered with Mozilla to develop an open source NLP engine[1], so I have high hopes for it.

[0] https://mycroft.ai/

[1] https://github.com/mozilla/DeepSpeech


It does look promising, but it also seems like it has yet to get to the point of working "just about as well as Alexa/Echo/etc." – which means it's an open question whether it will be able to.

"This behavior is clearly related to trying to provide the service that people bought."

OK I have a device that will let you do all your chores for free and for no work. Oh now I need slavery to be legal. It's ok tho - im trying to provide the service people bought.

See how a tangible example makes tech monoliths flaunting of the law sound ridiculous? Yes. You sound like a shill.


This argument makes no sense. How is this anything like slavery? And what law are they flaunting?

Yeah, I always assume that anyone who disagrees with me is a paid shill too. It makes things so much simpler!

Id refer you to the Wikipedia list of logical fallacies.

Amazon apart, what's inmmoral about labeling data that you got with the consent of the user?

It's hard to appreciate what you are saying when you're basically judging the incentive of economic/technological progress as an inmmoral behavior (in this case the intention of improving a technology)

Labeling data and training a machine learning model sounds far from inmoral. Perhaps for some people the idea of companies developing machine learning models and capturing data to train them is an ethically gray concept.


Does Amazon clearly says that everything you say to Alexa will be recorded and reviewed by its employees? Or they prefer to hide this somewhere in the legal documents using vague wordings? That's what is wrong. They know that if they say about it explicitly, their product will look worse than competitors', so they try to hide it.

Everything you say is not reviewed by employees, that would be impossible given the scale. It clearly says in the article that a very small sample are chosen for human annotation. Every single voice assistant uses supervised machine learning algorithmns, which require labeled data. It won't look better or worse than any of the others in that regard.

Yes, I meant "everything you say is recorded and might be reviewed by employees". But Amazon doesn't have to make every customer a beta-tester. And Alexa could give a warning when setting up for the first time, something like "Look, we are recording everything you tell me so please don't say something too embarassing or something that might get you into a trouble with police". While this is what any honest person should do, no way Amazon or any other major tech company does this.

I don't disagree, it probably should be made more explicit to customers that someone could listen to certain things said to the device. I just wanted to point out that there is a big difference between reviewing everything a customer says and having the possibility of certain phrases reviewed.

> Does Amazon clearly says that everything you say to Alexa will be recorded and reviewed by its employees?

Thing is, they can't possibly review everything. Tagging is hard. The taggers can accomplish 1000 data points per day, as the article says. They can only cover a tiny portion of the Alexa data, and probably they use some algorithm to select which part would benefit the model most.


> Amazon apart, what's inmmoral about labeling data that you got with the consent of the user?

They don't have my consent and I'm increasingly surrounded by the damn things. Take a look at the comments about hotels, new condos, etc. that have Alexa or Google Home installed. And, of course, friends and family.


When did I give anyone consent? Because I used a monopoly's search engine? Because Facebook has data on me when I don't have an account?

And more to the point - what kind of hell is this where getting permission to my data where I have no say on the unethical ways they use it?

You're doing exactly what I'm saying with the shilling tho.


Everyone who disagrees with you is a shill. Got it.

Yeah I heard another shill say that a minute ago. No. Disagree all you want. I love dialectic and I'm always ready to change my mind.

It's just that there's a retarded level of corporate love in the world.


Dismissing someone as a shill, an ad hominem, is not a dialectic argument.

Agreed. And it's pretty obvious too. If you ever have some serious misgivings about any of these IoT privacy invasions and voice it in a comment, a deluge of comments scolding you for being a Luddite start appearing within seconds.

Earlier today someone made a tongue-in-cheek comment about https://notnews.ycombinator.com. I thought it was flippant, but here this article's title proves that it was prescient, instead.


No surprise there


I’m shocked. Shocked!


I don't think the sarcasm is warranted, because as technical people, we already implicitly understand how the system works.

I was listening to Reply All [0] yesterday, and one of the hosts had to explain to the other that, of course, the device has to listen to you to constantly in some fashion in order to know if you've said 'Alexa' or 'OK Google' or whatever.

It hadn't even occurred to me that people wouldn't know that.

[0] https://www.gimletmedia.com/reply-all/139-the-reply-all-hotl...


I think the problem is not that the device is listening. The problem is that users' recording and seacrh queries are saved to the cloud, stored indefinitely and reviewed by other people. And some users might even not realise it and think that they are just talking to a robot, not to a team of Amazon's employees.

You're right, of course. I think I was just reacting to the tone of the earlier comment

Why do you assume this is sarcasm?

>Millions more are reluctant to invite the devices and their powerful microphones into their homes out of concern that someone might be listening.

Sometimes, someone is.

This wording is misleading, because it sounds like someone is listening through the device as the speech is being picked up by it, but in fact it's recordings.


What is a recording? Does a 10 millisecond delay mean it is a recording? What is the legally binding lock-out time for this audio stream before a human can listen to it?

A recording is end pointed audio captured and stored on a server and reviewed later.

There is no legally binding lock out time because there doesn’t need to be. Live-listening is impractical at scale and also worthless for what the article is describing they do with the data.

Remember, this work is being done to make it so humans don’t have to be in the loop.


Why is live-listening impractical at scale? How can Discord provide such a service? You seem to be conflating Amazon's intent with the unknowable intent of anyone that can touch such data, which is a common fatal flaw in reasoning.

>Why is live-listening impractical at scale?

Because there is no value in doing so. A person could collect all the spare change thrown into fountains at shopping malls, but they don’t.


If I worked on voice search I would wish to have full context to train my models on. Full context could be year long. Ethics are an issue here for sure.

Sure. But the devices don’t do that.

By this logic, if I watch a recording of a TV show hours or days after it's aired, I'm not actually "watching" it.

A better comparison would be if you watched 8 second clips of various TV shows from around the world without knowing the title and without having control over which ones you watched.

Whether you consider that “actually watching” is a matter of opinion, but your ability to process and take action on any of the clips is certainly diminished.




Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact

Search: