
Facebook's Not Listening Through Your Phone. It Doesn't Have To - ForHackernews
https://www.wired.com/story/facebooks-listening-smartphone-microphone/
======
zherbert
Today my girlfriend spoke about wanting a cinnamon roll, and later in the day
her iPhone Facebook app showed an advertisement for cinnamon rolls.

We were trying to determine if maybe she had seen that same ad earlier in the
day on Facebook, and it planted the craving in her head.

Regardless, it was a very freaky experience. I doubt Facebook is listening,
but it's likely their ads are inadvertently influencing us. As we scroll
quickly through our news feeds, we automatically ignore most ads. But they are
still making an impression.

~~~
jordigh
> We were trying to determine if maybe she had seen that same ad earlier in
> the day on Facebook, and it planted the craving in her head.

People don't seem to really understand how nefarious ads are. We think we're
so smart. We won't be influenced by ads. No way we're falling for that. But if
you know what "just do it", "the happiest place on Earth" or "think different"
refer to, regardless of you opinions of those products, you have already been
influenced by ads, and your next buying choices will reflect that.

We're frail, easily manipulated creatures, unprepared for the combined
centuries of expertise that hordes of advertisers can control us with.

~~~
throwaway613834
> But if you know what "just do it", "the happiest place on Earth" or "think
> different" refer to, regardless of you opinions of those products, you have
> already been influenced by ads, and your next buying choices will reflect
> that.

I beg your pardon? I know where the "happiest place on Earth" is and yet I
have only ever been there once... because I was taken there by others, not
because I chose to go. Not sure how my buying choices have ever even remotely
taken that slogan into account.

~~~
jordigh
Ads essentially control the domain of discourse. Suppose you _did_ want to go
to a theme park. You will have to consider Disneyland as a possible option,
because you're keenly aware of it. Ads aren't just trying to convince you that
something is great; they're also trying to make sure you just always know the
product is a possibility, that it always factors in your choices, even in your
choice to consciously reject it.

To use a cliché, the opposite of loving a product isn't hating it; it's being
indifferent to it.

Also, the fact that you've already been there means the ads are working.
Whether it was your choice or not is less important. You have already been
made a consumer.

~~~
throwaway613834
> Also, the fact that you've already been there means the ads are working.
> Whether it was your choice or not is less important. You have already been
> made a consumer.

You seem to be pretty adamant about shoving your wrong idea down my brain even
though I'm explicitly telling you you're wrong. I'm telling you I had _zero_
input into whether I go there the first time (others wanted to go while we
were around the area and I couldn't care less whether we went there or
anywhere else) and that I have since never been there since because I still
cannot care less whether I go there or not. I've literally _completely
forgotten_ about its existence every time I've been in the area. My reaction
to ever going there is exactly the same as it would be to any other theme
park: "okay" if I've never been there or if I've enjoyed it before, or "
_shrug_ " otherwise... irrespective of the ads.

In other words: unlike your claims, I am neither "keenly aware of it", nor do
I "always know the product is a possibility", nor does it "always factor into
my choices", nor is my choice to "consciously reject it", nor are the ads
"working" by "already making me a consumer". Ads or no ads, Disnelyland or
Foobarland, I would have been there the first time just the same, and not
there since then. The ads just got the slogan into my brain and literally did
nothing else.

~~~
jordigh
It's not a nice feeling to know your mind has been hijacked. But it has been.
Ads are very effective.

------
dfabulich
The "Reply All" podcast had a fantastic episode on this.
[https://gimletmedia.com/episode/109-facebook-
spying/](https://gimletmedia.com/episode/109-facebook-spying/) (Click the
"show transcript" link to see the transcript.)

One thing they realized is that it was impossible to _convince_ anybody that
Facebook doesn't listen in on microphone conversations, even given all of the
facts.

One of the co-hosts, Alex, gets on the phone with five people to try to
convince each of them that Facebook doesn't use the microphone. He's unable to
convince even one of them.

~~~
MarkMc
I suppose it would be good for Facebook if Android had the capacity to show a
log of which apps used which phone features when. eg.

12:36pm Google Maps used the location feature

12:48pm Instagram used the camera

1:16pm WhatsApp used the microphone for 34 seconds

etc.

~~~
mikeash
iOS doesn't have a log, but it does have a really obvious visual indicator (in
the form of a fat red status bar) when an app uses the microphone in the
background. I became convinced that this whole story was nonsense when I
realized a lot of people reporting it were talking about iPhones.

Something like what you describe would be wonderful on either platform,
though.

~~~
danso
Yeah that seems to me to be the strongest argument against FB using the
microphone. For iOS, it would require circumventing the Apple guidelines. FB
is clearly a big player but not big enough for Apple to bend their privacy
rules for.

~~~
Yetanfou
They created an exception to their rules for Über [1], a much smaller company
than Facebook. If Facebook came with some convincing reason why they really
needed something similar I don't see why they would not get their wish
granted.

[1] [http://fortune.com/2017/10/06/apple-uber-secret-
entitlement/](http://fortune.com/2017/10/06/apple-uber-secret-entitlement/)

~~~
mikeash
It would have to be a convincing reason _to Apple_ , though. In Uber's case,
that convincing reason was providing a better Uber experience on the Apple
Watch, which is something Apple would go for. I'm pretty sure Apple would not
go for "let us listen to the microphone at all times so we can tailor
advertising by spying on the user's conversations."

------
throwaway613834
I stopped reading the article after the first piece of nonsense:

> To make it happen, Facebook would need to record everything your phone hears
> while it's on. This is functionally equivalent to an always-on phone call
> from you to Facebook. Your average voice-over-internet call takes something
> like 24kbps one way, which amounts to about 3 kBs of data per second. Assume
> you've got your phone on half the day, that's about 130 MBs per day, per
> user.

No, this is complete nonsense. They could easily record the audio locally
(perhaps even sporadically rather than constantly -- you don't need constant
audio data), do some local speech recognition when the phone is plugged in,
and send over the resulting text in short bursts along with all the other data
they send over. There's no need for raw audio to be sent over the wire. It's
not like they need perfect accuracy.

~~~
aptwebapps
You should have kept reading, because that was discussed as well.

------
krishicks
If you liked this article, I recommend his book, Chaos Monkeys, about his time
at Facebook as an Ads product manager.

~~~
creaghpatr
Yes I agree, and he’s a great contributor choice for Wired which has been
going downhill lately

------
woodandsteel
The author says facebook isn't listening in on your smartphone, but it can get
all the information about you it wants by other means. That makes me feel so
much better.

~~~
torgard
That seems to be the point of the article.

People are in an uproar about being recorded, but not realizing they are being
'recorded' in many other ways.

------
teaneedz
It doesn't really matter what a former PM focused on ad tech says, because
true or not, the average user will not believe FB anyways. FB won't be trusted
because of it's anti-privacy history and experiments. That FB gets so much
data from other sources just increases the impression that FB is not to be
trusted. The first argument made in the article is disingenious and only acts
to discredit FB even more. It's not just a FB problem, it's an ad tech driven
business model problem. However, Facebook keeps on a path that makes it the
perfect poster of a company not to be trusted. Listening or not, FB is not
trustable.

------
yosamino
> In the bright-eyed naiveté of my first few weeks as Facebook's first leader
> of the ads targeting effort

Not to be too snarky, but this is facebook's (ex?) ad targeting leader telling
us that everything is A-OK? He might well be right, and he presents some good
arguments, but in the face of such an opaque system, that doesn't inspire much
confidence.

------
ravenstine
I don't think they're listening through the microphone, but I've been
wondering to what extent they use your photos to determine how to market to
you. For example, most of my photos on Facebook show me with either hair or a
hat on; as I've gotten older I've actually developed male pattern baldness,
and so one day I thought it would look best to just buzz off the rest. I had a
picture taken of me and use it as my Facebook profile pic. Since then, I seem
to get more and more ads in my feed for hair transplants. Maybe they classify
your photos and sell that data?

If you think about it, there's a lot more guesses they can make from your
photos than listening to a bunch of disjointed conversations. They really
don't need to hear what you say.

~~~
danso
Have you checked out your FB ad preferences?

[https://www.facebook.com/ads/preferences/edit/](https://www.facebook.com/ads/preferences/edit/)

(article with context: [https://motherboard.vice.com/en_us/article/3dk3y8/how-
to-see...](https://motherboard.vice.com/en_us/article/3dk3y8/how-to-see-what-
facebook-tells-advertisers-about-you))

The _Your Information > Your Categories_ tab might be the most revealing. It
is for me as it includes categories that seem to be inferred, not explicitly
stated, such as "US politics", "Frequent Travelers", "Expats (Vietnam)" (maybe
because I have a Vietnamese surname), and "Technology early adopters", which I
can only assume comes from accessing FB using iOS devices soon after their
launch date.

~~~
ravenstine
That's pretty fascinating... I've been totally oblivious to this feature.
Though it turns out I'm not very interesting. The classifications are pretty
rudimentary, listing what devices I use, frequent travelers, computation and
mathematics, and US Politics(liberal).

~~~
khedoros1
Mine are confusing. I saw references to a few things that could've been
harvested from comments, some connected to companies that friends and family
have used, and a lot of things that I think are celebrity gossip and such.
Even the classifications that might make sense on their own were mixed in with
enough noise that they almost seem like accidentally there.

------
samspenc
This part of the article where he describes crunching through some additional
user data unnerves me: "It was like pressing a field of livestock into the
sausage grinder, and getting out one hot dog as a result. And Facebook users
are a very large herd."

Sometimes I wonder if from a high level, that's how Facebook sees users' data
("field of livestock") that goes into the data processing engine ("sausage
grinder").

------
lichenwarp
I've seen this multiple times so it's totally not coincidence, you see a post
that has someone wearing glasses for example and the next sorted post is
someone wearing glasses but the post has no mention of glasses and is
completely unrelated to the previous.

~~~
jrockway
A lot of people wear glasses.

------
hawkilt
i was once talking to a friend that i should get a new laptop bag for my mac,
i never searched on anything. but after 2 days i’m abale to see the laptop bag
ad from amazon on Facebook

~~~
UncleMeat
Why would it wait two days? Have you never seen an ad for laptop bags before?
I see them often.

You can decompile the facebook app. You can install a cert and mitm the
network connection. You can do anything to the client. Yet nobody has found
any evidence whatsoever of this behavior.

~~~
throwaway613834
Ugh, I hate this nonsense. You say "you can do X" as if it's so easy. I assume
this isn't something you're just pulling out of thin air and that you're at
least saying this because _you_ really know how to do it (i.e.: you've done it
before). So, please do all of us a huge favor and explain how to do the MITM
on Android step-by-step, which you seem to think we're all so lazy & unwilling
to do. Not merely "in theory", and not merely on some random app, but in
actual freaking practice, on the Facebook app. Because every single person
I've caught saying this had evidently not tried it on the Facebook app himself
to realize how nontrivial it is. People don't have the time or energy to
switch their full-time jobs to being reverse-engineers of the Facebook app, so
if you think it's so doable, please do the world a favor and show us how to
put this myth to rest. (And to make it even easier, no need to assume zero
prior knowledge. You can assume people already know how to do this on a
desktop, and just teach them how to do it on the phone. I _assure_ you that
the knowledge does not simply transfer over.)

~~~
quesera
I think it's safe to say that "you can do" here means "it is possible to, and
people do".

The risk Facebook would take by pulling a stunt like this is ginormous. See
also: Amazon Echo.

The consumer and legal backlash would be swift and stunning, and the secret
would be _impossible_ to keep. Mobile app decryption is a well-established
process. Reverse engineering a large app is tedious, but fully comprehensible.
If you're specifically looking for recording (streaming audio out, or spooling
to storage), it's much more manageable.

Of course there's a danger in assuming that _someone_ , _somewhere_ has
already done this (or many someones). ~"With enough eyes, all bugs are
shallow" ... sure, if the eyes are open. I don't know anyone who has done this
work for the Facebook app, or for the Echo. But there are so many little
boutique security firms out there today, and the technical prerequisites are
so low...I just don't see how it's possible that it hasn't been done a hundred
times.

The genuine risk, I think, is that a Corp with all the tooling in place could
be compelled by some vaguely legal process in some sketchy jurisdiction, to
target an individual of interest with custom code. This isn't hard either. Of
course, the "tooling" is minor and any popular app could be subverted usefully
in this fashion, so Facebook is not special here.

Still, most of us are laughably uninteresting to LE, but proper opsec still
dictates caution.

------
halayli
author is relying on his own limited technical knowledge on wether it can be
done or not.

There are plenty of ways to do it like sampling / increase sampling when other
fb friends are around or maybe they just listen randomly and in the end
they’ll have a well built profile about you.

It’s not like it needs to be listening all the time or respond anytime a
keyword is said.

~~~
danso
The author was a Berkeley PhD in physics, modeled derivatives at Goldman
Sachs, and was a research scientist at Adchemy before going to FB. I don't
think we should just assume he has limited technical knowledge.

[http://www.antoniogarciamartinez.com/about-antonio-garcia-
ma...](http://www.antoniogarciamartinez.com/about-antonio-garcia-martinez/)

~~~
3131s
Ok, but the whole "Is it possible?" section is predicated on a totally false
assumption:

> _To make it happen, Facebook would need to record everything your phone
> hears while it 's on. This is functionally equivalent to an always-on phone
> call from you to Facebook. Your average voice-over-internet call takes
> something like 24kbps one way, which amounts to about 3 kBs of data per
> second. Assume you've got your phone on half the day, that's about 130 MBs
> per day, per user. There are around 150 million daily active users in the
> US, so that's about 20 petabytes per day, just in the US._

No, all they need to do is use the client device to perform speech-to-text and
send the tiny amount of resulting data.

I don't think Facebook records people's conversations, but I also don't buy
this article's other arguments that 1) conversation data is not useful to
advertisers and 2) Facebook would not at least make a semi-successful attempt
at analyzing natural language data with their existing NLP tools.

~~~
danso
It is misleading to argue that FB has to _store_ that data. But he addresses
the _" all they need to do is use the client device to perform speech-to-
text"_ part:

> _Because it has no specific trigger word for Facebook, your phone would need
> to listen for every targetable keyword. That means the speech-to-text
> translation code could only run on your phone itself, a taxing demand even
> for the beefy cloud servers that usually handle those tasks._

What app currently does speech-to-text translation with any usable accuracy on
the device? Even Siri and dedicated-devices such as Alexa don't seem to
attempt anything beyond trigger-word recognition locally.

~~~
flukus
This is 90's tech, we had speech-to-text working back then for full dictation
which is way beyond the needs of facebook for scanning for keywords. Having
high accuracy wouldn't be important in the same way it is for Siri et al.

