
Facebook Knows How to Track You Using the Dust on Your Camera Lens - huntermeyer
https://gizmodo.com/facebook-knows-how-to-track-you-using-the-dust-on-your-1821030620
======
aleem
From a comment I had left on a related thread,

> Though very uncanny, this is rather easy to extrapolate even using the basic
> facets FB has available. But the most overlooked is photos. They are offline
> beacons and are to offline tracking what websites are to online tracking.
> For example you and a group of 10 other people took photos at the same
> location which FB sees as a small gathering of intimate friends. It will
> need to qualify the location is not a public restaurant to be sure its an
> intimate gathering. The chances of you connecting with that person are then
> really high. And if you are and the other person are in each others photos,
> then it's almost a certainty you will end up connecting on FB should FB
> recommend the connection. The more connected FB's network is, the more it
> can extrapolate based on commonalities from the first degree graph of your
> network. It's also the reason why Google is betting so heavy on photos and
> offering free unlimited storage -- remember, there's no such thing as a free
> lunch. Google wants to build a social network graph based on location and
> facial recognition to draw on proximity. With AI this will get even more
> uncanny. For example a wedding will have a certain photography profile
> (number of pics taken, the time of day, the location and venue based on
> Google Maps or past photographing histroy, the lighting in the photos, etc).
> Once you throw AI into the mix you realize Google doesn't need to draw out
> any conclusions. It can throw all these parameters into the AI engine and
> draw up proximity. In this case, Google or FB will not be able to tell you
> how they drew the connection, because even they won't know. All you can
> assume is the AI engine will take dozens of parameters today and hundreds
> tomorrow. Google's deep investment in AI infrastructure is a bigger
> testament to this.

~~~
semi-extrinsic
On the exact day when my daughter started first grade this year, Google Photos
(which I use pretty often) notified me that it had made a new video montage.
It started with a title slide saying "Look how fast they grow", and then
showed photos and videos of her (not a single mistake with one of my other
kids or any of her friends) growing up from ages 0 to 6, ending with the photo
I had just taken of her in front of school. It was really nicely made, but it
also freaked me out quite a bit how they not only detect "today is first day
of school" and act on that, but also "this photo is of Kid Y, not Kid X" at
all ages growing up.

~~~
rbosinger
I've had it make me around 3 if those already for my 2.5 year old. It's the
same video but they just keep adding a few more months into it. I always save
them. It's amazing, creepy and well done. I had a Premiere project where I was
painstakingly trying to compile a similar thing and now I just gave up on
that.

~~~
ktsmith
I have dozens of these videos now for my two kids and even one or two that are
called "Dog days" or something like that with my dog featured instead of the
kids. They are usually pretty good but every once in a while there's a blurry
pic or they include a video instead of photos and there's some background
motion that's distracting. They've gotten much better since they were first
introduced.

------
pxeboot
_by comparing the accelerometer and gyroscope readings of each phone, the data
could identify when people were facing each other or walking together._

More evidence we need far more privacy controls in mobile operating systems.

~~~
gdulli
I've always been floored that people leave their location services turned on
by default. I have mine on for no more than a few minutes a month. I only turn
it on when I'm in an unfamiliar location, don't have my bearings, and looking
at the map alone isn't enough to help without GPS.

~~~
kraftman
What are you gaining in exchange for making your life harder?

~~~
gdulli
I don't know what you mean about making my life harder. I don't see what I
have to gain by using location services outside the rare moments I need it.
I'm gaining privacy by not keeping it on.

~~~
kuhhk
I think you’re fortunate to not need it, but I lived in an unfamiliar place
for a few months and I used my map apps to identify the best routes (public
transit, biking, should I walk, or should I Uber?) nearly 3-4 times a day.

Turning it off/on would be difficult, but I do set most apps to only allow
location services during use... but the maps I have set to always. I also
don’t have the Facebook app installed

~~~
Scoundreller
The alternative is to keep location on (while in use), but download offline
maps.

No idea if the app phones home with all historical location data if you re-
enable data though...

Maybe that’s why Google likes to expire it’s offline maps ?

~~~
emj
OsmAnd and MapsWithMe are good apps that you can use offline maps on
indefinetly once downloaded. They use Openstreetmap of course.

------
nlh
I was talking to a friend today who said, fairly succinctly, that Facebook's
single biggest problem is that they Totally and Royally Suck(TM) at PR.

I wonder what would happen if they were just straightforward about this.
"Friends you May Know is really cool! We use your phone's location data and do
cross-checks on your address book to make the most intelligent suggestions. We
even use some unique features of your camera to figure out if you know
people!"

They don't have to give away the store, but they could certainly avoid the
creepy "try and guess what ridiculously creepy stuff we're doing to suggest
friends" game.

~~~
drdaeman
> Friends you May Know is really cool!

I've got over 9000 suggestions and not a single accurate one. Maybe they have
the best technology and all the data out there, but I'm skeptical they can
really use all the data they supposedly have on me.

Same goes for any targeted advertisement, not just Facebook. Some come close
(based on simple topic preferences), but in all those years I've yet to see
any large number of the actually useful ones. All those data mining and AI-
driven targeting seem to utterly fail, suggesting me to buy a second TV after
I've just already got a new one. Or subscribe to Grammarly (every second
YouTube pre-roll, seriously?!), when I've already bought that.

~~~
JoeSmithson
I take it a kind of weird 21st century compliment that my machine learnt ad
suggestions are so poor. I think it indicates Google/Facebook don't have that
much data for me. Sometimes it's fun game trying to work out how the algorithm
possibly thought I would know people from other countries with no mutual
friends.

> subscribe to Grammarly

Oh my god. I considered installing it just to stop the ads, but I see that
doesn't help.

~~~
TeMPOraL
> _Oh my god. I considered installing it just to stop the ads, but I see that
> doesn 't help._

I have mixed feelings about Grammarly. It's a perfect example of a thing that
should not be an on-line service, but a fully off-line product. In its current
form, it's essentially a keylogger.

~~~
drdaeman
They don't integrate with my email clients anyway, and their browser
extensions are overreaching. I just use its desktop (Electron) app and
website. Copy the text I want to proofread there, edit it, then paste it back.

Wish their browser extensions would have a non-invasive on-demand mode, where
the user has to click the toolbar icon to initiate the verification
explicitly.

------
xg15
Pardon for the somewhat OT post, but the quotes about patent use in the
article made me realize once again that I don't get how the patent system
works.

> _[Facebook:] "We’ve often sought patents for technology we never implement,
> and patents should not be taken as an indication of future plans."_

> _“A lot of patents are filed at the idea stage rather than the actuality
> stage,” said Ranieri by phone. “A tech company that files a patent has,
> hopefully, at least thought about how to do it. You’d hope they could
> implement it if asked, but it doesn’t mean they have done so before.”_

So if registering a patent neither requires that I actually intend to make use
of the idea, nor that I even knew how to do it, what keeps me from patenting
moonshots such as "A general framework for DNA-editing based cancer therapies"
or "A method to construct a moon base using a mix of on-site material and
material sourced from earth" and start raking in cash once someone else
figures out the details and actually wants to do it?

~~~
cm2012
Companies do this all the time. Amazon succesfully patented a 1 click buy
button, for god's sake.

~~~
lolc
Though they actually used that one and it worked well. Not to say it wasn't an
absurd patent.

------
vlovich123
What's the news story here? Tech companies file innumerable number of patents
without implementing them to cover the space in case they get sued so they can
retaliate. Realistically this patent wouldn't actually be useful in practice
for the proposed application but would be useful if written carefully enough
to countersue for unrelated applications of the tech.

~~~
vertexFarm
What? You're saying companies file patents on tech they don't intend on using
so they don't get sued by patent trolls for their non-use of some other
preexisting patent? That doesn't make sense on any level.

And if you don't think such a thing would be useful to a company like this (if
it could really be made to work) you aren't using your imagination properly.

~~~
sprayk
> What? You're saying companies file patents on tech they don't intend on
> using so they don't get sued by patent trolls for their non-use of some
> other preexisting patent? That doesn't make sense on any level.

Why does that not make sense? This has been the case at every job I've worked
withe patent filing incentives. You get a few thousand dollars for coming up
with an idea and going through the motions of writing the technical parts of
the patent. The patent doesn't have to have anything to do with the business,
they just want it for defense, and to increase value of a potential sale of
the company.

~~~
vertexFarm
Okay, I get that it adds value to the company. But what do you mean by
defense? How is filing patents that you don't intend to produce so you can win
legal battles any different from patent trolling?

~~~
ALittleLight
The "defense" part is that big company X with a large patent portfolio might
sue you for infringing on a patent - unless you have a large patent portfolio
because then if they sue you, your lawyers can review everything they're doing
and compare it to your patent portfolio and find ways to sue them. This is
something like mutually assured destruction.

It seems preposterous to me, but this is also the basic idea I've got from
working at multiple big companies where they have training meetings to explain
these things. You're very much encouraged to come up with ideas and submit
them for the lawyers to look at and possibly patent even if it has no
applicability to anything you're doing.

~~~
vertexFarm
Ah I see, that makes more sense. So basically the court case looks at the pool
of patents each company has, and one that has a lot of patents related to the
industry it's involved in looks more legit than a potential patent troll
holding a bunch of totally unrelated stuff. Is that more or less correct?

Sorry for contradicting you earlier. That still seems like a crazy way to
manage IP, but after all I haven't got a better solution. Thanks for
explaining. In my career I haven't been expected to produce patents very
often. I'll try to remain a bit more humble.

------
a-dub
I'm not really sure if that's a scary facebook thing or merely just an
inconvenient reality of the world we live in now.

There's quite a few papers out on fingerprinting/identifying cameras from
images taken with them using intrinsics (dust, scratches, slight offset of
image sensor with respect to lens).

~~~
rhizome
_I 'm not really sure if that's a scary facebook thing or merely just an
inconvenient reality of the world we live in now._

These aren't mutually exclusive. The reality of the world may just be
acquiring more scary things, and ones that are created for profit.

~~~
ralusek
Is something being created for profit scarier/worse? I'm personally far more
concerned about government surveillance programs that I have no option but to
participate in.

~~~
AlexandrB
I'm always confused by comments like this. Not only is corporate surveillance
always one NSL away from being used by the government, but corporations aren't
democracies. As a user I can't vote on what data Facebook choses to collect.
Facebook's shadow profiles also put to rest the idea that you get to _choose_
whether or not to participate in corporate surveillance.

~~~
BigJono
Ask me and the other citizens of Australia what democracy has done to save us
from government mass surveiallance.

It may possibly be one of the worst guards against it. 99% of the population
don't understand it and don't give a shit. Co-incidently that's the only
reason Facebook and Google get away with it too, or it'd be far more
profitable for them to back off and play the "we value your privacy" PR card.

Literally the only difference is that Facebook is headed by software engineers
and governments are headed by a bunch of people with tons of power and no
knowledge. You tell me which is more dangerous. You can't act ethically if you
don't even understand the domain you're acting in.

~~~
pferde
> Literally the only difference is that Facebook is headed by software
> engineers and governments are headed by a bunch of people with tons of power
> and no knowledge. You tell me which is more dangerous. You can't act
> ethically if you don't even understand the domain you're acting in.

On the other hand, you can do a lot more damage if you have the knowledge, and
do not care about ethics.

------
aasasd
I'm pretty sure Google and other large networks also build 'shadow profiles'
on users to figure out who they are, who they're connected to, etc. And I
wonder if this shady profiling data should be available for users to download
under e.g. GDPR. Because it sounds to me like it should, but isn't.

~~~
hedora
If any europeans with spare time and no Facebook account are reading this, get
a lawyer (maybe related to the EFF, or not), and send a GPDR request.

When it comes back “no data”, record yourself signing up for a FB account,
since at that point, they’ll list suggested contacts that cover pretty much
everyone you know.

(Even if this doesn’t work as I predict, it’d document what they’re doing for
gpdr compliance.)

~~~
mehrdadn
Why does that require having a profile on you in particular? They could be
saving all the data from everyone else (e.g. their contacts, their search
queries, etc.) and dynamically assembling it when you make an account to make
you recommendations. They don't have to be doing anything related to you in
particular.

------
anonytrary
Jokes on them, I have electrical tape on my camera lens. Something I learned
from Facebook's CEO, ironically.

~~~
analog31
The image that's gathered when the camera is completely in the dark, is a
unique fingerprint of the camera chip. This is something called "fixed pattern
noise."

~~~
Laforet
In that case just put on a few more layers. Shot noise is fairly random.

~~~
analog31
True. But if you share enough images from the same chip, eventually the fixed
pattern noise can be computed to a decent enough accuracy to use it as a
fingerprint. Now, you could measure the fixed pattern noise and subtract it
from subsequent images. This is a widespread practice in scientific imaging.
I'm not devious enough to think of how this could be defeated by someone who
really wants to use your camera chip as a fingerprint, but it might be enough
to defeat an amateur.

------
mehrdadn
> “We’ve often sought patents for technology we never implement, and patents
> should not be taken as an indication of future plans.”

How is this possible? Wasn't it a requirement of a patent grant that you
actually try to put it to use?

------
pdxww
This is smart. Unethical, but smart.

~~~
remarkEon
Indeed.

I'm wishing for a "dumb" social networking company, where the people who run
it are idiots and I don't have to worry about this kind of invasive
engineering (I long ago deleted my Facebook).

There's always twitter, I suppose.

~~~
abootstrapper
Can we not have smart ethical people? When did getting away with unethical or
unlawful behavior become associated with “smart?”

~~~
ravenstine
Seems pretty rare. Most people want the money and prestige that comes from
working for Facebook or Google. It's easy to be a rebel when you've got
nothing to lose.

------
api
I've come to believe that privacy "by design" is impossible in the hyper-
connected age. There are too many vectors for tracking, linking, and de-
anonymization and software in general is too insecure.

The only solution to the privacy problem is legislation. My favorite idea is
HIPAA type regulations for intimate personal data like audio, location, etc.
Leak location data? That will be $10,000 per incident where an incident is one
record per person on a given day.

This would transform data like this into a liability rather than an asset,
pushing companies to store it only long enough to perform a given service and
to develop cryptographically blinded systems whenever possible to cut
exposure.

As it stands all the economic incentives encourage all vendors (even small
indy apps) to maximize privacy invasion at every opportunity.

~~~
superkuh
Nope. Just chose to not use services that suck yourself. It's easy. You can
say you peer group or whatever uses them but it is still your choice.

I strongly oppose your suggestion to bring in the government use of violence
to impose your ideological goals on others. They have to chose for themselves.

~~~
toufiqbarhamov
_Nope. Just chose to not use services that suck yourself. It 's easy._

Ok let’s test your hypothesis.

 _Camera noise_

Right, I’m tagging you in this group photo and uploading it. Where’s your
choice now?

~~~
superkuh
I don't see the problem. It sounds like I'm in a public setting. If not and
this is some friend group gathering I could just ask you not to since we're
friends. My friends and family that use facebook know not to include me. But
even if they do, so what? It's not like a single photo of me uploaded by a
friend impacts my privacy.

~~~
barrkel
_It sounds like I 'm in a public setting_

There's a difference between being a random bystander in an individual's photo
and mass surveillance of the public space through millions of photos. Just
like lots of photos become something else - movies - lots of photos of public
spaces become something else - a surveillance state.

You're sitting on one principle and riding it to the point of absurdity. There
are lots of other principles, like the right to live in a free society.
Sometimes freedom requires lack of freedom - sometimes we have to apprehend
criminals. You can't ride this principle all the way without taking off your
blindfold.

------
josefresco
Imagine being at a bar, and a guy engages you in conversation. You try your
best to indicate you're not interested, and your friends help shoo him away.
Then, that night when we gets home Facebook tells him your name and suggests
you connect. Creepy AF (and not the guy)

~~~
IshKebab
If they are using the dust on your camera apparently you'd have to both take
photos with the same camera and then upload them to your respective accounts.
Then facebook will correlate the dust and learn you are friends.

I don't see that happening... ever. Even if it were possible (I'm pretty sure
dust and scratches on a camera lens aren't in focus...).

~~~
brokenmachine
It could also be the sensor, say if certain pixels were slightly less
sensitive than the surrounding ones. You could run statistical analysis of the
image and spot the similarities.

------
smelendez
Is this actually a common scenario? Facebook users who aren't friends but
upload photos taken with the same camera?

------
edoo
I really don't get these patents. How can you patent ifchecks on data if it is
ascribed to an abstract process.

------
clubm8
Facebook's Achilles's Heel is it doesn't know the _strength_ of your
connections. There's lots of data points, but they currently do a poor job of
parsing them. Unfortunately given their recent privacy struggles it may be
difficult for them to leverage their dataset to answer that question without
driving off users.

------
bredren
New to some but this is regurgitated content as these patents were reported on
long ago.

------
tim333
Clickbait headline. The closest they get to it is

>One filed in 2015 describes a technique that would connect two people through
the camera metadata associated with the photos they uploaded. It might assume
two people knew each other if the images they uploaded looked like they were
titled in the same series of photos—IMG_4605739.jpg and IMG_4605742, for
example—or if lens scratches or dust were detectable in the same spots on the
photos, revealing the photos were taken by the same camera.

ie facebook mentioned the concept of tracking you by the dust on your lens. No
evidence that they can actually do it.

------
jcoffland
> or if lens scratches or dust were detectable in the same spots on the
> photos, revealing the photos were taken by the same camera.

This seems pretty useless. The only scenario where it gives FB more
information is where person A takes a picture, gives it to person B through
some route other than FB and then person B posts it on FB. Any other scenario
and FB does not need to compare dust specks.

~~~
bad_user
But this happens quite often, like every time people get together.

~~~
jcoffland
Your saying people who use FB regularly share photos outside of FB and then
upload someone else's photos to FB? I'm not saying it doesn't happen, just
that it's not the normal path.

I think people who use FB generally share photos with their friends who are
also on FB, through FB.

~~~
bad_user
Not in my experience and I have many friends outside of tech circles, since
I'm not living in a tech bubble. When people get together group photos happen,
or photos that you want, made with somebody else's phone.

The result is basically dozens of photos out of which only the best 2 or 3 get
shared on Facebook.

And in general Facebook is not how those photos get shared. Not even its
Messenger because it has annoying size limits, which matter for videos. If I
were to guess, out of Facebook's properties, WhatsApp is probably the most
popular photo sharing app by volume ;-)

------
the-watchtower
I am going to go out on a limb and say that this crosses the line from
"invasion of privacy" to "just plain evil." Seriously: this is the type of
stuff that you expect to find in state-sponsored malware, not software that's
forced onto the majority of consumers.

~~~
brokenmachine
_> not software that's forced onto the majority of consumers_

I agree with you on your other points, but not "forced".

I've left Facebook and encourage anyone who cares about privacy at all to do
the same.

------
sonnyblarney
Ok, this is definitely bad news upon bad news ... but you have to concede it's
pretty cool. Outside the scope of actually doing this: you wish you had of
thought of it!

------
antaviana
With the gyroscope and accelerometer feature implemented, you will be able to
friend with someone who stole your phone.

------
dompydumpy
So we'll regulate facebook maybe?

------
qsdevacc
Disabling locations services doesn't stop all forms of location tracking.

------
nkkollaw
We leave in scary times.

------
userbinator
_or if lens scratches or dust were detectable in the same spots on the photos,
revealing the photos were taken by the same camera_

With the prevalence of mobile phones, whose camera lenses are likely to be
exposed to a lot more "scrubbing" than a professional/dedicated camera, I
suspect this might not work so well.

~~~
jnnnthnn
Wouldn't it possible for it to work better if it is the case that more dense
scrubbing patterns are complex enough to differentiate one camera from the
other, and that such an algorithm can pick up on that?

~~~
vuln
I agree. It seems to me that everyone's scrubbing would be different thus
pretty unique.

~~~
lucideer
I thought the point was more about variance over time. Surely a camera would
need to have consistent patterns over time for this to work?

------
vertexFarm
Gross. Oh well, another one for the already massive pile of unreasonably
greasy facebook creeping. Massively parallel automated stalking. This shit
needs to be regulated soon. A lot of the people being data-fucked are not
actually people who have signed a EULA or anything, and it's getting more and
more dubious that this is readily-available information "in the public square"
that has no reasonable expectation of privacy. In my opinion at least, as if
that's worth anything.

Also forgive me if I don't 100% take their word for it that none of this shit
has been implemented except a few tests in 2015. These companies are never
forthcoming with the truth around these matters of privacy and security.
_Ever._ Any breach, any shady practice is denied until it's not possible to
deny anymore. Why should they be truthful? It can only bring bad PR and there
has literally never been consequences for lying about it.

