
Facebook confirms that it tracks mouse movements - shahocean
https://www.indiatoday.in/technology/news/story/facebook-confirms-that-it-tracks-how-you-move-mouse-on-the-computer-screen-1258189-2018-06-12
======
code_duck
I have always assumed that Facebook uses heat maps to track what is under your
pointer. Doesn't every serious site do that to gauge user behavior, interest
and interface utilization? I guess the difference is FB is putting it in an
available data set along with everything else about consumers' individual
lives.

More interesting is "a patent held by the company states that the Facebook app
uses voice recognition algorithm, which uses audio recorded by the
microphones, to modify the ranking scores of stories in users News Feed." and
their speculation that Facebook could soon reveal details about their use of
surreptitiously recorded user audio.

Facebook makes a curiously specific denial about audio, which is that it is
not used for advertising. Considering their entire business is basically
advertising, what does that leave? But all they mean is ad selection. When
they were found to be recording audio during the posting of statuses, I
believe they claimed it it was so they could recognize the music you were
listening to, and know something about your mood. So for a long time, I have
thought that they use audio to select other content, like friend suggestions,
or to inform the selection of stories that appear on your newsfeed.

~~~
bilbo0s
Like heat mapping a visual UI, there are a lot of people out there recording
audio.

However, I think that I disagree with you on whether or not sharing the data
is important. If you are heat mapping me, like facebook and probably everyone
else from Microsoft to CNN and FoxNews does, or you are recording me like
everyone from Facebook to Samsung does, I'm sorry, I've got a problem with it.
I don't care if you don't share that data. I don't want Samsung recording
what's going on in my living room. Doesn't matter if the data isn't shared.
It's just the principle of the thing.

It's gotten to the point where I actually purchased a certain model of Sony
TV, because the teardown verified that there is no microphone in it. Then I
tossed the remote control and got a generic remote with no voice control.

People joke about me being paranoid, but I'm not paranoid. Sheez... I'm old
and boring, I know that no one cares about what's going on in my house or on
my computers.

I'm just stubborn.

Why let the privacy invaders win?

~~~
code_duck
The difference is how the data is used and the whether it's associated with an
individual's permanent personal data. If it's only gathered anonymously and
used for internal UI improvement, that isn't objectionable. At the other
extreme, association with a real-world individual enables many uses that are
potentially harmful, such as unmasking those who legitimately prefer to be
anonymous.

Edit: I wanted to add that I didn't intent to focus on whether the data is
shared. I think FB having and using it is bad enough, especially if they're
ubiquitous. Also, once anyone creates such data, other entities such as
governments will seek to obtain it and likely do so eventually.

~~~
gbrown
Anonymous data is one identifier or clever match away from identification.
This is particularly severe for sites/services/hardware that records audio
(man, who would knowingly allow that kind of abuse??), but it can apply to
mouse tracking too. Mouse tracking fingerprints could be used to re-identify
all sorts of other things.

~~~
code_duck
It depends how much data is collected in the first place, and how much is
available to the person trying to break anonymization. If I'm not mistaken,
everything is deanonymizable with global traffic analysis.

------
titzer
I'm not surprised. Honestly, I stopped trusting them years ago, so pretty much
all the words out of their mouths I consider to be BS. I'm not even sure why
people apologize for creepy-ass multi-billion dollar companies that treat
their users like products and study them like labrats.

I'm looking forward to technical solutions to verifiably disable and prevent
this kind of tracking.

To those talking about "tracking UI usage": do a UX study. Sit some people
down and watch how they use the site. Ask them questions about what works and
what doesn't. Stop spying. You got all this damn money and you can't be
bothered to actually lift a finger to do some difficult work that involves
interacting with all those "dirty" people out there. FFS most people would
probably be happy to fill out a survey if it actually would impact the product
in a positive way. Creepiness is creepy.

~~~
brlewis
I am not affiliated with Facebook myself; I barely use it, but given how big
it is I bet they do user studies in addition to metrics collection. Metrics
alone leave out important parts of the human element, but user studies alone
are a biased sample.

~~~
rohit2412
Given the speed and comfort of not doing user studies, I expect "move fast and
break things" Facebook to not care about user studies at all

~~~
jartelt
I know several UX researchers at facebook. Yes, they do in person user
studies.

------
StanAngeloff
I'm not sure if this is actually how reCAPTCHA v2 works, but I've found moving
my mouse and highlighting text immediately after ticking the box almost
certainly passes the tests for human. I very rarely get asked to recognise
images or pick X out of Y. When I don't do this, i.e., I don't move my mouse
or highlight text on a page at random, I most certainly have to sit through a
couple of screens of tests (I'm behind a shared IP with lots of users).

All this leads me to think mouse movements tracking is much more widespread.

~~~
plopilop
I have a similar experience. Filling in fields and switching with the TAB
button then pressing ENTER always brings me to a visual recognition test,
while manually clicking on the fields (and adding a bit of sloppiness) is most
of the time an immediate validation from reCAPTCHA2.

However I'm pretty sure this was advertised or at least acknowledged by Google
in the launching of reCAPTCHA v2.

------
akerro
According to lcamtuf[1] how people move mouse and use keyboard are unique per
person, it's your personal digital-fingerprint that can be transferred to
track you elsewhere.

[1] From Book "Silence on the Wire"

~~~
dvfjsdhgfv
Unfortunately, it's not just JS, but several other indicators. In practice, if
you don't use the Tor Browser, it's as if you decided to leave tracks
everywhere, almost identifying yourself. Yet, this is not common knowledge
among web users. These are the things that children need to be taught in
schools. The society as a whole needs to be aware and learn how to protect
itself.

~~~
yoz-y
That is victim blaming, just because you do not know that everybody out there
is tracking you does not mean that you have decided to let them.

I agree that this is something people should be more aware of and school is a
good place to start. However it is up to browser manufacturers to fix this,
not users.

~~~
code_duck
The comment says "it's as if" you are intentionally leaving traces everywhere.
So, the effect is the same as doing it intentionally, not that users are to
blame somehow.

------
gkya
I have a network meter on my xfce taskbar, and it shows uploads and downloads
whenever I move my cursor on a website I view with a JS enabled browser. It's
almost all of them.

~~~
zimmund
Correlation doesn't imply causation.

(You can always open the browser inspector and check network traffic for each
page or, if you are using Chrome, dive into chrome://net-internals/ )

~~~
gkya
Yes, certainly. But I don't have background programs that connect randomly,
and I have observed this enough times that I can say this.

~~~
seba_dos1
As somebody who worked in the past on a piece of software that generated
heatmaps from cursor movements on websites, I can confirm that it's a very
widespread thing. Well, it was ~5 years ago, so I'd guess it's even worse now.

~~~
gkya
How did you use this data? Well, except for bot detection that is. I can't
think of a particularly useful way this data can be utilised.

~~~
burger_moon
For a non-nefarious use case, it can be used to iterate on the UI to create a
better user experience because it can expose areas that people aren't seeing
on the webpage. Your site might have the important content or useful
navigation in a place that users aren't noticing which causes them to leave
the site in frustration.

~~~
seba_dos1
Yeah, basically this. The data was aggregated and a graphical heatmap was
displayed on top of the website, with some fancy accomodation for responsive
designs. You could see heatmaps for hovers and for clicks. Customers then
optimized their shop flow, adjusted graphics that looked like they were
clickable but weren't, moved important content into more visible places, etc.

------
amelius
> Facebook said that it tracks mouse movements to help its algorithm
> distinguish between humans and bots.

Stupid cat and mouse game. How difficult would it be for a bot to simulate a
human's mouse movements? I suppose not very difficult.

Also, doesn't this conflict with rare types of input devices? Or people with a
motor function disability?

> to also determine if the window is foregrounded or backgrounded

Shouldn't there be an API for that?

~~~
TekMol

        How difficult would it be for a bot
        to simulate a human's mouse movements?
    

Very very hard. Because the bot author does not have the giant database that
FB has to analyze how humans move the mouse around. Also the bot author does
not know which aspects FB looks at to determine if it's a human.

And even if the bot author had all that information, it would still be super
hard to write an AI that accomplishes a given task in a way that mimicks a
human successfully. It would mean to win a 'mouse turing test'.

    
    
        Shouldn't there be an API for that?
    

What the API returns is under the control of the user. So the API does not
help FB to fingerprint you.

This issue touches on the real privacy problem the net is facing. It's not the
wrong cookies or privacy policies. It's fingerprinting. There is no technical
solution to it.

~~~
amelius
> Very very hard. Because the bot author does not have the giant database that
> FB has to analyze how humans move the mouse around.

Don't forget that Facebook's false positive rate should be very low. There are
lots of humans on their platform, and they should all pass the test.

This makes it easier to construct a bot that will pass the test.

~~~
Klathmon
It won't hurt if humans fail the test every so often, as long as it's under a
threshold that humans regularly can overcome.

I can imagine it would be easy to trick the system a few times (either as a
bot pretending to be human, or a human acting like a bot), but tricking it
consistently over months or years is going to be damn near impossible.

~~~
amelius
Also don't forget that Facebook probably has to do all detection in Javascript
on the client, i.e. with limited resources. I suspect they don't send every
mouse-movement to the server. This also means they probably don't have fine-
grained historical data.

~~~
Klathmon
Not necessarily.

I've only given it a few minutes thought, but position and time data is really
small, and easy to compress (you don't need to send anything while the user
isn't moving the mouse). If it's sent in batches or over an already open
websocket, it's not like it's using a ton of resources on the client.

Assuming all of their users (guessing a billion daily active users) are on
desktop half of the time (a wildly incorrect assumption I'm sure), and the
mouse position data is 1mb per person for the data you care about (which
again, seems like a lot), that's 500tb.

For $25k you could store it all. That's nothing compared to the benefits of
being able to identify bots on your platform.

~~~
taurine
Yes, the standard way to do this a few years back for conversion optimization,
was to RLE compress and send the data in intervals. Also the
resolution/measurement does not need to be in the milliseconds.

------
GarrisonPrime
Ironically, the more interested I am in part of a page the less likely my
mouse is going to be over or even anywhere near it. I find the pointer
distracting.

If I'm interested in the content of a page, I either swipe the mouse off to
the edge of the screen or put it on an area of whitespace so I can scroll with
the scroll wheel.

The thought that people are hovering their pointers over stuff they're
actively looking at strikes me as odd. Oh well.

------
sunseb
This is beyond absurd. Why would you do that? What could go wrong? Is this a
social network or spy agency?

~~~
andrewmackrodt
Lots of websites you would not expect do this; it's not for spying on users
but to improve the user experience.

Being able to aggregate data or inspect individual sessions is a useful tool
to learn how users navigate with a site.

Keyboard keystrokes get captured too but the systems are intelligent enough to
filter out passwords and payment details.

I don't really like this form of monitoring either but I've seen it in several
companies.

~~~
orwin
> Keyboard keystrokes get captured too but the systems are intelligent enough
> to filter out passwords and payment details.

Citation needed. But i did not realize that before, so thank you very much for
this information, i will desactivate js on every page with a password field
from now on.

~~~
r721
Relevant:

"Following the recent report that Mixpanel, a popular analytics provider, had
been inadvertently collecting passwords that users typed into websites, we
took a deeper look. While Mixpanel characterized it as a “bug, plain and
simple” — one that it had fixed — we found that:

\- Mixpanel continues to grab passwords on some sites, even with the patched
version of its code.

\- The problem is not limited to Mixpanel; also affected are session replay
scripts, which we revealed earlier to be scooping up various other types of
sensitive information.

\- There is no foolproof way for these third party scripts to prevent password
collection, given their intended functionality. In some cases, password
collection happens due to extremely subtle interactions between code from
different entities."

[https://freedom-to-tinker.com/2018/02/26/no-boundaries-
for-c...](https://freedom-to-tinker.com/2018/02/26/no-boundaries-for-
credentials-password-leaks-to-mixpanel-and-session-replay-companies/)

------
vladharbuz
This should be no surprise — received chat messages are marked as read on
Facebook if you wiggle your mouse after a few minutes of inactivity.

To be fair, this makes sense from a UX standpoint. You don't want messages to
be marked as read if you have your window open but have walked away from the
computer.

~~~
proto-n
This one is quite a big quality of life improvement IMO. It used to be that I
sometimes left my browser accidentally open on facebook, and people would get
offended because I 'read' their messages and didn't respond.

~~~
jhowell
Maybe telling your friends that you didn't actually see their message and that
instead the algorithm FB uses to record my mouse activity sometimes reports
that I have read messages that I have not. This sounds dangerous and could be
harmful to a person's credibility if another with bad intentions acts.

------
mlthoughts2018
Earnest question from someone without much front-end knowledge: how is
browser-based mouse tracking performed in a way that doesn't significantly
degrade performance?

Do they use some type of client-side library that caches data for a while and
asynchronously uploads it occasionally? Or occasionally try to asynchronously
sample the mouse position and just get a coarser set of data?

It seems like real-time requests that respond to mouse changes would create
huge performance problems and/or be easily stopped with browser extensions.

~~~
noiv
Capturing, collecting and forwarding mouse movement events can be done at
almost no cost. 10 locs max.

------
ckastner
> We collect information from and about the computers, phones, _connected TVs_
> [emphasis mine]

Personally, I find this even more disturbing. Does this "only" apply to TVs
where a user is logged on, or are they also building shadow profiles for any
smart TV that comes with the Facebook App preloaded and is connected to the
internet?

At least for Samsung and Sony, I can easily see them cooperating with Facebook
for a negligible fee.

------
benevol
Lol @ the Facebook army astro-turfing the comment section.

"But it's not nefarious..."

"But everybody else is doing it too..."

"It's not for surveillance, profiling and shadow profiling, pinky promise,
trust us..."

~~~
foobarchu
"Everyone else is doing it" is totally a valid thing to point out here. It
means that, while you can get up in arms about it, you need to get up in arms
about the practice industry-wide, using the facebook name to make it sound
worse than it is makes this either dishonest or ill-informed.

It's a lot easier to get one company (or person) to stop a practice only they
do than it is to get them to stop doing something that everyone else does too.

~~~
monkeynotes
Not sure I agree, there is a difference between a private company that can
collect covert private data on over 2/3 of the US population, and one that has
access to a handful of users.

Facebook has ties to and influence over government, it is so big and far
reaching that I feel it's right to be more concerned about FB doing something
like this than other smaller players.

The scale of FB is what makes it a special case.

However, I think that the proper approach to something like this is educating
users. Companies are gonna capture your mouse movements, it's not something we
should legislate over, but users should be informed as to what it means to
give companies like Facebook information about yourself.

Worrying about mouse movements when you freely send clear text messages to
their data pile about your most intimate feelings and thoughts is ass-
backwards.

------
ameister14
""We collect information from and about the computers, phones, connected TVs
and other web-connected devices users use that integrate with our Products,
and we combine this information across different devices users use," Facebook
wrote in the document adding that the collected information is used to "give
better personalize the content (including ads), to measure whether they took
an action in response to an ad we showed them on their phone"."

In other news, Google Tag Manager and crazyegg exist.

------
Rjevski
Is there even anything left that Facebook _doesn 't_ track?

~~~
ilikehurdles
They claim they don’t record and track what you say. My spouse claims
otherwise, saying she’s gotten hyper-specific advertising for medications that
she’s discussed with her patients with the phone in the room.

~~~
doubleunplussed
Facebook didn't claim they don't record what you say.

They said very specifically they don't record what you say for _advertising_.
It was specific enough that it almost seems an admission that they use it for
something else.

However, there are plausible explanations for your spouse getting ads for
things she's spoken with her phone in the room - for example, the patient
googled the medication later or wrote about it on facebook, and facebook knows
thay your spouse and the patient likely had a conversation based on location
data showing that they walked down a hallway together.

~~~
Karunamon
I still call maximum shenanigans on that whole concept. You'd think there'd be
one hacker out there with a packet dump or disassembly of the application
(yknow.. concrete evidence) rather than lame anecdotes that reek of
confirmation bias.

Furthermore, I don't believe that Google and Apple (well.. less Google, more
Apple) are in cahoots with Facebook to give them a backdoor to device
permissions.

~~~
dirkgently
> (well.. less Google, more Apple)

That's funny, because very recently, there was a discovery that fb gave most
of the device makers deep access to user data, including Apple.

Against all evidences, Apple still gets a pass.

~~~
Rjevski
Unless I’m mistaken the “deep access” is just bad reporting (the media are
surfing on the wave of Facebook bashing).

What they mean by deep access is that the Facebook sharing system extension
built into some OSes (including iOS pre iOS 9) had the possibility of
accessing a lot of information from the connected Facebook account, which is
not really anything to worry about (if your device’s OS is malicious you have
way more to worry about).

------
mattbessey
as does every person here using Mouseflow, LogRocket, or a dozen other off the
shelf solutions. is this really that surprising?

~~~
sti
Seriously my thought.. Pretty much every company does that. Not especially for
detecting if its a Human or AI, but for understanding if the User has problems
with the Usability of the Product..

~~~
princekolt
Yes, of course, I'm sure Facebook only uses this data for usability studies.
I'm also sure they could pretext all other tracking they do to some other
marginally positive use case.

------
zer0faith
Where I work we track: mouse movements, how long you spend on a specific page,
the path that you took to get to a specific page, what you clicked, ip
addresses, and basically all information included in the header of the
request. We also screen shot everything from when you login to when you
logout.

------
methodover
Um. This isn’t nefarious.

We’ve used inspectlet from time to time to help figure out where there are
problems in our UI. It tracks mouse movements as part of a complete session.
It’s been really helpful.

~~~
p49k
You should still only do this when the user is aware you’re tracking these
behaviors and you have their consent. That’s the problem here.

~~~
reustle
I'm not so worried about this, as long as it's only on their site and when I'm
logged in. They're using it to test features and engagement, and can't really
learn much about me as a person with it.

------
cbcoutinho
It isn't just Facebook that has been doing this. Web development tools are
able to track scroll and mouse movements as well, and have been used to test
website usage

~~~
angott
There is a big difference between using this kind of technology to run
usability studies or to provide more targeted advertising.

~~~
kaybe
Not really. If you record and save that data, the reason is secondary.

~~~
jhowell
Tracking an individuals mouse movement across devices for perpetuity,
packaging said data as a product offering to potential advertisers, seems to
be a more extensive undertaking than simply UI/UX improvements.

------
hmate9
Every website that uses google analytics does this.

------
sofaofthedamned
I assume from this they can infer if you're left or right handed and store
that in your profile.

------
fori1to10
The really sad thing is that all this data is not available for academic
research. Instead of understanding human nature to make a better world, it is
being used to make better targeted ads.

------
err4nt
Does Facebook just grab everything they can? Does anybody know if they grab
your device's battery status, or gyroscopic data about how you are
tilting/holding your device?

~~~
taormina
I mean, likely. Given the number of devices they are on, someone has to be
using the Battery Historian or similar tools to optimize for power usage.

------
learnstats2
Presumably Facebook can do this because Javascript gives up this information.

What's the easiest way to hack your browser to give no information or dummy
information here?

------
throwaway122378
At this point Facebook can come out and say they track any and everything.
They might as well because no one will do a single thing.

------
gambiting
I remember just few years ago there was an article here about a website
tracking mouse movement to predict what the user will click and pre-loading
the content in the background. It was seen as a huge improvement to the
experience and the comments section was full of people saying how cool it is.
But hey, now facebook is doing it, so it's suddenly the worst thing in the
world?

~~~
pavel_lishin
There's a difference between pre-loading content to improve user experience,
and tracking mouse movements to gather information about users that's then
used to show them advertisements, or potentially track them across the web[0].

There's also the difference between announcing "hey, look at this cool tech we
have to make the web faster!" vs "we are legally required to admit that we've
been watching you like a hawk, for... reasons."

[0]
[https://news.ycombinator.com/item?id=17301769](https://news.ycombinator.com/item?id=17301769)

~~~
anthonybullard
I can't speak to Facebook's intentions or policys(I don't use big blue app at
all), but many sites use tools like Full story to discover bugs or watch a
customers user journey to discover flows that aren't working well. Some
portions of the page are automatically filtered(inputs with type=password),
and the rest depends on the team being very thoughtful about marking sensitive
portions of the screen as such.

It's a very manual process, but probably one of the most powerful tools for
improving user experience I've ever seen. And typically for most businesses,
you are keeping these sessions for 2 weeks or thirty days at most.

~~~
pavel_lishin
I absolutely agree - I used something similar at my previous job.

But neither of us know Facebook's policy (although we can easily guess their
intentions, based on past behavior), and nobody ought to be expected to cut
them any slack.

------
lend000
Is there a more legitimate source for this announcement? Seems like HN has
fallen prey to fake news.

~~~
gvb
[https://www.judiciary.senate.gov/imo/media/doc/Zuckerberg%20...](https://www.judiciary.senate.gov/imo/media/doc/Zuckerberg%20Responses%20to%20Judiciary%20Committee%20QFRs.pdf)

Quick search indicates the tracking information starts on page 84.

Page 86: "Device operations: information about operations and behaviors
performed on the device, such as whether a window is foregrounded or
backgrounded, or mouse movements (which can help distinguish humans from
bots)."

------
codedokode
Using mouse movements to detect that tab is inactive is an old trick. Compared
to other facts, I think that tracking mouse movements is absolutely not a
problem. Collecting things like IMEI or IP addresses needs more attention. For
example, I think it is not necessary to keep IP addresses longer than week or
two if you respect your users' privacy.

------
goldhand
Every company and their mother is tracking mouse movements. How is this
trending???

------
diehunde
This is hardly interesting. A lot of websites have been doing this for a
while. Just open your network console while you are browsing and you'll see
all the things they are tracking

------
zerostar07
"Facebook confirms it receives your IP address!!!"

We need a new science of hysterias. The internet provides ample data for
robust analysis and prediction. Any takers?

~~~
badwolf
Seriously. The hysteria is getting a bit stale that this is what people are
now being outraged about? We've been using CrazyEgg, or any other number of
different mouse and scroll tracking tools for ages.

"But Facebook is using it to signal advertising!"

Well. Everything you do on Facebook is used to signal advertising. I thought
we were kind of all aware that was what happens with this free service. /shrug

------
erickj

      document.body.addEventListener('mouseenter', e => {
        console.log('this just in, erickj tracks mouse movements');
      })

------
cartercole
its just for security like he said to congress right?

------
MRonaldhino77
thats great step towards security and privacy

------
theweb1
Mark is tracking lots of user data

------
throwawayxxx12
Does this allow detecting drug addicts and recommending treatment?

~~~
zerostar07
"We think you may have Parkinson's, would you like to visit the doctor?"

~~~
Applejinx
"We think you may have Parkinson's, your driver's license has been suspended
until you can confirm your fitness to drive. Also you're no longer registered
to vote"

------
IshKebab
This seems like a reasonable thing to do to detect bots.

------
g105b
BREAKING NEWS: A free service on the web monitors how you use it.

