
Graphing when your Facebook friends are awake - adamch
https://defaultnamehere.tumblr.com/post/139351766005/graphing-when-your-facebook-friends-are-awake
======
ceocoder
My favorite part -

> This friend recommended nvd3.js, presumably because you’re not making real
> graphs in 2016 unless your graphing library is <something>.js and requires
> at LEAST one other <something else>.js as a dependency. Everyone looks at
> you like “what, you DON’T already use <something else>.js? Jeez say goodbye
> to your Hacker News karma. Just apt-get install npm && npm install bower &&
> bower install-” NO STOP IT THIS ISN’T WHAT TIM BERNERS-LEE WANTED”.

edit: as huckyaus mentioned in a different thread, author did
[http://swagify.net/](http://swagify.net/) as well. In completely unrelated
news, I'm changing my handle to [Tr1Ck$h0t][LEGIT][60x7]$$$C30C0DER$$$, that
will make me really popular among the cool kids.

~~~
tajen
Tho only drawback I see with nvd3.js is that we can't generate png images of
the graphs, server-side, for email of PDF export. Is there an nvd3-js-to-
png.java ;)?

~~~
seba_dos1
It's called PhantomJS :P

~~~
test1235
What? You don't already have it installed?

~~~
samstave
How can I tell?

------
542458
Some people might take issue with it, but the writing for this had me in
stitches. I very much agree with the author on graphing libraries - there are
a few good simple ones, but as soon as you want anything unusual you have to
jump to these big, hard to configure monstrosities. More than once I've just
given up and written my own server-side generator.

~~~
swampthinker
What's striking about this writing style is that (as a junior dev) it's a lot
easier to understand what he's doing.

By sticking to jokes and a simple style, I never felt confused or lost in
lingo.

~~~
baby
I was thinking at the same thing today reading some tutorial. You have these
serious ass tutorials that are an horror to read. And then you have these
gems, these very informal tutorials with jokes and slangs and ... and reading
them is a pleasure.

I guess if you're a serious company like Apple, you can't really have that
kind of tutorial for iOS dev. But that would really help.

~~~
davnicwil
> I guess if you're a serious company like Apple, you can't really have that
> kind of tutorial for iOS dev.

The fact that this is true is, to me, a slightly sad reflection on our
culture. I understand that too much silliness can get in the way of conveying
information, but as you point out, so can too much seriousness.

It's obviously a fine balance, and unfortunately it seems safer to err on the
side of seriousness when in doubt. It's one of those things that if you sit
down and ask any average individual, they will probably prefer a bit of
silliness and an informal style in any situation where it's appropriate, yet
somehow collectively we seem to have agreed to default to the opposite, and
stick with things being quite dry and serious as a norm.

Interesting, but ultimately a little sad. Why is this the case? Why is our
culture this way? Why can't things just be a little more fun by default?

~~~
a3n
I put jokes in my resume. You have to have a bit of restraint, but it can be
fun and effective. An oatmeal cookie wouldn't be very good if the whole thing
was raisins.

~~~
50CNT
But a chocolate chip cookie would be decent if the entire thing is chocolate.

~~~
johnchristopher
Except if you want a cookie.

(aaaaand this could go on for hours)

------
BinaryIdiot
> If you reload the page you’ll see approximately fifty-bajillion network
> requests go off as Facebook desperately tries to load all the junk that it
> needs to display facebook.com.

I like this part. As a developer I've often looked at the network usage of
large websites / web applications and it's always surprising to me just
how...unoptimized it is as far as network connections go.

I mean Facebook loads decently enough and all I'm just surprised the first
load isn't condensed into a small, handful of network calls to save on
latency.

~~~
onewaystreet
> I like this part. As a developer I've often looked at the network usage of
> large websites / web applications and it's always surprising to me just
> how...unoptimized it is as far as network connections go.

Have you considered that your opinion on the best way to serve data to a
billion users might actually be wrong? That maybe Facebook actually knows what
they are doing?

~~~
BinaryIdiot
> Have you considered that your opinion on the best way to serve data to a
> billion users might actually be wrong? That maybe Facebook actually knows
> what they are doing?

Certainly! But I've also done work where Humvees in the middle of Iraq use
satellite internet connectivity with massive latency and the more connections
it has to open to get started the slower everything gets.

Perhaps this is an advantage where latency is low so you can load parts of the
page faster but I'd love to know some of the more technical reasons behind
this!

~~~
thewarrior
Lets get rid of the notion that Facebook doesn't know what he's doing. He
knows exactly what he's doing.

~~~
adrianpike
"he"?

~~~
saryant
It's a not-very-funny attempt at a Rubio joke.

------
spydum
I like to do this sort of web spelunking all the time.. But the writing and
humor really make this more enjoyable than it should be! Of course Facebook
leaks info to you about your friends - that is the sole attraction for people
to use it! Seems like you could turn this thing into a browser extension as
well if you felt daring.. Like some sort of FB snooper.

~~~
giancarlostoro
Yes the author seems to have a bit of fun with what he does.

------
jonesb6
"If you I dunno, didn’t have a lot of friends in high school, you might
recognise that as a UNIX time stamp - the time in seconds since midnight,
January 1, 1970. "

Great article. And a further reminder why Facebook kinda sucks.

~~~
givehimagun
Why does unix timestamp imply that Facebook sucks?

~~~
MichaelApproved
Not sure if you're seriously asking but OP was likely saying it's a great
article because of the witty writing. The remark about FB sucking was related
to the article's substance.

------
jacalata
As someone who is not personally humiliated by my interest in
computers/tech/programming, I wasn't really entertained by the constant "oh
yea lol it's cause I'm a waste of oxygen that I know that, don't you hate me
as much as I hate myself?" Maybe I know too many nerds with actual self esteem
issues to find it funny.

~~~
Symbiote
This is standard British (also Australian, NZ) humour. Make fun of yourself,
if it's slightly hurtful so much the better.

Example, which I'd not seem before:
[https://youtu.be/taTSxDVEHRM](https://youtu.be/taTSxDVEHRM)

~~~
jacalata
hilariously, I'm British/Australian, so perhaps I've been overexposed to it.

------
christiangenco
This style of writing is so entertaining; it's like a funnier stream of
conscious of what goes on in my head when hacking things like this together.
If OP is the author, please write more.

------
a_bonobo
>Similarly, I’m not sure why there are these weird spikes every three minutes
(+- ~1minute) sometimes.

Could these just be keep-alive requests? For example, the mobile app checks
whether it's still connected?

~~~
defaultnamehere
Good point!

I didn't think of that.

3 minutes does seem like the kind of time an engineer would code into their
app.

So does that mean that the spiky periods are times when people are online the
whole time?

~~~
a_bonobo
>So does that mean that the spiky periods are times when people are online the
whole time?

I'd be inclined to think so, but I don't work for Facebook, there may be an
entirely different reason :)

Edit: or maybe, maaaybe, it's the point of time when people fall asleep over
their mobile - they've stopped interacting with the app, it still sends a few
keep-alive requests, and then logs itself out.

------
WilliamSt
You should try to add some sort of tracking of when people start to write a
message to you. If that's in any way possible. It would be really stalkery if
you knew whenever someone started writing a message and perhaps decided not to
send the message.

~~~
emilburzo
pidgin[1] has a plugin ("psychic mode") that I always enable.

It opens up a chat window when somebody start typing.

It always freaks out the few people that never send the message.

[1] [http://pidgin.im/](http://pidgin.im/)

~~~
WilliamSt
awesome, thanks!

------
gengkev
I don't have a Facebook account, but is there really no way to not share your
available status to your friends? In Gmail you can simply sign out of
Hangouts.

On a side note,

> If you’re wondering why the response starts with “for (;;);”, it’s to, among
> other things, encourage developers to use a quality JSON decoder, instead of
> like, y’know, eval().

This is wrong, as I commented on the linked StackOverflow post, perhaps a bit
too strongly. But it's really frustrating to see that people have
misconceptions because of incorrect answers on StackOverflow.

~~~
Fogest
You can just turn chat off:
[https://www.facebook.com/help/www/215888465102253](https://www.facebook.com/help/www/215888465102253)

Any messages you get when your chat is off just go to your Facebook inbox so
you can still get messages even with it off you just kinda lose that online
live kinda element. I mean this works just like most chat applications work.
They basically all say you're online or not too.

------
buremba
It would be real creepy if someone does the same thing for Whatsapp, you can
even predict who's talking to each other much better than Facebook. It's a bit
harder to collect data from web.whatsapp.com because it's using Websockets but
let me know if someone develop such tool and publish it on Github. :)

~~~
k-mcgrady
I'm interested in why you think you could do better predictions using
WhatsApp?

~~~
mcintyre1994
There's a subset of my Facebook friends, mostly older/family who are slow to
reply to messages on Facebook and only use Facebook in 'down time' or whatever
but are available by text all day. Since they got smartphones years ago, by
text has meant by Whatsapp.

~~~
k-mcgrady
The way I was looking at it was people will get messages on WhatsApp but won't
reply immediately. Same on Facebook. But people will also visit Facebook.com
when ignoring messages so you still get online status info even if they aren't
specifically using the chat. If you could look at WhatsApp and Facebook and
combine the data you'd probably get a really accurate overview.

~~~
buremba
Yep, I thought the same thing.

------
theon144
I know I'm in the minority here, but I just couldn't bear the writing style.
I'm sure the content is interesting, but this article tries way too hard for
my tastes. I had to give up after the first couple of paragraphs.

------
awjr
This is very well written, intelligent, and very entertaining. It's almost
like he channelled Deadpool. Kudos.

Oh and did not know about the Copy as cURL feature on Chrome!

------
anaphor
I did the same thing with the XMPP interface before they scrapped it and it
was obviously much easier...also I used the built in graphing that's in Racket
to visualize it. Also I made a thing to do desktop notifications whenever
someone came online, which is actually kinda useful.

------
bijection
Antimatter15 has a pretty cool clock style visualization of this from 2012 [1]

[1] [https://antimatter15.com/project/facebook-
clock/](https://antimatter15.com/project/facebook-clock/)

------
drdiablo
Nice work! I really like the idea that the web allows anyone to
programmatically dig into the UI and extract data to do things. A friend and I
actually made a whole API to interact with FB chat. You should check it out:
[https://github.com/Schmavery/facebook-chat-
api](https://github.com/Schmavery/facebook-chat-api). I'd really love to see
what you can come up with, with some of the stuff we support.

~~~
auscompgeek
I personally use BitlBee with bitlbee-facebook [1], and hook up my existing
IRC bot to it. It works for my needs.

[1]: [https://github.com/jgeboski/bitlbee-
facebook](https://github.com/jgeboski/bitlbee-facebook)

------
glossyscr
_" Graphing how addicted your friends are to Facebook and Facebook Messenger"_

------
dclowd9901
> when it’s midnight and your x-axis formatting function doesn’t convert UNIX
> times into JavaScript date objects properly because there’s no timezone
> information and I dunno JavaScript was written by some guy in two weeks
> (yeah I ain’t afraid to call it out what of it) and your binary-search based
> conversion of sparse timeseries data into uniformly dense timeseries data is
> causing so many data points to be graphed that it’s slowly crashing Chrome
> and you’re watching helplessly as your RAM goes up and Chrome won’t close
> the tab and it just doesn’t seem right that 2016, the year of the Linux
> Desktop has brought us this situation I mean I thought if you had enough
> <something>.js libraries this stuff was meant to just scale right up so tha-

So, did you forget everything you learned about memory management? Or do you
think Javascript really doesn't have sound memory management principles? Hell,
it's not like you need to retain references to rendered points. Just dequeue
them. Browser graphing libraries render to canvas which is just pixels.

------
pranaysharma
Great work and nice style of writing...felt like nerd Deadpool ripping away FB
:P

~~~
swampthinker
That's actually a great way of putting it.

------
karlding
You used to be able to do this with the Facebook Query Language (FQL) that
Facebook exposed, sending something like this query to the FQL endpoint.

    
    
       SELECT uid, name, online_presence
       FROM user
       WHERE online_presence IN ('active', 'idle')
       AND uid IN (
         SELECT uid2 FROM friend WHERE uid1 = me()
       )
    

Unfortunately, the current version of the Facebook Graph API[0] doesn't have
the online_presence field, so this is no longer possible. Maybe the Graph API
will be updated in the future to also return the online_presence fields?

[0] [https://developers.facebook.com/docs/graph-
api/reference/v2....](https://developers.facebook.com/docs/graph-
api/reference/v2.5/user)

------
gohrt
What determines whether the app is online? What happens when the user is using
the phone but FB is in the background? Does FB get some kind of update when
the user is active on device? Or do OP's friend live in the FB app all day
long?

~~~
defaultnamehere
Hi, I wrote this blog post.

I have no idea what the Facebook apps (Messenger and the Facebook app) do in
the background.

It's easy to test that, but I didn't because #yoloswag

~~~
frogpelt
You're a funny person.

------
mattlutze
I followed his GitHub link at the bottom of the post and see that he's
Australian. My preconceived stereotypes of Australians suddenly explains quite
a bit of his writing style and humor, and makes the post that much more
enjoyable.

------
Wingman4l7
Reminds me of the old-school user tracker _(whose name escapes me)_ that would
give you a bar graph of your friend's online/offline presence when AOL Instant
Messenger was the dominant chat client.

------
Matiss
This is awesome! Thank you for sharing the code for this. Overall I would say
that this could be very entertaining to watch over multiple sites. Potentially
gathering a good profile of your friends over time!

------
enig_matic7
So, I can mine when my friends are online.

Perhaps, buy some targetted ads about 'SleepCycle' and show them to the
naughty ones who sleep less than 6 hours. :P

------
dyscrete
Awesome and hilarious article. Id just like to note `for (;;);` is not to
prevent users from using bad JSON parsers like `eval` but prevent older
browsers with little to no cross domain policy from loading it with a script
tag and doing evil XSS by overriding Array or Object constructors or
prototypes to pull that data

~~~
nothrabannosir
Good point. Minor nitpick: I believe this is called CSRF, not XSS? But Im just
repeating stackoverflow comments at this point.

~~~
ascorbic
No, CSRF (cross-site request forgery) is where a page tricks your browser into
making requests to another domain in which you're already authenticated, in
order to perform some kind of action. e.g. an img or script with a src
"[http://example.com/message.php?message=you+are+hax0red&amp;s...](http://example.com/message.php?message=you+are+hax0red&amp;submit=1").
You can sometimes perform similar tricks with self-submitting hidden forms, or
XHR. Quite easy to mitigate using nonces and referrer checking.

~~~
nothrabannosir
But.. that's exactly what while(1); and friends in json responses protect you
against? someone overriding the Array constructor function and including your
JSON resource from a <script src=…> ? So this is, in fact, CSRF?

~~~
ascorbic
No, CSRF isn't about pulling scripts etc from another site, it's tricking the
browser into making malicious HTTP requests. So, it's not trying to grab
facebook .com/someinterestingdata.json, it's trying to trick your browser into
performing actions on the target domain by making it perform GET or POST
requests such as sending Facebook spam. It doesn't matter what the response
is, it's just interested in the action. A while(1) won't do much if it's
inside an img tag or hidden iframe rather than a script tag.

------
atrudeau
Cool hack and awesome, fun writing, but on a more serious note : how is
knowing your friends' usage/sleep patterns useful in any way? Could it be used
for some dark, machiavélique purpose?

And how about for advertisers? "Get your sleeping pills here" type ads?

------
soofy
Another neat FB analysis:

Data Science of the Facebook World

[http://blog.stephenwolfram.com/2013/04/data-science-of-
the-f...](http://blog.stephenwolfram.com/2013/04/data-science-of-the-facebook-
world)

------
Buetol
Small tip: If you don't want to be tracked, you can also turn off the chat.

~~~
gry
Turning off chat doesn't necessarily suggest that is true. While
`sticky_pool=atn2c06_chat-proxy&state=active` suggests the case, it's
something to explore and see if there is a larger issue at hand.

------
mariust
This is crazy, I had this idea like 1 month ago and I thought I will find some
free time to make this happen by the end of this month, I guess I should thank
you :) Nice story by the way :)

~~~
zuck9
I had the idea a month ago too. I even started working on this yesterday but
got distracted. Thought I would do it today, but woke up to see this. This is
even crazier.

------
lewapkon
Are you aware that there is actually no requirements.txt in your repo?

~~~
lewapkon
I see that it is fixed now ;)

------
dmichulke
Thank god HN doesn't have the same _feature_ (ehh, I suppose at least). But
how about linkedin and reddit?

I also bet that Whatsapp has this feature since I often see "Last seen at
...".

------
zimpenfish
You can use the MQTT Facebook plugin for Bitlbee and get similar
online/offline/active information right there in your IRC client without all
the screen-scraping faff.

------
davidwparker
Nice little investigating!

Personally, I have chat off all the time on FB, and I don't have the Messenger
(or FB) app on my phone either, so I guess I'm always sleeping :)

------
jordan801
Love it. I knew where you were going right off the bat, but your writing made
me read through it. Now I feel like a secret agent. I'm not sure why.

------
beatpanda
seriously though how are you not already using D3

~~~
baby
This is getting downvoted, but I really thought D3.js was the most popular
library for doing that kind of stuff? I'm a bit outdated I still use gnuplot
:/

------
obelisk_
inb4 facebook resolves this issue by banning anyone who's connected 24/7\.
(that wouldn't solve the problem either way, btw -- a small group of people
could conspire to pull this data at irregular intervals and then share the
data with one-another to get a more complete picture while still staying
reasonably undetectable if done right.)

~~~
defaultnamehere
mfw banned

~~~
auscompgeek
so this is why I always get disconnected from Messenger!

------
teen
You're really funny! Great post. Highcharts.js is the easiest js library to
make quick charts btw.

------
aham
This is brilliantly creepy, and so well written as to be both engaging and
informative. Thanks!

------
merb
Not sure if this blog post is great because of the thing he is doing or the
way he writes it.

------
vmateixeira
Very good post! And very well written, very humorous. Thank you for your
teachings as well!

------
Globz
Very good and fun to read article.

Thanks for all this amazing info!

peace - [2edgy4u][ev REE DAI][24x7BLAZEIT]|ggg10Bzzz|

------
pascalmahe
Loved the article. Great writing style!

Especially loved all the links to Facebook :D

------
AznHisoka
Does this work for anyone, even if you're not their friend?

~~~
defaultnamehere
Hi, I wrote this blog post.

Nope, you have to be their friend.

Well, you have to be their Facebook friend. What you do in real life is none
of my business.

------
meapix
I don't have facebook but my friends are on facebook.

------
yojoma
This was hilarious and really cool!

------
gchokov
Impressive. Kudos to the author.

p.s. I am hiring ;)

------
brightball
Great writing style. Hilarious.

------
xiphias
So to make it useful it just needs to find the pairs of people who ,,go to
sleep'' at the same time

------
LargeCompanies
This is random, but....

Nice with this data I might finally and truly finish my Social Alarm Clock
idea and do so in which it truly improves the sound of your alarm clock; one
that always makes you smile, laugh, etc.

There's been tons of social alarm clocks(from Justin Bieber to Nestle to Sony
to Wakie, etc) since releasing sleep.fm in 2007 (a century ago in Internet
years) yet no one has executed on the idea properly.

------
david90
This is creepy, but yet very good illustration of social engineering

~~~
aaronbasssett

      Social engineering, in the context of information security,
      refers to psychological manipulation of people into performing
      actions or divulging confidential information.
    

I don't see how this is social engineering at all.

~~~
erikbc
Because it's not.

