

Looking back on “Look Back” videos - slyall
https://code.facebook.com/posts/1474977139392436/looking-back-on-look-back-videos?hn=1

======
junto

      - Obtain enough computing capacity
    
      - Stay within the electrical power limits of our data centers
    

Somehow it feels like the world is so backwards. Here you have all these smart
hard working people with billions of dollars of resources at their disposal
and they made a load of kooky videos that people could 'like'.

The videos I'm sure are cool, and I realise that it was quite an achievement,
but still, our priorities as a Hunan race are pretty much fucked up.

~~~
grinich
What are the priorities of the Hunan race?

~~~
Keyframe
Food for everyone, diversify our habitable planets portfolio, higher education
en masse, stop killing each other, prevention of diseases, keeping up with the
Kardashians, a few others as well...

~~~
alttab
Or at least some spring rolls.

~~~
junto
This is turning into the funniest thread I've ever started on HN because of a
typo. Not very PC, but amusing nevertheless.

~~~
grinich
:P

------
qnk
The people from Statigram[1] came up with this idea last year. I have always
been interested in how they did it. It wasn't the same scale as Facebook, but
I assume it was a big challenge for them, especially not having the resources.

The feature consisted in generating a 15 second video with your most liked
Instagram photos and sending it to you as and MPEG email attachments. They had
to do this because third party apps cannot post to Instagram.

I initiated my video generation probably at their peak, and still, in less
than five minutes, the generated video was in my inbox. If someone from
Statigram is reading this, I would really love to hear your side of the story,
being able to compare it to facebook's approach now will make it even more
interesting to me.

Here's an example of the generated video:
[http://instagram.com/p/ibh7zXmvqm/](http://instagram.com/p/ibh7zXmvqm/)

[1] [http://statigr.am](http://statigr.am)

------
vjeux
To get a sense of scale, the largest DDoS ever recorded by Cloudflare is
400Gb/s[1]. Lookback video was 450Gb/s.

[1] [http://arstechnica.com/security/2014/02/biggest-ddos-ever-
ai...](http://arstechnica.com/security/2014/02/biggest-ddos-ever-aimed-at-
cloudflares-content-delivery-network/)

------
stedaniels
Really annoyingly, Facebook's selection of photos was way off for me. I've
been married over 3 years, with my wife for over 4.5 years, yet it still chose
to include photos of ex girlfriends. Facebook has all the data it needs to
discern that was a bad idea. Quite an oversight in my opinion as I know
several people who it affected too so they didn't publish the video.

~~~
kawsper
Ex-girlfriends are part of your past whether you like it or not. I can speak
with most of my ex-girlfriends, and it was a nice trip down memory-lane to
watch the pictures of me and them.

I am not running from my past, and I appreciate the time that we had together.

~~~
wavefunction
That's a nice sentiment but do you think your wife or their new beaus feel
similarly?

What exactly is the value of the "social graph" if it is never applied by the
one company in the prime position to do so?

~~~
adyus
A HN-specific analogy would be: would you work for a startup who can't accept
what you did prior to employment?

A wife or new beau who can't accept that you had a past may not be a good idea
for the future.

~~~
marcoagner
touche

------
jzwinck
I assumed the videos were created lazily, on first view. This article makes it
pretty clear that they decided from the beginning that they needed to render
all the videos in advance. I wonder how many videos they rendered which were
never viewed, and whether rendering on first demand would have been viable.

~~~
aragot
I was also expecting amazing optimization tricks, like:

\- Designing an MPEG encoder which can take a couple of pictures and render
them on the fly, so only saving the pictures,

\- Carefully spreading the views over a period of time,

\- Associating the soundtrack at view time, since the soundtrack is the same
for everyone,

\- Using EC2 for the purpose, deleting all vids afterwards,

But no, they just used the brute force method.

It would be a good job interview question: "How would you serve 700m videos?"

~~~
ruchirablog
\- Using EC2 for the purpose, deleting all vids afterwards

Facebook has their own infrastructure and very well capable of hosting their
own. And of course deleting the videos isn't a smart idea

------
jbverschoor
Why didn't they just create an html version of this? No need to render a
gazillion videos

~~~
sjtrny
Probably to ensure compatibility for old browsers and mobile users. Obviously
the more elegant solution is to do the animation client side and avoid
rendering millions of videos but it's not the solution that would have worked
for the most people.

~~~
Achshar
Considering how facebook has more than a billion users from all the corners of
the world accessing it from literally every internet connected device, they
had to make something that would work everywhere. Doing it the video way seems
like a good decision in this case, at the scale of facebook.

------
jw2013
Is there really the need to use 80 people for this feature? Why it seems to me
that is just because of excessive programming hours in total available at
Facebook rather than the real need for this new feature. Just curious.

~~~
xixixao
There are no excessive programming hours at Facebook. I assume most of those
people worked on the massive backend challenge, not on making the videos (but
that's my guess).

~~~
jw2013
I understand that the difficulty of making this new feature is scaling the
backend, but at this point wouldn't Facebook already have sort of some
processes or "frameworks" to follow when making a new feature.

------
chengiz
> We had anticipated that about 10% of the people who saw their video would
> share it. Instead, to our surprise, well over 40% of the viewers were
> sharing.

That 10% estimate seems way too low to me. If something like this catches on,
it catches on big time. If they didnt expect it to catch on, then why even
bother. That said, I never had a problem with videos loading etc during this
look back craze. And hardly ever any other time for that matter - it is
remarkable how stable that site is.

~~~
harmegido
That estimate might be a tacit admission to the problem of all the fake
accounts out there.

~~~
randartie
If the estimate was for people who would see the video, why would fake
accounts see the video?

------
rakoo
An impressive engineering task, for sure. But this shows how far we are from
the fundamental ideas of the internet and the web, and instead fall back to
the good ol' feudalism.

~~~
mahouse
Looks like that Schneier article made a dent...

~~~
rakoo
In France we had that thing called Minitel [0], which was heavily widespread
before the WWW took off (I think it was given to everyone or something like
that). The economic (and technical) model was completely different than the
internet: you had a passive, stupid terminal, and you connected to a server
that offers services for a fee. So you paid for connecting and using a service
that could technically not be done on your own minitel (at least in the
earlier version). Just like a browser that is completely stupid and allows you
to connect to Facebook, Twitter, Google et al. for things you can't do on your
own machine. You were depending on a third-party to do things.

We speak a lot about the Minitel model and how it is after all these years a
model for all the big WWW players and ISPs to monetize their services. But
it's not a common reference, so 'feudalism' works well in this context.

[0]
[http://www.reddit.com/r/netsec/comments/1shi77/textsecure_no...](http://www.reddit.com/r/netsec/comments/1shi77/textsecure_now_with_10_million_more_users/ce1uwc1)

------
mcintyre1994
It's easy to forget given some of the press they get just how active their
users are, 720m users who care enough about your product to watch a video
about how they use it must be incredible. The algorithm seemed fairly naive
but it certainly seemed to work well for some I saw, and really impressive
engineering too.

~~~
personlurking
> 720m users who care enough about your product

With the product (and the videos) being (about) the users themselves, it
starts to make sense.

------
yoavz
It unbelievable to think about how much synchronization must have gone into
this project for all the pieces to come together successfully. Just some more
evidence that goes to show facebook has some of the best engineering teams in
the world.

------
asharpe
While I understand they want their own server farms, it is interesting that
Facebook does not use any cloud compute. This is almost the perfect usage
example and pattern for cloud compute.

~~~
randartie
Facebook has some of the largest data-centers in the world, with plenty of
spare capacity. In the article they describe that running these HUGE jobs put
them at risk of not potentially not being able to provide power to all the
machines at once.

If you're thinking they could have just started up a ec2 job to process a
billion videos, I really doubt there's any cloud solution in the world right
now that can handle this scale without capacity planning before hand.

------
notastartup
They should make a video on how they served a billion Facebook likes and where
they came from for the people who bought them.

