Hacker News new | past | comments | ask | show | jobs | submit login
Netflix: What Happens When You Press Play? (highscalability.com)
517 points by yarapavan on Dec 25, 2017 | hide | past | web | favorite | 130 comments

> I pulled this chapter together from dozens of sources that were at times somewhat contradictory. Facts on the ground change over time and depend who is telling the story and what audience they're addressing.

You know old time journalists had a solution to this. Interviewing subjects with first hand knowledge.

I know I rag on highscalability a lot, but I feel like they deserve it. They've written multiple articles about topics in which I would be a first person resource (Reddit and Netflix), and not once have they ever contacted me or anyone I worked with for an interview. They've quoted me from presentations, but the quotes were out of context and wrong they way they quoted it.

All of this could have been cleared up with a single interview of anyone who is a primary source expert.

I get your point, but how many of these tech companies will actually authorize their employees who know what they're talking about to speak freely with tech reporters about how their system works? I would figure most would redirect you to a PR flunky to feed meaningless corporate-speak to any reporters, and then it's all "Netflix uses the latest internet technology to ensure the best possible viewing experience for all viewers around the world. Unfortunately, the specifics of our system are proprietary and we are unable to discuss that."

Reddit is constantly out presenting, talking, and even writing about architecture, culture, and our unique challenges.

Netflix seems to be similar. Their open source record speaks for itself, and their technical blog and presence at conferences and meetups is outstanding.

Source: though these opinions are my own, I work on Infrastructure at Reddit.

Netflix is particularly open about how it works. There are some confidential bits, but most of the infrastructure projects are not considered proprietary. (I work at Netflix)

Hi Jeremy,

I'm very sorry if I've misquoted you or quoted you out of context. I'd certainly correct them instantly if contacted. I've searched my email and I don't see any corrections, so I'm not sure how I can make amends.

I pull a lot of quotes from a lot of sources. Many thousands a year. My goal is always to highlight ideas that I think might help readers do their job better. Highlight is they key word here. That you say those sort of things often is not surprising. My hope is always readers will be made curious enough to follow the link and learn more on their own.

And unfortunately there is no "they" at HighScalability.com. There's only me and there's only ever been me. So all mistakes are mine and only mine. There's never been much money or time to be as comprehensive as I'd like.

I do the best I can, but I often fall short, no doubt. I'm not a journalist. I never said I was. I'm a programmer. Have been all my life. I started this blog a long time ago. It was just stuff I found interesting. For whatever reason it got popular for awhile so I tried to make something out of it. That, like most of the industry is petering out now, so we'll see what the future holds.

The point about interviews is a good one and one I've wrestled with. Over the years I've conducted a lot of interviews. Much less so lately, and that's for a reason. I've learned that presentations at major conferences are usually the best resources. Presenters put a huge amount of effort into making those talks. And everything they say is open and approved for public consumption by definition.

Interviews with someone like me are often a bother, a time sync, so I don't usually get as much out of them. Interviews have a high overhead, taking a lot of time to setup and there's often a lot of red tape and legal wrangling that must go on to get access and content approval. Which is again, why, presentations are such awesome resources.

This article was derived mostly from people working on a project making a major presentation or from primary sources writing an article. I couldn't do better than that.

And regardless, I would still make mistakes. Believe me. It's surprisingly hard to make sense of an interview and turn something from thread mode into document mode. That process is an error prone one because I have to fill in the gaps and there are always gaps.

It's rare that I've ever just published an interview. I don't think they are that useful in general. I want people to read a highly condensed document they can get something concrete out of with minimal effort or just ignore easily if it's not relevant to them. The strange style I've developed reflects that goal. This article was of course different because it has a different audience.

So again, I apologize. But I guarantee my mistakes are never intentional, never for money, and never for lack of caring or effort.

One solution would be to email your draft to all authors whose content you used in your article and see if they require any amendments. Thanks for writing the article!

Also want to thank you for highscalability.com- I've been a fan for years. I understand you're not presenting yourself as a journalist- I've always just taken your website as a blog. You've helped me understand a lot of concepts about building bigger websites and engineering practices of so many websites I admire. I very often follow your sources to watch all the videos and read all the posts they link to. Thank you again and keep up the good work!

I think the condensed format is great, but the initial disclaimer makes it sound like it was assembled in a vacuum from open sources. For a post like this with a very specific focus it might make sense to reach out to a couple Netflix engineers for clarifications or just making sure there are no errors. I have worked on some playback systems for a few years and would not mind answering technical questions, I am sure others would do the same

Just wanted to chime in and say thank you! I've enjoyed reading your blog a lot!

Thanks, much appreciated!

> For whatever reason it got popular for awhile so I tried to make something out of it. That, like most of the industry is petering out now, so we'll see what the future holds.

What industry are you referring to here?

Advertising supported technology blogs, magazines, websites, etc.

I want to concur with others, big fan of your work dude. Parent comment is being rather thankless to put it mildly

A really good way for you to get your point across would be to write a reactionary blog post detailing the actual information. Netflix seem to be pretty cool & open about things so it should be possible?

A good idea, except I haven't worked there in almost three years, so a lot of my knowledge it out of date.

I have to start by interviewing my old co-workers! :)

"Netflix: What Happens When You Press Play (2014)" - 2017/12/25

Which just goes to show how much of a market there is for easily digestible information in this space.

Netflix in particular seems to have some really interesting ideas hidden behind unclear blogs(at least for me personally) and long presentations I rarely have time to sit through, even though I often appreciate them when I do.

Try writing tech articles for money. You have to produce them quickly and to point. Interviewing people takes time. By the time the subject recieves the request and gets permission to talk to the reporter, the dealine has passed. An interview might also overturn the theme of the article, a theme dictated by a manager/purchaser based on thier brief reading of a few other articles/blogs. If the reporter wants to be paid they better generate the story as requested and on time. It isnt good jounalism but it is how you make rent.

I've actually heard a lot of people say this. At least a couple Ph.Ds, though I can't recall the links.

How many failures in journalism are attributed to "lazy journalism" where nobody will even leave the building these days to even ask someone involved in the story or topic. How the internet made it "easier" but they've become so over-reliant on it that many journalists fail to include information that isn't already online.

Then again, Netflix is probably a phone call away, so that may be a whole 'nother level of lack-of-followup.

I am all for video-on-demand, as long it's free. So YouTube, Twitch, etc are great value.

In case I need to spend money, I prefer to own the thing I pay for. So Blu-ray/DVD/CDs discs for me. The gathered video/music can be played on all my devices from car to phone, even in flight-mode offline. I trade a bit of less convince upfront to really own it, but then I can play it as often and exactly when and where I want.

Anyway every few years one could get a higher resolution version, unfortunately the films aren't sold in source versions, but downscaled, but the video-on-demand at always lagging behind and stream worse bit rates.

Modern journalism in a nutshell. Now realize that most journalism is this sloppy, not just the pieces about things you happen to know about.

not true. many print news sources call their sources and sometimes verify each quote twice. i’ve heard stories from people who were simply quoted in a new yorker article about movies and they were called twice to verify their quote when it was just a small part of the article

> The advantage of having three regions is that any one region can fail, and the other regions will step in handle all the members in the failed region.

At a previous employer, we met with AWS engineers who told us that this sort of thinking wasn't correct. Availability zones are for failure tolerance. Regions are conceptually different and for things like moving compute closer to clients, mitigating political instability or ensuring that data is held in a certain jurisdiction. To back up that claim, they told us that amazon.com ran out of a single region, us-west-2.

As an aside, another interesting point from that meeting was that autoscaling wasn't a good strategy for scaling up in the event of the loss of an availability zone. The reason why is that in those scenarios, a lot of other customers will be trying to fail over as well and autoscaling can get very slow. Instead of autoscaling, they recommended that we run 150% of our per-AZ capacity in 3 AZs so that if an AZ went offline, the remaining two would already have the necessary capacity. We still used autoscaling, but it was for responding to spikes and troughs in our own load, not failure scenarios.

I'm curious whether these two observations are correct since it seems like a lot of people view regions and autoscaling through a failure-recovery lens.

I think this is another case of "your not really supposed to...but it works ¯\_(ツ)_/¯"

For your first point, I think it depends on your usage scenario. With a big company that requires good performance to satisfy their users, like Netflix, having multiple regions is a given. But what the article probably missed is that they also most likely use AZs within the same region to mitigate failures, but since they have multiple regions at hand why not use them to mitigate failures as well? And their monthly tests are a simulation of a worst case scenario if an entire region goes down.

As for autoscaling I also think that people think of it as a magic solution. More users? Autoscale. Zone down? Autoscale. But the problem arises when they think that it's instant because it isn't. Things have to happen behind the scenes, but for many people, they only care about what happens on stage, so for them configuring autoscale is just better because it's easier than load balancing on 3 different AZs.

Sometimes there have been region-wide issues. Running cold has its own set of problems. We do not rely exclusively on auto scaling when we do failover.

Disclaimer: while I work at Netflix, these views are not necessarily those of my employer

Just my $0.02.

The Netflix client/SDK was configured to contact 3 different regions. I'm guessing somehow, on the backend, it was replicating the state of your request to the other three regions. If one regions were to fail, the client would query the other two regions and regain its state.

Having 3 regions available - the other two should be able to pick up the load of losing one without hitting capacity.

IIRC Netflix went down in 2012 when all of us-east went offline. I bet they now have multi-region followers to mitigate this scenario.

I'd be interested in finding out what happens when I press the thumbs down/up icon on Netflix, because the match ratings that are displayed to me seem to have nothing to do with either my ratings or with my watching history.

AFAIK from some conversations I had at a conference last year with a few Netflix engineers who work on this system, it's sort of realtime. Recommendations are build up using Apache Flink, which provides stateful stream compute.

So your click turns into an event on a Kafka topic which is eventually consumed by Flink, joined with whatever other data sources they need and then written out to a store somewhere. IIRC the various machine learning models exist inside their Flink architecture.


It records your like/dislike, but it doesn’t distribute the change to its database immediately. Same for YouTube or so many other services.

The thing is I don't feel like any of the match ratings in the last year and a half since I've been using Netflix have had anything to do with my interests, the ratings I've given out or with my viewing history.

Unless propagation to their database takes two years, I'm really interested in how they actually arrive at match ratings.

I believe largely because a single "like / dislike" is massively inferior to the old traditional Rating system where I am able to Rank content I like...

The recommendation engine seems to be mostly about pushing their original content. Not shocking there.

I love how Netflix is so open about their architecture, but I have one strong criticism.

After trying to get Spinnaker running, which is like 10 services for a fairly straightforward application, and reading about how Netflix has Cron jobs running to 'clean up' data inconsistencies... I started to think Google has a better approach to scaling the code.

Netflix is strong on eventual consistency, and making services as small as possible. Having tried to build something in that image, holy shit the issue of distributed transactions becomes a nightmare.

We had to build error handling in everything to handle the failure of everything else. Maybe half our code was 'just in case' to deal with exploding failure scenario complexity during service call fanout. If your service calls out to two or more services that also do data updates, God help you. There's no call ordering or way to make sure both succeed or fail, so if one does you could easily end up in an invalid state.

Contrast this to Google's approach. Maps is one service, Docs is one service. Their service boundaries are much larger so they can shove the complexity of consistency and rollbacks back into the DB where it belongs. And they avoid eventual consistency as much as possible.

If Google can make these big 'monoliths' work at huge scale I don't see any advantage to 'microservices', it's just a bunch of pointless overhead. I think Netflix has had a little too much industry koolaid ...

> Netflix is strong on eventual consistency, and making services as small as possible. Having tried to build something in that image, holy shit the issue of distributed transactions becomes a nightmare.

Distributed transactions are a sure sign that your service boundaries have been incorrectly drawn.

Not according to Google :). They actually support and encourage distributed transactions across boundaries.

Imagine all the data Google stores with your user profile across their hundreds of application boundaries. To keep things consistent you would either need to store all profile data in one massive service or support distributed transactions.

The Netflix model is fault tolerance, with every service supporting various failed profile update scenarios. Google just decided to add distributed transactions support to spare all that overhead.

I feel bit sorry for Netflix, for the lengths they have gone to get their high performance/reliable architecture but the clients are still at the mercy of the OS, browsers, CPU (decoding codecs) of-course assuming high speed Internet.

From their website - https://help.netflix.com/en/node/23742


Google Chrome Up to 720p on Windows, Mac, and Linux. Up to 1080p on Chrome OS. Internet Explorer up to 1080p. Microsoft Edge up to 4K. Mozilla Firefox up to 720p. Opera up to 720p. Safari up to 1080p on Mac OS X 10.10.3 or later. Streaming in 4K requires an HDCP 2.2 compliant connection to a 4K capable display, Intel's 7th generation Core CPU, and the latest Windows updates.

P.S Interesting to note that Chrome OS plays 1080p, I assume because it has HW acceleration enabled by default (in browser alone, not in android apps though).

Is it browser limitations, or just DRM limitations? Youtube seems to do 4K just fine.

It's DRM. That's why you see 4k available in edge (where they use special CPU instructions for secure enclave - that's why requirement of newer i7 CPU) while 720p only in Chrome.

I think it has to do with HEVC h.265 codec & HW acceleration support available to the browser in the respective platforms.


Not sure why Chrome doesn't support HW for it though albeit Chrome OS. My Celeron N3060 Chromebook can decode it.

I read Netflix has started using VP9 for streaming, newer CPU's have VP9 decoders in them.

No it's actually a DRM limitation. You're not allowed to watch high definition content of your platform doesn't support latest Widewine DRM even if you have the HW or SW decoding capability.

Yeah, IE and Edge are the only browsers with PlayReady DRM. Rest of them (besides Apple devices) are using Widevine DRM.

DRM, you should be sorry for the users

I know what it does when I don't press play: fills my memory with a boatload of bloated interface that has the shittiest navigation of any media player I've ever used. And I still can't just view a list of all series/movies by date.

I’ve hated it since pressing play stopped being necessary to start video playing. Now I have to keep the focus bouncing around or else whatever it’s currently on will start yelling at me. Can’t just stop and look around at what’s on screen. It’s stressful.

[EDIT] in fact, now that I think about it, I almost never just browse the catalogue anymore, while I used to a lot. I open it and go to search ASAP, then close it if I didn't find what I want or see something else interesting in the related results. The auto-playing is annoying enough that it's almost totally driven me away from that part of the interface. I knew I didn't like it but hadn't thought about how it'd affected my behavior, but there it is. Time browsing catalogue from "lots" to "nearly none".

[EDIT EDIT] I might add that full-screen ads you have to scroll past (with no indication that's what you need to do) and other general mucking with the plain grid-layout interface make it wicked confusing for my parents. They just got a Roku solely for Netflix and if it keeps doing that I bet they won't use it very much.

Yes. And it gets worse -- I really hate the new "fake auto-play".

On our Roku, if you sit on a movie or show for just a few seconds, Netflix starts auto-playing a video. But it's not the real show anymore, it's a weird custom-Netflix-made background advertisement for the show using random clips set to odd royalty-free stock music that is specifically not from the show/movie you are actually on and doesn't match. It kind of looks like they set some machine learning robot, or some poorly-paid Amazon Turk people, to generate these clips for much of the titles in their library.

Why? Why does this exist? What user testing or data could have happened for Netflix to decide to spend time/money/bandwidth/resources on this?

Not an earth shattering issue or anything. But it's a personal pet peeve to encounter this every time I turn on Netflix. It feels like they are trying to video-clickbait me, even after I've already paid them for everything.

On our PS4 the auto-fake-video thingy runs and is SO annoying, but on our Amazon Fire Box, inside the Netflix app, it doesn't play them, you just get a static image. I assume it's down to the horsepower of the relevant client device.

All of the complaints in this chain are things I like about the Netflix UX. Just to give some perspective.

I use the Netflix homepage primarily, rarely searching. I've watched those ads they play and then started watching the TV show/ movie they're for.

Amazon's UI on the fire stick is so much worse.

I'm not sure about that really. It's like bashing your shin against a coffee table vs walking into a lamppost. Both painful experiences in their own unique ways.

Which media player has an interface you like? I'm usually an outcast for it, but I actually like the iTunes Movies interface myself.

I like mpv the most. It's simple, but still has everything you need (and you can do rest with the keyboard shortcuts), and it's pretty. In my opinion, however, the problem with Netflix player, is not its design. It's that it's pretty laggy, it is not nearly as smooth as for example YouTube player.

Popcorn has a better interface than Netflix IMO

Many of the things Netflix does here google does too and has been doing for a long time like GGC (Google Global Cache) for free at ISP NOC or datacenters - their GGC nodes are many orders of magnitude time is distribution.

90% of the Google traffic through these devices are youtube videos.

They mention it in the article like Netflix pioneered this idea with open-connect. If anything it's harder to get OC devices from Netflix then it is from Google.

Even Akamai does have a similar program where they let you put their CDN on ISP NOC.

You react like Google pioneered these ideas.

Many of the things Netflix does here, Google does too... but VDNs did before YouTube was a twinkle in Chad, Steve, and Jawed’s eye.

The custom appliance proactive multi-tier video placement process outlined here, along with more techniques not mentioned (some of the most significant are only just now being rediscovered by non-CDNs such as Netflix or even AWS), were generally invented and deployed between 1998 and 2003 - 2005.

My beef was not more like "who did it first", I honestly don't know and I couldn't find a reference online of anyone did something like this at this scale before google/youtube/akamai.

The Point is the global scale of this on-premise network and the amount of data they move. I think Netflix is a blip compare to youtube/google/Akamai/facebook (I recently learned facebook also has a similar program, but I think they also use Akamai), on a global scale. I didn't like that glossed on over this as if Netflix is doing something exceptional or even unique. Obviously, it takes some serious engineering chops scale this amount of traffic globally and routes the traffic through on-premise nodes. But at this point, it's not exceptional - if anything its commodity.

Netflix isn't a blip.

For instance: https://fortunedotcom.files.wordpress.com/2015/10/apps-by-in...

Still true in 2017: https://techcrunch.com/2017/07/05/u-s-cord-cutters-watch-mor...

Regardless, long form video streamed globally without buffering is incredibly difficult. Short form clips are drastically easier. Do the math on how much content at what quality can fit in a given cache, or how long it takes to swap a piece of content in a cache, you'll start to see why.

It's surprising to me that streaming well is still not a commodity. Even less so if you consider transactional (paid) video, which is what Netflix does.

For example, just recently, tech press talked about cable companies losing $10B a year from account sharing.

This was a problem VDNs solved in late 90s / early 00s for streaming protocols, but the technique for handling this across a global footprint was sidelined when YouTube's "http" progressive downloads / pseudo streaming and 'ad supported' video became vogue.

Now that streams need to act more like streams again, and people want to charge for the content itself again, the solution is again needed, and no http-based "commodity" CDN seems to have it sorted out. Netflix does.

PS. A couple refs from before Netflix/Google/YouTube:

- https://www.google.com/patents/US20010027491

- https://www.google.com/patents/US20010047275

- https://news.microsoft.com/2000/06/12/microsofts-new-digital...

Those were first iterations. Later iterations were more sophisticated, and not patented in order to not disclose.

Thanks for the references. I was referring to who did a large _volume of traffic_ in this type of distributed node first on a _global scale_ - not who did it / patented it first.

Most importantly, outside the North-American/Europe echo chamber Netflix is a blip (a large blip nonetheless) compare to google/facebook/Akamai (probably even Cloudflare) traffic. I don't say this to undermine what Netflix does both as a service and a Network Engineering standpoint - I am personally a big fan of their service and their tech blog - I was trying to make a point on some info mentioned in the blog.

I am sorry if I wasn't able to express my point properly.

> I was referring to who did a large _volume of traffic_ in this type of distributed node first on a _global scale_ - not who did it / patented it first.

I am too.

For example, even back then we did live streaming events of millions of simultaneous viewers and billions of on-demand streams a month, from petabytes of content in 14 datacenters globally (including Southeast Asia). All before Netflix, Google, Facebook. Akamai and most of the top CDNs were clients.

> Outside NA/EU echo chamber Netflix is a blip...

Netflix is very much playing outside ‘the echo chamber’: http://www.popularmechanics.com/technology/a20138/netflix-in...

And growing: http://variety.com/2017/digital/news/netflix-250-million-hou...

If you have stats that show they are ‘a blip’ in traffic volume globally, please share.

> I was trying to make a point on some info in the blog

By continuing to insist Netflix is “a blip”, and offering no definition of what you mean by large scale, or why you’re so sure Google is doing video delivery networking better, I’m finding it difficult to have a substantive discussion.

PS. You started an ISP in Southeast Asia, and looks like you had Google/Akamai footprint for your ISP. So maybe your ‘blip’ assertion is your ISP data? From the provider point of view, different ISPs (even in US) had very different viewing patterns. Without looking into individual ISPs, we guessed this related to median connection speeds, peering headroom, and localized content/viewing preferences.

That sounds a bit like a violation of net neutrality avant la lettre.

There is a whole world outside the United States.

It's good for everyone -- if you are an ISP with 35%+ of your traffic coming from Netflix, you might be begging for these boxes in your network.

Yup, as an ISP owner, I can tell you 70-75% of my traffic come through GGC, for every 1gbps traffic from cache nodes 100mbps on average is pulled from the internet. Even with GGC nodes, it's still very expensive, BW-wise.

I saw another comment on here in the distant past that said an ISP owner decided to pull out the GGC boxes because they would sync vast amounts of data (largely YouTube IIRC) and it wasn't worth it bandwdith wise for them.

It sounds like 10:1 is a decent saving but still objectively a lot of data.

You can't survive as an ISP if you don't have GGC nodes or directly peered with google DC. If your customer wants to watch youtube videos and you serve them through internet only, which will either mean you will have to buy more BW (not financially sustainable) or you will have to cap youtube traffic resulting in inferior user experience and a customer leaving your service. Or just ban youtube - might as well stop running an ISP.

The best solution is to peer with their DC - let them do the heavy lifting and you can pull from their cache, then you can chuck the GGC nodes. Google is building a small DC close to our NOC - which means lots of saving for us and a better experience for our customers. We can see us chucking the nodes or in this case, we most likely have to give it back to them or something - they are very expensive and large supermicro servers.

10:1 is not a lot if you are serving a lot of traffic. For 50gbps, thats 5gbps bw - that's a lot of bw for free service - we don't cap youtube traffic (users likely to watch high res 1080p, 4k videos nowadays), we can't sell youtube traffic (license agreement from google) - so as far as ISPs are concerned its pissing away money. To be honest services like youtube, facebook and Netflix, even with their on-premise cache solution are a huge headache and expense for ISPs - it's not just the BW, rack-space, distribution switches, power, they all add up.

There is a reason why Google with all their expertise, knowledge, money and influence can't even make a dent in their ISP business - because even with all their money - it insanely expensive when you try to cover most of the country.

> You can't survive as an ISP if you don't have GGC nodes or directly peered with google DC.

Believe me, you can. Deutsche Telekom, Germany's AT&T and Europe's biggest ISP, does this and in the evening YouTube videos regularly stutter on FullHD. But the customers just don't care enough.

If all the big players do it, you really don't have a choice.

> The video comes in a high definition format that’s many terabytes in size. A terabyte is big. Imagine 60 stacks of paper as tall as the Eiffel Tower. That’s a terabyte.

What... No. An HD ProRes will be 150-200GB for a 1.5-2 hour film. That is a 1920x1080 Apple ProRes 422 HQ @ ~190Mbps which is the 'gold standard' accepted Mezz file for Netflix, Google Play, iTunes, AIV (Microsoft, Sony, TalkTalk etc). There are different codecs for broadcasters but this is the standard for VoD. If you deliver uncompressed (uncommon, few relative gains) then it might run into the hundreds of GB but never a multi-terabyte file for a feature.

Although most of this article has been informative, I am dismayed by the outright BS. It basically ruins any trust I would have of this otherwise well written piece. Can your exaggeration be explained or justified please?

There's something about this article, it kind of feels like a sequence of staccato thoughts. It feels like it kind of jumps around.

I did try to tell a story, but I don't think I succeeded as well as I would have liked. It definitely does jump around more than it should. It's hard to cover the important bits and link them seamlessly together. I'm just not that good a writer.

Practice makes perfect! It was frustrating because I think I saw what you were going for and it's a good idea.

Don't be afraid to drop things, even if they're important.

You've got a title, right? "What happens when you press play". Answer that question. There's a whole load there that is interesting but doesn't answer the question of "Given Netflix have a bunch of video ready to go, how does that reach me?". Don't get me wrong, the rest is interesting, and important when thinking about Netflix as a whole, but ultimately I'm unsure they fit in the article.

You might be worried about missing important bits, I personally worry about losing readers. It doesn't matter how important something is, if my target audience quits 15% of the way through the article.

Anyway, no sweat really, blogging is hard enough. It's been a long time since I've written regularly, and I just started a personal blog, and yeah, a few weeks later im sitting on a mound of drafts I need to take a 'minimum viable post' eye to. :)

Merry Christmas!

This article is a chapter from my new book Explain the Cloud Like I'm 10.

The normal expression is "explain like I'm 5", but in the cloud, you need to double everything, in case an AZ goes down.

I feel like this article could be cut in half if it had been given a sharper focus. I don't care why I am seeing a specific image for a film and the reasoning for image personalization. Save that for a different article. I just want to know what happens after I click play.

It ticks to 25% loaded even though your internet is down.

I guess the discrepancy between what I expected the article to be, based on the title, and what it actually was is a good reflection on the fact that development of these types of services is no longer built from first principles, low level to high level, but is rather constructed like a building from a set of abstracted lego pieces.

All this tells me is that Netflix is way too powerful a force in my life. Yikes.

> Netflix Saves Money In AWS

How much of it due to special discount that netflix gets from AWS. Would a company thats not highly visible as netflix save the same amount of money ?

> Recommendations.

I find netflix recommendations pretty bad. Does it really work for other people?

It's not visibility, it's how much you spend/use. I'll bet there are large companies out there you've never heard of that get just as much of a discount from Amazon as Netflix does.

I bet very few do I remember watching someone high up from Netflix presenting at a conference he quoted that they get 70% discount.

>>I find netflix recommendations pretty bad. Does it really work for other people?

They used to, the old system I found the recommendation to be pretty good and discovered some new things as a result

The new, like/dislike system is garbage and most of the stuff is recommends are not something I would enjoy... I wonder how much of that is even based on my personal rating anymore or it is just stuff netflix wants to promote...

The recommendations don’t work for me. The media selection by Netflix is not for me yet. Ultimately, I want a service that recommends me only three possibilities, and at least one is a total hit.

If they could please make it so that you can search on <any> subtitle languages, it would really help when the in-laws visit.

Author uses acronym CDN without first defining it. Explain Like I Do Not Already Know What You Are Explaining.

that article is pretty technical, beyond the level of technicality of simply knowing what a CDN is.

I red-circle the style-guide violating unexplained CDN acronym because the author titled the work from which this is excerpted “Explain Like I’m 10.” I have no objection to using terms-of-art in articles written for experts.

But even technical people sometimes don't know simple acronyms. It doesn't hurt to define them anyway.

mentioning cdn would be akin to a car review magazine mentioning ABS in an article and then dedicated a couple sentences explaining what ABS is

Hmm I dont know what CDN or ABS are.

Content Delivery Network

Anti-lock Braking System

Both are exactly what they sound like

OK thanks :-)

Crap, you're right. I thought I covered it earlier in the book, but I really don't. I do expand the acronym earlier, but that's way too early to be relevant here. I'll fix that. Thanks!

Nothing. So you go to your nearest torrent aggregator and are streaming it down in 3 minutes.

I read the word "nothing" and I thought to mysel: now THIS will turn interesting!!! (thinking that this will point to removal of Net Neutrality and the risks and conflicts it will bring to online services once ISPs turn aggro to both content producers/distributors and end-customers.

Buuuuuuuuut no.. it was a silly short-lived comment about piracy (not even an educating one)

Actually it was a comment questioning why they don't use torrents to provide their services. And also pointing out that p2p based solutions somehow outperform theirs in many cases (for me, not US based so the net neutrality stuff is maybe not relevant). Anyway sorry I was too silly and didn't clarify these points.

P2P has traditionally not worked out well for companies.

Even recently Microsoft tried it for updates, and people were quite angry.

I honestly don't see a situation where someone would prefer to use their own bandwidth for a paid service rather than having a server do the heavy lifting. Either as the provider, or the consumer.

This is true, and I wonder if someone has done an analysis of the many failures. It is impressive that something which works so well in the wild provides so little benefit when tamed.

Maybe Netflix or a streaming service could pay for bytes uploaded. Abuse would be difficult to mitigate but in principle this would encourage peers to provide the company's service to the network.

> people were quite angry.

I don't think this is true. Some enterprises were concerned about the security aspects and asked for greater controls, which Microsoft provided, but Windows 10 has not removed P2P updates, AFAIK.

It has not but I think it's not enabled by default now, and you can declare you want it but limit to a certain amount if on a metered connection, or just turn it on fully.

I think it was the metered connection bit people were annoyed about, especially those who were tethering to their phones and then it downloaded/shared a huge amount of data.

> I think it's not enabled by default now

This is easy enough to check with a simple Google search. (It's still enabled by default).

> Netflix has more than 110 million subscribers. > Netflix operates in more than 200 countries

The five largest public torrent trackers track 30-50 million peers at any given time, serving over 10 million torrents. Torrents are estimated to have over a quarter of a billion active users. Torrents can be streamed, and in the end you have a DRM free copy of the file that you can rewatch at your pleasure, on any device(adding custom subtitles if you need them). It amazes me how much effort has been expended on pushing malware to almost everybody(binary, unauditable blobs in every browser) solely so that massive cartels can make money on something that has been solved since 2001.

Subscribing to Netflix, or any other proprietary streaming (dis)service is not only foolish(in the sense of surrendering your own freedom, not to mention your hard earned money to nefarious corporate interests), but deeply unethical.

How much money do those top 5 torrent trackers generate for the content creators?

How much money do the "cartels" generate for the content creators?

I believe the tens, if not hundreds, of thousands of people involved in the production of the thousands of TV shows and movies earned much more money from those "cartels" than those torrent trackers

Also, how much original content did torrents create?


> We must use guns

Wait, we're still talking about Netflix's highly scalable architecture, right?

This shit sounds like its right out of n-gate.



The only reason Netflix exists in the face of superior alternatives is violence.

What. Are. You. Talking. About.

What happens when I violate IP laws? The police comes knocking on the door, even though I didn't deprive anyone of anything or harm anyone in any way. The only reason companies like Netflix are able to exist in the face of alternatives that are many times better in every single way(like torrents) is because of this underlying foundation of violence.

Curious. how do you earn a living? How do you expect others to earn a living?

Truth is you just don't want to pay for anything.

> not to mention your hard earned money to nefarious corporate interests

Yes, this is the basis of our society. I work for money to buy things, mostly from corporations. There's nothing nefarious about that.

I find it really hard to believe that paying for content is unethical.

Some people think that it doesn't cost anything to make movies/TV.

And some people think that downloading copyrighted content is the same as stealing a loaf of bread.

> There's nothing nefarious about that.


A small example of the nefarious behaviour you support by paying for Netflix.

> Yes, this is the basis of our society. I work for money to buy things, mostly from corporations

Intellectual Property is by definition "non-excludable", which means my having access to some information does not exclude you from having access to the same information, unlike land or food or clothing. It is very different from these, and using your Government to violate the sovereignty of other nations and individuals to impose your warped up notions of property is deeply unethical.

So did Stallman finally start using a web browser?


I think he meant to use exclusivity as a straw man to show that "IP is not food", instead of the more often misused: "not paying is not equal to stealing".

Both are obviously false, but I think that was the intend (argument by confusion?).

> I find it really hard to believe that paying for content is unethical.

Your money isn't going to the people who created the content. It is going to cartels that have no interests in common with you, and have massive lobbying power in order to force you to download malware and restrict the access of information from you.

Of course money goes to the creators. I have an actor friend who was in a 'Netflix Original'. He got paid for it. Sure, others are making profits on it but, again, that's not surprising. That's how our society works.

I wasn't 'lobbied' or forced to install 'malware'. I'm aware of what's going on (I too have heard of R.M.S.!) and I made a conscious decision to subscribe to such services.

I met RMS once. He spoke at my college's CS department and then gave a campus-wide talk.

At the CS reception he told a retired professor to switch banks and just not tell his wife because the bank didn't support a FOSS browser—while eating the dandruff in his hair.

Later, at the wider talk, he called our IT department fascists for requiring authentication to join the Wifi network, then put a halo on his head, proclaimed himself the Jesus Christ of software and tried to auction off a stuffed gnu to a dead silent crowd.

Good times.

> It is going to cartels that have no interests in common with you

To elaborate - we absolutely have something in common. These 'cartels' (businesses) don't give a flying fuck about DRM, they just want to be able to sell things. Often, they make good things that I want to buy. Good set of interests to have in common.

> Your money isn't going to the people who created the content

Patently false and easily verifiable.

How about "isn't primarily going ...".

Media is an investment, which is why were getting lots of "low creativity" movies, re-boots, extended franchises, sequels, etc.. People will risk money for art, but investors don't want to risk anything: some franchises can basically churn out anything and be sure to make a huge profit (which Hollywood accountants can spin as a loss by charging millions for IPR).

I think the spirit of the former comment is "when a movie makes $$$ profit how much of that is going to the colourist, or the make-up artist, the IT support, the animation assistants, ... all people essential to the movie. Versus how much goes to investors whose only part is being already rich?".

The creators get wages, but how many of the creators get profits, and how often is it more than simple investors get?

What is also true though is that The Pirate Bay or Megaupload are about as poor of a choice for content creators as it gets. As far as I know, the amount of money The Pirate Bay and Megaupload have paid content creators is diddly squat, and the amount of original programming produced by these two places is zero.

There are certainly reasons for not liking DRM, but "freedom to pirate" seems like a pretty poor one.

You are right about investors retreating to "safe low creativity", especially in times of downturn. Traditional movie theaters are in decline, so of course traditional cinema is playing it safe for now with reboots and sequels. Netflix, on the other hand, is more awash in investor cash... so at the moment, they are able to take greater risks.

As far as cutting out the "investor" side goes, it would be interesting to see a content host that is explicitly non-profit. Beyond major content hosts, I think the Soundclouds / Bandcamps / Patreons etc. definitely have their place, and have been helpful for some not-so-commercial / hobbyist artists. Even these platforms however seem to struggle with the conflict between making a profit vs serving the artistic community sometimes.

Agreed and disagreed. Netflix or YouTube prowess is indeed only because they have to push their agenda. Torrents solved that problem already. But torrents indeed did not solve the problem of paying to creators. I am looking forward to new decentralized solutions using Blockchain.

I am pretty sure much more malware was distributed over torrent than on Netflix. And for me at least Netflix is much more convenient than torrent ever was. I do not even have a torrent client on my machine, and I cannot even recall when I last had it.

Feels more like an ad for AWS than a technical post

What happens when i press play?

It checks DRM settings in my browser and gets me lowest quality because my browser is open source. That is what happens.

You have the freedom to watch something else

And content creators have the freedom to choose where they distribute content, many of whom choose Netflix.


It's what customers do in free market to get better service.

Or turn from government regulated monopolistic market to actually free market i.e. torrents.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact