
Netflix: Container Scheduling, Execution and Integration with AWS (2016) [video] - fullung
https://www.youtube.com/watch?v=4OLlKGT7aVQ
======
nkkollaw
I get scared every time a big company releases their framework, fearing that
it will become the new standard to build even the smallest of websites.

I've used React extensively and you're almost forced to use it if you want to
be hired anywhere (I'm talking front-end development).

The trend nowadays seems to be to adopt any framework that huge companies use
thinking that it'll be good for you. Honestly, do we really need all the
complexity of the what the biggest social network in the world uses, to build
a project 1/1000 the size?

I'm sticking to PHP/MySQL because I'm productive with them, I build small apps
used by 10-20 employees at the most, I don't need NoSQL to scale horizontally
to million of users and don't really need Node since it doesn't really matter
if it's real time and PHP is found on every hosting platform—including the one
all my clients have been using for years. I currently use React but all I do
is CRUD operations and some visual effects, I'm thinking about moving to web
components to lose Webpack, Redux, Babel, and a 200MB installation to save
100-200 orders/week on a database.

~~~
krylon
Personally, I really dislike PHP, but I agree with your sentiment. You do what
you have to do to solve your customer's problems, using tools you know will
work.

At work I have written a couple of CGI scripts in Perl, because that was the
easiest, fastest way to get a working solution to a relatively small problem.
I know CGI is totally '90s, and Perl has not been hip for at least a decade,
but damn it, it gets the job done quickly without fuss.

I kind of suspect that with PHP the situation is comparable to Perl - you have
a well-tuned, mature interpreter, a rich library ecosystem for almost any task
you can think of, and a community of experienced developers.

~~~
nkkollaw
Definitely.

PHP isn't perfect at all, with all its inconsistencies and counterintuitive
constructs, but it works great. Lots of libraries you can install with
Composer, most people know at least some PHP.

On top of that, unless WordPress isn't rewritten in a new language (which
won't happen) PHP won't go anywhere.

Perl is fine, too. We solve problems for clients who pay us to get the job
done, not be brag about how we use the latest, coolest technology.

------
aw3c2
Why should they release it?

~~~
fullung
This site was mostly created as a joke after I watched a couple of @aspyker's
great presentations about Titus and realized that we have a huge amount of
exciting work to do to get our own container platform on AWS to the same
level.

Netflix should do what makes sense for them. I understand that doing a good
job of releasing a very complex piece of software as open source is hard and
takes a lot of effort.

~~~
flurdy
Could be a more useful site if it had been on a domain like
[http://www.isitopensourceyet.org/netflix/titus](http://www.isitopensourceyet.org/netflix/titus)
or similar. (note domain does not exists. At the time of writing...)

~~~
fullung
I'll happily create a redirect once you build that for us. :)

------
moondev
Seems there are plans in the works via clouddriver:
[https://github.com/spinnaker/clouddriver/search?utf8=%E2%9C%...](https://github.com/spinnaker/clouddriver/search?utf8=%E2%9C%93&q=titus&type=)

I'm sure they will eventually, but it probably takes a big amount of effort to
prep and polish something for oss consumption. Thankful for all the netflix
oss tools that are already in the wild!

------
jacques_chester
There are other container schedulers that run on AWS, after all: Mesos,
Kubernetes, Diego, Nomad and probably a dozen others.

Several of these are built into platforms: Mesos in DCOS; Kubernetes appears
in Deis, Rancher and OpenShift; Diego appears in Cloud Foundry (for which it
was created to replace a previous scheduler).

Disclosure: I work for Pivotal, we contribute to Cloud Foundry and our Spring
folk do a bunch of work around Netflix OSS.

~~~
fullung
This thread has some of the motivation for why Titus is exciting in my
opinion:
[https://news.ycombinator.com/item?id=14526452](https://news.ycombinator.com/item?id=14526452)

Kubernetes, Mesos/DC/OS and the various other options are all valid solutions
if you need to run in many diverse environments. Titus runs (ran?) on top of
Mesos before they built an ECS integration.

That being said, there's interesting benefits to the approach of not trying to
abstract the IaaS and deeply integrating with one or two clouds instead. You
get to leverage much more of the cloud(s) you've decided to adopt if you don't
immediately try to abstract them away.

~~~
jacques_chester
> _That being said, there 's interesting benefits to the approach of not
> trying to abstract the IaaS and deeply integrating with one or two clouds
> instead._

I actually thought about mentioning Convox in this vein, as it is intended to
be for AWS only.

> _You get to leverage much more of the cloud(s) you 've decided to adopt if
> you don't immediately try to abstract them away._

It depends on the services. For the bulk services (blobstores, VMs,
networking, queues, databases etc) the big 3 support, you can usually have
some degree of migration without too much pain, if you're using one of the
platforms correctly.

For example, on migrating a Cloud Foundry platform from AWS to GCP, you can
switch your service brokers from AWS to GCP. Apps that ask for a database
won't, in general, know or care who's providing it.

Another thing is that a fair amount of the specialising services now have
portable alternatives. A good platform can integrate these on an equal basis.
At the moment there's an effort to extract the Cloud Foundry service broker
model so that it can also be used by other platforms[0], so this will become
even easier with time.

Where AWS still have a sustained lead is on sheer breadth of offerings. But my
hunch is that Google know this is the first real opportunity to produce a
second revenue stream comparable with advertising. They're going to brute
force their way to near-parity with a lot of engineering and money.

[0]
[https://www.openservicebrokerapi.org/](https://www.openservicebrokerapi.org/)

------
jsiepkes
[https://www.slideshare.net/mobile/aspyker/netflix-and-
contai...](https://www.slideshare.net/mobile/aspyker/netflix-and-containers-
titus) says: "Deeply support AWS (not trying to abstract IaaS)". I'm wondering
how much value Titus has outside Netflix as compared to say Hashicorp's Nomad?

~~~
sargun
So, disclaimer: I work on Titus.

We have a lot of odd integration points with _Netlfix OSS_ and AWS.

It's funny, but providing simple things like a metadata service is valuable.
Our internal Service Discovery relies on it (which is OSS). Also, VPC-
integration isn't in Nomad, and again, the Netflix ecosystem expects it. These
little integrations make your container appear more EC2 VM/Netflixy, and help
leverage the ecosystem.

~~~
i336_
That makes a lot of sense.

Releasing this even without abstracted integration sounds reasonable to me -
this is a system you built primarily for internal use, and you're engineering
strictly atop AWS.

If the community wants support for other providers, someone else can come
along and re-integrate with another provider, and maybe see if/where it's
possible to collaborate with you guys on small changes that allow for easy
maintenance for everyone.

