
What Makes Containers at Scale So Difficult - orrsella
http://www.theplatform.net/2015/09/29/why-containers-at-scale-is-hard/
======
jacques_chester
This is an advertorial for Mesos, which attacks some of these problems at a
deep level.

It's (not yet) a full PaaS, though Mesosphere is driving there along with
Docker and Hashicorp and everyone else who built parts of the solution and
then realised there's not much profit in selling cogs.

I like Cloud Foundry -- which _is_ a PaaS -- probably because I've worked on
and seen how that sausage gets made. But there's also OpenShift. And Heroku,
which people often forgot pioneered the whole space.

The bin packing problem is hard in theory, but at the scale we're talking
about, it becomes less about the perfect fit and more about the fact that
keeping up-to-date information is hard under realistic conditions. You can
show linear scaling for short tasks. Wait until you're running in someone's
whacky private DC.

Diego (the container scheduler in the next generation of Cloud Foundry)
doesn't try to maintain a perfect view of state in order to do its job. It's
basically impossible to really do so. Instead it satisfices by turning
container placement into an auction, then periodically goes out to sync up
with reality.

If you want to play with it, the best option is to install Lattice[0].

Standard disclaimer: I work for Pivotal Labs, a division of Pivotal, which is
the largest donor of engineering effort to the Cloud Foundry Foundation. So I
guess my comment is an advertorial too.

[0] [http://lattice.cf/](http://lattice.cf/)

------
pcunite
I am not sysadmin, so, thinking out loud here: why not build software services
in a way that does not require this large and sprawling infrastructure? The
future could be simple binaries that run in user space and cache everything in
simple data structures all in RAM. 64GB is cheap and will handle C1M easily.

I want my future server to abstract the hardware (memory, socket too), provide
a few simple API calls, run my runtime (C++, rust, .NET, etc.), and then I'll
build my application business logic to handle all my customers waiting on the
other end of the socket.

Perhaps it the open source way where you have 40+ dependencies. I recently
built my own webserver and found it very liberating.

Think outside the container.

~~~
stephengillie
The goal of Containers is to fully abstract the application. That can only go
so far - the underlying infrastructure is still a huge part of the
application. The responsiveness of a web application doesn't just involve
efficient algorithms and storage calls, but also backend call latency, and
transport time back to the user.

The infrastructure doesn't have to be large, but distributing parts will help
the entire service stay online. Front-end nodes could be distributed to
different datacenters, and back-end could also be mirrored or replicated
between locations. With some kind of global load balancing, users can be
routed to the nearest server for the fastest response.

The binaries have to run somewhere, either in the virtualized unit or its
host. And Open Source or Proprietary, you're going to have dependency bundles
- .NET, Java, etc. It might be more efficient to lump Containers by dependency
so fewer of them need to be loaded by a Host.

------
erikpukinskis
Generic containers are an awkward mid-way point between special-purpose
containers (a Wordpress instance or a rails app on heroku) and an actual
machine.

You get the hassle of maintaining your own instances, without the flexibility
or well-defined performance characteristics of an actual box.

I just don't see the market.

~~~
jacques_chester
The market you don't see is the market for PaaSes, which make the question of
"how do I run this, anyway?" basically vanish.

Continers are the fundamental building block for modern PaaS design. There's a
reason OpenShift 3 was built on top of Kubernetes, or why Cloud Foundry was
built on top of a pre-Docker container system.

~~~
narrowrail
The one thing I can say for Docker is that before it launched, it seemed most
companies using container technology were not that open about it. Docker
appears to be the company that spoke about the unspeakable and caused lots of
software to suddenly be developed in the open where it was all behind the
scenes previously. Maybe I was just clueless, but the more Docker pushed their
releases, the more other companies suddenly had all this code to release.

~~~
jacques_chester
I think Docker deserves pretty much all the credit for making containers a
mainstream concept. In particular, they tied it together the existing
primitives into a uniform concept with a developer-friendly interface.

The very fact we call them "containers" is due to the analogy Docker made to
shipping containers.

~~~
twic
> The very fact we call them "containers" is due to the analogy Docker made to
> shipping containers.

I don't think so. I imagine it stems from the use of that term by Solaris
starting in 2004:

[https://en.wikipedia.org/wiki/Solaris_Containers](https://en.wikipedia.org/wiki/Solaris_Containers)

The evolution of containers on Linux was heavily influenced by Solaris. More
so than it was by BSD, from what i remember.

Whether Solaris picked up the name from an even earlier use, i don't know.

~~~
jacques_chester
Well there you go. I remember them being almost exclusively referred to as
Solaris _Zones_ , but some Googling shows that Solaris Containers was used
interchangeably as early as 2005. You're right.

However, Docker did deliberately make the shipping container analogy -- the
name, the logo and the early blogposts all consciously focused on the
similarity with the invention of shipping containers. So I was half-right.

------
bestusername111
Because not enough memes and hype and buzzwords.

