
The Curious Case of Linux Containers - coloneltcb
https://medium.com/@sumbry/the-curious-case-of-linux-containers-328e2adc12a2#.ehqvuvenq
======
tinco
We solve all of those use cases in our cluster management system of which
Docker is a key enabling component, except for point 5, though we do have
other components that require multiple steps to provision. I bet we could run
Hadoop if we needed it.

It's easy to point at Docker and say it's a trivial wrapper around some
technologies that have been around for years and call everyone crazy for
buying into the 'Docker is going to change the (sysadmin) world' hype. Of
course in essence it's pretty trivial, many things are.

Docker, and Docker-like systems really are changing the sysadmin world. Easily
formalised software environments is what this hype is about, and the fact that
the Docker community isn't really looking at the stateful service problem
right, doesn't mean they're not enabled by Docker either.

To me, and to many 'devops' (i.e. developers rolling into sysadmin roles)
Docker was eye-opening. We reduced our chef scripts by 80%, and thus our
headaches by 95%. It's saving us months of ops work. If you were a seasoned
sysadmin and rolling your eyes at the fact that we were rolling out
application level services in the same chef codebase we were setting up our
system environments with, then yeah you already were better than us. But as
far as I could tell the way we used to do it really was the status quo, at
least in the startup/small it business world.

~~~
sumbry
Someone once said that the public cloud was never about virtualization but
rather automation. I would contend that the container revolution we're going
through now isn't really about containers but ultimately distributed system
platforms. And if it changes the way many have traditionally approached
problems great!

My main point is that we're really just at the beginning of all of this. I'm
not picking on Docker (I'm actually embracing containers) but rather pointing
out that many of these next gen solutions are only tackling the basic use
cases right now. While I'm excited there is so much more work left to be done.
Implementing many of these solutions still requires a bunch of engineering
effort and I'd like to see more turnkey solutions. The developer end is
definitely getting easier and more productive but the operational end is
getting more complex and still not solving some use cases everyone has. How
many different ways can different companies implement HA MySQL for example?!

There are tons of other platforms out there that have recognized and solved
many of this class of problems for years but the Cloud and microservices are
actually starting to make this worse as adoption skyrockets. Platforms are not
really a new thing, we went through this with JVM Platforms a ~decade ago,
Heroku style PaaS ~five years ago, and now containers + cloud today.

I guess this is what progress looks like :)

~~~
iheartmemcache
Ha you weren't kidding about the number of HA MySQL drop-in engine
replacements out there, damn. At least most people follow the ANSI-SQL
standard, which is something.

I'm not being snarky, but what are the use-cases that everyone have? I'd argue
authentication and authorization would be one of the few things, and it was
already solved with JSR 196 like 15 years ago. It worked, had a well defined
spec, and most importantly it had interop with PAM on UNIX, Kerberos, LDAP,
raw db's, mixed mode, you name it. Everyone complained it was too enterprisey
and no one used it, so they re-invented the wheel in the early 00s with Rails
and Devise and Authlogic and a billion other non-standards. Transactions?
Persistence? Java's spec took care of that too, and fairly rigorously. So I'm
with 100% with you on this being a solved problem.

We're making gradual progress (i.e., the option to develop in a language where
the correctness of our code can be formally verified thanks to more readily
available, mathematically sound type-systems) but like any society there are
trends, and where there are trends you'll have the recurrence of many old
things (Interpol ripping off Joy Division) and the invention of a few new
things (where are often the composition of two older ideas, or the
implementation of a new idea which wasn't computationally feasible previously
but now is, or a concept from another industry like signal analysis or three-
phase road traffic theory applied to our code-monkey'ing domain).

RE: Overall progress - Microsoft is doing some fantastic things in Powershell,
effectively taking concepts like package management, man pages, and the shell,
extracting the best elements from each of those, and implementing them in a
consistent manner. No more choosing between systemd or init.d or other holy
wars. If you want to do it differently, you effectively have a standardized
interface to write your new implementation against within most of the
platform. Don't like ASP.NET's templating system? No problem, it's all open
source, and you can swap your own in, but it's all modular so nothing will
break, and your co-workers can continue working in the traditional Razor
templating.

------
tobbyb
What I find curious about all the container discussions and narrative is the
strange lack of context. Sure discuss Docker but also discuss namespaces,
cgroups, overlayfs, aufs, and all the other critical enabling technologies
where a lot of major problems with containers exist and will be solved. For
instance user namespaces, cgroups are not namespace aware, how to integrate
overlayfs or aufs so they can be mounted by user namespaces seamlessly.

Surely these projects and developers need support and focus. Or else it become
mere marketing for companies that have the funds or ability to market
themselves. Do we just talk about libvirt without context or understanding of
kvm and xen, how would that be useful or meaningful?

An ‘immutable container’ is nothing but launching a copy of a container
enabled by overlay file systems like aufs or overlayfs, a ‘stateless’
container is a bind mount to the host. Using worlds like stateless, immutable
or idempotent just obscures simple underlying technologies and prevents wider
understanding of core Linux technologies that need to be highlighted and
supported. How is this a sustainable development model?

Docker chooses to run containers without an init. The big problem here is most
if not all apps you want to run in a container are not designed to work in an
init less environment and require daemons, services, logging, cron and when
run beyond a single host, ssh and agents. This adds a boatload of additional
complexity for users before you can even deploy your apps, and a lot of effort
is expended is just managing the basic process of running apps and managing
their state in this context.

Contrast that with LXC containers which have a normal init and can manage
multiple processes enabling for instance your VM workloads to move seamlessly
to containers without any extra engineering. Any orchestration, networking,
distributed storage you already use will work obviating the need for
reinventing. That’s a huge win and a huge use case that makes deployment
simple and slices all the complexity, but if you listen to the current
container narrative and the folks pushing a monoculture and container
standards it would appear there are no alternatives and running init less
containers is the only ‘proper’ way to use containers, never mind the
additional complexity may only make sense for specific use cases.

~~~
icebraining
_Docker chooses to run containers without an init._

I don't think Docker chooses one way or the other, and people do run Docker
with an init: [http://phusion.github.io/baseimage-
docker/](http://phusion.github.io/baseimage-docker/)

~~~
shykes
Correct, Docker doesn't care what you run inside the container. You provide a
command to run, and it runs it. That command may be your application server or
a traditional init process which in turn will fork multiple children.

Docker _does_ make it easier to follow an "application container" pattern, and
that pattern avoids (with good reason) booting an entire traditional init
system inside the container. But following that pattern is not mandatory. Not
forcing too many patterns upon users all at once was part of the original
Docker philosophy. Unfortunately that aspect was drowned in the cacophony as a
few loud and passionate people interpreted Docker through the lens of their
own favorite patterns.

In retrospect I wish we had been more assertive in reminding everyone to
respect the philosophy behind Docker: that you can share tools with people
without forcing everyone to use them in the same way as you.

------
anotherhue
Many of the author's desires are handled in SmartOS
([https://www.joyent.com/blog/triton-docker-and-the-best-of-
al...](https://www.joyent.com/blog/triton-docker-and-the-best-of-all-worlds))

~~~
zwischenzug
And OpenShift:
[https://enterprise.openshift.com/](https://enterprise.openshift.com/)

~~~
jacques_chester
I'll pile on!

Cloud Foundry is another opensource PaaS:
[https://www.cloudfoundry.org/learn/features/](https://www.cloudfoundry.org/learn/features/)

(Disclaimer: I work for Pivotal, which donates the majority of engineering
effort to Cloud Foundry)

------
jondubois
I don't like containers for the purpose of deployment. All they do is hide
complexity from DevOps people - But the complexity is still very much there.
Hiding complexity from DevOps people is useful in a PaaS context, but it is an
anti-pattern when you consider pretty much every other use case.

Docker encourages developers to keep adding more and more different
technologies to a system. If you consider most popular applications today,
they are usually made up of hundreds of different technologies - Each of which
requires their own bootstrap/setup logic.

Maybe if each micro-service was made up of fewer different technologies,
deployment wouldn't be such a headache and you wouldn't need Docker to begin
with.

------
vezzy-fnord
Author's terminology seems muddled. The first bullet is correct, but only on a
technicality, and it fails to point out that strictly there is no "Linux
container," only an emergent and weakly cohesive combination of features that
as an artifact create a so-called Linux container (as opposed to being a well-
defined atomic unit/OS resource). The third bullet equivocates archive formats
with application containers.

------
contingencies
The author's 'platform' concerns are in fact infrastructure automation
concerns generic to any service-oriented architecture and _not_ specific to
containers.

My take as an early LXC adopter (way pre docker) and from-concept builder of
two production clusters using custom automation:

1\. Service state. The author essentially requests master-slave promotion.
Corosync/pacemaker is an excellent place to look for well tested solutions
here. The normal approach used there is superior to the author's, ie. a
floating IP used by all clients is switched to an already live replacement
master, which is first jabbed in to action, and which shares the same backing
storage as the nodes which has failed. (A great solution for shared backing
storage on standard hardware is DRBD)

2\. MySQL update. Change management with SOA has to be far more rigid due to
complex interdependencies. Typically you version your entire service,
exhaustively test it against peer services, and only then deploy the updated
service. This implies a workflow process more formalized than the series of
manual operations the author hand-waves about. That said, in a typical
database scenario it is often possible to upgrade across minor versions simply
by updating the binary, since it will read the old on-disk database store
fine.

3\. Heartbleed. With an appropriately automated approach to service builds,
this should be trivial. Mask the affected packages or versions, rebuild all
services, test and deploy. This really goes back again to overall workflow
formalization and automation. (Solid infrastructure would have cryptographic
signoff on stuff before it runs, eg. production demands a signature from test
infrastructure that all tests have passed before executing a new service)

4\. Service dependencies. This is far more complex than people assume. My
advice is to use corosync/pacemaker, a well tested approach. (There are
others)

5\. Hadoop. Same as any other service.

There are many other problems with SOA-type infrastructure automation; some
ideas I wrote up ages ago can be seen at
[http://stani.sh/walter/pfcts/](http://stani.sh/walter/pfcts/) as well as the
sketch/documentation of the solution I built.

------
gingerlime
> Did you know that even with all the advances with technology the amount of
> time that a housewife spends maintaining a household today is the exact same
> as it was in the 1950s!

Unrelated to the core of the article, I know. But it seemed odd to me. Is this
true? Or was it a metaphor?

EDIT: did a bit of googling and found a couple[0][1] of not-extremely-academic
references that claim that it's less than half... but couldn't find anything
more authoritative.

[0] [http://www.telegraph.co.uk/women/womens-
life/9721147/Women-s...](http://www.telegraph.co.uk/women/womens-
life/9721147/Women-spend-half-as-much-time-on-housework-today-compared-
to-1960s.html)

[1]
[https://www.anglianhome.co.uk/50years](https://www.anglianhome.co.uk/50years)

------
cbd1984
Are cgroups meant to be secure against malicious code trying to get out and
install rootkits on the underlying system?

AFAIK, hardware-level virtualization is. If code running under Xen can get
out, that's considered a bug in Xen and it gets fixed, right?

OTOH, chroot is not considered to be secure. It isn't designed to be secure.
It is not a security tool. Code in a chroot jail is expected to be able to
leave without subverting the security mechanisms which, by and large, don't
exist in the first place.

So, are cgroups Xen or chroot?

~~~
kijiki
Containers (which are more than just cgroups) are supposed to be secure.

In practice, the attack surface is much bigger for containers than VMs, so
there are far more "break out of container" vulnerabilities out there than
"break out of VM" vulnerabilities.

Most containers are run in a VM, usually one being run for you by Amazon or
Google.

------
powera
"Also be warned — I have no idea what I’m talking about. Take everything in
this article with a grain of salt." \- pretty much. I'm not sure whether the
author thinks "platforms" even are. And the complaints that "linux cgroups
aren't the entire platform so why call it containers" seem pedantic.

I don't see why "handling MySQL failover", "resource management", "upgrading
OpenSSL" should all be handled by the same piece of software in any case.

