
Docker - the Linux container runtime - radimm
http://docker.io/
======
nicolast
I recently changed our Jenkins CI infrastructure to use something like this:
all jobs run in their own LXC container, using BTRFS and its snapshots/clones
instead of AUFS. Works like a charm!

The script is available at <https://github.com/Incubaid/jenkins-lxc> (no docs
for now, and several improvements are possible). It expects CI job scripts to
be contained in the job repository, e.g.
<https://github.com/Incubaid/arakoon/tree/1.6/jenkins>

~~~
niall_ohiggins
That's pretty awesome.

At FrozenRidge.co, we work on our own CI server called Strider and have
actually integrated Docker directly:

You can read about it at: [http://blog.frozenridge.co/next-generation-
continuous-integr...](http://blog.frozenridge.co/next-generation-continuous-
integration-deployment-with-dotclouds-docker-and-strider/)

------
shykes
Wow! Did not expect this to show up on HN before actual release! (I work at
dotCloud).

We're still polishing a few rough edges. If you want early access add your
github ID to this thread and we'll add you right away!

~~~
GuiA
There was a lightning talk about Docker at Pycon; I'd assume that's where OP
got the info from :)

~~~
ARothfusz
I've posted a video of the lightning talk here:
[https://plus.google.com/photos/115695491015706412558/albums/...](https://plus.google.com/photos/115695491015706412558/albums/5856171619418221505/5856171619150829938)

It should be public so you can view it even without a G+ account (I think)

------
asadjb
I'm not familiar with any of the technologies used in this. Anybody care to
comment on how strong the isolations would be security wise, compared to
normal virtualization?

If the security is almost at par and the isolation is good enough that one bad
process can't bring the whole system down, might this be a good alternative to
virtualization, since I imagine it would definitely use less resources.

~~~
teraflop
The Linux namespace stuff is evolving pretty fast, and I personally wouldn't
trust it as the main line of defense for anything important.

With virtualization, a buggy or malicious guest is still limited to its
sandbox unless there's a flaw in the hypervisor itself. With
containers/namespaces, the host and guest are just different sets of processes
that see different "views" of the same kernel, so bugs are much more likely to
be exploitable. Plus, if you enable user namespaces, some code paths (like on-
demand filesystem module loading) that used to require root are now available
to unprivileged users.

There's already been at least one local root exploit that almost made it into
3.9: <https://lkml.org/lkml/2013/3/13/361>

~~~
derefr
> The Linux namespace stuff is evolving pretty fast, and I personally wouldn't
> trust it as the main line of defense for anything important.

If I recall, Heroku uses cgroups (EDIT: and namespaces) exclusively for
multitenant isolation (and by the looks of this, dotCloud does too), so that's
two big votes in the "if it's good enough for them" category.

~~~
teraflop
Sure, but cgroups and namespaces are kind-of-orthogonal features that both
happen to be useful for making container-like things. cgroups are for limiting
resource usage; namespaces are for providing the illusion of root access while
actually being in a sandboxed environment.

And as far as I'm aware (speaking as an interested non-expert, so please
correct me if I'm wrong) cgroups have no effect on permissions, whereas UID
namespaces required a lot of very invasive changes to the kernel.

~~~
jpetazzo
That's correct: cgroups have no effect on permissions. They only enforce
resource usage limits.

Shameless plug: I work at dotCloud, and I wrote 4 blog posts explaining
namespaces, cgroups, AUFS, GRSEC, and how they are relevant to "lightweight
virtualization" and the particular case of PAAS. The articles have been
grouped in a PDF that you can get here if you want a good technical read for
your next plane/train/whatever travel ;-) <http://blog.dotcloud.com/paas-
under-the-hood-ebook>

------
wereHamster
I'm interested to see how that standardization works across linux
distributions. I mean, different distros use different versions of various
libraries (openssl for example). So if your app links with libssl.so.X but the
host only provides libssl.so.Y, then your app won't work.

Of course, you could bundle libssl with your app. But then the standardization
is at the level of kernel/libc ABI. In which case the container is basically a
full LXC guest.

But then why standardize an image format if you can create a small script
which builds the image with lxc-create + installs whatever else necessary for
your app. That script will be much smaller than the full image, even a
barebones ubuntu lxc guest (debootstrap quantal) is ~400MB.

~~~
wmf
Their base layer needs to specify exact versions of libraries.

Deploying from images should be much faster and less fragile (oops, is Github
down?) than from scripts.

~~~
shykes
Absolutely. Think of a Docker container as a universal build artifact. Your
build step can be an arbitrary sequence of unix commands (download
dependencies, build libssl, run shell scripts etc.). Docker can freeze the
result of that build and guarantee that it will run in a repeatable and self-
contained way, no matter where you run it.

So you get clean separation of build and run, which is a hugely important part
of reliable deployment.

~~~
wereHamster
So docker container ABI is basically the kernel? You just create a filesystem
image and docker starts that as LXC guest. You could build a statically
compiled binary and put that as /bin/init into the container image, right?

~~~
shykes
Yes! Exactly :)

You wouldn't need to save it as /sbin/init. You would just type:

    
    
        $ docker run MYIMAGE /path/to/my/static/binary
    

Here's the smallest image I've personally used to run a docker container:
<http://get.docker.io/images/busybox>

------
zdw
Are the aufs filesystems easily navigable from the host OS, ala Solaris's
zones feature?

~~~
zimbatm
The aufs mount is (probably) only mounted on the guest's VFS so I'm not sure
how you would access it from the host.

~~~
shykes
Docker lets you visualize changes on any container's filesystem live, as they
happen. It also lets you snapshot changes from any container into a new image,
and immediately start running new containers from that image. No manual post-
processing of the image, no configuration files to templatize. The whole thing
is 2 commands and maybe half a second.

------
buster
Wow, i need this! What would be the requirements for docker? I suppose it
wouldn't run on linux 2.6.x? (CentOS 5.x)? Any other requirements?

~~~
shykes
The main requirement is a modern kernel with the aufs module. We do most of
our testing on Ubuntu 12.04 and 12.10. But any modern distro should be fine.
There are a few people testing it on CentOS as I write this.

------
zobzu
Note that LXC DOES NOT PROVIDE SECURITY. It provides resource separation (to a
point) and so on.

Breaking out of a filesystem container is as easy as creating a root block
device. Breaking out of a network container is as easy as creating a network
device

And in all cases, you can just inject memory, load lkms, etc. That's without
mentioning the amount of weekly CVEs for Linux namespaces.

~~~
price
> LXC DOES NOT PROVIDE SECURITY

This is out of date. As of Linux 3.8, or with out-of-tree patches in older
kernels, LXC puts each container in its own _user namespace_ , so that root in
the container has no privileges outside. LXC also uses network namespaces, so
the user inside the container can only do on the network what the admin allows
them to do.

Because root inside a user namespace is unprivileged outside it, it can't
scribble on memory or load modules, etc., either.

See <https://wiki.ubuntu.com/LxcSecurity> for a decent summary of the
situation in Ubuntu's releases. Several Ubuntu contributors are also among the
main drivers of LXC upstream.

It's true that user namespaces and other kernel features LXC relies on are
beginning to get much more use than they used to, and probably still have
flaws, though I think you exaggerate how many CVEs are actually being found.
Ubuntu's LXC support also uses apparmor and seccomp to provide further
isolation. Conservative users will probably wait a while more to see what bugs
get shaken out.

~~~
throwaway1979
Thank for your comment! I've been afflicted by FUD related to security in LXC
for over a year. This really helps.

------
pshc
Will there be any possibility of running dev instances in OS X? Perhaps we'll
be able to do _brew install docker-compat_ at some point in the future and get
a best-effort emulation layer even though the Linux APIs are missing. I hate
messing around with virtualization.

~~~
niall_ohiggins
I run a Docker host fine on OS X w/ Vagrant. The client runs on OS X but not
Linux Containers.

------
epynonymous
fyi, aufs which is the union mount filesystem required by docker (and cloud
foundry warden), is not well supported on rhel, centos, and other linux
distros, seems mainly support for ubuntu.

~~~
kalmar
More than this, Ubuntu are trying to remove it from their distribution in
favour of overlayfs [0]. As mentioned there, the reason is that aufs is not
and will not be part of the mainline kernel. Precise 12.04 was going to be
without AUFS, but some issues cropped up, keeping it around.

It now looks like overlayfs will make it into the 3.10 mainline kernel [1], so
it may be a better choice in the future. I think that the _Under the hood_
stuff on docker's page are implementation details that can change, so a switch
to overlayfs when that becomes more suitable could be possible. (Confirmation
from the dotClouders present would be apprecaited.)

[0] [https://lists.ubuntu.com/archives/ubuntu-
devel/2012-February...](https://lists.ubuntu.com/archives/ubuntu-
devel/2012-February/034850.html)

[1] <http://lwn.net/Articles/542707/>

------
theatrus2
Whats the interaction between systems based on Mesos, which can (and many do)
use containers? Is this really designed for more multi-tenancy with lower
trust over same-org clouds?

~~~
shykes
Docker solves the problem of running things in a repeatable and
infrastructure-agnostic way. Mesos solves the problem of telling many nodes
what to run and when.

In other words, Docker + Mesos is a killer combo. There is already
experimentation underway to use Docker as an execution engine for Mesos.

------
duked
That sounds like a definitive improvement over LXC specially the isolation
properties but I'm not sure what is the added value of Docker compared to
OpenVZ ?

Any ideas ?

~~~
zimbatm
My guess is that the container specification is orthogonal to the sandboxing
feature. In fact they're using LXC.

~~~
shykes
Correct, Docker is currently based on lxc, but that is an implementation
detail. In theory it could be ported to any process-level isolation tech with
similar features: OpenVZ, Solaris Zones - you could also try using BSD jails
although I don't know if they have all the required features.

To answer the original question: Docker extends LXC with a higher-level API
which operates at the process level. OpenVZ helps you create "mini-servers".
Docker lets you forget about servers and manage processes.

~~~
zimbatm
Nice. Can't way to try it out. I've built a similar tool in Go for PandaStream
to isolate or encoding processes.

------
nwg
Does anyone know, or can anyone speculate on exactly how the "heterogenous
payloads" are specified?

~~~
shykes
The payload is anything which can be recorded on the filesystem by running a
unix command. For example:

    
    
         # Run command which adds your payload
         $ docker run base apt-get install curl
         5b4a1ee8
    
         # Commit the result to a new image
         $ docker commit 5b4a1ee8 nwg/base-with-curl
    
         # Run a command from the new image. Your payload is available!
         $ docker run nwg/base-with-curl curl http://www.google.com
    

Docker doesn't care _how_ the payload was added. apt-get, shell script, pip
install, gem bundle... All the same. In the end it's just a change to the
filesystem.

------
Harkins
This sounds like a reimplementation of virtual machines at the os layer
instead of hardware layer.

~~~
voidlogic
It sounds like they are just building on cgroups etc, which are already part
of the Linux kernel.

I would argue that virtual machines at a hypervisor/hardware level were just a
hack for OSs not living up to their isolation promises/obligations. Strong OS
level isolation implementations (cgroups, namespaces etc) allow people to put
isolation back where it belongs, the OS.

The job of the OS is to control the hardware, wrapping the OS is software to
emulate hardware is ridiculous and VMs generally have much more performance
overhead than isolation containers.

------
rlpb
How is this different from what you can already do with lxc on, for example,
Ubuntu Server?

~~~
derefr
It's not that it's any different, it's that it's _standardized_. The idea is
that a Docker container would be portable between different PaaS hosts (and
from your own staging environment to those hosts!) without rebuilding, because
they'd all be using the "Docker standard for deployment."

A PaaS host saying they supported Docker would imply that they'd be using, for
example, SquashFS for container format, AuFS instead of OverlayFS for union-
mounts, LXC instead of OpenVZ/Xen/KVM for isolation, and any other set of
things your container might subtly rely upon.

The culmination of this, I imagine, would be a PaaS host allowing you to
specify the "stuff" you want to run just by the URL of the container-image.

~~~
peterwwillis
Doesn't a standard involve, you know, standards? AFAIK a product name is not a
standard.

What if the namespace changes? What if AuFS changes? What if LXC changes?
Independently or all together? ABI changes? Version changes? Feature changes?
Are all the licenses compatible? Will it ever support platforms other than
just certain versions of Linux? Or languages other than Go?

I don't see a standard. I see marketing for a product and a mailing list to
collect potential customers. But maybe i'm missing something.

~~~
shykes
Hi Peter, this website was only meant to be seen once Docker is actually open-
source, which will be the case very soon.

I do think there is a need for a standard way to package and share software at
the filesystem and process level - we don't pretend to define that standard,
but hopefully we can contribute to it by open-sourcing a real-world
implementation.

~~~
peterwwillis
I guess I read too far into it when I saw the word "standard" everywhere and
got excited - sorry about that. Do you plan on adding to your implementation
the ability to differentiate between compatible versions/platforms, so one
could use this on several cloud instances that aren't built the same?

~~~
shykes
Yes, that is definitely something we would like to add. And we will gladly
accept pull requests :)

------
NateDad
Not to be a stick in the mud, but you're using a copyrighted image as your
logo without any attribution or acknowledgement of the original owner of the
copyright (the Lego Group). You should probably fix that.

~~~
shykes
You're right! We put it up there when it was an internal project, it's
probably time to take care of that. Taking it down until we found a correct
way to do it.

I'm a copyright noob: would simply acknowledging the copyright owner be enough
and fall under fair use, or should we not use it unless we get written
permission?

~~~
NateDad
I am not a copyright expert, however I am familiar with the Lego Group (the
company that produces Lego toys) ... the set is pretty old (1986), which means
they're not worried about making money off it. If you just stick a disclaimer
after it that it's copyright 1986 the Lego Group, that's probably fine,
especially since this looks to be an open source project, so you're not going
to be charging people for something with their picture on it.

This page has what is very likely the original image:
<http://www.peeron.com/scans/7823-1/> Peeron.com has special permission
directly from Lego to display the images, so if you wanted to be extra careful
you could email dan@peeron.com and ask for permission to deep link to their
picture (they'd probably say yes, the admin is a linux geek too). But
honestly, a simple copyright disclaimer is probably fine. Lego won't reach out
and swat you even if they do decide they don't like it, they'll just ask you
to take it down.

------
allerratio
Why should I use this when systemd offers the same (minus the AUFS root fs)?

~~~
shykes
We get that question a lot... Until people start playing with it. Then they
never ask it again :)

------
iand675
I'm not entirely sure that I understand what this does. Is it some sort of
hybrid between provisioning automation and deployment automation?

~~~
derefr
It's a white-labelling of dotCloud's implementation of Heroku's "slug" concept
(<https://devcenter.heroku.com/articles/slug-compiler>): basically, a SquashFS
image with a known SHA, storing a precompiled runtime+libraries+code artifact
that will never change, able to be union-mounted atop a "base image" (a chroot
filesystem, possibly also a known-SHA SquashFS image), then spun up as an
ephemeral LXC container. I actually use the idea in my own ad-hoc deployment
process; they're very convenient for ensuring repeatability.

(As a side-note, this is an example of an interesting bit of game theory: in a
niche, the Majority Player will tend to keep their tech proprietary to stay
ahead, while the Second String will tend to release everything OSS in order to
remove the Majority Player's advantages. This one is dotCloud taking a stab at
Heroku, but you can also think of, for example, Atlassian--who runs Github-
competitor Bitbucket--poking at Github by releasing a _generic_ Git GUI
client, whereas Github released a _Github client_.)

~~~
shykes
This is mostly accurate :)

I will add that our implementation predates Heroku's. Using a generic
container layer early on (first OpenVZ-based prototypes in 2009) is what
allowed us to launch multi-language support a year before any other paas. It's
also how we operate both application servers and databases with the same
underlying codebase, and the same ops team.

------
rolandtritsch
GitHub: rolandtritsch

------
felixr
curl get.docker.io | sudo sh

Has anybody tried this?

------
andyl
Looks cool. When is it gonna be open sourced?

~~~
niggler
"Docker will be open source soon"

Soon could be in tomorrow, a week, a month, a year, or longer ...

~~~
shykes
ETA 2 weeks maximum.

------
undoware
So what we have here is a presently-closed-source, linux-only implementation
of closure copying, which is a tiny part of Nix, which is the platform-
agnostic package manager behind the elegant NixOS (nixos.org). Or is there
something I'm not getting? If it's more platform-dependent, less well-tested,
and (ostensibly temporarily) less open, then why is this on the top of my
page?

~~~
shykes
As a fan of Nix, and someone who expected a lot from Conary/rPath back in the
day, I can provide at least a partial answer: these distros are great as long
as you use their packaging tools for everything. Which is simply not
practical. There's a reason Nix and rPath never took off, and it's the same
reason developers use virtualenv and rvm instead of system packages. They
don't like being forced to use a single packaging tool for everything.

Docker doesn't have an opinion about _how_ you package things. It only cares
about the resulting changes on the filesystem. So you are free to use the best
tool for each job.

By the way this means you can use Docker and Nix together. I would love to see
that :)

