How is process isolation not a significant security enhancement?
Nope, it very much is. The fact that "your pid should not have any significant access anyway" doesn't mean that having that made certain and very easy by namespacing is not a security enhancement.
Perhaps you mean something else by "security enhancement" compared to what others here mean. You seem to mean: "extra security that couldn't be achieved by totally finely tuned apps running on the host with all the proper pids and permissions".
Whereas by "security enhancement" people mean: "achieving the same level of security of finely tuned apps running on the host with all the proper pids and permissions with much better EASE, and without having to repeat the whole fine tuning for each new app I add".
But still, layers and all that.
Docker makes that way easier.
If I want 3 instances of nginx running for different projects, I don't really want to setup 3 nginx users (nginx1, nginx2, nginx3).
With Docker, I just start the container and it's isolated from everything else.
# sudo -u "#10000" -g "#10000" id
uid=10000 gid=10000 groups=10000
I am looking at the name of this website and I see that this website is named "hacker news".
As in "hackers"? People, that is, from all ages, that weren't necessarily born knowing everything, and are not afraid to ask around when they don't know how to do something?
If so, then this is the wrong website for this kind of snark.
adduser one; adduser two; adduser three
Significant compared to multiple threads running in same OS process, not significant compared to a fully virtualized OS.
You also don't need Docker to launch multiple processes, you can do it on any modern general purpose operating system.
It has helped us in situations where we wanted to try new configurations of nginx and email without fiddling with the production processes. Reverting changes was also handy sometimes.
It also helped us to be more flexible with versions. We're more confident about running a very recent version of nginx, which allows us to make use of the new features instead of waiting for debian to ship packages. Also less hassle with with dependencies, although the different docker instances need to be upgraded like any other VM.
That said, when we set things up we were missing some features (connecting docker instances to eachother). We had to fiddle to get things the way we wanted and we're still missing some features that we have made workarounds for. I wouldn't recommend it running it in production. However for us it's worth the effort especially in the long run when Docker can handle all of our use-cases.
Docker is awesome for several other reasons, one of which is "shipping."
Google uses not Docker (that I know of) but containers for a lot of their development.
Interest in it seems to have died down.
Not really. AFAIK, they are right now preparing the next release and there will be a lot of activities on lmctfy in 2014.
I have met with the lmctfy team, they are indeed awesome and doing very cool work, in particular around providing a higher-level and more ops-friendly interface to cgroups for resource limitations, one that emphasizes application profiles and SLAs over tweaking dozens of individual knobs.
I really want to make this available as a docker backend, and they seemed to like the idea - something was said about Go bindings possibly coming soon :)
Lets say I have a web app - and there's a production server which runs in docker. Cool! Some port-forwarding easiness with nginx or your favorite flavor of X and it's good to go.
When you want to upgrade, you run your integration tests on the new container. To "Ship" you just, well, change the port forward once you confirm it's good.
If you want to change it to an entirely new server - easy. Just pull your image and create your container - then change your IP information to point to the new server.
Docker is to contain things, not necessarily for security, but to make a build contained. Make it have no dependencies outside the container, other than being able to run Docker.
No application is that simple. You need services that persist between deployments. Like a database.
So you make the database run in it's own container and have the containers talk.
Now you want to upgrade the database. How do you do that?
So you have some persistent storage that can be attached to the containers.
Now you create a new container that has all the exact knowledge (in the form of ad-hoc scripts?) that it's job is to get the attached volume and upgrade the data there and then run the new version of the database?
How is that more convenient or in any way better than chef or puppet?
Also, there is no real animosity or conflicting choice between Chef/Puppet and Docker. While there's overlap, there's nothing preventing you from taking the best of both technologies and integrating them. In fact, there are projects (like deis.io) which attempt to do just that.
I have written about how to do basic data migrations with Docker. You can find the link in my profile.
Also I completely agree that data migrations are tricky, that's why I need a tool to help me there as best as possible.
I should have asked: Given that I'm familiar with or can learn Chef/Puppet what advantage do I get from (also) using Docker. Or what advantage will I get down the road from (also) using Docker?
For example limiting resources come to mind, giving the processes in the container different IO priorities, memory allowances etc. But that's all just cgroup settings, I can do that without creating a container (via Docker) for individual processes.
I just don't see any situation where a Docker container is more useful than a Chef/Puppet recipe. And for any "complicated" setup I feel the advanced features of Chef/Puppet that allow me to fine-tune the setup for each deployment make up for the ease of use of Docker.
Yet there's a huge amount of buzz around Docker and containerization in general. What am I missing?
Chef/Puppet don't solve the problem of running multiple apps with conflicting dependencies on the same machine. A docker image is kinda of like a more efficient virtual machine in that it isolates containers from each other. Maybe you're running 15 containers on one machine each running various different versions of rails, or whatever.
Chef/Puppet let you automate the setup of a machine so you can duplicate it; a docker image basically is a machine you just copy around, and like a VM, they're their own little worlds (for the most part).
That's my understanding anyway.
My view on this, basically, is that the role of automation tools in the Docker realm is going to be exactly like it is with most of the people who like to 'treat cloud like cloud' -- i.e. immutable services.
The config management tools -- whether Ansible, Puppet, Chef, Ansible, whatever -- are a great way to define the nature of your container and have a more efficient description of it.
You'll also continue to use management software to set up the underlying environment.
And you might use some to orchestrate controls on top, but the set of management services to manage docker at a wider scale are still growing in nature and very new.
I'm keeping an eye on things like shipyard but expecting we'll see some more companies emerge in this space that provide management software on top.
Is Docker right for everyone? Probably not. However I like how it is sort of (in a way) taking the lightweight vagrant style model and showing an avenue to which software developed in that way can be brought into production, and the filesystem stuff is pretty clever.
Hi, I'm the creator of Docker. You are not the only one asking this question :) Here's a discussion on the topic between me and the author of a fairly critical blog post titled "docker vs reality: 0-1", which should give you an idea of his state of mind :) I left a comment at the end.
I wrote a blog post about Docker and Configuration Management that elaborates on this:
And I wrote a blog post talking about using Puppet and Docker that talks about how they might interact:
Dependency management without having the high(er) cost of full VMs.
How do you upgrade the database usually? You can also do it the same way.
Not saying it really solves all the problems, just that it solves some of them.
Lots of people go years without upgrading the database that "shipped" with their application! As well, I know of many enterprise applications that literally ship with an entire _server_ as their method of production. You literally buy a server! Containers seem better than that.
They definitely aren't to be used for everything. I wouldn't use them in your situation at all - but they work well for many other things.
What about in a dev environment? It seems that configuration on a single local host would look vastly different, in spite of the same code running.
It shows how a container can be "linked" to another container with an alias, and that alias then causes environment variables within the container to point to the correct IP address and port:
When doing this, do you run the latest unit tests on the source as well? Do they pass?
Important: If you're targeting developers, and those devs use Docker, they might contribute to Docker. If they do, they'd need the tests to pass.
Currently, on DO, the tests do _not_ pass.
I'm not sure how the Docker devs handle non-AUFS-related contributions from contributors whose systems don't have AUFS (whether a failure on any tests means they refuse to consider patches, or whether they consider the tests relevant to the patch).
We kindly point them to our use of libdevmapper, the other storage driver for Docker.
And I should add, we should probably refactor those tests to detect storage capabilities like the Engine does.
I've seen enough projects that have a rigid "if your dev environment isn't perfect, your code's no good to us" mindset that I was wary.
Also, just wanted to say that the Docker installation process and the rest of the Docker docs have been amazing. Originally, I was looking at writing a quick guide on getting up and running with Docker on Linode, but we decided linking the official docs made more sense considering how comprehensive they are.
We are working on upgrading the integration test harness to handle a wider matrix of target systems, including those that don't handle aufs.
If you are interested in helping us out on this, come say hi on #docker-dev on freenode :) We are always looking for help and love to get new contributors and maintainers up and running.
But thanks for that! Are the integration tests not included? No insult, just wondering if they ran.
It's tough. Docker integration tests currently fail on stock Ubuntu from what I can tell. Trying to figure that one out...
If tests still fail for you would you mind filing an issue? Thanks!
I ran through the whole process linked from the Github repo (http://docs.docker.io/en/latest/contributing/devenvironment/), though I only gist'd step 5
EDIT: It looks like the integration tests reside here:
They're called when you run make test, via this chain:
It looks like the integration tests may not be running because the AUFS test failed?
I manually adjusted the Makefile to run the integration tests anyways, and here's the result:
It appears to have a very unhappy time, primarily because the memory cgroup is not enabled. That was a deliberate choice because from what I can see there is a performance hit for enabling that (as well as the swap cgroup), and we decided to leave that disabled rather than give all users a performance hit in order to benefit docker users.
Linode looks like it's in a lot better shape than DO currently. DO has some issues with Docker that prevent it from being very viable.
I'll have to give it a try, thanks again for being so diligent with the tests! The information is useful.
From what I understand the overhead in performance is negligible, while giving you very fine grained control over your resources. Say you have a web application, a front-end reverse proxy, a database, a cache, a background worker process, and a message queue. Throw all these components into separate containers on a single server while the load is low. Then move the DB container into a separate VPS as your traffic grows. Or try doing an upgrade of the message queue in a brand new container, and if it fails, just don't use it.
Sure, you can do the same thing with a VPS, but then you are paying the per-hour fee for it, while 99.99% of its resources will go unused.
I think what we're saying is that in addition to using awesome IaaS services, you can use Docker to solve any number of issues like utilization, deployment, etc., whether they be with your existing providers, on bare metal, or on OpenStack (which can do both.)
I personally have a Linode VPS where I continuously run about 30-40 containers powering the various services/demos I create. I'm looking forward to migrating to the new offering.
You're already running VMs with 1-service per-VM for the sanity of dev-ops, right?
Instead of 1 VM per-service, with docker, you can make 1 larger host, run 1-service per-docker container, and get more performance out of a single machine.
So it allows you to run Docker in an existing VM (provided you reboot the VM with the new kernel image).