It is unfortunate that so many people compare Vagrant and Docker. While there is overlap, Docker is mostly not viable as a dev environment tool alone, so it isn't a fair comparison. The main reason is because you have to be using Linux (and a recent Linux) as your main dev system, and in practice this is very rare. Move beyond indie developers and for all intents and purposes Linux desktops are non-existent (Vagrant is in use by companies like BBC, Expedia, Intuit, etc. and I can tell you most devs don't know how to use Linux let alone run it as their primary dev platform).
BUT, I agree that putting your dev environment in a Docker container is absolutely _amazing_, and there is a KILLER Vagrant/Docker combo.
The killer combo is actually running Vagrant to spin up Docker-ready VMs, then using Docker inside that to develop. This lets people use Docker on Windows, Mac, and Linux. You get the fast iteration time because all your state is actually in a container, so you just docker kill and run as usual.
In fact, an upcoming version of Vagrant is adding Docker as a provisioner, so you can `docker pull` containers down as part of `vagrant up`.
I'm really curious what these "don't know linux" devs are doing with Vagrant in their day to day jobs. Do they get one team member who 'knows linux' to set up a Vagrantfile and then force them all to use it?
First off, it's not so black and white. Many developer know some Linux but wouldn't trust themselves setting up a secure production server. I consider myself part of this group.
Using Vagrant, we can still:
* Write software that depends on POSIX-only applications, such as Redis and, yes, Docker
* Share development environments-that-look-like-production with other developers, thus avoiding "works on my machine" bugs.
You need very little Linux knowledge to do this. Just apt-get, a text editor and the occasional HOWTO/blog post gets you very far.
Additionally, with Docker-on-Vagrant, we can easily:
* Simulate a multi-server environment locally without hogging resources
* Do effective versioning on dev environment configuration before sharing stuff with colleagues
* As a result, learn Linux administration with easy rollbacks after fuckups.
All this without an on-team Linux guru.
Of course, once you go live, you'll need a decent sysadmin/devop type to un-suck the installations. And backport that to the dev setups. Or, just go to some PaaS and have the security/efficiency part done for you. But that's not my point: my point is that even without running your own hardcore-linux-guru-devopsed production environment, and without anything more than basic Linux skills, you can get a lot of value from Vagrant+Docker.
Yes, the ops side of things primarily sets up the Vagrantfile. Developers just know the Vagrant workflow (up, destroy, ssh). Usually developers know "just enough Linux" to `vagrant ssh` and do useful things, but definitely not enough Linux to run it as a primary dev platform.
And it is growing more and more common that most developers in larger organizations don't even know Vagrant is being used under the covers because it is being masked by higher level scripts ("click button to start dev environment"). Under NDA I can't say any names here, but it is more common than you think.
It's quite common. I've worked at several companies, including BSkyB where the developer team is supported by a devops team - the devops write Vagrant scripts which bootstrap a VM that closely mimics the production environment and include all the necessary services ready to run such as nginx and php or ruby, meanwhile the devs all work on Macs and have enough command-line-fu to do development work, but certainly not enough to provision a Linux virtual machine and set up said services.
One thing I keep seeing and feeling uncomfortable about, is this throwing layers ontop of layers ontop of layers until you are so far removed from what is actually happening, if one piece breaks down it's impossible to fix, and you have a ton of overhead. I want fast, simple, easy to hack, and something that just goes.
Isn't there a way to do this super slimmed down and light weight?
I've been keeping an eye on the ShipBuilder PAAS project - the minimalist Heroku clone written in Go. It is starting to look quite promising.
Quoting jaytaylor's comment [1] from below:
> You may be interested in checking out ShipBuilder - it
> is an open-source Heroku-clone PaaS.
>
> ShipBuilder is written in Go, and uses Git, LXC and
> HAProxy to deliver a complete solution for
> an application platform-as-a-service.
>
> http://shipbuilder.io [2]
>
> https://github.com/sendhub/shipbuilder [3]
Layers are completely optional (though thoroughly encouraged due to aggressive caching.) If it doesn't fit your model and you're not interested in using the features, that's completely fine.
As a dev whose 2010 MBP doesn't do so hot spinning up multiple Vagrants, I've been strongly contemplating Linux for my next laptop just so I can run containers natively.
Docker-on-linux-on-vagrant is a nice compromise though.
If you're just using Vagrant as a harness to run Docker-ready VMs, you only need to spin up one VM to host multiple Docker containers, so it works decently on earlier (definitely 2010!) laptops.
Yes, just to confirm I run loads of stuff on a 2010 MBP... not docker but similar and heavier workloads. With caching as advertised in the linked article, you should see no issues. Just be sure to fork over adequate memory to your VM to stop it swapping over a seriously slow, paravirtualized block device with parallel applications causing host OS contention.
This is my primary use of Vagrant. I think it's amazing to be able to set up a Docker-ready host.
Combining these technologies to have the Docker builds released in ready to go AMIs/VBox/VMWare images for easy deployment is on my dream list. Does anyone want to help?
Sure thing. I am working on something similar for a different LXC-based deployment layer. One of the biggest challenges is keeping the payload (ie. container) naieve of the actual hardware (ie. virtualized interface to the outside world), both in terms of kernel driver requirements and in terms of network topologies. Then, for any nontrivial deployment, you also have to hand over information about services to which it may wish to connect, which may not be appropriate to hardcode. If you would like to thrash this out with me I'd be keen.
I agree that the Vagrant/Docker combo is awesome. I see them as solving completely different problems. To me, Vagrant is for running complete systems while Docker is for running isolated processes (quite commonly within a system managed by vagrant).
Docker source itself ships with a Vagrantfile so that you can get an example development environment up and running should you be running on an old Linux kernel that doesn't sport Docker's current runtime requirements. Just 'git clone git@github.com:dotcloud/docker.git docker && cd docker && vagrant up'.
I'm still a bit confused as to how Vagrant, Docker and Packer all fit in together. I think using Vagrant should be better than interfacing with the virtualization/container layer directly.
Thanks for this post. It's very difficult to keep up with all the stuff going on in the DevOps world right now and figure out what goes together, what competes with each other, and when there's overlap.
I feel like we need some sort of standard nomenclature or diagrams for explaining different pieces of complex software systems. Reading through pages and pages of documentation to make decisions about what to use is too time-consuming.
I would really love to see this too. A diagram with layman description of all the different packer/vms/dockers/vagrants/deployment projects and how they work together.
Glad to see a Vagrant maintainer hop into the fray. Also, really excited to see Docker being added as a provisioner. I'm a senior internal dev at a marketing firm and I'm using Docker to prototype / develop testing environments / workflows and translating them to Vagrant / puppet setups when it's time to hand things off to the junior devs who are running god knows what ( Win 7 / 8, OS X, etc).
> using Linux (and a recent Linux) as your main dev system, and in practice this is very rare.
There is the problem right there.
Make it not rare, hire people who work on Linux and prefer it, this is a way for any company to ensure a good hire and not having to find new interview questions or other tricks to weed out the copy-pasters.
Unfortunately declaring "let it be so" isn't enough to make it so.
If you hire only people proficient with Linux, you won't create a sudden thirst for developers to grok Linux. Instead you'll have just limited your talent pool to the subset of developers who enjoy tinkering with OSes. If that's your goal, then great - but most companies are looking for people who can develop well, regardless of their OS preferences.
I still find the association of mac users with graphics design/video/audio editing interesting.
I've used a mac since 10.4 as my main workstation system and I still get people asking me what they should use for a video editor. I spend most of my time in Emacs, or the Terminal ssh'd into something, or looking at docs in Dash, or in chrome looking at webpages.
I couldn't edit an image to save my life, I leave that to professionals.
I met a few Apple fans in my life, and those who were programmers had a thing for fancy UI - they were not always right, for example the damned switch toggle, but at least they have an opinion and interest. So my small sample size led me to conflate UI and mac users.
So emacs ey, why Mac then? Why not a Linux flavor?
I've used all flavors of Linux (from Gentoo, when there was no installer and I had to use dd, to Ubuntu). I still love Linux, but I don't use it anymore as my main system. I have 2 problems with it:
(1) I want the computer to work for me, not the other way. I used to like to hack everything on my computer, but after years and years of doing that, I've lost my patience. I want things to just work and not break after every damn update. I want to be able to do my real job and use the computer as just a tool.
(2) It looks ugly, the fonts are really bad, nobody cares about UX when designing apps, etc. You can say that I can change that, but as I've said: I want the damn thing to just work out of the box. I understand that maybe you don't care about that, but that's why it's me who's using a Mac. Don't get me wrong, it's nothing wrong in using Linux, it's just not for me anymore.
I'm still using the command line more than the GUI, though. But that is a choice. It combines the best parts of both worlds.
It is unfortunate that so many people compare Vagrant and Docker. While there is overlap, Docker is mostly not viable as a dev environment tool alone, so it isn't a fair comparison. The main reason is because you have to be using Linux (and a recent Linux) as your main dev system, and in practice this is very rare. Move beyond indie developers and for all intents and purposes Linux desktops are non-existent (Vagrant is in use by companies like BBC, Expedia, Intuit, etc. and I can tell you most devs don't know how to use Linux let alone run it as their primary dev platform).
BUT, I agree that putting your dev environment in a Docker container is absolutely _amazing_, and there is a KILLER Vagrant/Docker combo.
The killer combo is actually running Vagrant to spin up Docker-ready VMs, then using Docker inside that to develop. This lets people use Docker on Windows, Mac, and Linux. You get the fast iteration time because all your state is actually in a container, so you just docker kill and run as usual.
In fact, an upcoming version of Vagrant is adding Docker as a provisioner, so you can `docker pull` containers down as part of `vagrant up`.
And I published Packer[1] templates to build Docker-ready VirtualBox/VMware/AWS images that are Vagrant-ready: https://github.com/mitchellh/packer-ubuntu-12.04-docker
[1]: www.packer.io