Nearly all the articles and examples seem super easy, but it's quite a challenge to use in an actual multi-host production environment supporting mission-critical systems.
This can send the wrong message to beginners and send newbies down the wrong path, which ends up as a disservice to Docker when they hit the complexities and become disillusioned with it.
To address those misconceptions, I've just posted the article "Docker Misconceptions": https://devopsu.com/blog/docker-misconceptions/
I spent about a month working on setting up Docker for production scenarios and reading everything I could on it (all the docs, courses, and well over 100 articles - see Docker Weekly for a wealth of resources). My post covers the main misconceptions I see popping up and also some advice for simplifying Docker use if you do decide to use it for multi-host production.
I'm sure I've probably missed some things, so please chime in if I've missed something important.
In the post you link above, it seems like you're mainly comparing Docker to traditional, non-virtualized setups. Do you (or does anybody) have thoughts on comparing it to virtualized environments? I recently experimented with Xen as a way to isolate services, and it seemed awfully heavy; does Docker end up being a lighter burden?
I'd also love to hear more stories about Docker vs the other virtualized setups. It generally seems to be a dramatic improvement in usability, performance, etc, but I'd love to hear more about the specifics from those who have made the switch.
Here's a few production users: http://www.docker.com/resources/usecases/
We provided CI for the 2500 teams and also ran the competition using Docker (both on top of AWS EC2 instances). I think that without Docker the implementation would have been a lot more painful and almost impossible given the time constraints we had for the project.
Fast starting and fast shutdown of build and test processes allowed us to run CI builds efficiently on a fairly small amount of nodes. This saved us money and also helped to keep the infrastructure manageable.
Also the additional level of security was really nice since we were running possibly malicious code from 2500 sources. The malicious part became even more evident when Kevin Mitnick tweeted that he might participate: https://twitter.com/kevinmitnick/status/446359450614378496
The Docker image repository was also really handy when we ran the races. We needed build the bots only once and then we could just use the prebuilt images on on race nodes.
Building the actual base image was fairly painful. We support about 20 different programming languages and also package in quite a few libraries. Once the base image build was fully automatized and we had the Docker related processes fully working, it was almost a revelation on how Docker could improve things over the classical virtual machine approach. I can't wait to try it out on another project :)
By the way, the competition finals are today (Tuesday) and streamed live @ https://helloworldopen.com/ (in English and Finnish)
* half git-syntax/half package manager-syntax is cumbersome to me.
* Inability to compile Docker from source without using the Docker Project's provided "black box" binary Docker.
* No formal explanation of how to create your own base image from scratch (ie. not using a Docker Project provided base image).
* Defaults to storing images in the Docker "cloud" repo. (would have preferred a git-like setup where pushing my images to a repo of my choice was more "normal")
* GO Lang is still pretty obscure for most developers, and an interesting choice given the language's youth and likely-hood to change rapidly as it matures.
* Docker is built on-top of existing Linux technologies, mainly LXC, making it more-or-less disposable in the future (as someone else figures out how to abstract/manage LXC better)
* Docker is mainly built by Dotcloud - a for-profit company. Dotcloud has been very generous in their effort... but, being for-profit, what is their take out of it? (It can't be "just for the goodness of Linux").
You can compile it from source (assuming you have a go environment and set GOPATH properly):
$ sudo apt-get install libdevmapper-dev libsqlite3-dev btrfs-tools
$ go get -u github.com/dotcloud/docker/docker
docker will be in your $GOPATH/bin
> GO Lang is still pretty obscure for most developers, and an interesting choice given the language's youth and likely-hood to change rapidly as it matures.
Go1 promise will be kept by Go team.
> Docker is built on-top of existing Linux technologies, mainly LXC, making it more-or-less disposable in the future (as someone else figures out how to abstract/manage LXC better)
No, docker right now is built on top of libcontainer.
Edit 1: Format
The go authors have committed to keeping major versions stable since v1 and are unlikely to massively change anything.
I'm sure much of google's infrastructure is coming to rely on go (and docker!) so they will be as unhappy about such drastic change as others.
However, this doesn't excuse the remaining gripes listed above.
You mean 14 months ago?
I also found this particular annoying.
To build it from source, make sure to install all dependencies (check the provided Dockerfile for details), add both docker and docker/vendor to your GOPATH and then execute `bash hack/make.sh dynbinary`
Offer them some suggestions on how to clean up the UI.
"Inability to compile Docker from source without using the Docker Project's provided "black box" binary Docker."
Addressed in a different comment.
"No formal explanation of how to create your own base image from scratch (ie. not using a Docker Project provided base image)."
As usual, documentation rarely keeps up pace with the rest of the project despite best intentions. Mind you, despite some holes, their documentation is quite impressive.
"Defaults to storing images in the Docker "cloud" repo."
This will probably change now that Google is embracing it.
"GO Lang is still pretty obscure for most developers, and an interesting choice given the language's youth and likely-hood to change rapidly as it matures."
"Docker is built on-top of existing Linux technologies, mainly LXC, making it more-or-less disposable in the future (as someone else figures out how to abstract/manage LXC better)"
This is a benefit of Docker! I'd rather see them build it with existing components wherever they can.
"Docker is mainly built by Dotcloud - a for-profit company. Dotcloud has been very generous in their effort... but, being for-profit, what is their take out of it? (It can't be "just for the goodness of Linux")."
They're following a tried-and-true pattern for generating revenue off an open source model (giving away the product for free and making money on support and consulting). Companies that try to throw out a crappy product and charge for support quickly find themselves with no customers. Also, now that Google is on-board, you can expect the product to improve significantly as they contribute to the repo even more.
Some links there for examples. Ultimately the base image is a tarball of a filesystem.
For what it does, the syntax seems pretty natural, but that's all subjective, so of course YMM(and clearly does)V.
> Inability to compile Docker from source without using the Docker Project's provided "black box" binary Docker.
I think this is more accurately phrased as "no simple documentation of the dependencies to build docker from source without using the pre-built dev container".
> No formal explanation of how to create your own base image from scratch (ie. not using a Docker Project provided base image).
Well, there's an explanation-by-example @ http://docs.docker.com/articles/baseimages/ which seems to indicate that the process is "take the root file system of the Linux system you want to make into an image, tar it up with specified options, and then import the tar."
Admittedly, this documentation could be improved.
> Defaults to storing images in the Docker "cloud" repo. (would have preferred a git-like setup where pushing my images to a repo of my choice was more "normal")
Its "Go" not "GO". And why is this a gripe? If it was true (and it doesn't seem to be, Go has a stable 1.0 release with a strong backward-compatibility commitment), that might be an added dev cost for the Docker team, but unless there was a reason to think that the cost didn't have sufficient benefits, shouldn't be a reason for a potential user to complain.
Docker is first and has momentum, and doesn't seem like its going to stop development just because it hit 1.0. So, yeah, something better might come along, but its not like Docker is a stationary target.
> Docker is mainly built by Dotcloud - a for-profit company. Dotcloud has been very generous in their effort... but, being for-profit, what is their take out of it? (It can't be "just for the goodness of Linux").
The whole list of services (training, consulting, and support) that they announced today , which were previewed in the earlier blog post about Docker 1.0 , makes that pretty clear.
 http://blog.docker.com/2014/06/its-here-docker-1-0/ : "In addition, to provide a full solution for using Docker in production we’re also delivering complete documentation, training programs, professional services, and enterprise support."
You still need to be able to configure everything inside the container (thus the need for a CM tool), and then also the environment and infrastructure inside which the containers live (thus the need for infra management tools).
Docker really doesn't make infrastructure management markedly easier—just faster, more flexible, and slightly more powerful. It's one layer of abstraction over a VM, IMO, and that's a good thing, but carries with it it's own set of complexities.
I'm missing something, but in what scenarios would you need to do something more complex than what can be expressed in a few RUN and ADD commands in a Dockerfile?
My limited experience is that indeed, orchestration is still completely needed. We underestimated that and now we have a bunch of hacky shell scripts that build and start and stop docker containers at various moments. But the docker containers themselves are pretty simple.
That can indicate some deeper problems, but in some circumstances (and especially when making updates to your container images), using configuration management for individual containers can be helpful for the same reasons even CM-ifying a small single-purpose standalone VM can.
I agree with the Docker Misconceptions article for the most part, but that might just be because it what makes the most sense to me (and I'm still likely ignorant of some important configuration to consider :D )
>For logs, you can either use a shared directory with the host or use a remote log collection service like logstash or papertrail.
I would not put Logstash (software) and Papertrail (SaaS) in the same bucket though. With Logstash (or any other data collectors like Flume, Fluentd, etc.) you need a backend system to go with it.
Sorry to nitpick, but logging is something I work on/think about all day, so I thought to point it out.
We're going to show the result of that tomorrow :)
We decided to ship 1.0 anyway just for the sheer volume of bugfixes. But don't worry we are still on a monthly release schedule. We are far from finished shipping stuff!
# systemd unit file
ExecStart=/usr/bin/docker run username/app
More examples here:
cat /run/docker/mypipe >/dev/log
I'm not sure this is the right approach, though. In my opinion, docker shouldn't log at all—containers are unix processes, and Unix processes log to ephemeral stream-sink endpoints (think AWS SNS topics) called stdout and stderr. To collect and persist those logs, the Unix Way is to use shell redirection to "attach" those endpoints to subscribers.
Now, in Docker's daemonized run mode, these pipes are just hanging loose. But docker provides the "docker attach" command precisely so you can reattach them to something (or as many somethings) as you like. All you have to do if to get sensible logging if you're using e.g. Upstart is to create a service that runs "docker attach <mycontainer>": the logs will flow from the container's stdout/stderr to the service's stdout/stderr, and Upstart will catch them and handle them sensibly from there.
So, having done something like that, the important bit would actually be making sure docker doesn't do its own broken logging at all, except in the case of test containers that will never be attached to. Actually, this also cuts across the use (or lack thereof) of the --rm flag: it'd make sense for containers started with --rm to not log. (The generalization of the --rm semantics would seem to put such containers in the same category as e.g. EC2 instance-store instances.)
I would hate to see docker go down a path where it folds in its own logging "framework", which, I feel, would be going too far (poor separation of concerns, etc).
1.) logrotate at the end of the day
2.) have your logshipper watch docker log folder for each container
3) Log shipper ( or collector) ships files as they are updated to a central server and you are free to archive or delete as you wish.
Many of the central logging systems can detect rolled logs, so this set up is not much of a stretch.
I do agree more configurable logging is a bit of an oversight, especially for something like Docker.
That being said, logrotate would sort of work with caveats since the json files are one entry per line last I checked.
The CHANGELOG states Production support as the 1.0 feature, but how am I supposed to be confident using Docker 1.0 in production if I can't use the client from my own machine? Or is there some other preferred install method?
TLDR: Homebrew does not appear to be an official install method.
I'm with you on this one. Not exactly confident about using this in a production environment quite yet.
You can check out our build and release tooling in the ./hack subdirectory.
Fwiw I use the Darwin build every day and it works great.
edit: yes I realise that the docs tell you to use a 64-bit version of a recent Ubuntu
docker ps -l --no-trunc
lxc-attach -n xxxxxxxxxx
Here's the video -- https://austin2014.drupal.org/session/mo-servers-mo-problems
Something more than a hello world.
Or you could ship a relatively large fully installed VM image of the OS+all the application parts and dependencies (this may be problematic if your target is a licensed OS).
Docker gives you something in between.
Think of a chroot'd environment (not that that's what it is, but it shares some concepts) with all the application code, libs and dependencies pre-packaged that should run theoretically out of the box.
That way you can have a 'war' like file that contains everything required to run your app and you're 100% sure that it will just work, rather than to have to field a million questions about how to make your app run on 'obscure distro x' or 'obsolete version y'.
Another use-case would be that if you're deploying something distributed across a large cluster that you can build your docker container on your desktop box, test it until you're happy and then deploy to your cluster without having to re-test everything. Since the container wasn't changed it should work.
It's a way of isolating dependencies between various pieces of software using one single container to hold them all.
Of course this approach has its own unique set of problems, but it solves certain things quite elegantly.
I hope that explains it and that if it is wrong in some aspect that someone will correct me.
Option 3: use a package manager.
Now, I realize this doesn't solve the existing software conflict problem you mention. Nor is it mutually exclusive with shipping an image: you could always construct an image by installing packages into it.
I've always been a little wary of the "ship a golden image" idea. How do Docker images get updated? What if there's a security issue with software in the Docker image -- can my package manager automatically apply security updates? How do I push other upgrades to the client once they have a Docker instance -- do they need a whole new image from me? Package managers have solved these problems, and I hope we're not reinventing those solutions in the Docker world.
If you're familiar with Docker, please enlighten me on these points.
This does seem to get a bit cumbersome. I'm at DockerCon today and Fabio Kung mentioned in his talk that this is one difference from Heroku's container platform---they provide the base image and can update it without requiring you to rebuild your application slug. He said there's been some discussion of a possible "docker rebase" command that would produce new images by replacing lower-level layers while keeping higher-level layers the same.
If you just have the application running in the container then you just replace it with an updated container.
I rebuild the image. At the moment with the giant stupid image that we use for rhell land it takes about 5 minutes (builds a vmware, virtualbox, aws, qemu, and docker image (packer.io is kinda cool))
By the way, I do have a basic idea of what Docker is, as well as some inclination with OS-level virtualization (FreeBSD jails and LXC), but OP's example wasn't particularly inspiring.
I'm an enormous fan of docker for solving linux sysad woes (stacking up magic numbers in /etc/passwd, etc), and it's also great as an asylum for badly designed or badly packaged applications, but let's not forget the utter magic of paths beginning in "./".
Notable features since 0.12.0
* Production support
To ease this pain across a large team, I'm considering using netfilter/iptables or a TCP proxy to make the old port available in our VM environment during transition.
I don't have a nice blog post to show you, but the basic idea is:
- Use the CoreOS Vagrantfile (+ extra files like user_data and any service unit files)
- Share your local directory with the vagrant box with config.vm.synced_folder (just uncomment a line in the Vagrantfile)
- set the DOCKER_HOST and FLEETCTL_TUNNEL env variables so that the local tools work transparently with the vagrant box.
- Then you have the option of running a docker image with -v and sharing the vagrant directory with the docker container, or
- You can dive into CoreOS stuff and write a unit file for your service, and that will share the directory with -v
Now local files sync into the docker container, and if the server is configured to reload on file changes you're good to go.
The hack that docker-osx (https://github.com/noplay/docker-osx) uses has been very effective for me. Basically it mounts your user directory into the vagrant VM at the same path. From there, you can bind-mount any folder in your home directory easily via something like "docker run -i -t --rm -v $(pwd):/srv ubuntu /bin/bash".
Not sure what "Production support" really means.
But, Docker is awesome software!
> First, while many organizations have cheerfully ignored our “Do not run in production!” warnings, others have been waiting for a level of product maturity before deploying production workloads. This release’s “1.0” label signifies a level of quality, feature completeness, backward compatibility and API stability to meet enterprise IT standards. In addition, to provide a full solution for using Docker in production we’re also delivering complete documentation, training programs, professional services, and enterprise support.
I was just a little skeptical since there wasn't an official blog post to go with it
For example, when working you end up with a lot of old items or layers hanging around, and its not clear how to remove or collapse them. I've seen posts complaining about this.
Also, on some commands it outputs the the id as the first column, and sometimes the fourth, or other column. This will make script pipelines harder to write.
I've not gotten far enough to worry about logs, but I hear there is an issue with them.
It is very true I may not be using it correctly... or there could have been lots of stuff still to be done for a .12 release?
Possibly I'm reaching here - but I thought I'd ask anyway, I don't know much about Docker.
Windows has App-V (http://en.wikipedia.org/wiki/Microsoft_Application_Virtualiz...) which seems like a similar technology (I believe Office 365 uses this). Note that Office plug-ins are specifically called out as something that is problematic with this tech.
When you start a container and forget to forward a port into it you can't do it at a later time. This is braindead.
The LXC driver is _not_ the default anymore as of ~0.9, Docker switched to pluggable execution and storage drivers at that time and shortly after, started using the new "native" driver for execution (which does not use LXC at all, but uses the underlying kernel services directly, like namespacing and cgroups, that are also used by LXC).
The execution driver LXC provides the same experience from before the pluggable driver model happened, and it's still available but not the default anymore.
Am I being pedantic, is my information wrong, or has this just not been heavily publicized before (maybe it's actually a very small distinction and I should stop repeating it?)
But yes, some framework is there.
Just as you might not trust Docker's native driver more than LXC, if you most cynically regarded LXC as an "example of a failed marketing experiment" and then Docker as "the first notable consumer of cgroups and linux kernel namespacing, and certainly too new yet to trust with anything important" -- but there are plenty of high-profile consumers of the FreeBSD jails and it is trusted by many to keep processes separate.
I mentioned emulation and binary compatibility, maybe because it's the thing that's missing in many other cases that wind up being responsible for alternative platform support getting back-burnered and second-classed in Docker community.
Where there might be lots of people who want to use Docker on i386, it's not supported probably most of all because it would lead to a bad user experience and fragmentation. Binary images are all built for amd64 linux at this point.
I bypassed the warnings, I use it, I'm not bothered that I have to rebuild for my specific architecture. The docker users at large can still use my images, and maybe I'm a bad citizen for pushing images compiled this way, but it works.
Similarly there are probably a lot of people testing Docker on ARM, but they can't use any of the pre-built containers that are provided by the majority docker community, tested and supported. Their architecture is even farther out from being supported. They have the drawback that they can neither receive blood from "O" nor give to "AB" if you'll indulge the analogy of images to blood types, they have one of those weird blood types that doesn't even register in the statistics with a reasonably sized sample. Anemic, maybe.
If you had a jails execution driver, you certainly don't have to use Linux processes or binary emulation, you can build your containers with a BSD base and BSD toolchain. The point is that in theory, you probably could skip that step and get working with existing binary images built from Linux distributions.
It would quickly be a better "alternative platform" than any of the other existing contenders (different architectures) who you could already see in the docker ecosystem at this point; it could probably run many existing container images out of the box, on a FreeBSD host with amd64 architecture.
The contained software really shouldn't care what execution driver keeps it isolated, at least in theory. In practice someone with more knowledge than me can probably say what a bad idea this is, maybe because either "congrats, you just invented Virtualization" or "wrong, jails won't keep those processes separated."
(someone should look at porting the netbsd amd64 linux emulation to freebsd...)
Every time I use Docker it leaves a grin on my face.
I got sidetracked by other things and haven't gotten back to it, but if I do and it's still as bad as it was, I don't think I'll want to continue on that path.
Fundamentally (1) LVM2 LVs are capable of single-depth snapshots only, forcing hacks like an actual full blockstore copy if code expects duplication. (2) Docker was written primarily against aufs which provides rapid arbitrary-depth snapshots.
I suspect this is the heart of the issue... ie. Docker expects features this storage subsystem wasn't designed to support.
Rest assured we're working on it and it's a priority.