
Docker 1.0 - dominotw
http://blog.docker.com/2014/06/its-here-docker-1-0/
======
mattjaynes
Docker is an amazing tool, but there are quite a few misconceptions about it.

Nearly all the articles and examples seem super easy, but it's quite a
challenge to use in an actual multi-host production environment supporting
mission-critical systems.

This can send the wrong message to beginners and send newbies down the wrong
path, which ends up as a disservice to Docker when they hit the complexities
and become disillusioned with it.

To address those misconceptions, I've just posted the article "Docker
Misconceptions": [https://devopsu.com/blog/docker-
misconceptions/](https://devopsu.com/blog/docker-misconceptions/)

I spent about a month working on setting up Docker for production scenarios
and reading everything I could on it (all the docs, courses, and well over 100
articles - see Docker Weekly for a wealth of resources). My post covers the
main misconceptions I see popping up and also some advice for simplifying
Docker use if you do decide to use it for multi-host production.

I'm sure I've probably missed some things, so please chime in if I've missed
something important.

~~~
wpietri
Thanks, that's exactly the sort of thing I was hoping to learn from the HN
comments.

In the post you link above, it seems like you're mainly comparing Docker to
traditional, non-virtualized setups. Do you (or does anybody) have thoughts on
comparing it to virtualized environments? I recently experimented with Xen as
a way to isolate services, and it seemed awfully heavy; does Docker end up
being a lighter burden?

~~~
Alupis
My gripes:

* half git-syntax/half package manager-syntax is cumbersome to me.

* Inability to compile Docker from source without using the Docker Project's provided "black box" binary Docker.

* No formal explanation of how to create your own base image from scratch (ie. not using a Docker Project provided base image).

* Defaults to storing images in the Docker "cloud" repo. (would have preferred a git-like setup where pushing my images to a repo of my choice was more "normal")

* GO Lang is still pretty obscure for most developers, and an interesting choice given the language's youth and likely-hood to change rapidly as it matures.

* Docker is built on-top of existing Linux technologies, mainly LXC, making it more-or-less disposable in the future (as someone else figures out how to abstract/manage LXC better)

* Docker is mainly built by Dotcloud - a for-profit company. Dotcloud has been very generous in their effort... but, being for-profit, what is their take out of it? (It can't be "just for the goodness of Linux").

~~~
lolo_
> * GO Lang is still pretty obscure for most developers, and an interesting
> choice given the language's youth and likely-hood to change rapidly as it
> matures.

The go authors have committed to keeping major versions stable since v1 and
are unlikely to _massively_ change anything.

I'm sure much of google's infrastructure is coming to rely on go (and docker!)
so they will be as unhappy about such drastic change as others.

~~~
Alupis
Fair enough point about GO Lang, however, should note Docker as a project was
started way back when GO's future was... perhaps iffy. Maybe good foresight on
the Docker project regarding GO Lang.

However, this doesn't excuse the remaining gripes listed above.

~~~
nl
_Fair enough point about GO Lang, however, should note Docker as a project was
started way back when GO 's future was... perhaps iffy._

You mean 14 months ago?

------
mmerickel
I'm glad to see docker stabilizing but it's very disappointing to not see any
changes addressing the logging support that was promised in 1.0
([https://news.ycombinator.com/item?id=7712138](https://news.ycombinator.com/item?id=7712138)
reply by shykes). The infinitely increasing-in-size and unconfigurable
logfiles generated by containers in docker are basically a dealbreaker unless
you bake your own logging solution into each container and avoid stdin/stdout.

~~~
derefr
I wonder if a very simple solution would be enough: with a one-line change,
the logs could be created as named pipes instead of files. Then you could just
write some upstart scripts/systemd units that do

    
    
        cat /run/docker/mypipe >/dev/log
    

And you'd be good.

\---

I'm not sure this is the right approach, though. In my opinion, docker
shouldn't log at all—containers are unix processes, and Unix processes log to
ephemeral stream-sink endpoints (think AWS SNS topics) called stdout and
stderr. To collect and persist those logs, the Unix Way is to use shell
redirection to "attach" those endpoints to subscribers.

Now, in Docker's daemonized run mode, these pipes are just hanging loose. But
docker provides the "docker attach" command precisely so you can reattach them
to something (or as many somethings) as you like. All you have to do if to get
sensible logging if you're using e.g. Upstart is to create a service that runs
"docker attach <mycontainer>": the logs will flow from the container's
stdout/stderr to the service's stdout/stderr, and Upstart will catch them and
handle them sensibly from there.

So, having done something like that, the important bit would actually be
making sure docker _doesn 't_ do its own broken logging at all, except in the
case of test containers that will never be attached to. Actually, this also
cuts across the use (or lack thereof) of the --rm flag: it'd make sense for
containers started with --rm to not log. (The generalization of the --rm
semantics would seem to put such containers in the same category as e.g. EC2
instance-store instances.)

~~~
mmerickel
In general you ought to just be able to "docker run" in the foreground from
your systemd script, and then stdin/stdout will go into the journal where you
can handle things like any other file. This works now, except that the logs
are also duplicated into /var/lib/docker.

------
davidcelis
Unfortunately, installing the Docker client is currently broken, at least via
brew:
[https://github.com/dotcloud/docker/issues/6256](https://github.com/dotcloud/docker/issues/6256)
— This made it into 1.0

The CHANGELOG states Production support as the 1.0 feature, but how am I
supposed to be confident using Docker 1.0 in production if I can't use the
client from my own machine? Or is there some other preferred install method?

~~~
ClifReeder
In fairness, nobody is using brew to install something on their production
environment.

~~~
Goopplesoft
Really? Never read/worked/etc in a os x server environment, but it seems like
brew would be a good package manager for automating installs...

~~~
justincormack
Hardly anyone uses os x server.

------
FooBarWidget
I was hoping that Docker 1.0 would have a 'docker login' or 'docker shell'
command for spawning a shell within the container. In baseimage-docker
([https://github.com/phusion/baseimage-
docker](https://github.com/phusion/baseimage-docker)), we run an SSH daemon so
that people can login to the container for administrative purposes. In the
past, it that approach was criticized because it was also possible to use lxc-
attach, but since Docker 0.8 or 0.9 lxc-attach is not usable anymore because
the default Docker backend no longer uses LXC. The Docker authors promised
they would have a solution in the future, but sadly it didn't make it to 1.0.
I guess we'll continue to use SSH until they have an official solution.

~~~
heavenlyhash
[https://github.com/polydawn/siphon-cli](https://github.com/polydawn/siphon-
cli) might address your needs for now. It's just a tiny little wrapper around
terminals that shuttles the connection over a unix socket... which in the case
of docker, you can put on a mount to access from the host. Author here; I
built it to replace that use case of sshd with something that doesn't have the
overhead of key management. Of course if you're doing something with network
gaps involve, sshd might still be the right answer for you.

------
jacquesm
For those wondering what docker is:

[http://www.docker.com/whatisdocker/](http://www.docker.com/whatisdocker/)

~~~
dpcan
Is there a very specific use case someone could share so I can visualize what
docker really is?

Something more than a hello world.

~~~
jacquesm
Let me try: Say you're going to deploy an open source application that
requires a whole pile of dependencies. Normally you would have two choices:
ship the source with a bunch of readme's and let the recipient figure out how
to install your software and all dependencies and to resolve any conflicts
with other software already on the machine (aka version hell).

Or you could ship a relatively large fully installed VM image of the OS+all
the application parts and dependencies (this may be problematic if your target
is a licensed OS).

Docker gives you something in between.

Think of a chroot'd environment (not that that's what it is, but it shares
some concepts) with all the application code, libs and dependencies pre-
packaged that should run theoretically out of the box.

That way you can have a 'war' like file that contains everything required to
run your app and you're 100% sure that it will just work, rather than to have
to field a million questions about how to make your app run on 'obscure distro
x' or 'obsolete version y'.

Another use-case would be that if you're deploying something distributed
across a large cluster that you can build your docker container on your
desktop box, test it until you're happy and then deploy to your cluster
without having to re-test everything. Since the container wasn't changed it
should work.

It's a way of isolating dependencies between various pieces of software using
one single container to hold them all.

Of course this approach has its own unique set of problems, but it solves
certain things quite elegantly.

I hope that explains it and that if it is wrong in some aspect that someone
will correct me.

~~~
sigil
_Normally you would have two choices: ship the source with a bunch of readme
's and let the recipient figure out how to install your software and all
dependencies and to resolve any conflicts with other software already on the
machine (aka version hell). Or you could ship a VM image._

Option 3: use a package manager.

Now, I realize this doesn't solve the existing software conflict problem you
mention. Nor is it mutually exclusive with shipping an image: you could always
construct an image by installing packages into it.

I've always been a little wary of the "ship a golden image" idea. How do
Docker images get updated? What if there's a security issue with software in
the Docker image -- can my package manager automatically apply security
updates? How do I push other upgrades to the client once they have a Docker
instance -- do they need a whole new image from me? Package managers have
solved these problems, and I hope we're not reinventing those solutions in the
Docker world.

If you're familiar with Docker, please enlighten me on these points.

~~~
grosskur
My understanding: If you provide an image then, yes, you're supposed to build
a new image every time there's an OS security update. Users of docker
containers aren't supposed to run 'apt-get upgrade' themselves inside
containers. In fact, people build who build app images on top of base images
aren't even supposed to run 'apt-get upgrade' in their Dockerfile---it's the
responsibility of the base image to be up-to-date. See:

[http://crosbymichael.com/dockerfile-best-practices-
take-2.ht...](http://crosbymichael.com/dockerfile-best-practices-take-2.html)

This does seem to get a bit cumbersome. I'm at DockerCon today and Fabio Kung
mentioned in his talk that this is one difference from Heroku's container
platform---they provide the base image and can update it _without_ requiring
you to rebuild your application slug. He said there's been some discussion of
a possible "docker rebase" command that would produce new images by replacing
lower-level layers while keeping higher-level layers the same.

~~~
voltagex_
docker rebase sounds like an awesome idea. Is there a mailing list thread or
GitHub issue?

------
Dorian-Marie
Here is the CHANGELOG:
[https://github.com/dotcloud/docker/blob/master/CHANGELOG.md](https://github.com/dotcloud/docker/blob/master/CHANGELOG.md)

    
    
        1.0.0 (2014-06-09)
        Notable features since 0.12.0
        * Production support

------
paulannesley
Note that Docker 1.0 (and the brief 0.12.0) changed its TCP port from 4243 to
2375. If you use a remote client (like `brew install docker`) and have
`DOCKER_HOST=4243` in your environment, you'll have to update it.

To ease this pain across a large team, I'm considering using
netfilter/iptables or a TCP proxy to make the old port available in our VM
environment during transition.

[https://github.com/dotcloud/docker/commit/5febba93babcf8c4b0...](https://github.com/dotcloud/docker/commit/5febba93babcf8c4b01862e88b6f6e11a1532bc8)

------
rmanocha
Do we know yet if mounting a directory from host as a data volume is available
in Mac OS X now? I think something along these lines was promised (not able to
find the source now) - it's the only thing blocking me from using docker as my
everyday dev environment.

~~~
evancordell
I've been working on getting a dev environment set up with docker over the
weekend, and I finally have something I'm happy with.

I don't have a nice blog post to show you, but the basic idea is:

\- Use the CoreOS Vagrantfile (+ extra files like user_data and any service
unit files)

\- Share your local directory with the vagrant box with
config.vm.synced_folder (just uncomment a line in the Vagrantfile)

\- set the DOCKER_HOST and FLEETCTL_TUNNEL env variables so that the local
tools work transparently with the vagrant box.

\- Then you have the option of running a docker image with -v and sharing the
vagrant directory with the docker container, or

\- You can dive into CoreOS stuff and write a unit file for your service, and
that will share the directory with -v

Now local files sync into the docker container, and if the server is
configured to reload on file changes you're good to go.

~~~
rmanocha
Thanks a lot, I'll give this a shot tonight. Not too familiar with CoreOS yet;
I was hoping that I'd simply be able to mount a volume similar to how I might
do it in Linux. I understand the problems with doing that though...

------
odonnellryan
See here:
[https://github.com/dotcloud/docker/blob/v1.0.0/CHANGELOG.md](https://github.com/dotcloud/docker/blob/v1.0.0/CHANGELOG.md)

Not sure what "Production support" really means.

But, Docker is awesome software!

~~~
derefr
Quote from the blog post:

> First, while many organizations have cheerfully ignored our “Do not run in
> production!” warnings, others have been waiting for a level of product
> maturity before deploying production workloads. This release’s “1.0” label
> signifies a level of quality, feature completeness, backward compatibility
> and API stability to meet enterprise IT standards. In addition, to provide a
> full solution for using Docker in production we’re also delivering complete
> documentation, training programs, professional services, and enterprise
> support.

------
sysexit
A release that seems somewhat rushed, and a Red Hat media event tomorrow. I
wonder if the two are related.

~~~
ritchiea
Today is the first day of docker con [1], that is surely the reason for the
announcement.

1\. [http://www.dockercon.com/](http://www.dockercon.com/)

------
MoOmer
Congrats to the team - I'm wrapping up a deployment on 0.11 now; and, though
there are definitely some pain points, docker wasn't even near the most
painful part of the process.

------
soccerdave
Version was changed to 1.0.0-dev in Github. I think this was accidental that
the version said 1.0.0.

~~~
opendais
Docker .11 was also Docker RC 1 for 1.0, .12 was RC 2. So I think it may not
be accidental.

~~~
soccerdave
Yep, I was wrong, [http://blog.docker.com/2014/06/its-here-
docker-1-0/](http://blog.docker.com/2014/06/its-here-docker-1-0/)

I was just a little skeptical since there wasn't an official blog post to go
with it

------
mmcclure
This is awesome! Now CoreOS just needs to follow suit and hit 1.0...

~~~
tonyhb
In their last article they said they're about a week behind, so by next week
we should be production ready on all fronts. Cannot wait.

------
marklit
You'd think IPv4 connectivity with containers would be high up the priority
list to get working:
[https://github.com/dotcloud/docker/issues/2174](https://github.com/dotcloud/docker/issues/2174)

~~~
jacquesm
It looks like it is pretty high on the priority list, the last update in that
thread is on Friday.

------
mixmastamyk
Docker has a lot of potential and I recently really dug into it. Seems like
there are still some rough edges though.

For example, when working you end up with a lot of old items or layers hanging
around, and its not clear how to remove or collapse them. I've seen posts
complaining about this.

Also, on some commands it outputs the the id as the first column, and
sometimes the fourth, or other column. This will make script pipelines harder
to write.

I've not gotten far enough to worry about logs, but I hear there is an issue
with them.

It is very true I may not be using it correctly... or there could have been
lots of stuff still to be done for a .12 release?

~~~
brianxq3
I think most (all?) of the commands can be performed over the API with JSON
coming back, so the order of output shouldn't matter too much.

~~~
sergiosgc
JSON over HTTP is not an alternative to a good CLI interface. As such, your
comment, although true and happily so, is a non-sequitur.

------
x_soda
Can someone explain to me the practical differences between vagrant and
Docker? I know they're implemented in quite different ways, but in my utterly
non-devops mind they seem to achieve the same thing. Wrong?

~~~
billmalarky
Scroll down to the bottom of this page to see Docker VS VM's.

[https://www.docker.com/whatisdocker/](https://www.docker.com/whatisdocker/)

------
CSDude
At last, it was already production ready, mostly bug free and fast enough, but
having 1.0 release makes one more comfortable when they highly depend on it.

------
philbarr
I am right in thinking that Docker can virtualize any environment, as long as
that environment is Linux, right? Because I have an office plugin I need to
test against various different version of office and content management
repository and so on and it feels like this would be ideal.

Possibly I'm reaching here - but I thought I'd ask anyway, I don't know much
about Docker.

~~~
nickstinemates
The Microsoft Azure team is talking at DockeCon tomorrow. It's just the tip of
the iceberg in terms of our collaboration together. More to come. I'm
extremely optimistic.

------
tomerbd
What is the difference between this and rpm+puppet with rpm+puppet I have no
problems installing my apps with my dependencies to any server and best of all
as its the developers who create the rpms+puppets they hand it as a building
block to production who install it in a few seconds on any machine.

------
jtchang
Gripe:

When you start a container and forget to forward a port into it you can't do
it at a later time. This is braindead.

------
tachion
Sadly there is no new things to support FreeBSD Jails, a competitive (and much
more matured) container technology than Docker's default LXC (despite earliers
mentions of plans for supporting FreeBSD in version 1.0)...

~~~
yebyen
I think that what you're asking for (a Jails execution driver) is really _not_
so far out of bounds, mostly given that FreeBSD has really good support for
binary compatibility and Linux emulation, so FreeBSD is actually the _one_
good platform to try this with (one of the best candidate platforms?) but one
thing, that people all keep repeating this about LXC makes me grumpy...

The LXC driver is _not_ the default anymore as of ~0.9, Docker switched to
pluggable execution and storage drivers at that time and shortly after,
started using the new "native" driver for execution (which does not use LXC at
all, but uses the underlying kernel services directly, like namespacing and
cgroups, that are also used by LXC).

The execution driver LXC provides the same experience from before the
pluggable driver model happened, and it's still available but not the default
anymore.

Am I being pedantic, is my information wrong, or has this just not been
heavily publicized before (maybe it's actually a very small distinction and I
should stop repeating it?)

~~~
justincormack
I don't think the binary compatibility and Linux emulation will be anything
useful - there is no emulation of containers, and no one would want to run a
Linux process in their container.

But yes, some framework is there.

~~~
yebyen
I think the point would be not to use linux containers or any abstraction
around the parts that fuel LXC, but to use BSD Jails on the idea that they are
"battle tested"

Just as you might not trust Docker's native driver more than LXC, if you most
cynically regarded LXC as an "example of a failed marketing experiment" and
then Docker as "the first notable consumer of cgroups and linux kernel
namespacing, and certainly too new yet to trust with anything important" \--
but there are plenty of high-profile consumers of the FreeBSD jails and it is
trusted by many to keep processes separate.

I mentioned emulation and binary compatibility, maybe because it's the thing
that's missing in many other cases that wind up being responsible for
alternative platform support getting back-burnered and second-classed in
Docker community.

Where there might be lots of people who want to use Docker on i386, it's not
supported probably most of all because it would lead to a bad user experience
and fragmentation. Binary images are all built for amd64 linux at this point.

I bypassed the warnings, I use it, I'm not bothered that I have to rebuild for
my specific architecture. The docker users at large can still use my images,
and maybe I'm a bad citizen for pushing images compiled this way, but it
works.

Similarly there are probably a lot of people testing Docker on ARM, but they
can't use any of the pre-built containers that are provided by the majority
docker community, tested and supported. Their architecture is even farther out
from being supported. They have the drawback that they can neither receive
blood from "O" nor give to "AB" if you'll indulge the analogy of images to
blood types, they have one of those weird blood types that doesn't even
register in the statistics with a reasonably sized sample. Anemic, maybe.

If you had a jails execution driver, you certainly don't have to use Linux
processes or binary emulation, you can build your containers with a BSD base
and BSD toolchain. The point is that in theory, you probably could skip that
step and get working with existing binary images built from Linux
distributions.

It would quickly be a better "alternative platform" than any of the other
existing contenders (different architectures) who you could already see in the
docker ecosystem at this point; it could probably run many existing container
images out of the box, on a FreeBSD host with amd64 architecture.

The contained software really shouldn't care what execution driver keeps it
isolated, at least in theory. In practice someone with more knowledge than me
can probably say what a bad idea this is, maybe because either "congrats, you
just invented Virtualization" or "wrong, jails won't keep those processes
separated."

~~~
justincormack
Isn't the FreeBSD Linux emulation still 32 bit only? I can't find any
reference saying that changed. In which case it there isn't any way compat
would work. I think docker will give in and support multiple repositories at
some point, as all these osx people are going to want native stuff, and the
arm (and arm64) people will want something, and IBM will have a big docker on
Power8 push and so on... Linux amd64 will do doubt be the biggest platform.

(someone should look at porting the netbsd amd64 linux emulation to
freebsd...)

------
jradd
Join #docker on IRC for anybody struggling with Docker. Everybody is very
helpful on there. And I have learned a lot just by supporting others, and
asking questions myself.

------
curiousDog
Looks very promising. Too bad their jobs page says they only want engineers
with Go experience. Why does language matter?

~~~
akgerber
Probably because they're developing very fast presently, and a fashionable-
enough company that they get a lot of good applicants and can afford to be
choosy with hiring.

------
kolev
Great news! I just wonder why Docker is getting so much attention and Google's
lmctfy doesn't get any?

~~~
shykes
Docker and lmctfy have largely joined forces - Rohit and Victor are now core
maintainers of Docker and kicking ass contributing all sorts of low-level
system awesomeness :)

[https://github.com/dotcloud/docker/blob/f4b60a385cbaae045674...](https://github.com/dotcloud/docker/blob/f4b60a385cbaae045674146644294e9c55129b3f/pkg/libcontainer/MAINTAINERS)

~~~
kolev
That explains it, thanks!

------
bhhaskin
I haven't given docker a try yet, but I think I might have to give it a go.

------
1qaz2wsx3edc
Congrats on the 1.0 Docker!

------
turnip1979
Anyone know if there is a livestream or videos available from Dockercon?

~~~
kylemathews
No livestream but they'll be posting videos after the conference.

------
muloka
Congrats to the Docker team on 1.0!

Every time I use Docker it leaves a grin on my face.

~~~
nickstinemates
Thank YOU. Users are amazing. Use more. Keep it up!

------
lavalla0
Is the devicemapper storage driver ready to use in production?

~~~
heyimwill
Why wouldn't it be?

~~~
nknighthb
When I was working on a docker-based build/test/deploy solution a few months
ago, its performance was so anemic I wanted to tear (what's left of) my hair
out, especially during a development/testing cycle.

I got sidetracked by other things and haven't gotten back to it, but if I do
and it's still as bad as it was, I don't think I'll want to continue on that
path.

~~~
contingencies
I haven't seen the code but have played with LVM2 LVs / device mapper myself
on my own containers-based infrastructure and also seen performance issues in
some situations.

Fundamentally (1) LVM2 LVs are capable of single-depth snapshots only, forcing
hacks like an actual full blockstore copy if code expects duplication. (2)
Docker was written primarily against _aufs_ which provides rapid arbitrary-
depth snapshots.

I suspect this is the heart of the issue... ie. Docker expects features this
storage subsystem wasn't designed to support.

------
ilaksh
Is there a way to add a Docker Webhook programmatically?

------
general_failure
Congrats! I would love to see user namespace support.

~~~
nickstinemates
"It's complicated" \- crosbymichael, my favorite maintainer of all time.

Rest assured we're working on it and it's a priority.

~~~
shykes
Yes, user namespaces are coming soon.

------
rsanaie
Great work!

------
dang
Url changed from
[https://github.com/dotcloud/docker/releases/tag/v1.0.0](https://github.com/dotcloud/docker/releases/tag/v1.0.0),
since there were two threads on the front page and this one has the more
extensive discussion.

