I also had a incident with Docker in 0.6, when creating a new container it used to traverse all the containers metadata and when the container count reaches thousands it froze my server and I had to manually reboot it on 3AM, but hey that's okay, it can be fixed, I reported it on Github and it does not happen anymore. It is still useful to me, and still necessary.
(copy) "However it's important to note that this article focuses on the daily, long term usage of Docker, both locally and in production"
You should consider doing your students a favor and not tie them into a single for-profit company's technology (most will go on to use whatever you have taught them since they will have come to be comfortable with it during their studies).
Consider using one of the App Container Specification's implementations -- there are already 3 prominent ones, all adhere to the standard which means their images are interchangeable and compatible with each other.
The standard is community-driven, and stable at this time. It could even become a project to implement your own implementation for your school, teaching your students not only a ton about Linux, but the principals of containerization in the process.
Flawed? sure.. useless hype?? yeah well are all the people loving it just stupid.. or is it just placebo?
The author makes a lot of conjecture with very, very little backing.
I love docker. I'm a programmer more than a systems engineer. I've used Linux as my sole computing environment for 6+ years. I've deployed countless LAMP stacks; and countable Haskell/postgres stacks.
For both having an extremely portable development/building environment; and dead-simple disbursement of binaries-with-prereqs, Docker has been INCREDIBLY useful to me.
For actual deployment of single-server apps; it might be a bit more trouble than it's worth, in some cases. I have a couple places where I develop and build in Docker; but actually deploy "raw"; because it is easier/fine.
But when you start considering Coreos clusters and docker containers to utilize them; deployment again is made congitively simpler (to us mere mortals) thinking in terms of containers.
I guess this is click-bait; but even as a passive user of Docker, I find it quite offensive; and not well-grounded.
I opened the article with an open mind; thinking someone smarter than me knew something terrible about docker that was going to bite me in the ass some day.. only to instead get the impression of, either being trolled, or that the author is my favorite type of neck-beard elitist.
I'm sorry to hear the article gave you the impression of being trolled, as I certainly did not intend this. If there are specific areas which you feel I did not justify, rather than an overall dismissal of the argument being put forward, then please call me out on it. If you read my other comments, you will see I am a reasonable person and happy to admit when I'm wrong. Though despite these corrections, I'm standing strongly by my conclusion.
PS) I have recently grown a neck beard, but it's going soon, itching the crap out of me.
For me, Docker "clicked" when I took a legacy PHP webapp, and was able to get the whole thing running in a Dockerized dev environment by setting some links and pulling down a Postgres and PHP container - which I can now launch into with a tmux script which lets me see the debug output of both while I'm developing, live updating while I edit and tinker in Webstorm.
That simply wasn't possible before - not nearly as easily.
> the author is my favorite type of neck-beard elitist.
Again, Seriously? The guy has an issue with docker, which is grounds for you to go all ad-hominem on him? What is this, Reddit?
At the very least it let me iterate on system configuration / installation procedures more quickly. I have learned a lot just due to tweaking Dockerfiles and rebuilding them. Virtual machines are much slower in this regard.
In Data Science Docker is becoming revolutionary. People tried distributing virtual machines to let others reproduce their work. Dockerfiles are much more reproducible, and don't need a few gigabyte of seperately hosted VM images. Also these VM images usually contain tons of undocumented "state", whereas a Dockerfile is easier to reverse engineer and ultimately very reproducible.
You can include Dockerfiles in all your projects. Other developers can then go in and get started with a minimal amount of guesswork. Turns out "sane" development environments, which probably means one supposedly optimal configuration/setup/framework for all your projects, is the exception rather than then the norm.
It's my opinion that Docker images from Docker Hub are less repeatable than your average VM; as pointed out in the article there's nothing stopping someone from re-using a tag, and the contents of the VM depend entirely upon on what the upstream image creators felt was necessary (which can include things like grabbing config files from the internet in order to run a service).
Although I agree this approach is absolutely flawed, and anyone doing so needs to re-evaluate their workflow, I feel this comment is more an argument of doing things properly, rather than a positive of Docker. Everything you mentioned above can be easily achieved by using Vagrant and a provisioner. But as discussed in other comments, the Docker ecosystem helps you get started quicker.
You seem to assume that everyone should be doing everything the "right way". Turns out they don't. I found it easier to deal with stuff not being done "perfectly" than to try and argue other developers into doing stuff "right".
As opposed to those who prefer imprecise, flawed and ugly solutions? Jesus dude, could you possibly come up with a more arrogant sentence?
Have you ever looked at the source code of a dependant library, followed by an overpowering and compelling urge to write your own from scratch? Suppressing the urge to rage quit because your tools are fighting against you, not with you. For me, it is a constant daily fight to force myself not to do things "properly", for it's mostly an unrealistic and unachievable goal.
Believe me when I say that being a perfectionist is nothing to be arrogant about, and despite it being one of my major strengths in some ways, it's also one of my biggest flaws.
(I'm taking it on faith that you're not trolling, and have taken time to give an honest reply)
It seems you are making an argument not for Docker itself, but for containers in Linux in general. To that point, something like the App Container Specification standard and one of it's implementations would suit your needs just as well, if not better. There are already 3 prominent implementations of App Container Specification, each doing their thing differently but all compatible and inter-changeable with each other and their images.
Creating your own base image is really the only way to ensure the base is trustable.
And to that end -- App Container Specification has image discovery and downloading covered
But honestly how hard do you think it is to integrate secure builds into a docker system? I would just stand up a private docker repository and lock the build system to that, problem solved. Or docker could roll out an update to leverage the existing namespaces and combine that with a user controlled whitelist of public key & namespace pairs. The reason docker has enjoyed this much success is because they understand sharing is #1.
Could, but haven't.
> The reason docker has enjoyed this much success is because they understand sharing is #1.
Except they have fought tooth and nail against a standardized specification until App Container Specification was released. Docker isn't about "sharing", it's about vendor lock-in.
Umm ok? Judgement, jumping to conclusions. Take out some of that kind of tone and you have some legitimate criticisms of docker that might be taken more seriously.
There are some real warts in the docker core & community, and a high barrier of entry for multi-node deployments and isolated networks, but on the flipside, deterministic image build & deployment is a big win compared to many other ways of doing this stuff (at scale).
IMO, one of the biggest positives that Docker-like provisioning encourages is clustered-by-default architectures. When you build around failure by assuming nodes come and go (as easily as Docker makes it) your platform availability is likely to be more resilient to some types of failure.
Have you tried Weave?  It gives you an isolated multi-node network for your containers. I'd be interested to know if you still think it has a high barrier of entry.
Disclosure: I work on Weave.
But - despite agreeing with your words - I wouldn't put them together to reach the same conclusion. The container model with managed images seems to be a big step forwards vs the VM model: it encourages having immutable 'machines' that are disposable, which works very well for cloud architectures.
I would check out the kubernetes project and where it is heading: containers orchestrated across a cluster of machines, with seamless/automatic networking between the containers.
Other people have raised a similar argument of Docker ecosystem, which is a fair point and I'll be amending the article to reflect this.
So basically, ocker adds nothing to these other solutions except complication, bugs, etc. (which is also my experience).
Moreover, and to push it a little further, if anything, it's a better idea to call some container library so that you use namespace for your very use case, inside a VM. That VM being an image that you manage and deploy of course (ie that you dont tweak when its running unless its for debugging)
It's certainly possible to do all of the same things with virtual machines, but the difference is the process is really cumbersome. No one has made an equivalent service of Docker Hub for virtual machine images and snapshot deltas.
I work at a company that leverages Hadoop, and on each developer machine we each have a baby Hadoop cluster (3 nodes) running in VMs at the moment, in addition to a master controller. Hence, at any given time, my machine is actively running 5 OS's (if we include the native OS) and all their associated background tasks. My computer generally idles at 25% CPU usage. Keeps my legs warm in the winter.
We're finally going to dockerize the project and I couldn't be happier. Granted, Mac OSX doesn't natively support the LXC's docker uses (c'mon Apple!), so we'll still be running 1 VM, but that's a huge improvement over our existing implementation. My CPU wouldn't be idling nearly as high, I'm going to recover a ton of RAM, and the boot sequence is going to be substantially faster.
Being that these are developer machines, and simply understanding our use case, we're not worried about malicious tenants breaking out of their environments. It's just not a factor in our situation.
If anything I feel the "article" was a waste of time. I thought I was going to read a great summary on the state of Docker, and instead I got this unexpectedly aggressive piece from developer that couldn't conceive how to fit this particular tool into his toolbelt.
If you look further into the article, you will see that I do give fair mention that Docker/LXC can result in faster startup times and lower memory usage.
The tone of the article is an accurate representation of my feelings towards Docker, writing anything else would have been a lie. I'm sorry you feel like the article was a waste of time, but I appreciate your honesty! Despite these corrections, I still stand by my conclusion, however it's been quite insightful to have peer review from a wide audience and so far 99.9% of the feedback has been constructive.
I just wish it were easier for people to read the contents on the label. Right now everyone thinks it is good for everyone. It isn't, and there are some sharp edges that require RTFM'ing.
I feel like these sorts of usage cases are the sweet spot right now. I have zero desire to run something like Postgres in Docker in a production environment. On the flipside, a Postgres Docker container is great for a local dev environment.
(paste from reddit) I agree that the Docker ecosystem makes getting started easier, and my argument of uselessness is focused on the long term usage of Docker, rather than the immediate "quick start" gains. Never the less, I'll amend the article with reference to this, as you make a fair comment
So I find this article to be the same as the hype around Docker, useful information surrounded by hyperbole that ruins it?
I don't need a full VM usually, and being just local dev the security isn't an issue, but used Vagrant before Docker for this.
Is there anything else similar?
Would you like it if I told people your blog post was useless because you buried your thesis in the third-last paragraph and led off with an either astounding or false assertion that you tested docker (which you had previously disliked) by putting it in production for six months? That sort of needs some explanation and story telling. If Docker induced so much bile in you, then how and why did you end up stuck using it in production for six months?
You are right that I've failed to justify my testing, other than stating that it was used in prod for 6 months, and in hindsight this was a mistake. I'll amend the article to reflect this, naturally it will be of limited use without reproducible code snippets, but it will at least complete the story.
As for polishing, I'd spent around 18 hours writing this article, with 4 full cleanups. I had considered putting a job out on fiverr to perfect the grammar, but that felt wrong.
What I like the most personally is how easy it is to install and experiment with 3d party tools. You want an elastic search stack? In one command you have a webserver hosting Kibana with Elastic Search and Logstash properly configured on your local. Jenkins, Elastic Search, Redis, Postgres, etc: they all have their dockerfiles and can be installed as one liners. Removing them is equally as easy.
Oh and I don't know why it is written that running a Docker registry is "extremely complex". Just like any Dockerized app it is a one-liner.
This new ease-of-install just by itself is worth of my gratitude to the guys that build it.
Also remember that Docker is not a container deployment system, and afaik, they don't advertise themselves as such (though they also don't make much effort to explain this explicitly either)
The challenges of doing this are less than many claim - the trick is to use a CM so you have a repeatable build process.
They are good at bringing systems to a known state; but not as good for cleaning systems up.
If not watched carefully, all of the cruft in your /etc directory will be ported to your CM because someone needs it.
>either useless or poorly implemented, and it's primary
>benefits can be easily achieved using namespaces directly.
any tutorial ? For me the docker is very easy.
and there are some new alternatives:
- LXD : https://lists.linuxcontainers.org/pipermail/lxc-devel/2014-N...
- Rocket: https://coreos.com/blog/rocket/
- Flockport : http://www.flockport.com/faqs/
- Spoon : https://spoon.net/docs/getting-started/spoon-and-docker
Each will have their own workflow, but be interchangeable and compatible with images.
Just like few years ago, everyone here on the front page is talking about nosql and seems like everyday, there's a new webscale nosql database being born...but now, I often see more people here inclined toward traditional technologies such as Postgres as it has been greatly improved. (Yes, a lot of aged software are still improving, e.g. MySQL, Apache, PHP or even Perl5)
I am not saying Docker is just a hype, I also don't think it is completely unnecessary, but it is not for me to solve my immediate problems and therefore I would rather review Docker..maybe few years later.
Say I have my python app, how do I transform that into a working container? If a Dockerfile is flawed, whats better?
For production I'd use kvm or vmware.
Lastly, you might consider systemd-nspawn.
It has gained snapshotting recently (on top of btrfs)
Docker is not a silver bullet, and it is definitely being over-hyped right now. But it does have some great usage cases, and it has got a lot of people working together on re-thinking how we build and deploy applications. I think you'd be more likely to bring about positive change by adjusting tone and trying to point out some usage cases where Docker excels. I can't speak for everyone else, but I'm a lot more likely to take someone to heart who shows some balance while criticizing.
Unless you just wanted to get lots of page views with an hyper-critical (though well-researched) article. If that is the case, carry on and disregard this.
However you're quite right to point out that this article fails to represent the positive impact of the Docker ecosystem, something which several other people have also commented. I'm going to amend the article to reflect this.
I feel like if you're going to criticize running your own registry you really need to devote more time to it. You can get an S3-backed, distributed private registry running in about 5 minutes using their provided image. It's really, really easy.
> Lets say you want to build multiple images of a single repo, for example a second image which contains debugging tools, but both using the same base requirements. Docker does not support this.
Of course it does. It appears that it doesn't support it the way you think it should, but to say that you can't do it is misleading. A base image, and two images that pull from it with the different requirements will solve the problem. You apparently don't like that solution, but that is not the same thing as not having a solution.
> there is no ability to extend a Dockerfile
Yeah, this would be nice. Maybe they will add it. But it is hardly... not even close to... a make-or-break feature. Honestly I think you might just need to refactor your stuff, or perhaps Docker just isn't a fit for what you're doing.
> using sub directories will break build context and prevent you using ADD/COPY
You mean if you include a bunch of stuff in subdirectories that you don't want uploaded to the demon. Again, man, not even close to make or break. You really need to log gigabytes to a subdirectory in your build context? There's _no other way_ you could set that up? We create gigs of logs too, but most of them are events that go to logstash and get indexed into ES. Our file-based logs go to mount points outside the container. We do have images we build using context, where we ADD or COPY multi-gigabyte static data files. Seems to work fine.
> and you cannot use env vars at build time to conditionally change instructions
No, you can't. I'm not sure I would want to. I like the fact that the Dockerfile is a declarative and static description of the dependencies for a deployment. I don't think I want to have to debug conditional evaluation at build time. There are other ways to solve those problems, like refactoring your images.
> Our hacky workaround was to create a base image, two environment specific images and some Makefile automation which involved renaming and sed replacement. There are also some unexpected "features" which lead to env $HOME disappearing, resulting in unhelpful error messages. Absolutely disgusting.
First of all, what exactly is hacky about having a base image and two environment-specific images? I don't know what sort of makefile automation you're talking about, but we do some environment specific sed manipulation of configs at build time, and in some cases at container launch time. Sometimes that makes more sense than having two different versions of the container just to have a very slight change to the config.
Secondly... absolutely disgusting? Is that the sort of language you regularly use in technical writing? Oh, hey, look at the third paragraph: "If you expect anything positive from Docker, or its maintainers, then you're shit outta luck." I guess it is. The strike-out font was a nice touch, man. "I don't really mean this, but you can't help reading it!" Nobody's ever done that before.
> These problems are caused by the poor architectural design of Docker as a whole, enforcing linear instruction execution even in situations where it is entirely inappropriate
You're not talking about linear instruction execution. You're talking about grouping instructions into commited layers. I would much prefer the proposed LAYER command to conditional execution or branching, which is what I assume you mean by non-linear in your comment. But I don't find this to be a serious problem either. That seems to be a pattern with this post: in a year of using Docker to containerize all our services - in-house python code, Django, redis, logstash, elasticsearch, postgresql - I haven't run into these issues that are deal breakers for you. Again, you might want to try to refactor and simplify some of your image builds. It's better to have a few simpler containers talking to each other than to try to cram a complex multi-service deployment into one. But then, I don't know what you're doing, and maybe it's just not suited for containers. You seem to have a strong preference for VMs anyway, so do that.
> However the Docker Hub implementation is flawed for several reasons. Dockerfile does not support multiple FROM instructions (per #3378, #5714 and #5726), meaning you can only inherit from a single image.
This whole post is like a laundry list of Absolutely Critical Things Nobody Ever Needed. I can't imagine a situation in which you'd absolutely have to be able to inherit from multiple images. If you have that situation I would agree it's an indicator Docker won't work the way you currently want to do things. I do agree with you about the occasional speed issues on the hub. But they're giving it to lots of people for free, and to me for a ridiculously low price. If I need better performance I can always run my own registry.
> There are some specific use cases in which containerisation is the correct approach, but unless you can explain precisely why in your use case, then you should probably be using a hypervisor instead.
There are some specific use cases in which virtualization is the correct approach, but unless you can explain precisely why in your use case, then you should probably be using containers instead.
See what I did there?
> If your development workflow is sane, then you will already understand that Docker is unnecessary.
I do like to read even-handed, unbiased reviews of technologies like Docker, even when I already use them. I like to have my world view challenged with an exposition of solid critical points. Maybe someone will write an article like that.
> The strike-out font was a nice touch, man
I've made it clear in other replies  that the comment was of poor taste, the strikethrough was intended to show that I was wrong to make the statement, but also not attempt to hide it from history. I have since moved it to the bottom of the article to draw attention away. To be absolutely clear, I admit I was wrong to make that comment.
You hold the opinion that most of the technical frustrations I mentioned are not deal breakers, for you these things don't matter, but to me they do. As I explained in other comments , my priorities are based on precision, perfection and beauty, not "getting the job done".
In my use case, Docker worked in production for 6 months and got the job done, but it left me feeling frustrated and dirty. As such, I'll refrain from responding to your individual technical arguments because we have a fundamentally different view on priorities and importance, and will result in both of us wasting our time.
However I've added a link to this reply in the article, as other users might find this rebuttal helpful. (If you would rather I remove your handle, let me know).
Thank you again for spending time writing this detailed reply.