I wouldn't even limit this to just the swarm feature. We've been running Docker in production for a year, using it in dev environments a year before that, and we've had major problems nearly every release. We had to upgrade directly from Docker 1.7 to 1.11 because every release in between was too unstable or had glaring performance regressions. We ended up running a custom build and backporting features which were worth the risk.
Speaking of 1.12, my heart sank when I saw the announcement. Native swarm adds a huge level of complexity to an already unstable piece of software. Dockercon this year was just a spectacle to shove these new tools down everyone's throats and really made it feel like they saw the container parts of Docker as "complete." One of the keynote slides literally read "No one cares about containers." I get the feeling we'll be running 1.11 for quite some time...
For example for some reason it's still not possible to ADD something and change its permissions/ownership in one layer, resulting basically in a doubling of the size of such layers.
I wouldn't go as far as saying it's in any kind of a 'sad' state though. It's a neat wrapper over some cool Linux kernel features, and it's been that way since before 1.0.
I'm curious how you even get performance issues from Docker, what feature did cause performance issues for you?
Our use case involves rapid creation and destruction of containers. Granted, this use case was pretty unheard of when we first adopted Docker, but it is becoming much more common.
Before Docker moved over to containerd, the docker daemon was riddled with locks which resulted in frequent dead-locking scenarios under load as well as poor performance. Thankfully, containerd now uses an event loop and is lock-free. This was a huge motivating factor for us to move forward to Docker 1.11.
To me, the sad state has more to do with Docker the company pushing new features out as quickly as possible and leaving stabilization to contributors. There are some days where it really feels like Docker is open-source so that Docker Inc can get free QA. To most users things may not feel in a sad state, but it can really suck for contributors.
I second this. We use Docker in a similar scenario for a distributed CI. So we spawn between 70k and 90k containers every day. Up to very recently we were running 1.9 and got a staggering 9% of failures due to diverse Docker bugs.
It's getting better though, since we upgraded to 1.12 a few days ago we're down to a more manageable 4%, but I'd still consider this very unreliable for an infrastructure tool.
edit: my metrics were slightly flawed, we're down to 4% not 0.5%
In the end, at this scale even with four or five nines of reliability, you'd still have to deal with 80 or 8 failures everyday. So we would have to be resilient to those crashes anyway.
However it's a lot of wasted computing and performance that we'd love to get back. But even with those drawbacks our Docker based CI still run 2 to 3 times faster than our previous one because containers make heavy CI parallelism quite trivial.
Now maybe another container technology is more reliable, but at this point our entire infrastructure works with Docker because besides those warts it gives us other advantages that makes the overall thing worth it. So we stick with the devil we know ¯\_(ツ)_/¯.
You spawn 200 containers for one build‽ Egad, we really are at the end of days.
> But even with those drawbacks our Docker based CI still run 2 to 3 times faster than our previous one because containers make heavy CI parallelism quite trivial.
Since containers are just isolated processes, wouldn't just running processes be just as fast (if not slightly faster), without requiring 200 containers for a single build?
The applications we test with this system have dependencies, both system packages and datastores. Containers allow us to isolate the test process with all the dependant datastores (MySQL, Redis, ElasticSearch, etc)
If we were to use regular processes we'd both have to ensure the environment is properly setup before running the tests, and also fiddle with tons of port configurations so we can run 16 MySQLs and 16 Redises on the same host.
See my other comment for more details https://news.ycombinator.com/item?id=12366824
But very little of our failures are accountable to AWS, restarting the Docker daemon "fix" most of them.
To put it simply when you run a distributed CI your performance is:
setup_time + (test_run_time / parallelism)
By setup_time I mean installing dependencies, preparing the DB schema and similar things. On our old hosted CI, we would easily end up with jobs spending 6 or 7 minutes setting up, and then 8 or 9 minutes actually running tests.
Now with our own system, we are able to build and push a docker image with the entirety of the CI environment in under 2 minutes, then all the jobs can pull and boot the docker image in 10-30 seconds and start running tests. So we were both able to make the setup faster, and to centralize it, so that our workers can actually spend their time running test and not pointlessly installing the same packages over and over again.
In the end for pretty much the same price we made our CI 2 to 3 times faster (there is a lot of variance) than the hosted one we were using before.
But all this is for our biggest applications, our small ones still use an hosted CI for now as it's much lower on maintenance for us, and I wouldn't recommend anyone going through this unless CI speed becomes a bottleneck for your organization.
Also it's not like hosted CI is without maintenance, if you want it to not be totally sluggish, you have to use some quite complex scripts and caching strategies that need to be maintained.
Side note.. I'm a production AWS user, with no plans to change, but I feel like AWS does this exact same thing with each reInvent. The announced products actually become available 6-12 months later, and actually "useable" and reliable 2 yrs later...
In practice, they wrap it in marketing speak to paint it as something to revolutionize your stack.
Then you jump in spending several days of engineering time diving into it, only to find late in the game that the one (or several) critical details you can't find in the documentation that are essential to making an end-to-end production ready pipeline, are not actually implemented yet...
And won't be for many months
Now that I think about it we did have one semi-serious bug in Docker, though that was also our own fault. Our containers would log a lot, and we hadn't configured our rsyslog very well so under some circumstances its buffers would fill up and the log writes would become blocking and be real slow. When this would happen some commands like `docker ps` would totally lock up, which messed with our (hand rolled) orchestration system. It wasn't until one of us noticed the logs would be minutes behind that we discovered killing rsyslog would make docker responsive again and thus found out what was happening.
Since it didn't actually affect our service I didn't remember it as particularily bad, but I can imagine that if our service depended on having fast interactions with Docker that would have hurt bad. IIRC they did recognize the severity of the issue and quickly had a fix ready.
I bet Docker Inc. has a tough mission, building out Docker services far enough to compete with the dozens of platforms that integrate Docker such as AWS or OpenStack so they can actually make money off the enterprise.
I'm curious if it'd be worth it to create some shim layer over runC (or adding the functionality) in-order to have a copy-on-write file-system that could be used to discard all changes when you're done with the container. Similar to how you can do a "docker run --rm -v /output:/output/34 mycontainer myapp" and all changes except those within the mounted volume get thrown away.
The use-case at my job needs the security of SELinux + CGroup/filesystem/network isolation. At a first glance, it looks like runC may handle most of the containerization bits, but not the copy-on-write filesystem stuff that I currently need. :s
With runC you can mount whatever filesystem you want, but it is up to you to setup that filesystem. So yes, you would need some kind of shim to set up your filesystem.
I still like Docker and can see myself, team and company using it for a long time if nothing MUCH better show up (rkt is promising to take some of the complexity pain away but we are not diving into it yet) but I can't say I've not been bitten enough to completely avoid upgrading Docker if it isn't needed, we follow a rule to only upgrade to ".1" releases as most of our problems have been with ".0" ones.
We have an "application" state template in our salt config and every docker update something would cause all of them to fail. Thankful the "application" state template abstracted running container enough were we switched from docker -> rkt under the covers without anybody noticing, except now we no longer fearing of container software updates.
How did you replace docker with rkt? Do you have an howto that you can share?
First off, this and the rest of the rkt docs is a good starting point https://coreos.com/rkt/docs/latest/rkt-vs-other-projects.htm...
Second, rkt runs Docker images without modifications, so you can swap over really easily https://coreos.com/rkt/docs/latest/running-docker-images.htm...
rkt uses acbuild (which is part of the application container specification, see https://github.com/appc/spec) to build images, and I had a very tiny Docker image just running a single Go process.
I just created a shell script that ran the required acbuild commands to get as similar image.
A good place to get started is the getting started guide https://coreos.com/rkt/docs/latest/getting-started-guide.htm...
Docker runs as a daemon, and rkt doesn't (which is one of the benefits). I just start my rkt container using systemd, so I have a systemd file with 'ExecStart=/usr/bin/rkt run myimage:1.23.4', but you can start the containers with whatever you want.
It's also possible to use rkt with Kubernetes, but I have not tried that yet. http://kubernetes.io/docs/getting-started-guides/rkt/
This and I can attest to 1.12 problems listed in this article.
Can't remember the specific with 1.10, but basically, nothing really ever works as promised which make people waste a lot of time trying to make something work when it can't and second, doesn't give much trust in the product's stability.
I really wish they would collaborate a lot more and fragment their solutions in smaller module while keeping everything simple. I think they have a great product, but too much growing pain.
I sort of agree, but that's not entirely realistic :)
> It would be great for AWS if Docker simply focused on open source bug-fixing and let AWS provide the profitable services....
Devs are mad ITunes is closed source. Mad windows is closed source. But happy to get a big paycheck if they work at Microsoft or Apple.
I wouldn't ever want to work for a proprietary software company. But I admit that I'm on the extreme end on this debate.
There are alternative runtime implementations (such as Mesos/Mesosphere DC/OS) that let you have best of both worlds: developers can still use Docker and produce Docker images but you use production-grade container orchestration (and that same Docker images) without using Docker daemon for your actual service deployment.
Cloud Foundry is currently running real applications with 10k+ containers per installation. We are on track to test it with 250k app instances.
Plus it's been around for, in internet terms, eternity. Garden predates Docker, Diego predates Kubernetes, BOSH predates Terraform or CloudFormation and so on. Used by boring F1000 companies, which is why it's not talked about much on HN.
Disclosure: I work for Pivotal, we are the majority donors of engineering to Cloud Foundry.
You are more likely to run into issues with containerization itself ( cgroup, namespaces ect) than abstractions on top of it.
Unless you are doing some sort of orchestration on top of containers, you can't go wrong with any of the container abstractions.
I tried it with the nginx and php-fpm Docker containers, but it wouldn’t work – because those containers assume specific process hierarchies (to log to the console where you issued `docker run`) that just aren’t present when using rkt. The advertised Docker compatibility only goes so far.
I still think rkt is a great idea, but I’m too lazy to develop my own containers. The documentation isn’t that good either.
I've found the single-service Docker containers from the Hub are useful for development (MySQL, Redis, etc.), but the "official" language run-time Docker containers that I've looked at are basically demoware. They are built to give you something that runs with the minimum of effort, rather than being efficient or anything else.
I agree that we need a better ecosystem of build tools and that is something we are looking to help build out. But, with rkt what we are trying to do is build an excellent runtime; and think the build side is an important and orthogonal problem.
It just works. Really well, actually.
Disclosure: I work for Pivotal, we donate the majority of engineering to Cloud Foundry.
AWS doesn't want to support Swarm, because it gives people portability off of their cloud. Google doesn't want to support swarm, because K8s is a trojan for Google Cloud. No one else wants to support swarm because it competes with their products.
That said, what's happening right now, if we are not careful, will fragment the container ecosystem, and it make it impossible for single containers to target multiple runtimes.
Docker is the only one who can deliver a universal set of functionality that is leveraged by all. From a technology point of view, Docker is going in the right direction. We got burned with Redhat in Openshift 1 & 2 land, and that's left us with a point of view that the only thing we can depend on is a container runtime itself, and 12fa applications.
K8s does not really work that way. It's huge and it's heavy, and it expects every app to be written it's way.
The technical direction here for Docker is good. But the implementation and early release is ridiculous. I was impressed by the first RC release, and then terrified that they released a RC as production.
Why do you say that? I have quite a bit more faith in the design chops of the folks behind Kubernetes (Google, Redhat, CoreOS, and many others) than Docker Inc.
Swarm really only touches the surface of the requirements for large scale distributed container orchestration.
Kubernetes is complex because the problem it attempts to solve is complex.
I'd also add that Kubernetes is dead simple to use. The difficulty is in setting it up - but even that is getting much better.
For it to be universal, it has to live in the container runtime.
I disagree. Kubernetes is quite lightweight, and its architecture is nicely modular. The core of Kubernetes is just four daemons. You can also deploy most of its processes on Kubernetes, which greatly eases the operational side.
> and it expects every app to be written it's way.
Kubernetes makes zero assumptions about how an app is written, as long as it can run as a Docker (or rkt) image.
It imposes certain requirements, such as that pods are allocated unique IP addresses and share networking between containers, but that doesn't really impact how apps are written.
> K8s is a trojan for Google Cloud
Doubt it very much. For one, the Kubernetes experience on GCloud (GKE) isn't particularly good at all — the "one click" setup uses the same Salt ball of spaghetti that kube-up.sh uses, the upgrade story isn't great, alpha/beta features are disabled, you can't disable privileged containers, ABAC disabled, the only dashboard is the Kubernetes Dashboard app (which is still a toy), and GCloud still doesn't have internal load balancers. Setting it up from scratch is preferable, even on GCE.
* Kubernetes has excellent support for AWS as well as technologies such as Flannel for running on providers with less flexible networking.
* Google makes a lot of effort to help you to set it up on other providers (also see kube-up).
* Projects like Minikube let you run it locally.
If Kubernetes is a "trojan" of anything, it's to improve the containerization situation generally, because this is an application deployment model where they can compete with AWS, which doesn't have a good container story at all (ECS is pretty awful).
I apologize for your experience with Red Hat OpenShift 1 & 2. OpenShift 3, which has been out for more than a year now, is natively built around both docker and kubernetes. Red Hat developers are among the top contributors to docker, kubernetes, and OCI. With OpenShift we seek to provide an enterprise-ready container platform, built on standard open source technologies, available as both software and public cloud service. I hope you will give us another look!
Like this fellow says, OpenShift 3 is lightyears ahead of 1 & 2.
(Obviously, my horse in this race is Cloud Foundry)
If people want to run Swarm or Nomad or Rancher on Compute Engine, then more power to them!
In fact, I even open sourced deployment templates to run Swarm on GCE and hopefully will add autoscaling and load balancing soon: https://github.com/thesandlord/google-cloud-swarm
In a nutshell yes. We recognized pretty early on that fear of lockin was a major influencing factor in cloud buying decisions. We saw it mostly as holding us back in cloud: customers were reluctant to bet on GCE (our first product here at Google) in the early days because they were worried about betting on a proprietary system that wasn't easily portable. This was compounded by the fact that people were worried about our commitment to cloud (we are all in for the record, in case people are still wondering :) ). On the positive side we also saw lots of other people who were worried about how locked in they were getting to Amazon, and many at very least wanted to have two providers so they could play one off against the other for pricing.
Our hypothesis was pretty simple: create a 'logical computing' platform that works everywhere, and maybe, if customers liked what we had built they would try our version. And if they didn't, they could go somewhere else without significant effort. We figured at the end of the day we would be able to provide a high quality service without doing weird things in the community since our infrastructure is legitimately good, and we are good at operations. We also didn't have to agonize about extracting lots of money out of the orchestration system since we could just rely on monetization of the basic infrastructure. This has actually worked out pretty well. GKE (Google Container Engine) has grown far faster than GCE (actually faster than any product I have see) and the message around zero lock-in plays well with customers.
I'm speaking about something other than k8s (Cloud Foundry), but the industry mood is the same. Folk want portability amongst IaaSes. Google are an underdog in that market, so it behooves them to support that effort -- to the point that there are Google teams helping with Cloud Foundry on GCP.
Not a chance. There is Packer  to get rid of all potential lock-in and monopoly. It's a universal image/container creation tool.
- It re-uses your ansible/chef/puppet/shell/whatever scripts for setting up the image.
- It outputs a docker containers, Amazon AMI, Google images, VmWare Images, VirtualBox Images. Whichever you like, with the same configuration.
I see docker-machine and docker-swarm as distractions. Reasons why doing all those other things, instead of focusing on containerisation and packaging may be harm-full for docker itself:
- Bundling-in the orchestration with docker make k8s or Mesos more inclined to fork docker and pull out unnecessary cruft.
- Churning out half-ready features causes Docker to be known as unreliable and leads to posts with titles like this. Such reputation tends to stay long after bugs are fixed. SV-esque launch and iterate works for web apps, but IMO not for back-end software.
From one of Spolky's finest, Fire and Motion: http://www.joelonsoftware.com/articles/fog0000000339.html
- Churchill 1931
It is absurd to consider morality on this topic when everybody used it in the last war without a word of complaint from the moralists or the Church. On the other hand, in the last war bombing of open cities was regarded as forbidden. Now everybody does it as a matter of course. It is simply a question of fashion changing as she does between long and short skirts for women.
- Churchill 1944
This analogy perfectly captures why in software, the second best thing always wins :-) 
The utopian 'ideal' systems that can only be built slowly and methodically get crowded out by the systems that start with some scruffy code just keep moving.
 UNIX vs Multics, Windows vs OS/2, MongoDB vs ?
Mesos 1.0 already introduced the universal containerizer, allowing to run many docker images natively without the docker daemon:
The downside, of course, is vendor lock-in. But that's only a problem if a) the vendor stops updating their products, which is unlikely if those products are popular, like Docker, or b) the vendor raises prices beyond what can be justified to remain that vendor's customer. But that's a problem for whoever takes over your project next year, not for you.
Docker Inc. themselves make a big play of the facts that containers have to be standardized to be portable, and the portability is the key value. We could have done a lot of this stuff years ago if the virtualization vendors had a totally portable format for transferring VMs between different systems.
Swarm etc. is part of the monetization strategy - other vendors in the ecosystem have already backed Kubernetes, or MesoSphere, or whatever, and do not want or need this stuff tied to the Docker run-time itself. Fortunately, Docker Inc. can add these without breaking compatibility of images or damaging the core features enough that a fork becomes necessary, but it does create market confusion.
Conversely, it's not been a surprise for those of us embedded in this community for a long time to see Kube and rkt join forces. There are a ton of both technical and political decisions behind this and unfortunately most of the political barriers end in the name Hykes.
If those companies would focus on securing position in the pie, rather than owning the whole pie, the pie would grow quicker and thus the absolute returns of each player could be better.
K8s is far ahead of Swarm - K8s has practically built its own language using YAML files - Swarm is still at a stage that all the configs for a service have to fit into a single command (and the options are much more restrictive than K8s).
To be fair, I do like some things about Swarm better than K8s (based on the docs), but in practice, Swarm is behind and they should tell you that up front. When I was just starting out, I literally had to install all of them; Swarm, Mesos and K8s to be able to make an informed decision because, in the case of Docker, the docs are like 6 months ahead of reality. I didn't realize that the Docker 'service' command didn't even exist until v1.12 and I couldn't install v1.12 on my machine (last time I tried, installation was failing - Obviously not yet stable).
I think Swarm has potential but they need to accept that they're just not going to be the first to market.
That's not a matter of moving too fast, it's a matter of broken processes. A user-visible change that does not update the docs ought to be rejected during review. If they don't have these basic development processes nailed down, that does not instill confidence in me regarding the quality of their shipped code. And that, of course, fits nicely with the reports of buggy .0 releases.
Kubernetes seems to be quite highly opinionated toward "clouds" and "microservices". I just wasn't able to wrap my head over its concepts' applicability to my "I just have one server that uses Docker for packaging, and now want to throw in another, for resiliency" case.
Its core data model, simplified, that of pods. A pod specifies one or more named containers that should run together as a unit. A pod's config can specify many things, such as dependencies (volumes, secrets, configs), resource limits and ports (including how to perform health checks). You can deploy single-container pods, and this is the norm, but it's entirely feasible to run a whole bunch of containers that conceptually belong together.
To expose a pod's ports to the world or to other pods, you define services. These simplify specify what ports should go which pods, and Kubernetes will assign a persistent, internal IP address to it. Kubernetes will (typically) configure iptables so that the service is round-robin-balanced at the network level across all containers that it serves; the idea is that the pod should be reachable from any other pod in the cluster. Together with KubeDNS, which resolves service names, you can do things like call http://mylittlepod/ to reach a pod.
To achieve resilience, Kubernetes lets you define replica sets, which are rules that says "this pod should run with N replicas". K8s will use the scheduler to enforce this rule, ensuring that a pod is restarted if it dies and always has N replicas running, and it can automatically ensure that pods are spread evenly out across the cluster. Replica sets can be scaled up and down, automatically or manually.
There are other objects, such as deployments (handle rolling upgrades between one version of a pod and another), ingresses (configures load-balancers to expose HTTP paths/hosts on public IPs), secrets (encrypted data that pods can mount as files or envvars), persistent volumes (e.g. AWS EBS volumes that be mounted into a pod), and so on, but you can get by with just pods and services, at least to start.
Kubernetes is a bit pointless with a single server, but adds convenience even if you have just two or three.
Yes, when you only have a couple of servers, that is not the sweet spot for K8s.
But few people stop at 2 servers. A few months in, someone asks for a staging environment, and/or QA environment. Someone eventually realizes that they need to regularly test their fail-over and backups. Someone hires a contractor, and wants to give them a copy of the setup that won't block anyone else. Someone realizes we can centralize the logs from all these environments... And so it goes.
Even with one server, sometimes you go to do an upgrade, and find your "one server" is actually a tightly coupled bunch of services. (Made-up example: I want to upgrade Varnish, but it requires a newer library that is incompatible with my WordPress version.) That one server could be a server for the Database, one to run the cron jobs, a few for the cache layer, etc. If you break up those into different boxes, you can scale them better -- Instead of one big beefy server, you can have each layer at it's own scale (one or more wimpy boxes, dynamically adjusted).
You don't do this to save money directly. But by simplifying things, you make it easier to maintain. That saves labor, plus prevents problems (and makes it easier to hire and train ops.)
When you have just a few servers, it looks manageable. As you grow, it gets a lot harder to manage. K8s helps.
The issues I have encountered with swarm mode:
* Some containers could not use hostnames to connect to other containers.
* Sometimes, in a 3 node swarm, containers on A could be reached from B, but not from C.
* After every reboot these issues could be fixed or start occurring.
* It automatically adds firewall rules for every service you port map to be exposed to the internet, without warning
In the end I switched to nomad, it isn't perfect either but at least it is consistent.
Docker is doing exactly what they should be, but in a manner that is destructive. Getting a built in consistent p2p routing mesh under every container is brilliant, and fixes one of the biggest problems with k8s and swarm (it's not really possible for these technologies to interoperate because of incompatibilities with the network model).
The big problem is the stability hit. 1.12 had no business loosing the rc label.
> If by google's age and experience, you mean the requirements of the Google Cloud
Quite on the contrary. Kubernetes flies in the face of Google's cloud APIs, and has to take advantage of every dirty trick it can. But it does that because the result is better. I can say that without hesitation, having worked on the logical conclusion of port-mapping (Borg).
> Getting a built in consistent p2p routing mesh under every container is brilliant, and fixes one of the biggest problems with k8s
That's hilarious to me, because what Docker calls "routing mesh" is a feature that Kubernetes has had since 1.0. It's different in some subtle ways, but again, for really important reasons.
Why do you think this is a requirement of Google Cloud?
As for marketing, it does seem a bit funny that a product would announce "deep integration with underlying infrastructure" for a cloud provider when they haven't written a single line of (public) code to actually support that cloud provider.
The fun thing is this arguably critical blog post praises features in "swarm mode" that have long been present in Kubernetes/Mesos/Nomad: [labels/constraints].
There could be a lot written about the fact that Docker ships "Swarm Mode" as stable in 1.12 despite virtually everyone's actual first-hand experiences. I would argue that if "Swarm Mode" were not shipped inside of Docker 1.12 and didn't benefit from riding along with the normal `docker` package, few would be talking about it.
I wish they didn't have to cram it all into single product docker though.
This type of bundling reminds me of Microsoft bundling IE with Windows. Initially, IE was much worse than Netscape, and this seems like roughly the same thing - monopoly in one market bundling an inferior product to try and achieve a monopoly in another somewhat related market.
Complicit in their own destruction.
Indeed, Red Hat have done exactly that with OpenShift 3.
Disclosure: I work on Cloud Foundry for Pivotal, ostensibly a competing project/product.
- Their built-in load balancing (HAProxy-based) is nearly impossible to debug. Literally no logging there.
- No locality awareness. DNS queries always return all addresses they know about (including those that don't even work - https://github.com/rancher/rancher/issues/5792) and I haven't yet found any good way to prioritize containers co-running on the local host to the more distant ones (https://github.com/rancher/rancher/issues/5798 - if someone been to this situation and has some ideas, would appreciate any suggestions!).
- Storage management was advertised, but can't find anything besides NFS (which is SPOF) and Amazon EFS (which I don't use). There was GlusterFS support, but it seems it was too broken so they had removed it or something like that. If one wants persistent storage, they'd better pin containers to hosts.
You may find that RFC3484 helps; it prefers the address with the longest prefix in common with your own address so will tend to pick your own address. And you are probably getting this behaviour already.
The problem is, the network space there is flat, not hierarchical - while I haven't looked at the actual implementation code, I believe container addresses are just randomly chosen from a single big 10.42/16 subnet and I'm unaware if there's a way that I can assign hosts, say, a /20 out of that space (yes, this would've solved things nicely).
(I'm asking, so the next time I'll have to make a choice between stacks I would be more aware about the finer details.)
My actual issue is, there's effectively no distributed storage support in Rancher/Cattle at this moment, be it GlusterFS or anything else (for all I know, MooseFS worked quite well for us on one project).
Just pointing it out, because for some reason I got quite a different impression from the website/docs.
Every point of advertisement statements like "Rancher provides a full set of infrastructure services for containers, including networking, storage services, host management, load balancing and more." is to be taken with a huge bag of salt.
(And that's by no means unique to Rancher.)
Docker promises too much and delivers too little. Story of every software project in the last 50 years.
I guess I just know some day Docker's investors are going to want their money back.
Yes sure there are other quasi opensource products that are super critical that I use but they all have alternatives (for example Java has plenty of alternatives).
Hopefully I don't get downvoted to oblivion for this comment. I am sure my trust issues are illogical and I would really like to remove the inhibition to use docker but articles like this do not help.
We also already have a functional replacement with rkt. It can use Docker images, and Kubernetes can use rkt as a run-time in place of Docker, so Docker is not irreplaceable.
I think that the most valuable bits of Docker today are the developer tooling - easy Windows and Mac installers, Docker Compose, the online documentation, and the Hub for grabbing ready-made images. None of which, AFAIK, makes much money for Docker Inc.
These seem to be way more mature than Swarm.
Other docker projects suffer from various levels of similar issues. Docker-machine is nice, but has a ton of rough edges, especially when spinning up host on AWS. It feels like the programmer(s) of machine never used AWS beyond the simplest use case.
place api, proxy, scheduler and controller definitions manifests folder.
If you're running infrastructure that deals with problems such as maintaining the health of applications then it's going to generally be a whole lot more complex than that.
1. sudo docker build -t quay.io/foo/bar
2. sudo docker push quay.io/foo/bar
3. <login to production>
4. sudo docker pull quay.io/foo/bar
5. sudo docker kill foobar
6. sudo docker rm foobar
7. sudo docker run -p 80:80 -p 443:443 -e FOO=bar --name foobar --net=host -d quay.io/foo/bar
I run a service using docker and have your steps 4 through 7 as part of a systemd unit file. Updating the application requires a single systemd restart command.
ExecStartPre=/usr/bin/docker pull app
ExecStartPre=-/usr/bin/docker kill app
ExecStartPre=-/usr/bin/docker rm app
ExecStart=/usr/bin/docker run --name app --rm=true -p 80:80 app
This will ensure that a restart of the docker service will trigger this service to be restarted.
Still, I'm liking many aspects our tools. Using Docker with Rocker (https://github.com/grammarly/rocker/) has greatly sped-up CI builds by caching results when the source hasn't changed (especially important in multi-language shops; the Python guys don't want to wait on the Java code to build every time.) Just upload a tagged image to ECR, generate an "application version" referencing those images, and deploy via the Slack bot ("@bula deploy develop develop-XXX-e83fc3bd").
We've started using a self-hosted Drone  install (not to be confused with the hosted drone.io service, which is not good) to build containers. Unlike Quay, it doesn't launch build VMs, but rather uses Docker containers, so it's very fast. It also supports the notion of build containers, so you can do things like compile C code or run NPM without ending up with any compilers or build tools in any of your image layers; it completely removes the need for a custom "base image" shared among apps. It also lets us add the Kubernetes deploy as a final step after publishing.
On the other hand I think Docker can probably be forgiven for my particular frustration. For one, the software was in beta. And two, working with Windows and all the different flavors must be a nightmare.
If you haven't already and can spare the time, could you file a pair of issues: perhaps one for the shared volume problem and another for the networking on https://github.com/docker/for-win ?
We've been fixing bugs in both areas and the fixes should arrive in the beta channel over the next few updates. Thanks for your patience!
Some of it is clearly very useful, but some of it strikes me as a tower of babel built upon workarounds to problems that have simpler solutions.
What I see here are several different vendors vying to make sure they remain relevant. While I understand this and even the need for it, I also understand that it can drive poor engineering decisions in the long run.
This is the reason the latest Mesos and DCOS has the universal containerizer. Docker is great for development but currently doesn't make sense for production. The latest DCOS uses the docker images without the docker process and provides the high scale production quality needed for a large datacenter.
Perhaps they should create some sort of firewall? I mean, the network packets coming through... they bring malware... they're spam... and some, I assume, are good traffic.
And we should make them pay for the firewall.
Your submission page is heavy on the linux-toys.com domain, implying that it's yours. That same site identifies the author as working at Apcera.
There is also a submission in your account that points directly to an Apcera blogpost, with the same author name.
Disclosure: I work for Pivotal, we donate the majority of engineering to Cloud Foundry. Apcera has beef with us.
I gave up after trying to spin up a cluster on DigitalOcean using v1.12. Like the author, I couldn't get my containers to see each other, something that worked before v1.12.
I've tested Docker Swarm a bit and it seems to work as advertised.
Is it just about node selection not being sophisticated enough? In that case I don't need to worry since all my nodes are the same, but if there's any words of warning I'm all ears. Thanks.
My old team had a production environment running in docker containers for about a year (this was pre-swarm, pre-kubernetes) and then transitioned to just using ansible for application deployment in a more "traditional" manner because we spent more time trying to fix broken things with docker than it was worth.
what is the current best practice to do this (with logging, etc.) ? should I even be considering something like marathon/k8s, etc ?
currently I have a fat docker VM with supervisord and all services running in a single VM with highly fragile logging. I dont think this can last.
getting started seems very intimidating in the Docker world.
Feel free to jump on our slack or irc: http://dokku.viewdocs.io/dokku/getting-started/where-to-get-...
Disclaimer: I am a maintainer of Dokku
Generally we moderate HN stories/threads less when a YC startup or YC itself is at issue. But we do still moderate them some, because the standards of the site still apply.
(We haven't done anything here besides this title edit, though, in case anybody is wondering.)
Unnecessary fear mongering by people who have an outside agenda.
What we've had to do at Red Hat is always stay a couple releases behind (we're shipping 1.10+patches right now) and backport all the fixes from upstream releases to make it stable (ie production ready). Docker keeps shipping new versions, with fixes for old issues but then a whole new set of issues added in.
Why would one use docker swarm vs just docker swarm mode.
Maybe docker-swarm is not supported anymore with 1.12? It was all pretty confusing.
I am using docker-swarm with 1.12 so it's definitely supported. Yea I am really confused too :D.
Do you know if there was any guidance by docker team on this? Seems like there is active development going on for both docker-swarm and swarm-mode.
https://github.com/docker/swarmkit is what gets integrated into Docker engine as of docker-1.12.0.
Their readme says
"Docker does not currently have a plan to deprecate Docker Swarm."
Do you know if they officially say otherwise elsewhere?
It's simple and stable!
If you can't take the bleeding part of bleeding edge, wait for things to mature before using them. Bitching and whining that they didn't create a perfect product out the gate only belittles the hard work it took to get a new product out the door.
This is how software releases work in the real world!
It's another thing to bundle beta software into an existing package and label it as stable in order to stay relevant in the orchestration game.
No doubt the guys working on Docker are super smart guys, though they too have bill's to pay and lives to live. So if you want to see Docker improve how about contributing back and or supporting their efforts with extra funding. Maybe they will have the funds to bring on more full time developers to hunt those obscure bugs or add those features that everyone wants?
Discussing and putting down the developer with article titles such as 'Sad state of docker' to me shows self entitled the community around these projects are.
Except for, you know, the people who pay for it...https://www.docker.com/pricing
> without any contribution by the users
Then who are the over 1,000 contributors to the Docker repository?
> supporting their efforts with extra funding
I think they're doing just fine. https://www.crunchbase.com/organization/docker/funding-round...
> how self entitled the community around these projects are.
As a member of the "self entitled" community, I see a lot of ignorance in this comment. The developers of Docker are not just the >100 employees of Docker Inc, but also the over 1,000 members community who have contributed to the Docker project.
Frustration with Docker isn't about entitlement, it is about frustration with a governing body run by a for-profit company with $180 million of funding who has shown a pattern of favoring shiny new features over stability and left their own community to clean up the mess of bugs left in their wake.
So no, I don't think the community is self-entitled, I think the community just wants to see the project they've worked so hard to help build be competently maintained.
> It nothing about that. The main point is the whole community bitching a moaning about software that in all intent and purposes is given away free > without any contribution by the users.
I remember that Jessie did a talk where she said that something like 4/5 contributions to Docker are from people who don't work for Docker Inc. I was one of them (I spend most of my time contributing to runC now because there's less politics there). So your point is categorically invalid.
> The source code of Docker is open source so why doesn't the guy go in there and contribute his time and or his own money to support the project that to him is so vital?
Docker Inc does not operate Docker like a community project. Maintainers are elected behind closed doors, most of the maintainers work for Docker Inc, Docker Inc has a roadmap that the community does not have access to, maintainers of Docker will not merge code if it is not strictly inline with Docker Inc's roadmap, they openly attack distributions like RedHat and openSUSE for applying out-of-tree patches to Docker.
> No doubt the guys working on Docker are super smart guys
The community is also fairly clever too, especially considering that we outnumber the number of people working for Docker Inc.
Thank you, thank you, thank you. This here is the truly sad state of Docker. Glad to see someone else in this thread sick of closed door governance bullshit and flat out bullying.
Why do you think "free" is used as a marketing strategy? Obviously there is a monetary benefit somewhere - or anyone attempting a "free" or even just a "low(er) price" strategy must be doing wrong. An example is Uber, which has just generated yet another billion dollar loss headline.
Apart from the data collection model, what it does is it removes the incentive of others to enter that market and possibly create something better.
And on an individual level, something can have a cost merely by existing - because I have top choose. And if I choose wrong I end up worse than where I could be.
Imagine I was an evil guy from a Disney movie. My next project to annoy people might be to create an app that seems to work - but has a lot of very annoying flaws. However, I never mention the flaws (of course not), instead I go all out on my marketing and make a lot of claims to get people to use my evil software. I then sit back and with an evil laugh enjoy the frustration that I created a million times - the (Disney-level) evil scheme succeeded. I wasted lots of people's time; I may also have sabotaged other (non-evil) projects by taking users away from them, and I managed to raise the world-wide frustration level.
I'd much rather pay for something that works than get something for free that doesn't.
The point is, if something doesn't work as claimed, it adds real 'cost' which is often much more than the relatively cheap cost of buying something that is more reliable.
That it is free is almost irrelevant - the cost/benefit of implementation, training, reliability, support, and possibly 'savings' using it vis-a-vis other solutions probably far outweigh any price tag it might have.
Apparently they launched a rev. with a bunch of problems - this is bad, and costly.