From day 1 rkt's approach always seemed like the right approach to me intuitively. Their process is for many reasons much more professional. Docker on the other hand consistently overpromises and underwhelms.
I tried to get docker to work for a CI pipeline and it failed miserably. Consistent lock-ups and just generically slow operation when spinning up and deleting 100s of containers per hour forced me to move to lxc which works extremely well for a CI type workload. Not that surprising considering TravisCI and CircleCI behind the scenes are using lxc as well. Volume mounts, networking, sandboxing and just general stability all work without surprises in lxc but to this day I have no idea what the proper patterns are for sharing host level mounts properly inside docker containers.
I really hope rkt becomes the default container substrate. From first principles and fundamentals perspective rkt is the right solution.
The bad experiences we have had with docker have been from devicemapper bugs. Since devicemapper is part of the linux kernel I can't exactly blame docker. We really need a (limited) multi-host solution and have been using rancher.
I really like CoreOS and rkt. At my company we're building our next generation infra on CoreOS and Kubernetes with Docker. We'd like to switch from Docker to rkt eventually. The two things blocking this for us are:
1. rkt is focused on production usage (which is good) but is completely lacks a development workflow akin to Docker Compose and Docker for Mac. (Kubernetes is not a suitable alternative to Docker Compose, before someone suggests it.) Since rkt doesn't use a client/server model like Docker, having a seamless experience with a VM like Docker for Mac where the client on the host is just sending requests to the server will require some work, or a different approach altogether. Yes, we could use rkt only for deployment and develop with Docker, but that complicates our stack in a way that just doesn't provide the benefit to justify it yet.
2. Integration in Kubernetes at the same level of maturity as Docker. As mentioned in that reddit thread, "rktnetes" is currently described as a "work in progress." I remember somewhat recently that official Kubernetes release notes introducing rkt support literally linked to a Google Doc filled with issues. Kubernetes itself has very bad documentation, so linking to a Google Doc isn't a huge surprise, but it still shows that rkt support is still very early. It will happen, certainly, but it's not there yet.
rkt support for a local kubernetes cluster should be in the next version of minikube[0]. A PR was just merged to master today. [1] Of course, theres still a lot of work to be done.
> (Kubernetes is not a suitable alternative to Docker Compose, before someone suggests it.)
Why not? I have been contemplating this exact thing right before I came to procrastinate on HN.
A few of my local development workflows are using minikube[1]. Kubernetes within minikube is a lot like docker-compose. The only use case I have for docker-compose is pycharm integraion[2], but someone will probably make it work with minikube one day. There is a tool to convert from docker-compose to kubernetes[3].
It has the technical capability to do what Compose does, but Compose is a tool specifically designed for development workflow, and this is not the case for Kubernetes. Application developers without any infrastructure/operations experience can have a tough time understanding and learning all this new technology, and to have to get a mental model of Kubernetes to do their work is a big barrier. One simple example is that there is no direct equivalent of `docker-compose restart` in Kubernetes. The only way to restart a pod is to remove it and recreate it (or scale an RC/RS/deployment to zero and back up to force recreation.) It may just be a couple commands more, but it's unnecessary cognitive load for many app devs. Even just comparing a Docker Compose manifest file to a Kubernetes manifest file, it's more verbose because Kubernetes is more powerful and needs more details specified.
> It may just be a couple commands more, but it's unnecessary cognitive load for many app devs.
My general approach (not only to interacting with k8s) is to create fabric files with list of commands that makes sense in given place. I've seen people do similar things with Makefiles.
You would type "fab -l" and see list of commands, including "restart_pods" with the docstring describing what it means. Or even broader command for "rebuild and restart". Not only it helps me as I get auto completion (https://gist.github.com/Fandekasp/1522827), but also others can get things done without understanding all the details. And it's easier to migrate everyone to use slightly different command, e.g. add a flag.
I've generally thought Alex is the most pragmatic of the Container Kids. Flo seems to have a pretty decent business, something that is realistically lacking in the go to market at Docker, recently because they seem very emo. However, Alex/CoreOS has good technology that I believe is often overlooked. HashiCorp is to my mind always the dark horse, I still firmly believe containerisation is stop gap and part of the reason I campaigned hard internally at DigitalOcean to not deeply productize a container solution. Fully expect ten thousand down votes for this post. :)
> I still firmly believe containerisation is stop gap
It's a primitive that can be composed into other things. It's a better primitive for many cases than VMs are.
If you're fretting about containers/not-containers, you're below the value line. What matters is application code that provides value. If you're thinking about how it gets launched and managed, you're losing time to someone who doesn't have to.
Disclosure: I work for Pivotal, we have a fairly close interest in this topic through PivotalCF.
In the sense that the uses of both have been about isolating apps from each other. The early fuss about VMs was about increased density -- but only because hardware used to be the boundary of isolation, meaning lots of idle capacity.
Containers are a cheaper way to isolate, but the point was and still is isolation.
Separate servers -> VMs -> containers. Same song, different instruments.
I'm not sure what you're driving at. Orchestration is an example of taking a lower-level primitive and building on it.
There are no real novelties in software architecture, just occasional shifts in algo-economics that cause previously burdensome ideas to become cheap and attractive.
So predicting the future is premature at this point. It always is, but particularly at the moment. Everyone, including Pivotal, is busy trying to invent it.
That makes sense to me. Right now, containerizing Web applications effectively adds a lot of complexity to the infrastructure, which means that hosting providers need to have a number of services in production, and the customers have to be willing to deal with that complexity. Lots won't, or can't.
We use a mix of providers for different things, and I actually love DigitalOcean because of your focus on making it super-easy to run basic Linux servers at reasonable cost.
Who's "Flo"? Are you talking about HashiCorp's Nomad? Why is containerization a stop gap? Maybe answering these questions will "contain" the downvotes ;)
FAIR! Florian Leibert is the CEO of Mesosphere, I'm talking about the HashiCorp toolchain generally as an ecosystem, containerization is stop gap because I believe in event driven architecture/applications, and there are/will be new interesting and unique ways to implement that///cloud, however, I'm often wrong. :) :)
I will say, I know all of those folks relatively well and they are all really really really good humans who I believe in and love.
I was in the opposite position I was working at a hosting company in 2010. At that time I was heavily pushing containers, at the times Jails, Zones, LXC, etc. Using a full hypervisor stack for VMWare was not the best use of resources. I honestly haven't kept up with containers too much, other than Kubernetes, Misos, etc.
I'm also thinking they're a stop gap. In the CI work flow it definitley helps with dynamic language, or slim executables. Where dependencies are seperate. But often I see infrastructure almost meshing with the code. A block, in the overall program design. I'm not sure the next step either, often as you said you're looking at a event or service driven architecture. It comes from the pain points you're trying to solve, and how you actually get your code running.
NixOS is awesome and I lost a very long internal battle years ago to support it heavily. Very very pro NixOS. I've not been in cloud for almost a year now so I've not been following, however, my company builds fully towards AWS like this: lambda+kinesis+sqs+rds+s3+kms
Steps in the 'Trying out rkt' [1] document worked like a charm on Fedora 24 (using the binary install method).
I saw v1.14 is tagged as 'f26' in Fedora's package builder so I'm not sure when this will be getting released there. F24 is currently offering rkt v1.0.
Loved the `rkt gc` command. Really excited to try this out with Kubernetes now [2]
I finally tried out rkt as a result of this post. But are there images other than the Alpine one? Other linux distros, e.g.? I'd prefer not to use docker images unless that's the only option.
I tried to get docker to work for a CI pipeline and it failed miserably. Consistent lock-ups and just generically slow operation when spinning up and deleting 100s of containers per hour forced me to move to lxc which works extremely well for a CI type workload. Not that surprising considering TravisCI and CircleCI behind the scenes are using lxc as well. Volume mounts, networking, sandboxing and just general stability all work without surprises in lxc but to this day I have no idea what the proper patterns are for sharing host level mounts properly inside docker containers.
I really hope rkt becomes the default container substrate. From first principles and fundamentals perspective rkt is the right solution.