I too played around with k8s but my issues were even more basic: It's darn near impossible to deploy it on a platform that deviates from their expectations.
The docs seem to expect that:
- If you're not letting managed services do most of the work for you, you're running on one of CoreOS, Ubuntu or a RHEL derivative
- You know all about "CNI" and how it magically makes everything work
- You know that the default "security" is none at all
Deployment is just so much harder than it should be. Fundamentally (I discovered far later on in the process), Kubernetes is comprised of roughly the following services: kube-apiserver, kubelet, kube-proxy, kube-scheduler, kube-controller-manager. The other dependencies are: A CA infrastructure for certificate based authentication, etcd, a container runtime (rkt or Docker) and CNI.
The steps for setting up a simple installation should be able to fit on a single page. If it weren't for a bug crippling my cluster, I would have done it myself (it seemed to be running but a bug with cadvisor's disk space detection messed it up to the point where it was useless).
To get a handle on what I mean, have a look at the docs for Ceph, another reasonably complex distributed system. Here are the manual installation docs: http://docs.ceph.com/docs/master/install/manual-deployment/. It's a list of commands to run and a basic idea of what the config files should look like. Following that guide actually results in a running Ceph cluster. Now have a look at the manual installation docs for Kubernetes: https://kubernetes.io/docs/getting-started-guides/scratch/. It's a bunch of links to other parts of the docs, pretty much no practical guidance on how to actually set up the cluster and the few commands that have been suggested are mostly outdated (for example "--configure-cbr0" doesn't exist on kubelet anymore). Following it can result in a working cluster but only with a lot of additional work and study (it doesn't give you anything on networking, which is essential).
You'd either use the ansible or salt recipes for k8s to install.
Everything is nicely documented there.
There are a few very nasty pitfalls on the way to your k8s install in production though...
We deploy kubernetes on hundreds of nodes in multiple locations and i've to say i've never ever looked at any install docs.
ACK, they are basically useless, but overall i find the documentation quite good (except not up to date with the code on may locations).
What's severly missing is the overall picture!
But lets not stop there, if you want to have k8s running properly in production you practically have to make all containers yourself, otherwise some shitty container from dockerhub will bring your node down one after another (hint: read-only containers).
The docs seem to expect that:
- If you're not letting managed services do most of the work for you, you're running on one of CoreOS, Ubuntu or a RHEL derivative
- You know all about "CNI" and how it magically makes everything work
- You know that the default "security" is none at all
Deployment is just so much harder than it should be. Fundamentally (I discovered far later on in the process), Kubernetes is comprised of roughly the following services: kube-apiserver, kubelet, kube-proxy, kube-scheduler, kube-controller-manager. The other dependencies are: A CA infrastructure for certificate based authentication, etcd, a container runtime (rkt or Docker) and CNI.
The steps for setting up a simple installation should be able to fit on a single page. If it weren't for a bug crippling my cluster, I would have done it myself (it seemed to be running but a bug with cadvisor's disk space detection messed it up to the point where it was useless).
To get a handle on what I mean, have a look at the docs for Ceph, another reasonably complex distributed system. Here are the manual installation docs: http://docs.ceph.com/docs/master/install/manual-deployment/. It's a list of commands to run and a basic idea of what the config files should look like. Following that guide actually results in a running Ceph cluster. Now have a look at the manual installation docs for Kubernetes: https://kubernetes.io/docs/getting-started-guides/scratch/. It's a bunch of links to other parts of the docs, pretty much no practical guidance on how to actually set up the cluster and the few commands that have been suggested are mostly outdated (for example "--configure-cbr0" doesn't exist on kubelet anymore). Following it can result in a working cluster but only with a lot of additional work and study (it doesn't give you anything on networking, which is essential).