I run a couple of small clusters and my Ansible script for installing them is pretty much:
* Set up the base system. Set up firewall. Add k8s repo. Keep back kubelet & kubeadm.
* Install and configure docker.
* On one node, run kubeadm init. Capture the output.
* Install flannel networking.
* On the other nodes, run the join command that is printed out by kubeadm init.
I'm definitely not knocking k3s/microk8s, they're a great and quick way to experiment with Kubernetes (and so is GKE).
I haven't done a manual deployment since. I hope it got significantly better and I may be an idiot but the reputation isn't fully undeserved.
The problem back then was also that this was usually the first thing you had to do to try it out. Doing a complicated deployment without knowing much about it doesn't make it any easier.
The thing that concerns me the most is managing the internal certificates and debugging networking issues.
I haven't yet set it up, but https://github.com/kontena/kubelet-rubber-stamp is on my list to look at.
> debugging networking issues
In this regard, I have had much more success with flannel than with calico. The BGP part of calico was relatively easy to get working, but the iptables part had issues in my set-up and I couldn't understand how to begin debugging them.