Because there's so much to properly running it. Individual component updates, network policies, certificates, backups, monitoring, authorization, authentication, secret management. Kubernetes makes it all relatively easy and k3s (or k0s) ain't that scary.
K3s, Calico (or whatever you prefer), Istio with cert-manager and self-manager CA (or whatever you prefer for your service mesh), kube-prometheus, OTEL or Jaeger for the mesh visibility, pick the operator (I used Crunchy Data operator but there are at least two other solid choices), will get one far at low cost. Of course, use reliable infrastructure provider of one's choice.
No need to think about placement (one still can if they want to), addressing, firewalling, DNS, IP assignment, and so on. Add nodes to the cluster as necessary and let it sort itself out.
Some understanding of Kubernetes is necessary, indeed. But it's a stack usable every time once learned.
Sorry but do you mean nodes or clusters? Anyway at this scale you are definitely looking into managed services from cloud providers or using Kubernetes. 1 and 2 is not an option because of the maintenance burden. Rolling out upgrades and resource allocation is going to be very hard without Kubernetes.
(after clicking through context...) There's no amount of money you could pay me to do this with Ansible. Like, you could offer me double my salary, and I'd laugh.
The only answer that makes any sense here is Kubernetes.
So... Ansible isn't the thing hosting the thing. It talks to APIs; K8s is one of many it can handle!
This reads like a brazen misunderstanding/misrepresentation of Ansible.
Like... imagine the position of Terraform, a Kubernetes darling, and that's it.
I could especially see Ansible being used when not buying toy clusters, but building your own. I don't know how useful CRUD on the objects like deployments would be. Enough to do the job.
Point being, Kubernetes doesn't exclude Ansible in the slightest, whatsoever.
This was just enough for me to post... they aren't mutually exclusive. It's yet another layer of yaml to throw at the declaration problem.
Where/what/how is entirely up to the user. I'll take your money and then move on to the next thing to manage
No, I know what Ansible is, thanks. I've written and maintained Ansible playbooks. I respect myself, my team, and my customers and deploy statelessly with NixOS. My life is significantly easier than it was in the salt/Ansible days and there's no amount of money worth those headaches.
> It's yet another layer of yaml to throw at the declaration problem.
Really, my head is already starting to hurt. I KNOW.
> This was just enough for me to post... they aren't mutually exclusive.
I'm sorry but there's no fucking good reason to templatize k8s yaml into Jinja/Ansible. Just hell no. Just more Ansible Things that look like a solution because folks don't actually know k8s.
This whole thing is based on a perceived false dichotomy of Ansible/K8s
I'm not saying using Ansible for K8s is necessarily a good idea, just that laughing because it's mentioned in the same neighborhood is hardly reasonable
I'd gladly use it for node-level tasks, for example - in the cluster I'd be less excited.
Custom facts are neat. The module library is neat. Possibilities are too, dismissive nature isn't
The only answer that makes any sense here is Ansible.
Arguments:
Way simpler
Can fall back to using plain SSH if things goes really wrong
No vendor lock in
One less bit of technology in the mix to go wrong
Very boring, which means very reliable