Obviously, something is still being run on bare metal. One area of consulting for is right now moving people off of VMware who don’t feel like paying the rather stiff Broadcom licence fees.
I do, still. It's often a lot cheaper and modern HW is very robust and reliable. But it comes with more responsibility, of course. Not happy with performance on typical cheap VMs.
With Kubernetes it makes a lot of sense to run bare metal. What advantage is there to running K8s nodes on a hypervisor? It's one more thing that requires maintenance and can go wrong.
And that's actually my concern with K8s still. It still has an unnecessarily high maintenance burden for projects that are limited in scope and few scalability requirements. Which can often still be an important project.
Often I find that a load balancer and a couple of docker compose file sis more rugged, because you'll be able to leave it for a year and it'll still work and easily be understood by just glancing at it.
Kubernetes will still work after a year too, it just won't be supported. I forgot about a K3s setup for well over a year...
If you don't need a cluster then a single node K3s is still better than Docker Compose. You might know how an ad hoc load balancer and Docker Compose works, but I wouldn't say this knowledge is any more given than knowing K8s these days.
But if you do need a cluster then maintaining a bunch of Talos nodes is easier than maintaining a bunch of full fat GNU/Linux boxen tbh.
I've never looked at K3s so this is intriguing. I find docker compose great but it lacks some things that add to the complexity if bolting on, e.g. secrets management, or even load balancing on the single node to achieve zero downtime deployments. Would k3s handle the proxy layer?
IMO a lot of the "K8s is complex" is just the fact that you _can_ go complex with it, but you can really just declare pods and stuff and avoid the abstractions
I feel like k8s is better as soon as your thing goes from simple self-contained thing to multiple things running on one box
especially with agents it's never been easier to work with k8s.
from my experience, k3s supports basically everything k8s has that you'd need for a single-node setup
not sure what "Would k3s handle the proxy layer" means but if you mean the load-balancing part, then yes, that's like one of the main abstractions that k8s provides. but again, it provides many knobs so you probably just wanna pick the simplest option available
I do :)
alongside cloud of course, hybrid is the way to go in my view.
I run a bunch of openshift clusters for very private container workloads and VMs.
You can't beat a local fiber channel SAN for speed and reliability, especially compared to cloud software defined storage, and even on the networking side of things if you worked out your local connectivity well using real networking vs cloud providers software defined networks is so much better.
CSIs are hit and miss, some vendors are really really good, some are meh (hitachi, even though they are improving a lot with the latest relases), but the moment you get an IBM nvme san and start to tune all layers you get performance that are out of this world, some of our dbms workloads are very io sensitive and they improved 10x with nvme fabric.