Hey ! I'm the creator of SSLPing... yup, I'm the guy who got famous when he killed his dream.
Switch to K3S, you won't regret it. I was hesitant to pour more work into my project to switch to K8S (which I know well, it's part of my day job !)... K3S is an excellent way to get into Kubernetes. Or use Digital Ocean's or other hosted implementations...
Kubernetes has a lot to offer... peace of mind is one of them !
K3s is pretty great, at least as long as you keep on the happy path.
In practice, that probably means a DEB based distro, quite possibly Ubuntu LTS and something like Rancher for management (if you prefer a UI of some sort and can afford the spare RAM), or Portainer, or even just Lens.
Sadly, i had a situation where i needed Kubernetes on a server at work, which had about 8 GB of RAM, so OpenShift was out of the question and most other "full" K8s distros also weren't viable, when that same server was also to run the actual containers. Moreso, i was stuck with Oracle Linux, on which getting K3s working properly was a bit problematic (though the same probably applies to most RPM distros).
Not only that, but instead of Let's Encrypt, i needed to use custom SSL/TLS certificates, installing which with the Traefik ingress and using them as the defaults instead of the self-signed ones was quite the mess, about which i wrote in another comment here on HN: https://news.ycombinator.com/item?id=30672765
In short, to get it working i needed:
- a ConfigMap for Traefik, knowledge about the structure of the ConfigMap (tls.stores.default.defaultCertificate)
- a TLSSecret for storing the actual certificate/key
- a TLSStore (which i also needed to actually use the secret, spec.defaultCertificate.secretName)
- a HelmChartConfig for Traefik to load the ConfigMap with the mounted secrets and config
none of which were documented as well as i'd like, because Traefik isn't necessarily aware of the intricacies of being used with K3s and K3s hasn't got instructions on such a setup because the happy path is using Let's Encrypt. Furthermore, attempting to use Nginx as the ingress instead failed and trying to uninstall all of Traefik resources hanged, something about Kubernetes waiting for the resources to do something so that they may be deleted, a callback that never seemed to happen.
Oh, attempting to use Rancher also failed due to recent changes with cgroups v2 and varying support for all of that, whereas it seemed to work properly on a throwaway Ubuntu LTS VM/VPS.
In short:
- i still think that the goals of the K3s project are really nice, the current resource usage is surprisingly decent
- if you have to use Kubernetes on RPM, go with OpenShift, if you don't have the resources for it, just use DEB distros
- the struggle with niche setups, like i needed, is largely not worth your time, the documentation for these still isn't there yet (example of mature documentation: Apache, which has been around for decades)
- ideally, just pay someone else to run a cluster for you, if you can (i can't, because mostly on-prem at work and am relatively poor in regards to personal homelab)
I'm still torn about migrating over to Nomad or K3s when i'll eventually retire my Docker Swarm clusters (maybe in the next 5 years), but for now i'm putting those plans on the backburner entirely to give both projects a few years to mature and become more stable/established. In regards to Rancher, they have RKE2 in the works as well.
I've been using k3s on Ubuntu (for development) for a while, and I switched it to ingress-nginx and cert-manager... It was always straightforward.
But I must admit it's easy/easier when someone else is managing K8S for you, be it DigitalOcean or Tencent (which I have experience with) or AWS / Google Cloud.
K3S is quite comparable to Docker Swarm though. I liked Swarm when I developed SSLPing because of its simplicity compared to K8S, but once you learn your way through K8S, there's no point sticking to Swarm I think...
I know you say you will let the projects mature, but I think it's a pretty good time to start picking up Kubernetes or at least getting your hands dirty with it. Nomad is a much simpler solution but not nearly as popular, in-depth or in demand as Kubernetes.
If you wanted an 'in' to Kubernetes and better automation of your homelab. I would suggest to check out this repo[1], it's nearly a A-Z guide on getting k3s running on Ubuntu (I am no distro-snob, Ubuntu just works well for most home workloads in Kubernetes). GitOps tools like Flux and Argo are really becoming popular to use with Kubernetes because all your configuration is stored in Git and the GitOps tools deploy manifests based on the Git repo state. My entire home cluster[2] is open source and there's many who are also doing something similar[3].
All of this comes at a cost of learning these tools which is not easy, but from my interactions with people who do take the plunge from docker-compose or docker-swarm most of them stay and once they see the benefits of GitOps and Kubernetes cannot go back to their previous ways. Automation is king and it is much easier using Kubernetes because most of the tools exist for it already.
Switch to K3S, you won't regret it. I was hesitant to pour more work into my project to switch to K8S (which I know well, it's part of my day job !)... K3S is an excellent way to get into Kubernetes. Or use Digital Ocean's or other hosted implementations...
Kubernetes has a lot to offer... peace of mind is one of them !