A long time back , I had gone down a rabbit hole to get Kubernetes manage my Mongodb. Thankfully , I got bitch slapped into reality within a few days and gave up because that rabbit hole goes deep. Now my mongodb stands outside of the K8 and I have no issues with it that cant be solved with a rebuild of a replica node.
Honestly I'd only think of running Mongo on Kubernetes if my organization is thoroughly bored with how well they've managed Mongo upgrades, scaling, outages, migrations, and tuning over the last year on a traditional VM setup.
Even so, if everyone's bored with the stability, why change it?
I'm an advocate for databases in dev/test environments on K8s due to ease of deployment, but there's too many moving pieces around storage for it to be a great idea for production out the gate.
Well. The benefits (monitoring, telemetry/observability) are nice enough that I might think about running it on k8s in prod. But only after sufficient number of dev/staging clusters have been sacrificed.
Don't use anything that doesn't come with a very-heavily battle-tested Operator (that does periodic backups and does at can restore). Simple StatefulSets are ... cute, but are very far from a reliable solution. Because when the shit hits the fan you won't be able to efficiently fiddle with containers, persistent volumes, secrets and configs (tons of YAML and so on) to try to somehow salvage the setup.
