
CRDs Killed the Free Kubernetes Control Plane - caleblloyd
https://caleblloyd.com/software/crds-killed-free-kubernetes-control-plane/
======
jacques_chester
I don't completely agree that this is the cause, as CRDs also create revenue-
generating demand from controller software that acts on them.

But I absolutely agreed with this:

> _There are applications out there that are storing their complete state in
> CRDs. Please do not do this. The Kubernetes API server is not a general-
> purpose datastore._

A thousand times, yes. The API server is good at what it does. It is _not_ a
suitable replacement for a real, honest-to-Codd database. The idea that laying
hands on a connection to a postgresql or mysql server is somehow more onerous
than spinning up _an entire goddamn cluster_ is bonkers. Bon. Kers.

~~~
shickys
If your apps run on k8s already, why is it a bad idea to store some simple
states in custom resources?

~~~
jacques_chester
There are a number of reasons. One is that states don't stay simple for very
long. Just the other day I saw someone ask "how do I find all the children of
a parent resource?". The answer was to iterate over everything that might have
that parent resource as an OwnerRef. In a normalised schema this would be
simple. In a document database it is not.

Another problem: there are no multi-document transactions. If I rely on
updating A and B together, I am out of luck. I will instead need to account
for A and B being out of sync, which involves 4 permutations: A,B; A, !B; !A,
B; !A, !B. As a CRD controller author I am now responsible for a shitty
database. Where Kinds have crisp hierarchies this is fine, the supervising
controller can inspect and repair the lower documents. But many things are not
hierarchical in nature and you can easily wind up with control loop fights.

Another is that there's a difference between "I already have a cluster" and "I
am using the API server as a database". Because CRDs wind up in a single
global document store, folks inevitably want them to have hard isolation.
RBAC, namespaces etc get you _some_ of the way but it's basically just hard.
Eventually some bright spark suggests putting things into a different cluster,
at which point the balance of ease flips the other way.

CRDs are super useful extension points (note, for example, I am writing a book
about Knative), but they are not suitable for most application and business
logic purposes.

~~~
shickys
Well, I agree that you can't really use CRD as an alternative to a database.
But wouldn't you say that there are cases where it makes sense to use CRD to
keep metadata? Keeping results of job runs for example.

~~~
jacques_chester
I think the best use for CRDs is for creating seamless extensions to
Kubernetes itself. I see Knative as being in this genre.

------
ojhughes
> Since CRDs are installed on the cluster level, it is not possible to
> namespace resource versions (and that’s unlikely to happen soon)

This is misleading, the object _instances_ of a CRD belong to a namespace,
unless they are cluster wide

