Hi HackerNews-ers! I want your thoughts on a concept for an internal custom-built AI platform (I work in a fintech in London).
We use Kubernetes as our container orchestration platform. We deploy machine-learning models using a CRD+Operator which wraps a (mainly) python model artefact into a webserver to serve inferences over the network.
Our ambition is to allow non-engineers to deploy models on the platform from a model registry. They would probably want to use a CLI/UI to select a model, a target environment and deploy the model there. Ideally, we wouldn't need a repository to do this and we were thinking of providing a lightweight abstraction over K8s API.
What patterns/tools would you recommend looking at? We looked at https://cdk8s.io/ as well as ways to create think wrappers around kubectl. Any thoughts?
