
Cortex – An ML model deployment platform that runs in your AWS account - ospillinger
https://github.com/cortexlabs/cortex/tree/v0.7.0
======
sandGorgon
You should be using Terraform as your configuration management language.
Please do not invent your own. Pretty much your yaml is a pseudo-configuration
management language.

That will become one of the biggest blockers because of the amount of
automation that already exists.

By leveraging Terraform, you will also have the added benefit of getting all
the other pieces of AWS/GCP/azure components for free - and is rock stable and
production tested.

~~~
fake-name
You should be using _an existing scripting language_ as your configuration
language.

Seriously, Every single fucking stupid infrastructure-deployment-
tool/"platform" whatever has it's own, dumb in-house language that winds up
basically re-implementing the programming language the tool is written in
_badly_.

    
    
      - Puppet: Has a stupid ad-hoc config language.   
      - Terraform: Has a stupid ad-hoc config language.   
      - SaltStack: Has a stupid ad-hoc config language.   
      - Ansible: Has a stupid ad-hoc config language.
    

If you're even _considering_ implementing a tool like this, use a goddamn
existing language for your configuration files.

You don't need to use the _entire_ language, but at least use the language's
lexer/parser (cf. json/javascript). That way, _all existing tooling for the
language will work for the config files_ (ask me about how saltstack happily
breaks their API because you're not "supposed" to use it, despite the fact
that they have public docs for it). Additionally, people won't need to figure
out all the stupid corner cases in your terrible piecemeal configuration
language.

Additionally, by making your configuration language an _actual_ language, you
also simplify a lot of the system design, because the configuration can act
directly against your API. This means using your tool from other tools becomes
much more straightforward, because the only interface you actually need is the
API.

~~~
sandGorgon
You are right - Terraform does have the ecosystem, but the new kid on the
block is Pulumi.

Pulumi = Terraform in Typescript. That's good as well - but i was not sure if
the OP is familiar with Pulumi

~~~
chucky_z
Pulumi made the mistake of immediately making remote state a paid-only
feature. Even if it's not, from all the recent marketing I looked at
everything useful required payment; for getting started with a project that's
a non-starter.

On top of that, most of the worst parts of Terraform are no longer an issue
with 0.12.

~~~
reilly3000
Its completely possible to host your state in S3 or a filesystem; it takes a
bit of setup and there may be a few rough edges, but the effort or
subscription is completely worth it. The secrets management alone makes it
worth it, but their programming model is definitely the future. I think the
fact that AWS just released their Cloud Development Kit is strong validation
of the approach.

------
personjerry
Man I can't way to pay for a ML dev service that handles the infrastructure
dev service that I pay for to host my ML dev services.

~~~
tixocloud
We're actually building infrastructure to serve ML as a Saas and would love to
get your input.

~~~
metabeard
_whoosh_ ;)

------
scribu
A comparison with AWS's own offering (SageMaker Model Serving) would be
helpful.

~~~
ospillinger
We solve a similar problem to SageMaker but we are focused on developer
experience and flexibility.

\- Deployments are defined with declarative configuration and no custom Docker
images are required (although you can use your own if you want)

\- You have full access to the instances, autoscaling groups, security groups,
etc

\- Less tied to AWS (GCP support is in the works)

\- We are working on higher level features like prediction monitoring,
alerting, and model retraining

\- It's open source and free vs SageMaker's ~40% markup

------
jmcminis
Compare and contrast with Seldon on Kubeflow?

~~~
ospillinger
My understanding is that Seldon and Kubeflow are more geared towards
infrastructure engineers. Our goal is to hide the infrastructure tooling so
that Kuberentes, Docker, or AWS expertise isn’t required. Cortex installs with
one command, models are deployed with minimal declarative configuration,
autoscaling works by default, and you don’t need to build Docker images /
manage a registry.

~~~
jmcminis
Thanks!

I bet you could get Cortex running on Kubeflow pretty easily since it's all
K8s anyway.

~~~
ospillinger
Good idea, I definitely think it's doable.

------
hamolton
Has anyone used this? It looks useful!

