Machine Learning Deployment Frameworks

[inf3cti0n95]

What are the different alternatives for ML Model Deployments and Pipelines and Pros and Cons for each.

I have came across few

- GraphPipe (https://oracle.github.io/graphpipe/#/) from Oracle

- Tensorflow Serving, TFX (https://www.tensorflow.org/tfx/) from Google

- Mlflow (https://mlflow.org) from databricks

- kubeflow (https://www.kubeflow.org/) from Google

- Seldon (https://www.seldon.io/)

I am looking towards some perspectives such as features, community support, ease of use, framework support such as (PyTorch, Tensorflow, etc), scalability.

[pplonski86]

Could you give some more details of what are you trying to build?

What kind of algorithm have you used for model building?

How frequently would you like to request the model for predictions?

Do you need scheduled model updates?

[inf3cti0n95 → pplonski86]

The problem is, I have to build a generic deployment system for the researchers. To deploy their Model.

Inference frequency would be quite low, but multiple models could be deployed at time.

[pplonski86 → inf3cti0n95]

I think generic deployment system can be very difficult to build. For example, there can be problems with library dependence or data format dependence. I think you should select one 'family' of models, for example narrow down to only python scikit-learn models, and try to build a system for them.