
MLflow: An Open Source Machine Learning Platform - r4um
https://databricks.com/blog/2018/06/05/introducing-mlflow-an-open-source-machine-learning-platform.html
======
rememberlenny
If you are looking for a dependable, scalable, closed-source option, check out
[http://www.comet.ml](http://www.comet.ml) (the thing I work on).

The focus with Comet.ml is more on experiment tracking and hyperparameter
optimization rather than model deployment. We make it very easy to compare
your experiments results, code, and hashed datasets for better
reproducibility.

We have a one-line integration with your existing machine learning code and
make it stupid simple to start tracking your experiments.

All you do is:

    
    
      > import comet_ml
      > experiment = Experiment(api_key="MY_API_KEY")
    

_ __boom ___

Comet.ml supports many libraries (keras, tensorflow, scikit-learn, custom-
built code spaghetti, and everything else that makes you a ML
wizard/unicorn/armored flaming hippopotamus).

    
    
      ++ Its free for public projects and academics.

------
denzil_correa
What are the current open source alternatives to MLflow?

~~~
hcrisp
Sacred ([https://github.com/IDSIA/sacred](https://github.com/IDSIA/sacred))

FGLab ([https://kaixhin.github.io/FGLab/](https://kaixhin.github.io/FGLab/))

Metricmachine
([https://github.com/danielwaterworth/metricmachine](https://github.com/danielwaterworth/metricmachine))

Non-open source:

Neptune ([http://neptune.ml](http://neptune.ml))

Aetros ([https://aetros.com/trainer](https://aetros.com/trainer))

~~~
__bee
How about SageMaker, Can we include it in this list. I played with SageMaker
sometime ago and it helps you build a whole pipeline to host your models, in
addition to host your notebook and bridge the gap between data scientists and
data engineers.

~~~
brylie
Anecdotally, we considered using the hosted versions of Jupyter and Apache
Zeppelin that are part of AWS SageMaker and EMR. We couldn't figure out a
simple/familiar workflow for keeping the notebooks under version control. So,
we agreed to run the notebooks locally, use a familiar Git-based workflow, and
interact with the AWS infrastructure through the local notebook instances.

~~~
garysieling
Does Zeppelin work naturally with git? I've been struggling to get the right
setup with just Jupyter

~~~
brylie
Well, good question. The file format for Jupyter is not ideal for 'code
craftsmanship', as pointed out by another comment. There are utilities to
strip out some of the metadata from the Jupyter files, such as rendered output
and run counters, but that is a trade-off to be decided by your team:

[https://github.com/kynan/nbstripout](https://github.com/kynan/nbstripout)

------
axilmar
There is also Orange, although I am not sure it is 100% related to MLFlow.
Orange is a joy to use though, so even if it doesn't solve all the problems
solved by MLflow, it's worth to be mentioned in this context.

------
riku_iki
So, what MLflow can do more efficiently, than Tensorflow?

~~~
nicolewhite
I can't tell if you didn't read the article or are asking for a comparison
between this and, say, TF's Experiment class.

------
tafycent
FYI typo: mlflow.log_atrifact("roc.png")

