
Show HN: Xcessiv – Fully managed web application for automated machine learning - reiinakano
https://github.com/reiinakano/xcessiv
======
zitterbewegung
This looks really neat. But, I would put a summary of the link you have at the
bottom of the README (See [https://mlwave.com/kaggle-ensembling-
guide/](https://mlwave.com/kaggle-ensembling-guide/) ) so that people know
what stacked machine learning is. People that already know this would be sold
on the software. For everything else though you have really clear examples and
I can see how user friendly everything looks like.

~~~
reiinakano
Thanks for the feedback! Will do.

------
daddyo
Only a few HN submissions each year are so complete that it is nearly
impossible to gather comments to start a discussion. Well done!

I wonder how close we are to running these "excessive" ensembles in a
production environment.

Like how we went from using decision trees to random forests, it seems to me
only a natural progression to move from random forests, to a random forest of
random forests.

Some Kaggle competitors use over a 1000 RF estimators in their ensemble, but
this is not yet possible/pragmatic to put in production for most use cases.
But an ensemble of 10 complex base estimators is already within reach for
applications that demand the highest accuracy.

About the Netflix prize, the engineers said:

> This is a truly impressive compilation and culmination of years of work,
> blending hundreds of predictive models to finally cross the finish line. We
> evaluated some of the new methods offline but the additional accuracy gains
> that we measured did not seem to justify the engineering effort needed to
> bring them into a production environment.

So it also depends on the additional gains, if going the route of complex
ensembles makes any business sense. But the next 20 years can make a lot of
difference.

Anyone have experience putting complex ensemble models in production?

Another progress I find really interesting is the
[https://arxiv.org/abs/1701.06538](https://arxiv.org/abs/1701.06538)
"Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts
Layer". It seems possible to learn how to selectively prune a giant ensemble,
to select a handful of experts that do well on particular samples. This makes
it computationally feasible to get predictions from a giant ensemble. In the
paper they solely use neural nets, but I guess there is no reason to not try
this with other models, like SVM's or gradient boosted decision trees.

------
pplonski86
I like the integration with TPOT. This makes whole solution more automatic.
Have you compare with [https://github.com/kaz-
Anova/StackNet](https://github.com/kaz-Anova/StackNet) ?

~~~
reiinakano
The two main differences are programming language (StackNet is Java) and the
fact that Xcessiv is more a tool (heavily inspired by the flexibility of
Jupyter notebook) and StackNet is a library.

