
Machine Learning 101: An Intro to Utilizing Decision Trees - markgainor1
http://www.talend.com/blog/2016/09/29/machine-learning-made-easy-with-talend-%E2%80%93-decision-trees
======
nl
Here's the truth.

If you want to work in ML, and are all excited about the (justified) deep
learning hype, you are much better off learning decision trees, random forest
and gradient boosting _first_ , and then learning neural networks.

Most of the time, for most non-binary data XGB/VW will outperform a NN, be
easier to use and more interpretable.

Plus you'll learn all the shitty programming stuff you need to do to get your
data in a reasonable shape for a NN anyway.

~~~
drxzcl
The upside of teaching decision trees first is that it allows me to spot the
people that dropped out of the online course after week one right away ;)

Seriously, I see an amazing amount of people that deploy basic vanilla
decision trees (no boosting/bagging, no particular thought about features etc)
without any real thought or reason. It has become one of my main "ML smells".

~~~
matchagaucho
In m experience, 95% of DT optimization comes from pruning; not
boosting/bagging.

------
iraphael
Blog post from a while ago, with nice visualizations, but heavy browser
computation:

[http://www.r2d3.us/visual-intro-to-machine-learning-
part-1/](http://www.r2d3.us/visual-intro-to-machine-learning-part-1/)

------
zump
In theory, aren't decision trees universal approximators like Neural Networks?

~~~
argonaut
Yes. And in theory, k-nearest-neighbors is also a universal approximator.
(Being a universal approximator doesn't tell you much since so many ML algos
are).

~~~
zump
On MNIST KNN will get you 95% but a deep net will get 98+. I find that
interesting.

~~~
argonaut
On MNIST KNN with clever feature engineering will get you 99.4%, and a crazy
deep net ensemble will get you 99.8%. I'm not sure what your point is.

MNIST is the todo list of machine learning - it's a necessary but not
sufficient condition for knowing which algos are good. In other words it's
only useful for finding out which algos _aren 't_ good (e.g. if your code only
gets 90% on MNIST you know there's a bug, but if your code gets 99% on MNIST
it doesn't really mean much).

