
A Course in Machine Learning - federicoponzi
http://ciml.info/
======
kmax12
A lot of easy to digest content in this! Always great to see quality free
material to help more people pick up machine learning and get involved solving
problems using data science.

One thing I didn't see covered in depth here was feature engineering, which is
the process of preparing your raw data for the machine learning algorithms.
They cover it briefly in the chapter on "Practical Considerations", but anyone
looking to apply ML in the real world should look into feature engineering
more on their own.

One resource I recommend (and I am biased), is a python library for automated
feature engineering called, Featuretools
([https://github.com/featuretools/featuretools/](https://github.com/featuretools/featuretools/)).
It can help when your raw data is still too granular for modeling or comprised
of multiple tables. We have several demos you can run yourself to apply it to
real datasets here:
[https://www.featuretools.com/demos](https://www.featuretools.com/demos).

------
jpamata
With all of these online courses around, I'm just curious: is there an ML
course that teaches you the right model to use? Say for example, the right
amount of layers/nodes for a neural net? As a newcomer doing one of these
machine learning MOOCs on his free time, it seems to me that it's about
chucking in a load of parameters into a black box hoping for the best.

~~~
joshvm
Usually you throw everything and see what sticks. There are some obvious
trends like if you're doing image classification, a convnet will beat just
about anything else. There are problems which are particularly well suited to
e.g. random forests and cases where you might want to use a simpler model
which will run faster at the expense of a bit of accuracy.

In terms of neural nets, people have more or less done the hard work for you.
Typically you'll want to take an empirically well-performing network like VGG,
ResNet, Inception, etc and then re-train the top few layers. There is ongoing
work in the field to try and train the structure of the network as well.

If you look at Kaggle, the vast majority of winners do so using ensembles. In
a recent example - Iceberg or Ship - the winning team used a bit of feature
engineering and apparently _over 100 ensembled CNNs_ :

> We started with a CNN pipeline of 100+ diverse models which included
> architectures such as customized CNNs, VGG, mini-GoogLeNet, several
> different Densenet variations, and others. Some of these included inc_angle
> after the fully connected layers and others didn’t.

[https://www.kaggle.com/c/statoil-iceberg-classifier-
challeng...](https://www.kaggle.com/c/statoil-iceberg-classifier-
challenge/discussion/48241)

It's an ugly kitchen sink approach, but it works.

~~~
vinn124
> Usually you throw everything and see what sticks.

most practitioners start with the simplest possible learner, then gradually,
and thoughtfully, increase model complexity while paying attention to
bias/variance. this is far from a "kitchen sink" approach.

~~~
joshvm
Certainly that's what most _sensible_ practitioners do. I somewhat doubt that
most people follow this to the letter every time.

It's a nice theory, and it works intuitively with models where you can ramp up
complexity easily (like neural nets). It's less obvious if you have a "simple"
problem that might be solved with a number of techniques. In that situation I
don't see why you would be criticised for trying say any of SVM, random
forest, logistic regression, naive bayes and comparing. Pretty much the only
way you can categorically say that your method is better than others is by
trying those other methods.

The simple approach actually came up in the iceberg challenge. The winning
team won because they bothered to plot the data. It turned out that the
satellite incidence angle was sufficient to segment a large number of icebergs
with basically 100% reliability. So they simply thresholded that angle range
and trained a bunch of models to solve the more complicated case when there
might be a ship.

------
cs702
I quickly skimmed a few chapters and like what I see: the language is very
friendly and the examples are easy to understand, but there is enough rigor to
make this a good introductory book on the subject. Nice!

------
connectsnk
I have not yet found a course that can cover machine learning and also
explains the Maths needed to understand this.This course is no exception. Its
extremely difficult for a programmer with no solid footing in Maths to
understand it. Any help is a lot appreciated

~~~
f4stjack
True, very true and you'll hit that language barrier in no time. The courses,
mostly, provide hands-on experience and don't explain, say for example, what
does standard deviation mean?

But this is a good news/bad news kind of thing. Bad news, you need some
statistics education to make a sense of what your computer is telling you.
Good news, that mathematics isn't "high-grade" mathematics involving integrals
and other stuff (as far as I can see anyway).

So what I am doing as a person who is trying to convert himself into a Data
scientist is I am taking a basic udemy course ([https://www.udemy.com/python-
for-data-science-and-machine-le...](https://www.udemy.com/python-for-data-
science-and-machine-learning-bootcamp)) to give me the basics and a book on
statistics to properly grok what I am seeing on the screen. On this part I am
following a different book than its peers though, so it might not be for you
but I like the story-telling aspect of it ([https://www.amazon.com/Adventure-
Statistics-Reality-Enigma/d...](https://www.amazon.com/Adventure-Statistics-
Reality-
Enigma/dp/1446210456/ref=sr_1_1?ie=UTF8&qid=1528016826&sr=8-1&keywords=an+adventure+in+statistics))

After I finish those I'll move to Kirill Eremenko's a-z courses on Machine
Learning, AI and Deep Learning. I've found that even though they teach cool
tricks and some basics before going in detail, that basics part didn't contain
enough information for me as a new student. So I feel if you have some proper
background in stats and python data analysis you can skip the parts I
mentioned and go straight to a-z courses.

~~~
connectsnk
Thanks for your insights. I was wondering if one also needs a footing in
vector calculus / linear algebra / integrals and other such words I have no
idea about.

Also Kirill Eremenko has 38 courses listed. In what order should one take
them?

------
rheide
Wow, this is a lot more in-depth than I was expecting. I started out with the
O'Reilly book (Hands-on Machine Learning)[1]. There's some overlap there, but
I'd say both this course and the O'Reilly book are well worth your time if
you're starting out with machine learning. [1]
[https://amzn.to/2kKaNiQ](https://amzn.to/2kKaNiQ)

------
Keloo
So many ML/AI courses out there, it's amazing.

Wondering if those ML/AI specialists can't find themselves a job and start
writing courses -.-

~~~
mlthoughts2018
There is some truth to this. Many jobs in machine learning are all bark and no
bite. The company may even have created a machine learning team solely out of
hype, and just equivocates machine learning with making d3.js visualizations
or maintaining Spark jobs that just tabulate summary statistics.

Yet these jobs will still require you to do outrageous things during the
interviews, like deriving the full backpropagation formulas for a 3-layer MLP
network, or explain some esoteric issue with vanishing gradients or offer from
memory a bunch of time complexity info about the SVM fitting algorithm, etc.

They require exponentially more impressive knowledge about machine learning in
the interview than what the job experience will actually offer once you’re
hired. Most positions will fundamentally make your skills atrophy.

As a result, a lot of people resort to writing blog posts or courses about
their experiences implementing toy models, studying trade-offs between
approaches, analyzing publicly available data sets.

In part it’s to help pad a resume and look relevant for getting hired. In part
it’s to build or exercise skills that the person’s day job won’t actually
enable them to use. And in part to try to get your name out there and
associated with a hyped up field.

~~~
thraway180306
I'm not questioning your analysis of behavioral psychology of the herd, but
this particular example, prof. Hal Daume of University of Maryland and
Microsoft Research has been long associated with the field, though not
particularly neural networks, since way before the current hype.

~~~
mlthoughts2018
I totally agree for this specific link. Many well-respected and established
people in machine learning write course materials.

I was only responding to the parent comment regarding why random blog posts,
articles, and course materials seem so widespread and constant in machine
learning overall.

------
scarce
Thanks for breaking down a concept that is completely overwhelming into simple
pieces of information!

Great work!

