
30+ Machine Learning Resources - mistaflu
https://medium.com/@karamanbk/39-machine-learning-resources-that-will-help-you-in-every-essential-step-b2696515ed9
======
mlthoughts2018
Machine learning is still a field that employers are very confused about and
the hiring practices vary from misguided to fundamentally misunderstanding the
entire premise of the technology.

If you really think about it, there are so many wide ranging tools in machine
learning that you simply cannot spread yourself thin enough to stay sharp and
"interview-ready" on all of it.

Just off the top of my head, I can think of:

svm, decision tree, random forest, gradient boosted trees, simple ensembling,
Gaussian processes, classical regression, complex treatment of standard
errors, Bayesian regression, hierarchical regression, mixed effects models,
structured prediction, outlier detection techniques like RANSAC or one-class
SVM, agglomerative clustering, k-means, approximate nearest neighbor methods,
missing data techniques, EM algorithm, non-parametric regression, locally
weighted regression, regularized regression, model evaluation concepts like
goodness-of-fit tests, likelihood tests, interpretation of R-squared, how
these metrics change with different data collection changes or problems,
nuanced understanding of the pitfalls of p-value reasoning, Bayesian p-values,
classical confidence intervals, Bayesian credible intervals, logistic
regression, softmax regression, ANOVA, linear discriminate analysis, graph
algorithms, graph centrality measures, Page rank, min-cut and norm-cut,
segmentation models, perceptron, and multi-layer perceptron, pros and cons of
different activation functions, pros and cons of different momentum-based
optimizers in deep learning, general backpropagation algorithm, nuances of how
complex recurrent neural networks like LSTM or GRU work, exploding or
vanishing gradient problems, nuanced understanding of weight initialization
techniques in deep learning, batch normalization, stochastic gradient descent,
nuanced understanding of learning rates, training diagnostics to detect
overfitting or sample biases or convergence, how to use deep learning
libraries to make customized network topologies, domain-specific topics like
word or character embedding representations in natural language models, bag-
of-words, language distributional assumptions, shallow neural models like
fasttext and word2vec, multi-task learning, computer vision models, face
detection, segmentation, inpainting, natural scene statistics, adversarial
networks, adversarial noise, techniques for exploring interpretation of high-
dimensional models like weights in a deep neural network, pivotal papers like
AlexNet, ResNet, Inception, glorot, cyclegan, style transfer, learned data
structures, dropout, etc. etc. Comvolutional neural networks and how the
shapes of output layers derive from inputs and filter sizes, the usefulness of
1x1 convolutions, regular max pooling as well as global pooling layers,
understanding of why max pooling is used to learn larger-scale spatial
relationships and why capsule networks are meant to address that in a more
rigorous but harder to train way. Reinforcement learning (both Q learning and
deep reinforcement), AlphaGo, monte carlo tree search, general MCMC methods
like metropolis and gibbs, and advanced ones like variational bayes and NUTS,
and how to use them with STAN and pymc, and diagnostics to monitor convergence
of sampled Markov chains, sampling methods like importance sampling (and in
what situations it reduces error), rejection sampling, inverse CDF, basic
pseudorandom number generation, sensitivity of all these things to IEEE
floating point specs and precision, digital signal process basics like FFT,
autocorrelation, time series modeling with ARMA, ARIMA, differencing, trade-
offs vs just using indicator variables or lagged regressors in more standard
models, hidden Markov Models, Kalman filters, particle filters, A/B testing,
as well as working understanding of many SQL RDBMSs, Hadoop, Spark,
appropriatness of GPUs for different methods and how to understand associated
cost trade-offs, and working knowledge of data visualization tools to
communicate about results.

... and this is what I can think of literally stream of consciousness. I am
certainly still missing huge topics and application areas.

And then consider that in an interview you can be asked about any of these,
about nuanced model fitting diagnostics, computational complexity, tricky
riddles about how does the model change if you resample the data or remove a
feature or rescale something... is it rotationally invariant, is the loss
function convex, what to do about overfitting?

And then, after all that grueling preparation and intense study, followed by
your unassailable interview performance where you heroically explain these
algorithms in great technical detail but simultaneously with "insights" and
"interpretability" that pleases the most ephemeral of fleeting political
interests... after it all, you are rewarded with a job writing maintenance
code for bullshit Spark jobs, or A/B testing a new purple button on the
advertising page that the product manager has already decided is better and
stops the A/B test early just as soon as it appears to give the desired result
with statistical significance... and you start to open your mouth to mention
multi-peeking bias in A/B testing, but you stop, because you remember that
nobody cares.

The reality is that statistical modeling practitioners should be learning the
trade-offs and nuanced evaluation details of the whole variety of possible
models _through their work_ which should (because it is value-additive to the
employer) include plenty of freedom to explore modeling trade-offs and engage
in work and professional development allowing you to stay current in a wide
variety of statistics topics.

If anyone finds any job like that anywhere, please let me know!

Until then, I guess interviewers will keep telling us we're not good enough if
we don't know all about the latest paper on arxiv that was published while we
were in the bathroom just now, and if through some miracle we get the job,
we'll be promptly asked to forget everything about statistics and just be ML-
literate devops engineers or stats-literate Excel admins.

What a field...

~~~
p1esk
To get the job you just need to be better than other candidates. If you are
good at what you do, you'll be fine. Same as in any other field.

~~~
mlthoughts2018
In practice this is not true at all. For example, I’ve been turned down for
senior-level roles in the past, roles requiring deep experience and rigour,
after doing weeks of difficult interviews and having a great rapport with the
hiring manager and getting lots of positive feedback. The reason for not
choosing me was, “we are looking for someone with less experience.”

It’s just a crap shoot in any given company. Behind the scenes you don’t know
what the power stuggle is and how it shapes which people have the power to
hire who they want vs. which people are commanded to minimize the budget at
all costs. Other companies want senior level experience at junior level prices
and are willing to leave headcount unfilled for years rather than pay
conpetitively.

And the broader point is that jobs like what I described (where you gain
meaningful experience from your actual work projects, rather than needing to
pass some outrageous barrier of expertise to get hired only to do menial work
that causes your skills to atrophy) doesn’t exist. It’s not a question of out-
competing other candidates.

------
jimmy1
Am I alone in wishing that the overwhelming majority of these weren't in
python? Are there some resources that are language agnostic?

~~~
mlthoughts2018
For language-agnostic do you mean just tutorials on the mathematics? Or do you
mean pseudocode? (which would be hard since a lot of the implementations have
to make assumptions about things in the language, like how arrays work, or
mutability, etc.)

Or do you mean just a resource that had implementations in multiple languages
from the start that spanned a wide range of practically used tools? (this
would be great, but seems like it would require a huge additional amount of
effort, both to create and also to maintain if a given example needs to
change)

------
diehunde
nice! But not even one book?

