
MNIST for ML Beginners: The Bayesian Way - tbs1980
https://alpha-i.co/blog/MNIST-for-ML-beginners-The-Bayesian-Way.html
======
Eliezer
Never use predicting stock prices as an example of anything. Pick literally
anything other than stock prices to display confidence bounds. This helpful
example diagram violates the EMH even worse by showing a large predictable
directional change. You might as well illustrate physics with a chart of
bowling balls falling upward.

~~~
kmangutov
Could you elaborate why predicting stock prices is an invalid example?

~~~
fnbr
It's almost impossible, for two reasons:

1) It's incredibly difficult to create strategies that work because there are
thousands of highly trained, highly paid people working to create strategies.

2) Any strategy that you do create will self-correct over time and become
useless, as strategies find examples where the market has misjudged prices; by
exploiting the strategy, the market should, over time, stop misjudging the
prices.

The "EMH" that the parent is referring to is the Efficient Market Hypothesis,
which states that the market price is the "correct" price, and that it is
impossible to predict a better price, as the current price incorporates all
possible information.

I, personally, don't subscribe to that version of the EMH; I believe the one
implied by 1), which is that it's difficult to compete with the thousands of
math PhDs working on Wall Street.

~~~
panarky
> It's almost impossible ...

You'd think so, and yet ML is transforming investment and trading strategies.

From The Economist:

    
    
      Castle Ridge Asset Management, a Toronto-based upstart, has achieved
      annual average returns of 32% since its founding in 2013. It
      uses a sophisticated machine-learning system, like those used
      to model evolutionary biology, to make investment decisions.
    
      It is so sensitive, claims the firm’s chief executive, Adrian
      de Valois-Franklin, that it picked up 24 acquisitions before
      they were even announced.
    

Source: [http://www.economist.com/news/finance-and-
economics/21722685...](http://www.economist.com/news/finance-and-
economics/21722685-fields-trading-credit-assessment-fraud-prevention-machine-
learning)

------
harel
I wish someone would come up with a tutorial for ML for the mathematically
challenged. Something more practical and less theoretical.

~~~
srean
I understand and am sympathetic towards the sentiment, but honestly, its a bit
like asking for a tutorial on swimming that does not involve water.

Something like that can be written, but it wont be very useful. Most of the
simple stuff, the non-mathematical parts would get automated away. You don't
want to be in a position where you are competing with someone's commodity
script (sometimes just a for loop), unless the situation calls for desperate
measures.

A bodybuilder got to lift them weights.

One genuine scenario could be that you personally do not do ML but want to
evaluate/understand what your hires are doing. Even then, its hard to do avoid
the math if you want to do a semi-decent job.

~~~
harel
I get it, and I've actually used the same metaphor just a couple days ago.
What I mean though, is that the maths are OK, and although the heavy notation
goes over my head (as I'm not academic), I understand the ideas. But I'm
lacking some practical examples of how this is put to use. I understand how
neural networks work for example, but I do not understand the maths behind how
they work. There was (is?) a site called ai-junkie which used (does?) have
very practical and layman docs about A.I. before ML became the buzz word du
jour.

~~~
jclos
Honestly, the best thing you can do is try to implement your own shitty neural
net with only Python + Numpy, from scratch, with only a basic understanding of
the math. It will make most of the math very concrete very fast.

~~~
srean
Cannot upvote this enough.

I think this is the only way to really grok backpropagation. The hours of
staring at the update formula till your eyes glaze over the subscripts and
superscripts and the summations would not give you as good an understanding as
implementing a toy neural net with just a single hidden layer. Its actually a
whole lot easier than parsing those low-level notation. It can be done better
with high level notation but then you would need familiarity with the relevant
mathematical abstractions.

~~~
annnnd
I respectfully disagree. Of course you need to _really_ understand
backpropagation for any advanced stuff, but it is easier to ignore it at the
beginning. Take Keras container, copy some MNIST example from somewhere and
tweak it. Then, when you have a general feel of how training NNs works,
gradually learn about each concept - BP should of course be one of the first.
By the time you get into math stuff it will probably make much more sense
because you will understand how it applies to your case.

But I guess the approach depends on how you best learn, so there is no wrong
answer. Just jump in!

------
calvinh123
Worth pointing out that the mean field approximation used in this example
means the posteriors are uncorrelated with each other - exactly what the
author finds in the "joint posterior distribution" figure.

------
msimpson
I wish people would stop using Raleway or implement a fix. This is what I see:

[http://imgur.com/a/locAq](http://imgur.com/a/locAq)

~~~
cromulen
Can you elaborate what the issue is? The site looks nothing like that to me. I
see much more contrast. [http://imgur.com/g49XOip](http://imgur.com/g49XOip)

I ask because I like and plan to use Raleway on my blog and would hate to have
readability issues.

~~~
msimpson
There is a known issue regarding Raleway due to aliasing in Chrome, see:

[https://webmasters.stackexchange.com/questions/74672/renderi...](https://webmasters.stackexchange.com/questions/74672/rendering-
of-raleway-google-web-fonts-in-google-chrome-browser)

I hear claims it has been fixed, but I still see the issue in Chromium 58 on
Linux.

Here's one bug which seems to have been tracking the issue with no resolution
as of yet:

[https://bugs.chromium.org/p/chromium/issues/detail?id=152304](https://bugs.chromium.org/p/chromium/issues/detail?id=152304)

You can see in the final comment from November of 2015 that the issue was, and
from my experience still is, massively affecting Raleway:

[https://bugs.chromium.org/p/chromium/issues/detail?id=152304...](https://bugs.chromium.org/p/chromium/issues/detail?id=152304#c148)

EDIT: I just checked and this issue no longer affects me in Chrome 59 on
Windows. Although I still have the issue on Linux.

