
Implementing a Gaussian Naive Bayes Classifier in Python - amallia
https://www.antoniomallia.it/lets-implement-a-gaussian-naive-bayes-classifier-in-python.html
======
pedrosorio
It seems odd that the key feature of Naive Bayes (conditional independence of
individual features given the class) is only mentioned in passing (and never
explicitly that they are conditionally independent).

This feature is then used when transforming probabilities to log-probabilities
in one go without any mention which would make it particularly confusing for
beginners.

I would recommend decomposing p(x|C) before applying the log transform for
clarity.

~~~
amallia
Thanks for pointing out! This is a blog post for people that know already a
bit about the theory and are interested in a possible implementation. But I
see your point and will add it.

------
nerfhammer
What does it mean to use the probability density function of the gaussian
distribution as a "probability"?

the probability of any x value in a continuous range is infinitely small. Is
it really using the probability of a value "as extreme or more" as x?

~~~
amallia
If you have a bag of [1.0 3.1 5.2 7.8 7.8 7.9 8.1 8.2 9.9 10.1] what is the
probability of picking 8.0? It is definitely higher then 1.1, isn't it? I hope
this clarifies the pdf usage.

~~~
nerdponx
You have an interval of real numbers [0, 1]. What's the probability of picking
0.3639401?

~~~
nightcracker
If we're using a uniform random number generator in [0, 1] with 32-bit IEEE754
numbers, the closest to 0.3639401 is

    
    
        2^-2 * (1 + 3823195*2^-23) = 0.3639400899410247802734375
    

The numbers directly before and after it are:

    
    
        2^-2 * (1 + 3823194*2^-23) = 0.363940060138702392578125
        2^-2 * (1 + 3823196*2^-23) = 0.36394011974334716796875
    

Giving a range of size 2.980232238769531250 * 10^-8 in which all numbers
compare equal to 0.3639401 in IEEE754 32-bit floating point. And since we're
looking at a domain of [0, 1], that's also immediately our probability.

(I'm fully aware this isn't what you were asking, but I found it fun either
way)

------
myegorov
Might be worthwhile to add conditional risk to it, to generalize it to the
minimum risk classifier. That way you'd also distinguish it from, say the
scikit-learn implementation.

------
4tomX
Great topic!

~~~
amallia
Thank you dude!

------
vinni2
What’s so special about this post? There are tons of ML tutorials online! Why
was it upvoted to front page?

~~~
amallia
I honestly couldn't find many implementations of Naive Bayes out there. The
famous ones are over engineered to use it as a learning too. I think people
appreciate the fact that an article like this for its step-by-step approach.

~~~
aaronbrethorst
I went looking for something just like this a few weeks back. The examples I
found were all either wildly over-mathematical or terribly unclear. This seems
to strike a good balance. Thanks for the writeup!

