
Why Mean Squared Error? - danijar
https://danijar.com/why-mean-squared-error/
======
tofof
I think [https://www.benkuhn.net/squared](https://www.benkuhn.net/squared)
offers a much more comprehensive view of the advantages of squared error in
about the same length.

Edit: turns out that article was actually featured on HN previously! See
[https://news.ycombinator.com/item?id=13032210](https://news.ycombinator.com/item?id=13032210)
for the thorough accompanying discussion.

------
get
TLDR:

Because it gives more weight to one big error then to multiple small ones with
the same sum.

We want the errors to be noise and not systematic. Noise usually has a
gaussian distribution. And in a gaussian distribution multiple small values
are more likely than one big one.

~~~
get
An example:

Imagine these two predictors:

    
    
        Reality: 1 1 1 1 1 9 1 1 1 1
        Predic1: 2 2 2 2 2 2 2 2 2 2
        Predic2: 3 3 3 3 3 6 3 3 3 3
    
        SumOfErrors(Predic1) is 16
        SumOfErrors(Predic2) is 21
    

So Predic1 was better then Predic2? No. Because correctly predicting the one
outlier shows more predictive power then staying close to the average.
Therefore we use SumOfSquerrors:

    
    
        SumOfSquerrors(Predic1) is 58
        SumOfSquerrors(Predic2) is 45
    

This shows that Predic2 is "better" and we are happy :)

~~~
scrooched_moose
It should be 16 & 21 and 58 & 45.

~~~
get
True. Fixed. Thanks.

------
Someone
The intuitively more logical “average absolute error” doesn’t necessarily have
a unique solution. For example: if your samples are x1 and x2, any estimator
between x1 and x2 has minimum average absolute error.

Squared error doesn’t have that problem (the midpoint beteren x1 and x2
uniquely minimizes it) and (very important historically) is easy to compute
for the linear regression case. That’s why linear regression and squared error
won. The rest, I think, is gravy. If absolute error were easy to minimize, we
might even have found/invented some other properties that ‘show’ why that is
nice.

