On: Why squared error? (2014)

[tonyedgecombe]

Well it will only amplify values > 1.

[esrauch → tonyedgecombe]

That's not correct. Even though the magnitudes of the value in isolation shrinks, the relative magnitudes are still amplified which is what matters.

Consider values 1/2 and 1/4: in the original space it's double but in the squared space it becomes 1/4 and 1/16 so the difference is 4x. Also relevantly if you compare eg 0.9 and 1, the gap between them is amplified after squaring.

[tonyedgecombe → esrauch]

Those values aren't compared individually, they are summed to calculate the deviation, the result of that sum will be reduced if the values are < 1.