
Smarter Ranking with Bayesian Average Ratings - amzans
http://www.evanmiller.org/bayesian-average-ratings.html
======
contravariant
The proof in the mathematical appendix can be simplified and generalized a
bit.

To avoid confusion I'll let 'S' be the score and 'R' the random variable
representing the _actual_ rating. In that case the 'loss' (denoted by L) as
defined in the article is (R - S) if R > S and k(S - R) if R < S.

Now the goal is to minimize the expected loss E[L]. Assuming everything is
well behaved, the optimal value S should satisfy:

    
    
        dE[L] / dS = 0.
    

and therefore

    
    
        0 = E[ dL / dS ]
          = E[ dL/dS | R>S ] P[R>S] + E[ dL/dS | R<S ] P[R<S]
          = (-1) P[R<S] + (k) P[R<S]
          = (P[R<S] - 1) + k P[R<S].
    

Solving this last line results in the formula:

    
    
        P[R<S] = 1 / (1+k).
    

Now note that P[R<S] is the cumulative distribution of R, which in the case
that R has a beta distribution is the incomplete beta function.

In fact, all the formula tells you is that if being too high is k times worse
than being too low, you should try to place your estimate such that it is k
times more likely that you're too low than that you're too high. Which seems
kind of obvious really, but I guess it's nice to have a proof.

