

Ask PG: More transparency on average karma? - gjm11

[Note: Yes yes yes, obsessing over karma is unhealthy, we should focus on actually making a valuable contribution, etc. Consider all that said.]<p>The "average" karma displayed on a user's page is computed as follows. (1) Take the user's 50 most recent comments. (2) Delete the latest 5 (because their scores may still be increasing, making their current values unrepresentative), and the highest-scoring of the others. (3) Take the average of the remaining comments' scores. This is done asynchronously; empirically it seems like it's every few days.<p>This seems like a sensible system -- it's not too sensitive to outliers, ignores comments whose scores may still be changing rapidly, etc. -- but because it ignores recent events <i>and</i> has an unknown, variable delay in it (because it's evaluated asynchronously) <i>and</i> isn't clearly documented anywhere, it's not great for user feedback.<p>I suggest (with the understanding that there might be one or two other calls on PG's time) that<p>1. there should be some documentation (e.g., a link saying "(what?)" after the average score, going to a short page describing the algorithm and mentioning that the number may be a few days out of date;<p>2. visiting a user's user-page should trigger an update of their karma if it's more than an hour out of date -- unless user-pages are visited so frequently that that would be too expensive;<p>3. when the average of the most recent 5 scores is &#62;= the average of the others, they should not be excluded from the calculation.
======
ColinWright
To further expand on the update interval, I believe that every 45 seconds or
so a user is selected at random and their average karma calculated. I don't
know if any measures are taken to prevent starvation or a user - there are (of
course) simple techniques to prevent that. Whether they're necessary is an
interesting question.

------
brudgers
[IANPG] Just out of curiosity, what would be the benefit of the proposed
changes?

~~~
gjm11
Benefit of change 1: It would be easier for users to understand the otherwise-
mysterious fluctuations in their "average" karma.

Benefit of change 2: Average karma estimates would never be far out of date.
(This would also make it easier for users to see what's going on.)

Benefit of change 3: Average karma would be more representative. The point of
ignoring the 5 most recent changes is to avoid giving a misleadingly low
average when recent comments are still gaining karma -- but if a user's most
recent comments happen to be high-karma ones, the result may actually be to
lower the estimated average, taking it further away from what the average
would be if the scores all had time to settle down. Again, this is liable to
be confusing to users.

(The specific situation that prompted this: My "average" karma is currently
substantially lower than (1) the values it's generally had in the longer term
and (2) the scores of recent comments I've made. I found this puzzling -- I
knew about the ignore-highest-score trick but not about the ignore-five-
latest.)

~~~
brudgers
[IANPG] Since it doesn't result in better posts or comments, it probably
doesn't improve HN.

~~~
gjm11
It's worth distinguishing immediate from indirect benefits.

The purpose of the karma system is to encourage better posts and comments. To
do this, users need to be able to see and, preferably, _feel_ the link between
karma and post and comment quality. Their ability to do this is impaired if
the link is subject to delay and confusion.

Of course this is only true in so far as users are being guided by the
"average karma" figure. But if they aren't, then either it shouldn't be there
or the principle you're appealing to is incorrect.

Just out of curiosity, do you think that the black bar displayed at the top of
HN pages when certain people die results in better posts or comments? Or would
you consider the possibility that, important as those things are, they aren't
_all_ PG cares about? I think they aren't; I think he probably cares to some
extent about how pleasant HN is to use.

~~~
brudgers
Of course, I'm probably as much of a karma whore as the next guy...but I take
my karma average with more salt than the score of individual posts because it
offers less direct editorial feedback.

The link between numeric score and quality of post is tenuous. Good comments
in low interest threads may still wind up as one point, and well timed snark
can score in the double digits (though less likely now that scores are
hidden).

The difference I see between the average karma mechanics and the black band is
that while the former is of individual concern, the latter fosters a sense of
community.

