
Ask HN: The HN karma paradox - johnnybgoode
Am I right about any of this?<p>It appears that a comment which makes a point that's already widely accepted, whether true or false, tends to get more upvotes than a comment which is more controversial - again, whether true or false. The controversial yet true comments are usually more interesting and valuable. (Maybe that applies even to false controversial comments.)<p>Fortunately, I think most people here care more about saying what they want to say than about getting more upvotes. So I'm not saying something needs to be fixed or that I have a much better idea. Still, I think it's a reason to be cautious about taking someone's karma score very seriously. Forgive me if I'm just saying things you already know.
======
gojomo
The problem is that a single upvote-downvote axis has to capture two
different, sometimes orthogonal, judgments: (1) is this comment valuable or
detrimental to the discussion?; and (2) do I agree or disagree with this
comment?

This entanglement is unavoidable as long as there's only one score per
comment, given the interface and established practice, at HN and similar
sites.

It might work to have a second axis/score that's specifically agree/disagree.
(This could take the form of left-agree and right-disagree arrows, perhaps on
the right of the comment-meta-line.) Then a comment could be be wildly agreed-
with without offering a karmic windfall to repetitions of obvious popular
sentiments, or wildly disagreed-with without the current undercurrents of
censorship (sinking/fading-out) and karmic punishment via disagreement-
downvotes.

Personally, I would expect to find the comments with both net upvotes
(valuable) and rightvotes (disagreement) to be most interesting -- because
they capture challenging minority viewpoints, but well-presented.

~~~
egor83
It might be interesting for you to look at the rating system that's used on
the site RSDN.ru (Russian Software Developer Network). The site has been
around for quite some time and they've formulated a very balanced approach to
rating.

On that site one has 7 buttons to rate a message: 1, 2, 3, '+', '-', '+1' and
':)'

1, 2 and 3 affect score (=karma) of the message and its author based on the
level of the person who rates it (i.e. 2 from a 5th level person would add 5*2
= 10 etc)

'+' and '-' show that you agree/disagree with that comment, but do not affect
the score.

'+1' adds 1 point to score independent of rater's level.

':)' shows that you find this comment funny.

I translated rules from this page
<http://rsdn.ru/forum/Info.aspx?name=info.forum.rating> and as you may see,
there are more rules that help to prevent excessive you-rate-me-I-rate-you
behaviour, keep score up-to-date (your level depends only slightly on your
all-times score but greatly depends on the marks that you've got during the
last month) etc.

If people are interested, I can translate the rest of the rules.

~~~
stcredzero
Interested!

This has some of the same qualities as Slashdot, where you tag one of several
reasons on your moderation.

EDIT: I just thought of a more user-directed mod/metamod system along the
lines of Slashdot, but this way would be more Web 2.0. (As opposed to web
1.99a like Slashdot.)

Have a choice of pre-defined tags (about 10), each with a karma score attached
to them. These tags would be long the lines of: "Funny" "Interesting"
"Insightful" "Logical Fallacy" "Troll" Users could choose to either moderate a
comment, which would be attaching 1 or 2 tags to a comment and thus altering
the comment's visibility rating either-or meta-moderating, which would be
voting on the fairness of moderation. So you could see what tags are attached
to a comment and vote fair/unfair on each of them.

Now here's the rub: you get Karma by fairly moderating. (And lose it by
unfairly moderating.) So while moderating changes a comment's visibility, only
meta-moderating affects Karma.

Karma would be rewarded by the number of tags one could apply to a comment.

EDIT: More ideas. Users should be able to define their own tags, but only the
admins would decide what point score should be attached to them!

------
nostrademons
I think it's more that long, detailed, well-supported points tend to get more
karma than simple one-liners. IIRC, PG posted once that there was a big
correlation between comment length and score.

I've noticed much less of a correlation between popularity and score than I
would've expected - certainly much less than at Reddit, where you can write a
detailed page-long comment that completely demolishes the parent's argument,
and then have it sit at -1 because voters on Reddit reflexively downvote
articles that don't fit their preconceptions. Here, that same comment is often
at +10 or more.

~~~
eru
> IIRC, PG posted once that there was a big correlation between comment length
> and score.

Yes, he wrote that. I wrote a small script to test that proposition. It does
not hold up. If there is a correlation, it is only very weak.

You can have the source, if you wish. The script's written in Python.

~~~
visitor4rmindia
I'm interested eru - could I see the script?

~~~
eru
I have now sanitized the program. Anyone else wants to play with it?

PG, is it OK to download such massive amounts of items?

------
mixmax
There are many interesting subtleties to voting and karma systems, such as the
one we use here on HN. Another one is that a post that has a child that
glorifies the parent _"Great insight, bla.bla."_ will inevitably be voted up.
You can probably find more if you look a bit.

You assume humans are rational beings and vote accordingly. But they aren't,
they are emotional beasts that are hard to control, and are full of strange
biases and ideas about what is right and wrong.

See for instance this list of cognitive biases from wikipedia:
<http://en.wikipedia.org/wiki/List_of_cognitive_biases> Your paradox is
explained by one of the biases listed, I'll leave it to you to figure out
which one ;-)

It's all about psychology.

~~~
johnnybgoode
When did I assume humans are rational beings and vote accordingly? What I was
getting at is that widely accepted points get lots of upvotes as people
express their agreement, but more controversial points get more of their
upvotes because of the comment's quality. For the controversial points, this
is offset by downvotes from people expressing disagreement. See gojomo's
comment for a possible solution.

I'm not saying any voting system can be perfect, but this particular problem
is explained by the fact that multiple judgments are being captured by a
single voting axis, as gojomo said.

~~~
mixmax
Reading my parent comment I see that it may look a bit arrogant, sorry that
wasn't intentional, I merely think the topic is interesting, and hope to add
some value to the conversation :-)

I think that basically what's going on here is that a simplistic voting
system, such as the one we use here, and also one that carries more parameters
and dimensions will always just be a simplistic overlay on an extremely
nuanced and varied set of psychological rules that humans use for evaluating
their surroundings, thus making it hard to accurately have karma map onto a
users insightfullness (in lack of a better word) I'm pretty sure I could get a
lot of karma by using psychology tricks to make people upvote me, but it
wouldn't be very interesting. Or ethical for that matter.

So I absolutely agree with you: Karma isn't a measure of a users
insightfullness, there's probably a correlation but it's weakened by a lot of
parameters, such as the ones you describe, age of the account, and lots of
other factors.

And hey, Welcome to HN, hope you like it here :-)

~~~
johnnybgoode
I agree; there's no perfect voting system. This is similar to what you said,
but people getting lots of upvotes for saying something many others agree with
probably means at least slightly fewer unpopular views are being posted.

Thanks for the welcome! I know the account is new but I've actually been here
for a long time. :)

------
DenisM
Soon you will realize the pointlessness of the score system (no matter how
it's implemented), and from there it's a short distance to leaving it behind
and getting down to business (whatever is business to you). Treat HN as a
class which you need to graduate from and it will make a lot more sense.

~~~
johnnybgoode
I'm not saying I expect the score system to be perfect and "fair". That would
be impossible. But it isn't _pointless_ either, because it helps us sort the
articles and comments. If it really were pointless, as you say, then even the
single voting axis that exists today should be removed, right?

~~~
DenisM
> it helps us sort the articles and comments.

Are you sure about that? I find it roughly as many good comments with low
votes as I find with high votes. My most subjectively valuable comments remain
at +2 whereas my smart-ass comments get upmodded uncomfortably high. Same with
stories - half of the stories I find valuable are lingering at low scores.

Voting seems to be about social status, values and group identity, not about
content quality. There are people who try to vote quality but I rarely see
them rising above the normal voters.

I suggest you give it a try and take notes while reading both high and low
rated stories/comments. Maybe you will come to different conclusions than I.

------
ScottWhigham
Are we talking "karma for a single comment/post" or "karma for a user"? The
reason I ask is that I've been using HN since close to the beginning and never
once have I looked at a user's karma score. User karma has no meaning to me,
good or bad. Post/comment karma matters in the sorting and what appears on the
front page so, to me, it's the only thing that I care about.

So is your discussion more about user or post/comment karma? I suspect the
latter.

~~~
johnnybgoode
I guess I had both in mind. The post/comment karma for the reasons you
mentioned, but also user karma - not just because people look at a user's
karma score, but because it's used to determine other things. If it's used for
even more purposes in the future (vote weighting, for example) its importance
will only increase.

------
ttam
HN and most karma-based forums follow the bandwagon effect. The exception is
probably slashdot, due to it's different way of power assignment and levels of
voting

------
chrisbolt
What kind of point is widely accepted and false?

~~~
eru
For example (but that's kind of a meta-point): If you select any skill (say
driving) and ask a sample of people in which quantile they place themself, you
are very unlikely to get a uniform distribution. (Like: "90% of people think
themself above median drivers.")

~~~
nostrademons
But they're often _right_. Most people don't think themselves above-median
drivers, they think themselves above-average (mean) drivers, i.e. if you take
all the car accidents and divide by the number of drivers, they've been
involved in fewer than that number. And they'd probably be right: the median
number of car accidents is significantly lower than the mean, because a small
number of bad drivers account for a large number of car accidents. Not
everything is uniformly distributed - in fact, most things aren't.

~~~
eru
I know that most things are strangely distributed. Even the normal
distribution is far from uniform. So I was carefully phrasing my comments in
terms of quantiles. Quantiles are evenly distributed by definition.

People think themself "better than the typical guy" i.e. above median.

