

Ask HN: How to design a karma reward system - yannis

I am building what is actually a large portal for a 'tutoring site'. Visitors will be able to join groups, create groups and post comments, lessons etc. They will also be rewarded for completing on-line exercises.<p>I would like to introduce a 'karma reward system' in order to a) motivate people to contribute and b) to crowdsource ratings both for people as well as content.<p>I am leaning to something that is in-between what http//thesixtyone.com offers and HN's or stackoverflow's floating method of ratings.<p>I am looking for suggestions, formulae and good links to resources, books etc as well as other sites than the above three that you favour.
======
JeffJenkins
What I've wanted to try but I have never had the opportunity to was having
karma be accumulated on a per-topic basis. I think any time you've got a wide
set of topics this is necessary to avoid the issues of people being treated as
credible outside of their domain of expertise, and it could help mitigate the
issue on Stack Overflow where people who answer lots of easy questions can get
much more karma than those who answer a few much harder ones.

~~~
yannis
Jeff Thanks! What you saying is very true of stack-overflow and although I
visit occasionally their whole karma formula does not appeal for the reason
you mentioned. I was thinking in similar lines and have introduced the groups
concept for example when you belong to the 'lisp circle' you will be able to
earn additional points when you answer or solve lisp problems. My concern is
not to let the users focus too narrowly, I was thinking that joining groups
should be limited i.e, when you accumulate 100 points you can join a group and
as you earn open up more groups... To link the whole thing in a sort of
'PageRank' algorithm to stop people from gaming the system is what is puzzling
me.

------
Perceval
I've posted this on HN before, but I wrote a long essay about problems with
account-karma-moderation systems:
<http://www.kuro5hin.org/story/2009/3/12/33338/3000>

Probably too involved for your project, but there are some problems you should
be aware of from the get go.

~~~
yannis
Perceval thanks and by the way an excellent essay. You hit the nail on the
head with ... 'our hypothetical comment-driven forum would push stories up
based on quality of conversation....'. Do you think that part of the formula
can be improved by including automatically a 'readability formula', where the
childish comments and language would automatically be penalized by the system?

~~~
Perceval
Some folks are already working on that: <http://stupidfilter.org/main/>

It's supposed to identify _formally_ stupid comments, i.e. those that have bad
spelling and txt abbreviations, etc. It doesn't do anything about substantive
stupidity, unfortunately ;)

~~~
yannis
I was thinking something like the _Gunning fog index_
<http://en.wikipedia.org/wiki/Gunning_fog_index>

------
anamax
> I would like to introduce a 'karma reward system' in order to a) motivate
> people to contribute

What evidence do you have that any karma system motivates people to contribute
the kind of content that you want?

Seriously - look at the top karma people on HN. What fraction of them post for
karma? Of the ones who do, how many of them do you want?

> and b) to crowdsource ratings both for people as well as content.

Again - what evidence do you have that karma helps with this?

A karma system guarantees that you'll have people posting for karma and
discussing it. Both are bad. What benefits that you can reasonably expect that
are more valuable?

------
babyshake
It depends on your choice of language and some specific implementation
details....and whether you are more interested in the user interface side, the
data modeling, or both.

There's a range of approaches to motivational systems, and I think in 2010
we'll see some real innovation in this area. For instance, I think Gowalla is
doing a particularly great job.

~~~
yannis
Thanks! I added Gowalla on the list! I am more interested in the data modeling
and algo and good examples like Gowalla.

------
Perceval
Here's a blog that might be relevant: <http://buildingreputation.com/>

~~~
barmstrong
Some great articles on there...

------
karanbhangui
I'm planning on working on something potentially very similar. Here's an
outline post I made a while back: [http://karanbhangui.posterous.com/course-
learning-aid-propos...](http://karanbhangui.posterous.com/course-learning-aid-
proposal)

Lemme know if you're interested in collaboration.

~~~
yannis
Thanks I will send you an email! Is that UWaterloo like in the UK?

~~~
selven
The University of Waterloo ( <http://uwaterloo.ca/> ) is in Ontario, Canada.

------
barmstrong
Related question: does anyone have any info on using a super simple voting
system (thumbs up/down, or just up) that does NOT suffer from time bias. In
other words, an item might have more up votes just because it's been around
longer.

~~~
selven
Bias your comment ordering system in favor of new comments so that every
comment gets its 5 minutes of fame before staying around with a good score or
floating to the bottom with a bad score. it's not perfect, but it should
mitigate the problem.

------
dustingetz
stackoverflow has a few interesting incentives:

\- activity is renamed to 'rep' which implies that your activity ranking
reflects on you personally, even though everyone knows its really a measure of
how long you've been active and whether you answer broad beginner questions
that get a lot of google hits

\- it makes sure you know when you are downvoted

\- though the numeric penalty of a downvote is negligible, but since it is a
personal criticism of your reputation, you have ego incentive to improve your
contribution

------
Mathnerd314
<http://public.research.att.com/~volinsky/netflix/>

For all your rating/karma/rewarding needs :-)

------
gojomo
Read/watch all the Amy Jo Kim presentations you can find online.

~~~
yannis
Thanks lots of videos - will need three days to view them! :)

