
Fixing Hacker News: A mathematical approach - spion
http://gkosev.blogspot.com/2012/08/fixing-hacker-news-mathematical-approach.html
======
citricsquid
This doesn't address the real issue. The issue isn't that people are voting up
comments or submissions that don't _fit_ with what HN is _supposed_ to be, the
problem is that those submissions and comments are being made.

Even if the top submissions and top comments fit squarely with what HN is
_supposed_ to be there is still going to be comments and submissions that
don't and they will continue to be a problem. If a post has 50 comments and 25
are "bad" it's a problem, even if the 25 "bad" comments are at the bottom of
the page because people will still reply to them and they will still be a part
of the discussion.

~~~
rachelbythebay
Isn't this the same pattern as yesterday's "bozo" post? The general principle
there was "B players hire C players and so on". Granted, we're talking about
posters and topics instead of "players", but couldn't it be the same thing in
a nutshell?

"People are voting up comments or submissions that don't fit..." sounds just
like "People are hiring people who don't fit..." to me.

Or am I wrong?

~~~
debacle
Can you link to the post from yesterday? I didn't see it, but it sounds
interesting.

~~~
rada
The Bozo Event Horizon: <http://news.ycombinator.com/item?id=4409760>

------
acabal
Interesting, but with an important flaw the author acknowledges:

> The important downside of these results is that the people using the system
> were not aware that points are calculated in a different way.

As I mentioned in that HN-meta-whining thread, you can't engineer away jerks,
and social problems need social solutions. People will naturally adapt to any
sort of engineered restrictions and find away around them regardless.
Personally I think the only way to maintain a very high standard of quality in
a community is strict hands-on moderation and swift ejection of members who
don't match the community's goals and vibe.

(In either case I think the thread this post refers to was about the
negativity in comments, not the quality of submissions. Though I might be
remembering wrong.)

~~~
buro9
"Personally I think the only way to maintain a very high standard of quality
in a community is strict hands-on moderation and swift ejection of members who
don't match the community's goals and vibe."

Many communities I've seen have done this in a way that drops-off over time.

Screw up any of your first 3 posts... you're out of here.

Posts 3 > 10... you're going to get a warning, and may still be out of here.

Posts 10 > 25... warnings only, only extreme cases get banned.

Above 25 posts you're largely left to it, though once in a blue moon someone
will screw up so spectacularly that they'll get pulled up by the community.

Even then, the "pulled up by the community" is literally that. No moderators
or banning, but the communities that enforce rules strictly for new members
and than relax once you get comfortable within the constraints, generally are
also the ones in which if you suddenly made an extremely racist post the
solution is to be ripped to shreds by your peers rather than to censor and
hide it... it reinforces the "that behaviour isn't acceptable here".

This is currently the most successful pattern I've tried. It works far better
than other systems I've seen.

~~~
buro9
Of_Prometheus: There is a deep irony in your post, in that it's marked as dead
and not visible to many.

The question was, "How does that system deal with the old members > new
members hierarchy that would develop? I think the problem with giving a member
a longer stick the longer they stay/comment is that older members are given
greater leniency when they do decide to be jerks, and new members who may very
well develop into excellent members are kicked aside for possibly minor
errors, thus creating an unfair advantage."

That hierarchy does exist. I'm not seeking to eradicate it.

The older members tend to be the ones that have established the tone and
quality. This can be used to determine the rules that are used in the early
stages of membership of a site.

But generally that hierarchy refers to inner circles and cliques who are
protective of what the site evolves into for them, and this isn't addressed in
any way by the system to integrate new members.

Old members will eventually leave, the question is more whether the value of
the community is preserved when that happens.

~~~
viraptor
Of_Prometheus: you're probably hell banned. You'll see your posts, but others
who didn't explicitly enable the option will not.Which is also something that
I see a lot lately... pretty well written comments from hell banned accounts.
Makes me a bit uneasy - maybe there was some reason for those actions, but I
rarely see one in the author's history.

------
thornofmight
It's so upsetting being a 21yo just getting started in the industry. I finally
find this awesome site with like-minded, motivated, and successful people, but
all anybody can talk about is fixing it!

It seems like I'm always on the tail-end of communities. Is this just simply a
case of the grass is greener for you veterans of HN?

~~~
debacle
It happens. If you're a risk taker and willing to be a part of multiple social
networks at once, you can be on the bleeding edge of good communities but how
much are you really losing out on? The next good community will spring up
sooner rather than later, and we'll all visit it, for a while.

Communities seem to have a 2-5 year lead in cycle, followed by 1-3 years of
good, healthy community. Then they get mainstream, bloated, and meta.

But they never really die, which is the saddest part. I can go to social
networks I signed up for over a decade ago, and people are still stomping
there.

And they are angry. Angry at being deserted, or angry because they put all
their eggs into one basket, but either way you wind up with little hovels all
over the Internet that are remnants of the gold rush for good discussion.

Some day, the Internet as a whole will find its El Dorado. One thing's for
sure - it's probably not Facebook.

------
lysol
Another approach is to limit submissions via some manner. Metafilter still
retains a similar level of content as it did when it started around 12 years
ago, and they accomplished this by a) Requiring signups to pay $5 b)
Maintaining strong, public moderation where deletion reasons are clearly shown
and frequently discussed ad nauseum on a section of the site clearly
designated for talking about the site itself.

It may not be everyone's cup of tea but the site has grown leaps and bounds
and hasn't yet strayed from its original goal, which is albeit more open-ended
than HN. I'm not suggesting users pay $5, but there are more nuanced
approaches than trying to squash the problem with the math hammer.

------
zerostar07
I 've seen many solutions, and personally i believe it's a psychological
matter of attention overload: too many submissions / comments -> less time
spent on critical review -> hasty "cat picture" upvotes.

Let's create artificial scarcity: let each user be able to vote only on a
small random sample of submissions/comments

Other ideas: Require approval by 2 blindly selected users before showing, or
allow multiple dimensions in voting[1]; but these don't seem to work.

Anyway, this is a deep issue, essentially political [2]. It's also fun to
consider the dizzying election procedure that the Venetians used [3].

[1] <http://textchannels.com/>

[2] <http://en.wikipedia.org/wiki/Criticism_of_democracy>

[3]
[http://en.wikipedia.org/wiki/Doge_of_Venice#Selection_of_the...](http://en.wikipedia.org/wiki/Doge_of_Venice#Selection_of_the_Doge)

~~~
ansible
_...let each user be able to vote only on a small random sample of
submissions/comments_

Slashdot first pioneered that. In case you are not familiar with the scheme,
all logged-in users are occasionally asked to vote with their 5 allotted
points. Users with high karma are given a chance to vote more often. The
points expire in a day or so if they are not used. They also have a meta-
moderation system, where you judge how others have used their votes.

They've had the present system for a while (I want to say over 6 years, but I
don't frequent the site as much these days). The overall quality of comments
there is... OK. Not great, but not as bad as it was for a while around the
turn of the millenium.

~~~
icebraining
The quality of Slashdot comments depends a lot on your default threshold. If
you hide every post with score < 2, it's decent. If you include lower scores,
it drops considerably. -1 is almost masochism.

~~~
ansible
Yeah, I have my account set to show 3+ comments by default.

I also appreciated the changes they made when comments marked as 'funny'
didn't count towards your karma, and made it possible to filter based on that
too.

------
politician
One problem is that there isn't a feedback loop describing why a comment or
article is bad, just that it _is bad_. Negative context-less signals lead to
_superstition_ which induces drift away from the values of the Creators.

One way to provide context could be to allow people to tag their downvotes
with an adjective like "irrelevant", "mean", "notreddit", etc. Then,
critically, only the author of the comment would be able to see this
additional feedback.

------
brudgers
I remain unconvinced that HN needs "fixing" though I am certain it could be
improved. I also believe that the improvements are largely along the lines of
better HTML (e.g. hiding comment scores) than more complex functions for
calculating the value of a karma point.

Karma has the most influence on long term behavior - HNers have to learn
community mores over time, and one vote one point is well suited for that
task.

------
markkat
I think this system could work.

One problem with community forums is that there is a drift to the 'majority
taste'. As the population changes, so does the majority taste. This could put
some drag on that drift, but you'd probably need to use historical voting data
if it were to pull things back to the HN center. Otherwise it might just hold
the community in a place where HN doesn't want to be.

------
robomartin
HN isn't perfect, but it isn't that bad. I've seen countless fora degenerate
over the years. This one seems to be in check for now.

I see the voting system as one of the problems. More accurately, the obvious
emotional or fanboi voting trends along some topics. Blind down-voting, as I
see it, is a problem. If you down-vote you ought to be required to provide an
explanation, otherwise, don't. Topics such as many subjects surrounding Apple
draw in people who will mercilessly down-vote well reasoned comments simply
because they happen to be anti-Apple. The same applies to the occasional
political discussion in which anything that isn't aligned with liberal
thinking is castigated.

The biggest problem I see with this is that it does not promote intelligent
contrasting posts. Those with different opinions either leave (or stop
posting) or fall-in with the crowd and choose to become part of the
collective. If one was after collecting points and accolades all you have to
do on HN is be all over anything that relates to Apple and talk like a
Liberal. That's the formula. Oh, yes, you also have to be 100% pro FOSS.

The fact that it is so easy to define is kind of sad. Then again, maybe that's
what the YC guys wanted HN to be. If that's the case, there are no issues. If
the idea behind HN was to bring together pro-Apple, pro-FOSS, politically
Liberal hackers, then there's nothing to fix.

If, on the other hand, if HN is intended to promote diversity of thought along
a wide range of subjects of interest to hackers and entrepreneurs, the voting
system needs to reflect this.

~~~
Someone
_"If you down-vote you ought to be required to provide an explanation"_

Here is an idea:

\- require downvoters to provide that explanation.

\- Use some simple automated method (e.g. run the text typed through a
language recognizer, and see whether it thinks it is English) to prevent
simple key-banging as a response.

\- by default, do not show the explanations, but do show a 'has downvotes'
indicator and a 'show downvotes' control (could be a single UI element)

\- when showing downvotes to a user, do not show author names.

\- allow users to flag downvotes as inappropriate.

\- let the powers that be manually check flagged downvotes (hopefully, there
aren't many), and take appropriate action against either the downvoter or the
flagger, or both.

------
dinkumthinkum
Maybe it would help if we decided what the problems really were? Everyone
seems to agree that it is completely broken; I'm not seeing it. I see a few,
what I consider to be, very bad comments occasionally and they tend to be
down-voted out of visibility, so it seems like the current system sort of
works? Is the problem more related to there being people going against the
accepted groupthink? It feels a little like "grass is greener" to me. No
community is perfect; to me it seems like the main issue, and I don't think
it's a real problem, is the there seems to be a growing amount of non-
technical people, which is OK but it will tend to change the content of
discussions quite a bit if the trend continues. Overall, I'm obviously in the
minority, I don't think there is a dramatic problem.

The post that started this recent wave, someone was, I suppose, genuinely
upset about not receiving good comments to a "look at my project post."
However, a user looked through posts and actually found a lot of praise ...
So, I don't know what to think about that.

------
zobzu
/install slashdot ;-)

oh yeah, i might get down votes - but - its still one of the massive comment
systems that more or less works. more.. or less. :)

~~~
stcredzero
Unfortunately, the most famous installation of this code is saddled by the
editors of Slashdot.

------
think-large
I would be very interested to see this implemented in a side by side
comparison. I think it would be great to see the type of content that would
float to the top of HN.

I also have to say that I love this site. I feel that the content of the posts
have made me a better programmer in general and I really appreciate everyone's
input. Even if the discussion threads spiral out of control quickly.

It would be nice to have expandable comments so that when people begin name
calling we have an easy way of filtering them out on screen and just viewing
the people who directly address the post.

------
debacle
You can't fix the HN problem without people.

If pg really wants to preserve the feel of HN, he needs to develop a system
that analyzes the behavior of users to determine their acceptance level of
other users. This will create a network effect, where the most influential
users will drive the site.

He then needs to seed the system with 5-10 users who he considers paragons of
HN (I can think of five off the top of my head, but his are almost certainly
different), and tweak as necessary.

------
rossjudson
Seems to me the missing element from your analysis is simple: Users don't need
to have exactly the same experience. Picture a graph where each user is a
node. When I freshly arrive at the site, it tells me to pick out a few users
to place in my "circle", to use Google terminology. I do this by finding users
who write comments I think are good, or by looking at a list of some of the
most popular users. I can then set the degrees of Kevin-Bacon controller to
set expand my "circles" outward by joining in people that others in my circle
trust, multiplied by a reduction factor (farther out means less influence).

Only votes by members of my extended circles count towards my "score" for a
given item, weighted by distance. If something is upvoted in my perspective, I
can see who and why, and break that connection (remove a person from my
circles).

Finally, I can publish a group of users I trust and give it a name, which
makes it easy for others to follow well-known groups.

I think it would work. A user has to put in a little effort, to find people
they think have good judgement. It's very easy to knock out bad judgments by
breaking links.

If I were less tired I'd think this through. The downside to all of this is
processing time, of course.

------
eranation
Newcomer question, how can I know if my comment / post fits the HN "good" or
"bad" qualification? is it something describable? or is a bozo doomed to be a
bozo?

~~~
SkyMarshal
Everything is learnable. For the poster-child of good posts, see Patio11's
comments:

<http://news.ycombinator.com/threads?id=patio11>

Also, know the official etiquette/guidelines:

<http://ycombinator.com/newsguidelines.html>

------
stefanve
I always liked the idea of adding meta data to your up vote (funny, insightful
etc) just choose from a list, and people can filter out category's they have
no interest in. together with a system where you have to earn voting rights
(to become a 'citizen' of a site), for instance: you have to be a member for
at least a year and you start with a veined number of up/down votes.

~~~
parfe
Slashdot solved that problem over a decade ago. Semi-Random users receive
moderation powers and can assign a post to several categories (Flame-bait,
Troll, Funny, Redundant, Insightful, Informative). Any other user has the
power to review moderator choices. Get enough negative feedback and you lose
the chance to be chosen as moderator in the future.

You also cannot moderate and participate in the same conversation. Posting a
comment revokes any moderation in the thread.

You can change your user preferences to augment the moderation choices, such
as -5 to funny and +3 to insightful.

CmdrTaco provides history on how the system evolved:
<http://slashdot.org/moderation.shtml> (1999)

~~~
snogglethorpe
The can't-mod/post-simultaneously rule is super annoying though, and I wish it
were at least a bit more subtle. For instance, maybe something like "can't mod
the same sub-thread you've posted in" or whatever (to stop people from modding
down their opponent in an argument).

------
drblast
I'd posit that threaded discussions are a problem.

Particularly with the nerd community, the ability to reply and pick apart what
the parent post said almost ensures that every discussion devolves into
inanity.

I try to make it a point, if I'm going to comment, to just leave a top-level
comment and not respond to replies. It works well enough, and it might be
worth considering as a rule.

------
sesqu
Beyond the problem of not counting votes as contributions (the knights of /new
should be rewarded), I feel this scheme has a major downside in using the
Wilson lower bound: namely, it rewards quick comments.

It's well known that the number of votes a submission or comment gets is very
time-sensitive, and the Wilson bounds are substantially affected by that
number. This leads to little-seen content gaining their author very little
influence, which may be purposeful but probably isn't.

A further little-mentioned complication is the existence of thresholded
voting: some people distribute votes freely, while others manage their vote
histories by limiting themselves to voting on exceptional content, and yet
others use contextual voting, touching only misrated content. Given the
dependence on voting, I feel this aspect has not been sufficiently
investigated.

------
shell0x
I'm not sure if it would be a good idea to switch the algo behind HN.It worked
well for several years,but the spectrum of users changed. I'm not against a
new solution, but you will run into this issues again and again, because it
are social problems, not technical ones. Just my 2 cents.

------
Tichy
Just had an idea: fixing parts of HN by allowing ads. But not for products,
the ads would be for discussions in other communities. The idea being to lure
trolls and other unwanted members away into other fishing grounds.

------
efsavage
I'd like to see a system where your vote is simply part of an estimate as the
ultimate quality of a comment. If things Bob votes on tend to also gather
other votes, Bob's vote should be seen as an indicator that other people will
be voting for that item soon. So his vote basically counts for more early on,
but as others join his vote effectively counts less as the actually tally
reaches the estimated target.

You could take this to the level where if Bob and Mary both vote for
something, it's a sure hit, but either one on their own means little. Some
kind of Bayes implementation.

------
russtrpkovski
We need to access to user activity data in order to determine root cause(s)and
come up with potential recommendations. All proposed solutions without any
empirical evidence is based on speculation.

------
PaulHoule
This isn't too different from an algorithm I used to clean up mech. turk.

------
jedharris
Lots of interesting and potentially useful points. But everything depends on
knowing or controlling the quality of HN: Is it deteriorating? How do we
increase it? Etc.

So: We need to be able to measure the quality of HN. This doesn't sound any
easier -- maybe it isn't. But the process of figuring out how to measure the
quality will take us a long way toward telling us how to maintain and increase
it.

Without measurement, we don't really have a basis for agreement on whether a
change to the mechanism is good or bad.

------
Kilimanjaro
Here is a simpler formula:

    
    
        vote = max(10,floor(karma/1000))
    

If I get an upvote from tptacek, I'd get 10 pts but a down vote will send me
to oblivion at once.

At the same time noobs don't have voting power until they get enough karma,
increasing comment quality.

Trolls who get below -100 will be automatically hellbanned.

~~~
zerostar07
Via that strategy old users become tyrants; even if we assume they are wise,
there is the danger that karma whores will become the next generation of
tyrants.

------
kzahel
I think it makes more sense to just update the posting and commenting
guidelines to more accurately reflect what kind of posts and comments are
"desired" by the old guard. Not all of us have been around silicon valley for
10 years and understand the unwritten etiquette of HN!

------
tokenadult
The Hacker News guidelines

<http://ycombinator.com/newsguidelines.html>

have been updated pretty recently, as I see one significant change in the
guideline text that responds to controversies that came up over the last two
months. The Hacker News welcome message

<http://ycombinator.com/newswelcome.html>

gives an overview of the community experiment here, summarizing the site
guidelines.

<http://ycombinator.com/newsguidelines.html>

The Hacker News FAQ

<http://ycombinator.com/newsfaq.html>

gives some additional details about how Hacker News is administered. The
welcome message distills the basic rules into a simple statement: "Essentially
there are two rules here: don't post or upvote crap links, and don't be rude
or dumb in comment threads."

In recent discussion,

<http://news.ycombinator.com/item?id=4397542>

several HN participants proposed further revision of the guidelines, and also
proposed making the guidelines more visible during the article submission and
comment submission process. For example, have the "add comment" form field
prominently display a link to the guidelines, perhaps with a snippet of text
referring to the guidelines most related to comments, and similarly have the
submit form

<http://news.ycombinator.com/submit>

prominently display a link to the site guidelines and a brief description of
what kind of submissions are most desired.

Weighted voting fixes based on user behavior signals do seem like a good idea
(that is what pg is thinking about), and we might as well discuss how to make
those fixes mathematically correct. The technical tweaks can best be
reinforced by ongoing efforts at user education, including possible revision
of the guidelines, and, in the opinion of several HN users, simply making the
guidelines more visible to everyone who submits an article or posts a comment.

Edit to reply to first comment kindly made to my comment:

What kind of comment is most often upvoted, and by whom, is an empirical
question. I don't have access to the data to resolve the question of what kind
of comment is most readily upvoted. The bestcomments page here on HN

<http://news.ycombinator.com/bestcomments>

shows a current snapshot of what the current community, based on current
communication of the guidelines and current technical features of the HN
software, has upvoted the most in the most recent several days.

~~~
transpostmeta
The problem with votes being weighted upon previous upvotes of content
submitted by that user is as follows: The chance of upvoting a post is related
to its length and complexity. One-off puns or silly remarks are much more
readily upvoted than long, insightful but demanding comments on complex
matters.

~~~
ljf
Well the difficulty is really that we don't know that any more (since we can't
see comment vote counts). I'd say it's not inverse to your suggestions - some
'higher' HN posters do seem to get away with 'comedy' comments, but many newer
members who don't get the 'HN tone' get downvoted into oblivion.

I think since we can't see the vote numbers people are less likely to upvote
longer comments now - but I don't imagine them losing out to jokes.

I'd still think that over all, good comments are going to attract more votes
that joke comments, and with less risk of being downvoted for 'making this
like reddit'.

------
robertskmiles
I really think that this could work. It reminds me of a "Web of Trust" system,
where you have a few known trustworthy individuals, and the reputation
propagates out from them.

------
jhuckestein
I think posting this on YC's Demo Day is unfortunate timing. I'm pretty sure
this won't be noticed by anyone at YC.

------
6ren
Similar to PageRank.

------
tkahn6
There is no way for the community to effectively exert influence on the
behavior of new members. That is the problem.

A complicated algorithm or formula will not fix this.

New members have the ability to upvote. If there are enough of them with
interests and standards contrary to the current culture of HN, they will
upvote each other and bootstrap their own culture. Eventually they and the
people they subsequently attract to HN will overwhelm the original culture.

This problem will not be fixed unless there is a threshold system for upvoting
or PG and the Y Combinator mods take a more aggressive approach in removing
articles.

~~~
vannevar
_There is no way for the community to effectively exert influence on the
behavior of new members. That is the problem. A complicated algorithm or
formula will not fix this._

Sure it can. If you hide low-scoring comments from low-ranking users, but show
them to high-ranking users, the high-ranking users can exert influence on the
low raking-users by upvoting quality comments and downvoting poor ones. It
becomes a self-organizing system with a bias towards quality, rather than the
open loop 'majority rules' system currently in place.

------
peterwwillis
Moderators.

Moderators, moderators, moderators, moderators, moderators, moderators,
moderators, moderators. Moderators, moderators, moderators, moderators,
moderators!

