
How Hacker News ranking really works: scoring, controversy, and penalties - jseip
http://www.righto.com/2013/11/how-hacker-news-ranking-really-works.html
======
swombat
Hilarious that the original article was flagged off the front page, but this
one isn't...

I find it very disheartening that the negative voices are being given so much
weight. Everything that's worth doing will have detractors, and when it's
something _really_ worth doing it will have vocal detractors. Back when I had
comments on my blog, every article I wrote that was any good had at least one
person commenting that I was a moron or some equivalent statement.

Great things arouse passion - on both sides.

Giving 10x the power to the people on the negative side just creates an
environment where new ideas are discouraged, where important but difficult
discourse is pushed aside, where things of true import are penalised out of
the group's attention by a few detractors.

There does need to be a system for flagging and removing spam articles, but if
this system can (as it plainly regularly is) be co-opted to remove articles
from sight just based on not liking them much, then it is broken. The people
who have flagging powers are not responsible enough to use them wisely,
perhaps.

I see at least one simple solution: lift the flagging privileges so it only
becomes available to a much smaller segment of the population. Perhaps making
the limit 10'000 instead of 500 would do that. That would still include
hundreds of people, based on a quick extrapolation from
[https://news.ycombinator.com/leaders](https://news.ycombinator.com/leaders)
). An even better model would be to make it dynamic - perhaps the top 200
commenters...

~~~
pg
We don't let users abuse flagging. We have software that identifies users who
flag excessive numbers of stories, and we take away their ability to flag.

~~~
drinkzima
Just an FYI, but accidental flagging happens to me all the time on my iphone,
would be nice to have a flag-confirm on mobile (I'd bet mobile flagging
correlates much worse with controversy if others are having that problem too)

~~~
alxndr
Downvoting, too. Have to zoom in so the up triangle is unmissable.

~~~
lostlogin
What about instead having the ability to change your vote?

~~~
alxndr
Yeah, being able to change it within a minute of the initial vote would solve
my use case.

------
DanielBMarkham
At some point over the past 2 years HN has stopped being my friend. The folks
here? Great people. Very happy to have gotten to know many of them. But the
system itself? Not so much.

People expect machines they interact with to behave in some kind of logical
manner. After 2 or 3 times of submitting an article that HN has traditionally
liked -- and watching it tank -- just not that motivated to submit more. After
submitting my own articles, having people stop me in the hall and tell me they
liked it and voted up for it on HN, only to see it have no votes? Not so
motivated to submit more. After the tenth conversation about how people expect
HN to act one way and instead it acts another? Not so crazy about it.

I think the problem here is that PG wants folks to participate, but only to a
certain extent. People want to interact with the system, but on some kind of
mutually-fair terms. I'm not sure PG's goals line up with the average user any
more. There are good reasons for this, and I'm not trying to trash the entire
effort. It's just that this is a tough problem. I don't think you can code
your way out of dealing with messy human issues at scale. If you could, we'd
all be managed by computers in 50 years, and that's not a future I would wish
for my children.

~~~
larrys
"I think the problem here is that PG wants folks to participate, but only to a
certain extent."

Short answer is HN is basically deal flow for YC. To that goal it obviously
works very well.

PG and YC need "everybody" but yet they need "nobody".

As a business HN works to achieve the greater goals of YC. I don't think it is
about anything else and I haven't been around long enough to know if things
were ever different. They may have been back at the start because I guess if
you were to arbitrary you would never build up an audience or fan base.

On a personal level I've always thought that people who get up to the karma
level you are at actually deserve some special treatment because to me the top
commenters are some of the glue that makes a place like HN special. In any
business I have ever run you always give special treatment to the better
customers.

------
mgunes
How HN page rankings really work: you vote stuff up, and then the flag mob and
hidden moderators axe right off the front page whatever irritates the pro-
capitalist internet-libertarian techno-optimist idelogical sensibilities of
the white male Californian HN hive mind even in the subtlest of ways.

~~~
JonnieCache
Isn't that the point of this site? That's mainly why I come here, to peer into
the collective unconcious of the californian ivy-league portfolio-owning
hivemind. It's like a zoo but with tesla owners instead of chimps.

~~~
mgunes
I don't imply an ideal of neutrality; no online community is ideologically
neutral, and neutrality isn't a merit to strive for. It's just that most
participants seem to have internalized a supposedly meritocratic "whatever is
interesting to good hackers floats to the top" mental model regarding how HN
works, when that simply isn't the case. Every online community turns into a
self-censoring echo chamber with time, and given its origins and initial
purpose, the precise _kind_ of echo chamber that HN has been turned into
shouldn't be a surprise to anyone.

~~~
phillmv
The part that bothers me is that it's not _self_ -censoring. It's being
_actively_ censored, and sometimes the motives being the penalization are
obnoxious; everything that doesn't directly promote california imperialism
gets shunted to the side.

I believe it to be uncontroversial that all communities need some moderation,
but the lack of transparency or involved around these parts has always made me
feel uncomfortable.

I distinctly recall some articles on gender inequality in the industry…

~~~
davidgerard
Well, duh. The actual purpose of HN is as an advertising and recruitment tool
for YC. It's certainly not for the _benefit_ of casual commenters like
ourselves. I think of it as a more tech-oriented form of television. Possibly
one of those high-hundreds cable channels.

------
david927
The post this morning, "Ask HN: What kind of side projects are you working
on?"* got a lot of great responses and was killed because of it. There was
nothing "controversial" about the post; it was merely popular with comments of
what people are working on. This algorithm needs to be tweaked or HN risks
losing its base.

*[https://news.ycombinator.com/item?id=6799694](https://news.ycombinator.com/item?id=6799694)

~~~
brudgers
The number of links within it probably drags it down. Though I am ambivalent
whether it's good or bad, I sort of see the point...there's not much
discussion or analysis in the comments.

~~~
david927
It's bad because this is exactly the kind of thing that many on HN come here
to see. I hate sifting through the "YC startup raises $50" news, but I love to
see what people are working on, and I'm not alone.

~~~
brudgers
I think the issue is that it's mostly just links, and not much meaningful
feedback. People are mostly talking past one another and there was [when I
looked] very little dialog.

It was more like the monthly job thread and HN might have legitimate reasons
for discouraging analogous threads for everyone's side projects.

It is also possible that the thread's pattern of growth triggered some
heuristic.

------
sethbannon
I find it really disheartening to learn that any article with "NSA" in the
title is pretty severely penalized by HN's algos. This seems like one of the
seminal issues of the decade, for this community in particular.

~~~
ximeng
On the other hand there's an article with "N.S.A." in the title at the top of
the front page now.

~~~
joering2
This remind me initial war with Gmail on trying to change colors of links in
your HTML email body.

Google was replacing #000000 with blue links to make sure some won't deceive
users that underscored black text is or is not a link. So of course people
start using #000001 color, then this one got blocked they went into #000002.
Gmail quickly realized they wont win this one :) so they revert it back to
replacing only 000000.

------
flexie
The avoidance of controversial topics when talking together is one of those
things we Europeans are typically not so good at. I know from many Europeans
who like me lived in the US for a while that they had to learn the art of
talking without touching controversial subjects. At first it seemed
superficial but then I realised that it makes discussions that are not
controversial but nevertheless important possible and I came to appreciate it
every now and then.

Anyways, it would be nice if we in the settings could apply our own penalizing
to subjects that we don't care about or that we find controversial instead of
having others decide for us. But that would mean that submissions ranked
differently for different users, of couse...

~~~
clarkm
Do you have any examples of specific topics?

~~~
bnegreve
The fact that HN is trying to fight so hard against negative feedback is
actually a good illustration of that. For me, this has always been strange.

------
minimaxir
Not sure why the HN submission didn't link to the original post:
[http://www.righto.com/2013/11/how-hacker-news-ranking-
really...](http://www.righto.com/2013/11/how-hacker-news-ranking-really-
works.html)

Discussion here:
[https://news.ycombinator.com/item?id=6755071](https://news.ycombinator.com/item?id=6755071)

~~~
micampe
Because duplicates are _useful_. I hadn't seen the original post, I've seen
this one. Disallowing reposts is one of the HN policies I don't agree with.

~~~
lnanek2
Agreed. I'm glad I caught this on the repost.

------
tehwalrus
Interestingly, the best strategy for keeping an article you like up high is to
upvote it and not comment (or, if you must comment, do so only once.)

If you do comment, however, you can be as verbose as you like (as long as you
are bland enough not to provoke replies.)

I wonder if this will change the strategy some post authors have of "hosting
comments on HN" (and replying to every comment, even just to say "thanks".)

EDIT: and to edit your posts instead of replying.

I think this is penalisation of comments is a shame - I certainly come to HN
for the comments, not the articles (although they're interesting stimulus for
discussion).

~~~
Zaephyr
Agreed - often I read the comments and never get to the original linked
article.

------
lnanek2
Interesting. So if you want to get rid of the stories on the front page, and
see more stuff, you should comment a lot. Because on HN a lot of comments is a
death sentence for an article.

On the other hand, if you are an article writer and add a "discuss this on HN"
link in your articles, you should remove the link as soon as you get a good
ranking. Or actually don't ask people to discuss at all, because it is
harmful, just ask them to vote and have your own comment system for
discussion.

HN basically reinvented "sage", the concept from 4chan and its Japanese
origins where people sometimes comment on a thread just to get it closer to
the comment limit before it would no longer be bumped up to the front page
when replied to.

~~~
unreal37
Articles with a lot of comments without a lot of upvotes are probably rightly
labelled "controversial". I sometimes comment without upvoting the OP on
something I don't like. So this behavior by the algorithm matches my use of
the site.

------
grey-area
This article was more interesting than I anticipated. While I admire the
tinkering which goes on with moderation here in an attempt to keep discussion
civil and interesting, sometimes it has counter-productive effects. In
particular this rule doesn't seem to work very well:

 _In order to prevent flamewars on Hacker News, articles with too many
comments will get heavily penalized as controversial. In the published code,
the contro-factor function kicks in for any post with more than 20 comments
and more comments than upvotes._

Is a vigorous discussion bad? Should everyone commenting also upvote?

~~~
md224
I find this interesting as well. Flamewars are counterproductive, but if I had
to pick between flamewar and no discussion at all, I'd probably pick flamewar.
After all, it seems like flamewars sprout when ideologies clash, and shielding
ourselves from ideological clashes feels like it encourages insularity of
opinion. On the other hand, one could argue that heated arguments increase
polarization, so I'm not really sure what should be done. I just think there
needs to be more dialogue on topics that people disagree on, not less.

~~~
danielweber
I've seen communities that banned flame wars. They succeeded, in the same way
that ghost towns have no crime.

------
ChuckMcM
Its a wonderful analysis, kens if you would ever like to come work for me
identifying robot search clients just give a shout :-). I chafed a bit though
at calling it a 'penalty'. Isn't it really a 'moderation' ? The scoring is
adjusted by the moderators to be more the site they want to have and so they
moderate articles that they feel aren't appropriately more heavily than those
that are appropriate?

I understand that for some people the moderation choices offend them, I think
that is unavoidable, but the goal is, I believe, to make a 'better' collection
not to shoot down particular articles.

------
jader201
Assuming this article is correct regarding the penalization of comments, I'm a
bit surprised (maybe even disappointed) that it is assumed that discussion is
a sign of controversy. And maybe it is, historically?

It's a shame for those articles sparking insightful discussion though.

It seems like a weighted penalization could be implemented, potentially
looking for red-flag words like "pedantic", or "not to be *". Or maybe it
already is.

Hope I didn't just set it off. :)

------
hmsimha
According to this article, the link currently in #3 [Vote Now: Who Should Be
TIME’s Person of the Year? Edward
Snowden]([https://news.ycombinator.com/item?id=6800145](https://news.ycombinator.com/item?id=6800145))
must be the victim of a very harsh penalty. I'm guessing 'Snowden' is a
heavily penalized term as well, and having less than twice as many upvotes as
comments might not be helping either.

------
brador
Blog spam link. Real content from the article at
[http://www.righto.com/2013/11/how-hacker-news-ranking-
really...](http://www.righto.com/2013/11/how-hacker-news-ranking-really-
works.html)

------
cloudflare
It would be interesting to know if there are 'flagging rings' in the same way
there are 'voting rings' and whether HN actively detects the former as it does
the latter.

~~~
mcintyre1994
For what it's worth, there are a lot of stories around of people losing flag
rights after flagging too often.

------
Samuel_Michon
TL;DR: “If an article has more comments than votes, don’t add your comment to
it or you may kill it off entirely!”

Rings true to me and, if indeed accurate, it seems like a good practice for
HN.

~~~
krapp
Comments should matter more than votes. A thread with all votes but no
comments is a meaningless number, but a thread with all comments but no votes
is exactly the same as that thread _with_ votes.

That particular weighting doesn't make sense to me. Clearly some people vote
by commenting. Which is as it should be.

~~~
sejje
But many people are downvoting with their comments.

~~~
krapp
Unintentionally, yes, that's the problem.

~~~
sejje
No, I meant intentionally.

They disagree with the article or what have you and say so in the thread.

~~~
krapp
Oh. Well, good. I think civil, negative comments have more value than just
downvoting.

------
twotwotwo
Not everyone has the same criteria for what content they want to read. Nor
does it really help the world for all HN visitors to read the same content.

Would love to see ideas that broke from the model of a single ranked list: let
folks tune their personal penalty amounts and gravity; add random jitter to
rankings and throw a couple random new stories onto each list;
classify/cluster users by their votes, so people who vote for jokes or NSA
articles or their neighbors' articles (automatically) see more of those
things.

It's maybe a bit much to ask PG and co. to architect radical alternatives to
HN, because HN is a handful as it is and, besides, I hear they have day jobs.
It could be cool to let a thousand flowers bloom: publish most of the now-
hidden ranking data (maybe not all, because it can be useful to obscure how
anti-spam algorithms work); let users opt in to publishing anonymous
votestreams for clustering, etc.; then let other folks use all of this to make
their own homebrew HN frontends within certain limits.

I suppose that, too, is kind of a pipe dream, because opening HN up for people
to easily build their own frontpages is far-from-trivial for both tech and
policy reasons. But it's a nice pipe dream.

------
alexkus
If someone wants a 'weekend project': an interesting browser plugin would be
one that undoes the effect of the penalties and reorders the page accordingly,
it could even allow the user to choose the level of penalties enforced against
stories in their view in case they did want some to disappear off the front
page.

------
damon_c
The question is: Deep down, whether we realize it or not, are these
unspeakable manipulations the reason we come here?

------
sixtypoundhound
Using score adjustments as automatic community management makes a lot of
sense; there are certain topics which are more likely to upvote/rank than
others. Similarly, there's also a good bit of research that controversial
articles generally do better on Reddit and other social news sites.

Applying an automatic penalty to certain topics / tactics which are likely to
gather excessive upvotes, due to the nature of the content vs. it's quality,
helps ensure you've got a diverse mix of content occupying the front page.
Which is generally good for the overall user experience.

Otherwise, the front page will be a massive list of shock jock posts about the
NSA.... [since controversial posts about those subjects will get sympathy
votes, regardless of their actual contribution to the community...]

------
alok-g
Instead of using ad hoc scoring rules like that, HN could use a machine-
learning based system. This can also help solve another issue -- automatically
determining the initial score of the new stories.

------
atmosx
Hm, this post was kinda bad for me to read. I thought this was a completely
community driven board. But I'm a fairly new member and some things never
change, apparently.

------
vijayboyapati
What I find quite strange is that I have posted articles which show up in the
new section, then later when they make the front page, it shows them posted as
someone else. Either dupe detection isn't working (these are very simple URLs
typically, like from the New York Times) or HN is rewarding the post to
someone else after I've posted it. Weird.

------
franstereo
As other folks have mentioned there is a raw feed if you want to see a non-
penalized version.

It would be interesting if it somehow incorporated other elements to determine
article "value": \- Open rate \- Ratio of comments to opens \- Time spent on
article or comments \- Depth of comments

~~~
anaphor
Sentiment analysis is becoming pretty accurate so maybe you could take into
account the number of negativew/positive comments as well. Maybe that will
eventually replace voting systems, who knows?

------
pearjuice
I think what is more interesting, is what drives people to upvote certain
things.

------
mikeevans
Interesting that github.com links are automatically weighted down, especially
since almost every blog post they make hits the front page, and often projects
hosted there show up fairly regularly as well.

~~~
skeletonjelly
HN actually stands for Holman News

------
brudgers
This article is worse than the original, and the original article was crap
because it never addressed flagging. I have on occasion flagged articles on a
particular topic...sometimes I just get frustrated with recycled topics. It
also doesn't address the possibility that penalties are applied when HN's
heuristics suspect vote rings.

And given that the hackaday article is blossoms, don't be surprised to see it
fall.

------
Eye_of_Mordor
Why doesn't everyone vote for this article and leave a comment?

------
muyuu
I see this is being penalised already >:-D

------
analog31
Beautiful MatPlotLib work. ;-)

------
brosco45
And censorship

------
michaelochurch
I get a personal penalty, because someone with moderation powers (possibly PG,
but I'll give him the benefit of the doubt) is a wolfbagging pissant.

Apparently being brutally honest about VC means that _everything_ I say is of
low value.

For more, go here: [http://michaelochurch.wordpress.com/2013/11/03/heres-why-
pau...](http://michaelochurch.wordpress.com/2013/11/03/heres-why-paul-graham-
probably-owes-me-an-apology/)

------
001sky
_On average, about 20% of the articles on the front page have been penalized,
while 38% of the articles on the second page have been penalized. (The front
page rate is lower since penalized articles are less likely to be on the front
page, kind of by definition.) There is a lot more penalization going on than
you might expect._

==Why there will never be a Flat tax...

