
Solving The Hacker News Problem - tianyicui
http://al3x.net/2011/02/22/solving-the-hacker-news-problem.html
======
edw519

                     Quality of HN Comments Over Time
       |                   . .
       |                  .   . 
      q| . .             .     .
      u|    .           .       .               . . .
      a|     .         .          .           .       .
      l|      .       .              .      .           .
      i|       .     .                  . .               .    
      t|        . . .                       you are here -->. .
      y|                                      (that's all)
       |________________________________________________________
        N D J F M A M J J A S O N D J F M A M J J A S O N D J F
           '09                     '10                     '11
    

(It must be that time of year again...)

<http://news.ycombinator.com/item?id=926604>

<http://news.ycombinator.com/item?id=1646871>

~~~
Alex3917
When HN first started it was kind of like r/truereddit but with an emphasis on
startups. HN today has almost nothing in common with this. That's not
necessarily a bad thing, but I don't think it's correct to say that the
quality level is just an oscillation.

~~~
_delirium
I guess I don't really agree; HN of several years ago typically had _much_
briefer and less "weighty" articles and discussion than r/truereddit, and a
lot more offhand stuff, like the typical techcrunch stories, and "here's how
you do a thing in vim" type quick-tip articles. Certainly _some_ good, in-
depth technical articles and discussions, but that's the case now too. I've
been digging through archives to try to remind myself of whether I had missed
some sort of golden age (I was a lurker then, and considered it an interesting
but not _amazing_ site), and I still don't really see the golden-ageness, even
in retrospect. Nothing like the Kuro5hin golden age, anyway. ;-)

~~~
redthrowaway
Having never heard of Kuro5hin, I decided to google it. If it ever had a
golden age, it must have fallen _far_. The current front page reads like a mix
of 4chan, freerepublic, and the occasional tidbit of interesting content.

Seriously, a selection of headlines:

"OFFICIAL NOTICE: SITE CLOSED DUE TO AIDS"

"Judiciary Abdicated. Republic Dead. All Hail Emperor Obama"

"The New York Times Editorial Board Should Be Executed For Treason"

"Our lunatic Military"

What kind of content did it have originally?

~~~
_delirium
It's sort of a dive bar these days; a surprising proportion of the good
posters are still there, but they just post different things, while they post
"serious" stuff on their blogs or somewhere else. Most discussion goes on in
the diaries (which are also dive-bar-ish, mixture of serious and unserious
chat), while the front-page is a bit of an active joke. Hasn't had much
serious content in probably 3-4 years, with the occasional exception.

(Actually that's a strangely common thing on the internet; the offensive-joker
and troll types sometimes are the same people as the good-technical-content
types. For example, the Rands of <http://www.randsinrepose.com> is _also_ the
Rands of <http://www.jerkcity.com.>)

As a half joke, I collected a bibliography of k5 articles that have been cited
in academic papers, which includes some of the better ones:
<http://www.kuro5hin.org/story/2007/12/8/23637/9295> ,
<http://www.kuro5hin.org/story/2010/10/17/5512/5360>

------
pg
I think if we could see random frontpages from days a few years ago, we'd find
that the top stories weren't that different, and that there was the same "jack
of all trades, master of none" aspect to the site that Alex complains about.
It may be that a site whose design spec is to satisfy hackers' intellectual
curiosity would necessarily feel that way.

Maybe I'll write something to regenerate past front pages, so we can check if
things are different now. That should be possible, because news.arc has always
logged vote times.

~~~
bootload
A diff of past front pages would be interesting. Some historical front page
entries:

2007AUG15 ~ <http://www.flickr.com/photos/bootload/1119991071/>

2008FEB08 ~ <http://www.flickr.com/photos/bootload/2250089864/>

2009JAN18 ~ <http://www.flickr.com/photos/bootload/3205568708/>

2009MAR12 ~ <http://www.flickr.com/photos/bootload/3347109733/>

2010JUN24 ~ <http://www.flickr.com/photos/bootload/4728476663>

~~~
barrkel
Of that sampling, it hit the apex for me with Jan 2009 page. The Mar 2009 page
is when there was an influx, and we started posting Erlang stories as a joke
to stem the flow.

~~~
KirinDave
They were a joke? I thought finally people were interested in something cool.
:(

~~~
mechanical_fish
What brings a tear to my eye is that, back in those days, it was hard for me
to tell at first that it was a deliberate plot. It felt quite natural to have
multiple discussions of Erlang on the front page.

------
jacques_chester
Ah yes, the cycle of website life.

* Hot new community forms at Site X.

* Site X residents refer to themselves as the New Wave of whatever. Much better than older Site W because of features/members/dynamic/demographics 1, 2 and 3!

* Site X's reputation spreads to former hot new sites T, U, V and W. Site X begins to attract more and more new users.

* Site X denizens begin linking articles at T, U, V, W and vice versa.

* Site X begins to exhaust natural topics of conversation. Denizens of more than 3 months standing become sick of 100th "What does Site X think about AlphaGamma?" post and begin to slap down newbies.

* Someone reminisces out loud about the Golden Days of Site X.

* Discussions on Site X become more and more about Site X. Extremely intelligent individuals begin to earnestly argue that their proposed feature will save Site X from itself.

* Someone proposes or launches Site Y. A how new community begins to form there ...

I've been watching this same story play itself out since Slashdot circa 1998.

~~~
JonnieCache
_> I've been watching this same story play itself out since Slashdot circa
1998._

And every social subculture in history. See: the constant evolution of the
social scenes around music genres. It's just a fact of social psychology we
have to live with.

 _"Dubstep was sooo good back in 02 before the students found out about it! I
mainly go to future-garage raves now man..."_

~~~
jacques_chester
Good point.

On reflection it resembles flocking behaviour. I imagine that website members
can be modelled as simple agents with two rules:

1\. A cool site has few, impressive members.

2\. Move towards the cool sites.

Given an arbitrary distribution of "impressive" members, this should cause a
constantly moving flock of members to move from site to site _regardless of
its purpose_.

~~~
groby_b
It strikes me that a logical step to prevent #2 is "allow only new
contributing members that have been vouched for by n old members".

Yes, I'm that elitist. And yes, I'm well aware I probably wouldn't get voted
in :)

But here's the other part - keep the site open for _reading_. That means it
(hopefully) self-selects for quality, and other people can _observe_ a high-
quality discussion.

~~~
mkr-hn
Do you have any examples of a site where treehousing has improved things?

~~~
steveklabnik
Metafilter is often cited. They have a $5 fee for accounts.

Then again, Something Awful is $10...

~~~
mkr-hn
That's clubhousing, not treehousing. Treehousing would be putting social
barriers on entry. Clubhousing would be financial.

These terms are 100% arbitrary and invented an hour ago, but these seem like
good definitions.

~~~
steveklabnik
Dribbble, then.

~~~
mkr-hn
It'll be interesting to see how that model holds up to another 2-3 years. The
few invite-based communities I've been a part of started to degrade as the
social graph grew out and started to pick up the occasional undesirable.

It was harder to get rid of them because they knew someone who knew someone,
and trying to push them out had a network effect. It might be obvious to
someone removed by 3+ levels that a bad element is bad, but the person who
invited them might not see it.

~~~
groby_b
I am seriously considering pruning branches, not just leaves of the social
graph to keep communities healthy.

I.e. if you invited a bunch of bad apples, you _will_ be pruned too. I have no
idea if it would work, but it's pretty obvious that open social graphs beyond
a certain threshold are not conducive to high-level discussions.

~~~
die_sekte
Zed Shaw's Utu had that idea. Doesn't seem like he's still working on it.

~~~
groby_b
I'm not surprised that I like how Zed Shaw thinks :)

------
tptacek
For what it's worth: I feel safe saying that _most_ high-karma users of HN
have a variety of severe concerns with it. My experience asking this question
over email has generally been one of getting gigantic essay-length responses.

In my official capacity as "representative of people dorky enough to have
karma this high", we do officially declare: stuff's broken. Needs unbreaking.

~~~
DanielBMarkham
In my official capacity as commenting wind-bag, I'll second that.

HN started as a way for YC guys to communicate. It was about _startups_ , the
tips and techniques to make one, to grow one. Once PG opened it up, it became
"stuff hackers like", which meant cool deep tech stuff. But the tech stuff was
there because after a while, all the startup stuff just ran together.

Then "stuff hackers like" kept drifting wider and wider. I was happy with
that, but now that the traffic numbers are through the roof, what I'm seeing
is the 1% snark factor is out of control. If each member of HN said something
truly snarky only once a month, there would still be thousands of snarky
comments a day on here. It's just too much noise.

HN has always been driven by emotion: the only reason to click that button is
to express your emotional response, good or bad. Sure you can rationalize that
in various ways, but in principle it's about how the thing makes you feel.

Now we have the love-fest stories, the hate-fest stories, the snide jokes that
get upvoted, the trashing of "rate my startup" posts (I still can't believe
how some of the startups are treated). It's becoming less of a community and
more of a mob.

I think maybe the story _titles_ might look mostly the same, perhaps, but the
quality of the stories and the quality of the conversations have changed quite
a bit.

~~~
NickPollard
Maybe upvotes/downvotes could be weighted by the karma of the person giving
them, thus putting more emphasis on those with high karma (which might sound
elitist, but the only reason those people have high karma is because people
think they talk sense).

This would concentrate power slightly in the hands of those long time
dedicated commentators ('the community'), and slightly lower the strength of
newcomers who are still getting used to the ettiquette ('the mob').

Hopefully this would go someway to lowering the effect of newcomers, until
they rack up enough karma themselves to 'graduate' into the higher karma
ratings.

~~~
gbrindisi
Wasn't this the same mechanic behind Digg?

------
redthrowaway
All of this has happened before, and all of this will happen again. HN is
succumbing to the problem pg tried to address: dilution. Thing is, reddit
already came out with an excellent solution with their subreddit system. This
simply wouldn't be an issue if there were different sections for tech,
hacking, programming, startups, science, finance, and general interest. Keep
all of the deeply technical stuff in one place, the cruft in another. Let's
face it, the people who are complaining about lack fo deep tech are also
likely to read and enjoy one of Spolsky's blog posts. There's no need to ban
the latter to protect the former, just keep them in separate sections.

Now, HN isn't trying to grow, so there's no need to have user-created
subreddits (sections, I suppose). Just make 8 or so that people care about,
and add another if there's sufficient demand.

I really shouldn't be crediting reddit with this, as the solution existed long
before them. All HN needs to do is follow the forum model and have different
sections. It's too big to only have the front page.

~~~
lwat
We have become exceedingly efficient at ruining online communities.

------
sachinag
As a longtime MeFite and a longtime member of this community, I believe that
most of the issues could be dealt with by having obvious and active
moderation.

As MetaFilter, not only do we know who the mods are, we know which mods are on
call at what times. (And there's 24/7 coverage.) HN relies very heavily on a
flagging system, but it's just not as responsive to stuff that is broken as is
a human who's responsible for what's on the front page and what's in the
comments. Having a handful of humans who are responsible for curating the
front page (and possibly also pinning really good stories from new onto the
front page) would solve most of these problems. Is this less democratic? Sure
it is. Would the unfairness be worth it? In my opinion, yes.

This problem just isn't solvable with code; it takes benevolent dictators.

~~~
tptacek
What's the threshold HN crossed where it started needing active moderation?
Because '08 HN threads were just better than '11 threads. Is it just a number
of users, past which no set of guidelines restrains pathological
conversations? Why? If you can trace it back to what thing happens when you
get to your 50,000th user, maybe there's some passive moderation mechanism you
can identify to keep it in check.

~~~
sachinag
For me, I noticed it when people started posting things from investor blogs
and the VCs and angels themselves started commenting here. I think that
spurred some small handful of users to start showing off how smart they were
in hopes that they'd get a response from fredwilson or whoever, and that just
naturally spread the way it did. (There's a certain high when you get
fredwilson or msuster to respond to your comment on their blogs; I think it's
the same dynamic here.) I'm not blaming the VCs at all here - but that's when
I started noticing it. (joshu and yegg are the exceptions that prove the rule
because they're hackers first.)

Also, I've been told the Something Awful community is surprisingly pretty
awesome, although I'm not a participant there. If there's some intersection of
structure/moderation between MeFi and SA that HN could implement, I think
that'd be a great place to start.

 _sighs_ We complain because we care. It doesn't mean that we have all the
answers or we're pointing fingers. It means we're so invested in finding a
solution to this problem that we won't just move on. Given just how good HN
was (and remains), minute changes are more noticeable. I don't think a
comparison of then-and-now really gives us useful data. There's just a
gestalt, a feeling, you get that something's not quite the same. The closest I
can think of is when you're out to dinner with someone you've fallen out of
love with; your routine isn't any different, but you just know that it's not
the same.

~~~
JonnieCache
SomethingAwful was (is?) good because of the membership fee, and because of
the _viciously strict moderation._

You could have your account deleted, with no warning, for:

    
    
        * Persistent lack of punctuation or grammar
        * Posting tired memes
        * Persistent inability to cite sources for claims (in some subfora)
        * Dozens of other things I can't remember
        * Basically anything the mods decided to ban your for
    

The reason this worked is because the mods were drawn from the community. The
admins would look at a subforum, see who was trusted on there, had good
judgement and was there a lot. They would then make that person a mod, without
asking them or _even telling them!_ That person would just log in next time to
be confronted with lots of extra widgets on the UI.

Because the community was so cohesive across the site, that person would then
know immediately it was their duty to post a thread saying "Hi, I'm XYZ, your
new Moderator" and christen their new powers by stickying the thread.

Again, they were not told to do this, but they would immediately and without
fail, because the criteria for being chosen as a mod means that anyone who is
chosen would be aware of this tradition, and in fact every other aspect of
their role, having learned by example.

The SA forums were operating lots of quite deep game mechanics stuff nigh on
ten years ago. It's very interesting. Most of the measures wouldn't be
appropriate for here though, a lot of them involved strategically fostering
antagonism for the benefit of wider community cohesion. It really worked over
there for a long time though.

EDIT: I seem to remember that for a long time they only took payment by actual
credit cards, visa and so on, and not debit cards, paypal etc. This was
deliberate to prevent kids from signing up.

------
Mz
I'm probably one of those darned newbies who isn't a real hacker and is
screwing the place up. (Sorry.) So I wasn't around in 2008 (or whenever the
Glory Days were). I don't feel like this article or other discussions about
the issue have really given me a good idea of what HN supposedly once was that
it isn't anymore. I wish I could get such info. I think that kind of
information would hold out some hope of figuring out a real solution -- a
means to raise the bar or deepen the discussion or whatever it is that people
are wanting.

I know there are other large forums on the internet but this is the largest
one I have personally participated in. I think such large forums are breaking
new ground, socially, in ways that do not compare to sites like Facebook.
Where else can I actually speak with my 80K closest friends? If I am in a room
of 500 at work (and not on the stage, because I am not one of the big wigs),
only a handful of people around me can hear anything I say. We all can listen
to the presentation, but we cannot converse. Here, any and all of us can
converse. It is unlike anything you can do "IRL". I suspect that is part of
the issue: No one really has a model for how you manage that kind of social
interaction. And the models we do have break in that setting.

Just thinking out loud.

~~~
groby_b
Part of "the glory days" was a much more interesting debate. Currently, lots
of HN comments are either going for the cute/snide/sarcastic joke, or argue
dogmatic points.

Worse - the snide remarks get modded up. Being guilty of them myself, let me
post an example: <http://news.ycombinator.com/item?id=2202958>

The discussion was about Google's two factor auth. Somebody asks for a good
API, poster #2 suggests OpenID. I joke "you read the part about 'good' API".
It's by _far_ the highest-modded comment I have. (Which makes me feel guilty
every time I see it). Sure, I don't have a lot of comments or karma, so it
might be a fluke, but it exemplifies what's wrong with the discussion.

HN'08 would've modded me to 0 for that and moved on. HN'11 rewards playing to
the audience.

~~~
Mz
A couple of real quick and dirty thoughts:

A) Meatier comments/debates require a certain level of trust that the debate
is about ideas and not about pecking order. That is enormously hard to achieve
in most settings. It gets harder when the community grows. A smaller group is
much more able to know each other and all that.

B) My best understanding is that formality is what older cultures with larger
populations moved towards as a solution for such problems. I know I find some
stuff at work endlessly annoying because of the disconnect between the setting
and the social behaviors/assumptions. My mom is European and my dad was career
military and my ex husband was career military. Europe and the American
military are both more formal cultures than the American Deep South where I
grew up (and live currently). I find some of the assumptions of more 'casual'
cultures to be very uncouth.

I don't know how to raise the bar on formality for a place like HN but my best
guess is that would help with this particular issue. I like a good, healthy
debate. I rarely engage in it because most "debates" are really fights (ie
they are "arguments" in the other sense of the word -- emotional, social,
ugly), not intellectual discussions.

Anyway thanks for tossing me a clue. That does help me think about the issue,
unlike most of what I read in these discussions.

~~~
groby_b
That is an interesting thought.. I've never looked at it from that angle, but
formality is certainly a buffer against "pecking order" fights. Then again, as
much as I'm happy in a less casual setting, I wouldn't go so far as addressing
fellow HN posters as "Sir" or "Ma'am" :) (But maybe I should...)

It's also interesting that in the last 6 months or so I ran across a couple of
sites that take your HN karma as a predictor of your geek-worthiness. Well-
intentioned, I'm sure, but it encourages gaming the system. HN karma now is
not only something valuable to the HN community, but outside.

It would be very interesting to see if kuro5hin/slashdot/reddit "decline"
periods correlate with wider acceptance of their karma metric.

~~~
Mz
I don't know that formal culture necessarily has that much to do with using
titles (though that can be a part of it). I think it just doesn't assume
familiarity. Informal cultures assume a degree of familiarity which you
logically can't achieve with your 80K closest friends and I think that is one
of the roots of evil here, so to speak. I read a story once about a young
American man who had an affair with a British woman and then ran into trouble
because when he met her in public, she was offended that he behaved in a
familiar fashion merely because they had slept together. Her view: "We've
never been formally introduced."

That may make no sense to many people here. I can't think of a better example
though right now. Formality gives people a certain degree of social space that
a "small town" mentality (for lack of a better phrase) fails to provide. That
space reduces friction enormously.

------
malandrew
Forking of sites has the same problems as forking open-source projects. It
exacerbates conflict, forces people to choose sides, and ultimately both forks
typically end up poorer because of members lost.

Instead, the best solution is to evolve Hacker News as a product.

My personal opinion is that we should put Hacker news in the hands of the
YCombinator alumni. Founders and first employees (CEO, CTO, lead designer,
first engineer hire and first design hire) of YC startups would probably make
the best moderators and admins.

In fact, I would say that it's probably time that PG spin off YC as a full-
time startup, assigning control of the design and codebase to one talented UI
designer, one talented developer and one talented product manager.

For the site to keep growing in a way that maintains quality, it needs more
functionality that it has. The two features that lack the most are filtering
and combinatorial game mechanics.

Filtering is necessary so it is easy for the the hardcore tech articles to be
easily found by high-karma members, so they can vote those articles up. If
it's not findable, it's not voteable. Filtering is also necessary for people
to extract the most value out of hacker news. Most users don't want 100 front-
page articles everyday. They probably want 10-20 of the highest value
articles. Less is more.

Combinatorial game mechanics like those on StackOverflow would help as well.
Upvoting/downvoting is limited in that it will always fall victim to the
masses. Giving special voting/tagging/burying rights to distinguished members
(very high-karma users and YC founders and employees) would go a long way to
helping eliminate the crap.

I think I speak for most members here, when I say that I don't want Hacker
News to be a democracy. I want it to be a technocracy. I want the smart and
accomplished people to control what is good and should be visible to all. I've
got only 260 karma points, and personally I don't think that should be enough
karma points to allow me to upvote a submission. 500+ karma points should be
the threshold to be able to vote an article to the frontpage.

~~~
erikpukinskis
_both forks typically end up poorer because of members lost._

I strongly disagree. Off the top of my head: XFree86/Xorg, Mozilla/Firefox,
Debian/Ubuntu, Rails/Merb... these were all cases where the existing community
had calcified in some key area and a fork was necessary to create a space
where important work could get done.

And in most of those cases, not only was the fork stronger than the trunk in
(some) specific areas, the fork actually led to the trunk getting stronger.
Not only were patches merged, but entire projects, community values,
development practices, etc were.

XFree86 is an exception, but organizational paralysis was what necessitated
the fork, so it's not surprising they were also unable to respond to the fork.

But I just don't buy your notion that forking is somehow unhealthy and
destructive. It's one of the most valuable ways we have to keep open source
code, ideas, organizations and practices fresh, healthy, and vibrant.

~~~
malandrew
I agree that there are exceptions. In the case of HN, I don't see how forking
would be of benefit, especially since HN is so strongly linked to YCombinator,
the startup incubator/school that tends to attract the most talented up-and-
coming startups.

------
johnrob
Another reason HN may be boring: we've beaten a lot of the common topics to
death. It only takes a handful of articles about "how to pitch a VC" to soak
up most of the relevant advice on the subject. While posts often present a
unique combination of previously mentioned ideas, it's becoming increasingly
rare to actually find something new if you are a regular here.

~~~
erikpukinskis
It's worth noting that this is also true in Magazines, where they have strong
editorial review, and an actual staff of people creating content.

Once you've read 12 months of Runners World, you've basically read every
Runners World that has been, or ever will be, written.

------
jmm57
As a low-karma, long-time lurker, I'm not sure I've ever really seen the kind
of submissions he is looking for. Can someone provide examples of submitted
content that would meet his criteria of deeply technical discussion worthy
news?

~~~
paulbaumgart
<http://news.ycombinator.com/item?id=1304354>

<http://news.ycombinator.com/item?id=1321443>

------
dschobel
How to solve the signal/noise problem? Amplify the signal.

Call it undemocratic, but insight and perspicacity is not uniformly
distributed so it's absurd that pg/$whoever_you_respect's upvote on an article
counts as much as anyone else.

As a simple experiment, it would be interesting to see a view of the frontpage
based only on the upvotes of people who are above a certain avg-comment karma
threshold (since the site is predicated on karma as a quality indicator) and
the idea that people who write insightful comments won't upvote crap stories.

~~~
SoftwareMaven
The problem with using karma is that it is affected by the regression to the
mean. As more "average" people come in, more "average" people will give them
karma. Solve that problem, and you might have a solution to the whole problem
(I, for one, am not smart enough for that ;).

~~~
richchan
Alternatively, perhaps we can let users select (follow) a set of like-minded
people and make the up-votes of the people they pick weigh more on a
"customized" front page.

But then people may have not like having their up-votes be so transparent..

------
doron
Gated communities are effective means of preserving the identity of
communities, they all employ some bar of entry whether racial, religious,
ageist, or economical.There are many social maladies that are also unique to
the gated community, the insularity often breeds all sorts of creepiness.
Preservation all to often morphs to Stagnation.

Artists are often the shock troops of a neighborhood gentrification, after the
studio loft, comes the artisan coffee, some renegade youths, a young lawyer or
two, and before you know it, the neighborhood just ain't what it used to be.

I would Posit that a website calling itself "Hacker News" immediately opened
itself to all kinds of interpretations. The term "Hacker" seems to be as hotly
debated as "Artist" and justifiably so.

The Hackers, introduced others who identify with the Label, and still others
who probably do not, but nevertheless find it of value to their venture.

When the neighborhood changes, you are free, within your means, to move to
another place. Sometimes you yourself change and require a change of scenery.

When a startup grows to a full company, many times you lose something while
gaining another, and vice versa. Many in this forum have made those choices on
their own, so it should be familiar ground.

It is almost heretical to mention it here, but perhaps there is no algorithmic
solution (if there is a problem) to the complexity of human relation,
expression, and motivation.

More people, more heat, Entropy.

------
pclark
Come now, I can't be the only one that finds the Hacker News quality "good to
great"?

If Hacker News is about hackers in a startup sense, it's _good_ that the front
page has everything from: Movies being in decline - Ruby concurrency explained
- A torrent meta search engine - Windows 7 SP1 launch - iPad2 being unveiled.

There are far more elements to hacking than programming, just as there are far
more elements to startups than programming. And I dig that Hacker News is so
varied.

I think there is a _vocal minority_ of people that get irritated by bicycle
shed debates (+1 from me to allow collapsing comment threads on my machine) or
people wanting to only read about programming or hacking - the latter of which
is laughable because I am pretty sure you'd be sick of Hacker News if it was
100% a specific topic (I have some scars in the field of sorting content users
will enjoy...)

Guess what: there are millions of non technical silent people on the internet,
and a _huge_ amount of those people visit Hacker News _every day_ \- and love
this destination. The amount of random non computer scientists I meet in
Cambridge that love Hacker News is staggering.

------
petercooper
It's definitely not what Alex is semi-proposing but I've been running RubyFlow
- <http://rubyflow.com/> \- for a few years now and it totally stole the
MetaFilter model, just for Ruby-only stuff. No "votes" and points scoring -
just interesting posts from people in the Ruby community coupled with me
editing posts for format and deleting anything that's blatantly spam or
offtopic. Seems to work though I have been _tempted_ to go in the
voting/Reddit/HN direction with it.. maybe I shouldn't!

~~~
mechanical_fish
So long as you have the personal bandwidth, you will almost certainly be a
better editor than any algorithm you can come up with.

Thanks for the link, by the way.

------
davidhollander
_Simplest solution_

Limit the number of links submitted per account per day to 1.

 _Why_

Prevents spammers and karmafarmers from submitting the entire TechCrunch\Wired
back-catalog at a rate of 25+ a day.

 _Further Analysis_

Increasing the scarcity of a resource (link submission ability) will increase
the value of items it is traded for (links).

HN already gets the independent code submissions people want. They just die an
early death on the new page due to overcrowding by webzines\newspapers with
builtin linkbait titles. This reduces the rate of dropoff for independent
news.

------
doorhammer
If the crowd has cycled so much, I wonder if maybe this isn't the best
solution for the desired outcome.

Granted, I haven't been visiting tech-specific boards for more than a few
years, but I'd generally agree that the more technical articles are what I'm
interested in.

I think I'd be interested in a board that was geared toward
programmers/hackers, but didn't use a typical karma/point system. I'd like to
see one that perhaps utilized karma, but under a collaborative filtering
system. So, in a simple for-instance, if a small subgroup of people tend to
upvote articles that I do, those articles would be given more weight, and
similarly those who downvote articles I upvote would be, from my perspective,
given less downvote weight, while at the same time there might be a different
subgroup that was weighted to value their downvote more. Perhaps give people
the ability to tweak the tolerances of their collaboration. Give them the
ability to say "if this guy has X karma and ignores someone's articles and
votes, then I want to ignore them too"

Of course, this might be 1\. a completely naive idea, 2\. an idea that's
already been tried and failed 3\. an idea that's already being used 4\.
something to time-consuming for people with real work to do or 5\. an idea
that's unworkable and that I'm only having because I just started reading
books on, and experimenting with, machine-learning ;)

Though even if it existed, I probably wouldn't use it. I already waste half my
day reading the few articles that interest me on hacker-news, heh

it sucks that when you design any system or any set of rules, and humans are
going to interact with it, you have to think "how are these shady bastards
going to subvert my beautiful creation?"

------
peterbraden
I think that if you stay at any online community long enough, you begin to
perceive a drop in quality - even if that drop does not exist.

IMHO opinion, there is plenty of signal in the stream. What has happened is
that the interests of the community have diverged. I'd be far more interested
in ways to focus on things that I was interested in, within the stream, than
narrowing the flow of information.

On my wishlist is a way to pipe the HN stream through a Bayesian filter based
on articles I've enjoyed, and make an RSS feed of articles I'd be interested
in.

------
teyc
Where is the data that shows HN has degraded? We aren't seeing kitten
pictures. A scan of the front page shows the mix of articles being
programming, startups, tech.

I'm not sure what Alex wants? More discussion around PG's hackers and
painters?

------
jefe78
I've come to realize in my short time here, that dissenting opinions are
dangerous. I've learned to respect the karma gods and pander or, post my
opinion and delete it before taking too hard a karma hit.

Its sad to see that an informed, but non-conforming opinion is taken as fact
and karma-nuked.

~~~
astrange
Please cite examples. Your post is practically an example of karma-whoring
without them - anyone can make this claim.

------
kedi_xed
It's simple really. Digg was good, then it got popular, Reddit was good, then
it got popular. I've increasingly visited HN more than Reddit to get my old
Reddit fix, as I assume others have, and so popularity has increased and now
the quality is degrading as people want their karma fix or 2 cents.

There should be a brainstorm on this. I'm starting to realise I want comment
submissions from well known or quality submitters. Not just your average kid
or someone who is trying to troll.

The other issue is one-off opinion pieces on some guys blog. HN feels like
every programmers chance at 15 mins of fame. Why Ruby On Rails is X times
better than this (adudecodingblog.com), My way of speeding up Python
(pythonlover.com), etc. having someone like pg, of Joel, or big wigs viewing
items or articles like these, offering actual real world advice, and providing
comments.

Maybe a subscription based hackernews, where the kudos goes to the legends of
the industry, interns are made, and I get my intelli-fix and boredom disguiser
because I'm stuck in a cube-farm polishing PL/SQL wondering how the hell I got
here and when can I play that stupid COD:Black Ops with its really crappy hit
detection. Why do I keep playing it?! Why haven't I asked for a bigger
paycheck? Why am I not contracting? How is it that the kid I use to teach
programmer is now earning more than me? Oh well, keep surfing...

------
doron
The Illustrated Guide to Flame Warriors is a handy reference:
<http://redwing.hutman.net/~mreed/index.htm>

------
krschultz
In theory, centrally planned things make a lot of sense.

In practice, democracy usually comes to a better solution, even if it is not
perfect.

HN is driven by votes, the community is getting what the majority wants right
now. The only way to really improve HN is to change or limit the community.
You can tweak the rules only to limit certain actions to high-kharma users,
but if there is pent up demand for some kind of story it will make its way to
the front page.

------
tianyicui
IMO, a tag system like StackOverflow or Quora seems a good way to go.

~~~
tptacek
Tags concede defeat, fragmenting one dysfunctional community into a whole
spectrum of subcommunities. How will we handle the tag for "good enough for
'08 Hacker News"?

~~~
dschobel
If you can sideline all the people who want to bullshit about how the ipad2 is
the future of personal computing or the quality of the writing at TC, you've
already won.

------
danenania
I'm fairly new, but I think HN is great. The level of discussion is way, way
higher than other similar communities I've seen, and I think the general focus
on tech issues with splashes of other topics is perfect. Seems like a
successful self-regulating community if I've ever seen one. People just like
to complain.

------
jcsalterego
It seems there's always been reluctance to add features, for the sake of
simplicity. Personal messages, for example, would have been useful in many,
many instances, but instead we find ourselves checking out the plain-text
profile and finding alternate methods of communication.

This limitation has also sprouted ancillary sites attached to the HN Tree of
Life, such as searchyc.com, hackermonthly.com, and hnrecap.com as mentioned in
the post.

In a similar vein, carving out a sub-HN seems to be: a) downloading the source
code, b) bringing it online at another domain and c) announcing via "Tell HN".

All in all, unless someone with >10^5 karma decides to take the time and add
some community features to HN (for various values of "community" and
"features"), we're all going to continue and see more noise and many different
signals.

As an aside, I wholeheartedly appreciate the name, "Bloomfilter."

------
hammock
Everyone here has seen this same lifecycle play out at just about every online
community there ever was. Doesn't matter whether it was open or closed. It's a
fact of life.

The solutions offered are top-down culture modification and just plain don't
work. Adapt, and wait for the next HN to come along. You can't stop the train.

------
protomyth
Almost all sites that have comments and user moderation concentrate on the
comments and up voting / down voting them. Normally, when dealing with people,
I don't remember the individual quote that made me think they were brilliant /
a troll, I wrap that up into my sense of them. If I'm flipping channels and
see someone who has struck me as a brilliant commentator, I stop based on the
visual cue of their face. Names are kind of hard (is this the guy who called
me a $%$% or was he the one who really knows python?).

I guess I wonder if the same thing keeps happening on "comment moderation"
sites, isn't it time to look at the ways your view could be based on your (not
the group's) opinion of your fellow commentors? I don't have a technical
suggestion, but I will probably think a lot on it.

------
vidar
Perhaps pg is too busy these days to really tend to HN? God knows I would be
if I were carrying his load.

~~~
dools
Nope, I got an email from him the other day telling me not to editorialise in
submission titles after I posted that Nissan Leaf HTML5 extravaganza with the
heading "Behold: We don't need flash for anything anymore. Ever".

I'm sure that post itself rankled our OP.

~~~
rudiger
Question: Why'd you give it such a sensational title?

~~~
dools
Well, I guess because I was editorialising!! I found it sensational at the
time. That was the first time I had looked at a site and said to myself "well
that must be Flash(TM)", and then a second later in my head thought "well fair
enough, you'd HAVE to use Flash(TM) to get that kind of experience for a major
brand", then right-clicked on the page only to see the standard context
menu... no "Zoom in"!!

I was really quite blown away. I could have just put it on my blog then posted
_that_ to HN (which is actually what pg said I _should_ have done when he
emailed me), but my initial thoughts, were that I really just wanted to share
this with my pals on HN (similar to the way someone else had posted Ben the
Bodyguard as a "Gorgeous HTML5 implementation" or something) and that putting
it on my blog then linking to my blog from HN would just be viewed by the
community as me jockeying for traffic and being self-promotional!

------
RiderOfGiraffes
In response to some of the points being raised in the thread, and to provide
some sorely needed data, here are some snapshots of the HN "newest" page taken
since Feb 2009:

<http://news.ycombinator.com/item?id=2254397>

------
bootload
_"... I think HN does a crappy job with general tech news and a so-so job with
content that’s specifically relevant to startup founders and employees. These
days, HN does a downright terrible job with deeply technical topics; that’s
the area I hear the most complaining about on Twitter and in private. Since
deep tech is HN’s weakest point, let’s go after it. ..."_

The weakness of the argument is that the engineer/developer/programmer view is
a subset of the interests hackers, founders and entrepreneurs. I draw a clear
distinction between tech guns for hire who only want depth as opposed to those
who want to solve technical problems and maybe innovate which requires both
depth & breadth.

------
nbashaw
It's important to separate concerns, and I think there are three main things
people are complaining about:

1) quality of submissions 2) quality of comments 3) quality of community

It's the second and third that I think have declined. It's not because the
people are any less smart, it's that there are just too many of them. It
becomes difficult to keep a mental model of everyone in your head, so you
start seeing everything as disembodied text, rather than human beings speaking
to one another, with a history of shared experiences.

IMO, this is a solvable problem. You can use avatars and display our locations
next to our comments, or even just make our names a little bigger. Anything to
humanize the conversations.

------
cfontes
Every single community driven website on earth have being thru this kind of
cycle the only way to stop this is to start a new one that will end getting
into this cycle again. It's a matter of the amount of people using it, on the
beginning only very early adopters and people passionated about that specific
topic (here tech startups)are in, as years go by more people that have more
interests start to join and post things they think is good, and then the topic
changes to a more general subject.

I like this community and I think the quality will always fluctuate but the
most of it will always be very good content for people in a hurry.

Thanks for all of you who help this place being nice.

------
jpwagner
Well, obviously posts/discussions like this can actually be contributing to
the "problem" as some see it, but I'll make one point I don't see made here.

For me personally, I've learned a lot and grown a lot over the course of the 4
years I've been lurking and occasionally contributing here. So for me, a
smaller percentage of the stories/articles/posts/discussions appear as
insightful as they once did. I don't mean to knock HN in any way, in fact my
point is that that fact is not a "problem" to me. New users are joining
everyday and everybody who makes the effort to learn and contribute gets
something out of HN.

It's what brings me back 17 times a day.

------
mcav
HN would be better if it were invite-only like Dribbble. I'm not a member of
Dribbble, but it's a good example of why restricting community growth is
beneficial.

We're too late for that here. I don't think PG has enough bandwidth or
interest to truly solve the problem. New users will continue to join, adding
noise to the signal, unless HN changes course. It's going to become more
generic and more biased the longer the site stays open.

I hesitate to suggest more moderation as some posters suggest. I'm already
uncomfortable with the murmurs of unfair moderation in the system here.

~~~
pg
Actually HN is the reason I don't have enough bandwidth for other things, not
the other way around.

------
alienreborn
One solution can be to give read-only access to new users and charge a very
low one time fees for upvoting, submitting and commenting. Only people who are
serious about contributing will pay for it.

------
alexknight
Honestly I'd love to pay for a Hackernews account if it meant weeding out some
rather distasteful people. Not saying that is the be all/end all solution to
the problem though.

------
cmars232
There is no hacker news problem if you realize that all sites up to this point
in internet history are either heavily curated for quality and limited in
scope or self-moderated with a one-dimensional, imperfect karma game, and thus
chaotic and ephemeral.

Such is the nature of suchness until someone figures out a better game that
more properly engages human nature.

Crying over the demise of HN is like crying over a naive hill-climbing
algorithm when it gets stuck.

------
mickeyben
Very well written article. I think he has some good points. I'd really like to
see a 'deeply technical' alternative to HN and hope he'll find the good guys !

------
reedlaw
Could this phenomenon be in any way attributable to nostalgia? Personally, I
find any online community loses attractiveness after a certain period of time.

------
JoshCole
One thing worth noting is that reversion to the mean doesn't have to be a bad
thing. For example todays mean level of education compared to the mean level
of education a few hundred years ago is very different. A good question to ask
might be, would submitting this increase the mean level of discourse on Hacker
News? It is the same sort of thing as what is in the guidelines, but reworded
for greater relevance.

------
T_S_
Put some teeth into karma. Make more karma mean bigger upvotes and downvotes,
say 1 extra point for every 500 points of karma. It doesn't have to be linked
to when a person joined. It's elitist like a journal or university, but at
least anyone can read HN, and good posters will rise. Problem solved?

EDIT: Oh yeah. 1 week comment lockout for negative karma, with a grace period
for newbies to learn how to comment.

------
nhangen
I just want to speak up as a relative newcomer that feels I've learned enough
about this place to speak my mind without being afraid of retribution and can
do so with a basic understanding of what works/doesn't work here.

I really like it here, and it's my 1st stop after Gmail every day, and often
more than once per day. Nothing is perfect, but as far as I'm concerned, this
is as good as it gets.

------
howlingmime
This has probably been said before (and/or above), but perhaps each article
should be tagged and users should subscribe to only those tags of interest to
them. In addition, a social component to HN would be useful -- for instance,
allow me to recommend a story for someone or for stories of interest to my
friends to be ranked above the norm. Our friends make great filters!

------
adrianwaj
I agree with Alex, I think the success of HN has led to an overflow of people
dissatisfied with HN who have made HN a success in the first place (and some
with YC itself.) I posted two ideas that could be of interest to such people
here:

<http://news.ycombinator.com/item?id=2253752>

------
wyuenho
Why can't HN force categories on every posts? Have the community create and
curate the categories, and select their own categories on HN to read. Just a
blanket vote up/down button hardly measures how valuable anything is for any
particular group if that group is not constant.

------
SeanLuke
This was discussed before: <http://hackerne.ws/item?id=1934367>

I had a comment which I guess I should link to rather than repost:
<http://hackerne.ws/item?id=1934605>

------
icey
I would very much like to have <http://startups.ycombinator.com/startups/> and
have a more narrow focus on startup related news and entrepreneurship.

The constant stream of front-paged political arguments and noticeable increase
in mean-spirited commentary in the threads has caused me to spend most of my
time on HN logged out. It used to be that I'd read the comments before I'd
even read the story to see if the story was worth reading. I wonder if it's
possible that pg doesn't notice the degradation in comment quality as much
because the trolls have been here baiting him since the very early days.

I don't think HN is irrevocably broken; I'm glad that pg is helming the ship
and I think he's doing an admirable job of it so far (I think the ranking
algorithm in use for the front-page stories is one of the best anywhere). But
HN used to be great, and now it's merely good.

I think that a lot of people who have been here for a long time have thought
about what's changed here, and how it could be fixed. I know I've littered
more than a few mailboxes with lengthy emails about what I think is wrong, and
what I think the solution is. Reading this thread kind of tells the story - a
point has come where the community is large enough to have factions that value
different things. "Anything that good hackers might find interesting" works
when you have a small group of people engaged in conversation. It's less
useful when you have mobs of people who have come with different ideas of what
they want to get out of this site.

In the early days, HN felt like it was a problem solving tool; a way to find
out what cool things people were working on, and occasionally to ask for
advice. The community was humble, competent, and full of people who actually
made things. Those people are still here, but there's a self-aggrandizing
element here as well. The group of people who seem to think that someone
else's success somehow reflects poorly on themselves, the bloviators and
blowhards who believe that a volume of arguments somehow makes up for the
measurable factuality of arguments. I don't really know what the solution is
to this. I thought if there was a way to ignore people it might make a
difference, but after some experimentation I think that that's a dead end -
there is too much chance of missing something truly interesting from doing
that.

All this being said, HN has had an immeasurable positive impact on my life -
The people that I've met through HN (both in person and virtually) are some of
the smartest, most amazing people I've known. I'll get to use the things I've
learned from HN (and more importantly from the people in it) for the rest of
my life. I can't think of another site on the net that has come even close to
making such a huge impact.

I can't imagine missing out on all of this if HN had been invite-only when it
launched. I didn't know anyone when I first came here. I didn't even know who
Paul Graham was.

Instead of complaining about it, I think those of us that have been here for
awhile owe it to pg to actively try to improve the community. It's become too
large for him to handle on his own. Yes, there are moderators, but they're an
invisible hand that only act as a corrective force.

We're a creative lot. I'm sure we can figure out some way to improve this
community from the inside.

------
zaidf
Or may be after a while our _perception_ about something changes
disproportionate to the actual change?

That's basically boredom--and it can happen even if you consume something
_good_ for a long time. That "good thing" doesn't change so much as your
perception of it.

------
d0m
Maybe experienced HN (read as a mix of high karma + there since the
beginning), could have a bigger impact on which articles are chosen. I
honestly don't mind a "dictatorship" selection where chosen members could
remove useless post / select useful one.

------
jaekwon
Hi, I'm taking on this challenge. Al3x, can you send me an email at
jkwon.work@gmail.com?

Also, I'm taking suggestions for seed users. There will also be a HN Karma
cutoff where everyone above a threshold can join. You can nominate HN users or
yourself here.

------
tomrod
Wait... he complains on his post about not submitting his blog posts to the
community.. which then gets submitted to the community and upvoted (albeit
probably not by him). Does that sum it up?

------
newguy889
Political stories simply need to be killed with prejudice.

------
aDemoUzer
Part of the problem is the basic UI look, hence I am working on a new UI for
it: <http://peri.me/2B1A/>

------
gabaix
what about tagging? automatic or crowd-powered. Seems a great way to sort
through the noise. tags could be "technical", "startup", "YC" etc.

------
dave1619
Another HN challenge... as discussions get longer (like this page), it gets
more chaotic and more difficult to follow.

------
ck2
Remove points from users (keep on submissions/replies for positioning).

Problem solved by changing motivations/behaviors.

------
pdaviesa
Pretty soon you guys will be telling the kids to turn down that damn music and
stay off your lawn :)

------
jsmcgd
I now check 'new' articles more often. There are a lot of gems in there that
are not voted up.

------
adrianwaj
Does anyone here think crowd-sourcing due-dilligence on startups would be a
good idea?

------
mkr-hn
Sounds like (s)he wants a Less Wrong for startups.

------
mkramlich
My suggestions for tweaks to improve the site:

1\. hard ban on purely political news ("Egyptian leader stepped down! OMG!")

2\. hard ban on gender-specific things ("i'm female, went to bar during hacker
conference, got groped, OMG!" -- yes it was hacker conference, but gosh
subtract the 'during hacker conference' and you have real life, it's
independent of tech, not specific to it or due to it, just a life thing with
guys and gals)

3\. particularly if hard bans (enforced by a set of trusted admins) on the
above topics are not added, then allow submitters and admins to add/edit
content tags for each post; then allow logged-in users to submit content
filters so that when they see, eg., the front page, it can suppress all posts
with certain tags (eg., pure-politics, gender, sports, religion, etc.)

4\. optional for-small-periodic-fee premium accounts, which allow those users
to exercise extra features like smarter content tagging/filtering, sorting,
user following, user submission/comment filtering (so you can blacklist
blowhards and pedants from what you see, even if they are not banned from the
site overall)... I'd personally love to blacklist anybody that ever does a
comment reply to me that is (a) rude, or (b) idiotic, or (c) overly pedantic
(some is fine, we're nerds, goes with territory, and some precision is
valuable, sometimes). Blacklists could be flat files, one user per line. We
could share them among each other privately. I've bookmarked a few "a __hole-
or-idiot" users but I'd love it if I could have them automatically stripped
from anything I see on HN in the future. Actually, I'd love to have this
feature on all social/forum/news sites I visit.

5\. fix the "type comment, hit submit, get error page saying something doesn't
exist, so you have to go back, copy your text, hit Refresh, paste the text
back in, hit Submit again" bug/feature. that drives me nuts. feels like impl
side-effect rather than intentional UX

6\. don't have the up/down arrows so close together when viewed on iPhone

7\. don't allow just anyone to downvote any comment. or at least, they can't
downvote it beyond 1 point, below which is penalty land. right now, any
dumbass can downvote a comment of mine from 1 to 0, which then reduces my
overall lifelong site karma by 1. Just because they disagreed with me. Or
they're an asshole. Or they accidentally hit the downvote button (see 6).
Instead, have a minimum karma requirement to issue downvotes, and/or only
admins.

HN is great, despite it's imperfections. But I'd gladly pay up for premium
features. HN Gold? HNGold.com (YC-W11)?

EDIT: added a few items

~~~
maxklein
My suggestion would be simpler : have a "this is not hacker news" queue. It's
like the frontpage, but it's for the stuff that is "not hackernews". Whenever
something hits the frontpage that people feel does not belong, instead of
whining, they just hit the "this is not hackernews" vote button. With some
algorithm that can combine vote numbers, influencers or whatever, when it
crosses the threshhold, it gets moved over to the alternative page. So there
is the community moderated hacker news and there is the alternative everyone
hacker news.

~~~
mkramlich
good idea. i've been using 'flag' for that now, but your idea is better
because it more precisely signals why one is flagging something. makes
distinction from reasons like obscene, duplicate, etc.

------
rubashov
The basic complaint is that social sites grow into mobs. The solution is
rather obviously: halt growth. You have to limit the number of active users to
some vaguely Dunbar-ish number or you inevitably wind up with a lowest common
denominator mob.

Metafilter did this, right? For a couple years they said "No new accounts."

I think scaling a social site to a very large number of members without
deteriorating badly is impossible. It's a matter of human nature and mobs.

------
Pooter
The solution, ultimately, is for the site to wither and die, and be replaced
by something else that will have the same fate. This is what happens to all
things, and to all human social groupings in particular, from ancient Rome on
down to your nuclear family.

If you're tired of it, start something else. Or hang out and jump ship when
the next great thing comes along. Trying to preserve the golden age is
rearranging the deck chairs on a sinking ship.

------
mthreat
Case in point - this article.

------
georgieporgie
If sites want to claim some sort of community and continuity, they're going to
have to place newbies into virtual reeducation camps. Want to see the newest
links? You have to read through 10 comments from an '07 post first. Posted a
link to an internet meme? Back to the virtual reeducation camp with you.

------
ddkrone
Reeks of elitism.

------
p90x
This.

