That's why I don't think down votes should be used to express disagreement (although pg doesn't share this opinion). I personally frequently up vote comments I disagree with if I feel they are thoughtful and bring an interesting perspective. The variety of ideas is important and should be encouraged.
Up/down is the wrong direction for the arrows I think. Left <--> right, meaning on-topic/off-topic ? One of the problems I think is that there are so many dimensions represented by two simple little arrows that it is difficult for the system (the web software itself) to discern the intent, so it turns into a popularity (I agree,I disagree) button.
But if you had a button for each piece of information it would look like
[] well written
[] well thought out
[] agree
[] disagree
[] poorly written
[] funny
[] off topic
For bonus points they would be in a random order in the UI so people wouldn't develop muscle memory voting on comments.
I have thought about the On/Off-Topic button as well for some time now. I think it's a good concept to try out, mainly because it is not judgmental - Sometimes I like following an Off-Topic discussion and I've had a couple of good ones on HN. But most of the time, it can be very distracting - separating it from the main discussion is worth a shot.
As for more detailed buttons - I think that would make it too complex. Slashdot has established the same concept, probably based on the same reasoning (mostly to distinguish between "just" funny and actually insightful comments). Can't speak that much about how successful it is, maybe somebody else has more insight here.
Yeah, the buttons get out of hand, but the information they would represent is greatly needed. And how do you encourage people to rate what they read and interacted with? Based on the stats people post about the amount of traffic driven by an HN or Reddit submission ... the vast majority of users don't interact via any sort of voting mechanism at all.
I am working on a site now that is heavily based on passive activity voting, users have no idea how their actions are being used by the site to rank itself. Their inputs have much less weight than purposeful actions where both parties are aware of the event.
It is almost like you need an agent that votes in certain public ways (sentiment analysis,summarization,etc) to decide if comments snarky, funny, off topic, etc.
I guess that would fall down once you get to mixed comments - something can be snarky, funny, off- and on-topic at the same time. Human conversational interaction is just bloody hard to quantify. The first few steps are always simple and then it gets hard fast.
Passive voting sounds interesting, though - would that work via Javascript? How would you determine what the user is focused on?
Finally - I think it's important not to get ahead of ourselves - because I think that's what happened at Slashdot. Talking about new classifications is all nice and well, but it's useless when you don't have a clear plan what it should result in and how it would engage people. You need a clear use case.
For instance - my proposal about an off-topic discussion button is pretty straight forward. People would see the immediate benefit: Hey, this can help us make conversation less wasteful and save me scanning time as it separates the cream from the crop. I think it wouldn't have a problem enticing engagement, too - nothing drives engagement in nerds like being annoyed, so they'd click that button.
On Slashdot, I can theoretically reduce a conversation to exclude all the funny-only comments so I'm left with just the cream. But seriously - how often does anybody use that? Once you give people elaborate personal filters, you actually end up having them worry whether they're missing something. Which is why they can only be consistent and transparent. If I reduce a comment thread on Slashdot to exclude the funny business and see that one of the nodes has a HUGE conversation, I wonder whether I should check it out and maybe end up annoyed because it's just a chain of memes after all. If there was a way in HN to mark things as off-topic and have that a clear process that is always applied, I think I would be more confident in skipping. Mainly because if it was broken, the community would complain until it is fixed.
It's a social problem unlikely to be solved by a technical solution. When it's an open flat democracy and votes from hordes of new users have the same weight as elder ones it is pointless. They'd just start flagging for spam or offtopic to hide dissenting comments (like it's done in youtube).
On the converse, if the older people can amass power and karma new people can't move it.
I don't have a solution. I am not sure there is one or that it could be "fixed"
Maybe semi-intelligent Eliza like agents that constantly clean the place up.
Stackoverflow has done a pretty decent job I think. Wikipedia could use some work.
----
I think the base set of rules need to be codified by the system. The users can guide each other and the system, but the system itself should be stable.
