
Design: Voting up or Down is Dead - DanielBMarkham
http://www.whattofix.com/blog/archives/2007/10/voting_up_or_do.php
======
waleedka
It's not a voting problem, it's a scaling problem; I believe. Simply put:
social news sites don't scale very well.

As communities grow:

1\. You and I can't get things on the home page any more. It becomes only for
the well connected.

2\. Subjects lose focus.

3\. It becomes a gold mine for spammers. They'd do anything to game the system
if they can get 234,000 visitors from one post [1].

The solution, I believe: Keep communities smaller and focused. If need be,
create many small communities.

[1] [http://blog.mindvalleylabs.com/marketing/how-to-get-
traffic-...](http://blog.mindvalleylabs.com/marketing/how-to-get-traffic-from-
web-20-like-digg-reddit-stumbleupon/)

~~~
aston
At the same time, monetizing eyeballs requires lots of numbers. I guess the
solution then is to make a site that feels like a ton of people are there, but
still operates quality-wise as if there's a small community actually
controlling the content.

~~~
waleedka
True; monetization requires big numbers. But why not split them into smaller
communities? Rather than one huge home page, have a lot of smaller ones. And,
then, allow the user to select the 5 or so that they like to participate in.

Restrictions are sometimes good. Compare Facebook to MySpace. With MySpace
it's one huge community; everyone can see everyone. With Facebook, you can
only join a small set of networks. You cannot even join San Francisco and
Silicon Valley at the same time. True, people hate it, but it keeps the system
manageable as it scales.

This is a subject I'm interested in and trying to build a startup to address
it.

~~~
DanielBMarkham
Do people really know which topics they are interested in? I mean, there's
stuff I like, such as astronomy, philosophy, or politics, but would I want to
choose a group to limit my discourse there? I like philosophy but I'd hate
reading a thesis in the subject. But a little light rationalist vs.
subjectivist or philosophy of science reading would be great. Except for those
times I'd just like to see pictures of naked famous people. How would these
sub-groups come into existence? I just can't see it being topic-based.

~~~
waleedka
Edited: A web site doesn't have to cover ALL types of stories. This one
doesn't cover politics or astronomy, for example. And, in fact, your point is
more of an argument for a multi-community approach. Don't you think? For
example, if you have multiple communities to choose from, you can read about
marketing in the morning and about programming in the evening. Whatever you
like.

Google News is organized this way, and it seems to work.

~~~
DanielBMarkham
Edited:

I think my point was more that people have "favorite" topics, but it's a loose
and free-flowing thing, not a rigid structure. Computers are really good at
rigid definitions, while people may be amenable to all kinds of fuzziness.
Perhaps my group is better defined as "people who like science, astronomy,
politics, and startups" -- but that wouldn't be the topics of the articles!
The topics of the articles I like could be anything. And over time, I might
add or subtract the stuff I like, even without being consciously aware of it.
Heck, I'm not even sure I'm able to give you a precise list of topics I'm
interested in, much less self-select against a preconceived ontology.

~~~
waleedka
I see your point. So how about this: a community for people who like science,
and another one for those who like politics, and so on. And, then, you choose
to be a member of the science, astronomy, and politics clubs, and I choose to
join the science and startups clubs. At least this is what I want to build.

------
alex_c
Seems to me like Slashdot's comment system does two things that might address
many of the problems you mention:

\- instead of "Up" or "Down", the voting is a bit more granular - "+1
Informative", "-1 Flamebait" etc. The UI isn't MUCH more complicated - instead
of a "vote up / vote down" link you have a dropdown.

\- Voting points are a limited resource - assigned to "reliable users"
randomly and expire after a few days. This makes it a bit harder to get a core
of power users who are practically guaranteed to be voted up. As a bonus, the
scarcity probably means that people are more likely to use their votes
"wisely".

I wonder if something like this has been tried for selecting the front-page
stories on a social news site, and how well it would work. As far as I know
everyone's following Digg's model.

~~~
whacked_new
Yay for mentioning Slashdot!

Slashdot has a good system, but for increased granuarity users actually need
to think more about what they're voting, instead of what is possibly a
subconscious, wholistic, instantaneous judgment, which seems pretty applicable
for Digg. So there's a tradeoff for a system to be "easy to use" rather than
"intelligent."

In this sense the UI becomes a bigger challenge than the data processing. See
<http://x-o-o.com/news.php?sort=new> It's a dictionary-supported rating system
with granular voting, in the sense that you can vote 10 or 0.1 if you choose
to do so; it also supports weighted voting.

It is much harder to use than digg though so such a system probably will not
reach popularity. But I think theory wise it is the best implementation.

~~~
toisanji
Slashdot's commenting system is one of the best I've seen. One thing I wish
they had though is the ability to filter out types of comments. You can
already label comment as "funny","interesting", or whatever, but you can't
filter by those labels. I want to be able to read at a threshold of 3+ and not
funny.

~~~
testapplication
Are you crazy? You can just set a -1 funny modifier (I set -2, Slashdot's meme
humor rampant and unbearable). If you do that, and browse sorted by highest
score, Slashdot has by far the best comments anywhere on the net, aside from
certain niche newsgroups.

------
pg
I think you must have arrived since the explanation of how we're going to
avoid drift.

<http://ycombinator.com/hackernews.html>

Vote weighting isn't turned on yet, but for the last couple weeks I've been
thinking it was time to.

Incidentally, the poll problem can easily be fixed by changing the frontpage
ranking algorithm to treat poll votes as worth e.g. a third as much.

~~~
earthboundkid
I think one thing that might help is if the choice was a little less binary. A
one to five star rating system might help sort out the difference between "a
decent way to kill some time at work, so up vote" (3 stars) and "something
that will actually change the way that I look at the world, so up vote" (5
stars). Something to think about.

~~~
ijoshua
"Maybe" is one option too many

[http://www.zeldman.com/2007/06/20/remove-maybe-from-
invitati...](http://www.zeldman.com/2007/06/20/remove-maybe-from-invitation-
systems/)

------
reidman
Since my YC app deals with this directly, I've been thinking about it a lot.
My idea has to do with voting quotes up/down (kinda like Jyte,
<http://jyte.com> ).

In allowing people to vote up/down, I realized that there needed to be a
separation between 'Yes this quote is accurate' and 'Yes I think this quote is
important', since the two can be exclusive.

In the end I decided to implement a two-step process. The first step is to
Agree/Disagree with the quote at hand. Once a user has voted in this way, they
have an optional second step to vote for how 'Important' the quote is
(important or unimportant).

I have yet to see how it will work out, but I'd love any comments you guys
could provide.

~~~
waleedka
By doing so, you make the UI complex. Only a small percentage of people vote
in social news sites as it is, even with the simple one-click up/down.

~~~
reidman
I've designed it so that the user doesn't see the voting option for importance
until after they've voted for the accuracy of the quote.

~~~
waleedka
I'm not sure this fixes it. Try to factor the human emotional side into the
equation. For example, for an article that says "RoR sucks", what do you think
Ruby enthusiasts will vote? Even if it's accurate, they'll still mark it as
inaccurate, because they'd be hating it and wouldn't want it to go up on the
home page. It's human nature.

------
kn0thing
To the argument about reddit suffering from "Vote up if..." syndrome, we've
thought about building a "poll" submission type (like a self-post) that would
create an actual poll on reddit as the submission. We'd like to see fewer of
these polls on the front page, too, but still give worthwhile poll a chance --
they're just in the minority.

We figured this would make the poll have to earn the "up vote" instead of just
sucking people into clicking up because they "want Bush in prison." It could
even graph the results! Histograms are sexy.

Incidentally, newsvine has something like this. We're in the middle of getting
new reddit online, so who knows when/if this would actually happen. Perhaps
voting up/down will already be dead by then ;)

------
udfalkso
Explicit rating & ranking will inevitably crumble under its own weight. You
have to use implicit data that isn't easily subject to collusion. Look for
trends among disjointed groups that are acting in their own self interest.

I wrote about this a while back: [http://breasy.com/blog/2007/07/01/implicit-
kicks-explicits-a...](http://breasy.com/blog/2007/07/01/implicit-kicks-
explicits-ass/)

------
yters
In my opinion, this is as strong AI problem, and only people have strong AI.
Therefore, large social sites are always going to need human moderators to
have a good signal to noise ratio.

~~~
aston
That's a bit generous, I think. Most people aren't smart at all. And even the
ones that are don't act objectively and without bias. Ever.

Although I will admit that moderated solutions almost always come out better.
See slashdot's complete lack of this problem.

~~~
Xichekolas
Slashdot may not have the problem on a story/post level, but have you ever
tried to read every comment on a story? A third are hate speech or spam,
another third are repetitious or illogical, a sixth are insightful/funny, and
the other sixth would be if they didn't happen to be against the wider
political views of the people that had mod points that day.

I think the only real solution is to have a small community whose members are
actually heavily invested in the topic... basically, experts.

If I try to go add to a discussion about some new Myspace feature, my thoughts
are likely going to be off topic, misguided, or an outright waste of space...
all because I don't have a Myspace account, and never have. For the same
reason, if a bunch of Joe Sixpacks show up on Hacker News to discuss startups,
they are probably going to bury any thoughts from actual startup founders or
people involved in the 'scene' (whatever the scene is).

I'm not sure how you maintain that, short of making a community invite only.
Even Hacker News is starting to show the pain of size. The front page often
has pretty off topic stuff on it, when I, for one, came here originally to
keep up on the startups funded by YC and the thoughts of the people involved
in them.

Sure, PG still comments here regularly, but how long until this place grows
and he becomes an absentee owner like CmdrTaco? He may not stop commenting,
but there will be so much crap in the comments that few may notice what he
says.

~~~
waleedka
> I think the only real solution is to have a small community

I share the same point of view, and that's what I'm building right now with my
new startup. There has to be a balance somewhere. Too small, and you get a
dead community that rarely updates. Too big and it loses focus. The sweet spot
is somewhere in between.

------
aston
I voted up because I agree. Would I have done so if I disagreed, despite it
being decently well-presented? Should I have?

So here's the question: How do you do better?

~~~
DanielBMarkham
I would form a startup that had a better idea and I would try to adapt it to
make more users happy!

Not trying to be a pain, just that it's a big question, and whoever makes
progress in this area is going to be accomplishing something big, in my
opinion. I have my ideas, which I'd like to try out in the market and not kick
around on a board. Hope that's an okay answer. I'm not a "cool idea makes the
business work" guy, but neither am I a "give away the store" guy either. I
think there's going to be some proprietary magic going on when this nut gets
cracked, much the same as when search was cracked there was a lot of behind-
the-scenes magic that Google did.

BTW -- thanks for the vote. Karma whores unite! :)

------
Readmore
I had up and down ratings on <http://www.klipboardz.com> and then decided to
switch them out for a system that just rates on number of times the story is
read and the number of comments. I think it's a better system than most social
sites. The only problem I expect is burying content, but that isn't a problem
since the site is so small.

------
sbraford
This is why for a new project I just launched I'm using pageviews as the
metric (it's a pic sharing site) instead of voting up/down.

Each IP address can only register one unique "vote" per 24 hour period. This
makes the system still vulnerable to proxy servers and botnets, but it's not
nearly to the point anyone would want to game it yet. (blacklists will go a
long way against proxy server attacks)

------
utnick
the problem is that even though WE don't like poll stories or random bush
bashing on the front page, a TON of people do like them obviously, and you
need to accomadate them too

anyone ever used pandora.com?

you give it an artist, it plays you a song by the artist or in the same genre,
you vote the song up or down, eventually pandora learns enough about your
tastes to only play songs that you enjoy

what if a social news site kept track of what kind of stories you voted up or
down, did some magic to draw connections between types of stories, so the
front page of the site would look different for each user... people that don't
like poll stories wouldn't have poll stories on the front page, etc, etc

~~~
pg
_even though WE don't like poll stories or random bush bashing on the front
page, a TON of people do like them_

Not quite, unfortunately. The problem with voting, as the reddits realized
from the very beginning, is that people will vote something up not just
because they liked it, but because they want other people to read it. And a
site full of what the majority want everyone else to read could be
significantly less interesting than one full of what people like themselves.

And with polls there's not even a pretense that they reflect anyone's
interest. The voting mechanism is being hacked for another use.

~~~
aston
People voting for whatever they want other people to read is basically the
same as voting for what they want themselves. I know surveyors occasionally
ask questions like "How do you think most people would answer the following
question: ___" in order to get an honest answer in cases where people might
not want to fess up to some position personally.

~~~
pg
_People voting for whatever they want other people to read is basically the
same as voting for what they want themselves._

Only if the Bible is basically the same as porn.

~~~
aston
Not the best comparison. I don't see why a person would vote something up that
they have no interest in personally. Altruism isn't all that common.

~~~
pg
There's a difference between what people want to read and what they want other
people to read. I know of no precise word for this. Altruism is too
charitable, and hypocrisy too uncharitable.

Imagine yourself inside the head of, say, an abortion rights advocate. A story
appears with a headline saying that abortion is just as common where it's
illegal. You don't read it; you already know that; but you upvote it as a way
of saying "hear hear." Now apply the same rule to articles about Bush, Ron
Paul, the RIAA, police abuses, etc, and you've filled up most of the frontpage
of a site.

~~~
DanielBMarkham
Seems to me like we're confusing the users and the system. The users want to
read the best material for them. The system only has two states: yes/no. If
people are voting _knowing that their vote will influence what others read_,
then they aren't working for the best interests of the users, only their own
best interests. As you indicate, I don't think there's a word for that, but if
there is a recommendation-consumption impedance mismatch (a gap between
recommender interests and reader interests) then the system will always swing
out of whack. That's true no matter how much it is weighted, because all
you're doing is switching the impedance mismatch from a user-to-user scenario
to a user-to-editor one.

------
dyu
I think up/downs are to judge the worthiness of the link, and agreements
should be expressed in the comments.

~~~
DanielBMarkham
Would you vote up for a link to a story that you violently disagreed with? Be
honest now. And if you would (I _think_ I would) do you really think that's
the way others are voting?

~~~
reidman
I just experienced this dilemma with the 'Facebook Sucks' post by Dave Winer
(heh) that went up on YC news earlier today or last night.

I thought the linked story was pretty lame, but at the same time, I thought it
was an interesting conversation. In the end, I didn't vote up the link, but I
did participate pretty heavily in the comments.

I would like a way to distinguish between liking the story and thinking that
other people should see it. But at the same time, I realize how complicated
that would make things.

This question/dilemma actually relates directly to my YC app/idea, so I'm
going to be following the comments in this thread pretty closely.

~~~
DanielBMarkham
I have a hard time voting anything up, mainly because I have no idea what I'm
signifying.

I literally have to remind myself go through the news list and comments and
vote up every so often -- after I've consumed the material. It's not part of
the consumption process for me at all.

There have been a lot of stories on YC that I agreed with, but I thought the
linked article was poor. And what about good AskYC questions that have no-so-
good conversations? Would you vote up the question because it's good, or
ignore it because there's nothing of value there for anybody else?

This is kind of one of those seemingly pointless philosophical conversations
that have a great deal of meaning once they turn into solid solutions.

