This is something that I seem to disagree with a lot of people on. Dupe-police have existed on reddit, digg, slashdot, here, and just about every other social bookmarking website I've ever used (including my own).
I hate them. Why? Because the entire point of social bookmarking is to find things you find interesting, not things that you find unique. That little arrow to the left of the title means "I found this link interesting. I think other people will find it interesting as well."
It absolutely does not mean "This link is unique. Nobody has seen it before." If that were the point, we could just pipe RSS feeds into the URL submitter, couldn't we?
The very fact that links are appearing on the front page means that a lot of people haven't seen them yet. It means that they got some utility out of reading them, and it means that they thought others would too.
Sometimes, dupes are good. I don't remember who said it, but that Louis CK interview that gets posted every once in a while, where he is talking about how we're surrounded by wonderful technology and yet nobody cares, they said something to the effect of "I wouldn't care if this was stickied to the top of the page and everybody had to watch it every single day before they post. He is making an excellent point."
Now, I think this is a bit excessive, but the point stands. It isn't about being unique, it is about being good.
The difference between something like DupDetector and someone like MrOhHai on Reddit is that DupDetector was polite and was more like "here's the earlier discussion" as opposed to "you should not have posted this".
It’s still an understandably sensitive topic, hence my recommendation to make DupDetector much more polite and friendly so that there can be no misunderstanding about the intent of the bot.
When folksonmies were the current hotness Clay Shirkey made a great point about things that seem similar but aren't.
"Synonym control is not as wonderful as is often supposed, because synonyms often aren’t. Even closely related terms like movies, films, flicks, and cinema cannot be trivally collapsed into a single word without loss of meaning, and of social context. (You’d rather have a Drain-O® colonic than spend an evening with people who care about cinema.) So the question of controlled vocabularies has a lot to do with the value gained vs. lost in such a collapse. I am predicting that, as with the earlier arc of knowledge management, the question of meaningful markup is going to move away from canonical and a priori to contextual and a posteriori value."
This is a false analogy. We're talking about identical content (a particular website) as compared to two different linguistic terms. The point about folksonomies is interesting in valid in its own right, but should not be used as a justification for duplications on News.YC.
The very fact that links are appearing on the front page means that a lot of people haven't seen them yet. It means that they got some utility out of reading them, and it means that they thought others would too.
Assuming the theory works. People may upvote for a number of reasons, not necessarily because they received value from the link. There remains a lot of work to do in the unexplored space of social, user-powered spaces like Reddit and HN. In theory, the upvote/downvote method maximizes utility of the site users, because those who received value from the article upvoted. Unfortunately, what maximizes utility for one set of users isn't the same for others, which is why we can have such a diversity of sites like HN, Reddit, and Digg.
HN has established itself as a high quality, technically and entrepreneurially oriented site, which is why we don't see the same content as say, reddit.com/r/funny. Defending this position by submitting and upvoting high quality links is critical to maintaining the caliber of HN. This is part of why I don't like seeing resubmitted or duplicate content. Duplicate content (even if very high quality) still takes up a valuable front page location. If enough of the community received new value from the piece (ie. haven't seen it before), then they can upvote it and the article can stay. On the other hand, if you upvote something you've already seen for the benefit of others, you aren't necessarily helping them out. Instead, you are tampering with the ranking algorithm.
If enough new users see an old article and upvote it, then the utility for the aggregate user is maximized, and I am happy with duplicate content when this happens, even if I'm not receiving any value.
Still, there is the problem of users new and old not seeing or knowing about these great quality articles from years ago. Inevitably someone will see an old repost and say "I didn't see this the first time around. I'm glad it was reposted." However, I think using a site like HN to get these old articles new life is using the wrong tool to solve the problem.
I hate them. Why? Because the entire point of social bookmarking is to find things you find interesting, not things that you find unique. That little arrow to the left of the title means "I found this link interesting. I think other people will find it interesting as well."
It absolutely does not mean "This link is unique. Nobody has seen it before." If that were the point, we could just pipe RSS feeds into the URL submitter, couldn't we?
The very fact that links are appearing on the front page means that a lot of people haven't seen them yet. It means that they got some utility out of reading them, and it means that they thought others would too.
Sometimes, dupes are good. I don't remember who said it, but that Louis CK interview that gets posted every once in a while, where he is talking about how we're surrounded by wonderful technology and yet nobody cares, they said something to the effect of "I wouldn't care if this was stickied to the top of the page and everybody had to watch it every single day before they post. He is making an excellent point."
Now, I think this is a bit excessive, but the point stands. It isn't about being unique, it is about being good.