We started doing this in late 2014. There's an explanation at https://news.ycombinator.com/item?id=11662380, with links back to others. We've talked about it in comments and whatnot (https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que..., https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...), and have intended to publish the list, but only did so recently. We're slow.
If you see a submission that didn't get attention and which you think is particularly good for HN, please tell us at email@example.com! We love getting those requests and usually add them to the pool. It's fine if it's your own article, but we like it better when it's just something you ran across and recognized as good. That's more the kind of interest that HN is for.
A related list is https://news.ycombinator.com/invited. Those are old submissions that we ran across and thought deserved attention, so we emailed and asked the submitter to repost it. Yesterday's top story was one of these (https://news.ycombinator.com/item?id=26982286). They all go into the second-chance pool, but maybe it's interesting to see them broken out as a subset too. (If you don't have an email address in your profile, please put one in so we can send you repost invites!)
If you read the old explanations I linked to, you'll see that the original plan was to turn this system into software that anyone can participate in, likely as a new way to earn karma: users who discover second-chance links that hit the jackpot (that is, which interest the community) would get karma along with the original submitter. That is still the plan! We're just slow.
I think that's about everything there is to say about the second-chance pool. Questions, feedback, ideas, and views are welcome as ever. And please, everybody keep an eye out for obscure, out-of-the-way stories that got overlooked and let us know when you run across them. It's one of the best things you can do to help make this place more interesting. Best of all are the kind that can't be predicted from any existing sequence: https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor....
Shout-out to dang & Scott, who are doing fantastic work!
I'm biased on this though, as someone this might impact (https://news.ycombinator.com/from?site=thume.ca). From talking to other technical bloggers the consensus does seem to be that when we put a lot of effort into a technical article it nearly always makes it to the top of https://lobste.rs/ and /r/programming because it starts on the front page there but will sometimes flop off new on HN and maybe only make it months later if someone else resubmits the post.
I'd rather have eclectic ideas and projects from HN users not be overlooked (thus encouraging more of such content), and am less worried about GAFAM announcements, CNBC/Axios/BBC news, or things already popular on Twitter/Reddit.
Would this be a doable change to try?
Also, a lot of those media and BigCo stories really are of interest to the community. We try to dampen the stuff that's repetitive, and most of those sites are downweighted by default, but HN would not get better if they were excluded. It's all just more complicated than it seems like it might be.
What ultimately matters is how interesting a story is, not what site it comes from. I'm suspicious of encoding proxies for that, because it would be easy to end up optimizing for the wrong things. https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor...
Rather I think obscure sites should get more opportunities to be organically upvoted on (and if they don't get voted up, then fine) and not just fall off /new after a few hours only to be seen by a few people. The BigCo stuff naturally gets posted often several (different) links from different people, whereas obscure stuff is only posted by a single person once. So this is about evening the odds.
Would be great if non-commercial blog domains that have produced good discussions on HN in the past have that somehow reflected in future submissions. Sure, not every piece we write will be worth a front page discussion, and obviously we don't want to recreate digg where some people start getting disproportionate power. Writers can put hours and hours of thought into a piece, and we'd probably be fine if jo one thought it was interesting, but it's discouraging when it feels like a coin flip.
How much of "page one", ie. the top ranked stories, is made up of stories from those two pools?
I feel good when a re-upped post or an invited repost makes #1, like Yayagram did yesterday, and Internal Combustion Engine has now (https://news.ycombinator.com/item?id=26991300), because that's such a strong indicator of the community finding a story interesting.
Is it just that the ratio of submitted stories to people browsing new is too high, causing a low chance of any particular story to be looked at? If I had novel ideas on improving this I'd write it out, but I think others have already suggested things like randomly mixing new stories into the hot pages.
IIRC, the second-chance pool was our next experiment after that one. It has worked much better. The difference is human judgment or, if you will, taste.
It's not easy to come up with new mechanisms that might help with this problem. Every software mechanism we've tried allows many things through that don't pass muster. Community mechanisms, as soon as you open them up, get overwhelmed by people trying to game them to promote their own stuff. There's a feedback loop with that: the more interesting HN is, the more attractive it is to a high-quality audience, and therefore the more attractive a target it becomes for manipulation, which makes the site less interesting again. So there's a cap on how good it can ever get (https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...).
I know it took a long time for me to start looking at new, a little nudge would have made me start much earlier.
When you mixed new stories onto the front page, were they clearly marked as new or could they easily be mistaken for upvoted content?
Maybe this could be a karma gated opt-in feature that individual users can enable?
I'll try to add /pool to hnrss.org this weekend, too, if time allows.
Also, forgot to mention this before, make sure to subscribe to https://hnrss.github.io/updates.xml if you want to be notified of project updates!
I don't see how the voting system enables anyone to block stories hitting the front page. Mods can block things (presumably) but that could still happen with a more personalized feed.
One simple solution to the problem of articles you think should be on the front page not getting there is for you to look at /newest more and upvote the articles you like. Once an article gets a vote it seems to attract more votes even just within /newest. If you're not seeing the articles you like then maybe submit some?
Ultimately HN is about squeezing millions of readers in to the top 30 posts. That's probably why it works quite well. I have no doubt that a lot of people find their tastes aren't well catered for and leave for other sites, but that's OK. No one should stay on a website they don't like.
That's a vivid way of putting it (good writing on your part!) but - yes, that's pretty much how HN works. I usually describe it as "non-siloed": https://hn.algolia.com/?query=silo%20by%3Adang&dateRange=all.... The DNA of the site is that everyone sees the same things—it doesn't get sharded. Other sites have social graphs and follow lists and whatnot, but this one doesn't. More on that at https://news.ycombinator.com/item?id=23308098.
Maybe teaching millions of HN readers not to be too lazy to click past the first page?
That shouldn't be too much to ask of a community premised on intellectual curiosity.
My frustration with sites like HN and Reddit is that I may spend a few days crafting a question but once I post it, it usually gets one or two, often not very good, answers at best. It is hard to figure out if a question or a post is what the audience of HN or a particular subreddit would enjoy.
More specifically, the frustration comes from the fact that, e.g., HN, attracts tons of smart folks who have answers to my questions, but there is no reliable way to reach them. Unfortunately, I have no inkling of an idea of how to solve this issue and how widespread it is.
Having said that, big thanks to dang and the nameless mods for shepherding this great community!
I visited so many great blogs, that last year, I fell in love with blogging. I started on Twitter, then micro-blogging wasn't enough. So I wrote first article. To make it harder, I set the rule that I have to implement what I write about and make a video(s) about it. This way, I can avoid writing about things I don't understand and keep quality high enough. According to analytics, almost no one read it, but I love it anyway. I can't wait when 20, 30, 40 years from now I will re-read it.
I assume it's a manual decision, not strictly algorithmic. (Though it probably contains algorithmic elements, the final decision is done by a human.)
If true, then the inexorable outcome of this logic is that the front page isn't really controlled by the community, nor would we want it to be.
If not, then the details of the algorithm would be fascinating to learn about. It's not an easy problem to solve in general, and making a few dozen (hundred?) editorial decisions each day is probably more efficient, and more precise.
HN's system consists of community, software, and moderators, just as it always has. Moderation is a feedback module tacked on to try to prevent the system from landing in any of its default failure modes. That, alas, requires humans.
2 - ciechanow.ski's post on Internal Combustion Engine is currently #1 on HN, but also #8 on the second chance pool - whats the thinking behind offering duplicates? i can see an argument for it (makes /pool a mod-curated HN frontpage) and an argument against it (confusing as to whether or not this page is really for second chances)
I was always curious what makes the difference between what goes into the second chance pool directly vs what gets invited to the second chance pool if you can elaborate on that?
edit: thank you for making the lists public, I will browse them in the future!
edit: I also think this creates a charming distinction.
Have noticed a lot more repeat submissions that didn't get that much attention first time around getting a ton of more upvotes lately. A comment on the nature of the audience changing. Annoying somewhat.
But what's worse is stuff that already got submitted a year ago, two years ago, 5 years ago, 10 years ago, got a ton of discussion and upvotes, but now is much reduced as far as relevancy and newsworthiness and gets submitted again. Need to maintain balance of that oldie-but-goodie stuff with actual new topics.
Vested interest in this sort of because I often find myself digging up those old discussion threads to share like Dang does also, mainly because old is old, but also because we all end up having the same endless repeated conversation points and people can see all the discussion before posting the same stuff again
The point of combing through obscure submissions is to find things that haven't been repeated much, and to prevent HN's front page from collapsing into the same handful of hottest and/or most sensational topics.
Btw, to digress a bit: this is really a tragedy-of-the-commons problem. The few hottest topics get upvotes because individuals optimize for what momentarily attracts their attention. Left to its own devices, that mechanism reaches a state which is less interesting, and would probably burn out the system. Local optimization produces a global sub-optimum. https://hn.algolia.com/?dateRange=all&page=0&prefix=false&so...
If the same individuals were giving HN their full, sustained attention, thinking about the site as a whole, then a different result would be possible—but why should they? Users have better things to do, and it's HN's job to interest them. The solution is to have a small number of humans a.k.a. moderators whose job it is to give HN their full attention, thinking about the global optimum and trying to steer things in that direction, and jigging the machine when it gets stuck in one of its failure modes.
Maybe this is too shop-talk to be of interest to many readers, but I find it interesting because I think this business of full vs. partial attention is the solution to the puzzle of why upvotes alone, i.e. community plus software alone, doesn't produce the most interesting community. Previous thoughts about this, if anyone cares:
An example, to my first point, is like some old cyphernomicon txtfile from 1994 that's been submitted every few years for like 10 years and never really gotten many upvotes nor discussion, now gets submitted a day ago, I'm assuming not much traction again, and then submitted to Pool and now has 74 upvotes. It's weird. Never got much attn before, over and over and now all of a sudden all these things are getting large amount of upvotes. It shouldn't really get that much more attention and my gripe is that it shouldn't be on my radar as something new/interesting for the day because overall it really isn't. It's old and never warranted much interest repeatedly over time. Changing audience (a decline imo) or like something else driving more HN votes in recent year...hmm
I agree with you that it's weird for some old file from 1994 to suddenly get traction, but I think it's good for HN to be weird. I wish it were weirder.
Mods, or "community mechanics", have to be our better nature, because our natural instinct, the things we're wired to prioritize under partial attention, is for indignation and sensationalism. In other words information about risk or excitement. because for fast paced life in a group where partial attention is the only possibility, such information probably has the highest utility.
but our better nature relative to hn has to be something that gratifies intellectual curiosity, something genuinely interesting. Something that if you were to spend your full attention immersed in a comment thread or reading an article would have the highest utility for you.
It's not exactly trivial. I don't know how they achieve it at all.
But they do somehow. Because I keep coming back here to learn something. and I look forward to coming here more than I do to a news site because I think there might be something interesting I can learn here. but on a news site it's probably more of the same.
People might complain about the power imbalance that moderators have about how they're this "privileged few" poking the community this way and that from up on high. But this can't possibly be true. Who wants the "privilege" of having it to be their job to stand against face and invite the abuse of the resentful? How can one person with a stick stand against the madding crowd? They can't. Any respect the moderators have from the community is earned certainly not "given" as a "privilege". Do they have power? yes they have power but that power only comes from the community respecting their decisions. And they use software to give that stick a multi-local reach.
Such no-moderation advocates would prefer absolute unmoderated freedom: an idyllic community self-regulated and self-moderated, we're no individual has the power besides the social power they can accumulate through their interactions, every other form of power must be trashed and rejected. such criticisms of moderation are necessarily selfish because the people advocating them can only see their local corner of the community. And they don't concern themselves with the well-being of anything outside of that. If anything outside of that is a miss they simply shrug it off return to their safe corner. It's their privilege to not have the responsibility to have to moderate. a privilege given to them by the good work the moderators do.
But if they had their way, you end up with a partitioned world of all isolated tribes and no global optima . so the cost of this freedom (that no-moderation supporters want), is the freedom of other people, everyone else, to be genuinely interested and have their intellectual curiosity gratified when they visit the site.
By corraling the community to work as a whole you can approach the global optima of submission quality for the metric of genuinely interesting.
Do you still do that? I've always felt that the difference between "hey, this submission is cool, we auto-repost" and "hey, this submission is cool, here is a link so you can manually repost" is confusing and unnecessary.
Will the second-chance pool be added to the top bar?
Adding as a menu at the top will mostly ignored by all except by a select few. Having it pinned to the front page as the first post would be a good way.
Pinning things to the front page in some new way would be the sort of UI disruption that users hate, and we kind of agree with them.
Are you using the plural "moderators" just to be abstract in case this changes again or does HN currently have another mod?
Just curious, as for the most part I only read about you
Also reminds me of Last Chance Kitchen.
Great to see it evolve to a place on HN software!
When we're putting an article in the second-chance pool, we do prefer the original submitter where possible. Often, if there's time, I'll go through the submission histories of the other submitters too, to see if there's something else we could put in the pool for them.
In your case, though, you just run into the fact that /newest is a bit of a lottery and it's unpredictable which submission of a story will be the one that gets traction. We do have plans to do something about that someday: https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que.... In the meantime, the lottery does at least even out in the long run if you submit a lot of interesting stories.
When I say we usually add them to the pool, that's because the emails are usually from high-quality users with good taste. If that stops being the case usually, we'll stop adding them to the pool usually.
The purpose is to find more interesting things, where "interesting" means things that the community finds interesting. We use our own intellectual interest as a proxy for that, but only as fuel for the first stage.