I like this solution, and would like to point out another way to look at it.
It can be viewed as simulated annealing. A brand new article starts at a "high temperature", it has a chance of showing up randomly for a selection of people. As time goes on the temperature lowers. And it either made it or didn't.
That is suppose we have a function called rand_hash that takes an article id and a user id and produces a random number between 0 and 1. Suppose that T is the time in minutes that a new article has been up. For position we treat a new article as having a vote of:
votes + C * rand_hash(article, user) * e^(-k * T)
Where C and k are factors that affect how much we want a random boost to be able to raise a new article, and for how long we want it boosted.
Then what happens is that every person will see, mixed with the popular articles, a random selection of new ones. The new ones that are particularly interesting will organically pick up a few votes and then snowball. The new ones that aren't will disappear from the random mix.
When we tested a variant of this idea, we found that it didn't work for what I guess you could call psychological reasons: readers reacted negatively, even violently, to seeing junk on HN's front page. By "junk" I mean stories that were placed there randomly and that weren't of high-enough quality to otherwise be credible candidates for the front page. Basically, it was an "ow my eyes", "get that shit away from me" effect. (I don't mean to demean the submissions people posted that got randomly placed in this way; it's just that words like junk and shit reflect how users reacted to the random placement thing.)
HN users have an intense emotional relationship with the front page. This shows up in the inevitable "How is this on HN?" comments when a story somehow violates the commenter's personal contract or mental model of HN. Randomly placing candidate stories on the front page created a version of this that was like 100x worse. It violated people's model of what HN is supposed to be, in ways that the randomness exacerbated. Two lessons I drew from that experience: (1) people's relationship with HN is surprisingly personal; and (2) never jeopardize that bond. Runner-up lesson: don't introduce noise into an emotional relationship.
If we had software that could measure how credible an article is as a candidate for HN's front page, I'd be willing to try this again. But I suspect that would amount to software that could measure how good an article is, and that would be the P=NP of internet news sites.
I'm still convinced that a lot of the best content—i.e. submissions that would gratify intellectual curiosity and that the community would have a great time reading and discussing—languishes in obscurity in the lower echelons of /newest, so HN's front page is still far from optimal. The second-chance process has helped (https://news.ycombinator.com/item?id=11662380) but I'm convinced that there are more things that we can do, mainly by building ways for the community to do them. It's important that we not build new mechanisms that just reproduce the upvote system, though.
It feels like using the homepage traffic to validate /new has a lot of promise, and maybe some tweaks could solve the problems you had. One idea I had from reading your comments and dan's article is to have a permanent slot on the homepage with something from /new in it.
I think you could address the issue that "readers reacted negatively, even violently, to seeing junk on HN's front page" by making it clear with a special visualization, that this slot is special. You could also add a "report low quality" button which would make that story less likely to appear in the random slot in the future.
You could even add a "show me another" which fetches another random story instead.
So roughly:
- somewhere near slot 5 we add a special "new" slot, visually identified as such
- on each page load, the slot is given to something randomly posted in the last 8 hours
- the slot is different for each request/user, so everything on new gets seen a little bit
I don't always like looking at "/newest" also for the reason it can take time to sift through to find the good ones.
What if there was a "/random" or "/custom". First would maybe weigh "newness" into the formula, but "/custom" could be based on user preferences. Might take a min to find right mix of "settings / allowable combinations" that would allow the site to remain performant. That way you all would be able to monitor trends in how users set up their preferences.
The response you saw probably wasn't very surprising to anyone who has spent time looking at the /newest page. It's fun to find the good submissions early, but there's a lot of low-quality submissions to sift through to find the good ones.
I would suggest as a followup that you allow people to opt in to getting a mix of new stories. My suggestion would be that "people are automatically opted in, and every new story comes with a link to opt back out". I do understand if that version scares you though.
I suspect that only a vocal minority of people will have a strong negative reaction. And if objectors have a painless way to solve the problem, you shouldn't get many complaints.
What about tacking a seriously "new boosted" and possibly randomized 10-30 submissions, clearly marked with different background to the end of frontpage, maybe after "next page" buttons? It would hardly make a dent to the load time, and people would in theory not feel as cheated since they know what to expect. It would still give "new" section a lot more eyeballs to find the golden nuggets.
I second this one. Having one-maybe-two new random links at the bottom, marked with a different color (maybe brown or golden, given the associations? ;)) would probably solve this.
One thing that might make /newest a little more user-friendly would be an option to have a showdead just for it. I have global showdead on and often it means staring at a /newest full of autokilled spam. showdead in /newest just keeps me out of /newest.
This is a good tweak. In fact: there's an argument that showdead doesn't fit /newest at all; aren't the people reading it deliberately not discriminating that way?
There are a few things that are flagged for no apparent reason and that need to be vouched for, and a lot that needs to be flagged manually. The [dead] flag is useful. Were vouching not a thing, you'd be right, though.
As always, thanks Dang for your presence. HN is the most civil place I’ve found on the internet in the better part of 30 years.
I’m wondering if the mistake was letting things make the front page, when maybe page 2 is acceptable even for people who feel vehemently about trash submissions. If we as a community can expect some pain in order to find better content it might be worth it (especially if we can train people not to abuse mods at the same time). I dunno. The idea of adding noise to smooth out results is a very common technique in signal processing, I’d like to think it’s useful in this context as well.
I will say, as an anecdote, I find the variety of interesting content isn’t as high as it was 6 years ago (let alone 10). And weekends tend to be better IMO. I assume the community has shifted, but even just being bigger is going to tend to lower the variance of topics. So I really appreciate any effort made towards helping quality interesting topics surface.
Can you try this again, but this time add around 4 of these types of stories the bottom of the home page in a separate section? (i.e. not mixed into the "good" stories)
I don't always remember to check /new and vote up good stories, but I feel if I saw random stories while anyway browsing the home page I might vote up any good ones.
Seconded. This would probably remind me to check /new more often, too. I also like the suggestion elsewhere in the thread of hiding the vote counts for this section.
I know HN sticks to a very simple UI, but I wonder if giving readers a way to control the mix for their personal front page would work. Maybe a little slider to adjust the threshold somewhere discreet (but on the front page, not in settings).
Would it be possible to mark the "candidate" stories in some way, to explain to the user why they were seeing this possibly lame post?
I used to be bothered by seeing off topic stuff and memes in a subreddit I visit but the sub started adding flare for different kinds of posts. I found this completely alleviated my psychological revulsion to the things I didn't like once they were categorized correctly. So, a meme on the front page would have annoyed me, but that same thing labeled "meme" didn't bother me at all.
Perhaps half way down the list inject, advert style, "Three from /newest:" add in three rows from newest, styled slightly differently so viewers can continue to scroll past. You could even try putting these at the bottom instead of half way down. If it was at the bottom, you could increase it to 5 or more. That would be the least intrusive and get the most eyeballs.
I'm curious, do you have any theories as to why the job ads don't elicit this negative reaction? Is it just because there's no comments, so people have no place to complain?
The job ads have been around forever (or almost forever, a few diehards will insist), so they stand out less than something being introduced for the first time. Also, the noise level is lower because they are sort of all the same and easier to tune out—whereas random posts from /newest bring new forms of annoyance on a regular basis.
It isn't just that, though. It's also that the community has a sense of fairness about HN giving some things back to YC in exchange for funding this place. I say this because when we introduced Launch HNs for YC startups about 3 years ago, there were few if any complaints.
IIRC, the job ads are a perk of getting into Y Combinator. So even if they are seen as annoying by some, it's expected that they will just suck it up and deal. After all, Y Combinator is why the forum exists.
i had thought of using simulated annealing for a meme-based social sharing platform around 2014.the idea was after "cooling" to a point it would be removed from the stack. honestly i never developed it past that line of thought.
HN could implement an interview style vetting. A new post is shown to a N interviewers who vote yes/no. The stddev is sqrt of number of votes. Once the number of upvotes crosses the 95 tile, the post is admitted. Once the number of downvotes crosses the 95 tile, the post is rejected. Otherwise the post stays in the undecided area until it expires.
The simplest solution in my opinion would be to expand the length of the new page. Good stuff simply lives there for too short a time to gather sufficient upvotes and people rarely click through to page 2.
I think it works fine like it is. Things easily get on the frontpage so they can get some exposure, but they only stay there if enough people continue to vote for it. If someone flags it, it also disappears quicker.
The value is not about how fast it gets to the frontpage, but about how long time it lasts there.
If an article manages to get a few votes, quickly, then it gets on the front page fairly easily. (This one got there with 5 votes.) Once on the front page it snowballs. The obvious solution to that is to ask a handful of friends to vote for it. HN has a solution for that workaround by identifying vote rings and banning them, plus banning the site that was attempting to do that. So getting on the front page remains a crapshoot, even for good material.
> If an article manages to get a few votes, quickly, then it gets on the front page fairly easily.
In principle yes, but HN's anti-voting-ring software often drops a lot of those votes. If you see something on /newest with a bunch of votes and it's not on the front page, that's likely why. A side-effect that we never anticipated is that some eagle-eyed HN users figured this out and started calling out the egregious cases. So we got a secondary hivemind ring-detector out of the deal.
Yeah this is confusing to me because they imply you need a certain score to get on the front page but I'm seeing plenty of links on my front page with very few upvotes.
The decay rate for front page items has always felt off to me. Quite often you'll see articles with 20-30 points and no comments camp in the middle of the front page all day, while more actively discussed items drop off.
Comments don't affect story rank, with the exception of the flamewar detector mentioned in the sibling comment by krapp. It's not uncommon for certain kinds of story to get plenty of upvotes but no comments. Those stay on the front page for a while. I think that's fine; something can be interesting even though we don't have anything to say about it.
Items with active discussions are often implicitly penalized, because the number of comments are often greater than the number of upvotes, which is interpreted as a sign of low quality discussion or possible flamewar.
I find it to be a pretty good gate of quality submissions and I don't have to sit on the site all day... the other bonus is I catch stories that get flagkilled after that threshold that I would otherwise miss which is sometimes nice.
You may also be interested in Hackerer.news[0] (mind the extra 'er'). It groups posts by day. Clicking on the heading switches from popular to chronological, then Ask/Show views.
Speaking as an Australian raising Swedish kids, it's going to be weird watching this with them in s couple of hours and acting like it's a natural thing to do!
(Christmas is of course really about eating mangos and drinking cold beer in the sweltering Queensland sun...)
Make front page participation accessible through participation in the newest section. You have to earn the privilege of participation. Upvoting the first X items on the list, so as to just get the work done, is an issue I haven't figured out yet.
It can be viewed as simulated annealing. A brand new article starts at a "high temperature", it has a chance of showing up randomly for a selection of people. As time goes on the temperature lowers. And it either made it or didn't.
That is suppose we have a function called rand_hash that takes an article id and a user id and produces a random number between 0 and 1. Suppose that T is the time in minutes that a new article has been up. For position we treat a new article as having a vote of:
Where C and k are factors that affect how much we want a random boost to be able to raise a new article, and for how long we want it boosted.Then what happens is that every person will see, mixed with the popular articles, a random selection of new ones. The new ones that are particularly interesting will organically pick up a few votes and then snowball. The new ones that aren't will disappear from the random mix.