
How reddit tried to solve the "new link" problem. Why HN doesn't need a new algo - jedberg
This morning I saw two articles on the front page about how HN should change their algorithms.  I would contend that an algo change is not the right solution.<p>Here is what we did to try and solve the problem on reddit.<p>First, there is the &quot;organic&quot; box at the top of the page.  The first link in that box is always an ad, but after that, it shows pseudo random links from the new page (more on that in a second):<p>https:&#x2F;&#x2F;github.com&#x2F;reddit&#x2F;reddit&#x2F;blob&#x2F;89f6f1ad9c1babbf520b83c49fa27f509bb5d0ef&#x2F;r2&#x2F;r2&#x2F;lib&#x2F;organic.py<p>What this does is give exposure to up and coming links to a lot of people all at once, which helps overcome the luck factor of who is looking at the new page at any given time.<p>The second solution is the &quot;rising&quot; sort on the new page:<p>https:&#x2F;&#x2F;github.com&#x2F;reddit&#x2F;reddit&#x2F;blob&#x2F;89f6f1ad9c1babbf520b83c49fa27f509bb5d0ef&#x2F;r2&#x2F;r2&#x2F;lib&#x2F;rising.py<p>The rising sort accounts for how many times the link has been shown in its ranking algo, which helps better new links rise to the top.<p>The organic box on the front page uses this rising ranking to choose what is in the box, and also contributes to the view counts.<p>So I would humbly suggest that HN should do as it has done often in the past, and copy reddit&#x27;s solution here by implementing the rising sort and the organic box.
======
jedberg
Why was my title edited?

It said "How reddit tried to solve the "new link" problem/ why HN doesn't need
a new algo"

How is this new title an improvement? I would at least expect a comment here
when a title is edited as to why so I can learn for next time.

~~~
ClayFerguson
At least they didn't censor you when you posted the note saying they edited
your content. If they wanted to they could have just deleted that too. I think
any sort of censorship at all, once the word gets out, will be damaging to
HackerNews, and they should stop doing it before it becomes their
"reputation". Communities like this can form elsewhere. Hopefully they know
they have no monopoly, and should err on the side of NON-censorship at all
times.

~~~
jedberg
It wasn't censorship. Only governments can censor.

HN is a private organization and has every right to do anything they want with
the content.

My only request was for feedback.

~~~
markburns
> Only governments can censor.

Bit of a side issue, but that's an interesting statement, I've never heard
that before. A quick google doesn't seem to back that up, so I wonder what I'm
missing, or why you said that?

~~~
arjie
When Americans use the word 'censorship' they seem to explicitly refer to
censorship in the context of the First Amendment to the US Constitution. By
the general meaning of the word, of course anyone can censor anyone else but
in America it is meant to be "censorship that violates the First Amendment".

I'm not American. It's just the best explanation I can come up with why this
view is so common.

~~~
markburns
Thanks, yeah that satisfies my curiosity and seems like a reasonable
explanation.

------
jawns
I think the techniques Reddit uses to give exposure to deserving links are
nice.

Here are just a few other ways you can fine-tune a "link recommendation"
algorithm beyond just the standard "show highly rated links at the top"
technique:

1) Devote a portion of prime real estate (e.g. homepage) to new or trending
links, as Reddit does.

2) Give higher placement to submissions that come from someone whose previous
submissions the user has upvoted.

3) Give higher placement to submissions that come from the same source as
previous submissions the user has upvoted.

4) Give higher placement to submissions on which a person has commented whom
the user has previously upvoted.

One way I think HN, Reddit, and other link-recommendation sites can put power
into their users' hands is to allow each user to tweak the recommendations
algorithm to suite their own preferences.

For instance, one user might want half their homepage to be filled with
trending stories, rather than popular stories. Another user might find
Technique 2 above to be useful but might not want to enable Technique 4.

~~~
jedberg
Your suggestions sound great on the surface, but I suspect 2-4 would increase
the echo chamber problem.

Also, it's computationally difficult to compute 2-4 in real time (reddit used
to do a similar calculation a long time ago, under the now defunct recommended
section).

~~~
jawns
Yeah, just throwing stuff out there.

One other technique that springs to mind:

5) Set a minimum number of views or clicks a link must get before "falling
off." So, if a ton of links are submitted around the same time, sprinkle them
back into the mix -- perhaps using a version of Technique 1 -- until they've
hit the minimum, then let them die a happy death.

------
strict9
This issue isn't what need solving. The much larger problem is the "unknown or
expired link" page.

What year is this? Why are we still accepting an implementation detail as an
excuse for an awful user experience?

~~~
arh68
Well what's the solution?

Do you simply redirect all "expired link" GETs to the home page? Should we
have a fixed # of 10 first pages (like some imageboards)?

~~~
randallsquared
The solution is to avoid capturing the state of the user on the server,
passing it in the URL instead. That way nothing need expire about the link.

------
eggbrain
I think the biggest issue is that the Hacker News admins want to have as few
moving pieces as possible. It's why Hacker News doesn't have collapsable
comments, why it doesn't have a mobile layout, etc. There's been some tweaks
here and there, but I think the biggest change I've noticed over the past ~3
years I've been here is that they removed karma count from comments, and they
made the up-vote triangle high resolution.

They seem feature-adverse, and I assume it is because A) Adding more features
lead to more causes of failure, B) Front-end/back-end code additions leads to
higher page file size/more computation on the backend (meaning higher costs
for them to deliver content) and C) the K.I.S.S Principle

~~~
icefox
I would have to put my money on the way simpler answer D) A few users might
complain, but the traffic keeps growing so why bother and E) YC is eating up
all my time and is more interesting and F) The site clearly works (the traffic
keeps growing) so it isn't as interesting to hack on anymore.

------
recuter
I understand the endless desire to tweak, I really do, but HN simply doesn't
have anywhere near the volume of posts Reddit does.

This place has the same feel as:
[http://www.reddit.com//r/depthhub](http://www.reddit.com//r/depthhub)

Most of the action is in the comments and a lot of the traffic is from
lurkers. Its a slow roll in other words and frankly there is a very finite
amount of good quality new posts to be had.

That is the real problem - try and rank this place more optimally by hand as
an experiment, it simply won't take that long. Where is all the great content
you are trying to algorithmically float?

~~~
amerika_blog
That place is the worst of the self-congratulatory, narcissistic, Reddit hive-
mind.

~~~
recuter
I'm afraid I have to stand by the analogy. ;)

------
gabemart
Does the "rising" page on reddit actually work? On subreddits with millions of
subscribers it seems to show one or two submissions, at best, and often the
votes cast on those submissions don't reveal why they're considered to be
rising. For example, many submissions on the rising page have scores, using
the format (upvotes,downvotes), of (1,0) or (1,1)

Is this the intended behavior?

~~~
jedberg
Yes. Remember, it is a function of both votes and views, so something with two
votes that's only been viewed a few times will still be higher than something
with more views and two votes.

It's not perfect, but it is better than just straight chronological.

------
Houshalter
The solutions presented were better and more mathematically sound. This might
work, but that algorithm maximizes the exposure of deserving posts and
optimizes the tradeoff between testing new posts and sticking with proven
ones. This is just arbitrarily throwing new links to a top box that everyone
ignores.

And reddit's solution doesn't seem to be working much better IMO, tons of
posts get buried with little exposure. The rising section seems to be usually
empty or just 2 or 3 totally random posts.

Reddit's algorithm is often criticized for heavily favoring quickly consumed
content like images because they get vote quicker. Also easily manipulated by
bots/sock-puppets.

------
mburst
Not sure if a rising sort is really needed. A link only needs a few points to
get on the front page of HN. Based on my experience if a link gets 3 votes in
the first 15-20min or 5 in an hour and it'll get some frontpage time.

I think it may almost be easier to just show a random new link from the past
hour rather than doing anything fancy. I'm sure a ton of good content misses
the frontpage just because of the sheer lack of visibility that links on the
new page get.

------
shaunrussell
I think the real problem is that most of us come to the site too often... take
a stretch and go outside :)

------
joshdance
I think just giving a small portion of the front page to new stories could
help a lot.

