Hacker News new | comments | show | ask | jobs | submit login
How Does HackerNews Pagination Work?
11 points by wpeterson 2653 days ago | hide | past | web | favorite | 8 comments
I'm familiar with traditional SQL list pagination (limit/offset) using a page number query parameter.

Hackernews seems to generate a token for the next page that has a limited cache/lifetime.

Is this a common design pattern? Where can I find more information.

The code that generates list-structured pages spits out a page of n items, then saves a closure that will keep going if asked. Since closures can't conveniently be written to disk, you have to gc them after a while or you'll have a memory leak.

It may be overkill to use this approach if you just want to generate the frontpage, but the advantage is that it's very general. You're not limited to displaying a range of items stored in a list somewhere; you could be displaying things you're computing on the fly.

That's fascinating.

Is the stored closure working on a snapshot of potential articles or does it have a state-less algorithm that can generate next n articles?

People complain often about traditional offset/limit pagination, but this the most interesting alternative I've heard.

> People complain often about traditional offset/limit pagination

I haven't heard any. What kinds of complaints have you heard?

I appreciate the generality and elegance of the closure approach, but the links expire far too quickly for my taste. About once a week I read a page slowly enough that the "next" link is dead by the time I click it.

I was working on a caching problem and it reminded me that caching requests for a resource with standard pagination sucks.

You cache Page 1 with some set of objects, by the time you cache Page 2 you have Page 2 of a different overall set. It's inconsistent and wonky.

I was hoping there was a more clever way to tease out an elegant solution here.

I agree. It is occasionally disruptive to the user experience.

For pagination, this is a very minor inconvenience. However for topic threads such as this one, expiring links is irritating.


When you removed or upped the limit on how far back one can look (this was done a year or a bit more ago?), that change was made to news/ and newest/, but not to classic/ (still cuts off at 210 items). Any chance of also lifting/upping the limit on classic/ ? Once in a while, I like to dig back a bit through it.

(If not, hope you don't mind my asking.)

I didn't increase the upper limit for news and newest. I changed the code so there was no limit. I can't do that with classic because I have to do a sort before I display anything. The front and new pages are simply displaying already sorted lists, so they can be written as if the lists were endless. But with classic I have to pick some number of items in advance to sort and then display.

Oh. Thanks for the explanation. Sorry for any bother.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | DMCA | Apply to YC | Contact