Poll: Unknown or expired link on YC

[bayareaguy]

I'm seeing this a lot more often these days, even when just pressing "More" at the bottom of the main page without being logged in.

It makes me think the current continuation-based session scheme isn't enough to guarantee a good experience.

What do you folks think?

[cousin_it]

Voted for #1.

Maybe a heretical opinion, but why use continuations here at all? Their drawbacks aren't worth it in a simple app like news.yc. Use URL parameters, like /news?skip=30 for paging. Make the "reply" link lead to /item?id=12345 , it already has a reply box.

In my own apps I give a lot of thought to URL design. And yeah, don't explain, I do know about closure/cont-based web frameworks - used to be a contributor to Borges back in the day, then wrote my own one ;-)

[vthakr → cousin_it]

I can totally agree with you here cousin_it. I've noticed this problem a few times on the main user-facing [more] link at the bottom of the news listings and I don't see any reason for it. From what I've seen/read/advocated it's best to use continuations for the backend things such as the admin pages, but keep all of the forward-facing pages as continuations-free as possible in order to avoid just these types of problems. As a user of the site, I should be able to bookmark any page I desire and return to find it still exists rather than the continuation has timed out and I lost a list of really great articles that I was hoping to read.

[kf]

I usually only get it if I have a comment reply open for a long time before clicking add. Feels like the continuation lasts for about 20 minutes.

[pg → kf]

There's no set timeout. They're kept in a queue.

[keating → pg]

With the load going up so much (causing the crashes) it also means the links expire faster. I would guess in under 10 minutes sometimes, maybe under 5.

[aston → kf]

Pretty sure that's what pg's said it is. Personally, that's way too short, as I'll leave threads open for half a day without reading it, then can't comment.

I don't see any fundamental reason why continuations have to expire so fast.

[gnaritas → aston]

Because they take up ram on the server. 20 minutes is standard session timeout for most session based web frameworks.

[axod → gnaritas]

Reason enough not to use a 'session based web framework'.

If PHP can do it...

[phaedrus → gnaritas]

I wonder - would it be technically feasible to page the continuations out to temporary files on the server? Kind of like cookies, but on the server instead of the client!

[shiro → phaedrus]

Technically possible. There are a few Scheme implementations that have persistent continuation (all I know is JVM based ones). Also this page contains a link to the slides that talks about persistent delimited continuation in OCaml: http://logic.cs.tsukuba.ac.jp/Continuation/program.html

Making continuations serializable may cost its implementation, so whether it scales well for yc.news is anothoer question. I don't know the answer for it.

[LogicHoleFlaw → shiro]

In lua at least, continuations have a 1k stack overhead in memory per continuation. I believe they are serializable as well... I'll have to double-check that. At 1k+ a pop they're not cheap, but are manageable.

Having my tabs open last for a couple of days instead of 30 minutes would make the site more pleasant for me.

[ericb]

I edit my replies a lot. I rarely manage to post without seeing expired link and needing to refresh.

[h34t]

Almost every time I write a comment I get this error.

update: Including this time.

[TrevorJ]

I see it a lot when I get down to the 5th page or so. Yes, I am really a nub with nothing better to do than hope that there is a 6th or 7th page of submissions on here :-) I really can't get enough of this site.

[pius]

They've been hard restarting the server a lot lately, so the sessions are getting lost more frequently.

[andreyf]

It usually isn't a big deal, because the fix -go back, copy, refresh the page, paste- is trivial. For a less technical audience, this would be absolutely unacceptable.

Are there elegant solutions to fix this? Could someone link the explanation to how the continuation sessions work, please?

[bayareaguy → andreyf]

I believe the specific code has to do with "fnid harvesting". See the comments here: http://github.com/nex3/arc/tree/master/srv.arc line 330 or so.

The general idea is described here: http://dresese.thehyatts.net/archives/000253.html

pg's original patent on this scheme: http://www.freepatentsonline.com/6205469.html

[rsheridan6 → andreyf]

Explanation: http://pagesperso-systeme.lip6.fr/Christian.Queinnec/PDF/www...

A possible elegant solution is suggested in the paper: store the continuation on the client. I don't know what problems this presents in practice.

[nostrademons → rsheridan6]

With GET requests, you frequently run into URL size issues. IE has a roughly 1800 character limit, which is pretty easy to hit with machine-generated data. I remember going through hell with JSF charting frameworks, trying to squeeze the serialized data down under 1800 characters. Eventually we gave up, pulled everything out of the serialized data (it still took up over 1K though, just through framework metadata, and so only left us around 500-800 bytes to play with), and threw it into query parameters that we could control ourselves.

This isn't an issue with POST, but then you run into the normal POST issues with refreshability, bookmarkability, and portable URLs.

[jcl → rsheridan6]

I would imagine that serializing your server's execution state to the client would require some pretty careful sandboxing to avoid security problems.

[aston → andreyf]

It's pretty similar to other session-based systems. Users get an id that they resubmit at some point to the server, and the server pulls the data associated with that id. In the case of Arc, that data is a closure (basically, a function and its bound variables).

[gommm]

As I clicked to vote "I spend a lot of time on YC and I'm seeing this a lot too." I just got and unknown or expired link error.....

[quellhorst]

I tend to leave tabs open and when I come back later and click the more link the error often pops up.

[alex_c]

I read a lot, but I've only really noticed it when posting comments. I've got it maybe 3 times over the past week or two... which I guess is a lot, since I don't comment that often.

[ralph]

Continuations used to be used more on news.yc but it was trivial to flush the cache by making a few requests from a single client causing everyone's sessions to be hosed. That causes a switch to URL parameters for lots of things; why not all things? It's a fundamentally flawed model that I've never seen pg defend. It's certainly not scalable. Either the server stores info for all N clients, which fails for large N, or you give each client their own piece of info to store.

[ralph → ralph]

Let's take http://news.ycombinator.com/threads?id=pg as an example. When I visit that I get dealt 75 fnids. They're different every time I go to the page. So I'm using a new 75 slots in the cache every time. It's little overhead in CPU or bandwidth for me to get the page but it has a big impact on the other users of the system if I flush their fnids out of the cache before they try and use them. I first pointed this out at the end of April 2007. In response, pg cut the number of fnids used, but they've crept back in.

The technique's flawed, trivial to exploit, and should be dumped.

[ralph → ralph]

Here's the last time I pointed it out. http://news.ycombinator.com/item?id=18083 pg made the post dead and emailed me about publishing a DDoS. Yet here we still are, using a broken concept.

[sah]

I frequently get interrupted while writing comments and finish them a few minutes later, so I get bitten by this all the time.

[michael_dorfman]

I get this often, but since refreshing the page always fixes it, I've assumed that it's just because I've used the "back button" to get to an expired version of the page.

[xenoterracide]

Haven't noticed it. but I mostly access YC through rss.

[edu → xenoterracide]

This issue raises when commeting.

[tstegart]

definitely having problems with the more button

[lst]

And no one ever tells us the drawbacks of continuations...

(Why is this so?)

[jcl → lst]

These error messages are a consequence of any scheme that stores session state on the server: the session data must expire, or the server will fill up. The problem would remain even if some other method of serialization was used instead of continuations.

That said, I expect that continuations store more data than a hand-tuned serialization and thus need to be expired earlier. I'd be curious to know how much more space is used storing continuations versus the bare minimum.

[LogicHoleFlaw → jcl]

Some sort of stateless method of interacting with a site helps alleviate the session problem. That's one of the touted advantages of REST architecture.