Hacker News new | comments | ask | show | jobs | submit login
Ask HN: Downtime. What was changed?
71 points by Kiro on June 10, 2014 | hide | past | web | favorite | 62 comments

Global hacker productivity rose by 25% as a significant outlet of procrastination was temporarily placed out of reach

Refresh. Nope. Refresh. Nope...

For some of us this disturbed the whole working day: Having to log into twitter at work to check what is happening with HN.. no good.. ; - )

Anyone notice that all the threads contain nothing except the usual "productivity gained while down" comment, and the "how dare you keep the site down for so long" comment?

Someone writing a Turing Test bot to imitate HN readers?

I had to play more 2048!

I laughed!

Downtime message:

  Changing how profiles, passwords, items and votes are 
  stored on disk.

changed how passwords are stored on disk is something that might be worth elaborating on.

rot13()? :-)

Edit (for the benefit of the downvoter(s)): Yes, I do understand that HN is not really that poorly written; it was a joke.

Twice, just to be extra secure!

My productivity this morning.

The worst part is the lack of fresh content in my morning.

I really think it's a disgrace that HN is down for hours just to do some (probably important) back-end work. Here we are, talking about scalability, high-availability, what have you - and the medium we're using to host those discussions is built on 90s web technology and operated as one.

I understand that Hacker News is probably run by a small crew, but I think it's a disgrace anyway. If you guys need money to make these things go away, then I'd be more than happy to pay. But to have such an important forum go offline for what is basically internal plumbing - that won't do.

EDIT: To the down-voters: please explain yourself? I know this message is harsh, but the truth is that having a site go offline for hours for internal maintenance is highly unprofessional and unbefitting to a high profile outlet such as Hacker News. Also, I'm offering and more than willing to pay (on a subscription basis or something) to make sure they have the capacity to properly deal with things like this in the future.

Look, I think the message delivery here is very poor -- there's no need to be rude. And getting angry is uncalled for.

That said, I do agree with the sentiment expressed here: it is pretty amazing that a site that is really a nexus of high-tech Silicon Valley companies and figures -- quite possibly the most popular one in that segment -- still has downtime and a web interface reminiscent of the 1990s.

And I disagree with those people who say that the site is operated entirely altruistically. There's definitely a huge benefit to YCombinator -- not least of which is raising awareness of their investment opportunities. Also, YCombinator uses HN as a very effective advertisement medium for job offerings for the companies it has invested in. Companies would pay big money for that kind of exposure of their job ads here.

To be clear, I have absolutely no problem with the above motivations. I think it's great that Y Combinator is able to leverage HN is such a way. But I think HN does owe its audience a site that is run and managed in a manner befitting its status as a focal point of global high tech and entrepreneurship.

  That said, I do agree with the sentiment expressed here: it is pretty amazing that a site that is really a nexus of high-tech Silicon Valley companies and figures -- quite possibly the most popular one in that segment -- still has downtime and a web interface reminiscent of the 1990s.
I actually like the UI. It's simple, doesn't use tons of bandwidth, and gets the job done.

I quite like the UI in terms of interface.

I rather dislike the UI for a couple of specific things: expiring links, low contrast in places, and not seeing points.

How many other websites can be browsed with Links?


Let's make sure dang is wearing a shirt and tie everyday. Maybe we should be asking to see some timesheets? Oh, and about those TPS reports? We're putting cover sheets on the reports before they go out now. So if you could go ahead and try to remember to do that from now on, that'd be great. All right!

I realize you're being sarcastic, but what does a corporate environment have to do with being efficient or competent? I don't associate the two -- in fact, I find that very traditional corporate atmospheres are often a strong signal of incompetence.

I have no idea what TPS reports are.

In any case, please realize that I took the time to write a respectful and non-emotional comment, and it clearly struck a nerve since it's been upvoted to the top subcomment above dozens of other comments.

I am very grateful that a place like HN exists, and I appreciate all the work that has been done building it up and maintaining it. I'm just pointing out an area that I think needs some work. And that's just my (respectful) opinion, nothing more.

> I have no idea what TPS reports are.

"Office Space" is an important movie for anyone reading HN.

Oh no - now you have me thinking about what movies are important for HN types.

e.g. "Primer" - would having a time machine help with downtime?

It's a forum. We come back. They don't lose revenue when it's down. Why would they do anything any differently? HA when it makes sense, sure, but were I running the HN stack, I'd tolerate 1% of downtime happily. On our eCommerce platform, 0.01% downtime is an unmitigated nightmare, however - because it generates revenue.

I understand that they don't have any incentives to do this differently, and (given that this forum is offered to us for free) that we should have no reason to complain, but I'd rather have them staffed properly so we can prevent this situation in the future.

Hacker News is providing an important service to me, and because they're offering it to me for free, it means that they don't have the capacity to do these backend changes properly, that it's unavailable some of the time or that we don't have a reasonable mobile version.

That sucks, and I'd rather pay them to make sure they are in a position where they are able to offer these things. To me it's worth money to make sure it stays online - always and forever. If you don't feel like that - fine. I'm happy for you to freeride on my contribution.

I am just really disappointed that a service that's important to me goes offline for hours, and I am willing to contribute to make sure that they don't have to.

First of all, thanks Dan for all the hard work.

Think about it this way: you need your fix so badly that within minutes you're back here to vent your frustration.

And yet you have nowhere else to go. So HN can afford the downtime easily, and with what I know of the way it is put together the reliability is astounding.

Now imagine it shuttin HN down for a month! (I sensed a disturbance in the force there when you read that.) Within an hour of switching it back on the traffic would be right back where it was before.

Hm, maybe there is an idea in there somewhere. A website for hackers with 'opening hours' that simply refuses connections from your locality when you're supposed to be at work.

It would probably be sponsored by some VPN supplier ;)

> "A website for hackers with 'opening hours' that simply refuses connections from your locality when you're supposed to be at work."

I'd argue that HN already has this kind of functionality with the noprocrast setting [1].

[1] Last question at http://ycombinator.com/newsfaq.html

Yes, but that's easily defeatable. I know that because err. never mind.

I actually submitted a bug report because noprocrast stopped working properly for me, so it became pretty much unusable to actually go on hackernews for more than the allowed time and post comments, which is really more in the spirit of noprocrast... but incredibly frustrating.

It took years (weeks) to fix. I disabled noprocrast in the mean time. (I guess I should turn it back on now)

If you use this feature and you noticed the bug, thank @kogir I guess!


Yup, I posted it more because it's easy to miss those features of HN if they're not occasionally pointed out.

> thanks Dan

Thank Nick! I did keep him company though.

Ok, thanks Nick!

I hope you guys did not lose too much sweat. Messing with a live site like this always makes me lose weight.

If we're talking business ideas, then I think it should be "Reddit Gold" but for Hacker News. Because: Why not? I'm not paying for Reddit, but I most certainly would for Hacker News. It's a crucial news source for me, more important than any other, and I'm very (very) willing to pay for it.

If it keeps their servers up and the downtime low, and it gives them time to develop new things (such as a proper mobile version, or a non-table-based design), or time to fix the "Unknown or expired link" bug... then why not? Is there any reason to run this site like it's 1999?

Throwing more money at the problem is likely not going to make a big difference but I do appreciate your willingness to put your money in. Most people that bitch about free services only bitch that they'll leave if they don't get what they want when they want it, very few of them offer to pull their wallet to help solve the problem. So kudos!

Well, adding a monetary component into the mix would achieve two things:

1. HackerNews would have more resources to do maintenance and development on the site. Both of which are relatively necessary: There's a lot of low-hanging fruit.

2. It would create an incentive to improve the service and prevent downtime. Would they really put the site down for multiple hours during (European) office hours if they had quite a few paying customers?

Primarily, though, I think it boils down to the fact that Hacker News was originally run as a service towards the YC alumni, and that it serves that purpose very well as it is - and as a result, product development has stalled. The original author (PG, the product owner) is busy doing other things, and has employed other people to run the show.

Truth is, Hacker News has outgrown its original audience (YC alumni) and would be ready for some changes or improvements. Nothing major, because it's great as it is, but it could use some bug fixing, or maybe getting the HTML and CSS up to date, or a mobile version, finally fixing that stupid 'up' arrow... I could go on. I care about Hacker News, and I want it to improve and to be available permanently, and most certainly not be offline when I wake up and am ready for my 21st century equivalent of the morning newspaper. That's why I wrote that angry message, and that's why I'm offering to pay to help achieve that.

I'm sure the team that runs HN will happily refund your subscription fee in full.

I can't tell if you are being sarcastic. The site is free, and free from ads. They owe you nothing and they are free to maintain this resource as they see fit. You have no business calling it a "disgrace."

I agree with the content of your message, but:

> free from ads

It is not, to insert news to hire hackers for YCombinator startups is an incredibly useful thing who run this site is getting back, and it is a form of advertising.

I see that as a service to the Hackers just as much as to the start-ups. Everybody wins.

Said every advertiser ever (and every recruiter that ever cold-contacted you). I do appreciate your point and although I've no problem with the job posts, they are still advertising.

The fact that an ad is well-placed makes it no less an ad.

I totally agree, but it's still a very valuable form of advertising. Companies would pay big money for featured ads on HN.

They don't have to pay, they can just submit.

Or wait for the 'who is hiring' thread that rolls around the 1st of the month.

YCombinator companies clearly get boosted ranking because there's no way some of those ads would otherwise be on the front page. I have no problem with this, but I don't understand why you're trying to argue with it being advertising.

>They don't have to pay, they can just submit.

Yes, they could, but there's no guarantee at all that it would be upvoted to the front page and stay there for multiple hours.

And 'who is hiring' threads are great, but it's easy for an employer to get lost amidst the other comments.

I think the who is hiring threads have far more visibility than the YC calls for employees.

Interesting observation: that model does not scale anyway. As the YC portfolio grows you can only advertise so much before you start to hurt the pool of founders for the next batch and before you fish the pool empty of new talent. So likely even YC companies will face some kind of rate-limiting for these kind of posts, or alternatively, they should simply join the masses and post in the Who is hiring threads. That would level the playing field. Or, yet another option, an open 'jobs' section that any company could post to.

I fucking loathe the attitude that HN should be anything other than what it is. If you like it, use it. If you don't like it, don't use it (and hopefully make something that you do like).

But don't, when someone has spent a few hours tinkering to fix some stuff, complain about unprofessional. Or do, but don't then complain about the downvotes.

I think this shows that scalability and high-availability can be overrated. HN is back online and so are the users. No harm done.

Not to mention the dreaded "Unknown or expired link" message.

Not sure why you were downvoted, but I completely agree. I get the 'Unknown or expired link' message all the time because I'll write long comments, or write part of a comment, get sidetracked, then finish. There's no reason for this error message, and to have tokens time out after what feels like a matter of minutes.

Then of course this happens on the homepage too, when you click 'more', after being inactive for a while. Why?

And why is there no pagination?

Why do a click reply on a comment and get redirected to a completely separate page, instead of replying inline? And then when I submit that comment, it just redirects me to the top of the comment page, so I have no idea where I left off, and now the comments might be shuffled since votes have changed.

These feel like critical changes that benefit everyone, and require little work. Why are they never addressed after years?

Its because this website was designed using a neat and novel model, that ultimately doesn't work.

(Most links on this website (ones with ids at least) are associated with state, stored on the server as opposed to in the link / cookie / etc. As there are an absurd number of links accessible at any one time, the number of serverside state objects that need to be stored gets excessive, so they get garbage collected after a while.)

It's a way of warning you that someone else might have made the same point that you're about to make and that you should reply in a fresh thread.

Prepare your replies in a text editor and copy 'n' paste them to the new text entry box.

I simply can't believe that's the reason. It's an error message, not a warning. It doesn't check if new comments have been posted to that thread, it doesn't prompt you with the new comments, and ask if you want to continue with the post, and it doesn't make it easy to go back to the original thread to check if there are new comments.

Asking users to type in an external text editor, and then copy/paste isn't a reasonable solution for a message board.

Well, have fun with your continued error messages I guess.

I think it's hilarious how angry people get when a website goes offline for an hour or two! The bane of any experienced webmaster is suffering this sort of criticism.

It's not. No damage done, possibly quite the opposite, but It's an overkill to have own hardware and in-house http for it. Rewriting it for heroku with Flask or Node would be quite simple.

Oh c'mon, HN isn't that mission critical to anyone's life. I don't mind if it goes down periodically for a few hours at a time.

There are other nice things on the internet to go and look at, it's a mighty big place. Or, read a book.

There are applications where 99.99% uptime matters. There are applications where it doesn't.

Maybe there is a lesson here about engineering to a set of requirements.

Maybe there is a lesson about customer selection.

I suggest you ask for your money back and cancel your subscription.

Perhaps you should upgrade your subscription plan for better uptime. Oh wait.

Yep, the downtime is hypocritical.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact