
Is Your Site Hacker News Ready? - ger_phpmagazin
http://www.brandonsavage.net/is-your-site-hacker-news-ready/
======
jenius
If you're hosting some sort of blog post, the best way to ensure that all goes
well with high traffic is to make it static. Something like jekyll works great
for this (publishing especially), and you can host any static site with a node
server in like 3 lines of code on nodejitsu for $3/month.

Nodejitsu can handle hacker news traffic without even the slightest blink,
with one dyno - I had a post up on the homepage for hours with over 200
concurrent visitors and like 13,000 total for the day and never had a single
issue. Again, $3/month for that.

No need to over-complicate things : )

~~~
pablasso
Really, I don't know why dynamic languages became so popular for blog
platforms.

It's an overkill all you need is static content.

~~~
steveklabnik
Non-programmers find it much easier to log into a site, type in a text box,
and click 'submit' than to run some sort of static site generator.

~~~
Osmium
If you're at the stage where you're maintaining your own server anyway, it
doesn't seem like too much to ask.

Static websites are glorious. Much more secure, much faster to load, far fewer
things that can go wrong. Cheaper, too, since they require fewer resources.
And, as luck would have it, there's a perfect Hacker News thread to get one
started:

<http://news.ycombinator.com/item?id=4857473>

~~~
nthj
Or, if you really want or need a dynamic website, throw CloudFront or similar
in front of the site.

Then lock down your CMS at a secret, SSL-only URL.

It's about 2 lines of Ruby or PHP to ping CloudFront and request an
invalidation when you change a page (which only happens rarely anyway.)

Secure, fast, dynamic, only goes down if CloudFront does.

------
thraxil
I'm constantly amazed at how poorly Wordpress is able to perform. 2.5GB
virtual server with a cache in front of it and it went down?

My blog is a Django app running on a 512MB Rackspace VM with a basic Apache,
mod_wsgi, and Postgres setup. I have a dozen low traffic applications all
running on the same server alongside it. I have zero caching set up so every
hit is doing the full process of querying the database and pushing the results
out through a template.

I was on the front page of HN last year and the server load was negligable. I
have graphs: [http://thraxil.org/users/anders/posts/2012/06/02/How-
Django-...](http://thraxil.org/users/anders/posts/2012/06/02/How-Django-deals-
with-a-Hacker-News-frontpaging/)

~~~
lemcoe9
I bet if you switched to nginx, your server would use even less resources to
handle the same amount of traffic.

~~~
thraxil
Having used nginx on other projects, I'm sure you're right. My blog setup is
pretty old and hasn't really needed any improvements.

On newer deployments, I'm still using Apache+mod_wsgi to manage the Django
processes (it's a setup that has been reliable for me and that I know how to
debug), but I'm sticking an nginx proxy in front as a matter of course. Not
for performance reasons, but just to leave myself a hook for handling later
changes. I can trivially spin up additional application backend servers and
load balance across them, or quickly flick a switch to have nginx cache a
particular resource in an emergency.

------
krapp
Mostly as an educational exercise (perhaps in futility) I decided to try
writing my own small php framework specifically to build and serve static
files, load external libraries only when necessary and automatically send
caching headers. The point being, for instance, not to even load Dwoo except
if compiling the static pages (as opposed to fetching data from a view,
building them and serving them with each request.)

Now i'm wondering if i'm not both overoptimizing and missing the point
entirely.

And yes I do realize 'custom php framework' is missing the point of frameworks
and that php is an abomination. Getting Laravel to do what I wanted (awesome
as it is, and it is) started to seem like trying to tweak a cannon into a
flyswatter.

It's still fun, but I dread it ever getting posted here. Assuming it gets
finished.

~~~
AlexCP
Building a static site generator is always a good learning exercise.

------
Udo
NginX + php-fpm + APC did the trick for me, at one time even in an EC2 Micro
instance under heavy load.

There is a very nice hack I applied to my WordPress installs, but it's not for
the faint hearted: hack the WP index.php to serve pages directly from Memcache
for all non-authenticated GET requests. The beauty of this is that for most
requests, the WP environment (which is horribly bloated) does not even get
loaded.

It works about an order of magnitude better than all those cache plugins, but
the downside is of course that software updates require some special care.

~~~
EwanToo
You could just pop Varnish cache in front of Wordpress to do the same thing -
any unauthenticated GET's will just hit Varnish, and come nowhere near
Wordpress itself.

That way you wouldn't need to worry about software updates.

~~~
Udo
Totally. I guess I could also use some advanced Nginx directive/plugin to do
that. The hack is just something that occurred to me spontaneously, it took
about 2 minutes to implement, and I didn't even have to install anything :)

~~~
EwanToo
You could give these 2 config files a try sometime:

[https://github.com/ewanleith/Wordpress-Server-
Configuration-...](https://github.com/ewanleith/Wordpress-Server-
Configuration-Files/blob/master/nginx-caching/default.conf)
[https://github.com/ewanleith/Wordpress-Server-
Configuration-...](https://github.com/ewanleith/Wordpress-Server-
Configuration-Files/blob/master/nginx-caching/nginx.conf)

Basically they make nginx cache itself, which combined with this Wordpress
plugin:

<http://wordpress.org/extend/plugins/nginx-champuru/>

Makes things very minimal, no need for even a dedicated cache :)

------
fekberg
First time I reached the front page I had ~250 concurrent users and
increasing, then the server crashed.

After that I wrote a cronjob that runs a script which automatically increases
my virtual machines RAM. When I have little or no visits it uses 1GB RAM the
most it went up to was 9GB RAM (increasing with 1GB at a time).

It also sends me an e-mail when increasing/decreasing the RAM so I can ensure
that it doesn't crash.

I also use Wordpress with caching enabled and the above helps me keep the
costs down, don't want the server to run on 8GB RAM all the time nor 1GB RAM.

~~~
eli
Which provider is that? AWS? I feel like it's only a matter of time before
auto-scaling works well enough that it's just a standard feature.

~~~
fekberg
Glesys <http://www.glesys.com/> they have server centers in Sweden, New York
and Amsterdam (from what I can remember...)

------
Jabbles
200 hits per minute? Is that all it takes?

(I guess the answer is "No, only if you've misconfigured", but still...)

~~~
brandonsavage
A misconfigured site that serves all its own assets will perform terribly
because 200 hits per minute x 20 assets = 4,000 hits per minute. Wordpress has
tons of assets too.

~~~
eknkc
Never used WP, but aren't those static assets? The bottleneck would be
bandwidth then, should not be a huge load on the server. If it's compiling
stuff or fetching them from DB though, then that could cause some problems.

~~~
brandonsavage
It has to do with the number of requests plus the number of allowed processes
plus the requests each process was allowed to serve. At a limit of 300
requests per process, the processes fell over at about 3 seconds time. With
only 20 processes, dying every three seconds, the server was overwhelmed. So
it's really a combination of issues.

Looking at the records for MySQL data, there wasn't that much of an issue
there with MySQL connections. This is probably due to the fact that the page
cache was working correctly.

------
flexterra
My blog (elweb.co) has been on the HN home page a few times. I use Wordpress
with WP Super Cache, hosted on cheapest available shared plan from Dreamhost
with no problems.

------
robin_reala
We had a blog article on the front page of HN recently (
<https://news.ycombinator.com/item?id=5133906> ) and I can’t say we had much
trouble. Pushed the Heroku dynos up to two for an evening then dropped them
back down to one at the end of the day. The site assets are hosted on S3 and
we didn’t have to touch that. Ended up with about 16k hits.

------
tritchey
We recently launched a simple website monitoring app for the iPhone where we
ping sites from our servers, and send push notifications when we detect
something is wrong (a poor man's pagerduty or pingdom where you do everything
from your phone and can subscribe using IAP).

Probably the biggest lesson we've learned from collecting data on latency and
timeouts from our backend is that there are a LOT of very poor hosts out
there. Trying to strike a balance between intermittent failures, and actual
"Your site is down, you need to do something" is pretty difficult, unless you
are willing to simply wait long enough. We'll regularly see timeouts from
places like Tumblr and Github Pages--they never go down, but things can get
spotty. We can also tell when certain hosting providers are doing their weekly
backups (or something - just guessing) because things will just go to pot at
the same time every weekend for about an hour.

I can second the S3/CloudFront recommendation. Pretty rock solid (modulo
downtime in specific regions).

For anyone curious about the app: <http://vigil-app.com>

------
lousy_sysadmin
I've read so many times on HN that people find it hard to scale WordPress.
Really? Running WordPress on 128MB VPS and theoretically I can handle more
than 250 request per second without problem[1]. Granted that number is just
theoretical value but please, if you have gigabytes of RAM and your WordPress
site fail after some hundreds of request per second, you certainly have done
something terribly wrong. Especially for site with text and very few media
content.

Forget any other setup you've ever used, go with this for high performance
WordPress

1) Nginx [set so that request to static content to completely bypass PHP]

2) PHP-FPM

3) APC [Object & Database cache]

4) W3 Total Cache or WP Super Cache

5) Varnish

There many other high performance WordPress setup you could find out there but
in term of simplicity and manageability, this is the one I fancy most. The
bottleneck in this setup is RAM, not PHP, WordPress or MySQL read/write
operation as I've seen in many server setup.

Disclaimer : I run small operation of managed WordPress hosting called
KittySensei[2]

[1] <http://i.imgur.com/m0Dg9YR.png>

[2] <http://www.kittysensei.com>

------
challengee
Good tips for Wordpress users. Another great option we went for is hosting
your entire site/blog on S3/cloudfront. This can be done using tools like
Jekyll (<https://github.com/mojombo/jekyll>) or Middleman
(<http://middlemanapp.com/>).

~~~
brazzy
If your site is entirely static content and mostly text, you don't really need
a CDN. I've had a static site survive the HN frontpage without any problem on
cheap-ass shared hosting.

~~~
untog
If all you need is static hosting then S3 is great. The benefits of a CDN, and
it's incredibly cheap too.

~~~
danielhughes
If you want the benefits of a CDN I think you'd have to link S3 to CloudFront.
S3 alone does not provide CDN capabilities AFAIK.

------
danso
If you are self hosting Wordpress, the w3 Super Cache plugin is solid:
<http://wordpress.org/extend/plugins/w3-total-cache/>

My otherwise poorly maintained blog didn't have a problem with a simultaneous
HN/Slashdot hit, and I'm not even sure I'm using all the settings correctly.
But the basic page cache seems to do what it promises.

Edit: oops, I see the OP is using Super Cache as well but still flopped. Then
he is definitely doing someone wrong. My blog is hosted on the cheapest
hosting plan that Dreamhost allows.

------
dangoldin
My solution was to move my WP blog from being hosted locally to Github pages
using Jekyll. They'll do a much better job keeping it up than I will and more
importantly it's less for me to worry about.

------
Cherian
I wrote about my experience related to this in detail - Post mortem of a
failed HackerNews launch [1]. This blog post provides insights into the
caching, concurrency and stability issues I’ve faced.

[http://www.gigpeppers.com/post-mortem-of-a-failed-
hackernews...](http://www.gigpeppers.com/post-mortem-of-a-failed-hackernews-
launch/)

------
TallboyOne
For <http://pineapple.io> I had 250 simultaneous users at once, and I believe
16,000 over the course of the day.

Server cut through it like butter :) Just lots of caching, optimization,
sprites, etc. That traffic isn't even that bad, but I was happy my rails code
performed nicely.

------
kens
Tip: if you have an image-heavy blog, don't host the images somewhere that
charges extra for exceeding a network traffic limit. Getting an overage charge
email was how I discovered that my fractal article full of pictures was on HN.

As an aside, I found that being on Reddit can give much, much more traffic
than HN.

~~~
krapp
I'd also like to suggest, as a matter of politeness, don't just hotlink to
images on other people's sites either. Then you're just screwing someone else
over. I'm still getting requests for an image I posted to an old wordpress
blog years ago because random j. user liked it and reposted it to a forum.
It's a popular forum, judging from my logs.

------
jonascopenhagen
2.5 GB virtual server? You don't need that at all. I use shared hosting for my
blog, and yet I survived several hours as #1 on the HN front page. The trick?
Static HTML pages (made with a homemade PHP blog system, but I could just as
easily have used one of the many static site generators).

------
niggler
Is Wordpress.com itself capable of handling a slashdotting? If so, why not use
it?

------
arikrak
You can just use free services to host your wordpress blog without any
trouble. Set it up on AppFog or OpenShift, use cloudflare too, and use a cache
plugin if needed. I don't think any HN-effect could take that down.

------
speeder
I am preparing a new blog, that I do have the intention to have it featured on
HN sometimes.

For now what I will do is use a very lightweight Wordpress theme, and
cloudflare.

Is that enough?

~~~
narcissus
If you're just blogging, is there a reason you want to do it all yourself? I
just set up a Blogger account with a custom domain and let it be...

~~~
speeder
I used to have a wordpress.com blog, but after they charged to everything I
wanted to do, I got annoyed with them.

Also, I like to sometimes fiddle with stuff, and not let my "webmastering" get
too rusty.

It is good to sometimes ssh to something that you own and wreck it at your
will :)

To me, part of the fun of setting up a blog, is not only writing, but also
taking care of it. I don't know really why (I have no intention of doing that
for other people)

~~~
narcissus
That's fair. I have a similar thought process around other servers and sites
that I have, so I can appreciate the need for fiddling.

Funnily enough, I think the reason I went with Blogger (alongside not needing
to worry too much about traffic etc.) was specifically to take away my
complete ability to fiddle... otherwise I tend to do all that instead of
actually writing.

