

Ask HN: How do you handle sudden huge traffic spike like that of Obama IAMA? - prasanna83


======
tvdw
1\. Cache. Cache, cache, cache. Cache. Consider turning off most of these
caches for normal production work, then turn them on when you really need
them. This way if you normally run at 90% of your capabilities, you can turn
on caches and still be able to handle a massive traffic spike.

2\. Backup servers. Make sure you have a way to quickly spin up more machines.
If you're using EC2, just launch a few extra machines, it's really easy as
long as it's not the database being the bottleneck.

3\. Turn off anything unimportant. During the traffic spike of Obama, there
were undoubtedly some background processes that could be ran later. Just pause
those for a while and make them continue later. As long as it's a spike and
the background processes aren't that important, this works well.

4\. Turn off some front-end stuff. Reddit, for example, disabled commenting on
Obama's IAMA for a while, instead forcing a cached version of the homepage. It
really helps performance and doesn't really hurt the visitors too much. Sure,
they can't comment, but with 15000 comments it's not likely that your comment
gets read by Obama anyway.

On 1 and 2: always make sure you have room to handle spikes. Get backup
servers ready but don't turn those into real production servers. Always ensure
you have a buffer. Maybe embed some code that makes your site extra heavy,
then give it a switch to turn off that extra code in case of a big spike? If
you always run your site on full capacity hardware, you can't plan for spikes.

