

Ask HN: Preparing for the TechCrunch/HN Effect - tce

I'm a n00b when it comes to developing web apps, but the one I've been working on lately has gotten a fair amount of really positive feedback.<p>Again, I'm not a sysadmin or anything near the sort, so I'm quite unsure as to how to go about ensuring my app is robust. This has been a learning experience for me to develop a cool app from scratch, using RoR and Heroku. In the event that my app <i>does</i> go viral, I'd like my app to not crash and burn.<p>How could I ensure my app is robust and able to withstand thousands of potential users simultaneously? Even though getting featured on TC or other sites might be wishful thinking at this stage, I'd still like to prepare for the worst-case scenario in terms of handling unexpected traffic spikes etc.<p>Any pointers will be helpful!
======
mryan
) Stress-test your app by simulating loads of users performing realistic
actions on your site. See what breaks first. Optimise that. Start again.

) Offload your static media to a CDN. The cheapest and easiest way to do this
is to copy your CSS/images/javascript to Amazon's S3, then change your HTML to
reference the new location.

) Put a caching server in front of your current server (which then becomes the
"application server").

) Watch Jacob Kaplan-Moss' (one of the core Django devs) Deploying Django
workshop [1]. While it is obviously focused on Django, there are loads of
useful language-agnostic tips on deploying web/caching/DB servers.

[1] [http://ontwik.com/python/django-deployment-workshop-by-
jacob...](http://ontwik.com/python/django-deployment-workshop-by-jacob-kaplan-
moss/)

Edited to add: Forgot the obligatory "don't optimise too early" thing. The
time you spend making sure your app can sustain a million uniques per day
might be better spent elsewhere.

------
mooism2
Investigate caching, both at http level (and put a caching reverse proxy, e.g.
varnish, in front of your webapp) and at component levels.

Ask yourself which database queries are essential for the basic experience and
which are for enhancing the experience beyond that. It may be worth doing
without the second class of queries (or being more willing to use out of date
cached values) when your website is under heavy load.

Ensure your sshd runs at a higher priority than your webserver/database/etc,
so you have a chance of fixing things should your site actually crash and burn
in the end.

------
mskierkowski
[http://blog.phpfog.com/2011/03/16/scaling-php-up-out-and-
aro...](http://blog.phpfog.com/2011/03/16/scaling-php-up-out-and-around/)

I want to share this article (which I wrote at PHP Fog) about scalability (in
PHP but applies to all web apps) and the scenario you are asking about. I
agree with travisglines (and everyone else on the thread), that this scenario
you are talking about is in fact rare. It's one of those good problems to
have. ("I have so much money, how am I going to spend it all")

Most people who I talk to care about scalability, not because they need it
now, but rather because they want it as insurance. It sounds like you are in
the same boat.

My recommendation is to pick a hosting platform that allows you to build your
application the same way you normally would, but allows scalability if you do
need it.

Heroku already does this for you for RoR; Windows Azure does this for .NET
apps; GAE for Java/Python; and PHP Fog for PHP. Those platforms basically make
caching and scalability really easy.

Other than scalability and caching, you should build your application with
performance in mind; as having a fast site is important regardless of the
number of users. mryan & mooism2 referenced some good resources for doing
exactly that.

------
travisglines
I wouldn't assume that your app goes viral by default. In fact the chances of
this happening are very slim. The guys that made breakup notifier almost went
off the new page when they went viral.

What I would do is put it out there and be ready to take the steps you need
to, to keep it alive if it does go viral. Planning too far ahead for this is
probably a waste of your time as you most likely wont be able to see the weak
points before hand. Just move quickly if it does.

~~~
tce
I didn't say that I'm assuming my app will go viral by default. In fact,
there's almost a 100% chance that it will not, of course, and I'm very
cognizant of that fact. And I definitely do understand that planning too far
ahead would be a waste of time.

My issue is that I'm pretty clueless as to all this sysadmin stuff in the
first place. If not for my current app, I'd at least like to have an idea on
_what_ steps to take _should_ a future app go viral.

~~~
atgm
With this in mind, (price) scalable-CDNs are a good idea. Don't spend a lot of
money up-front when you're not sure you'll get the traffic to merit it. I'm
glad I didn't, because my site got all of five hits from HN... though,
granted, it's not tech.

~~~
tce
Did you have any recommendations (e.g. Cloudfront)?

My google-fu is pretty weak, it seems. I'm not able to find any examples of
how apps built on Ruby on Rails and deployed with Heroku were ensured to be
robust.

