Ssshhhhh... lets keep the mystery for the suckers that read scoble.
Therefore, you need to have some kind of distributed storage architecture, and automated recovery for said system.
For extra points, do the whole thing with no external server with a stable IP.
Should be doable. In principle, ARP over SQS.
And what's the alternative going down the Twitter road? I think not...
Always have a reasonable scalability plan preplanned, it doesn't have to be fully implemented but you should devote some thought cycles to the problem and have a road map for scaling.
Sure, it's important to have a scalability strategy planned, but that's nowhere near the same thing as wasting money on excess capacity before you need it. I suppose a few years ago you would have advised the HotOrNot guys that if they couldn't afford to maintain a standing account with Akamai before launching, they were in the wrong business too.
The entire advantage of Amazons services is that you don't need to pre-purchase too much capacity and you can scale relatively painlessly so long as you designed your server architecture for it.
I admire the HotOrNot guys for their "beg, burrow, steal" approach, I wish I had even a fraction of it. And that was exactly the right thing to do back then. But now the infrastructure has improved so much that you don't need to do it for servers.
Keep your energies for the other hundreds of battles you'll need to fight on your way & good luck
Also, multiply by 4 or 8 depending on what instance size you are using; so it's $72, $288, or $576 a month, per box.
This is their "small instance" setup.
1.7 GB of memory, 1 EC2 Compute Unit (1 virtual core with 1 EC2 Compute Unit), 160 GB of instance storage, 32-bit platform
The usual thought is to use dynamic dns to point to a server you have setup in an EC2 instance, so yeah a third party.
However, this isn't really an all or nothing proposition. You can certainly plug and play different components in to complement your existing setup and still save money / increase reliability.
If you have a solid setup, but are exceeding bandwidth limits you can move static files to S3.
Want better testing rigs, you can make EC2 instances for each and just turn them off when you're done.
EC2 and S3 are great tools, but you still have to do the homework to make them fit your needs.
I recently signed up w/Joyent and am happy with it so far, have also heard good things about SoftLayer, ServerBeach, RackSpace, Layered Tech...some of the other names you hear thrown around on here.
EC2 is probably a better option once you've been around for awhile and that economy of scale starts working in your favor.