

Scaling on EC2 - WebMynd's experiences (YC Winter '08) - amirnathoo
http://webmynd.wordpress.com/2008/06/23/scaling-on-ec2/

======
bprater
I notice that he mentions that his database is remote. I would really like to
try this so that I can establish scale with EC2 without moving my database.

Has anyone done this? Do you establish a tunnel between the two locations? How
does this kind of latency affect an application? Any other pros and cons?

~~~
jsjenkins168
If your schema is simple, you might want to check out SimpleDB. No need to
setup tunnels (queries are handled over HTTP), and latency is super low when
querying from an EC2 instance with both end points being inside AWS networks.
You are also not charged for traffic between EC2/SimpleDB.

------
thingsilearned
What do you guys do to manage your servers? Have you scaled to a point where
you automatically scale and downsize servers?

~~~
jamesbrady
Great question, I might well write another post about this because it's a
really interesting problem.

The software and services that we looked at to address this problem all left
us pretty lukewarm. We needed distributed configuration and resource
management, logging and alerting at a minimum.

To cut a long story short, we've written our own system, which draws on ideas
from things like rush, capistrano, munin and nagios and tailors it for our
needs.

At the moment, we have a system which gives us the information we need to make
decisions on commissioning new servers etc, and wakes us up in the middle of
the night when something goes wrong. I expect the capabilities to grow to
include auto-scaling, auto-healing and some other bits and pieces as I get the
time and our needs mature.

------
siculars
the author hits the nail on the head. isolate, isolate, isolate all the moving
parts of your system. as every talking head pundit has pointed out since they
were slapped in the face by amazon and their massive 800 pound cloud, the
cloud is where computing will be done in the future.

even today in my it group we use a citrix cloud to provide remote access
services for our users. the cloud concept isnt new but it is being used in new
and interesting ways. now... where is the hp/ibm/microsoft/sun cloud? where is
the emc storage cloud and the vmware compute cloud?

~~~
jamesbrady
My view is that you probably won't see the HP/IBM/... companies enter this
market as it's already commoditised (even though there's only really one
player): they prefer to focus on high value, high margin products and
services.

Sun is the only one in the space:
<http://www.sun.com/service/sungrid/index.jsp> \- they're currently task-
centric grid computing rather than machine-centric cloud computing.

In many ways, running applications on a grid rather than machines in a cloud
is a more pleasing idea, and fits better with the point I make in the post,
but pragmatism wins out: the edge cases and dynamic requirements mean running
virtual machines is often much more convenient.

~~~
whyleyc
Actually IBM has "Blue Cloud" (it even made Techcrunch ! see
[http://www.techcrunch.com/2007/11/15/ibms-blue-cloud-is-
web-...](http://www.techcrunch.com/2007/11/15/ibms-blue-cloud-is-web-computng-
by-another-name/))

But you're right it's a very high-end offering, so you're not gonna' see it
mentioned much in the bootstrapped startup world.

btw, when are you coming back to Hursley ? ;-)

~~~
jamesbrady
That's interesting, I stand corrected - it is interesting how the trend has
cycled between centralised and distributed compute resources over the decades,
and IBM's been on the ride the whole time.

btw, when are you leaving Hursley ? ;-)

------
tlrobinson
Great article, thanks guys.

------
ycseattle
It's off-topic newbie question but I haven't find the right place to ask: does
anybody know if HAProxy work with ASP.NET websites?

~~~
brk
It sure does (I've personally done it).

