

Ask HN: What are the benefits & pain points of using EC2 for daily development? - freerobby

I've been thinking about using EC2 for my main development environment so that I can get quick and easy access to heavy resources during those times that I need them. I'm aware of a few startups that do this as an internal standard (most notably Quora), but I'd like to learn more about the benefits and pain points before I invest the time.
======
aonic
I find EC2 to be very expensive for development environment needs. The micro
instance is a joke because of its slow IO and CPU availability, yet the small
instance costs more (compared to Linode) and gives you more RAM than you'd
need in a development server.

I find Linode 512MB hosts more than enough for development need, and the
automatic backups make me feel better about leaving unfinished code and
scripts on my server.

Also, if you gain a team member, you can easily clone your VPS to make a new
development environment for the team member.

With all that being said, I use and encourage all my employers heading to the
cloud to use AWS for production servers.

~~~
code
I completely agree here. There are limitations with the micro and even running
on a small server is insufficient in most cases with memory limitations. For
development purposes before launch, AWS is probably not ideal.

On the other hand, after launch, AWS is amazing to scale with. I use AWS in
conjunction with Rightscale which I highly recommend. Much easier to deploy.

~~~
freerobby
Thanks (and karma) to you both for your perspectives.

------
pw
What I like most about developing on EC2 is the ability to save an development
environment as an Amazon Machine Image (AMI), and then spin up instances of it
as needed. I can fine-tune the environment over time (just spin up a new
instance, make changes, and save a new AMI), as well as have different
environments for different types of development.

I think you could do something similar with Slicehost/Linode using disk images
and configuration profiles, but it doesn't seem they offer anything like
Amazon's machine images. Correct me if I'm wrong, though.

Of course, the flip-side of all this is that any individual instance of an AMI
is inherently ephemeral, and you need to treat it as such. Even the
reliability claims of EBS are fairly weak. [1] That means EC2 instance
storage, even EBS storage attached to an instance, isn't truly persistent, and
I haven't quite ironed out my system for dealing with this.

You'll probably want to run your own Git server [2] and commit code as it's
completed. That still leaves the possibility of data loss if your instance
dies after you've written code, but before you've made a commit. You could
just take periodic snapshots of the EBS volume where your in-progress code
resides, but whatever period you choose for the frequency (like every 30
minutes) is equal to the amount of work you could lose. And that sort of
constant backup isn't really what snapshots are meant for anyway. [3] I think
the ideal solution would be to run Dropbox to get continuous backups of the
files you're working on. That might also make recovery easier. Attaching an
orphaned EBS volume to another instance isn't hard, but it's certainly not as
easy has having Dropbox download the file you were just working on.

\--------------------

1\. <http://blog.rightscale.com/2008/08/20/amazon-ebs-explained/>

2\. <http://alestic.com/alestic-git/> and make sure you're taking frequent EBS
snapshots. The right way to do this is with ec2-consistent-snapshot and XFS
(see <http://alestic.com/2011/02/ec2-consistent-snapshot-sceencast>).

3\. I might be wrong about this. The fact that you should lock the file system
to ensure a consistent snapshot makes very frequent (say, every minute)
snapshots _seem_ like a bad idea. Maybe you could get around this by not
locking the file system for snapshot when dealing with plain files (and not a
database), as consistency should be less of issue. Maybe I just don't know
what I'm talking about ;-)

