

Extensive Benchmarks Of Amazon's EC2 Compute Cloud Performance - spahl
http://www.phoronix.com/scan.php?page=article&item=amazon_ec2_exhaustive&num=1

======
shizcakes
These benchmarks were done by someone who seems to have no idea about the ins-
and-outs of EC2. For example, there's a lot of IO bound tests... but they're
on instance storage. What about Instance storage vs EBS? What about network
performance over time?

All I can get from these is that Amazon isn't lying that their instance sizes
are different in terms of compute units and memory.

------
listic
It looks strange to me that High-CPU Instances didn't come as clear winners in
compilation tests. I thought of choosing only between c1.medium and c1.xlarge
for my CPU-intensive tasks. Looks like I should look closely into all types of
instances instead, except for micro and small.

According to the specs Amazon gives us, "High-Memory" instances have 3.25
Compute Units per virtual core, while "High-CPU" only 2.5. It would be
interesting to know what is going on: are their compilation tests unable to
properly utilize many cores, or the nodes assigned to "High-Memory" instances
tend to have more powerful hardware, or what?

------
jcborro
How about a comparison to a known standalone unvirtualized box?

------
ericb
I was hoping for more extensive and definitive. Bandwidth? Latency? Startup
time? A meaningful sample with variation from day to day and an idea of
variation across instances running at the same time?

For all the dollars being thrown at EC2, I'm surprised someone hasn't gone
nuts with benchmarking this for the authority, notoriety and seo link juice.

~~~
snewman
I've done some long-running benchmarks of EC2, mostly for kicks and personal
edification. For about 8 months now, I've had a job running on an EC2 "small"
instance, that performs a suite of microbencharks every 10 seconds and records
the results. Lots of raw data can be found here:

<http://amistrongeryet.com/dashboard.jsp>

This covers a variety of AWS operations, ranging from simple EC2 CPU
benchmarks to SimpleDB and RDS operations. I also include some measurements of
Google App Engine. I've put a lot of focus on measuring variation over time,
and measuring outliers (i.e. 99.9th percentile, not just mean). I'd like to
measure variation across instances, but haven't gotten around to it.

The web site linked above has no documentation at the moment, and is a bit
broken in other ways, but the data may be of interest. I have a little blog
(<http://amistrongeryet.blogspot.com/>) where I've discussed some of my
findings, e.g. [http://amistrongeryet.blogspot.com/2010/04/early-results-
on-...](http://amistrongeryet.blogspot.com/2010/04/early-results-on-gae-and-
aws.html), [http://amistrongeryet.blogspot.com/2010/04/beginnings-of-
rob...](http://amistrongeryet.blogspot.com/2010/04/beginnings-of-robust-data-
collector.html), and [http://amistrongeryet.blogspot.com/2010/04/cloud-data-
storag...](http://amistrongeryet.blogspot.com/2010/04/cloud-data-storage-many-
choices-all-bad.html).

If there's interest, I'll put a little effort into cleaning up and documenting
the data reporting site.

~~~
listic
Hi, I'm definitely interested in benchmarking the EC2 instances. Are you doing
some benchmarks that are CPU-intensive? You're measuring the one instance that
you've got to use as a server, aren't you? (I'm interested more in, "what
perfomance would I keep getting if I keep requesting instances") How can I
contact you?

~~~
snewman
You can reach me at e-mail address "public" on domain "snewman.net". I'd be
happy to share more of what I'm doing.

------
DrJosiah
<http://cloudharmony.com/> has at least as extensive benchmarks, and includes
other cloud providers.

------
samuel1604
I wonder how those compare to others IaaS providers like Gogrid or Rackspace

------
eclark
How about cc1.4xlarge we have been debating using them for the IO.

------
snissn
i'm much more interested in ec2's variation over time

