

Exploiting Hardware Heterogeneity within the Same Instance Type of Amazon EC2 - EwanToo
https://www.usenix.org/conference/hotcloud12/exploiting-hardware-heterogeneity-within-same-instance-type-amazon-ec2

======
chrisacky
(tl;dw) This talk discusses how to get the most cost effective servers from
EC2 taking into account that EC2 instances are heterogeneous (that is to say
they don't run the same hardware so it is almost certain that they will
perform differently).

Each instance has three-four subgroups of different hardware types depending
on when the configuration was installed. (2006/07/10 etc).

Also, each zone has different configurations, especially as new zones are
being installed and newer hardware is being added to start with.

Variations of performance can approach 60% between the same instance types
(processor,memory,disk,network).

Cost savings can be achieved by seeking for better performing instances, check
it it performs well, if it does, keep it, if it does not, drop the machine and
request a new instance.

Instances using E5645 are 1.6 times as effecient as the E5507, however, if you
are requesting m1.large servers, you only have a 5% probability of getting a
E5645 server if you rented in 2011, whereas now you have a 42% probability.

You can also wiew the slides here:
[https://www.usenix.org/sites/default/files/conference/protec...](https://www.usenix.org/sites/default/files/conference/protected-
files/ou_hotcloud12_slides.pdf)

~~~
pella
from the slide:

 _"Saving money by seeking for better performing instances, simply using
“trial-and-failure” method

– Applying for instances randomly;

– Checking if performing well;

– If not, drop and apply for new ones."_

------
newhouseb
We got bit by this recently when we were running code on some m1.large
machines and incorrectly assumed that all m1.larges supported SSE4.2 (which
this paper doesn't mention). The only way to guarantee that you get a specific
processor type is to use cluster compute instance or play roulette while you
are booting instances, neither of which are ideal (which this talk presumably
says).

I'm not sure what the ideal way to resolve this outside of further subdividing
instance types or rebalancing CPUs so that more advanced CPUs are used at the
high end in a consistent manner or if users should be able to request a
specific CPU type. (but hopefully jeffbarr will read this and put it on a
todolist somewhere)

~~~
jeffbarr
> hopefully jeffbarr will read this

Done, from our development center in Iasi, Romania. I'm fairly sure that our
EC2 team is already aware of this, but I'll send the thread to them just to be
sure.

~~~
newhouseb
Awesome, thanks!

------
EwanToo
There's a fairly well hidden PDF link at the bottom which has the details

[https://www.usenix.org/system/files/conference/hotcloud12/ho...](https://www.usenix.org/system/files/conference/hotcloud12/hotcloud12-final40.pdf)

------
spindritf
I liked William Vambenepe's comment on that -- "Are you the dumb money in the
Cloud?" <http://stage.vambenepe.com/archives/2061>

------
EliteBob
A similar paper was recently presented at SOCC: <http://tinyurl.com/8hgvaq9>

This one focuses more on a model for how to take advantage of said
heterogeneity, and has a few examples where they actually implement the model
on EC2 and see some performance improvements.

------
sudhirj
How is this usable information, though? Do the authors recommend playing
roulette to figure out which of your instances are new? Or even if they're AZ
based, everyone has differently coded AZs.

~~~
EwanToo
Well if you were planning on having an instance run for a long time, and it's
a CPU bound process, then it would definitely be worth experimenting with a
roulette system.

Some people already do this for EBS disk I/O issues, you fire up an instance,
run some sample benchmarks, and kill the instance if it's a problematic one.

