
Google Shows How To Scale Apps From Zero To One Million RPS, For $10 - zafirk
http://www.forbes.com/sites/reuvencohen/2013/11/26/google-shows-how-to-scale-apps-from-zero-to-one-million-requests-per-second-for-10/
======
bhauer
I only skimmed the gist with the instructions for reproducing the test [1],
but in that skimming I could not figure out what the request and response were
in this simulation. Were these responses 100 bytes, 1k, 10k, 100k? I guess
small responses, but I'm not certain.

I know the goal of the test is to demonstrate how capacity can be scaled
upward in _very little time_ and with comparatively little effort. And for
that this demonstration is impressive!

That said, the requests per second number is not especially impressive given
that a modest single server can easily saturate gigabit Ethernet with 100-byte
responses (and it's even easier to do so with larger responses) [2]. I am left
wondering, does the $10 cost cover the cost of server instances _and_
bandwidth? If so, that is a very good deal. The bandwidth charges for
exceeding the capacity of a gigabit Ethernet connection (1M RPS with even the
most trivial requests requires more than 1 gigabit) would be substantial with
many hosting providers.

[1]
[https://gist.github.com/voellm/1370e09f7f394e3be724](https://gist.github.com/voellm/1370e09f7f394e3be724)

[2]
[http://www.techempower.com/benchmarks/#section=data-r7&hw=i7...](http://www.techempower.com/benchmarks/#section=data-r7&hw=i7&test=plaintext)

~~~
scoot
Couldn't agree more. As I (poorly) tried to point out in my earlier off-the-
cuff comment, requests-per-second is a meaningless measure without the context
of the size of those requests. The Forbes repost and the original google
marketing article are both vague on details, and appear to conflate requests-
per-second with throughput.

~~~
p3rfguy
Marketing Post.. Ouch :) One of the things I love about working at Google is
we in engineering are empowered to share our knowledge. For those wanting the
details check out the step-by-step instructions in the Gist
([https://gist.github.com/voellm/1370e09f7f394e3be724](https://gist.github.com/voellm/1370e09f7f394e3be724)).

The goal was to measure the speed of scaling and load balancing vs egress.
Bigger egress would not change the load balancing decisions.

Anthony F. Voellm Google Cloud Performance Engineering Manager @p3rfguy

~~~
simplemath
I wish there were more posts here on HN where the OP linked items such as
yours, but alas, clickbait rules apply.

I think I'm going to play with your script package over the long weekend in
our dev cluster - Thanks! And nice work!

------
danhoc01
$10 for a 10 minute test, okay. A month of 200 US-hosted n1-standard-1
machines would cost 200 * $.115 * 30 (days) * 24 (hours)... over $16,000. That
doesn't include load-balancing, bandwidth, or any additional charges there may
be. The price shouldn't be mentioned anywhere in the article.

Demand is elastic of course, and if you really want to scale in a cost-
effective manner you also need to do auto-scaling. As far as I can tell (I
have no direct experience with GAE), it's much easier on AWS. It would also be
interesting to see if you can scale your pool of webservers from 1 -> 200
faster on AWS or GCE.

The article does quote @cloudpundit, who hits on the true point of the
exercise: _Relevance of GCE LB load test: With AWS ELB, if you expect a big
spike load, must contact AWS support to have ELB pre-warmed to handle load._ I
would also guess that Amazon is working to improve ELB to behave similarly,
especially now that Google's product has less restrictions than theirs.

~~~
espeed
GCE instances are charged in 1 min increments
([https://cloud.google.com/pricing/compute-
engine](https://cloud.google.com/pricing/compute-engine)), not 1 hour
increments like AWS
([http://aws.amazon.com/ec2/pricing/](http://aws.amazon.com/ec2/pricing/)).

1 min increment billing enables you to spin up massive Web clusters to handle
spikes or Hadoop clusters so big that you can process the entire dataset it a
few minutes, while paying less than you would for a smaller AWS cluster that's
billed by the hour.

~~~
_delirium
It's definitely finer-grained, but there's still a 10-minute minimum per
instance: billing is only per-minute above 10 minutes. So if you run a
massively parallel 2-minute Hadoop job, you pay num_instances x 10min.
Admittedly that's still a win, since AWS would bill you for 6x as much
instance time. But it's not a 30x differential as you might guess from the
headline "hourly" vs. "per-minute" pricing.

~~~
simplemath
A fair point, but the scaling granularity advantage is a powerful cost win
nonetheless

------
yeukhon
I am pretty excited and yet shocked to see what Compute Engine has evolved
into. The original Compute Engine was about scaling (as cluster) in scientific
experiment and yet this news makes me think Compute Engine is becoming the new
App Engine but with full control of the VM (plus the amazing autoscaling
feature). I always like to control my own VM because I can do much more with a
VM than a sandbox (to me an App Engine is just a sandbox loaded with X
framework and X database). I have always wanted to work on Compute Engine :(

This is an interesting marketing stragery. People who wish to launch a VM can
choose CE and people who just want a sandbox quickly they can use App Engine?
Though I am really skeptical about the future of App Engine if the CE became
cheaper. I am sure if that happens, Google will do everything it can to
migrate things over to CE. This is probably many years down the road...

I still think CE is really good for computations.

~~~
enneff
App Engine is not just a place to run your code. It has a nice deployment
system with the ability to flip between versions instantly. It provides a few
great pieces of core infrastructure: the datastore, task queues, and memcache
(there are many others, but those are the essentials). The SDK gives you a
stand-alone development server for testing locally. If you want to build web
apps (or a backend for mobile apps), App Engine is a great way to go.

Compute Engine is for all the other stuff that you can't run on App Engine.

Incidentally, the datastore is now available as a stand-alone service, Google
Cloud Datastore:
[https://developers.google.com/datastore/](https://developers.google.com/datastore/)
This should benefit Compute Engine users.

~~~
hrjet
I like App-Engine and now that there are open-source, API compatible
alternatives, there is no lock-in to worry about either.

The biggest drawback for App Engine is lack of async support. The only ways to
scale are: multiple-threads (slow) or multiple instances (costly).

~~~
enneff
Or you can use Go and get high performance concurrency in a single-threaded
instance. The best of both worlds. ;-)

------
dudus
Original post [http://googlecloudplatform.blogspot.ca/2013/11/compute-
engin...](http://googlecloudplatform.blogspot.ca/2013/11/compute-engine-load-
balancing-hits-1-million-requests-per-second.html)

------
chuhnk
Having beta tested the Google Compute Engine while working at Google, I have
only good things to say about it. They've come along way in an extremely short
space of time and knowing what they have internally, this is just the
beginning. They'll far surpass AWS for performance, ease of use and cost.

~~~
threeseed
> They'll far surpass AWS for performance, ease of use and cost.

Most likely. But those aren't the only things developers care about when
looking for PAAS.

There is that whole "Platform" aspect. And AWS destroys Google in this
respect. It has far more offerings and more importantly it has a very large
ecosystem of companies who will be in the same data centre who you can
leverage e.g. Iron.io.

~~~
toomuchtodo
Don't forget support. I _know_ I can get decent AWS support if I pay Amazon.
Google? HAH!

~~~
bgoldy
We actually have come a long way in Support and now offer affordable paid
support options and some very high end packages for mission critical
operations. Check out
[https://cloud.google.com/support/packages](https://cloud.google.com/support/packages)
where we have the details.

-Brian Head of Marketing, Google Cloud Platform

~~~
tracker1
I know these are entirely different departments, and likely have little
correlation to reality.. but google has a worse track record than microsoft
for supporting some of their most popular products (iGoogle, Reader, free apps
account levels)... it's hard for me to trust services to be around in the long
run (this is probably more of an emotional bias here).

I know that there are api compatible systems out there, and one could roll
your own (so to speak), it will get very interesting over time. I'm also
curious what happens in the application space for docker.io based cloud
offerings in the next year.

------
scoot
The only reason I can see for this being this high on the front-page [18
points and only one comment(from a googler)] is the google employee block vote
(apparently tech-savvy isn't a prerequisite in the google upvote marketing
department):

 _1 million requests per second is 20 times greater than the throughput in
last year’s Eurovision Song Contest, which served 125 million users in Europe_
No, r/s != Gb/s.*

(Cue the down vote from the google up voters)

~~~
bgentry
Have you worked extensively with ELBs? They need pre-warming in advance of
significant traffic changes, they can't handle spikes well because they take
many minutes to respond to changes in traffic volume, and they use an ever-
changing set of IP addresses for your load balancer nodes.

The GCE load balancer has none of these problems, which makes it a huge
advantage over AWS and ELBs.

Disclaimer: I'm an engineer at Heroku. We manage dozens of ELBs for ourselves,
and thousands of them for our customers.

~~~
ancarda
>They need pre-warming in advance of significant traffic changes

Why?

~~~
bgentry
Because they consist of a set of EC2 instances, the vertical and horizontal
scale of which is determined automatically based on the average traffic
profile of each node. Once the traffic has increased enough to warrant a
scaling event, it takes minutes for new ELB nodes to come online and go into
DNS rotation before they can start serving traffic.

~~~
toomuchtodo
It's obnoxious that AWS hasn't developed an API call or web interface option
for the ELBs to pre-warm them yourself, vs having to contact AWS support to
get the pre-warming done "manually".

------
recuter
I keep checking to see whether or not I can run my own images yet but Google
Compute Engine so far supports less things than even Digital Ocean.

What if I don't want to run your version(s) of CentOS/Debian, guys? Hmm? (I
hope somebody from there reads this) :)

Still, its a very very interesting offering and something to keep an eye on. I
bet nobody here is using it yet because its still early days and the thread
devolves into such old classics like "you'll never be able to reach a human at
Google" and "they changed up their pricing with Google App Engine that one
time and my app was no longer free to run all of a sudden!". Sigh.

I personally think it would be great if you could spin up the same instances
on either cloud and load-balance/fail-over as needed/cheapest. Docker makes it
doubly exciting.

~~~
bgoldy
Stay tuned - Getting custom image support and custom kernels so you can run
things like Docker is just around the corner.

-Brian (@bgoldy)

------
bad_user
AWS has a really interesting feature in their auto-scaling groups, coupled
with their ELB (elastic load balancer). You can configure an auto-scaling
group with a variable number of instances that ads or removes instances
automatically, based on the ELB's measured latency or the number of requests
coming in or many other such metrics.

This works out great and results in cost savings, as it can survive spikes,
plus during the night the traffic is at most half or even less than the
traffic you get during the day. I had a setup that was handling over 30,000
reqs/sec and during the night it kept about 6 h1.medium instances active,
while during the day it could go upward to 20 instances, but was usually
stable at around 14 instances.

This article mentions ELB, but I don't understand - does Google's Cloud
Compute offer something similar? Can one vary the number of instances based on
the incoming traffic or other metrics?

~~~
krallin
Google Compute added an ELB equivalent (the "GCLB"), but it doesn't have
autoscaling built-in.

To get autoscaling, you can either roll your own — Google explains how here[0]
—, or use a multicloud autoscaling solution (disclaimer: I work for such a
provider).

[0]: [https://cloud.google.com/resources/articles/auto-scaling-
on-...](https://cloud.google.com/resources/articles/auto-scaling-on-the-
google-cloud-platform)

~~~
bad_user
Thanks for the link. I wonder how well it works. Did you try implementing it?

~~~
krallin
Well, that's what our software does too, so I didn't really have an incentive
to do so!

Now, judging from how busy the engineers are here, I'd say this is a bit
harder than it seems!

------
judk
RPS and $ are not interconvertible units, unless that 1M RPS is financed by
the interest on a $10 investment.

------
drakaal
This is the reason that our site.
[http://plexisearch.com](http://plexisearch.com) uses Google App Engine.

Google has allowed us to run "Code" rather than worry about infrastructure.
Because it is an "Engine" we don't have to configure much, and the autoscaling
is fantastic.

Google Edge Cache means we get Better than CDN performance on static assets.

There are limitation to AppEngine, but because we can also leverage virtual
servers we can create a hybrid environments that let us do things AppEngine
won't like install C Libraries, or run Windows (we aren't doing that, but we
could).

We have been very happy with Google App Engine and since we are running
millions of pages a day through our Natural Language Engine it has worked out
really well for us.

-Brandon Wirtz CTO Stremor.com

~~~
bgentry
AFAICT this article is talking strictly about Google Compute Engine and its
GCLB load balancer product, not about App Engine.

~~~
drakaal
How do you think App Engine works? AppEngine runs on the same hardware and the
same process described as a how to is nearly identical to what is obfuscated
and provided to AppEngine users via a GUI.

------
tzury
This is the original post
[http://googlecloudplatform.blogspot.ca/2013/11/compute-
engin...](http://googlecloudplatform.blogspot.ca/2013/11/compute-engine-load-
balancing-hits-1-million-requests-per-second.html)

------
wiremine
The gist describing how to reproduce the test yourself:
[https://gist.github.com/voellm/1370e09f7f394e3be724](https://gist.github.com/voellm/1370e09f7f394e3be724)

------
laz
The big thing they're crowing about is because you can spin something like
this up quickly on demand in the G cloud.

Most sites aren't remotely close to this artificial traffic pattern (1 packet
request, 1 packet response).

It's kinda cool from an L4 load balancing perspective that it's only one fault
tolerant IP address. In terms of L4 LB throughput though, a single box with
IPVS will happily do 1M pps.

------
nl
BTW, the Gartner analyst (Lydia Leong[1]) who's tweet was quoted is worth
following. She tells it like it is, with no bullshit (read some of her stuff
about the various OpenStack debacles for example).

[1] [https://twitter.com/cloudpundit](https://twitter.com/cloudpundit)

------
X4
The cake is a lie, they don't teach howto scale yourself. They teach howto use
THEIR infrastructure to scale. I mean it's still really useful to know and
all, but the headline makes the impression that Google teaches you howto scale
your own company's App from Zero to 1Million RPS.

------
tobyn
This is misleading, $10 is just for the duration of the test. Roughly $58k/mo
if $1.33/min.

------
jheriko
you do need context... a million things a second is not a great metric. these
are small numbers compared to a lot of things. e.g. how many pixels your
graphics card is pushing right now.

