
Saving Hundreds of Hours with Google Compute Engine Per-Minute Billing - izzym
http://omerio.com/2016/03/16/saving-hundreds-of-hours-with-google-compute-engine-per-minute-billing/
======
obulpathi
Google Cloud wins big on simplicity and ease of use. That can save lot of
developer time. Here is the article that I am writing on "Ease of use
comparing Google vs AWS":
[https://docs.google.com/document/d/1B5XUiQUClxdoFpl2Md5MZOv6...](https://docs.google.com/document/d/1B5XUiQUClxdoFpl2Md5MZOv642StuOW0ULvfIkgOE1I/edit?usp=sharing)

Any feedback is welcome.

Citing from OP on ease of use with Google Cloud:

> Over the past couple of years, I’ve never struggled to grasp or understand
> any of the GCE offered services. The predefined machine types are also very
> clear, shared core, standard, high memory and high CPU, I know them all by
> heart now with their memory configurations and pricing to some extent. I
> clearly understand the difference between the IOPS for a standard disk and
> an SSD disk and how the chosen size impacts their performance. There is no
> information overload, disk pricing and other information is kept separate,
> it’s simple and easy to understand.

Now compare this with EC2 VMs, it’s overwhelming, current/previous, etc…
generation VMs. Disk information and network configurations all lumped
together with VM configurations, paragraphs and paragraphs of different price
configurations. For me, it was painful just trying to understand which VM type
is suitable for my needs. My first encounter with SSD configurations and
maximum provisioned IOPS for AWS RDS was one of pain. Instead of spending time
working on my project, I found myself spending valuable time trying to select
which IaaS offerings best fit my needs. Things like trying to figure out if
low to moderate or high network connectively is best for my needs!. No wonder
I still hear many say they find Cloud offerings confusing!, I think this is no
more with GCP.

~~~
gcb0
i think it's mostly historical baggage. as soon as gce gets as old as aws, the
offering variations they have to offer for historical contacts and what so
ever will be as confusing as aws today

~~~
williamstein
I've been using GCE a lot since 2012 (early beta tester) and the web interface
has only improved with time.

~~~
darkr
Same with AWS, I've been using that since 2008 or so. At that time, many of
the services, or functionality within certain services were either not usable
(as in couldn't be accessed) via the web interface, only via the APIs.

------
fpgaminer
Wow, I never realized that Google offers per-minute billing. We have a video
decoding, encoding workload, and it has been a major pain point handling it on
AWS. Decoding/Encoding 4K video requires quite a bit of beef, with a hefty
minimum memory requirement on the order of 3+ GB for h264 encoding. Less if
you use ultrafast preset, but then you pay for larger files and larger egress.

So, we can spin up a beefy EC2 (c4.8xlarge or larger) for each video, and get
them handled quickly, but we get charged for the full hour which is quite
expensive. All the machines we would target for spot instances have had very
rocky pricing lately, so that often doesn't save much. We could spin up less
beefy instances, trying to encode/decode in ~1 hour so as to maximize economy,
but then, obviously, each video takes much longer to process.

Lambda would be an okay solution, approximately double the price of equivalent
EC2 solutions, while providing two very big wins: granularity and 5 minute
encoding/decoding time regardless of video size/length. That second point is
pretty insane; we could process an entire 4K movie in 5 minutes by unleashing
a swarm of Lambdas on it. The problem is that Lambdas have a max memory of
1.5GB which isn't enough for 4K encoding (h264). Also they have limited disk
space, so it's challenging to get frames and video in-out (and no FUSE module
in the kernel :/). We're still experimenting, but even if we do get it to work
it will probably be with the ultrafast preset, which is non-ideal.

Elastic Transcoder doesn't handle decoding/encoding, only transcoding, so its
a no-go. We thought to use it in a larger pipeline, like have lambdas sew
frames up into a lossless codec, then hand that to Elastic Transcoder to be
compressed. But ET won't encode 444 formats, and it incurs the cost of both a
lambda swarm and ET, so that's not great.

Per-minute billing would be ideal. Now I have to consider whether it's worth
it to port everything over to Google. Oh how I wish these companies didn't use
ludicrous egress pricing to enforce vendor lock-in.

~~~
Eugr
What's even better, you can use preemptible instances in GCE which are a rough
equivalent to Spot instances in Amazon, but have fixed pricing (no bidding
nightmare). They can live up to 24 hours, but can be shut down earlier, so
your processing pipeline should be able to handle it. But with up to 70% lower
prices and the same per-minute billing it's a steal!

Keep in mind that GCE has 10 minute minimum, so if you run an instance for
less than 10 minutes, you'll be charged for 10 minutes, but after that billing
is per-minute.

~~~
wstrange
+1. Preemptible images are insanely cheap. If you can tolerate the possible
loss of the machine (video processing seems like it would fit) they are a
bargain

------
jedberg
I'm seeing a lot of folks here talking about how to-the-minute billing would
save them so much money on Amazon, but I think you're not thinking about the
problem correctly.

If you want to build for scalability, then it's best to separate the jobs from
the machines. Having one job per machine isn't granular enough. It's best to
have a farm of machines that can take jobs and process them, and then scale
the farm if the jobs aren't processed fast enough. (Incidentally this is what
Lambda does for you)

If you do that, then once you have enough jobs to fill a single machine, to-
the-minute billing doesn't really save you much money, if any, and you're then
on a path to much better future scalability.

~~~
chimerasaurus
I think it depends on the use case and service.

As a disclaimer, I work for Google on Cloud Dataproc - a managed Spark and
Hadoop service. So, I am passionate and focused on Spark clusters from 3
CPUs/3GB ram to 5k+ CPUs and TBs+ of RAM.

If you are running Spark (or Hadoop, Pig, Hive, Kafka, etc.) jobs than per-
minute billing can save you quite a bit. Unless you can seperate your jobs
over time (and balance them) to keep n clusters saturated, you're probably
paying for an idle Spark/Hadoop cluster to sit around. Moreover, in terms of
Cloud Dataproc (only) that's why you can scale clusters up/down, use
preemptibles, and custom machine types. You can custom shape your clusters
_and_ pay for exactly what you use (disclaimer - there is a 10 minute minimum
like most Google Cloud services.)

As a practical example, if you have a 100 node Spark cluster and only use 25
minutes of it, you can stand to save considerably in a given year. Yes, you
can possibly rebalance your work internally to saturate a cluster at the
optimal n minutes but at that point, you're paying to do the engineering work
for it. :)

~~~
Eugr
... and with some use cases you just can't saturate the cluster 24/7\. For
instance, if you have to run ad-hoc computational jobs.

Not every use case revolves around customer-facing websites or processing
streaming data...

~~~
chimerasaurus
+1

All the more reason that 15 minutes * 100 nodes * x days can add up very
quickly. :)

------
mark_l_watson
The author makes a good point.

Many years ago I was crunching (doing NLP analysis) on parts of the Twitter
firehose (or sometimes just the garden hose) on Amazon Elastic Map Reduce. EMR
was a fantastic service that made getting work done much easier. However, I
was always trying to game the system by having my runs end in just less than
one hour, which was a nuisance. Per minute billing would have made my life
easier.

------
ak217
Amazon should definitely be feeling the pressure from Google on this. Per-hour
billing has always been a pain to manage for offline/batch workloads.

If you're interested in leveraging advanced cost-saving techniques on AWS
compute and storage, we're currently building something in the cloud HPC space
that does exactly this. Nothing public yet, but email in my profile for more.
Our system automatically manages elastic fleets of instances and right-sizes
instances to their workloads (including spot and lambdas), and also makes it
easy to reduce TCO on storage. It's agnostic in terms of deployment (hosted
vs. bring-your-own-VPC).

------
vgt
When it comes to Big Data, per-minute billing + rapid startup is significant.
When you can have a 10,000-core Hadoop cluster in 45 seconds, you stop
thinking about it in terms of clusters and start thinking about it in terms of
jobs.

Before: Start cluster > submit many jobs to the cluster > manage resources

Now: Submit a job to Dataproc > Start a cluster for each job > shut down the
cluster

And, of course, BigQuery gives you the equivalent of per-second billing and
0-second scaling to thousands of cores:

[https://cloud.google.com/blog/big-
data/2016/02/understanding...](https://cloud.google.com/blog/big-
data/2016/02/understanding-bigquerys-rapid-scaling-and-simple-pricing)

~~~
chimerasaurus
In addition, you can also use preemptibles and custom VM machine types
(specify the exact ratio of CPU/RAM you want for master + workers) for your
clusters. This gives even more control for cost vs resources.

Disclaimer - I work at Google, on Cloud Dataproc (and we are passionate about
making the Spark/Hadoop ecosystem both super fast, but also extremely cost
effective) :)

------
scott_s
This is a great analysis, but I'm missing some basic background. Do all of the
other cloud providers charge per-hour? Do people typically design their
applications around this pricing model, or ignore it and eat the cost?

~~~
zbjornson
AWS rounds up to the nearest hour. Azure is by the minute. Not sure about the
other smaller providers.

------
gegtik
Wait till you try per-100ms billing on AWS Lambda

~~~
zbjornson
Except Lambda is quite expensive. Compared to equivalent EC2 t2 instances,
Lambda is 4.6x the cost of on-demand, or ~20x of spot instances. I really
don't understand why anyone would use it for a high-volume web app. For low-
volume sites you have the advantage of not paying for idle time, but the
inflection point is somewhere around 13 minutes per hour.

~~~
fndrplayer13
Lambda is great though at handling services that see unpredictable spikes of
traffic throughout the day where its difficult to provision quickly. Its also
really good at reducing machine-management issues where your software doesn't
utilize resources very well on a given machine. While its a bandaid on writing
well-designed scalable software, it does allow you in the interim to take
somewhat inflexible software and scale parts of it independently of physical
machines.

edit: To clarify, I mean if you're running software that does something on a
given set of physical machines and you cant get job throughput higher than
some number, n, on a given machine it can be prohibitively expensive to scale
machines ahead of time or during load. With Lambda we have the option to run a
ton of independent processes on theoretically independent machines to vastly
increase throughput while we slog through improving our design to improve per-
machine throughput on our legacy EC2 infrastructure. Lambda allows you to
isolate your scale problem to 1 job per container at a time instead of looking
at machines as capable of only `n` jobs at a time.

------
jononor
Heroku charges per second of compute, with no minimum. Using AMQP with
[https://github.com/the-grid/guv](https://github.com/the-grid/guv) our average
server stays up for just couple of minutes, directly in proportion to current
demand

------
sergioocon
I would love to see how you automate the full cycle, not only spinning new
instances

------
homageyellow
Awesome.

------
nickbauman
Can anyone spot a dateline on this article?

~~~
AnkhMorporkian
Well, the URL signifies that it was published today.

