
GPUs Are Now Available for Google Compute Engine and Cloud Machine Learning - boulos
https://cloudplatform.googleblog.com/2017/02/GPUs-are-now-available-for-Google-Compute-Engine-and-Cloud-Machine-Learning.html
======
boulos
(Disclosure: I work on Google Cloud and contributed to this effort).

As I said back in November [1] for our initial announcement, the most exciting
thing (for me) is that we let you mix and match cores and GPUs. You can see
that spelled out in a nice table in the docs [2].

[1]
[https://news.ycombinator.com/item?id=12963902](https://news.ycombinator.com/item?id=12963902)

[2]
[https://cloud.google.com/compute/docs/gpus/](https://cloud.google.com/compute/docs/gpus/)

~~~
geoka9
Sorry for being off-topic: can you guys consider introducing something similar
to AWS's free tier? It doesn't have to have a lot free resources but it should
be lengthy (6+ months, ideally at least 1 year). I recently started a new
project, and while I'd prefer to use GC, I had to go with AWS because they
offer one year for free (and GC's 2 months is not nearly enough to validate an
idea).

~~~
vgt
That's valid feedback. GCP's free tier is more valuable and more flexible than
AWS, but only for 2 months instead of 12.

There are also perpetual free tiers for several GCP services - AppEngine,
BigQuery, Firebase.

No comments on your specific feedback, but please do join GCP NEXT March 8-10
for some exciting updates [0].

(Work on Google Cloud, NOT in marketing :))

[0] [https://cloudnext.withgoogle.com/](https://cloudnext.withgoogle.com/)

~~~
wsh91
Cloud Datastore has a perpetual free tier, too:
[https://cloud.google.com/datastore/](https://cloud.google.com/datastore/) :)

(I work on it.)

~~~
vgt
Thanks, I forgot.. there are probably others I'm missing :/

------
bsaul
A bit off topic, but i recently realized how much the cost per cpu is really
competitive in cloud offerings compared to reserved instances in regular
hosting providers, but the network costs are absolutely outrageous.

Cpu costs may be something 5 to 10 times more expensive in the cloud, but
network are close to 100 times. Any hosting provider will offer you a 250 mbps
unlimited network access for your macine, whereas consumming that much
bandwidth in google cloud for a month will cost you more than 1000$.

How come is the difference that big ?

~~~
mastazi
> the cost per cpu is really competitive in cloud offerings

> Cpu costs may be something 5 to 10 times more expensive in the cloud

So are CPUs more or less expensive in the cloud?

Also I assume by "cloud offerings" you mean AWS, Azure, Google Cloud and the
like, and by "regular hosting providers" you mean GoDaddy, BlueHost and the
like? Or perhaps you mean Paas vs Saas offerings?

~~~
hyperknot
I believe he means OVH (+Kimsufi, Soyoustart), Online.net, and Hetzner for
example.

If you are interested in what 1/100th of AWS price looks, have a look at
Online's server order page for example [1].

[1]
[https://console.online.net/en/order/server](https://console.online.net/en/order/server)

~~~
gorodetsky
Yes and no: yes you can get a Quad-Core 64GB RAM + SSD server for $55/month
(source:
[https://www.hetzner.de/de/hosting/produkte_rootserver/ex51ss...](https://www.hetzner.de/de/hosting/produkte_rootserver/ex51ssd)).
But there's one more spec that matters: networking. It takes completely
different amount of effort to provide 1Gbps connectivity versus 40Gbps per
server.

Most of the providers like Hetzner/OVH provide former, while GCE provides the
latter. I'm not saying it's bad, in fact for most of the people 1Gbps would be
more than enough. But it's not something that is fair to omit.

Disclaimer: I don't work for Google, just from my experience.

~~~
jorangreef
OVH offer 1Gbps, 10Gbps and 40Gbps plans: [https://www.ovh.com/us/dedicated-
servers/storage/](https://www.ovh.com/us/dedicated-servers/storage/)

I am sure Hetzner would be able to offer the same as an add-on if you
contacted them directly.

That being said, you would continue to pay orders of magnitude less for your
bandwidth then you would at Google or AWS. There's a bubble in cloud bandwidth
pricing, and I don't think it's value-related.

------
sheerun
FYI Google's K80 ($0.7, pay-per-minute) are cheaper than AWS's K80 ($0.9/h,
pay-per-hour), as well as Azure's K80 ($1.08/h, pay-per-hour).

~~~
minimaxir
Google's $0.70 rate is _in addition_ to the normal instance cost, so math is
more complicated if you don't have a machine already. (Alhough, a low-end high
memory machine is $0.126/hr, so the sum is still lower than Amazon)

~~~
boulos
Yes, but you can connect any number of GPUs to an n1-standard-1, so feel free
to add $.05/hr :).

~~~
gwern
Is the n1-standard-1 instance fast enough to not be the bottleneck for the
K80?

~~~
boulos
It really depends on what you're doing. If you are doing large transfers over
PCIe back and forth, not really. But lots of things work just fine.

The bigger challenge for large ML models is the memory you'd need to back it.
But with GCE you can happily do a custom machine with up to 6.5 GB per vCPU,
just so you can fit the output ;).

------
SomeStupidPoint
Slightly off topic, but I have two questions:

1\. Are GPUs covered by the free trial (ie, can that $300 be spent towards GPU
instances)?

2\. How is support for GCP?

I've been curious about trying GCP, but held off over GPU support (since AWS
was covering my needs and I do ML stuff mostly) and general support (since
Google doesn't have a great reputation for supporting products).

Also, perhaps an affiliated person can chime in with something about the
roadmap to stable GPU support. (It currently says there may be breaking
changes.)

~~~
boulos
1\. At this time, we don't grant _quota_ for GPUs for free trial customers
(hello various coin miners!). However, if you upgrade from your free trial,
you do keep your $300, and that's just money :).

2\. Unlike consumer-facing products, GCP is focused on business. We offer
_paid_ support plans [1] with high (measured) customer satisfaction. I know
several of the people in the support teams (and you see them here on HN as
well), and we're really trying to defeat the meme of "Google doesn't do
support".

As far as "stable GPU support", this is just confusing language surrounding
our usual "Beta" terms [2]. Once it becomes Generally Available, no changes
would be made. But moreover (for GCE anyway), we don't make API breaking
changes from Beta to GA (Beta to GA is "just" about stability in production).

[1] [https://cloud.google.com/support/](https://cloud.google.com/support/)

[2] [https://cloud.google.com/terms/launch-
stages](https://cloud.google.com/terms/launch-stages)

~~~
chipperyman573
>1\. At this time, we don't grant quota for GPUs for free trial customers
(hello various coin miners!). However, if you upgrade from your free trial,
you do keep your $300, and that's just money :).

What do I have to upgrade to? Isn't GCE all per-minute? Do I just have to pay
$0.05 for a standard instance and then use the $300 from my trial?

Also, is coin mining against any TOS? I'm not planning on doing it, I'm just
curious.

~~~
boulos
Sorry, upgrading to a _paid_ account (abuse risk is just too high for the free
trial, so we keep the quota limits low).

Coin mining is _not_ against the TOS. However, because it's usually
economically irrational, it's usually abuse. If you don't pay for your GPUs
(fake credit card) it's awfully economically rational though ;).

~~~
Buge
It wasn't always economically irrational.

[http://web.archive.org/web/20160704054747/http://www.complet...](http://web.archive.org/web/20160704054747/http://www.completefusion.com/profitable-
litecoin-mining-on-ec2)

------
Xcelerate
This is great! Just out of curiosity, how does the $0.70 / hour rate compare
to the per-hour electricity cost of running a K80 (at full load)?

~~~
ucaetano
[https://images.nvidia.com/content/pdf/kepler/Tesla-K80-Board...](https://images.nvidia.com/content/pdf/kepler/Tesla-K80-BoardSpec-07317-001-v05.pdf)

"The board is designed for a maximum input power consumption of 300 W"

So one hour at home:

Power used: 300 Wh

Price: $0.20/kWh

Total cost: $0.06/hour

Google cost: $0.70

You could say it's 10X more expensive, but you need to include all additional
costs, power is just a fraction of it. The GPU itself is $5k, which over -
let's say - 3 years would cost $5/day, or $0.20/hour, and if you're using it a
third of the time, just that amounts to $0.60/hour. Add everything else (you
might use less than 1/3 of the time) and it would be far more expensive than
using it in the cloud. As it is expected to be.

~~~
rafinha
Unless you are using Geforces, which are much cheaper than 5k$ and have near
performance (single-precision).

~~~
arnon
The only problem is GeForce 1080s (for example) burn out much quicker under
heavy load. They're not designed to sit inside an enterprise chassis

~~~
patrickg_zill
Do you mean that they fail? I have built a mining setup with multiple gpus in
the past and haven't lost a card. I am 100% sure that Google is smarter than
me at such setups.

------
minimaxir
Looking at the docs, it appears that the CUDA drivers have to be manually
installed on the host image, which takes time/money.

Is there a GCP Image with CUDA drivers pre-installed? Or is that not possible
with the hardware architecture?

~~~
arnon
Amazon Linux AMI for their GPU instances also does not include the CUDA
driver.

~~~
gigatexal
Just run a cuda container.

~~~
AlexCoventry
Can you point to an example of this? I have been installing nvidia-docker, and
it sounds like what you're proposing is substantially simpler.

~~~
mnbbrown
[https://hub.docker.com/r/nvidia/cuda/](https://hub.docker.com/r/nvidia/cuda/)

Edit: still requires nvidia-docker, or a hand crafted docker command that
replicates nvidia-docker.

~~~
gigatexal
Yeah my use case is nVidia docker but still a lot easier than configuring it
on your host system imo

~~~
arnon
Takes 10 minutes to install nv-docker correctly, and you're golden.

------
kozikow
I know you are working on better GPU support in kubernetes, but it would be
awesome if I could just grab my image that already run in nvidia-docker and
run it on GKE.

~~~
fest
You can do that on Nimbix/Jarvice. Submit a Docker image, start batch job and
get an e-mail once it's finished. 4 core 32GB RAM machine with K80 costs
~$1.06 there.

------
tomovo
Anyone checked if this is compatible with Blender GPU rendering? On my old Mac
machine GPU rendering doesn't work; I can still get a 16-core VPS for a few
hours when I'm in a hurry but this has the potential for more performance I
guess..?

~~~
boulos
I believe Cycles (the Blender renderer) only relies on CUDA [1], so it should
work. Depending on which AMD card you had on your Mac, it sounds like it might
not be supported by Cycles. Because the NVIDIA K80 doesn't do Display though,
you'd need to run say vnc or run Cycles from the command line.

[1]
[https://docs.blender.org/manual/en/dev/render/cycles/gpu_ren...](https://docs.blender.org/manual/en/dev/render/cycles/gpu_rendering.html)

~~~
tomovo
My iMac 2011 doesn't work with GPU Cycles but that's OK. And I've successfuly
boosted render times by renting the multicore VPS by the hour already
(commandline, specify different frame range on each machine to render).

~~~
tech_man7
You might want to check out Golem
([https://golem.network/](https://golem.network/)). They're working on very
cheap Blender rendering. They're on Slack:
[http://golemproject.org:3000/](http://golemproject.org:3000/)

------
porker
Hopefully soon someone will create a GCE image for WebPageTest. It's been a
pig to get up and running, but Amazon's per-hour billing is expensive for a
machine needed 10-15 minutes every hour.

~~~
theDoug
(I work at Google, I have my biases, but…)

You might find good fun with our Preemptible VMs, which I think will land
about ~$7/month:
[https://cloud.google.com/compute/docs/instances/preemptible](https://cloud.google.com/compute/docs/instances/preemptible)

~~~
luka-birsa
Preemptible VMs + Kubernetes = Awesome.

If you manage to build you app/infrastructure in away that can survive nodes
shutting down at random times (or if you don't care about restarts) then you
can reduce your infra costs for 80%.

We have a staging cluster running on preemptive instances and as soon as one
instance goes away we get a different one. Everything gets deployed
automatically. Regular internal users checking out various webpages don't even
notice.

We're looking into changing our 24/7 infra (which needs to be 24/7) to
something that can be run mostly on preemptive (with a couple of normal
instances for services that can't be randomly killed).

Super happy about our move to GCP and our K8S experience.

------
wiradikusuma
Sigh. When I first use Google cloud (lowercase c), it was only App Engine
(CMIIW). I woke up one day, now it has more than a dozen offerings, confusing
like AWS.

Does anyone have, specifically for AI/ML, a list of "If you want to do X, use
Y" for Google Cloud offerings? The official list
([https://cloud.google.com/products/machine-
learning/](https://cloud.google.com/products/machine-learning/)) doesn't help
much. Would appreciate if it's explained by layer (higher such as ready-to-use
Speech Recognition and lower where you possibly need to setup some infra
stuff).

EDIT: I'm looking something like this (explains AWS offering by layer) but for
Google -> [https://aws.amazon.com/blogs/ai/welcome-to-the-new-aws-ai-
bl...](https://aws.amazon.com/blogs/ai/welcome-to-the-new-aws-ai-blog/)

~~~
vgt
Try this:

[https://hackernoon.com/what-are-the-google-cloud-platform-
gc...](https://hackernoon.com/what-are-the-google-cloud-platform-gcp-
services-285f1988957a#.jyn2n8l1s)

Or for the really lazy:

[https://twitter.com/gregsramblings/status/832223967096090624...](https://twitter.com/gregsramblings/status/832223967096090624/photo/1)

------
danjoc
What happened to TPUs?

[http://austriantribune.com/informationen/165669-googles-
new-...](http://austriantribune.com/informationen/165669-googles-new-tpu-
accelerator-chip-speeds-machine-learning-systems)

~~~
frankchn
TPUs are for Tensorflow-based computation only, rather than being generally
CUDA compatible so they can run everything from Deep Learning to fluid
dynamics simulations. I believe they are also for Google internal applications
right now, so not generally available.

------
diamonis
O/T I guess...Does Google have plans for FPGA instances preferably with design
tools (for $.70 an hour :) !)?

------
tomarshubham24
AWS has spot instances which are roughly ~1/10th the price of the regular EC2
instances. Anything equivalent here?

~~~
wsh91
There are discounts if your instances can be pre-empted:
[https://cloud.google.com/preemptible-
vms/](https://cloud.google.com/preemptible-vms/). The pricing isn't variable
like EC2 spot instances, however.

~~~
candiodari
From the site:

> You cannot attach GPUs to preemptible instances. GPUs do not receive
> sustained use discounts.

So sorry, no.

~~~
wsh91
The question didn't even mention GPUs, and there's no reason that what you
mention can't change in the future. It was a pointer.

------
arnon
Are the cards physically in the same machine, or are they in a remote chassis
and connected through the network?

I think on P2, Amazon's cards are remote, and there is pretty significant
latency when using them for time-sensitive computing.

What kind of performance can we expect from GCP compared to AWS P2?

~~~
thesandlord
They are connected directly to the host with PCIe, so you should get "bare
metal" performance.

~~~
arnon
I've heard that before. I'll believe it when I see it :)

~~~
thesandlord
Anyone can try it today :)

I'd love to see your findings, I'm curious as well!

------
rahimnathwani
I know it's early, but has anyone tried using one of these instead of the AWS
p2.xlarge recommended for use in
[http://course.fast.ai/](http://course.fast.ai/) ?

------
nswanberg
Awesome.

Separately, can tensorflow models trained on Cloud ML be downloaded yet?

~~~
boulos
As Cloud ML is "just" hosted TensorFlow, once you train the model it stores a
.meta file in GCS for you. You can import this in TensorFlow [1] for serving
elsewhere if you so choose. Is that what you're after?

[1]
[https://www.tensorflow.org/versions/r0.11/how_tos/meta_graph...](https://www.tensorflow.org/versions/r0.11/how_tos/meta_graph/)

~~~
nswanberg
Thanks. That works in Cloud ML now? I'll try it.

The copy on [https://cloud.google.com/ml/](https://cloud.google.com/ml/) under
Portable Models says "In future phases, models trained using Cloud Machine
Learning can be downloaded for local execution" so I hadn't looked into it
further.

~~~
boulos
Huh... maybe I'm mistaken. Lemme ask the experts and get back to you.

[Edit: Okay, we've decided that the exported model it produces is what you'd
expect. We're going to update the landing page, once we can agree on what it
_should_ say.]

~~~
nswanberg
Thanks!

------
krona
When (if ever) will we see Pascal cards in the cloud?

~~~
boulos
> AMD FirePro and NVIDIA® Tesla® P100s are coming soon.

from cloud.google.com/gpu (our landing page). That's currently limited on
availability of hardware, testing, etc. but it really should be "soon". Note
though that P100s are _massive_ and expensive, so we don't intend to get rid
of K80s or anything once we have P100s.

------
seangrogg
Out of curiosity, are there strong reasons to leverage GPUs in standard web
application development from a general-purpose standpoint? Can I leverage this
to enhance a general-purpose server or database? If so, anywhere I can read
more?

Always interested in what kinds of things one can do when new offerings like
this are made.

~~~
yeukhon
Short answer: no. Long answer: if and only if you can really parallelize your
processing into very small calculation each doesn't take a lot of memory, and
you have a lot of time of writing special code using special framework.
Perhaps streaming encoding in real time could be one, but that's already sort
of beyond general web application.

GPU is powerful because a GPU usually has 100+ cores. Each core is weak and
inefficient, but power adds up when you have 100+ cores available.

[https://en.wikipedia.org/wiki/General-
purpose_computing_on_g...](https://en.wikipedia.org/wiki/General-
purpose_computing_on_graphics_processing_units)

~~~
arnon
GPUs have over 3000 cores nowadays
([http://images.nvidia.com/content/tesla/pdf/nvidia-
tesla-p100...](http://images.nvidia.com/content/tesla/pdf/nvidia-
tesla-p100-PCIe-datasheet.pdf))

------
ortekk
Out of curiosity, can you... run video games with it?

~~~
curuinor
... if you can deal with the 250-2000ms display lag sure

and you have to use a remote windowing dealie

~~~
wickedlogic
Is that number based on your experience using it this way with something like
RDP/TurboVNC, or anecdotal based on other cloud providerS?

~~~
curuinor
I tried with AWS GPU once and only once. Probably if you spend time and effort
you can get it down a fair bit

~~~
NegatioN
From my experience, network in GCE is way more stable and usable than it's AWS
counterpart. So maybe we can see another OnLive rise again ;)

(I'm joking, but there has to be a day when it's possible, right?)

------
8166284
Does anyone else find it kind of odd that Google, a company famous for having
no human support at all anywhere who shuts down any attempts at contacting a
human within its ranks for support on ANY of their products or services,
always seems to deploy a small army of senior tech people to answer questions
every time there is an article posted on HN. Strange double standards afoot,
some users matter, others not so much.

~~~
dragonwriter
> Does anyone else find it kind of odd that Google, a company famous for
> having no human support at all anywhere

While a number of people push that false meme repeatedly, everything I've
heard from people who've used it (or worked on it, but the latter comes with
obvious bias) is that the paid human support on GCP is good.

Heck, on the consumer side, I've gotten good (quality and speed) human support
on Google Express, too.

~~~
poooogles
GCP gold support is great as you actually get through to an SRE, silver (what
I use)... Not so great, often you'll submit a ticket with a ton of technical
detail only to have them come back with something asinine rather than
escalating it if it's past their understanding. The difference in price is
pretty striking though, so it's no surprise the different in quality of
service is different.

Honestly though I can't say I've ever really needed support on GCP, most of
the times I've raised tickets its been due to funny behaviour (slow spinning
disks in us-central1 was the last one).

