
GCE vs. AWS in 2016: Why You Shouldn't Use Amazon - mikecarlton
https://thehftguy.com/2016/06/15/gce-vs-aws-in-2016-why-you-should-never-use-amazon/
======
doh
We moved back and forth between AWS and GCE (based on who gave us free
credits). Once we ran out, we chose GCE and never regretted it.

GCE has many quirks, for instance the inconsistency between API and the UI, it
misses the richness of the services offered by AWS but everything GCE does
offer is just faster, more stable and much more consistent.

One of the biggest problems with AWS is that once you outgrow the assigned
limits, it becomes a hell to get more resources from them. We're running on
average around 25k servers a day, majority of them preemptible (spot). AWS
requires that you request exact type and location for your instances. GCE only
asks for a region and then overall resources (e.g. number of CPUs).

Also the pricing is much less complicated. 1 core costs y, 32 cores cost 32*y.

~~~
geggam
At 25k servers / day one would think pulling this into real hardware and doing
capacity planning would be cost effective.

~~~
Turing_Machine
That's what I was thinking. Unless they're really bursty (so they only need
the 25k servers for an hour at a time or whatever), I would think that 25k
servers would be well beyond the number where it makes more economic sense to
build out your own data center.

~~~
user5994461
> Unless they're really bursty

Last I checked. AWS has hourly billing and Google Cloud has sub-hour billing.

If they are really bursty, they are in troubles with AWS pricing model.

> the number where it makes more economic sense to build out your own data
> center.

I would think that this number doesn't exist... unless you have already built
datacenters for yourselves and you have a major internal expertise in that.

~~~
geggam
I can tell you less than 1000 servers in AWS runs around 200-300k US / month

I am not sure what he is doing at 25k servers but you can see real world math
puts this in a serious money range. Data centers can be built for less.

I wonder if the down voters on the hardware comments actually run services at
scale.

~~~
doh
What kind of servers run at that price range? I've listed [0] some of the
reasons why are we hosting in cloud instead of our own datacenter.

[0]
[https://news.ycombinator.com/item?id=13260805](https://news.ycombinator.com/item?id=13260805)

~~~
geggam
Various servers, i2 for Cassandra eats most of it. Application at various
levels, multi region etc etc

We too are hockey sticking but not ready to go to metal .... yet.

Your reasons make sense. I would grow that team soon :)

~~~
doh
We have some large servers, but not too many of them. We're running 32 core
nodes that are holding our Postgres, but the majority is smaller.

~~~
geggam
We are a data company :)

~~~
doh
I would claim we're also
[https://twitter.com/synopsi/status/811755523091480576](https://twitter.com/synopsi/status/811755523091480576)

------
inopinatus
I have serious trust issues with Google. Their history of discontinuing
services, dismal support (even for a paid service), and neglect of bugs in
SDKs/APIs - all three of which I have experienced first-hand - has left a
long-term bitter taste. No doubt some individuals are fantastic, but the
organisation as a whole gives me an enduring impression of being
systematically arrogant and aloof.

AWS by contrast have demonstrated fanatically helpful support (even on
business level, the cheapest), fixing issues within days of reporting, and a
willingness to maintain obsolete/deprecated services (like SimpleDB) long
after I'm sure they'd wish everyone had migrated away.

~~~
lobster_johnson
It seems to me that Google is highly siloed internally. The GCP team is very
visible (they engage on HN and elsewhere), and GCP has excellent support. Very
different from other parts of Google.

I have problems with Google as a whole, but I have as much faith in their
cloud as I have in AWS or Digitial Ocean (we currently use all three).

The Kubernetes team is also highly engaged on Github (they also occasionally
show up on the official Slack channels). The Go community suffers from a
depressing abundance of hostility, intransigence and arrogance, and some of
Google's projects (e.g. Protobuf, the Go project itself) reflect this, but I
was delighted to find that the Kubernetes people are not like this at all.
It's a very friendly, quality-focused community. I think they have to be since
Kubernetes is still emerging tech that's craving adoption. Same goes for GCP
-- Google doesn't hide the fact that they're aggressively courting customers
to migrate.

~~~
scrollaway
Not to be too cynical but none of this is reassuring. It's making it sound
like the moment that GCP stops being the underdog they'll stop being as
helpful and courteous :)

------
hhandoko
One reason why I still can't use GCE in 2016. No PostgreSQL support for
CloudSQL.

I can find alternatives for other services, but I don't want to compromise on
the choice of relational database.

Note: I understand there are third-party providers for PostgreSQL, but I'd
rather have Google's.

~~~
lobster_johnson
Googlers on HN have commented before that they're working on it. No ETA,
though.

With AWS now offering both plain-old PostgreSQL and souped-up-PostgreSQL-on-
Aurora [1], whatever Google produces needs to be great in order to compete.
However, I fear they'll initially come out with something that's on par with
the current MySQL support in Cloud SQL, which is just a vanilla MySQL server
behind a UI/API. (For example, Cloud SQL's read-replica stuff is reportedly
just MySQL binlog replication.) Better than nothing, of course.

Cloud SQL also has some annoyances (such as not supporting private IPs and the
need for the Cloud SQL Proxy [2]) that I hope they're working on.

[1] [https://aws.amazon.com/blogs/aws/amazon-aurora-update-
postgr...](https://aws.amazon.com/blogs/aws/amazon-aurora-update-postgresql-
compatibility/)

[2] [https://cloud.google.com/sql/docs/sql-
proxy](https://cloud.google.com/sql/docs/sql-proxy)

~~~
hhandoko
Yeah, I heard this too.

TBH, I don't need Aurora-level performance (although it would be a nice option
in the future), I'm just happy to have a vendor-managed PostgreSQL instance
with granular price points.

C'mon Google! :)

------
gdgtfiend
Two points here really hit home with me about AWS (not in comparison to GCE
though since I've never tried it).

1) Reserved Instances: I think the pricing model for this has become very
outdated since the beginning of AWS, and it is definitely becoming cumbersome
(and therefore scary) to use.

2) ELB + Traffic Spikes: I have tried (unsuccessfully) to pre-scale an ELB to
prepare it for the traffic it was about to receive. I tried to pre-scale for
this project 3 different times, in coordination with support and without them.
I could not do it. Very frustrating.

I think these are all signs of extreme growth, and a strange organization of
engineering units inside of AWS. However, as OP descried.. we are much to
heavily invested in AWS to consider an infrastructure shift at this point

~~~
tejasmanohar
We also manually sync up with AWS support to "warm up" our ELBs... I guess
they don't expose an option to avoid abuse, but it really does seem like an
implementation caveat.

------
abvdasker
God I hate any piece of writing that doesn't define its acronyms at least
once. Google Compute Engine isn't popular enough that people should be
expected to know immediately what it means.

~~~
imafish
Also, GCE is a service while AWS is a suite of services. The correct
comparison should be Google Cloud Platform (GCP) vs AWS. Guess it's
nitpicking.. but still.

~~~
user5994461
I don't think this is nitpicking. This is probably the single biggest issue
with Google right now: Don't know how to call their services.

GCE - GCP - Google Cloud - Google Compute Engine - Google Cloud Platform ???

I can see on that blog's analytics that people are looking for various terms
which result in different articles and ranking.

This needs to be unified by Google. Personally, I think I'm gonna call
everything "Google Cloud".

~~~
manigandham
It is unified, they have some of the simplest naming. GCP is Google Cloud
Platform which is the entire ecosystem. GCE is Google Compute Engine which is
the VM/IaaS offering within.

You can look at the official products page for all the names and descriptions:
[https://cloud.google.com/products/](https://cloud.google.com/products/)

------
ungzd
> AWS Premium Support is mandatory

Is Google Cloud support even acceptable? Google is known for poor or no
support for most services.

~~~
Veratyr
Google's known for poor or no support on their __free __services but their
paid services often have decent support. Personally, I 've found GCP, GSuite,
Project Fi, Google Store and Pixel support to all be pretty great but haven't
found any support at all for Gmail, YouTube etc.

~~~
GordonS
Hah, Adwords must be an exception then - support is total crap. You're lucky
if you get a canned response that doesn't begin to answer your question..

~~~
rahrahrah
Hmm sounds like you might have to increase your bids.

~~~
GordonS
We already pay around $5 per click - I think that's more than enough! However,
it's true our business is tiny compared to that of large corporations - but I
don't think that excuses the _abysmal_ 'support' I and others have received
from Adwords.

~~~
rahrahrah
It's a joke... you said that you always get a canned response, didn't you?
That's the canned response I always get from them. Doesn't matter what the
issue is, raising bids is always the solution.

------
aluminussoma
AWS premium support is the best support experience I have encountered so far
and should absolutely factor into choosing a cloud provider. Reading frequent
stories of Google support nightmares across all their services makes me think
twice about using GCE. GCE must find a way to counter this.

Also, the AWS premium support fee is negotiable for some customers from what I
have heard. They don't like to negotiate down, though!

~~~
user5994461
Strange. I'd qualify them as the most useless support I've ever encountered.

In the year 2016, among maybe a hundred tickets, there was only ONCE where
they could change something (an ELB issue).

And well, I'm not sure whether the fix was related to their changes or if it
was just an intermittent error that happened once. Thus their implications in
the only time something happened has yet to be proven.

------
nodesocket
My biggest complaint about AWS is still EBS and having to guess about for the
right provisioned IOPS. Throw in confusing extras like EBS optimized
instances, enhanced networking. GCE just abstracts away all these details.

------
jasonlotito
This was referenced in the previous thread, but here is what amounts to a
stl;dr:

> "Unfortunately, our infrastructure on AWS is working " > "I learned recently
> that we are a profitable company, more so than I thought. Looking at the top
> 10 companies by revenue per employee, we’d be in the top 10."

~~~
dastbe
I'm a bit interested in what their company's response would be to this
article.

~~~
inopinatus
The author(s) have a high proportion of strongly pro-GCP articles; so many,
and so emphatically worded, that I began to distrust the source.

------
tetraodonpuffer
previous discussion at

[https://news.ycombinator.com/item?id=12038298](https://news.ycombinator.com/item?id=12038298)

------
AlexCoventry
Looking forward to the day Google Cloud provides on-demand GPU-backed
machines. Currently AWS is the only game in town for that, as far as I know.

~~~
zbjornson
Coming early 2017:
[https://cloud.google.com/gpu/](https://cloud.google.com/gpu/)

~~~
user5994461
You should add that Google will allow to attach up to 8 GPUs to any type of
instance.

That will put Google Cloud seriously ahead of the competition in terms of GPU
computations.

Long term: Google has a better trajectory. IMO.

Note that AWS announced something similar at reinvent 2016 but it ain't coming
any time soon. Not sure about the status, is it even real anymore?

------
patwolf
What reason is AWS premium support mandatory? I ask because I'm currently
building out SaaS offering on AWS and haven't yet hit any issues requiring
support. Can I expect to start seeing issues as traffic scales up to a certain
level?

~~~
user5994461
Author of the article here.

It's not. It's utterly useless. After an entire year, where the support has
never been of any help, my last action for the year 2016 was to call a meeting
with everyone, subject line "we should cancel our support subscription with
AWS".

It's clear that we (especially me) are way more qualified in all AWS
offerings, from basics to special quirks, than they are. And they can't do
anything that we can't do ourselves.

I think the support is useful when you first start out, they can answer a lot
of general questions. You should subscribe for support the first year and see
how it goes.

Note that some AWS managed services (e.g. RDS) can only be debugged by the
support so you might be forced into support if you use these services. We
don't.

------
tscs37
I've used AWS for a bit over 9 months now and it's quite terrible to be
honest.

I don't need it for anything professional and it's quite terrible for just
some amateur hosting plus the immense fees if you somehow manage to get decent
traffic together.

Once my reserved instances run out I'll probably either check out GCE or DO,
either seems to be a better option, though GCE seems to be more expensive.

Anyways, the console in AWS is a mess and I'm quite sure that I leaked my
entire IAM settings to the internet because some switch somewhere isn't set
right.

Since everything recommends to setup IAM users you'll have to setup the
permissions, a procedure which I enjoyed about as much as getting my
fingernails slowly removed by a glowing red iron.

Calculating any sort of sustained cost is a pain in the backplane if the total
doesn't exceed three digits a month.

And lastly the login process is probably the biggest pain I've encountered
across many many providers. There are atleast 4 login forms I've discovered, 2
of which I have to use and one of those always asks for a captcha with such
low quality that a brain-damaged AI running on my calculator could figure it
out, not mentioning never knowing if the 2FA setup was correct or maybe
probably blew up somewhere because giving some feedback from the UI is plain
impossible.

TL;DR Don't use AWS, anything else is better.

~~~
swozey
I mean, you're basically setting up a datacenter if you aren't using
Beanstalk.

I see complaints all the time about how complicated the networking is, IAM,
etc is. But it's far more simple dealing with VPCs then having to buy and hop
onto a bunch of F5s/Brocades etc that tend to require their own network
engineers on staff.

The issue always seems to be that someone tries to move a company onto AWS but
they lack the experience in actually running infrastructure to that degree. If
you know Change Management you can figure out Cloud Formation templates, if
you don't you're likely completely lost and rolling out instances by hand. If
you don't know any network engineering you're likely going to have issues with
load balancers and VPCs.

You can really see the experience difference in people when you work on
multiple AWS infrastructures. And that's the huge benefit to it. You can
practically roll out whatever sort of infrastructure your business requires.

If you're experienced enough you can basically do every single thing in
AWS/GCP without having to hire ancillary staff (network engs, etc).

I personally use Cloud Formation and create EVERYTHING by it. Load balancers,
VPCs, instances, kubernetes, etc. But yeah, if you haven't touched CF before
it can be like walking into a spider web of confusion - but believe me, I'd
rather do that over again than take over someones hardware that has no APIs,
no dashboards, no centralization. I spent years in datacenters where
everything was done manually. I wouldn't ever want to go back now that I've
learned to treat infrastructure as code.

But hey, if you want to run Vsphere servers, manually configure instances and
databases and not do any automation via AWS you can do that as well.

That's why it's incredible.

~~~
tscs37
I totally agree, AWS is great if you are basically screwing in the racks
yourself anyway (metaphorically speaking).

But for running a few hobbyist/amateur servers on it, it's absolutely
horrifyingly complicated, especially when I'm not really a networking
engineer, I like tinkering in the backend but not to that degree.

~~~
swozey
You could look into Beanstalk, it depends on what the code your app is written
in, though. But yeah, I really wouldn't suggest AWS for something simple that
DO or Linode could easily handle.

GCP is also far more simplified than AWS if you haven't tried it. The way they
do server auth via Google accounts (using gcloud) is pretty awesome and
simple. You can make project-level SSH keys that are automatically placed onto
every server and you can block specific servers from receiving them all
through the UI.

I do greatly miss AWS's breadth of service offerings, though.

------
bdcravens
Evaluated GCP, and 2 main issues made it hard to consider moving:

1) quickly bumped into project limits just doing some tests, and the fact that
you have to wait until billing cycle to reset the counter was quite jarring (I
presume there's a way to increase)

2) Better tooling for S3 than Google Cloud Storage - non-technical members of
our team need to work with files, and there's many nice third-party tools for
s3.

~~~
Veratyr
1) AWS has similar limits:
[https://aws.amazon.com/ec2/faqs/#How_many_instances_can_I_ru...](https://aws.amazon.com/ec2/faqs/#How_many_instances_can_I_run_in_Amazon_EC2)
and it's pretty simple to fill in the form and request more.

2) GCS has an S3 compatible API so you can use your S3 tooling:
[https://cloud.google.com/storage/docs/interoperability](https://cloud.google.com/storage/docs/interoperability)

~~~
bdcravens
AWS's limits are in real-time however. I don't mind the limits, I just don't
like the fact that I can't run a quick test without it impacting me for the
billing cycle.

I'll check out the GCS XML compatibility with the current tooling we use; it
looks promising.

~~~
vgt
One more comment on storage.

Google Cloud storage has the same API for all its tiers. So to go from Multi-
Region to Nearline, you just change the bucket designation. One API, one
service, one interface and set of tools.

(work on Google Cloud)

------
bhouston
I really like OVH's SoYouStart in terms of its pricing for CPU/memory
intensive computations, their prices just destroy AWS/GCE:
[https://www.soyoustart.com/ca/en/essential-
servers/](https://www.soyoustart.com/ca/en/essential-servers/)

~~~
Veratyr
In terms of raw machine cost it generally goes owning a DC < owning machines +
leasing racks < leasing dedicated < VPS < cloud.

But each level down you go is another level of expertise you have to employ
someone for and a loss of flexibility.

The strength of the cloud has never been cost, it's always been flexibility,
the ability to scale up 1000 servers in 5 minutes without any preparation or
management.

~~~
devoply
Yea but those 1000 server are going to do the job of 100 dedicated servers.
You still have to provision them, and there aren't too many use cases (there
are some) where you have to all the sudden jump 10x of demand. The reason
cloud soars is because in developed economies the cost of human resources is
astronomically high. The savings are from having fewer people deal with stuff.
Even 100k/year bill is okay if that means that 1 person has to deal with it
rather than 2 or 3. So if you have money then cloud makes all the sense in the
world, if you don't have money and have a bit of free time dedicated servers
make a lot of sense until you have money.

------
nhumrich
Because of speed of light/latency issues, I won't move to GCP until they have
an RDS-postgres equivalent

~~~
lobster_johnson
According to comments from googlers elsewhere on HN, they're working on it.
It's pretty much the number one thing people are asking for.

------
blam
Having used AWS in production for about 2 years, I can say that scaling in the
cloud is hard to get right. Spikes of traffic and load are hard to scale with,
especially if you're scaling down to just the right amount of instances to
manage the current load. We try to keep 50% capacity free for such spikes.
Speaking with AWS support, we can fix issues with limits and scaling elbs, in
hours not weeks, so I haven't had the same experience as you in that respect.
One thing I would suggest, is that you have 2 AWS accounts. One for dev and
one for prod. This way you should be able to see the limits that you're
hitting before you get to production, and can raise tickets to get them
ammended.

------
themihai
GCE has its own issues. The raw price is different than the real/total cost.
In a better world you shouldn't have to choose between just a few cloud
peoviders.

~~~
aluminussoma
How have you observed the total cost being different from the raw price? In
what ways were you surprised?

~~~
themihai
If you are after the raw price you should sign up for bare metal and
install/manage your own tools. When you sign up for a service such AWS you pay
mostly for the tools/ecosystem and management. I think AWS is still the
undisputed leader in that regard, closely followed by Azure. GCP is something
I would not recommend due some personal experiences namely the appengine
platform and early cloud sdk releases.

------
ilaksh
You should use AWS if you _really_ need something like an infinitely scalable
database such as DynamoDB or some other uber-scaling AWS service that you
can't replace with some open-source software on a VPS provider. 95-100% of
startups don't actually need an infinitely scalable database or whatever. So
they shouldn't subject themselves to Amazon's horrible pricing and throttling.

------
rmykhajliw
GCE - piece of shit in case of support and solving customers issues. They
didn't manage to FIX billing issue for 8!!! months. Support is useless. Missed
access rules boundaries - everything you have to share with everyone. It's
nightmare for companies.

------
cpncrunch
The cost comparison link is broken.

~~~
user5994461
fixed

------
holografix
I'd be interested to know why the OP never considered Heroku

~~~
lobster_johnson
Heroku is hosted on AWS, and so you inherit the performance, but lose the
ability to control it (no way to select local SSDs or enable IOPS, as far as I
know, for example), while at the same time paying a large premium for the
convenience of using their platform.

~~~
holografix
Thanks lobster_johnson, yeah I know Heroku is ephemeral so data is stored in a
DB or somewhere else, often S3.

Just wondering what kind of work he's doing that took him from AWS straight to
GCE without stopping to think about Heroku.

Maybe you've hit the nail in the head, he's writing to disk a lot for some
reason?

~~~
lobster_johnson
A lot of the article's focus was on network performance, local SSD
availability and I/O performance. I think it's safe to say his use case
involves a lot of I/O. If "thehftguy" means HFT == high frequency trading,
then that's not surprising.

