

Google Kubernetes Engine is introducing a cluster management fee on June 6 - agoell
https://cloud.google.com/kubernetes-engine/pricing
EDIT: Link on Google&#x27;s Official Documentation here <a href="https:&#x2F;&#x2F;cloud.google.com&#x2F;kubernetes-engine&#x2F;pricing" rel="nofollow">https:&#x2F;&#x2F;cloud.google.com&#x2F;kubernetes-engine&#x2F;pricing</a><p>Just received an email from Google Cloud -<p>&quot;On June 6, 2020, your Google Kubernetes Engine (GKE) clusters will start accruing a management fee, with an exemption for all Anthos GKE clusters and one free zonal cluster.<p>Starting June 6, 2020, your GKE clusters will accrue a management fee of $0.10 per cluster per hour, irrespective of cluster size and topology.<p>We’re making some changes to the way we offer Google Kubernetes Engine (GKE). Starting June 6, 2020, your GKE clusters will accrue a management fee of $0.10 per cluster per hour, irrespective of cluster size and topology. We’re also introducing a Service Level Agreement (SLA) that’s financially backed with a guaranteed availability of 99.95% for regional clusters and 99.5% for zonal clusters running a version of GKE available through the Stable release channel. Below, you’ll find additional details about the new SLA and information to help you reduce your costs.&quot;
======
agoell
Just received an email from Google Cloud -

"On June 6, 2020, your Google Kubernetes Engine (GKE) clusters will start
accruing a management fee, with an exemption for all Anthos GKE clusters and
one free zonal cluster.

Starting June 6, 2020, your GKE clusters will accrue a management fee of $0.10
per cluster per hour, irrespective of cluster size and topology.

We’re making some changes to the way we offer Google Kubernetes Engine (GKE).
Starting June 6, 2020, your GKE clusters will accrue a management fee of $0.10
per cluster per hour, irrespective of cluster size and topology. We’re also
introducing a Service Level Agreement (SLA) that’s financially backed with a
guaranteed availability of 99.95% for regional clusters and 99.5% for zonal
clusters running a version of GKE available through the Stable release
channel. Below, you’ll find additional details about the new SLA and
information to help you reduce your costs."

------
rolleiflex
This is awful - I don’t think GCP is fully aware of their position in the
market as the second, inferior choice. I took a bet on the underdog by using
GCP and they bit me back in return. Especially considering their ‘default’
kubernetes config automatically sets you up with three(!) control planes in
replication, that’s, as far as I understand, $~300 added to our monthly bill,
for nothing.

Oh, and per their docs, three-control-planes decision is _not_ reversible - I
cannot in fact shut two of those down without shutting down my production
cluster and starting a new one. [https://cloud.google.com/kubernetes-
engine/docs/concepts/typ...](https://cloud.google.com/kubernetes-
engine/docs/concepts/types-of-clusters)

Awful. Just so awful.

Edit: To answer some questions below - we have a single-tenant model where we
run an instance of our async discussion tool
([https://aether.app](https://aether.app)) per customer for better isolation,
that’s why we had bought into Kubernetes / GCP. Since we have our own
hypervisor inside the clusters, it makes me wonder whether we can just deploy
multiple hypervisors into the same cluster, or remove the Kubernetes
dependency and run this on a Docker runtime in a more classical environment.

~~~
sethvargo
Thank you for the feedback. The management fee is per cluster. You are not
billed for replicated control planes. You can use the pricing calculator at
[https://cloud.google.com/products/calculator#tab=container](https://cloud.google.com/products/calculator#tab=container)
to model pricing, but it should work out to $73/mo regardless of nodes or
cluster size (again, because it's charged per-cluster).

There's also one completely free zonal cluster for hobbyist projects.

~~~
rolleiflex
Seth — I appreciate you being here to take feedback, and for the clarification
as well. The very surprising email I’ve received this morning is very hazy on
the details, and the docs linked from the email are not updated yet.

The main issue is that _not charging for the control plane_ and _charging for
the control plane_ leads to two very different Kubernetes architectures, and
as per your docs, those decisions made at the start are very much set in
stone. You cannot change your cluster from a regional cluster to a single zone
cluster for example. So you have customers who built their stacks taking into
account your free control plane, and you’re turning the screws in by adding a
cost for it — but they cannot change the type of their cluster to optimise
their spend, since, per your docs, those decisions are set in stone. That’s
entrapment.

You should keep existing clusters in the pricing model they’ve been built in,
and apply this change for clusters created after today.

That said, many of us made a bet on GCP. For us in particular, we made a bet
to the point that our SQL servers are on AWS, but we still switched to GCP for
‘better’ Kubernetes and for _not nickel and diming_ , since AWS had a charge
that looked like it was designed to convey that they’d much rather have you
use their own stuff than Kubernetes. It is a relatively trivial amount, but it
makes a world of difference in how it feels and you guys know more than anyone
how much of these _GCP vs AWS_ decisions are made based not on data sheets but
for the ‘general feel’ for the lack of a better word.

AWS’ message is that they’re the staid, sort of old fashioned, but reliable
business partner. GCP’s message, as of this morning, is _stop using GCP._

~~~
sethvargo
Thank you <3\. I apologize the email was hazy on details. I can't un-send it,
but I'll work with the product teams to make sure they are crystal clear in
the future. I'm interested to learn more about what you mean about outdated
docs? The documentation I'm seeing appears to have been updated. Can you drop
me a screenshot, maybe on Twitter (same username, DMs are open).

These changes won't take effect until June - customers won't start getting
billed immediately. I'm sorry that you feel trapped, that's not our intention.

> You should keep existing clusters in the pricing model they’ve been built
> in, and apply this change for clusters created after today.

This is great feedback, but clusters should be treated like cattle, not pets.
I'd love to learn more about why your clusters must be so static.

~~~
yongjik
> clusters should be treated like cattle, not pets.

Off-topic, but is this really how people do k8s these days? Years ago when I
was at Google, each physical datacenter had at most several "clusters", which
would have fifty thousand cores and run every job from every team. A single
k8s cluster is already a task management system (with a lot of complexity), so
what do people gain by having many clusters, other than more complexity?

~~~
slovenlyrobot
The most common thing I've heard is "blast radius reduction", i.e. the general
public are not yet smart enough to run large shared infrastructures. That
seems something that should be obviously true.

People had exactly the same experiences with Mesos and OpenStack, but k8s has
decent tooling for turning up many clusters, so there is an easy workaround

~~~
yongjik
I still feel like that would only work in very niche cases.

I mean, if people aren't smart enough to run a large shared infrastructure,
how can I trust them to run a large number of shared clusters, even if each
cluster is small. The final scale is still the same.

~~~
iampims
Updating 100 clusters bares less risk than updating a single giant one.

------
jerendy92
Time to dump GCP then. It's not even that the fee is that large, but rather
that this is once again Google failing on a long term commitment and shafting
those on their platform once again. This was one of the benefits that was
pushed by their sales team when they called us up to market GCP over AWS and
their EKS offering. Doesn't matter that they are price matching, Google's
inability to actually commit to long term support, servicing, pricing or
features across any of their products is tiresome. Time to move business
elsewhere to AWS or Azure. They may be more expensive, but at least we know
what we are paying for, and that it's going to stay that way for a significant
length of time.

~~~
thockingoog
If the main value of GKE over DIY is $73, you should totally DIY.

I mostly try not to be too Google-focused here, but I have to say...

I'm pretty proud of GKE, and I think it offers a lot of value other than just
being cheap. Managing clusters is not always easy. GKE handles all of that for
you - including integrations, qualifications, upgrades, and patching clusters
transparently BEFORE public security disclosures happen.

We have a large team of people who deal with making GKE the industry-leading
Kubernetes experience that it is. They are on-call and active in every stage
of the GKE product lifecycle, adding value that you maybe can't see every day,
but I promise you is there. When things go sideways, there isn't a better team
on the planet to field the tickets.

I don't understand the anger here - you're literally saying you'd rather pay
more for a service of lower quality because... why? Because they will continue
to charge you more? Does not compute.

For those people who use a large numbers of small clusters, I understand this
may make you reconsider how you operate. As a Kubernetes maintainer, I WANT to
say that a smaller number of larger clusters is generally a better answer. I
know it's not always true, but I want to help make it true. GKE goes beyond
pure k8s here, too. Things like NodePools and sandboxes give you even more
robust control

GKE is the best managed Kubernetes you can get. And we're always making it
better. Those clusters actually DO have overhead for Google, and as we make
GKE better, that overhead tends to go up. As someone NOT involved in this
decision, it seems reasonable to me that things which are genuinely valuable
have a price.

Also, keep in mind that a single (zonal) cluster is free, which covers a
notable fraction of people using GKE.

~~~
atombender
Just a data point:

I'm the CTO at a very small company. All our stuff is running on GKE. Our
monthly bill tends is a lot less than $10,000/mo. We're currently in the
process of splitting our stack into separate projects and clusters, because
co-locating projects in a single cluster has gotten messy. We'll probably end
up with 4-5. That will increase our bill by $292/mo, worst case, assuming the
first cluster is free. For a company our size, it's not a huge expense. But
_these things add up_.

Since moving from DigitalOcean, our Google Cloud setup has more than doubled
our monthly bill. We're paying for more compute, but certainly not twice the
amount, as we've only gone from 14-15 nodes to around 20; it's just more
expensive across the board, both node cost and ingress/egress. We're even
cost-cutting by using self-hosted services instead of Google's; for example,
we use Postgres instead of Cloud SQL. I ran the numbers earlier today; the
equivalent on Cloud SQL would be 3.4 times more expensive.

In short, Google Cloud is expensive, and it's not like the bill is getting
smaller over time.

Developments like these factor in my choice of cloud provider for future
projects.

~~~
hn_throwaway_99
Not sure what size a "very small company" size is, but I'm just curious as to
why you chose GKE. I make tech decisions for a (probably) much smaller
company, and I found things like App Engine Flexible Environment, Cloud Run
and Cloud Functions let me do much of the stuff I can do with k8s but with
much, much less complexity (at least on my side of things). The main factor is
that I don't have a full-time infrastructure expert, and my experience in the
past is that k8s essentially requires that.

~~~
atombender
Less than 15 employees. Several products, two teams, <= 20 nodes.

We migrated our stuff from DigitalOcean around 2018. At the time, we briefly
toyed with the notion of self-hosting Kubernetes on DO, but it's complex to
manage, and we don't have any dedicated ops staff. GKE is significantly easier
to manage.

At that time we migrated, the things you mentioned weren't available/mature, I
think. Even today, I'd choose Kubernetes over a complex mishmash of different
systems. I like the unified, extensible ops model. In fact, I'd go so far as
to say that I wish all of GCP could be managed as Kubernetes objects.

~~~
briangrant
Re. managing GCP as Kubernetes resources: [https://cloud.google.com/config-
connector/docs/overview](https://cloud.google.com/config-
connector/docs/overview)

~~~
atombender
That's very cool, thanks! Note that this allows selectively _creating_
Kubernetes resources backed by GCP resources. Looks like it will not
automatically sync everything that already exists, which seems like a missed
opportunity.

------
jeroenvisser101
> Last modified: November 27, 2018 | Previous Versions

> As of November 28, 2017, Google Kubernetes Engine no longer charges a flat
> fee per hour per cluster for cluster management, regardless of cluster size,
> as provided at [https://cloud.google.com/kubernetes-
> engine/pricing](https://cloud.google.com/kubernetes-engine/pricing).
> Accordingly, Google no longer offers a financially-backed service level
> agreement for the Google Kubernetes Engine service. The service availability
> of nodes in a Google Kubernetes Engine-managed cluster is covered by the
> Google Compute Engine SLA at
> [https://cloud.google.com/compute/sla](https://cloud.google.com/compute/sla).

> Uptime for Google Kubernetes Engine is nevertheless highly important to
> Google, and Google has an internal goal to keep the monthly uptime
> percentage at 99.5% for the Kubernetes API server for zonal clusters and
> 99.95% for regional clusters regardless of the applicability of a
> financially-backed service level agreement.

[https://cloud.google.com/kubernetes-
engine/sla](https://cloud.google.com/kubernetes-engine/sla)

Interesting that they're now walking back from this...

~~~
developerdylan
Google, walking back their commitment to software? _gasp_

~~~
sneak
[https://twitter.com/SwiftOnSecurity/status/12341533883188715...](https://twitter.com/SwiftOnSecurity/status/1234153388318871555)

------
praveenperera
Good time for people to look into DigitalOcean managed Kubernetes (DOKS). I've
been using it since it was in pre-release and its been great so far. Their
support has been very responsive as well.

[https://www.digitalocean.com/products/kubernetes/](https://www.digitalocean.com/products/kubernetes/)

~~~
developer2
Care with DigitalOcean's Kubernetes offering. Unless they have completely
revamped their entire stack since their initial release, it's a security
nightmare. The way they launched it was unacceptable for any company to use. I
immediately migrated my projects away and closed my account upon understanding
how abysmally they screwed up the security of that offering. The fact they
were willing to launch that means they cannot be trusted to host _anything_.

Not only are your cluster's administration ports exposed on public addresses
with no ability to firewall them, but each node pulls an Admin-level auth
token to manage the DigitalOcean account behind the scenes. A single http
request made to an internal IP, _from within any Docker container running
within the cluster_ , grants the attacker full read/write access and control
to the underlying DigitalOcean account. This includes any developer who does
so from code deployed into the cluster, giving the developer full access to
the business's DigitalOcean account.

~~~
sho
Are you referring to this? [https://www.4armed.com/blog/hacking-digitalocean-
kubernetes/](https://www.4armed.com/blog/hacking-digitalocean-kubernetes/)

I'm curious as to whether that has been fixed and a proper security evaluation
done. I've been avoiding k8s on DO for a while because of that (perfectly
comfortable with their other services) but it would be good to get an update.

~~~
developer2
Yeah, that's the one. I'd done my own analysis before that came out and
figured it was an unacceptable risk, and then that article came out with the
actual attack vector in all its glory. The simple fact the k8s/etcd ports are
exposed on a public address with no ability to firewall it off is bad enough,
as you're relying on the security of the software running on those ports
rather than a firewall restricting which source address(es) can even connect
to begin with.

The credentials (certificates) being exposed via
[http://169.254.169.254/metadata/v1/user-
data](http://169.254.169.254/metadata/v1/user-data) – from within any
pod/container, not just from a physical node – was the final straw. I'd
forgotten that the DO token wasn't directly listed there, but can be extracted
from the etcd instance where it is stored (explained under "DigitalOcean
Account Takeover" of that article).

Again, all of this may have been (and hopefully has been) mitigated since the
original release. For me, it's too late to reevaluate; the fact that was
considered releasable in the first place destroyed any credibility in my eyes.

------
jfrankamp
This strikes me as an attribution problem e.g. Within large companies business
units are treated as standalone financial entities so you are either making
money or a cost center. I think what could be happening here is that the
kubernetes group is generating what look like un-paid-for costs, instead of
being back-credited for ALL of the compute they are selling via the worker
VMs. I would argue that people come to GCP for GKE which means it should be
seen as a revenue generator, in that light it makes sense to hold the loss
leader of the control plane open and 'free' to grow GKE as quickly as
possible. This decision is bad for business.

~~~
nojvek
Well then this is perhaps a big failure by the execs at GCP. It’s making the
individual teams look good at the cost of unhappy customers.

Sometimes I think Google has phenomenal engineers, they have this massive ad
cash cow from search, YouTube etc. However when it comes to business units
that need to handhold their customers and need human interaction, they are
pretty terrible at it. Empathy is perhaps a skill that’s not important to them
when they are hiring.

------
marvinblum
We moved our cluster into the Hetzner Cloud for Emvi [1]. Yes, they don't
offer a managed solution and it took us about two weeks to set up and test the
cluster properly. But the cost is less than 1/6 of what gcloud costs us. If
you have the resources and knowledge to maintain your own cluster, check it
out. They have insanely good pricing (2,96 €/month for the cheapest instance
which is faster than Googles $15/month VM).

Here is a very good tutorial on how to set up your own cluster:
[https://community.hetzner.com/tutorials/install-
kubernetes-c...](https://community.hetzner.com/tutorials/install-kubernetes-
cluster)

[1] [https://emvi.com/](https://emvi.com/)

~~~
sphix0r
Hetzner is solid from a perf / price perspective. Mind that the network
peering outside Europe is not that great. So could be a very good choice
depending on your user base' location.

~~~
marvinblum
True. In our experience it's fast enough. Additionally traffic is free unless
you hit the 20 TB outgoing traffic per node (which probably will never happen
to us). In contrast gcloud costs about 12ct per GB outgoing traffic (!!!).

------
agoell
After this, I've been exploring other places to host our team's clusters...
copied pricing below

\- EKS: $0.10/hour/cluster

\- Digital Ocean: Free (only charges for the nodes)

\- Azure: Free (only charges for the nodes)

In the long run we'll probably try and build our stack on vendor-agnostic
tools..

\- Rancher -
[https://rancher.com/products/rancher/](https://rancher.com/products/rancher/)

\- Infra.app - [https://infra.app](https://infra.app) (mentioned a few weeks
back on the Kubernetes podcast)

\- Prometheus [https://prometheus.io/](https://prometheus.io/) \- metrics

The cloud providers all include their own tooling (logging, monitoring) built-
in but I'm worried this will only lock us on to further price increases.. has
anyone found a good vendor-neutral logging system? We don't really want to use
ELK stack right now since it's really heavy and costly to run...

~~~
thockingoog
[https://news.ycombinator.com/item?id=22487110](https://news.ycombinator.com/item?id=22487110)

~~~
rcconf
Honestly I understand the hard work it takes to manage all the clusters, but
this was a total bait and switch and hurts the reputation that everyone has
with Google Cloud. Telling us to DIY because we cannot pay $71 just sounds
like someone who works at Google would say, which you do work at Google.

The sentiment with my clients before was that Google Cloud was a great choice
because of the security and expertise with GKE. It's also free!

Meanwhile, in the back of my head I've always had this fear because of your
reputation that you do not keep your promises and that you do not care about
your users. Because of this fear, we have tried to make every infrastructure
decision not use a managed service by Google even though it may be easier to
do so short-term.

For the product I'm working on, we decided to use Kubernetes just in case you
baited and switched us with the reputation you have. In terms of monitoring,
we really wanted to use Stackdriver, but now we're 100% using fluent-bit +
prometheus + loki + grafana. It's the only way to protect ourselves from your
reputation which is becoming a reality.

So yeah, this is pretty sad and a bad decision. Should have priced GKE at $70
/ month to begin with and we would have been fine with it. Now we're
(actually) looking at EKS since Amazon doesn't seem to have this reputation
and you've spooked us. We never would have thought about using any other
provider until today.

~~~
thockingoog
I understand the emotional response here, but I don't think it's rational. GKE
has to work as a business, or else the whole thing is in trouble.

I think GKE provides tons of value, but people tend to under-estmate that. In
order to keep providing that value, we need to make sure it is sustainable.

I'm really, truly sad that you perceive it as bait-and-switch, but I disagree
with that characterization. If you want to move off GKE, I'll go out of my way
to help you, but I urge you to take a big-picture look at the TCO.

~~~
ohyeshedid
I think part of the optic's issues is your peers seem to be offering similar
services for free, while being sustainable.

~~~
thockingoog
EKS has always had a fee.

AKS, well, I don't have any insight into their business, but I have my
suspicions.

------
rcconf
Oh wow, one of the biggest reasons we picked Google Cloud was that you did not
have to pay a flat fee for their managed Kubernetes service. Luckily there is
Kubernetes support across all Cloud Providers so we're happy we're not vendor
locked in. (biggest reason we picked Kubernetes in the first place.)

We were thinking of using Stackdriver for logging, but we were scared of
vendor locked in due to price increases or other changes that we've been
warned about with Google. In this case, I think it's safe to say we'll be
using Prometheus + Grafana + Loki instead since there may be a random
Stackdriver flat fee introduced or some other weird fee and we may need to
migrate out of Google.

~~~
atombender
Stackdriver is terrible, and super expensive for what it is. We ran a
dedicated Fluentd in GKE for a long time to work around its shortcomings (GKE
also uses/used Fluentd to shuffle logs into Stackdriver), then switched to
using Loki + Promtail + Grafana, which has been excellent.

~~~
physicles
+1 for Loki + Promtail + Grafana. Really low maintenance once it’s set up.

------
nimos
"Let's trade goodwill for short term profits." I suppose not really surprising
given the maps fiasco and Oracle appointment but this really comes off poorly
to me.

Flat fee is gonna suck for people running a lot of clusters. I bet there are
some people out there spinning up a cluster per x who are going to be real
unhappy about this.

------
yashap
I know the HN crowd hates this sort of thing, but seems reasonable to me.
$70/month is a very reasonable price, and most businesses likely have well
under 10 clusters (I’d think many just have 2-3, a single cluster for prod and
then 1-2 dev/test type environments). This is probably mostly to cut down on
edge cases users who are spinning up crazy numbers of clusters for weird
reasons, and costing Google a bunch of $$$.

~~~
sneak
Furthermore, anyone spending enough on compute to warrant k8s shouldn’t balk
at all at $70/mo. I think the threshold for introducing the complexity and
overhead of k8s isn’t probably until at least $5-10k/mo of spend (and probably
3-10x that in the normal case). Less than that and k8s is a whole lot of
overkill.

~~~
briffle
We use google cloud projects to isolate customers and environments. (Some of
our clients are old school and VERY scared of cloud, and multi-tenant). so for
our pretty small company, we have >60 projects, each with a k8s cluster. that
is a pretty good bump in costs come this summer.

Historically, we have powered down all the compute in a project that isn't
needed, but left the k8s cluster in place (with its compute nodes powered
down) because we could then bring it back up in 2 min or less. That dropped
our cost for each project that was powered down to ~$25-$50/month, depending
on how much disk space, etc, they were using. this will more than double (or
triple) the cost of those projects. If we have to rebuild a full cluster from
scratch, we then have to wait for the global load balancer to build our
ingress, and then go authorize a TLS cert. This adds 20-30 min to re-
activating our projects, which will suck.

~~~
zomglings
Perhaps the right thing for GKE to do is introduce a cluster snapshot. Sounds
like a great feature.

~~~
briffle
That would actually be pretty awesome.

------
etchalon
Time to start looking into DigitalOcean more seriously.

G Cloud is already unreasonably expensive and nearly impossible to price
manage. It's cool to see them double-down on that.

~~~
itake
DO is pretty famous for losing data.

[https://news.ycombinator.com/item?id=20064169](https://news.ycombinator.com/item?id=20064169)
[https://news.ycombinator.com/item?id=17225665](https://news.ycombinator.com/item?id=17225665)

~~~
whalesalad
DO is famous for being a pain in the ass. They’re great for tiny hobby things
but honestly I’d never run a prod/serious/client workload there. Too many
issues. I’ve had _multiple_ clients lose a droplet due to a simple credit card
expiration.

It’s a race to the bottom on price so this doesn’t surprise me. They chose
this life.

~~~
geerlingguy
To be fair, what is the best way to handle expiring credit cards? For one of
my SaaS products, I give a 30 day grace period, then delete the data. If they
didn't have a backup, that's on them...

If they delete the droplet the second a single CC payment fails, that's one
thing, but I don't believe that's how their system works.

~~~
whalesalad
If you are literally _in the business_ of enabling, storing, and protecting
production workloads, data, etc.. then catastrophic data loss should be an
asbolute last resort.

In both of these instances I am referring to a balance of less than $20.

So for less than $20 (a few weeks late) DO says, welp fuck this customer we
are going to terminate all of their resources immediately.

This is what DO and others need to do: Put it in your terms that you will keep
racking up charges and then send it to collections. Charge interest, charge
fees, do whatever you want. Turn $20 into $40. Why? Because businesses do not
give a shit... if it is between losing everything or a slap on the wrist
(monetary fee) they will chose the latter every time.

One of my clients had to painstakingly trudge through archive.net to recreate
their missing blog posts. How fucking miserable is that? Over a few hundred
megabytes of disk that DO could have kept around...

Also, actally _make an effort_ to reach out before doing anything serious.
Call phone numbers, email other members on the team to alert them to the
issue, etc...

Too many times I have seen some script kiddie throw together a client's WP
site and toss it on DO because it is 'so cheap and cool' and yet they forget
about everything else: backups, security, managing the box, etc... and
inevitably shit will hit the fan.

I was really rootin' for DO in the beginning. I even applied to work there
when they were first starting out but did not want to relo to NY. Now I am
moving three clients OFF of DO because they are all very unhappy with the
level (or lack) of service they've received.

~~~
nuggien
Or...if it was important enough to you that losing it hurts, then maybe pay
attention to your emails and don't let things expire and pay your shit on
time. And of course, a sane person would backup anything important.

~~~
imtringued
Ok, lets say I store all my backups at AWS, Google and Azure. My credit card
expires and all backups are gone. What's the point of additional backups in
this scenario?

~~~
nuggien
Aws, gcp and azure are unlikely to delete your shit the moment your card
expires?

------
youngdynasty
I run a hobby project in multiple zones because low latency is important, and
Kubernetes makes it easy to do so.

There's no way I'm going to pay an extra $73/mo -- I already pay for the
computing resources, this should be free.

Looks like I'll be moving away from GKE. It's a shame, I _was_ a big advocate.

~~~
sethvargo
We offer one free zonal cluster, which is specifically designed for your use
case :)

~~~
sofaofthedamned
I have personal experience of a few companies in the UK where GCP are offering
90+% discounts to onboard. GCP are spending hundreds of millions to do this.
K8S control-planes are a rounding error compared to this.

You could have grandfathered in current deployments but - nope. In the tech
world this is up there with killing Google Reader.

------
zmmmmm
It's interesting how naive the gcloud folks in commenting here seem to be
about how it is to be an enterprise customer evaluating their service.
Deploying to a cloud vendor is a bit like deciding to build a house on a
rented lot. You place an incredible amount of trust in your landlord -
unbelievable really. It's a miracle people do it at all in some ways. This
kind of action is like Google going to their land for rent and posting
earthquake signs on front of it. It is toxic to the most fundamental element
of trust people rely on when they choose a cloud vendor. It can't possibly be
worth whatever miniscule revenue they will accrue from implementing this.

------
simonkafan
It's not so much about an additional fee (honestly, 0.10$ per hour are
nothing) it's more about Googles practice of suddenly charging services,
shutting down services as they like, not giving a shit about us customers.
Azure and AWS are much more customer friendly here.

------
adityapatadia
A long time Google cloud customer who migrated away to DO recently. Here are
the things Google gets completely wrong

\- They are way behind compared to AWS in terms of stability and features,
even then they always demand similar premium (ref. recent incedent:
[https://www.cbronline.com/news/google-cloud-
down](https://www.cbronline.com/news/google-cloud-down))

\- Tax issues in foreign countries are never resolved. Due to Indian laws, we
must pay TDS on our business payment and google has to refund them to us. It
takes almost a year to get those refunds from Google. Not startup friendly.

\- No support!! If anything goes wrong, AWS or DO solved issues even technical
ones at times even when you have no support plan purchased. Google literally
is useless when it comes to support. They carry that attitude from their
search business.

\- I see someone mentioned egress charges. In reality, cloud providers get
super cheap bandwidth and they charge 9-10 cents for egress which is
outrageous. It's not only Google but AWS as well but Google being new in
market could have done better job here.

\- DO Kubernetes is free and as someone mentioned, control plane downtime does
not matter until workloads are not affected.

\- In a nutshell, if I need stability, I will go to AWS, if I need better
price, DO is not a bad choice. I am not sure where is Google positioned.

------
sethvargo
Hey everyone - Seth from Google here. Please let us know if you have any
questions! You can learn more about the pricing changes at
[https://cloud.google.com/kubernetes-
engine/pricing](https://cloud.google.com/kubernetes-engine/pricing).

~~~
Glyptodon
This change is pretty huge for non-revenue units & small teams at institutions
and SMBs. These smaller teams often seem to run two clusters rather than try
to split their production and dev environments within one cluster (I think
this is even widely recommended for smaller, less experienced outfits). For
many the management fee will probably be a large percentage cost increase for
units that are very cost sensitive and require significant re-engineering to
avoid for units where engineer hours a scarce resource.

Seems weird, given that GKE is basically the main reason people seem to use
Google Cloud. These kinds of users aren't big fish, but I suspect a lot of
them are going to run.

~~~
donmcronald
Yeah. Having 1 free cluster will just encourage SMBs and hobbyists to resort
to the bad practice of commingling dev and production, wont it? It's the same
attitude that GitLab has with a lot of their CI stuff. It's not only huge
enterprises that want to try to follow best practices.

------
wereHamster
> One zonal cluster per billing account is free

For hobby projects nothing will change.

------
kwils
Note: Azure does not charge a master/cluster management fee (bias disclosure I
work for Microsoft)

~~~
sethvargo
Attempting to capitalize on a competitor's announcement isn't cool and doesn't
look great.

~~~
jchiu1106
I don't work for Azure and having used both GKE and AKS, I can say GKE is
superior, however, I don't understand why this comment isn't cool. It's not
like they're capitalizing on your misery. It's a decision you have taken and
it's a fair game IMO that they want to emphasize their advantage over you.

------
jjoonathan
Oh, so now in addition to Google's reputation for killing services, GCP wants
a reputation for raising prices?

~~~
justaguyhere
They have done it before. Remember Google Maps?

------
ratherbefuddled
I don't like the direction this is heading, it seems like the SLA and the
accompanying charge could easily have been optional.

We have a couple of dozen clusters, two per client, and can't change the
architecture. We use helm and terraform and can build new clusters quickly but
we can't treat them entirely like cattle because we don't own all the DNS. Our
clients are not the sort to do things quickly - or even slowly.

Does anybody have any good and up to date resources comparing the current
options for K8 providers? I'd like to get a feel for what it would take to
switch.

~~~
vpEfljFL
As many have mentioned here already, $72/mo most likely a rounding error on
workloads kubernetes is designed for.

I think, most customers love the change because of SLA where even 1 minute of
downtime per year is amplitude more costly than 10 years of cluster managing
fee.

This also showing the commitment from google to provide great and reliable
service.

If you're looking to run k8s "for free", Digital Ocean looks like the way to
go, but again, it's two completely different set of offerings and if you've
chosen the google cloud in the first place then DO doesn't look like suitable
alternative.

~~~
ratherbefuddled
> As many have mentioned here already, $72/mo most likely a rounding error on
> workloads kubernetes is designed for.

There are _many_ reasons to use k8 beyond just workload scale and that amount
per cluster per month isn't anywhere near a rounding error for many
deployments.

~~~
vpEfljFL
I would be interested to know when you may need kubernetes for small types of
workload and simple architecture.

For my understating, support contract alone for google cloud will cost you
around $150/mo.

For small to medium workloads there are plenty of tools if you want to use
containers: docker swarm and nomad. Docker Swarm is really simple and most
engineers already know it because of `docker-compose.yml` they're using each
day.

I can't really understand what type of workload you have because kubernetes
cluster management requires you to have at least 2 full-time DevOps team. What
is $72 compared to salary of two employees?

If you have an issue with $72 for SLA guarantee then I can't really understand
why you need google cloud at all. They are literally bound to maintain zero
downtime because of huge losses if something go down for their entire customer
base.

~~~
ratherbefuddled
One simple example - of many: Enterprise clients with low workload but very
sensitive data strict infosec and segregation requirements - several clusters
per client all managed with infrastructure as code.

There are lots of benefits to k8 other than just scale, and there are
architecture choices that are made for good reasons that rely on separate
clusters with no particular need for a legal uptime SLA on a control plane.

This decision will cost us a lot more than $72 a month.

------
caleblloyd
I think that CRDs are partially to blame for this. CRDs can tax the API Server
and backing data store, without directly mapping to a revenue-generating
activity.

I’ve noticed a trend where teams spin up new clusters for each application.
Since CRDs are installed on the cluster level, it is not possible to namespace
resource versions. It is easier for teams to take the cluster-per-application
approach as opposed to mandating a specific version of cluster tooling.

More small clusters means more control planes, and more subsidizing if a cloud
provider is giving away the control plane.

I just finished a blog post on this opinion that goes into more detail-
[https://caleblloyd.com/software/crds-killed-free-
kubernetes-...](https://caleblloyd.com/software/crds-killed-free-kubernetes-
control-plane/)

------
pnathan
If the trade is a SLA for a management fee, that is a reasonable business
decision, and largely a rounding error for a decently sized company with a
well designed clustering system. Lack of SLAs is a major issue IME with cloud
providers.

------
klingonopera
"Cloud computing is a trap, warns GNU founder Richard Stallman" [2008]

[https://www.theguardian.com/technology/2008/sep/29/cloud.com...](https://www.theguardian.com/technology/2008/sep/29/cloud.computing.richard.stallman)

~~~
klingonopera
I know this is kinda against the guidelines, but just to highlight what an
ideological war is being fought, at this moment, my parent post has received 5
upvotes and 6 downvotes, with the last 4 downvotes all occuring in the last
minute.

On the content, RMS chose a colorful (insulting) language, yes, do hold that
against him, because _that_ is wrong, but in my opinion, at the core, his
statements are quite legit.

~~~
cmhnn
I am grateful for Stallman's efforts but claiming cloud is a nefarious plot no
matter the style is not "legit".

As for the voting, join the club. Like all communities online HNN is an often
hypocritical cliquefest where the mob will have its way. But at least it's not
r __ __;)

------
whatakaruvad
This is seriously such a bummer. This was the main reason for us moving out of
AWS

~~~
MightySCollins
We also jumped over for this reason and had to deal with a large number of
gotchas from GCP. I kinda wish I never spent the long nights on this...

------
lain
That's about $72 a month, which matches Amazon EKS's lowered pricing. I guess
that rules out my hopes of EKS not charging for the management plane in the
near future.

~~~
state_less
It looks like Google's offering one free cluster per billing account if I read
that right. If that's not the case, I'll be turning off my cluster before this
kicks in.

~~~
dman
One zonal cluster per billing account is free

------
stevencorona
Wow, this is a huge bummer. A lot of our infrastructure assumptions have been
based around having several small GKE clusters.

~~~
thockingoog
I think this trend is not good overall, and people will eventually be very
unhappy with it. I'd rather help you figure out how to use fewer clusters.

~~~
seneca
Tim, would you be willing to elaborate on why you dislike the "many small
clusters" pattern?

~~~
thockingoog
Many small clusters just do not deliver on a lot of the value of Kubernetes.
Clusters are still hard boundaries to cross (working to fix that). Utilization
and efficiency are capped. OpEx goes up quickly.

There are reasons to have multiple clusters, but I think the current trend
takes that too far.

TO BE SURE - there's more work to do in k8s and in GKE.

~~~
seneca
As always, the insight is appreciate, Tim!

------
sladey
This is really disappointing. GKE was a staple amongst Kubernetes adoption,
not only for the feature-set but also that there were no overhead costs.

I hope GCP re-thinks this.

~~~
thockingoog
For folks just trying it out, 1 cluster is still free.

~~~
mleonard
For folks just trying it out, 1 cluster is still free... in a single physical
data centre. Sadly you'll be charged for running a cluster across two or three
data centres in the same availability zone (eg London).

~~~
thockingoog
This is a fair point. We don't have an HA (multi-master) zonal offering
either, because mostly people don't want that.

------
d_watt
$72/month per cluster, regardless of the size. It's interesting they're not
charging per managed node (outside of the regular machine cost), makes it
steep if you want to have a small cluster up.

~~~
tssva
Each billing account gets 1 free zonal cluster so the cost for keeping a small
cluster up won't change.

~~~
enos_feedler
How do if i know if my cluster is zonal? Does it mean all nodes are vms
provisioned in the same zone?

~~~
zaat
>Single-zone clusters

>A single-zone cluster has a single control plane (master) running in one
zone. This control plane manages workloads on nodes running in the same zone.

[https://cloud.google.com/kubernetes-
engine/docs/concepts/typ...](https://cloud.google.com/kubernetes-
engine/docs/concepts/types-of-clusters)

------
zelly
You Probably Don't Need Kubernetes

If you're small, you can probably just run a few containers/VMs with automatic
restart.

If you're big, you already made something custom like Borg.

Who's it for? Temporarily embarrassed unicorns?

------
pcj-github
Thomas Kurian is going to be the Steve Ballmer of Google, and he's currently
tearing up Google from the inside. Google leadership need to wise up and give
this guy his walking papers.

~~~
GKE_Greed
he is like GKE, once you start him, you can't stop :)

------
MightySCollins
I am really disappointed by this. One of the reasons we moved to GCP from AWS
was the ability to create multiple clusters at no extra charge. Now it looks
like the pricing matches EKS.

------
marcinzm
I don't offhand remember AWS increasing prices for a service before but I
might be wrong. How often does Google increase prices?

For a business I prefer a company that starts with higher prices and then only
lowers them to one that may increase them at any time.

~~~
saltysugar
Having worked for AWS, one of the things we got a lot push back from was
offering something free and then walk back from it.

AWS really strongly focuses on gaining customer trust, and they will only
lower price, and never increase price. They won't turn off things until the
last customer stops using it (they might stop new customers from onboarding)

I did not enjoy working for AWS so I left pretty fast, but some of the
customer obsession there really impressed me.

~~~
pdelgallego
For a AWS product manager, pricing is a one-way door once you cross it there
is no way back from it you.

------
predictmktegirl
I was an early adopter of Google App Engine when it was first released and the
consistency guarantees of the datastore burned me pretty badly. What I
experienced in the test environment was not even remotely close to what I got
in production.

Just last week, I was so tempted to give Google another shot over AWS. By all
accounts, they have come so far, and more importantly, started using more of
my preferred toolset as well. This is exactly the kind of thing that really
scares me off.

I have a huge amount of respect for the world class systems scientists and
engineers at Google. Unfortunately, they seem to counter it with a customer
experience full of uncomfortable, often expensive surprises.

------
maktouch
We're a little bit too locked to GCP to migrate out.... But I'll definitely
stop evangelizing GCP to people.

I have now lost faith in you, GCP.

~~~
GKE_Greed
same here - losing face now with my team when I argued for GCP (and won)
against AWS.

------
jaimex2
And this is why when Google's team comes knocking on your door enticing your
company to move to GCP from AWS you should ask for at least one years worth of
credit at minimum.

While you transition leave as many doors open to switch back.

------
nojvek
The email definitely scared me and put a sour taste in my mouth. Sudden fee
changes with somewhat vague details aren’t a great way to build trust.

Especially when GCP doesn’t have a great rep like Azure and AWS.

To clarify, this is hourly charge per entire cluster right ? If it’s a small
cost, ~100/month, why can’t GCP absorb it. Digital ocean, which is a smaller
player has much simpler and saner pricing.

AFAIK GCP has higher compute pricing for similar instances compared to AWS.
(Last I checked). How is this a good deal for the customers ?

------
minimaxir
I think this is _fair_ : for hobbyists, a free zonal cluster is sufficient and
you probably wouldn't use more than one cluster. For businesses/revenue
drivers, the $7.30/mo/cluster is nothing (EDIT: actually $73/mo/cluster, which
may be a tougher sell but if the business is in a case where it benefits from
Kubernetes, it's still likely insignificant relative to the cost of actually
running the VMs on it )

~~~
sladey
If I'm not mistaken, it should be $73.00+/mo

~~~
minimaxir
Oops, I can't math. Fixed.

------
partingshots
The guarantees for uptime with 99.5% for regional and 99.5% for zonal clusters
seem pretty good. Is this above the average in terms of cloud providers?

~~~
seslattery
Small typo: 99.95% for regional. It's similar to AWS with a 99.9% for EKS
clusters. Also not sure what GKE's new SLA reimbursement structure will be
going forwards.

------
sandGorgon
There's this theory - "CRDs Killed the Free Kubernetes Control Plane"

[https://caleblloyd.com/software/crds-killed-free-
kubernetes-...](https://caleblloyd.com/software/crds-killed-free-kubernetes-
control-plane/)

Basically running etcd in production with reasonable SLA and more than a
little bit of data is super hard.

------
mchiang
I haven’t done cost calculations yet, but this might actually make me want to
explore to EKS pricing. I do love the experience of GKE.

~~~
lain
It's the same as the new pricing that EKS has at $0.10/hour. Guess that rules
out my hopes of EKS not charging for the management plane any time in the near
future. :'(

~~~
sethvargo
It's slightly different than other cloud pricing because we include a free
zonal cluster as a hobby tier.

------
beaker52
Careful Google, free Kube is GCP's only big-shiny draw and you've just cut
that into quarters.

------
kwils
Note: Azure does not charge a master/cluster management fee (bias disclosure I
work for Microsoft)

------
GKE_Greed
This is penny wise pound foolish GCP.

------
whatakaruvad
At one side EKS is slashing their prices and on the other GKE is increasing it
:(

------
ljsmith93
"Anthos GKE clusters are exempt from this fee"

Adopt Anthos or pay, basically.

~~~
jasonvorhe
As long as they don't support kvm Anthos is going to be tough sell,
considering the VMware premium.

------
jwegner
There actually was a 15/cent/hr fee a few years back:

[https://www.forbes.com/sites/janakirammsv/2017/11/29/google-...](https://www.forbes.com/sites/janakirammsv/2017/11/29/google-
drops-cluster-management-fee-from-its-managed-kubernetes-service-offering/)

YMMV, but I think the value prop of using a managed GKE cluster vs the raw
costs and engineering time to run your own control plane is still strong.

------
eatmyshorts
I guess I need to start migrating my clusters, then. $73 per cluster on my
prod, dev, QA, and test clusters is a significant increase for me. Looks like
Google is doing their best to kill off GCP -
[https://www.cnbc.com/2019/12/17/google-reportedly-wants-
to-b...](https://www.cnbc.com/2019/12/17/google-reportedly-wants-to-be-top-
two-player-in-cloud-by-2023.html)

~~~
briffle
Curious where you are going to go that is going to be cheaper for the
management plane and work nodes.

~~~
eatmyshorts
AWS+kops is less expensive. Digital Ocean and Azure provide services without
charging for master nodes.

I will likely choose AWS+kops.

------
tedk-42
The number of people that still trust google not to screw customers over is
amazing.

They got you into their tech by offering something for free - now they're
upping the price to match the leading competitor (in cloud that is).

Better than getting you into their ecosystem and discontinuing a service I
suppose.

I hope this backfires on them and they provide it 'free forever' as they would
have received a lot of customers drawn in from their free GKE control plane
offering

------
taormina
Ahh, wonderful. You're taking away the only thing that made me pick GCP and
GKE over AWS and EKS?

AWS used to charge me for the master node, and it was more than the worker
nodes cost. So the logic reaction to AWS lowering that cost to a flat $.10 an
hour was for Google to raise their own price point to that??????

As a GCP customer, I've got one simple question. Why exactly should I continue
to use GKE?

------
miguelmota
The good ol' bait and switch tactic; lure new customers in with zero fees and
switch out the fee structure once they're locked in.

------
JMTQp8lwXL
I've been mentioning Google's penny pinching for awhile here. Simple things
like Chrome's address bar showing google searches before my bookmarks. It's
all part of the monetization. Are fees like this because Google is struggling
to continue to grow? That's probably the most concerning thing for Google's
future, rather than a $73 fee.

------
cbushko
One of the reasons I recommend GKE over EKS is because of the lack of fees on
the control plane.

I guess that one advantage is gone now...

Bad move Google Cloud. Bad move.

------
reilly3000
AKS still has a free control plane. GCP won my business for a bit, but quickly
lost it based on some features. I still love StackDriver and BigQuery, but
don’t love doing business with GCP- the sales/support experience was pretty
lacking, networking for serverless was immature for multi-region, and what
they are doing with Anthos feels like Oracle (it is).

------
w9jds
There are so many issues with this. I used GCP because I liked the service and
was able to build all of my stuff well on it. My personal projects already
make me shell out over $250 a month because of running costs and now you want
to gouge me for hourly running of a cluster I alraid pay hourly of compute
time for? What in the actual hell...

------
busterarm
The SLA guarantee for stable is interesting because at every turn the
documentation and our TAMs encourage us to switch from Stable channel to the
Regular.

Workload Identity, for example, is pushed hard, but it's Beta so there's no
SLA and it's broken in Stable so if you use it you're going to have outages or
be forced onto Regular.

------
londons_explore
Are there any other companies doing managed Kubernetes in GCP?

If there was interest I could try to build a third party clone of GKE running
in all GCP regions and still managing GCP VM's, and I could run it for a lot
less than $0.10 per hour (although obviously it would take many months of
refinement till I could offer a decent SLA)

~~~
the_duke
Azure and AWS both have managed Kubernetes ...

~~~
londons_explore
But a lot of businesses have a reason to stick to GCP. For example if you need
access to TPU's... You can't easily use Azure kubernetes with the actual VM's
running in GCP...

------
manigandham
Years ago, GKE had a $0.15/hr charge for clusters with more than 5 nodes. They
dropped that and other clouds matched them. Now they're adding back a fee for
all clusters.

Bringing back that original min cluster size would be a good idea and probably
solve a lot of the issues with people running dev/stage clusters.

------
spicyramen
I'm not gonna beat on the dead horse, but with GKE increase I need to explain
why I choose GCP for GKE and other services like new KFP. The increase is not
much TBH but the point is that there is an increase and tomorrow could be BQ
and then later APIs...I don't know what to say to my manager now...

~~~
zapita
I honestly can’t tell if this is satire.

------
vhmaster
Those who had a chance to look deeper into k8s control structure, can
comprehend such movement. There's no free lunch. I mean master nodes, etcd and
all that stuff in HA has its costs. That's it. Surprisingly, AWS announced 50%
price reduction for EKS control plane January this year :)

------
gigatexal
This is not cool. One of the best parts of GCP is that the control plane is
free for k8s. :( this change is simply a money grab at least optically.

How does the exemption for anthos clusters work? If I enable it on my cluster
but it’s still a GCP native cluster and not on non-premises one, am I still
billed?

------
rudolph9
I wish there was a "Preemptible" or more cost effective tier for GKE. I get
why they're doing this and $70/month seems reasonable for a premium service
but would be nice to have a more cost effective option similar with tradeoffs
similar to "Preemptible" nodes.

------
geraldC13
Now is the right time to Terraform stuff and get some portability in. We can
help you out if you need to be quickly up and running on Infrastructure-as-
Code \---> [https://www.cloudskiff.com/](https://www.cloudskiff.com/)

------
domlebo70
Does anyone have any opinions on alternative stacks to GKE within GCP? My side
project is all dockerized. We love that aspect. Google Cloud Run looks
promising, but it's not available in Sydney, which rules it out for us
unfortunately.

------
hughpeters
This makes GKE pretty much impossible to use for side projects. Given that it
will cost $100 per cluster per month without including the instance costs.

Edit: A single zonal cluster per GCP account is exempt from this fee, so this
comment is inaccurate.

~~~
thockingoog
Free (zonal) cluster per account, regardless of size, should cover a lot of
this, no?

~~~
hughpeters
Great point! Didn't catch that one. I'll edit my comment

------
b34r
That’s kind of a lot. $876 per year for a toy cluster that might not do
anything...

------
nikolay
Why is the third in place cloud (GCP) making their offering worse than the
second place one (Azure)? DigitalOcean also doesn't charge per cluster. Not to
mention how behind the Kubernetes releases they are...

------
jb775
I'm not sure why everyone is so surprised. Part of Google's monetization model
is to offer developer-friendly software for free, then charge a small fee once
it crosses the headache-to-replace threshold.

------
timplant
GKE is the best Kubernetes offering according to my investigation of
Azure/AWS/GCP. However adding extra $73/mo is not trivial for small teams like
us. That is price of a medium-classed VM...

------
outime
Not that I invested much in GCP but this was just what I needed to completely
stay away from it myself and my clients. Really awful decision. I'm sorry for
those affected who trusted them.

------
rb808
What resources would the cluster mgmt require? I always thought k8s was
perfect cloud platform as the management was minimal and you're actually
paying for the pod resources you require.

------
tebruno99
Now I'm not sure what sure what we were paying for when we paid for our
cluster? Isn't the price we pay to use/build the cluster the payment for the
cluster?

------
duncangh
And I just cancelled all my Kubernetes side projects on GCP

------
topherpalmtree
Wait until people realize gcp might not exist after 2022 if it’s not
profitable. Better pay for it than have your platform yanked out from under
you.

~~~
yjftsjthsd-h
Better still to take this opportunity to get out before Google does what
Google does best.

------
MuffinFlavored
How many businesses really need Kubernetes? Can't you orchestrate
infrastructure + rolling deploys with Terraform + Docker containers?

~~~
jjeaff
I'm not aware of any rolling deploy functionality with terraform or docker.

Docker swarm does, but they are primarily using swarm with kubernetes these
days.

------
thefounder
Just a reminder to everyone: avoid proprietary services such Google
datastore/firestore, etc! There is no (easy) migration path.

------
johnvega
So what makes anyone think this is the last?

------
tuananh
i can't remember the last time AWS raise the price of their services.

in fact, i can only recall when they reduce price.

------
dmitryminkovsky
Just got this email. $0.10 per hour is so much for something that was 0 per
hour before. Wow these guys. The chutzpah! This isn’t just offensive
monetarily, it’s offensive like price gouging is offensive. It’s emotionally
offensive. Gotta look somewhere else. Shame.

------
_jezell_
Ah, the sound of former Oracle executives counting their GCP bonuses.

~~~
toomuchtodo
Why people think cloud providers are benevolent provider of infra is beyond
me. Their margin is because people are willing to pay it. Either run your own
metal (k8s arguably makes this easier than VMs along in the past) or form a
cloud coop, but absolutely don't be shocked when a business is taking money
off the table because they can.

Don't marry yourself to a provider, stay portable, it's just good risk
management. Pricing changes? Spin up a cluster elsewhere, migrate data,
migrate traffic, profit. The terms of the agreement can change at any time.

~~~
marcinzm
Different companies make money using different approaches and it's perfectly
valid to be upset at the approach a particular company is taking. Just because
they provide a service doesn't mean they'll try to f __* you over at every
chance. For some it 's bad long term business to do that.

AWS, for example, begins with high prices and then lowers them over time. It
costs money but you know the maximum.

Google seems to be grabbing you with cheap prices and then jacking them up
when you're committed (google maps is another example offhand). Maybe no on
purpose but bad initial pricing and ill-intent have the same impact
externally.

~~~
toomuchtodo
> AWS, for example, begins with high prices and then lowers them over time. It
> costs money but you know the maximum.

Past performance is no guarantee of future benevolence or reasonable behavior.

~~~
matwood
> Past performance is no guarantee of future benevolence or reasonable
> behavior.

Sure, but pushing prices down and eating competitors margin to the benefit of
the customer has been Amazon's MO since inception.

Google seems to make a lot of missteps wrt pricing and the cloud. Remember the
geo pricing change that put a lot of projects out of business??

~~~
toomuchtodo
Amazon's bandwidth charges are still extortionate. Have those ever gone down?

------
zinlencer
I guess I won't be using GCP then.

------
cosmotic
Alternatively, don't use kubernetes.

[https://doineedkubernetes.com/](https://doineedkubernetes.com/)

~~~
nineteen999
Thems fightin words around these parts.

------
larosalia
That's it, I'm out of GCP.

They could at least start working on fixing their horrendous documentation.

------
msohn
Try project Gardener.

It's fully open source and uses kubernetes to run kubernetes control planes
and manage underlying infrastructure across many infrastructure providers.

Manage homogeneous kubernetes clusters across Azure, AWS, GCP, Alicloud,
OpenStack, VMWare at scale. Kubernetes on bare metal with Packet Cloud or
using open source metal-stack coming soon.

Extensible for other infrastructures, contribute support for your favorite
infrastructure.

Automation of day 2 operations e.g. etcd management including automated
backup/restore.

Choose kubernetes version, DNS provider, operating system, network plugins,
container runtimes.

Extended cluster services like DNS management and TLS certificate services.

[https://gardener.cloud/](https://gardener.cloud/)
[https://github.com/gardener](https://github.com/gardener)
[https://kubernetes.io/blog/2018/05/17/gardener/](https://kubernetes.io/blog/2018/05/17/gardener/)
[https://kubernetes.io/blog/2019/12/02/gardener-project-
updat...](https://kubernetes.io/blog/2019/12/02/gardener-project-update/)
[https://landscape.cncf.io/selected=gardener](https://landscape.cncf.io/selected=gardener)

[https://github.com/gardener?q=extension-
provider](https://github.com/gardener?q=extension-provider)
[https://github.com/gardener?q=gardener-extension-
os](https://github.com/gardener?q=gardener-extension-os)
[https://github.com/gardener?q=gardener-extension-
networking](https://github.com/gardener?q=gardener-extension-networking)
[https://github.com/metal-stack/gardener-extension-
provider-m...](https://github.com/metal-stack/gardener-extension-provider-
metal)

[https://www.packet.com/developers/integrations/container-
man...](https://www.packet.com/developers/integrations/container-
management/gardener/) [https://github.com/metal-
stack](https://github.com/metal-stack)
[https://knative.dev/docs/install/knative-with-
gardener/](https://knative.dev/docs/install/knative-with-gardener/)

------
dead_man
slowly killing the competition

------
justlexi93
A very basic truth that AWS understands is that you introduce a product at
price X, and then either keep it at X or reduce the price moving forward. You
never, ever, EVER increase prices on established products. You also basically
never discontinue products, even if there's only a handful of customers still
using it (ex: SimpleDB).

Apparently it's just too tempting to Google management to juice revenue in the
short term, but screw their customer trust and adoption in the longer term.

~~~
qzx_pierri
We see it again and again and again and again. I’m amazed that people still
put trust into Google’s services.

------
battery_cowboy
I saw someone mention Digital Ocean k8s products, right now Linode has a
Linode Kubernetes Engine (LKE) "Beta" program going on where they give you
some credits to fool around. I've been using it to learn k8s myself, and it
couldn't be easier to use for a beginner.

[https://www.linode.com/products/kubernetes/](https://www.linode.com/products/kubernetes/)

I'm not even going to post or give anyone my Linode referral code, because I
think their product is so good I'm happy to promote it for free, for no
credit, especially if it can help someone leave Google for good.

------
rajacombinator
What’s the betting line on GCP shutdown? Q3 2021? Q2?

