
Extending per-second billing in Google Cloud - boulos
https://cloudplatform.googleblog.com/2017/09/extending-per-second-billing-in-google.html
======
mythz
The primary driver that's going to get me to switch to Google Cloud from AWS
is their simplified pricing. AWS's reserved pricing is fraught with
inefficiency and prone to manual errors where it looks like they've
intentionally made it hard to determine what's covered by reserved pricing.

Everything else in AWS looks to be automated except for their pricing which
I'm assuming they're avoiding adopting an automated sustained usage model so
their prices appear cheaper then they are by taking advantage of manual
inefficiencies from trying to map reserved pricing to services used.

~~~
ChrisAntaki
If you have time, please elaborate on what you'd like to see.

~~~
mythz
A "no-touch" effort-free pricing model similar to GCP's sustained usage where
the hourly rate automatically drops based on continuous usage for each month.

Reserved pricing is an unnecessary waste of time/effort turning us into manual
book keepers where we have to try carefully match our reserved pricing plan to
each service we use. It's time consuming and frustrating when you purchase the
wrong reserved pricing SKU or are not notified when you have instances not
covered by a reserved pricing so you end up paying the highest rate despite it
running 24/7.

It's frustrating enough that I've started moving servers that don't require
access to other AWS services to hetzner.

~~~
endymi0n
...and EC2 reservations are still a Godsend compared to some other offerings
out there, I mean at least they're interchangeable by node class:

We've got Redshift reserved instances and due to growth, we need to upgrade
our node type soon (even in the same class!!). But because you _can 't reuse,
recycle or sell_ the reservations between node types, we have to let all of
our small node reservations expire before, complicated by the fact they were
already bought in several layers.

There's _no_ reserved instances market for Redshift (all of this looks like
not very well thought out from the beginning) and it took us hours to finally
come up with a semi good plan to do this (which will still cost thousands of
dollars in lost expired instances). All of this was completely unproductive
time in regards to our product, so the antithesis of what AWS stands for.

Managing financial matters on AWS is such a royal PITA, I'm so glad we
switched 90% of our stack to Google.

They even give you a big fat yellow warning sign directly in the console:
"Want so _SAVE_ money? One click here and we'll resize your instance online
instantly." Leave it on for a month and get nice discounts automatically. It's
heaven.

(Yes, BigQuery. I'd _LOVE_ to, however there's a lot of very good reasons
we're still on Redshift...)

------
dantiberian
I think dropping the minimum charge to one minute will turn out to have a
bigger effect than the per second billing. I build app images with Packer, and
these generally take a few minutes, but I'm charged for the full ten. It's
nice to know only pay for exactly what I'm using.

It also opens up opportunities for spinning up a fresh VM for every CI build.
GCP startup latency is pretty good, so this might be doable for some CI
systems.

~~~
andoma
Even though a GCE instance can be made to boot pretty quicky it's a bit sad
that it takes around 40-50 seconds before any network connectivity can be
established.

I asked a question about it here a while ago
[https://serverfault.com/questions/845298/no-network-
connecti...](https://serverfault.com/questions/845298/no-network-connectivity-
until-one-minute-after-boot)

Checked again just now, approx. 40 seconds between pressing START in the
console UI and until I can ping 8.8.8.8, using serial console on Debian
Stretch.

If anyone have a clue how to improve this I'd be most happy to hear about it.
I'm using this for CI builds.

~~~
boulos
We're working on it. It's a major initiative for us, because as you see we get
the damn thing "booting" in a handful of seconds and then reachability to/from
the internet is the long pole. Fwiw, we at least got to/from *.googleapis.com
way down, so if you need to say fetch something from GCS, that should be a bit
faster.

Disclosure: I work on Google Cloud.

~~~
_asummers
Curious (if you can go into it) what the current technical hurdles are in
getting the connectivity times down, and/or what you did to get the internal
connectivity going quickly.

~~~
boulos
At a high level, it's the result of having global, flat Networks and not
wanting to declare the network "up" until you've "programmed" all the routes.
So if you have 1000 VMs distributed globally, you get to make sure that
they're not "connected" until your new VM in asia-east1-a can talk to all
other VMs in your Network (and vice versa).

With the to/from API path this routing is much simpler since you don't get the
N^2 behavior.

Again, Disclosure: I work on Google Cloud (but I've never contributed
engineering-wise to Cloud networking).

~~~
_asummers
Would that connectivity happen regardless of firewall rules and such (thinking
network tags specifically)? Like if I have 1000 VMs as in your example, and I
spin up a new box but have a tule that says only one of those 1000 can talk to
it, would the route programming still go through and assert that connectivity
to all 1000? Or would it be smart enough to realize that only one of those
needed to connect and finish more quickly? If the latter, what happens if I
then remove that firewall rule immediately after boot?

~~~
jsolson
Today's control plane is fairly smart about which routes are required for a
given config change event (there's a lot one could speculate about here,
especially around the word "required") -- fwiw, I _also_ don't work on the
control plane -- I have spent a lot of time on the GCE hypervisor network
dataplane. So a fair amount of hand waving follows -- just assume details are
missing because I don't know what they are :)

We aim for global convergence of network state as sort of an ongoing goal, but
it's a distributed system with failure domain isolation, so that goal is
necessarily flexible. There are different rates of convergence, and Solomon is
certainly right that routes to first party services are some of the easiest to
converge. Internet connectivity is to a certain extent the hardest thing to
converge, as in our premium tier (which up until recently was our only tier)
we aim to keep data on Google's network for as much of its journey as
possible. At the extreme, this means a lot of edge nodes learning that a given
external IP belongs to your VM.

Part of it is also just the mundane business of reconciling what a given
configuration event means and propagating that to interested parties. With
respect to your firewall example, it's really just another config change event
with some set of implications for routes that are added or removed.

Anyway, that's my hand wavy explanation. Hopefully it's helpful!

~~~
Lukas_Skywalker
OT: boulos/jsolson, thanks for being so open about the platform. Getting
information like this has become rare, but it makes a very interesting read.

~~~
jsolson
Certainly!

I love what I do, and I love talking about it to anyone who will listen. I
find Google's infrastructure is fantastically exciting -- sometimes too
exciting, rarely boring. Most of the time I wish I could share more. Always
happy to hear someone has found what I can share interesting!

~~~
_asummers
I'm not sure how well this would go over internally, but blog posts outlining
cool stuff in the Google infrastructure would be really helpful to others,
especially GCP users, to know how things work under the hood. I'd read that.
And not even just success stories. War stories and "we tried this but it
didn't work because X" stories are really valuable to others without Google's
scale. Just a thought :)

------
fhoffa
Specially interesting given other similar recent announcements, Google's per-
second billing also covers:

| "premium operating system images including Windows Server, Red Hat
Enterprise Linux (RHEL), and SUSE Enterprise Linux Server"

(disclosure: I work for GCP)

~~~
DannyBee
I'm completely unsure why that is worth special mention?

(disclosure: I don't consider what I do to be work)

~~~
rch
That's the only helpful use of the disclosure line I've seen in a few years ..
thanks.

~~~
unshift
many employers request that employees provide a disclosure when discussing
their products on social media

~~~
TheIronYuppie
We don't, but generally do it out of an abundance of caution.

(disclosure: I work with the above two)

------
nimos
Really enjoying this price and feature war between Amazon and Google. Now who
wants to drop their egress pricing to be reasonable?

~~~
boulos
Working on it! We released our new "Standard" networking tier last month _and_
switched to src x dst pricing that saved some people ~10-20% depending on the
src x dst pair.

Disclosure: I work on Google Cloud.

~~~
Veratyr
Are you ever going to be competitive with transit? I can buy gigabit from a
number of providers for $400/month, which saturated the entire time is 330TB
or $0.0013/GB. I can also see 10Gbps for $999, which is $0.00030/GB.

There are so many things I'd love to use Compute Engine for but storage is
obscenely expensive for my needs and egress pricing makes it impossible to get
data out after I send it in.

~~~
ucombinator
Although the difference isn't as big as you think: when you buy from an ISP,
you typically pay for a router (and space/power to host it), a $250/mo cross-
connect fee, two ports if you want reliability, and you have a >2x unit price
increase just from running a port at <50% average utilization. But yeah, it's
still cheaper if you have >1Gbps of traffic.

------
andrioni
One thing of note is now both Google Cloud and AWS charge at least one minute
per instance (AWS used to charge a full hour, and GCP ten minutes).

~~~
CobrastanJorji
Where'd you see 10 minutes? The very first sentence of the post says "one
minute minimum."

~~~
sagadotworld
He said it _used_ to be 10 minutes.

(disclosure: The cake is a lie)

------
nealmueller
AWS wasn't first with per second billing. As the GCP blog states, GCP
Persistent Disks had it since 2013.

(I work for Google)

~~~
RKearney
AWS is miles ahead in IPv6 adoption though.

Why do GCP employees always post in HN articles about how much "better" GCP is
than AWS or how GCP "did it first"?

Our company literally can't use GCP, full stop, because Google is completely
ignoring IPv6 support. I heard load balancers are now in beta, which just
terminates IPv6 and connects to the back end with IPv4.

When am I going to be able to make an outbound IPv6 connection from GCP?
Specifically the Compute Engine VMs.

~~~
samstave
Can you ELI5 what we ppl who are deploying to any cloud should have as First
Principles WRT IPv6 and why we should be architecting around it?

Disclosure: I like turtles.

~~~
blasdel
Apple's App Store review has a hard requirement that everything works on a
pure IPv6 network

~~~
soccerdave
The only thing that you have to do to satisfy this requirement is don’t
hardcode IPv4 addresses in your app. If you are using domain names for
everything then it doesn’t matter if those are only resolvable to v4
addresses.

~~~
snuxoll
Correct, using DNS on a IPv6 only network is enough since the point is to use
DNS64/NAT64 to handle the transition until services are IPv6-native.

~~~
samstave
Please unpack this comment...

------
plandis
Is it okay to have the comment section of a HN post be just one giant ad? The
majority of the posts seem to be Google employees.

There is basically no discussion. Unless we are counting misleading companions
between GCP products and their competitors as discussion. :-/

~~~
boulos
Disappointingly, there haven't been any material questions or comments
(perhaps because it's straightforward). I think it would actually be
inappropriate for us to have attempted to lead the conversation too much, so
_I_ prefer to respond to questions.

Disclosure: I work on Google Cloud.

------
MightySCollins
This feels very much like copying the person who copied you. I am not sure
that I like the lack of notice though (unless I missed something)

~~~
dantiberian
What is the issue with the lack of notice? While the net effect of this is
pretty small for most use cases, it's only going to be cheaper, never more
expensive?

~~~
jgritty
(disclosure: he works for Amazon)

j/k

------
fiokoden
Minimum 60 seconds = per minute billing.

Not per second billing.

Nice try google.

~~~
Buge
Same criticism can be applied to AWS.

[https://aws.amazon.com/blogs/aws/new-per-second-billing-
for-...](https://aws.amazon.com/blogs/aws/new-per-second-billing-for-
ec2-instances-and-ebs-volumes/)

------
zimbatm
translation:

we are pleased to announce that we're extending per-second billing across our
platform.

we were first to do it.

it's actually useless (but we were first!).

let me show you why: ...

but we are forced to do it because of <unnamed competitor>

bunch of ads of why we are cool and doing "real innovation", compared to that
stupid other competitor that forced us to enable per second billing.

we are still better than competitor so..

signup!

(disclosure: I _don 't_ work for Google)

------
nodesocket
Your move AWS. Google improves to per-second with a 1 minute minimum.

~~~
boulos
They announced last week :).

Disclosure: I work on Google Cloud.

~~~
nodesocket
Yes I know about AWS per second, I was being somewhat playful. Huge GCP fan.
Keep it up boulos and the rest of the GCP team.

