
All of Google's cloud database services are now out of beta - smb06
https://techcrunch.com/2016/08/16/all-of-googles-cloud-database-services-are-now-out-of-beta-and-nearline-cold-storage-gets-faster/
======
vgt
TL;DR:

1\. Cloud Bigtable is GA [0]

2\. Cloud Datastore GA [1]

3\. Cloud SQL 2.0 GA [2]

4\. Persistent Disk SSD now gets 25,000 IOPS [3]

5\. SQL Server images available [3]

6\. Custom Encryption Keys in Google Cloud Storage [3]

7\. Google Nearline Storage access latency now real-time [3]

POV:

\- Bigtable can reach some serious scale for Hbase-style workloads (ex:
Sungard got 34m reads per second and 23m writes per second[4] ).

\- Datastore serves almost 6m qps on average [1]

\- Cloud SQL 2.0 compares favorably against AWS RDS and Aurora [2]

\- Persistent Disk SSD compares favorably against options in the
marketplace[5]

\- Nearline is archival storage with real-time access characteristics

[0] ([https://cloudplatform.googleblog.com/2016/08/Google-Cloud-
Bi...](https://cloudplatform.googleblog.com/2016/08/Google-Cloud-Bigtable-is-
generally-available-for-petabyte-scale-NoSQL-workloads.html))

[1] ([https://cloudplatform.googleblog.com/2016/08/google-cloud-
da...](https://cloudplatform.googleblog.com/2016/08/google-cloud-datastore-
serves-over-15.html))

[2] ([https://cloudplatform.googleblog.com/2016/08/Cloud-SQL-
Secon...](https://cloudplatform.googleblog.com/2016/08/Cloud-SQL-Second-
Generation-performance-and-feature-deep-dive.html))

[3] ([https://cloudplatform.googleblog.com/2016/08/why-Google-
Clou...](https://cloudplatform.googleblog.com/2016/08/why-Google-Cloud-
Platform-is-ready-for-your-enterprise-database-workloads.html))

[4] ([https://cloudplatform.googleblog.com/2016/03/financial-
servi...](https://cloudplatform.googleblog.com/2016/03/financial-services-
firm-processes-25-billion-stock-market-events-per-hour-with-Google-Cloud-
Bigtable.html))

[5] ([https://medium.com/google-cloud/new-google-cloud-ssds-
have-a...](https://medium.com/google-cloud/new-google-cloud-ssds-have-amazing-
price-to-performance-2a58e7d9b433#.3ckq4e4iz))

(Disc: work on Google Cloud)

~~~
jhgg
Does the SSD iops increase happen automatically? Or do we have to turn it on
somewhere or recreate the disk?

Also any plan to allow objects in GCS bucket to automatically move to nearline
after a specified time, like s3 to glacier?

~~~
thesandlord
Should be automatic. Remember your max IOPS are also limited by core count.
[https://cloud.google.com/compute/docs/disks/performance#rela...](https://cloud.google.com/compute/docs/disks/performance#relationship_between_size_and_performance)

~~~
jhgg
Makes sense. Did the iops cap for fewer than 15 cores increase?

~~~
boulos
15000 was the previous limit for any instance type. I think it's correct to
say: it's the same as before for smaller instances, and has been increased for
bigger ones.

------
mark_l_watson
I am a fan, and a user, of Google Cloud, Azure, and AWS: because of extra APIs
and services.

However I find the price differences for plain VPSs between these platforms
and more bare bone VPS providers like OVH and dedicated server providers like
Hetzner to be surprising. I understand that sometimes you get what you pay for
in reliability, fast networks, etc. but still the price difference is huge.

I like having a single large memory, multiple core VPS for fast development
and earlier this year I would spin a GCE instance up and down as needed. I
transitioned several months ago to an always on large memory OVH VPS, much
more convenient. When I need more cores and memory Hetzner has incredible
pricing on dedicated servers.

------
samhamilton
Dear Google, pretty please add some Postgres support

~~~
mythz
+1 don't see why they're putting in the effort of providing managed SQL Server
hosting when PostgreSQL still needs doing.

~~~
numbsafari
A lot of companies use SQL server. I wouldn't be surprised if it has double
the installed base of PostgreSQL. Especially among the more established firms
that Google seems to be targeting in their sales effort.

~~~
redwood
Double? More like 20x

------
mxuribe
This is good news; even if only for upping the competition with azure and aws.
The more _legitimate_ options in the cloud that exist for enterprise, etc.,
the better!

------
jjnoakes
I've noticed what I will call inconsistencies with some of Google's free
products.

I see see space taken up in my drive account by pictures but I can't find them
in my Google pictures, drive, hangouts, Picasa Web albums, nowhere.

I also see things via the drive API that were shared with me long ago, but
which I can't access (or remove) via the web interface.

These don't give me confidence in the overall infrastructure.

Has anyone noticed anything similarly disconcerting about their cloud
services? Or are those more stable?

I'd like to migrate some work to them but I'm justifiably uneasy about what
quality to expect. Are things more solid in the paid space?

~~~
quwert95
>I see see space taken up in my drive account by pictures but I can't find
them in my Google pictures, drive, hangouts, Picasa Web albums, nowhere.

I ran into a similar issue with Google Drive/Gmail. Have you checked the
trash? Things stay in the trash until you delete them (and some Google
services are configured to keep things in the trash for up to 30 days for
automated pruning). The 'quota' page may help you pinpoint where the space
takers are hiding:
[https://drive.google.com/drive/u/0/quota](https://drive.google.com/drive/u/0/quota)

~~~
jjnoakes
Yep, trash was one of the first things I checked.

------
caseyf7
However, their billing system is definitely not out of beta.

~~~
kyrra
Care to elaborate on the issues you have with GCP billing?

~~~
markdown
I manage a few accounts for different clients of mine. Now and then (not
often) I get an email about some billing service being discontinued (or
perhaps a free trial of something has ended).

When this happens, I have no idea which account it refers to as the emails
provide no details. I just have to cross my fingers and hope nothing goes
offline.

EDIT: Here's what they look like:
[http://i.imgur.com/XcatLQm.png](http://i.imgur.com/XcatLQm.png) AFAIK the
account id denoted there as XXX-XXXXX-XXX is some behind-the-scenes account
identified that isn't displayed anywhere in the UI. To hunt it down, I would
have to log into a google account, then click on the link. Then repeat over
and over with a dozen google accounts until I log into the one actually tied
to that billing account.

Ridiculous.

~~~
kyrra
Is the id there the billing id listed here[0]? AdWords does something similar
but their account ids are sorter.

One hack that may work: use + in the name part of your email (ex:
name+account@example.com). Gmail at least routes them all to the "name"
account. Then you can filter or label as you would like.

[0]
[https://console.cloud.google.com/billing](https://console.cloud.google.com/billing)

~~~
markdown
> Is the id there the billing id listed here[0]?

Looks like you're right. It's still a meaningless number in this context. It's
not something I can or should have to memorise. To make it worse, an account
can have multiple billing accounts.

The suggested hack should work, thanks. Hopefully Google improve the content
of their email though. Hacks shouldn't be needed.

------
a_imho
Does Google offer any sensible support though without paying consultant
prices? My pain with Google services that it is very rare to have any contact
address where you can get more than basic information. No, clicking through a
FAQ does not help, not to mention when you are forwarded to out of date pages
because Google decided to override your language of choice.

~~~
brianwawok
If you want free support you generally do the mailing list. The times I have
asked I got an exact answer from someone on the team the same day.

------
myth17
On Cloud SQL : If you start with 10GB amount of storage, you can always get
more storage but never decrease your storage amount. Why?

~~~
nordsieck
They almost certainly don't want to deal with races involving incoming data
and a request to decrease storage volume.

~~~
boulos
That's almost right. Most modern filesystems are okay with resizing upwards,
but very unprepared for downwards (even when lots of bytes are not yet
touched!). So it's really just the "restriction" from GCE that comes from
"normal file systems don't like this".

Disclaimer: I work on Google Cloud, but I'm not a filesystem expert.

------
ffggvv
Why is a business name required?

~~~
boulos
I'm guessing you're in Europe. First, let me say we're sorry. The explanation
unfortunately is VAT collection and reporting, which we don't (currently) do
for you. A business is already required to do so, but for individuals would
have to do their own / represent that they're doing so. My understanding is
that it's also per country, requiring either a per-country holding company or
some sort of extra accounting.

tl;dr: VAT (and yes we should handle this for you, but don't today).

Disclaimer: I work on Google Cloud, but I am not a tax lawyer (nor even a
lawyer).

------
kevindeasis
This is awesome. I hope they start releasing features about geoqueries for
datastore.

~~~
colink
While it is not part of Datastore, Cloud does currently support geographical
queries via the Search API:
[https://cloud.google.com/appengine/training/fts_adv/lesson1#...](https://cloud.google.com/appengine/training/fts_adv/lesson1#location-
based_queries_geosearch)

~~~
chippy
note that this is currently a very simple radius search for points. So it
cannot do point in polygon, intersections, allow shapes for data points etc

------
fasterthanlime
Sorry to hijack that comment thread but since the Google Cloud team is around,
who should I talk to about this?
[https://github.com/itchio/butler/issues/71](https://github.com/itchio/butler/issues/71)

Not seeing anything on the status page nor any outside confirmation, but many
independent reports from our users, all in EU pushing to US zones.

I'm looking forward to using more Google Cloud services, but unreported
(partial) outages are making me wary :(

------
perlin
Great, now fix PubSub and Stackdriver, and GCP starts to look attractive...

~~~
euyyn
What specific issues do you have with PubSub and with Stackdriver?

~~~
pacala
I've setup a Stackdriver account today. My colleagues [Editors on the
monitored project], when navigating to [0], were prompted to create a new
Stackdriver account. They could not see the one I just created, nor were they
presented with the GCP project we are using. There was no obvious way to
invite them to join the project, more generally there was no obvious way to
manage user accounts, though I could see and edit my Profile, which does show
[readonly] that I'm an "Admin", though I don't know how to make other people
"Admin", or what other roles are available. It was not obvious to check which
project "hosts" the Stackdriver account. There was no obvious link to a forum
or chat to discuss the issue, only a "Send Feedback" link who-knows-when
somebody will look at.

We eventually solved the issue by passing around [1], which is not obviously
displayed anywhere. I scrapped it by right clicking and <Copy Link Address>
somewhere in the UI [I forgot where].

I'm bullish on Google Cloud [founder on GCE & long departed, hi there guys],
so I put a solid 30 minutes of sleuthing in it, but someone less determined
would have gave up a whole lot sooner.

[0] [https://app.google.stackdriver.com](https://app.google.stackdriver.com)

[1] [https://app.google.stackdriver.com/account/login/<project-
id...](https://app.google.stackdriver.com/account/login/<project-id>)

~~~
pacala
PS. After getting past the initial hiccup, we've setup our basic monitoring
and infrastructure in a few hours. Coming to working with ops after a couple
years hiatus, Stackdriver is incredibly comprehensive and easy to use. Let's
see how the systems will hold moving forward.

------
douche
Next step, end-of-life...

~~~
daveguy
Normally I would agree, but not in this case. It is unlikely they will EOL
these services any time soon. They are paid services and could represent a
significant revenue stream for Google.

~~~
chippy
> _could_ represent a significant revenue stream

If it does not, we should, like other Google offerings, expect it to be cut.
Google has a strict internal business case (new from around 2010 or so) - what
they do now has to make money, it's no longer enough to be getting more people
to use the internet, or increase use in the brand, the services has to make
business sense.

Let's hope it is used and profitable, eh.

~~~
daveguy
> what they do now has to make money

That's a fair concern. However, as of 2015 they already have between 3 and 5 %
of the cloud market [0][1]. It is likely they are already profitable with the
usage they have (cloud is pretty high margin). There have been very few google
products that were officially taken out of beta. Google Search and Google Mail
are two of those.

True, Google has a terrible reputation for dumping small dev and consumer
facing products. Their cloud offerings are a completely different category.
Not least of which, because they have _no free tier -- only free trial_.
Anyone who continues to use Google Cloud products is paying for them.

[0] [http://www.businessinsider.com/why-amazon-is-so-hard-to-
topp...](http://www.businessinsider.com/why-amazon-is-so-hard-to-topple-in-
the-cloud-and-where-everybody-else-falls-2015-10)

[1] [http://dazeinfo.com/2015/05/22/amazon-aws-google-cloud-
micro...](http://dazeinfo.com/2015/05/22/amazon-aws-google-cloud-microsoft-
azure/)

