
Cloud SQL for PostgreSQL now generally available - CoachRufus87
https://cloudplatform.googleblog.com/2018/04/Cloud-SQL-for-PostgreSQL-now-generally-available-and-ready-for-your-production-workloads.html
======
optimusclimb
Started down the path of using this when it was in beta, but had to abort when
we saw there was no option to connect to it from Python App Engine Standard.

Now that it's GA...it looks like that hasn't changed. Is the classic, Python,
App Engine standard becoming a second class citizen? Or was there some reason
why this wasn't considered GA worthy for Postgres?

Trying to understand if going forward Google is trying to push everyone to the
flexible environment or not - as I would have really expected connectivity
between these two products.

~~~
litewulf
[I am a Googler and my team works on part of this]

You can connect to postgres from app engine standard... as long as its Java.
See this doc
[https://cloud.google.com/appengine/docs/standard/java/cloud-...](https://cloud.google.com/appengine/docs/standard/java/cloud-
sql/using-cloud-sql-postgres)

And no, appengine standard is not a second class citizen. Hand-wave-ily, the
connectivity path that flex uses works for postgres with minimal changes, but
unfortunately some additional work is required to get appengine standard for
other languages working for postgres. :(

~~~
kirubakaran
Thanks. Could you please explain in a little bit more detail how it works for
Java GAE Standard and not Python?

------
pritambarhate
Is point in time recovery available now in GA? I had checked it a couple of
weeks ago for the beta service and it was not available. I think for a managed
DB hosting point in time recovery is a critical feature.

~~~
flamingcow
[I'm the Cloud SQL TL] No, it isn't. We agree with you that it's an important
feature for managed databases, and we're working to get it right. We decoupled
it from this launch to get PostgreSQL to GA faster.

~~~
tacticus
Unrelated to pg but could you badger the spanner team to make a mini spanner
product :)

Related to postgres. We have many many concurrent connections but a load
satisfied by an n1-standard-4 atm do you recommend a connection pooler or
something to help us get down to the 100 to 200 connections we need to be at
to use cloudsql?

~~~
renchap
Connection pooling is recommended for any non-trivial PG deployment. I can
recommend pgbouncer, worked flawlessly for us.

------
kdv
Does anyone have insight or experience using this in production? We're
currently running PostgreSQL 10 w/ pg_partman on our own hardware but looking
at a several options for cloud migration. Unfortunately, Citus Cloud on GCP
doesn't appear to be an option (yet?)

\- Google Cloud SQL (PostgreSQL)

\- Citus Cloud (AWS Only)

\- Citus (managed ourselves) on GCP

~~~
manigandham
We did, but now use [https://aiven.io](https://aiven.io)

Highly recommended if you want a fast and featured managed db service.

~~~
misterbowfinger
what's the benefit of using them vs. AWS or GCP directly?

~~~
manigandham
Latest software (using v10.3), better performance (nvme SSDs), better backups
(point in time, instant cloning), better features (more extensions, cross-
region replication even across different clouds), better flexibility (migrate
master across different clouds), better monitoring (logs and datadog metrics
export), and more focused support with a smaller team.

~~~
bpicolo
AWS and GCP both have nvme SSDs for instance types intended for big-ol DBs

[https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ssd-
inst...](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ssd-instance-
store.html)

~~~
manigandham
Yes, I know. This thread is about managed database services, specifically
about Aiven vs cloud-direct. Aiven still runs VMs on AWS or GCP but offers
nvme disks while neither RDS nor CloudSQL have that available.

~~~
bpicolo
Ah I see, i3 isn’t in the RDS pool yet. I imagine it will be eventually?

------
manigandham
We actually just migrated off yesterday... we went with an independent vendor
like Aiven instead of the clouds because they move too slowly and don't have
enough features.

~~~
gingerlime
Like aiven or aiven? :) I’m not aware of too much competition in this space.
We’re happy with aiven so far, but they’re quite unknown / under the radar it
seems...

~~~
manigandham
They're the best we've found as well, but there are lots of managed database
providers of all kinds. [https://www.compose.com](https://www.compose.com) is
another big one.

~~~
gingerlime
We’re using compose for redis, but we might switch it to aiven as well,
despite aiven having some missing functionality...

For Gcloud and especially in Europe, I think these two are the only options
that I’m aware of (at least for PG and redis)

~~~
manigandham
If you want managed Redis, there’s nothing better than RedisLabs

~~~
gingerlime
Yeah, I was practically begging them to offer service on GCloud in europe ...
:) but they don't cover it...

------
zawerf
How do you deal with custom extensions (for example plv8)?

I started off with heroku and they don't support the same subset:

[https://cloud.google.com/sql/docs/postgres/extensions](https://cloud.google.com/sql/docs/postgres/extensions)

[https://devcenter.heroku.com/articles/heroku-postgres-
extens...](https://devcenter.heroku.com/articles/heroku-postgres-extensions-
postgis-full-text-search)

~~~
contrahax
They don't have a great story around extensions - the ones they do have are
unsupported/buggy. For example, PostGIS is missing "ST_FromGeoJSON" because it
was compiled with the wrong flag - and has been this way for over a year
despite hundreds of user complaints.

~~~
aserafini
This is what I expect from Google. They are not customer-centric and no-one is
really prioritising or cares about customer feedback.

~~~
TurningCanadian
You clearly have never talked to GCP support. They're in a different league
from AWS support.

~~~
manigandham
They're all the same quality. Sometimes great, sometimes terrible, but will
get the job done as long as you put in the effort. Also bigger customers will
obviously get more.

GCP does win on pricing now with role-based support that is a flat rate per
user: [https://cloud.google.com/support/role-
based/](https://cloud.google.com/support/role-based/)

------
brightball
Will we be able to use the Citus extension? Been itching to get Citus running
on GCP.

~~~
jadbox
Does any major cloud provider support it?

------
eloff
I'm looking around for more details on these "regional" disks that replicate
between two zones at the block level. Is that just a fancy term for os level
mirrored disks using the cloud persistent disks?

~~~
laixer
There's a previous blog post[1] about HA replication. It uses block-level
replication managed by PD infrastructure.

[1] [https://cloudplatform.googleblog.com/2017/11/Cloud-SQL-
for-P...](https://cloudplatform.googleblog.com/2017/11/Cloud-SQL-for-
PostgreSQL-adds-high-availability-and-replication.html)

~~~
nh2
Block device based replication for Postgres seems a bit unconventional given
that Postgres has native synchronous replication support with WAL streaming.

Intuition tells me that you might get better performance if you let the DB
itself do the replication but I can't really justify that without real review
of what happens.

The postgres docs ([https://www.postgresql.org/docs/10/static/different-
replicat...](https://www.postgresql.org/docs/10/static/different-replication-
solutions.html)) say that the WAL solution has no "Master server overhead" in
contrast to the File System Replication solution, but it's not explained and
I'm not sure what is meant by that.

I guess with a block device based solution, recovery takes longer, because
failover entails you have to actually mount the block device (as no 2 machines
can mount it rw at the same time), and then start the DB (or in a more basic
implementation, just boot the entire second machine as part of failover),
while with WAL streaming both postgres instances would already be running? Wo
failover would be faster with WAL streaming?

I would be great if somebody from GCP could elaborate what the tradeoffs here
are, how long failover takes, and whether we can expect similar performance
and behaviour as with WAL shipping.

~~~
Erwin
Amazon's Aurora Postgres database does a similar thing: your master in one
zone replicates to a disk that is in all the other zones. Unlike normal
Postgres RDS instance it also auto-scales storage to what you use.

Amazon claims better scaling then ordinary Postgres for this.

------
theptip
Great news! This has been a long time coming (over a year since the beta
started), looking forward to migrating my MySQL instances over.

------
kerng
Great to see all three major cloud providers offering this now. Should be good
for pricing!

------
lobster_johnson
Has anyone used the beta and got any feeling for how maintenance downtime
impacts things? A bit nervous about how you can only set a "maintenance
window" and not be able to plan ahead for disruption; as far as I can tell,
they won't even tell you ahead of time. The HA seems really solid (zero-lag
"regional disks"), but it's still a bit disconcerting.

~~~
daj40
The updates take the entire instance down for 2-5 minutes each month. While
you can't avoid them, they can be scheduled for particularly low traffic
times. If you're trying to avoid downtime, its a giant PIA. Even with HA
enabled, you still lose master, slave and read replicas. Not entirely sure
what they define HA as, but a mandatory monthly downtime doesn't usually fit
into mine.

[Update] That said, from what I understand, they have a road map to
maintaining read replicas and queued writes. Not sure what the date on it is
though.

~~~
flamingcow
[I'm the Cloud SQL TL] I can't comment on timelines, but we're aware that
customers are interested in more features around maintenance window
scheduling, deferral, and notification, as well as shorter downtime for
updates and smarter scheduling within a group of replicas.

~~~
andydd
Can you confirm that it's impossible to avoid downtime, even with HA, because
of forced updates?

Surely that's what HA is? no downtime as you update each node one at a time?

If it's impossible then it's a dealbreaker.

~~~
flamingcow
[I'm the Cloud SQL TL] Confirmed. We know it's a problem that we need to fix.
HA reduces downtime in unexpected failure cases (live migration for your
primary only helps in planned shutdown cases, not if the physical machine
fails), but doesn't currently help with maintenance-related downtime.

------
elvinyung
Postgres wire-compatible Spanner when? :P

~~~
Crash0v3rid3
Wondering what kind of use case you would have for this compatibility?

~~~
elvinyung
I know that it's definitely not going to be 100% the same (especially since
Spanner doesn't even support SQL DML right now), but I think a drop-in
replacement into a managed autoscaling database is a really nice alternative
to manual sharding.

Right now basically the options are Aurora, Citus, and running CockroachDB
yourself.

------
Lunatic666
Last time I was at Google for a workshop (If you have the chance to visit
Google, do it. The food alone is worth it), they didn’t seem to push CloudSQL
a lot, because they wanted to guide people more in the spanner direction.
Without a solid RDS counterpart however, I don’t think bigger companies will
consider moving from AWS. Happy to see they changed their mind and continue to
expand their SQL services. The competition by Google put a lot of pressure on
AWS who seemed to be gotten a bit lazy. Google was ahead of the game with
their global load balancers and network speed and quality. Now AWS countered
with their 5th series C5/M5, which solve the bandwidth problem of the smaller
C and M instances

------
renchap
Finally! Nice to see more PG support. Any news about PostgreSQL 10 support?

~~~
briffle
I asked that yesterday in a meeting with my gcloud rep, they said all work has
gone into getting to GA, and once that is done, look for them to start doing
things like updating, more new features, etc.

------
sirn
Since there seems to be some Googler who work on Cloud SQL here, I wonder is
there any chance Cloud SQL will be available in asia-southeast1 soon? It's the
only region (I believe) where Cloud SQL isn't available at all, and one of the
main reasons we can't fully migrated from AWS to GCP just yet.

~~~
litewulf
[I am a Cloud SQL-er, and worked on region expansion... among other things]

Cloud SQL has only been available in regions with at least three zones (since
we believe that is the minimum to make sure we can maintain HA in the event of
a single zone failure). asia-southeast1 currently only has two zones, when a
third zone is launched, Cloud SQL will become available in that region.

~~~
sirn
Thank you. That makes sense and totally understandable. Looking forward to
when there's third zone in asia-southeast1. :-)

------
cellis
I tried using this, but couldn't get the root certs to work. So I went to AWS
RDS instead. A pity, it's much cheaper ( especially in beta pricing )

~~~
litewulf
[Cloud SQL person, SSL+connections is my jam]

When you say you couldn't get the root certs to work... what do you mean?

Cloud SQL automatically generates server certificates, and we offer UI+API for
creating additional client certificates. The two should not share a root CA.

------
foobarbazetc
Do these use the standard persistent disks (capped at 240mb/s)?

~~~
litewulf
[I am a Cloud SQL-er]

Yes, you can use both standard and SSD persistent disks. If you create a
larger instance with more vCPUs and a big enough disk, you can achieve greater
than 240mb/s, see the docs:

[https://cloud.google.com/compute/docs/disks/performance#ssd-...](https://cloud.google.com/compute/docs/disks/performance#ssd-
pd-performance)

------
TeeWEE
How expensive is this compared to, for example AWS RDS?

~~~
michaelanckaert
The cost is virtually the same.

A db.t2.small instance should compare to a db-pg-g1-small instance. Pricing is
around 90$ on AWS vs 93$ on GCP.

I’m an AWS consultant so I could have messed up on the GCP instance type.

