
Python 3 on Google App Engine flexible environment now in beta - arouzrokh
https://cloudplatform.googleblog.com/2016/08/python-3-on-Google-App-Engine-flexible-environment-now-in-beta.html
======
jo909
I'm really glad this big fat blemish is now finally going away.

Anybody has a reasonable theory for why this took so long, despite users
asking for it for years?

~~~
theptip
Especially amusing was that Guido was working on the GAE team for a while, and
even that wasn't enough to get them to support Python3.

~~~
branchly2
I wouldn't say amusing, though it is interesting, and taking note of Go and
Dart, it may reflect what role Google thinks Python plays.

------
pokstad
Does anyone have any numbers on the cost difference between standard vs
flexible app engine usage?

Flexible environment supports any language/environment you want as long as you
implement the HTTP endpoints it expects. I would be much more impressed if
they brought Python3 to the standard environment which seems to be much more
cost effective for many types of web apps. Flexible environment essentially
requires you to run a VM all the time, which can get pricey, versus a much
more managed sandboxed environment that can sleep and be awaken quickly as
needed to serve requests. Also, Python 3.5 would be awesome to leverage
asyncio, but that's probably still a bit too bleeding edge.

~~~
kujenga
We use a combination of both at my company. We use the standard environment
for the majority of our services, doing things like proving APIs, serving
webpages, performing datastore interactions, etc. The flexible environment
starts to make a lot of sense when you're doing heavier weight processing,
text analytics in our case. App Engine instances are relatively cheap when
you're using the smallest instance sizes, and have even more benefits if your
application has idle time when they can spin down completely, but costs start
to go up very quickly if you need to move to the bigger instances in the
standard environment [1]. The flexible environment is current billed at the
rate of the machines you're using, so is much cheaper for bigger instances
[2].

[1]
[https://cloud.google.com/appengine/pricing#standard_instance...](https://cloud.google.com/appengine/pricing#standard_instance_pricing)

[2]
[https://cloud.google.com/compute/pricing#predefined_machine_...](https://cloud.google.com/compute/pricing#predefined_machine_types)

~~~
alooPotato
From the pricing docs you qouted - the smallest standard instance is an F1
which costs $0.05/hr and has a 600mhz cpu + 128mb ram.

Compare this too flex VMs where the smallest machine is $0.038/hr and have
beefier CPU and 3.75Gb of ram.

Doesn't it always make sense to use the flex VMs? The only downside being min
1 instance and slower autoscaling.

~~~
kujenga
waprin's comments are on the nose with what we've experienced.

There are pain points with the flexible environment that we don't have to
worry about at all with standard runtimes. The two biggest ones for us are
slower deployments and a lack of monitoring tools.

Just today we actually switched one of our App Engine modules to Flexible
runtimes out of necessity as we we're hitting quotas for App Engine
infrastructure, and in doing so we no longer have easy access to metrics on
how many instances are running or memory usage for the service. The "Cloud
Trace" tools are also impaired when you aren't using standard runtimes. In
short, you lose visibility into your instances. It ends up being a tradeoff
between developer time and cost.

With certain runtimes in the flexible environment, you also lose out on pieces
of the App Engine infrastructure like memcache, which are invaluable in
optimizing services.

~~~
gbin
We preload a memcache proxy on your VMs for your non-gae runtimes:

[https://github.com/GoogleCloudPlatform/appengine-sidecars-
do...](https://github.com/GoogleCloudPlatform/appengine-sidecars-
docker/tree/master/memcache_proxy)

In a nutshell, you can use a standard memcache library from your favorite
language.

Disclosure: original author of this proxy when I was on the App Engine team.

------
rch
> you can use the tools and databases you already know and love

But not Postgres or Redis?

Edit: OK, I'm seeing some references in the docs. Still seems limited to
deployment on generic instances though.

~~~
jonwayne
You can use Postgres and Redis. :) I'm working on writing new tutorials for
both of those, but in the meantime:

You can host your own Postgres on Compute Engine [1] or use a hosted provider
like ElephantSQL [2]. You can use whatever client library you want to connect
to postgres in App Engine flexible.

Same story for redis - you can host it on Compute Engine [3] or use something
like redislabs.

[1] [https://cloud.google.com/solutions/set-up-
postgres](https://cloud.google.com/solutions/set-up-postgres)

[2] [https://www.elephantsql.com/](https://www.elephantsql.com/)

[3]
[https://cloud.google.com/launcher/?q=redis](https://cloud.google.com/launcher/?q=redis)

[4] [https://redislabs.com/](https://redislabs.com/)

Note: I'm in Developer Relations for Google Cloud Platform

~~~
dabeeeenster
How do you deal with securing the DB's in this setup? Is it possible to
isolate the compute engine DB ports to only your appengine instances, or do
they have to be open to the whole world?

~~~
waprin
App Engine Flexible runs on GCE instances, which all have internal IPs. So you
can just firewall off external traffic but allow internal, which is the
default anyway.

Disclosure: Also GCP Developer Relations

~~~
dabeeeenster
OK great - thanks. Just to be clear, you'd still be opening up DB ports to all
internal global GCE instances?

~~~
waprin
Global GCE instances within your project, yes. Not every GCE instance
everywhere, that would not be very helpful :)

If you wanted to have some instances that couldn't talk to your database,
easiest thing to do is probably throw them in a separate project or create a
new subnet that you firewall off and create your non-App Engine instances in
that subnet.

------
coldtea
This changes everything!

Just kidding. Python 3 adoption will continue to be slow, and judging from
PyPI stats, nothing to write home about even atm.

~~~
dswalter
I wish there were a clearer delineation between developer downloads on PyPI
and automated builds, since it seems there's been a bit of a cultural sea
change over the past year or so in favor of Py3. I'd love to know which
version humans are downloading.

~~~
coldtea
Wouldn't Python 3 ALSO have installations doing automated builds though?

~~~
gbin
Yes but if automated pip installs numbers are way larger than manual installs
you end up with projects supporting both with roughly 50-50 numbers.

I agree with the parent post: now is the time. Open source projects should to
stop supporting py2 in their new major release and cleanup their codebase from
the hacks made to support both languages.

~~~
coldtea
> _I agree with the parent post: now is the time. Open source projects should
> to stop supporting py2 in their new major release and cleanup their codebase
> from the hacks made to support both languages._

The danger with this for Python is that it can give the impetus (is that the
word?) and motivation for people with py2 legacy projects (which is the
majority of people) to say "fuck this" and go try another language that
doesn't pull this shit on them. Go, JS with Node, Ruby, whatever...

If they're not gonna get compatibility anyway, they might jump ship
altogether. At least for green-field projects...

------
ben_jones
Really cool. Would be interesting if App Engine becomes a spot for larger
framework driven web apps like Django, or if/when ruby is supported, ROR. It
could be a very efficient way of doing so because you could have a single
instance running with a gazillion gunicorn/unicorn worker processes to
distribute the load across processes.

~~~
rrdharan
Note that Django has worked on App Engine standard for many years now and in
fact the App Engine SDK still distributes a Django build.

More info on the different Django+AppEngine options here:
[https://cloud.google.com/python/django/](https://cloud.google.com/python/django/)
[https://code.djangoproject.com/wiki/AppEngine](https://code.djangoproject.com/wiki/AppEngine)

Disclosure: I'm an engineer at Google.

------
borplk
I recently signed up for a trial and tried to use Google Cloud Platform. Oh my
god the UI was amongst the worst I have ever experienced on the web.

Super slow and hard to navigate. Stupid "tutorial". I tried for a good two
hours, tried to accept it but finally gave up on it.

Google, cut down on your Angular and material design kool aid and try harder
next time.

~~~
brianwawok
Have you compared it to AWS?

~~~
borplk
Yes I have used AWS a lot and been happy with the UI, it gets the job done.

Google Cloud Platform UI (and speed) was just actively hostile towards me.

For example I went to the App Engine section to create a simple flask app. It
keeps forcing you through a stupid tutorial trying to guide you around the
site. Believe me being a developer myself I'm no stranger to web and its
interfaces and can quickly navigate my way around this stuff but this had me
hitting my head on the desk.

