
Python 3 is now available on App Engine standard environment - stewart27
https://cloud.google.com/blog/products/gcp/introducing-app-engine-second-generation-runtimes-and-python-3-7
======
anteater_alex
We're a SaaS platform running on App Engine and were part of the early-access
program. Porting over our Python 3 based platform from App Engine Flex to AE
Standard took just a few hours - only tweaks needed were to the YAML config
files and rewrite our deployment scripts after creating a new environment and
project.

We still maintain a docker-based environment on AWS EBS which we were able to
port over as well. Overall a fantastic experience. The App Engine Standard
team provided stellar support too.

We did notice pure CPU performance on AE standard to be less than dedicated AE
Flex instances, but on a price/performance ratio AE standard is 10X better for
infrequently accessed micro-services which will allow us to migrate some of
these to standard for great cost savings - as well as ramp up auto-scale to
handle spikes automagically.

With Python 2 becoming officially deprecated in 2020 this means App Engine
Standard just got a huge boost in long term relevance. Great for developers
who want to deploy without having to deal with infrastructure.

Good to hear that AE standard - i.e. this vision of a 0-effort Python
deployment support - is still relevant after all the Kubernetes-related
announcements at Google Next.

~~~
deepsun
Just curious, why did you migrate from Flex to Standard? Only for the 10X
price/performance boost for infrequently accessed services?

PS: Regarding automagically handling spikes I have a warning that might be
useful for you. The way AE load balancer works is that it:

0\. A request comes.

1\. AE checks for available instance. If none available:

2\. Spins up new instance.

3\. !!! Assigns the request to that instance.

4\. Waits for /_ah/warmup to finish.

5\. Adds that instance to its serving pool so other requests may be routed
there as well.

Note that step 4 is after step 3, so at least one request will wait for
/_ah/warmup. So occasionally you will see requests with high latency (if your
warmup is slow, of course). Our latency requirements were quite high, so we
ended up switching to manual scaling and heavily over-provision to handle
spikes. That costed us good money, though.

~~~
thesandlord
This should be a little better now:
[https://cloudplatform.googleblog.com/2018/05/Increase-
perfor...](https://cloudplatform.googleblog.com/2018/05/Increase-performance-
while-reducing-costs-with-the-new-App-Engine-scheduler.html)

But yeah, you should definitely do your own load testing. GKE or Flex are
probably better choices if you need more control.

(I work for GCP)

------
indosauros
> At this time, the original App Engine-only APIs are not available in Second
> Generation runtimes, including Python 3.7.

This means that if you already have a significant app running on Appengine
Standard, none of the built-in APIs are supported (memcache, images, search,
task queues, email, etc). That's a significant blocker to anyone wanting to
upgrade their existing Standard 2.7 app to 3.7.

~~~
anteater_alex
Correct, but for us (migrating from AE Flex to Standard) it meant that it was
working out of the box.

~~~
indosauros
It seems to me this product is mainly targeted at 1) new customers, and 2) AE
Flex customers wanting to switch to Standard.

The third category (Existing 2.7 Standard customers wanting to just upgrade to
3.7 -- which is me unfortunately) are going to be disappointed and stuck in a
hard place, I think.

~~~
dudus
What would be the incentive to migrate from AE Flexible to Standard if these
apis are not there? The only thing I can think of is scale to 0.

~~~
thesandlord
Scale to 0, faster scaling up/down (no need for full VM spin up), faster
deployments, less surface area for attack (maybe?).

------
demygale
Thanks for supporting this cutting edge technology. I really hope it catches
on.

~~~
make3
I can't tell if you're sarcastic or not. Python 3 is pretty old now,
definitely not cutting edge technology.

~~~
4di
Pretty sure they're being sarcastic. But I'm guessing the commenter hasn't
used Google Cloud anyway - Python 3 was supported on Flex enviroments for a
long time. The news here is that they're supporting it on Standard
environments, and there seems to be good technical reasons for taking so long.

------
sayhello
Exciting release of the 2nd generation of App Engine runtimes. I'm really
excited to see how the community receives this product.

It uses the gVisor sandbox, allowing sandboxing at the process level. This in
turns allow for fast turning up and scaling of apps.

Disclosure: I work for Google, on the App Engine Runtimes team.

------
stewart27
Hi all,

Product Manager for App Engine here. Happy to answer any questions about
Python 3 on the App Engine standard environment.

~~~
peacetreefrog
Can I run pandas in it?

~~~
stewart27
Yes, this runtime supports arbitrary dependencies specified via a
`requirements.txt` file. If you find a library that doesn't work, please let
us know. In general, anything that works using an open source Python 3.7
distribution should be supported.

~~~
aleaxit
Incidentally, the best way to "let us know" about bugs in GCP (App Engine or
whatever else) is via the Issue Trackers, as per
[https://cloud.google.com/support/docs/issue-
trackers](https://cloud.google.com/support/docs/issue-trackers) (disclaimer:
making this and similar processes work is my current job at Google).

------
Rapzid
I believe the title should be updated to say 3.7 as the article title does.
The .7 is pretty significant to those who would be interested in this.

~~~
e12e
Any highlights of why 3.7 is significant vs 3.6,3.5,3.4?

~~~
aleaxit
If I had to guess, I'd say the most relevant changes for App Engine use may be
the optimizations documented at
[https://docs.python.org/3/whatsnew/3.7.html#whatsnew37-perf](https://docs.python.org/3/whatsnew/3.7.html#whatsnew37-perf)
: the `sort` method of lists about 50% faster in common cases, the `copy`
method of dicts up to 5 times faster, etc.

------
rukittenme
Out of curiosity, why did this take 10 years?

~~~
stewart27
Long story short: security is hard.

Launching these new unmodified Second Generation runtimes required us to
develop new security and isolation technology (based on gVisor [1]). This
allows us to securely run arbitrary code on shared data centers with isolation
guarantees. This took us significantly longer than expected. The good news is,
now that we have this new stack in place, we should be able to deliver runtime
updates significantly faster.

[1] [https://github.com/google/gvisor](https://github.com/google/gvisor)

~~~
makmanalp
In case anyone else is wondering why you'd not just use LXC:

[https://cloud.google.com/blog/products/gcp/open-sourcing-
gvi...](https://cloud.google.com/blog/products/gcp/open-sourcing-gvisor-a-
sandboxed-container-runtime)

That said, they don't quite go into the details of what type of isolation is
missing from standard containers - I'm curious. It does seem like it would
have been ideal for everyone if LXC would have had better isolation, rather
than having to run a userspace kernel emulator thingy for each container, but
c'est la vie!

~~~
iangudger
I work on gVisor. The answer is that having a separate kernel is required to
achieve a high degree of isolation and by definition Linux containers share a
kernel with the host. A separate Linux kernel could work as well, but gVisor
tries to achieve a different set of trade-offs.

------
lostmsu
.NET Core?

------
wasd
Where can I find more information on Cloud Scheduler & Tasks? Having a hard
time googling it :) and the links go to a sign up form.

~~~
aleaxit
For Cloud Tasks, [https://developers.google.com/api-client-
library/python/apis...](https://developers.google.com/api-client-
library/python/apis/cloudtasks/v2beta2) should help.

~~~
wasd
Thanks! Unfortunately, there's a link to Cloud Tasks API on that page that
404s and there's not much else info on it. As far as I can tell, it's alpha.
Maybe I can find the Google Next presentation on it.

~~~
aleaxit
The links to APIs I see on that page are
[https://developers.google.com/resources/api-
libraries/docume...](https://developers.google.com/resources/api-
libraries/documentation/cloudtasks/v2beta2/python/latest/) and
[https://developers.google.com/api-client-
library/python/](https://developers.google.com/api-client-library/python/) ,
both work for me. The latter in turn points to
[https://cloud.google.com/tasks/](https://cloud.google.com/tasks/) (Tasks
Alpha) which also works for me. Maybe it only works for customers enrolled in
the limited-access Alpha and as a Googler I'm automatically enrolled without
knowing that...? But, I can also see the latter in an incognito window in
Chrome...

~~~
wasd
Last link doesn't work for me. Tried multiple devices and browsers. Just 404s.
I didn't find the first two links very helpful.

------
chocks
Would be nice if Python was available in the cloud function environment.

~~~
stewart27
This is supported today!
[https://cloud.google.com/functions/docs/writing/](https://cloud.google.com/functions/docs/writing/)

~~~
ProblemFactory
Do you have suggestions on whether to deploy simple (functional - no state or
external resources used, except perhaps usage tracking) APIs as cloud
functions or on the App Engine? What would be the "turning point" when it's
best to start considering App Engine?

------
qtXJ9EM
Awesome! Have really been looking forward to this release.

------
make3
It wasn't already?

------
qtXJ9EM
Awesome!

