Hacker News new | comments | ask | show | jobs | submit login
Beta release of PHP 7.2 in the Google App Engine standard environment (cloud.google.com)
185 points by sl0wik 5 months ago | hide | past | web | favorite | 63 comments

This is exciting because the second generation standard environment allows Google App Engine to provide a more "idiomatic" experience.

Using gVisor, instead of NaCL, as the sandboxing technology, shipping new runtimes becomes a much more tractable task with less differences from the "plain" runtime. In fact, gVisor allows us to use the runtime unmodified.

This in turn, decreases the "lock-in". There are still decisions we need to make depending on the runtime, e.g. which package manager to use if there isn't a community default.

Hopefully, the narrative for "Serverless" moves beyond lambda and functions. Sometimes (!) we want apps, databases, caches and more to be "Serverless" :-)

Disclaimer: I work at Google, on the App Engine Runtimes team. I helped ship this

Thank you for all your hard work! I'm so excited about thew new app engine. Pleaaase bring Ruby next :).

More runtimes coming out soon. Stay tuned :-)

Am I correct in understanding the new environment is built on essentially a ptrace sandbox? How does that impact performance? Is it still single threaded? If you're moved beyond single threads, how does the it cope with security problems endemic to userland sandboxing? (TOCTTOU issues as the 'hypervisor' verifies arguments, etc)

Are you willing to compare your approach to for example Lambda's approach, where the user gets an actual real Linux VM. with zero differences (Work that, incidentally was in part contributed to Linux by Google!), as opposed to a Linux emulator?

Hi. I'm the PM at Google Cloud on the Second Generation runtimes.

This is based on gVisor (https://github.com/google/gvisor), but it does not use the ptrace sandboxing that is in the mainline open source project. Our First Generation runtimes (aka nacl) were based on ptrace. In our informal testing we have observed the Second Generation runtimes to be about 20% faster than the First Generation. I don't have benchmarks on how it compares to a raw VM.

These new runtimes are capable of multithreading, so long as the language is capable of it. And you will need to use an instance larger than the default (F1 / B1) to have access to more than one thread.

Two have access to more than one hardware thread or more than one OS thread?

I work on gVisor. gVisor supports concurrent multi-treading, but should not be vulnerable to TOCTTOU, even in ptrace mode. gVisor handles system calls itself, it does not simply verify the args for app system calls and then pass them on to the host. That said, if you find any security issues, please let us know. There is a security email address in the github issue template.

App Engine Flex is a version of App Engine that uses VMs with Linux. You can find a comparison between the standard and flexible environments here:


gVisor can use KVM for isolation instead of ptrace, and I would guess that Google is using it.

Correct. We do not use ptrace in production.

Thanks for clarifying.. it sounds thoroughly tragic, and the kind of endemic overengineering that made so many people (myself included) run screaming from the original App Engine over the years. I guess GCP really turned a page in Google culture and learned to finally listen to customers

AWS> we put your code in a container, like everyone else puts your code in a container. We bikeshedded the Linux distro a bit, so you might need to recompile

Customer> OK. Fine

GCP> our web scale open source userland kernel hypervisor written in Go hosts your code in a Linux-but-not-quite-Linux microservice

Customer> Why does <X> system call return a strange error code?

GCP> You're too dumb to understand. Trust us

Customer> oh, I think I understand well enough. Calls back Azure sales rep

Random question, is it possible to have a python2 default service and a python3 service in the same app engine project?


What about search and cache?

it's still Python 2.7 only :(

dis·claim·er, noun, a statement that denies something, especially responsibility. [1]

Did you mean "disclosure"?

[1]: https://www.google.com/search?q=define%3ADisclaimer

I suppose colloquially on this forum in the context of comments, the word does deny something: usually of the possibility of bias.

Disclosure would work just as well, and is in fact more accurate!

> the word does deny something: usually of the possibility of bias.

That's precisely why I objected to its use in the context it was used here (and other places where it is colloquially used incorrectly): by announcing you worked on something, you reinforce the possibility of bias in the post while taking partial responsibility for the work--it does not disclaim or subtract bias or responsibility.

I made a logical error, I meant to say:

> the word does deny something: possibly objectivity.

i.e. there is a possibility of bias.

That said, I've made my peace about the "proper" use of language, or rather, the lack thereof, and now espouse it.

Like, literally.

The meanings of words and idioms shift over time. As long as we're understood well enough, that's fine by me.

Instead, could they not be dis-claiming their objectivity?

Not sure why you're getting down voted. Perhaps because people find the tone too snarky. You're right though. It's a mistake I see all the time here (and it secretly bugs me haha). Disclaimer in this case would mean: "I say this, but beware I'm not involved in this so I'm probably wrong". Which is exactly the opposite of what's intended here.

The state of App Engine environments seems super confusing, but I _think_ this uses the "second generation standard environment" like the Python 3.7 and Node 8 beta environments, which means that unlike App Engine of Yore, you have access to the full package ecosystem and full network access. I'm not actually 100% sure on this because it's not listed on the runtime page (https://cloud.google.com/appengine/docs/standard/appengine-g...), but the docs show that you can use composer, unlike the PHP 5.5 runtime.

The documentation for all these second generation runtimes is shockingly bad. I consider Google to be _generally_ pretty good at documentation, but this is one of those cases where their propensity for confusing naming and classification of their products makes it hard as hell to figure out what is going on. It doesn't help that the top-level "product overview" page is useless marketing junk (microservices! serverless!) instead of a practical description, which is buried several links deep.

Having never used App Engine but occasionally wandering into its docs when a new product gets announced like this, am I right in assuming that App Engine's "second generation standard environment" is more or less equivalent to the feature set you'd get from Heroku? The flexible environment also continues to confuse me - it talks about being a managed Docker container, but it doesn't seem like you actually get access to anything like a Dockerfile and that's more of an implementation detail on their end.

> The flexible environment also continues to confuse me - it talks about being a managed Docker container, but it doesn't seem like you actually get access to anything like a Dockerfile and that's more of an implementation detail on their end.

You're right that you don't get access to the Dockerfile. Docker isn't a 100%-isolated environment for multitenant use-cases, so Google needs to control the stack themselves, using specific trusted versions of base-image layers and app layers. There's maybe extra "Docker-image compile time" security restrictions made on top of that as well.

However, it's not just an implementation detail—it has pretty visible effects.

1. A Flexible Environment app slug gets baked into a regular Docker image and placed into your Google Container Registry alongside container-images you've built yourself with GCR. (And your GCR registry, in turn, is backed by a Google Cloud Storage bucket in your own account—meaning that you're paying for storing the resulting images just like any other uploaded objects. You should prune old App Engine Flexible Environment image versions if you don't want to pay!)

2. You can do whatever you want with the filesystem of your container, because it is just a Docker container. You can mount volumes to the container in the container spec (app.yaml), just like if you were writing a k8s container spec. (In fact, sharing IPC sockets through a Docker volume-mount is how the Google Cloud SQL proxy works.) You can internally auto-update your container by downloading new Python/Node/etc. code into the container and getting the interpreter to re-exec(2) itself. Pretty much anything that can run in a "Python Docker container environment" can run in Google's "Python Flexible Environment", because they're pretty much the same (save for those container-build-time security restrictions.)

Disclosure: I work for Google on the App Engine team doing documentation.

Firstly, a sincere thank you for reading and caring about our documentation.

With regards to the page you referenced, would you please elaborate on what information would help you come to the appropriate conclusion about using a second generation runtime? Your feedback would be very useful to us in order to shape this page into something that provides you with actionable information rather than "useless marketing junk" :)

Also to address a point you made in the first paragraph of your post, the PHP 7.2 beta runtime is a "second generation" runtime.


Hi, I couldn't agree more with the parent comment about the AppEngine docs. I built a fairly small app and it was utterly excruciating to sort out the nuances.

There should be right up front, a crystal clear page detailing the three possible environments.



- Standard Second Generation GVisor

For example, in Python Standard (old standard), it's extremely confusing to work out which gcloud libraries are bundled, and which need to be vendored and packaged into the app (yes I know there is a page that tries to explain. It's unclear).

Thank you for working on the documentation. It is the number one area that GCP needs to invest in. All the technology is miles ahead of AWS in the core areas. The docs however, are very very far behind.

Happy to elaborate more if there's any specifics you need.

> For example, in Python Standard (old standard), it's extremely confusing to work out which gcloud libraries are bundled, and which need to be vendored

I agree. I spent a lot of time trying to figure this out myself with various APIs, and in some instances, I gave up and looked for an alternative after I failed to figure out how to correctly vendor some Google Cloud libs (even following what documentation I did find, which made it look so simple).

Sure, and apologies for the harshness in my original post.

My main complaint about the marketing landing page at https://cloud.google.com/appengine/ is that it doesn't contain any immediate information about the different environments. It does have a link to "Choosing the right Environment" but it's _waaay_ down below the fold. In general I wish Google Cloud product pages had their big "View Documentation" buttons up at the top next to the free trial button, instead of way down below the pricing section.

In addition, neither the "Choosing the right Environment" nor the "App Engine Standard Environment" docs linked at the bottom of the product page contain detailed information about the second-generation standard environment, which is frustrating. I do appreciate that the language-specific pages in the main docs link contain a short comparison that includes the beta environments, at least.

That said, the primary document comparing the environments (https://cloud.google.com/appengine/docs/the-appengine-enviro...) seems very out of date. For example, it mentions you should use the flexible environment if your app:

> Depends on other software, including operating system packages such as imagemagick, ffmpeg, libgit2, or others through apt-get.

However, the Node standard environment contains these three packages and more (https://cloud.google.com/appengine/docs/standard/nodejs/refe...).

There might be other nit-picks to be said, but really, my main criticism of the documentation is that trying to understand the split between "standard/flexible/second-generation standard" environments at first glance is tricky. I'm sure y'all have plans for this going forward as these environments come out of beta, so I'll hold off on whining too much more. Thanks for asking for further feedback, and for clarifying that the new PHP runtime is a second generation one.

Thank you all for the feedback, it is extremely useful for us.

From this, I believe the action items for us (apologies if I have missed out anything, please correct me):

* Work on making it clearer which environment and which runtime you should choose for your apps and cut out the marketing prose.

* Provide more technical details on the differences between first generation and second generation runtimes.

* Be explicit on what libs are provided and what need to be vendored.

If y'all want to provide more feedback please feel free to add to this thread (I'll be checking it periodically for a few days) or contact me via email - lla@(you know the domain).

Disclosure: I work for Google on the App Engine team doing documentation.

For the moment, unfortunately it appears this second generation also lacks some of the great parts of the older platform. For example, with the shiny new GAE Standard Node Beta, you can't declaratively put your app behind authentication, you can't access the (extraordinarily useful) work queue feature from the older GAE, and probably more I have forgotten about for the moment.

I hope that stuff is missing just temporarily - there is a lot of genius in the original GAE.

Anyone know if Cloud IAP can handle at least some of the auth wall use cases in this environment, and how to do it?

I suspect yes, but the last time I checked the Cloud IAP docs (for GAE standard's recent Python 3.7 launch) they didn't yet address the unique situation of second-generation App Engine standard environment runtimes.

Based on some comments from Python 3 on GAE Standard [1], I think Cloud Task supersedes queue. DSL Authentication sounds cool though.

[1]: https://news.ycombinator.com/item?id=17717045

The first-generation GAE Standard Environment felt a lot like† programming in a port of the given language's runtime to a "cluster OS" like Mesos or Plan9—except it went even further: rather than the "GAE OS" just exposing things like "the cluster's KV storage" as a character device, and then making language runtimes implement a userland protocol library to speak to it over that device, the GAE OS's ABI just has system calls for things like key-value storage or job-queuing (calls which aren't even parameterized by a handle, since there's only one possible kvstore you could mean—the cluster's.) The language runtime ports are, then, essentially just exposing those system-calls, 1:1.

It reminds me a lot of being back in the 80s, when you could write raw assembly that simply called BIOS interrupts to read and write from a disk (by logical disk number!), rather than there being this whole edifice of an OS kernel in between, mapping high-level conceits like logical disk requests to low-level SCSI/ATA/etc. wire protocol messages. That mapping was instead happening at a firmware level, such that the hardware (the CPU) could expose intrinsics that would look like any other ISA instruction, but which would turn around and use the firmware (the BIOS) to execute the instruction.

That BIOS-interrupt style really feels, to me, like how computers should be: rather than everything compiling down to RISC code that inevitably gets bogged down in kernel context switches as it attempts to express its intent through thousands of low-level system calls, why not have an ultra-high-level CISC ISA that can just express its intent to the hardware [and firmware] directly, with no impedance mismatch? All the "expansion" of intent to RISC would happen in ring 0, with no switching back and forth. (A good example of this kind of CISC-expression-of-intent: kernel packet filtering.)

Or, if you don't want to go that far: why not an abstract machine, like the JVM, with these hyper-CISC semantics? I know there's already one example of this approach in the way Erlang's BEAM VM implements its "send" instruction. In the expensive cases—when a module is not loaded in the local VM, or when you need to send the message to a remote node over the distribution protocol with a custom transport—the "send" op can bounce back down into calling other Erlang modules (essentially "firmware"!) to get its job done. I'd love an abstract machine that had that approach... but for everything.


† I say "felt a lot like", but from my understanding, this is how it actually was/is! GAE Standard Environment v1's interpreters are compiled as Portable Native Client [PNaCl] binaries, with extensions to the "ABI side" of the PPAPI for the specific services the Native Client VM host provided to its guest. The language runtimes then 1:1 expose these PNaCl instruction-set extensions as the functions of modules like ndb, taskqueue, etc.

genius… or vendor lock-in?

Member of the App Engine team here. In the context of the second-generation runtimes, this is one of the big reasons we're pushing people to use standard libraries and public cloud services rather than App Engine specific services accessed through syscall magic that doesn't work elsewhere. We want you to be able to run your app locally or on a non-Google server (or on Flex, GCE, GKE, etc).

You should really consider either very detailed porting guides or compatibility libraries, at least from what I hear those required changes will keep some projects on first-gen otherwise (and thus Python 2, and thus requests to dependencies to please keep Python 2 supported and ...)

For those not familiar with benefits, it's mainly about scaling from zero to heavy traffic fast. If for example, you are running CRM behind call center with crazy traffic during TV ads, and no traffic on weekend your infrastructure can scale from 0 to multiple instances instantly.

This is on the heels of the Python 3.7 AppEngine release from 2 weeks ago[0]. gVisor[1] (user-space kernel) seems to be ready for prime time, which is allowing rolling out new AppEngine runtimes waaay faster.

[0] https://news.ycombinator.com/item?id=17717045

[1] https://github.com/google/gvisor

What explains the vast price difference between App Engine standard and flexible environment? In the standard environment, $0.05/hr gives me a 128MB, 600MHz container. Flexible environment is priced at GCE pricing, giving me 3.75GB and 1 vCPU for the same price, before even applying committed use discounts. It seems hard to justify using the standard environment, even if scaling up/down is faster.

Standard scales down to zero instances, so for intermittent loads this can be very cost efficient.

I ran in to a couple of show stopper issues with the new nodejs standard env which were fine in flex (waiting on Google support on these, but very slow datastore queries and odd request queueing behaviour), so we're running flex at the moment, but for our particular use case standard would otherwise be cheaper.

That is a very good question.

Another thing to note is that GAE standard api does not work in Flex or GCE (no portable api for datastore, memcache, runtime apis for versions and modules, etc.) So basically any code that you write in standard gae is locked in.

Another interesting note is that standard gae is covered in free tier but flex is not covered in free tier.

It seems like google is trying to lock you in with standard gae and then make you pay order of magnitude more for the infrastructure once your service gets some traction.

Gen1 Standard used proprietary GAE APIs that locked you in.

Gen2 Standard (PHP7, Node8, Java8, Python3, etc) uses the standard GCP APIs that you can call from anywhere, including other clouds. It should be relatively straightforward to move from Gen2 Standard to Flex and vice versa.

(I work for GCP)

Excuse my complete and total ignorance here - but is this essentially a 'quick deploy' php instance? if so, what might a none-Google alternative be?

I've only ever 'rolled my own', on small-scale virtual or shared servers, but I've got an upcoming project I could do with trying to install on something completely seperate - something like this might fit the bill.

Or not!


AppEngine is a "Platform as a Service" although they are re-phrasing it as "serverless" these days. It has benefits like you dont have to worry about the "roll your own" side of things - you just deploy your code and it runs (Google takes care of the servers - patches, security etc).

The "killer" feature of AppEngine is the scalability - i.e. your small-scale virtual or shared servers are ok for small usage, but what happens when you have more traffic than that VPS can handle? Perhaps call the hosting provider and ask for more CPU/Mem/Bandwidth, or maybe move to a new bigger server elsewhere. But that takes time, and what if it was just a spike and now your traffic is back to normal but you've got a beast of a server running the site. You could be wasting money even with the small VPS sitting idle for most of the day if you have very infrequent traffic.

With AppEngine it just scales automatically to handle the load, from zero QPS to many thousands QPS and back down to zero again without any effort from the developer apart from the initial config (... of course you can limit that scaling too if you want to control cost).

There are also a bunch of other nice-to-have features around traffic splitting (i.e. A/B testing), some nice task-handling tools (e.g. cron-style stuff, task queues), centralised logging, and the easy plug-ins to the other Google Cloud stuff.

Deployments themselves are not especially "quick" and you have to do a few things "the appengine way", but the trade-off is you have nothing to manage on the server at all.

I've used AppEngine a lot over the past few years (mainly Java and Go) and its pretty decent I've found, although I do use a lot of the other Google Cloud services (e.g. their database stuff) so it makes sense. App Engine has been pretty rock solid in my experience.

I'll add that the "free tier" makes it a great place to host hobby apps that get little traffic without paying a monthly fee. They are very stable; I've only had to upgrade every five years or so.

Good run down what GAE offers. The point I'd like to add and emphasise on is zero downtime deploys.

Thank you very much for your response here, fellow LDNer - it's illuminated a domain I had completely overlooked for years!

What specific problem are you're not sure how to solve the right way ?

Compared to some other language a proper PHP env is relatively dumb to set up, have a build script in your system of choice to install fcgi PHP in the version you need and the extensions you expect and setup the relatively simple conf file in your browser of choice, and then have a composer file to setup the libraries you need, tada you're pretty much down (everything else is not very much PHP specific)

Thank you for all your responses here - I am red-faced with shame to realise I'd for years completely ignored exactly what containers are and associated terms such as 'Heroku' or 'Kubernetes' I'd simply assumed they were curious alien creatures, completely irrelevant to me!

After a long journey down various rabbit holes last night, where I at points got quite excited by the idea of leveraging Heroku with perhaps a soupçon of Cloudflare in the background or AWS' myriad of services, I ended up back where I've always been, and think I'll stick with what I know for my upcoming project.

However, I think this'd be a great way to dabble in things I really don't have any famiiarity with (eg. Ruby), so I'll have a play yet.

Thanks, all!

Laravel Forge [1] offers what you're after though I'm not entirely sure it's worth the added cost imo.

[1] https://forge.laravel.com/features

Forge is a laravel only platform and doesn't scale automatically. It is more useful for maintaining a standard deployment process for your laravel application. It made setting up queues / redis / mongo a breeze when we onboarded new employees.

> Forge is a laravel only platform

That's incorrect. The page says: "We don’t discriminate: Laravel, Symfony, WordPress, Statamic, Craft and any other PHP application can easily be installed on a Forge server."

> doesn't scale automatically

That's true but I the person to whom I was responding isn't looking for auto-scaling (nor am I presenting it as a GAE competitor). It sounds like he just wants a button to push that'll create a fully operational PHP app server.

If you wanted say wordpress cpanel hosting will provide a one click button, auto update plus allow you install most popular php software packages easily.

Paying 10 or more dollars a month probably doesn't make a lot of sense unless you are installing laravel and for some reason can't do it yourself.. Or if you need to replicate the process. It is not really one button click either you need to find/create the right script(s) if you are doing anything complex and if you are not just have the developers get a copy of the app from github and clone a dev box on digitalocean.

Non Google alternatives are for example AWS Elastic Beanstalk or Heroku.

There's always Heroku too.

I was wondering what people generally think on using AppEngine when compared to rolling the API gateway+Lambda for public side projects. Somehow there seems to be much more movement on Lambda front, but I can't seem to find a lot of opinions from people running their apps in app engine, which I am quite sure there are more than Lambda? Or am I mistaken? The standard environment seems to be quite feature rich, but with a lot of hidden limitations for a newcomer, but at the same time the developer story for Lambda is not that great at all.

I've used App Engine for side projects, and small personal projects that only I use. It's great. I've wrote mostly everything in Go, and a little Python.

So far I've been billed $0.00. The free tier is quite generous, and probably plenty for most low traffic endeavors.

Not sure why you compare GAE to lambda?

Well lambda+API gateway is somehow what AWS is pushing in the serverless, scalable and pay per use app model, so for me it seems quite logical to compare. Cloud functions can't really compete feature set and integration wise in my opinion.

AWS's offering is more mature, but has a concomitantly steep learning curve; using something like serverless[1] can reduce the exposure to that, but there's a lot of stuff to learn to get things working nicely with AWS.

[1] https://serverless.com/

Now if they only would support Perl...


now .net core in standard environment!

bborud 5 months ago [flagged]

I feel slightly queasy :-)

You should see a doctor if it doesn't go away :-)

Are you sure you found the right thread? Your comment seems more related to the panic attack thread.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact