- Java 8 is already generally available on GAE standard and, on top of that, previous limitations wr/t class whitelisting, threading capabilities and others have been lifted
- appcfg is pretty much outdated these days; all GAE management operations have been progressively integrated into gcloud app subcommand, which also fully supports newer GAE features like Flex, firewall rules, etc.
- GAE standard instances are indeed limited in computing power, but it should be enough for most web-based applications. Again, GAE Flex fills in the gap here in case more muscle is needed, since it leverages GCE-class instances
- Besides the already mentioned background work options (background threads, task queues, cron jobs), there's also Pub/Sub which can be used to inter-communicate GAE services or even other GCP services like GCE or GKE
Slow adoption of new Java versions is a pretty big downside. Even if you are satisfied using Java 8 on GAE today, you may want to use Java 9 in the near future, and you won't be able to. By choosing GAE you are committing yourself to being stuck with Java 8 for a long time.
Also, not having access to Java 8 was painful. Not being able to use lambdas or streams for that long really sucked. There's no similar pain from not having access to Java 9 yet; it's not nearly as big of a change in the language.
> it's not nearly as big of a change in the language
True, but there were quite dramatic changes to the JDK.
Disclaimer I work on GAE.
We do monitor the public issue tracker (https://issuetracker.google.com/savedsearches/559750) and will prioritize appropriately according to vote number.
Tack this on as one of the reasons people don't trust google cloud...
No more information from users is needed to implement it, so the only useful comments can come from Google employees who can still comment.
This is standard procedure on Google bugtrackers.
for me that is a big blocker.
Is it that volatile that they don't want to go public with their intentions? That's not really a good sign, IMO.
*(Just one of many reasons not to use Java on the JVM in general)
Within less than a day I had a prototype up and running that handled inbound requests, did some processing, offloaded the rest to background workers and returned the request. The workers asynchronously stream data to BigQuery for real-time reporting.
The only thing I found baffling is that I could not load Googles' own cloud sdk from within the GAE environment and had to "vendor" it in, doing a bunch of workarounds/patching to get it to run. That probably consumed 3 of 4 hours it took to get this running.
Load testing this thing, GAE automatically managed everything and I never saw any error responses throughout the load test with. It's amazing how little ops work is needed.
AWS lambda and kinesis firehose/s3/Athena could provide part of this, but it's far from turnkey/conclusive. GAE/GC are just so much more complete.
This always happens when I try to use one of Google's APIs/SDKs. I dread using them, I feel like they're overengineered most of the time.
The SDK works as expected once imported as the outside library and various outbound requests things are configured and patched.
I think I overstated the complexity. All I needed to do was enable SSL for outbound connections (one-liner in the yaml file) and add the requests library patch to use URLFetch service (python/GAE specific step). Searching SO and Google documentation is what took up 95% of the 3 hours I spent on this.
So if you would like to use GAE to host your web app, Go is most preferred. Go instances consume much less memory than Java instances (less than 1/5) and need much less time to fully warm-up (less than one second, so you don't need to handle the cold instance problems at all).
Just my personal experiences.
The instance warmup time tends not to be an issue at scale, but it does mean you end up paying for more idle instances than "optimum". It's still cheaper than hiring a devops team. The cool thing about GAE is that you can build a multi-million-dollar business with a couple engineers and they can both go camping for a weekend at the same time.
Go is nice on GAE because you have near-zero instance start time. But then you must work in Go, which is a poor language for general business processing (despite its other merits). The Go runtime also lags behind the Java and Python runtimes on GAE Standard. The GAE community does not think "Go is most preferred"; actually the GAE/Go community tends to whine a bit about support. It's not perfect, but then again nothing is.
I disagree on "Go is a poor language for general business processing". The current Go runtime on GAE is very mature to build all kinds of websites.
The lack of generics, and a nice map, filter, reduce, flatten workflow cause this for me.
Sometimes I'll just use goderive then, but it still looks clunky. (No pipe or fluent API)
But for everything else just start adding APIs and Services and see how your costs balloon out of control.
Cloud is way more expensive than other options, but I'm serving 5,000 customers for less than $100 a month, so spending more time to get that price down is definitely not the best use of my time.
When you have a simple stack and reduced uptime requirements, getting something like 3 servers from a provider like Hetzner (https://www.hetzner.com/dedicated-rootserver/matrix-px) can give you a lote of compute power for little money.
Of course building something on top of GAE or AWS EBS / Lambda is a lot easier on the OPS side and gives you much more confidence, but at a much higher price.
You have to sacrifice a lot though, like with python there are tons of libraries that are unusable because the have native code which is only allowed with the flex runtime which can never be free.
Think about it this way. When you use appengine Google has an sre making sure your application is up and running. It makes sure it auto scales up and down. Deals with zero day security updates. And gives you a release rollout procedure. Logging is hooked in as well.
Sure you can do all of that, but at most companies it's not unheard of to cost about 50-100$/hour of SWE work.
I’d argue if cloud is convenient for you and cheap as hell, no reason not to try. However, it is not cheap because you would need to setup your own cloudfoundry or kubernetes in AWS and whatnot (or the cloud provider’s native container solution), which often means renting a machine.
So that’s why most people don’t do development in the cloud. They test their “just committed to my feature git branch” latest version in the cloud by means of automatic (Jenkins etc) or by pushing to the cloud manually.
Over the years they've introduced a whole bunch of stuff/services, simplified a bunch of things (e.g. datastore pricing is now simpler).
They've also made things more complicated (in my opinion). Years ago, downloading and using GAE was straight forward. Now they have all sorts of versions (standard, flexible, etc) and running a program is no longer as simple as before.
Their documentation is also extremely poor and they sometimes change/remove services without enough notification (switching from modules to services, changing support for openID/Federated login even though it still showed up in GAE).
At the end of the day, I would still recommend it. It just takes more effort for me to explain to new folks how to use it.
Average RPS * $15
So for 100 RPS for one month it costs around $1,500.
Also, you can optimize for cost by adding the following to your app.yaml:
Also Golang and Node costs less than other environments.
My two cents.
It's not much for anyone who is used to high scale system but it is a lot more than the average application can take without effort. It's gonna require quite a few cores, disks and bandwidth to achieve. Definitely not gonna run on a single cheap instance.
Cloud Functions for Firebase
I think the promise was never actually realized. When first released, you didn't have to worry about load balancing instances or scaling, it was "internet scale", out of the box, dependent only on your traffic. Then they went to the per instance pricing and raised prices.
I was really disappointed no service ever arose to fill this gap; internet scale as service you could say.
And AWS Lambda has sort of picked up on this. Not exactly the same but very close.
Flexible environment does not have that restriction.
Stick with Java or Go, I'd say.
This has always been one of the benefits of Heroku to me: I don't have to architect my application for a single vendor's environment. If I use a hosted Postgres backend, then even more so.
What Google should consider doing is making App Engine "classic" services like Memcached, etc. available to any service running across GCP. Flex is fairly different from classic, but I think there's some confusion generated from the "App Engine" name and the baggage (good & bad) it has gathered over the years.
Already being done. First was Datastore, now Cloud Tasks is in Alpha. Expect to see others (Cron, Cache, etc) come as well.
Shameless plug, but for Go, there is an unofficial third-party (Chromium team) datastore package that feature near-seamless AppEngine standard/flex datastore support: https://chromium.googlesource.com/infra/luci/gae/+/master
It currently couples to any hosted memcached service, but should be forward-compatible with AppEngine if/when they release a Flex-accessible cloud version of their hosted memcache.
"luci/gae" also features a fully-compliant in-memory datastore mock for 1000x+ faster datastore testing and uses a memcache implementation based on "qedus/nds" which fixes several consistency issues present in "ndb".
Many AppEngine standard services are accessible using the "cloud" versions of APIs/endpoints. You may have to do some more heavy lifting, although on Flex you get GCE self-authentication for free, and most "cloud" API packages know how to recognize and utilize this.
Memcache is apparently on the list to receive a cloud API, although it's been in the "pre-alpha" state for over a year now. See: https://cloud.google.com/appengine/docs/flexible/java/upgrad...
Given Google's record with other products, I'm particularly worried Google might discontinue or stop improving GAE. Does Google provide any offical reassurance that GAE is a high-priority product that will be quickly improved and not put on autopilot? If so, do their promises seem to match the reality observed by the author for GAE standard?
quote from the end of the article:
It feels like they only invest the bare minimum anymore. Actually, I have had this feeling for the last two years...It is infuriating if there are known issues but they are not fixed. It is depressing to receive so little information on where the platform is heading. You feel trapped.
the 'old' App Engine is on its way out. I do not think it is a good idea to start new projects on it anymore. If App Engine Flexible Environment on the other hand can actually fix its predecessor's major issues, it might become a very intriguing platform to develop apps on.
Even today surprisingly few people know about it. So I finally authored and published a book on building products using the Google Cloud this year. (Free to read on the site)
Then I found out that I couldn't run my own sidecar container (nginx reverse proxy), and also that Cloud CDN doesn't support App Engine Flex yet. Those were dealbreakers, so I switched to GKE. So far things are working well except a Kubernetes issue not terminating Job/CronJob sidecars correctly which gets tangled up with Google forcing you to run a "magic" proxy sidecar container to access your Cloud SQL instance(s).
The appeal of App Engine is simply being able to type `gcloud app deploy`, which tars up your local source code, builds a Docker image from it, then does a rolling deploy of the new image to replace your old app containers - all from that one command, no complicated build system needed on your part (i.e. Google Cloud does it all for you).
My point was the moment you stray from the App Engine happy path (`gcloud app deploy` fails for obtuse reasons), you're out in the weeds. That plus the fact that App Engine has some weird missing integrations (e.g. Cloud CDN) often makes it easier to just jump to GKE in my opinion.
Sounds like you're doing something totally different and the heroku-ness is failing you.
And yes, although this is essentially vendor lock-in, the access to the many services provided is really really a timesaver that boosts productivity tremendously. I just wonder whether Google is spending much time on it, given how slow they support newer versions of languages.
p.s. it is pricey, but probably this will improve soon.
For PaaS products, GAE is comparable to AWS Beanstalk (or Heroku, or CloudFoundry, etc).
For "serverless" products * , AWS Lambda/Step Functions are comparable to Google Cloud Functions.
* Serverless in the product sense rather than the literal sense.
PaaS & FaaS are quite related; so it is not unreasonable to discuss in this threat. e.g. see informative comments by @indigodaddy on the same threat.
p.s. article also talks about other non-PaaS topics e.g. Data Store, Memchase, BigQuery, etc.
> Comparing GAE to AWS step functions (which is some wrapping around AWS Lambda) is moot since they are different categories of products solving different problems.
Many of the problems that they solve are the same; but they do so in a different manner.
AWS Step Functions is a lot more than "wrapping around Lambda"
> For "serverless" products * , AWS Lambda/Step Functions are comparable to Google Cloud Functions.
AWS Step Functions has no comparable service on Google Cloud
We did some contingency planning a while back, which included moving a low volume service to a LAPP (Linux/Apache/Python/PostgreSQL) stack to see how difficult is was. The biggest job was re-writing the NDB API calls, but it was not difficult.
For me, knowing the world's best sysadmins are looking after our services is worth a lot. I've heard a couple of talks by Google's system relability engineers, and they are fanatical. Visible evidence of this is the brutally honest post mortem they produce every time there is a significant outage, including the vitally important "what we're going to do to stop this ever happening again" section. This focus works - GAE is way more reliable than AWS or Azure.
So YMMV, but GAE is well worth it for us.
So I hold that it is the scaling that locks you into GAE, but only to the extent that you desire scaling. Scaling gives you redundancy and reliability too, don't forget.
> GAE -> easy to get in...very hard to get out!
If your codebase is really 70% datastore API calls, I'd strongly suggest writing your own wrapper so you can change datastore by updating a few primitives. But in any case, GAE is no harder to get out of than any other scalable platform.
> and expensive...
We run three services with millions of users, for about 250 USD per month. Given a single dedicated server starts at $75 USD per month, and that we'd need at least four servers for redundancy and scaling, and that we'd need to hire a system architect/administrator for say ten hours a month to maintain them, I think GAE is excellent value.
> and risky...
You don't say why you think GAE is risky. In terms of uptime, our experience is that it is an order of magnitude more reliable than AWS or Azure. The other risk people sometimes suggest is that Google will discontinue the platform, but Google seems strongly committed to Appengine and Appengine has a three year deprecation policy for any part of the service.
> it's not worth it
We obviously disagree on this point. I would accept "it's not worth it if you will never need scaling and are happy to do your own system admin for free".
And you can trivially ensure you never exceed them simply by setting a $0 daily budget. (Although I recommend setting a low but non-zero cap instead, so your app doesn't go down if there's a legit but unexpected traffic surge.)
Disclaimer: I work at GCP.
I didn't register a card and all applications have been shut down. I will not register a card.
Basically, GAE cannot be used anymore for amateur, high school and student projects.
That said, I had once linked a card from my uncle with the private Google Wallet to buy a Nexus 7.
And yes, the CC requirements are insane. Every professional service supports SEPA transfers, except Google. It basically is an extra 30-60€ a year cost just to use Google services, or to develop for Google Play. 30€ CC cost plus 25$ registration fee. (CCs are basically unused in Germany, where I live).