
Hidden limitations of Google App Engine - zeynel1
http://stackoverflow.com/questions/565963/hidden-limitations-of-google-app-engine
======
nikcub
I have completely fallen in love with AppEngine since the beginning of this
year. It is a brilliant dev platform for prototyping and the BigTable DB is a
pleasure to work with.

My prototyping platform is essentially: tornado.web with the tornado request,
response, escape, auth and locale. OpenID from web.py. My own WSGIApplication
that inherits from the built-in so that it uses my Controller class (which has
a plugin arch), Django for templates, JSON, BeautifulSoup, Markdown, a generic
'asset' module implementing something akin to ActiveRecord on the standard DB
with memcache support, a generic HTML5 getting started template, a CSS toolkit
(can't recall the name) then jQuery and backbone.js with some of my own JS
glue code that maps the server models to client models.

Using this stack, I can prototype form-driven web apps in hours.

I wrote a relevance view for Twitter (with Twitter oauth support) in half a
day. I don't think I will use the RDBMS even when they release it - BigTable
is excellent (it will be confusing if you schema the same way you schema with
an RDBMS).

It is great to be able to have an idea for a quick app, register an AppEngine
app, fire the code up, commit and deploy and have the app live for others to
try out within hours.

The admin interface allows you to spot errors and choke points in production
and fix them quickly. The pricing is better than AWS or any of the others and
the platform includes hidden gems such as the MapReduce lib, multitenancy,
xmpp and Google data libs (including auth)

The only real limitations that I have bumped into are:

* The max request time is around 20 seconds, so you can't have open Ajax requests

* No HTTPS support as yet (coming soon, apparently)

* Limits on the number and size of files. It isn't ideal for serving a large number of static files, but it does have zipimport support for libs

* For best performance, you need to define the indexes yourself rather than relying on the defaults that the lib generates

* There are a lot of sys calls blocked, so you can't run Tornado or any cpython modules

~~~
thezilch
Is your last point the reasoning behind not using Tornado's templates? And why
not Jinja? Django templates have always rubbed me the wrong way in performance
and flexibility measurements.

~~~
nikcub
Using Django with my prototyping framework is by no means an endorsement, it
just happens to be what I fell into because when I designed the generic HTML5
template I plugged in Django vars and helpers (stubify, etc.) without really
thinking through a decision on which templating lib to use.

That said, I did create a generic sketch.template wrapper for Django
templates, and I have managed to swap in web.py, tornado and cheetah (sketch
being the name of the framework). I have never used Jinja in production, but
have been meaning to go back to it, your comment will prompt me to do just
that (I assume by asking about it, it is your pref).

I will eventually push the entire framework public once I clean up some parts
- it is simple and has a narrow scope/purpose but it is working really well
for me

------
mmastrac
AppEngine has made some great leaps in the last year. We built <http://gri.pe>
on it earlier this year and have been impressed by the platform.

The last two months have seen the platform mature a lot as well. The latest
1.4 release has some amazing features: async datastore queries, app keep-
alive, longer timeouts for background tasks, etc.

~~~
nl
And 1.4 has the Channel API for everyone!

------
grandalf
The architecture limitations exist b/c App Engine essentially scales
effortlessly once you abide by them.

If you insist on using SQL, then use a company like Heroku, but realize that
if your app gets popular you will be paying Heroku a lot of money for the same
level of performance you could get for a few bucks a month using app engine.

After I wrote my first toy app a few years ago, App Engine doesn't really feel
like it has any significant limitations other than the absence of naked
domains and SSL. The recent reliability problems have been a nightmare, but as
of Nov 6, I feel like they have been resolved.

I should add that I love app engine and use it whenever I don't need naked
domains or SSL. _(SSL is only available via_.appspot.com domains)

~~~
bad_user
On Heroku ... I chose to build by own stuff on EC2, and it is a lot cheaper.
And now with the free tier account I also get some freebies.

And yeah, app engine is great for entry-level prototipes, since you can forget
about sysadmin tasks.

------
rubypay
The biggest limitation I had was while writing to the database, the database
would sometimes return an error for no apparent reason. I'd then be forced to
catch these errors, and resubmit. But if you had lots of traffic and resubmit
too quickly, Google App Engine would throw more errors back at you. This was
all over a year ago, not sure how it is now, I would never go back to using
it.

From <http://code.google.com/appengine/articles/life_of_write.html>

_There is an expected failure rate on writes as Bigtable tablets are sometimes
unavailable, for example, when they are being moved or split. The presence of
more indexes increases the probability of hitting an unavailable tablet as an
exception will be raised if a write fails for any of the indexes. In those
situations, your application will need to decide how to handle the exception.
One option is to add a task to the task queue to retry the write at a later
point in time. Another idea would be to respond with an error from the app and
have the client retry. This tends to work with things like AJAX requests where
there is client side logic which can handle an error message from the server._

~~~
follower
This might have been reduced by the release in February:

[http://googleappengine.blogspot.com/2010/02/app-engine-
sdk-1...](http://googleappengine.blogspot.com/2010/02/app-engine-
sdk-131-including-major.html)

See the section "Reduced error rate with Automatic Datastore Retries".

I haven't kept a close watch on what's changed since then though.

~~~
nl
That helped some, but there was a release (to the backend infrastructure, not
the AppEngine SDK) about a month ago that made a huge difference.

Edit: Here's the link that shows the improvement:
[http://code.google.com/status/appengine/detail/datastore/201...](http://code.google.com/status/appengine/detail/datastore/2010/11/06#ae-
trust-detail-datastore-get-latency)

~~~
fharper1961
Yes that day looks impressive, but if you look at the following days, the
improvement does not look anywhere near as dramatic.
[http://code.google.com/status/appengine/detail/datastore/201...](http://code.google.com/status/appengine/detail/datastore/2010/11/18#ae-
trust-detail-datastore-get-latency)

------
SkyMarshal
Notice the date of this question is Feb 20th, 2009. Almost 2 years out of
date. I imagine at least some of the limitations mentioned are gone by now.

------
endlessvoid94
If you're sick of these limitations (and use Django) head over to
www.djangy.com and sign up for the private beta :-)

------
makeramen
I wonder if the simplenote guys can provide some input?

<http://simplenoteapp.com/about/>

------
rorrr
The biggest limitation is that they only support two languages - python and
java.

Not that there's anything wrong with these particular languages, but I'd
prefer more choice. It would be cool if they supported C++, PHP, Javascript
(V8), Ruby, Erlang.

~~~
bad_user
You can run Ruby and Javascript stuff on top of the Java support. The pipeline
has to be pretty light though and I wouldn't run Rails on it.

