

App Engine Performance Hacks You'll Probably Never Need, Part 1 - marcia
http://bjk5.com/post/11766098283/app-engine-performance-hacks-youll-probably-never

======
socratic
Most of these "hacks" involve working around how slow it is to store
persistent data on GAE. However, other than the protobuf one, they all seem to
rely on undocumented or uncertain semantics.

For example, one very common performance optimization in web applications is
batching writes. Why not send the very compact results of a few A/B tests to
the database/store once every ten requests, rather than once every request?

However, GAE doesn't really provide a way to do this. You could just store the
results in local memory and then persist to the datastore every 10 requests,
but there is no guarantee your instance won't get shut down. You could also
store to memcache, and do what the author suggests, persisting from memcache
to the datastore periodically. But my understanding is that there aren't any
guarantees as to what the memcache size or revocation policy is, so it's not
really possible to guarantee that all data gets persisted. (Losing random data
intended to be persisted may be fine for A/B testing, but probably not for
many other things.)

Likewise, it would be nice to delay datastore puts as the author does, or put
them in a task queue, but both of those seem to take as long as doing a
datastore put in the first place.

Is this analysis wrong? Does Google's memcache implementation provide some
guarantees that mean that your data won't get removed as a result of your or
someone else's actions without you knowing? Is there some reasonable pattern
for storing persistent data on GAE but not having to force the user to wait?

~~~
kamens
No, your analysis is not wrong -- except putting objects in a taskqueue to be
persisted later is actually much faster than doing a datastore put in the
first place (one reason being that you don't have to write to any indexes).

