

Who's using Google App Engine? - micaelwidell

I'm looking for a scalable web hosting provider which is free or cheap at entry level, and which isn't a VPS (I don't want to bother with server maintenance). Google App Engine is the only one that comes to mind.<p>What big sites use Google App Engine? Have you tried it? Is it flexible enough to run big, serious projects on?
======
waleedka
I use app engine for several projects, a couple of which have decent load. My
biggest one is networkedblogs.com, which I started on php/mysql and recently
migrated big parts of it's back-end to the app engine. My experience with it
has been good, and I always build new projects on it, unless there is a good
reason that prevents it. It has it's own pros and cons, but overall it's a
good choice:

\- It's easy to start with and fun to use. You shouldn't need more than one
day to learn it.

\- If you like to focus on coding and not worry about managing the server then
you'll love it. But if you're a control freak and like to have access to low-
level settings then you'll not.

\- If you have prior experience with relational databases such as mysql, then
you'll need an adjusting period to forget much of what you learned about
databases and start thinking in new and different ways. Some things that are
really easy in mysql, such as getting the number of records in a table or
joining tables, become hard on app engine and require writing code.

\- When it goes down, you'll feel helpless that you can't do anything to bring
your site back up, but you'll also feel lucky that a Google employee is
carrying that beeper and have to wake up and fix it.

\- It scales, but it's not super fast. Due to its distributed nature,
individual requests rarely finish in less than 50ms and usually take 100 to
150ms (that' my experience, yours will likely differ). But it can run a huge
amount of requests at the same time.

\- It solves the scaling problem, but it's not magic. You still need to
optimize the heavily used parts of your code for scaling. But at least you do
it with code, rather than having to fiddle with server and cache settings.

\- It's a paradigm shift that will eventually eliminate most of database and
system admin jobs if it catches it (I think it will).

\- It's very different from Amazon's EC2. Amazon gives you servers that you
have to manage, Google manages everything for you. With ec2 you pay per server
hour, with App Engine you pay for resources you use: CPU Cycles, storage,
bandwidth, ...etc.

\- It's free to start with, but if you have heavy load it's a bit more
expensive than hosting dedicated servers. I think it's worth it, though,
because you'll be saving a lot of your own time.

\- It's good for the majority of projects, but if your project has special
not-too-common requirements, then it might not. Check first.

\- It only supports python and Java right now.

If you're in Silicon Valley and if you need more in-depth information, you can
find me at the next App Engine meetup at Google on the 6th.

~~~
gurop
_You shouldn't need more than one day to learn it._

Depends how well you want to learn it of course.

~~~
khelloworld
This reminds me of <http://norvig.com/21-days.html>

------
kaffeinecoma
I've been using GAE+GWT for about 6 months now, mostly for small, "do it in a
weekend" type projects. I've also used it to host content for my iPhone apps,
so that I could update the content (mostly ebooks delivered as PDFs) without
going through the awful App Store update process.

It's been great for me. I love how quickly I can bootstrap new ideas, put up a
prototype, and then just hand out the URL for to people to try.

A couple of recent apps:

<http://create-a-password.appspot.com> <http://slimpoints.appspot.com>
<http://mealminder.appspot.com>

That last one was mostly a proof-of-concept in using Google's account and
payment infrastructure. Really handy if you decide that you'll want your
customers to pay you some day.

I guess the biggest hurdle is getting used to the Datastore, which is not
really intuitive if all you're used to is SQL. Overall, a very positive
experience for me.

~~~
araneae
Meal Minder is cool, but you might want to work on the automatic-filling-in
bit. It read my directions as an ingredients list... AND as directions. I
listed my ingredients with bullet points (*) which is a pretty common format,
and they weren't recognized at all.

------
mbenjaminsmith
We're using app engine for www.jukaroo.com. We've been using it for a year now
and after a lot of invested time are migrating away. The number one reason to
use it is the one click deploys but we've found that time saved there is lost
working around app engine's quirks. The service is buggy (set a page to
refresh every ten seconds or so and watch how many random 500's it will throw
- there's a monkey patch floating around for the random datastore timeouts
that are partly to blame). It's also slow compared to other setups I've used
(currently running vanilla django at webfaction). The limitations are many and
absolute (dig around but off the top of my head no threading, no long running
threads, no push, no naked domains, no python c libs, no ssh no advanced image
processing, no support, no support and no support).

Core apis I've had to patch: imaging (black lines on edges of pics), datastore
(random timeout errors), inbound mail (improper body and attachment handling)
and probably others that I've forgotten about.

There is also no portability at this point. I've seen one project that will
run an app engine app on ec2 but I haven't used it.

I think it's a cool project but if you want the unhindered ability to build
really good products I can't recommend it. Also don't mistake app engine for a
hosting company. The experience for us has been 'you are lucky to be using our
nifty cloud' and not 'thanks for your business, we're here to help'.

------
adora
I talked with one of the guys that works for Google App Engine, and he
confirmed that BuddyPoke is one of the biggest (if not THE biggest) apps that
run on it. That app serves millions of DAUs on various social networks and
runs just fine.

I've set up things on app engine and amazon as well. I found GAE just to be
much, much easier and saved me a lot of time and money (yes I know EC2 is
cheap but it's not cool to see your bill growing when you're just testing
things).

I actually took the time to switch to EC2/S3 for a couple days because I was
worried that GAE was not really serious about making it a productive place to
run serious applications. Portability was a big factor too. But after a while
I figured that all this worry was nonsensical. If I ever have to move things
over, it must be a good sign and either someone will have built a tool for it
or I just spend a week building a tool for it.

Also, re: SSL. You can actually do it right now on GAE but it involves a few
tricky workarounds. That said, the GAE team just pushed a more straightforward
feature for this to the top of the roadmap. Beforehand they had said they
weren't going to work on it anytime soon, but presumably after so many people
complained they decided to go ahead with it -- another good sign that at least
they're responsive to developers' needs.

------
jeff18
I've been using it for about a year for <http://www.wolfire.com> and several
hobby projects and am a big fan. If you have any specific questions, I am
familiar with most aspects of it. It certainly has a lot of limitations, but
if your app happens to coincide with what App Engine currently offers, it is a
pretty sweet deal.

~~~
micaelwidell
About where do you usually hit the limit for free usage? Are we talking
hundreds of daily visitors? Thousands? Tens of thousands? I know this varies
heavily depending on your code, but does anyone have a good rule of thumb?

~~~
jeff18
We rarely go over the free limit -- I think we've stayed under $10-20 for the
entire year, while serving up several million pages. Most of this is due to
bandwidth overages from our comic: <http://www.wolfire.com/comic>

The times we are charged tend to be for burst traffic when something gets
picked up on Digg or Reddit. Recently, one of our blog posts was featured on
Digg, Reddit, Slashdot, etc. at the same time and Google dutifully served up a
couple hundred thousand visits over the course of a few days. They charged us
about a nickel! Granted everything is nicely cached so we were not doing much
heavy processing.

It definitely depends on your web app. If you can cache it, App Engine is
essentially free. If you are doing something CPU intensive for every visitor,
it's going to be expensive.

~~~
micaelwidell
Thanks! One more question: Had any problems with using third-party Python
libraries? I mean like PIL, cjson or stuff like that? Heard something about
C-based libraries not working?

~~~
skybrian
No, you can't install your own C libraries. There's an imaging library built
in:

[http://code.google.com/appengine/docs/python/images/usingima...](http://code.google.com/appengine/docs/python/images/usingimages.html)

------
gte910h
I've found it just fine for whatever clients want to do....EXCEPT....when they
need to contact a server over a non-http port. It just can't do that. So for
my client who wanted to do apple push notifications for his iPhone app, we
needed to use both app-engine and dumb endpoint nodes on rackspace 11 buck a
month linux servers.

Amazon Ec2/S3 works well as well and has the highest security bar of all the
cloud offerings.

Rackspace is pretty good, but everyone seems very dis-satisfied with the S3
competitor, cloudfiles, so just use them for servers. Don't put anything super
valuable there though, they know your root passwords (as does google, if you
think about it, on GAE).

The users are on python. Apologies for the poor formatting/spelling, having to
rush out the door.

~~~
jeff18
That is no longer a limitation as of the 1.3.2 release. I quote:

 _URLFetch API - We’ve expanded the number of ports you can access with the
URLFetch API. You can now access ports 80-90, 440-450, and 1024-65535._

~~~
gte910h
Well that will be nice that I can then talk to 2195

------
gr366
I'm using it for small, very low-traffic projects, so I'd also be interested
in hearing about larger sites using GAE. Also curious whether they're using
Python, Java or something running on the JVM like Clojure.

------
mayanks
I have used google app engine for several projects. Most recently for
<http://www.smstweet.in> This basically helps Indian mobile phone users to
update their twitter status by sending a simple SMS at negligible price.

Overall I feel GAE is very good to quickly validate your core ideas without
having to worry about scalability at all. In SmsTweet I created the app in
couple of days and as traffic grew, started optimizing little by little by
using Tasks and Memcache. Yes there are a tonne of restrictions if you chose
GAE, but there are decent workarounds for every one of them. You also have to
learn stuff specific to GAE (datastore, tasks for example), but I guess they
are trivial to pick up. And finally if your application does pick up traffic
you can easily convert it to paid version and pay only for the extra traffic
that you get. So a win win situation for you.

------
fisadev
<http://www.subituqueja.com> (in spanish) <http://pythonregex.appspot.com>

Both running Python + Django 1.1.1 (with Google App Engine Django Helper,
<http://code.google.com/p/google-app-engine-django/>)

I don't know if they are "big", but you can see app engine in action there.

------
grandalf
If you're disciplined about how you design your app it's an amazing platform.

I recommend doing a fairly trivial project in your spare time to get the hang
of it. Try out the following features before deciding:

\- large data set related stuff like cursors, paging lots of records \-
transactions \- GQL (look at the various limitations of it)

GAE gives you a set of solid conventions for writing a highly scalable app
that you'd eventually have to deal with if you used a more traditional
approach. Some can be a bit annoying at first but it's fairly obvious why each
choice was made.

You might also check out jinja2, django, etc. But personally I love the simple
webapp framework. It's clean, obvious, stays out of the way, and it's very
fast.

~~~
olsonjeffery
I used webapp pretty faithfully, myself, until I came across bottle.py .. I
haven't looked back, since.

~~~
camperman
I saw a note in the bottle docs that it's easy to set up with GAE but no
further info. Any tutorials out there?

------
derefr
> I'm looking for a scalable web hosting provider which is free or cheap at
> entry level, and which isn't a VPS

Heroku! Basically the Ruby equivalent of GAE—you get a Postgres database
instead of Bigtable, though (whether that's a good or bad thing is up to you.)

~~~
evandavid
I am loving Heroku for the exact reason described in this post: no
sysadmining. At first I thought that wasn't possible and expected all kinds of
issues with gems etc, but so far everything I've tried really does Just Work™.

This may warrant a separate question, but I've seen various posts here and
there about running Ruby/Rails on GAE using JRuby. Does anyone have any
experience as to just how realistic that is as a deployment option? I can
imagine it being full of headaches, but I would love to hear from someone who
has either tried it or is actually running it on a production app.

~~~
skybrian
I've not run it myself, but a neighboring team tried it and switched to Java
due to latency issues. Remember that anything you do at startup will affect
latency for some requests when App Engine has to spin up a new JVM. It's to
your advantage to make startup times as fast as possible.

------
dhess
While we're on the subject, does anyone here have experience using Clojure
with GAE? I'm considering that configuration for a useful-but-not-mission-
critical application, and wondering if Clojure on GAE is robust enough for
that.

~~~
jeandenis
I've been using Clojure w/ GAE to build an enterprise knowledge-base
webservice for about 3 months now. It's my first serious project with Clojure
on the GAE although I've been playing around with it since last summer.

(1) On GAE you don't get to use some of the good parts of Clojure because you
can't spawn any threads. So the rationale for Clojure has to be something else
than agents and the Clojure concurrency model. I'd say use Clojure if you want
functional programming and all of the Lispy goodness: code as data, macros...

For me Lispyness is a huge + so it was a no brainer.

(2) One of the things I really like about traditional Clojure web-development
(using Ring or Compojure) is that if you set things up properly you can
experiment with and patch a live system using the REPL. I haven't been able to
set that up just yet on the GAE.

On your development server using the GAE jars, however, you can do it (see
[http://www.hackers-with-attitude.com/2009/08/intertactive-
pr...](http://www.hackers-with-attitude.com/2009/08/intertactive-programming-
with-clojure.html) for some guidance). There is nothing like being a C-c C-c
away from updating your development server app :)

I can't emphasize enough taking the time to get Slime, your GAE development
server, lein, and Clojure well-integrated. Your development cycle will make
good ol' Java GAE development using the Eclipse plug-in (which I used to think
was pretty sweet) feel really really clunky.

(See also [http://elhumidor.blogspot.com/2009/04/clojure-on-google-
appe...](http://elhumidor.blogspot.com/2009/04/clojure-on-google-
appengine.html) for some pointers on setting up Clojure on GAE)

(3) GAE's datastore works extremely well with Clojure because it's schema-
less. You can store arbitrary hashmaps and structmaps in GAE with zero effort
-- it's a match made in heaven. There are a couple of branches of appengine-
clj (<http://github.com/duelinmarkers/appengine-clj> is the original,
<http://github.com/r0man/appengine-clj> is a pretty good branch) that you can
look at to get a feel what I mean.

(4) With GAE's transaction support and automatic index generation (it's good
enough most of the time), you can dynamically create types (kinds), put them
in the datastore, and query on them without having to do any of the
traditional NoSQL stuff (thinking queries first, denormalizing ahead of time,
etc.). It's very sweet. I've done some Clojure+Cassandra and Clojure+GAE is a
lot more painless (as long as you can live with the other limitations of the
GAE) -- although things are getting much easier in Cassandra world with Hector
+ automatic index generators.

(5) In terms of performance and robustness, nothing about Clojure's held me
up. But since I haven't launched yet, that doesn't mean anything.

I don't know of any "high-traffic" Clojure+GAE apps out there -- if there are
any, I'd love to get some numbers on costs (CPU, especially) to see if there
is a significant difference with similar java projects.

(Actually, the guys behind <https://the-deadline.appspot.com>, see
[http://www.slideshare.net/smartrevolution/how-a-clojure-
pet-...](http://www.slideshare.net/smartrevolution/how-a-clojure-pet-project-
turned-into-a-fullblown-cloudcomputing-webapp), just launched three days ago
-- you may want to shoot them an email, they seem nice enough)

My guess is that this is a non-issue. Clojure holds up really well against
Java and JVM languages like Scala in terms of performance (especially if you
give type-hints), and I'd expect the same thing to be true on GAE.

(6) General Clojure limitation with web development: Clojure web frameworks
are light. You don't get to start with anything as nice as Scala's Lift.
You'll have to work and build your own tools -- the good thing is that Clojure
is a blast and macros will, in the longer term, allow you to do much cooler
stuff.

Compojure is a great light framework to start with. Also look at Enlive for
templating.

On the flip-side Clojure immutable nature works very naturally with request /
response.

(7) For the OP: before you get on GAE, spend some time reading about its
limitations:

\- exploding indexes (makes implementing text search very tough -- there is
some irony in there I think)

\- no threads (it kills me, I want to use pmap)

\- not real message queues. GAE is a pull/request-based system, not a push
system. It is, however, getting much better (tasks, comet on the coming
features list). Still, if you are building a real-time system that needs
constant background processing, I'd think twice right now.

\- if GAE goes down, you're down until they fix things, and will be totally
helpless.

\- some of the GAE goodness/crack means you're locked in (e.g., index building
-- to move away you'll have to build equivalent facilities for whatever DB you
move to). Although you can protect yourself by being smart in how you abstract
away the GAE.

(8) Can't think of anything else.

If there is interest, I have been thinking about writing some blog posts about
my experience setting up a Clojure+GAE development environment, working with
the data store, working the the GAE API, etc. Let me know here and I'll get a
first post out of the way by week's end.

~~~
frognibble
The one problem with using Clojure on App Engine is that the time for a
"loading request" is at least 5 seconds. It makes your app look bad for the
random user stuck on the loading request.

~~~
jeandenis
I'll interpret that as a dig against the App Engine and not Clojure... even
though it's clearly aimed at either Clojure alone or the combination with GAE.

Anyway, are you being serious (i.e., in your experience is clj+gae noticeably
slower than java+gae or python+gae?) or just funny? Since I've never done any
serious profiling of Clojure on GAE, I'll bite and do some brief performance
tests and report back.

~~~
frognibble
I am serious when I say that the GAE loading request time for a Clojure app is
noticeably longer than the time for Java or Python. I searched for some
benchmarks and found this: <http://github.com/garyburd/app-eng-cold-start-
test> See [http://googleappengine.blogspot.com/2009/12/request-
performa...](http://googleappengine.blogspot.com/2009/12/request-performance-
in-java.html) for more info on loading request perf.

~~~
jeandenis
Both of those are about "loading requests" which happen pretty rarely in the
larger scheme of things (on upload, new server for app, too much traffic so
need to scale jvm). On my current project, it's 3.5-4 seconds. Definitely not
ideal, but it's not that common an event.

I'd like to run some benchmarks for normal requests, where I doubt there'll be
any real difference.

Thanks for the search.

------
nir
<http://therealurl.net> runs on on GAE and handled >1m reqs/day a few times
with no problem. Billing is turned on, but limited to $1/day - Google only
charge for the resources you use and you can assign X cents for bandwidth, Y
for CPU etc.

I posted a screenshot of GAE quota usage on 430k reqs/day, might interest you:
[http://niryariv.wordpress.com/2009/11/27/google-app-
engine-q...](http://niryariv.wordpress.com/2009/11/27/google-app-engine-
quotas-on-43k-requestsday/)

------
okaq
GAE is ideal for nano web apps. I wrote an HTML5 Canvas Plasma Benchmark using
GAE/P:

<http://speed.okaq.com/>

It clocks in under 10kB compressed, is very fast, very accurate and I use it
to test devices all the time.

Under current quotas, I'm able to serve ~200k users per day at no cost and am
actively developing 2 more nano web tools.

The development environment is certainly restrictive, but the risk for web app
prototyping is absolutely minimal.

------
adamfisk
LittleShoot is all built on Google App Engine with App Engine Patch using
Django:

<http://www.littleshoot.org>

I'll soon update that to use "Django non rel" though -- Django running on no-
relational data models like Big Table:

[http://bitbucket.org/wkornewald/django-nonrel-
multidb/wiki/H...](http://bitbucket.org/wkornewald/django-nonrel-
multidb/wiki/Home)

------
mattdennewitz
i've used appengine from time to time @ pitchfork, and it's been great. with
the recently relaxed restrictions on uploads, result sets length, and the task
queue, _and_ ability to read and process larger uploads, _and_ xmpp and
incoming mail handling, etc, i'd say it's in good shape for bigger, more
"serious" projects. absolutely capable for smaller projects, too.

------
ordinaryman
_Have you tried it? Is it flexible enough to run big, serious projects on?_

I believe it is and am betting with my startup with hosted apps targeting SMBs
with CRM (<http://crm.ifreetools.com>) and Online Database App Creator
(<http://creator.ifreetools.com>).

If you are from RDBMS, at first you will have to become flexible to
accommodate to the BigTable way of things - no group-by, joins, etc., - and
find alternate ways of doing things. Once you make the switch, if will indeed
be good enough for _big_ or highly scalable projects.

------
khelloworld
Jumping URL (<http://www.jumpingurl.com>) runs on app engine.

FYI, there are few key things the GAE team is working on which can be found at
<http://code.google.com/appengine/docs/roadmap.html>

Things like SSL for custom domains and background processing would make GAE
even awesome to have your project running on.

------
snprbob86
Although we aren't a big site by any measure; we're using it and loving it for
<http://www.classlet.com>

~~~
clemesha
Great site you have going there. Went through the whole thing, and then demo.
Clean and straightforward.

Quick question, a bit off topic. Here:
<http://twitter.com/Classlet/status/3854900110> you say "We finally rescued
our Twitter user name from a squatter!". How did you manage to do this?

~~~
snprbob86
Thanks!

As per some instructions on the web, we created a "classlet2" Twitter account.
When logged in, there is a help form where you can open a ticket. It took a
few weeks, but they eventually released the account to us.

------
frognibble
Google's Panoramio runs on App Engine: <http://www.panoramio.com/>

------
johnnytee
I built a small app on it Tweetc.com using python and webapp framework. It was
my 1st gae and python app. So far no problems. I know Twibes is built on it
which gets moderate traffic. I like it can I can quickly build and test ideas
for free.

------
blasdel
<https://www.nearlyfreespeech.net/services/hosting> is the only service I know
of that does this kind of thing other than GAE and Heroku.

They support a whole bunch of languages, but PHP is the only one worth using
because their execution model is pure-CGI — no persistent processes. The
pricing model is just like GAE: pre-pay into an account that is debited for
the resources you actually use.

------
simonk
I really only know sites that use Google API's like
<http://www.socialwok.com/>

We're working on moving our application over to App Engine but waiting until
they get SSL on your own domain before doing that. You can see the roadmap for
it here <http://code.google.com/appengine/docs/roadmap.html>

------
chipsy
I'm considering working with GAE to make a persistent browser-based game. Not
serious about it yet, but maybe in another month or two.

I have some doubts about staying within the CPU quotas, since I have to take
on a processing load in the tick-update loop of game time, but I think I can
design it to be spread out somehow.

~~~
bd
I did a very simple game on GAE in its early days and the biggest trouble were
very costly writes.

Basically you could forget about applications that needed to write game state
to datastore in each step.

Even the simplest possible writes were already triggering CPU quota warnings
[1].

I didn't touch GAE recently, so maybe they already relaxed it a bit, but when
it started, it was usable practically only for read-heavy applications (for
which Google strongly suggested to use memcached wherever possible).

Also moving larger amounts of data between your computer and GAE datastore was
massive pain (e.g. backups or initial state load). You had to be very careful
not to blow over your quotas, as such imports were doing a lot of writes.

\---

[1] For example, in early days they counted CPU use in "megacycles":

    
    
      * memcached based ajax request costed tens of megacycles
      * datastore-read based ajax request costed hundreds of megacycles 
      * datastore-read-write ajax request costed thousands of megacycles

~~~
grandalf
this has gotten much better. Also, they now let you specify what level of
consistency you want, which makes writes go a lot faster if you're just
putting in a bunch of data or don't need to wait to be sure it's consistent
before you put in the next item.

------
RoyceFullerton
I have used it for prototype projects, but nothing that I have been completely
serious about yet. I know that <http://www.shoesofprey.com> uses it and I
believe it is working well for them and scales when it needs to.

------
Concours
I use it for 2 sideprojects: <http://www.websitecheck.org/> and
<http://www.serversproxy.org/> are both running on GAE , it's very easy to
"manage"

------
eplanit
Sounds like from all these postings that its appropriateness for "big, serious
projects" just isn't there. Now that it's over 2 years old now, I'm
underwhelmed (yet not at all surprised) by it's lack of adoption/popularity.

------
robspychala
We're using it for <http://www.scienceslides.com/> ... a place for scientists
to download signaling pathway slides in powerpoint format, etc...

