

Ask HN: Who's using Google App Engine? - shawndumas

What big sites use Google App Engine? Have you tried it? Is it flexible enough to run big, serious projects on?
======
sean_lynch
Disclaimer: In a past life I was a PM on App Engine (and not surprisingly,
we're using it for our new company), so the following comment has a fairly
high bias.

Though it doesn't have the same HN buzz that Heroku does, App Engine has a
fairly large developer community and a lot of applications using it. Like
almost all PaaS product, there are restrictions that limit what you can do
(library support, native code support, etc) and these limits frustrate
developers that go in expecting the same experience as having a virtual
machine. The limits mean that App Engine isn't the right solution for every
application under the sun. But if your application doesn't have super unique
requirements, the value that App Engine provides is hard to beat.

Ten second deployments, one click version management (and roll back), logs and
monitoring, and autoscaling mean that we can be really agile pushing new
versions of the application. App Engine's recent changes to the scheduler and
Always On feature have really solved the slow instant start issue that still
plagues a lot of Heroku developers. Last year the datastore had a ton of
performance and reliability issues, but the new high replication configuration
appears to have solved the problem. The biggest issue for us is the one that's
mentioned here a couple of times and that's SSL.

For now, App Engine is nearly perfect for us. If we end up with unique
requirements, we can definitely spin up a few EC2 instances and hook them up
to App Engine's Task Queue. But I'm not excited about having to manage
machines. Personally, if we hit a limit with App Engine, we'd be looking to
another PaaS offering like Heroku or dotCloud, or Infrastructure as a Service
like MongoHQ/RedisToGo before bothering to deal with machines again.

~~~
jcampbell1
> For now, App Engine is nearly perfect for us.

Have you launched yet? My experience is that it becomes very difficult to
manage once you have customer data.

~~~
sean_lynch
Launched, although not particularly large (www.splitterbug.com). What becomes
difficult to manage when you introduce customer data?

------
vosper
We use it at bynd.com for a bunch of projects; we've deployed at least 30
applications on it now. It has taken me quite a while to get used to the AE
way, and I don't love it, but it is very good for some things. I can think of
lots or pros and cons, but off the top of my head:

Pros:

\- scaling is amazing; we've had sites go from 1 to over 1000 instances in a
few minutes; from <1 to 1600 queries per second in the same time.

\- reliability is pretty good now

\- if you let go of the idea of doing everything synchronously life gets a lot
easier, and deferred processing is really easy to do.

\- the development process is great: download SDK, checkout code, point SDK at
code and you're off. Then you develop locally and have it run exactly the same
(with a handful of caveats) as it does appspot. It's a really low barrier to
entry.

Cons:

\- webapp (the default framework) sucks. Once you find something like tipfy,
webapp2, or flask life gets a lot better

\- changing your datastore structure can really suck with a live site. There's
still no simple way of getting data on and off, or of doing backups. Messing
with your live database if you need to fix some bug can be _scary_

\- it takes some time to grok the right way to model data, and to use the
datastore.

\- there's a lot of "here's how to get started / build a simple blog" info,
very little on best practices, how to model complex data, how to do
maintenance...

~~~
okal
What framework did you wind up using? Tipfy seemed great at first but the docs
aren't quite up to date, so I wound up sticking with webapp for the prototype
for my project.

------
grandalf
App engine has the following awesome characteristics:

\- the datastore is extremely scalable if you use it as intended.

\- your app can automatically scale up and down (as if an invisible genie were
moving the little "workers" slider that Heroku uses)

\- always on instances make your app mega fast, even at low volume (but cost a
tad bit more).

\- there are a lot of useful apis (memcache, queue, email) and a nice
development environment.

\- it usually "just works" and "just scales".

a few bad things:

\- they had some very annoying downtime while they sorted out the datastore
architecture problems (where theory and practice had differed with horrible
results)

\- the OSX dev tools don't bundle the correct version of python and associated
libs, which makes it a pain to get a working dev environment on OSX without
breaking your system's python.

\- no naked domains, SSL support for normal apps that aren't priced per user.

All in all I'd rather write an app for AppEngine than any other cloud system,
though Heroku is catching up -- both could really benefit from prioritizing 24
hour support via irc though.

------
OmarIsmail
App Engine has a lot of really awesome features going for it. There are some
pains for sure, but not the ones you find people typically griping about. The
datastore and all of that aren't a big deal. What's annoying is the lack of
library compatibility, and no real SSL suport. Having access to a relatively
small subset of available libraries for Java/Python sucks, each time you find
a library you'll have to find out for yourself it it works on AE or not.

As for SSL. I mean come on. It's 2011, it's absolutely unbelievable they don't
support SSL for custom domains. Unbelievable.

~~~
axiom
There are ways to hack around the lack of SSL on third party domains with
Ajax, but yeah it's pretty insane that they still haven't got that in place.

~~~
marcc
Completely agree that it's a much needed feature. We've worked around that by
setting up an nginx server in AWS which has proxies to the
<https://*.appspot.com> name. So it's still SSL all the way, but we just
didn't want appspot.com as the URL we gave out.

------
edhgoose
The UK's Met Office - who provided Weather Forecasts use it for their new
Weather Observations Website (WOW): <http://wow.metoffice.gov.uk>

It's a site that allows the public to submit weather observations from
anywhere in the world. Since launching 31 days ago, we've got 1.7 million
weather reports and probably spend about 0-30 seconds a day (on average) on
maintenance.

App Engine has some big pains (SSL has already been mentioned) but if you
don't require it (we don't), I think it's a fantastic tool.

The multiple versions, quick deployment (new build inc all
testing&configuration can be completed by us in <5 min), automatic scaling and
the fairly good dashboard are fantastic. Plus, our infrastructure costs in
development (before heavy load testing) was $10 for 3 months of work -
absolute bargain! Even after heavy load testing, it was a fraction of what a
real server would have cost.

I'd say the biggest difficulty (or barrier to entry) is the datastore design.
It's very easy to think you're in SQL land, and you're not. And because of
that, it's very easy to burn yourself quickly. But, the design gives great
scalability. We wouldn't be able to manage the size of the data we're
expecting in 2 years+ in a typical environment.

So, I think it's great :)

(Disclaimer: I was part of the team at PA Consulting that built WOW)

------
sktrdie
I don't know what big sites use App Engine but I personally use it for
everything. Every weekend project, every work prototype, everything! I just
love the fact that it's free to start with, and the idea that it's on Google's
server sort of brings a level of confidence for up-time and scalability.

Sure the fact that are some limitations (no IO and not many libraries) are a
bit of a down-side, but the payoffs of using such a service instead of
managing your own servers is really great. You're just able to crank a lot on
the app code part without worrying at all about system administration work.

The simplicity of the datastore is also a great benefit, and it scales like
nothing out there.

------
jbaker
I use App Engine a ton. I won't try to list all the positives. Of course "is
it flexible enough to run big, serious projects" depends on the requirements
of your big serious projects. Feel free to get more specific here, or (better
yet) in the google app engine forums.

One thing not mentioned yet that is a legitimate concern applying generally to
PaaS offerings including App Engine - that is vendor lock-in. There are some
way to address that including choice of apis and frameworks and also projects
like AppScale and TyphoonAE.

There is much to evaluate when considering App Engine for your projects. the
flexibility of the platform from a technical perspective is one of things.

Also note, watch closely the changes that are coming in pricing and the way
that Google is offering the service. You don't want to get caught evaluating
the App Engine of 2010 for 2012.

------
petekoomen
We use it here at Optimizely and have been very happy. It hasn't been smooth
sailing all the time, and we've gradually grown our stack to include EC2, S3,
and several CDNs as well, but overall we're very happy with it and have no
plans to leave.

The biggest benefits for us are the scalability (huge), ease of deployment,
and the monitoring. The biggest cons have been lack of support for SSL on
custom domains (HUGE hassle) and lack of official support.

~~~
vosper
I'm interested to hear that you think monitoring is easy - it was something I
forgot to add to my "cons" list; all you get for free is the web dashboard
that lets you look at logs, and ereporter to send you daily exception
summaries. One of my biggest "want"s in fact is API access to logging.

------
znt
Google itself is using App Engine on both internal and external projects as
far as I know.

The websites below has tens of thousands of registered members, there are also
similar websites for some other European countries (Italy, Spain, Germany etc)

Getting British Businesses Online: <http://www.gbbo.co.uk/>

Getting Australian Businesses Online:
<http://www.gettingbusinessonline.com.au/>

So yeah, App Engine seems to be suitable for big projects if you play by their
rules. Only problem seems to be backing up data, as their App Engine's
bulkloader tool has rate limitation, so it takes quite a while to
download/upload all data.

------
detst
It would be great if Google would allow for managing of a custom reverse proxy
with unique IP into an app.

This could solve the naked domain and SSL issues and enable other cool
customizations.

Has anyone done this themselves? Just put Varnish or Nginx in front of GAE?

~~~
marcc
We've put Nginx in front of GAE and it works without any issue. I haven't
tried any other methods such as Varnish though. We aren't really throwing too
much traffic at the GAE app which is "behind" Nginx, but we fully expect it to
handle a reasonable amount of traffic.

I'm sure there are many people who have done this. I'd be interested in
hearing how other people have solved the lack of SSL...

------
krisrak
Appengine is extremely good for auto scaling, if u have 100 users today and
and u get 1 million users tomorrow, it will just work without u having to do
anything, u dont need a scaling engineer.

I use appengine for all my web apps, it works for most of basic webapps, here
are a few I have used appengine for: <http://gramfeed.com>,
<http://shaloc.com>, <http://checkinmania.com>, <http://misotrendy.com>

------
code_devil
I have used AppEngine for a few fun projects where I have tried to use FB
Login/Connect for authentication. I like the fact that you can rapidly
prototype and improve your skills and have it all published to the web for
free.

Question: I was looking for a library for authentication mechanism other than
FB or Google . Any suggestions ? (All I need is basic user/password)

~~~
StavrosK
I use Django exclusively on it, and thus use Django's authentication
framework, which works very well.

~~~
code_devil
so are you using one of those helper libraries/project django-norel ?

------
erebrus
I'm using Google App Engine in a new project ( <http://www.homeyplanet.com> \-
allowing travelers to share their favorite spots off the beaten track) and I
have to say I am very happy about it. This project requires geo spatial
queries, so at first it wouldn't be a good choice given how inadequate the
datastore is for it, but the trick is always to think how things work in GAE
and see if you can optimize it. It only scales well if you do it their
way...so it won't work for everything, but most problems can be solved quite
well, you will just have to scratch the way you normally would solve them.

Note: I noticed that some have accessed the site from this link. The site is
beta and is newly launched, and for now we are only covering Europe. If you
want to check out how fluid are the geo spatial queries, you can try checking
out Portugal, where half our spots are now.

------
reso
AppEngine is great for rapid prototyping, but it still has several limitations
that make me hesitant to base a real infrastructure off of it.

I've used it for <http://www.minehub.net>, <http://www.quickcite.it>, and a
few others.

------
babs474
I used appengine for my now defunct <http://www.unedditreddit.com> project,
which sucked up all reddit comments for a few months. It was extremely easy to
use and seemed to preform just fine, with the exception of those few weeks
where the datastore had stability problems.

------
wolfrom
Windsoc uses App Engine for our Unified Social API, and while it has many
benefits, we've always had our minds on how to move off of it someday. SSL is
a big issue for us, and another is the support model (which should be changing
soon from what I've seen).

So what we have now is a product that runs both on App Engine and on DotCloud.
Since DotCloud is awesome, too, and we know that we can talk to a specific
person when something goes wrong, it's currently a front-runner for us. We're
also looking at Joyent, too, as opposed to EC2.

But even if we switch large parts of our product to somewhere else, I wouldn't
be surprised if we still used App Engine for some things.

------
dsulli
I was using this on a project for a while. But compared to running the
application with PHP, it was always slower. Sure, I've developed applications
with PHP since 2001, so I really know how to tweak it and optimize it. But,
with App Engine it was really frustrating. Requests really took long to
complete, and there was no real way to figure out what was going on behind the
scenes. We created an 'always on' instance, but it was still quite sluggish
compared to the exact app running on a standard linux, apache, PHP system.

I would still try it again down the road, but it's really not a good fit for
what I'm currently doing.

~~~
jbaker
Just out of curiosity, how would you describe what you are currently doing?
Can you elaborate on why it might not be a good fit? I'm honestly just trying
to learn from your experience, not disputing anything.

~~~
dsulli
Sure - no problem - the application is Generic Domain Finder - just search it
on Google. The current app in the search results is still running GAE.

The things I loved: \- tight integration with Google accounts \- easy
integration with other Google apis (we used a few)

The frustrating thing was dealing with the black box. We used an api to get
domain availability. Every time it returns more slowly than the equivalent
call on our linux based cloud server.

With GAE, you're basically stuck with what you get. With a generic Linux based
cloud server - you have more room to optimize your environment, or even take
your code and run it anywhere... AWS, Rackspace, Voxel...

I still believe in what GAE is trying to do - and I'll definitely try it out
again for future projects.

------
cfontes
It takes you a bit to get used to the catchas... I am using it in one of my
side projects and it's working very well although we had some throuble to move
from SQL to the BigTable and that slowed us down a bit in the architecture
phase.

But I am betting on it because I think it will make my life easier in the
future...

The memcache is ok to get.

The java subset is the part that bothers me as I had a lot of already made
stuff I liked to use in my new site, and had to redo a loooooot of stuff to
fit the dawn java subset.

------
PanosJee
We use it at www.bugsense.com and we love it. It has lot of great apis
included but it requires a mind switch and you should study the limitations.
Datastore is really scalable but slow.

------
gsharma
Most, if not all the comments are from people/companies that use GAE. I'd love
to hear from people who evaluated it and then decided not to go for it.

------
ww520
I've used App Engine for web projects and backend server for Android apps. My
web TV site <http://www.topchan.tv>, and the webpage snapshotting site
<http://www.previouslook.com> are in App Engine.

------
netmau5
I built <http://www.sparkmuse.com> and <http://thesparkfoundry.com> on
AppEngine. You'll have to write a little more code to handle not having a
RDMS, but that's only if you need joins.

------
marram
I have been using GAE Python since it came out. Used it to build
www.thesponty.com and www.gamedaytycoon.com (social game on FB). The latter
has thousands of users and GAE scales super well. We deploy > 10 times a day.
GAE is really awesome.

------
evolution
Optimizely. I can't remember other big ones but there more. App engine has
proven to be serving millions of requests per day without any issue. There are
some case studies around, also some web caches hosted on app engine.

------
bchjam
I've only just started using it (this week) as an excuse to get my feet wet
with Go. It seems easy enough to get started, things are well documented so
far though I've admittedly only touched about half the packages.

------
slig
What about the channel API, any experiences? I want to do a multiplayer quiz
app as a side project and I'm wondering if I should go with the GAE or if I
should dive into node (using socketstream, maybe).

------
HowardRoark
I built <http://www.onesong.me> in appengine using the java playframework, and
I quite like what I get for free.

------
davisml
We use GAE for Pulp Sync (acrylicapps.com/pulp). It's scales very well if you
handle requests quickly and take advantage of the memcache and taskqueue APIs.

------
drawkbox
At 2XL we built the backend for Ricky Carmichael's Motocross Matchup on Google
AppEngine. We might switch to EC2 as it grows.

------
len
we use it at ligertail.com and dont see a reason to switch to aws so far :)

