
Show HN: Zappa – Serverless Python/Django on AWS Lambda - Mizza
https://gun.io/blog/announcing-zappa-serverless-python-aws-lambda/
======
sergiotapia
If you're asking yourself: "Why should I care about Lambda? How does it help
me build a fast website? In what context?", there's an interesting use case
from a merchandise store that made real hats and such for Counter Strike
Source.

Valve mentioned them on the official CS:S page, and things went haywire. The
team restructured into a lambda friendly architecture, they scaled without
breaking a sweat, and ended up paying pennies in costs.

Link:
[https://www.reddit.com/r/webdev/comments/3oiilb/our_company_...](https://www.reddit.com/r/webdev/comments/3oiilb/our_company_did_a_collab_with_valve_for_some_new/)

------
vangale
It seems like having local lambda emulation would be useful as a companion to
this. If I ever do some lamba development I'd love to see integration with
emulators like [https://github.com/HDE/python-lambda-
local](https://github.com/HDE/python-lambda-local),
[https://serverlesscode.com/post/emulambda-testing-aws-
lambda...](https://serverlesscode.com/post/emulambda-testing-aws-lambda/), or
[https://ashiina.github.io/2015/01/lambda-
local/](https://ashiina.github.io/2015/01/lambda-local/) (some or all of these
may work already, I can't tell just by eyeballing the projects from a
distance)

~~~
Mizza
This would have been useful for me during development..

But honestly, the big problem is the API Gateway. The product is a mess, the
docs are a mess, and the whole system is kind of insane. Lambda is awesome,
but API Gateway still seems half baked. 90% of my development time was
fighting with APIGW. There are some kind of crazy hacks in there (base58
encoding cookies, regex based on b64 encoded status codes) that shouldn't have
been necessary.

Still, now that the system works and it's easy to use, I think this is ready
for real usage. And I'm sure Amazon will get their stuff together for future
releases of APIGW. They probably weren't completely anticipating that people
would use it in this way.

~~~
dexterdog
Yeah, I fought with APIGW for a while in the beginning because I had to make
scripts to create my route structures to I could call Lambda in a fully
RESTful manner for my data model. A lot of stuff just didn't work when calling
from the AWS CLI. After a lot of trial and error and a lot of bug reporting I
got to a script that would quickly build my APIGW for me because that web
interface is horrible.

------
Mizza
Happy to answer any questions that people might have about this!

To head of a few questions at the pass:

Here are the hacks necessary to make this work:
[https://github.com/Miserlou/Zappa#hacks](https://github.com/Miserlou/Zappa#hacks)

Here's how to avoid the cold-start problem:
[https://github.com/Miserlou/django-zappa#keeping-the-
server-...](https://github.com/Miserlou/django-zappa#keeping-the-server-warm)

~~~
jjoe
This is cool. On this:

 _To ensure that your servers are kept in a cached state, you can manually
configure a scheduled task for your Zappa function that 'll keep the server
cached by calling it every 5 minutes._

The cost of running the warmer (300s * ~3M * 512MB) comes out to about $18 and
that's not counting the number of actual requests from end users. It's
interesting but costly as a substitute.

~~~
dexterdog
The call should only run for under the minimum 100ms each time as it's just a
ping and the instances is usually already loaded although AWS will likely
recycle the instance from time to time outside of your control. There would be
roughly 8640 calls/month which is about 864 seconds of execution time. You get
800,000 free seconds per month. This is totally negligible because even if you
are counting it above the free level it is about 7/10 of a cent per month.

Disclaimer: I am a Lambda fanboy.

~~~
jjoe
From my understanding of the pricing (I could be wrong), even if the call is
100ms, because the timeout is set to 300s so the app stays in cache, you'll be
charged for 300s (app is "running" for that long).

~~~
dexterdog
That is not correct. You can set the timeout to the maximum if you want and
you only pay that when the call does not complete.

------
vog
_> Where normal web servers like Apache and Nginx have to sit idle 24/7,
waiting for new requests to come in, with Zappa, the server is created after
the HTTP request comes in through API Gateway_

To me, this sounds very much like reinventing the plain old CGI, just with
different names ("Web server" -> "API Gateway", "CGI script" -> "server").

Am I missing something here?

~~~
rakoo
There are a few differences:

* CGI is the closest UNIX thing you can find, fork a process, write on STDIN, read from STDOUT. Lambda actually is a framework for running javascript/java/python code, _from which you can call actual binaries_.

* Lambda's attractive point is that containers are reused, which means that instead of paying the full price of a new process your context is already "hot", even for the actual binary you run. That means less latency and less CPU usage, allowing you to scale far more easily

I'm no Lambda user so I'm possibly wrong, but the idea and execution behind
sure looks nice.

~~~
toomuchtodo
Both your statements are correct: Amazon is most likely running lxc containers
under the hood though, so anyone could duplicate what Lambda does using an
evaluator for javascript/python/etc with a REST API containerized in a docker
container.

Disclaimer: I am a heavy Lambda user.

~~~
Eridrus
I haven't done my own experiments, but reading the Lambda docs on Security has
made me think these are real VMs and not containers.

------
siscia
I am toying around the idea of writing an open source implementation of Lambda
on top of docker (for isolation) golang (for custom code to run) and
erlang/elixir/OTP (for coordination). Are people interested ?

Feel free to contact me on the address on my profile...

~~~
Mizza
I think there is a strong need for something like this. It's worth noting that
API Gateway is also a huge component of this project, not just Lambda. Lambda
alone wouldn't provide the benefits of Zappa.

------
vincentdm
Interesting! I've been playing around with Serverless/Lambda this weekend (the
node version), but a main thing that struck me was that I couldn't use a
connection pool for database connectivity. This means that each request to a
lambda-backed API will need to create new connections.

Anyone who wants to share their thoughts about this?

------
cdnsteve
"Serverless" is a framework used by AWS:
[https://github.com/serverless/serverless](https://github.com/serverless/serverless)

And it now supports Python. Would be interesting to see someone using AWS
Serverless framework with Django Plugin support.

~~~
Mizza
The point of this is that you don't need to learn any new frameworks - it
works with your existing code. You can deploy your existing apps on Lambda
without having to change anything, and you're not locked in to AWS if you want
to go back.

~~~
StavrosK
I rather doubt the "without changing anything" part. For example, I write
static media to a subdirectory of my Django installation. Will that work
without any changes? How about Django uploads? How about temporary files? Etc
etc.

I used to use Django-nonrel for GAE so I wouldn't be logged in, and guess
what: By the time I wanted to move away, I had so much GAE-specific behaviour
that I pretty much had to rewrite the app.

~~~
Mizza
Well you really shouldn't be serving static content through Django, you should
be serving it through a CDN. If you use something like Django-Storages for
uploads, it'll Just Work.

I'm not guaranteeing that everything will work out of the box on the first
try, but I bet you'll be very close. You will have to make a few design
decisions, but if you're making the right ones it should just work. At the
very least it should be far, far easier that GAE.

(The one thing you'll have to watch out for are C-extensions, which, for now,
require that you do your deployment from an x86_64 machine.)

~~~
StavrosK
> Well you really shouldn't be serving static content through Django, you
> should be serving it through a CDN

The two aren't mutually exclusive. My CDN points to Django for retrieving the
static media.

> if you're making the right ones it should just work

"If you're making the ones that Zappa requires", you mean.

> The one thing you'll have to watch out for are C-extensions

Eh, that's to be expected, though.

------
bake
Fantastic! We had been considering using lambda, but were worried that doing
so would irreversibly lock us into AWS. Being able to develop for lambda using
standard web frameworks (django already being our favorite) goes a long way in
increasing our willingness to use it.

~~~
Mizza
Yep! This should make it super easy, there are no code changes needed so you
can go back and forth between Lambda and traditional hosts as often as you
please. No "lock in" \- although one of Lambda's advantages is how well it
ties into the rest of the AWS ecosystem with RDS/S3/CloudFront/etc.

------
hendi_
Wow -- the fact that this allows deployment of a "regular" Django application
to Lambda is awesome!

Have you checked if Postgres works via the py-postgresql driver? It's
implemented in only Python 3, with all C optimisations optional.

~~~
Mizza
Haven't tried that yet - want to give it a shot? Pull requests welcome! :D :D

------
gitaarik
Love the name! Django Reinhardt and Frank Zappa are 2 of my favorite artists!
^^

------
trymas
1\. sounds nice.

2\. what the hell is this
[http://imgur.com/a7FevNF](http://imgur.com/a7FevNF)? where's at least the
close button? thank god ESC works.

~~~
Mizza
I think you're right about that. Increasing conversion rates by 5% at the cost
of pissing off the other 95? Dumb. Sorry.

~~~
StavrosK
Why is it dumb? You increased conversion rates by 5%.

------
herbst
The website sucks. It asked me twice about my email. Fucking twice. Cant read
that shit.

