Seriously, 20ms isn't good even for Python. That's not something to be proud of, that's reason to rip up whatever overelaborate mess you've constructed to figure out just how you're blowing so much time.
As a reminder, a 200 byte HTTP request can be parsed in around 600 instructions or less (about 250 nanoseconds at 2.4ghz).
Bet you didn't see that one coming
I'm content with 20ms.
Hell, I'm content with 100ms, as long as its consistent and you can argue the value.
On my servers I have one upstart job whose role is simply to start the emperor daemon and make sure /var/run/uwsgi permissions are correct for writing sock files.
Each django project deployed on the server has a uwsgi.ini file in it's etc folder. This ini file holds the config for that app only. It defines where the socket file is located, where the virtualenv is located and where the project.wsgi file is located.
It's simple because you can host multiple websites on a single server, but you uwsgi configuration is separated into 'system' and 'project', so your 'project' specific config can live in git and your 'system' config never has to change.
Nginx is setup in a very similar way.
If you're a control freak who bases engineering decisions primarily on flexibility (and are bound to python/WSGI) why not use Mongrel 2 with a simple python backend instead? You can escape having to shoehorn your code into a gevent/gunicorn/uwsgi/whatever container if it causes headaches to do so. Here's a teaching example for how to play with mongrel 2 from python (feel free to change it into something that would work for you) https://github.com/tizr/belasitsa/blob/master/belasitsa.py
I'm not aware of anyone actually using Mongrel 2 in a large scale Python application, but I'd be curious to know how well it works out.
I've never seen any articles about it since, so I wonder if others know something I don't. Can anyone compare gevent's webserver vs. uwsgi, gunicorn, etc behind nginx?
One of the main things I'm talking about in this post is load balancing, which gevent (and uwsgi, and gunicorn, etc) don't implement very well. Nginx does a pretty good job at this, in addition to many other things (like buffering).
You can definitely get a lot of performance out of a coroutine, async, etc framework, but at the end of the day it's not going to be the entry point to a production application.
Gunicorn+gevent has been painless for me.
Gevent's monkey patching approach allowed me to convert a traditional blocking web app (built on Flask/SQLAlchemy) into an evented model without having to rewrite the entire app around an event-driven approach (ala Twisted or Tornado). All you have to do to get Gevent working with Gunicorn is to pass it a command line argument to use the Gevent worker instead of the default synchronous worker.
Gunicorn + Gevent also makes it quite easy to integrate WebSockets into your existing web app, which is something uWSGI doesn't handle very well yet.
All this is to say that though uWSGI is a very solid piece of software, Gunicorn is definitely a reasonable alternative to uWSGI depending on your use case.
We're happily using tornado in production. Between the heroku router and tornado's built-in support for multi-processing and asynchronous processing, I don't have to worry about nginx + uwsgi + whatever framework. I just run tornado. And we're immune from the heroku random routing debacle , something that neither rails nor uwsgi can say.
Maybe I'm misunderstanding uwsgi though?
If for example, you randomly routed to a bunch of tornado servers that were overloaded/slow serving requests, you'd hit the same issue as that post.
Async/coroutines dont solve the problem, better routing does.
In this case, we use HAProxy, and you ideally would say "I know node X can handle 100 concurrent requests), and within HAProxy you could weight the node/set the maximum concurrent requests to said node, and it would distribute based on that.
On top of that, tornado has multi-processing support built-in, which adds intelligent routing at the dyno-level.
HAProxy is another solution, although it can't be setup on heroku so that's irrelevant. Also, it doesn't change the fact that your workers are blocked while executing long-running API requests. uwsgi won't save you either.
The linked doc literally says as much, toward the bottom: "So the only solution is for Heroku to return to routing requests intelligently. They claim that this is hard for them to scale, and that it complicates things for more “modern” concurrent apps like those built with Node.js and Tornado. But Rails is and always has been Heroku’s bread and butter, and Rails isn’t multi-threaded.
In fact a routing layer designed for non-blocking, evented, realtime app servers like Node and its ilk — a routing layer that assumes every dyno in a pool is as capable of serving a request as any other — is about as bad as it gets for Rails, where almost the opposite is true: the available dynos are perfectly snappy and the others, until they become available, are useless. The unfortunate conclusion being that Heroku is not appropriate for any Rails app that’s more than a toy."
Don't get me wrong, I think it's cool he made it work.. but seriously?
The options that a standard application will use are almost identical between the two. UWSGI just provides numerous lower-level options that I (and it sounds like you) probably don't care about.
That'll teach me to burn through a post quickly
'There is no magic rule for setting the number of processes or threads. It is application and system dependent. Do not think using simple math like 2*cpucores will be enough. You need to experiment with various setup an constantly monitor your app. uwsgitop could be a great tool to find the best value.' (from uwsgi doc)
edit: Hm, maybe there's a different thread or I was discouraged from using it in IRC or I'm inventing memories because this ml post is fairly lackluster either way: https://groups.google.com/d/msg/golang-nuts/2vHlfkpS-m0/MZ-X...
Here's the docs on UWSGI's Go support:
There are a bunch of supported languages and platforms, including Go (thought its dev): http://uwsgi-docs.readthedocs.org/en/latest/LanguagesAndPlat...
Also, uWSGI's core team doesn't have any faith in their own code; they use Apache instead of dogfooding. (And they're moving all hosting and development to sites like Github.) Twisted, at least, is confident enough in the quality of their code to self-host.
unbit.it (an Italian hosting company) employ him full time to work on uwsgi and they use it extensively throughout their production infrastructure.
(Sorry PyPy guys!)
I dont have numbers/charts to pull up for this right now, but I was testing it out to see if I could push more cpu on a server, and the memory cost was insane for the minimal test case.
Twisted also is not a web server, it's a framework. Same with Tornado, and honestly, the same with gevent/etc.
There's a ticket I opened on the PyPy tracker, but this is sort of expected if I understand JITs correctly.
I didn't exhaustively benchmark it, but the initial results were too far outside the bounds of my limitations.
Of what he was seeing without PyPy.
He doesn't want advice on how to lower that. He wants to get the same memory usage or better out of the box.
e.g. 20% faster execution for 100% more memory
In my case it was 15% faster execution (give or take), with a 400% memory increase.
This was on a simple view which did nothing, so it's hard to tell how it'd handle more complex views in the app.
Either way this is a huge detour from what this actual topic is about, whether you use PyPy or CPython
I'm optimizing for my application. If you want to use PyPy, use it. I couldn't care less.
If I was having a hard time already maxing CPU due to memory constraints, how does switching to something that will use a very large amount of memory (relative to the CPU gains) make any sense at all? Do you really think "more complex code" is going to use less memory?
If you want benchmarks, go find some, or go create some. It has absolutely nothing to do with the gains I'm pointing out by simple webserver choices and minor tuning.
$ twistd -n web --wsgi your.wsgi.app