
10 000 concurrent real-time connections to Django - mYk
https://github.com/aaugustin/django-c10k-demo
======
bayesianhorse
I dabbled in realtime connections for a while now, and this is certainly a
sexy option to avoid the fragmentation happening in the Python async world.

~~~
hoov
Well, I think that was Guido's goal in creating Tulip.

------
kapilvt
why do people persist in mangling django into doing websocket persitent
concurrency when there are better py or otherwise well suited tools to such
tasks...

~~~
LeafStorm
Because it's a popular, well-written framework with a huge volume of existing
code, and integrating WebSockets into Django means you don't have to maintain
two codebases if you want to have a realtime Django app.

Also, it's fun.

~~~
kapilvt
how much of that existing code has any application in a websocket context.. if
it uses the orm layer throw it out.. if it uses the template layer..
optionally throw it out for efficiency (jinja and friends).. what are you
really left with... routing.. trivial.. a familiar api with the guts out..
overrated and a hobgoblin of consistency.

~~~
mYk
This system doesn't replace Django; it complements it.

You could build 95% of an application with the traditional request-response
model and add the 5% of real-time featurs with a system similar to my demo.

(Of course, given your opinions on Django, I don't recommend you build
anything with it.)

~~~
eyepulp
Well said =) It's incredibly frustrating to so easily build the entire app
with a "traditional" django stack and then be faced with solving for a whole
new stack just to avoid wasteful XHR-polling for some simple server-side event
driven UI updates.

We've implemented some SSE based solutions lately with gevent & nginx in front
of django, and it's been great to keep it all in the django family.

Maybe switching to py3k has some value after all... =)

~~~
notdonspaulding
I concur.

------
est
For Python2.X, I think gevent.wsgi.WSGIServer could handle C10k or more, no?

~~~
hoov
In my experience -- yes. I've pushed it further than that.

In practice, debugging is an unpleasant experience. If you're doing anything
slightly out of the ordinary, you should take care.

------
eliben
Very cool, well done.

------
stefantalpalaru
The C10K problem[1] is about real clients accessing real web servers, not
about writing short strings over WebSocket.

I'll associate "C10K" and "Django" when I'll see a dynamic web page being
served using the template system, the standard middleware and the ORM (with
the default behavior of creating and destroying a database connection for each
request).

[1] <http://www.kegel.com/c10k.html>

~~~
vampirechicken
Why would you insist on creating and destroying a db connection on each
request? We stopped doing that decades ago.

~~~
jaytaylor
Django hasn't stopped insisting on that... v1.5 might have pooling, but it's
taken them a long time to get connection pooling in.

~~~
obviouslygreen
Forgive my ignorance, but isn't this the point of something like pgpool? I
didn't think connecting to a connection pool would be as expensive as
connecting to the database itself (if it is, then my next question would be:
What's the point of a connection pool? Because clearly I've missed something
one way or the other).

~~~
jaytaylor
Yes, but pgpool isn't always an option. If you use Heroku's Postgres service,
you do not have sufficient privileges to run the commands to attach to
software like pgpool.

The only way we were able to get connection pooling working in django 1.3/1.4
was to use django-dbpool[0]. It works okay, but is still pretty sketchy
compared to the connection pooling libraries available for the JVM like BoneCP
or C3P0.

[0] <https://github.com/gmcguire/django-db-pool>

~~~
obviouslygreen
Understood, thank you for the clarification.

