

Hire YC: Need high-traffic Django help ASAP - mattdennewitz

hey,<p>we just re-launched out site w/ Django, and are cant handle the peak traffic load. we have apache in the back (mod_python), nginx in the front, memcached. the database doesn't seem to be the bottleneck.<p>restarting apache yields fast response times up front, but gradually drifts to about 6-12 seconds per request.<p>we can't handle getting crushed with unacceptable load times much more without serious consequences, so if anyone is available to help, email me at `mattdennewitz at gmail` and lets talk.
======
maney
The first thing you want to do is measure the problem in enough detail to
identify the cause. I can recommend Rob's Debug Toolbar (though with
reservations about the possible overhead, though I'd have been happy to chance
that when I needed to instrument an underperforming site):

<http://github.com/robhudson/django-debug-toolbar/>

though his tree doesn't have the injectable measurement points I've added to
the timer panel yet. I hesitate to suggest this for a production setting only
because it's not had much third party exposure (or if it has no one's let me
know), but "it works for me":

<http://github.com/mmaney/django-debug-toolbar/>

You might want to avoid the most recent commit (so use da1429...) - Rob had
asked about adding support for the cache API's "add" method, and I threw that
together based on the spec; that one part is completely untested other than
that it doesn't cause a compilation error in my test setting.

------
suhail
buy a server load balance with nginx and throw a duplicate copy of everything
on that and use the same db since it's not your bottleneck.

Start looking for code you can optimize such as bad loops.

------
chiffonade
... why can't you just add more application servers?

------
erlanger
mod_wsgi. If nginx is serving as a proxy or a file server too? If the former,
I'd go with HAProxy. If the latter, Lighttpd works very well with Python and
FastCGI.

~~~
mattdennewitz
just switched to modwsgi

response time is up to 30 sec with 1 thread and 50 processes. nginx is up
front as a proxy and a file server.

~~~
erlanger
Have you benchmarked the same app running on Lighty and FCGI? You shouldn't
have to change much, if anything.

~~~
mattdennewitz
no. is that a common solution for hi-traffic service?

what would lighty/fcgi replace?

~~~
a_toker
Lighttpd would serve both static files and the application. Reddit uses web.py
with Lighty+FCGI, and I see no reason that you couldn't run your Django app on
it. I use Apache + mod_wsgi for the Apache modules, but I'm taking a good look
at lighttpd+FCGI to get more out of my server for my Django and Trac
installations.

