

The rise of lighttpd (excellent web server) - nickb
http://pinderkent.blogsavy.com/archives/79

======
willarson
Well, server optimization has been my primary hobby for the past month, and
I'd like to make some clarifications about a few things the article says.

Saying that Lighttpd supports "CGI scripts, Ruby on Rails, Pylons, TurboGears,
Django, Quixote, webpy, Mongrel, Catalyst, Mason, Joomla, AWStats, and so much
more" is kind of misleading. Lighttpd can do a few things: it does fast cgi
(fcgi support explains about 75% of the examples in that list), it can serve
static files, and it can proxy requests to other servers. Lighttpd doesn't
"support Mongrel", because Mongrel is another server. What Lighttpd does is
use something akin to mod_proxy to send certain requests somewhere else, that
somewhere else might be a Mongrel cluster that is running on port 8080 (hence
it "supports" Mongrel).

That proxying service is where the majority of Lighttpd usage has been. A
typical efficient setup these days (YouTube does along these lines) is to have
lighttpd accept incoming requests and if the request is for a static file then
it serves the request (because it is very lightweight (i.e. minimal) it is
more efficient than Apache at serving static media), and if it is for dynamic
content the request is passed to Apache, which then passes it back to
Lighttpd, which passes it back to the requester (this means that "heavy"
Apache threads will be dedicated to a request for a shorter period of time
since it only has to communicate/wait for Lighttpd instead of the actual
requester).

Lighttpd on its own isn't necessarily the best solution (an example from my
experience is that mod_python and Apache become more efficient at serving
dynamic content than fcgi and Lighttpd as the numbers of requester per second
get very high). Usually a combination of a light frontend server and Apache in
the backend can provide the best performance (again this is at high numbers,
at lower numbers lighttpd is probablly equally or more efficient).

As a final note, I'd really recommend Nginx over Lighttpd if you are looking
for a light http server to serve static content, support fcgi, and/or proxy to
other servers. The biggest reason? Lighttpd leaks memory and has to be
restarted occasionally because every couple of days (under heavy usage) it
dies. Not a great feature in a production server.

------
staunch
The big win of lighttpd/nginx is that they're single-process [1] multiplexing
servers. Apache spawns dozens (or hundreds) of processes/threads and relies on
the OS to distribute resources. This critical difference is why they're so
much more efficient and generally faster. The epoll mechanism in Linux has
widened the gap even further.

Brad Fitzpatrick created a software load balancer in Perl that handles all the
requests for LiveJournal based on this principle.[2]

1\. A small number of processes may be used to take advantage of multi-
processor machines.

2\. <http://danga.com/perlbal/>

------
tx
The "rise" of lighttpd? I think it's dying now. Apache 2.2 and Nginx are the
kings of the hill.

~~~
davidw
It seems every 6 months or so, one of these single process, select based
servers comes out as the "apache killer", only to fall prey to a later
successor. Anyone else remember boa?

~~~
staunch
That's not really true at all. BOA did almost nothing but serve static files
and there was no consistent development. Same is true of thttpd and all the
rest.

Lighty and Nginx are becoming full-featured multiplexing servers. It's a lot
of work to add features when all the code must be super reliable and non-
blocking. No serious open source effort has ever been made. Zeus has been the
only option for years, but it's commercial and expensive. They've made
millions filling this void.

------
mhartl
If you're running Rails using Mongrel, beware that Zed Shaw (Mongrel's
principal author) has stopped recommending lighttpd:
<http://mongrel.rubyforge.org/docs/lighttpd.html>

------
ivankirigin
I love it. Though I'm not an expert in optimized configuration of servers. I
guess that's sorta the point.

------
plusbryan
we moved from lighttpd to mongrel a few months and couldn't be happier.

