
How to handle 1000s of concurrent users on a 360MB VPS - joelg87
http://markmaunder.com/2009/how-to-handle-1000s-of-concurrent-users-on-a-360mb-vps/
======
jrockway
Why use nginx instead of Varnish as the frontend proxy? If you are using nginx
as your http <-> fastcgi gateway, that's one thing, but if you're just using
it as a reverse proxy, it seems like a lot of extra stuff to maintain.

I like living on the bleeding edge, so I do varnish -> mongrel2 <-> apps
instead... but varnish -> nginx <-> apps or varnish -> apache <-> apps seem
equally reasonable. The key is caching and being able to handle slow users
without tying up your app. All three of these setups do that. nginx to apache
only saves your app server; it doesn't do much caching.

~~~
mmaunder
I must admit I've never used varnish, but from what I gather, it's function is
caching, not handling large numbers of concurrent connections. Nginx's
strength is handling tens of thousands of concurrent connections with very
little incremental CPU load and a tiny memory footprint. It then reverse
proxies those connections to a threaded-model server occupying each thread for
very little time and quickly freeing them up for other work.

From what I gather, varnish uses a threaded model with 1 thread per connection
which is what nginx solves. So it might make sense to put nginx in front of
varnish so you get high concurrency with great caching.

Nginx does offer some caching but I've played with it in high load
environments and wasn't impressed with the performance.

~~~
StavrosK
Varnish is arguably more performant than varnish. I've had amazing success
with both, and I don't think both are necessary together, but they're both
screaming fast. I noticed one day that varnish had served 100,000 visitors per
day from the cache due to a Google mistake only because I saw it in my
analytics. The server load was at 0.05 all day.

I love varnish. Hard.

~~~
koko775
Er, would you mind correcting your post to indicate which one is more
performant?

Thanks.

~~~
StavrosK
What do you mean? I say they're both very, very fast, but I think Varnish
might be a bit faster (or better written). In any case, it's a really close
call, and you should use the right tool for the job, as they aren't both
written for the same job.

~~~
axiak
I think he means that you should read the first sentence of your original post
again.

~~~
StavrosK
Ah, I meant "than nginx". The weird thing is that I read it again after the
comment. I can't edit it now, unfortunately...

------
mmaunder
This is surreal. I just wrote a draft blog entry about the changes in startup
economics due to event-servers like node and nginx (publishing tomorrow), and
I hit HN just before heading to bed and this blog entry is #1.

Thanks for all the love today guys. Have an awesome memorial day weekend if
you're in the States.

~~~
PawelDecowski
Or bank holiday weekend if you're in the UK ;)

I've been using nginx for about 2 years and it performs so much better than
Apache.

I'm not using Apache as a backend though, I use FastCGI. It performs even
better.

------
geuis
I use nginx exclusively as my server and I run node instances and wordpress
behind it. Why even use apache?

~~~
etherealG
mind if I ask what you use to tie wordpress in to nginx? cgi?

~~~
ognyankulev
As written in another comment, nginx + php5-fpm is a great combo, no need for
Apache at all. I run both Drupal and WordPress (the latter against my will :-P
) in such a setup.

I think the only reason people use nginx -> Apache mod_php is because there's
a lot of documentation and knowledge about Apache. nginx requires a little bit
different mental model how HTTP requests are served but it's worth it.

EDIT: You can think of PHP5-FPM as Apache+mod_php but only serving PHP
requests. So nginx handles static files and URL magic and when there's request
for PHP, it passes it to PHP5-FPM, just like it would pass it to
Apache+mod_php. This is oversimplifed but it's kind of like that.

~~~
chopsueyar
I had a similar configuration for a Wordpress blog, and another Linode
instance running ExpressionEngine.

I have nginx configured with php-fpm to use sockets to communicate between
each other.

------
stock_toaster
At the end of the article, why is apache even being used anymore?

Just use nginx and php-fpm (or php-cgi/fcgi).

~~~
ramidarigaz
That's what I use. I thought about the method in the article, but it seemed
too complicated, and nginx + fcgi is plenty fast for me.

Are there any performance benefits to having Apache as well?

~~~
stock_toaster
Performance benefits? No, I wouldn't think so. I can think of too many
performance downsides to it.

The only benefits I can think of offhand would be if you had a requirement for
something that Apache specifically provided, or custom modules you needed run.
Something like suphp, modsecurity, or shared hosting configuration, etc. All
policy type stuff, not performance.

------
jacques_chester
Missing: caching. Caching. Caching. And more caching.

I speak of Wordpress and database-backed PHP applications of its ilk.
Wordpress loves to spray MySQL in a thick stew of queries for every single
page load.

Without sensible -- preferably aggressive -- caching, your Wordpress site will
die in the arse.

My setup generates cached .gz files, which nginx can serve directly. It
_flies_. It didn't used to.

~~~
aonic
Or if you dont want to deal with Wordpress to implement the caching, use
Varnish. (also mentioned few times in this thread)

I've used Varnish for some high-traffic Wordpress sites by sitting it in front
of Apache running Wordpress. With well planned TTLs, and logged-in cookie
detection, most of the inconveniences of caching can be eliminated.

Also, using Disqus or similar services for comments will be helpful. You wont
have to worry about comments not showing up until the cache clears, leading to
confused users. That is, if you use the javascript implementation of these
comment services.

Alternatively, plugins like WP Super Cache have automatic post cache flushing
if there are updates, and you can also tell it to bypass cache for logged in
users. But it is IO-bound.

~~~
euroclydon
WP Super Cache will serve up cached files via PHP with the flip of a switch.
While this isn't as fast as letting the server serve up cached files, it's an
easy win.

------
otterley
This technique naïvely assumes the only thing that is slow is the client. If a
backend data provider that the web server communicates with (e.g., database)
is also slow, this arrangement merely adds complexity without much
corresponding benefit. The origin server will still block waiting on the data
provider, which can cause process starvation in the pathological case.

Moreover, issues with slow clients often can be solved by raising the TCP send
buffer size. As long as the response size is less than the send buffer size,
it really doesn't matter how slow the client is: write() will return
immediately, leaving the webserver free to serve the next request. Getting the
data to the client then becomes the kernel's responsibility.

~~~
SaltwaterC
That would work, unless you're using a container-based VPS. OpenVZ is
notorious for standing in the way of system tuning. Using nginx makes a lot of
sense in this case, especially with the ey-balancer patch.

~~~
otterley
The TCP send buffer size can be controlled at the application level (see
SO_SNDBUF in tcp(7)). There's no need to adjust kernel sysctl settings.

~~~
SaltwaterC
By the manual page you mentioned: <http://linux.die.net/man/7/socket>

> SO_SNDBUF: [...] The default value is set by the wmem_default _sysctl_ and
> the maximum allowed value is set by the wmem_max _sysctl_.

You can't use sysctl under OpenVZ. QED.

~~~
otterley
Fair enough - then you'll have to switch to a different VPS if you're sending
> 128KB responses. Use the right tool for the job.

------
jschuur
Worth noting the the story is from Dec '09.

~~~
gaius
People were doing this with Squid years before that, in the 90s.

~~~
iphoneedbot
Yeah.. squid is actually great. I do wish squid is a packaged that just needs
to be turned on for most VPS

------
rkalla
As a good number of folks have pointed out, this article (helpful for sure) is
_basically_ nginx + PHP-FPM.

nginx servers up static content (e.g. if you are using WP-SuperCache and it's
generating those static cache files for you, JS, CSS, images, etc.) then you
configure a pool of warmed up PHP VM instances via PHP-FPM (Check
/etc/php5/php-fpm or some equiv dir on your server).

Then you setup a rule that directed PHP requests to the FPM service, that
would probably look something like this in nginx:

    
    
      location ~ \.php$ {
        fastcgi_pass   127.0.0.1:9000;
        fastcgi_index  index.php;
        fastcgi_param  SCRIPT_FILENAME  /var/www/mysite.com/$fastcgi_script_name;
        include fastcgi_params;

}

and instead of adding the layer of Apache servicing the PHP requests, you are
having nginx pass through the PHP requests _directly_ to PHP processes to run.

Going from an Apache2 + FCGID configuration (somewhat similar to this, but
with Apache) to nginx, I saw a 75% drop in server load.

I'm almost certain _this_ still isn't a totally tweaked out setup that someone
more familiar with this process could do better, but for my needs it took my
server crashing under load no matter what I did, to typically being idle most
of its life (~900k/month pageviews)

So I'm a big fan of nginx. I'm not saying you couldn't configure Apache to do
the same... just 5 years of attempts to do so never got me anywhere with it.

------
gsharma
Repost: <http://news.ycombinator.com/item?id=970682>

Interesting comments on the original thread.

~~~
stock_toaster
oh wow. I didn't even notice the posted date on the original article until I
followed the repost link you provided, and saw how long ago _that_ had been
posted.

------
james4k
I just find it odd that Apache hadn't employed the likes of epoll already. IRC
servers have been using async sockets like this for yeeeears.

~~~
peterwwillis
Well, it does since apache 2.1 ('event' mpm module, replaces worker)

~~~
SaltwaterC
Event MPM is still a large pile of fail. Unless you're unwilling to switch
from Apache, the benefits over Worker are mostly theoretical from the
performance point of view. Hopefully this will change with 2.4, but only for
those who didn't kick Apache out of their production machines.

------
chopsueyar
Are the loopback requests limited by the TCP stack?

Is it possible to set this up where nginx and apache communicate via a socket?

~~~
arianb
Yes, and that's the recommended way of setting it up. :)

------
muppetman
You can do the same on a really high traffic Drupal site using the boost
module. It's basically a module that will, for non-logged in users, generate a
static html page and serve that instead. PHP isn't even called, the .htaccess
just serves up the cached content.

And that's using Apache!

~~~
tobylane
I've heard a lot of bad things about htaccess, is its slowness comparable to
php over static, or apache over nginx?

~~~
true_religion
htaccess is just a configuration file used by Apache to determine what should
be served to whom.

~~~
tobylane
Which can slow things down, seems much less of a problem now than when I last
dealt with them. There's a lot to read on using them to speed websites up.

Minus two for an on topic question?

~~~
true_religion
The amount of slow-down is relative to what you actually write in the htaccess
file. You can have two very different files that actually do the same thing.

------
ericb
Dropping keepalives actually slightly hurts your best-case performance. As a
config change, if you are using the prefork mpm (I wouldn't if scalability is
your goal) it will generally help you though. With prefork the scaling math on
worker memory * connections vs your box's memory is often the first place
things will fall apart if you have keepalives on and real traffic.

I suggest the worker mpm if you don't want to switch to nginx.

------
peterwwillis
Hint: you can just run Apache twice; once with a mpm_prefork config for your
apps, once with a mpm_event config to handle client connections.

------
mise
I've been on Apache servers for years, so I come from that frame of mind. My
managed VPSs are running Apache, and one of them is running nginx for static
resources. I think my managing host (Servint) doesn't even support nginx. How,
for argument's sake, would I move my PHP/MySQL sites to run on a managed nginx
server?

~~~
ComputerGuru
At the risk of stating the obvious, you would move to a better VPS that
charges as much or less for a better package :)

~~~
chopsueyar
He means Linode.

------
mp3geek
I use straight nginx on Fanboy Adblock (which gets hundreds of SSL connections
per minute), no need for Apache.

------
iphoneedbot
This approach really works well... I would add that off-loading MySQL to a
different VPS within the same network works well too; latency within the same
network is very negligible. Doing simple optimization such as moving/disabling
mail and stuff you probably dont need works too.

~~~
jacques_chester
Indeed, pushing MySQL onto a different VPS in the same network basically
halved page generation times for my sites.

This is partly because when a table contains a TEXT field, MySQL performs all
joins against that table _on disk_ , regardless of what fields you're
selecting or joining on.

Which is a problem for, oh, I dunno, every PHP blog engine / bulletin board /
CMS ever.

------
nwmcsween
nginx is actually pretty bad at all benchmarks I've thrown at it, you can see
some here: <http://www.webhostingtalk.com/showthread.php?t=918355> this was on
an 8 core intel Xeon 5520 server system it maxed 8 cores (18ghz) @ 7500
requests per second for static content.

~~~
stock_toaster
I found that forum thread quite hard to follow, and there are blobs of ab runs
sprinkled all over. I clicked a few pages through, and didn't see any configs
posted, sysctl values, etc. Was it more of an ad-hoc test you were running
just to get some vague ideas about performance, as apposed to a comprehensive
performance comparison?

My personal (anecdotal) experience shows that nginx is _amazingly_ fast. Some
nginx configs can impact performance if you aren't careful (I am thinking of
try_files vs if-tests[1]), and there are various nginx tunables (worker
process count, cpu affinity, etc) as well.

I have personally seen issues with apachebench when there are lots of
connections (not to mention linux system tunables such as max file
descriptors!), iptables state tracking, etc...

[1]: <http://wiki.nginx.org/IfIsEvil>

~~~
nwmcsween
This was stock RHEL 5, I tested nginx compiled with gcc -O2, -O3, -O2 -ffast-
math epoll was enabled, I even specified the cache line size at build time and
tested hard enabling AIO the nginx config consisted of simply serving a simple
static html page. I tried every option under the sun at the time tcp_nopush,
etc. I spent _days_ testing apache vs nginx vs litespeed, instead of drinking
the kool aid.

~~~
guylhem
Did you try others things like thttpd ?

