

Optimizing Nginx, Node.js and networking for heavy workloads - gnw
https://engineering.gosquared.com/optimising-nginx-node-js-and-networking-for-heavy-workloads

======
aroman
Good read, but despite being a major proponent of Node.js and many of the
ideals it seeks to embrace, I'm not sure I'm comfortable with calling Apache
"archaic". It's not like IE6, which has objectively no redeeming value as a
modern platform target -- it did back in the day for sure, and is still
relevant in some spheres, but overall I don't think anyone (even Microsoft)
would argue that IE6 is "archaic".

But to call Apache, one of the most popular and successful _actively
developed_ webservers _archaic_? I think that's a bit much. It's not
inherently _bad_ just because it's not really targeting the C10K problem...
just different.

[A minor nitpick to be sure, but it bothered me nonetheless as I feel like I'm
seeing this "Threads bad. Async good." rhetoric passed around as fact all over
the place and it's starting to feel a bit like Animal Farm ;)]

~~~
jiggy2011
Precisely , I've used Apache for yeeeeeears and see exactly 0 ROI in switching
to something else.

~~~
d0m
An apache expert would probably prove me wrong, but I've found it quite
simpler to use nginx to dispatch multiple domains to different backend types.
For instance, I have a couple nodes, one wordpress blog and lots of small
django websites.

~~~
jiggy2011
Apache reverse proxies are very easy to setup once you have done it once. I've
reverse proxied Tomcat,PHP and ASP.Net apps just by copy-pasting a few lines
in the config.

The main strength of Apache though is the ability to separate apps run on the
same system by different users without reverse proxy via .htaccess files and
stuff like mod_php.

Apache also has an excellent security track record considering it's vast
numbers of deployments and years of service.

Perhaps though if you are a green field developer with no Apache experience
and deploy with stuff like EC2 you may as well just skip apache and go
straight to nginx.

What nginx's security record will look like in 10 years if it becomes as
popular as Apache though remains to be seen.

------
No1
For the tl;dr people:

On the nginx side, author discusses tweaking sysctl.conf, cutting down the
number of sockets stuck in TIME_WAIT, some other tweaks for performance
resulting in a 90% reduction in occupied sockets. On the node.js side, author
uses the cluster module to fully utilize available CPU cores, arriving at N-1
for the magic number of node processes to spawn, where N is the # of CPU
cores.

Definitely suggested reading for anyone running Nginx + Node.js

------
silenteh
enabling

net.ipv4.tcp_tw_reuse

net.ipv4.tcp_tw_recycle

can create unexpected problems with NAT, so use it with caution.

~~~
gnw
I only suggested changing tcp_tw_reuse but you are right, especially
tcp_tw_recycle can have adverse effects.

According to this reference: <http://www.speedguide.net/articles/linux-
tweaking-121> "[tcp_tw_reuse] is generally a safer alternative to
tcp_tw_recycle"

~~~
cpleppert
I believe the section about TCP_FIN_TIMEOUT is wrong. tcp_fin_timeout has
nothing to do with the time wait state at all. TCP_TIMEWAIT_LEN is the value
that holds onto the TCB

------
r4vik
have you tried making nodejs listen on a unix socket
<http://nodejs.org/docs/v0.5.4/api/net.html#socket.connect> and then set your
proxy upstreams in nginx to use that?

edit:

scratch that, it seems you're using node on cluster of servers (not on same
box as nginx). In which case the article is good advice.

------
zerop
should nginx be only used for serving static files. Does it have any advantage
when used to serve plain data API. I want to expose REST api (django + uwsgi)
over web, but not sure if should use nginx for it.

~~~
jiggy2011
I think this basically boils down to: "are you likely to have many users
connected concurrently at any one time"

If your API basically involves a client connecting, quickly getting a small
JSON/XML response and then disconnecting again you are probably absolutely
fine with Apache unless you have truly enormous numbers of users.

OTOH if the socket is likely to be held open for a while, because maybe the
API responses can take some time to be returned or the client is likely to
hold the connection open in order to get a stream of data over time then you
may get more mileage out of nginx.

~~~
zerop
service returns quick & short JSON responses and huge number of users are
going to hit it. So basically there are going to be enormous concurrent
connections each returning quick and short json response. No heavy work by
each connection, just that there are too many.

------
niggler
Does this setup work with web sockets?

~~~
simontabor
nginx doesn't support websockets quite yet, so no, but will do soon -
<http://trac.nginx.org/nginx/roadmap>

~~~
themgt
nginx can support websockets via the tcp_proxy module:
<https://github.com/yaoweibin/nginx_tcp_proxy_module>

