
Why Nodejs serves a file with 80x more CPU usage than Nginx? - devrim
Take the same code that sits on nodejs.org home page. Serve a static file that is 1.8Mb. And do the same with Nginx, and watch the difference.<p>Code       : http://pastie.org/3730760
Screencast : http://screencast.com/t/Or44Xie11Fnp<p>Please share if you know anything that'd prevent this from happening, so we don't need to deploy nginx servers and complicate our lives.<p>ps1. this test is done with node 0.6.12. out of curiosity, i downgraded to 0.4.12 just to check if it's a regression, on the contrary, it was worse. same file used 25% twice.<p>ps2. this post is not a nodejs hate - we use nodejs, and we love it, except this glitch which actually delayed our launch (made us really sad), and seemed quite serious to me. i've never read, heard, seen this mentioned before so we could prepare ourselves better.
======
IsaacSchlueter
First of all, writing bug reports to Hacker News is usually a bit like
complaining about the government at a bus stop. It's fun, and you'll get lots
of agreement from crazy people, and maybe start a little riot, but writing
your representative is a better way to effect change.

We have two mailing lists, and an issue tracker on github. If someone hadn't
emailed this to me, I probably would not have seen it.

There are three problems I see with your test.

1) It serves the file twice, since you don't filter out the /favicon.ico url

2) you're converting the file contents to a string, which needs to be
converted _back_ into raw bytes to send.

3) your gist is in JavaScript, but I see that you're actually running
CoffeeScript. I'm not sure what kind of wrapper CoffeeScript is adding to the
equation.

I ran the same test serving the ChangeLog file from node's source folder with
node v0.6.15 (to be released tomorrow)
<https://raw.github.com/joyent/node/v0.6/ChangeLog>

Here's the source code of the server.js: <https://gist.github.com/2338851>

It did use more CPU than nginx, but not 100x as much. Requesting the /buffer
url made it spike up to about 0.2% cpu usage. Requesting the / url made it
spike up to about 0.3%. Even when I disabled the /favicon.ico check, its
behavior is nowhere near what you're seeing.

When I ran it with coffee script, it actually uses this JavaScript:
<https://gist.github.com/2338874>. So, there's an extra Function.call in
there, but otherwise, it's pretty much identical. My coffeescript version is
here: <https://gist.github.com/2338879>

So, something is odd with your situation, and from your test, it's not clear
what. I'd love to get more details.

On the more general note, clearly we in the node world are perfectly fine with
using nginx to serve static assets. Qv. the nodejs.org home page. But node's
performance here should be within an order of magnitude, or something is
broken. This is not expected or normal performance that you're seeing. We run
the "serve a big string over http" benchmark quite often, and so this was very
surprising to me.

~~~
IsaacSchlueter
Aha, I take back some of what I said. ChangeLog is not big enough, that's all.

When running with a 16MB file, there's some more CPU spiking, but only if it's
served as a single write.

Investigating further, I found that there's a bug in my gists, and I'm testing
res.url instead of req.url, so I was making all the exact same mistakes in
your test! Hah!

With this test, we see it spike up to 10% for a big string, but a big buffer
still stays well below 1%. <https://gist.github.com/2339010>

I did it in coffeescript to just rule that out as a possible source of work.

The moral of the story is that Buffer<->string conversion is WAY too expensive
in Node, and leads to situations where the easy approach leads to poor
performance. That's a problem. Thankfully, it's one we already are aware of,
and plan to fix, and it's easy to work around by just using a buffer rather
than doing extra conversions.

~~~
devrim
hi isaac,

thanks for responding, it's nice of you to acknowledge it as a bug, and say
you're working on it.

> First of all, writing bug reports to Hacker News is usually a bit like
> complaining about the government at a bus stop.

just a little note on this one, i've made clear in my original writing this is
not hate and i sent an email to substack, and ryan to some others i know. i
guess i will send you an email next time as well.

not filing as a bug: you are right, i should have done that.

Buffer<->string conversion is WAY too expensive in Node: yes it is, hope you
can make this way better, because we don't want to hit cpu each time we pull
data from a e.g. database.

writing it to hackernews/stackoverflow: it's because i'm asking for solutions
outside of nodejs domain, varnish,nginx or any other way this issue can be
worked around. i'm sure you understand. and this was quite helpful as we're
now deploying a few varnish and nginx servers to mitigate this problem.

~~~
IsaacSchlueter
Sorry if my response was a bit overly strong. I know you're a good guy and we
all just want these things to work well. I wasn't offended, just a bit
confused by the results that weren't matching what I was seeing.

Of course, poking a bit further showed that converting a 16MB string to a
buffer actually is quite expensive. You can avoid this by using buffers
directly when you're serving files (or better yet, just pipe a fs.ReadStream
directly into the response, so it'll stream it as it reads from disk, and
buffer as little as possible.)

------
benologist
You could put nginx in front of nodejs and let it handle the static files,
it's far better at it.

This is worth reading as well:
[http://engineering.linkedin.com/nodejs/blazing-fast-
nodejs-1...](http://engineering.linkedin.com/nodejs/blazing-fast-
nodejs-10-performance-tips-linkedin-mobile)

~~~
1SaltwaterC
Besides that, node can listen on UNIX domain socket, while nginx can proxy the
connection over UDS, therefore people can avoid the TCP stack. That should
also shave off some latency. It may throw some 502 errors if is not configured
properly, but with sysctl tricks or Ryan's patch for nginx
(<https://github.com/ry/nginx-ey-balancer>) it is doable.

------
staunch
"complicate" your lives? nginx is an amazing piece of technology and it's
extremely simple. It's _built_ for serving static files (and doing other
things) better than anything else that exists. Use the right tool for the job.

~~~
devrim
i wasn't saying nginx is bad (nginx is awesome). i just wouldn't want to
deploy and configure nginx with my app, if i can avoid it. in fact i got an
awesome suggestion over at stackoverflow, that put the usage down to 0.3%
<http://d.pr/dXVR>

~~~
bmelton
If it's an app you in tend to package up, or is portable, that might be an
acceptable solution, but in the long term, using Nginx or Varnish (or even
Apache) is likely the better answer, unless it is your objective to build a
static file server in Node.

Otherwise, even if you can speed it up, and/or minimize its memory usage,
you're still wasting cycles and memory in your app that needn't be spent on
it. To a point, this is perfectly fine, but at some point, you're (hopefully)
going to want to scale, and I promise that this is the first thing you'll want
to look at.

------
jameswyse
Node isn't really built for serving static files.

I'd say the easiest solution would be to stick varnish in front of your node
application.

~~~
cheald
That's true if you also want a caching proxy in front of your stack, but if
you just want to serve static files, nginx in front of node.js is a perfectly
acceptable solution.

------
aristus
[http://www.kernel.org/doc/man-
pages/online/pages/man2/sendfi...](http://www.kernel.org/doc/man-
pages/online/pages/man2/sendfile.2.html)

------
thinknlive
// response : http response stream // filename : the static file we want to
serve

response.writeHead(200 / _, ...set http header info (mime, length etc)..._ /
); stream = fs.createReadStream( filename, { flags: 'r', start: start, end:
end }); stream.pipe(response);

