
.NET and Node.JS – Performance Comparison - bencxr
http://www.salmanq.com/blog/net-and-node-js-performance-comparison/2013/03/
======
pilif
It's funny how everybody posts their theories of why the benchmark is wrong
(or done by the wrong person) and nobody actually tries out the code they
think is better. So instead of guessing, let's implement that corrected
version of the node.js application and run the benchmark again.

In my case, I admit, I don't have Windows ready to reproduce the result, but
then I'm also not complaining about how the test was done, nor do I care about
this at the current point.

If I ever have issues with performance of an application, _then_ I will be
doing the profiling and evaluating other options. I certainly wouldn't use
some benchmark I'm too lazy to reproduce as a basis for selecting the
technology to use. Instead, I use the tool I'm most comfortable with, I see
the most advantages from or I just like best.

While the OP's benchmark might have been (and IMHO probably was) flawed, at
least they provided source code and documentation for us to improve and re-run
it. That's way more important than to know who they are and what their
motivations might be.

~~~
JPKab
Still, there is nothing that annoys me more than when someone who is well
versed in one toolset, and new to another toolset has the audacity to conduct
a comparison and pretend like they did it correctly.

I'm glad he provided source code, but he clearly doesn't know Node well enough
to provide an Apples to Apples comparison. Maybe .NET will blow Node away?
Fine, but make sure you know Node well before you do the test.

Still, it's a nice conversation to start. I don't care how much I hate the
company that builds it: if a tool is best for a job I want to know about it.

But your complaint that nobody is doing the tests: C'mon, how many Node
experts even have Windows laying around?

~~~
angersock
So, what's wrong with his Node code? What's the tell?

~~~
thedufer
He doesn't know that node is single-threaded. Take an embarrassingly-parallel
task (serving multiple unrelated requests, for example) and pit a single
thread against multiple threads. Guess who's going to win?

Depending on the size of the .NET's thread pool, this probably more than
accounts for the difference. The drastic inefficencies from not knowing what
the async library does are just icing.

------
chjj
Another "hey look, node.js isn't good at everything" post. I figured these
would have stopped by now.

Why are you using an async sort function when you're not doing anything
asynchronous in the sort callback? That's going to entail some overhead
obviously, and it also can't take advantage of the sort optimizations v8
implements. Are you trying to show the overhead the asynchronous sort entails?
Any overhead it causes would be easily outweighed by whatever asynchronous IO
it's doing in that sort function anyway. It is negligible in comparison: not
even worth pointing out.

> The key point I want to make is that I am using the async NPM instead of the
> default blocking Array.Sort.

I don't think you realize how this works. That async sort function is there to
help deal with sort callbacks that already have to do something asynchronous.
You're doing something synchronous and using that async sort function for no
reason. If you're going to run these benchmarks, use the synchronous sort
which can take advantage of certain optimizations. Or, you can also just do
something actually asynchronous, which would justify using an async sort in
the first place.

------
eknkc
YAY! .Net sorts bytes faster than Async.js sorts floats. In similar news, Go
seems to split strings faster than .Net sorts bytes on my laptop. Fascinating
stuff.

~~~
bazillion
And PHP serves an empty page faster than Go splits strings. I declare PHP the
winner by default.

~~~
eknkc
I have bad news:

\------------------

% time ./test.go "test1,test2,test3"

["test1" "test2" "test3"]

0.00s user 0.00s system 79% cpu 0.005 total

\------------------

% time php -r ''

0.03s user 0.01s system 93% cpu 0.044 total

~~~
dexen
...and I have good news :-)

    
    
        cd ~
        > empty
        php -S localhost:8080
        ## on the other console
        time { echo -e 'GET /empty HTTP/1.1\n\n' | nc localhost 8080 >/dev/null; }
        
        real    0m0.003s
        user    0m0.001s
        sys     0m0.000s

------
jrajav
.NET native list sorting and Node.JS async.sortBy - Performance Comparison

Fixed that for ya.

~~~
masklinn
And it's important to note that the comparison does not make sense: TFAA seems
to believe "async.sortBy" performs an asynchronous sort (e.g. delegates the
sorting to a threadpool) and that's how he implemented his own "SortAsync": to
perform a regular Array.Sort in its own task.

But that's not what async.sortBy does according to its documentation,
async.sortBy is used to sort using an async comparator/key function e.g. to
sort file names using a stat(2) call, the .Net version would have to sort
using an asynchronous IComparator of some sort.

edit: plus, for some reason the benchmark explicitly parses the input strings
to floats in javascript (and sorts using that), but seems to sort the raw
bytes in the .net version. The C# version further performs the string split in
the sparked task, where it's performed in request in JS (not that it'd help
since — as I mentioned — the sort itself is incorrect)

second edit: to be fair, I like neither technology and am not a specialist in
either, I have a pretty basic basic grasp of both and thus may have made
mistakes in my cursory reading, if I have and somebody knows better don't
hesitate correcting me.

But as you may see from the paragraphs above, as far as I understand the code
this comparison is complete nonsense (even ignoring the rather dubious higher
level case of reading a single big file in memory at once, sorting it and
extracting a bit of stat, which is unlikely to be IO-bound)

------
dexen
The author notes about himself:

 _> I work for Microsoft as a software engineer for Bing in Silicon Valley. My
group specializes in building platform and experiences for consumers._ (
<http://www.salmanq.com/> )

~~~
viraptor
Unless you can point to something incorrect or unfairly biased in that article
- why does this matter?

~~~
masklinn
It's a benchmark of .Net's Array.Sort to NPM's third-party async.sortBy, not a
benchmark of .Net to Node, and the test does not make sense: async.sortBy is
used to sort an array on an asynchronous function, it does not perform an
asynchronous sort (which is what the asynchronized Array.Sort does)

~~~
viraptor
I agree with that and it's a fair comment. I'm just opposed to bias just
because someone works for X.

~~~
masklinn
And I agree, character assassination is no good.

On the other hand, when the author has a hand in one of the pots it very
commonly results in a biased or incorrect comparison if only due to incorrect
interpretation of the other tech, so defaulting to dismissal is not a bad
heuristic either as first-order filtering to know whether there's a point in
spending time in evaluating the post.

Especially when there's no clear up-front disclaimer of the conflict of
interest, and here there is not (I went back to check, the post does not
specify he works at Microsoft, you have to go and check his about page and/or
blog history to realize that he's posting about MS tech all the time)

------
d4nt
A common cause of slowness in .net web app is multiple database round trips
during a single request, all happening in sequence. You _can_ write async
database calls in .net but they're not the standard / path of least
resistance, and by the time you app is hitting these issues, refactoring it
all is a big undertaking.

The results do not surprise me, but what I find more interesting is how
different ecosystems and frameworks can encourage your average developer to
write scalable or non-scaleable apps. And I still think node's model has a lot
going for in this regard.

~~~
CodeCube
With C# 5's async features ... a lot of code can now easily be non-blocking by
default. Even making asynchronous actions in asp.net mvc can have an
enormously positive impact on server scalability. Just because you can write
synchronous/blocking code doesn't limit the power of the platform when it's
written using the latest features.

------
tinco
This is terrible. You are benchmarking node.js on exactly the thing everyone
knows it is terrible at. Besides that yiur scenario is completely unrealistic
and your google efforts Should have tipped you off. There are no results for
sorting algorithms on node because nobody sorts on node.js. This is CS102..
Why would anyone ever sort more than 50 items on the server side of a web
application.

.net is faster than node but you absolutely failed at showing it.

~~~
bagosm
I don't program in nodejs but this is wrong on so many levels. First an honest
question: what exactly is it that nodejs is terrible at? Please define it.

Now granted every tool has its strengths and weaknesses this benchmark shows a
very large difference in operations that aren't uncommon in web servers.

Sorting in memory should concern you, since caching happens in the memory.
Again, I don't know the state of ORM technology in nodejs but any NHibernate
user will be able to tell you that level 2 caches are a common thing and they
will boost the benchmarks even further if you were to compare it to a nodejs
stack.

The only scenario where I see nodejs faster and better than .NET are the very
smallish script scenarios like a log receiver etc, where no state is needed
and minimal memory footprint is required

~~~
jeswin
It wasn't wrong on many levels.

\- For very obvious reasons, JS (or Python or Ruby) would be slower than Java,
C# at number crunching.

\- Node.js would handle concurrent connections better than most Asp.Net apps
since the framework is non-blocking. Now you could write non-blocking code
with .Net, but that isn't how most people write Asp.Net code. OTOH, with
Node.js that's the only way you could write an app.

\- I am impressed that Node.js actually performed this well in that benchmark.
For a dynamic, hard-to-optimize language, being 2.5x slower than .Net (or
Java) code is a fairly good result.

~~~
bagosm
So you are saying that nodejs is slower in general, and .NET is slower when
people don't know how to program in .NET.

To add on top of that, mvc 4 has extremely easy ways to set up async
controllers/tasks. However, these are not needed or even WANTED most of the
times. They are only really needed in I/O operations, a sector in which .NET
excels anyway since you can get ORM magic happening easily.

Also, don't think non-blocking is really non-blocking. What can nodejs when
it's waiting for the SQL lock to be lifted? (If there is any)

~~~
jeswin
\- Correct. I am saying that nodejs is slower in general. The reason you would
use nodejs is to save development time, and not to extract the last drop of
performance. It is for the same reason that people have written so many apps
on Python and Ruby.

\- I/O in nodejs is always asynchronous. I/O in typical .Net apps is almost
always synchronous, due to programmer choice. This approach is followed in the
vast majority of Asp.Net apps. So according to you, this would imply that most
.Net programmers don't know how to program in .Net.

\- I can't follow how "ORM magic" is related to I/O. The ORM magic I remember
has been a terrible experience for me when I had to use .Net (3-4 years back).
We have to write our own ORM to replace EF, since it was dog slow. Including
the LINQ expression tree parser, design-time tools etc. You can find this
result here: <https://github.com/jeswin/AgileFx>

I know that things finally got better after EF4. But I haven't worked much on
.Net of late.

~~~
bmuon
> I/O in nodejs is always asynchronous. I/O in typical .Net apps is almost
> always synchronous, due to programmer choice. This approach is followed in
> the vast majority of Asp.Net apps

Nitpick: actually Node lets you do synchronous IO
([http://nodejs.org/api/fs.html#fs_fs_readfilesync_filename_op...](http://nodejs.org/api/fs.html#fs_fs_readfilesync_filename_options)).
Sometimes you can chose to use synchronous IO, for instance to do something in
the initialization phase of your server.

------
antihero
Until Mono becomes 100% supportable, _none_ of the advantages of .NET outweigh
the misery of having to use Windows for development and serving.

~~~
viraptor
What do you mean by supportable? I was happily using mono for a number of
custom servers in a similar way a couple of years ago. You don't need any huge
frameworks to make mono really useful.

~~~
shadowmint
What, you mean, unless you're doing something useful like trying to write a
web app?

~~~
viraptor
Writing web apps may seem like the only thing people do lately, but it's not.
Given a choice of a number of languages and runtimes available 5 years ago for
example, .net/mono was a perfect choice for a daemon dispatching jobs to a
number of usb dongles where having a good ffi (p/invoke), nice io abstraction,
async actions and database interface was important.

This is something you write from scratch. No big frameworks are involved. If I
ever find I need MVC.net, I'll probably go with a completely different
language.

~~~
antihero
What if you already have code written in MVC.NET?

------
manojlds
One main thing I found missing was the OS that the tests were run on. I am not
even sure if Windows can be seen as intended OS for node.js. Was the
comparison between .NET on Windows vs Node.js on Windows? That might not be
fair on the node.js side.

PS: I am a .NET fan, so I want fair comparisons, if I were to use them :)

~~~
trust-me
I've run a few small test to see how Windows and Linux implementations of Node
compare and Windows came on top every time. I think Microsoft put a lot of
effort into a proper port, using Windows excellent I/O Completion Ports. Plus
I don't think Google could ignore Windows performance when developing V8.

------
Wintamute
So .NET with its massive, stable, mature native sorting libs is faster at
sorting than async.sortBy, which is about 25 lines of JS. Well, no surprises
there.

~~~
masklinn
Worse, it's a complete misuse and misunderstanding of async.sortBy: TFA
implements a "SortAsync" by performing a Sort call in its own task, byt
async.sortBy uses a task (an async callback) as its key function, so it calls
and waits on a callback for each item in the original array where SortAsync
TFA's SortAsync just blasts a single native Sort when the task runs.

------
quarterto
async.sortBy is for sorting the results of an array of asynchronous functions.
It is not an asynchronous sort. Let's see how it fares with
process.nextTick(function(){array.sort()}) (or better still, threads-a-gogo).

------
jroseattle
I find all the framework comparisons or this technology vs. that technology
somewhat interesting, but quite often useless in terms of decision making.
Evaluating very specific routines and functionality of framework A vs.
framework B is roughly equal to evaluating the performance of carburetors in
two racecars. If the only thing I'm interested in is carburetor performance,
then I doubt I'm going to win many races.

What comparison do I find useful? The one that incorporates everything my
application must address -- db queries, page rendering, cpu activity, memory
consumption, operations, maintenance, etc. Given that I need lots of things
for my application, single-point comparisons just really don't provide me with
much value.

------
davidjhamp
Javascript is slower at sorting numbers then .Net!

 _mind blown_

------
nicwise
Well, THIS is going to bring on the haters.

(keep in mind, microsoft haters, that microsoft is currently a HUGE proponent
of node.js - a lot of the Azure Mobile Services and the like are built on
node)

------
benmmurphy
i might have missed something but it looks like the .NET version sorts by the
string values and the nodejs one sorts by calling parseFloat on the string
values first. presumably this is going to slow down the nodejs version.

------
jeswin
In terms of VM performance, nobody would doubt that .Net (or the JVM) would
outperform V8; especially in number crunching. If anything, the benchmarks are
only proving that Node.js is fast enough.

However, I would guess that typical Node.js apps would be better at handling
concurrent connections than typical Asp.Net apps; since Asp.Net apps are not
usually written in non-blocking style.

------
meryn
I posted this comment: "It's obvious that you're new to Node.js.

First of all, you should be aware that Async.js is a mere flow-control
library. It does not offload work to separate threads, and neither is it able
to parallelize work. Internally, it mostly does bean-counting (but very
helpfully so).

As you can see in the source
[https://github.com/caolan/async/blob/master/lib/async.js#L35...](https://github.com/caolan/async/blob/master/lib/async.js#L359)
async.sortBy simply uses Array.sort for the actual sorting. The only reason
you'd want to use Async.sortBy is if the values that the array of "keys" is
not known beforehand (and needed to be loaded through io - asynchronously).
This is clearly exemplified in the documentation.
<https://github.com/caolan/async#sortBy>

The implication of this that your call to async.sortby can be replaced by a
call to array.sort. This will remove two unnecessary runs of async.map,
inflicting a potentially huge performance penalty.

You do need to pass array.sort a comparator function, otherwise it will sort
lexicographically (see [https://developer.mozilla.org/en-
US/docs/JavaScript/Referenc...](https://developer.mozilla.org/en-
US/docs/JavaScript/Reference/Global_Objects/Array/sort) ). That said, I'm not
sure what the actual contents of your input file is. In your .Net example, you
do not seem to bother to convert the array of strings to an array of ints (or
floats). I think that .Net sort will sort an array of strings
lexicographically as well. Furthermore, in the node.js example, you seem to be
content with returning the resulting median as an int, not as float. Do the
input "decimals" in the input file represent ints or floats? Do they all have
exactly the same amount of decimals? Are both the Node.js and .Net algorithms
doing the same thing? I think not.

Finally, we get to Array.sort. Array.sort blocks. Depending on the multi-
threading efficiency of the underlying algorithm of Array.sort (which I don't
have insight in), the code may not be able to use all available system
resources. Keep in mind that Node.js is single-threaded. I practically don't
know anything about .Net, but I assume it will magically start new processes
and or threads if the runtime deems this beneficial. For Node.js, you may want
to try the Cluster api, <http://nodejs.org/api/cluster.html> . You could try
seeing if performance increases by adding one or more extra server processes.

I can't comment about the quality of the .net code since I don't have any
experience with it.

I think it would be fair (and very educative to others) if you'd rerun the
benchmarks with 1\. Async.sortBy replaced with array.sort 2\. with both .Net
and Node.js algorithms fully doing the same thing (i.e. let them both sort
either floats, ints, or strings), and 3\. at least one extra server process
for Node. I think most interesting would be if you'd made the changes step-by-
step, and run the benchmarks at each step.

My guess is that step 1 would give the biggest difference. Depending on how
you decide to resolve the differences in the two algorithms, performance of
your .Net code may be slightly affected. It could potentially be speed up in
fact, if somehow it's able to sort ints (or floats) faster than strings. The
actual job of sorting probably overshadows it all though."

What do you guys think of this?

~~~
base698
Good break down of .NET Array.sort

It's kind of funny he talks about it being fast for due to non-blocking IO:

 _One of the key reasons most argue is that node.js is fast, scalable because
of forced non-blocking IO, and it’s efficient use of a single threaded model._

...then goes on and sets up a benchmark which is more dependent on CPU than
IO. Not to mention as mentioned here that benchmark itself is flawed.

In my experience Node is faster in the case of most web apps that select a row
from the db, read network requests to do aggregation, or update a column in a
db. Anything that does do CPU computation generally uses a native hook or a
different tech altogether.

~~~
JPKab
"...then goes on and sets up a benchmark which is more dependent on CPU than
IO. Not to mention as mentioned here that benchmark itself is flawed."

Thank you. It amazes me how some people can write code in a framework like
Node without even understanding the event-driven paradigm's strength and
weaknesses. I was sitting there reading it, and then he states he is using a
file sort??? WTF? As if anyone would use Node for that purpose.

------
octatone2
Parsing floats rather than letting JS loose typing handle it for you kills
your sort algorithm: <http://jsperf.com/parse-or-no-parse-sort>

~~~
quarterto
"<" on strings is lexographic.

~~~
thedufer
As noted elsewhere, lexographic sorting is how the .NET version is doing it.

------
tempoman
Technologies are tools in a toolbox. The bigger your toolbox is, the more
complex and wonderful things can be built. For any given problem there are a
number of parameters involved: available resources (skillsets, fun factor,
hardware platform, OS, budget restrictions), time to market, expectations on
scalability/security, functional requirements and so on. Thinking that one
tool is inherently more suited to all tasks and problems is the equivalent of
the old saying about the hammer.

------
richyzhang
I want to ask several question:

1\. How many threads does the .Net Runtime use?

I know little about .Net, but I know C# is like Java. The Jvm with servlet
will use many threads to serve, so that many cpu cores will be involved.

However, this is not the case for node. One node process will use just one
thread(thought it is not the exact fact).

So it's unfair int programming model.

2\. What's the hardware you use, how many cores is there?

------
meryn
It seems the blog had a kind of "invisible" filter on the comments, which made
it appear there were zero comments, or one (your own) if you had posted one.
Now a whole pile of comments have become visible (manually approved I think).
Funny to see all the reactions. Most commenters were probably not aware of one
another.

------
nabaraj
Comparing 'Subject A with one thing that is good at' with 'Subject B with one
thing that is bad at'.

Interesting!

------
malachismith
If you do research and analysis NOT to find out the answer to a question but
rather to JUSTIFY the answer you have decided (without evidence) is correct
then the value of the results is nil.

------
jasallen
I'm sure someone can write an optimized algorithm for Node.JS that will be
much faster than his (seemingly naive) implementation. Until then, I see a lot
of bluster.

------
brenfrow
It looks like right around 35 parallel request .net reaches the balmer spike.
<http://xkcd.com/323/>

------
scriptproof
The benchmark should has a new try further, when Asm.js will be added to V8.
Sorting arrays of numbers could have another speed.

------
htilford
Can't we all just get along: <http://tjanczuk.github.com/edge/#/>

------
suhair
On which platform (OS) the NodeJS is running?.

------
Glyptodon
And here I thought perhaps the largest driving force behind node was the
ability to use shared code on server and client.

------
rrouse
Is there some tech law that states that when someone does a benchmark or
comparison, it is almost always incorrect?

There should be.

------
hallomac2013
Microsoft trolling HN? trollface.jpg

------
sdogruyol
This benchmark makes no sense at all. Just a trolling of an official troll.

------
omniwired
nice try Microsoft!

