
How Raygun increased throughput 20x with .NET Core over Node.js - aliostad
https://raygun.com/blog/increased-throughput-net-core/
======
markrendle
Here's what I read: they had a solution written in Node.js; they rewrote the
solution in .NET Core; it was 20x faster. It's anecdotal, sure, and Microsoft
prompted them to write it, and YMMV, but they did a thing and it had a
remarkable result so they've remarked on it. It's not realistic to expect them
to take the time to build a representative sample that doesn't expose any of
their proprietary business logic; they're running a business and have more
important things to do.

Anecdote of my own: I was working on a web bug that had to generate a few v4
UUIDs on every request, and using a version of Node for which the libuuid
wrapper wasn't working, so I was using the fastest script-based generator I
could find, but it was still too slow. A .NET Core version of the same code
handled something like 40x the number of requests on the same hardware.

If nothing else, it demonstrates that using the same solution for all your
different problems is A Bad Thing, because there are surely things that .NET
Core is not particularly good at either.

~~~
jrs95
The biggest problem with .NET Core imo is that .NET just seems to be
permanently uncool, despite how good it is. Maybe one day this industry will
be less fashion driven...

~~~
GordonS
Yeah, but not everyone is a hype-train riding, super-cool hipster intent on
switches languages every 6 months as some kind of fashion statement.

~~~
jrs95
They aren't, but there's still a lot of irrational hate for anything even
remotely associated with Microsoft, even amongst people who aren't chasing the
hype. I can't really think of any other major general purpose platform that
has this kind of PR problem.

------
endorphone
This, and the AgeOfAscent piece it links to, both read like PR pieces
commissioned by Microsoft. Both are absent of actual information to really
determine what exactly they were testing, or what this means to anyone else's
problem space.

Because, of course, when you get to the fun of benchmarks there's always a
faster options. With .NET Core offering them 20,000 requests per second (on a
C3.large), that is a terribly low bar to hit. Again, maybe they're doing
something amazing, but many frameworks have rates in the seven digits on that
sort of hardware. And I know .NET Core can process basic requests in the six
digits, so even it is hardly the limiting point, and it comes to the logic.

[https://www.techempower.com/benchmarks/#](https://www.techempower.com/benchmarks/#)

Still terribly flawed, but better than someone rewriting an app and then
gloating about speed improvements.

~~~
venning
With respect to the TechEmpower benchmarks, if you look at Round 13 [1] they
note that ASP.NET got 859 times faster in their benchmarks since Microsoft
dedicated effort to improving it.

[1] [https://www.techempower.com/blog/2016/11/16/framework-
benchm...](https://www.techempower.com/blog/2016/11/16/framework-benchmarks-
round-13/)

~~~
endorphone
Indeed (though it's worth noting that they're talking about the performance of
ASP.NET Core versus Mono, which was notoriously disastrous performance wise,
and that story got somewhat misrepresented). In no way am I saying that the
.NET Core is slow.

But if we want to use that benchmark, doing the most trivial useful thing of
all -- serializing a simple object to JSON -- and node.js beat core .net. In
the recent iteration it beat it by almost 2x (which makes the "node.js is slow
we all know that" bit in the linked piece humorous given that it beats
aspnetcore in every test but plaintext, which is an irrelevant test anyways
given that they put their service behind nginx).

I don't use node.js. I don't advocate it. I personally think most node.js
solutions end up being a spaghetti mess. But these sorts of "the details are
hidden but look at the magic we wrought!" benchmarks are worse than useless.
They're fools gold for people choosing platforms and thinking this sort of
advocacy is guidance to learn from.

------
scandox
> when we started to look at .NET Core in early 2016, it became quite obvious
> that being able to asynchronously hand off to our queuing service greatly
> improved throughput. Unfortunately, at the time, Node.js didn’t provide an
> easy mechanism to do this, while .NET Core had great concurrency
> capabilities from day one.

This is a mildly maddening article - not because I have any emotional
attachment to either platform - but because it never really gets to the heart
of the matter. It's like they can tell us, but they'd have to kill us
afterwards.

~~~
andrewstuart
I'm with ya. All hype, no details.

A 20X improvement cannot be attributed to .NET core, it must attributed
substantially to some implementation/algorithmic issue that they resolved once
they got to the .NET core platform. There's a big problem somewhere with
node.js if it is 20X slower. If true, then its simply a bug or series of bugs
to be fixed in node.js.

~~~
UK-AL
Just a basic request/response with barely anything thing in it, shows the .NET
core is faster than node.

[https://stackoverflow.com/questions/43920942/unexpected-
outc...](https://stackoverflow.com/questions/43920942/unexpected-outcome-of-
node-js-vs-asp-net-core-performance-test)

Obviously this isn't very scientific without a benchmark suite.

~~~
jamra
It's not 30 times faster. Also, it doesn't matter which one is faster unless
there is. Huge speed improvement. Your web server is not the bottleneck.

~~~
UK-AL
"Your web server is not the bottleneck." \- Highly depends on your setup.

But removing any 3rd party libraries, and external requests. Generally .NET
core is faster.

~~~
rubber_duck
Actually 3rd party libraries and application code is where .NET is likely to
be faster unless node lib is wrapping a C library - .NET can easily beat JS an
order of magnitude simply by having control of memory layouts and high
performance/specialized data structures. Meanwhile JS has crap structures
(doesn't even have primitive arrays) and everything is it's dynamic object
where you pray the JIT eventually figures out a constant structure and tries
to optimize.

~~~
Klathmon
Just to nitpick a bit, JS does have primitive arrays using TypedArray.

~~~
rubber_duck
This is a good point - it's also why you can still optimize JS on
microbenchmarks but for real world code where you'd want to use an array of
structs in say C# for eg. there's just no JS equivalent other than write
obfuscated stuff around typed arrays.

------
moomin
I'm a .NET fan, but it sounds strongly like these guys knew what they were
doing on .NET and not so much on node. It's amazing how many of these rewrite
stories actually boil down to "and the new version didn't do some incredibly
dumb things the old one did".

If you really wanted throughput and large numbers of simultaneous connections,
you'd be looking at Erlang, anyway.

------
vorpalhex
The fact that they were only getting 1k reqs/sec with Node gives me concern.
It clearly shows something went wrong there very early on at a very
fundamental level. By no means is Node the end-all be-all for performance by
any measure, but you should definitely be getting much much higher throughput
than 1k reqs/sec.

Simply booting up a single core http server should net you around 4-5k
requests per second. Spin up an instance per core and you should be _at least_
in the 10k realm.

~~~
kompuuter
> The fact that they were only getting 1k reqs/sec with Node gives me concern.
> It clearly shows something went wrong there very early on at a very
> fundamental level.

Because the program is performing CPU compute for each request. Node is single
threaded.

It's a shame that the Node project killed off the multithreaded web workers
pull request. It sorely needs that functionality. Pools of node processes are
a poor substitute.

~~~
ruffrey
Looking at my Prometheus stats right now, I've got a Node TCP server doing 12k
concurrent connections with 3-5% CPU and 130 MB memory. At least 4 DB queries
are done per request, sometimes 10 queries. Raygun either has some nonobvious
stuff going on, or Microsoft paid them to write PR fluff. After 6 years with
Node it seems odd for their performance to be so bad. They should dive into
more technical details to explain what specifically was made faster.

~~~
uppity
You've described a classic I/O bound server application (waiting on the
database) that Node handles well. As soon as you introduce serious
computations or server side rendering into the request handling performance
would fall dramatically.

~~~
zamalek
And Raygun, according to what the website says it does, is doing a significant
amount of work.

------
JTenerife
Obviously they did something terribly wrong in the first place and now are
showing off being all right. Netty would probably faster. As there're no
details at all, I feel free to just guess :-).

I recently found Node much faster then .NET Core for a very specific scenario:

Elasticsearch - Node / ASP.NET Core (Kestrel) - NGINX - Client

where every connection must be TLS. So, Node / ASP.NET Core have to decrypt
traffic from Elasticsearch and encrypt to NGINX. For a minimal workload the
whole trip took 50 ms with .NET Core and 20 ms with Node. Obviously there's
something wrong with the .NET setup - maybe some setting with Kestrel and TLS.

Anyway, that's my anecdote. And I wouldn't dare writing a block post how node
is 2,5 times faster than .NET.

------
bluedino
>> Node is a productive environment, and has a huge ecosystem around it, but
frankly it hasn’t been designed for performance.

Isn't/wasn't part of the hype behind node that it's fast? Compared to what?

------
rikkus
Has anyone any code they can show which performs the same job on both and is
close to comparing apples to apples? It'd be interesting to me, not for
performance comparison, but as a real world application structure comparison,
a kind of more advanced Rosetta Code[1] example.

[1]
[http://rosettacode.org/wiki/Rosetta_Code](http://rosettacode.org/wiki/Rosetta_Code)

------
zbjornson
It looks like they ran their last node.js benchmark around Sep 2015. I'm
curious if this was still with node.js v0.x, or if they had already adopted
the latest version of io.js (3 or maybe 4).

~~~
drew-y
Or node.js v6.x for that matter. Seeing as io.js has merged back into node.

------
ruffrey
There is no technical information in this article. As a result it reads like
Microsoft marketing and is off-putting to .NET Core, something I'd otherwise
be interested in.

------
draw_down
> From the questions we received around the specifics of our performance
> improvements, there seems to be two schools of thought:

> 1\. Of course it’s faster, Node is slow

> 2\. You must be doing Node wrong, it can be fast

Allow me to offer a third :)

When you rewrite something, it had better be faster! You know a lot more about
how it works and how it's used than you knew at the outset. There was a recent
post, about going back to Ruby after creating the first version in Clojure,
that touches on this point.

~~~
GoodbyeEarl
Hey! Where could I find this post?

~~~
draw_down
[https://news.ycombinator.com/item?id=14405132](https://news.ycombinator.com/item?id=14405132)

------
xutopia
I never trust any benchmark stories from Microsoft ever.

~~~
cooldude67
Well there are plenty of public not run by Microsoft benchmarks that show this
same trend:
[https://www.techempower.com/benchmarks/#section=data-r14&hw=...](https://www.techempower.com/benchmarks/#section=data-r14&hw=ph&test=plaintext)

------
garganzol
The article is an excellent example of sneaky PR practices. Not only it lacks
the technical details, it deceives the masses with insane amounts of
artificial hype. "How AstroTurf invented astroturfing" would be a better title
for it.

