
C# .NET Core versus Java fastest programs - throwaway13337
https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/csharp.html
======
runfaster2000
.NET Team member here. We don't look at these specific benchmarks. Our belief
is that they are wild west w/limited moderation of what is reasonable/real-
world and not. Not necessarily a reasonable comparison, but we focus on
TechEmpower[1]. You can check out the latest .NET performance improvements in
our latest blog post on the topic [2].

[1]
[https://www.techempower.com/benchmarks/](https://www.techempower.com/benchmarks/)

[2] [https://devblogs.microsoft.com/dotnet/performance-
improvemen...](https://devblogs.microsoft.com/dotnet/performance-improvements-
in-net-5/)

~~~
jonathanoliver
I am thoroughly impressed with you what your team has accomplished. Roughly
7-8 years ago, the company I work with switched away from .NET (to Go) largely
because our performance with IIS for HTTP traffic running .NET was...poor by
comparison to other available options. I wrote a blog post[1] on the subject.

It's been exciting to watch the hard work you and your team are investing to
make .NET (Core/Framework/etc) faster. In fact, virtually every single issue
or point I raised in my blog post has been addressed to my sanctification and
then some. The Techempower benchmarks show the massive strides you've all made
and now I'm delighted to see .NET consistently performing at or near the top
of the benchmark even without heavy customization/modification. To this day, I
still really love C# as a language and there are a number of significant
things in C# that I wish the Go ecosystem would adopt. If/when bootstrap a
technology stack again, you can bet that C#/.NET will be the top contender on
the list of options.

Thanks again for all of your effort. It is very much appreciated and admired,
even if I'm not using that tech stack at this point.

[1] [https://blog.jonathanoliver.com/why-i-left-dot-
net/](https://blog.jonathanoliver.com/why-i-left-dot-net/)

~~~
whycombagator
> Roughly 7-8 years ago, the company I work with switched away from .NET (to
> Go) largely because our performance with IIS for HTTP traffic running .NET
> was...poor by comparison to other available options. I wrote a blog post[1]
> on the subject.

After reading this quote, and as a former .NET developer, I was really
interested in the performance issues/story your blog would go in depth about.
Instead, I found it strange, as I skimmed your blog, that I didn't see any
specific scenario or perf numbers/bottlenecks discussed (seeing as you
specifically state your company moved away from .NET due to IIS perf issues) -
it was very generalized/hand-wavey. Do you happen to remember anything
specific? Or were the reasons your company moved away from .NET literally the
contents of the blog post?

It seemed to me that your frustration was more surrounding the MS bloated
tooling &/or windows.

Some quotes from your blog:

"Once I used TFS on a small job and it was a monster--just like everything
else from Redmond. It infected my project files and contaminated my source
directories. Yuck. No thank you. Give me command line git any day...or perhaps
SourceTree if you need a little love from a GUI."

You could definitely have used git back then.

"As a .NET developer, when you start a new project there are a few things
you'll typically do (by hand, of course):

    
    
        - Create a new solution
        - Add it to Team Foundation Server
        - Create a corresponding website entry in IIS
        - Create a new SQL Server database
        - Associate the Entity Framework with the solution (typically for projects created after 2010)
        Start designing your database and your ActiveRecord entities."
    

You absolutely did not have to use EF or TFS, even back in 2014.

The post ends with:

"And that's why I left the .NET Framework because it kept reasserting itself
and wanting to be more than it was: an implementation detail."

~~~
fendy3002
It's usually the company policy that use TFS for C#, not developers. Usually
it's the Microsoft marketing scheme to include .NET solutions with TFS.

~~~
whycombagator
> It's usually the company policy that use TFS for C#, not developers.

I agree, but remember we are specifically talking about a company that
switched it's whole business off .NET. A company that makes such a decision is
probably one that could also make the call to switch version control systems
without changing stacks.

------
magicalhippo
Only a few seems to be plain code vs plain code. For the C# code there's use
of unsafe, SSE/AVX intrinsics and calling C code to do the job (PCRE for the
regex).

Certainly it's nice that you have these capabilities in C#, but at least for
me it limits the usefulness of the results.

~~~
zamalek
Handicapping one language to the featureset of another is unreasonable. You'd
also have to disable a host of JVM features (such as hotspot) for a "fair"
comparison, which is ridiculous.

Calling C is unfair, though. JVM can do the same and it should be implemented
the same way.

~~~
jsiepkes
> Calling C is unfair, though. JVM can do the same and it should be
> implemented the same way.

If you want to benchmark C code you can just benchmark a C application. You
run stuff in a VM like the JVM or CLR because you want things like memory
safety. If you allow native code in something like a regex benchmark then all
your benchmarks are basically about: How fast can the JVM/CLR call native
code. And say absolutely nothing about how fast the VM itself is.

So using native code in these benchmarks is just silly and basically makes the
whole thing meaningless.

~~~
plorkyeran
How fast your managed code can call native code is an extremely relevant
performance detail sometimes. It's far from the only thing you should
benchmark, but it is a useful thing to benchmark.

~~~
magicalhippo
Absolutely, but then that should be a separate benchmark, designed to test
exactly that.

Same with intrinsics. Have a benchmark that tests how good are the intrinsics,
are there any significant overhead etc.

When I look at generic names like "n-body sim" I'm thinking ok, let's see how
generic mixed floating-point code compares between the languages.

------
xiphias2
Just looking at some .NET programs, they all use unsafe code, which is always
faster than using garbage collected memory management, so the results aren't
surprising.

~~~
zelly
It's using features of the language. Is that not what this is meant to
measure/compare? .NET has unsafe, SIMD, and value types which opens it to a
whole world of programming that Java is currently unsuited for.

~~~
derefr
One would think that a benchmark described in reference to the performance of
the virtual machines themselves, would be measuring the performance of pure
bytecode (or a JITing thereof), not the performance of native code linked into
the runtime.

~~~
jashmatthews
Using a native regex extension lib is the norm for "scripting" languages so
totally reasonable to benchmark like that. How do you define when it's OK to
use a native extension and when it's cheating?

------
est31
Seeing the debian.net domain name, I'm a bit saddened to be reminded that
DotNet Core is open source, yet still not available on Debian without
installing packages from external sources (like the deb packages MS provides):

[https://bugs.debian.org/cgi-
bin/bugreport.cgi?bug=779970](https://bugs.debian.org/cgi-
bin/bugreport.cgi?bug=779970)

[https://bugs.debian.org/cgi-
bin/bugreport.cgi?bug=779969](https://bugs.debian.org/cgi-
bin/bugreport.cgi?bug=779969)

Right now the official install instructions want you to trust Microsoft keys
and I'm not really a fan of that. Further, it installs a custom package repo
and in the past I've had very negative experiences with PPAs so I tend to
avoid them.

[https://docs.microsoft.com/en-
us/dotnet/core/install/linux-d...](https://docs.microsoft.com/en-
us/dotnet/core/install/linux-debian#debian-10-)

~~~
moron4hire
> Right now the official install instructions want you to trust Microsoft keys
> and I'm not really a fan of that.

Why?

~~~
est31
Because I don't want to trust Microsoft more than I have to. It'd be the only
non-distro provided keys on my computer.

I would subject myself to them. Maybe in 5 years they decide that to download,
you need a Microsoft account login?

This weakens one of the core strengths of GNU/Linux, that every software
installation is only an apt get away. Also, Debian packaging does a level of
vetting that the software complies by their packaging policies, which ensures
that it is truly libre.

I know that Microsoft has their UEFI signing keys in all the IBM computers
around the world and has signed the desktop Linux bootloaders as well, and I
respect that they do this, but that's a different part of the stack. I
understand that _some_ entity needs to sign the bootloaders, and Microsoft has
since shown itself to be a reasonable guardian.

------
smabie
Everyone just games these benchmarks by calling out to C libraries. It's
better to actually compare the slowest programs for each language instead of
the fastest ones, because those are usually idiomatic.

~~~
igouy
> Everyone just games these benchmarks by calling out to C libraries.

Really?

Please say which of the n-body programs do that.

[https://benchmarksgame-
team.pages.debian.net/benchmarksgame/...](https://benchmarksgame-
team.pages.debian.net/benchmarksgame/performance/nbody.html)

Meanwhile, sort by program source code size —

[https://benchmarksgame-
team.pages.debian.net/benchmarksgame/...](https://benchmarksgame-
team.pages.debian.net/benchmarksgame/performance/nbody-gz.html)

------
shrewduser
I took a quick look at the code, it's nothing alike.

It seems they have people submit solutions to the problem and compare the
fastest implementation for each.

~~~
bArray
And the fastest implementations aren't the fastest either, I was able to knock
about 10% off one of the Java examples from 2010. I suspect the C# folks were
a lot more motivated to "prove" that C# was no longer slow.

As others have pointed out, if you want to test C#'s ability to call
C-functions in an unsafe way, then fair enough. Otherwise, with memory safety
enabled, they are roughly comparable.

That said, I did port the C# version to Java and it added about 20% extra
time, even with some of the cheats (algorithm optimizations) it has
implemented.

~~~
igouy
> I suspect the C# folks were a lot more motivated to "prove" that C# was no
> longer slow.

One person became curious and was brave enough to contribute their programs.

------
5986043handy
Take with a grain of salt, JVM tuning is an entire art and science, probably
same with CLR.

~~~
adamnemecek
Presumably, these are comparably tuned.

~~~
bremac
The .net benchmarks use architecture-specific SIMD intrinsics. There's no
equivalent on the JVM, though JEP 338 aims to add a vector API.

------
Dork1234
Maybe I missed something, but there doesn't appear to be a benchmark between
JNI, JNA, and pInvoke. Most my experience with .NET/Java has shown it is fast
enough in most cases, but then run into areas that need to be optimized into
Native Code. This calling of Native "C" code can be slow with JNI and
generally just fast enough for .NET pInvoke. I haven't ever had a reason to
test JNA.

------
sercand
Unity also shows how you can use C# for high performance game code with their
custom burst compiler. They even switch some of their C++ code to C#.

~~~
Churb
They’re moving away from the managed runtimes and compiling straight to native
code — it’s not really comparable to what is being discussed here.

------
zucker42
I thought I remember a post about them shutting it down from a few years back.
Did they reopen it, or did I just fabricate that memory?

~~~
Legogris
Who are "they" and what is "it" here?

~~~
zucker42
Oops, I reworded my question and removed the antecedents.

"It" is the benchmarks game and "they" are the maintainers of the benchmarks
game.

------
Churb
.NET core was slower than mono which was slower than JVM for running real time
multiplayer games (3d math, serialization, reading and writing from multiple
streams) as of a couple of years ago, did something big change With the
runtimes very recently or is this just coding to benchmarks?

------
tyingq
The regex-redux one looks like more of an endorsement for the PCRE2 jit.

~~~
igouy
Like compare `regex-redux #8` and `regex-redux #5` ?

[https://benchmarksgame-
team.pages.debian.net/benchmarksgame/...](https://benchmarksgame-
team.pages.debian.net/benchmarksgame/measurements/csharpcore.html)

------
exabrial
These are pretty hilarious.

~~~
exabrial
Did you guys actually read the code in the benchmarks before downvoting me?
Both sides are terrible.

~~~
igouy
Please be specific.

------
chromedev
So do Go, C, and many others.

------
axaxs
JVM clone edges out JVM. Fantastic.

