
Benchmark: Java, Python, PHP, C, JavaScript, Swift, Ruby, Perl, Go, Lua, Rust - filipmandaric
http://benchmarksgame.alioth.debian.org
======
FrozenVoid
Years ago i was naive enough to believe these benchmarks. Now i know these are
Potemkin-village level stunts and political maneuvering by language activists
and staff of that site. If you really want actual benchmarking you have to do
it yourself and read the .asm output for comparison. Another caveat is that
some languages are genuinely faster in one hyperoptimized case(even faster
than C) but fail horribly when things go more complicated and abstraction
creep makes them much slower in most cases.

~~~
olegkikin
You provided zero evidence for your claim. I'm not saying you're necessarily
wrong, but so far you have only made an assertion.

If you think some language is misrepresented in that benchmark, you can submit
your own solution.

~~~
jaredklewis
The problem is subjective and thus more difficult to provide evidence for.

I don't think the parent doubts the runtimes or is claiming that the code for
his pet language wasn't written properly and therefore the site is a sham. The
issue is: what is reasonable code?

Benchmarks invariably become a game of massaging the code to produce the
optimal assembly. And that's the problem. The reason that we invented higher
level languages was so we could stop thinking about assembly. The point of
most higher level languages is not to optimize CPU cycles, but to optimize
human brain cycles. If we only cared about speed (and not at all about
development time), we would just write all code in assembly.

I am not particularly interested in how fast meticulously fine tuned code in a
given language runs. I have no doubt that, given enough time, an excellent
programmer can finely craft a block of code to run very fast in any half
decent language. And this site is evidence of that.

What I want to know, and what benchmarks never show, is which languages make
writing performant code simple, idiomatic, and natural. Which languages give
me good performance, without having to really work for it?

This is an amazing site, but it is a stretch to think that it compares
languages. It compares programmers and their implementations. The
implementations are more like their own art form, like Chess or something.

~~~
olegkikin
Have you looked at the winning submissions code? Every single one I saw (not
all of them) was just a straightforward implementation of the algorithm /
solution. I don't see much weirdness.

~~~
jaredklewis
Yes. For example I noticed the typescript one. The implementation is
straightforward, yes, but it is just plain javascript. It doesn't need TSC at
all. So why call it typescript? If you're going to say it is typescript at
least use types, no?

Or the ruby ones. Not much like ruby I have ever seen. It's not particularly
difficult to write C code that can be easily called from ruby which is why I
guess most ruby programmers wouldn't bother to write code like this in ruby.
If you want C, just write C!

I think all the implementations are straightforward as implementations. But
few are the most idiomatic way to express the alogorithm in that language.

~~~
igouy
Specifically which "typescript one"? (Currently 15 typescript programs are
shown.)

Specifically which "ruby ones"? (Currently 37 programs are shown.)

>> the most idiomatic way <<

One person's idiomatic is another person's idiotic.

------
SwellJoe
Hey, the benchmark game got a new website (at some point in the several years
since I looked at it last)! Almost simplistic, but nice to read.

As someone that works predominantly in a language long considered passé
(Perl), I always kinda relish seeing that it is still faster than Python and
Ruby for many problems. As Python and Ruby have gotten faster, I'd sort of
assumed they were both notably and clearly faster than Perl, by now, but
that's not the case though the difference is now quite small and probably
debatable.

~~~
Waterluvian
Python was never really designed for speed. It was designed for simplicity and
clarity of behaviour. The beauty of the age we live in is that we have such a
broad array of tools to apply to different problems. Performance is often not
the problem that needs solving, but when it is, you've got many options too!

~~~
SwellJoe
I'm not disparaging Python (or Ruby). Merely taking small pleasures where I
find them. I work in Perl because of a huge array of existing code in Perl,
not because I hate Python or Ruby (I've built stuff in Python and Ruby, too,
and like them both).

------
Xoros
So PHP is faster on most of those test than python or ruby ? I'm I missing
something ? I thought everyone hating PHP ( also) because it was slow.

~~~
nodesocket
No, PHP is actually really fast. Especially HHVM[1] which is crazy fast.

[1] - [http://hhvm.com](http://hhvm.com)

~~~
mappu
Ever since PHP 7, there's not much difference anymore.

------
dpratt
Any benchmark that includes a JVM language always makes me look for a
disclaimer that they allowed for proper JIT/warming time by running the
benchmark a few times in the same VM instance before collecting the numbers. I
may have missed it in this article, but I didn't see it.

Yes, startup time is a concern with the JVM, but that's not what it's built
for - it's meant for long-runnning repeatable code paths.

~~~
alayne
He doesn't do warmups for JITted languages.

~~~
peterashford
...which means the measurements are meaningless. They will include the time to
analyse the hot paths and then compile them.

~~~
igouy
They _will_ include a few tenths of a second.

Note the timings are seconds and tens-of-seconds.

------
z1mm32m4n
I see that the point of this site is supposed to be that a change in
implementation (keeping the language itself held constant) greater affect on
performance than rewriting in another language.

My problem with this site it makes poor support of that claim. The data is
hidden behind loads of links, and none of the data is visualized at all. Even
a few simple bar charts, keeping the hierarchy of the site otherwise unchanged
would vastly aid comprehension. The points being made would come across much
more easily.

This sort of data is the poster child for visualization, yet this site
relegates the data to tables of numbers.

~~~
igouy
What on the website tells you _that_ is the point?

>> Even a few simple bar charts <<

[http://benchmarksgame.alioth.debian.org/u64q/which-
programs-...](http://benchmarksgame.alioth.debian.org/u64q/which-programs-are-
fastest.html)

>> would vastly aid comprehension. <<

Is it possible that you are just wrong?

Is it possible that simple bar charts do not aid comprehension but encourage
thoughtless instant conclusions?

------
nodesocket
The Go vs JavaScript lineup is quite stunning. Go destroys Node.js in memory
usage and performance on every test except regex-redux (edge case?).

[http://benchmarksgame.alioth.debian.org/u64q/compare.php?lan...](http://benchmarksgame.alioth.debian.org/u64q/compare.php?lang=go&lang2=node)

~~~
Safety1stClyde
> The Go vs JavaScript lineup is quite stunning. Go destroys Node.js in memory
> usage and performance on every test except regex-redux (edge case?).

Why would it be stunning that a programming language compiled to assembler can
beat a broken-by-design interpreted language which doesn't even have integers
or arrays? The stunning part is that the people behind node.js have managed to
get that much performance.

~~~
alayne
That's not a very fair or accurate description. JavaScript implementations use
a JIT which is different than what people usually mean by interpreted. Types
can be inferred without explicit type declarations. JavaScript has Array and
integer semantics can also be inferred.

~~~
weberc2
Also JS has received many times the investment of Go.

------
joliu
Ruby's fastest time for pidigits was 3.14, haha.

------
frik
With this new revision this great benchmark got less useful.

Why can't I compare:

    
    
      Lua vs LuaGIT
    
      PHP5 vs PHP7 vs HHVM vs Hack
    
      Perl5 vs Perl6
    
      GNU GCC vs Intel C vs CLANG
    

This new rule "one implementation per language" is so stupid!!!

@debian: change it back

~~~
igouy
_" If you're interested in something not shown on the benchmarks game website
then please take the program source code and the measurement scripts and
publish your own measurements."_

------
jackyb
Does anybody know why Go binary-trees is so much slower than Java's?

~~~
trishume
Binary-trees is allocation bottlenecked, and Java has bump pointer allocation
where Go has slow allocation. The root cause being that Go's GC is non-moving
so it can't do bump pointer allocation without fragmentation, so it has to do
a whole bunch of computation to find a place for each node.

~~~
jackyb
I see, thanks!

------
insulanian
Wow, I expected Erlang to score better, considering the claims of Elixir
community how fast it is... compared to Ruby. OTOH maybe that tells more about
Ruby.

~~~
rubyn00bie
Erlang makes no claims about being fast in these sorts of tests. They aren't
using what it was designed to do.

Also some languages like, Ruby, do really well on some of these tests because
the C-code underneath has been written well (example regexes are just as fast
under ruby as they are anywhere else because the underlying C-library kicks
ass though I forget the name of it).

~~~
smitherfield
You're probably thinking of PCRE, although Ruby actually uses Onigmo:
[https://github.com/k-takata/Onigmo](https://github.com/k-takata/Onigmo)

------
bedros
where's D-lang? it would be interesting to compare D with C, C++, and Rust

~~~
wulfklaue
Good luck on seeing D on that site. There seems to be a active war between the
sites maintainer and D. There used to be D benchmarks on the old site and
people tries to release new code to the site, but for some reason the current
maintainer has a issue with D.

And he thinks its no unfair that D is not resented when a lot of people look
at his site and think: "If your not here as a language, then the language must
be bad".

Good luck on changing his mindset because i have seen topics going back years
regarding D not being there.

If you want a more impartial benchmark set:

[https://github.com/kostya/benchmarks](https://github.com/kostya/benchmarks)

And where is D on those tests ;)

~~~
igouy
> There seems to be a active war between the sites maintainer and D.

No.

> There used to be D benchmarks on the old site

Back in early 2008.

[http://web.archive.org/web/20080730204710/http://shootout.al...](http://web.archive.org/web/20080730204710/http://shootout.alioth.debian.org:80/gp4/)

Look how many other language implementations were also shown back then and
have not been shown since.

> … for some reason the current maintainer has a issue with D

No.

> Good luck on changing his mindset…

Make the measurements you want to see and publish them for everyone else to
see.

------
rhlala
I readed python was slow, but i had no idea it was relativly That slow!

------
mike986
they should have included luajit it's almost a dropin replacement for most
projects

~~~
trynumber9
It was included in the past [0]. It was removed to simplify the process and
due to a "one implementation per language" rule.

[0]: [http://i.imgur.com/0xxOfVm.png](http://i.imgur.com/0xxOfVm.png)

------
rilut
No Scala anymore?

~~~
igouy
No success getting programs updated for newer versions -- so, no Scala
anymore, done.

------
rhabarba
No Lisp? :-(

~~~
susi22
They used to have Lisp & Clojure. No idea why it's been removed. Some is still
accessible:

[http://benchmarksgame.alioth.debian.org/u32/lisp.html](http://benchmarksgame.alioth.debian.org/u32/lisp.html)

[http://benchmarksgame.alioth.debian.org/u32/clojure.html](http://benchmarksgame.alioth.debian.org/u32/clojure.html)

~~~
igouy
>> No idea why it's been removed. <<

Lisp hasn't been removed.

(Clojure has. So's Scala.)

------
RandyRanderson
If you're a competent programmer, one should be able to make any _real-world_
task take about the same time in any two languages.

Therefore, any large-margin differences are benchmarking the programmer not
the technology.

~~~
clouddrover
> _one should be able to make any real-world task take about the same time in
> any two languages._

I don't think that's a practical claim. Interpreted or JIT compiled languages
depend on the interpreter or VM. JavaScript has become much faster over time
as JavaScript VMs have improved, independent of the details of any particular
JavaScript program. Ruby has become faster over time for the same reason. I've
seen Ruby programs I've written speed up significantly without any significant
code changes when I moved from an older to a newer version of Ruby.

