Those benchmarks are extremely suspicious. I remember comparing Perl and Python for bioinformatic work (non-trivial computational workloads), and finding that Perl was about 2x the speed of Python, on average.
Later, I did similar, non-trivial benchmarks with Python and Ruby, and found a similar, 2x factor. Ruby has improved since then, but unless Perl has become dramatically slower in the same interval, I suspect that these benchmarks are either trivial (i.e. simple loops), or badly written.
If you want to debug the metrics they're using, you're more than welcome to do it. I've got plenty of direct experience to question the results shown by a random webpage on the internet, and not much incentive to figure out why this particular set of benchmarks looks wrong.
A quick inspection of the tests suggests a high enough bogon count that it already begins to confirm my suspicions: there's a "fasta-redux" test for Perl which dramatically outperforms the "fasta" test that's implemented cross-language. Also, many of these tests are written in least-common-denominator style, which doesn't reflect the way the languages are actually used.
Your benchmarks are likely very biased. The vast majority of the work done by your bioinformatics programs should be done in C code, with thin wrappers so that the code can be accessed through Perl and Python. If anything, your benchmarks only comparing specific implementations, such as BioPerl vs BioPython.
Indeed. Peak RPS is what is going to be interesting. Depending on where traffic is from, it's very possible to have 12 mil requests in a day only from 9-5 type of hours and a big huge empty space during the night (This is the kind of traffic my company sees).
$ time I=$((I+1)) curl -s "https://duckduckgo.com/d.js?q=test&t=A&l=us-en&p=1&s=0" >/dev/null
I=$((I+1)) curl -s "https://duckduckgo.com/d.js?q=test$i&t=A&l=us-en&p=1&s=0" 0.01s user 0.00s system 3% cpu 0.256 total
I saw it go over 800ms a couple of times, anyway, I don't intend to run a DDG benchmark.
However, just for the sake of the discussion, about architecture and language choice etc, the interesting part would be to see how many rps before starting to degrade. There is no point bashing it and comparing it to other search engines that have more resources behind them.
DDG is mostly a front end for a bunch of search engines. Perl, PHP and Python are great to make front-ends like that. Because you are for the most part only dispatching queries, parsing xml feeds and build the gui for the end user.
Writing an internet scale search engine on the other hand would require different tools. Most successful projects has probably been done i C, C++ and maybe Java.
This shows Perl higher up the chart with it being 49% better (on average) than Ruby.
Some other things to note:
1. Perl has never been fast with binary trees. But Ruby 1.8 is even worse (I recall it being many times slower than Perl at this). So hats off to the Ruby 1.9 VM guys because they've turned things around and now it's even out pacing Lua on this benchmark! - http://benchmarksgame.alioth.debian.org/u32/performance.php?...
2. Fasta in Ruby maybe twice quicker than Perl however it uses over 100 times more memory! This is be because it's inlining some code (eval) giving it a huge speed bump at expense of a bigger program footprint. If I do a port of this to Perl then this fasta.pl runs 2.6 times faster than alioth's perl version (which now means the Ruby version is about 30% slower than it's direct equivalent Perl version).
3. My make_repeat_fasta idiomatic subroutine is actually a little slower than the one in fasta.pl on alioth. So both my fasta.pl & alioth's fasta.rb programs could be speeded up more :)
4. I may even be able to shave a little bit more off fasta.pl time. Same might be true for spectral norm & binary-tree however the Perl versions aren't showing up on alioth site at moment :( ... IIRC, when I last looked at the perl binary-trees code on alioth a couple of years ago I was able to shave 10% off.
On the contrary my dear igouy, specialisation can be key and knowing which tool in my toolset is best for different tasks is indeed insightful & helpful ;-)
re: contribute - Looking on my hard disk I see that I downloaded the bencher/shootout-scm back on 1st Jan 2010. However IIRC the process of contributing code back was a bit unwieldy. If some free tuits come my way then I may relook at it.
Frankly, i'm not convinced there's any point in investing time on improving the benchmarks there, considering the overall comparison directly contradicts the detailed comparison, by putting perl in a worse position than languages it outperforms.
Replying to myself because I want to add something interesting (to posterity) that I noticed today on Alioth:
Perl (or OS) was upgraded from 5.14.* to 5.16.2. From cursory glance this gave all the Perl benchmarks a little boost. For eg. My fasta is about 3 secs quicker and the "interesting alternative" fasta dropped below 2.0 barrier (now timed at 1.96).
However on the summary Perl slowed down a few points (here's the new bottom five on u32 single-core benchmark):
I think the drop is because the Perl pldigits benchmark is now failing. The Math::GMP module can't be found. Pretty sure this wasn't a core module so perhaps a Perl dependency has been removed in the OS (Debian).
PS. This maybe a temporary glitch so that dependency maybe restored soon. If not then I may amend pldigits benchmark accordingly.
PPS. I see that Python pldigits is using gmpy and is working fine. This means that GMP is installed (as is gmpy Python library) so it's just the Math::GMP perl module that's missing :(
> You don't seem to understand what the overall comparison shows.
I told you i don't. It looks entirely nonsensical. I asked you for clarification. So far your only response has been to parrot my saying that i don't understand why your data representations are dissonant.
> Are you familiar with descriptive statistics?
Possibly under another name, but i don't know what you mean when you say that.
In theory yes, i am unsure how you're applying it here, since we're not talking about binnable quantities.
>> I told you i don't. It looks entirely nonsensical. I asked you for clarification. So far your only response has been to parrot my saying that i don't understand why your data representations are dissonant. <<
It would have been better if you had said -- "English is not my primary languages and especially english maths are hard for me to grasp." -- instead of saying "it certainly seems deceptive".
You say you are familiar with box plots, so you should have no difficulty understanding that box plot shows - the Perl and Ruby programs have very similar performance when compared to the fastest programs.
"Visual Presentation of Data by Means of Box Plots"
Those two sentences are not a contradiction. I may not be good with reading english descriptions of math, but i am good with applied math. The calculations i did with your numbers disagree with what your graph showed. So to me the graph seems deceptive. There is no contradiction in this.
Further, if you show me the actual calculations done, i will understand it perfectly fine. Yet you refuse to do so. I do not understand why, and i hope you can understand how that makes me even more distrustful.
On the graph in the overview page Perl was shown to significantly outperform Ruby in a number of benchmarks, yes, i could see that. Yet the median of Perl was still set higher than the median of Ruby, which could possibly be explained by perl also being outperformed significantly in one benchmark, but which was not supported by the actual direct comparison numbers.
So i ask again: Please show me the actual calculations performed to arrive at the median values shown in the overview graph.
Okay, i worked it out, no thanks to yoo. Actually, i fucking worked it out IN SPITE of you. All your condescending hints and links and such were entirely bullshit and did not even remotely lead in the direction of explaining why the data seems dissonant. They were flat out orthogonal to the entire problem.
The important thing which you did not bother to point here even once is that the comparisons on the overview page are done against the fastest programs of all languages, thus weighting the results by a factor that is simply not present when one language is compared directly against another.
So, alright, the graphs do entirely make sense.
Would you be open to a patch that reworks the language vs. language comparison pages in such a manner as to make this relationship obvious?
On that page, for all benchmarks that can be compared, Perl has used an overall time of 9255 seconds, while Ruby has used an overall time of 10662 seconds. As such Ruby is actually 10% slower than Perl.
You go too far -- your lack of understanding is simply your lack of understanding ;)
What are you told the table shows?
>> Where does this difference come from? <<
Check the same thing for 2 other language implementations were the arithmetic should be easy. For example, Java median 2.04 and Smalltalk median 21.22 -- the direct comparison shows 11x as the rounded median of the Smalltalk/Java program times.
So, basically: Because Perl is considerably slower in one single comparison, even though it is faster in 7 others, it gets judged as slower overall?
Seems like your graphs up top in the language versus language comparison need to be reworked to make it clear how bit the difference in reality between x3 and 1/3 is, because right now it is deceptive.
English is not my primary languages and especially english maths are hard for me to grasp. That's why i am asking you to demonstrate, using the actual numbers for Ruby and Perl, what calculations should be performed to gain the numbers your site is showing.
Also, in addition, after reading your link, the situation seems even worse. Using the median Perl outperforms Ruby by ~15%, but the main site does not reflect that at all.
This is simply not true. Not every problem can be solved with caching. If DuckDuckGo would be a real search engine, they would not even touch with a 10 foot pole to Perl for any non trivial algorithm/function.
Well, sure if you want to use 1000 servers instead of 100, go ahead and use it. No wonder Facebook noticed this and desperately trying to compile Php to C. Perhaps if Blekko ever gets popular we will see if their choice will bite them.
Also an anecdotal example. Two months ago a friend of mine wrote a rather complex algorithm in Perl (He is very fluent with it). It was a novel sentence alignment algorithm using Gibbs sampling. Algorithm was hard to make parallel and not cache friendly. He needed to wait more than a day for training the system in a server. Well, long story short, with his help another developer converted it to Java in short time it worked around 200 times faster. So there. Moving bits around did not help Perl here at all.
>>Blekko ever gets popular we will see if their choice will bite them.
The only reason why Blekko or Facebook is even able to launch and turn around things quickly to survive in competition is because they opt to use dynamic languages like Php and Perl. If they started doing their projects in C, with the current growth in rate of complexity they will never finish their projects ever.
>>No wonder Facebook noticed this and desperately trying to compile Php to C.
Dynamic languages aren't slower because they are not C. They are slow because they do a lot of magic. C is fast because the magic is left for you to perform. I can't see how any one pull out pace out of compiling Php to C directly, unless they sacrifice things along the way. Which really defeats the purpose of using Php at the first place.
>>Well, long story short, with his help another developer converted it to Java in short time it worked around 200 times faster. So there. Moving bits around did not help Perl here at all.
Number of programmers who even need to here the word 'bits' in their day to day activities(Talking of application programmers) are rare enough to make their case totally exceptional.
Besides there are places where C makes perfect sense. There is hardly any other language heard of in the embedded programming world.
I wrote the first prototype of Boitho (Norwegian internet search engine, now defunct) in Perl in 2000. That did not work at all because memory access and sorting was so slow. We had to write the next version in C. Of course a lot has happen with Perl in the last 13 years, but search kernel in Perl still sounds odd in my ears.
Modern day software architecture is too complicated to run on one and only one programming language.
But regardless of that Perl continues to power some very serious work happening some very important places all over the world. And that is not likely to change sooner. No matter how many web frameworks get written in Php, Ruby or Python.
The reason for that is Perl has little or almost no competition in the niche it occupies. And anything that is likely to be invented to replace Perl will by and large like 99% look like Perl(Read: Perl 6 or whatever). This being the case we are likely to be using Perl(or a Perl like language) very far into the future.