

      An empirical comparison of C, C++, Java, Perl, Python, Rexx, and Tcl - CaptainMorgan
http://page.mi.fu-berlin.de/prechelt/Biblio/jccpprt_computer2000.pdf

======
koblas
While this article is 10 years old, it contains a few very good points -- I've
referenced it many times over the last 8 years. Specifically:

\-- Scripting languages relative run time and memory consumption overhead will
often be acceptable and they may offer significant advantages with respect to
programmer productivity.

\-- For all program aspects investigated, the performance variability due to
different programmers is on average about as large or even larger than the
variability due to different languages.

------
houseabsolute
I've always kind of felt that any claim to empiricity in programming language
comparison will be fatally undermined by a focus on only one task. The choice
of task itself is very opinionated in a way that makes it difficult to apply
the conclusions of this study more broadly.

~~~
m0th87
FTA: "It must be emphasized that the results are valid for the phone code
problem only; generalizing to different application domains would be
haphazard"

------
lg
should maybe note that this was written 10 years ago.

~~~
cgranade
That's bound to throw off the Java and Python numbers quite a bit. JDK 1.2.1
was problematic, and I think a lot of its poor performance there could
probably be fixed with no more than a simple recompile. The past 10 years have
also seen a wide diversification of Python implementations, including
IronPython, Jython and the JIT-based interpreters. It would be very
interesting to compare the performance of the same programs across platform
implementations to try to knock down the variations and get to the effects of
the choice of language itself.

~~~
kllvql
Both of those languages have seen vast improvements in speed throughout the
past ten years. Java 1.4.2 was immensely faster than any previous Java
implementation. I would be very interested in seeing this study redone with
the newest Java, Python, Ruby, and something Lispy.

------
nomen
Indeed; for some aspects of the study ten years ago was a different epoch.

------
antileet
I've seen so many benchmark comparisons that only focus on conciseness (that
too involving lines of code), and speed. Yeah, they all re-affirm what is
already known - C is fast, python/ruby is concise, C#/Java is somewhere in
between and there are the ones which do well in both - haskell, clojure, etc.
(And then there's APL)

All the benchmarks are algorithm and processor intensive, while quite a few
applications are idling most of the time. How about a comparison regarding the
standard libraries, ease of debugging, external tools, code readability,
available support, external libraries, etc. In short - developer productivity.

I'm not saying performance is not important - it is quite critical, but there
are other things code does as well (At least most of my code is mundane, and
is just calling APIs, updating files, serializing objects, etc. Nothing too
fancy.) </rant>

~~~
dalke
And your rant is relevant to this paper precisely how? As the paper also
covers time to implement and memory use it, it doesn't fall into the category
you complain about.

~~~
antileet
I guess it's rather off-topic from the paper's content. I apologize for that.

------
patrickgzill
In many respects, TCL is dead at this point ... there is still a lot of legacy
use for it but few new users in comparison to Python, Ruby, etc. Rexx is dead
outside of IBM shops, and was ever since Amiga and OS/2 died.

~~~
davidw
It's written "Tcl", not "TCL". This has been true for at least 10 years, as
the article nicely demonstrates.

Nitpicking aside, I agree that it's not real lively, but I don't think I'd
call it 'dead' quite yet either, as there is still a fair amount of active
development, and anything that widely used doesn't just up and go away from
one day to the next. They even published a new Tcl book recently:-)

[http://journal.dedasys.com/2009/09/15/tcl-and-the-tk-
toolkit...](http://journal.dedasys.com/2009/09/15/tcl-and-the-tk-toolkit-2nd-
edition)

------
thisrod
One section of the paper describes which algorithms the programers used. It
appears that the solutions in scripting languages made heavy use of
associative arrays, so the results might change if you swapped Tcl's hashing
algorithm for Perl's.

The algorithms described don't seem very clever. I expect you could do much
better by splitting the input number into digraphs or trigraphs, sorting those
by frequency in the dictionary, and matching the rarest ones first.

------
balu
It's always nice to see papers published by my profs on HN. I would not have
looked at those on my own.

------
cmallen
Am I the only one tired of mathematical tests? I'd rather see a test comparing
the relative speed and scalability of various web app implementations.

And don't give me that line about scaling not mattering, I have to cope with a
cantankerous server and database at work, I know full well what my problem
sets are.

