

A comparison of programming languages in economics - scarmig
http://marginalrevolution.com/marginalrevolution/2014/07/a-comparison-of-programming-languages-in-economics.html

======
minimax
Their code is on github [1]. I am a C++ programmer so I'm happy that C++ came
out on top, but the comparison is obviously flawed (as these sorts of
comparisons always are). The problem is that the implementers don't have the
same level of skill in each language and who knows how much time they spent
analyzing and refining each version. See Russ Cox's response [2] to a Google
paper [3] of a similar nature comparing C++, Go, Java, Scala for an example of
a similar boondoggle.

The R code, for instance, is obviously unidiomatic. If you have a four levels
deep nested loop in R, you're doing it wrong, and it's taking every ounce of
willpower to keep from spending the rest of the afternoon trying to fix up
their code and make it faster.

1\. [https://github.com/jesusfv/Comparison-Programming-
Languages-...](https://github.com/jesusfv/Comparison-Programming-Languages-
Economics)

2\. [http://blog.golang.org/profiling-go-
programs](http://blog.golang.org/profiling-go-programs)

3\.
[http://research.google.com/pubs/pub37122.html](http://research.google.com/pubs/pub37122.html)

~~~
bradfordarner
It is unfortunate that the authors of this paper didn't reach out to experts
in each programming language in order to write high-quality code against which
they could have run the analysis. This seems like the perfect use-case for
open-source: academic study, subject-matter experts in one area who are
stepping outside their areas of expertise and an interesting subject matter.

They should have crowd-sourced the code for this study with a Top Coder-like
contest.

~~~
k2enemy
From the paper...

 _Second, to make the comparison as unbiased as possible, we coded the same
algorithm in each language without adapting it to the peculiarities of each
language (which could reflect more about our knowledge of each language than
of its objective virtues)._

They have invited others to fork the code on github and write the fastest
possible for each language.

~~~
igouy
This is not "as unbiased as possible" \-- it is biased towards the initial
form of algorithm description.

To make the comparison "as unbiased as possible" would be to code the same
algorithm but adapt it to the peculiarities of each language implementation.

------
zissou
I used to be gung-ho about convincing economists to use Python (started
econpy.org in the 1st year of my PhD in economics, stopped updating it after 2
years). But now I just don't care. The vast majority of economists are
horrible programmers, however most know how to script in at least 1 language,
and a small number of them are actually really good at scripting. An extremely
small number of economists know they're way around at least 1 entire general
purpose programming language.

It seems that every time this gets brought up online a sea of infamous
Fortran-programming-economists always tell you how fast Fortran is compared to
everything else, signing it with "Fortran or GTFO". I've met
hundreds/thousands of economists and econ grad students at schools ranked from
1 to 200, and I don't recall a single one that actually used Fortran (I'm sure
there is a non-zero quantity -- I probably just haven't cared to talk with
them b/c they are most likely macro theorists).

The fact of the matter is that in economics graduate school, your professors
could care less what language you use. It isn't an algorithm competition, it's
a story telling competition. But the type of story telling economists do is no
less noble than writing elegant algos. Writing a story based on economic
mechanisms and behaviors and supporting it with quality data, sound
statistics, and logical/exact theories is no simple task.

~~~
bachmeier
"The vast majority of economists are horrible programmers, however most know
how to script in at least 1 language, and a small number of them are actually
really good at scripting. An extremely small number of economists know they're
way around at least 1 entire general purpose programming language."

As an economist whose hobby is the study of programming languages, you can
imagine the frustration that I feel. What I have found works best is to make
grad students use basic functional programming techniques. Everything they do
is just a few lines long. As opposed to a 250-line heap of garbage with three
nested for loops.

~~~
JAlexoid
I'm sorry, but that it's a wasted emotion. Economists are not software
developers. They do the minimal they need and nothing more. It's like
complaining that a masseur has poor baking skills.

Get over it! It's a support tool for them and until they demand that you take
their code as a perfect example, developed have no need to complain.

~~~
avemuri
It's more like saying a masseur doesn't have a deep knowledge of anatomy. For
most healthy people it wouldn't be an issue if the masseur just did what they
were taught, but they'll consistently be suboptimal and will occasionally do
some real damage. See the recent high profile errors with economists using
excel.

------
nemoniac
Two economists judge the relative suitability of a bunch of programming
languages.

I have to wonder how it would be received if two computer scientists wrote an
article judging the relative merits of a bunch of economic models.

------
cschmidt
Direct download link of the original paper:

[http://economics.sas.upenn.edu/~jesusfv/comparison_languages...](http://economics.sas.upenn.edu/~jesusfv/comparison_languages.pdf)

------
oniTony
This seems more a survey of compilers than programming languages. Python is
represented by 3 different compilers, spanning 2 orders of magnitude in
performance: CPython (155x), PyPy (44x), Numba (1.57x). It's hinted that C++
similarly depends on compiler choice:

> although one needs to be careful with the choice of compiler [for C++ and
> Fortran]

------
MrBuddyCasino
In case anyone missed the Java results, I do think it is worth mentioning that
it is very competitive:

"Our fourth result is that Java imposes a speed penalty of 110 to 169 percent.
Given the similarity between Java and C++ syntax, there does not seem to be an
obvious advantage for choosing Java unless portability across platforms or the
wide availability of Java programmers is an important factor."

~~~
dodders
In my professional life I have rarely (if ever?) seen a language chosen on the
basis of it's syntax.

~~~
adrusi
It's true, most good programmers won't choose a language based on just syntax.
And even though most economists aren't good programmers (in my experience),
they're still smart people who can see past the syntax.

The paper is probably misusing the term syntax though. C++ and java really are
quite similar in many ways, far beyond just syntax, java just letting the
programmer forget about some annoying things like memory management and
portability. If most of what they're doing in the language is numeric
analysis, then there probably isn't much of a difference at all, and they will
probably get slightly better performance from C++.

------
civilian
What about Excel?

I know, I'm also horrified of how economists use excel. But it does tend to be
their go-to tool, and they are all comfortable in it. It'd be nice to show
them just how poor the performance is in excel, to tempt them into learning
scripting languages if they have an excel workbook that will barely refresh.

------
georgewfraser
There are two glaring problems with this paper:

1\. The algorithm is just a bunch of arrays and loops in one giant block. This
is the easiest possible case for compilers that optimize through
specialization. Even a bad specializing compiler will be good at this but will
blow up when faced with a big piece of code with lots of functions and data
structures.

2\. The runtimes are only ~2s, which is not long enough for the JIT-compiled
languages to "warm up". I suspect Java in particular would have had the exact
same runtime after warming up, because in an easy example like this Hotspot
will generate the exact same instructions as the C program.

EDIT: I stand corrected. I downloaded his examples off Github and ran them the
same way as the paper. I get 1.87s for C, and for Java... also exactly 1.87s.

~~~
Alupis
> EDIT: I stand corrected. I downloaded his examples off Github and ran them
> the same way as the paper. I get 1.87s for C, and for Java... also exactly
> 1.87s.

Your code probably got JIT'ed... which essentially makes your java code into
native C... So... it's no surprise they perform around or exactly the same
(given the JVM is allowed time to warm up, discover "hot spots", and then
perform the JITing)

~~~
georgewfraser
True though oddly I compiled and ran them exactly the same way he described in
the paper -- one time through, no warm-up time.

~~~
Terr_
I think the command-line option -XX:+PrintCompilation should show you if/when
the JIT kicks in for a method.

------
larrydag
It seems like this is a comparison of speed. Not a comparison of languages.
Each language has their own pro's and con's. For instance, I use R on a daily
basis. I don't require speed to do my job because I'm analyzing data and
drawing inferences to help make business decisions. I like the fact that there
is a huge library and an active community that can help me implement
solutions. Perhaps if a solution/product I come up with needs speed then I'll
implement a tool that will do that job well. But my "toolbox" requires fast
prototyping and development.

------
wcember
I think the paper should be called "A comparison of programming language run
times in economics." It's not obvious to me that run time is an important
quality of a programming language for economic research.

------
adrusi
It's important to note that the available libraries for each language varies a
lot, and the best performing ones are proprietary for the most part. Fortran,
for example, has a set of proprietary libraries that many employers buy that
have been continually optimized for nearly 50 years. Both C and Python can
readily bind to Fortran libraries, but for other languages like R, Matlab,
Mathematica, etc. this is a major disadvantage.

~~~
bachmeier
Not only is it easy to call Fortran functions from R, the inline package
actually allows you to embed Fortran functions inside your R program.

------
psychometry
Economic is a large, diverse field and it would be reasonable to assume, for
instance, that R is used in very different ways than C++, so this study is
already of limited use. Furthermore, they claim to be comparing the "strength
and weakness of each language" yet only analyze the performance of naively-
written, unoptimized code.

~~~
zissou
You bring up a good point in that the use cases for each language varies. R is
a good example because one of the main reasons it is chosen is due to the fact
that it already has the largest set of drop in econometrics operations for
tons of extremely specific situations. In any other language, you'd often have
to code a bunch of complex linear algebra by hand, which would totally negate
any marginal benefits due to computation time.

In terms of econometrics, Python is going to surpass closed source software
packages like SPSS, SAS, eViews, etc soon simply because the rate at which
econometric procedures are being implemented in Python is growing steadily
(e.g. via a statsmodels/pandas/numpy/scipy based stack). I don't think Python
will ever pass R in this respect though as R has a large share of the
statisticians helping add the implementations relative to Python.

------
mikecb
Another interesting comparison from the github repo[1] is the proportions of
lines that it took in each language.

[1] [https://github.com/jesusfv/Comparison-Programming-
Languages-...](https://github.com/jesusfv/Comparison-Programming-Languages-
Economics)

~~~
igouy
Programs for some languages are tall and narrow, while comparable programs for
other languages are short and wide.

------
plumeria
It would be cool that people from different areas of expertise (whether C++,
Rust, Go, etc...) committed their most idiomatic implementation for the
benchmark.

------
JAlexoid
Wasted time on a wasted subject. The person that compiled it had nothing
better to do... Economists are not developers and should not imagine to be
ones. It takes a lot of time and effort to be one.

