
Energy Efficiency Across Programming Languages - foob
https://sites.google.com/view/energy-efficiency-languages
======
flavio81
According to their normalized "global" results, something interesting i see:

1\. Pascal, surprisingly, the most memory efficient of all. I should take a
look at the implementation they used.

2\. Rust a good alternative to C which leads in "energy efficiency" and speed.

3\. Common Lisp most energy-efficient _and_ fastest _and_ smallest memory
footprint of _all_ the dynamic programming languages in the list -- like
Python, Ruby, Lua, Perl, and even Racket (which fares pretty well)

4\. PHP, JRuby, Ruby, Typescript, Perl, Python, being _massively_ slow than
the fastest languages, for example Ruby being 59 times slower than Rust or C.

I agree that these languages (in item 4) are "acceptably fast" for many
applications, but we can't say they are "close to the speed" of the fastest
languages, even more if we consider the fastest dynamic+interactive languages
like Lisp (Python being 21 times slower than Lisp.)

5\. I wonder which implementation of Lua they used. Lua can be pretty fast,
one of the fastest dynamic languages out there.

~~~
mamcx
> 1\. Pascal

Pascal is a "hidden" gem in the area of languages. Sadly not enough pus for
it, but imagine if it have the push that other languages have...

~~~
mac01021
But why should it's memory utilization be so much lower?

Pascal and C give the programmer basically the same control over memory
layout, no?

~~~
subwayclub
Basically, C's semantics assume too little - everything is a primitive type or
a pointer dressed up to look like something else. Pascal has a stronger notion
of types and this allows for additional automatic optimization where they're
relevant - memory layout just happens to be one such area.

~~~
addHocker
I miss the ability to pair structures as "antagonistic" \- aka if one exists
the other is not, thus allowing for unions of various datatypes who guarantee
that if they exist- the shared memory they reside in is theirs and theirs
alone.

------
cwzwarich
I can't believe they used the Computer Language Benchmarks Game. Those
benchmarks don't reflect real-world workloads at all, and the contest has
fairly arbitrary rules about implementations and widely differing
implementation quality between languages. This should have been rejected by
peer review.

~~~
igouy
> Those benchmarks don't reflect real-world workloads at all…

That's a very definite claim, for which you provide no supporting evidence ;-)

~~~
oconnor663
Maybe we could get away with a couple smaller claims:

\- It would be weird to expect programs specifically optimized for a
performance benchmark, to also be optimal for energy usage or memory. Maybe
less weird for energy, if runtime is the biggest factor in how much energy
gets used. But I'd expect there to be huge tradeoffs between runtime and
memory usage, once we start really optimizing for memory.

\- Running the Benchmarks Game has meant doing a lot of work to figure out
what the relationship between "optimal" and "representative" is, and what the
rules of the game should be as a result. Even if we take it for granted that
we've answered those questions in a way that gives us useful benchmarks for
performance, it would be weird to expect the answers to be exactly the same in
benchmarks for energy or memory.

~~~
jackmott
performance usually == energy efficiency on modern cpus. race to shutdown.

~~~
igouy
Does your comment agree or disagree with what they measure?

~~~
jackmott
The implementations in the benchmark game are all targeting performance.

This will almost always also be the most energy efficient thing on modern
cpus.

It will not always be the most memory efficient way. I neither agree or
disagree with what they are measuring, its just data. I would not look at this
if I was trying to get an idea of what languages are most memory efficient,
maybe.

~~~
igouy
I wondered if you'd looked at all ;-)

------
trhway
For a bunch of years i was writing in Java and driving V8. Now i'm back to
native (C++) and am driving Prius these days. I guess that it is my eco-
consciousness that is making me shudder in disgust every time i look at
Python... pretty much the same way like when looking at Hummers. While of
course i love Perl, an M1 Abrams which gets things done despite anything :)

------
mtm
A real shame that Forth wasn't included in the tested languages. Chuck Moore
has been an advocate for more energy efficient computation for a while now.

~~~
fnord123
The tests come from the benchmark game. If you want to see Forth, implement
the tests in Forth:

[http://benchmarksgame.alioth.debian.org/](http://benchmarksgame.alioth.debian.org/)

~~~
mncharity
@igouy points out that while the old shootout site is gone, its language files
are still available[1]. There are about 70 benchmarks, in about 70 languages.
I remember them as interesting multilingual browsing.

Unlike the old shootout, adding languages directly to
benchmarksgame.alioth.debian.org is not an option. "Because I know it will
take more time than I choose. Been there; done that."[2] :) Instead, the code
is available[3], and communities are invited to document their own
benchmarksgame comparisons[2], such as Nim''s[4]. The OP used different
code[5].

A brief glance at the current benchmarks, suggests there is some overlap with
the old ones. Or at least an overlap of names - the requirements may have
changed. So it might be possible to get started fairly easily? Perhaps even to
do several languages...

[1]
[https://alioth.debian.org/scm/viewvc.php/shootout/bench/?roo...](https://alioth.debian.org/scm/viewvc.php/shootout/bench/?root=shootout)
[2]
[https://benchmarksgame.alioth.debian.org/play.html](https://benchmarksgame.alioth.debian.org/play.html)
[3] [https://github.com/Byron/benchmarksgame-cvs-
mirror](https://github.com/Byron/benchmarksgame-cvs-mirror) [4]
[https://github.com/def-/nim-benchmarksgame](https://github.com/def-/nim-
benchmarksgame) [5] [https://github.com/greensoftwarelab/Energy-
Languages](https://github.com/greensoftwarelab/Energy-Languages)

~~~
dom96
It's a pity that the OP didn't include Nim despite benchmark files for it
being available.

~~~
mncharity
If a summary of the test data is/were machine readable in the individual
language repos, perhaps one could create an automated aggregation? A
distributed/federated version of the old shootout. One with lower maintenance
requirements. bendmarksgame-results.json?

The data would be crufty, of limited comparability. But combining easily
browsable links to colorized source code, with "just for a rough feel" speed
relative to C, might be sufficient for the use case of raising language
awareness - "What is this C-speed-like language I've never heard of? Oh, that
looks pretty! I think I'll explore this language's web page..."

Or alternately, use the language files on github to create a new, broader
benchmarksgame. That is, distribute the work of benchmark revisions and
makefile compiler options, but keep the testing centralized. I've no idea of
the relative costs of those tasks, or of others. But a continuous integration
shootout sounds intriguing.

~~~
eeZah7Ux
> A distributed/federated version of the old shootout

It would be wonderful but how do you ensure any form of hardware consistency?

------
winter_blue
Note: The PDF of the paper has the a lot more detailed listing of the results
(with graphs, explanation, etc) than the results web page:
[http://greenlab.di.uminho.pt/wp-
content/uploads/2017/09/pape...](http://greenlab.di.uminho.pt/wp-
content/uploads/2017/09/paperSLE.pdf)

~~~
pjmlp
Only while printing it, I noticed my university name there! :)

------
phkahler
Rust is fairly competitive with C, but it's memory use was 50 percent higher.
I bet if they focus on getting the memory usage down it will perform better
too.

~~~
steveklabnik
Rust uses jemalloc by default; I wonder what it would look like if you swapped
the allocator.

------
int_19h
I'm kinda surprised to see Go score so low on some of these, given that it's
AOT-compiled to native code, and that its memory model and safety guarantees
are optimization-friendly.

I'm also kinda surprised that OCaml scored that high, considering how high-
level it is.

~~~
tyoverby
OCaml has an incredible optimizer, and many of the abstractions that one would
use in OCaml are transparent to the optimizer.

You could say the opposite about Go.

~~~
int_19h
Kinda makes me wonder why OCaml didn't become more popular for Unix systems
programming, the way Go seems to be becoming now. Looking at the two
languages, it seems like pretty much everything that Go has to offer in that
department, OCaml has as well - and if it also optimizes better, why is this
even a contest?

------
matt_wulfeck
Higher-level scripting languages. Fast for programmer's time, slow for run
times.

~~~
pjmlp
Not really, if you look into Lisp results.

------
danieltillett
On of the things with the C set is they didn't use Intel's icc. On our main
application (which involved a lot of heavy duty number crunching) it speeds up
our application by 100% over the best we can get out gcc.

~~~
fnord123
I've seen 20% speedups with icc over gcc; but not a doubling. Is this
including mkl over some other numeric library as well?

~~~
danieltillett
No nothing extra. The code does vectorise well. But even if the result is 20%,
it would make quite a bit of difference to the results presented in this
paper.

------
gwenzek
I'm suprised at the Java-C# comparaison. When are this numbers from?

The following blogspot explains how the author improved C# implementations
after seeing defavorable C# results.

[https://anthonylloyd.github.io/blog/2017/08/15/dotnetcore-
pe...](https://anthonylloyd.github.io/blog/2017/08/15/dotnetcore-performance)

------
stewbrew
So go (a garbage collected language) is about as memory efficient as pascal,
the most memory efficient language. That comes surprising.

~~~
brlewis
Not really. "Manual" memory allocation is usually malloc/free, which leaves a
lot of gaps.

~~~
bbatha
Go's garbage collector isn't compacting.

~~~
brlewis
My point wasn't that garbage collection is more efficient than people think,
but that "manual" memory allocation is less efficient than people think.

~~~
pjmlp
To the point that people used GC enabled systems programming languages to
write full OSes[0], but as Joe Duffy puts on his last keynote, winning
prejudice is an uphill battle.

[0]- Done at Xerox PARC, DEC/Olivetti, ETHZ, Erlangen, MSR

------
nur0n
I have just started looking into Ada and it is cool to see up there with C.
Does anyone know a good resource for writing fast Ada?

------
goatlover
No Julia?

------
kcarnold
I see they monitor a complete process lifetime, not just the active workload.
A quick grep of the paper didn't turn up anything discussing this. And it
would seem to hurt dynamic languages and runtime-JIT languages a lot. Perhaps
the active workload takes long enough that the transient is washed out. Anyone
have insights?

~~~
fnord123
If you want to tease the startup out, then you can run the game with a null or
extremely small input set and then discount the time it takes to run a
small/null workload from the actual run.

But really, if they hurt, then they get hurt. There's only so many knobs you
can turn and so many runs you can run before you have to assemble a table and
submit your paper. They put all the code online so you can try it yourself.

------
boznz
Considering most CPU cores (Arm/X86/etc) are optimized for C/GCC its an
unsurprising result

~~~
nur0n
Can someone elaborate. I always thought that the compiler was optimized for
the architecture, not the other way around. Why would an architecture be
optimized for a specific compiler?

------
flukus
I'd love to see some scatter plots of CPU-time and energu-time, I expected a
fairly linear correlation but there looks to be quite a lot of outliers, some
but not all are explained with parallelism. Memory seems all over the place
too.

------
BenoitEssiambre
Why is TypeScript so much less efficient than JavaScript?

~~~
ashark
Guess: its output has more lines of JS than the vanilla JS versions, and since
a lot of the variation between scripting languages in these things often come
down to how fast they can stop running themselves and start running some
underlying C, those extra lines hurt it.

This may also explain why JS is _really_ fast in a couple of the examples.
Those script likely do almost nothing but call some built-in function that
does all its work in C (or C++, or whatever).

[EDIT] OK, if I'm reading these[1][2] files correctly, it has more to do with
the the TS version being written in "modern" "idiomatic" "beautiful" JS with
promises and crap, and the vanilla JS one being written in "ugly" "bad" old-
school 90's-lookin' C-ish Javascript.

[1] [https://github.com/greensoftwarelab/Energy-
Languages/blob/ma...](https://github.com/greensoftwarelab/Energy-
Languages/blob/master/JavaScript/regex-redux/regexredux.node-2.node)

[2] [https://github.com/greensoftwarelab/Energy-
Languages/blob/ma...](https://github.com/greensoftwarelab/Energy-
Languages/blob/master/TypeScript/regex-
redux/regexredux.typescript-3.typescript)

------
equalunique
Haskell did not shine here. Ocaml did surprisingly well.

------
Apocryphon
One wonders how Kotlin (or other popular JVM languages) would compare. And
also Objective-C or Smalltalk.

------
ram_rar
I am curious to know, if there is any explanation for why java performs better
than Go.

~~~
topspin
Two decades of optimization probably account for that. From the paper:

    
    
        Moreover, the top 5 languages that need less energy and
        time to execute the solutions are: C (57J, 2019ms), Rust (59J,
        2103ms), C++ (77J, 3155ms), Ada (98J, 3740ms), and Java
        (114J, 3821ms); of these, only Java is not compiled.
    

That is an admirable accomplishment. JRE 1.8 has the most efficient "managed"
language runtime tested.

The PHP result is interesting; they used 7.x which is roughly twice as
efficient as older (5.x) versions; PHP 7 was primarily a (successful) effort
to improve performance. If you extrapolate the PHP result to the old version
(multiply by 2) it's nearly as bad as Perl and among the slowest/most costly.
That certainly corroborates Facebook's several PHP re-implementations.

~~~
jaawn
The Java performance was the most interesting part of the results for me. I'm
glad it got some attention. I have been developing the opinion for awhile
that, as an industry, we've become far too avoidant of "pre-mature"
optimization. It is astonishing to me how much slower a lot of this stuff is
than Java.

Granted, I also think considering Java to be "not compiled" is a slight
misrepresentation since it does get compiled to Java bytecode.

------
problemsmith
It'd be nice if these werent images of the tables ...:-\

~~~
knolan
They're just screen capped from the actual tables in the paper PDF.

------
br1n0
brief: C is the winner of all the races ( time, energy and memory)

~~~
milcron
Pascal won on memory.

------
mapletree
Nim compiles to C

