I care much more about how long it takes to create/modify a program, than I care about how fast it runs or how concise it is.
But for idiomatic code I'd find speed of execution a questionable benchmark, simply because you rarely find idiomatic code in performance-critical places. That holds for C as well as for Java. There are all sorts of rules that have to be followed to make code fast, such as no allocations in the hot path, etc. that don't usually hold for normal code.
I don't know whether this is actually a situation borne out by reality or merely a hypothetical, though.
Generally there is a complexity cost associated with performance tuning so there's value in avoiding it if possible.
Whether or not that value trumps other factors e.g. convenience, expressivity will depend on your usage but I think it's a useful data point.
From anecdotal experience, I've written some Haskell code, and compared it to optimized versions. There is a whole lot of knowledge that goes into optimizing Haskell, since you need to know how to make things eagerly evaluate, etc. Plus functional languages make it really hard to reason about memory usage (doubly so for lazy, functional languages).
For some languages, the main person who does all the work is not very skilled. Perhaps that person is just doing it as a learning exercise. Or in other cases, the person just doesn't have it on their agenda to make the program as short as possible.
All else being nearly equal, languages whose Rosetta contributors fall into the above categories will "lose" to those languages for which the "Rosetta person" is an expert programmer and advocate of the language (perhaps one of the authors of the language, even!), who is working deliberately to make the code as slick as possible to promote the language, and regards Rosetta to be a competitive playing field.
Although what are the new hipster languages now?
For execution speed and simple (or no) interface I'd choose Go, or if I could handle some platform instability I guess I would go for Rust (I still love C, but it feels like a dead weight for modern development).
>> "… the average Go program is 18.7 times slower than the average C program. Programs in other languages are much slower than Go programs … 4.6–13.7 times slower than Go on average" page 7 <<
Elephant in the room -- What kind-of programs did the Rosetta Code program authors intend to write? Did they write their programs as-if run-time performance did not matter? Did they trade-off program performance for pretty-code / shorter programs? Did they trade-off program sophistication for program clarity?
For example, look at the gap between the Shortest C++ programs on the benchmarks game and the fastest. (I pestered a C++ program contributor to write shorter programs.)
>> " …every large project is likely to include a small set of core functionalities whose quality, performance, and reliability significantly influences the whole system’s; Rosetta Code programs are indicative of such core functionalities." page 10 <<
Large projects are likely to identify that small set of core functionalities and the programmers are likely to make different trade-offs for those parts of the project, for example -- optimize performance over short code, optimize performance over programming time.
>> To have reliable performance measurements, the scripts repeat each execution 6 times; the timing of the first execution is discarded (to fairly accommodate bytecode languages that load virtual machines from disk: it is only in the first execution that the virtual machine is loaded from disk, with corresponding possibly significant one-time overhead; in the successive executions the virtual machine is read from cache, with only limited overhead). page 4 <<
The method does not take into account JVM dynamic optimization, and because we are not told the actual run-times of the programs there is no way to even guess whether the program has run long enough to gain the benefits.
>> "Which programming languages are less failure prone? To answer this question, we measure runtime failures of solutions of tasks T EXEC marked for execution that compile without errors or timeout. … Thus, a terminating program fails if it returns an exit code other than 0." page 8 <<
Why would program failure be a property of a programming language rather than an indication that the programmer did not test their program? (Or an indication that the program was written for some other implementation of the language? Or written for a workload smaller than the workload tested, given available memory?)
Even if the program did not fail, is the result correct?
>> "… we maintain that paradigms still significantly influence the typical “style” in which programs are written, and it is natural to associate major programming languages to a specific style based on their Rosetta Code programs. For example, even if Python offers classes and other object-oriented features, practically no solutions in Rosetta Code use them." page 11 <<
Not whether the Python programs should be considered imperative because they did not use OO features, but whether the Java and C# programs should not be considered OO because they did not go beyond the boiler-plate required by the language.