Taking the opportunity to show off some of the profiling tools and walk through the process of optimizing the program is a classy way to respond, first of all, and secondly I didn't know those tools existed. Very cool.
This post basically shreds the original paper. The classiness is the icing on the cake. If his version is 15x faster than the original Go version, that would make it about twice as fast as the super-optimized C++ version from the paper -- the one that was "was heavily dependent on several Google internal data structures and could not be open sourced". Lots of brainiacs at Google probably now have egg on their face courtesy of Russ Cox.
> If his version is 15x faster than the original Go version, that would make it about twice as fast as the super-optimized C++ version from the paper
The article you just (sort of) read explicitly states that it's not much faster than the C++ version because the C++ version lacked many of the optimizations they did in the article. To make a fair comparison, they applied those same optimizations to the C++ one code and got similar performance:
Of course, it's no longer fair to compare this Go program to the original C++ program, which used inefficient data structures like sets where vectors would be more appropriate. As a sanity check, we translated the final Go program into equivalent C++ code. Its execution time is similar to the Go program's.
It turns out that Google is a collection of people with similar aims but differing views, not a cabal of braniacs. The original paper was written by a Googler who likely published it on his own for fun (and clearly without review from a Go developer); Russ Cox (also a Googler) "shreded" the paper here without himself passing it by the Scala enthusiasts for their take on it, etc.
Russ masterfully also demonstrated the Google cultural value of politeness, so I expect the results of this blog post will be taken in the gracious manner in which they were written.
You're right. Here's what Ian Lance Taylor (one of the Go developers) said:
"Robert asked me to take a look at his code and I hacked on it for an hour to make a little bit nicer. If I had realized that he was going to publish it externally I would have put a lot more time into making it nicer."
> Russ Cox (also a Googler) "shreded" the paper here without himself passing it by the Scala enthusiasts for their take on it, etc.
But russ didn't claim anything regarding Scala, the problem with the original paper is that it made explicit and implicit claims regarding Go, Scala, Java, etc. without really bothering to make much of an effort to make the comparison minimally realistic.
The original paper was a joke, as anyone who read it knows, for example the Scala version submitted didn't even implement the same algorithm as the other submissions.
That is not accurate. The performance numbers they reported were the idiomatic implementations. The optimized versions were discussed in a different section.
Except that the "idiomatic" implementations were not idiomatic, and the "optimized" versions were not optimized either. The whole paper was a huge joke.
I'm very glad to see this more-serious optimization of the Go and C++ implementations of the original benchmark, which was less than thorough, and not equally fair to each language. It's interesting that the optimized Go and C++ code ultimately wound up with similar performance characteristics.
