

Benchmarking the Go frontend for GCC 4.8.0 - CoffeeDregs
http://www.alsonkemp.com/programming/synthetic-performance-of-the-go-frontend-for-gcc/

======
CJefferson
This seems too simplistic. I am almost tempted to download the two frontends
just to see why it is so much bigger.

Lots of inlining (which -O3 would cause, it always does. I often find -O2
produces faster executables)? Debugging information? Pulling in the standard
library?

A compiler doesn't produce an extra 3MB of code for no good reason.

~~~
blinkingled
Yeah, static linking could explain the code size difference. On my machine
where I have gccgo 4.8 built and installed the Gccgo produced executable is
way smaller than go 1.1 rc1 built. Go builds static executables whereas
installed gccgo with just -O3 builds a dynamic one.

~~~
hrez
Run 'strip' on it.

~~~
blinkingled
'strip' will remove symbols - debug or unused, but that doesn't change the way
static libraries work - all the code, instead of being in the shared library
dependency is stuffed into the executable. That's still going to be there.

------
onan_barbarian
Why would anyone care about relative code size or compile time for a program
this small? This is obviously a "cost of entry" issue.

I have trouble imagining that doubling the code size and complexity of the
"mandel" example would double the compile time or code size with either
example. Why not try a handful of programs to get a scatterplot between the
two toolchains?

~~~
CoffeeDregs

        Why would anyone care about relative code size or compile
        time for a program this small?
    

Because it can be extrapolated [obviously, not perfectly]... The nice thing
about a mandelbrot benchmark is that it's FP and optimization driven, so does
a decent quick-and-dirty job of illuminating the parallelizable performances
produced by different compilers. Once you add in regexes, databases, etc, you
move beyond the compiler and on to the performance of secondary or tertiary
systems.

    
    
        Why not try a handful of programs to get a scatterplot
        between the two toolchains?
    

Because relative to the question I was trying to answer ("as someone
investigating Go, should I also investigate GCCGo?"), that would be a waste of
time. Were I to have gotten a result that suggested that GCCGo was markedly
better along _some_ dimension, I would have done more testing to quantify
"markedly".

Go is a lovely, young language, but the GCCGo implementation is very young.
The GCCGo folks will catch up soon enough with Google. Until then, Google has
produced a nice toolchain for you.

~~~
onan_barbarian
I'm not arguing that it's a bad microbenchmark for FP run-time performance.

I'm arguing that it's a bad microbenchmark for compile time and code size. I'm
not all that familiar with go, but in most systems I've used, if you write a
really tiny program, you will find that there's a "fixed cost of entry' where
the standard libraries get munged through the linker (or whatever) and that
writing a program 10x larger will not cost you 10x as much. Since you took
only one measurement of one program, you have no idea what the trend lines
might look like here.

It seems like you are going out of your way to do this badly and make a snap
judgement that you can proclaim to the world.

I am surprised at the extent to which you seem to think you need to ration
your investigation and avoid wasting your time by the exhausting task of "also
investigating GCCGo".

Also, in the world of high-performance computing, 10% for zero programmer
effort is _huge_.

------
mseepgood
The article doesn't mention which version of gc and which version of gccgo was
used. Why do people post benchmarks without such basic information?

~~~
CoffeeDregs
Apologies. Though the HN post title says 4.8.0, I didn't mention anything
about it or the Go version in the blog post. I'll add those. GCC was 4.8.0. Go
was 1.1rc1. Both AMD64.

------
Xorlev
I'm also curious about a larger swath of tests -- a single algorithm can
hardly characterize a whole compiler.

~~~
voidlogic
Not only that, but I have heard people say that for compute (code generation)
bound tests that are single threaded GCCGO does better; however, if you run
code with many goroutines (which most Go programs have) the standard Go
compiler is faster.

As of 1.1 Go code generation is also drastic improved. I would love to see Go
1.0.3 vs Go 1.1 vs GCCGO tests with a wide sample of programs.

~~~
gillianseed
Did some of the go benchmarks on the lunch-break, I used standard 'go build'
for the go results, and "go build -compiler gccgo -gccgoflags '-O3
-march=native'" for the gccgo results.

The compilers where : Go 1.1rc, GccGo version 4.8.0 20130502 (prerelease), on
an Arch Linux 64-bit system, Core i5

    
    
      binary-tree.go
      gccgo-4.8:  0m0.932s
      go-1.1rc1:  0m1.835s
    
      binary-tree-freelist.go
      gccgo-4.8:  0m0.165s
      go-1.1rc1:  0m0.307s
    
      chameneosredux.go
      gccgo-4.8:  0m10.163s
      go-1.1rc1:  0m5.620s
    
      fannkuch.go
      gccgo-4.8:  0m44.533s
      go-1.1rc1:  0m56.389s
    
      fannkuch-parallel.go
      gccgo-4.8:  0m26.525s
      go-1.1rc1:  0m27.855s
    
      fasta.go
      gccgo-4.8:  0m1.126s
      go-1.1rc1:  0m1.250s
    
      mandelbrot.go
      gccgo-4.8:  0m30.004s
      go-1.1rc1:  0m30.537s
    
      nbody.go
      gccgo-4.8:  0m11.423s
      go-1.1rc1:  0m12.620s
    
      pidigits.go
      gccgo-4.8:  0m7.333s
      go-1.1rc1:  0m2.321s
    
      spectral-norm.go
      gccgo-4.8:  0m12.259s
      go-1.1rc1:  0m13.389s
    
      spectral-norm-parallel.go
      gccgo-4.8:  0m3.142s
      go-1.1rc1:  0m6.695s
    
      threadring.go
      gccgo-4.8:  0m37.421s
      go-1.1rc1:  0m9.800s
    

Overall I'd say Go did better as in the instances GccGo won (although more
numerous) it was typically with a small margin while on those GccGo lost we
had larger margins.

Overall Go 1.1rc seems to have improved quite a bit from my previous test
(1.0x) unless my memory betrays me.

Note that these are far from all the 'computer language benchmarks game'
tests, only those which I managed to get done during lunch break, and as such
they may skew the results compared to a full benchmark comparison.

~~~
igouy
Note, the programs in go/test/bench are not necessarily the same as the Go
programs shown on the benchmarks game.

Most to the point, the mandelbrot program shown on the benchmarks game is not
the same -- someone contributed an improved program to the benchmarks game.

~~~
gillianseed
Good to know, as such it would be a more apt comparison to use the currently
best Go versions from the benchmarks game as they better reflect where Go
stands performance-wise (with the usual micro benchmark disclaimer).

~~~
igouy
And to be fair, that's what CoffeeDregs did originally.

------
songgao
IMHO, both compile fast enough. Either gccgo or gc, it's acceptable, and is
much faster than compiling a similar-size C program. What matters would be the
code they produce. Compiler optimization is what makes the difference.

------
AYBABTME
In the snippets for `time`, by the end of the article, the `>` symbols have
been escaped to their HTML equivalent:

    
    
       time ./mandel.golang 16000 &gt; /dev/null 
       time ./mandel.gccgo 16000 &gt; /dev/null 
    

They also happen to be within two `<pre>` tags. I guess there's something
wrong in the rendering of your snippets.

I say 'your' assuming that OP is the author.

~~~
CoffeeDregs
You're correct about "your". Fixed. Thanks.

------
ericflo
How long did each take to compile?

~~~
CoffeeDregs
I'll add that, but: GCCGo Direct [not called by go]: 0.3s Go: 0.5s

