From experience, development in Go can definitely be classified as rapid. The type system stays out of your way, and compile times are insignificant. Try compiling on play.golang.org, or tour.golang.org to see what I mean.
About this issue: yeah, its unfortunate, but its a property of the class of garbage collector that Go uses atm. My servers are all 64-bit, so doesn't really affect me. But I do feel bad for those who are trying to run Go on ARM, or on 32-bit servers.