
Go at Google: Language Design in the Service of Software Engineering - enneff
http://talks.golang.org/2012/splash.article
======
cromwellian
I think the "return an error code" approach is just as unreadable as nested or
multiple try/catch blocks when the number of failure states grows large. I
think the "Maybe Monad" approach of functional languages and Scala, combined
with pattern matching in the language tends to look cleaner, e.g.

some_expression match { case Some(x) => do_something_with(x) case None => no
result (e.g. null in C/Java/etc) case Error(x) => handle error x }

One of the nice things about the monad approach is they can be combined
together. Consider the following chain of function calls which may return null
at any point:

int val = foo().bar().baz().blah();

To deal with this with if/else statements, you need 3 nested if-statement for
a single line of expression. In a language like Scala, you can just write:

for (a <\- foo(); b <\- a.bar(); c <\- b.baz()) c.blah();

I think there is an inherent tension between explicitly detailing everything
and boilerplate in terms of readability and writeability. I haven't seen
enough Go code or written much of anything to say anything about it, except
that it worries me. I do think the multi-return stuff can limit the amount of
accident ignoring of error conditions, but the Go language itself doesn't
provide any high level syntactic constructs to make dealing with the errors
less painful, whereas with the Monad approach (which essentially is multi-
return), the pattern matching tends to look cleaner IMHO.

~~~
irahul
> "Maybe Monad" approach of functional languages and Scala, combined with
> pattern matching in the language tends to look cleaner, e.g.

It looks the same to me.

    
    
        case maybeValue of
          Just value -> ...
          Nothing    -> ...
    

isn't any different from:

    
    
        if (val != null) {
            ....
        } else {
            ...
        }
    

<http://news.ycombinator.com/item?id=4721748>

> Consider the following chain of function calls which may return null at any
> point:
    
    
        int val = foo().bar().baz().blah();
    

Like val = foo.bar.blah.boo rescue nil

Or

    
    
        try {
            val = foo.bar().blah().boo()
        } catch(NullPointerException ex) {
        }
    

Or

    
    
        try:
            val = foo.bar().blah().boo()
        except AttributeError:
            pass
    

<http://news.ycombinator.com/item?id=4726266>

Personally, I do see Maybe/Option as cleaner, but it isn't novel as people
make it out to be.

~~~
yummyfajitas
_It looks the same to me._

The key point is that because Maybe is a monad, you can chain things together
and handle failure at the end much like a try/catch:

    
    
        let result = do
            x <- f(t)
            y <- g(x)
            z <- h(y)
            z
          in
        case result of ...
    

You are correct that this isn't particularly novel.

~~~
irahul
And as Go lacks both exceptions and Maybe/Option type, chaining or computation
expressions will look more clumsy. I am still a Go beginner and am wondering
if idiomatic Go designs programs in a different way to make up for this.

------
millstone
I continue to be confused by how Go gets positioned. From the article:

 _"Go is a programming language designed by Google to help solve Google's
problems, and Google has big problems. The hardware is big and the software is
big. There are many millions of lines of software, with servers mostly in C++
and lots of Java and Python....And of course, all this software runs on
zillions of machines, which are treated as a modest number of independent,
networked compute clusters"_

The author moved effortlessly from "Google's problems" to "servers" that are
"networked compute clusters." That's quite a leap, because it's certainly not
true that " __all __this software" runs in that manner. Android does not,
Chrome does not, Chrome OS does not, the iOS YouTube app does not, Google
Earth does not, etc. And there's plenty of not-big hardware that Google writes
for, like laptops and phones.

And the reason I bring this up is because Go was explicitly pitched as a
systems programming language - see the PDF
[http://www.stanford.edu/class/ee380/Abstracts/100428-pike-
st...](http://www.stanford.edu/class/ee380/Abstracts/100428-pike-
stanford.pdf). Programming tools, IDEs, web browsers, operating systems
("maybe") are among the tasks specifically called out as suitable for Go. But
now this article tells us that Go was designed for "large server software".

So the question is, what changed? Was Go found to be unsuitable for tasks like
web browsers? If so, in what way?

~~~
enneff
Nothing changed. The complete list from the linked PDF:

    
    
      - web servers
      - web browsers
      - web crawlers
      - search indexers
      - databases
      - compilers
      - programming tools (debuggers, analyzers, ...)
      - IDEs
      - operating systems (maybe)
    

Of that list, "web servers" and "search indexers" are both the kinds of "large
server software" mentioned by Rob in this paper. (Most Google programs are
effectively web servers of some kind, even if they don't speak HTTP directly.)

Sorry if our "positioning" of Go is a little confusing at times. We're
programmers, not marketing people. We didn't set out to "market" Go at all,
and had we set out with a clear message from the beginning we might have done
a better job at it.

This paper best describes the initial motivation for Go's existence. However,
like any general purpose language, its uses extend well beyond its design
goals.

~~~
millstone
Thanks for your reply!

Go was originally presented as a language for servers, system software, and
application programming. The last two were what got me excited, because
server-side languages are already well represented, but there's not many
languages you could consider writing a web browser or an IDE in.

However, from my perspective, Google refocused Go exclusively on server
software. For example, at Google I/O 2011, the talk was "Writing Web Apps in
Go." and in 2012, Go was organized under the "Cloud platform," and Rob Pike's
talk was about "high performance network services." Support for Go on App
Engine appears to be a higher priority than fixing its known problems on 32
bit. Rob has also discussed how he was surprised that Go attracted more
interest from Python and Ruby programmers, who mainly work on server software,
than from C++ users (and had some not-very-kind words to say about C++ users).
These are some of the reasons I'm discouraged about Go's future as a systems
and application language.

In the end I guess you give users what they want, and if Go is mainly
interesting to people writing servers, so be it. Keep on rockin' (but I
reserve the right to be disappointed).

~~~
enneff
You're reading tea leaves.

The Go team are not a homogenous group of programmers who can do anything.
Each of us has our own particular areas of expertise. That's one of the
reasons we work so well, but it also means that prioritizing tasks is
complicated. Without going into it too much, the idea that we prioritized App
Engine over "the 32-bit issue" is a nonsense.

Incidentally, that issue, to the extent that it was experienced by some users,
will be resolved entirely in Go 1.1. This is timely as a lot of interest has
developed in the ARM port recently.

Rob Pike's talk at I/O 2012 wasn't about "high performance network services,"
it was about concurrency. Network servers are a familiar context in which to
understand concurrency problems.

There's no "refocusing" going on. It's just that we're putting our best foot
forward, and right now that's in server programs and tools. Writing
applications (and I'm assuming you mean non-web applications) requires good UI
toolkit support. That's not something we're working on, but it is something
that others in the community are doing. We all believe Go will make a
wonderful language for writing native apps, it's just not there yet. (And I
suspect Rob does in particular, having built Inferno with Limbo, which used
Go-like concurrency to great effect in its UI APIs.)

It will come.

~~~
cpsales
The reality is that people working on non-google projects keep leaving because
the community is being shaped by some toxic personalities. We get hooked by
the evangelism, buy the party line, build a proof of concept, become disgusted
with the community, leave, and concurrently realize that C++ is the way
forward.

~~~
enneff
Wow, you are the first such example I've heard of. Please email me
adg@golang.org. I would be very keen to hear about your experience with the
community.

~~~
cpsales
No thanks, Andrew. I've already contributed enough and I really don't think I
owe your company anything at all.

~~~
secure
I find it rather rude to publicly say bad things about a community and then
not backing it up with at least a subjective detailed report of what happened.

~~~
cpsales
And I find it rude to make sudden demands of strangers. Maybe we can agree to
disagree.

~~~
enneff
Not a demand, just a request. Your "no thanks," is ok with me.

------
swdunlop
Slide 14 is particularly important for new Go programmers. Go's garbage
collector does not have access to certain optimizations because of the
requirement to support pointers to internal members of structures and arrays.

The loss of these optimizations are compensated by nesting structures instead
of just always using pointers to imitiate pass by reference semantics found in
other GC'd languages, such as Java or Python. Using them correctly can reduce
the number of objects to be collected and analyzed.

------
onan_barbarian
From the article:

"In 1984, a compilation of ps.c, the source to the Unix ps command, was
observed to #include <sys/stat.h> 37 times by the time all the preprocessing
had been done. Even though the contents are discarded 36 times while doing so,
most C implementations would open the file, read it, and scan it all 37 times.
Without great cleverness, in fact, that behavior is required by the
potentially complex macro semantics of the C preprocessor."

"Great cleverness" apparently means "the ability to detect that a file has a
standard 'include once' pattern and only process it once". It hardly seems
like rocket science.

~~~
JulianMorrison
No, that's called "dirty nonstandard corner cutting that will cause you to
curse up a storm at $VENDOR who wrote their compiler to be fast but it breaks
on $POPULAR_LIB which gets cute with the preprocessor, and yes they shouldn't
do that but are you going to rewrite it?"

~~~
cjensen
That's incorrect. Optimizing the guard macros works in all cases as long as
you use the standard pattern _and_ the C Preprocessor memorizes the name of
the guard macro.

~~~
jlgreco
How does this happen then?

 _"The construction of a single C++ binary at Google can open and read
hundreds of individual header files tens of thousands of times."_

~~~
cjensen
An excellent question which deserved an answer in the paper.

~~~
georgemcbay
Walter Bright (who created D to solve some of the same problems Go aims to
solve) wrote something a few years back which touches on this topic:

<http://www.drdobbs.com/cpp/c-compilation-speed/228701711>

------
cpeterso
Here is a recent Mozilla talk about the Firefox build system. The speaker,
Gregory Szorc, provides LOTS of data about the performance of parallel and
recursive makefiles and Mozilla's investigation of new build tools.

[https://air.mozilla.org/the-future-of-the-firefox-build-
syst...](https://air.mozilla.org/the-future-of-the-firefox-build-system/)

------
rwj
I enjoyed this article. Pike is an advocate for the language, but he openly
discusses the trade-offs that were addressed by the language. Too often, the
downsides to a language/technology are ignored, leading to technically boring
talks.

------
bishop_mandible
_"For the record, around 1984 Tom Cargill observed that the use of the C
preprocessor for dependency management would be a long-term liability for C++
and should be addressed."_

And in 2012 Apple tries to address it:
<http://news.ycombinator.com/item?id=4832568>

~~~
myg204
Go is 3 years old. Also, golang addresses more than just this issue, the
concurrency approach is interesting for example.

~~~
cjensen
D is 11 years old and solves the exact same problems plus it uses a more
traditional approach to objects.

The big difference is Go has Google+Pike pushing it and might actually get
widely adopted. Getting some improvement is not a bad thing.

~~~
agentS
Considering one of the goals of the language is simplicity, I don't see how D
has solved this problem.

~~~
pjmlp
Because everyone else doesn't want to throw away the last 20 years of language
design.

Go's search for simplicity is seen as a solution looking for a problem in the
language design communities.

~~~
danieldk
_Go's search for simplicity is seen as a solution looking for a problem in the
language design communities._

But most programmers are not in language design communities. There is a big
potential for a 'safer C' or a 'faster Python'. Yes, that's a simplification,
but also how people will see Go.

~~~
pjmlp
> 'faster Python'

Why throw away Python's capabilities to execute it faster? Just make use of
PyPy.

~~~
Symmetry
PyPy is geometrically about halfway between CPython and Go, performance-wise.

~~~
pjmlp
For the type of applications Google is showing off with Go, I doubt it
matters.

------
paupino_masano
I found this article really interesting, though I would like to see more
information on the design decisions for the syntax (i.e. types/return types
come after). They did say it was easier to parse from a human and computer
perspective, however personally (being used to C based languages I guess) I
find it a little more difficult to parse. I guess I'd just like to see some
references to this rather than insinuation. Anyone have any?

~~~
enneff
<http://blog.golang.org/2010/07/gos-declaration-syntax.html>

~~~
paupino_masano
Excellent - thank you for that.

I'm not sure I completely agree with their reasoning: the example only really
got complicated when dealing with function pointers, however that could be
simplified with a macro (or a delegate in other languages).

I do agree that C can get complex to read, however I'm not sure switching the
type/parameter option is the answer. But then again, I've probably become
accustomed to this format. The real test will be using the language for a few
weeks... I better start some small hobby projects in Go just to see if I end
up liking it!

~~~
enneff
Go doesn't have macros, as they're a whole other can of worms, so the
simplification one might use in C doesn't apply to Go. :-)

Go has great support for closures, so you tend to use function types more
often (or perhaps just more naturally) than in C.

It is just a matter of getting used to it. I switch back and forth between Go
and other languages, and it doesn't bother me. (The thing that does get me is
forgetting to use semicolons in other languages.)

~~~
paupino_masano
Thanks again, I really appreciate your input! I'm going to have to switch
between the two to get a feel for it (and like you say, get used to it). I'm a
big fan for language grammars (I enjoy writing them myself) so am looking
forward to digging into it some more to see how it all works :)

------
cjensen
What gets my goat about this paper is the amount of misleading information in
it.

Re-parsing files multiple times is a solved problem using the standard header
guard macro pattern combined with a slightly intelligent C Preprocessor. Yet
quotes about the horror of header blow-up pervade this paper as if this is an
exciting new problem that has been solved. It isn't; it was solved more than
20 years ago.

~~~
akkartik
Yes, the guard macro pattern works. But it's something programmers manage, so
it's a band-aid rather than a solution.

If you change a header you need to recompile all files that include it, even
if the include was conditional. Now your build system starts to look inside
files rather than just check timestamps. Similar tool bloat happens over and
over again in large codebases.

I'm not convinced Go has the right solution. But it's disingenuous to claim
that it's competing with the C preprocessor. It's _leagues_ better.

You should enumerate all the information in the paper you consider to be
misleading.

~~~
jeremyjh
Can you name a language developed since the 80s that hasn't also solved this
problem?

~~~
enneff
Python? Ruby? JavaScript?

There are many more, but I name these three as they are among most popular.

~~~
jfb
How has Javascript even _addressed_ this problem?

------
aschroder
I've been reading about Go a lot lately, starting with Effective Go, and now
the tutorial book Learning Go. (and periodically referring to the FAQ when I'm
stumped by design decisions)

It's been enjoyable learning it, though I'm yet to write any code in anger
with Go. I find some of the concepts around concurrency and OO design have
really made me think about the way I write in other languages.

One thing I cannot quite make sense of though is the treatment of
errors/exceptions. This article says: "Exceptions make it too easy to ignore
them rather than handle them, passing the buck up the call stack.."

But the example given seems to me to do exactly that:

if err != nil { return err }

Worse still if you forget to do something with err, or don't handle a specific
err !=nil situation, it seems easy for control to flow past the error handling
and into code that expects no errors.

Are there small, easily understood open source projects written in Go, that
would provide tangible examples of the benefit of this error handling
approach?

The learning materials thus far have not really convinced me, maybe code in
the wild would.

~~~
jlgreco
The idea, as I understand it, is to use panic/recover in truly _exceptional_
cases to unwind the stack and handle error conditions, but to never do so
across package/api boundaries.

So essentially, use panic/recover as you would a much nicer version of
longjmp.

(I believe) the only thing panic/recover lack compared to a more traditional
try/catch is typed catching, but idiomatic usage is much different.

Honestly, I really like it. The typed-catching would be nice but the idiomatic
usage of exceptions in languages like Java has always rubbed me the wrong way.
Exceptions crossing package/api boundaries just seems wrong to me.

~~~
EdiX
> (I believe) the only thing panic/recover lack compared to a more traditional
> try/catch is typed catching, but idiomatic usage is much different.

You can use reflection with recover to do it, the upside is that it's a more
general mechanism the downside is that you need to manually rethrow exceptions
you don't handle.

BUT the bottom line is that the standard library doesn't throw exceptions.
What you do in your code is really up to you, if you really think panicking
across api boundaries is the right thing to do you can, there is no go police
coming after you.

------
bfung
I find the guard macros discussion in this post most amusing. Just use Go and
there's no point in talking about guard macros! =P

------
haberman
The dependency story becomes significantly less compelling if something like
this proposed module system for C/C++ takes off:
<http://news.ycombinator.com/item?id=4832568>

------
ChuckMcM
I wonder sometimes if Go will become to Google what TAL was to Tandem. A
language uniquely suited to solve problems unique to the design space in which
they operate. In Tandem's case it was non-stop computing, in Google's case its
kilocluster distributed algorithms.

I really appreciate watching it evolve.

~~~
burntsushi
I wrote a fully featured X window manager in pure Go [1] that I use full time.
And it was pure joy compared to doing the same in C.

[1] - <https://github.com/BurntSushi/wingo>

------
pretoriusB
> _Go is a programming language designed by Google to help solve Google's
> problems, and Google has big problems._

I believe that this part of the speech is wrong, or overstated (and Rob did it
on purpose). There ARE actual texts about how Go came to be, and none of it
involves Google, as a corporate entity, wanting something particular and
asking for some specific set of requirements, the way, say Ada was designed by
the defence departments, of Erlang might have been designed by Ericsson. The
story, as told also by Rob Pike elsewhere, speaks of some guys getting
together t hack on a new language on some pain points THEY had, and mostly as
a side project.

And, really, Go was never put front and center by Google, or that Google seems
to care a lot about it and advertise. They've done far more for V8 and even
Dart, including building a large team of full time paid compiler guys to work
on those, whereas the did nothing of the sort for Go.

Not much adoption at Google either: some mysql load balancing service in
YouTube, a caching layer for Google downloads, a Google doodle served by Go,
and beta support in GAE pretty much sums it. Sure there should be some other
small projects, but nothing earth shattering, the way C++/Java are used.

~~~
joelgwebber
I think you're fundamentally misunderstanding the way these kinds of things
happen at Google. Most projects of this nature have to work to get internal
adoption just like any external toolchain. There's almost never an "edict from
on high" that a project will use a particular technology (except to the extent
that most code in production has to be written in a set of approved languages;
a language created by Google itself still has to surmount this hurdle).

Pike, along with other Go team members, have been heavily involved in solving
_very_ large-scale problems (e.g.,
[http://static.googleusercontent.com/external_content/untrust...](http://static.googleusercontent.com/external_content/untrusted_dlcp/research.google.com/en/us/archive/sawzall-
sciprog.pdf)) for a long time now, so they are solving the pain they've
personally experienced. But other teams at Google will treat Go just like
developers in the world at large do -- with skepticism, until it's proven
itself. It's doing so now, but it will take some time before you see heavy
adoption, which is the way it should be with any new system of this magnitude.

~~~
pretoriusB
> _I think you're fundamentally misunderstanding the way these kinds of things
> happen at Google. Most projects of this nature have to work to get internal
> adoption just like any external toolchain._

Well, Dart and V8 are too examples to the contrary.

~~~
joelgwebber
Not really.

V8 was simply a drop-in replacement for existing Javascript VM's, so there was
no "adoption" curve, per se. It ran something like 40x faster than the one in
WebKit when it was released, so everyone just said "Awesome, thumbs up! Stick
it in there."

Google is spending some developer relations resources promoting Dart, probably
because it's the kind of project that requires more public buy-in (including a
VM that ultimately needs WebKit hooks, which is not solely controlled by
Google). But it still has to surmount the same hurdles to internal adoption
that other tools and languages do. I'm no longer privy to the details, but I'm
fairly certain that there are just a few internal teams trying out Dart right
now, to help it get its "sea legs".

And that's really the way it should be. These things don't happen overnight,
and edicts from on high that a particular team will use a new technology don't
usually work out well.

~~~
pretoriusB
> _V8 was simply a drop-in replacement for existing Javascript VM's, so there
> was no "adoption" curve_

I never said anything about adoption curve. I'm talking about money spend by
Google on the project and promotion done by them on it. V8 had a large team
assembled, with a star compiler guy, was heavily promoted with marketing
material, website and even several videos.

Of course V8 was part of their plan to take over the web browser, whereas Go
is just a language for them. Well, my sentiments exactly.

------
jebblue
What is with all these Go articles? I wish Go would Go the way if the DoDo
bird. Java rocks client and server, web and think client.

