
Go best practices, six years in - krat0sprakhar
https://peter.bourgon.org/go-best-practices-2016/
======
nvarsj
The stages of go enlightenment:

1\. Holy crap, this is like coding in early Java days, what the heck were the
language designers smoking?! They just ignored everything! Where's my testing
framework! DI?! Build system, dependency management?! WTF.

2\. Holy crap, this is like coding in the early Java days! This is awesome! I
can understand all golang code I read! Everything is so simple and easy. I
finally get "less is more", and "worse is better"!

3\. (Many months later) Oh god, I'm getting so sick of writing this test
assertion in 4 lines over and over, and writing 100s of lines of mock structs.
My tests are 5% substance and 95% setup. Okay let's write a junit/mockito
assertion library. Oh crap, I'm having to manually wire everything everywhere
and it's such a pain. Let's write a DI library. Ugh my project is getting too
big and `go get` just doesn't cut it - time to use a dependency management
library. Ugh, makefiles only go so far, time to write a build tool. Ugh code
generation and metacoding is nonexistent, let's write our own code generation
tool so we don't have to hand build 100s of types. ETC.

4\. (Distant future) As golang's stubborn designers finally accept some flaws
in their original thinking, they add features to make the language look a lot
like another widely used language at the Googplex. Mission accomplished?

P.S. This is somewhat tongue in cheek - I do enjoy using golang day to day. It
is a refreshing change :).

~~~
gtrubetskoy
Java was:

1\. Wow, this program runs on any platform!

2\. Wow, no need to free memory!

3\. Wow, threads and locks are so easy to do!

Then many years later

4\. Who really cares about cross-platform code if all I do is on Linux?

5\. Why does this program require 4GB of ram?

6\. Must I really download and install the JVM all the time, and what is it
good for anyway?

Go:

1\. Wow, I can write stuff that previously could only be done in C/C++, and I
can do it almost as easily as writing Python or Ruby.

2\. Wow, I don't need _anything_ installed for my program to run, it just
runs!

~~~
incepted
> 4\. Who really cares about cross-platform code if all I do is on Linux?

Most developers write their code on Windows and MacOS and then deploy on
Linux. I'd say this "Write once run anywhere" thing was and still is a pretty
big deal.

> 5\. Why does this program require 4GB of ram?

Oh no! I need to buy a $5 piece of hardware to run this Java code? What kind
of nonsense is that?

> 6\. Must I really download and install the JVM all the time, and what is it
> good for anyway?

Once or twice a year, if you really want to. You update your browser much more
often.

~~~
throwaway_exer
> Oh no! I need to buy a $5 piece of hardware to run this Java code? What kind
> of nonsense is that?

4 GB per program absolutely destroys VM consolidation. I love the look on
managers' faces when they ask for a 1 GB VM and I ask them, "Is this for a
Java program?" :)

~~~
nixusg
Just adding some ram is also not so easy in an embedded system.

~~~
Hydraulix989
But the Java JRE update installer wizard for Windows touts the fact that Java
is "in" thousands of embedded systems -- parking meters, toasters, etc.

~~~
dig1
Imagine, Java is run on your phone's SIM card. Enough?

On other hand, on truly constrained embedded systems where real time is
essential, no GC-based language has any business, no matter how GC is good.
Even C++ is sometimes avoided.

~~~
pjmlp
You mean like missile radar control systems?

[http://www.pr.com/press-release/136232](http://www.pr.com/press-
release/136232)

Or do you prefer Aegis battleship Weapons System ?

[http://dl.acm.org/citation.cfm?id=2402699](http://dl.acm.org/citation.cfm?id=2402699)

These are my favorite examples for Java being used in life critical
situations.

~~~
geodel
Well 2 thing stand out for me from links you mentioned:

> The applications use a combination of Thales proprietary middleware written
> in C combined with Java code enabled by the Java™ Native Interface (JNI).

> AONIX has additionally delivered professional services to help PERC Ultra
> users at Thales optimize their applications and improve execution
> performance including, a customer-specific training course handled the needs
> of coding in Java™ with hard real time constraints.

Seems like with helping of C and Consulting on 'How to do Java right' they
made it work.

> End-to-end distributed thread timing constraints measured from stimulus to
> response are typically under 100 ms.

Since it is slightly faster than typical (per char) typing speed on keyboard.
It appears nice but terribly impressive real time Java use case.

~~~
pjmlp
For me it is impressive, because if the system doesn't work, the wrong guys
die.

------
themartorana
The article sort of glosses over IntelliJ with the golang plugin as an "other"
IDE, but it's best in class hands down. At one point in my past I had sworn
off of Java-based IDEs, and used Sublime Text for years with primarily Python
and Go. Something convinced me to try Go in IntelliJ, and really it's
fantastic.

It also covers "important-to-me" features the author mentions. I haven't tried
VSCode yet, but have tried all the others listed. IntelliJ is fantastic for
Go.

~~~
sagichmal
In my experience most Go developers seem to favor a terminal-heavy workflow,
sometimes frequently shutting down and re-opening their editors in new paths
and projects. The barebones simplicity of Go-the-language seems to be a nice
match to this workflow. It's a different thing altogether to how large IDEs
like IntelliJ expect you to work: opening up a project and staying in the IDE
for a long period of time.

Both approaches are totally valid, of course, but I suspect Go developers are
biased toward lighter-weight editors.

~~~
nedsma
I concur. Go syntax is dead simple and even the most basic tools like
[https://play.golang.org/](https://play.golang.org/) can be used without
hesitation for some quick and dirt POCs. In the discovery mode, when you're
learning a new library, any tool that supports code completion and provides
documentation links is nice to have.

~~~
xixixao
Surprises me it doesn't have syntax highlighting.

~~~
tujv
Possibly because Rob Pike is not a fan of syntax highlighting, calling it
juvenile.

[https://groups.google.com/forum/#!msg/golang-
nuts/hJHCAaiL0s...](https://groups.google.com/forum/#!msg/golang-
nuts/hJHCAaiL0so/kG3BHV6QFfIJ)

~~~
ramchip
Wow, his second response is really demeaning. Or maybe it's just a shock to me
after hanging out in really friendly communities (Elixir etc.) for a while...

~~~
twr
Pike literally quoted a passage from the bible:

[https://en.wikipedia.org/wiki/1_Corinthians_13](https://en.wikipedia.org/wiki/1_Corinthians_13)

I won't argue that it was in good taste.

~~~
QuercusMax
If by "literally quoted", you mean paraphrased... Or is this just another case
where literally doesn't mean literally?

~~~
twr
No. Sorry. I should have linked to the NASB translation, which was the source
of his (literal) quote.

[https://www.biblegateway.com/passage/?search=1+Corinthians+1...](https://www.biblegateway.com/passage/?search=1+Corinthians+13%3A11&version=NASB)

------
MereInterest
> Only func main has the right to decide the flags that will be available to
> the user.

This one applies to every language. I was working with a python package that
decided that, since `sys.argv` was available from anywhere, it should parse
`sys.argv` to configure its own behavior. For a long time, their suggestion
was to first parse `sys.argv` yourself, then to modify it before going into
the library.

~~~
bbrazil
This defeats the purpose of a good flags library though.

Where flags shine is when you've some obscure tunable deep in the dependency
tree that you need to tweak (particularly in an emergency). Plumbing through
potentially thousands of flags through to main is nonsensical in that
scenario.

~~~
dlor
The flag libraries I've used always make this easy, but the problem comes in
when you try to reuse that dependency in another binary as a library.

Maybe you've seen a better flags library than I have, though.

~~~
bbrazil
No, the flags libraries I've used had exactly that issue.

When that came up what you'd do then is refactor that to be a class parameter
or config option or whatever (and we'd usually ask that the flag be kept in
some form).

Until then though you get the benefits of a quick and easy way to both expose
and use tunables, which is much better than not having it at all.

------
alexellisuk
After working with .NET/Java/Node.js/Ruby/Python etc the move to Go involved a
larger investment in time. I found this really informative and it's great to
have the learnings condensed down.

~~~
nedsma
Check out YouTube conf. talks from the same author, where he shared his
experience developing services in Go, while working at SoundCloud. Also, very
helpful.

~~~
Philipp__
Could you please provide the link?

~~~
Hates_
I'm assuming they are the ones listed here:
[https://peter.bourgon.org/talks/](https://peter.bourgon.org/talks/)

~~~
alexellisuk
Checking these out now. I use SoundCloud.. cool to hear about their
infrastructure. It is a bit weird how you can't see his hands in the shot:

[http://www.infoq.com/interviews/bourgon-crdt-
go](http://www.infoq.com/interviews/bourgon-crdt-go)

------
wtbob
Good article on the whole, but I have a few quibbles:

> If your repo foo is primarily a binary, put your library code in a lib/
> subdir, and call it package foo.

IMHO, that's just ugly, uglier than foo/foo or foo/foolib or foo/libfoo.

I also think that anything which has commands other than a single-command
project which will _always_ be a single-command project (there are fewer of
these than one might think …) should put all commands, even a single one, in
cmd/.

I think that inline config objects should be used with useful zero values to
try to emulate Lisp's or Python's keyword arguments: if one always needs to
provide a value for each config object member, then just use arguments after
all.

I think a testing library can be a great addition, since it can turn a three-
line if check into a one-line assertion.

~~~
curun1r
> IMHO, that's just ugly, uglier than foo/foo or foo/foolib or foo/libfoo. > I
> also think that anything which has commands other than a single-command
> project which will always be a single-command project (there are fewer of
> these than one might think …) should put all commands, even a single one, in
> cmd/.

It's this kind of inflexibility in the directory structure of ones repo that
really turned me off of Go. It's such a trivial thing, and yet the fact that
Go lacks a level of indirection (package.json, Cargo.toml, pom.xml, etc) to
map a chosen directory structure to the standard build tooling causes problems
that get really annoying. For instance, there's a ton of Go repositories out
there that are not go gettable because the author wanted to put source code
under a src directory and hacked that together using make. And good luck with
organizing any repository that contains multiple languages where go is one of
them...go's "code lives at the repository root" doesn't play well with others.

It all makes me sad since Go's decentralized dependency management (i.e. go
get being able to pull/build based on a meta tag) is brilliant and I always
hate having to rely on a single, centralized repository to deal with
dependencies.

~~~
wtbob
> For instance, there's a ton of Go repositories out there that are not go
> gettable because the author wanted to put source code under a src directory
> and hacked that together using make.

Are there? If someone's that ignorant of the language, I don't think I'd want
to run something written by him …

> And good luck with organizing any repository that contains multiple
> languages where go is one of them...go's "code lives at the repository root"
> doesn't play well with others.

I've actually built libraries which involved both Go & Python without a
problem.

~~~
curun1r
> Are there? If someone's that ignorant of the language, I don't think I'd
> want to run something written by him …

Yes, there are. I don't know any off the top of my head, but I see them all
the time. So, just for fun, how about a little test:

Step 1: Google 'notable applications written in golang'

>> first result: [http://www.infoworld.com/article/2843821/application-
develop...](http://www.infoworld.com/article/2843821/application-
development/10-open-source-projects-google-go.html)

Step 2: Click through to github repos

>> Result 9/10 of them won't work with go get.

So, are Docker, Kubernetes and Etcd the kind of software you wouldn't want to
run because their creators are too ignorant of the language?

~~~
sagichmal
The etcd binaries are go gettable. The others are large enough to easily
justify their own build process.

------
alblue
It was a great talk at QCon London earlier this year - the video is due to be
published towards the end of next month. I'll try and come back to this thread
when it is with the URL.

(Disclaimer: I was the track host for Perer's talk)

------
dllthomas
_" those parameters should be part of type constructors"_

I'm not sure if this is a nit, a misunderstanding on my part, or a difference
in terminology, but I think what is meant here is "value constructors" (or
more commonly, just "constructors"). As I understand the term, "type
constructors" construct _types_.

~~~
sagichmal
Well, there are no type constructors in that sense in Go.

~~~
catnaroek
Not even for pointers, maps and slices?

~~~
dllthomas
Oh, good catch!

------
hesdeadjim
Libraries using log.Printf are incredibly tough to work with in a production
environment if you want anything more interesting than looking at stderr.
Libraries that let you provide a logger API at time of construction are better
than log.Printf, but they still fail at letting you include contextual
information inside method calls.

We inject a logging interface into all our methods that take a context.Context
object [1]. This allows us to push contextual information onto the stack of
context objects, and then when we log we can do it at the point of failure and
have access to an immense amount of useful information. Check the attached
gist to see an example of how this works in practice [2].

Given that the context library originated out of Google and is now part of the
stdlib in 1.7, I would love to see other libraries embrace it instead of
relying on much less flexible solutions.

[1]
[https://godoc.org/golang.org/x/net/context](https://godoc.org/golang.org/x/net/context)

[2]
[https://gist.github.com/justonia/f81eead323d2b23eca1c485ed8e...](https://gist.github.com/justonia/f81eead323d2b23eca1c485ed8e5db5d)

------
dllthomas
> No statements go by where the object is in an intermediate, invalid state.

This seems kind of misleading. Per my understanding of Go, omitted fields in a
struct initialization are defaulted not reported as errors. So you're equally
likely to be passing invalid state, in the two situations.

One pattern I like in Haskell, for this kind of thing, is to define a
defaultConfig value that contains typical defaults and which can then be
tweaked as desired.

One big advantage this has is that when some functionality is made newly
configurable, you set the default to be the old behavior and existing code
continues to work correctly unmodified without any additional effort.

~~~
sagichmal
Hopefully you'll combine that idiom with "make the zero value useful" so that
omitted fields still give valid behavior.

~~~
dllthomas
Reading on, I see that.

It still breaks down when a zero value is meaningful for a configuration
parameter, but means something different than what was done before the
parameter was configurable. I think that's probably not super often.

In light of all this, I still don't really see what harm we're avoiding by
preferring the inlined struct initialization. I do agree it looks a little
prettier, but the article seems to give it a greater import.

------
krat0sprakhar
> _That advice still holds today: vendoring is still the solution to
> dependency management for binaries._

This might be a stupid question, but can someone explain to me what this
means? Thanks!

~~~
sudhirj
To vendor a dependency is to store a copy of all the source code you're trying
to use inside your project itself. That way, when you compile your project,
you also compile the code it depends on.

Updating code you depend on is a semi-manual process - you choose to copy the
latest version of the code you depend on into your project again.

This is contrast to stacks like Java, where I can give you a pre-compiled JAR
file that you can link your code to.

~~~
fludlight
Is this different from statically linked libraries (as opposed to dynamically
linked)?

~~~
NateDad
This is about code, not libraries. Vendoring the code means copying it into
your own repo... that way you're guaranteed that everyone who builds your
project is building the exact same code, and if one of the third party repos
disappears, your project doesn't all of a sudden fail to compile.

------
transfire
What you need is... a
[goat]([https://github.com/mediocregopher/goat](https://github.com/mediocregopher/goat))

------
kinkdr
Looks a lot like a language designed by a committee, but I generally like it.

I don't love it, like I love Python, but it does its job and it is fast.

What I really like is the defer(), the channels(having the option for
asynchronous channels would be awesome), the go routines(async callback hell
is getting old).

What I find completely awkward however is the enforced first letter
capitalization(don't tell me how to live my life, go!), the interface{}, and
the GC.

~~~
calcifer
I get - and agree with - the complaints about interface{}, but what's wrong
with the GC?

~~~
kinkdr
No control over it. It will run when it decides to run, stalling my app.

------
dineshp2
Slightly tangential, but, could someone share their experience using Go
specifically for building websites?

How does Go (including Go frameworks specifically geared towards web
development) compare in terms of performance, ease of development with RoR,
Laravel?

Is building websites using Go a good use case or is Go better suited for
building high performance microservices?

~~~
dalyons
I've done a lot of ruby and go. Honestly, I would think twice before building
a major website/product API on go. Development time is markedly slower in go,
some causes off the top of my head:

\- testing is way harder (ruby has some amazing mocking/stubbing/testing
tooling).

\- debugging is way harder. Hope you like printf.

\- Json; the strict typing & struct searialization rules make dealing with
Json a pain in the arse, and heavy on boilerplate

\- interfacing with Sql (if you have to do a lot of it) is a pain in the arse,
for similar reasons to json.

\- having no stack for errors is insanely frustrating.

\- templating in go is primitive

Go is amazing for services. The parallelisation primitives are fantastic, and
allow you to do things that would be hard/impossible in ruby/Python. Go is
undeniably much faster, execution wise. (Although our super high volume ruby
product api is at 50ms median so, it matters less than you think)

Decide what you care about most; if your prime goal is to get a product in
front of customers as soon as possible, I'd pick a higher level language. If
you want to make a rock solid service that sips on resources & scales to the
moon, use go.

~~~
jsmthrowaway
Median latency is a completely useless number. There is literally no use for
it, ever. Satan invented median latency to confuse people, because it lies to
you and whispers sweet nothings in your ear. Averaging latencies in your
monitoring leaves the outliers out in the cold because you will never see
them, and tail latency is _way_ more important for user experience.

Quantile your latency. I suspect you'll find 99th tells an interesting story
between Ruby vs. Go. The best latency graph has five or six lines at different
percentiles (yes, including 50) stacked in a latency sandwich. I'm willing to
bet your app has a rough 99th. Maybe even a rough 95th. Nothing against Rails
(honest, this applies to most interpreters), but most scripted stuff has a
pretty rough latency tail, and you ignore that with median. (Go is not off the
hook either; GC pauses can blow out high percentiles if you're memory-heavy.)

I'm pretty passionate about this, sorry. I'm on a mission from God to purge
our mortal plane of average/median latency, because it's one of those
misapplications of statistics that everybody does without a second thought.
Don't even collect it. Average latency is unimportant and misleads people, and
misleads people _more_ as you get more popular. It's that 1% of clients
sitting there for 2sec that impacts perception of your UX.

~~~
yxhuvud
> Median latency is a completely useless number. There is literally no use for
> it, ever. Satan invented median latency to confuse people, because it lies
> to you and whispers sweet nothings in your ear. Averaging latencies in your
> monitoring leaves the outliers out in the cold because you will never see
> them, and tail latency is way more important for user experience.

Umm, you do not get the median latency by averaging anything. Median latency
is just the 50th percentile. It is definitely not one of the interesting ones
to reason about or care about improving, but not valueless to measure. It is
interesting to have if you are graphing your latency curves, to make an
example.

~~~
jsmthrowaway
I think if you reread my comment you'll find that what you're saying does not
actually disagree and is a restating of what I said.

~~~
yxhuvud
It is not. Your comment confuses the median with the mean latency. The comment
you replied to did not mention using averages - it mentioned only using the
median. You introduced scathing critique of using averages when those are only
used for means and therefore totally irrelevant here (even if I totally agree
that people that use them should be educated. But that wasn't the case here).

~~~
jsmthrowaway
I actually don't, and you're reacting to the use of "averaging" as a verb.
That's why I encouraged you to reread. When you're discarding 50% of the
samples in a 50th percentile median, I think "averaging" is an acceptable verb
to proxy the situation in English since "medianing" isn't a word. I could have
said "computing the median," but that's just tedious.

Notice later in the comment I say average/median, implying that they are
separate but related concepts. I think it's safe to assume that someone who
can conversationally use the word "quantile" is not confusing median with
mean. You're assuming that my (intentional) selection of a lower-fidelity term
to describe a concept, which I carefully illustrated with ancillary points to
give specificity to said concept, demonstrates a misunderstanding of the very
field I'm explaining. That is pretty obviously wrong and a bit condescending.

We agree, which is what's frustrating. You're just latching on to a pedantic
correction of my point, and rather than belabor that correction I encouraged
you to reread to see that we do actually agree. Now, I _do_ see average
latency far more often than I'd care to admit, which is why I got lazy and
just said "average" at the end there once I started referring to generality
instead of specificity, but I think it's pretty clear that I understand the
distinction regardless.

~~~
skj
In statistician-land, the median is simply one way of averaging, as is the
mean. Introducing (or at least using) the extra term takes care of some
ambiguity.

------
pred_
The site blocks Tor users; ugh.

~~~
dlsniper
Try a different exit node, the website uses Cloudflare so if you could access
HN you should be able to access that one as well.

~~~
cyphar
Tor creates new routes every X minutes (usually 15). It also uses a different
route for every IP it routes to. So they'll always go through a different exit
node.

