
Go best practices, six years in (2016) - metmirr
https://peter.bourgon.org/go-best-practices-2016/
======
bbatha
This is generally great advice, and aside from the package manager
recommendations is still relevant today. I do take issue with a few things
though.

1\. Don't put your internal libraries in /pkg. /pkg has special GOPATH meaning
as "compilation cache". Its not an actual name conflict, but why bother
risking it.

2\. This is just wrong:

> fmt.Printf is self-contained and doesn’t affect or depend on global state;
> in functional terms, it has something like referential transparency. So it
> is not a dependency. Obviously, f.Bar is a dependency. And, interestingly,
> log.Printf acts on a package-global logger object, it’s just obscured behind
> the free function Printf. So it, too, is a dependency.

stdout (and the buffer, and mutex on it) are exactly the same as the global
log object. In fact, `log.Printf` is more or less just an alias to
`fmt.Printf`[0]

3\. I wish it had mentioned the functional options pattern in the part about
constructors[1]

0:
[https://github.com/golang/go/blob/b77aad089176ecab971d3a72f0...](https://github.com/golang/go/blob/b77aad089176ecab971d3a72f0357ed0abc81f4a/src/log/log.go#L294)

1: [https://dave.cheney.net/2014/10/17/functional-options-for-
fr...](https://dave.cheney.net/2014/10/17/functional-options-for-friendly-
apis)

~~~
atombender
> the functional options pattern

Functional options are controversial in the Go community. I don't like them
personally. The idea of having a bunch of functions that exist only to mutate
internal state on init is... an odd choice. Google code is riddled with this
sort of style.

It's annoying to write client code for another reason: It's hard to discover.
For the "term" example in the article, you can't sit in your editor/IDE and
type "term." and get a list of setter suggestions, since everything in a
single namespace. "term.Speed()" does not _sound_ like an option to me.
Function names are generally verbs, and having one called Speed() doesn't read
well. And it causes issues if you want to have a _type_ called Speed. Then it
has to be term.WithSpeed() or term.SetSpeed() or something. Neither of which
is really self-explanatory.

The fact that options are functions has other downsides. The moment you pack a
option into a function, they've lost their ability to be introspected. You
can't dump the options to a debug log before you invoke the constructor
function. You can never ask an option function what it contains.

I prefer representing options using actual data:

    
    
      type (
        Option interface {
          isOption()
        }
    
        Speed int
        RawMode bool
        SomeOtherSetting struct {
          A, B int
        }
      )
    

Unfortunately, due to how Go interfaces work, you'll have to provide a dummy
private method to tie the types together, but it's a small loss. (Go itself
uses this pattern in a bunch of places, such as the compiler's AST.)

~~~
discoursism
My biggest object to the functional options argument is the step where

    
    
        NewServer(..., Config{})
    

is considered bad for some reason. As far as I can tell, the entire purported
utility of the thing is based on avoiding this obvious approach.

~~~
lobster_johnson
A big config struct is great, but it comes with a big downside: You have to
rely on Go's zero value semantics to provide defaults. It's not always clear
that 0 or an empty string means "no value".

It means that sometimes you'll have to use dummy values (-1 for "no value",
for example) or pointers (nil means optional). It means that any boolean has
to be phrased in such a way that false is the default (so if it feels natural
to call the option UseKeychain and it should default to true, you have to
invert it and call it DontUseKeychain).

Config structs also means it's easier for a caller to construct invalid
configs:

    
    
      type Config struct {
        // Either stream or channel can be set, but not both
        OutputStream io.Writer
        OutputChan chan Event
      }
      // ...
      pipelines.New(pipelines.Config{
        OutputStream: myFile,
        OutputChan: myChan,
      })
    

The semantics cannot be guaranteed by the type system, so won't be caught
until runtime. Neither will this, of course:

    
    
      // ...
      pipelines.New(
        pipelines.OutputStream(myFile),
        pipelines.OutputChan(myChan))
    

...but in this case we can quite sensibly mandate that the following option
overrides the previous one, flipping the internal switch over to the setting
you specify. With the Config struct, there's no way for pipelines.New() to
know which one should have precedence.

(In this particular case, you can work around it by having a single Output
field that uses an interface, but there are other situations where mutually
incompatible fields can occur.)

~~~
discoursism
Yep! It was probably overstepping on my part to say that the _entire_ benefit
depends on that argument. I should instead have said that the tradeoff does
not seem good to me.

~~~
Groxx
It depends on the purpose. Config structs are indeed simpler and make tons of
sense for cases where you control all the related code. So yeah, for code
where you can do a mostly-automated refactor if you need to change the API, I
totally agree that Config structs make more sense and functional options are
_huge_ overkill.

For stuff that needs finer-grained control, or long-lived API stability,
functional options give you a lot more flexibility.

------
nzoschke
I simply love the go toolchain. The defacto standards for code formatting,
linting and vetting, and the speed of builds and tests save tons of time. Both
clock time and overall developer productivity.

I have a GitHub CI service that verifies the standard go tools pass on every
commit.

The bot itself is naturally written in Go and executed in a Lambda function.

Everything is just so fast! You get feedback in seconds. Which keeps you in
the flow of coding.

Maybe some other gophers will find it useful:

[https://www.mixable.net/products/bios/](https://www.mixable.net/products/bios/)

------
0xmohit
50 Shades of Go: Traps, Gotchas, and Common Mistakes for New Golang Devs [0]
might serve as a nice supplement to this article.

[0] [http://devs.cloudimmunity.com/gotchas-and-common-mistakes-
in...](http://devs.cloudimmunity.com/gotchas-and-common-mistakes-in-go-
golang/)

~~~
kylequest
I have another post that covers a few project structure best practices:
[https://medium.com/golang-learn/go-project-
layout-e5213cdcfa...](https://medium.com/golang-learn/go-project-
layout-e5213cdcfaa2)

It has a little more than Peter's post, but it's still not too opinionated :)

------
zalmoxes
At [Kolide]([https://kolide.com/](https://kolide.com/)) we're heavy users of
Go Kit, and as a result have also adopted a lot of the style Peter recommends
here. We've been slowly expanding on it with a style guide and company
specific set of common libraries [here]([https://github.com/kolide/kit#kolide-
kit](https://github.com/kolide/kit#kolide-kit)). My coworker also wrote a
[blog post]([https://blog.kolide.com/using-go-for-scalable-operating-
syst...](https://blog.kolide.com/using-go-for-scalable-operating-system-
analytics-cb170d85b1c5)) on how Go has been fantastic for us and references
the above style guide.

The code from go kit and
[oklog]([https://github.com/oklog/oklog](https://github.com/oklog/oklog)) are
great examples of idiomatic Go. Unfortunately the community at large doesn't
really follow the "no init"/"no package global vars", which can sometimes lead
to bad experiences importing opensource Go libs.

~~~
junkscience2017
I feel like go-kit is quite antithetical to the Go mindset...it presents a
lossy abstraction as a means of future-proofing against eventualities that
will almost certainly never be encountered

to be honest it strikes me as the sort of library that excites intermediate
developers who tend to over-architect

~~~
lima
I vouched for this comment - why would it deserve to be flagged?

~~~
grzm
This comment in particular wasn't flagged: 'junkscience2017 is banned. Note it
was marked [dead], not [flagged].

~~~
lima
Thanks for the explanation!

------
jjuel
This is great, but I wish it would get updated for 2018 as it is now 2 years
old.

~~~
HankB99
Search for "update:" in the article. There are three and the most recent
references a post from June of 2017. It appears the author has modified it for
developments he considers important. It would be helpful if he indicated the
revision date near the top of the article.

------
tuespetre
I love the way the headings and their 'anchors' are styled on this blog post.
I don't know, it just looks so pleasing.

------
Goopplesoft
Is anyone else using golang dep[1] as their vendor/dependency tool?

I’ve used it and it works great (much better than glide, gb et al) but I don’t
see it in the wild too often.

[1] [https://github.com/golang/dep](https://github.com/golang/dep)

~~~
pjmlp
Well, it is on its way out, have you missed the news regarding _vgo_?

[https://research.swtch.com/vgo](https://research.swtch.com/vgo)

~~~
kitd
Er, thanks. We've just moved to go dep :)

Go's dependency management story is starting to resemble JS's modules story :(

~~~
dilap
Yeah... In defense of the Go folks (1) they are working hard to make the
transition as smooth as possible (2) dep was always just considered an
experiment (though with aspirations by the author to have it become the
official solution) (3) vgo almost certainly marks the end of the churn, and
looks really nice.

------
indescions_2018
Author of go-kit microservices framework, which has several nice examples with
which to get up and running

[https://gokit.io/examples/](https://gokit.io/examples/)

Another interesting recent project is Truss which allows you to autogen go-kit
handlers from protobuf definitions

[https://github.com/TuneLab/truss/blob/master/TUTORIAL.md](https://github.com/TuneLab/truss/blob/master/TUTORIAL.md)

~~~
wiremine
I found Gokit's stringsvc tutorial particularly helpful:
[https://gokit.io/examples/stringsvc.html](https://gokit.io/examples/stringsvc.html)

------
sdhgaiojfsa
Pretty good advice, except the preference for interfaces to support mocking.
You should avoid mocking when possible in favor of real objects or fakes.
Interfaces are still cool, though.

~~~
sagichmal
This is straight-up bad advice.

You don't test the http.Client as part of your business logic, even
(especially) if your business logic makes HTTP requests. Instead, you trust
that package http is already well-tested, mock out http.Client as an abstract
HTTP Request Do-er, and you downscope your tests to your business logic
exclusively.

~~~
sdhgaiojfsa
Lots of people have different opinions on this. You're entitled to yours, but
I think it's hasty to call it bad advice.

I'm not so worried about http.Client behaving incorrectly, but I am worried
about the server I'm targeting behaving in a way that's different than my
expectations. I'm also worried about getting cookie, post-data, url, etc.
formats wrong. Even if they match what I put in my mock, they won't
necessarily match what's out there in production. I'm worried that I'll handle
download resumption wrong, for example.

If I have the capability to bring up the target server locally, that's what I
will do. If I can't, but it's straightforward to create a work-alike, I'll
bring that up in process. Only in the most dire of circumstances will I use
mocks to test my code.

Think about it this way. What harm is done by using the real object, under the
circumstances I described? Sure, if the dependency code breaks, my tests will
too. Is that bad? Is there any point in my tests passing when my code won't
actually work in production? OK, so maybe the tests will run slower? This
matters in the limit, but often times you can use the production dependency
without materially increasing test time. OK but what about the CI system using
more resources? That will only be a problem of significant economic import if
you're running at a scale like Google or Facebook.

My opinion is there are a few main valid objections to my approach. One is an
argument about the size of the test code. In some cases, bringing up the
production dependency will require more code than a mock. Sometimes a lot more
code. My response is that we should make it easier to bring up the production
dependency! But if we can't, I can see the case for a mock or a fake. The
other counterargument is for when we really are interested in testing the
sequence of interactions the code under test has with the production system.
In that event, I guess I'm ok with using a mock, but I think most of the time
we're more interested in testing state. We can also use things like testvalue
injection to test behavior sequences, which is normally a lighter-weight
approach, and in some cases can allow you to reproduce paths that you'd
otherwise have trouble hitting, even with a mock.

~~~
tigershark
It seems a bad advice also in my eyes. I guess that you have never worked on
big systems if for you it is normal to bring up all the production
dependencies for an integration test. For small projects it may make sense,
although I would still prefer to write unit tests to catch exactly were the
bug happens and to be independent from breakages in the external systems. The
correct approach is to rely heavily on unit tests that run continuously at
every build and run the full integration testing suite at regular intervals or
overnight. In a _quite big_ project on which I worked many years ago the full
integration test suite took 11-12 hours to run, I would be curious how you
would proceed in that case given that from what you write it seems that unit
tests are pretty useless to you..

~~~
sdhgaiojfsa
> I guess that you have never worked on big systems

I work on search and indexing systems at Google, and have done for more than a
decade. I'm by no means the best, nor an authority. But I have spent some time
living in this problem space, and this is my conclusion from my years of
experience.

FWIW a single run of the tests for the system I work on now takes . . . I
don't know. I would guess hours or maybe even a day of computer time? But,
computers are cheap, and defects are expensive. So you split the test suite
into bits small enough that the whole thing runs in ten minutes, and run them
all in parallel. And, yes, there have been cases where I have had to debug the
slow startup of a dependency so that I can make it fast enough to run in my
test. But, that's what they pay me for, I enjoy that type of work, and it has
ancillary benefits to the dependency when it needs to start up in production.

~~~
tigershark
In the real world you don’t have pretty much unlimited resources. If I
remember correctly those tests I was speaking about were already running in
parallel on 2 or 3 machines otherwise the total time would have been 2 or 3
times more. Obviously we couldn’t use hundreds of machines to make them run in
10 minutes. In most of the places were I worked even getting a couple of new
machines for _production_ use would take months, I think if I requested even a
couple of machines dedicated only for the integration testing I would have
been laughed in the face. So if you can run a full suite of integration tests
in minutes on hundreds/thousands of machines then good for you, but don’t
expect that every company has the resources for doing it. And in those cases
unit tests that run continuously are more useful than integration tests that
run once a day.

~~~
sdhgaiojfsa
> In most of the places were I worked even getting a couple of new machines
> for production use would take months

OK, well I guess this would be another caveat for my advice. If bringing up a
production simulacra would take more resources than you have, then you might
have to lean more on various types of doubles. This may be more often in
environments that have insane prioritization of dev time vs. machine
resources.

If I may ask, what was the bottleneck in your test suite? Did it saturate the
cpu, ram, disk?

~~~
tigershark
I don’t really remember. The software was a _kind of_ ORM but it was built to
interface the java code with an object database, so it doesn’t make much sense
to call it ORM. Probably the main bottlenecks were cpu and disk if I have to
guess. I arrived very late in the project, at the point where we were going to
migrate everything to oracle and we had to add oracle support to that
behemoth.

------
petercooper
[2016]

~~~
sctb
Thanks, we've updated the headline.

------
erikb
Here are also some Go proverbs. Some of them are a lot older than 6 years.
[https://senseis.xmp.net/?GoProverbs](https://senseis.xmp.net/?GoProverbs)

------
ctdonath
Rule of thumb: anyone using the term "best practices" doesn't understand them.

(Yes, some do. On the whole I find most don't.)

~~~
spraak
Are you saying that speaks on this article or not? I don't care about a rule
of thumb if it doesn't apply here.

