
Just tell me how to use Go Modules - boyter
https://www.kablamo.com.au/blog-1/2018/12/10/just-tell-me-how-to-use-go-modules
======
nstart
Just going to share a sentiment I've felt while diving into Golang over
several months. The canonical way to do things feels like it's all over the
place in several areas of Golang. Not in the fundamental "this is how you
should indent" bit but in the "this is how you should unpack dynamic json
content" sense.

I really wish golang had more canonical content. But at this point, I feel
like I'm working against the grain. Golang doesn't feel like a language that
wants to prescribe a lot more than fundamentals. How you structure your code
beyond the src/git/path/your-code/ is completely up to you and every repo you
visit could be different. It's not my thing personally, and I'm leaving this
comment just in case any "want to try golang" folk stop by. This is something
you should be aware of going in. Don't bang your head when you can't find a
clear way of doing something that probably seemed simple in other languages.
Embrace the new concept and see if you can get used to it.

That said, if anyone ever breaks into the eco system with an es lint style
tool that allows experts to share their best practices via plugins, that would
be awesome. If that exists already, please do let me know. I'd love to give it
a go.

~~~
weberc2
> How you structure your code beyond the src/git/path/your-code/ is completely
> up to you and every repo you visit could be different.

This is an unusual criticism. Typically Go is faulted for being too
opinionated, and it’s projects too predictable. In fact, this is why I took to
the language—no need to see past every maintainer’s personal flourishes just
to build the code or run the tests. No ceremoniously nested directory
structure. No figuring out how to point the build tool to the source files or
how to convince the build tool to output or run a test target. Everything is
right there, rather plainly and consistently across all projects. This does
mean I have less room for creativity which is an okay tradeoff for me because
I have just enough creativity to spend on the application and its architecture
and the rest should just work, but I recognize that some may disagree and
that’s okay too.

~~~
Felz
This sounds like tooling and formatting consistency versus abstraction
consistency. Go isn't great for building powerful abstractions, so at a higher
level implementations will trend towards inconsistency, however consistent the
low level syntax.

~~~
weberc2
A few thoughts:

1\. I don't think powerful abstractions are often that useful. I use Python in
my day job, and we mostly don't use many high level abstractions. We never use
monads, for example, and while we do use async futures, these are strictly
worse than Go's concurrency features.

2\. Pretty sure abstraction power drives _less_ consistency, not more.
Effectually, it means there are more ways to do the same thing (and more ways
to screw it up and make a mess), and consistency depends on organization
discipline and less on the rails of the language. I don't mean to say
abstraction is bad--only that consistency is one of the costs, not an
advantage.

I'm interested to hear your thoughts.

~~~
zeugmasyllepsis
> Pretty sure abstraction power drives _less_ consistency, not more.

It depends, I think. Abstractions allow users to specify part of a program's
behavior independently from others. This is _not_ intended to start a flamewar
or pick on Go, but for better or worse Go does not support user defined
generic functions. One consequence is that certain kinds of methods are
difficult to write in a generic, performant, and type-safe way. If users want
performance and type safety, they need multiple implementations of those
methods for each desired type. This leads to multiple copies of very similar
code (which can evolve independently, lowering consistency) or the use of on
of the community-driven code generators (which all have their own differences,
again lowering consistency).

Adding generics _could_ largely make all of this inconsistency go away, but as
you point out would enable a whole slew of other programming patterns more
ways to make a mess.

I prefer to think of abstraction techniques as _shifting_ consistency.
Ideally, they provide an obvious way to factor out one piece of a program's
behavior, allowing developers to focus on the rest of the behavior
independently.

~~~
weberc2
I largely agree, but I think people generally underestimate the abstractive
power of (typesafe) Go and overestimate the cost of its abstractions.

Generally Go's abstractions are based on dynamic dispatch (virtual function
calls) which are quite cheap and are the basis for most other languages'
abstractions. There are languages like Rust and C++ which offer zero-cost
abstractions and generally beat Go in performance and abstraction power, but
lose in other ways. If you're comparing Go to Java, C#, Haskell, OCaml,
Python, JS, and Ruby, Go performs on par or better.

As far as abstractive power, Go does quite well, although there is still some
_boilerplate_ (for loops and closures); however, this is generally not where
your bugs come from (certainly not your troublesome bugs) and boilerplate is
consistent by definition. I think the biggest reason Go's lack of generics
irks people is because they want _terse_ code, because terseness is the most
obvious (but least useful) kind of DRYness. The next biggest reason is more
legitimate--getting a lot of typesafe abstractive power out of Go requires a
different bag of tricks (and a different way of thinking about abstraction)
than languages with generics.

------
triodan
Thank God, finally a proper "Getting Started" guide for Go Modules. The
official docs and wiki page always felt so obtuse and unhelpful about how to
actually _use_ the feature, while dep at least offers the obvious "dep init
then dep ensure and you'll be fine" solution.

------
shanev
Today I had to write some deployment scripts and the value of a fixed GOPATH
really dawned on me. No matter which machine I use, I always know my code will
be at GOPATH/src/...

I didn't have to fiddle with any paths or env vars in my scripts. It made my
day easier. This is an example of something that seems like a good practice
only after you've gotten some gray in your beard.

~~~
rauhl
Yup, that’s been my own experience too. I’ve really come to appreciate the
virtue of GOPATH, and I honestly don’t understand the complaints about it.

Why _wouldn’t_ one want all of one’s code in a single place? What’s wrong with
organisation? If one needs to work with different versions, then that’s what
version control is for, isn’t it?

Obviously I’m missing something, because my opinion seems to be in the
minority. I wonder if it’s because most people are using GUI file managers,
which make using directories of code more difficult, while I use emacs and the
shell, and so can comfortably manipulate files & directories at scale.

~~~
_rs
Because different projects need different versions of their dependencies. Do I
need a separate GOPATH for each of these projects? I haven't wrangled Go
enough to know the right answer to that yet. In my experience with other
languages: I'd really prefer to have the dependencies local to the project and
not in some global folder that affects all of my projects. I want to be able
to make a random project folder anywhere and have the entire thing be self-
contained without worrying about anything outside of that folder.

~~~
wbl
That's what vendor and modules are for.

------
maddyboo
Something which isn't mentioned here and is difficult to find precise
information even in the official docs is how versioning works - specifically,
how do you specify what version your module is? Semver is mentioned, but not
how to _define_ your version.

The short answer is to use git tags. You need to include the `v`, e.g.
`v1.2.3`. If you don't use git, I don't know the answer.

~~~
gcb0
versions? 2018 is the year of the evergreen module dependency injection!

~~~
alexandernst
Absolutely stupid... I don't want my poduction pipeline to be injecting
evergreenshittery I haven't revised/tested.

~~~
bpizzi
Not sure if you were being sarcastic...

But this is exactly what 'go mod' is NOT doing. It's called 'Minimal Version
Selection'. Quoted right from the lead core dev's blog: ' For stable
development, today's build list must also be tomorrow's build list.'
([https://research.swtch.com/vgo-mvs](https://research.swtch.com/vgo-mvs)).

From what I gathered, 'go mod' is the first mainstream tool which is going
that way (but I would be glad to be proved wrong!). Other package manager are
using lock files which have their caveats (see
[https://research.swtch.com/cargo-
newest.html](https://research.swtch.com/cargo-newest.html) for an in depth
examination of the revered cargo approach on low-fidelity builds).

~~~
greysteil
The version resolution algorithm is very similar to NuGet's "lowest applicable
version" logic, which has been around since 2010. I've seen surprisingly
little discussion of the fact they're so similar.

[https://docs.microsoft.com/en-us/nuget/consume-
packages/depe...](https://docs.microsoft.com/en-us/nuget/consume-
packages/dependency-resolution#dependency-resolution-rules)

------
teknologist
Go newb here but I personally found it easy to get started with Go Modules and
migrate over a fairly large project using Dep. `go mod init` was very helpful
here in picking up the Gopkg.toml and doing the magic to make it all work. No
complaints at all.

------
ra7
A more detailed intro to Go Modules: [https://roberto.selbach.ca/intro-to-go-
modules/](https://roberto.selbach.ca/intro-to-go-modules/)

------
laurent123456
There should be a website like "justtellmehow.com" with useful articles like
this.

I found that lib and tool documentations written by their authors tend be to
be too long. It's understandable since they are passionate about the subject
and want to talk about every details, but it's not great when you just want to
quickly add a lib to your project or use some tool.

~~~
SlowRobotAhead
This can of course go beyond programming. But, yes, there is a major lack in
CS and CE of relaying on the INTENT.

\- I can see the datasheet where this micro has 32 message objects and a FIFO
mode - but please tell me how you INTENDED it to be used because after weeks
of figuring out the obscure memory and buffer system it turns out you never
INTENDED a fifo rollover. Ok, that would have been excellent info on Day1.

\- Your framework has a couple of examples and each function is documented.
Great, but imagine my surprise when it turns out I can’t access a ‘global’
memory object outside of your function because you never intended that to
happen - and never thought to let me know.

\- This is the entire concept of a WHY comment vs a WHAT comment. Don’t write
_/ /increments X+1_ instead use _/ /counter ticked here because new value is
used during the loop, must be placed at the top, uint8 / rolls at 256_ for an
x++ example. WHY comments made a big difference in the readability of my code.
The goal is to relay intent.

The thing is, you don’t want to rigidly be stuck to someone else’s idea - but
it’s extremely helpful to know how they planned for you use this because maybe
the choices they made do make sense. _Oh use an array of this memory type
instead of casting it all to a class /structure! Yea, that’s fine and will be
more efficient to search through_ etc etc.

I don’t need know why but engineers are really bad at telling you what they
intended when making the product.

------
stubish
For projects with all dependencies hosted on github. I still need to work out
if I can use the new tooling with our internal repos. I imagine I'll need to
be editing that go.mod file.

~~~
josteink
So not only are modules tightly coupled to a specific version control system
(git), it’s also tied to a single centralized online service of this
decentralized VCS.

Maybe I’m just old fashioned, but to me this is a code smell. It reeks of
needless coupling, and I suspect one day (sooner rather than later) the golang
designers will come to regret it.

To top it off, despite being tied to a VCS, it seems to lack a useful means of
enforcing versioned dependencies.

This simply does not look like a good design. Is this really the new and
improved version? If so, how god awfully bad was the previous iteration?

~~~
jchw
It is not tied to Git nor GitHub.

~~~
e12e
So when you're building on a host without internet access, the "github.com"
namespacing isn't an issue?

~~~
jchw
Depends. If you use vendoring, which is supported by Go modules, then you
should be able to avoid needing any internet access. If you want to grab the
dependencies fresh, then obviously if you have a github dependency, it will
require git and the ability to access GitHub.com.

------
raverbashing
Thank you. Go Modules seems to suffer the same disease as emacs where people
trying to explain it go on and on without getting to the point.

------
dana321
Two commands:

go mod init mymodulename

go build

This will download all the dependencies needed automatically.

You can also do this (from scratch as well):

go mod init mymodulename

go run main.go

And it will still download the dependencies.

If you use a modulename like github.com/dana321/test you can include a
subfolder of your project by importing github.com/dana321/test/mysubfolder for
example

If you bring down the wrong version (ie. an older) version of a module, just
edit the go.mod file and modify the version (after the space where the url is)
and change it to master on the line where the module is. When you next run go
build or go run it will re-download and re-populate the line(s) you've changed
in go.mod with the latest version.

There really isn't much else you need to know!

~~~
kasey_junk
There is if you want to vendor your dependencies, which given my experience
with golang dependency management over the years, I’ll be doing for the
foreseeable future.

This guide outlines the set of incantations required to accomplish that &
frankly having fought through figuring it out myself from the doco, this would
have saved me a bunch of time.

------
giancarlostoro
After half a decade of not touching Go I went back to learning it. This was
the one thing tripping me up. Not because I didnt understand it but because I
cant even compile or pull in packages without defining some compiler variable
flag witchcraft. I miss how simpler Go was. The compiler UX there really needs
more TLC. Otherwise I still love what I loved about Go: a really incredible
out of the box standard library which I highly attribute to Go's success since
it allowed people to hit the ground running with Go much quicker.

~~~
yesimahuman
Was it ever easy? I remember hours trying to figure out what to set GOPATH and
GOHOME to and how to get “go get” to actually work. I can’t help but think go
is a wonderful language with a poor package manager unfortunately woven
directly into the language. Really makes npm feel wonderful in comparison.

~~~
giancarlostoro
I cannot for the life of me understand why they don't make setting up all
those variables part of the installer (in the case of Windows, and possibly
Mac?) / first time setup process, or define some sane defaults that don't get
in your way. I can't help but feel most people likely have a similar enough
setup to how I have my environment variables configured at the minimum, why
not automate that repetitiveness at the installer / default configuration
level.

~~~
ptman
There is a default for GOPATH these days, it's ~/go on unix.

~~~
kkarakk
wasn't the case on mac when i dled golang 2 months ago

~~~
sethammons
Not sure what your experience was. Release 1.8 started the default GOPATH in
Feb 2017, which is a tad over two months ago. Maybe you somehow got an older
version.

------
jonathanstrange
One question not answered in this article is whether I will need to use
modules in the future or not.

I'm perfectly fine with not using them, but I've heard conflicting information
about the future roadmap. If I have to use them later, I'd rather switch now,
but if I don't have to switch even in 10 years from now, then I'd prefer not
to switch to modules at all.

Does anyone know more about this?

------
kishansh
Do add the awesome "go mod tidy" command to your blog :-) This removes unused
modules and adds missing ones.

------
bpizzi
> Unfortunately, a quick guide like this doesn’t seem to exist, so I’m writing
> one here. You’re welcome.

Well, `go help modules` is a good combination of exhaustiveness and
conciseness, IMHO.

------
shaklee3
This site needs better formatting. It looks like the author has section
titles, but they're the same font and style as the body.

~~~
valtism
Also, the text being set at alpha of 0.5 makes it very low contrast and
difficult to read, even as a nearly fully sighted person.

------
jfolkins
Please understand that if you vendor your dependencies inside your go.mod
project, you own them and will need to support and audit them in the future.

The mod command does not vet any computed hashes of the dependencies found in
your vendor directory.

~~~
jerf
You own your dependencies and will need to support and audit them in the
future.

Note the lack of conditions on that statement, up to and including the fact
this statement isn't even about Go, nor is it even about open source! It is
simply a fact. You own your dependencies and you need to support and audit
them. You are free to support and audit them by not supporting them at all,
not auditing them at all, and blindly pulling in whatever random mutations
they may go through in the future. You own the consequences of such terrible
support. You own the consequences of good support.

~~~
jfolkins
You are correct and I was unclear. My statement is that you and ONLY you now
own them. You've essentially forked the project and lost your community.

------
vorpalhex
Thanks for writing this! I just picked up go again after several months and I
had to read the formal guide a few times to correctly figure out how to use
modules.

I do like the new modules system so far though!

------
whizzkid
Golang's not being an opinionated software is the main reason I was not able
to passionately start learning this software. Most of the engineers have a bit
of a picture in their mind when it comes to implement a specific function. But
when the question comes "what is the best way of implementing it in Golang?"
then unfortunately the answer is "Do as you want". This prevents me learning
the power of language from the beginning.

Being opinionated is about onboarding people with the best experience from the
experts of the language, then let people overwrite or choose their own way of
doing things when they know what they are doing.

~~~
bpizzi
> Golang's not being an opinionated software [...] "what is the best way of
> implementing it in Golang?" then unfortunately the answer is "Do as you
> want".

I'm really not sure. Idiomatic Go is something well covered. Gofmt is ensuring
consistent formatting. The official docs have an entry dedicated to this:
[https://golang.org/doc/effective_go.html](https://golang.org/doc/effective_go.html).

And what would be an opinionated language if Golang is not?

~~~
kenhwang
Rust comes to mind. Almost all projects fetch dependencies and build the same
way (cargo), with very similar looking project structures and entry point. The
community has rallied around a set of libraries that's used by everyone.

I couldn't care less about formatting differences between repos. Ignoring
superficial formatting difference, Ruby's community also has one package
manager (bundler) and build tool (rake) with a widely followed naming
convention and directory structure and community consensus set of libraries.

Both the above languages I can build a codebase I've never seen before and
have it running within minutes even without README.

Go feels like the wild wild west. Everyone has their own directory structure,
with different ways of fetching dependencies (modules vs dep), building (seems
to have consolidated around go build vs make), with everything hand rolled or
whatever NIH selection criteria they used for their libraries.

~~~
bpizzi
> ways of fetching dependencies (modules vs dep)

Yes, an 'official' package manager have clearly not been a target for the core
team for a (way too) long time, whereas Rust has envisioned that 'community'
part from the beginning.

> I couldn't care less about formatting differences between repos.

Whereas, to me, an official, no-question-asked tool like 'gofmt' is godsend.
It really depends on what you're building and in which context.

> , building (seems to have consolidated around go build vs make)

Not sure here, 'go build' is clearly the idiomatic building way from almost
day one.

> Both the above languages I can build a codebase I've never seen before and
> have it running within minutes even without README.

That was rather not (enough?) straightforward under %GOPATH% era, indeed.
Today, with Go modules, its a one liner 'git pull; go build', granted you have
Go 1.11 available in %PATH% and that the codebase already have a go.mod file
(if not, a 'go mod init <pkgname>' should fix it).

------
SaulOfTheJungle
I am new to Go and I want to start a new project.

In which cases should I adopt modules versus just sticking with $GOPATH?

~~~
SamWhited
Everything is moving to Go modules; unless you need compatibility with legacy
tools I'd start updating and go ahead and use modules whenever possible. It
will make your life easier in the future when you have to transition fewer
things.

------
astockwell
Is there a decent convention for naming your specifically-for-deps module?

‘go mod init vendored’?

~~~
SamWhited
You should probably still be using a full module init path (eg.
myrepos.example.com/mymodule) to make it go get compatible, and regardless of
whether it will only be a dep for a single project or not, the name should be
descriptive of what's in the module or package. Having a module that's
"specifically for deps" sounds wrong to me in general, but without knowing
what you're actually doing that is harder to give advise on.

------
merb
actually

> go get -u ./...

should probably be

> go mod tidy

