
Introduction to Go Modules - robteix
https://roberto.selbach.ca/intro-to-go-modules/
======
bobjordan
Recently found a command line library written in go that was so useful to me
in a python webapp I'm building, I decided to just wrap the go app and call it
from a python subprocess. It is a bit slow to do it this way, but premature
optimization, and all that. Anyhow, the bottom line is due to this I was
forced to get a handle on Go this week so I could at least build the go
library and make a few changes as needed. Wow, was I absolutely shocked to
find out current package management in this go app is just pulling libraries
from master on github. It really made me feel like go is far behind python in
maturity. Pulling all these dependency libs from master definitely does not
feel production ready. Maybe this will help resolve things.

~~~
tomohawk
I was likewise shocked at how good go compared to python in this regard.

Resolve your deps once, at build time, and you're done.

Compare that to dealing with the externalized deps that is deploying a python
app on a machine that might have the correct version of python and all
dependencies, or not. virtualenv? good luck. You end up bringing in docker and
other tools just to deal with it. But what do you do if the machine is locked
down by corporate IT and they don't allow docker or the installation of other
python deps?

~~~
collinvandyck76
> at build time

Unless you're using CGO, in which case you still have to make sure your target
environment has the shared libraries your bindings will make calls to.

~~~
bitwalker
Unless you statically link your dependencies, but that comes with its own can
of worms.

------
dom96
Isn't it a little odd that these are called modules when they sound very much
like packages? In many languages, a module is just a source code file that can
be imported. Why is Go redefining this term?

~~~
masklinn
Go's terminology matches java, and is the exact reverse of your comments: a
package is a namespace, a module is a bundle of 1..n namespaces (packages).

~~~
paulddraper
Java "packages" are namespaces, Java "modules" are modules, and Maven/Ivy
"modules" are packages.

~~~
_old_dude_
Maven modules are a bit of a mess, it's a set of packages but a package can be
split into different Maven modules.

I agree with the GP that go package/module are similar to Java package/module

~~~
snuxoll
Maven modules aren’t necessarily packages; you can have POM only maven
artifacts for things like BOM’s, for example.

Thankfully the whole split-package nonsense will eventually go away, since the
JPMS doesn’t allow a package to reside in multiple modules.

------
rukenshia
Do I have the ability to somehow specify "use git+ssh for this dependency"
with the new modules system?

Right now it seems near impossible with go to do that other than manually
cloning the repositories into the correct path. We can't host our things
publicly and have to use SSH to clone the repositories at my company.

It is especially frustrating in our CI/CD process if we need to manually clone
our packages for setting everything up.

~~~
BillinghamJ

        git config --global url.ssh://git@github.com/.insteadOf https://github.com/

~~~
pritambaral
This is what we use too, but it's still magic (in a different tool) and Go
shouldn't be relying on it.

~~~
BillinghamJ
I think it should actually. Go’s module system should not have to give
consideration to the particular transport you want to use.

Its references are canonical and it’s up to you to set up the relevant process
for it to retrieve the source for those references.

I get the impression that that’s what the proxy stuff is about - you’ll just
set up a proxy which deals with retrieving the code.

~~~
pritambaral
> Go’s module system should not have to give consideration to the particular
> transport you want to use.

> Its references are canonical and it’s up to you to set up the relevant
> process for it to retrieve the source for those references.

Git has a proper notion of references; it properly separates transport from
references of remotes. Go uses just `[https://`](https://`) links, and expects
the website to have a single `<meta>` tag containing a vcs clone URL (i.e.,
transport) to use. VCSes have had separation and multiplicity of transport for
a long time, but Go will deal with only one URL (i.e., transport), with no way
for the user/distributor to specify preferences otherwise.

For example, everything from git.kernel.org to github.com allows the user to
chose which URL to clone with. The remote is _not_ supposed to know or dictate
a single transport.

I think `go get`'s limited method of transport discovery is just a hack that
got released into production and stagnated in its form. It was a perfectly
fine hack for public repos (on the internet or on Google's intranet), but it
just never got any features (let alone documentation) specifically for repos
that need an authenticated way to clone them. The fact that git has a nifty
way to rewrite URLs is just a luck.

~~~
BillinghamJ
Bear in mind, Go doesn't just support git... so HTTP and SSH aren't the only
options.

~~~
pritambaral
> Go doesn't just support git

I didn't claim Go supports only git.

> so HTTP and SSH aren't the only options.

My entire point is that VCSes support multiple transports since before Go came
about. I never claimed Go should support HTTP and SSH only. In fact, I never
claimed it should support either. I claimed it shouldn't force the VCS host to
chose _for the user_ which clone URL (and thus transport) to use.

------
kromem
The v2 library suffix on the package path seems rather hackish.

Feels like it would have been better to use a different delineating character
instead of a slash to make it abundantly clear at a glance that it's
effectively a major version tag and not a subpackage.

Other than that, seems relatively straightforward. Hope this is the last "new
and completely different" attempt to tackle dependency management in Go.

~~~
BillinghamJ
Particularly the fact that it only applies after the second version. If we're
going to require /v2, /v3, etc - shouldn't we also require /v1 for the first?

Also it's unclear how initial development (as per server) is to be handled. If
the first few versions are v0.1.0, v0.2.0, etc. - how are these handled on the
import paths.

What if a package URL actually contains "/v2" at the end, but this isn't
actually referring to the version and should be used when fetching the
package. Seems very odd. Agree that it should have been more like "@2".

What if you want to include two different minor versions, for example, as you
can include two different major ones. Is that possible/allowed?

~~~
cube2222
Requiring v1 would break backwards compatibility. If you create a new library,
feel free to do that. The point is to treat different major versions as
different packages, enough obviously lets you use multiple major versions in
your project.

v0 and v1 are treated as the same package, you're supposed to use v0 as long
as you're adapting to vgo / potentially doing breaking changes and use v1 when
you know you'll keep a stability guarantee.

As I mentioned in the sibling comment, it's all described in Russ's blog post
series, long, but well worth the read:
[https://research.swtch.com/vgo](https://research.swtch.com/vgo)

------
tamalsaha001
Lot of thanks to the author for this post. I have 2 questions:

1\. How does one use a branch of a dependency in go.mod? (scenario: local dev
where api & impl repos are separated)

2\. How does one use branch from a fork in go.mod? (scenario: fixed bug in
someone else's repo and now want to use my fork)

~~~
ainar-g
1\. IIRC you should be able to do

    
    
      go get example.com/repo@branch-1
    

2\. Something like

    
    
      go mod edit -replace example.com/repo@version=example.com/fork@version
    

It seems like the second command doesn't understand branches, so you'll need
to specify a "pseudo-version" like "v0.0.0-20171125154426-754f7301a386". Or
maybe my go1.11 is just out of date.

------
pests
Does vgo miss a nuance in the semantic versioning of 0.x.x branches in that
any bump to minor or patch allows a breaking change? Specifically point 4 of
the spec. Some projects or libraries have a huge psychological barrier to
releasing a 1.0.0. React comes to mind when they finally went from 0.14.x to
15.0.0.

\---

4\. Major version zero (0.y.z) is for initial development. Anything may change
at any time. The public API should not be considered stable.

[https://semver.org/#spec-item-4](https://semver.org/#spec-item-4)

~~~
fpoling
Go developers made a reasonable choice. Unstable API is just another way to
say that there is no API. And with zero API any change is compatible and
cannot break anything. So there is no need for v0.

~~~
cesarb
If there is no API, how can the library be used? Every function a library
exposes is part of the library's API, so for a library to have no API, it
cannot expose any function.

------
e12e
So, if I want to use the new code in v2 for my v1 legacy consumers, would I
let v1 (or a new v1.1) depend on v2, and wrap the new hi-method passing in a
default parameter of "en" for language?

Or is my only "supported" option to push consumers to the new major version -
with the possible bit-rot of the old code?

[ed: that is manual backport fixes/improvements to a v1.x version]

~~~
ainar-g
Russ has explicitly written that yes, redefining v1 in terms of v2 is not only
the way to go, but in the future it might help automatise the switch to v2
with the "go fix" command.

See [https://research.swtch.com/vgo-import](https://research.swtch.com/vgo-
import), section "Automatic API Updates".

------
Jyaif
"Go modules is a first step in potentially eliminating $GOPATH entirely at
some point."

Hooray!

~~~
ainar-g
Ech. I will prefer GOPATH over the hell that are C/C++ include paths any day
of the week. How many are there on a modern Linux system? Six?

Holy hell, it's eight!

    
    
      $ gcc -xc++ -E -v -
      #include <...> search starts here:
       /usr/include/c++/7
       /usr/include/x86_64-linux-gnu/c++/7
       /usr/include/c++/7/backward
       /usr/lib/gcc/x86_64-linux-gnu/7/include
       /usr/local/include
       /usr/lib/gcc/x86_64-linux-gnu/7/include-fixed
       /usr/include/x86_64-linux-gnu
       /usr/include

~~~
masklinn
I'm not sure I see the difference. $GOPATH can be a list, all this tells you
is that C and C++ libraries come from various sources (system, compiler,
package manager, …), you could have the exact same crap in your system-
provided GOPATH if there was such a thing.

~~~
majewsky
Well, you have a system-provided GOPATH, in GOROOT (on my system, the stdlib
source is in /usr/lib/go/src). But you just surprised me: I never knew that
GOPATH is a list.

------
sheeshkebab
Why can’t google just maintain a npm.js style repo for go packages (together
with a package manager program)? And a proper simple to read manifest file for
a go programmer to list out external dependencies with their versions

I really don’t see these git based hacks can improve situation much.

~~~
Yokohiii
Because a centralized repo is a disaster.

~~~
ofrzeta
Why? It has worked out quite well for pretty much every other ecosystem like
CPAN, PyPI, Maven etc. don't you think?

~~~
Yokohiii
It worked out for the time being, but npm destroyed the illusion. I don't even
see the problem. A package manager could rely completely on git with the
advantage to not have an single authority that isn't giving a flying fuck
about quality and security of the hosted packages. Yes it forces you to dig
out a good package, evaluate it and look at the track record of the devs.
Someone has to do it, but it is currently hardly happening.

------
BillinghamJ
Is there a recommended process or guide on the best way to switch from dep to
modules?

~~~
cube2222
vgo mod can read your lock or vendor file (it supports most of the go
vendoring solutions) when initting.

~~~
robteix
Happens automatically when you run 'go mod init' in a directory containing
Gopkg.lock. Caught me by surprise the first time as I had no idea it did that
:)

------
kalekold
Go modules are a step in the right direction but still doesn't protect you for
when a developer deletes their repository!

