
Experimental Dependency Vendoring in Go 1.5 - NateDad
https://groups.google.com/forum/#!msg/golang-dev/74zjMON9glU/4lWCRDCRZg0J
======
NateDad
I really like this proposal. It's basically the best of all current dependency
tools - you get reproducibility without needing external configuration files,
without hugely long vendor import statements, and still supporting raw go-get.

------
AYBABTME
This is pretty much what we've been doing at DO, but we were dealing with
GOPATH manipulation to achieve it. By having `GOPATH="third_party/:docode/"`,
`go get` pulls packages into `third_party` automatically, and `go install`
finds our binaries in `docode/src/services/foosvc/cmd/foosvcd`.

Migrating to use `vendor/` instead of `third_party/` will be trivial, and not
having to modify GOPATH will be nice.

Having a GO_VENDOR=1 env var on top of the `go get -vendor` feature flag would
be nice, so that the feature works with minimal disruption.

~~~
technoweenie
How do you handle a custom GOPATH like that when the current repository needs
to be in there too? We've tried manually symlinking the repo dir into the
custom '$GOPATH/src/github.com/user/repo' dir, but it feels really hacky.

~~~
anezvigin
You can separate multiple paths with a colon.

~~~
bradfitz
More specifically, with os.PathListSeparator. (colon on Unix, semicolon on
Windows and \x00 on Plan 9)

------
lobster_johnson
Am I correct in thinking that this proposal only lays the foundation for other
tools to actually _manage_ the vendor folder?

In other words, you would still want something like godep or gb, because
manually copying dependencies into a folder (or using git submodules, ugh),
and then tracking their versions through their lifecycle, isn't a workable
solution in most situations.

This is clearly an improvement over "go get", but I don't understand why Go is
so insistent that dependency management is something for the community to
solve, and not something that should be provided by Go itself.

Go is so opinionated about so many things, and yet so little interested in
having a strong opinion here. The Unix philosophy is great, to be sure, but
the Unix philosophy is about increasing modularity through simplicity, and
this problem is _about_ modularity.

~~~
rsc
Is the criticism here really that we have declined to express a strong opinion
about something we don't know a lot about?

~~~
ngrilly
A quick comment to tell you that I really really appreciate how you solve
problems step-by-step, without rushing. The improvement you proposed for
supporting vendoring in Go 1.5 is great.

------
vruiz
This is nice. Finally it seems like we are going somewhere. The call for
proposals had not yielded anything groundbreaking so far. Tools like gb are
nice but as Russ says it's not gogettable. This seems like a good compromise.

------
drewolson
Am I understanding correctly that this proposal again side steps / ignores
pinning dependencies for libraries? To me, this is the biggest current problem
in the ecosystem and, as a library author, this doesn't solve any of my
issues. I hope I misread somehow.

~~~
enneff
With this scheme libraries can vendor their dependencies, if necessary.

~~~
drewolson
Can you reference the part of the proposal that states this? I'm reading the
following points:

> $GOPATH/src/foo will be imported as foo by non-main package
> $GOPATH/src/mypkg/p.

> $GOPATH/src/mypkg/external/foo will be imported as foo by a non-main package
> p anywhere, when p is being built as a dependency of command
> $GOPATH/src/mypkg/c.

It seems to me this is saying that libraries will use external packages only
when built as a dependency of a main package. Again, this means library
authors are out of luck.

~~~
jooon
There are two proposals.

You seem to refer to the Google doc proposal that was linked to in the first
message of the mailing list thread several months ago.

The link on hackernews points to an answer by Google that was written just a
few hours ago that instead "propose" the following:

> If there is a source directory d/vendor, then, when compiling a source file
> within the subtree rooted at d, import "p" is interpreted as import
> "d/vendor/p" if that exists.

~~~
drewolson
Ah, thanks.

------
Lethalman
How we solved it in Nix: [http://lethalman.blogspot.it/2015/02/developing-in-
golang-wi...](http://lethalman.blogspot.it/2015/02/developing-in-golang-with-
nix-package.html)

We don't need go get or any other tool, and we have about 174 packaged
libraries in our repository.

------
ngrilly
This is a great and necessary improvement. Being able to vendor dependencies
without having to rewrite import paths is like having the best of both worlds.

The proposed mechanism looks similar to what Node.js does with the
node_modules directories:

[https://nodejs.org/api/modules.html#modules_loading_from_nod...](https://nodejs.org/api/modules.html#modules_loading_from_node_modules_folders)

------
wumbernang
That's really nice.

This is a royal PITA with .net and nuget and I've get to find a half decent
solution that allows you to actually have the source rather than a bunch of
symbol-less dll dependencies or a ridiculous build stack.

~~~
romanovcode
Since MVC6/VS2015 beta you can actually get the source instead of dlls.
However your build times will increase dramatically so it's a bad idea unless
very needed.

~~~
wumbernang
Interesting - thanks for pointing it out. I was completely unaware of that.
Going to go and play with it on my win10 test box now :)

------
0xdeadbeefbabe
A vendors B and C, but C vendors D and E. According to this proposal do you
lay out the files like this: (A (vendor (B C D E)))?

Edit: I guess HN tree markup is not working :)

~~~
jooon
Yes. That would work. However, I also believe this will work: (A (vendor (B (C
(vendor (D E)))))).

[https://groups.google.com/d/msg/golang-
dev/74zjMON9glU/LjrVU...](https://groups.google.com/d/msg/golang-
dev/74zjMON9glU/LjrVU8dMKbcJ)

[https://groups.google.com/d/msg/golang-
dev/74zjMON9glU/0e7M7...](https://groups.google.com/d/msg/golang-
dev/74zjMON9glU/0e7M7FHipusJ)

[https://groups.google.com/d/msg/golang-
dev/74zjMON9glU/qGzsS...](https://groups.google.com/d/msg/golang-
dev/74zjMON9glU/qGzsS8RfjnEJ)

~~~
jooon
and if you have both: (A (vendor (B (C (vendor (D E))) D E)))

C would pick D E in its C/vendor before D E in A/vendor before D E in
$GOPATH/src

ps. I have edited these comments about 5 times to get the parentheses right.
:)

