
The vgo proposal is accepted. Now what? - stablemap
https://research.swtch.com/vgo-accepted
======
eberkund
I wish they had just copied Rust/Cargo. I remember reading a comment on GitHub
somewhere from one of the Go maintainers who responded to someone expressing a
similar sentiment and his reply was basically that Go is somehow different
than every other language and they need to explore and find a unique custom
solution for their particular use case. Has it ever been addressed anywhere
why the tried and true "list of packages" \+ "lock file" paradigm is not good
enough for Go?

~~~
amasad
There are a lot of suboptimal design decisions that's shared across the state
of the art in package management (cargo, pipenv, npm, yarn, etc) that the Go
team is not taking for granted.

One is reproducible builds. The standard answer is lock files which is extra
bloat and leads to merge conflicts. vgo is trying to avoid it with the
"minimal versions" strategy ([https://research.swtch.com/vgo-
repro](https://research.swtch.com/vgo-repro)).

Another is shared (or recursive) dependencies. The semver answer to this is
that if the versions match then they should be shared if not then they should
be duplicated. But what do you do if it's a singleton, listens on a port, or
exposes a port. Right now, with npm for example, good luck with that, you're
in for a world of pain. On the other hand with vgo's "semantic import
versioning" they're trying to make version interop more explicit.

Just because 90% of the time mainstream package mangers work doesn't mean it's
a solved problem. Kudos for the Go team for trying to advance the state of the
art.

~~~
kibwen
_> The standard answer is lock files which is extra bloat and leads to merge
conflicts._

Bloat how? A lockfile is a handful of bytes. A thousand lockfiles could fit in
the space of a single Go hello world binary. And I've never heard of a merge
conflict from a lockfile. What are the actual arguments against lockfiles?

~~~
blaisio
It's pretty easy to get a merge conflict from a lock file. If two developers
add a dependency and then try to merge their changes together, it can happen
easily.

~~~
kibwen
In that scenario, you're going to get a merge conflict anyway in your manifest
(e.g. in go.mod).

~~~
grahamedgecombe
It's probably easier to manually merge the manifest (as they're much more
human-readable) than a lock file though.

~~~
steveklabnik
Lock files are easy enough to resolve that npm, in newer versions, will do it
for you automatically, if you ask it to.

------
adwhit
For those who missed it, Sam Boyer (maintainer of dep) wrote a detailed post
about why he thinks vgo (or rather Minimum Version Selection) is
inadequate[0].

The key argument is

 _With dep, it’s usually easy to point to failures - they’re explicit, verbose
(and, currently, often difficult to understand, and printed out at the end of
a dep ensure run.

The primary failure mode in vgo, however, is silent false positives - a vgo
{get,test,run,build} command changes your dependency graph, and exits 0. Maybe
everything’s fine, maybe it isn’t, but it’s incumbent upon you to take
additional steps to understand that your build is broken._

I haven't been following this debate very closely, and this post doesn't make
it clear - have these points been addressed? Is there still room to maneuver,
or is the design mostly settled now? I know the post states that any design
flaws will be fixed, but it sounds very much like the more typical lock-file +
solver solution has been definitively decided against.

[0] [https://sdboyer.io/vgo/failure-modes/](https://sdboyer.io/vgo/failure-
modes/)

~~~
tmpz22
I just hope they figure out a solution which satisfies the community for the
long run. Maybe they have already with vgo, but this is starting to feel vary
Javascript-y the way people have been pushed from $OLD_PACKAGE_MANAGER -> dep
-> vgo.

~~~
ljm
Quite the contrary: NPM's reputation in these circles is less than stellar and
it's been incumbent in the JS ecosystem ever since Node became a thing. Well,
with a few attempts at competition along the way before those working with the
browser jumped on board, but nobody is pushing anybody to use anything but NPM
and that attitude hasn't changed for years.

This story isn't unique to Go: both Python and Ruby have had their own ordeals
with dependency management, there was an article about CMake just the other
day...

Maybe Go's mistake is searching for the one-true dependency resolution system
and deprecating everything along the way until something good-enough turns up.
Maybe it'd be better that developers are encouraged to use dep while vgo is
still in proposal stage so that there is an easy migration path from one
standard to another, as opposed to immediately rendering obsolete.

I don't really know, neither do I know why Javascript is the scapegoat for
this kind of shit, it's practically a rite of passage for a language gaining
increasing mind-share.

~~~
weberc2
> Maybe Go's mistake is searching for the one-true dependency resolution
> system and deprecating everything along the way until something good-enough
> turns up. Maybe it'd be better that developers are encouraged to use dep
> while vgo is still in proposal stage so that there is an easy migration path
> from one standard to another, as opposed to immediately rendering obsolete.

The odd thing is I get the feeling that their search for the one-true system
is causing them to repeat every other package manager's mistakes. I'm likely
misinformed, but I get the feeling that they think they can do the same thing
other package managers have considered or tried, but it will work for them
somehow. I don't get the feeling that they seriously considered the criticism
of VGo's approach. It smells like hubris. Also, there's Cargo, which is lauded
by all, but the proposal doesn't seem to consider that perhaps the Cargo folks
had a reason to make their dependency resolution scheme complicated. Again,
this smacks of hubris.

I'm happy to be persuaded otherwise, and it would really only take a link to a
thread in which some VGo proponent thoughtfully addresses Sam's criticism and
the "why not copy Cargo?" criticism (and no, the "Cargo's dep resolution
scheme is overly complicated" rationale from the proposal doesn't constitute).

~~~
ljm
Feels like Go is trying to carve its own path, for better or worse, against
conventional wisdom. Plan9 was pretty much the same in a lot of ways and it
came up with some really interesting concepts (that sadly haven't panned out
that well - I'd love to see a modern OS attempt at Acme, I found that
incredibly intriguing and I actually wonder if it could be revived in VR.)

If they come up with something innovative with vgo then great. It's just a
shame that the community is opting for a monopoly on the system before it even
physically exists. They should be encouraging dep to thrive while doing this
experimentation on vgo behind the scenes, because then at least they've got
community alignment on dep and not dep, glide, godep, `git clone some-repo
vendor/some-repo`, `go get` and whatever else you can do to pull in external
code.

~~~
weberc2
> If they come up with something innovative with vgo then great.

For sure, but I actually like Go and I'm unnerved that none of the concerns
raised by folks like Sam (read "folks with experience in package management")
are being addressed. This approach doesn't inspire confidence in vgo.

------
throwaway243425
To an outsider this may sound like there is some sort of process that resulted
in this solution, there actually isn't any.

The committee is nothing more than a simple bureaucracy to the point that it
is almost a joke how Russ makes a proposal, community is against it, then it
gets accepted by the Committee.

It is all just a funny joke.

~~~
jimmy1
> To an outsider

> Russ

> throwaway

Something tells me you aren't really an outsider.

Anyways,

> community is against it, then it gets accepted by the Committee

Is this your primary gripe? Because this has happened pretty consistently in
the history of almost all open source languages. Think of how many JSRs were
hotly contested, only to be accepted. Open source does not mean democratic
development, and thankfully so because you might have ended up with this
[https://i.redd.it/7t1p88ct13ez.jpg](https://i.redd.it/7t1p88ct13ez.jpg)

~~~
dikaiosune
FWIW, I read "To an outsider..." to mean "if you are reading this on HN and
aren't involved in the community, this might seem like" as opposed to "I am
pretending to be an outsider, let me tell you what about my perspective."

------
dstroot
I first used $GOPATH and “go get”. Was amazed at how simple and easy it was
and it “just worked”. Then you start to run into issues... so I started using
Glide. Which worked pretty darn well. Then I switched to Dep because “it was
the future” and it didn’t cause me to break out in hives. Now vgo... but Dep
works for me so at the moment I don’t plan to switch until vgo is good and
baked. Given Boyer’s concerns I hope he maintains Dep for a while as vgo is
polished. Too bad this was not part of the vision at the start. Not sure where
JS would be without NPM, etc.

~~~
kodablah
> Not sure where JS would be without NPM

Where it was before NPM, i.e. where Go is today. No real versioning or
discovery (granted Go has qualified URLs for discovery).

------
gkya
I still wonder why solving a solved problem took so long to solve for the Go
community, given they already solved it anyways?

That is, in more proper words, first of all, language specific package
management is mostly a solved problem. There are possible improvements, and
maybe vgo realises some of them, but that's mostly a bikeshedding problem.
What users need is to be able to declare what packages they need, in what
version range. And their search for an alternative to fetching source repos is
like searching for the cure to ilnesses that already have proven vaccines: you
just put up a server and fetch from there. Decentralisation? Put up mirrors.

Then the way this vgo thing happened is the opposite of nice. Tools already
existed, and they had to conform to the restrictions of the project (like the,
excuse me but, idiotic idea of a $GOPATH); but then one of the Go deities come
around and goes, um, I deprecate all of you, break the rules that you had to
comply, and because I-am-who-I-am, this is the way to go.

Now Cox's solution might indeed be better (though I think it's an overkill,
and do agree to Boyer's articles I read), but this is not the way to run a
community. From my PoW, this would not preclude me from using the language if
it came up, but I'd definitely be reluctant to send patches to them.
Communities with deities and dogmas are always unhealthy. Those that also,
additionally, are deep down in yak shaving and bikeshedding are even more so.

~~~
lobster_johnson
> Now Cox's solution might indeed be better (though I think it's an overkill
> ...

vgo is actually much, much _simpler_ than dep. The sheer number of words in
Russ Cox's series of blog posts belies its simplicity. vgo doesn't need a SAT
solver. If you look at many of the issues dep is struggling with, they're
related to solving N libraries with transitive dependencies up the wazoo.

Cox's long treatise reflects the complexity of the problem space. Developers
tend to brush off package management as being simple. But once you include
range-based version constraints and transitive dependencies, it gets a bit
messier. Look at NPM and Yarn; they're _still_ struggling to get all the
details right. On the other hand, there's Ruby's Bundler. It came out in 2009,
RubyGems in 2004, and I've never had a single issue with the toolchain (other
than messing up my own constraints). I don't know what kind of magic elixir
they were drinking, but somehow those guys managed to nail it from day one.

~~~
Lazare
> Look at NPM and Yarn; they're still struggling to get all the details right.

I think that's a bit unfair. NPM has been a horrible package manager in a
multitude of ways since day 1. My default assumption if it gets something
wrong it isn't because it's hard, but because npm gets a _lot_ of things
wrong.

Yes, RubyGems got it right, but so did Composer. And Cargo. And every other
language specific dependency manager I've used in over a decade. The lesson
I'm drawing isn't that dependency management is uniquely hard, it's that npm
is uniquely bad. :)

~~~
gkya
Comparison to NPM is also unfair because of the rather unique community around
it.

------
ospider
Besides the technical details between vgo and dep. The go team just announced
and accepted its own proposal, completely ignoring the community's solutions,
dep was even called the official experiment, which is sad.

------
jrs95
> Now what?

I keep using dep for as long as it's reasonable for me to do so, because I
don't like MVS and I don't like how MVS has been basically forced upon us.

~~~
wuliwong
What is MVS?

~~~
mali9
Minimal Version Selection

[https://research.swtch.com/vgo-mvs](https://research.swtch.com/vgo-mvs)

------
rahenri
There is a lot of criticism on the proposal with a lot of good reason. But,
I'm honestly very excited about the deprecation of GOPATH, that was one of the
most annoying features of go for me.

------
matte_black
What does this mean for someone who is starting out with Go?

~~~
eosrei
Nothing. Use dep now. In fact, since you are just starting out, see how far
you can get using only the standard library. Later, there will be a seamless
upgrade to vgo.

~~~
marcus_holmes
Seconded. Unlike Node/Ruby/most other modern languages, Go devs actively avoid
including dependencies if at all possible.

And contrary to rumour, this is not because Go's dependency management sucks.
It's more about the pursuit of simplicity, and avoidance of magic.

You can write pretty much anything using just the standard library (and some
of the official packages, like the crypto ones). As the parent said, it's good
practice to go as far as possible with just the stdlib.

~~~
oceanswave
So go is about recreating the wheel, over and over again

~~~
sethammons
Not really. Generally, the wheel you need and the wheel I need are a bit
different. Instead of using some bloated "all-wheel," developers choose their
custom wheel.

Simple example, SyncMap. This is a general purpose all-wheel, and as such, due
to Go's type system, you have to use runtime type assertions against it. If I
need a lockable map, I just make one and it is of the type I need, say
map[string]*Foo. I just wrap it in a struct with a lock and I'm done.

For more complicated things, most everyone will pull in a package. I'm not
going to waste some time making a Redis or Kafka package. For a web server, I
may pull in a different muxer, but only if needed.

~~~
matte_black
What is the Go alternative to generics?

~~~
marcus_holmes
specifics?

Joking aside, that's pretty much it. The usual method for go devs is to solve
the specific problem in front of you (and accepting a certain amount of
duplication) rather than creating generic solutions that have a wider scope
than they need. Once it's all working, refactoring can often remove the
duplication and provide a better solution than the generic one would have
provided (because by then you know the problem domain better).

I've been coding in Go for a few years, and only a couple of times run into
the "shit, I need generics here" problem.

And yes, I get that this means that the Go answer to generics is "you don't
need generics" ;) Which sounds like such bullshit of course.

------
Avi-D-coder
How does Haskell's cabal compare to vgo?

~~~
danharaj
Cabal uses a SAT solver. It's in the middle of a beta for replacing it's
global package management with per-project package management by default (so-
called new-style builds). There were some warts in the past when it couldn't
manage different versions of the same package, I think because of a ghc issue
but that is in the past.

Most industrial users use stack or nix on top which reduce the use of Cabal's
version solver.

------
anonfunction
I really do not like the “semver-like” versioning string requirement. My
packages are already tagged with valid semver releases and now I need to
change them by adding a “v” prefix.

If you don’t know what I’m talking about vgo requires the release to be tagged
like “v1.0.3” which is not standard semver.

~~~
cesarb
> vgo requires the release to be tagged like “v1.0.3” which is not standard
> semver

Are you talking about git tags? Tagging a release with a "v" followed by the
version number has been done since the very first git repository.

I don't think semver has any official standard; the closest I can find is
[https://semver.org/spec/v1.0.0.html](https://semver.org/spec/v1.0.0.html),
which does say "When tagging releases in a version control system, the tag for
a version MUST be “vX.Y.Z” e.g. “v3.1.0”."

~~~
anonfunction
Yes I am talking about git tags.

I purposely followed semver 2.0.0 which doesn't mention anything about a "v"
prefix. Thanks for finding an old mention of version control tagging!

At the end of the day it's not a big deal. I will just duplicate the tags so
it won't break any dep configurations.

------
markrages
This article is about the Go programming language (and environment).

------
gyrgtyn
As someone who just recently started using go, this seems really dumb.

~~~
reificator
The churn in dependency management is one of the worst issues with go. The
other is not having a canonical GUI option, instead mostly just bindings to
other toolkits with varying completeness and quality.

That said, I think it's worth sticking with go for the quality of the language
itself. If you don't need a GUI _(or plan to use a web GUI)_ it's a really
nice balance of design decisions that let you actually get things done.

