
Pkg.go.dev is more concerned with Google's interests than good engineering - trulyrandom
https://drewdevault.com/2020/08/01/pkg-go-dev-sucks.html
======
amscanne
> Go is a pretty good programming language. I have long held that this is not
> attributable to Google’s stewardship, but rather to a small number of
> language designers and a clear line of influences which is drawn entirely
> from outside of Google — mostly from Bell Labs. pkg.go.dev provides renewed
> support for my argument: it has all the hallmarks of Google crapware and
> none of the deliberate, good engineering work that went into Go’s design.

Putting aside the criticisms, I think a new word is needed to describe anti-
Google sentiment of this flavor. The idea that Google is incapable of doing
anything positive, so anything “good” coming out of Google must be
attributable to influences outside of Google.

I’d like to think about it as a “no true Googler” argument. The Go team are
not true Googlers, because they produce something useful. When they built that
module proxy however, they were clearly so doing as Googlers. (As someone who
works at Google, I can humorously imagine that when I’m acting as a true
Googler, my only goal is to foist new crapware onto the world, get promoted,
and kill it off.)

Or maybe we recognize that companies are collections of individuals, and all
individuals bring their own history and influences. The culture shapes them
and they shape it back. Playing semantic games to divide and attribute isn’t
useful.

~~~
ddevault
This is not a "no true Googler" argument, but a factual statement supported by
the history of Go. The design of Go is _heavily_ inspired by Plan 9, Inferno,
and Limbo, none of which came from within Google. In fact - name any of the
flagship features of Go and I'll draw a line for you from that feature into
the historical influences. Goroutines, channels, garbage collection, GOPATH...
none of these were invented by Go.

~~~
amscanne
Right, but Go itself, which _builds_ on those ideas was built within Google. I
have no objection to attributing ideas, but you seem to want to attribute some
general “quality of engineering” attributes to “not Google”.

The people doing things you like are just as much Googlers as all the people
doing things you don’t like, and they all have an influence on the company and
vice versa. In many cases, it’s even the same people.

So the distinctions you are trying to draw are silly. Just let the criticisms
stand on their own.

~~~
ocdtrekkie
FWIW, Drew's position on Go is hardly even unique. Someone who is quite fond
of Go was telling me last week about how I should consider learning it despite
it being from Google, because it is not very Google-like of a project. And
that wasn't the first time I'd heard it either.

(Myself, I am usually just trying to build stuff people wrote in Go, and I
usually end up fighting the tooling for a while and getting mystified by the
error messages and having no idea where to look to solve them.)

~~~
arp242
I avoided Go for years because it was from Google until I quite accidentally
landed a Go job several years ago, and I haven't looked back. Amusingly, I am
now using Go to build a competitor for Google Analytics. Heh.

It was an important lesson in a way, as it taught me that Google – or indeed,
any organisation comprised of thousands of people – is not a single monolithic
entity.

Also, feel free to drop me a line if you've got problems with a specific
error, and I can perhaps point you in the right direction.

~~~
ocdtrekkie
I did get the thing last week to build! :) My Go-recommending friend seemed to
think the script I was using was pulling in Go 1.11 where modules were not as
well supported as they are now? But I got around it.

It was just that the error message I got really gave me nothing useful to
search for. Literally was a "permission denied" error... while "go build"ing
stuff with effective superuser. It wasn't really pointing me at what it didn't
have permission to do or how I would go about getting it.

~~~
arp242
"Permission denied" usually happens when it tries to build stuff in "GOROOT",
that is, the standard library installed in /usr/lib/go or some such. This
normally shouldn't happen, so there's probably something wrong with your
installation or setup. Hard to be 100% sure though, that's just the usual
cause in my experience.

I think Debian ships with Go 1.11 in current stable (and by extension, Ubuntu
too, I assume?). The entire GOPATH to modules switch is messy, ugly, broke a
lot of stuff, and is confusing for newcomers, even though modules is clearly
better. I think we're finally nearing the end of it though.

You probably want to just use Go 1.14 (the latest version) if you can. The
language itself is very stable (just not the surrounding tooling).

~~~
ocdtrekkie
I can definitely say it's the tooling that throws me every time. And I do
think we pull from Debian for that script currently, though we're going to
update it soon. (I think I was told 1.15 was near anyways?)

~~~
arp242
Overall I find that the tooling is actually really nice, and better than
anything I've worked with previously. It's just this weird "old method vs. new
method" schism that makes it hard :-/ It's restricted to just tooling, but
kind of Go's Python 3 moment.

And yeah, I believe Go 1.15 should be released in August. rc1 was released
last week or so.

------
dingdingdang
These two points alone are honestly just bad news for Golang on the whole

>> pkg.go.dev fetches the list of modules from proxy.golang.org: a closed-
source proxy through which all of your go module fetches are being routed and
tracked

>> pkg.go.dev hard-codes a list of centralized git hosting services [that are
the only allowed ones]

~~~
LukeShu
_> > pkg.go.dev hard-codes a list of centralized git hosting services [that
are the only allowed ones]_

So how in the world did git.lukeshu.com get to be on that list?

[https://pkg.go.dev/git.lukeshu.com/go/libfastimport](https://pkg.go.dev/git.lukeshu.com/go/libfastimport)

~~~
ddevault
Note that some features are absent here. You can't click on any of the
declarations to view their source, for example. This is the degraded view for
unknown hosts.

~~~
throwayws
That does not work in the old godoc.org either:

[https://godoc.org/git.lukeshu.com/go/libfastimport](https://godoc.org/git.lukeshu.com/go/libfastimport)

So that's not against pkg.dev. Did somebody look into why?

Anyway it seems that the repo was not added to pkg.dev automatically:

[https://github.com/golang/go/issues/38326](https://github.com/golang/go/issues/38326)

and the automatic fetching should work now:

[https://github.com/golang/go/issues/37002](https://github.com/golang/go/issues/37002)

~~~
ddevault
See my comment here which explains how "go-source" works:

[https://news.ycombinator.com/item?id=24024130](https://news.ycombinator.com/item?id=24024130)

It doesn't work on godoc.org not because they don't recognize your domain, but
because cgit doesn't have go-source meta tags.

~~~
throwayws
There was a proposal to add a new go-source-2 meta tag, but that was abandoned
because a lot of code hosting sites would not have been supported yet:

[https://github.com/golang/go/issues/39559](https://github.com/golang/go/issues/39559)

[https://github.com/golang/go/issues/40477](https://github.com/golang/go/issues/40477)

That is why they decided to turn it into an internal/source package inside
pkgsite first to have test cases for a later go-source-2. If your site does
not show source links on pkg.go.dev then add a comment to issue 40477 above
and an exception will be added.

\- internal/source:
[https://github.com/golang/pkgsite/blob/master/internal/sourc...](https://github.com/golang/pkgsite/blob/master/internal/source/source.go)

Note1: Maybe mention this on your website because that took a little to find.
It does not fit into the narrative though.

Note2: Requiring frontends to offer meta tags is the only way to make code
discoverable with the url alone. cgit not offering this is on cgit and not on
Go/pkgsite (haven't checked whether they do)

------
warent
These are fair criticisms. Still though, there are some pretty sweeping
generalizations about Google being made here. The company is as big as a city.
Not all of their engineering culture is like this.

Take what I'm saying with a grain of salt; slightly biased since I worked
there for a bit in the past.

~~~
finnthehuman
>Not all of their engineering culture is like this.

Sure. I'll grant you Junio; he seems like a cool guy.

The rest of google though? Google's engineering culture is by and for google.
They're not doing the rest of us any favors.

Googlers need to admit that. Not to me, but to themselves.

~~~
IfOnlyYouKnew
I'm seriously astonished by the depth of these Freudian analyses of the dark
underbelly that is Google's "engineering culture", and how everything bad with
the world can be linked to ...

... a meaningless decision to have a transparent proxy in some network path
some free code takes on its way to you?

~~~
finnthehuman
>a meaningless decision

If it's so meaningless, now that Drew has complained they can just make the
alterative the default for their software right? Right? It is MEANINGLESS
afterall.

When faced with a decision, someone chose the option that took extra
operational expenses to keep services online. If you're going to call that a
meaningless, then you've either a huckster or have got quite a lot to learn my
friend.

~~~
IfOnlyYouKnew
Exactly! It's so meaningless they shrugged and flipped a switch when asked to
do so in a single blog post. Try that feat with any policy decision google
actually does care about!

------
lenkite
One of the reasons that despite Go's several advantages, I still prefer an
open language like C++ for native programming. Go is Google's language and
people should _never_ forget it.

------
thayne
Probably another instance of someone making a new shiny to get promoted, after
which it will be abondoned.

------
arghblarg
Remember, Go is MIT-licensed. If things become too bound to Google, it can and
will be forked. People thought it was the end of the world when multiple Java
SDKs arose didn't they? (Not to say that didn't hurt Java, but the language
survived).

~~~
marcus_holmes
Is anyone forking it to remove the Google taint?

I'm aware of TinyGo, but not any other forks

~~~
arp242
What Google taint is there in the Go compiler/command?

------
jrockway
There is some complexity here. Go is one of the few programming languages with
a fully-decentralized module system. I can upload a module to example.com, and
you can use a module from example.com with no centralized coordination. This
is very different from systems like Node/npm or Perl/CPAN, where if you want
to write a module, you have to beg them for permission first. (This is a
moneymaking opportunity, too. You can charge people for your package system if
they want to keep their code private. And npm does! With Go, you can have
private modules for free!)

There end up being some problems with this model. Popular modules might be
hosted on some server that can't stand the load, and if you can't pull
dependencies, the entire module system breaks down. So, you probably actually
want the modules on a CDN, because high availability is important for CI
systems. Before Go automatically pulled from Google's module proxy, I had my
own. I had to, because Github rate-limited our pulls (and when it wasn't rate-
limiting us, it was slow). Setting that up was a few hours I had to spend for
no good reason -- it didn't make my product better, except by making some
incidental step between idea and deployment slightly faster. You can of course
vendor your modules (now your git clone is slow), or add caching to your
builds (go's caching is very good, but doesn't play well with Docker). All of
them take your time and mental energy to do something unimportant. So, Google
made a module proxy that Go pulls from by default. It works every time. It
probably costs them almost nothing. There is no business value in it. Google
doesn't need Go for anything. They just have some employees that want to
improve the lives of average programmers. You can turn it off. You can host
your own -- go/x has an example proxy, and there are full-fledged open source
projects that add more features (Athens). It's actually a very good situation.
You can be as centralized or as decentralized as you desire.

Going back to centralized package databases, they do have some really useful
features. The main one I use is looking at users of a package that I'm reading
documentation for. No example code in the repository? No problem. You can go
see how other people are using it. The problem there is getting a list of all
possible packages. You can poll common repository hosts for them. You can
convince everyone to use a certain proxy that gets the names of the modules
you are using so you can go index them. That's what Google does, and the user
experience is great. You trade Google the information "the IP address 1.2.3.4
uses github.com/foo/bar" to get the knowledge of every Go module in the world.
Seems like a good trade. And you can easily fork go and tell it to use your
proxy, and make your own package database. The code is open. The license
allows it. Go for it!

Would it make more sense for Go to be owned by something like the CNCF instead
of Google? Sure. But they did make it, so it's kind of their choice. It is
pointless for a large company to make a brand new programming language, but
they did it anyway, and it's nice that we have it. They give us lots of free
stuff and ask for nothing in return. The fact that your CI system pulled a Go
module is not something that advertisers are salivating over. If you want to
hurt Google because you don't like them, ask your lawmakers to make
advertising illegal. That would show them! But caching Go modules is not a big
moneymaker for Google, it's just kind of a nice thing they do because a few
employees thought it would be good. I appreciate it. If you don't, set up
Athens and forget about them. I am sure you can render godoc just as nicely as
pkg.go.dev with only a few hours of work. The thing is, it's boring and nobody
would care.

~~~
abraham
You can install node packages from any git host or tarball url.

[https://docs.npmjs.com/cli/install](https://docs.npmjs.com/cli/install)

You can also use any registry you want.

[https://docs.npmjs.com/using-npm/registry.html](https://docs.npmjs.com/using-
npm/registry.html)

~~~
jrockway
True! This does exist, however:
[https://www.npmjs.com/products](https://www.npmjs.com/products)

~~~
Xylakant
So they do offer private package hosting as a business like GitHub offers
private git repos side by side with public plans? I’m not seeing what you’re
trying to point out. They’ve built something and they’re charging money for
it. You’re welcome to use any other private npm registry.

------
dpifke
Is anyone aware of any third-party project which attempts to audit
sum.golang.org (and/or sum.golang.google.cn)?

Its design is similar to that of certificate transparency, in that changes or
deletions (e.g. DMCA takedowns or FISA warrants) are supposed to be
detectable; see:

[https://go.googlesource.com/proposal/+/master/design/25530-s...](https://go.googlesource.com/proposal/+/master/design/25530-sumdb.md)

I'm curious if anyone has actually tried to audit it.

------
htmlproplus
A little off-topic. I like Go but haven't used it in a while.

While offline I remember a couple go command operations were surprisingly
giving me proxy.golang.org or network unavailable errors. In the sense that it
had no reason to touch the network. Can you refresh my memory what they were?
Or if they are still present? I can't for the life of me find or remember the
answer.

It was what made me discover some of the things the article mentions, like the
package index.

~~~
ridv
If you download a package that supports go mod and try to build it, it'll
automatically try to fetch dependencies through proxy.golang.org

You can override the proxy, and there are open source implementations of the
module datastore[1], but it's not made clear front and centre to the end user
that building software will call a Google owned service.

It was a source of contention for some when the default toolchain moved in
this direction.[2]

[1] [https://github.com/gomods/athens](https://github.com/gomods/athens)

[2]
[https://news.ycombinator.com/item?id=20870264](https://news.ycombinator.com/item?id=20870264)

~~~
marcus_holmes
also, you can choose not to use the module system entirely, and just `go get`
packages the old-skool way.

------
kgraves
Not surprised at all, what did you expect? it's Google after all.

If it were me (or my company), I would refuse to use a programming language
created by a surveillance capitalist like Google.

What's next? telemetry in the Go language?

------
throwayws
I'm thankful they open sourced pkg.dev and that they continue to bring it on
par with godoc.org. Keep up the good work!

------
hermanradtke
I understand the complaints, but Go is Google’s language just like Ruby on
Rails is Basecamp’s framework. This is not a secret! Also, I think this is
fine? If your interests align with the owners, then you can really benefit. If
not, then you are better off choosing a different solution.

------
zelphirkalt
This article names, what I have had as a feeling for a long time: "Oh it
created by Google … ah this will suck." Take whatever you want. GMail,
YouTube, Google Search, Google's online office products (they are sooo soo
basic, so that even the most basic used can handle it, with no way to unlock
great functionality), Google Hangout, you name it. It all sucks. You are the
product, a used and not a user.

Mostly the reason is probably, that this kind of software is only written with
the most basic and common used in mind, not with the one, who wants to
customize everything to their own personalized needs or a user, who does not
require big corp to tell them "what they want". Google targets another group
of people.

So I think the wording "crapware" pretty much covers it.

Of course ethical questions like the ones raised in the article … I guess they
don't usually even enter the equation at Google.

