
Idiomatic Generics in Go - bouk
http://bouk.co/blog/idiomatic-generics-in-go/
======
discreteevent
If a language is not dynamically typed then it's cumbersome not to have
generics, that's for sure.

On the other hand what I really like about go is that they are the only
popular recent language that pushes back against a lot of modern features and
says: "It doesn't matter how clever those features are, overall simplicity and
consistency matters more."

The point is that we really need this experiment. It's in everybody's interest
that the go people push back and continue to keep the language minimal. In a
few years time someone will say on the internet that unless you have features
x, y, and z in a language you simply cannot be
productive/concise/maintainable/performant/safe/not-a-blub. At that point it
may be possible to point at some evidence and say "Well I noticed that those
machine learning guys at myAppsTheGreatest.com wrote a large system in go".
Then you can go study it and see if they really needed those features. You can
also ask the developers who used it and hopefully some of them will also have
experience in [insert crystalline functional language here] so you can get
them to weigh up the pros and cons. Go is an experiment that is needed.

~~~
perturbation
It's interesting that the pendulum has swung the other way in favoring
readability over features. In many ways, Go seems to have something Pascalian
lurking under its surface - WOCA with static compilation (and fast compile
times), readability favored over terseness/expressiveness, and deliberately
trying to limit what can be done (i.e., no pointer arithmetic) so that there
is a 'sane subset' by default.

I've played around with Go (worked through the Go Tour and implemented some
trivial site scraping/JSON client stuff). It's an enjoyable, straightforward
language, and it makes it very easy to get started, but I fear that it will be
(like Pascal) eventually overtaken by something hairier. (Go : Rust(?) ::
Pascal : C++)

~~~
kibwen
It's a miscalibration to compare Rust to the complexity of C++. Rust is more
complex than Go, absolutely and unequivocally, but that's because Go is
hanging out with Lua waaaay out over on the simple side of the spectrum of
programming language complexity. Given the enormity of its task, Rust is
actually a rather minimal language itself (though there are definitely still a
few features that I think could be made simpler).

FWIW, I think it's misplaced to fear that Go will be "overtaken" by Rust. They
both excel at different domains, and the systems software that you'd write in
one is probably not the systems software that you'd write in the other.

------
timtadh
I also miss generics in Go. It is especially a pain to implement a nice high
performance collections library without them. I have been working on and off
on a data-structures library and you have to go through a lot of hoops to make
it nicely usable and your still end up with `interface{}` everywhere.[1]

It should be noted that this is not the first attempt by any means. droundy
implemented basically this several years ago.[2] The problem is these things
are non-standard and I don't feel comfortable using them. I want generics but
I want them as part of the language not as some weird third party CGI script.
I also want them integrated into the template system. That way I can create
type variables which must implement an interface. However, when the code is
actually run, it isn't an interface object it is the actual object. Features
like that would make them fit much better with the rest of the language.

[1] [https://github.com/timtadh/data-
structures](https://github.com/timtadh/data-structures) [2]
[https://github.com/droundy/gotgo](https://github.com/droundy/gotgo)

~~~
barrkel
The key innovation here is that it uses the URL to do the parameterization.
That's not what is implemented in your reference [2].

I think this approach is actually quite clever. It's clever because URLs
belong to a global namespace. If you have third-party code that is also using
an instantiated generic, your code should be compatible with it because you'll
be using the same URL. Whereas locally-generated templates are more
problematic - compatibility depends on build scripts etc.

~~~
timtadh
I agree this does solve the `go get` problem. However, a better thing would be
to improve `go get` and make it use an actual build system. `go generate` is a
step in the right direction but they expect you to check the generated sources
in. I would prefer a tool chain that allowed your API customers to do code
generation from your library. However, these things are more complicated and
the Go Author's are probably right to do the simplest thing first.

------
jerf
It is worth pointing out that if the Go community settles on something like
this as the solution, it negates one of the underlying arguments that the Go
designers have against generics in the language. They have a set of criteria
necessary for them before they are willing to put a solution in the language
[1], and one of them is that they aren't willing to do C++-style template-
based generics that involve compiling separate functions for every generic
instance, causing the binary to grow in size.

But all the other solutions that are headed into that gap are just one level
or another of convenience wrapper around exactly that. If the Go community
just starts doing this at scale, then there's really no reason not to pull
that up and make it official.

[1]: Whether or not you believe that they're just against them full stop and
can't be budged no matter what, there's at least some claimed reasons.

~~~
chimeracoder
For context: Go has been my primary language both at work and for personal use
for over two years now.

> It is worth pointing out that if the Go community settles on something like
> this as the solution, it negates one of the underlying arguments that the Go
> designers have against generics in the language.

It wouldn't really negate the underlying arguments; it would just mean that
the community had grown and attracted developers who may not necessarily share
that part of the language creators' original vision of the language.

> But all the other solutions that are headed into that gap are just one level
> or another of convenience wrapper around exactly that.

Well, focusing on "the other solutions" misses two things:

First, many (I dare say most) Go developers[0] don't really use any "solution"
for this other than the features built into the language itself. In other
words, they're not out there writing alternative solutions because they
genuinely don't perceive a lack of generics to be as big of a problem as other
things, which they do write tools for.

Secondly, there _is_ a way to use interface{} to accomplish some (not all) of
the same things that generics do, while still preserving type safety. Look at
the functions in this Twitter client library I wrote:
[https://github.com/ChimeraCoder/anaconda/blob/master/users.g...](https://github.com/ChimeraCoder/anaconda/blob/master/users.go)

All the endpoints use the same apiGet() function ("generic"), but they all
return concrete types, _and_ they are all typesafe - you will never get a
runtime panic from any of those functions. This is _more_ than Java's generics
guarantee, due to the quirks of the way type erasure works there[1].

No, this isn't "generics", but it provides an alternative way of solving the
problem most of the time (in practice), and isn't that what matters? And yes,
this is not quite as clean as some languages which focus on having a very
expressive and precise type system (ML, Haskell, etc.), but it provides enough
type safety for me without requiring me to think consciously about the types
that I am using (which I have to do when writing ML or Scala).

[0] I mean people who write primarily Go at least 4-5 days a week, for work,
as opposed to people who dabble with it for side projects but primarily use
other languages.

[1] TL;DR: Java's generics are converted to "Object" after compilation, which
means that you can actually end up with runtime type errors that are
detectable at compiletime, due to inheritance. (It's been a while since I used
Java, so I won't provide an example, but they're easy to find online).

~~~
jerf
I write a reasonable amount professionally (not full time), and while in my
personal code I don't feel the lack of "generics" all that often, I _do_
frequently feel the lack of data structures, which I'd note is what the web
page uses as its example.

As I've been thinking about this issue, it occurs to me that there are a few
problems that get wrapped up into "generics", and "generics" aren't always the
only solution, and people tend to conflate all the subproblems in their head
when they say "Go doesn't have generics". For instance, one aspect of
"generics" is to write "general algorithms" that don't care about the
underlying data structure. Interfaces basically solve that problem. You write
the "general algorithm" to a specified interface, and provide things that
match that interface. It's pretty much "concepts" in C++. This is what the Go
community correctly refers to when it says that we don't always need
"generics" and that interfaces can be used instead... there is a specific and
important _use case_ for "generics" that is indeed covered by the interfaces.

The community is correct when it points out the problems are not as large as
some external people are supposing, and the external critics are also correct
when they observe that not all useful use cases are covered by interfaces in
Go.

And I'd say the biggest uncovered useful use case is type-safe containers that
aren't slices, arrays, or maps. What's more, Go's core target, network
servers, is also precisely where the runtime characteristics of many of these
more exotic data structures are important in the first place. Entire scripting
languages are perfectly happy with arrays and hash tables as the only two
effective data structures (or even just hash tables with arrays emulated on
top of them!), but they pay a level of speed a Go programmer is not
necessarily willing or able to pay, because network servers very often become
performance sensitive. So my opinion is that whether or not that solution is
"generics", Go badly needs _some_ solution to the type-safe container problem.

(And I consider the "you can do it with interface{}" argument to be a dodge.
It's a very convenient moment to suddenly decide you don't need static typing
in an otherwise statically-typed language. Why stop there with that argument?)

Cover that one use case, and what's remaining isn't worth worrying about...
it's not like the world is full of languages with perfect, quirk-free generics
implementations either, once you start studying them they all have issues of
their own. There is a reason the Go designers are resisting them, after all.

And I can't resist generalizing a bit... what's important is not the _how_ of
a solution, but the _what_. If you have a problem than in some other language
you'd hit with generics, then try to pick up Go, then scream because Go
doesn't have generics, that's not interesting if in fact interfaces would
work. The problem with "Go lacking generics" isn't, well, Go lacking
generics... the correct question is, _what set of problems are there than Go
lacks good solutions to_? And, as I show above, I'm not saying that because
there are no answers to that question... I named one, after all, and there's a
couple of more I could at least argue sensibly about. The point is that the
conversation ought to be focused on solutions to the specific problems rather
than insisting that "the stick I know from another language" has to be ported
in. And, it _shouldn 't_ be focused on things that Go does indeed have
perfectly cromulent solutions to, because interfaces _really do_ cover a lot
of the use cases of generics, in much the same way that in C++ templated
generics can be used to implement "concepts". There's definitely some overlap
here which careful thinkers should be careful to keep separate in their heads
as they think about the problem.

That said, I have to admit I haven't been able to sketch a solution to my
type-safe data structure problem that doesn't look an awful lot like generics.
But the solution space to this problem isn't bounded by my imagination.
(There's some possibility that one could restrict the "genericity" to the
point where it could only be used for type-safe data structures and dodge the
other issues, but I haven't had enough time to noodle that over.)

~~~
ansible
_And I 'd say the biggest uncovered useful use case is type-safe containers
that aren't slices, arrays, or maps._

For that, my current preferred approach is to wrap a generic implementation
(using interface{}) with a concrete type. Like this:

[http://play.golang.org/p/LkVWkyph75](http://play.golang.org/p/LkVWkyph75)

There is some boilerplate code needed for each concrete type, but it is
relatively small and easy to verify.

~~~
pjmlp
Just like Java, Object Pascal, Oberon and C++ back in the old days...

------
Animats
Go, which claims not to have either generics or objects, has both for built-in
types. Channels and maps are both opaque objects and parameterized types. The
argument that generics are unnecessary would be more convincing if those
built-in types were not needed. The problem is adding generics without making
a mess of things. This is harder than it looks.

In the example, there's a set of "int", and a set of "string". A new set is
created with

set := set.New()

in both cases. It looks like you could not instantiate "set" for both types in
the same program without a name clash. That's no good. Parameterized types for
Go make sense, but the types need to be given a user defined name each time
you create one. With "generics as a service", you'd have to encode that name
into the "import" statement, which is getting a bit clunky.

Go's lack of generics is a reaction to the mess C++ made of them. In C++ the
generics system turned into an entire compile time programming environment
based on term rewriting rules. Nobody wants to go there again. But Go went too
far.

~~~
burntsushi
> Go, which claims not to have either generics or objects, has both for built-
> in types.

Who is claiming that?

> The argument that generics are unnecessary would be more convincing if those
> built-in types were not needed.

This is a straw man that you're tearing down. The Go devs have always
acknowledged the utility of generics. They just have a different set of trade
offs that they are targeting.

More to the point, there's absolutely nothing inconsistent with the position,
"If we can make writing code easier with a little blessed generics without
sacrificing our project goals, then let's do it!"

------
wyager
Go seems to be re-inventing C++ and Java one step at a time.

Go didn't learn from history, and is doomed to repeat it.

~~~
zwieback
That was my thought, unless the Go guys send a signal what they will add in
the future. Otherwise you'll have tons of competing solutions and a lot of
anger when a winner is chosen a year or two from now.

With C++ you have boost as a funnel, C# has MS giving previews of what they
will add, similar for Java. If Go chooses to stay purist their world will
fragment.

------
bkeroack
Whenever this issue comes up (and on HN, it never appears to die), I'm
reminded of an eye-opening moment I had reading Rob Pike's excellent essay
"Less is Exponentially More" ([http://commandcenter.blogspot.com/2012/06/less-
is-exponentia...](http://commandcenter.blogspot.com/2012/06/less-is-
exponentially-more.html)).

An excerpt (original emphasis): "Early in the rollout of Go I was told by
someone that he could not imagine working in a language without generic types.
As I have reported elsewhere, I found that an odd remark...What it says is
that he finds writing containers like lists of ints and maps of strings an
unbearable burden. I find that an odd claim. I spend very little of my
programming time struggling with those issues, even in languages without
generic types...But more important, what it says is that _types_ are the way
to lift that burden. _Types_. Not polymorphic functions or language primitives
or helpers of other kinds, but _types_."

It seems to me that people program heavily with types because that's what
compilers are really good at. Therefore that's what most languages give us.
But I don't want to be constructing huge, complex type hierarchies with every
application. If I'm spending more time on that than the actual problem domain,
generic type systems are more of a problem than a solution. I don't use a
hammer because I love driving in nails, I use it because it's the best way to
stick two pieces of wood together.

(edit: perhaps I should clarify that I'm referring to type hierarchies within
the application itself as an intrinsic part of design, not _language_ type
hierarchies)

~~~
masklinn
> It seems to me that people program heavily with types because that's what
> compilers are really good at. Therefore that's what most languages give us.
> But I don't want to be constructing huge, complex type hierarchies with
> every application

That makes no sense, most languages with good generics support barely _have_
type hierarchies (let alone complex one).

~~~
NateDad
I must be using a different definition of a type hierarchy.... Don't C# and
java both have generics and type heirarchies?

~~~
masklinn
Why in god's name would you take these as references when discussing generics?

~~~
NateDad
Because they're the most commonly used statically typed languages with both
type hierarchies and generics?

------
alkonaut
If workarounds like these are common in the use of Go then leaving it out of
the language was a mistake. The argument that generics comes at a complexity
cost is nonsense: unless used it has no cost, the code would look like code
does today. The complexity added by NOT having generics in a strictly typed
language is enormous (as these examples show).

------
metafex
I honestly have mixed feelings about this. It is a really nice showcase of
what the standard library can do (walking the AST and modifying it) with
little amount of code.

OTOH it would be better to not make this a web-service. As much as it may seem
to be convenient, integrating it with "go generate" (when it's ready) would
probably be easier to use and not pose any security risks ;)

~~~
bouk
I agree the web-service is a terrible idea, I just wanted to see if I could
implement it. It's also super insecure, as go fetches the package over HTTP
(which you can see when you do go get -x gonerics.io/d/graph/string.git)

~~~
barrkel
Actually, I think the web service is the only interesting part of the idea.
Generics via string substitution are as old as the hills, I remember doing the
same thing for Pascal 20 years ago.

(The factual accuracy of the statement that it was 20 years ago disturbs me.
I'm getting old!)

~~~
crazychrome
100% agree. it's a break through not because of the problem it's trying to
solve, but the approach.

------
BruceIV
What happens if you want two different types of set? As-imports? (I've looked
at Go, but not written in it.)

~~~
bouk
You can alias imports! [http://learntogoogleit.com/post/63748050636/aliasing-
imports...](http://learntogoogleit.com/post/63748050636/aliasing-imports-in-
golang)

~~~
wffurr
But then I'm right back to having types named "IntSet" and "StringSet". Though
at least they're not using copy/paste code implementations.

~~~
NateDad
Well, they are using copy and paste, it's just automated.

But is Set<int> really better than IntSet? Don't they carry the same
information?

------
liveoneggs
being "idiomatic" is now the most important thing in Go, apparently.

~~~
norswap
Idiomatic thinking in an idiotic language.

(I don't really think Go is idiotic, although I do think it is lacking -- even
when you consider its aim to be minimal.)

------
crazychrome
How about my own types?

~~~
bouk
You can use them like this:
gonerics.io/d/set/github.com/crazychrome/package.git.Type

The things it supports can be found in the tests
[https://github.com/bouk/gonerics/blob/master/template_params...](https://github.com/bouk/gonerics/blob/master/template_params_test.go#L51)

~~~
crazychrome
Awesome!

Although I'm not a believer in generic but I think it's a bfd in the
programming field which will be as important as the REST thesis, or ror.

How about a service with similar approach that offers Parse to appengine
migration: user calls a rest point like:
generics.io/parse/github.com/crazychrome/package/user to configure fields and
types requirements, then the service generate a repo contains codes can be
deployed to appengine and 100% compatible with what Parse offers.

------
mserdarsanli
"If you think it's simple, then you have misunderstood the problem."

Bjarne Stroustrup

------
zerr
I found a better solution - abandoned the language.

~~~
swartkrans
What do you use instead now?

~~~
zerr
Mostly C++. As for Go's "niche", I'd still fallback to Erlang.

------
theflubba
Throwing type-safety out the window is so 2006. Just use Scala.

------
lorddoig
CGI...shell script...

~~~
bouk
:)

~~~
eximius
anyone tested it yet? I mean, you can't put an unpatched shell scripted site
in HN right after a new CVE and TELL us its a shell script... right???

~~~
bouk
Of course I patched it, and I'm not using bash in the first place
[https://github.com/bouk/gonerics/blob/master/cgi.sh](https://github.com/bouk/gonerics/blob/master/cgi.sh)

