
Channels Are Not Enough - kachayev
https://gist.github.com/kachayev/21e7fe149bc5ae0bd878
======
spion
Go doesn't let you build abstractions - it offers what it does, and if its not
enough - tough luck.

What I dislike worst is the denial of the Go community and creators, claiming
that generics are too complex and that you don't really need them.

I dismissed Go not because of its lack of abstraction power, but because its
authors and community is incapable of admitting problems when they see them. A
similar problem with CoffeeScript (conflation of declaration and assignment +
scope rules) and the authors' refusal to admit that there is a problem also
made me dismiss it entirely.

Every language/platform has problems. But not every language is in denial of
them. We should all avoid those that refuse to acknowledge their problems -
because that points to a much deeper, much more serious problem - a problem
that cannot be eradicated with technical means.

~~~
eloff
Lack of generics != lack of abstractions.

The Go community freely acknowledges that generics are a nice feature, and
that lacking them is a pain point of Go sometimes. Although many Go developers
will tell you that sometimes turns out to be not that often in reality, which
has also been my experience.

Rob Pike has outlined the tradeoffs inherent with generics here:
[http://research.swtch.com/2009/12/generic-
dilemma.html](http://research.swtch.com/2009/12/generic-dilemma.html)

The bottom line is it's not a feature that comes for free, and if most of the
time you don't really need it, maybe the costs aren't worth it. That's a bold
statement for a programming language these days, since generics is a central
feature of every popular, modern, statically typed language. Whether they are
right or wrong I won't attempt to say. The language may well get generics one
day, but it's early days still and the team is rightly focusing on more
important features for the moment.

~~~
ufo
Correct me if I'm wrong but doesn't the current state of affairs (using
Interface{}) have all the downsides of Java's boxing but without any of the
type safety?

~~~
Pxtl
It reminds me of the wonderful fun days of Java 1.2 - anybody remember how
proud Sun was of how everything derived from object so you could use the
untyped containers for everything?

~~~
JulianMorrison
The main difference with casting to Object is that you can use type switches
or safe casts which return an additional success boolean, to restore type
safety quickly after taking the item out of its container. Yes, you can type-
test in Java before you type-cast, but it isn't idiomatic, and it isn't
syntax.

~~~
frowaway001
"It's fine in Go to do the wrong thing, because they added syntax to make it
even easier to do the wrong thing."

------
djur
This is a really great read, and goes beyond just channels and concurrency. It
seems to me that Go is designed to discourage developing higher-level
abstractions. That was my sense using it in the past, and it's only gotten
stronger over time.

Remember that one of Go's primary stated goals is "speed of compilation"[1].
Simplicity and ease of learning are paramount, and it's easier to learn a
language where any chunk of code basically follows the same rules as any
other.

[1]:
[http://golang.org/doc/faq#Why_doesnt_Go_have_feature_X](http://golang.org/doc/faq#Why_doesnt_Go_have_feature_X)

~~~
voyou
"Go is designed to discourage developing higher-level abstractions."

Yes, that does seem to be the distinctive thing about Go. It's almost like
someone read Paul Graham's "blub" essay[1] and thought, "what would it mean to
take seriously the idea that blub _is_ the best language?"

Everyone knows that building abstractions has a cost - the cost of building
the abstractions themselves, the cost of figuring out the particular
abstractions employed in a given project, and the cost of comprehending a
language flexible enough to support these abstractions. The hope is that the
cost of abstraction is an investment: the time you put in will be rewarded in
faster development when you put the abstractions to use. But at some point
increased abstraction is going to give diminishing returns.

Now, most programs written today don't involve that much more abstraction than
would have been possible with programs written in ALGOL; that is, we haven't
seen a huge widespread increase in the power of abstraction used by most
programmers in about 50 years. People like Alan Kay and Brett Victor[2] decry
this stagnation, and maybe they're right to. But maybe the current low level
of abstraction is so durable because it's a sweet spot between the benefits
you get from abstraction and the costs involved in coming to grips with that
abstraction.

Most people, particularly most people who develop programming languages,
assume that we're nowhere near the point of diminishing returns for increasing
abstraction. Go seems like an experiment to test the possibility that the
maximum efficiency occurs at a much lower level of abstraction than we usually
think. It will be interesting to see whether (or in what domains) that
hypothesis turns out to be true.

[1] [http://www.paulgraham.com/avg.html](http://www.paulgraham.com/avg.html)
[2] [http://worrydream.com/dbx/](http://worrydream.com/dbx/)

~~~
nickik
I think what you are missing is that people do use abstraction if you offer
it. Java or C# generics or even when C++ added templates. C++ templates where
abused but it was done so often that is now standard.

If you look at code from the newer fancy languages people do use these fancy
features.

------
chimeracoder
It's funny that OP uses Twitter as the case study at the end. I wrote what was
then the first (and think still the only) Go Twitter client library that works
with v1.1 of Twitter's API[0]. It implements automatic rate-
limiting/throttling behind the scenes, and it returns values of concrete types
(not interfaces) ready for immediate use.

Keep in mind that, were I to write this again today from scratch, there are a
number of things I would do differently (since I started it as a relative
beginner expanded on it as my familiarity with Go developed, it's grown to be
a bit over-engineered in places). But I still think it's a worthwhile example
in this discussion.

For concurrency, I wouldn't say that what OP is trying to do is going to be
easy in _any_ language, because OAuth in general kind of sucks[1] the Twitter
API itself has a number of quirks that make it cumbersome in general,
irrespective of language[2]. That said, Go was by far the easiest to work with
here, because channels allowed me to abstract the pagination and the rate-
limiting in a way that it would be invisible to all callers, but "magically"
handled behind the scenes.

Without going into too much detail, I can see that the way OP has designed his
code looks a bit cumbersome. That said, while it's a reasonable way of
approaching it, I don't think it's actually the best approach in Go given the
language's idioms.

One other thing I want to draw attention to is the use of the general-purpose
function for issuing a GET request to Twitter, and how that is shared among
the various functions that use it to return values of varying, but known,
types.

I don't want to use the word "generic" here because people expect a certain
thing when they hear that word, but I will say that this function is (A)
general-purpose, and (B) type-safe - it involves _no_ type assertions, and the
functions all return concrete types instead if interface{}.

[0]
[https://github.com/ChimeraCoder/anaconda/](https://github.com/ChimeraCoder/anaconda/)

[1] Don't get me started on this

[2] I've written client libraries for Twitter in a few different languages, so
I actually have a reasonable point of reference on this - at one point, it was
my personal "hello world" for testing out a new language.

~~~
dkarapetyan
I'm curious to see this general purpose function of yours. You are also
addressing a different point than the article. The main point is that patterns
like parallel map are impossible to implement in Go in a type-safe manner.
This is a valid complaint depending on what is meant by type-safe and at this
point the arguments for/against Go usually devolve into name calling and
matters of culture. If you could factor out all that general purpose
functionality from your code base, e.g. the rate-limiting and other things,
and turn it into a re-usable library then that would be an entirely different
matter. Whether that would qualify according to the author's definition of
type-safe is another matter.

~~~
jordwest
It looks like this is the function:
[https://github.com/ChimeraCoder/anaconda/blob/master/twitter...](https://github.com/ChimeraCoder/anaconda/blob/master/twitter.go#L162)

This looks like a good example of interface{} used correctly - the consumer of
the apiGet function passes in their strongly typed object, which then is
handed in to json.Decode. No type safety is lost from the perspective of the
consumer, while the json.Decode can operate on any struct type.

Sure this addresses a different issue, but for all the complaints about
interface{} I think this is a good use-case.

------
skybrian
Rather than arguing about generics in Go again, I'd be interested in reading
about how experienced Go developers solve the problems posed in this article.
(He may be wrong that there's no elegant solution.)

Also, if it can't be solved elegantly, perhaps adding merge() and a few other
important functions to the language would be good enough? After all, Go
already has the magic append() function for slices and we get quite a lot of
use out of it.

~~~
jerf
There's no generalized solution to every generic problem. The "cast to
interface{}" is the closest to a linear mapping of generics, but if you're
reaching for that all the time, you're doing it wrong. So the only sane answer
is, "it depends".

The only use case that I have encountered that is essentially impossible in Go
is the "generic data structure". Mind you, that's a bit of a big deal, even if
it covered over by "generic" arrays and maps (which, further, _does_ cover
over a lot of the cases), but it is also the only case where it is actually a
problem. Despite the fact Go is not functional, like, _at all_ , I still find
myself using my functional training in minimizing the surface area of an
"object" as much as possible incredibly useful when programming in Go. I think
if I were only an OO programmer trying to come over to Go I'd have a much
harder time of it.

Most of the rest of the time you can solve your problem by asking what the
code is _really_ trying to do, pull that up into an interface, and carry on. A
substantial portion of the rest of the use case can be covered by providing
wrapper objects that compose in another object, probably using an interface as
the composition point.

I think in practice, about 80% of the problems that people trying to jam
generics into Go are encountering are because they are trying to program C++,
Java, or C# in Go. Despite their substantial superficial similarities, Go is
not any of them, and idiomatic answers differ substantially. Let me
rhetorically underline that... Go _really_ looks like a Java or C# clone if
you just read down the bullet points, but there are enough important
differences that it's a substantially different experience to program in it.
Trying to program Go as C# is only slightly less frustrating than trying to
port FP idioms into Go (which is a complete waste of time). The remaining 20%
are real problems that Go hasn't got a good answer for beyond duplicating
code.

Now. All that said, despite the fact that people not used to idiomatic Go are
significantly overstating the problem that is the lack of generics, it _is_ a
problem and were I in charge, I'd be trying to work something out on this
front. (I actually have a proposal I've probably put about 4 or 5 hours of
thought into. In defense of the Go developers, this _is_ a nontrivial problem
when you stop just waving the word "generic" around and start trying to
seriously create an implementable suggestion, accounting for all the use
cases, grammar interactions, semantics, etc.) My best argument is that for a
language as mature as Go, this is embarrassing:
[http://golang.org/pkg/container/](http://golang.org/pkg/container/) Those are
the "generic" containers in the standard library that ship with the
language.... all three of them as of this writing, plus array and map. At this
point that ought to be significantly richer.

(So, that ought to just about piss everybody off...)

Oh, I ought to add for context that I'm actually pretty fluent in Haskell, and
my primary work programming language up to this point have been dynamic
languages (Perl, Python, etc). So it's not like I'm not comfortable with
generics or incapable of understanding how to use them or anything. If
anything, I dare say my openness to other answers and then finding ways to
express them in Go may be part of why this doesn't bother me too much. There
_are_ other options, most of the time, and they are not generally
"compromises", either... they are often perfectly sensible options, or even
_better_ options than are available in other languages on a bang/buck basis,
such as the trade for making composition very easy and inheritance something
fairly difficult.

~~~
skybrian
Talking about how to implement generics generically isn't really responding to
my question. The original post had questions about how to implement specific
concurrent constructs like pipelines, futures, and so on. So the question is
how do you handle _concurrency_ in Go? Are we missing important features that
should be part of the language or standard libraries?

Some of these may be one-off generic functions like append() that could be
added to the language without adding full-blown generics.

~~~
jerf
I know this is going to sound like an evasion, but the answer is that
idiomatic Go doesn't involve pulling in patterns from other languages, but
using ones native to the language.

Futures I can be specific about, though. You simply don't use "futures" in Go.
Futures (as the term is used by most languages) are a hack around missing
concurrency constructs, and in Go, they aren't missing. Use channels instead.
This covers rather a lot of patterns from Javascript and the weaker languages,
actually... you don't port them, you simply _don 't use them_. They aren't
that appealing once you learn the native idioms, which are, generally, quite a
bit better.

Putting a "future" library up on /r/golang is a pretty common occurrence...
but they're so trivial they're worthless. It's harder to "use" a future than
it is to just do thing the future library is doing.

For pipelines, you just "do" them... you don't set up some declarative
pipeline, you just... program them. Sort of like how in the Python world they
prefer the for loops to map statements? Same thing, really... if you want a
pipeline, just hook things together manually. They may miss out on some
abstraction, but, honestly, in most imperative languages I'm underwhelmed by
the concept of really generic pipelines. Haskell's making some progress on
that front and I've actually got an "in the wild" five-stager, but in the
imperative world I'm not sure I've ever encountered _in the wild_ some sort of
6-stage pipeline that was better written other ways.

If any major concurrency pattern is missing from Go, it's an explicitly
asynchronous concurrency pattern. However, this can be fixed by library code:
[https://github.com/thejerf/reign](https://github.com/thejerf/reign)

------
georgemcbay
"I want all this <-done synchronizations and select sacramentals to be entire
hidden"

Then perhaps Go is not the language for you. There are a lot of design
decisions in Go that seem arbitrarily restrictive at first but are there,
AFAICT, to (as much as is reasonable) force programmers to write code where
what is happening is explicit and obvious without having to dive down into
layers and layers of abstraction to find "the magic".

This is, IMO, a feature and not a bug, but YMMV.

~~~
djur
Go has plenty of "magic": garbage collection, maps, slices, channels,
goroutines. It just wants you to stick to the abstractions provided by the
language and not create your own.

This is fine, it's a design goal of the language, but it's unsurprisingly
frustrating for people used to more expressive languages.

~~~
burntsushi
> but it's unsurprisingly frustrating for people used to more expressive
> languages

Speak for yourself. I know you certainly don't speak for me. I love Go. I love
Haskell. I love Rust.

There are things in _all_ languages that frustrate me. I can appreciate the
particular trade offs made in each language. Believe it or not, I think the
language specifications for both Haskell and Go are things of beauty. It's
amazing how close the core of Haskell is to just a simply typed lambda
calculus. It's amazing at how Go can be so well specified in such a short
document with _near_ perfect orthogonality in the language.

Many of your comments in this thread read (to me) as if you're speaking from a
position of authority on what the right design for a language is. I think your
comments would be better received if you expressed your thoughts as opinions
rather than as things you consider facts.

~~~
lobster_johnson
I find your response condescending and hostile, actually. The parent was
making a basic generalization; he didn't categorically claim that his
statement applied to all people.

People _de facto_ speak in the form of opinions, and if we are all to qualify
our statements with "...in my opinion" for fear of accidentally making an
authorative statement, then HN would become a very unhappy place.

~~~
frowaway001
> I find your response condescending and hostile, actually.

Welcome to the Go community.

~~~
kitsune_
A snarky comment, but I experienced the same on the golang-nuts mailing list.
If you dare to question conventional wisdom be prepared to be shot down rather
unceremoniously.

At least by now, they updated their docs a little bit.

I still think the "How to Write Go Code" article, which most beginners will
encounter, is absolutely misleading by its advocacy of 'go get' \- I cannot
envision a universe where 'go get' makes sense. You absolutely have to vendor
your dependencies if you want any kind of stability with your project.

~~~
burntsushi
> You absolutely have to vendor your dependencies if you want any kind of
> stability with your project.

I have several open source Go projects and I have been maintaining them for
years by just using `go get`. I've never once had a stability problem because
of it.

Of course, your point is absolutely correct. But I'm pointing out that `go
get` can absolutely be useful in some universes. In fact, it's one of the
things I love most of the Go toolchain.

So, umm, can we stop presupposing that everyone else's opinion and experience
is just wrong?

------
rdtsc
Yeah it seem channels are basic primitives. Kind of a like a class in OO
programming.

Just like Erlang has explicit processes and message sending primitives So it
is pretty simple and concise at that level. In order to build large systems
there is OTP (or e2) that embodies and abstracts away some common
patterns/behaviours: gen_event, gen_server, gen_fsm, supervisors, logging,
distribution between nodes etc.

I imagine over time go will acquire those kind of libraries (maybe it already
has them?).

There is also an interesting duality between Erlang and Go. In one case there
are explicit processes (identified by PIDs) + an anonymous (implicit) mailbox.
Where go has anonymous goroutines but explicitly identifiably channels. It
seems they are duals. You can simulate one with another. And you can build
concurrency patterns on top of them.

I personally prefer Erlang's model to think about concurrency. There is an
active entity -- a client request, a server, a socket handler, it has a an
id/name, it can be monitored for liveliness, it can be halted, upgraded,
killed, can send messages to it. Somehow to me that makes it easier to map to
concurrent problem domains. Channels can ultimately do the same things but it
is harder for me to design using goroutines.

~~~
jorlow
The problem (which the article touches on) is that you have to resort to
interface{} to make things reusable since Go doesn't have generics. So you
have to pick between using a library (which will handle edge cases better) or
compile time type checking.

~~~
NateDad
You don't have to resort to an empty interface, in fact, mrust experiences go
programmers cringe when they see anyone using empty interfaces. The fact of
the matter that much code can reused without being "generic" and often your
code never gets reused. YAGNI and all that.

~~~
tel
The point of generic code is not purely that it's easier to re-use. The almost
more important fact is that generic code has less information about its inputs
and outputs—this leads to a smaller design space and, consequently, an easier
time designing the implementation and an easier time avoiding bugs.

~~~
NateDad
I think a _lot_ more bugs are introduced from people trying to make their code
too generic rather than too specific. The complexity of the code goes way up
when you stop being able to rely on certain values being certain types. If I
"hardcode" my ID to be an int64, there are a lot of assumptions that become
safe to make, like the fact that it's a relatively small value, that I can bit
shift it, that the keyspace is of a particular size.... if you then take the
same code and decide that the ID should be genericized so it can be anything,
now I have to account for the fact that the keyspace could be unbounded (as in
the case of a string ID), I can't assume it's safe to copy around the value
(it might not be thread safe and/or it might be a really large value).

Saying having less information about the data makes for easier coding is
almost always not true.

~~~
tel
> _decide that the ID should be genericized so it can be anything_

You say that, but you probably don't actually do it. If it truly could "be
anything" then you would not be able to do _anything_ to it. This is the
nature of polymorphism.

So instead, your algorithm constrains the polymorphism based on what you need.
If you make use of the properties {small, bit-shiftable, sized, bounded, copy-
safe} then you must prove (exactly and only) that whatever the generic
variable gets instantiated to satisfies {small, bit-shiftable, sized, bounded,
copy-safe}.

Indeed, this is part of how such a system prevents bugs---it forces you to
express exactly how much information about the types involved is needed. You
can thus reflect better on the kinds of contracts/laws things must uphold and
are prevented from accidentally making use of a property of your concrete type
which you do not demarcate.

In a truly expressive language you might write

    
    
        foo : forall id n . 
              (Bits id, Size id = n, n <= 1024) 
              => id -> {Copy id} id
    

to indicate all of those properties needed (bounded being subsumed by
constraining the size)

Today, you can get promises very similar to the above by using a language like
Cryptol[0].

[0] [https://galois.com/project/cryptol/](https://galois.com/project/cryptol/)

------
toleavetheman
The interesting thing about Go, that can help make sense of all its oddities,
is that it was not created to assist the developer.

Go was created for businesses, not developers. The holy grail of a corporate
programming language is that all individual developer personality is
restricted, such that _you cannot tell from reading code who wrote it_. This
is all about long-term, large-scale maintainability for massive code bases at
massive corporations.

Disclaimer: I work at Google, I do not represent Google, and this is just my
opinion after spending time "on the inside".

~~~
the_af
I understand not having any fancy features in a "corporate programming
language", but wouldn't language features aimed at reducing code duplication
(like generics) help with maintainability?

~~~
toleavetheman
I think the extra flexibility gained would be perceived as a negative. The
ideal (from their perspective) is that there is only one way to do a
particular thing, no matter how verbose it is. Google has recently been mostly
Java (Guice, dependency injection everywhere), and has long preferred
incredibly verbose code at the cost of occasional duplication.

~~~
the_af
Agreed, it's likely what you say is the perception. But the thing is:

> _The ideal (from their perspective) is that there is only one way to do a
> particular thing_

The lack of useful tools like generics (and others) means there is no single
way to do a particular thing: things must be duplicated everywhere (or ugly
workarounds to avoid doing so must be employed, like casting from Object or
using interface{} or whatever). Surely the benefits of better tools outweigh
their inconveniences, even in a corporate environment? This is not a nerdy
programmer's whim, but a major software engineering principle of direct
consequences for any business.

Taken to an absurd extreme, you can simply copy & paste code everywhere --
that's the simplest programming model there is, and even the most junior of
programmers can handle it without having their learning skills taxed in any
way. It just leads to maintenance hell, which is why it's frowned upon even in
corporate environments.

------
ericflo
I think the author could use a technique like this to build the higher level
abstractions they want: [http://commandcenter.blogspot.nl/2014/01/self-
referential-fu...](http://commandcenter.blogspot.nl/2014/01/self-referential-
functions-and-design.html?m=1)

~~~
ufo
I remember seeing that post a while ago on Reddit. The impression I got from
the discussion is that the whole "self-referential" thing was unnecessary and
that the code would be much simpler in a language with generics.

[http://www.reddit.com/r/programming/comments/1w2y01/rob_pike...](http://www.reddit.com/r/programming/comments/1w2y01/rob_pike_selfreferential_functions_and_the_design/)

------
wilsonfiifi
Wouldn't a library like zeromq address the complications/shortcomings the OP
highlights in his article, when working with Go channels on a regular basis?
Using the 'inproc' for message transport with zeromq should give similar
performance to pure go channels no?

This Go client implementation
[https://github.com/pebbe/zmq4](https://github.com/pebbe/zmq4) by Peter
Kleiweg also includes all the examples from the online zeromq guide which is
great.

I know it's not a pure ago solution but the OP does mention that

 _' 99% of time I don't really care if the response is delivered with a
channel or a magical unicorn brought it on its horn.'_

I don't know, maybe I'm missing the point of the article.

~~~
dkarapetyan
Sure, but at that point what is the point of using Go? You might as well use
some other language like D if all the parallelism and message passing patterns
are going to be handled by a library like zeromq.

------
zak_mc_kracken
Pretty much every article that is posted about Go ends up in a never ending
discussion about the absence of generics, which completely drowns the
discussions about Go. As a result, the material on the web about Go has a very
high noise/signal ratio, which is unfortunate.

Hopefully, the Go team will see this as one more reason to add generics to
their language, but until they do that, Go will remain a niche language with a
severely crippled potential.

~~~
ianlancetaylor
I have to credit this as being one of the more original arguments for adding
generics to Go: we should do it because it will increase the signal/noise
ratio in discussions about Go.

(Personally, I don't mind the ongoing discussions about generics in Go, I just
wish they were less repetitive. It's very easy to say "add generics to Go!"
It's a little bit harder to actually do it well.)

~~~
zak_mc_kracken
Yeah I was trying to insufflate new life in this tired debate, glad someone
noticed :)

------
mavelikara
In an essay titled "Why Pascal is Not My Favorite Programming Language" Brian
W. Kernighan wrote:

<quote> The size of an array is part of its type

If one declares

    
    
         var     arr10 : array [1..10] of integer;
                 arr20 : array [1..20] of integer;
    

then arr10 and arr20 are arrays of 10 and 20 integers respectively. Suppose we
want to write a procedure 'sort' to sort an integer array. Because arr10 and
arr20 have different types, it is not possible to write a single procedure
that will sort them both.

The place where this affects Software Tools particularly, and I think programs
in general, is that it makes it difficult indeed to create a library of
routines for doing common, general-purpose operations like sorting. </quote>

Kernighan was one of the early C/Unix developers from Bell Labs. It is amusing
to note that Go, whose authors come from the same background, getting
criticized for something very similar.

[1]: [http://www.lysator.liu.se/c/bwk-on-
pascal.html](http://www.lysator.liu.se/c/bwk-on-pascal.html)

------
disputin
I stopped reading. The further I got the more the article seemed to be
complaining not about channels but that Go isn't a functional language. That's
right, it isn't.

