
Golang: channels are not enough - azth
https://gist.github.com/kachayev/21e7fe149bc5ae0bd878
======
tshadwell
> How to make it reusable? `<-chan interface{}`? Welcome to the land of types
> casting and runtime panics. If you want to implement high level fan-in
> (merge) you’re losing type safety. The same (unfortunately) goes for all
> other patterns.

I can understand arguments for generics, but they're complaining about trying
to warm a whole pizza with a toaster. You need to put individual slices in, it
doesn't work like an oven. Cook it with an oven if you want, but don't try to
make it out like the toaster is offensive and useless.

Yes, you might need to re-implement these patterns each time with separate
types (slice up the pizza), but it's not really a lot of work for the extra
speed and crunchy pizza it gives you.

I'm pretty tired of the frequency of these articles where someone takes a
concept from a language, tries to force it into a different language with
little care for its idioms and then complains that it doesn't work.

Maybe I like digging holes with a trowel, maybe I like the precision it gives
me, maybe I've worked out a way to with a little extra effort accomplish the
same work. If you work with a spade and a bucket all the time don't complain
that you can't throw a trowel around like a spade.

~~~
nickbauman
It reminds me very much of the history of the Java language. The interesting
thing about Go is increasingly the runtime/VM and decreasingly the language
itself. The same became true with Java vs the JVM. Remember that people like
Gosling and Van Hoff used to proudly proclaim how hard they worked to keep
_out_ features from Java? Remember how eventually languages like Ruby, Clojure
and Scala kept getting more interesting while Java festered (and eventually
began stealing from these other languages)?

It sounds a lot like the Go culture. Go 1.6 has _no new features_. Ho hum.
Eventually Gisp will get STM, refs, agents and atoms. Just like no sane person
would start a new SaSS using Java when they could choose other languages to
run on the JVM that are much better, for example; eventually nobody will write
new code in Go when they can use Gisp or some other language that has the
missing features. It will happen.

For me (today) I just want to be able to add an element to the middle of a
slice in Go without having to read the documentation.

~~~
geodel
Go runtime is not independently available. This is first time I heard that
people are more interested in Go runtime than Go language. It is likely many
people care about language being interesting but, where I work, most debate /
arguments are about product features not language features. We have been using
Java quite productively for long time and do not see any need to change.

> Just like no sane person would start a new SaSS using Java when they could
> choose other languages to run on the JVM that are much better..

I think Scala is 12 yr old and fair to say it has not taken software industry
by storm e.g like Swift. It has maybe few per cent point market share of all
JVM languages. And the superiority of their language that Scala developers
claim all the time may ensure that Scala will not get any more popular than it
currently is. In my interaction with Scala developers are they are more
interested in talking how cool is Scala rather than what cool software they
have developed with it.

~~~
nickbauman
Swift isn't a fair comparison at all because it's a mandated language by
Apple. For me Scala is just an improvement on Java. It's valuable to people
who fetishize complexity, IOW, people who use Java and like it.

~~~
ane
Scala is a highly versatile hybrid language that can be used in a way that
doesn't promote needless complexity. Just like you need not use the most
advanced OOP features, you need not use advanced FP features, nor do you have
to use macros. _If you don 't want to._

This choice is what makes it an interesting and pleasant language to work
with. I've often dealt with Scala code that is much simpler to understand than
comparable Java code, but still more often than not, Scala code is made
needlessly complicated.

That choice implies a certain kind of _respect_ for the programmer: here's a
very, very powerful tool, and use it wisely.

Go is the antithesis of this, and I get its point, it's the opposite
philosophical direction. I think it's a direction that will not move the
industry forward, it's a direction that will undoubtedly help us produce value
---software, reliably---instead of doing something amazing.

------
avitzurel
Beyond Go and channels, this article/post touches on a subject that is
personally a real pain to me.

> It’s so cool to find yourself writing tutorial for beginners! And it's a bit
> painful when you are trying to implement big and sophisticated system(s).
> Channels are primitives. They are low-level building blocks and I highly
> doubt you want to work with them on daily basis.

Posts/talks that relate to beginners.

You know the drill, you only have a limited time to talk about something in a
conference and you only have limited time to write a blog post. It's easy and
tempting to cover the basics without diving in too much into the details.

What you are left with as a professional is very thin data on how to
accomplish complex stuff.

If you look at Docker and all the talks/posts around docker, they cover the
absolute basic. Where's deployment to production? where's alerting/monitoring?

One of the few people I see solve this problem is:
[http://patshaughnessy.net/](http://patshaughnessy.net/).

I've been trying myself to do more in depth posts that cover more than just
the basics, it's really hard work.

------
andrewvc
This is why clojure is awesome. It's a nice enough language, but where it
really shines is that it gives you pretty much every concurrency tool under
the sun. You have the entirety of java.util.Concurrent + all the cool clojure
tools like agents,core.async,atoms,transactional memory, etc.

Can you shoot yourself in the foot here? Sure. But you also have the right
tool for the right job if you know what you're doing.

~~~
nickbauman
Parallel concurrency is the Great White Whale of our time. Languages like
Python and Ruby give you a toothpick to hunt with. Java gives you a letter
opener. Go gives you a decent bowie knife. Clojure gives you something like a
proper harpoon. But something eventually has to come along at some point to
make it possible to not have to take down Moby Dick in the first place. I
don't know what that is but I'd love to see it.

~~~
kasey_junk
There is quite simply no argument that Java doesn't provide better complex
concurrency support than Go. You can argue that the golang defaults are
better, but there is _nothing_ you can do in go that you can't do in Java and
quite a bit the other direction that reverts to "program C with FFI".

~~~
skj
Using that argument you can assert that all languages are equal in every
aspect. So, not a useful argument.

------
avz
I used to miss generics in Go, but after a few years of coding in both Go and
Java I recognize the costs, but also the benefits of their absence.

We use generics to define new abstractions. This allows us to reduce
redundancy in code and provide succinct way of expressing complex concepts.
The main cost of not having them is that we have to fully write out the
algorithm more often.

At some point I realized that when I read someone else's Java code I often
find myself digging through countless layers of abstraction, factories,
builders, suppliers, visitors, delegates and each each time I have this
thought in my mind: "so where the hell is this place where the stuff _actually
happens_?" In Go, this place is right there inline in front of you or very few
hops away. This is particularly important if you're trying to understand a
corner case, e.g. how does this function/class behave when I give it an empty
list or null.

Faced with the choice between having to understand an abstraction or an
algorithm I prefer an algorithm. Most mainstream languages have generics. I'm
happy there is one that does not.

(Disclaimer: I am by no means a Go apologist: I have very mixed feelings about
the language overall. One thing I was hoping would be discussed in the article
was that channels don't compose well with other concurrency primitives, e.g.
one cannot select on a channel and a condition variable)

EDIT: formatting

~~~
groovy2shoes
> At some point I realized that when I read someone else's Java code I often
> find myself digging through countless layers of abstraction, factories,
> builders, suppliers, visitors, delegates and each each time I have this
> thought in my mind: "so where the hell is this place where the stuff
> actually happens?"

This has nothing to do with generics and everything to do with Java's heavy-
handed approach to OOP.

~~~
acjohnson55
I think I fat fingered the down vote button. Sorry. I completely agree with
your point.

------
blablabla123
High level abstractions are nice and I see myself rewriting certain patterns.
But that's not only true for concurrency patterns, for object oriented
patterns as well. Still most people write these things each time by hand, even
if some language have baked in support for simple patterns like Delegate.

I'm actually quite fine orchestrating concurrency with channels and the sync
package but I use Go since 2011. I suspect that people new to the language are
tempted to write things more complicated than they really are. I understand
that it's difficult to find out when to close a channel. But in reality, when
it's complicated figuring that out, probably channels aren't used optimally.

Regarding Generics, there have been plenty of discussions on the Golang Google
Group. Keeping the language lean and consistent also seemed to be the main
priority.

~~~
TheHydroImpulse
If people are still writing it by hand each time, it's a problem with tooling,
specifically dependency management. If it's that much of a pain to distribute
a library, then people are going to write it themselves.

That's not a reason not to include generics, however. It's an entirely
different problem.

------
divan
This post is from Sep 2014. I know the author, he's a nice guy, but when he
was at Golang Meetup, he was desperately trying to convince people to switch
to Clojure. It's all you have to know about the bias of this article.

~~~
eternalban
Curious: Can you expand on his "desperation"? Handing out money? Begging on
his knees? Giving away little fluffy stuffed animals? A promise of a date with
Rich Hickey?

~~~
divan
Hah. "Begging" is close :) He wasn't a speaker, but asked for a mic and
started 15min speech for the newcomers, explaining that "Go has no immutable
types", "channels cannot solve all the problems", "don't use Go, switch to
Clojure" and so on. He also made up some example of board game, as a proof
that language without immutable types is flawed and channels are not enough.
It was pretty awkward, really.

~~~
bsaul
That's funny... Usualy those types of people are just a bit crazy and unable
to make any kind of descent reasonning. But in that case i found the blog post
pretty in-depth and well said.

------
tedsuo
A point often missed by new Go programmers is that channels are only for
select. In other words, channels are a primitive for handling nondeterminism
at the runtime level, not a general purpose queue. If there is no select
statement, there is no actual need for a channel.

A well designed interface, not a primitive, is the correct way to express an
Object or an Abstract Data Type in Go. Specifically, prefer a Queue or
Collection interface over a raw channel. (Unless you need to use select.)

This is our fault though, for putting channels in every beginner example,
while never really explaining select properly.

~~~
madgar
What about range over a channel?

~~~
tedsuo
you don't need a range or a channel to iterate over a collection or read a
stream to the end.

------
placeybordeaux
I would have much prefered the title

Golang: I still want generics

This post has little to do with channels.

~~~
kasey_junk
You aren't wrong, but the strongest arguments _for_ generics in go (and in
many other languages) have to do with the abstractions around concurrency.

In a language that supports generics you have a fighting chance of adding good
concurrency pattern support above and beyond what the designated language
gatekeepers designate. In go, you have to resort to codegen.

------
alpb
This content is widely discussed 2 years ago:
[https://news.ycombinator.com/item?id=8315996](https://news.ycombinator.com/item?id=8315996)

~~~
kasey_junk
And yet, the complaints stand.

------
spriggan3
People need to stop trying to force themselves using Go. Yes, good concurrency
perks and easy deployment are handy, but Go basically demands people to code
1993 C-Object style since its authors basically rejected 20+ years of type
theory. If it feels dirty don't promote that, don't promote a community that
is in total denial when it comes to its language shortcomings.

Plenty of alternatives like D,Nim,Vala,Crystal,Erlang and many more. You want
the same developer experience as Go (go get,gofmt,golint,go-code...)? then
develop the tools for these languages and drop Go. Don't settle for an half
baked language just because "it's easy to start with",it's easy to start with
because the standard library is dumbed-down. Just look at Go projects on
Github where developers spend their time trying to fix the Go std lib in their
own code...

~~~
AnimalMuppet
> ... don't promote a community that is in total denial when it comes to its
> language shortcomings.

The point of Go was not to be "up to date" on type theory. The point of Go was
to have a language that minimized some of the problems that you run into when
you try to maintain 10 million lines of code for two decades. The
authors/designers of Go didn't feel that "modern" type theory was the key to
doing so. Of course, the modern-type-theory people think the Go designers are
wrong.

Who's right? It's too early to tell. But for what it's worth, I'm pretty sure
that the designers of Go have more exposure to modern type theory than the
modern-type theorists have experience with maintaining ten million lines of
code for two decades.

~~~
spriggan3
> I'm pretty sure that the designers of Go have more exposure to modern type
> theory than the modern-type theorists have experience with maintaining ten
> million lines of code for two decades.

See, the problem right here."Go designers know better than everybody else". No
they don't. The standard library itself is totally inconsistent and goes
against what Go designers themselves promoted as being idiomatic.

> The point of Go was not to be "up to date" on type theory.

Sure it's a dynamically typed language that mascarades itself as a statically
typed one, a bit like C with its void pointers everywhere... Except it doesn't
have C macros, which makes it even more painful to program than C.

~~~
_ak
Nobody claims that they know better than everybody else, that's just your
words, which you then contradict without the slightest bit of evidence. Truth
is, the Go developera have a proven track record over several decades, and
that gives their ways of doing things certainly more weight than those of
young whippersnappers who have never built several operating systems,
programming languages, or other large and complex pieces of software.

------
hardwaresofton
There seem to be a bunch of blog posts popping up recently criticizing golang
channels (speed/ease-of-use advantages of mutexes, stdlib prefering mutexes,
channels forcing awkward code, all the points in this post), and the lack of
generics.

I'm starting to think I might have drank the kool aid too early (grateful,
however, to read something that caused me to reassess), as I have recommended
Go at least once to coworkers.

Is it fair to say that these are some of the last remaining warts of the
language? I still like a bunch of other things about go (super ez cross
compilation, package management system, fantastic stdlib & testing support)

~~~
vessenes
I love go, we couldn't have built out our current software without it.

Channels are .. not great. They are too slow for many use cases.

That said, I'm building out a tool right now that uses channels in a nice way;
it takes input from a bunch of spots, and the channels magically force it into
an orderly queue which is operated on. That's quite a lot of code I don't have
to write.

If the channel messages went up past a certain amount, mutex contention and
throughput worries would push me onto a different angle of attack though.

Anyway, push go all you want. It compiles fast to a wide variety of statically
linked binaries and is performant and easy to deal with. Most of the
complainers about go are coming at it from what I would call a craftsman
approach: go is a modern Java, not a Haskell with C friendly syntax. It
doesn't want to be Haskell or Lisp, those languages can be very hard to use
and maintain for junior developers in large teams.

~~~
TheHydroImpulse
It's not a modern Java, it's an antiquated Java. It might aim to be modern,
but Go is far from being a modern programming language, even if it was
implemented in modern times.

~~~
vessenes
I disagree; it's modern in its sensibilities about which features to restrict
in order to fulfill its design goals, and it is informed by experience
distilled from millions of developer hours at large companies since Java was
designed.

Go will never have functions (well, this is a class in Java) like
RequestProcessorFactoryFactory.RequestSpecificProcessorFactoryFactory, an
actual apache xmlrpc server class name.

------
jerf
Given the generalized complexity of getting sync stuff done correctly, I'm not
going to try to bash this out fully in an HN comment, but you can do less-
boilerplate Promises (in the sense the term is used in the blog post) by using
composition instead:

    
    
        type Promise struct {
            *sync.Condition
            err error
            finished bool
            // note no "payload" here with interface{}
        }
    
        func NewPromise() *Promise {
            // return a new promise
        }
    
        func (p *Promise) Wait() error {
            // code here to use the condition and bool to wait, or get the error
        }
    
        func (p *Promise) Finish(err error) {
            // code here to mark the promise as finished or errored
        }
    
        type AccountPromise struct {
            *Account
            *Promise
        }
    
        func GetAccountPromise(f func() (*Account, error)) *AccountPromise {
            promise := &AccountPromise{nil, NewPromise()}
            go func () {
                var err error
                promise.Account, err = f()
                promise.Finish(err)
            }()
            return promise
        }
    

The boilerplate would only be that last function. The promise is used as:

    
    
        err = accountPromise.Wait()
        // accountPromise.Account is now valid if err is nil
    

That's the reflect-free version. If you're still feeling crabby, I believe you
could safely reduce the boilerplate even farther with some reflect usage. (By
"safely" I mean that you can statically guarantee the loosely-typed reflect
usage would still be correct.) That can be used with arbitrary callers of
.Wait(), without an explicit queue (shared lock for the condition functions as
the de facto queue).

Though I still would ask serious questions about Go code that has extensive
use of "promises". Generally I'd expect there's somewhere that sync.Once could
be used, or some other solution native to Go rather than porting in promises.
You don't really need promises in Go; for instance, to head off the objection
that I'm not showing how to do any sort of ".Then()" function so this isn't
"real" promises, Go's code "color" [1] is already fully asynchronous, so you
don't need the usual array of Promise combiners to create a new function
color. Go is already basically promise-colored itself.

Not that it's perfect. Generics to put together useful channel patterns would
be legitimately useful. I've missed that several times.

But the composition is generally underrated in Go. The programming community
has decades of experience in inheritance-primary language, but composition-
primary languages afford a different style, even if both are OO in some sense.

[1]: [http://journal.stuffwithstuff.com/2015/02/01/what-color-
is-y...](http://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-
function/)

~~~
pcwalton
> You don't really need promises in Go; for instance, to head off the
> objection that I'm not showing how to do any sort of ".Then()" function so
> this isn't "real" promises, Go's code "color" [1] is already fully
> asynchronous

That _is_ a legitimate objection. Because what you wrote isn't a promise: it's
a condition variable. The point of promises is to write _asynchronous_ code.

I think the defensible argument here is not "you can write promises in Go" but
rather "Go doesn't need asynchronous programming because threads are efficient
enough in its implementation". That is an interesting argument, and there's
some good evidence for it, but it's not the same as "you can write promises in
Go with little boilerplate".

I always found the "what color is your function" article a bit confused for
similar reasons. The points it makes are valid, but it goes off the rails when
it claims that Go succeeds in marrying the two models (asynchronous and
threads) without friction, when what it really did is just to optimize threads
in an effort to make asynchronous programming not needed. If you've got a
stack and a program counter, you're a thread, regardless of whether your
implementation maps 1:1 to the kernel's notion of a thread or not.

~~~
jerf
"Because what you wrote isn't a promise: it's a condition variable."

Yes, I'm sort of cheating, because what the original blog post implemented
isn't really promises either. It's hard to "really" implement promises in Go
because whereas in Javascript you're implementing a different "color"
language, in Go you're simply re-implementing the same color again. (And quite
poorly, given Go's very weak/simple type system.) If you do good software
engineering and remove redundancy, promises simply collapse down to
conditions, or disappear entirely.

I've tweaked my post to clarify I'm propagating the original post's usage.

The point I'm really shooting for is that you can avoid a significant amount
of, but not all of, interface{} usage when you're trying to use something like
that Promise by writing the logic in an object, then composing that in to
another type-safe struct. You can not, of course, implement true generic
containers that way.

I dislike not having true generics in Go, but I have found I like the
composition instead of inheritance default quite a bit, once I adjusted. I
think there's room for another new language to adopt that default and make
some progress in spaces where Go simply refuses to play.

Re: threads, I agree completely. What I care about is having a stack, so I can
do structured programming even in my "async" code, not the hardware
representation of the threads. Really need to finish up my post on that
someday.

~~~
pcwalton
> What I care about is having a stack, so I can do structured programming even
> in my "async" code, not the hardware representation of the threads. Really
> need to finish up my post on that someday.

When you write that post, make sure you talk about performance. :)

In particular, there is always a performance cost to having a stack, and not
having one is much of the reason why nginx is so fast. nginx could not switch
to a goroutine-like model without losing performance. There is no free lunch;
threaded programming (goroutines, etc) is better for developer ergonomics,
while stackless/state machine programming is better for performance.

~~~
jerf
I'm not planning on putting it in there, because it's one of those things that
programmers use as an excuse. "Oh, I can't afford a _stack_ , I need my code
to go quickly. Even if it takes me longer to write. And crashes more often.
And is harder to debug. And I can't use half my language features. And I'm
just writing CRUD apps for my bank."

It's a last-ditch optimization for when you've got nothing else left, or
perhaps in your case, something that a language designer worries about. But at
the moment, I see a lot more people grossly underestimating the costs of
throwing away the stack than underestimating the benefits. It's a rare program
that has been so hyperoptimized that there's nothing left to do but throw away
the stack, and even if you get that far, you need to understand the full
import of what that means, which is actually a great deal more than just "oh,
I don't have these bytes in RAM anymore".

~~~
pcwalton
I completely disagree. :) Stacks have a large performance cost when you get to
this the last level of optimization; we're talking 2x or more in tight syscall
loops when you take cache misses into account. Moreover, the memory costs of
stacks compared to the most optimized state machine can _easily_ be 10x. You
may not need this level of performance, but some people do.

This leads me to the exact opposite conclusion that you have. I think that, in
terms of language design, we should be figuring out how to design stackless
coroutines in a way that is as ergonomic as the stackful ones instead of
throwing up our hands and saying "we're going to eat this large performance
cost because there's no way we can possibly design a language that makes
stackless programming ergonomic and robust".

As language designers, we shouldn't be giving up before even fighting the
battle. Because, if we do, nginx will be around and winning the performance
fight, and people will (rightly in many cases) never switch away from it, and
that old-timey C will persist forever. My former colleague Robert O'Callahan
had a great view on "abstraction taxes": if your abstractions aren't zero-
cost, you're incentivizing programmers to not use them, and you're forcing
programmers to choose between good engineering practice and good performance.

------
logingone
Why isn't Go the same as the other languages?

Because then it would be the same as the other languages.

------
elcct
I stopped after reading "reason for panic?" paragraph. Basically, if you don't
like the tool or you are not too comfortable with it, don't use it. If you
want to pass arbitrary stuff through a channel you can for example serialize
to JSON or even use things like protobuf if you are religious about memory. If
you really want to send something random, then there is something wrong with
your project.

~~~
mikelward
Did you not understand the

func merge[T](cs ...<-chan T) <-chan T

reference?

The author doesn't want to send something random, they want to send things of
a single type, where the type is defined at compile time, without re-writing
functions like 'merge'.

~~~
elcct
I think I was not clear - I mean essentially what you replied. I think this is
still a try to apply mindset of one thing to another. You can't do generics in
Go (well, you can use template system if you really want), so why would you
try to solve the problems with generics in mind? It is like you were really
upset that getting family in an industrial truck to Disneyland is not
comfortable. If you have a problem that requires generics, use something else
than Go.

