

Topo – A library to create in-process topologies of goroutines - mdmarek
https://github.com/mdmarek/topo

======
epimenov
I think you misunderstood the blog post and go channels. Shuffle is absolutely
not want you want to use (in the README example). Sending to a channel blocks
until the message is accepted by the receiver (unless you're using buffered
channel).

What you want to do instead, create 1 channel, and make multiple goroutines
(Sinks) read from it, then whenever goroutine is finished with the task it
would take a new message from that channel.

By picking a channel to dispatch yourself you don't take into account busy-
ness of it, so you might wait on a channel even though there are others that
are idling waiting for messages to be accepted.

------
mdmarek
The library handles a lot of what is covered on this golang blog post:
[http://blog.golang.org/pipelines](http://blog.golang.org/pipelines).

------
patcoll
I've been using this library which helps with composing channels, and also has
some nice implementations of useful patterns:

[https://github.com/eapache/channels](https://github.com/eapache/channels)

Unfortunately Go's lack of generics hurts in letting me use these helpers in a
way that guarantees the data types going in/out of the generic channels.

~~~
Animats
Right. Everything gets converted to Go's "any" type, "Interface{}". At the
output end you have to use a type assertion to get it back into the actual
type. Note that an "Interface{}" is a pointer; you're not queuing the data,
you're queuing a pointer to the data. So you're sharing data between
goroutines. The sender must allocate a new variable for each send.

(This is my biggest complaint about Go. It's too easy to accidentally share
data between goroutines, because so many things are implicitly references to
mutable objects. Slices, for example, are references. Erlang and Rust, which
also use queues heavily, avoid doing this.)

------
mdmarek
Hi @epimenov thank you for the feedback. The shuffle method starts a goroutine
for each inbound channel, so one busy outbound channel will not block the
whole system. That being said I probably should add two more features, one of
which is to add a channel size rather than always using a 0 size channel, and
the other is to change it to select the first non-blocking outbound channel. I
don't consider the blocking part always negative, because handling back-
pressure is need at one point or another, but you are right, the method should
not introduce starvation.

~~~
epimenov
You're using one input source in the example. I think the main issue that it
promotes bad usage pattern.

------
davidw
You probably know this already, but "topo" means mouse in Italian.

Looks like an interesting project. It'd be interesting to see someone who
knows both Erlang and Go well explain this in comparative terms. I know Erlang
reasonably well, but not Go.

