
Show HN: Chan – pure C implementation of Go channels - tylertreat
https://github.com/tylertreat/chan
======
eps
Good stuff, thanks for sharing. Random comments based on a quick glance
through the code if I may.

    
    
      chan_t* chan_init(int capacity);
    

should be "size_t capacity", since negative capacity values are invalid. This
also eliminated the need for the first check in the function.

    
    
      chan->buffered
    

is duplicate in meaning to chan->queue being non-NULL.

Why chan's mutexes and pcond are malloc'ed and not included in chan's struct?
They are also not freed when channel is disposed, unless I'm missing
something. Mutexes are also not destroyed when buffered chan is disposed, but
they _are_ created for it.

(Edit)

Also, my biggest nitpick would be that chan_t structure should really not be
in chan.h header since it's meant to be transparent to the app and the API
operates exclusively with pointers to chan_t. Then, if it is made private
(read - internal to the implementation), then you can divorce buffered and
unbuffered channels into two separate structs, allocate required one and then
multiplex between them in calls that treat buffered and unbuffered channels
differently. I.e.

    
    
      struct chan_t
      { 
          // Shared properties
          pthread_mutex_t m_mu;
          pthread_cond_t  m_cond;
          int             closed;
    
          // Type
          int             buffered;
      };
    
      struct chan_unbuffered_t
      {
          struct chan_t    base;
    
          pthread_mutex_t  r_mu;
          pthread_mutex_t  w_mu;
          int              readers;
          blocking_pipe_t* pipe;
      };
    
      ...
    

or better yet, just include a handful of function pointers into chan_t for
each of the API methods that need more than the shared part of chan_t struct,
initialize them on allocation and put type-specific code in these functions.
It will make for a more compact, concise and localized code.

(Edit 2)

Here's what I mean with function pointers -
[https://gist.github.com/anonymous/5f97d8db71776b188820](https://gist.github.com/anonymous/5f97d8db71776b188820)
\- basically poor man's inheritance and virtual methods :)

~~~
tylertreat
Thanks, I'm not much of a C programmer, so I appreciate the feedback. I'm not
sure what you're referring to with the mutexes and pcond not being on the
struct because they are. Can you clarify? You are right with the disposal
though, I need to clean that stuff up :)

~~~
eps
You don't need to allocate pthread_mutex_t instances on heap, you can include
them into the struct directly -

    
    
      struct foo
      {
         pthread_mutex_t bar;
         ...
      };

~~~
tylertreat
Got it, that makes more sense.

------
kersny
Libtask[1] by Russ Cox (one of the creators of Go) may provide some
inspiration. It implements channels but uses coroutines instead of threading.
As a consequence it has some cool stuff like networking and it does all of the
hard parts of saving/restoring registers for task switching.

[1]: [http://swtch.com/libtask/](http://swtch.com/libtask/)

~~~
byuu
Just a forewarning: that library, like most C cooperative threading libraries,
is using ucontext.

ucontext is unbearably slow, to the point where it's just as fast to use real
threads and mutexes, even on a single-core system. There is nothing
lightweight about it. (The technical reason is because they call into kernel
functions to perform their magic.)

The actual logic of saving/restoring registers and the stack frame requires
about 5-15 instructions per platform, and is very easy to write in assembly.
And for the platforms you don't do this for, there's a non-standard trick to
modifying jmpbuf which works on x86/amd64, ppc32/64, arm, mips, sparc, etc.
These techniques are literally _hundreds_ of times faster.

Try libco instead. It's the bare minimum four functions needed for cooperative
threading, which lets you easily build up all the other stuff these libraries
provide, if and only if you want them. And if you benchmark libco against all
of the other stack-backed coroutine libraries, I'm sure you will be stunned at
just how bad ucontext really is. That people keep using ucontext in their
libraries tells me that they have never actually used cooperative threading
for any serious workloads.

[http://byuu.org/programming/libco/](http://byuu.org/programming/libco/)

~~~
ezy
libtask isn't using the ucontext syscall (normally), just the ucontext_t
struct. If you look at the source code, it provides local assembly versions of
the context swapping code for the most common architectures.

That said, your library is quite nice in that it just provides the co-routine
wrappers and nothing else, which I appreciate. Also, your switch code has less
instructions (are there cases it doesn't handle?), but if you look at
libtask's code, it probably not going to be 100x faster. :-)

~~~
byuu
Oh ... I looked at libtask/task.c and found:

    
    
        static void
        contextswitch(Context *from, Context *to)
        {
        	if(swapcontext(&from->uc, &to->uc) < 0){
        		fprint(2, "swapcontext failed: %r\n");
        		assert(0);
        	}
        }
    

But it looks like you're right, context.c is actually defining its own version
of swapcontext to replace the system function that can use other backends. I'm
sorry for the mistake, thanks for correcting me.

> are there cases it doesn't handle?

No, it conforms to the platform ABIs. I even back up the xmm6-15 registers
that Microsoft made non-volatile on their amd64 ABI (compare to how much
lighter the SystemV amd64 ABI's switch routine is; the pre-assembled versions
are in the doc/ folder.)

Their own fibers library even has a switch to choose whether to back those up
or not, which I think is quite dangerous.

Still, nothing can top SPARC's register windows for being outright hostile to
the idea of context switching.

...

Oh, and it also supports thread-local storage for safe use with multiple real
threads.

> it probably not going to be 100x faster.

No, definitely not compared to his ASM versions. Even with a less efficient
swap, once you get past the syscall, most of the overhead is simply in the
cache impact of swapping out the stack pointer, so his will likely be very
close. In fact, even I had to sacrifice a tiny bit of speed to wrap the
fastcall calling convention and execute non-inline assembly code (to avoid
dependencies on specific compilers/assemblers.)

I do still strongly favor jmpbuf over ucontext for the final fallback, but
with x86, ppc, arm, mips and sparc on Windows, OS X, Linux and BSD, you've
pretty much covered 99.999% of hardware you'd ever use this on. That and
libtask lacks Windows and OS X support.

------
srean
Quick question: isn't channel's raison d'etre the fact that they need not be
backed by system threads, that they are considerably cheaper and one can fire
tens of thousands of them without worrying too much about performance. In
other words aren't they primarily intended as a tool for solving a concurrency
problem rather than a parallelism problem.

Although, as far as I recall Go does map a bundle of them to a thread and can
thus handle parallelism as well.

Just to clarify, this is not a complaint, I am trying to get a better idea of
these abstractions.

@tylertreat

> I believe you're confusing goroutines with channels. Goroutines are
> lightweight threads of execution. Channels are the pipes that connect
> goroutines.

That is indeed correct. I was thinking more in the line of fibres that meld
the concepts of a coroutine and a channel into a single abstraction. So the
idea is Chan connects threads rather than coroutines.

~~~
chc
Channels are useful because they allow for easy synchronous communication
between threads of execution. Whether the threads of execution are dedicated
POSIX threads or drawn from a thread pool or simulated by an event-driven
state machine (as in ClojureScript) is a different consideration.

------
edsiper2
One of our students[0] at Monkey Project Organization[1] through Google Summer
of Code[2], wrote a complete co-routine implementation for the Duda I/O[3] Web
Services stack.

The feature is called "dthread" or "duda-thread" which aims to expose Channels
and generic co-routines functionalities, you can check the following relevant
links:

a. The co-routine implementation:

[https://github.com/monkey/duda/blob/master/src/duda_dthread....](https://github.com/monkey/duda/blob/master/src/duda_dthread.c)

it does not need mutex or anything similar, short code but a lot of thinking
behind it.

b. Example: web service to calculate Fibonacci using channels:

[https://github.com/monkey/duda-
examples/blob/master/080_dthr...](https://github.com/monkey/duda-
examples/blob/master/080_dthread_fibonacci/main.c)

the whole work will be available on our next stable release, if someone want
to give it a try, just drop us a line.

[0] [http://blog-swpd.rhcloud.com](http://blog-swpd.rhcloud.com)

[1] [http://monkey-project.com](http://monkey-project.com)

[2] [http://www.google-melange.com](http://www.google-melange.com)

[3] [http://duda.io](http://duda.io)

~~~
sircomatose
That's very interesting work. I did a similar thing for Kore for its
background task implementation so that page handlers could talk to its tasks
it fired off and read its responses accordingly.

See example on
[https://github.com/jorisvink/kore/tree/master/examples/tasks](https://github.com/jorisvink/kore/tree/master/examples/tasks)

~~~
edsiper2
nice!, its good to see many people working on that area. We got a lot of
support from the author of LWan ([http://lwan.ws](http://lwan.ws)) project
during the development process.

------
arjovr
The circle is now complete... Plan 9 from user space implementation of
channels.
[http://swtch.com/usr/local/plan9/include/thread.h](http://swtch.com/usr/local/plan9/include/thread.h)

------
Rapzid
Here is some interesting reading:
[http://golang.org/src/pkg/runtime/chan.goc](http://golang.org/src/pkg/runtime/chan.goc)
Go's implementation
[https://docs.google.com/document/d/1yIAYmbvL3JxOKOjuCyon7JhW...](https://docs.google.com/document/d/1yIAYmbvL3JxOKOjuCyon7JhW4cSv1wy5hC0ApeGMV9s/pub)
Dmitry's "Go channels on steroids" post

~~~
tylertreat
Thanks for sharing, that's a really interesting read. Also would have been
very helpful while writing my own implementation, although maybe it's better I
didn't beforehand. It's more fun to work on a problem without any
preconceptions.

------
mseepgood
[http://plan9.bell-labs.com/magic/man2html/2/thread](http://plan9.bell-
labs.com/magic/man2html/2/thread)

------
BrandonM
I think you need to add

    
    
        pthread_cond_destroy(&chan->m_cond);
    

After
[https://github.com/tylertreat/chan/blob/master/src/chan.c#L1...](https://github.com/tylertreat/chan/blob/master/src/chan.c#L136)

------
copx
> #include <pthread.h>

That is not "pure C". "pthread.h" (POSIX threads) is not part of the C
language standard, it is a system-specific header commonly found on UNIX
systems. Windows has a different threading system for example.

~~~
readerrrr
I agree, OP could have used the C11 feature, <threads.h>

~~~
DSMan195276
To be fair, you'll probably have better luck finding people who can compile it
if you use pthreads.h vs. threads.h. IIRC glibc doesn't have threads.h
implemented, and I don't know of any C standard libraries that do. pthreads is
fairly universal even if it's not standard C, and at the very least it's much
more universal then threads.h is at the moment.

~~~
justincormack
Musl libc will have one in the next release it seems. It is the only one I
know of.

~~~
jevinskie
Correct, I was just going to chime in about this. Musl is _very_ impressive
with its compact size, correctness, Drepper-less-ness =P, and speed of
implementing new standards.

[http://thread.gmane.org/gmane.linux.lib.musl.general/5952](http://thread.gmane.org/gmane.linux.lib.musl.general/5952)

------
chi42
Pretty nifty. I like the names of the functions chan_can_recv() and
chan_can_send(). Made me think think of the old cooking show "Yan Can Cook."

------
WalterBright
Very nice. Any chance of Boost licensing it?

------
vhost-
This is pretty awesome. I do a lot of Go and C and this just might come in
handy. Thanks for sharing!

------
SeoxyS
How does this compare to using ZeroMQ inproc sockets?

