
Channels in Common Lisp - codr4life
http://vicsydev.blogspot.com/2017/01/channels-in-common-lisp.html
======
oskarth
Surprised that a thread about CSP in Lisp doesn't mention Clojure's
core.async. CSP in Clojure is implemented as just another library and it's
rock solid. While I don't claim to understand the implementation in detail,
it's one of the more interesting and high leverage uses of macros I've seen in
Clojure. It makes concurrent programming a breeze in Clojurescript as well,
despite JS being single-threaded.

As someone who programs in both Clojure and Go there's absolutely nothing I
miss from CSP in Go, and I would much rather do concurrent programming in
Clojure.

Rationale: [http://clojure.com/blog/2013/06/28/clojure-core-async-
channe...](http://clojure.com/blog/2013/06/28/clojure-core-async-
channels.html)

Code:
[https://github.com/clojure/core.async/blob/master/examples/w...](https://github.com/clojure/core.async/blob/master/examples/walkthrough.clj)

Presentations:
[https://github.com/clojure/core.async#presentations](https://github.com/clojure/core.async#presentations)

~~~
jgalt212
As I understand it, there are multiple ways to do concurrency in Clojure. Is
there a best way or a way that fits most use cases so that you should at least
try feature X before features Y, Z, and/or roll your own?

~~~
nickik
There is no easy path you can follow sadly. It depends on your use case quite
a bit. Clojure gives you a lot of options and you have to find your own way.

What Clojure gives you by default, are thread safe mutation constructs, atoms,
agents and refs that help a lot if you want to do simply concurrent things
without worrying about race conditions.

Clojure also has support to get some data parallelism, you can easily do a lot
with reducers and transducers. Its quite nice.

For more complex stuff, core.async gives you a powerful CSP library, its as
powerful as what the Go programming language gives you (I think its nicer
because Go has statements instead of expressions).

Because Clojure is on the JVM, you get access to all the Java stuff,
java.util.conncurrent, and you can get a lot of powerful tools there
(ThreadPools and stuff like that).

------
haspok
Sorry for the nitpicking, but:

"One of the things that Go (and Erlang) got right was using channels as the
main mechanism for synchronising and communicating between threads."

Erlang doesn't have threads and doesn't use channels either.

Erlang has lightweight processes, which might map onto OS threads (but user
code has no control over this).

Erlang uses async communication which is always non-blocking (of course, you
can emulate sync communication, the gen_server:call behaviour in the OTP
library does that, for example).

Otherwise, spot on! :)

~~~
zzzcpan
Only Erlang got it right, not Go. And channels were not even invented for real
life programming, they were just an easy notation for mathematicians. But an
interesting thing about concurrency is that the Erlang myth is actually true,
getting deeper into concurrency always leads to reinventing Erlang. The road
typically goes from synchronous multithreaded applications to asynchronous
single threaded to event loops to higher-order abstractions around events to
wrapping them into killable process/context abstractions to adding message
passing and to reinvented Erlang.

~~~
codr4life
Cool, just wanted to add that there's nothing wrong with reinventing Erlang
either. We're all in the same boat, ideas are universal.

------
wtbob
This illustrates the power & the problem of Common Lisp: the power is that it
really is easy to add channels (and pretty much every other feature) to the
language yourself; the problem is that it's generally easier to write your own
code to do this than to use (or fix …) someone else's.

I've experienced exactly this: I too looked at ChanL. In my case it had a bug
for one of my use cases, and it was honestly easier to roll my own channels
(with, of course, their own bugs) than to understand & fix ChanL's bug.

The problem is that this leads to a proliferation of different libraries, all
doing similar things. The power is that one really can write just about
anything, and it will work well enough for your use case.

~~~
reikonomusha
As you sort of imply, it doesn't help that each implementation is half-baked,
incomplete, and unmaintained after 16 months.

I've seen similar libraries that do the following, for example:

* Serialization and deserialization

* Promises

* Generators

* CPS

Somehow none of these "100 line wonders" make it into idiomatic usage, despite
their purported utility.

These channels as presented don't solve many problems well. They will be very
slow and contentious, they will be expensive to make, and they will not guard
against memory ownership issues because you're just passing pointers to
objects around, which message passing is supposed to solve.

But, it'll become another library on Quicklisp that 3 people use and that
people's applications will inadvertently depend on.

Lisp is a fantastic language for many things, including a plethora of
production applications, but it being a local optimum for exploratory
programming means we get lots of very incomplete, scratch-an-itch code that is
pawned off as a library intended for mass consumption. In fact, the author
links to his GitHub repository, where you find exactly this random assortment
of disparate utilities.

If I had my way, the rhetoric of this post would be closer to a pedagogical
exposition of channels, how they're used, what their benefits are, a proof-of-
concept implementation, and a non-trivial example application. I would
vehemently stay away from having the thesis be anything about Lisp or the ease
of implementing small POCs in Lisp.

~~~
codr4life
This implementation is yours to maintain, that's why it's simple enough to
make that possible. There is plenty of room for simple. They allow building
networks of cooperating threads, which is the purpose. Erlang isolates, Go
doesn't, in the end I like the choice.

~~~
reikonomusha
Copy-pastable blog post code is the antithesis of good software engineering
practices. DRY, encapsulation, reusability, broadly applicable abstractions,
and so on should ideally be heralded as the thing to strive for.

~~~
codr4life
I don't agree. As long as you understand the code and it solves your problem,
owning is an advantage. Why is it so important to you that no one steps out of
line and tries to use their brain as more than an answering machine?

~~~
smallnamespace
> As long as you understand the code and it solves your problem, owning is an
> advantage

Because the cost of acquiring an understanding of code that solves your
problem might be much higher in Lisp than other languages, if the hypothesis
is correct that the language facilitates the creation of bad half-baked
libraries.

Just to give a somewhat figurative example, the jungle is a much more hospital
environment for life in general than temperate environments (it's warmer, full
of nutrients, bathed in constant solar energy).

However most humans didn't manage to really expand into the jungles
successfully until the advent of modern medicine, because the life there was
so prolific that there were all sorts of nasty parasites and diseases, and
everything you build sinks into the jungle within years.

Lisp is like that jungle -- everything just grows fantastically easily, but
paradoxically that makes it harder to separate the chaff from the wheat.

~~~
codr4life
But these are symptoms; you're still basically arguing that Lisp gives the
user too much power. And that's a valid perspective. I don't agree that a
language can give the user too much power. Building tools for someone who's
supposedly not as smart as your self is a great ego boost, and that's about
it.

------
jsjolen
I think a lot of people in this thread are being absolutely ridiculous. It's
not easy to implement channels in Lisp because of Lisp, it's because the work
is already done for you.

What mainstream language doesn't have structs, condition variables, locks,
threads and... Wait, I think that was all of it? Oh, and linked lists. I mean
come on, you could do this just as easily in Java.

And all this talk of the Lisp curse, Jesus Christ. I mean, there's a point in
that there are a lot of lackluster libs (but what language doesn't have that),
but Quicklisp and a switch in Lisp culture means that there's a large amount
of collaboration (and usage of libs) between programmers. There's the
potential to start to use the same libraries and a will in the community to do
so.

~~~
junke
Thanks.

------
lispm
The book by Hoare on CSP
[http://www.usingcsp.com/cspbook.pdf](http://www.usingcsp.com/cspbook.pdf)
published in 1985 actually used Lisp:

> The proposed implementations are unusual in that they use a very simple
> purely functional subset of the well-known programming language LISP. This
> will afford additional excitement to those who have access to a LISP
> implementation on which to exercise and demonstrate their designs.

------
maxekman
Really interesting read for me as an experienced Go developer currently
learning Lisp. Not sure if I will ever use Lisp or this for a real project, I
see it more as some kind of Sudoku in code, and a indirect training ground for
Elixir (which is a wonderful complement to Go for backend tasks).

~~~
progman
I use Common Lisp for real projects. My recommendations:

\- Always follow the KISS principle
([https://en.wikipedia.org/wiki/KISS_principle](https://en.wikipedia.org/wiki/KISS_principle)).
Don't use Lisp's "super features" too much. The same applies to Haskell, by
the way.

\- Use QuickLisp as much as possible. In many cases you don't need to add your
own lib. Read Quickdos [QDC].

\- Comment your code well, even if others don't ever see it lest you could not
be able to understand your own code later. What's the purpose of the function,
which arguments are expected, of which expected type are they, what are the
possible results, possible side effects, etc.

\- Read and understand how Lisp's package system really works. This is
important to avoid strange behaviors in your own code.

\- Use SLIME ([https://common-lisp.net/project/slime/](https://common-
lisp.net/project/slime/)). It is an awesome interactive debugger which catches
bugs and presents different options how to deal with that. SLIME is also
useful as manual for Hyperspec
([http://www.lispworks.com/documentation/HyperSpec/Front/](http://www.lispworks.com/documentation/HyperSpec/Front/))
which is a basic tool to look quickly for code examples.

\- Don't use Hyperspec too much because it is really confusing. Use proper
reference manuals like Peter Seibel's Practical Common Lisp [PCL] and Edi
Weitz' Common Lisp recipes [CLR]

\- If you cannot get used to Emacs (which is a wonderful tool with a steep
learning curve, really the best editor ever) then you can use proprietary
IDE's like LispWorks or Franz Allegro.

[QDC] [http://quickdocs.org/](http://quickdocs.org/)

[PCL] [http://www.gigamonkeys.com/book/](http://www.gigamonkeys.com/book/)

[CLR] [http://weitz.de/cl-recipes/](http://weitz.de/cl-recipes/)

~~~
maxekman
Thanks for all the advice! Really helpful that you aimed it towards real world
usage.

------
farray
I once tried ChanL but it was buggy, and concurrency bugs are the worst to
debug.

lparallel on the other hand was solid to me, though I didn't like its API and
had to (trivially) build my own message-passing abstractions on top of it.

~~~
fiddlerwoaroof
I've been a bit interested in this, basically an attempt to bring some of the
nice things of the Erlang/OTP platform to CL:

[http://mr.gy/blog/erlangen-intro.html](http://mr.gy/blog/erlangen-intro.html)

