
Concurrency kit – Concurrency primitives and non-blocking data structures in C - misframer
http://concurrencykit.org/
======
Jweb_Guru
I have nothing but praise for this library... it's a great resource for
writing high-performance parallel programs.

------
virmundi
How does this compare to omp? It just started learning about this in my HTC
course.

~~~
exDM69
OpenMP is a bunch of compiler directives (and a runtime library) for easy data
parallelism. E.g. #pragma omp parallel for.

This library is a bunch of concurrent data structures (hash map,
producer/consumer queue, etc) for multi threaded applications. This is more
oriented to task parallelism than data parallelism (e.g. servers and such).
You'd use this library together with pthreads or other threading library.

Does this answer your question?

------
mingodad
I looked at this library in the past and I could not find any usage of it in
any open source project. That made me a bit scary, how a library like this
with the supposed well thought construction is not used anywhere ?

~~~
sbahra
It's used in a lot of mission-critical commercial applications which I think
is a stronger indicator. It just so happens those aren't open-source. This
ranges from mission-critical enterprise software to internal infrastructure
tied directly to revenue generation. There is a serious cost to failure there
(money, liability, jobs, etc...).

Few software uses these advanced techniques in general, and it isn't so much
about Concurrency Kit. There is a barrier to entry as folks have to invest
non-trivial amounts of time in learning more about these techniques.

Some open-source examples:
[http://www.machinekit.io/](http://www.machinekit.io/)
[https://labs.omniti.com/labs/reconnoiter](https://labs.omniti.com/labs/reconnoiter)
[https://github.com/Litres/turboxsl](https://github.com/Litres/turboxsl)
Aerospike

I'm sure there are others, if there's anything I'm missing, would love to
hear.

------
reza_n
It seems like a lot of these data structs are not thread safe. Other than
ck_bitmap which seems to be read/write thread safe, ck_array, ck_hs, ck_ht,
and parts of ck_ring only allow a single writer in the presence of many
readers. Did I misread something here? Still a lot of good stuff here, but not
sure if these data structures can be used without external locking.

~~~
alexforster
They're mostly all some combination of lock-free/wait-free single-producer
multi-consumer datastructures, meaning one thread can produce some data
(think, an epoll/kqueue I/O thread) and then many other threads can consume it
(think, threads that actually perform your server logic).

I've found that the "SPMC" concurrency model tends to fit into my pipeline
perfectly when I'm working on problems that need a specialized library like
this. CK is for achieving unimaginably fast data-parallelism, like the kind
you need when writing high performance drivers atop some DMA slab, or (in my
case) using userspace networking to do something useful with millions of
packets per second. Check out
[http://concurrencykit.org/articles/ck_hs.html](http://concurrencykit.org/articles/ck_hs.html)
for a better idea about just how fast this library aims to be (spoiler: tens
of nanoseconds per op).

------
Keyframe
OK, but where do I start?

PS I may be dense today.

~~~
justifier
if you already have a need of concurrency in your project and want to know how
to utilise the library there is a documentation section on the linked page

a good starting point would be the man pages

if you want to know where to start on getting info on how or why you would use
a library like this check out the articles or slides section, or wiki around
unfamiliar words in the man pages

~~~
Keyframe
I was hoping for a mini usage showcase / sample. Maybe I'm spoiled, but I was
looking through man pages and it was a meaningless dump to me as a newcomer to
the library. I then looked at the articles, but it was a long read I didn't
want to do at first.

~~~
sbahra
I agree. Such an article would be more emphasizing use-cases for different
advanced synchronization techniques in context of Concurrency Kit. You may
just not have a use for it too.

I'll put up a presentation soon that may be helpful in these regards.

------
gadrfgaesgysd
What a macro fest:
[https://github.com/concurrencykit/ck/blob/master/include/gcc...](https://github.com/concurrencykit/ck/blob/master/include/gcc/x86_64/ck_pr.h)
[https://github.com/concurrencykit/ck/tree/master/include/gcc](https://github.com/concurrencykit/ck/tree/master/include/gcc)

This might not be the intention of the author, but if you want someone to
use/hack with your library, please make it human-readable. Those macros reduce
the amount of duplication, at the cost of legibility. At least include useful
comments... /rant off

~~~
corysama
To each his own. As far as macro fests go, that file looks very reasonable and
legible to me. It's a single pattern repeated consistently for each function
family. Writing out a dozen variants of each function explicitly would not
only be much more error-prone, but would also be much harder to read and
understand as a whole IMHO.

