

MCS locks and qspinlocks - signa11
http://lwn.net/Articles/590243/

======
kev009
It's kind of interesting, Linux is perhaps the penultimate example of standard
"critical section" based kernel design. Thanks to brute force from the likes
of SGI and IBM this has been proven to around 4096 thread system images.

I wonder how hard it will be to get to the next level. I wonder if it will
continue to be logical structures on top of critical sections (I think RCU
kind of fits this characterization), or if currently academic solutions of
radically different design pay enough dividends that they start to make sense.
And if so, what are some of those possible solutions? Perhaps there's
inspiration from HW disciplines that can serve as inspiration?

~~~
bmm6o
"penultimate"? What do you consider to be better?

~~~
kev009
Vocabulary error s/penultimate/ultimate/

------
nkurz
How does this handle the case where a waiting process is killed before the
lock becomes available? Is there a kernel cleanup procedure at process
destruction as a backstop?

~~~
caf
Spinlocks are only used for low-level locking, and held for very short
periods. A killed process won't die while spinning on a spinlock (or while
holding one, of course) - it will find out the next time it goes into an
interruptible wait or returns to userspace.

------
Flow
If they got an improvement of 116% perhaps a spinlock isn't the right
solution? It sounds more like the subsystem protected should be autonomous and
respond to messages in a message queue instead.

~~~
exDM69
> If they got an improvement of 116% perhaps a spinlock isn't the right
> solution? It sounds more like the subsystem protected should be autonomous
> and respond to messages in a message queue instead.

You need some kind of a locking mechanism to be able to build a message queue.
And you need some kind of spinlock to be able to implement a proper locking
mechanism.

Lock-free queue mechanisms can be constructed using atomic operations but they
exhibit the same problems as spinlocks (moving cachelines from cpu to cpu) and
there needs to be special handling for cases when the queue is empty or full.

In the case of kernel programming, there is a lot of code that must be able to
run in an interrupt handler context and other contexts where you can't rely on
a scheduler for waiting or use any high level constructs.

A spinlock is one of the most fundamental building blocks for synchronization
in a multi-core system. It is difficult if not impossible to build a kernel
without some kind of spinlocks.

Message queues are a good high level construct and the kernel is full of
different kinds of message queues, but they can't function without spinlocks.
Or do you have a suggestion how would you get a packet from the Ethernet
adapter interrupt to the IP stack using a message queue without locking?
(edit: this might be a bad example)

~~~
kev009
For your last para, I think it is actually pretty common to use a lock free
ring buffer. An atomic instruction takes the place of locking for swapping out
data pointers. In fact I think modern adapters write directly into a buffer
(DMA) and atomically update the data pointer in the ring.

Not an expert on these topics, feel free to correct. But lockless programming
is pretty much critical section programming, just offloading the critical part
to the consistency model.

~~~
exDM69
> For your last para, I think it is actually pretty common to use a lock free
> ring buffer. An atomic instruction takes the place of locking for swapping
> out data pointers.

I think you're right, that particular use case is perhaps not the best
example.

Ethernet -> IP is a single producer situation where the Ethernet driver is
free to drop a packet if the input buffer is full. So that can be implemented
without locking.

