

PyPy STM work update - fijal
http://morepypy.blogspot.com/2012/04/stm-update-and-thanks-everybody.html

======
seunosewa
People think optimistic STMs are faster than locks, because "locks are slow",
but optimistic and pessimistic approaches are roughly equivalent when latency
is low. Under low contention, both locks have low overhead, just like
optimistic STMs. Under high contention, locks have high overhead, and so do
optimistic STMs (due to wasted work).

Pessimistic STMs are built on locks. They make code involving fine-grained
locking composable. So you can have many more locks instead of a global one,
and recover gracefully from read/write conflicts and deadlocks. When latency
is high, e.g. in distributed systems, optimistic strategies are more
efficient. When latency is low, e.g. in SMP memory, pessimistic strategies
more efficient. So pessimistic lock-based STMs are generally more efficient.

This implies that STM is not really a faster alternative to locking. It's a
simply form of fine-grained locking that composes well. I believe STM will not
eliminate the slowdown that results from fine-grained locking in freely-
threaded Python interpreters, but it might just make them easier to write,
which isn't a bad thing.

------
kbd
> On a few smaller, more regular examples like richards, I did measure the
> performance. It is not great, even taking into account that it has no JIT so
> far. Running pypy-stm with one thread is roughly 5 times slower than running
> a regular PyPy with no JIT (it used to be better in previous versions, but
> they didn't have any GC; nevertheless, I need to investigate).

Oy. Armin is hopeful that STM will become for concurrency what garbage
collection is for memory management. I'm really hopeful he's right. While I
know there's still lots of work to be done, that kind of speed hit is,
however, discouraging :(

~~~
sp332
Even if it's 5 times slower on one core, it might make scaling to 16+ cores
possible in some cases where it wasn't before. That at least gives you the
option of throwing hardware at a problem to speed it up.

~~~
seunosewa
The last attempt to remove the GIL was just 2 times slower on one core, and it
was rejected.

~~~
oconnore
Do you have a link?

That seems incredibly shortsighted.

~~~
sanxiyn
There were many attempts. The last one I remember is
<http://code.google.com/p/python-safethread/>

