
Transactions in Redis - DrJosiah
http://www.dr-josiah.com/2015/07/transactions-in-redis.html
======
joeskyyy
Interesting concept! Just curious if you have any reports on how this affects
the overall execution of a script, as well as the memory consumption and CPU
usage? (Not in a place the moment to test it myself :P)

~~~
DrJosiah
The existing implementation is a proof of concept. It works as advertised, but
any key that is written to as part of script execution gets dumped using the
same internals that the DUMP command uses.

For a handful of small keys, where EVALTXN/EVALSHATXN calls are coming one at
a time from your clients, the performance difference is relatively minor - on
the order of a few percent.

But for large keys, or large numbers of modified keys, because every key is
being dumped before being written to the first time (and because the current
implementation uses a singly-linked list to manage the rollback buffer,
leading to a linear search for whether a key has been changed), the slowdown
is substantial, and there is no upper bound on the slowdown.

But how can there be no upper bound on the slowdown? How big are your keys,
and how many are you changing? If you're dumping 5x string keys all of under a
few hundred bytes as part of each script execution, Redis is not going to have
a problem. But if you are dumping 1000x 1M element ZSETs, of course it's going
to be slow!

The point of the patch is to get people (like you) asking questions and
hopefully wanting to help. There isn't any good reason for Redis to not have
transactions, aside from someone not taking the time to do it. The current
implementation is crap. But the crap implementation is substantially better
than not having it.

The next step? Teaching data structures how to offer the ability to reverse
operations on themselves. Or fetching versions from caught-up slaves. Or (now
that fork() performance is good due to no transparent hugepages) pull the data
from a fork()ed child. Or write changed structures to disk in a secondary
thread in the background, force structure syncs on write (if there are
changes), and pull them from disk on rollback (as necessary).

There are several different ways that this can be improved substantially, and
the only thing stopping it right now is time and effort. I will be doing my
best to work on it further (I had been sitting on the original version of the
patch for 2 months because I didn't have time to work on it), but that doesn't
mean that Redis has to wait for me. I know there are people out there,
Salvatore included, who can help take the idea and turn it into something that
everyone uses and expects to be available, because the current state of
affairs is terribly unfortunate and a recipe for data loss/corruption (which
has happened to me).

~~~
sitkack
I'd save the commands to a buffer and apply them at the successful termination
of the script. See "batched staging", trying to reverse a data structure
already mutated will be problematic. This is actually how one should write lua
scripts in Redis anyway.

~~~
joeskyyy
I'd agree to this whole heartedly. At first glance (if I'm understanding your
implementation correctly), it appears like you're already going to use up the
same amount of space a buffer would anyway. In this case you're just tasking
the LUA best practice, but making it a feature within Redis itself.

Regardless I think you're really onto something that people want. There's
going to be a trade off one way or another, but the Redis community is all
about speed, so the faster and more efficient you can do it, the better (:

~~~
DrJosiah
I agree on the speed side of things... but see my reply to the other comment.
TL;DR; Buffer or not, mutations are necessary to be visible as part of script
execution, which precludes "only write at the end on success".

