
Concurrency and Redis - slig
http://santosh-log.heroku.com/2011/07/02/concurrency-and-redis/
======
eis
A lot of (especially web) applications suffer from race conditions. Luckily
for most of them the effects are either temporary and/or not severe.

It does not only apply to Redis but basically all data stores unless something
like transactions in RDBMSs are used and even then you can get into trouble at
times.

The last example still has a problem when an exception is thrown and the
'redis.del "setting_up_auth_token"' fails for whatever reason. Furthermore it
has a race condition if a process wants to read the content of
'web_service_auth_token' because it sees 'setting_up_auth_token' has been set
but the other process who is setting it up has not reached the redis.set line
for it.

The example itself is not very good as I'd rather do a little bit of extra
work (would not hurt here) than being vulnerable to failures.

Assuming the work really needs to be done strictly once - either because it's
a lot or the logic requires it - you would probably have to do something like
the following pseudo code:

    
    
      while (true) {
        watch("setup_key")
        v = get("setup_key")
        if v == 2 {
           break
        }
        if v == 1 {
           sleep(1)
           continue
        }
        multi()
        setex("setup_key", 10, 1) // 10s timeout
        if !exec() {
          continue // someone else got the lock in the meantime, try again
        }
        // we now have a temporary exclusive "lock"
        ... do the real work ...
        set("setup_key", 2)
      }
    

What does this do? Well, setup_key acts like an exclusive lock which has 3
states: non-existing, 1 and 2.

 _non-existing_ means we are the first, lets do the work. _1_ means the lock
is being held, sleep and check again. _2_ means the work has been finished.

The watch(),multi(),exec() work as a CAS (really, whytf is this not a single
command?) to atomically acquire the lock. If the lock is held by someone, we
just sleep 1 second and try again. I use a timeout on the key in case the
client holding the lock dies to prevent a deadlock.

But this is STILL not perfect. What if your setup work takes longer than the
timeout specified for the temporary key? I haven't spent enough time thinking
about it right now to come up with a solution if there even is any for redis,
so I'll leave it as an exercise to the reader :)

To be honest: with slightly different commands/semantics, Redis could be MUCH
easier to write correct applications for. Especially the "transaction" support
is mediocre imho.

Conclusion: Many web applications are inherintly and severely concurrent.
Concurrency is HARD to get right. There are many patterns like locks, atomic
operations, transactions etc. - all with completely different semantics. Redis
is singlethreaded for a reason :)

~~~
atamyrat
I've used counters in Redis for mutex control. If incr returns 1, it means we
have succeeded. Otherwise it decrements it and tries again after 1 second
until it suceeds.

Here is the code: <https://gist.github.com/1062584>

~~~
eis
What if the decr() fails because of for example a network issue? What if the
process who is holding the lock dies?

:)

~~~
atamyrat
Good point, never thought about decr failing!

------
lamby
Can't seem to comment on the blog itself, so:

> This code assumes the existence of a UUID library, which returns a unique ID
> on every call. Now, if more than one process were to run, they would each
> create their own temp_list’s

Wouldn't this still race in sense that a second update to the SafeIps table
may be overwritten by the atomic move?

(A inserts into SafeIps and generates unique temp list, B inserts into SafeIps
and generates another temp list, B moves its temp list to "safe_ips", A moves
its temp list to "safe_ips" => oops, we've lost B's insert.)

~~~
Detrus
You have to use Redis' INCR to generate the UUIDs.

~~~
eis
That does not help at all with the issue at hand.

The problem is not colliding UUIDs, it is a race condition between getting the
list of IPs and then writing them to redis.

~~~
Detrus
Oh, well in the pseudo(?) code he seems to push the content from temp lists
into the main list, so it seemed that as long as you have separate temp lists
for A,B you wouldn't lose anything, but I probably misread it.

But anyway, as the other comment mentions, there is some transaction support
<http://redis.io/topics/transactions>

I'm not sure how much Redis will focus on transactions but if it doesn't
you're out of luck or reinvent the SQL wheel on top of it.

