

When to Redis - transmit101
http://www.paperplanes.de/2009/10/29/when_to_redis.html

======
antirez
Related: Redis 1.1 (currently in beta but ready for some test) implements a
sorted type supporting range queries. It's pretty useful when the need is to
take million of items sorted with very fast top-N items operations or items
with score between min and max, especially in presence of continuous updates
to the items scores.

If you are interested check
<http://code.google.com/p/redis/wiki/CommandReference> (All the commands
prefixed with "Z" are about the Sorted Set data type). Btw both insert /
remove / and update-score operations are O(Log(N)). Top-N items is O(N) since
the list of items is already sorted.

Redis 1.1 will be released as stable in one or two months at max.

For data structures and algorithms geeks: the sorted data type of Redis is
implemented with a skip list, hacked to actually build a doubly-liked list,
with the backward pointers being only at level 0 (we have reverse range
operations, ZREVRANGE). Every element is also taken inside an hash table, so
that it's possible to update the score of an element already inside the sorted
set in O(Log(N)) time because we can get the old score in O(1) from the hash
table, and then find the element in the skip list.

Even when the scores are not distributed but there are large clusters of
elements with the same score, the update is still O(Log(N)) thanks to some
interesting trick.

------
transmit101
Redis is particularly useful for implementing simple message queueing systems
where the full functionality or power of something like RabbitMQ is not
required.

For example, when parsing incoming data into a database, push incoming data
onto one end of a Redis list, and have a seperate worker process popping them
off the end and into CouchDB, MySQL or whatever.

The fact that Redis list operations are atomic mean that multiple worker
processes (possibly on seperate physical boxes) can process jobs
simultaneously.

Result: the internet is entirely decoupled from both the hard drive and the
database, and extremely high performance is possible.

Twitter Streaming API, anybody?..

------
tptacek
There was a post earlier in the week about why Redis was "awesome" and a
"lifestyle" and I posted a kind of snarky comment about why posts like that
scare me away from Redis.

After this post, I had an hourlong discussion with my lead dev on swapping out
memcached for Redis so we could use it for our job scheduler. This post makes
an excellent, compelling case for Redis.

Just an interesting example of how two different evangelistic approaches can
play out. I don't know if I'm representative of "normal people", though.
(Don't say it.)

~~~
silentbicycle
Indeed. I went to a CouchDB presentation at a local conference, and afterward
I asked about some transaction-related details. The presenter didn't
understand, but kept repeating that the technology behind it was "way sexy".
It was useless _and_ off-putting.

~~~
janl
Which conference was that?

~~~
silentbicycle
A BarCamp in Grand Rapids, MI, US.

------
almost
Redis is really cool, I can think of lots of uses for fast and lightweight and
there's at least one project where it would have saved me a lot of effort if
it had existed then!

Watch out for the Python library though! It doesn't even try to sanitize key
names so malformed names cause all sorts of problems (including executing
arbitrary commands). Plus unicode strings with non-ascii chars cause it to
blow up.

All very easy to fix of course, as soon as I get a chance I'll submit a patch
if it hasn't already been done by someone else by then.

~~~
bham
Please submit a few reproducible cases to the mailing list. I'm sure Ludo or I
(or you) could even patch it quickly. Thanks!

~~~
almost
I've committed a quick fix for the problems to me clone of the Git repo here:
<http://github.com/almost/redis>

Check the changed doctests for examples of why it was a bad idea not to
sanitize key names and other params. Ignore the changed doctests involving
Decimals, I had to change those to get them to pass on my machine but they're
not related to me fixes.

------
chubbard
Is Redis stable enough to replace using a database as persistent storage.
There seems to be some indication that it is, but then there are paragraphs in
the documentation that makes it seem like it's not. Particularly if your
database size exceeds the machine's memory then you could lose data.

Is one of the goals of the project to replace the need to use a database? Or
will it always be more suitable cache or worker queue replacement?

~~~
antirez
Hello, the documentation should be updated indeed. Now Redis supports the
maxmemory directive that will start to reply with an error on write commands
when the database is using more than the specified bytes of RAM.

