
Hacking Redis: Adding Interval Sets - jacobparker
http://www.starkiller.net/2013/05/03/hacking-redis-adding-interval-sets/
======
jimktrains2
Just to add, the redis code base is pretty clean and straight forward. I was
able to digest enough of it to implement a multi-user system based on the
redis databases[1] (1 db per user; I've never seen more than one used anyway).
antirez and co do a good job at keeping it readable and understandable, which
is awesome and I'm sure is part of the reason it's as reliable as it is.

[1] <https://github.com/jimktrains/redis/tree/multiuser>

~~~
hox
I completely agree. One of my more immediate projects is to possibly find a
way to refactor some of the work I outlined in the article to make adding
datatypes even more straightforward, but I am very concerned on adding
complexity to the Redis codebase rather than removing it. Sal has done a great
job of making it easy to understand the code of relatively powerful utility.

~~~
dvirsky
I thought this thread might be a good place to write about a little project
I've been working on for the past few months (in the shadows so far):

I've written an experimental abstract database loosely based on redis (or its
base concepts) that is entirely based on a plugin system; and it's written in
Go.

I love redis to bits, but it always bothered me that it doesn't have and
doesn't seem like it will ever have a proper plugin system. So what I did was
take the redis protocol (you use it via redis-cli or any client) and a few of
its basic ideas: key=><data struct>, snapshot persistence, the overall command
syntax idioms, master/slave replication, pubsub - and stripped everything but
the very basic system commands. Oh, and also the concept of single
threadedness seemed to be wasted on Go.

The rest is implemented inside isolated plugins that do not need to know about
network or disk, and are just responsible for managing creation, serialization
and manipulation of data structs. PubSub and such are achieved via Go
channels, and locking is done on a key level, with reader/writer locks. The
rest is pretty straight forward.

Besides some basic commands (strings, hashes, pubsub, monitor), I've added
prefix search and json storage with sub node manipulation, with (relatively)
very few lines of code. Although Go doesn't support dynamic loading of
libraries, the idea is that you compile the database with the plugins you
want, since Go's compiler is amazingly fast.

I haven't opened it yet, and pressure at work prevents me from progressing
lately, but if anyone is interested in collaborating on this, I'd be glad to
write about it in more detail and/or put it out there.

~~~
hox
Sounds interesting, to say the least. Bill (llimllib above) was actually
considering writing a Redis implementation in Go as a learning exercise right
after we finished with the interval tree stuff.

I personally would be interested in the progression of the project. But yes,
finding time for yet another project is always fun...

~~~
dvirsky
Thanks. Yeah, it was a great learning experience about Go, and it also turns
out to perform pretty well (50%-90% of the throughput of redis, using redis-
benchmark). My hope is to use it for a work related project soon to put more
development time into it.

But I'll try to write up some docs and put it out there, incomplete as it is,
it beats having it sit in a closed repo.

------
yankoff
Very cool. Did you guys consider achieving the same functionality with Lua
scripting that Redis 2.6 supports? I wonder what are the tradeoffs and if that
was much slower.

~~~
llimllib
Yup, Ken didn't mention it but we didn't undertake the work until we had shown
that an implementation with sorted sets and an implementation with lua were
too slow.

