Yeah, that blog post is also me :) hehe. Anyway glad to hear you find it interesting. If you want to help out ping me. There's tons of stuff on the roadmap, and currently I'm almost entirely focused on the secondary index.
I still see some issues with it:
1. redis being single-thread, where inverted-indexes are ~easily sharable by cores (i think based on what I've read). This making hotspots easier(since it's a single core not single machine)
2. sharing of data(terms) between multiple indexes on 1 server (like in elasticsearch you use _routing, but all things are in 1 index, though some people like separate-per-user like dropbox does)
3. cluster not fully nice yet (losing writes)
4. no option to merge from different nodes (or even redis-api to do so as far as I know)
Salvatore just added support for asynchronous operations in side threads in modules. This would allow merging results from the cluster possible, and I want to get to it soon. I'm now implementing it in the client side and it works great, but I want it to be as transparent as in elastic.
The recent additions would also allow more parallelization of the indexes for reads. I could create RW mutexes in the engine and allow multiple clients to work on the same term index. It won't be trivial but it's not super hard as well.
Anyway, if you want to continue the discussion further, we can take it to reddit (I haven't posted the benchmarks there yet, it's a good idea anyway), or just email me, dvir at redis labs.
