
A few key problems in Redis persistence - wglb
http://antirez.com/post/a-few-key-problems-in-redis-persistence.html?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+antirez+%28antirez+weblog%29
======
fizx
I guess you're in the right spot when everyone else is writing the love
articles and you have to write your own hate. :)

------
jules
Could you memory map your memory to disk and let the kernel take care of it?

~~~
reitzensteinm
The reasons for not doing that are discussed here:

<http://antirez.com/post/redis-virtual-memory-story.html>

~~~
jules
So basically:

1\. Redis in memory data structures have poor locality, which hurts a lot when
a "cache miss" means going to disk.

2\. Redis in memory data structures take up to 10x as much space as the on
disk structures, so writing them to disk verbatim wastes a lot of bandwidth.

So apparently using memory mapped files wouldn't work for Redis. But what if
you used data structures that do have good locality and are optimized for
space usage? Has this been tried? I think it is an interesting approach and
has the potential to outperform Redis' approach, especially for bigger
databases. If your data structures use say 2x as much space as Redis' on disk
structures, then you would still be able to fit much more data in memory than
Redis because you don't lose half your memory as described in the OP, plus the
data structures are more compact.

