

Ask HN: Redis... but not in-memory? - leoh

I&#x27;ve grown to love redis quite a lot. Its data types are relatively easily understood, yet combinations thereof can be quite powerful. I would love to design an application using Redis&#x27; data-structure paradigms, yet I am concerned about limits regarding memory usage. Surely I could, at some interval, dump data to some other on-disk data store. Two questions then:<p>(1) what would be a good on-disk alternative data store?<p>(2) are there any alternatives to redis that are performant, yet have similar data models?<p>Of course, there is the VM route. But as the redis documentation itself says, &quot;An important concept to take in mind is that Redis is not able to swap the keys, so if your memory problems are related to the fact you have too much keys with very small values, VM is not the solution.&quot;
======
stevekemp
I was recently hit by a service running Redis exceeding RAM for the first
time. I looked around and found two compatible alternatives:

[https://github.com/mpalmer/redis/tree/nds-2.6](https://github.com/mpalmer/redis/tree/nds-2.6)

[https://github.com/ideawu/ssdb](https://github.com/ideawu/ssdb)

Beyond that I realized I only used get/set in my applications so I used a perl
module that understands the Redis protocol and wrapped it up as a network
service:

[https://github.com/skx/predis](https://github.com/skx/predis)

In terms of speed, writing the data to a local SQLite instance, it averages
3-4 times slower than native redis. I'm happy enough with that, although
obviously I'm sure that some other projects wouldn't be.

I realize this doesn't directly address your main questions but as it is still
fresh in my mind I hope it was slightly useful.

------
meowface
From what I know, Redis's virtual memory mode isn't bad at all. Odds are that
unless you have an absurd amount of keys or have very little RAM, Redis will
likely be able to fit all of the keys into memory. To help with this, use
short key names, and use collection data types (so one key may be able to hold
hundreds or thousands of values).

I'd say experiment with Redis, with actual testing, before looking at an
alternative solution. You can set a maximum memory limit in Redis's config and
see what performance is like. You can also have it prune entries once the
memory limit is reached.

At the scale at which Redis stores data, RAM is pretty cheap.

------
joeskyyy
I mean, nothing is really going to be as performant as Redis if you put it on
disk.

The closest thing I can think of off the top of my head is MongoDB. Mongo will
combine with on disk and in-memory storage. But you can do pretty much all the
same things within mongo, just slower. It might not be that huge of a
difference in performance depending on exactly what your doing of course. But
if you're doing something in Redis that is, say, doing 100k+ ops/sec, you're
highly unlikely to get that in mongo.

------
schmidtc
How much of the Redis API do you rely on? You could write a client library for
say any of a dozen data stores that mimics the redis API. Disclaimer: I've
only used redis for small side projects, and I'm sure some of the in-memory
data structures would be difficult to replicated on disk while maintaining
performance.

I use redis mostly for stream processing, keeping the latest few gigs of data
in memory while streaming the old data to mongo for archiving.

