

Redis-dump: Backup and restore Redis with JSON - delano
https://github.com/delano/redis-dump#

======
antirez
It's a good effort, but it is not the right approach IMHO. Redis is very good
at producing point-in-time snapshots via RDB files, so I think the right thing
to do is to create a tool, or add this feature to redis-cli, so that it is
able to translate RDB files into JSON and the other way around.

Also I think that a "generic" JSON is not a good idea, since a single JSON
value can be a multi million elements list or alike. The best thing to do is
to create a JSON that is also specifically conceived to be both valid but to
also have a single element per line, so that can be parsed much faster and
with minimal state without a real JSON parser if needed.

~~~
delano
Salvatore, thanks for taking a look.

 _or add this feature to redis-cli, so that it is able to translate RDB files
into JSON and the other way around._

Absolutely, that would be fantastic. I'm not a C guy so I wrote something
quickly in Ruby to solve my problem.

It does do two things that aren't possible with RDB though: it can backup by
database and/or a filter on the keys and it can output to STDOUT so it can be
piped to other processes (I use it to encrypt the data without an intermediary
file).

I use it in production but it's a proof-of-concept more than anything.

~~~
antirez
Thanks for the reply delano, actually it is possible to build both features in
the RDB -> JSON dump (filter by key pattern, type, single DB, and so forth).
But that would be much faster.

Your approach is also interesting but IMHO the main limit is that with the
current Redis API there is no robust way to iterate keys in a good way if the
key space is very big, without blocking the server and using a lot of
additional memory, and it is also not possible to easily get big sets in
ranges. Also what happens if somebody is writing to the DB while you need the
dump? It is no longer point-in-time.

Btw since this is an interesting feature to have I'm adding a feature request
in the github issues for a redis-cli feature able to do this work.

EDIT: feature request added -> <https://github.com/antirez/redis/issues/288>

~~~
delano
_actually it is possible to build both features in the RDB - > JSON dump
(filter by key pattern, type, single DB, and so forth). But that would be much
faster._

Absolutely, that would be awesome and definitely a lot faster.

 _feature request added_

Thanks for doing that. I'll continue my reply there.

------
redbad

        def dump filter=nil
          filter ||= '*'
          entries = []
          each_database do |redis|
            chunk_entries = []
            dump_keys = redis.keys(filter)
            . . .
    

<http://redis.io/commands/keys>

_Warning: consider KEYS as a command that should only be used in production
environments with extreme care. It may ruin performance when it is executed
against large databases. This command is intended for debugging and special
operations, such as changing your keyspace layout. Don't use KEYS in your
regular application code._

~~~
delano
Yep, good eye. It's not an ideal approach but using KEYS is the only option
for a non-specific tool.

The right solution is to have something similar build into redis-cli (see
antirez's link above).

------
midnightsun
Why not just copy the rdb file?

