Hacker News new | comments | ask | show | jobs | submit login
Show HN: RedixDB – a fast persistent key-value store with the Redis protocol (alash3al.github.io)
75 points by alash3al 27 days ago | hide | past | web | favorite | 24 comments

Badger is an lsm-tree storage engine which makes some unusual tradeoffs. Good background info here:


Relatedly, there are some active efforts to make a Redis on top of TiKV, which is a distributed persistent Key-Value store with transactions.

Titan: https://medium.com/@shafreeck/titan-a-distributed-redis-prot...

Tidis: https://github.com/yongman/tidis

A somewhat more accurate title, in case it saves someone else the time I just spent: "RedixDB - a Redis frontend to the Badger key-value store".

So, mysql is a frontend to InnoDB? Cockroach is a frontend to RocksDB?

Note: Redix has a pluggable storage engine, the current engine is "badgerdb", the next is "bolt" and it is already integrated and I'll release it soon.

So, mysql is a frontend to InnoDB?

I would put it the other way around: InnoDB is a backend for MySQL. But an SQL database is far more complicated than a key-value store, and has a lot of "middle end" between the "talk to the client" front end and the "store stuff on disk" back end.

I'm not saying that RedixDB isn't interesting -- having a network front-end to Badger seems like it would extremely useful for most people who want to use Badger. It's just that I spent a while digging through the code trying to figure out where the back end was and what data structures you were using, only to discover that you didn't have a backend (or rather, you're using a pre-existing one).

Second that. It would help if you included on the website some info about the technical implementation. That would help instill confidence in the solution.

None of these backends currently fsync() correctly on MacOS, due to a Go bug fixed in 1.12.

To split hairs, the bug is in MacOS - its fsync() does not flush data to stable storage. Go 1.12 includes a workaround for this MacOS bug.

To split further, it's a MacOS design flaw, with a published workaround. It is a bug to fail to use the workaround :-)

I don't know if Apoke really acknowledges it as a bug or flaw. Their man page for fsync reads like an intentional misunderstanding of fsync semantics.

There is also Facebook's MyRocks which is MySQL frontend with RocksDB backend. It has lower write amplification than InnoDB, which is the main reason why it was created.

RedixDB appears to provide a lot of new functionality that is not available to a plain Badger database.

I disagree that Badger plays such a big part in this project that it needs to be included in the title. Badger is only the storage component and it can be easily swapped out for an alternative.

Do you suppose it will ever be possible to support transactions with MULTI?

Is the DB strongly consistent?

Are there performance gains compared to Redis due to multi-core?

1- Multi: yes, I'm planning to support it in the next releases.

2- Consistency: Yep, also there will be test cases soon. 3- Performance: at first, nothing will be faster than RAM, so don't expect that an on-disk store is faster than an in-memory store, but that doesn't mean that on-disk stores are very slow, I selected two storage engines called bolt and badger, badger is the default because it follows RocksDB design and rocksdb. for redix server compared to redis server, redix doesn't use an eventloop, but each connection runs in its own light-weight thread, so there no possibilities that you run a command and block the server itself, another note is: the underlying implementation of internal datatypes makes redix faster in some cases especially when you try to load large datasets, you will notice that redix has very low response time than redis, that means, in a highly loaded environment you find redix performance better than redis especially with the coming version because redix will lower the memory usage more and more. Every day I improve it more and more, and thanks for the issues creator on the repo, they're helping me too in organizing my thoughts.

> for redix server compared to redis server, redix doesn't use an eventloop, but each connection runs in its own light-weight thread, so there no possibilities that you run a command and block the server itself,

How does this jive with the “happens-before” transactional guarantees of Redis? Since it’s single threaded it’s easy to reason about the order of operations. Once an operation begins subsequent ones can assume the full impact of the changes.

In Redix, if a “big write” happens to a key in one connection/thread, what does a different reader see for the value of that key if they issue a read after the write begins but before it completes? The old value or a wait followed by the new value?

Any suggested/safe backup/restore strategy?

Currently I didn't create commands for that, but you can easly do the following:

- Backup: Copy the storage directory as it. - Restore: stop the server, move the previously backup data directly to redix storage.

It is very simple, but I may add commands that will help you with that operation.

Nice work on this. I wish more databases used the Redis protocol as it would simplify the numerous different interfaces out there. There are so many key/value systems out there yet many that aren't very usable outside of specific embedded use-cases.

Thanks for this!

Is there a suggested way to get a list of keys based on wildcard? I note that the infamous KEYS command is not listed as supported (and fair enough). Is there a suitable non-Redis backend command that can get this info?

Noted, and will do it or a similar one in the next release :), v1.6 is now released, and 1.7 will include a support for your case :)

Cool, I did this several years ago. You can check LedisDB or RebornDB. But now I switch to TiKV, hope you can support it later :-)

Thank you ^_^, Greg Weber from PingCap mailed me and I told him that I'm planning to support TiKV :)

Do you plan to support sentinel and replication?

I started redix with redis protocol to make use of redis ecosystem out there. There is also a plan to make HA a native feature in redix with no external requirements, so I can keep redix a standalone DB server.

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact