
Show HN: rqlite v2.2.2, replicating SQLite via Raft consensus - otoolep
https://github.com/rqlite/rqlite
======
tyingq
Well done. I am curious, though, if you tried implementing this as a sqlite
VFS. That would allow it to be more plug-n-play with existing software that
already talks to sqlite.

I suppose you would have had to hack up something to map offset ranges into
key/value pairs, but at a high level, it seems feasible.

~~~
sargun
I built one recently. We have a SQLite VFS built on top of Zookeeper for doing
metadata-like stuff. We can pull between 1000-100000 reads per seconds out of
it.

I realize that this is a total abuse of what Zookeeper does, and how it is
supposed to work, but hey, this is a solution that's _simple_ and _correct_.

Here are the benchmarks comparing the traditional VFS with the ZK VFS:
[https://gist.github.com/sargun/7503ba636c1a0eb2e2f5cda7a6a23...](https://gist.github.com/sargun/7503ba636c1a0eb2e2f5cda7a6a23db2)

PS: Feel free to ask me any questions

~~~
otoolep
Pretty interesting. Do you have source code you can share?

~~~
sargun
Not yet open source, and it probably wont be for a while, given it's in
$EMPLOYER's proprietary product.

------
zmmmmm
Is this a potential way to work around the single-writer nature of SQLite?
It's probably the main limitation I hit when using SQLite, which otherwise
fits a lot of jobs very well. Obviously you would not want to use it for
situations where there is contention, but SQLite locks the whole database even
if you are writing to completely separate tables.

~~~
otoolep
No, unfortunately not, as every change must go through a single node i.e. the
leader. The same limitations as a standalone SQLite database apply.

------
ganitayu
_rqlite ensures that every change made to the database is made to a quorum of
SQLite files, or none at all._

So that means if a single node in the cluster fails then everything stops?

~~~
otoolep
No. As long as (n/2)+1 nodes are up, where n is the size of the cluster,
everything continues to work.

------
asimuvPR
Stupid question: What if I want the databases to be built with optional
extensions (like the json extension)? Is there a way to deal with such
scenario?

Cool project!

~~~
otoolep
I'm not really sure I follow you. But if it requires compiling other stuff
into the source code, probably not. The SQLite source is built along with the
SQLite Go layer, without that control AFAIK.

~~~
asimuvPR
Ok, that answers it perfectly. Thank you!

------
alexnewman
I don't think now works. Great idea though! Great demonstration ifnot nothing
else. Can we do one over 2pc as well?

~~~
otoolep
Yeah, certain non-deterministic functionality can't be used. This is called
out here:

[https://github.com/rqlite/rqlite#limitations](https://github.com/rqlite/rqlite#limitations)

Though with now() you could calculate that before you issue the query.

------
FatAmericanDev
This is cool. On a related topics, I'm still waiting for a sqlite for graphs
to emerge.

