I'm not sure if you know what CRDTs are but they're a family of data types that allow different actors in a distributed system to edit data concurrently even during network partitions. If enough data is shared, they will deterministically agree on the same value. They kind of give that "google docs" behavior if you're looking for an analogy. They're perfect for peer to peer and offline-first systems.
However there is actually more to it, and a much more detailed write up is coming soon. Ditto is a distributed database, each peer has it's own database. The database is organized into collections and each collection is a Ditto Document (this does not work like most NoSQL document databases). Each property of the document is it's own CRDT, you as the user can pick which CRDT you'd like to use, our current catalog includes:
* Registers (Causal Last Write Wins)
* Counters (sums of each writer's numeric values)
* Binary Attachments (same as a Register but you can put large arbitrary data like say video files, images, PDFs whatever)
* AddWinsMaps (coming soon) - This type allows for concurrent upserting and removing of values based on a key.
* ReplicatedGrowableArray - this is an array type that allows for concurrent insertions while preserving some semblance of order. It behaves rather closely to a collaborative text editor merge behavior.
Our AddWinsMap and ReplicatedGrowableArray are more special than you might think. They can actually host nested CRDTs. Think of it like a folder within a folder in Google Drive that can hold synced documents nested within.
I'd love to show you over perhaps a call! We tend to be perfectionist when it comes to documentation and have been so busy that we haven't fleshed it all out. Perhaps we might just open source our CRDT system.
Email is in my profile, I love chatting and sharing about this stuff!
A CRDT is a way to solve multi-leader replication without having the application code resolve conflicts.
This is what is required to build an app where any instance (node) can be offline for an arbitrary amount of time, but still be able to share state with the rest of the nodes when it's reconnected.
To implement this, every application node keeps a vector clock per register (an atomic piece of shared state). The vector clock allows any node the compare its own version of the register with the state received from any other node. Two values of a vector clock can either be causally related (in which case the most recent write wins) or concurrent. However the concurrency is from the system's perspective, but not necessarily from the user's perspective. An extra physical timestamp can be kept at the register level to order concurrent updates in a way consistent with the user's time perception.
Now, having the hybrid clocks in place to version each register on each node, the system must implement a protocol to ship every register update to all nodes (reliable broadcast).
Once all updates are shipped to all nodes, it's guaranteed that all nodes have the same (most recent) state.
(I built an offline-first product and had to roll my own protocol)
I've read the CRDT paper but never implemented it. Question - if you're not using LWW (instead you have concurrent values of a vector clock), this is where you have your CRDT and merge the states coming from every node?
Can you go into more detail here?