This is an interesting idea, but it's not clear to me that this is going to work at sufficient scale. This isn't clear to me because instead of getting a decent white paper on the implementation, we need to learn how this protocol works through what amounts to a PowerPoint. Sure, the slides are nice and simple, but I need more concrete answers on things like how consensus is determined and how available nodes recover from a disaster.
The slides suggest that a sufficiently complex system will almost immediately get out of sync. The dependency on upstream nodes asserting state about objects before they are acknowledged and successfully applied by the "owner" node means that some subset of nodes will be working on assumptions that haven't been validated by objects they don't in fact own. This seems to conflict with what prior slides say about who can mutate objects.
I think distributed object management is a useful concept, but instance state needs to be owned and managed by multiple nodes for redundancy in anything remotely approximating a production-level system. That doesn't seem to be readily addressed anywhere, either. I'd love to read more about this with the level of detail you'd need from a system that readily has use cases and reference architectures instead of possibilities.
It doesn't look like this project deals with consensus or linearizability at all. The "distributed" part seems to be about just (1) listening to changes from some master source and (2) issuing fine-grained mutation requests to that master (presumably, you then listen for your update to come through your subscription channel). The notation for mutations seems very basic, too.
As far as I can tell, this isn't really about distributed objects. It's a simple async client/server protocol with change subscription.
The creator basically says "with dop you have an easy/clean API. Is like using socket.io but easier", so yes, it's more of a thin layer over Socket.io to propagate changes. It doesn't handle consensus and definitely isn't a library for shared document editing or more substantial cases that require OT or other algorithms.
"definitely isn't a library for shared document editing or more substantial cases that require OT"
You are right, is not a library that provides a way to resolve sync conflicts. Consensus algorithms is a difficult topic, and I haven't seen an approach in JavaScript that resolve this problem. Maybe gun, but I haven't tried.
If you just need client-server architecture with reactivity, why don't you give a try to dop?
Does anyone have recommendations for node packages that perform similar client-server pub/sub or even sophisticated syncing?
Besides dop, DerbyJS/Racer seems to be dead, Apollo might be a solution but seems to require some setup, Meteor is too heavy, and Feathers with RxJS seems like a possibility. I'd be interested in any tools for getting easy client-server sync or reactivity.
Statebus can easily synchronize N clients to N servers, and has a very nice wrapper on top for reactive UI widgets to respond to changes in state. Here's a hello world example:
dom.BODY = -> # Let's define the body tag as a function
DIV
backgroundColor: '#eef'
'The time is '
fetch('state://server.com/time').now
' and the weather is '
fetch('state://another.org/weather/oakland,ca').brief
This defines the <body> tag as a function that will reactively re-run whenever the state of the time (fetched from server.com) or the weather (fetched from another.org) changes.
Interesting project with very simple core API. I saw some feedback about more sophisticated syncing through OT/WOOT/CRDT, though I don't know if that fits your approach.
If it can be used seamlessly with Vue/React (as mentioned in older discussions) such that changes are propagated into them, I'd be interested to try it. (Those libraries are in heavy use already so your solution would have to work with them for a decent chance at traction, I think.)
(1) It does work with React natively, and in fact the example I gave above uses React under the hood. Here's how to do the same thing with raw react:
<script src="https://stateb.us/client6.js"></script>
<script>
var body = statebus.create_react_class({
render: () => { // This function will re-run whenever
return React.DOM.div( // any state fetched below changes
{style: {backgroundColor: '#eef'}},
'The time is ',
fetch('state://server.com/time').now,
' and the weather is ',
fetch('state://another.org/weather/oakland,ca').brief
)
},
displayName: 'body'
})
setTimeout(_=> React.render(body(), document.body))
</script>
You can put that into a .html file and run it directly in your browser with a file:// URL.
I would love to help you give it a try and see what you think! Please send me an email or text (toomim@gmail.com or 510-282-4312) for feedback and assistance.
2) We are currently OT/CRDT/Version Control into it. We've come up with a new approach here, which I'm very excited about. I'm also looking for folks to beta-test or give early feedback on our approach as we polish it up.
I'm also interested in adding Vue support. If you'd like it, I'll add it.
PouchDB isn’t a client, it’s a whole implementation of CouchDB on top of IndexDB running in the browser or a Node server. It speaks the CouchDB sync protocol.
Maybe check out Gun (https://gun.eco)? It's CRDT-based and the docs make it look like they're taking things like consistency seriously. I have no experience with it, so can't make a recommendation either way, though.
Another interesting project with decent-looking adoption. The focus on p2p sync is a little off for typical server-client usecases. But it's nice to see that it can work with Vue/React.
Agreed on all points. The first question I have is “what problem is this trying to solve?” And it’s not immediately obvious to me. As far as I can tell that question is also not addressed on the site. From the github repo, this project is at least a few years old, but its README also doesn’t attempt to describe any use cases.
Imagine a Chess game, Trello, a multiuser ToDo list app.
If you are familiar with state management libraries such a Redux, Mobx, etc. Dop is more or less the same. With the difference that the initial state of your app can be initialized in the server.
As mention by others, the protocol doesn't handle consensus synchronization. The implementation of the protocol (JavaScript in this case) must apply all the mutations in the correct version order:
https://github.com/DistributedObjectProtocol/protocol#exampl...
This looks really pretty but the protocol seems not to be well-thought-out from a cursory glance. The example itself shows a case where the non-owning nodes are out of sync with the owner node (and sitting happily in a stable state!!!) and yet actions taken (possibly based off of the out-of-sync data...) are designed to propagate into mutations for the owned state which might be completely different to begin with! Instead of discarding alterations that are made on potentially bad/stale information, the system is designed to just accept any and all changes. There isn't even a rudimentary attempt at versioning to deal with conflicts and race conditions. Instead of dealing with errors from conflicts, they're swept under the rug and mutations are applied. Anything goes!
For a simple example of why that's a big problem imagine you have a state which has some boolean and an rcp that toggles that boolean which you fire based on that state (where both just want to turn it off). If two non-owning nodes say to toggle before the state has propagated from the other's change, it would turn off and back on again, cancelling each other out.
If this is just meant for toy projects where the worst case scenario has no real consequences that may be fine, but the polished look of the project is what's a little scary. It looks too professional for something that simply won't work.
Even still, if a node is connected to the owner through another (malicious) node, how do you prevent the malicious node from making mutation requests on behalf of the legitimate node?
I can see a number of ways that it can be accomplished that would take a good amount of work but it begs the question, is this worth the effort? You now have lots of security concerns, issues with nodes disconnecting, and nodes becoming overloaded. What benefit does this have over plain old fashioned AJAX? What problem does this solve?
This project is just about exchanging JSON data. CORBA "objects" are not pure data, but addressable RPC interfaces. Much more complex, and ultimately a bad fit for the modern web.
The slides suggest that a sufficiently complex system will almost immediately get out of sync. The dependency on upstream nodes asserting state about objects before they are acknowledged and successfully applied by the "owner" node means that some subset of nodes will be working on assumptions that haven't been validated by objects they don't in fact own. This seems to conflict with what prior slides say about who can mutate objects.
I think distributed object management is a useful concept, but instance state needs to be owned and managed by multiple nodes for redundancy in anything remotely approximating a production-level system. That doesn't seem to be readily addressed anywhere, either. I'd love to read more about this with the level of detail you'd need from a system that readily has use cases and reference architectures instead of possibilities.