
Flux over the Wire using Websockets and Immutable JS - elierotenberg
http://blog.rotenberg.io/flux-over-the-wire-3/
======
wereHamster
I see that Remutable uses 'patches'. How does it compare with ShareJS? Does it
allow collaborative editing (ie multiple people changing the same data
structure concurrently, is patch application commutative)?

The idea behind ShareJS (or OT for that matter) is awesome. You never have to
think about what to do if two people edit the same document concurrently. It
just works. You get undo for free. And you can write the code that it saves
automatically in the background. Never again do users have to press a 'save'
button.

A while ago I wrote a library which builds on top of the same ideas as ShareJS
(ie. OT) but with slightly different focus. You define object types and their
properties. This allows the library to automatically parse JSON into
JavaScript class instances. It also tracks changes to these objects in a very
similar way as ShareJS: each change has a 'path' (where something was changed)
and then the actual chang record (currently set and splice).

I use Avers in an SPA which is an editor of fairly complex data structures (90
different object types, nested ~5 levels deep). For rendering I use React. The
integration is really simple: Attach a change listener to the root and re-
render when anything changes.

[https://github.com/wereHamster/avers](https://github.com/wereHamster/avers)
[https://caurea.org/2015/02/08/the-future-of-
avers.html](https://caurea.org/2015/02/08/the-future-of-avers.html)

~~~
aidos
Interesting. I'm surprised at how little work seems to have been done around
OT (operational transformation, for those who haven't heard of it before).

I have an application where I hold a lot of synced data in the client and the
server but I've struggled to find good libraries to help manage that.
Generally the work seemed to be around text, rather than arbitrary objects.
Sharejs was the best I could find, but my BE system is python (and the object
support in ShareJS was a bit sketchy when I looked at it).

Unrelated but looking at your project I saw you were using Sets and Maps. I
didn't even know JS had those yet (and they're something I really miss coming
from python). Will be switching immediately.

~~~
wereHamster
Avers supports only two operations, _set_ and _splice_. It is very simple to
implement in any language. My backend is written in Haskell. The code dealing
with applying the patches is about 200 lines.

With these two operations you can edit arbitrary data structures. The user
experience may not be the best, for example if two people edit the same text,
because if you only have set then latest change wins. But maybe your data
structures are not text heavy. Or they have many small text fields where the
'latest change wins' semantic is actually desireable.

At my previous company we also used and OT based patch system. And we didn't
have text-editing support, we only had an equivalent to _set_ in Avers. And it
worked alright. Which operations you need depends on the application, ShareJS
is good for one particular type, but too complex (or maybe even lacking) for
others.

As for maps and sets, see [http://kangax.github.io/compat-
table/es6/](http://kangax.github.io/compat-table/es6/) how good the support
is. There are polyfills though if you need to support older browsers.

~~~
aidos
I'd be happy with set and splice. Text isn't really a big deal for my use
case. Will have more of a read through your code later today too. Thanks for
posting.

Great support list! Thanks for that. My customers all use the latest Chrome,
so I'm ok with fairly bleeding edge features.

------
amelius
Hmm, I don't really get it yet, but "single source of truth" sounds to me like
it needs extra round-trips to this "source of truth" in order to update the
display (even if the updates come from the client itself). Is that correct?

~~~
elierotenberg
You're right, although its fairly easy to implement optimistic updates on top
of this. Since patch objects are reversible, you can use a simple logical
clock and an undo/redo stack to perform optimistic updates locally, and undo
them/redo them when the remote SSoT sends updates.

~~~
amelius
Well, it could be me, but that doesn't sound very simple. Isn't that something
the framework should take care of?

Also, what happens when simultaneously transmitted actions/patches are in
conflict?

~~~
elierotenberg
Absolutely. I plan to include it in the framework ultimately, although it
requires implementing an optimistic client-side dispatcher in JS which mocks
the optimistic behaviour of the actual, eventually true server-side dispatcher
(which may not always be implemented in full JS, eg. if it needs to sync with
a db).

Conflicts are handled by the server-side dispatcher logic. Many applications
can handle conflicts easily (eg. chat rooms which only append new messages),
others can be more complex (eg. document editing). As mentionned in another
comment, Remutable.Patch leaves room for rebase-like algorithms. Its
definitely a very promising further work! :)

------
zzzaim
Alas, I've been working on something like this for the last few weeks on and
off, especially the "patching" of immutable data, my solution was to use
jiff[1], which generates JSON Patch operations[2], which is then applied to
the Immutable object using immpatch[3] (a lib I wrote).

A cursory look at Remutable seems it to be the better and faster option :)

[1] [https://github.com/cujojs/jiff](https://github.com/cujojs/jiff)

[2] [https://tools.ietf.org/html/rfc6902](https://tools.ietf.org/html/rfc6902)

[3] [https://github.com/zaim/immpatch](https://github.com/zaim/immpatch)

(edited formatting)

~~~
elierotenberg
That was nearly exactly my initial approach! :)

However I realized actual diffing was really slow, especially for large
collections, as it involves scanning the entire structure for changes, when in
fact all changes could simply be flagged upon mutation. Conceptually, it
performs diffing (ie. it gives you a diff object), its just much, much faster!

------
rattray
Remutable looks really impressive. I'm a little leery of the rest of it,
though, as it means my backend is no longer a simple REST API. I'm also rather
nonplussed that the full-stack Flux doesn't include optimistic display with
loading icon, error state, and rollback handling out of the box. It doesn't
even seem like an easy thing to do manually with this setup, though I haven't
played with it enough to know.

Few minor questions: it looks like a base Remutable must be an
Immutable.Map(). Can child items be any of the Immutable datatypes? Do we get
updateIn() and similar?

~~~
elierotenberg
Thanks for your feedback. Several other comments point in the same direction:
automatic (or assisted) handling of optimistic updates. I think this is
actually relatively easy, since there are clear hooks in the Nexus Flux API to
implement dispatch-time and update-time behaviour, but I will definitely
provide an example, or even better, an implementation of this that just works
out of the box.

~~~
drabinowitz
This is a really cool implementation! I'm wondering if it wouldn't be possible
to handle optimistic updates by submitting a request to the server and then
submitting an optimistic response from the client to itself. Then, when the
server comes back, you hook into the already present optimistic response and
just confirm or deny it based on the response from the server. That way you
keep as little state on the client as possible.

------
mandeepj
> We will rather record the mutations (or more accurately, the transitions
> between immutable states), and just replay them on the client.

Few questions -

How can we replay all the mutations?

How about instead of replaying, if we just show the last mutation? Will it not
be the same thing?

I think this flux architecture will work best only in chatting applications.
This approach is also making the app itself very chatter so performance
concerns in case you do not have strong machines.

I think on form pages this flux approach may not be required.

------
elierotenberg
Wow, a crashed site presenting a web architecture surely isn't classy! Sorry
about that, I'm working on it.

~~~
ac2u
Y'know... I quite liked jumping in and reading right away! Didn't notice it
was lacking certain assets at first.

------
Rapzid
The best example of single source of truth may be chat, but I believe chat is
the best example of a replicated log. This throw me off a bit while reading
the article :| Left it, came back to the comments later to see people talking
about OT and realized I missed something. Just an obesevation.

------
verroq
Synchronising state over the wire? Surely this is a solved problem in games
development already - serialise your state into a consistent binary form, send
over the binary diff of the previous state (AKA delta compression),
deserialise on client side.

------
mikaelemtinger
Take a look at swarm.js for a fantastic way to keep stores in sync - even
works in offline. Matrix.io could also be an interesting option for this, but
focuses more on communication than shared data.

------
stucorbishley
The single source of truth diagram is so great! :) Was getting tired of
turning my head sideways..

------
sehr
cache:
[http://webcache.googleusercontent.com/search?q=cache:http://...](http://webcache.googleusercontent.com/search?q=cache:http://blog.rotenberg.io/flux-
over-the-wire-3/)

------
rattray
slightly offtopic question; what are all those `should.be.exactly(whatever)`
calls in the Remutable readme?

~~~
elierotenberg
Its shouldJS. An assertion lib with a fun chainable DSL. Basically
whatever.should.predicate desugars to assert(predicate(whatever)). See
[https://github.com/shouldjs/should.js](https://github.com/shouldjs/should.js)
:)

