First off, we're huge fans of both Meteor and Asana. I spoke with Geoff @ Meteor a couple years ago when we were first starting to build out the Airtable product and was very impressed by their approach and vision.
We've closely followed the developer blogs of both those projects (and in Meteor's case, their source code). With those learnings, we built our own realtime database engine that supports relational data (which Meteor doesn't yet support) and also some other major features like the ability to undo any user action out of order (like git revert), which is necessary to support undo in a multi-user context (because the last thing that you did may not be the last change globally if other people are concurrently making changes). Undo is a particularly challenging feature to implement in a structured relational database context, because it can't be reduced to a set of simplistic character insertion operations as is the case for a google word doc, or a spreadsheet (which is a simple 2d array of values without type constraints, foreign key relations, etc).
Very impressive, one more congratulation from me. And a couple quick questions:
How do you handle conflicts? E.g. If two users change the same scalar value (of an integer cell, let's say) at the same time. Do you accept one and throw a warning back to the unlucky users?
How much to you cache locally? I had begun a framework such as this at some point, with the ultimate goal of being usable offline via the localStorage cached version (and sync upon connect), so I have a good idea of how much pain you must have gone through sorting this out :)
You wrote your own real-time database engine. You're nuts! (in a very good way). Have you written anything about this? It would be interesting to hear both about the database and the experience of "rolling your own" at such a low level.
Hi, yes--we definitely want to get a dev blog up and start talking about some of this stuff!
Re: conflicts, in the case of a scalar value, last in wins, and the system (if you consider the server and each client as nodes in a distributed system) is eventually consistent (i.e. everyone will eventually see the exact same state). In that particular case, we show the other user's profile picture and highlight the cell to flag that change so you realize somebody else is editing at the same time as you.
Nice. There is probably an argument to be made for a notification when you update something and someone else also updates it and your update loses (i.e. small time difference updates or conflicts + your client wasn't last)
Although to be fair, I suppose "undo" covers this. Can you undo an action taken by someone else? (User A commits "aa", User B commits "bb", User A commits "yy"; Can User A, undo to both "bb" and "aa"?)
>Undo is a particularly challenging feature to implement in a structured relational database context, because it can't be reduced to a set of simplistic character insertion operations as is the case for a google word doc, or a spreadsheet.
I personally hit this roadblock while trying to build a revision management system for a spreadsheet like Google Drive has. I had a pretty fragile implementation that works within a tangled mess of has-many relations. It was a tough time generalizing the whole mess. I'm looking forward to your insights on this particular problem on your future developer blog! Congratulations.
We've closely followed the developer blogs of both those projects (and in Meteor's case, their source code). With those learnings, we built our own realtime database engine that supports relational data (which Meteor doesn't yet support) and also some other major features like the ability to undo any user action out of order (like git revert), which is necessary to support undo in a multi-user context (because the last thing that you did may not be the last change globally if other people are concurrently making changes). Undo is a particularly challenging feature to implement in a structured relational database context, because it can't be reduced to a set of simplistic character insertion operations as is the case for a google word doc, or a spreadsheet (which is a simple 2d array of values without type constraints, foreign key relations, etc).