
A Certain Tendency of the Database Community - cmeiklejohn
https://christophermeiklejohn.com/lasp/erlang/2015/10/27/tendency.html
======
jacques_chester
> _In this model, the results of the executions of programs are also
> mergeable._

Not all updates are commutative.

The ordering of events in (add 1, add 2, add 3) does not affect the outcome.

The order of events in (add 1, multiply 2, divide 3) _does_ matter.

The problem of linearity exists as soon as you have a commutative operation in
a relativistic universe where information cannot be simultaneously updated in
two coordinates simultaneously.

Until now there have been heroic engineering efforts to hide this from
software developers. Hardware has conspired to conceal that even in a space a
few centimetres wide, values can become inconsistent.

Once you have independent memory spaces, all bets are off and you are forced
to pay a high price for losing the abstraction.

~~~
marknadal
Not all updates are commutative, but there exists an entire class of
algorithms that solve for each of these scenarios - see CRDTs, DAGs, even
linked lists.

~~~
jacques_chester
I'm skimming, so I am probably getting it wrong.

But on my reading, CRDTs just kick the can down the road to the implementer of
the operation or merge function. The way they solve for commutativity is to
... require commutativity.

That's a win _if_ your changes are commutative. Lots of things aren't.

~~~
marknadal
From my research the only thing you truly can't handle is The Double Spending
problem, blockchain ledgers help against this but are not immune to it.

Any and all arithmetic operations can be fully solved with CRDTs without
further can kicking. The trivial solution is to give every number you are
adding/subtracting a unique identifier, then as the values come in you do a
sum across them, deduplicating in the process. This means you can resend the
math operations as many times as you want and you still get an idempotent
result.

~~~
skybrian
"Fully solved" seems like a bit of an exaggeration, considering that
mathematical operations on computers often aren't commutative if you look into
the details. Consider catastrophic cancellation in floating point and overflow
in fixed-point calculations with range checks.

To get a consistent answer, you'd also have to sort the numbers consistently,
so instead of a sum this becomes an insert into a sortable list. This is far
more expensive than the typical kinds of aggregate operations people want to
do at scale - the point of aggregation is not to just save all the original
data.

------
dikaiosune
Uber is mentioned here as an interesting case of treating the edge device as
the source of truth, which was interesting to read about a little while ago.
However, Uber has a certain amount of guaranteed control over driver phones,
and even then I imagine that it represents a pretty promising attack vector
into their systems.

How would this system work when treating untrusted clients executing arbitrary
code as the definite source of truth for a fact? Further, it doesn't seem like
this addresses (unless I'm missing something) the problem raised in the
airline ticket example where split-brain systems may hand out resource
reservations that they aren't actually allowed to give out. If both
passengers' devices (or the plane? or some other edge device?) are sources of
truth, I don't see any easier way to negotiate the resource contention than
current distributed databases already have.

~~~
GauntletWizard
Split brain systems can easily deal with being disconnected: They don't give
out new tickets, they don't modify system state, and they treat the data they
do have as what it is: Out of date. An airline reservation system shouldn't be
able to give out multiple conflicting tickets, because it doesn't have a lock
on them; RAFT and PAXOS are designed to combat precisely this.

However, in the real world, waiting for consensus before making decisions is
hard, so airline systems, designed in the era before realtime communication
back to the mothership was possible, faked it. They made a reservation, issued
a ticket, and then the airline developed protocols to deal with the resulting
overbooking. In fact, since airlines already had incentive to overbook and
deal with overbooking because of no-shows, they simply built the failures of
their locking system into their cost-models.

There is a 'correct' solution to this: When you believe you're going to be
split-brained some of the time, pre-allocate tickets to entities. Those
entities can use their local tickets, but cannot make use of remote tickets
without approving them with the systems that own them. Sharding the state of
the plane down to individual seats has system complexity cost, but allows
perfect booking.

~~~
fallous
Changing behavior in split brain systems requires an understanding of the fact
that the brain is in fact split.

And even if you design the system to operate in your 'correct' solution, you
may still result in imperfect booking if one side of the split exhausts its
pre-allocated tickets and reports as unavailable when other sections of the
system still have available tickets.

------
fallous
Unhelpfully he switches analogies in midstream to avoid the problem of the
"two customers buy tickets when one seat is available" originally posited.
Additionally, "eventually consistent" begs the question of what is
"consistent?" How do you know that there is an outstanding commit that would
affect the system without a final arbiter of truth? In the comments there is
mention of air traffic control and the fact that two 747s cannot occupy the
same spacetime safely. Both planes can express "intent" with regards to flight
paths, but either these intents are resolved by a single arbiter of truth
operating in a timely fashion, the flight controller, or they are not and
result in a collision which is also resolved by a single arbiter of truth,
being reality. One arbiter is far more preferable than the other.

There are large numbers of problems that are easily amenable to casual
consistency, but there are other that simply are not. The possible solutions
to the former are many and varied but the latter requires a solution far more
in tune with the "database community."

------
marknadal
This isn't theoretical, I've implemented this system already in code:
[https://github.com/amark/gun](https://github.com/amark/gun) . Just spoke on
it in London, Germany, and the USA.

I think people need to know that this discussion is no longer happening only
in the hands of academics, this is now available to developers - with or
without the academic side.

~~~
tsantero
> I've implemented this system already in code

It doesn't look like you have. I watched the demo video "GUN Survives a
Primary Fault" and the conflicting updates were resolved with a LWW semantic,
which is non-deterministic.

(EDIT: I read some docs, and it looks conflicts are resolved by choosing the
highest lexical order of the value. So, apologies for the assumption. I
assumed LWW because the BBBBBB update was seen last after the fault was
resolved. Nonetheless, I think it is interesting you're treating the browser
as a peer in the system, and not simply a client)

> this is now available to developers - with or without the academic side.

I don't understand this attitude. Working closely with academics ought to be
what industry strives for. They take the time to research, formalize theorems
and write proofs to advance the current state of the art in Computer Science.
This research helps move industry forward. A tight relationship between
industry and academia is mutually beneficial, for what should be obvious
reasons.

~~~
marknadal
Thank you for looking at it! But I do need to correct you, it is not using LWW
semantics. To be clear, LWW isn't possible in a decentralized system because
"last" is different for every peer. Only a centralized system can have a
"last" because it has an objective place to measure last from. GUN is using a
deterministic merge function, not LWW.

Academia - I agree with you! Good words, sir. I was just wanting people to
know that it does exist in code land as well, because often times academic
designs aren't. So I think it is important to highlight when they are
available.

