
Exactly-Once Delivery May Not Be What You Want - luu
http://brooker.co.za/blog/2014/11/15/exactly-once.html
======
covi
> One is that end-to-end system semantics matter much more than the semantics
> of an individual building block, and sometimes what seems like a very
> desirable semantic for a building block may end up making the end-to-end
> problem harder.

This is a wisdom whispered __3 decades ago __. The end-to-end argument haunts
us again [1]!

[1] END-TO-END ARGUMENTS IN SYSTEM DESIGN
[http://web.mit.edu/Saltzer/www/publications/endtoend/endtoen...](http://web.mit.edu/Saltzer/www/publications/endtoend/endtoend.pdf)

~~~
akkartik
I made a more superficial version of this argument a year ago:
[http://akkartik.name/post/readable-bad](http://akkartik.name/post/readable-
bad)

 _Edit_ : "Safety is a characteristic of systems and not of their components"
\--
[http://www.ctlab.org/documents/How%20Complex%20Systems%20Fai...](http://www.ctlab.org/documents/How%20Complex%20Systems%20Fail.pdf)

------
autarch
The point about unique IDs is well taken, but there's a cost to _tracking_
these unique IDs too.

For example, if you generate hundreds of message a second, you quickly end up
having to track millions of unique ids. Then you have to manage the storage of
these IDs somehow, so you end up implementing a system to wipe out any ID
older than a certain time, and so now you not only have track the IDs you've
seen, but when you've seen them.

That said, this is probably better than making an incredibly complex message
queue system just to support exactly once delivery.

~~~
alexk
Cassandra is very good at this, works perfectly in similar case of tracking
unique keys with TTL.

------
kiyoto
As a maintainer of logging middleware
([https://www.fluentd.org](https://www.fluentd.org)) I am asked all the time
whether we have "exactly once" semantics. I try to be honest and say we err on
the side of "at least once" message delivery, but even then, there are edge
cases where messages can be lost.

It really vexes me to see how a lot of distributed systems projects claim to
support "exactly once", and it makes me happy to see a grounded, intelligent
summary like this one.

------
vishnugupta
> One is that end-to-end system semantics matter much more than the semantics
> of an individual building block, and sometimes what seems like a very
> desirable semantic for a building block may end up making the end-to-end
> problem harder. The other is that simple, practical, solutions like unique
> IDs can make really hard problems much easier, and allow us to build and
> ship real systems that work in predictable ways.

These assertions that I tend to follow most of the time. I keep warning my
team members to never assume that a request for payment/fulfillment/some-
business-function is going to arrive exactly once. If they build systems
_assuming_ that underlying scaffolding is going to behave in certain way then
they won't be designing for failure. In any distributed system it's very
important to define correctness from the end user's perspective. And then
design for failure ensuring that the correctness guarantees are not violated.

------
bkirwi
This is a nice article, but the title strays into 'Fox and the Grapes'
territory. Exactly-once delivery is pretty much always what you want -- in
contexts where all manner of delivery guarantees are possible, from Java's
concurrent queues to Go's channels, pretty much everyone delivers a message
precisely once.

Of course, you can't always get what you want, and in a distributed system
exactly-once delivery is not possible in general. But in the cases where it
_is_ possible, it's often worthwhile: it's just a whole lot easier to reason
about than a system where messages can be arbitrarily re-delivered.

------
akkartik
Might be a straight-forward application of Buridan's Principle:
[http://research.microsoft.com/en-
us/um/people/lamport/pubs/b...](http://research.microsoft.com/en-
us/um/people/lamport/pubs/buridan.pdf)

------
EGreg
Obviously, exactly-once delivery is a side effect of possible overproduction
and then filtering for uniquenessz

Some people might also say that creativity is a side effect of brainstorming
and idea selection. But it's more complex than that - over time our brains
work out heuristics and figure out patterns that work well, and use that as a
language to construct other things. That's why people who have been doing
something for years can do it on a much higher level.

------
ChuckMcM
This is a fantastic summary of the challenge of distributed execution. I also
really enjoyed the quote "there is always room to slot in another turtle" I am
definitely going to keep that one handy.

