This seems like another odd definition of "exactly once". The underlying assumption appears to be that all you work is happening insides the sql transaction, so all the work the consumer does is safe and can be rolled back if a failure happens. Many systems are going to be doing at least some work that can't be easily rolled back by a transaction (e.g. calling an external API). In this more interesting world, you really can't get exactly once since we already did something outside of our transaction.
In my experience, it's easier to reason about and build systems when idempotency is an application level concern. For example take a bank that has some messaging system to update account balances. While a exactly once system, if designed perfectly, might achieve this, you could also achieve this by building an idempotent "update balance" system. With application level idempotency, you have more flexibility to later add different paths or technologies without as many re-write headaches.
Also -- the messages per day stat seems irrelevant. I've yet to encounter many real world systems that don't have irregular bursty patterns. With this slow processing rate, you could basically have a single large burst and then normal traffic, but be unable to return to realtime latency for hours/days.
> In my experience, it's easier to reason about and build systems when idempotency is an application level concern.
This is exactly what we are doing in my company :) But it’s always good to have an option and know that we can do it in specific cases without external calls. In this scenario it can be nice simplification.
In my experience, it's easier to reason about and build systems when idempotency is an application level concern. For example take a bank that has some messaging system to update account balances. While a exactly once system, if designed perfectly, might achieve this, you could also achieve this by building an idempotent "update balance" system. With application level idempotency, you have more flexibility to later add different paths or technologies without as many re-write headaches.
Also -- the messages per day stat seems irrelevant. I've yet to encounter many real world systems that don't have irregular bursty patterns. With this slow processing rate, you could basically have a single large burst and then normal traffic, but be unable to return to realtime latency for hours/days.