
What do you mean by “Event-Driven”? - Garbage
https://martinfowler.com/articles/201701-event-driven.html
======
btilly
One common anti-pattern is that "event driven" can lead to a push architecture
that works on paper, but is unreliable in practice.

The idea looks deceptively simple. If you need something to happen, you just
broadcast the right event, someone else acts on it and everything is fine. You
code it, it works in the demo, and you deploy to production.

And then you run across network failures. Missed events. Doubled events.
Events that arrived while a database was down. Events that were missed due to
a minor software bug. And so on. You layer on fixes, and the system develops
more complex edge cases, and more subtle bugs.

Then you rewrite from a push based system to a pull based system, and all of
your complex edge cases disappear. :-)

~~~
virmundi
I've used events successfully for an important fraud detection system. There
are a set of known events. A fraud model could hook into the event chain at
any point. It would then produce an alert event with its findings. Downstream
another handler listened for it. Coupled with RabbitMQ, the system is a
pleasure to expand. All the fraud models are micro services. Just a start up
script and boom, hooked in.

We planned for duplication of messages. Operations are idempotent. If the
message already ran, we could rerun the calculation safely. With Rabbit the
only duplicate messages were the ones humans sent to rerun a model that broke.

~~~
Floegipoky
Sounds like a system I built, but what I didn't account for is how often the
RabbitMQ cluster would get tanked by my coworkers. The system fails when
messages start getting dropped on the floor due to server reboots. Still
haven't come up with a great solution, but what we have is "good enough" for
that project's requirements. It was a great learning experience though, an
uncomfortable reminder of the difference between reliability and durability,
and it definitely helped inform subsequent architectural decisions.

~~~
jononor
Separate out another RabbitMQ cluster for those other applications/people?

And why are they rebooting all the time anyway??

------
jph
Event-Driven software comes up on many of my projects. What I find works well
is using command-query separation (CQS, CQRS) to start because CQS is a lever
into two complementary ideas: backend implementation and continuous
improvement.

The backend implementation ideas involve REST, caching, replication, eventual
consistency, CAP, and especially PACELC.

The continuous improvement ideas involve upgrading existing software projects
from imperative styles to functional styles, phasing in publish-subscribe
modules, growing codebases to be more event-driven, and planning for
evolutionary architecture.

In practice, I see these areas need management help, such as funding and time
for training, and help for teams creating infrastructure as code (IaC) to
deploy many event-oriented modules as well as event monitoring and management.

Related links:

[https://en.wikipedia.org/wiki/Command-
query_separation](https://en.wikipedia.org/wiki/Command-query_separation)

[https://www.wikipedia.org/wiki/REST](https://www.wikipedia.org/wiki/REST)

[https://www.wikipedia.org/wiki/PACELC_theorem](https://www.wikipedia.org/wiki/PACELC_theorem)

[https://www.wikipedia.org/wiki/Publish-
subscribe_pattern](https://www.wikipedia.org/wiki/Publish-subscribe_pattern)

[https://www.wikipedia.org/wiki/Infrastructure_as_Code](https://www.wikipedia.org/wiki/Infrastructure_as_Code)

~~~
sethammons
I had not heard of PACELC and was excited to hear something new, only to see
it as a very natural extension of CAP. It reminds me of, back in math class,
you would learn of a new theorem that was only different than another because
they singled out a particular case, like for 1 or 0 :)

~~~
sundvor
I hadn't heard of either, but they both make perfect sense - so I for one say
thanks for that great post. :-) Did study up on CQRS/ES year and found it very
exciting.

------
Terr_
> a potential confusion between event-sourcing and CQRS

IMO this comes up _a lot_ in HN/Reddit discussions about either term. Event-
sourcing implies CQRS, but the reverse is not true.

> I'd love to write some definitive treatise [...] Sadly I don't have the time
> to do it.

The "Fermat's Last Theorem" approach to computer architecture :P

~~~
to3m
I'm not sure... sounds like CQRS (though it's a term I don't think I've ever
seen before...) is independent. Like, you can have one data structure for
read/write, and you just don't get public access to the write interface
directly, only through functions that add to the event list and then do the
event. Along the lines of something vaguely like this, perhaps:

    
    
        void Stuff::AddThing(Thing *thing) {
            this->AddEvent(new AddThingEvent(thing));
        }
    
        void Stuff::AddEvent(Event *event) {
            m_events.push_back(event);
            event->Execute(this);
        }
    
        void AddThingEvent::Execute(Stuff *stuff) {
            stuff->m_things.push_back(m_thing);
        }
    

and maybe AddThingEvent is a friend of Stuff, or it's an inner class, or
whatever you like.

Maybe this is what he meant though.

~~~
Terr_
I think that _most_ of the time event-sourcing will impact your
model/business-logic layer, because you often want to capture the _intent_ of
changes, rather than calculating a naive combined delta of all changes that
occurred in memory.

If so, that means that instead of a CRUD interface like:

    
    
        if(customer.getApprovalLevel() < 5){
            customer.setApprovalLevel(5);
            customer.setApprover(currentUser);
        }
        saveChangesSomehow(customer);
    

So you'll end up with an API that's a bit more CQS (no 'R' yet) with:

    
    
        if(customer.getApprovalLevel() < 5){
            customer.promoteApprovalLevel(5, currentUser);
        }
        saveChangesSomehow(customer);
    

That way you capture each change-event at the right granularity with the
necessary data. Finally, CQ _R_ S comes into play because you want to _do_
something with that rich event data, such as eagerly populating a table:

    
    
        recentApprovals = getRecentApprovals(); 
        if(recentApprovals.length() > 0){
            sendApprovalAuditReport(recentApprovals);
        }

------
urbit
Blatant plug: Urbit (urbit.org) is a general-purpose OS built on the event-
sourced model. Urbit's state is a small, frozen function of its event log.

I don't know of any other generalized event-sourced OSes. But almost every
principle Fowler describes in special-purpose event-sourced designs also seems
to apply in the general-purpose case.

For example, the connection between CQRS and event-sourcing seems very deep
and natural. The connection to patches in a revision-control system is also
completely apropos.

When Fowler starts pointing at advanced problems like "we have to figure out
how to deal with changes in the schema of events over time," you start to see
a clear path from special-purpose event-sourced apps to general-purpose event-
sourced system software.

Broadly speaking, one category of answer is: abstract the semantics of your
app, until it becomes a general-purpose interpreter whose source code is
delivered and updated in the event stream. Then the new schema is just a
source patch. And your app is not just an app, but actually sort of an OS.

~~~
dozzie
> Blatant plug: Urbit (urbit.org) is a general-purpose OS built on the event-
> sourced model.

I've looked through its documentation, and while I couldn't make heads or
tails of it, Urbit seems to be everything _but_ an operating system.
Espeically that it needs unix system to run.

~~~
urbit
Linux on Xen would also fail this test.

Urbit is an OS in that it defines the complete lifecycle of a general-purpose
computer. It's not an OS in that it's the lowest layer above bare metal.

Sorry you had a bad experience with the docs. Urbit is actually much stupider
than it looks.

Unfortunately, premature documentation is the second root of all evil, so some
of the vagueness you sense is due to immature code. It's always a mistake to
document a system into existence -- that is the path of vaporware. Better to
run and have weak docs, than have good docs but not run.

~~~
fb03
I agree with you on the documentation thing but, what we see (and I experience
also) is that most of the time, when things get serious enough (production),
there isn't any available time to document the core of the software properly,
let alone document it fully.... "lack of time" in this context meaning either
(1) management won't let you stop implementing stuff they need "for yesterday"
(everything is critical priority) and document for 10-20 hours or (2) your
self run project needs more features and it's always cooler to study/code than
to document puzzles you have already solved.

I'm not criticizing, I am exposing that I am part of the problem and lost, and
I'd like to know how people handle this on their lives. I want to document
more but there's always this rush to implement implement implement and not
once, during some catastrophic breakdown of something, management wants
everything solved quickly but they expect you to remember instantly of
everything you did 2+ years ago :-)

PS: ... and they'll get mad if you say "you didn't let me document this damn
thing"

------
osullivj
My first contact with event driven programming was coding for the original
Windows C API, back in 1990. Nowadays we call that API Win32, but then it may
have been win 16. Switching from the traditional imperative style to callbacks
was a big deal: don't call us, we'll call you! [1] and [2] show the difference
in coding style. Later in the 90s C++ wrappers like MFC and XVT simplified
things, but the paradigm was still an inversion of control. And that's what
event driven means to me.

[1]
[http://www.paulgriffiths.net/program/c/srcs/hellosrc.html](http://www.paulgriffiths.net/program/c/srcs/hellosrc.html)

[2]
[http://www.paulgriffiths.net/program/c/srcs/winhellosrc.html](http://www.paulgriffiths.net/program/c/srcs/winhellosrc.html)

~~~
barrkel
Unless you do async operations, UI framework programming with callbacks
doesn't really approach the full horror that can exist in some styles of
event-based programming.

UI events are typically of two kinds: user performed an action and you need to
update the model / execute a command, or the view is asking for information
and you need to translate it from the model. Both are fairly encapsulated; the
larger operation doesn't extend beyond the scope of the callback, and if it
does (e.g. showing a modal dialog), you can use a nested event loop to take
care of event dispatching while showing the modal.

The events in a UI are also naturally scoped to the screen being shown. When
the screen goes away, you can rely on those events not being dispatched any
more. Events get to party on a shared state (the model, perhaps attributes of
the view) but scoped within the view being displayed. It's all pretty flat.
You also don't need to worry much about concurrency, unless doing async.

------
barrkel
It's very easy to create an event-driven program that is effectively a larger
program implemented in terms of "come-from". The decoupling provided by
eventing is often illusory; if there's a higher-level semantic that's supposed
to be implemented in terms of the outcome of coordinated event handlers, then
the dependencies still exist - they're just implicit.

If event-handlers are genuinely decoupled from one another, and there isn't a
higher-level state machine that's being driven by events, then it can be an
excellent way to structure logic.

Events-as-deltas that are durably stored and can be reliably replayed is
another fine way to go for a system with checkpoints, auditing, history,
rollback, time-series reporting and similar requirements. But like the article
says, code will be much simpler if it deals with an eternal present and the
calculation and application of the deltas is centralized in code that doesn't
change often.

There's an isomorphism between building a bigger program out of event
handlers, and distributed programming. Distributed programming is known to be
difficult, and doing it at the message send level is IMO too low a level - one
should build more expressive and composable primitives and work at a higher
level. That's if it's warranted at all.

~~~
guptron
"The decoupling provided by eventing is often illusory; if there's a higher-
level semantic that's supposed to be implemented in terms of the outcome of
coordinated event handlers, then the dependencies still exist - they're just
implicit." \- I've experienced the same myself and converted my "events" into
coupled calls. More expressive.

------
TheAceOfHearts
I find the topic of event sourcing / CQRS a little frustrating. Somewhat
tangential, but I've slightly similar experiences with rxjs as well.

For both, whenever I read about em or I hear someone discussing em, it makes
so much sense in my head! But later when I try to execute on the ideas, I
can't make it "come together". It's like some puzzle piece refuses to click
into place.

In the cases when I've tried hacking something together, I've always ended up
with something that seemed more brittle than if I'd just gone with a safe
relational database. With that said, I'm also completely open to the
possibility that I've just been trying to apply the pattern in cases where
it's not a good fit.

I'd seriously love to poke around in a real-world application showcasing CQRS.
I don't expect something perfect, but getting a chance to look at what
tradeoffs and concessions a bunch of engineers made would surely be incredibly
insightful.

~~~
akiselev
_> In the cases when I've tried hacking something together, I've always ended
up with something that seemed more brittle than if I'd just gone with a safe
relational database. With that said, I'm also completely open to the
possibility that I've just been trying to apply the pattern in cases where
it's not a good fit._

There is a 95% chance that event sourcing and CQRS were entirely the wrong
patterns for your use case. If you're not dealing with a very complex domain
(like, say, automatically orchestrating a global logistics or drug discovery
pipeline with audit logs for regulatory compliance), that chance becomes
99.9%. If you're working on something that only you or a few people will ever
work on, it's 110%. Just the tooling for a reliable event sourcing system
would be a herculean task for a single developer and that's without even
writing a single line of business logic!

I don't know of any open source real world examples of CQRS off the top of my
head and the only successful ES/CQRS systems I've seen in the wild were
projects that involved hundreds of domain experts and programmers, dealing
with extremely complicated fields like biotechnology and electrical
engineering where nonprogrammers spent about as much time looking at code as
the programmers. Any good ES/CQRS project will contain a lot of the business's
secret sauce so it's not something Google or Facebook would open source.

------
grandalf
I'd like to see a simple e-commerce cart written using all of the patterns.
That's really the minimal "hello world" for comparing the patterns he
discusses.

~~~
kpmah
Not sure how much this will help, but a couple of years ago I wrote about
using event sourcing for a simple blog application in PostgreSQL:
[http://kevinmahoney.co.uk/articles/immutable-
data/](http://kevinmahoney.co.uk/articles/immutable-data/)

I wasn't aware of the term 'event sourcing' at the time though.

------
aryehof
A good summary. However too many choose to view it only as Event-
Sourcing/CQRS, a silver-bullet applicable to a minority of applications. One
particularly attractive in Microsoft projects, since it is the only
alternative enterprise architecture "practice" they propose to data modeling.
Object domain modeling continues to be an anathema, because I think of a
wizard and code-generation driven historical perspective.

In commercial systems, transactions as actions and events, recorded for
business or legal reasons, are central. That includes significant "events"
like purchases, sales, reservations etc. In that sense, they all are event
driven.

Event-Driven as an architecture that is based on recording changes to a
baseline state, should be applied only where _really_ suited.

------
mbesto
For anyone who has implemented a system using CQRS patterns, do you recommend
it? What are the pros and cons as the codebase and application grow?

------
jknoepfler
generally I mean interrupt-driven, but maybe I'm being obtuse.

~~~
kasey_junk
Meh. All events are just an abstraction 1 level above polling^, interrupts
included.

^may not be true for spooky actions

------
snissn
I think of it as using epoll

[http://man7.org/linux/man-pages/man7/epoll.7.html](http://man7.org/linux/man-
pages/man7/epoll.7.html)

------
OJFord
Idk, all my code's results-driven. (And goal-oriented.)

------
Flenser
Reading this it struck me how React/Flux uses Event-Sourcing whereas Angular
uses Event Notification.

------
whyileft
I would highly recommend spending an evening reading through Martin Fowler's
blog. Has has a gift of breaking down concepts into very pragmatic and
digestible explanations.

A couple of my favorites:

[https://www.martinfowler.com/articles/injection.html](https://www.martinfowler.com/articles/injection.html)

[https://www.martinfowler.com/articles/refactoring-
dependenci...](https://www.martinfowler.com/articles/refactoring-
dependencies.html)

------
jaequery
i was expecting to read about node.js

