
Show HN: Learn what DDD, CQRS, and event-sourcing are all about - goloroden
https://docs.wolkenkit.io/1.1.0/downloads/brochure/
======
victorNicollet
The DDD/CQRS/ES topic appears on Hacker News every so often, and never fails
to stir a strong debate. In fact, I was going to "Show HN" an Event Sourcing
library I open sourced yesterday, but I guess I'll wait until everything dies
down a bit :-)

I believe that the core principles of DDD/CQRS/ES are flawed and are the cause
of a lot of pain. If your team plows forward and ignores the pain, you will
have a bad experience working there. If your team decides to fix the problems
with the architecture (usually in a way that's rather specific to their
situation), and still insists that they're doing DDD/CQRS/ES, then you will
have a great experience. The same can be said of many approaches to SQL
database design.

When I joined Lokad, most of the code had been written by a strong CQRS
advocate, and there were significant pain points caused by this approach that
would have been nonexistent with a plain old SQL architecture. The author
himself did a retrospective that covers a few issues:
[https://abdullin.com/lokad-cqrs-retrospective/](https://abdullin.com/lokad-
cqrs-retrospective/)

Instead of dropping everything and moving back to a more conventional SQL
architecture, we dropped everything and moved to a solution which, if I were
to describe it as DDD/CQRS/ES, would have me branded as a heretic by the
DDD/CQRS/ES community: it does fit the definition, but it does not follow the
best practices. There is a single data model shared by both commands and
queries. Commands can sometimes return data. Each (micro?)service has access
to exactly one aggregate. There is no uncertainty on event ordering, and no
eventual consistency at the aggregate level. Materialized views for each
aggregate are kept in-memory at all times. All of these intentional weakenings
of the CQRS architecture were driven by pragmatic needs, and made with
knowledge of the consequences.

~~~
UK-AL
I'm CQRS advocate. None of are weakening the CQRS. CQRS is in fact pretty damn
lose.

"There is a single data model shared by both commands and queries." \- Nothing
wrong with that, especially in simple cases.

"Commands can sometimes return data" \- Well commands can succeed or fail.
Greg young himself admits commands are really synchronous, and you need to
communicate that.

"and no eventual consistency at the aggregate level." \- This is the
recommended approach. Between aggregates and it becomes loser.

"Materialized views for each aggregate are kept in-memory at all times" \-
Thats some advanced CQRS usage there.

~~~
victorNicollet
Maybe I'm not as much of a heretic as I thought I would be :-)

~~~
UK-AL
Starting from fresh these days I would use something 2 way for commands,
queries like HTTP REST.

But a messaging system for events.

If i'm using event sourcing use a dedicated event sourcing database with
catchup subscriptions etc

~~~
codebeaker
> a dedicated event sourcing database with catchup subscriptions etc

Can you recommend any?

~~~
victorNicollet
We're rolling our own:
[https://github.com/Lokad/AzureEventStore](https://github.com/Lokad/AzureEventStore)

------
he0001
While event sourcing is a quite alright tech for certain problems, this idea
of "one solution fits all" is just ridiculous. Also the idea of "eventual
consistency" is highly situational particularly in an distributed solution.
Paradoxically looking at one event source log this looks like a great idea,
but there are no guarantees that events propagate evenly and "in-order" so
even if events happen and there's "nothing you can do about it", systems will
still be out of order, when read in relation to each other because the systems
won't know that any other event actually have happened. The system will
happily consume it because it won't have the proper info or acted on the wrong
information. No matter how you try to mitigate this this will happen and you
won't have the slightest idea that it happened. Worst of all, you can't
recover from it because the information is gone.

It's just fundamental aspect of asynchronous programming, conveniently
forgotten trying to sell solutions.

~~~
josephg
This is a problem in how lots of people do event sourcing, but its not a
fundamental problem with event sourcing. If you enforce strict ordering of
events throughout your stack and do operation catchup properly, these sorts of
bugs are completely avoidable.

~~~
he0001
Well that's the entire problem, you are not the one making the requirements,
the solution of the problem does... You can't eat dinner before you have made
it.

Edit: I would argue that you can't write a system which is open (several
domains) and believe that you will have entirely independent events throughout
the system and claim that your system will be "bugfree"/correct.

~~~
nightski
I could be missing something but I feel like you are inventing an impossible
scenario. You may not be making the requirements, but the requirements have to
be technically feasible. CQRS/DDD give you a lot of flexibility on where to
choose your boundaries/aggregate roots and these are designed around the
business/requirements. If you can give a valid scenario where this model
breaks down but others would succeed I'd be very interested.

~~~
victorNicollet
Here's an example from the Wolkenkit brochure:

> user.login({ login: 'user', password: 'secret'});

> const isUserLoggedIn = user.isLoggedIn();

If you look at this purely from an API design point of view, this stinks: by
forcing the command to return void, the code ends up twisting its design to
return the result of the command through an artificial channel (the state of
the 'user' object). In the end, we are still interested in "the result of the
login attempt", and the easiest way to do this would be a function that
returns a value:

> const isUserLoggedIn = user.login({ login: 'user', password: 'secret' });

~~~
gregyoung1
Why are you using CQRS for logins? Nothing says derp like "we rolled our own
security system"

~~~
victorNicollet
That is a question worth asking, but I don't think it changes the underlying
point.

------
perlgeek
There is a disconnect in your messaging. Here on HN the title is "Learn what
DDD, CQRS, and event-sourcing are all about", but the tagline in the PDF is
"The semantic JavaScript backend for event-driven development".

So, what's this about? about DDD, CQRS and event-sourcing, or about a
particular Javascript framework? Those two aren't the same at all (even though
I suspect there is a little overlap).

~~~
goloroden
The brochure is to 2/3s an introduction to DDD, CQRS, and event-sourcing in
general, not being related to a specific JavaScript framework.

The last 1/3 shows how to apply these concepts using wolkenkit, which is a
specific framework.

When we created the brochure it seemed important to us to first explain the
concepts in a neutral way, because we want to help people gain a better
understanding of these concepts, no matter whether they are going to use
wolkenkit or not.

~~~
perlgeek
That's great. My point is that the title page does not properly represent
that.

You might have an easier time getting people to read it when they don't get
the impression "it's just another javascript framework".

------
shock
> In fact, not even credit institutes work with transactions, they too rely on
> eventually being consistent.

Is this really the case? I was always under the impression that everything
regarding financial transactions needed to be transactional.

~~~
andreareina
Consistency here is CAP consistency[1], not ACID consistency. In short, if
datum _d_ is visible to Alice at time _t1_ , it MUST also be visible to Bob at
time _t2 > t1_.

So financial transactions are transactional, but there's no guarantee that
every node has a fully up-to-date state at all times.

[1]:
[https://en.wikipedia.org/wiki/CAP_theorem](https://en.wikipedia.org/wiki/CAP_theorem)

~~~
jen20
This is also not how bank transfers work inter-bank. I as Bob (a recipient)
may have zero knowledge of a transaction initiated by Alice regardless of
time.

------
gspetr
I apologize for making a meta comment, but I had to modify the way the article
is displayed because it's an all too common example of this:
[https://www.wired.com/2016/10/how-the-web-became-
unreadable/](https://www.wired.com/2016/10/how-the-web-became-unreadable/)

Which is very annoying because the article about CSS can't get it's UI right.

They call Svelte "The magical disappearing UI framework" and indeed, their
fonts look like they are written in an invisble ink about to dry up.

------
aaronmu
Can anyone explain why eventual consistency is so scary? The way I see it,
eventual consistency is everywhere. Even in your non-distributed synchronous
CRUD app. The minute you start two db transactions within one script you have
eventual consistency. You have to deal with the fact that one transaction can
fail and that your state can be inconsistent.

~~~
jorgeleo
It really depends on the application. Eventual consistency means that
eventually, the databases will be consistent.

Some bank transactions, for example, do not have that luxury because it might
allow 2 possible answers to the same query if the eventuality has not been
met.

Some trading transactions don't have that luxury either because the
milliseconds that you waited for the consistency a stock might have changed
its price dramatically.

It really depends on the Domain if the time to wait for the eventuality is
acceptable or not.

~~~
nightski
Banks have daily lag (pending transactions). They also stop business in the
evening and resume in the morning to process all these items. That isn't
feasible in a lot of scenarios.

Rarely do you have an up to the microsecond view of the markets (unless you
are located appropriately and pay significant amounts of money). There is
always a delay in the information's accuracy and relevancy.

~~~
he0001
This "lag" is there mostly because historical reasons, not technical.

~~~
nightski
I'd rather argue they are there for "domain" reasons. For one, banks often re-
order transactions. I know my bank will process deposits first, then debits
for the day. This way you are less likely to incur overdraft fees. It's also
to fight fraudulent transactions. Although with the move to "real time" debit
processing (which is actually two batch jobs a day) they are placing limits on
the types of transactions that can be processed.

A domain that can't handle any delay in consistency is atypical in my
experience. Either way you can do DDD/CQRS without eventual consistency.

~~~
he0001
They are there because they are used to think that way. It's conways law
written all over it... There's no actual reason for them to do it. They
replaced their paperwork with a system that does the same thing. Fraudulent
transactions can you find without the need of summarizing it.

Yes you can do DDD/CQRS without ES but CQRS is not needed, it's just a
technical construct not solving any verifiable problem. DDD on the other hand
is more sane and to some extend, verifiable.

~~~
jorgeleo
Agree, the initial question was on eventual consistency, not on DDD.

------
raarts
Reading this.... It's just like react and redux, combined with redux-sagas. My
head is spinning.

~~~
gregyoung1
Only made 10 years prior.

------
bauerd
Offtopic, but was the PDF typeset with layouting software or a markup
converter such as Pandoc and the like? Looks really slick, kudos. Also is
there a way to tell by PDF metadata?

Edit: Had a second look and it's layouting software

------
goloroden
Hey :-)

I'm one of the authors of wolkenkit, so if you have any questions, feel free
to ask here :-)

~~~
codebeaker
As a non-native German speaker who's lived in Germany for coming up on 10
years, aren't you concerned that it'll be pronounced like "wol-kenkit" not
"volkenkit" for non-germans?

Naming things is hard but after watching UBER Inc destroy the pronunciation,
spelling and etc of a German word, I'm always cautious about unusual names.

(Also because, as now I'm hung up on the name before giving your product a
fair chance)

~~~
goloroden
Well, yes, there's a certain risk. We had the idea that people liked the word
"wolke" (German for "cloud"). Anyway, of course, I would be very happy if you
gave wolkenkit a second chance ;-)

~~~
codebeaker
I did - reading the brochure now, the word "semantic" keeps cropping up and I
wonder if it's an unsympathetic translation from German, or I'm too far away
from the JS world to understand that "Semantic" means:

> That‘s why we have built wolkenkit, a semantic JavaScript backend that
> addresses three important aspects of software development, and that empowers
> you to build better software faster.

There's a bunch of things wrong with this claim, "a javascript backend" sounds
like an alternative JS VM to me, a backend for javascript (I presume it's
meant that this is server side JS not client side?) I also always get my back
up when I read "build software faster", surely it ought to be about
maintainability when it comes to the perks of DDD/CQRS.

~~~
goloroden
Actually, you're right. What we mean is that it's a backend for JavaScript
applications, which itself uses JavaScript.

------
Yuioup
Run away. Run far, far away from CQRS and Event Sourcing.

~~~
codebeaker
I'm currently writing a book about CQRS/ES I'd be glad of your notes about
it's downsides.

~~~
k__
I think many people just dislike DDD and CQRS/ES because it's associated with
over engineering.

Either you are really good and get things right without much help, which isn't
often the case, or you end up in two other directions.

You over-engineer with DDD/CQRS/ES.

You under-engineer without them.

Most developers I met preferred the last solution, probably because they end
up with their "own" mess and not the mess of some process.

And I have to admit, the only people I met, that used these techniques, were
some "digital transformation" consultants who made their living by selling
this stuff to big corps, which isn't too convincing either.

~~~
naasking
> You over-engineer with DDD/CQRS/ES.

CQRS/ES definitely seems like a correct foundations for a distributed system,
I just don't think we have the right abstractions to properly model this
without wanting to stab yourself in the eye.

------
EdSharkey
Reshaping data is the source of so many costly bugs and communications
breakdowns in this world. CQRS is like descending to the 8th circle, pit 7 of
reshaping hell. What fetish drives one to such tortures?

If your schema is any more complicated than a plain stream of characters, and
you pick CQRS for your app, then you are an overpaid, bored, architecture
astronaut consultant looking to pad his resume. Nevermind the multi-million
dollar dumpster fire of an app you spearheaded and ditched out on before
completing (and that got cancelled, of course, a complete loss.)

If I ever saw CQRS on a resume, I'd unconditionally pass, you're not setting
foot in the door.

~~~
UK-AL
CQRS just simply means your queries, and commands have separate paths.

This can be as simple or as complicated as someone wants to make it.

It could be as simple as having commands operate on a domain object, but the
queries returning a fully populated view object from some custom sql.

~~~
EdSharkey
Are those objects in the queries and queues separate and distinct types then?
And, what about the object store, what types go in there? How do we handle
data migrations?

I like queues as much as the next developer. Queues let you scale performance
Mt Everest and deal with concurrency.

CQRS as an architecture is like queue abuse.

~~~
UK-AL
You don't have to use queues, separate objects/NoSQL store or anything like
that to use cqrs.

I mean you can use them, but all CQRS means is a separate code path for
queries and commands.

In my system I have command objects, they operate on domain objects which
contain some business logic.

I have query objects which just execute custom sql code which returns a
populated view.

This is useful because views often display specific data from multiple domain
objects.

This system allows you to bring back the entire view in a single query and
nothing else, rather trying to retrieve multiple domain objects which are
really designed for business rules rather than queries.

Often yes, people do starting adding queues, materialised views to CQRS which
can make it complicated.

~~~
EdSharkey
> Often yes, people do starting adding queues, materialised views to CQRS
> which can make it complicated.

You don't say?? You _don 't_ say??

Sorry to be flip, I hope you can tell that I've been burned. Badly. Thanks for
your detailed response. You obviously care and are not a stupid person nor do
you come off sounding vain.

But, you're not selling me on CQRS at all. What you've described is just
classifications and organization for types in a sane sounding UI design that
happen to have similar roles as types in a CQRS system.

If you were _selling_ me on CQRS, you'd start by talking about data flow.
Something like, "it's a realtime, live multiuser system where all commands are
queued, transformed to system events, and those events are committed to log
(via _another_ queue, we have to synchronize after all) that stretches back
into time. We can replay those events, rewind time, events are merged ..." And
it goes on and on into the stratosphere of architecture astronaut-hood...

My problem with your response is I honestly can't trust what you say here
until I do a code review. Is your actual code pathological, or are you
whitewashing the constant twizzling and niggling you have to do to keep your
ship afloat? How bad, honestly, do you have to work to add a new feature?
What's your velocity like? I want to see how much work and layers there are in
your code to do data type transformation.

Data transformation is where El Diablo lives, because data transformation
requires lots of people to agree and business terms to meet code - it's the
unholy marriage of garbage disposal, forest fire, and whirlwind. CQRS seems to
invite _needless_ data transforms that other architectures do not require, and
that's the pukey taste in my mouth when I have discussions about CQRS.

~~~
UK-AL
"it's a realtime, live multiuser system where all commands are queued,
transformed to system events, and those events are committed to log (via
another queue, we have to synchronize after all) that stretches back into
time. We can replay those events, rewind time, events are merged ..." And it
goes on and on into the stratosphere of architecture astronaut-hood..."

But that isn't CQRS. That's event sourcing, a complicated implementation of
event sourcing.

CQRS can work on a standard SQL system, you can design the database using a
standard normalised database.

All that has to be different is that commands and queries are separate.

A simple implementation;

A command can literally be

    
    
      var command= new AcceptOfferCommand(db);
      command.Execute();
    

A query can be

    
    
      var query = new AvailableOffersQuery(db);
      var viewModel = query.Execute()
    

Inside the command you may retrieve a domain object from repository, and
execute a method on it.

Inside the query method you may just perform a raw sql statement which grabs
data from multiple tables, populates a viewModel and returns it. Rather trying
to retrieve multiple domain objects from multiple repositories and then
shuffling that data into a result..

That's one way of doing simple cqrs. It can be simpler than this. But this
code just made it obvious.

CQRS allows you do very complicated things, but it doesn't mean you have to.

~~~
EdSharkey
Ah, well CQRS is only justifiable when you're _selling_ the work if you couple
it with the ES, and the more complex it is, the better the sales job and the
budget. It's like peanut butter and jelly.

My main point in replying to you was that you weren't selling me on whatever
brand of CQRS you think you've got. Why? It seems like you're atomizing your
business logic into a fine mist, why is that good? And you still haven't
talked about all the myriad transforms you do: where's the payoff? Does YAGNI
apply in your case?

~~~
UK-AL
Well each command is a use case. Each use case can be tested independently
without having setup controllers and their dependencies, or big service layers
and their dependencies.

Plus I can run the usecases from a console frontend for quick testing without
much setup.

I don't really do any transforms other than mapping domain objects into the dB
or database results into a query result.

Why do I need it? Because I practise DDD, and domain objects are not designed
for queries. To populate a single view you have pull multiple domain objects
from repos. This can be slow and adds complexity.

Query objects allow me to query the database using standard SQL or stored
procedure and put the output into query result using only a single query.
These queries can as complicated and optimised as they need to be.

Meaning I can avoid all DDD abstractions that I don't need for querying, but I
do need for enforcing complicated business rules on the command side.

~~~
EdSharkey
If you're able to reuse (through encapsulation or extension) your domain
object types in your commands and queries without transformations, then your
pattern sounds sane to me.

~~~
bonesss
The entire point of the Separation in CQRS is to avoid unnecessary coupling
between reporting concerns, GUI concerns, and user concerns.

The act of starting an order does not look like an order. Asking for an order
cancellation does not look like an order. Using GraphQL to pull an order
summary with report-specialized drill down capabilities does not look like an
order. These specialized domain objects should be modeled individually. By
trying to encapsulate all these facets in a single object what you get is
tight coupling, compromises, domain impedance, logical errors, and slow
development.

In context: One of the root principles of DDD is that each domain represents
its entities as is befitting that domain. Domain entities should be meaningful
in their domain and _not_ corrupted by every niggling redigestion of the same
facts through tens, or hundreds, of simultaneous conflicting views...
Reporting concerns should be first-class citizens when solving reporting, for
example.... View concerns and high-load systems often require denormalization,
materialization, transformation, and replication that has very little to do
with the user of origin while also _destroying_ the premises of traditional
ORM driven RMDB interactions.

At scale, under load, while supporting multiple N-tiered applications and
their accordant services, unified domain models break down. Aggregate
boundaries map these epistemological distinctions.

CQRS on top of that is just good engineering. In the same way modern car-
tunnel builders build two separate tunnels for traffic in each direction, CQRS
isolates low-frequency user interaction from high-frequency reporting (or vice
versa!), allowing independent scaling and direct delivery of features without
impossible coordination activities between teams or finding silver-bullet
technologies that may or may not exist.

That said: these are Enterprise solutions to Enterprise issues... Scale is the
reason they exist. If you aren't looking at a horrible mess around a unified
domain model then these solutions might not apply. If you aren't dealing with
multiple simultaneous data delivery platforms (mixing SQL server and Oracle
and Mongo DB and S3 and DocumentDB and DynamoDB all at once), these solutions
may not apply. If you're not balancing multiple teams working on multiple
products that have a constant need for harmonic data exchange, these solutions
may not apply. If you're not juggling BigData, if you're not dealing with
complex business logic, if scaling up an extra ten thousand users isn't
something you deal with, if you're not supporting multiple client types, and
so on: these solutions may not apply.

~~~
EdSharkey
Sounds really complicated, like "usually badly designed and written code"
complicated.

GraphQL and Falcor are interesting, and I like the sort of query-as-befits-
the-view aspect to working with it. Taking a wait-and-see on those.

~~~
bonesss
Commands are objects that express a command. A query is something that
provides a value.

You can code it up in minutes, and most of the code is pure plumbing to
connect to datasources. If that sounds complicated to buy independent control
over data exchange then you're simply not exposed to the complex multi-layered
and distributed applications that demand a structured approach to the issues
CQRS handles. Unquestioned and implicit assumptions are great for small
potatoes.

Domains should model domains and reflect the mental models of the people who
work on, and with, the system. If that is experienced as complicating the only
conclusion is that you're working with subject material neither complex nor
interesting enough to warrant a considered architectural approach.

For someone who deals with computing (currying, monads, SOLID, etc), you
should know that trying to meaningfully translate superficial domain
complexity to generalizations of code quality or design quality of any given
solution in all domains is laughable.

~~~
EdSharkey
Laugh away, I'm not interested in flashy code nor resume padding. I _am_
interested in SOLID, clean code. If you have clean code that is easy to add
new features to, refactor, and teach to new developers, then good for you. I
know good code when I see it. It is clear to me when developers lose sight of
the forest for the trees.

Your sales technique makes your project sound like torture.

~~~
bonesss
I'm not selling anything, and you clearly are missing the forest for the trees
while making assumptions & assertions about unfamiliar things.

CQRS boils down to a single principle: two one-way roads make for easier road
maintenance and improvement that one two-way road.

Two individual channels are too complicated to code? Torturous? Inherently
badly designed? Hard to add new features too? Unclean? ... Crazy talk,
unsupported by reality. In fact: CQRS specifically addresses all these points
you've brought up through its decoupling.

Those kinds of prejudiced assumptions do not hold water, cannot be logically
supported by the underlying design nor in-production systems, and reveals only
domain ignorance.

Successful engineering around distributed Enterprise systems tells a clear
story: those maintenance and improvement concerns are predominant at scale.

Opinions may vary... Informed opinions on this matter do not.

~~~
EdSharkey
And now for the coup de grace: you produce a link to sources containing a
shining example of CQRS.

Truly, one look at this archetypal wonder, (which certainly every programmer
worth his salt has already studied its types and patterns at length), and all
doubters shall melt away. If there ever were a more killer app demonstrative
of the state of the art, it should be the muse of comp-sci journals or even
popular culture!

A toy Microsoft CQRS spike on github from a couple years back does not count,
neither does a book reference. I want a canonical example of something real:
deployed code to users, thriving and successful. I'll be a happy to learn
something new.

My suspicion however is you're all talk - a super-consultant who sold some
CQRS work and has an axe to grind because you have co-workers reading this
thread, including some of the client's devs. You arrogantly bragged to them
and then waded out to sea because you thought me an easy mark to prove your
claims to expertise on the subject.

I am prepared to gracefully concede to my betters.

