
EventStore: Open-Source, Functional Database with Complex Event Processing in JS - juancampa
https://github.com/eventstore/eventstore
======
tobr
I've played around recently with a Redux inspired frontend architecture where
the complex, nested reducers are replaced with a single one that simply adds
every action to an immutable linked list (i.e, `(state, action) => ({state,
action})`. This means all the business logic that would normally live in
reducers can be moved over to memoized selectors, which fixes the awkward
difference in how reducers and selectors are composed that tend to happen in a
typical Redux app, and gets all the declarative benefits of selectors. It
makes it easy to load code for new selectors asynchronously (for example when
navigating to a new route), and because the state is a long list of every
state your app has ever been in, they can "rewind" as far back as they need to
sync up correctly with the rest of the app.

What I've realised recently is, this is basically event sourcing. Replace
actions with events, selectors with projections, and memoization with
snapshots. So this submission is certainly timely and of interest to me.
Thanks!

~~~
arnioxux
I wrote a similar comment before, viewing redux as a realization of the event
sourcing pattern:
[https://news.ycombinator.com/item?id=17061827](https://news.ycombinator.com/item?id=17061827)

You can go even further and use the exact same "events" or "actions" for event
sourcing on the server side too. You just need to save the list of actions to
the server. This lets you do realtime sync and multiplayer really easily!

~~~
atoko
This.

It makes Implementing CQRS in a single service really easy. GETs load the
event log, replay and display the results while POSTs send a command that gets
translated into an event. If it is valid it gets appended to the log. This
approach is storage agnostic: It doesn’t matter how the log ultimately gets
stored

------
megakid
We use this heavily in production and to good effect. It has its downsides
(doesn't every product) but we have found it works well once you've grasped
them. If you have a primarily .NET focused team, have plenty of event-driven
applications and want to do CQRS / Event Sourcing for some of your
applications I would recommend it as a starter step towards that. It gives you
a lot 'out of the box' which is great when starting out but over-reliance on
some of its features can make it hard to replace. Make sure you have a good
handle on retention and an archival strategy. Storage maybe financially cheap
but it's certainly not maintenance friendly once your database grows beyond
certain sizes (we're currently running our own fork to solve some of these
issues whilst the PR process is going on).

~~~
rjbwork
We just wrote our own on top of Sql Server. Runs fine for thousands of
accounts with many events per day. We currently host it on the smallest azure
sql scale level. We have virtually unlimited scaling with azure, and should
have no problem sharding it in future because we include extra metadata (event
type, accountid, etc.) so we could choose to shard on any of those dimensions.

------
TheLudd
I do event sourcing in my company and we had a look at this when we started
out 4 years ago. What I don't understand is why build a database? Why not
build something in the application layer that uses your fav database as a
storage mechanism instead? Aren't the existing databases more mature and
better to use?

~~~
BillinghamJ
There are a lot of non-trivial aspects to scaling event sourcing systems.
Particularly if you want atomicity or asynchronous two-phase committing.
Having implemented them ~5 different ways for different services at my
startup, I’d definitely welcome a reliable DB-level abstraction.

~~~
Fire-Dragon-DoL
Sorry for my inexperience in this kind of implementation but wouldn't an sql
transaction achieve something similar? And with table partitioning (at least
on postgres) you should be able to go quite far, instead of neediness a new db
(and a lot of related stuff to study)

~~~
BillinghamJ
No because in a distributed system, you can't rely on SQL transactions. i.e.
if you need to make one API request to start the transaction and a second API
request to commit it/rollback - you can't hold the SQL transaction open
between those.

------
platz
Event Sourcing seems like the ultimate YAGNI architecture.

You make an architecture that optimizes for future features at the expense of
a coherent flow for the current features that you know you have.

it's like if you made pub-sub the core architecture for your system. very
actor-system like, but also probably harder to understand the runtime behavior
- control flow.

~~~
hudo
Try to solve complex architectures that can handle race conditions without
some kind of event sourcing, good luck with that

------
Yuioup
I've had to break it to my client that had built a CRUD application using CQRS
and Event Sourcing that they have wasted a huge amount of resources on a
misguided architecture. A rewrite is pending.

~~~
root_axis
> _A rewrite is pending._

So another huge amount of resources wasted.

~~~
honestlyidk
rewrites are really the on thing all programers enjoy doing. If it works dont
touch it, should have caught it sooner. Make the next project better and move
on.

------
twic
Some ex-colleagues were considering using this a few years ago. They were
getting interesting in CQRS, got Greg Young in to give a talk, and came away
filled with enthusiasm. I think they ended up doing something incremental on
top of the MySQL they already had, rather than switching over to this.

Is there a particular reason this is suddenly exciting again now?

------
kyleperik
"Functional Database" is contradictory. Honestly this sounds like a bunch of
buzzwords concatenated together. Is there an example or a purpose for this?

~~~
lojack
It isn’t that contradictory. You can have data in a functional language. One
of the common features of a functional language that separates it from
imperative is data immutability. In this case it’s a database of immutable
data. I personally understand exactly what they’re talking about when they
call it a functional database.

~~~
kyleperik
I don't understand how this is useful. The purpose of functional programming
isn't to make everything immutable, but to use immutability to prevent
unnecessary change of state which can result in flaky code. So where's the
usefulness in a functional Database?

Also this still isn't functional. Is adding records to a set not mutation?

~~~
federicoponzi
I think your question is answered here: [https://eventstore.org/docs/event-
sourcing-basics/event-stor...](https://eventstore.org/docs/event-sourcing-
basics/event-store-as-a-functional-database/index.html)

~~~
kyleperik
Oh I see I misunderstood then. But still, how is this a database? Without
looking into it too much it seems that would either be an ORM or simply state.

~~~
naasking
A database is simply an organized collection of data with a restricted
interface for accessing and/or querying said data. This seems to qualify.

~~~
kyleperik
Should we call PANDAS a database?

~~~
naasking
Add "durable" qualifier to my definition, since otherwise you could say that
it's just a data structure. Which it is, but with durability properties.

------
Nelkins
Good article on how EventStore can be used in production:
[https://medium.com/@eulerfx/scaling-event-sourcing-at-
jet-9c...](https://medium.com/@eulerfx/scaling-event-sourcing-at-
jet-9c873cac33b8)

------
fulafel
What does functional mean here?

------
mattdeboard
What's it good at? What is this the right tool for?

~~~
mrdoops
Persisting a log of events. Used with CQRS and event-sourcing to decouple the
Read model(s) from the Command layer for benefits like: multiple read models
(projections), time-travel and audit-ability (rebuild the read model to
certain point in time), and decoupled contexts.

Event-sourcing can be a really useful tool in many domains, but especially
where having a state-of-the-art audit log is helpful.

~~~
slowmovintarget
In the parlance of Domain-Driven Design, every aggregate (think object
instance as a loose equivalence) is its own stream of events. Loading
aggregates involves replaying events until you are caught up.

The key design requirement here is to deeply understand the reads your
application will need to perform.

The downsides are you have to push the data somewhere else to digest and
report on it. Relations are tough to model, too, as events that happen to more
than one aggregate essentially have to repeat the data in each stream in the
form appropriate to that domain entity. (You can often model this as one event
causing another. Projections work less well for this circumstance.)

This can make it hard to draw data from the underlying store. Instead you must
"hydrate" it into objects in memory to ask questions of it, though projections
can help. EventStore is definitely not the right choice if you need ad hoc
query capability. (I used EventStore in production for 2 years with F#.)

I'd classify this as an exotic data store. Use it if you have a really strong
need for event-sourcing. Me... I'd likely choose Datomic instead.

------
circleoncircles
It's not a new concept.

~~~
topspin
Was some claim to a new concept made somewhere? I haven't seen this.

A new realization of an existing concept can be notable.

