But what about data overwrites? This is basically the same as deleting data, since information will be destroyed. Using soft delete is a somewhat naive solution if there is no mechanism for restoring overwritten data.
Append-only tables are my preference. If you're worried about infinitely growing space, you can have some garbage collector clean up the least relevant records periodically for very little cost.
Can confirm, dotnet works great on Linux. We have a lot of dotnet production systems running on Linux and Docker for many years now with zero issues. Also recommend Rider in Linux for development, even our Windows folks prefer it over VS nowadays.
This was maybe a problem on the early days of .NET Core, but from .NET 6 or so they have reimplemented (almost) all the stuff from the old .NET Framework and that in turn enabled relatively easy porting of third party libraries, so most of them are ported as of today (atleast more or less maintained ones).
How do you refactor code, if you have a poor test coverage? Also for me actually the most importan benefit is the instant feedback I get, when I write the unit tests before the implemenation. I can move faster with more confidents.
Probably by working in an environment where that confidence is conveniently provided by static type analysis. If the parts are sufficiently reshaped to fit together again you just know that it will work. And chances are that feedback is an order or two of magnitude more instantaneous.
Event sourcing can be a very powerful pattern if used correctly. You don't need to combine ES with Eventual Consistency. ES can be implemented in a totally synchronous manner and it works very elegantly capturing nicely all the busines events you need. You get a very detailed audit for free and you don't loose importan business data. You can travel back in time, construct different views of data (projections) etc. Most of the complications arise when you bring in Eventual Consistency, but you absolutely don't have to.
I agree. Audit and history functionality have been the motivating features for me in building systems that are "ES-lite," where the event stream is not a general-purpose API and is only consumed by code from the same project.
Sometimes the ability for an engineer to fetch the history of a business object out of a datastore and explain what it means checks an "audit trail" requirements box. Sometimes showing the history of a business process in a UI, with back-in-time functionality, is a game changing feature. If so, using event sourcing internally to the service is a great way to ensure that these features use the same source of truth as other functionality.
Where you get into trouble is when you realize that event sourcing will let you distribute the business logic related to a single domain object to a bunch of different codebases. A devil on your shoulder will use prima facie sound engineering logic tell you it is the right thing to do. The functionality relates to different features, different contexts, different operational domains, so keeping it in the same service starts to feel a bit monolithic.
But in practice you probably don't have enough people and enough organizational complexity to justify separating it. What happens is, a product manager will design an enhancement to one feature, they'll work out the changes needed to keep the product experience consistent, and the work will get assigned to one engineer because the changes are reasonable to get done in one or two sprints. Then the engineer finds out they have to make changes in four different systems and test six other systems for forwards compatibility. Oops. Now you have nanoservices-level problems, but only microservices-level capabilities.
If you aren't tempted down that path, you'll be fine.
For an interactive application it's an absolute nightmare when you don't have read-your-own-writes support (including synchronization for "indexes"). Async background updates aren't always appropriate or easy to do.
few questions:
- what was your biggest ES system that you worked on?
- how many people worked on it?
- how did the ES system particulary solve your problem?
- what was the tech stack?
No, I'm not a consultant. Maybe lead developer is the most accurate title for me:) By what criteria the biggest system do you mean? They certainly weren't toy projects, these are real systems in active use solving real world problems for thousands of users. The ES part of these systems is mainly implemented in .NET using excellent Marten DB as ES store on top of PostgreSQL. I would say that ES changes drastically how you model things and see business. It forces you to identify and define meaningful events. These are often actually something that also non programmers understand so it also improves creatly communication with your clients(or in DDD terms domain experts:)) as a by-product. Scaling also has not been a real issue, these systems can handle thousands of operations per second without exotic hardware and setups, all this mostly synchronously.
And I must add, use the right tool for the job, there are many cases where ES is not a good fit. Also, if you choose to use ES in your project, you don't have to use ES for everything in that project. Same thing actually applies to asynchronous processing. If something in the system doesn't scale synchronously, that doesn't mean you now should do everything asynchronously.
Not the OP, not a consultant, the biggest ES system I work on is an order management system in one of the biggest investment bank in APAC, which processes orders from clients to 13 stock exchanges.
40 people approx work on it in Asia, maybe around 100 globally at the raw dev level.
I feel it's a sort of false good idea for our particular problems. Clients trade ether at the 10ms latency for high value orders or sub-ms for latency-sensitive ones. They conceptualise what they want to buy and a bunch of changes they want to apply on that bulk: for instance, change in quantity, price limits, time expiry and try to make money by matching a target price either by very closely following a prediction curve or spending as little time on doing so (giving us only a constraint and asking us to fit it by doing our own curve fitting algos).
The tech stack is pure java with a kernel of C++ for networking, with as little external dependency as possible. And no GC beyond the first few minutes after startup (to prealloc all the caches). Everything is self built.
How does it solve the problem: it simply is the only way we can think of to over optimize processing. If you want to fit a millisecond or sub millisecond target (we use fpga for that), you must cut the fat as much as possible. Our events are 1kb max, sent in raw tcp, the queue is the network switch send queue, the ordering has to he centrally managed per exchange since we have only one final output stream but we can sort of scale out the intermediary processing (basically for one input event how to slice in multiple output events in the right order).
I'd say it doesnt work: we run into terrifying issues the author of the main link pointed out so well (the UI, God, it's hard, the impossible replays nobody can do, the useless noise, solving event stream problems more than business problems etc), but I d also say I cant imagine any heavier system fitting better the constraint. We need to count the cycles of processing - we cant have a vendor library we cant just change arbitrarily. We cant use a database, we cant use heavier than tcp or multicast. I'll def try another bank one day to see how others do because I m so curious.
Are you really using event sourcing, or just message queues?
I worked for a large US bank, and had close contact with those who worked on the asset settlement system. I don't have as deep insight as you do, obviously. But the general architecture you describe sounds very similar. Except they clearly used message queues and treated them as message queues, and not event streams / sourcing / whatever. They used a specific commercial message broker that specializes in low latency.
reply