Yeah exactly! The Kafka use case is a great one--specifically writing consumers that perform real-world processing on events from Kafka.
In fact, one of our first customers used DBOS to build an event processing pipeline from Kafka. They hit the "wall of complexity" you described trying to persist events from Kafka to multiple backend data stores and services. DBOS made it much simpler because they could just write (and serverlessly deploy) durable workflows that ran exactly-once per Kafka message.
Great question! Yeah, the biggest difference is that DBOS doesn't require a centralized workflow server, but does all orchestration directly in your functions (through the decorators), storing your program's execution state in Postgres. Implications are:
1. Performance. A state transition in DBOS requires only a database write (~1 ms) whereas in Temporal it requires a roundtrip and dispatch from the workflow servers (tens of ms -- https://community.temporal.io/t/low-latency-stateless-workfl...).
2. Simplicity. All you need to run DBOS is Postgres. You can run locally or serverlessly deploy your app to our hosted cloud offering.
Author here--thanks for the feedback! We'll update the docs to clarify the first three points assume that the database and application always restart and return online if they go offline.
For the last point, we'll clarify we mean that the code of the workflow function must be deterministic. For example, a workflow shouldn't make an HTTP GET request and use its result to determine what to do next, even though that's technically idempotent. Instead, it should make the HTTP request in a communicator (https://docs.dbos.dev/tutorials/communicator-tutorial) so its output can be saved and reused during recovery if necessary.
Yeah, I see, you mean deterministic in the strongest sense - no state, no side-effects. A pure function. Might be worth phrasing it that way as React devs are familiar with that terminology but not the former.
Author here--can you break that down a little more? A DBOS workflow can call out to external APIs using communicator functions (https://docs.dbos.dev/tutorials/communicator-tutorial), so it works well with other loosely-coupled services.
Co-founder here. Thanks for the questions! A couple advantages DBOS has:
1. It's a serverless platform. It manages application deployments, providing a simpler experience than trying to bolt a durable execution framework onto an app deployed on Kubernetes.
2. Transactional guarantees. DBOS does durable execution in the same transactions as your business logic, so it guarantees exactly-once execution for most operations while other workflow solutions are at-least-once. More details here: https://docs.dbos.dev/explanations/how-workflows-work
Our goal in these design decisions is to make DBOS look familiar to developers by supporting one of the most popular stacks right now, building an application with TypeScript and SQL on top of Postgres. Under the hood of our cloud platform (mostly Go + SQL), we're building on ideas from the academic project to provide new features like reliable execution/time travel, but like a good OS, we want to hide that complexity from users.
Thanks for the good question! Our free tier offers fixed resources per application (under the hood, a Firecracker microVM with 512 MB of RAM and 1 vCPU) that scale to zero when not in use. For paid users, we'll offer autoscaling per application--we'll share more details on that soon.
Durable execution is good for atomicity, but this approach also gives you isolation. If you're doing updates on both Postgres and Mongo and you want to guarantee both updates happen, Temporal can do that. But if you want isolation, for example the guarantee that any operation that can see the Postgres update can also see the MongoDB update, then you need distributed transactions like in Epoxy.
While we haven't tested this, I suspect the main scaling bottleneck in practice is the primary database, as every Epoxy transaction goes through it. If you wanted to run Epoxy at large scale, you'd need a primary that supports very high transaction throughput.
More seriously, I suspect that plain SQLite by itself, running in process in the main service, writing to a ':memory:' database for ultimate speed or perhaps to a local file with WAL mode for slightly more durability, could achieve pretty nice efficiency and work as the main transaction manager database.
In fact, one of our first customers used DBOS to build an event processing pipeline from Kafka. They hit the "wall of complexity" you described trying to persist events from Kafka to multiple backend data stores and services. DBOS made it much simpler because they could just write (and serverlessly deploy) durable workflows that ran exactly-once per Kafka message.