That's why their entire business model -- like Astronomer's -- is geared toward cloud hosting. The architecture is so complex it takes a full time team to run it.
Have you looked into DBOS? Same thesis: durable and reliable workflows are hard to manage -- it just doesn't have to be as hard as Temporal makes it be :)
Have not. For my workflows this was fine. Good to keep in mind thoUgh. I don’t plan to manage a truly distributed system with it. Plus my only reason to do so is professional and we rolled our own system here due to our size solutions like DBOS or Temporal would not work well.
I've talked to dozens of engineers who built their home grown "durable" stack. Most of them eventually moved on to buying vs building, when their system actually scaled. It's just not a side-hustle to build a foundational reliability layer.
That argument comes down to the scalability of RabbitMQ or one's database, both of which can scale fairly well, but require tuning. In the absolute worst case, one would have to use a distributed cloud database, e.g. AWS Aurora or AWS DynamoDB, otherwise a self-hosted one, e.g. TiDb or YugabyteDb, but far less than 1% of users would even need anything like it.
In the pre-AI era, the argument of using a third party tool or service even had some weight, but today, AI can even do much of the heavy work when pointed in the right direction wrt using the aforementioned. For the majority of users, a SQLite database will do the job.
Yeah, we've observed that too: people start implementing their own retry logic, idempotency, etc. But then they grow a hard to maintain, complex stack that's not their core business logic. There's a reason why there is a dedicated team building DBOS, every day. Because it's not that easy to build a solid durable workflows engine on Postgres.
In pg19 https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit... will land, which significantly improves NOTIFY performance. Right now LISTEN/NOTIFY doesn't scale to very busy instances because a `NOTIFY` within a transaction takes a global lock.
Well another POV is, AWS sells RDS instances capable of global lock NOTIFY. Clearly people have been using it despite it being really slow.
It's a terrible architecture but does it matter? This article should really say "AWS is a useful but expensive way to run your apps," which isn't say much of anything at all.
reply