Not sure why these not issues with state machines...
With RSM you often need to keep the history as well, just in a different form, that is the message/event log that your state machine processed.
With RSM you need your code to be deterministic to rebuild the materialized state during failover, unless you snapshot and replicate the state on each processed event/message.
I love KDE Connect, I use it both with KDE and Gnome (GSConnect). Both from the reliability perspective, but also on the shell integration side of things "Send to ...".
Hands down one of my favourite Desktop apps for Linux, kudos to the developers.
At Restate.dev (workflow as a code engine) we use Rust/WASM to code-share the implementation of the durable execution protocol between the restate-server and the SDKs to develop workflows. This code is essentially a "pure" state machine: events in, events out, no side effects.
We use WASM in some cases:
* Our Typescript SDK uses wasm-bindgen to compile and release the Rust part of the code. We have chosen WASM over Node native extensions because we wanted to support deno, cloudflare workers, bun, and potentially other runtimes. We've stumbled on few issues related to packaging, but except that it was a smooth process.
* Our Golang SDK uses WaZero + manual bindings using protobuf's. It was a bit of a manual process to set it up, especially surrounding concurrency issues wrt accessing the WaZero runtime plus you need some wasm runtime pooling to get decent performance, but at the end it works well and reliably.
My lesson learned from our experience is that Rust is the real player here, because it delivers on the promise that you can develop libraries that can be easily embedded in high level languages. It's really "write once, bind everywhere".
WASM is just a "packaging"/"distribution" detail in our case, and we picked it just for lack of alternatives. For example in our Python SDK, we could have used WASM but we didn't, instead we went with PyO3 which is an amazingly well done Python -> native code bindgen, and it works without hassle for users as opposed to, for example, CGO in Golang. If Golang had a PyO3-like solution without CGO/any hassle for users, I would have taken it as opposed to doing the WASM bindings myself.
Plus, from what I've tried myself, the whole WASM experience is great only when used in combination with TS/wasm-bindgen, in the other scenarios it's a lot of manual tedious memory moving code involved. Maybe when WIT gets more broadly adopted, and there will be bindgens available in most languages/engines, this will be different.
(Disclaimer: I work at Restate on SDKs) This is a very interesting point. I did some investigation myself, and so far I'm torn apart on whether a novel language would really make such a big difference for durable execution engines like Restate.
Let me elaborate it: first of all, what would be the killer feature that justifies creating a whole new PL for durable execution? From what I can tell, the thing that IMO can really make a difference would be the ability to completely hide durable execution from the user, by being able to take snapshots of the execution at any point in time and then record those in the engine transparently. Now let's say such language exists, and it can also take those snapshots reasonably fast, it is still quite a problem to establish where it's logically safe to take a snapshot, and when the execution cannot continue because you need to wait acknowledgment for stored results. Say for example you have the following code:
val resultA = callA()
val resultB = callB(resultA)
Both A and B do some non-deterministic operation, e.g. they perform HTTP calls to some other systems. Now let's say that when callB() completed, but before you got the HTTP response, your code for whatever reason crashes. If you didn't took any snapshot between callA() and callB(), you will completely lose forever the fact that B was invoked with resultA, and the next time you re-execute A, it might generate a result that is different from the one that was generated the first time. Due to this problem, you would still need to somehow manually define some "safepoints" where it's safe to take those snapshots. Meaning that we can't really hide the durable execution from the user, as you would still need some statement like "snapshot_here" to tell the engine where it's safe to snapshot or not.
In our SDKs we effectively implement that, by taking the safe approach of always waiting for storage acknowledgement when you execute two consecutive ctx.run().
Oh wow, thanks for the depth in your reply. well I don't know anything about programming languages but something just made me ask this question out of curiosity. I may have to play around with restate a bit
(Disclaimer, I work for Restate on SDKs) We definitely plan to introduce helpers for the common use cases, for example right now we provide helpers to generate random numbers or uuids. We could definitely provide a wrapped HTTP client that records the HTTP calls you perform and store them in Restate, in particular in Golang this should be easier than in other languages, given the std library provides itself an HTTP client we can wrap/integrate on.
In general, we aim to make our SDKs as tweakable as possible, such that you could easily overlay your API on top of our SDKs to create your own experience.
With RSM you often need to keep the history as well, just in a different form, that is the message/event log that your state machine processed.
With RSM you need your code to be deterministic to rebuild the materialized state during failover, unless you snapshot and replicate the state on each processed event/message.