Looks very nice from the ergonomics point of view. Trying to figure out the "call stack" of a bunch of Actors communicating via promises and futures is a mess (the physical call stack doesn't match the logical one at all) and their solution looks great. Also great work on making Swift and C++ seemingly so interoperable--I didn't know that was possible.
I wish they talked about performance. FoundationDB has a lot of low-level optimizations to make the overhead for the concurrency primitives low, custom arena-based allocators, etc. Given that C++ is often chosen for systems code because of potential for low performance overhead, it would have been interesting to see some benchmarks of how well the new Swift code performs compared to the Flow/C++ code.
TLDR: they're porting it from C++ to Swift for the safety, ergonomics, and multi-threading guarantees.
Cool parts are that they compile C++ and Swift to LLVM bytecode, and then compile all of this together, so they get inlining and whatnot. In practice this means they can have C++ and Swift files side-by-side.
They add some special C++ attributes so that Swift understands how to refcount the given type (or maybe not refcount at all).
They also use Swift protocols and extension methods to improve the ergonomics of using C++ types in Swift.
Ergonomics were especially emphasized, so that people writing on FDB write new parts in Swift because they want to, not because they have to.
Strange... Presumably production deployments of FDB are on Linux. Last time I heard, Swift support on Linux is second-class. Is the plan to deploy on Mac OS instead of Linux? Or is Swift support on Linux sufficient for FDB needs?
I must say I am not a fan of this decision. FDB looked like a promising solution but not if it is tied to a niche language.
I'm not experienced with Swift development, but did some readings recently (sparked by this talk). It seems like the dev story on Linux is improving, part of it being the rewrite of the Foundation Framework in Swift[0]. The Arc browser developers are afaik even trying to make their SwiftUI-based codebase work on Windows!
Also, it might be niche on the backend, but it's definitely not niche overall.
It may be improving but it's not there yet, let alone battle-tested with years of production use in serious applications on Linux. And we are talking about a distributed database here, where a bug can very well lead to it eating your data. Imagine PostgreSQL developers decided that, say, Zig will be their successor language. We would rightfully call them insane, no?
> Also, it might be niche on the backend, but it's definitely not niche overall.
I think it's safe to say it will always be a niche language on Linux, which is what matters here.
Here is recent report of trying Swift on a non-Mac OS platform, it's pretty damning:
The great thing about FDB is that they have a testing infrastructure that is unlike any other database: https://m.youtube.com/watch?v=4fFDFbi3toc
If there is a Swift compiler bug that causes data loss I will bet money they will catch it (I will also bet they will find at least one Swift compiler bug)
That statement is just not true. Swift on Linux is more than ready and battle tested. I personally have ran a bunch of work with Swift on Linux already and companies like Apple or Amazon are using it as well in production systems.
Personally the great thing about Swift is that it brings the performance of a C like language but compared to Rust has way better ergonomics.
Of course there are some problems that need solving like the foundation stuff but it got a whole lot better in the recent years.
In my experience, it depends a lot on the sort of libraries you need. The library ecosystem is still relatively small and client centric. Swift can be a fast language (if you avoid reference counting) but many libraries are incredibly slow and clearly unsuitable for server-side use.
For instance, I recently needed reasonably fast JSON and CSV parsers with predictable memory usage for incrementally parsing potentially large files. The libraries I found were either too slow or clearly written for client side use cases like mapping a few kilobytes of config files to objects in memory. I ended up writing my own, which was absolutely not what I wanted to spend my time on.
The upside is that Swift's C interop is excellent and now it's even getting C++ interop. So one way around library issues is using Swift like Python, as a thin layer on top of C. But as this FoundationDB talk shows, no one is really happy with this state of affairs.
I think server-side and cross-platform Swift is moving ahead at pace. Things will be looking lot brighter in two or three years.
> Swift on Linux already and companies like Apple or Amazon are using it as well in production systems.
Hm, any citations on the latter? That is, companies like Apple and Amazon using Swift in production systems on Linux, especially for something similar to FDB? Thanks!
It strikes me a little surprising that such a mature package as FDB is making such a large move.
For example, I understand that Deno's key-value store is based upon FDB, which certainly lends credibility to FDB.
Because rust has no real c++ migration story. Apple has been putting a lot of work into swift/c++ interop specifically to support migration from c++ to swift.
Rust seems utterly uninterested in supporting gradual migration of C++, which makes porting millions of LoC untenable.
Additionally rust has no interest in ABI stability, which is actually important (the rust ABI stability story is to my knowledge “expose a C FFI layer”)
> Additionally rust has no interest in ABI stability, which is actually important (the rust ABI stability story is to my knowledge “expose a C FFI layer”)
There certainly are problems for which ABI stability might be important. It would probably be desirable to do better than "Expose a C FFI layer" and deliver such stability. It's not at all clear that you can actually do that today without a terrible cost.
In C++ the situation is that de facto lots of things are frozen in order to deliver ABI stability. Having no actual ABI layer to stabilize, in practice major C++ implementations just... froze everything that seemed like somebody might ever possibly expose it, and any time they weren't conservative enough they got shouted at by developers who had, inevitably, depended upon a stability nobody even realised could be cared about.
The cost of that freezing is that everything affected is worse than it should be, for everybody, not just those who needed all, or even some, of the ABI stability. Where Rust is entitled to re-arrange a data structure (A,B,C,D) as (D,A,C,B) because that's 5% faster and has 8 byte smaller aligned size, in C++ they daren't do any such thing, even if they never specified this structure because you can just expose it anyway and then rely upon that never changing thanks to ABI stability...
Maybe, Rust will adopt an actual ABI mechanism which lets you deliberately expose some non-C flavour data structures or functions in a stable way, thereby promising they will keep some particular shape forever. It's certainly possible for Rust to adopt such things without having the inadvertent problem C++ got because it has things in place which let it write that down cleanly which C++ didn't really do.
Rust's repr attribute controls how a type is represented in memory, so today you'd use the attribute [repr(C)] to indicate that you want the structure laid out the way C would for ABI compatibility reasons, but you could imagine a repr(Awesome) for some hypothetical Awesome agreed ABI with shinier features or a repr(Swift) or whatever.
> There certainly are problems for which ABI stability might be important.
Like when you're a library, as FoundationDB is.
C and C++ have a specific approach to ABI stability that induces a specific kind of misery (more generally the ABI is fragile), but Swift is ABI stable without that misery.
If the first thing you need to do to in order to be a library is lose all your language safety features by marshaling to and from C/C++ to create the actual API, even when both sides of the API are in the same language that's just a sad waste.
Also, Rust does not have C++ interoperability the way Swift does and instead requires a binding layer. Some projects are choosing to completely replace C++ with Rust, but there isn’t a good way for the two to coexist, or to incrementally transition a codebase.
I wish they talked about performance. FoundationDB has a lot of low-level optimizations to make the overhead for the concurrency primitives low, custom arena-based allocators, etc. Given that C++ is often chosen for systems code because of potential for low performance overhead, it would have been interesting to see some benchmarks of how well the new Swift code performs compared to the Flow/C++ code.