It's not recommended--the assumed model is that every workflow finishes on the code version it started. This is managed automatically in our hosted version (DBOS Cloud) and there's an API for self-hosting: https://docs.dbos.dev/typescript/tutorials/development/self-...
That said, we know sometimes you have to do surgery on a long-running workflow, and we're looking at adding better tooling for it. It's completely doable because all the state is stored in Postgres tables (https://docs.dbos.dev/explanations/system-tables).
"Starbucks can have LGBT mugs but hell no to unions". I think you hit the nail on the head. There is a whole chapter to be written about pro/anti "wokeness" stances used by companies / politicians to divert attention from the deeper class vs class issues.
Wokeness is also a way the media can smack down candidates like Corbin and Sanders, labeling them sexist or an antisemite for focusing on class instead of identity politics.
It's surprising to see an article with such a large encompassing of different techniques, hybrid techniques and design interactions with the type system, but is more surprising that a whole dimension of memory (un)management was left out: memory fragmentation
Three big differences in comparison with Erlang:
1- Cannot externally kill a process (yes, ergo process have a Kill method but the process will be in a "zombie" state until the current message handlers returns... maybe stuck forever)
2- No hot code reloading.
3- No per-process GC.
Most beam developers will tell you they don't use hot code reloading, but if you're an elixir/phoenix (especially live view) you are using hot code reloading affordances in dev, though it's not the full suite of capabilities
I have used a portion of that capabilities for live debugging on prod machines to test things out before putting it through source code and using the regular CI/CD for deployment. However, the CD deployment does not rely on hot reloading because it hasn't been necessary.
On Kubernetes, running with regular pods it was not desirable. Maybe if we were deploying with StatefulStates.
(Not affiliated with Ergo) I have been watching Ergo repo for a while now and you wrote my observation concisely.
A while back, I tried to solve no.2 on your list by having a dynamically expanding and shrinking go-routines. POC here: https://github.com/didip/laborunion. It is meant to be used in-conjunction with config library & application object that's update-able over the wire. This idea is good enough to hot-reload a small IoT/metrics agent, but I never got around to truly solve the problem completely.
That could be another way to have AWS not poach a popular Open Source project. Name it something extremely pro-unionisation, such that every time they write the name they'd be reminding their developers that unions exist. :)
Also, for anyone not completely familiar with Erlang's terminology, the translation of "per process garbage collection" to Go would be "per goroutine garbage collection". As mentioned in a sibling comment, this allows Erlang style garbage collection to avoid pausing the entire operating system process when running garbage collectin.
Per-process GC is an optimization similar to nurseries in regular collectors, esp any object that has been sent in a message must be visible globally (yes there could be small object optimizations but that would increase sender complexity).
Also an overlooked part here is that the global Erlang GC is easier to parallellize and/or keep incremental since it won't have object cycles sans PID's (that probably have special handling anyhow).
TlDr; GC's become way harder as soon as you have cyclic objects, Erlang avoids it and thus parts of it being good is more about Erlang being "simple".
Erlang avoids object cycles because it's impossible to make an old term point to a new one; data is immutable, so new terms can only referenece previous terms. This means the GC doesn't have to consider cycles and keeps things simple.
But that's separate from per process GC. Per process GC is possible because processes don't share memory[1], so each process can compact its own memory without coordination with other processes. GC becomes stop the process, not stop the world, and it's effectively preemptable, so one process doing a lot of GC will not block other processes from getting cpu time.
Also, per process GC enables a pattern where a well tuned short lived process is spawned to do some work, then die, and all its garbage can be thrown away without a complex collection. With shared GC, it can be harder to avoid the impact of short lived tasks on the overall system.
[1] yes yes, shared refcounted binaries, which are allocated separately from process memory.
> GC's become way harder as soon as you have cyclic objects
This may be true only for some implementations. Good GC implementations operate on the concept of object graph roots. Whether the graph has cyclic references or not is irrelevant as the GC scans the relevant memory linearly. As long as the graph is unrooted, such GC implementations are able to still easily collect it (or, to be more precise, ignore it - the generational moving GCs the cost is the live objects that need to be relocated to an older/tenured generation).
I'd like to see a reference to some GC actually doing explicit optimizations of this kind in a multithreaded scenario (not just as an indirect effect of theuir regular scanning), more or less linear scanning of memory is a natural consequence of a moving GC.
The Java gc's are doing some crazy stuff, my point however was that the acyclic nature of the Erlang object graph enables them to do fairly "simple" optimizations to the GC that in practice should remove most need for pauses without hardware or otherwise expensive read barriers.
It doesn't have to do a lot of things to be good, once you have cycles you need a lot more machinery to be able to do the same things.
Personally, I'm disappointed that there is too much superstitions and assumptions about GC designs going around, which lead to the discussion like this one that treats specialized designs with explicit tradeoffs, which both Go and BEAM are, as universally superior options. Or discussions that don't recognize that GC is in many scenarios an optimization over malloc/free and a significant complexity reduction over explicit management of arenas.
When it comes to Java - it has multiple GC implementations with different tradeoffs and degree of sophistication. I'm not very well versed in their details besides the fact that pretty much all of them are quite liberal with the use of host memory. So the way I approach it is by assuming that at least some of them resemble the GC implementation in .NET, given extensive evidence that under allocation-heavy scenarios they have similar (throughput) performance characteristics.
As for .NET itself, in server scenarios, it uses SRV GC which has per-core heaps (the count is sizing is now dynamically scalable per workload profile, leading to much smaller RAM footprint) and multi-threaded collection, which lends itself to very high throughput and linear scaling with cores even on very large hosts thanks to minimal contention (think 128C 1TiB RAM, stometimes you need to massage it with flags for this, but it's nowhere near the amount of ceremony required by Java).
Both SRV and WKS GC implementations use background collection for Gen2, large and pinned object heaps. Collection of Gen0 and Gen1 is pausing by design as it lends itself for much better throughput and pause times are short enough anyway.
They are short enough that modern .NET versions end up having better p99 latency than Go on multi-core throughput saturated nodes. Given decent enough codebase, you only ever need to worry about GC pause impact once you go into the territory of systems with hard realtime requirements. One of the better practical examples of this that exists in open source is Osu! which must run its game loop 1000hz - only 1ms of budget! This does pose challenges and requires much more hands-on interaction with GC like dynamically switching GC behaviopr depending on scenario: https://github.com/dotnet/runtime/issues/96213#issuecomment-... This, however, would be true with any language with automatic memory management, if it's possible to implement such a system in it in the first place.
I'm not gonna say it with 100% certainty, but having started using C# a few years ago i think that much of the smaller latency is simply because C# probably produces a magnitude less garbage than Java in practice (probably less than Go as well). The C# GC's generally resemble the older Java GC's more than G1 or especially the Zgc collector(that includes software based read-barriers whilst most other collectors only use write-barriers).
Small things like tuples (combining multiple values in outputs) and out parameters relaxes the burden on the runtime since programmers don't need to create objects just to send several things back out from a function.
But the real kicker probably comes since the lowlevel components, be it the Http server with ValueTasks and the C# dictionary type getting memory savings just by having struct types and proper generics. I remember reading some article from years ago about re-writing the C# dictionary class that they reduced memory allocations by something like 90%.
When you say "C# GC's generally resemble the older Java GC's more than G1 or especially the Zgc", what do you have in mind? I'm skimming through G1 description once again and it looks quite similar to .NET's GC implementation. As for Zgc, introducing read barriers is a very strong no in .NET because it introduces additional performance overhead to all the paths that were previously free.
On allocation traffic, I doubt average code in C# allocates less than Go - the latter puts quite a lot of emphasis on plain structs, and because Go has very poor GC throughput, the only way to explain tolerable performance in the common case is that Go still allocates less. Of course this will change now that more teams adopt Go and start classic interface spam and write abstractions that box structs into interfaces to cope with inexpressive and repetition-heavy nature of Go.
Otherwise, both .NET and Java GC implementations are throughput-focused, even the ones that target few-core smaller applications, while Go GC focuses on low to moderate allocation traffic on smaller hosts with consistent performance, and regresses severely when its capacity to reclaim memory in time is exceeded. You can expect from ~4 up to ~16-32x and more (SRV GC scales linearly with cores) difference in maximum allocation throughput between Go and .NET: https://gist.github.com/neon-sunset/c6c35230e75c89a8f6592cac...
Erlang is much more likely for GC overhead to grow sub-linearly, because more logic means more isolates (processes) rather than more state, more deeply nested, in the existing processes. Say at the square root of total data.
That makes sense. I wonder how important this is versus Go, considering Go has a sub-millisecond GC even without per-process GC? (Go also makes much more use of the stack which might be sort of analogous to per-process GC?)
I have some production experience with Golang and one thing that helps it emulate Erlang's BEAM VM (also used by Elixir, FYI) is to have the goroutines be short-lived and/or disposable. No need for persistent workers most of the time anyway unless you are chasing every last millisecond of performance -- in those cases persistent workers waiting to snatch a job definitely perform better (though only by something like 1-2% in my limited experience; but that can be still a lot depending on hosting and workloads and customer expectations f.ex. in finance forgoing 1-2% perf is almost criminal).
So the BEAM VM definitely handles things a bit better but Golang can get quite close.
One reason also is I believe it has something to do with fault tolerance even at a hardware level. A process has its data isolated somewhere in memory, if something happens to that memory, the process will crash next time it runs and starts causing supervisors to start attempting to recover the system
Much lighter impact on system performance that world-GC. Simpler algorithms as well, so lower risk that you’ll ever get performance regressions - or worse.
You don't. Go is generally significantly faster than Erlang and unless you are deeply concerned about the pauses themselves, you will more than recover GC time in generalized performance in almost all, if not all, cases.
Go's GC already has a lot of work done to minimize "stop the world" time down to very small values.
As much as I am a fan of Erlang's BEAM VM (and made a career out of Elixir that is still going today) I have to say that you are right -- Golang is doing extremely well and the incremental improvements over the last several years truly pulled it ahead. Unless you're constantly copying big objects and just produce a lot of trash then having your loaded program in Golang is going to be a super smooth sailing. And even if you stumble upon some of the traps, there is a lot of tooling to help you out identify a pain point and remedy it.
I am a big believer in the idea that engineers, real professionals, need to have a clear view of what kind of performance technologies deliver, and should never ever view performance statements as a political statement. They may be right or wrong, but they aren't political. This includes both not writing a project that needs very high performance and reaching for a known-lower-performance tool, and also not reaching for the absolutely highest performance tool which generally comes with a price when you are orders of magnitude away from needing it.
Erlang/Elixir has plenty of performance for plenty of problems, but Go is definitely generally a cut above. And there's definitely another cut above Go in performance, it's not the fastest, and there's a cut below Erlang/Elixir as well because they're generally faster than the dynamic scripting languages. And even the dynamic scripting languages are often fast enough for plenty of loads themselves.
This is true, but they come with a different set of tradeoffs w.r.t ecosystem, tooling and performance (which turns into per-node efficiency in this case). There is also a matter of comfort and team/personal preferences.
A brain is more energy efficient than a chip, but how a couple of cells connected to wires has more economic sense than a chip with gallizions of transistors?
This is closer to rent a transistor, and the main use case is to find out what its voltage curves are.
It is not a computer.
I did work on a version which used in vivo brains for actual computation. It worked but no one wanted to invest in building the matrix for rodents. With the benefit of hindsight I shouldn't have added the slide explaining ethical considerations: 'Container Labs in international waters help us manage regulatory risks'.
Yeah. My understanding was that you'd need constellations of neurons to achieve a complex result, rather than a couple individual ones. Perhaps this article is light on details and it's more involved.
You are right on the point. Also note that antisemitism was the ideological way to unify the image of a nebulous jewish german finance class with bolshevism, painting them as both sides of the same jewish conspiracy coin. This way the hatred to the capitalist class was diverted to hatred against bolshevism. Antisemitism was what made that ideological manoeuvre possible.
Using the latest advances in technology and computing to plan and execute an ethnic cleansing and genocide? Sounds familiar? If not, check "IBM and the Holocaust".
reply