Where I work we don't use any process level parallelism at the ruby level, we hoist that up to the kubernetes level and use signals (CPU load, job queue sizes, etc) to increase/decrease capacity. Workloads (replica sets) are segmented across multiple dimensions (different types of API traffic, worker queues) and are tuned for memory, cpu and thread counts according to their needs. Some heavy IO workloads can exceed a single cpu ever so slightly because db adapter isn't bound by the GVL, but practically speaking a pod/ruby process can only utilize 1 CPU, regardless of thread count.
One downside of this approach though is it takes a long time for our app to boot and this along with time to provision new nodes can cause pod autoscalers to flap/overprovision if we don't periodically tune our workloads.
In a perfect world we would be able to spawn processes/pods that are already warmed up/preloaded (similar to forking, but at the k8s level and the processes are detached from the root process) in a way that's not constrained by the CPU capacity of some underlying k8s node it is running on and instead is basically an infinite pool of CPUs that we only pay for what we use. Obviously serverless sort of offers this kind of solution if you squint but it is not a good fit for our architecture.
In my past experience with a large rails monolith, memory usage was always the limiting factor. Just booting the app had significant memory overhead. Using in-process concurrency would have led to massive infrastructure savings, since a lot of that overhead could be shared across threads. Probably 2-3x the density compared to single threaded.
In the end, we never got quite there, due to thread safety issues. We did use a post-boot forking solution to achieve some memory savings thanks to copy-on-write memory, which also led to significant savings, but was a bit more complex.
All that to say, the naive "just let kubernetes scale it for you" is probably quite expensive.
Very interesting talk, I remember trying to setup Falcon for a simple app once but the docs was lacking and I didn't get it to work, but this was years ago. It seem to have matured a lot since then.
I also remember the Parallel gem being higher performing than Ractors, is that still the case? Does anyone know?
I would love to use a fiber based app server for Rails to gain parallelism with db queries and remote service requests. I've been burned so badly by thread safety issues that I do not want to deal with threads.
it just listed the ways you can handle concurrency and parallelism in ruby 3 and what they each do.
Aka Threads which are backed by OS threads but are limited by the GIL when it comes to parallelism.
Fibers which are green threads aka windows 3 style concurrency.
Forking Processes.
and Ractor which is backed by a thread pool and offers real parallelism.
I would say that fibers, when used with Samuel’s ecosystem of gems act as node’s event loop (which I am guessing more people on here would be familiar with). They can auto switch on I/O like Ruby’s threads if you’re using a new enough version of Ruby.
Cool, thanks! Always curious to hear if there's something new out there.
I have always enjoyed using the BEAM ecosystem for concurrency, but Ruby has so much code and is great for "getting stuff done", so it's mostly what I keep using.
Where I work we don't use any process level parallelism at the ruby level, we hoist that up to the kubernetes level and use signals (CPU load, job queue sizes, etc) to increase/decrease capacity. Workloads (replica sets) are segmented across multiple dimensions (different types of API traffic, worker queues) and are tuned for memory, cpu and thread counts according to their needs. Some heavy IO workloads can exceed a single cpu ever so slightly because db adapter isn't bound by the GVL, but practically speaking a pod/ruby process can only utilize 1 CPU, regardless of thread count.
One downside of this approach though is it takes a long time for our app to boot and this along with time to provision new nodes can cause pod autoscalers to flap/overprovision if we don't periodically tune our workloads.
In a perfect world we would be able to spawn processes/pods that are already warmed up/preloaded (similar to forking, but at the k8s level and the processes are detached from the root process) in a way that's not constrained by the CPU capacity of some underlying k8s node it is running on and instead is basically an infinite pool of CPUs that we only pay for what we use. Obviously serverless sort of offers this kind of solution if you squint but it is not a good fit for our architecture.
reply