Rayon abstracts away the parallelism for you. Yes Parallelism is easier in compi...

chrismorgan · 2024-10-22T06:10:26.000000Z

> Someone could write "rayon" for webworkers.

Not true. JavaScript’s threading model is entirely insufficient for something like Rayon. At best, you could get either something that only worked with shared byte arrays, or something that was vastly less efficient due to structured clone. At best, you have something far more manual and somewhat slower, or something a little more manual and much slower.

Rayon is a magnificent example of making something that is impossible in scripting languages easy. Of making something that you can only feebly imitate with some difficulty, trivial.

jpc0 · 2024-10-22T08:57:54.000000Z

> only feebly imitate with some difficulty, trivial.

My point was the API simplicity not the technical correctness, which is why my post discussed threading in the first place.

Yes Rayon isn't possible in JS, but a "rayon" api like multi-threaded library that you can reach for in cases where it makes sense is absolutely doable.

chrismorgan · 2024-10-22T10:16:00.000000Z

Having thought further about this, I’m going to double down on this, because it’s worse than I was even contemplating. What you describe would be as like Rayon as a propped-up, life-size cardboard cutout of a house, is like the depicted house.

Rayon’s approach lets you write code that will run in arbitrary other threads, inline and wherever you want to. That’s absolutely essential to Rayon’s API, but you can’t do that in JavaScript, at all: workers don’t execute the same code (it’s not based on forking), and interaction between workers is limited to transferable objects, or things that work with structured clone, which excludes functions.

No, you can’t get anything even vaguely like Rayon in JavaScript. You could get a feeble and hobbled imitation with untenable limitations or extra compilation step requirements (and still nasty limitations), and that’s about it.

With Rayon, you can add parallelism to existing code trivially. With JavaScript, the best you can manage, which is nowhere near as powerful or effective even then, requires that you architect your entire program differently, significantly differently in many cases, and in ways that are generally quite a bit harder to maintain.

If you wish to contest this, if you reckon I’ve overlooked something, I’m open to hearing. I’m looking for something along these lines to work:

  import { f1, f2 } from "./f.js";
  let n1 = Math.random();
  let n2 = Math.random();

  await par_iter([1, 2, 3, 4, 5])
      .map(n => f1(n + n1))
      .filter(n => f2(n + n2))
      .collect();

Where the mapping and filtering will be executed in a different worker, and collect() gives you back a Promise<Array>. The fact that f1 and f2 are defined elsewhere is deliberate—if it didn’t close over any variables, you could just stringify the function and recompile it in the worker.