On: A look at the J language: the fine line between genius and insanity (2012)

[p1esk]

If your primary interest is in parallel code that runs on a GPU, and not autograd features of Pytorch/Jax, take a look at CuPy: https://cupy.dev/

[dTal → p1esk]

That's extremely cool. Julia has something very similar, in the form of CuArrays [0].

I do wish we had some better cross-platform solutions though, instead of cementing nVidia's hegemony. There's no technical reason why GPGPU should be any more difficult on AMD hardware.

[0] https://github.com/JuliaGPU/CuArrays.jl

[marmaduke → dTal]

OpenCL was pretty good in this a few years ago. I had some kernels which ran across AMD and Nvidia GPUs as well as many core Xeon systems without changing anything. To squeeze the highest performance out, you still had to tune for the device, but it was a great start.

Now, all vendors seem to have dropped support for OpenCL, though it will live on nicely in PortableCL, which implements it atop LLVM, allowing it to target anything LLVM does, which finally goes further to achieve the open-ness the original standard aspired to, than the vendors themselves at the time.

[eigenspace → dTal]

> I do wish we had some better cross-platform solutions though, instead of cementing nVidia's hegemony. There's no technical reason why GPGPU should be any more difficult on AMD hardware.

Then look no further than GPUCompiler.jl. Tim factored out all the platform agnostic parts of CUDA.jl (successor of CuArrays.jl and CudaNative.jl) into GPUCompiler.jl and that is now being used by the AMDGPU.jl package. It works today!

[dTal → eigenspace]

Thanks! I had hit some of those pages before, but their ontology isn't immediately obvious, so thanks for the clarification. In particular I've definitely looked at AMDGPU.jl recently, but as the page says: "not all features (and especially performance) are up to par with CUDA.jl" - hence my impression of AMD being a second-class citizen. Still, it's great that we have anything at all, and I'm extremely grateful for the hard work of those in the Julia community. It's a remarkable achievement.

[eigenspace → dTal]

> In particular I've definitely looked at AMDGPU.jl recently, but as the page says: "not all features (and especially performance) are up to par with CUDA.jl" - hence my impression of AMD being a second-class citizen.

That just takes developer time to fix. Currently it’s basically just a one-man show.

[marmaduke → p1esk]

Right, but does CuPy fuse kernels? I can’t find anything in the docs about it.

[p1esk → marmaduke]

Not sure, but you can write your own cuda kernels inline if you want to.