
High-Performance GPU Computing in the Julia Programming Language (2017) - open-source-ux
https://devblogs.nvidia.com/gpu-computing-julia-programming-language/
======
maleadt
Author here, happy to answer any questions! We've been developing and
maintaining this toolchain for a while now, so the relevant packages
(CUDAnative.jl for kernel programming, CuArrays.jl for a GPU array
abstraction) are much more mature. Our focus has recently been on implementing
a common base of array operations that can be used across devices (GPU, CPU,
etc), so that users can develop using the base CPU array type, quickly benefit
from a GPU by switching to CuArrays, only to rely on specific CUDA-specific
functionality from CuArrays/CUDAnative when they need custom functionality.

------
adamnemecek
Julia is one of my fav languages. For numerical computing, neither python +
numpy, nor matlab come even close. The interop is nuts. To call, say numpy
fft, you just do

using PyCall

np = pyimport("numpy")

res = np.fft.fft(rand(ComplexF64, 10))

No casting back and forth. This is a toy example, julia ofc has fftw bindings.

Interop with C++, MATLAB, Mathematica etc is similarly simple.

~~~
siproprio
In theory Julia is supposed to be fantastic.

In practice, things either don't exist, or are poorly implemented:

Plotting simple things take 30 seconds.

And that's if you don't count the time it takes to `] add Plots`, especially
on Windows!

And the REPL is broken.

And the editor is slow and annoying (Juno or vscode).

And documentation ranges from poor (no examples, buggy between platforms,
broken links due to version updates) to non-existent. For example, lots of
tutorials will often link to broken links to official documentation, links
that one time were thought to be working but now aren't.

And so on...

~~~
socialdemocrat
Yes plotting in Julia is slow upon first invocation due to the JIT. Annoys me
too, but to say the REPL is broken is profoundly puzzling to me. It is the
best REPL I have ever used. It beats anything I have used for Python, Ruby,
JavaScript, Lua etc.

Also your documentation issue is also strange. Yes certain things don’t exist
but I would say the Julia docs is quite well made. In particular if you use
the REPL documentation I find it much better than Python. Tends to be quite
nice examples, color coding etc.

~~~
Oreb
> It is the best REPL I have ever used. It beats anything I have used for
> Python, Ruby, JavaScript, Lua etc.

This is true, but that's a _very_ low bar to pass. Julia is a Lisp, and
deserve to be compared to other Lisps rather than to lesser languages. Every
Common Lisp or Scheme I have used has a vastly superior REPL experience than
Julia. Even Clojure is better.

Don't get me wrong: I love Julia, and I hope it will eventually replace Python
as the main language for scientific computing, data science and machine
learning. But the REPL experience, at this point, leaves a lot to be desired.
I'm sure it will improve in the future.

~~~
StefanKarpinski
I’m curious what specifically you would want improved in the REPL.

~~~
siproprio
Does the REPL on Windows have all the features and niceties and quality of
life of the REPL on bash or other OSes?

If so (which isn't), then we can start suggesting new features, perhaps better
text editing capabilities, or introspection, better access to documentation.

~~~
StefanKarpinski
The REPL on all platforms is the same. There’s an issue with old buggy
versions of cmd.exe, but that’s only on such old versions of Windows that
they’re not even supported by Microsoft anymore.

------
systems
Just pointing out that this is an article from 2017 and was discussed before
on hn
[https://news.ycombinator.com/item?id=15564639](https://news.ycombinator.com/item?id=15564639)

------
vasili111
I always glad to see topics about Julia. I think it has good potential to
replace several other languages with one better language.

------
sytelus
Why this infrastructure is so tightly coupled with CUDA? CUDA is very specific
and closed APIs for NVidia hardware only. Programming languages should focus
on more general primitives that might work on NVidia or TPUs or something
else. PyTorch also has CUDA all over in its APIs and its frustrating to see
such tight binding with closed one company API. Also take a look at OpenCL.

~~~
darknoon
It's because CUDA performs better. It's not nice, but it's the situation we're
living in. Particularly AMD support and performance are lot.

~~~
shmerl
It doesn't perform better than what you can do in Vulkan. It's simply more
entrenched.

------
m4r35n357
Julia is presented as a simple language, but is is anything but that in
practice.

~~~
ddragon
Julia is not presented as a simple language, it's presented as a "I want
everything" language [1], a Python-Ruby-Perl-C-Fortran-Lisp-Matlab crossover
with it's own unique spice. Which is completely opposite from something like
Go. You can start programming knowing only one of Julia's inspiration, for
example programming Julia like Python, but if you want all the language brings
you'll have to dive in a lot of the other sides (which might clash a little
with the cleverness of the compiler, as it will accept such varied styles it
will not guide you to the one through way of idiomatic Julia code).

Still the Julia team did a great job in making all those diverse features feel
part of one connected philosophy instead of an ad hoc pile of functionality,
even if it does take a little while to fully internalize it.

[1] [https://julialang.org/blog/2012/02/why-we-created-
julia](https://julialang.org/blog/2012/02/why-we-created-julia)

------
shmerl
_> The performance possibilities of GPUs can be democratized by providing more
high-level tools that are easy to use by a large community of applied
mathematicians and machine learning programmers._

How exactly CUDA is "democratizing" anything, if it's tied to Nvidia? Vulkan
backend would make more sense for that purpose.

~~~
rrss
That sentence explains perfectly well what it means by democratizing, and how
is independent of the platform being tied to nvidia.

~~~
shmerl
Can you elaborate please? I was under the impression that CUDA is tied to
Nvidia, unless you mean there are now working shims for other GPUs.

~~~
Athas
CUDA is ultimately an API. AMD even has a converter for transforming CUDA cuda
to something more portable[0].

While it would be better in a democratic sense for GPUs to be accessed using a
fully free API, having an easily usable proprietary API is still more
democratic than a difficult-to-use API (especially when, as here, the easy-to-
use layer is actually fully free, and can perhaps be retargeted to fully free
lower layers later).

[0]: [https://gpuopen.com/compute-product/hip-convert-cuda-to-
port...](https://gpuopen.com/compute-product/hip-convert-cuda-to-portable-c-
code/)

~~~
shmerl
It still looks like porting idea, not like a shim that makes CUDA run on AMD.
So I'd say CUDA is still locked to Nvidia. AMD are trying to ease up the
transition to portable options - that's surely good, but it's not a full
fledged lock-in unlocking.

I'd say, Nvidia are being hypocritical here, with this whole "democratizing"
claim. They are direct beneficiaries of the lock-in they are advancing with
it.

