

Ask HN: Best GPU (scientific) programming tutorials? - tomrod

Interested to see what the real hive mind (at least its temporal lobe) would look for a GPU programming tutorial for a (relatively) amateur programmer.
======
manvsmachine
Here's what I used while doing an undergrad independent study a couple years
ago (except for the books; at the time, books on the subject didn't exist
yet):

 _Online Tutorials_ :

Dr. Dobbs - CUDA, Supercomputing For the Masses: <http://www.drdobbs.com/high-
performance-computing/207200659>

GTC Video Tutorials / Presentations:
<http://www.nvidia.com/object/gpu_technology_conference.html>

_University Courses_ :

GaTech ECE4893 - Multicore and GPU Programming for Video Games (not CUDA
specific, but covered): <http://users.ece.gatech.edu/lanterma/mpg/>

UIUC ECE498AL - Applied Parallel Programming:
<http://courses.engr.illinois.edu/ece498/al/>

_Books_ :

Programming Massively Parallel Processors: [http://www.amazon.com/Programming-
Massively-Parallel-Process...](http://www.amazon.com/Programming-Massively-
Parallel-Processors-
Hands-/dp/0123814723/ref=sr_1_1?ie=UTF8&qid=1296443374&sr=8-1)

CUDA By Example: [http://www.amazon.com/CUDA-Example-Introduction-General-
Purp...](http://www.amazon.com/CUDA-Example-Introduction-General-Purpose-
Programming/dp/0131387685/ref=sr_1_2?ie=UTF8&qid=1296443374&sr=8-2)

If you decide to go the CUDA route, as opposed to OpenCL, then NVIDIA has a
lot of solid documentation that will help you get to a "Hello, World" level
pretty quickly. This StackOverflow thread should help if you choose to go with
an AMD OpenCL solution: [http://stackoverflow.com/questions/780978/opencl-
books-tutor...](http://stackoverflow.com/questions/780978/opencl-books-
tutorials). Personally, I would stick with NVIDIA; they seem much more
dedicated to advancing the cause and you have support for both API's.

------
octopus
If you use Mac or Windows search iTunes for CUDA programming, or search Google
for "Programming Massiveley Parallel Processors with CUDA" you should find
videos and slides for this Stanford course.

A good book about CUDA is:"CUDA by Example"

~~~
octopus
If you want to program for general GPU devices, you could search for OpenCL
(my first comment is valid only for NVIDIA GPU's).

However you should bare in mind that coding for OpenCL is, at least now, an
order of magnitude more complicated (personal opinion) than CUDA programming.

With CUDA if you already know C is straightforward to get started.

~~~
tomrod
Will look into it. I have experience with python, matlab/octave, R, and an
itsy bitsy amount of C#/SQL/Java from another life (figuratively).

~~~
octopus
I think you could use Python for CUDA programming (I didn't tried that), see
PyCUDA at:

<http://mathema.tician.de/software/pycuda>

even Matlab has some bindings for CUDA:

<http://developer.nvidia.com/object/matlab_cuda.html>

Bare in mind however that the original CUDA implementation uses C, so if you
want the last drop of performance you will use C.

~~~
tomrod
Can OpenCL be implemented on systems which do not support CUDA? I have a
relatively inexpensive HP bought about 3 years ago that, well, just won't run
CUDA even with an nvidia card :-)

~~~
manvsmachine
If you're still talking about about NVIDIA cards, then no; their OpenCL
implementation is based upon the CUDA architecture (technically, CUDA is the
device architecture that supports GPGPU programming, not the language itself).
That said, videocards are cheap these days, as long as you don't need the
high-end: you can snag an entry-level Fermi card (GTS450) for right around
$100 these days, or less if you go with something GT200-based.

------
wladimir
As you mentioned scientific, and a lot of scientists use numpy/scipy these
days, these links might be interesting to you:

Open source Python library that automatically generates linear algebra code
for the GPU, pretty awesome, seems to be a bit slanted toward machine learning
but is not specific to it: <http://deeplearning.net/software/theano/>

Simpler GPU programming with Python (somewhat more low-level, but more
friendly than using the plain C API):
<http://mathema.tician.de/software/pycuda>

