
Introduction to Parallel Computing - r11t
https://computing.llnl.gov/tutorials/parallel_comp/
======
eliben
This is an excellent resource for learning parallel programming, on several
levels. Don't miss these related tutorials from the same source:

* Pthreads: <https://computing.llnl.gov/tutorials/pthreads/>

* OpenMP: <https://computing.llnl.gov/tutorials/openMP/>

* MPI: <https://computing.llnl.gov/tutorials/mpi/>

------
l0nwlf
I liked the way the content is presented. I had Parallel Computing as one of
my elective this semester, and the subject isn't kool any more when you dive
into it. Parallel Computing is tough IMHO and it takes some brainstorming to
solve even the basic problems like computing prime numbers or sparse matrix
multiplication. Also we use OpenCL, which itself is quite a complicated deal.
( TBB, OpenMP are simpler ). The tutorial makes the basics clear as well as
the post have an aesthetic feel to it. All in all - a nice read.

~~~
Untitled
Stupid question (I know embarrassingly little about GPGPU programming). Is
OpenCL mostly used with ATI cards? How common is OpenCL? It seems to me that
almost everyone is using CUDA.

Also, isn't copying everything to the GPGPU memory and then performing tasks
there (and when it is completed copying it to main memory) a bit stupid? What
happens in a cluster setup when two or more hosts work on the same dataset?
Won't this result in a needless amount of copying? Double memory requirements?

~~~
eliben
OpenCL is different from CUDA in two important points:

1) OpenCL is an industry standard supported by many vendors (although not all
support it fully currently, since it's a relatively new standard). OTOH, CUDA
is a proprietary system developed by NVIDIA for its GPUs. CUDA is much more
mature since it has been in use for quite some time now.

2) While CUDA addresses GPUs, the aim of OpenCL is more general. It's a
"heterogeneous" language/platform, aimed by design at various computing
devices: CPUs, GPUs, special CPUs like Cell, accelerators (FPGA, etc.), DSPs
and so on

~~~
phamilton
CUDA actually is open, but ATI chooses not to implement CUDA support in their
drivers.

------
_corbett
the website is a little hokey but I found the tutorials/mini courses on MPI
and OpenMP on <http://ci-tutor.ncsa.illinois.edu/browse.php> very useful

------
phamilton
here's a link to the current course I'm taking, in case anyone is interested
in working through some assignments.
<http://dna.cs.byu.edu/CS484/schedule/schedule-winter2010>

It uses the llnl tutorials for most of the reading.

