

CPU and GPU trends over time - TalGalili
http://www.r-bloggers.com/cpu-and-gpu-trends-over-time/

======
cyrus_
There are many architectural differences between CPUs and GPUs that make it
difficult to draw the kinds of conclusions suggested by these graphs.

Data parallel algorithms with high computation-to-communication ratios, good
data locality and low local branch divergence will see these kinds of speedups
when rewritten for GPUs. This is an important subset of algorithms, but don't
expect GPUs to replace CPUs for more general-purpose computation any time
soon.

~~~
manvsmachine
Very true. It is for the same reason that the article shows GPU progress
outpacing CPU's in regards to FLOPS. The limited scope of a GPU's
functionality allows most of the processor's silicon to be arithmetic units.
CPU's, on the other hand, have huge sections of their real estate dedicated to
handling conditional logic (something most GPU's fail miserably at), legacy
instruction handling, virtualization, etc.

------
ChuckMcM
Thanks for posting this, its an excellent insight.

It was interesting when Intel fail to deliver on their roadmap. Here is a link
from 2000 where Intel was alleged to be saying 10Ghz by 2011
[http://www.geek.com/articles/chips/intel-predicts-10ghz-
chip...](http://www.geek.com/articles/chips/intel-predicts-10ghz-chips-
by-2011-20000726/)

Which of course lead to the great Multi-core debate (AMD started it, Intel
followed with two physical die on a package later to integrate into a single
die).

Which lead to the great Memory channel bottleneck (Intel was in-order single
channel, AMD was multi-channel) and resolution (everyone now is multi-
channel).

GPUs on the other hand have always had memory bandwidth issues so they went
wide, the GPU equivalent of multi-channel, from the start. They also benefit
from a feature of graphics which was 'do this same algorithm on all of these
texels' which is straight forward to parallelize.

And as nuclear weapons designers, and genomic researchers will tell you, GPUs
are also useful in doing MIMD simulations.

Its less clear what the parallel architecture breakthrough is for OLAP type
problems (or parallel file systems as an even easier example) These have lots
of operations in parallel but they also have high connectivity of data
synchronicity across those parallel threads which, at the moment, brings
everything down to earth.

Fun times!

------
Rexxar
I would have preferred a logarithmic scale for the last graph
(<http://csgillespie.files.wordpress.com/2011/01/gpu3.png>).

~~~
lwat
Here's one I just made using his data: <http://imgur.com/TSFYB>

~~~
eru
Thanks. Looks more similar now than on the linear scale.

------
6ren
If someone figured out how to write regular apps on a GPU (which would amount
to parallelizing the app), it could change everything.

1\. What would be a first step towards this? It seems the big problem is that
most code is sequential... and the opportunity is those parts that don't need
to be, but just happen to be written that way.

2\. Perhaps another opportunity is to try to remove the layers that make
modern software so slow. Can they be distributed across all pipelines somehow?

3\. Finally, a radical approach is not to try to convert existing apps, but to
think about the whole problem of an app from the very beginning (what inputs
will you get, what outputs do you want), and try to write _that_ directly on a
GPU. By making it concrete, you might see new tricks, perhaps by redefining
the problem (the app's features).

~~~
sedachv
This is what parallelizing Fortran compilers do. The space of programs that
can be automatically parallelized is all computable programs (see
<http://en.wikipedia.org/wiki/Church–Rosser_theorem>). The space of programs
that can be _efficiently_ automatically parallelized is very small.

~~~
adrianN
Many problems aren't parallelizable for reasonable definitions of the word
(i.e. polylogarithmic depth and polynomial work: Nick's class) unless there
are some major breakthroughs in complexity theory.

~~~
sedachv
As a class of problems, probably (although it hasn't been proven either way),
but in terms of programs it comes down to the vagaries of data structures and
algorithms - if the dataflow in your reduction graph makes it look like a
linked list, there's only one way to reduce it.

Guy Steele gave a really good talk about this at ICFP 2009:
[http://labs.oracle.com/projects/plrg/Publications/ICFPAugust...](http://labs.oracle.com/projects/plrg/Publications/ICFPAugust2009Steele.pdf)

~~~
eru
The good thing is that functional programmers will barely notice that they are
using the new data structures. (It's still just filters and maps. Just your
fold-functions will take slightly different arguments.)

------
Synaesthesia
If this chart from nVidia is to be believed, the CPU speed increase will
continue for many years to come. They're projecting almost a 100x increase in
speed from Tegra 2, in 2014!
[http://images.anandtech.com/reviews/SoC/NVIDIA/Kal-
El/roadma...](http://images.anandtech.com/reviews/SoC/NVIDIA/Kal-
El/roadmap_sm.jpg)

~~~
Klinky
How are they going to get there? They could very easily pump up the shader
count to get something that is "100x" performance, while not actually being
100x as powerful for general purpose operations. The graph looks shady to say
the least.

------
Derbasti
Shit, I did not realize GPUs packed such a tremendous performance advantage
over CPUs (FLOPs wise). They seem to be several orders of magnitude ahead--
that is positively mindblowing!

And yet, all GPUs and CPUs of the whole world can just barely match one human
brain ([http://arstechnica.com/science/news/2011/02/adding-up-the-
wo...](http://arstechnica.com/science/news/2011/02/adding-up-the-worlds-
storage-and-computation-capacities.ars)). If that's not mindblowing, then
what?

~~~
jules
They compare instructions per second with nerve impulses per second. That's a
pretty useless comparison. I'm sure that all GPUs and CPUs could simulate a
human brain more quickly than a human brain could simulate all CPUs and GPUs.

~~~
adrianN
I'd venture to say that depends strongly on the problem the brain is trying to
solve and the problem the processors are trying to solve.

------
endlessvoid94
What does "computational power" mean in this context? There's a graph but no
explanation...

~~~
Synaesthesia
Probably number of floating point operations per second.

------
GICodeWarrior
This seems like an apples to oranges comparison with different scales and
units...

