Additionally, his premise that CPUs will continue to get more and more cores is probably also flawed. The diminishing returns of the extra cores due to Amdahl's law will make them less and less economically viable.
Intel and AMD want to continue making processors that cost more than $150. If feature size continues to shrink, what else are they going to do with the extra room on the chip? (other than slash their own prices using ever-more-expensive fab equipment)
In single threaded code, the unused cores can be powered off.
Recently I heard a talk by a guy from Oracle designing Sparc CPUs. IIRC, he said that while they could potentially put bigger caches on their CPUs, for their test-cases it wasn't that much of a benefit above their chosen size.
Another interesting note from that talk was that they had spent a large amount of time making an 8 socket architecture with quick-connect paths between all pairs of sockets for memory transport. Unfortunately that was not as important as believed either, since the programs that typically benefit from 8 CPUs normally have good memory properties anyway.
Not sure if either data-points are true, but interesting nonetheless.
And with programs for which Amdahl's Law isn't a huge concern, multiprocessing isn't necessarily even the most appropriate form of parallelism. Multiple cores is great for situations where the sequential fraction of a program is relatively high, because it helps minimize the cost of synchronization.
But it can also bring some problems that might limit performance, such as bottlenecks on shared resources such as memory and I/O. If the problem can divided into fairly large and independent chunks of work, on the other hand, then distributed computing becomes a lot more attractive.
That would only be true if systems are only capable of running a single program. Luckily I think there will be utility in running 10,000 processes on a box at the same time.
