This is something I've been looking into and I'm becoming convinced that given the wide range of (expert level competency in) domains required (memory models, processor architectures, concurrent data structures) to effectively address the issues, it is likely that after a short period of language/software layer solutions, the issues will be addressed at the hardware level. Its an intrinsically hard problem that I doubt can ever be made practical for the general (IT level) programmer community.
What I mean is, every algorithm or solution designed with software can always, by definition be implemented in hardware, bringing vast improvements in performance and scalability. This will always happen as long as hardware continues to get faster and there is no reason to believe that it won't continue to get faster exponentially for the foreseeable future. Quantum computing, 3D processors, light based data storage, carbon nanotubes, dna circuits...
Soon, we will have Databases and Web Servers implemented in hardware... When graphics was the rage, Intel put MMX in the CPU. Now we have two CPUs in the CPU, or four. We put SSL in hardware to speed that up. We can and do put it all in hardware eventually.
There are already lots of instant hardware solutions you can burn with EEPROM, PLAs, and the like. I don't see any reason Intel or AMD won't let consumers upload programs to burn right into the silicon. It'll be like embroidering polo shirts.
Hardware tends toward being general-purpose over time. It only makes sense to do dedicated hardware for things that aren't feasible to do using general-purpose silicon at the time (notable exceptions are for simpler things like ethernet controller chips, although if a general-purpose CPU is cheap enough to replace the chip, maybe that will change). As you pointed out, hardware speeds do get exponentially faster over time, which makes it less desirable to go to the effort of designing purpose-specific hardware.
The problem with current FPGA and CPLD solutions where you would be able to upload your own algorithms is that currently they operate much more slowly than mask manufactured ASICs. Not to say that couldn't change at some point, but even then the line between software and hardware is pretty blurry. Tools for designing correct implementations of an algorithm in hardware that take full advantage of being in hardware (very fine-grained concurrency, etc) are difficult to master.
Sorry, it is hard to hit these points in depth in a short post, but hardware has a set of concerns that are separate from software, and treating them as exactly equivalent is not really accurate, even if functionally and algorithmically they are the same.
You are right about the tools being difficult to master. I've been doing some work with FPGAs lately, and I see one big problem. The hardware is really cool. But the software is horrible.
I'm working with Altera tools. To do any work, I need to download about 5Gb of stuff. The IDE is painfully slow, badly designed, and crashes about once every hour. Everything is closed-source. Basically, the development process is at least a decade behind that for software work.
Xilinx tools are also closed (don't know if they are as unusable though). I think that if one of the FPGA companies opened up their toolkit, they could win big. But they are too worried about their proprietary routing algorithms to do that.
Of course, that's a given. This whole business with Ln caches and cache-line misses is a sort of déjà vu in scale (and now people are talking about MVCC and transactions at the memory level ;)
So yes, any software system can be embodied in hardware, but when a problem is so fiendishly difficult and optimal solutions require (sophisticated) algorithms matched to hardware specifics, then the cost equation tips heavily in favor of hardware embodiment. A significant barrier to the n-core future is pedagogical and I'm guessing that will further tip the balance towards a systemic solution embodied in hardware to hide the concurrency issues.
Well, that will be okay. Do you want to spend your time building the game or writing ray tracing algorithms? You want to call Array.sort(), not write a sort. It just takes up time.
It's interesting that CISC won out over RISC considering the conversation we are having. It's simpler for the builder. You can repeat yourself less. Imagine doing a quicksort in assembly. No way, right? Not today. Why?
And beneath that CISC instruction set is a RISC core and a translation layer. The fact that the instruction set is CISC has more to do with backwards compatibility (hello, Windows) than an actual advantage of CISC.
It also has to do with CISC code being more compact. With memory getting slower and slower relative to the CPU, this is increasingly an (overlooked) advantage. Maybe at some point CPUs will do automatic code decompression in hardware.
Until you look at anything other than PCs. I'm reasonably sure there are more ARM chips in operation than x86, and ARM is (if I recollect correctly) a RISC processor.
It's an intrinsically hard problem, but one that can't necessarily be solved at the hardware level.
For a large number of processors on the right tasks, just getting instruction level parallelism (e.g.) is not even tiny bit as good as using an optimized algorithm.
I understand and agree with you, but that is by definition accessible only to a very specialized expert group.
The differentiation aspect of hardware is that it involves both logical and physical domains, where as software is limited to logic. We may require new approaches to hardware and architectures as the "intrinsic" aspect of the problem is very much rooted in underlying hardware architecture.
(I also wouldn't be surprised at all if the initial successful applications of quantum computing are memory related.)
I think the general (IT level) programmer community doesn't require more processing power to do their jobs than they would have had on a single core desktop in 1999, anyhow.
It may be true that programmers don't strictly require the power, but interesting things become possible when you can afford to waste a lot of power. This should be evident by looking at Rails etc. In another 10 years who knows what will be possible by wasting another 100x processing power?
