This will be a niche area for few very good programmers. A general purpose developer should not risk this. If it does not pan out, then you've wasted a lot of time.
Frankly, the vast majority of applications don't have these classes of concurrency, capacity and computational concerns and can be adequately scaled in the traditional fashion.
Exactly what type of application are you referring to? Most server software does a lot of work in parallel. Many computationally complex algorithms are parallelizable to one degree or another. Video games usually have a lot of parallelizable number crunching, analytics very often run quite slowly, UI code requires at least a couple threads of execution, on and on goes the list... it's time to adapt the idea of traditional to include more than sequential programs
I presume that most applications that most people use most of the time do not need to scale. Most websites don't need to scale, most business apps don't need to scale, and most userland applications don't need to scale. Single-core sequential programs do just fine on comparatively resource constrained hardware for "everyday" applications. with the exception of maintaining a separate thread for UI, the examples you gave are edge cases.
I am into game dev, music dsp and highly concurrent web application development... but I recognize that these are all fringe activities.
parallelization is not only about scalability, but also performance. If a task takes 20min and can be parallelized 4X, it now takes 5 minutes. There's only a certain portion of each program that is inherently sequential. We're at 2-4 cores now, soon it will be 8-16. I'm not really in a position to comment on average joe computing... but my social circle runs PCs down to a halt regularly. If I had 8 cores and all of my apps were setup to exploit it, I'd be a very very happy man and I don't think I'm in a minority. There's analytics on large databases - every enterprise does this. There's gaming - imho not a fringe activity at all. Exactly what are these day to day applications? CRUD web apps with no analytics? word processing which certainly do parallel processing of spellchecking etc? firefox? photo editing? I honestly can't think of a program worth writing that couldn't stand to use multiple threads of execution - even gedit polls files for modification.
I agree that increased parallelization of common computing tasks could increase performance, but I believe that performance is acceptable (by the consumers and management) for most applications.
Programming games is fringe programming. I believe that most PCs are used for: CRUD web apps, VB/C# apps, productivity apps, document editors, ecommerce clients, collaboration, and communication clients.
Sure, faster programs would be great! I want it as much as the next person. Sadly, I don't see it being the Next Big Thing. I hope I'm wrong! I do think that the present high-performance community is going to get nicer tooling for parallel programming, and some of that may find its way into your everyday consumer apps but that will be in forms like Grand Central Dispatch or other library adjuncts to existing platforms, and therefore not a technological leap but rather a step in the right direction.
Polling files for modification is an example of not taking advantage of platform capability, where existing performance is "good enough." Most platforms now have filesystem hooks for modification, and if you want to improve the speed of gedit you can register an event handler.
This is something I've been looking into and I'm becoming convinced that given the wide range of (expert level competency in) domains required (memory models, processor architectures, concurrent data structures) to effectively address the issues, it is likely that after a short period of language/software layer solutions, the issues will be addressed at the hardware level. Its an intrinsically hard problem that I doubt can ever be made practical for the general (IT level) programmer community.
What I mean is, every algorithm or solution designed with software can always, by definition be implemented in hardware, bringing vast improvements in performance and scalability. This will always happen as long as hardware continues to get faster and there is no reason to believe that it won't continue to get faster exponentially for the foreseeable future. Quantum computing, 3D processors, light based data storage, carbon nanotubes, dna circuits...
Soon, we will have Databases and Web Servers implemented in hardware... When graphics was the rage, Intel put MMX in the CPU. Now we have two CPUs in the CPU, or four. We put SSL in hardware to speed that up. We can and do put it all in hardware eventually.
There are already lots of instant hardware solutions you can burn with EEPROM, PLAs, and the like. I don't see any reason Intel or AMD won't let consumers upload programs to burn right into the silicon. It'll be like embroidering polo shirts.
Hardware tends toward being general-purpose over time. It only makes sense to do dedicated hardware for things that aren't feasible to do using general-purpose silicon at the time (notable exceptions are for simpler things like ethernet controller chips, although if a general-purpose CPU is cheap enough to replace the chip, maybe that will change). As you pointed out, hardware speeds do get exponentially faster over time, which makes it less desirable to go to the effort of designing purpose-specific hardware.
The problem with current FPGA and CPLD solutions where you would be able to upload your own algorithms is that currently they operate much more slowly than mask manufactured ASICs. Not to say that couldn't change at some point, but even then the line between software and hardware is pretty blurry. Tools for designing correct implementations of an algorithm in hardware that take full advantage of being in hardware (very fine-grained concurrency, etc) are difficult to master.
Sorry, it is hard to hit these points in depth in a short post, but hardware has a set of concerns that are separate from software, and treating them as exactly equivalent is not really accurate, even if functionally and algorithmically they are the same.
You are right about the tools being difficult to master. I've been doing some work with FPGAs lately, and I see one big problem. The hardware is really cool. But the software is horrible.
I'm working with Altera tools. To do any work, I need to download about 5Gb of stuff. The IDE is painfully slow, badly designed, and crashes about once every hour. Everything is closed-source. Basically, the development process is at least a decade behind that for software work.
Xilinx tools are also closed (don't know if they are as unusable though). I think that if one of the FPGA companies opened up their toolkit, they could win big. But they are too worried about their proprietary routing algorithms to do that.
Of course, that's a given. This whole business with Ln caches and cache-line misses is a sort of déjà vu in scale (and now people are talking about MVCC and transactions at the memory level ;)
So yes, any software system can be embodied in hardware, but when a problem is so fiendishly difficult and optimal solutions require (sophisticated) algorithms matched to hardware specifics, then the cost equation tips heavily in favor of hardware embodiment. A significant barrier to the n-core future is pedagogical and I'm guessing that will further tip the balance towards a systemic solution embodied in hardware to hide the concurrency issues.
Well, that will be okay. Do you want to spend your time building the game or writing ray tracing algorithms? You want to call Array.sort(), not write a sort. It just takes up time.
It's interesting that CISC won out over RISC considering the conversation we are having. It's simpler for the builder. You can repeat yourself less. Imagine doing a quicksort in assembly. No way, right? Not today. Why?
And beneath that CISC instruction set is a RISC core and a translation layer. The fact that the instruction set is CISC has more to do with backwards compatibility (hello, Windows) than an actual advantage of CISC.
It also has to do with CISC code being more compact. With memory getting slower and slower relative to the CPU, this is increasingly an (overlooked) advantage. Maybe at some point CPUs will do automatic code decompression in hardware.
Until you look at anything other than PCs. I'm reasonably sure there are more ARM chips in operation than x86, and ARM is (if I recollect correctly) a RISC processor.
It's an intrinsically hard problem, but one that can't necessarily be solved at the hardware level.
For a large number of processors on the right tasks, just getting instruction level parallelism (e.g.) is not even tiny bit as good as using an optimized algorithm.
I understand and agree with you, but that is by definition accessible only to a very specialized expert group.
The differentiation aspect of hardware is that it involves both logical and physical domains, where as software is limited to logic. We may require new approaches to hardware and architectures as the "intrinsic" aspect of the problem is very much rooted in underlying hardware architecture.
(I also wouldn't be surprised at all if the initial successful applications of quantum computing are memory related.)
I think the general (IT level) programmer community doesn't require more processing power to do their jobs than they would have had on a single core desktop in 1999, anyhow.
It may be true that programmers don't strictly require the power, but interesting things become possible when you can afford to waste a lot of power. This should be evident by looking at Rails etc. In another 10 years who knows what will be possible by wasting another 100x processing power?
Parallel execution, map/reduce, hadoop, noSQL datastores;