What is dark silicon?
"Dark silicon is the fraction of chip that needs to be powered off at all times due to power constraints."
Long and short is that because of a combo of energy use and heat dissipation at the scale of upcoming transistors, maybe upwards of 50% of the transistors on a chip may need to be off. This then begs the question, why continue developing smaller scales of transistors if the technology can't be utilized because of heat and power needs.
Things like hashing/encryption functions, hardware encoders and decoders, even full blown FPGAs are all candidates.
You can also make more, slower cores, running at lower voltage, if you've got any embarrassingly parallel problems to work on.
It's not as attractive as just getting faster and more energy efficient across the board like we have been doing, but there's still room to move there.
Not yet, perhaps. New processes are effectively guaranteed to be needed at some point though.
So while for awhile now we knew we were going to hit the physical limitation of silicon at some point, the reality might actually be even worse because due to dark silicon and the economical constraints it imposes, there may not be enough incentive to keep shrinking.
This does not in any way rule out that the parts of the brain that are not showing up in the fMRI are participating in the thinking process at that moment. We have no idea whatsoever of how the thinking process itself unfolds.
Just because you may only be able to power x% of your chip, doesn't mean x% of your chip is useless. Some times you need to turn off parts of the chips anyways. Examples include cache misses or particular hardware flows. Other times you can decrease frequency due to program characteristics. For everything else we throttle because we cant power all the transistors at all times.
Curious how the author doesn't mention the upcoming Memristor  computing paradigm. Memristor computing will be based upon an entirely different architecture, which will likely revolutionize computing. There is much more to it than I can describe here, but here's at least one company commercializing this tech as we speak .
The first question of the interviewer is where the (misleading) HN title comes from:
Luke Muehlhauser: Could you please explain for our readers what “dark silicon” is, and why it poses a threat to the historical exponential trend in computing performance growth?
The most interesting part was that to combat this lack of general-purpose performance increase, specialised computing units, fpgas, and Neural Processing Units (NPUs) have been proposed. Linked paper on npus: http://www.cc.gatech.edu/~hadi/doc/paper/2013-toppicks-npu.p... . Linked paper on "an architectural framework from the ISA (Instruction Set Architecture) to the microarchitecture, which conventional processors can use to trade accuracy for efficiency." http://www.cc.gatech.edu/~hadi/doc/paper/2012-asplos-truffle...
- energy efficiency at transistor level will cause hassles, it doesn't scales down well
- gains in performance from multiprocessors are smaller than from improving performance of cores
It's a long article, so I will sum up what I think is the most important point: we are going to have to think of ways of creating value that no longer leverage exponential deterministic data processing capability. The next venues of profit and growth will be in figuring out how to deal with non-deterministic or open-ended questions where an exact answer is not necessary, or better yet, may not be optimal.
This is a particularly poor example: Microsoft controls the supply of Windows licenses and installation media. We have all ran out of Windows 2000 several years back and I don't remember if the supply of Windows XP still exists.
Any free as in speech software would be a better example.
What happens when computers no longer get any faster? People have been so used to the 40 year exponential trend that they can't even comprehend that it will end. We're talking about the next 4-5 years.
getting to the point where silicon development stops and full-on quantum computing begins has been a long time coming. these d-wave machines are still a long ways from a proper quantum computer.
Diamond could operate at higher temperatures and frequencies (giving us better single-threaded performance again). Graphene would be more suited to the current manufacturing process. I don't know much about nanotube processors other than that they are in research too.
Multicore in essence is breaking up the hot spot and spreading it out. Our choices are limited simpler CPU's and more of them to flex this technique. GPU design is more geared toward this, and unified address space on newer AMD chips as well as potentially Nvidia's Tegra design evolution (project Denver?) both point to smaller, simpler CPU's or something like sub-CPU's (instructions are already translated to micro-opcodes and scheduled differently than they appear in the program text) that only do parts of the work instead of operating on whole threads. CPU's are binary code runtimes implemented in hardware, so this kind of abstraction is like changing the runtime without changing the bytecode fed to it. We might end up at a situation where CPU's work on 100 threads with a blurry definition of what a core even is anymore. It will happen slowly, as microprocessors retain major similarities over the years. Convergent evolution and too much engineering and experience to start anything from scratch. Can you find the FPU's in each generation? http://chip-architect.com/news/AMD_family_pic.jpg I always marveled at how much silicon is necessary for SIMD FP.
I somewhat doubt this is relevant at the chip scale, but we're at 10cm per clock cycle at speed-of-light to put the frequency into perspective.
Maybe he doesn't think it would ever be fast enough, or maybe he thinks it can only apply at really small scales (i.e. nanotech) and there's no smooth incremental path there from where we are now?
so in 10 years haskell maybe as fast a C/C++ program without parallelism, not exactly my definition of future proof.
This is where Haskell comes in. Haskell allows you to program in a way that makes parallelization relatively easy.
It seems like you want to advertise a tool, instead of actually considering the issues.
His argument is that scaling silicon to multiple cores is facing physical barriers. You're no longer getting more cycles, improved transistor density, and decreased power consumption simultaneously. Exponential scaling of cores will not result in exponential scaling of computational throughput because we're facing energy-related tradeoffs.
On a chip there is now lots of redundancy to ensure chip functions correctly. This gets worse as variation at smaller geometries increases. So over margining because no one wants another floating point bug.
Design effort and cost is a huge issue too. Margins at chip companies are shrinking - design automation license costs, large runtimes and much more difficult verification process from design to fab to test.
A new programming paradigm where there is a certain probability of error can be tolerated should go a long way in making the underlying hardware much more efficient.