
End of Moore's Law: It's not just about physics - davidiach
http://www.cnet.com/news/end-of-moores-law-its-not-just-about-physics/
======
reitzensteinm
For a nice dose of doom and gloom, I quite like the dark silicon paper[1],
which explores the limited use we'll get out of Moore's Law, _even if_ it
manages to continue (and as far as I understand it, transistors/$ has already
flat lined so it's at least temporarily over).

Since we're no longer getting good power scaling out of shrinking, if Moore's
Law keeps up, we're just essentially getting price discounts.

A nice thought exercise for me is what would computing look like if you could
fab wafers for free, today. Sort of the ad absurdum take on Moore's Law
continuing.

We'd have more memory, and a ridiculous amount of flash storage, and high end
graphics cards would become cheaper (but not faster). Desktop CPUs might look
more like server CPUs, but single core performance wouldn't change one bit.

We'd probably see heterogeneous computing a whole lot more. Xeon Phi like
processors on package next to Haswell cores at the very least.

Probably computers would start to have FPGAs in them, as well as large amounts
of niche circuitry to compute hashes, encode videos.

We might see computation embedded inside memory (I think Hynix did this
recently but I couldn't find a link). Maybe memory that accelerates garbage
collection in order to accelerate modern workloads.

So there are some interesting places to go, especially from a Hacker News
perspective, but even if transistors became _free_ , we still wouldn't see the
rate of progress we did in the 80s or 90s (in terms of speed up to the tasks
we're doing).

Of course, this is all near to medium term stuff. I personally believe we'll
see that rate of progress again when we move off of silicon, to a different
computing paradigm, or (most likely) both - I can't believe we'll inch our way
to science fiction computing, or that science fiction computing won't be
possible at all.

[1] Dark Silicon and the End of Multicore Scaling
(ftp://ftp.cs.utexas.edu/pub/dburger/papers/ISCA11.pdf)

~~~
sliverstorm
Oh, if transistors became free we could realize a huge boost in performance.
We could basically take the cream of the crop and use only 9-sigma parts. (The
range of part quality is pretty wide, cost/rarity is the challenge)

We could also abandon yield-oriented design constraints, and a few other
things.

Now this probably isn't something that would keep doubling every year, but
there _would_ be an initial large bump.

~~~
reitzensteinm
Not so fast! I must object to the loophole you claim to have found.

I said the wafers would be free; testing ~100 trillion of them to find the
best costs extra :)

I never even thought of that angle - do you know how much better such a part
would realistically be? I thought binning was mostly a matter of defect
reduction (where some defects are slight). But at that extreme level, it could
be a different game altogether.

~~~
marcosdumay
People keep demonstrating 1THz transistors on the lab. So, I'd say that such
part would get up to a around 100GHz clock (a few switchings per pipeline
stage), with our usual architectures.

And it would use an insane amount of power.

------
lsc
I have, perhaps, a different perspective. I have been spending a lot of my
income on computer hardware and power for the last decade or so; and being
involved in spending other people's money on same for another half-decade
back.

My take? When Intel thinks they are ahead, compute doesn't get cheaper.

In the DDR2 days, If you were on Intel, you had the choice between the
stunningly inefficient and expensive rambus ram, or a stunningly shitty memory
buss with (not very many) low-power ddr2 modules.

At the time, the AMD HyperTransport system was absolutely beautiful. Even on
cheap boards, you could get more than 2x the low-power ddr2 ram modules per
CPU that intel could. (at the time, lower-density modules were dramatically
cheaper, per gigabyte, than higher-density modules) It worked way better when
you had multiple CPUs, too.

Then ddr3 came, and Intel came up with their QPI systems, which were awesome.
AMD came back with a competently built ddr3 platform, too; the G34 systems
were a huge upgrade from the mcp55 chipset socket F platform.

But the benchmarks came out in Intel's favor, even when AMD had twice the
cores. I mean, you could argue that the AMD systems had advantages in some
limited situations, but they had lost the dramatic advantage.

As far as I can tell, intel has been largely resting on their laurels, price-
wise. The E5-2620 is better than, but really not radically better than the
E5520. Now, some of the higher-end E5s are pretty nice, but they are priced
accordingly.

Until Intel gets some real competition again, we have to pay for our
performance gains.

So yeah, really, until AMD gets their legs back under them, and I hope the
A1100[1] will do it, I don't expect dramatic performance per dollar gains from
Intel.

[1][http://www.amd.com/en-us/press-releases/Pages/amd-to-
acceler...](http://www.amd.com/en-us/press-releases/Pages/amd-to-
accelerate-2014jan28.aspx)

~~~
lplplplplp
I agree. HyperTransport was stunning when it came out. Revolutionary, even.
Ditto for AMD64, still the standard as we speak.

I'm not sure, however, if this is due to Intel resting on their laurels vs an
entire Intel generation being shown the door because of epic (excuse the pun)
fuckups. With the (largely wasted) effort expended on Itanium / Itanium 2 /
EPIC / IXP / Netburst / etc, no wonder other vendors excelled. The MHz wars
took a horrible toll on Intel for mainstream x86 with things like Prescott,
with its 31 stage (!!) pipeline.

Stalls on Prescott were horrific for performance. On IXP, microcode screwups
(often due to explicit caching) were horrific for customers. On Itanium,
everything was horrific. I doubt we will ever know exactly how much these
escapades cost humanity. On the other hand, maybe we're all richer for lessons
learned.

It seems Intel is not interested in screwing up so badly anymore, so I think
it's the competitors' turn to sweat. Intel still has a long way to go in
recapturing territory it could have already had; ARM and MIPS have come a long
way in the phone/server and NPU/packet processing space respectively, and they
don't look as easily dislodged as AMD...

~~~
hga
It's even worse than you enumerate: echoing lsc and his mention of "stunningly
inefficient and expensive rambus ram", the highest level architects at Intel
were petrified by DRAM size, I think it was, concerns, and ordered some
stunningly stupid things that at least in some cases the engineers under them
knew wouldn't work. Intel had not one, but two *1 million part recalls", one
of which was for motherboards just before OEMs were going to start shipping.

And AMD, which only occasionally manages this, did everything right for a
short period of time with their K8 microarchitecture (P6 style, 64 bits,
HyperTransport plus on-chip memory controller) while Intel was screwing up so
much.

I wonder how history would have gone if they hadn't then taken 2.5 years to
start delivering the successor K10 microarchitecture, and another half a year
to deliver one that didn't have a screwed up TLB. Intel is not the sort of
adversary you can just give three years to get its act together, especially
with their historical manufacturing prowess keeping them at least a process
node ahead of you (and pretty much everyone else?).

------
tedsanders
It's always been economics. EUV already works. E-beam lithography already
works. Carbon nanotube transistors already work. III-V transistors already
work. It's just that none of these technologies work as cheaply as double
patterned silicon.

~~~
hershel
The fact the we can't manufacture transistors cheaper doesn't mean we won't
see big jumps in compute value.We could seem improvements from fpga's(500x
max),approximate computing(100x-1000x), packaging(3x-10x) or the radical
option of analog computing(6-9 orders of magnitude for brain simulation etc).

Also, there's still huge amount of value to unlock from software, interfaces
and the rest.Just look at the iphone where most of the value comes from a
great touch interface and an app store. Moore's law is just a bonus.

~~~
sliverstorm
500x performance improvement from using fpgas? Where do you get this number?

~~~
hershel
Asics can max out at about 1000x , and zvi orbach has a recent column talking
about how to make an fpga with 50% efficiency of asic.

[http://www.eetimes.com/author.asp?doc_id=1322021](http://www.eetimes.com/author.asp?doc_id=1322021)

~~~
sliverstorm
I must have misunderstood, I thought you were saying FPGA's will surpass CPU's
500-fold.

------
PeterisP
We still have a magnitude order or two of computing power that we can squeeze
out iff the semiconductor density/price stops.

We don't implement many optimization possibilities in each hw/sw layer because
the "layer below" keeps changing all the time and we need to keep
compatibility. Once we could say "this is it, this layer is as good as it will
ever get", then (and not a day before) you can start to re-architecture
everything above it to maximize performance by throwing away flexibility that
won't be needed anymore.

E.g., instead of transistors being spent on translating x86 to the underlying
microcode and cycles being spent on translating JVM/CLR bytecode (or
javascript) to x86, we'd be able to define a single standard and adapt both
processors and compilers to that. You can't break compatibility at every
technological change - but if you have a reason to believe that it will
finally be stable (which hasn't ever happened yet) then it does make sense to
make a single final switch that disregards all compatibility and legacy issues
- even if the benefits are small, they accumulate for each such layer and you
only have to do it once.

------
tambourine_man
I've been hearing about the end of Moore's Law ever since I was a kid. I
remember smart people citing wave lengths and economics convincing me that
there as no way we'd go beyond 300nm. Oh yes, and that would mean the end of
x86 as well (since intel wouldn't have the process advantage to compensate for
the less efficient CISC).

Well, maybe they are right this time, who knows, but I'm way more skeptical.

~~~
bsder
Moore's Law broke a while ago. Note the _past tense_.

nVidia couldn't get their economics at 28nm. RAM cells are now _more_
expensive in 20nm than 28nm. There are lots of other examples.

Remember: the strong form of Moore's law was that transistors get _cheaper_
ever 12-18 months. The original formulation of Moore's law was always about
economics, not technology.

The problem is: nobody knows what to do.

You now need clever circuit design and humans who can squeeze another 15-20%
out of existing technologies. Unfortunately, all the humans who knew how to do
custom VLSI design are dead, retired, or doing something else since custom
VLSI was fool's errand for so long (see DEC, IBM, Silicon Graphics, etc.).

~~~
hga
I'm not sure you can extrapolate that from one company's use of one foundry's
process nodes. Intel at least seems to be doing well with their 22 nm process
node.

~~~
marcosdumay
Yep, they seem to be doing well, but they are almost an entire doubling time
later if you compare the two previous doublings with to Moore's law.

~~~
hga
This might in part be explained by their moving to nonplanar FinFET
transistors. And in practice their geometry doesn't quite match the idealized
drawings:
[http://www.eetimes.com/document.asp?doc_id=1261761](http://www.eetimes.com/document.asp?doc_id=1261761)

------
skywhopper
One thing I can predict with complete confidence is that we have no clue what
the path forward for improving computer processing power will be from 2022.
It's fun to speculate but that's a long time for new technologies, approaches,
and architectures to take hold. Intel isn't the only company with an interest
in improving the state of the art here.

------
DanielBMarkham
I think maybe we might have been mis-reading Moore's Law for years, or rather,
Moore himself might have mis-stated it.

The radical changing reality that it describes is the number of transistors a
person could economically employ at any one time to perform work on their
behalf.

So yes, physics and economics might provide us with a limit (or increasing
slope of difficulty) for the construction of single chips, the _practical
effects_ of the Law continue unabated, at least as far as I can see. The
average person continues to be able to employ more and more electronics to
perform work for them. This is increasing geometrically.

I'm not trying to dismiss either this DARPA guy or Moore, just point out that
the specific details of Moore's Law may not be as important as we make them
out to be.

In my mind, the big obstacle we have now to continued growth is small-system,
imperative thinking. Systems of the future will be massively parallel. I have
no idea how long it will take the IT industry to truly transition, but that's
the next big hurdle, not counting atoms inside a switch.

~~~
Detrus
Yea I remember reading that DWAVE could tackle classical computing problems at
20 petaflops. All of Google today is 20 petaflops, across many cartoon colored
datacenters.

If you can produce small numbers of expensive computers that are more
efficient than millions of cheap computers cobbled together, you are still
advancing how much computational power each person has access to.

But a significant problem with this approach is most real time video games
aren't practical when the computer is far away.

~~~
nullc
Latency isn't the only issue— do we want to be building a world where people
are constantly trusting third parties to process their data privately and
correctly?

Certainly there are also cryptographic workarounds to those problems but their
overheads are so high as to make the outsourced computation somewhat
pointless.

------
hershel
For the talk by the DARPA guy:

[https://www.youtube.com/watch?v=JpgV6rCn5-g#t=15](https://www.youtube.com/watch?v=JpgV6rCn5-g#t=15)

------
anigbrowl
Worst formatting I've ever seen. Anyway:

 _Colwell said that for the Defense Department, he uses the year 2020 and 7
nanometers as the "last process technology node." But he adds, "In reality, I
expect the industry to do whatever heavy lifting is needed to push to 5nm,
even if 5nm doesn't offer much advantage over 7, and that moves the earliest
end to 2022. I think the end comes right around those nodes."_

Let's assume that in 10 years time everyone agrees we have hit the wall on how
small we can go. What then? Is there any reason to believe that popular
architectures of today are necessarily optimal? I'm curious to hear people's
ideas about what we do instead of shrinking dies.

My personal guess is that we move towards massively parallel systems with
large numbers of low-power cores, but bare-metal programming completely and
work on developing smarter and smarter compilers to take advantage of
parallelism. My personal hope is that we find some sort of cold optical
switching technology that lets us build ridiculously fast computers that look
like glowing crystal cubes. Of course, I have no idea how that would work, or
I'd be out pitching it XD

~~~
2muchcoffeeman
Have you heard of photonics crystals?

[http://optoelectronics.eecs.berkeley.edu/eliy_SCIAM.pdf](http://optoelectronics.eecs.berkeley.edu/eliy_SCIAM.pdf)

But I think that since the features of the materials are on the same order of
magnitude as the wave lengths of light you are trying to switch, this actually
makes things bigger. But still really cool.

~~~
anigbrowl
I had heard of it and I know that has been some progress with optoelectronics
recently, but hadn't looked into it in this detail - thanks. Most of his
recent work seems focused (groan) on things like solar cell efficiency and
suchlike rather than my fanciful goal of a completely optical processor, but I
need to read more.

Maybe I should ask him to lunch, since I live within walking distance of UC
Berkeley and don't really take advantage of it :-/

------
kstenerud
Yeah, I'm sure someone had similar doom and gloom to say about vacuum tubes.

The thing is, you just don't know where the next technological revolution will
come from. Yes, we've about hit the limit for silicon, and we may very well
stagnate for another decade because of it, but there's always going to be
someone scrappy enough to try what the big slow incumbents won't.

------
beloch
There are gains yet to be realized besides reducing the size of the process.
For example, reversible (a.k.a. isentropic or adiabatic) computing offers a
way to reduce heat generation, which might combine with 3D construction in
interesting ways. New ways of designing chips might allow progress to
continue, but they're hard and risky. They're not terribly attractive as long
as shrinking the process offers predictable advances and remains economically
feasible.

Still, it's worth taking a moment to appreciate how crazy just getting under
10 nm is. The wavelength of light that is visible to the human eye starts at
around 380 nm. Looking at a 10 nm chip with violet light would be like trying
to navigate your house by sonar using a sub-woofer as your emitter!

------
williadc
One thing that seems to be glossed over in his argument is the impending move
to 450mm wafer technology. That should allow Intel and others to continue
shrinking at reasonable cost.

------
lucb1e
> it's time to start planning for the end of Moore's Law

Do we? Because all current software runs on current hardware. Even if in the
next ten years we only get another 1% increase in speed, then the current
software will run at the current speed (something which we are all perfectly
fine with right now). I think it's indeed a doom and gloom article.

~~~
RogerL
I want it. Python is 100x slower than C. I want a language that doesn't need
to worry about performance. I want computers optimized for humans, not them. I
want computers that can model the weather, the ocean. I want search algorithms
that optimize car engines, electric plants. a million x isn't nearly good
enough.

------
hyp0
smaller devices will provide the economic drive, just as they did when 14"
HDDs had enough performance.

------
pmorici
Either this article or the guy's speech perhaps both are pretty lame. No
supporting facts are presented as to why he thinks the 7nm generation will be
the last frontier. He just basically states the obvious that if there is no
profit motive or the technology isn't there then it won't happen. No kidding.

The whole article reads like an appeal to authority.

Are there any factual reasons to believe this time is actually different than
the past 35 years?

~~~
PhantomGremlin
> The whole article reads like an appeal to authority.

You might be right. But Colwell isn't just a country rube who hitched a ride
into town on the turnip truck. He's been there, done that, _literally_ made
billions of dollars in profit for Intel. I reiterate:

    
    
       Bob was the chief architect for a number of design
       teams who, *LITERALLY*, no exaggeration, generated
       billions of dollars in profit for Intel.
    

You might have heard of a few of the products he architected. [1]

"Authority" is frequently wrong. But this guy certainly deserves a little more
respect than your flippant dismissal.

[1]
[http://en.wikipedia.org/wiki/Bob_Colwell](http://en.wikipedia.org/wiki/Bob_Colwell)

~~~
pmorici
Interesting, I wonder why it doesn't say that in the article which would lend
a lot more credability to it. It just talks about his position with DARPA

------
transfire
For decades I have heard that 11nm would be the end of Moore's law as we know
it. This is proving to be true. CPUs have been frozen at 1 to 4GHz for nearly
a decade, with the latest advances going to power savings. Currently at 22nm,
the next step is 14, and then 11. Perhaps they can eek out one more jump to
7nm or 5nm, but I expect that will barely be worth the effort and thus will
drag out for at least decade itself.

But Moore fans need not worry. There is a clear next step, and I am wondering
why no one is talking about it: Optical Interconnects. Connecting chips and
circuits within chips with optical channels should allow plenty of room for
speeding up processors and reducing power requirements well into the mid-21st
century.

~~~
dippyskoodlez
Clock Speed is not relevant. In that "decade" our single thread performance
has increased a fair amount, but we now have an additional 4-8 CPUs in the
same amount of space.

An Ivybridge will very handily beat a Northwood based system even in single
threaded performance.

~~~
autokad
Dont get me wrong, I love having all of my 12 cores, but I dont really count
that as improved speed. More (no pun intended) importantly the rule was every
18 months if I'm not mistaken, not 10 years. Lets double it and add a little
bit, 3 years ago you pretty much could have bought the same machine.

in the last 3 years, processors have been about 3.4ghz, 3 years before that it
was still around 3.1ish. I personally haven't noticed any improved performance
on single threads. they pretty much do things at the same completion time.

I have a 12 core 4.5 ghz machine (3.4 overclocked), and I ran a compute
intensive process on a 2010 machine that was 8 cores 3.4ghz and they finished
about 5 minutes later at about the exact same time.

i'm not saying all things are equal and that things havent improved, just that
i would have expected a lot more from something that was supposed to double
(and get cheaper) every 18 months after 4 years

