
The chips are down for Moore’s law - Stefan333
http://www.nature.com/news/the-chips-are-down-for-moore-s-law-1.19338
======
Symmetry
I really ought to do a blog post on the math behind this but it's not really a
simple matter of power density because drive currents have traditionally
shrunk along with the area and hence capacitance of transistors. For a long
time people didn't reduce voltage to achieve classical Dennard Scaling[1]
because back in the day nobody really cared about power but you could always
go back to scaling that if you wanted. I suppose that the reason it looms so
large in our heads is that Intel was having problems with power on the Pentium
IV right as things were starting to change around 90nm.

Some things that make increasing performance with feature size more difficult
recently are that transistors spend lots of time velocity saturated, there's
increased leakage current, and the capacitance of wires has come exceed
transistor capacitance.

~~~
clickok
Please do. I understand why chips can't continue to improve following Moore's
law from some fairly high level perspectives, but I haven't seen anything that
provides more detail than the typical pop-science articles but without
requiring me to crack open a text on condensed matter physics.

------
gnarbarian
I am excited to see if this evolutionary bottleneck will take us beyond the
von neumann architecture.

The article implies we will get even more specialized hardware that will speed
up common categories of algorithms like gpus asics and FPGAs have been used.

I wonder what kinds of performance gains could be had from a merge sort asic
for example.

I think the real interesting thing will be if new architectures beyond von
neumann will allow us to think outside the box and break out into new
algorithmic territory that is not obvious from our current perspective.

It's very hard for us to throw away our toolchain towers of Babel but I think
the departure has already begun when you look at bitcoin mining and gpgpu
programming.

~~~
nostrademons
I think it's more likely that software development practices will change to
recover a lot of the speed that's been lost. Not dramatically - certainly not
a new architecture - but in lots of little ways.

There's a _lot_ of low-hanging performance fruit in programming language
design, compiler implementation, technology choice, and application design.
For the most part, nobody takes it because it doesn't make economic sense.
With Moore's Law, just wait a couple years and your shitty implementation is
good enough.

It's possible for a skilled engineer to write code that is _much_ more
efficient. You have to compromise in other ways, like developer time or
feature set. Up till now, those compromises have not been worth it, because
somebody will beat you to market and make you irrelevant. In a world where
"fast as an initial prototype" is not "fast enough", however, you could see
firms actually start to compete on performance.

I predict that even without hardware changes, we'll see a bit of a maturation
of the software industry, where people look at what folks actually do with
their computers and then work to make that actually perform well. Possibly not
for a few years, though.

~~~
CuriouslyC
In my opinion, there is still a ton of room to scale performance by improving
paralellism.

Compilers will of course get better and better at automatically parallizing
code, but a big thing that needs to occur in my opinion is the introduction of
the concept of explicit partial ordering into mainstream programming
languages. Traditional programming languages assume that the order statements
occur in code is the order in which operations should occur (operator
precedence for the generation of the AST excepted). Imagine instead a
programming language where statements separated by a newline implied temporal
equivalence, while statements separated by a semicolon implied a temporal
ordering. This would enable the compiler to automatically parallelize code
much more efficiently.

~~~
nostrademons
Most modern compilers do data-flow analysis to identify which statements are
independent of each other, and they'll rearrange instructions so they can
saturate a processor's pipeline. That's what superscalar processors are - they
have multiple instruction decode units, so they're executing multiple
instructions at once even within a single thread.

A bigger problem is that many _algorithms_ aren't really parallelizable. Throw
a single mutable shared variable into the mix - even something as innocuous as
a counter, or memory allocation pointer, or a logging buffer - and suddenly
you need locking and your threads may end up starving each other. Once of the
biggest breakthroughs of MapReduce is that it forces you to write your
algorithm as functions that operate on independent data. But as someone who's
written a decent amount of MapReduces - the thought process about how you
design a MapReduce-based algorithm is very different from (and significantly
more difficult than) single-processor programming. You can't just go to an
algorithms textbook and copy some pseudocode, you need to actually think
through what the algorithm and what it's doing, and then rewrite many state
updates as message-sends, merges, and CRDTs. (This, BTW, is why Google
interviews focus so heavily on algorithms - in many other jobs "How would you
find the shortest path between a trillion graph vertices?" is totally
academic, but in big data it comes up all the time.)

------
Aardwolf
> Top-of-the-line microprocessors currently have circuit features that are
> around 14 nanometres across, smaller than most viruses.

Well, that gives room for improvement, a virus has quite a bit more logic
inside than a single gate feature :)

~~~
jostmey
True, but transistors are already smaller in size than a human synapse. But
synapses don't produce so much heat, so you can stack as many as you want in
3D space without overheating, unlike silicon transistors.

~~~
jacquesm
There may be an interesting lesson buried here. What if there is efficiency to
be gained from drastically reducing the clock rate of processors while at the
same time increasing the number of interconnects? A bit like an ultra low
power version of the connection machines.

~~~
jostmey
There is more than one way to skin a cat. Somehow, the heat problem needs to
be solved.

Perhaps what would help is to store energy locally, like our brain. Each cell
has its own store of ATP, so a constant feed of ionic current is not needed.

~~~
88e282102ae2e5b
That ATP has to come from somewhere though. In cells the ultimate source is
usually glucose. So you'd still have to supply some source of energy, in one
way or another.

------
alexnewman
"Top-of-the-line microprocessors currently have circuit features that are
around 14 nanometres across, smaller than most viruses"

Yea this is not true. The 14nm or 11nm stuff from chip manufactures is more of
a marketing thing. It's much more complicated than that.

~~~
bbtn
People think it is transistor size that is 14 nm, which is not. It is metal
pitch on high density memory chips that have repetitive patterns all over the
chip.

That sentence is not wrong. It looks well stated.

~~~
dbcooper
14nm is _supposed_ to refer to the gate length (which may be larger than the
node name).

The gate pitch of Intel's 14nm CPU process node is more like 70nm. [1]

[1] [http://www.anandtech.com/show/8367/intels-14nm-technology-
in...](http://www.anandtech.com/show/8367/intels-14nm-technology-in-detail)

~~~
creshal
I think the last process where process name = gate length was _90_ nm. Ever
since the naming scheme was more a historic artefact than anything else.

------
zekevermillion
Where do chipmakers go from here? When the best consumer chips on the market
add very little to the cost of a machine, then you have to come up with
another way to win market share that isn't just based on speed claims. I'm
sure there will be more spend on marketing fluff, attempts to brand slightly
different products for different segments and price discriminate between them
-- enterprise, gaming, mobile, IoT. But I also wonder if we will see more
pernicious behavior, in chipmakers keying chips to specific applications
through DRM, so they can sell locked appliances with software built in.

------
crimsonalucard
I think the upper bound of Moore's law foreshadows another impending boundary:

There is a fundamental boundary on all technological progress.

------
hga
_None of this was inevitable: chipmakers deliberately chose to stay on the
Moore 's law track. At every stage, software developers came up with
applications that strained the capabilities of existing chips; consumers asked
more of their devices; and manufacturers rushed to meet that demand with next-
generation chips._

Moore's law is stated in terms of a doubling of _lowest cost_ transistors you
can fit on a wafer, and results in massive yield gains if you shrink an
existing design when moving from node to node (the "tick" in Intel's tick-tock
cycle). So whatever customers want, you'll still move to the next node if the
economics allow you to deliver the same at a lower cost. This is one of the
things that drives game console economics, mid-life refreshes which allow
delivering the same capabilities with cheaper and fewer chips (fewer decreases
other manufacturing costs).

The heat problem and related end of "free" speed increases is due to the
failure of Dennard scaling
([https://en.wikipedia.org/wiki/Dennard_scaling](https://en.wikipedia.org/wiki/Dennard_scaling))
after a good 30 year run.

 _Every time the scale is halved, manufacturers need a whole new generation of
ever more precise photolithography machines. Building a new fab line today
requires an investment typically measured in many billions of dollars —
something only a handful of companies can afford._

Ignoring the far out scaling limits, this is what I would have focused on. As
far as I can tell, the biggest issue right now is the skyrocketing costs of
lithography due to the multiple masks required for multiple patterning
([https://en.wikipedia.org/wiki/Multiple_patterning](https://en.wikipedia.org/wiki/Multiple_patterning)).

If, and it's a big if (see Intel continuing to push out they date they plan to
adopt it), when EUV lithography become practical that should change this very
recent economic issue, substituting manufacturing costs in buying and running
$$$ EUV machines for many fewer masks required for each design, lowering non-
recurring engineering (NRE) expenses.

That could return a lot of the game to what it was like for a long time,
albeit with only a very few foundries being able to afford these machines. The
first generation the leading company in the Netherlands is trying is pretty
insane in how they generate the light, by exciting tin vapor (!), imagine the
inherent contamination potential for both the focusing optics and the
wafer....

One other detail: to the extent a lot of companies start focusing on the
economic ~22nm or so node that's just before costs skyrocket, we may be seeing
some serious economies of scale as people focus on aspects other than the
design shrink. Sure, you'd like smaller nodes, especially for lowest power
applications, note how Intel is leading with mobile SKUs for it's newest
smaller node chips, but that node is perfectly fine for a whole lot of things.
Another economic factor to focus on, not everyone is an Intel, nVidia, Apple
etc. that needs cutting edge nodes.

~~~
Symmetry
When Gorden Moore wrote his first paper on things doubling every fixed time
period it was on the lowest cost transistors back in 1965 but the term _Moore
's Law_ didn't come into use until the mid 70s by which time he had written a
number of other papers pointing out a number of other doublings including
feature size. In the old days these all doubled in lockstep so the original
coinage of the term didn't really distinguish economics from minimal feature
size.

So there's really no legitimate way to insist that Moore's law has to refer to
the lowest cost number of transistors. It actually wasn't that by coinage and
it certainly isn't that by common usage. I've even got a copy of a memo by
Moore in my 6.175 lecture notes endorsing a broader conception of Moore's law.

~~~
hga
Well, my conclusion after many years, echoed by one of the themes of _Nature_
article including the lock step nature of node moves, is that Moore's Law is
more one of economics than anything else. So I focus on that original
definition of it, it seems to be the essence of it and its implications.

Now, if you're in the weeds of designing a chip or node, yeah, things like
feature size are paramount, but why are you going to the trouble of moving to
a new node in the first place, especially with the end of Dennard scaling a
decade+ ago? Power consumption could be an alternative to the raw economics,
or maybe taking up less space in the smaller mobile form factors. Intel's
leading with mobile chips for _some_ reason.

------
tempodox
I had the idea to go 3D in chip design thirty years ago, but wasn't in a
position to trigger any research or experiments.

I'm curious to see how this development will turn out.

------
legulere
> “So far as I know, there is no example of anything like this in any other
> industry, where every manufacturer and supplier gets together and figures
> out what they are going to do.” In effect, it converted Moore's law from an
> empirical observation into a self-fulfilling prophecy: new chips followed
> the law because the industry made sure that they did.

Probably because in every other industry this would have been seen as anti-
competitive behaviour.

~~~
coldtea
> _Probably because in every other industry this would have been seen as anti-
> competitive behavior._

Not really, since nobody prevented any company from coming up with even faster
speeds / bigger transistor counts.

The whole coordination was to push the limits.

Not sure which industry would consider ... exponential improvements as "anti-
competitive". More likely they would kill to be able to have them too.

~~~
marcosdumay
While the GP is seeing collusion where there are only aligned incentives, you
are not getting some of the point of Moore's Law.

The thing is, most industries begin with a short period of super-exponential
improvements, most of what happen in the lab before the products even get
sold. In just a few years, the industry is mature, and further improvements
happen in a much slower rate.

Semiconductor industry is different. Because of economical factors, it
couldn't have that super-exponential beginning, and had to slow down for
decades of merely exponential growth. (It also may had the biggest rate of
mature/immature efficiency we've ever seen - but that's another thing.)

~~~
coldtea
> _The thing is, most industries begin with a short period of super-
> exponential improvements, most of what happen in the lab before the products
> even get sold._

Can you name any of those industries? Because most (if not all) industries and
products I know, aside of IT, have none such "super-exponential" improvements
at their beginnings, not even exponential ones year over year.

We don't have 100x better cars than in 1950.

Film cameras didn't get x better in their first years -- it was a slow and
painful evolution that lasted decades to get to something like the 35mm film
and standardized functionality.

We don't have 10x better air-conditioning, watches, guitars, etc.

The only places I can think of where that holds are things like memory,
storage, CPUs, GPUs etc -- all IT related. And maybe batteries, though I doubt
it.

~~~
marcosdumay
Well, you came out with a great example. Cars are one product with a very
visible evolution, and expensive enough for slowing this evolution down so it
could be documented. How much are the 1915 cars better than the 1900 ones? I'd
venture a guess at the hundreds of times.

Film cameras are at the cheap/fast evolving end, so I wouldn't expect to see
most of the in lab models documented. Yet, the first released ones were very
reliable - I doubt the first prototyped ones were so.

Thermal machines and watches were two examples of products that were slowed
down by economical factors. Different economical factors than semiconductors,
so they didn't got that the smooth exponential development.

------
robotcookies
> Moore's law, the principle that has powered the information-technology
> revolution since the 1960s, is nearing its end.

I don't understand why people keep saying Moore's law will end soon... as if
it's a forecast. I just checked an online site for cpus and the fastest one
available was at 3.7 GHZ. It was maybe 3.2 GHZ two years ago. In 2001, I
bought a computer that ran at 1 GHZ. That's 15 years ago! Moore's law is
usually defined as doubling every 18 months. Even if we stretched it to every
24 months, we should be at well over 100 GHZ. Even if we account for the 4
cores, it's still not happening.

What am I missing here? It seems like Moore's law has been dead for years.
Please don't tell me about experimental cpus that are faster or theoretical
processors. Moore's law is defined as speed increases while the price stays
the same. What matters is what cpus are available to consumers.

~~~
ajross
Moore's Law properly expressed is about transistor density and not switching
speed. Clock rates have been static over the last decade, but register widths
have climbed by a factor of 8 and we're now packing a dozen or more cores on a
die that used to hold one.

