
Moore’s Law Is About to Get Weird (2015) - sjayasinghe
http://nautil.us/issue/21/information/moores-law-is-about-to-get-weird
======
ansible
Well, none of those listed technologies is practical.

Ternary logic is really cool, but it only gets you about a 10% efficiency
improvement. I think it would be awesome to go in this direction, but
retooling our entire tech stack for that isn't going to happen. I mean, look
at x86.

The only thing that will really make progress is molecular nanotechnology. I
had always believed that reaching the end of silicon process technology would
necessitate the research into MNT.

What's funny is that we have arguably already reached the end on the silicon
process technology, if Intel's desktop product offerings are used as a guide.
There are gamers who are building new rigs using chips that are 2 or 3
generations old, because there isn't really anything better available. What's
really hilarious are the guys building systems with used Xeon CPUs from
retired servers!

So I don't know what's going on any more. You would think that the profits and
stock prices of all the silicon companies would collapse if they weren't able
to continue producing improvements on their existing products. Intel is
spending massively on R&D, but the gains are nowhere close to what we saw 15
years ago.

Note that keep referencing Intel, because they've been at the leading edge.
AMD doesn't have 14nm in production, for example.

~~~
swalsh
"retooling our entire tech stack for that isn't going to happen. I mean, look
at x86."

I wonder if today it's easier than people imagine, but the lessons of Itanium
are used as a signal to not try.

Sure there are plenty of apps which make low-level use of x86 features, and
those probably aren't going to be updated. However a lot of modern software is
written in languages that go through interrupters. Java, C#, Python.. if you
can update the runtime, in theory the apps can be supported anywhere.

~~~
pjc50
Switching to ternary means retooling not just the programming stack but the IC
design stack. IC design is actually quite a conservative business, mainly
because failures are so expensive.

You'd need to spend several years writing a ternary RTL synthesis, place,
route and verification suite before starting.

~~~
Symmetry
I would expect that once silicon stops shrinking and both chips and fabs can
be allowed to pay for themselves over longer time horizons we're going to see
a lot more architectural experimentation.

~~~
mmmBacon
I hope you're right. The costs of 7nm are so high that even the "big guys" are
being very careful about what they do. It's become so expensive they just
can't afford a miss in the market. I worry the high costs will stifle
innovation and that we'll see something that's more like a tweak here and
there for a while.

------
steego
For all the interesting things happening today, this is an incredibly dull
article. I'm convinced I read this same article in the 90's.

If we're going to see any developments in alternative architectures and
hardware, it's going to happen in the cloud computing space. From my
perspective, this is the only sector that is in the unique position to sell
specialized computing services to customers on a mass scale and make great
margins in the process.

There are many interesting and useful things you can do outside of Von Neumann
architectures and Turing Complete systems that were never practical on PCs.
Simple services like Reddis could probably be implemented with alternative
memory architectures like content-addressable memory. Many useful algorithms
in the data analytic and machine learning space can be compiled down the
specialized hardware using using a combination of transistors and memristors.

For companies like Amazon, Microsoft + Google, there's actually a financial
incentive to harmonize hardware and software much in the same way Seymour Cray
harmonized the software and hardware of Cray's supercomputers. These cloud
companies don't want to be constrained to selling commodity virtual machines.
They want to sell you the next generation PaaS solution like DynamoDB, SQL
Azure or Firebase so they can lock you into their cloud platform.

If there's a future where developers are writing code in special DSLs that's
compiled to gates on an FPGA, the cloud computing guys will be exploring it
because they have a huge incentive to get their customer's into the walled
garden that yields better margins. If you can create one platform service that
developers love and runs on cheaper specialized hardware, you'll be able to
destroy your competitors and signal an alternative hardware arms race.

~~~
erdevs
I think you are right on, on every point. (Including it being a dull article,
at least for the audience of folks here.) This is great insight and it's
something I've thought for the past few years as well.

Big cloud computing players will almost certainly move this way. Actually
surprised we haven't seen more of this already. But TensorFlow and the Tensor
Processing Unit were smart moves by Google and I think this is just a preview
of many, many things to come.

I think IBM probably belongs on the list of top contenders to make a move.
Between SoftLayer, IBM's hardware and chip design and fab partnerships, their
cloud/saas businesses, and their Enterprise penetration, they're well-
positioned to lead here.

I think that you're right as well that all these folks will push to create
developer-friendly services and APIs backed by hardware. However, I think the
real wins will come from capturing market share in the Enterprise sooner than
later. If I were them and I had to prioritize between the two, I'd target
Enterprise customers first and foremost. You can sell services to help them
migrate and that helps you get out to market more quickly than creating a
fully robust self-serve platform. Moreover, if your custom hw-backed platform
can offer _either_ better price _or_ better performance, Enterprises will pay
big bucks for it. Harder to get it right with self-serve first, but you
certainly need to open up to self-serve as quickly as possible over time.

------
erdevs
From the article:

> "...Moore’s Law, which states that the number of transistors that can be
> squeezed onto a semiconductor chip of a given size doubles roughly every two
> years, has held true since the mid 1960s..."

I thought it was the case that Moore's "Law" hasn't held up since ~2012.

I believe Intel and others have stated that as of 2012 or so, with 22nm
processes, transistor density now doubles every ~2.5 years and is likely to
slow further within 5 years without more fundamental breakthroughs.

The related trend that processor clock speeds doubled every 1.5-2 years--
which is commonly mistaken as Moore's Law in lay press but is actually more
related to Dennard Scaling, Koomey's Law and historical statements by Intel--
had ceased to hold as of the late Aughts as well, I believe.

Memsistors have been a promise for some time. I wonder how we will make the
leap from current dense-transistor IC tech and processes to anything new,
given the huge infrastructure investments in fabs, quality control processes,
etc. It seems it will be difficult to scale out a new technology, whatever it
is, and get it price-competitive with existing transistor-dense IC technology.
Perhaps I misunderstand here though, and would be interested in thoughts from
those with greater expertise in the field.

~~~
VT_Drew
>I thought it was the case that Moore's "Law" hasn't held up since ~2012.

I thought it was before that, like 1999. I pretty sure I had an Intel Pentium
III 2 GHz processor in 99. The computer I am using to type this comment is 2.5
GHz. More's law is long dead, we aren't going to see 12 GHz machines in the
next 6 years.

~~~
km3k
Moore's Law has nothing to do with clock speed.

------
rm999
This is one of the most insightful things about Moore's Law I've read:
[https://www.quora.com/How-long-will-Moores-Law-continue-
to-a...](https://www.quora.com/How-long-will-Moores-Law-continue-to-
apply/answers/605270)

Moore's Law is a limit that ultimately arises from economics. The costs to
keep Moore's Law going are exponential (just like the gain), and fewer and
fewer companies are willing to pay the increasingly high price.

I was always primed to believe the exponential gains were essentially "free"
and came with a fixed amount of investment into the industry. Understanding
that this isn't true explains why Moore's Law will end unless demand is
exponentially increasing.

------
protomok
The chemical, wetware, fluid and ternary computing concepts mentioned in the
article I guess make for a good headline but seem highly unlikely.

In the short term I really think we are entering the era of accelerators.
Accelerators like Micron's Automata Processor (1) for graph analysis,
accelerators for compute intensive applications like Convolutional Neural
Nets, continued innovation in DSPs, GPUs, etc.

Accelerators that can either outperform traditional processors or use
significantly less power I think represent one of the major next steps after
Moore's law. What comes next may well be a slime mold computer but I think at
this point is pure speculation.

(1) - [http://www.micronautomata.com](http://www.micronautomata.com) \-
actually I heard about this on HN!

~~~
erdevs
Yes, almost certainly the next phase for performance-intensive computing is
more specialized chips, systems-on-a-chip, and whole specialized rigs. The
TensorFlow Processing Unit is a good recent example of this, as are ASICs in
bitcoin mining, specialized hardware in molecular biology simulation, and more
examples. This is just the beginning.

I think the interesting question is what happens after that. Not too long from
now, specialized systems will run up against the same physical limits that
general purpose processors have begun to hit the past few years as well, as
specialized fabs increase in capability and sophistication. We see this
already with GPUs.

~~~
ghaff
Specialization is essentially trading off additional hardware design work--and
additional software complexity--for performance. When Moore's Law was going
strong, it usually didn't make sense; just wait a couple years and the next
x86 generation will probably more or less catch up to whatever performance
gains your specialization was going to buy you.

I think it's pretty clear that we're going to see the introduction of all
sorts of specialized hardware that we already more or less know how to do but
it just hasn't been worth the trouble before. But you're also right that, so
long as we're still talking CMOS, this is probably a more or less one-time
boost that's maybe worth a few CMOS generations depending on the specific
case.

------
GrumpyYoungMan
One interesting avenue of research is embedded reconfigurable computing
technologies like eMIPS [0]. Unlike other reconfigurable computing approaches
that simply place a FPGA side-by-side with a CPU in the same package to act as
a co-processor [1], eMIPS integrates reconfigurable logic directly into the
processing pipeline, allowing the creation of custom processor instructions on
the fly on a per app basis to accelerate apps. Pretty cool stuff.

[0] [https://www.microsoft.com/en-
us/research/project/emips/](https://www.microsoft.com/en-
us/research/project/emips/)

[1]
[http://www.theregister.co.uk/2016/03/14/intel_xeon_fpga/](http://www.theregister.co.uk/2016/03/14/intel_xeon_fpga/)

------
api
One thought that isn't on here:

Once the process is "mature" and isn't changing every few years, the error
rate could be squeezed way down. That would make it practical and economical
to make larger chips. There are cooling and power issues too but I'm sure
those are solvable.

Make way for the 64-bit 128-core 1024gb RAM 12cm^2 SoC?

Call it More's Law? Or a BFC? (for Big F'ing Chip?)

~~~
n00b101
>Make way for the 64-bit 128-core 1024gb RAM 8cm^2 SoC

This is already happening?

Intel Xeon Knights Landing already has an enormous 6.83cm^2 die size. It has
72 "cores," each of which has a 512-bit vector processor. In some sense, you
could say that this chip has 9,216 "single precision cores." It also supports
up to 384GB DDR4 RAM, which is not far from your 1024GB.

The term "core" is becoming vague, as NVIDIA refers to thousands of "CUDA
cores" on its GPUs which are really vector processor elements.

China's Sunway SW26010 is also kind of a "SoC," 260 "cores" communicating via
a Network-on-Chip interconnect. I think one could say that the Cell Broadband
Engine processor had a similar design.

> make it practical and economical to make larger chips

I do wonder if die sizes can increase.

I'm not a hardware engineer, but my understanding is that latency is the main
barrier to larger chips. The time it takes for a signal to propagate across a
chip limits the clock frequency. One possibility is to replace the global
synchronous clock design with an asynchronous chip design. However, from what
I understand, this would make things worse because some form of
synchronization is always needed - you could send back an "acknolwedge"
message, but that would double the signal traffic.

On the other hand, supercomputers are regularly built with multi-socket
systems and interconnects like QPI and PCIe between processors and nodes. It
would seem logical to put multiple processors their interconnects on a single
"SoC" and I don't know why that doesn't happen. Maybe it is because of error
rates, as you say.

~~~
pjc50
Asynchronous does not help you on latency at all - it's not the latency of the
clock that matters. It's the data latency. This is already kind of crippling,
as it takes hundreds of cycles to get anything from RAM if it's not in your
cache.

Only way to improve that is to make programming languages push either data
locality or dataflow like designs, which may mean giving up or reimagining OO.

Error rates (yield) are a serious problem, but if your design is uniform
enough you can very well get away with just disabling bits. There's also a
less obvious problem with on-chip variation (OCV) - some physical properties
have a gradient across the chip. At runtime, there's a temperature gradient as
well.

~~~
api
Or... continuing with the BFC concept... make chips with monster caches or
even with all SRAM running at core speed. That's not been done in a while.

~~~
pjc50
Caches effectively are SRAM. It's inherently much less dense than DRAM; I had
a quick look on Intel and the largest I could see was 37Mb of cache.

~~~
n00b101
It's not SRAM, but Kights Landing has 16GB of "MCDRAM" which is high bandwidth
3D-stacked DRAM. 400+GB/s throughput but latency is similar to conventional
DRAM.

------
carlob
Adamatzky seems to be churning out cookie cutter articles on slime molds and
highways

[https://scholar.google.com/scholar?q=Adamatzky+highways&btnG...](https://scholar.google.com/scholar?q=Adamatzky+highways&btnG=&hl=en&as_sdt=0%2C39)

------
amelius
How about the breakthrough in silicene production? [1] Could we use this to
start stacking transistors in the vertical direction?

[1]
[http://spectrum.ieee.org/nanoclast/semiconductors/materials/...](http://spectrum.ieee.org/nanoclast/semiconductors/materials/breakthrough-
in-silicene-production-promises-a-future-of-silicenebased-electronics)

~~~
imtringued
That will only work for passive components like caches otherwise you will see
higher power consumption and heat dissipation.

~~~
jacquesm
That's true, but there are other options. For instance, clockless computing is
one avenue worth pursuing.

------
tkinom
Moore's Law seems to reach its limit only for CPU clock. But it still moving
very fast for GPU, FPGA, DDR, SSD, etc, right?

Or even in term of Network connection speed - 100mbps, 1G, 10G, 25G, 40G (at
lease in data center where it matters a lot.)

The simple usb connection speed also continue to increase at a fast rate.

------
worik
"Nautilus uses cookies to manage your digital subscription and show you your
reading progress. It's just not the same without them"

Well, no thanks. No, it is not the same.

------
james-watson
As usual, memristor based computing architecture is left out.

Intel's 3DXPoint has already brought memristor memory units to market. The
beauty of memristor architectures is that they can be used for logic and
storage, so you can create some radical new architectures with them.

I'm too lazy to link the talk by Stan Williams of HP, but if you google it
it's the ~45min one.

This is the next architecture because it's already here. All that other stuff
is vaporware.

~~~
grkvlt
From the article: "However, ternary computing may be spurred on by a new
device called the memristor. [...] Memristors might become one of the
industry’s hopes for extending Moore’s Law beyond the age of the silicon
transistor."

------
TheArcane
What about Quantum computing?

~~~
GrumpyYoungMan
Quantum computing has nothing to do with what what we would consider "regular"
computing. Even if it became practical, at most it would be a specialized
accelerator for certain types of computation, like factoring numbers.

A decent explanation can be found here: [https://uwaterloo.ca/institute-for-
quantum-computing/quantum...](https://uwaterloo.ca/institute-for-quantum-
computing/quantum-computing-101)

