
Huang’s Law Is the New Moore’s Law, and Explains Why Nvidia Wants Arm - bookofjoe
https://www.wsj.com/articles/huangs-law-is-the-new-moores-law-and-explains-why-nvidia-wants-arm-11600488001
======
TazeTSchnitzel
Spoiler: the “Huang's law” this author is trying to coin is that AI processors
(surely this means ML) are still following something like Moore's Law in terms
of rate of improvement of processing speed per year.

Which is cool I suppose but comparing it to Moore's Law seems a bit unfair.

~~~
CyberDildonics
I feel that I took the days where headlines just said what an article was
about for granted.

Also it is extra silly because Moore's law is about transistor density and
transistor cost.

~~~
colejohnson66
I have heard Moore’s law generalized to “ _computing power_ doubles every two
years” which could then apply to “AI”

~~~
CyberDildonics
That's not a generalization, it's a misunderstanding. Not every program can
benefit from doubling transistors because not every program is written to
minimize serial computations and the impact of memory latency.

The reason Moore's law was insightful was because he didn't make the
simplistic generalization of computing power (which is more difficult to
quantify anyway) and looked at what was really happening on a fundamental
level from an engineering perspective.

------
paulsutter
They're both really Wright's law. Theodore Wright published a paper in 1936
describing how aircraft production costs decline at a constant percentage with
each doubling of production. The formal term is manufacturing learning curve.

[https://ark-invest.com/wrights-law/](https://ark-invest.com/wrights-law/)

~~~
supernova87a
Well, to be specific that seems more about manufacturing efficiencies
(optimizing waste, batches, waiting time, etc as a line learns how to be more
efficient) while this law and Moore's law are more about the intrinsic design
of the chips (hardware and software).

~~~
klelatti
Plus Moore's law has a time based element. I don't think anyone could have got
to a 5nm process in 1970 say simply by deciding to up the volume of
transistors being produced.

~~~
xiphias2
Generallys Moore’s law is time based, Wright’s law is production based. Both
are great models, but Ark Invest’s researcher found Wright’s law to be more
precise.

------
bookofjoe
[https://archive.vn/gCPp3](https://archive.vn/gCPp3)

------
klelatti
The rationale put forward in this piece for Nvidia wanting to acquire Arm is
essentially that AI compute will move to the 'edge' and that as Arm dominates
compute in the edge Nvidia needs to defend its position and to follow that
compute. So in one sense it's defensive of Nvidia's existing position.

Once again though did Nvidia need to buy Arm to do this? It could license its
own IP and /or compete with SoC's of its own for this market. As in the server
market the advantage of ownership (and what is worth $40bn) is the advantage
it gives Nvidia in information and control of what is done with Arm IP.

------
devilmoon
It's interesting that they picked Amazon Go as an example of the power of AI
(or ML) when in actuality most of the grunt work in those stores is done by
"dumb" sensors AFAIK. The store "knows" that an item has been picked up from a
certain shelf and thus it assumes that the item is what is usually on that
shelf, it has little to do with either CV or AI in general, just careful
placement of the items and weight sensing. I think that if you let a few kids
roam free in there picking stuff up and setting it down in an unexpected place
the magic of AI would dissolve into thin air.

~~~
psanford
Have you actually visited one?

I've been a couple times to one of the SF locations with coworkers. We
definitely tried to trick system by passing items between us, and putting
items down in random locations and then picking them back up later. When we
left the store everyone got charged for the items they had on them.

I think you are too dismissive in saying its "just careful placement of the
items and weight sensing".

~~~
trimbo
How do you know there aren't humans on the other side of the cameras doing the
item-customer matching?

Go has relatively few stores. Seems like the sort of thing you can still do
manually, especially if the model expresses doubt.

~~~
tracer4201
I went to one of these stores in Seattle on a weekday afternoon with a couple
friends. It was literally packed with people. We tried breaking the system by
leaving items in random places and exchanging them between us. It still
charged me accurately for a couple bottled waters, a sandwich, and some other
snacks.

~~~
trimbo
Again, how do you know there aren't humans on the other side of the
cameras/sensors facilitating this?

At the scale they're operating at and the accuracy you point out, to me it's
more plausible they're having humans assist.

~~~
nicoburns
I think they're saying that humans behind cameras likely wouldn't be _able_ to
do this accurately.

~~~
trimbo
Except that cameras in stores have been used for anti-shoplifting for decades.
They've been used for more subtle things like monitoring tables at casinos.

I mean, I'm just asking the question how people know for sure Go isn't (at
least partly) a Turk. It seems like an obvious thing to ask? But I first came
to wonder if there's manual component involved after shopping a few times at
Go. When it was busy, it took sometimes well over an hour to get a receipt.
When it was empty, it was immediate. How come?

Also, the accuracy is completely anecdotal both ways... at least one person I
visited the store with never got billed.

------
dvfjsdhgfv
[https://www.marketscreener.com/quote/stock/INTEL-
CORPORATION...](https://www.marketscreener.com/quote/stock/INTEL-
CORPORATION-4829/news/Huang-s-Law-Is-the-New-Moore-s-Law-and-Explains-Why-
Nvidia-Wants-Arm-31323896/)

\- the exact same article for those who don't have a wsj-account

------
ksec
I read the whole article a few times. But nothing in there explains why Nvidia
wants ARM.

Is this one of those PR articles [1] that try to convince public about the
ARM-Nvidia deal being great for the industry? ( While I prefer ARM to stay
independent, I am not exactly _against_ Nvidia acquiring it )

[1]
[http://www.paulgraham.com/submarine.html](http://www.paulgraham.com/submarine.html)

------
lmeyerov
Relevant chart I did mapping how much $1 gets you in GPU compute on AWS over
the last 10 years:
[https://twitter.com/lmeyerov/status/1232937998464901120](https://twitter.com/lmeyerov/status/1232937998464901120)
(note the log scale)

------
xiphias2
The key to doubling AI operations is changing the architecture significantly,
which sadly makes it harder to use the platform itself as well.

After CUDA we had to use Tensor cores, later float16, now sparce matrix
multiplications that work because deep learning uses lots of RELUs.

~~~
mikorym
CD players are a more legacy example of hardware doing magic, in this case, a
whole lot of XORs. [1]

[1] [https://youtu.be/fBRMaEAFLE0?t=651](https://youtu.be/fBRMaEAFLE0?t=651)
"Free with every CD, a Cray supercomputer"

------
baybal2
The author fundamentally misunderstands how the industry functions.

Money wise, the ARM acquisition for this amount of money makes no sense if
they want to save on core, or architectural licenses.

With the amount of money, and engineers Nvidia has, they can easily bootstrap
a CPU ISA on their own.

It was by far not their top priority.

And in their function relevant to the "AI" trend as giant linear algebra
crunchers, GPUs don't need much help from anything else.

And in longterm, basically anything you do pales to logarithmic scaling of
PPW/relative performance. A s..ttier, and cheaper chip few years down the line
invariably beats current best.

In practical terms, the winner in the GPU/CPU/WhateverPU is decided by who
gets the better fab contract.

------
lsorber
Where's the data that says Moore's law no longer holds? I see comments and
articles asserting this but everytime with evidence. The data that I do find
certainly still suggests Moore's law is doing fine.

~~~
pedrocr
Moore's law is still on pace but Dennard scaling has broken down for ~15 years
now. That means we're not getting the same speedups as we were used to and
that gets explained by "Moore's law has broken down" as a title.

[https://en.wikipedia.org/wiki/Dennard_scaling](https://en.wikipedia.org/wiki/Dennard_scaling)

~~~
dorfsmay
I did not know about Dennard law, thanks for that link. Any idea why it has
slowed down?

~~~
blp
Dennards law held until the approximately linear relations that made it work
ended. Delay stopped being entirely gage, and more importantly, voltage
couldn’t drop forever, due to material limits and intrinsic silicon limits.

------
microcolonel
This is silly. Just because you can build bigger chips doesn't mean you have
anything similar to Moore's law, which was more about the economics of
semiconductor manufacturing.

~~~
beervirus
Moore’s Law is literally about the number of transistors per IC. Building
bigger chips is... extremely similar.

~~~
blp
Yes, except the important parts. In Moore’s law scaling, you reduce the cost
per transistor, and because of that there is money for capex and the consumer
benefits.

In Huangs law, si is commoditized, so the benefit accrues to the designer and
software makers, and really only in narrow domains.

------
coverband
I’m wondering if there are any firewall/router/load balancer type of devices
that use GPUs for their parallel processing capabilities (or whether that’s
even possible). Anyone familiar with this ecosystem to comment?

~~~
sipherhex
Disclosure: I work at NVIDIA.

This is a developing area with a number of research and development efforts
ongoing looking at basic packet filtering flows, for example:
[https://ieeexplore.ieee.org/document/7019193](https://ieeexplore.ieee.org/document/7019193)
[https://arxiv.org/pdf/1312.4188.pdf](https://arxiv.org/pdf/1312.4188.pdf)

and AI-driven approaches like CyBERT showing promise:
[https://medium.com/rapids-ai/cybert-28b35a4c81c4](https://medium.com/rapids-
ai/cybert-28b35a4c81c4)

One of the main system challenges has been efficient high-bandwidth and low-
latency delivery of packet flows between NICs and GPUs. NVIDIA has their cuVNF
library (part of the Aerial SDK) that works with GPU Direct NICs and extends
the DPDK toolkit to accomplish this: [https://developer.nvidia.com/aerial-
sdk](https://developer.nvidia.com/aerial-sdk)

However, as for integrated devices, I'm familiar with at least one that claims
to be on the market:
[http://www.h3c.com/en/About_Us/News___Events/News/201907/121...](http://www.h3c.com/en/About_Us/News___Events/News/201907/1218274_294554_0.htm)

------
PaulHoule
On NVIDIA and ARM:

1\. Huang was seen sitting at the wheel of a Mercedes Benz S class, which has
a stupendous amount of NVIDIA silicon. That's great, but the bulk of the
market is for the electronics that go into a Chevy.

NVIDIA owns the highly profitable top end, but competitors (for gfx, robot
brains, ...) are so disorganized that NVIDIA should move downmarket quickly to
capture that terrain even if the profit margin is less.

2\. A cynical take is that of "Patronage Networks". That is if Softbank can't
manage "Return of Capital" to its investors, Masayoshi Son might come to his
room and find a katana to cut himself with. The $40B from ARM goes a good way
to "saving his ass" and might mean Huang owes him one in the same sense that
politician owes something to voters.

------
k__
Didn't Carmack say Moore's Law still holds? At least long enough to get VR
where he wants it to be.

~~~
kungato
Didn't Carmack recently have a talk where he said he was done with VR and
moving to AGI

~~~
person_of_color
Money talks

------
aey
Nvidia buys ARM, Apple builds its own GPU, and the dance goes on.

------
Larrikin
Why were either allowed to become laws. Moore's law was a little confusing in
high school

~~~
jasode
_> Why were either allowed to become laws. Moore's law was a little confusing
in high school_

Nobody _allowed_ Moore's Law to become a law.

The so-called Moore's Law is really just Gordon Moore's _Extrapolation_ of an
observation. In 1965, he _observed_ that transistor count was doubling every
~12-18 months. He _extrapolated_ that this trend could continue for ~10 more
years to 1975.

But journalists and readers like snappy poetic words. "Extrapolation" is a
long-winded 5-syllable noun so it's no surprise that the 1-syllable "law" was
used instead. Wikipedia says: shortly after 1975, Caltech professor Carver
Mead popularized the term "Moore's law".[1]

Although calling it _" Moore's Extrapolation"_ would be more accurate, it just
doesn't roll off the tongue as nicely. Sometimes humans value the sing-song
poetry of a meme more than its dictionary accuracy.

[1]
[https://en.wikipedia.org/wiki/Moore%27s_law#History_of_the_c...](https://en.wikipedia.org/wiki/Moore%27s_law#History_of_the_concept)

~~~
dathinab
Or Moore's Assumption. Which is less of a mouthful, through also less precise.

~~~
vezycash
Moore's observation.

~~~
ghaff
Absolutely right. An observation that became something of a self-fulfilling
prophecy.

~~~
ordu
I'm not sure about "self-fulfilling" part. If Moore didn't do his observation,
would doubling of number of transistor every 18 months still work?

~~~
ncmncm
The expectation drove capital investment.

The business application of it meant profits from each generation could fund
the higher costs of each succeeding generation.

Now we are in the tens of $billions for a new generation, with much smaller
increases in performance, and it is faltering. Will we spend hundreds of
$billions on factories for smaller transistors? Or will we find a more cost-
effective way to get more performance?

~~~
ghaff
It appears as if chipmakers will continue to drive things as small as they
can, e.g. with EUV (finally).

But the expectation is that a lot of continued performance gains will have to
come through other means. These are some of the things that were talked about
at the Hot Chips conference:
[https://enterprisersproject.com/article/2020/9/moores-law-
wh...](https://enterprisersproject.com/article/2020/9/moores-law-what-means-
today)

------
richajak
I still cannot comprehend the significance of Moore's law. Although the
hardware keeps getting more sophisticated, I cannot perceive that today's
computer is 100x faster or more than 20-30 years ago. Perhaps the software has
been getting bloated to counteract or the speed increase is not real.

For AI chip, will we fall into the same situation? will AI algorithm still
focus on efficiency?

~~~
throw_m239339
While processors have become more powerful, we're now coding our apps in
Javascript, which negates any progress described by Moore's Law. Of course I'm
being hyperbolic but you get my point.

~~~
richajak
Thanks, finally someone agree with my observation. :-) I got deduction point
for saying that statement. For me, it is not right to use 100x speed increase
for eye candy feature or using bloated Javascript libraries.

