
The New Intel: How Nvidia Went from Powering Video Games to Revolutionizing AI - sonabinu
http://www.forbes.com/sites/aarontilley/2016/11/30/nvidia-deep-learning-ai-intel/#5ded7d2a39cc
======
cs702
Yes, Nvidia dominates in deep learning and AI, but the risk of disruption by
Intel is _much_ greater than a lot of non-technical people realize. Intel has
to do only two things to start taking market share away from Nvidia in DL/AI
_quickly_ :

1\. Add "good enough" functionality to its high-end processors. Intel is
already working on that:
[https://news.ycombinator.com/item?id=12709220](https://news.ycombinator.com/item?id=12709220)

2\. Contribute open-source code to the main branches of the most popular DL/AI
frameworks (Tensorflow, Theano, Torch) so these frameworks support the new
chip functionality "out of the box," without requiring any additional
tweaking. This is not yet happening, but I'm hoping it will soon.

Many DL/AI developers would be content with "good enough" performance out-of-
the-box from CPUs if it means not having to pay extra for Nvidia cards or deal
with Nvidia's proprietary drivers.

It would be great for Nvidia to get real competition in this space.

~~~
modeless
There is zero chance that Intel can add "good enough" functionality to their
CPUs without changing their memory architecture. A high end i7 might have 35
GB/s of memory bandwidth. Even the biggest, most ridiculously expensive
($6000) Xeons have only about 100 GB/s. Nvidia's GTX 1080 has 320 GB/s. Their
P100 has 732 GB/s.

The threat to Nvidia from Intel is in the Nervana chips they recently
acquired. Those are presumably using HBM2 and could potentially beat GPUs for
neural net training performance.

~~~
boulos
A modern Xeon has 10s of MBs of cache, which has drastically higher memory
bandwidth. What matters is how big the matrices (the "model") is not the input
data. With native 8-bit or 16-bit support, you can fit an awful lot of model
in 30MB.

I'd recommend reading this paper about matrix math on GPUs from friends of
mine way back in the day:
[https://graphics.stanford.edu/papers/gpumatrixmult/gpumatrix...](https://graphics.stanford.edu/papers/gpumatrixmult/gpumatrixmult.pdf)
. While NVIDIA has built much larger register files and L2 caches since then,
a modern Xeon still is unbeatable when something fits in L2 or even L3 cache.

I believe streaming the input data in to update the weights is usually done
with non Cache polluting instructions (mm_stream equivalents with the NT
hint), so it's not hard to keep it fed.

~~~
modeless
SOTA models are much, much larger than 30 MB. You can't use 8 bits for
training (yet?), and Intel doesn't support 16-bit float (they need to catch up
with Nvidia on that, it's really a travesty, we could have used it years ago
for audio and imaging). Besides, the top end Xeons with the most cache are
fully 10 times more expensive than a GTX 1080.

~~~
varelse
IMO the _travesty_ is crippling FP16 on consumer GPUs for exactly the reasons
you cite. And given that AMD's Vega GPUs apparently will possess full speed
FP16, it appears to be yet another in a long line of arbitrary performance
cripplings for the consumer space that is IMO the most annoying feature of
CUDA development.

But while FP16 _is_ useful for audio and imaging, FP16 nearly killed NVIDIA
over a decade ago* when it lacked the dynamic range for DirectX 9 HDR effects
in contemporary games without banding. FP32 was more than enough for the task,
but power hungry, and thus NV30 could be used to figuratively fry eggs while
AMD GPUs had FP24, which was just enough for these effects.

These days, it's all going in the opposite direction w/r to deep learning, but
I find it ironic that INT8/INT16 is missing from the Tesla flagship P100, but
present on GP102 and GP104, the consumer GPUs (and yes, I know about Tesla
P40, but that lacks fast FP16).

I agree with the top poster that Intel could make quite a comeback here given
how far it's currently behind. I also agree that it would be hard to dethrone
NVIDIA without higher bandwidth memory, but I don't think that's necessary, a
bloody nose is more than enough to turn heads IMO.

[https://en.wikipedia.org/wiki/GeForce_FX_series](https://en.wikipedia.org/wiki/GeForce_FX_series)

------
johansch
This is pure and utter bullshit. Their core market is by far gaming, as this
PDF (that I found via NVidia IR) shows:

[http://files.shareholder.com/downloads/AMDA-1XAJD4/341168899...](http://files.shareholder.com/downloads/AMDA-1XAJD4/3411688990x0x917045/7D7AA043-67F6-4D04-A178-B87C3BAA2F0B/Rev_by_Mkt_Qtrly_Trend_Q317.pdf)

Also: The level of their gross profit margin indicates that they are in fact a
monopolist:

[http://marketrealist.com/2016/11/driving-nvidias-profit-
marg...](http://marketrealist.com/2016/11/driving-nvidias-profit-margins-new-
highs/)

(These are not the financials of company operating in a healthy, 20-year old
industry - they are the financials of a company that after operates without
meaningful opposition in a 20 year old business.)

~~~
sien
Nvidia isn't a monopolist. They have two big competitors in AMD and Intel.

Intel ship about 2/3 of the graphics chips for laptops and desktops. Nvidia
have more than half of the remainder. That isn't a monopoly.

In the discrete GPU space AMD are also there and ship almost half as many unit
as Nvidia. That's no monopoly.

The graphics systems for the current consoles are shipped by AMD aren't they?

Nvidia is a profitable company. Calling them a monopolist because they are
profitable is unwarranted.

* Edited to fix grammar.

~~~
dogma1138
AMD ships primarily low and mid range cards, NVIDIA has an effective
"monopoly" in the enthusiast market share which has the highest profit margins
by far.

That said AMD hasn't managed to ship a 350-400$+ card that's worth spending
money on compared to the competition since probably the 7950...

~~~
hueving
That's a ridiculous definition of a monopoly. Using that methodology, you can
claim any company has a monopoly that happens to have the best performing
thing at the time.

Nobody is claiming Tesla has a monopoly on electric cars because they make the
best. And nobody should claim Nvidia has a monopoly on GPUs since they are on
top.

~~~
dogma1138
I didn't claim they had a monopoly, I said they had an "effective monopoly" in
only one, albit very critical market segment, in which they ship over 75% of
the hardware.

------
vegabook
Just like this fawning article, a very optimistic scenario for Nvidia is in
the market pricing of its stock. Nvidia trades at a 45 trailing PE vs Intel's
15. It's valuation therefore assumes at least 3x more growth than Intel. In a
market which it seems literally _everybody_ is trying to target. For example,
Qualcomm, (trailing PE 14), just announced a 48-core ARM server processor with
vector capabilities for launch H217. And if AMD is allowed to be purchased,
and properly funded, by an Asian (or other) entity (already has licencing
agreements in place in China), the space could get very crowded for NVDA.

I wouldn't be so sure that the comparison to the 40-year behemoth of silicon,
which extracted quasi-monopoly profits for 35 of them, is yet valid.

------
spangry
If Nvidia are looking to supplant Intel, I think they key, weirdly, is i/o. In
a sense, GPU acceleration of general purpose computation is just a specific
case of the more general case of distributed computing (albeit the computers
are extremely 'close'). For embarrassingly parallel workloads, i/o is not an
issue; the data and computations can be batched, distributed and no further
communication required (until you collect the results).

But for mixed, semi-sequential/semi-parallel workloads, I/O becomes the
bottleneck. Having to retrieve intermediate results, combine and then
redistribute can eat up your gains from parallelisation. It will be
interesting to see if Nvidia start pushing for a new standard to replace PCIe,
and maybe invest in low-latency, high throughput networking R&D. Who knows?
Maybe they'll just build some kind of networking functionality in to their
GPUs directly, so data can be transferred NVRAM -> NVRAM without having to
travel up and down the stack (until they need to interact with userspace; once
on the way in and once on the way out).

Given there are RDMA adapters that can manage ~20ns latency already, it would
be interesting to see what could be achieved before we start pushing up
against the laws of physics. Even at the physical layer, my understanding is
that most networking fibre optics manage about 60% the speed of light.

So, just to make a wild prediction: within the next year, Nvidia will acquire
a HPC network adapter company (e.g. Mellanox).

~~~
jburgess777
Nvidia already has high speed GPU-GPU links:

[http://www.nvidia.com/object/nvlink.html](http://www.nvidia.com/object/nvlink.html)

~~~
jamesblonde
They aren't widely deployed yet - the DGX-1 uses it, though.

I assume it will be infiniband that will be the commodity hard adopted to
distribute Deep Learning. However, Nvidia are currently trying to segment the
market by providing GPU->GPU direct access via infiniband on only the Tesla
cards - not the commodity ones (titanx, 1080 gtx). That strategy will probably
be the death of them in Deep Learning.

~~~
greglindahl
Why are you assuming Infiniband is the winner? There's Intel Omni-Path and
there's Ethernet.

(Disclaimer: I'm the system architect of InfiniPath, which is one of the
things that evolved into Omni-Path.)

~~~
spangry
Ah, just the man I've been wanting to talk to then :)

As I mentioned in another post, I really think Intel are missing an
opportunity when it comes to high speed networking. Why don't they drop all
the crazy server market lock-in vendor shenanigans (e.g. brand to brand SFP
incompatibility, 'call us' pricing etc.) and just jump straight to the
consumer market? I know this means cannibalising CPU revenue, but that's going
to happen anyway.

Developers are consumers. In many cases the direction that their skills
develop in is largely determined by the consumer hardware they're running at
home. The same seems to be true for Ops folks, who all seems to be running
vSphere homelabs atm. Based on what others have said here, it sounds like
NVIDIA have chosen to cross-sell, and work their way down from HPC -> ... ->
Consumer, presumably to extract maximum rents, despite not yet having strong
enough network effects o lock out competition.

This seems like the perfect opportunity for Intel to cut the legs out from
under that strategy, by aggressively pushing widespread consumer (or at least
'power-user'/developer) take-up of high speed, low-latency networking. And
hasn't iWARP been endorsed as an IETF standard (or is on its way to be)? It
seems like there's a window of opportunity here, albeit one that gets smaller
and smaller as Nvidia slowly work their way down market.

~~~
greglindahl
Well, Omni-Path has a 48 port switch ASIC. So that's it for consumer
accessibility.

But if you want to buy the PCI Express cards, Newegg has 'em. Intel has a
great channel organization.

~~~
spangry
It's certainly impressive tech. But, and bear in mind this is just from a
consumer perspective, here are the reasons I wouldn't buy it:

\- ~$8000 USD for a 24 port switch is way beyond my price range. I doubt any
consumer market could support this
([http://www.newegg.com/Product/Product.aspx?Item=9SIA6ZP53G35...](http://www.newegg.com/Product/Product.aspx?Item=9SIA6ZP53G3510))

\- ~$550 USD for a single port adapter is also way outside of my budget,
especially since I'd need to buy at least half a dozen of these for my home
lab
([http://www.newegg.com/Product/Product.aspx?Item=N82E16833106...](http://www.newegg.com/Product/Product.aspx?Item=N82E16833106290))

\- It's very difficult to find solid information on the product. The Ark page
doesn't tell me anything ([http://ark.intel.com/products/92007/Intel-Omni-
Path-Host-Fab...](http://ark.intel.com/products/92007/Intel-Omni-Path-Host-
Fabric-Interface-Adapter-100-Series-1-Port-PCIe-x16)). The link to the
whitepaper (which I'm assuming is actually a marketing brochure) 404s.

\- So even if I could afford this _as a consumer /dev_, I probably wouldn't
buy it due to the uncertainty around total cost of ownership. Will it accept
non-Intel branded QSFP transceivers? Same question with optical cabling.

\- Given the product brief mentions 'fabric performance [will] scale
automatically with ongoing advances in Intel Xeon processors...', it leaves me
wondering if this is somehow dependent on some specialised Xeon CPU
instruction sets, making it useless for most home labs (which, at best, will
be running older generation Xeons from second-hand servers, but more likely
CPUs from Intel's consumer line-up).

On costs: If these are just reflective of the cost of production, then fair
enough, I guess I'll just have to wait (for either Intel, or another
manufacturer, to devise a cheaper production process). If it's not, then I
think you're missing the opportunity outlined above, and you'll lose the HPC
fight given Nvidia occupy the high ground here.

On compatibility: If this is just some idiot 'multi-channel cross sales'
strategy that some genius with an MBA has cooked up, you're going to have a
bad time. It falls apart the second someone releases a commodity adapter
(which may already exist, for all I know).

------
ksec
Slightly off Nvidia and thinking Intel for a bit.

Where is Intel heading? I dont see Intel gaining a foot in the AI / ML market.
Not Knight XXX. And all these supposedly weapon such as next gen AVX, Nervana
aren't coming any time soon. And Nvidia knows full well, the same thing about
GPU and AI/ML isn't in the silicon, it is in the drivers and library.

So not only this is a huge first mover and mature software advantage, the AI /
ML is difference in gaming, where the human cost of Nvidia AI/ML knowledge is
significantly more then what the Hardware is worth, which is next to nothing
when compared to those salary.

So in the next 2 - 3 years, I dont see Intel getting much from Nvidia. They
failed Mobile. Windows 10 is now slowing working towards ARM.

It leaves them with 1), a shrinking, or downtrend PC market. 2) Their Server
CPU market with heavy margin which will finally be getting a lots of
competition from AMD Zen x86 and Qualcomm ARM Server Chip.

Zen Server chip is highly unlikely to be competitive against intel in
performance. But there are lots of margin for AMD to attack. It means a lot of
pressure from Intel to lower price.

A lot of ARM Server will be coming in the next few years. Not low end blade,
but powerful Xeon like chips. I have always been skeptical of this, but it
seems ARM, which is owned by Softbank, whom is also the largest shareholder of
Alibaba, whom also operate a gigantic Cloud infrastructure, are All in on
this.

This is not saying Intel will die in 5 years time. They are likely to be
around for decades after, but i dont see where they could grow and head.

------
agumonkey
It's funny when you think about NVIDIA erratic debut (NV1, SEGA). Then the
constant progress to the top of the GPU field. I remember how quickly pro CAD
graphic cards became obsolete market-wise (ES, wildcat) when Transform and
Lighting was implemented. Now "AI", car vision, HPC .... really "funny".
Thanks to y'all Quakeheads I guess. And Crysis.

~~~
Keyframe
E&S, 3dlabs... I do not miss those at all!

~~~
agumonkey
I could never afford one, they were only luxury icons to me. Were they
complete scams or just overpriced ?

~~~
Keyframe
At the time they were the only game in town if you wanted OpenGL to run
decent(ish) on Windows. It was still a time when you could buy an SGI. Drivers
were a mess and price was steep. They weren't a scam, their products just felt
lazy-done. Luckily, 'gamer' cards came quickly to OpenGL land.

~~~
agumonkey
I remember the weird span when 3DS tried to work with vendors with the heidi
interface.

------
boxerab
Puff piece. Yes, Nvidia dominates at the moment, but as many have said, this
could easily change with a resurgent AMD. They have made all the right moves,
IMHO, since Lisa Su took over:

0\. Finally paying attention to software part of equation : hardware for
compute has always been very good

1\. drivers are now excellent

2\. whole line of GCN devices (since 2012) are supported and still optimized,
unlike notorious Nvidia nerfing of older cards

3\. Has x86 license - if Zen is a success, can build high-bandwidth fabric
between CPU and GPU, unmatchable by nVidia except on exotic Power8 arch.

4\. new project focusing on HPC on Linux

5\. they keep pushing OpenCL, which will win over Nvidia-only CUDA

6\. New tools to semi-automatically port CUDA code to run on their hardware.

Well, here's hoping AMD gives them a run for their money in 2016 </fanboy> 6.

~~~
dogma1138
0\. Finally paying attention to software part of equation : hardware for
compute has always been very good

 _By now supporting porting CUDA "natively", still no real answers to the
datacenter CUDA features like DMA, proper virtualization, and networking._

1\. drivers are now excellent

 _The Crimson driver suite is a step in the right direction, if only every 6
months they didn 't had a release that actually physically damages cards._

2\. whole line of GCN devices (since 2012) are supported and still optimized,
unlike notorious Nvidia nerfing of older cards

 _Yes that 's an admirable feat, but then again not one really cares about 5
year old hardware for either gaming or ML_

3\. Has x86 license - if Zen is a success, can build high-bandwidth fabric
between CPU and GPU, unmatchable by nVidia except on exotic Power8 arch.

 _NVIDIA also has the x86 license it 's more limited and AMD (w/ Intel) did
bash them when they tried to add x86 interoperability to their HPC parts but
NVIDIA has a very very large IP portfolio._

4\. new project focusing on HPC on Linux

 _AMD has a new project every 2 years, they tend to not die, CUDA on Linux is
excellent it 's also the recommended platform._

5\. they keep pushing OpenCL, which will win over Nvidia-only CUDA

 _OpenCL performance on NVIDIA GPU 's is still better, and OpenCL will beat
CUDA has been touted for nearly 10 years now..._

6\. New tools to semi-automatically port CUDA code to run on their hardware.

 _With still pretty poor results in many cases, and it only works if you use
the most basic use cases under CUDA and this was the bare minimum they had to
do so people would even look at AMD hardware at large scales these days since
virtually every high performing library is written in CUDA simply because it
offered a better solution and NVIDIA actually provides whatcha call it - ah
right support..._

Well, here's hoping AMD gives them a run for their money in 2016 </fanboy> 6.

~~~
boxerab
Let's compare notes end of next year :) I think things will be quite different
by then.

~~~
mastazi
Unfortunately, AMD executives have signalled in several interviews that they
just don't care about deep learning. Also, I don't think AMD can start
focusing on this field 10 years later than Nvidia and close the gap in a year
as you suggest.

~~~
boxerab
apparently, times are changing: [http://wccftech.com/amd-vega-mi-25-instinct-
gpu-accelerator/](http://wccftech.com/amd-vega-mi-25-instinct-gpu-
accelerator/)

~~~
mastazi
Oh yeah, it seems AMD wants to prove me wrong (I hope so):
[http://radeon.com/en-us/instinct/](http://radeon.com/en-us/instinct/)

------
deepnotderp
Well, people seem to forget that all it will take from Intel is to make an
ASIC like Nervana's (hint, hint:which they acquired) and suddenly Nvidia's in
serious trouble.

~~~
pjscott
Intel plans to release their Nervana-based chip, which they're calling "Lake
Crest", next year.

[http://www.forbes.com/sites/moorinsights/2016/11/21/intel-
co...](http://www.forbes.com/sites/moorinsights/2016/11/21/intel-commits-to-
nervana-roadmap-for-ai-first-new-architecture-in-decades/)

~~~
deepnotderp
That has its own problems... Not allowed to tell you, don't ask me.

------
j1vms
> "I always think we're 30 days from going out of business," Huang says.
> "That's never changed. It's not a fear of failure. It's really a fear of
> feeling complacent, and I don't ever want that to settle in."

I dunno where this company is going next or even if it will be around
(independently) in 10 years, but if that's _really_ the attitude of its CEO,
then at least today it's definitely in good hands.

------
bitmapbrother
When AMD acquired ATI a lot of people speculated that Intel would acquire
Nvidia. It never happened. What followed was a disastrous series of
acquisitions from Intel that never really amounted to much with McAfee being
the worst of the lot.

------
crb002
Von Neuman bottleneck is ripe for disruption. Ultra low power processors in
memory, a few 3ghz cores, and a fat matrix multiply ASIC.

------
stefantalpalaru
non-paywalled link:
[https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&c...](https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=3&cad=rja&uact=8&ved=0ahUKEwje0aaIgu3QAhUBnxQKHacFAQwQFggoMAI&url=http%3A%2F%2Fwww.forbes.com%2Fsites%2Faarontilley%2F2016%2F11%2F30%2Fnvidia-
deep-learning-ai-intel%2F&usg=AFQjCNFGcF9I_0uRQq4qwUsQv_SlO_5fwg)

------
BoysenberryPi
I'm not really ready to call Nvidia the new Intel. Nvidia seems like such a
volatile company and I have no idea why. I just feel like I could wake up
tomorrow and they could be gone.

~~~
okreallywtf
Care to elaborate? I don't follow them closely but they have seemed to be
consistently on top of the GPU market for some time now, at least in the
gamer/enthusiast market.

~~~
mschuster91
Just look at how fast AMD fell. Everyone thought they 'd profit big time from
Bitcoin... I remember that I couldn't buy an AMD GPU for weeks in any store in
Munich, because everyone and their dog bought them as BTC mining rigs. And
then, couple of months later? ASICs entered the field and prices for used AMD
GPUs fell through the floor.

The same can happen to NVIDIA, especially when Intel decides to launch
something inside their CPUs that is "good enough" for many people.

~~~
TillE
The vast majority of Nvidia GPU sales are to gamers. Intel has been trying to
build a "good enough" iGPU for years and years, with almost zero progress
relative to their contemporary discrete GPUs.

Even the best current GPUs can't handle the inevitable future high-res VR.
This is going to be a stable business for a while yet.

~~~
mschuster91
> Intel has been trying to build a "good enough" iGPU for years and years,
> with almost zero progress relative to their contemporary discrete GPUs.

Intel hasn't had real pressure to innovate for years. They watched the battle
and fall of NVIDIA/AMD, and in the meanwhile enjoyed good sales of CPUs for
all the cloud computing.

Now, they have intense pressure to innovate - as in the server space ARM is
going to eat them up, due to the superior power efficiency, laptop/desktop
sales are down (again, because a four-year old computer works just fine if you
upgrade its RAM to accomodate Chrome).

Intel needs something mind-blowing and that quick. And NVIDIA too, casual
gamers are shifting to console (where AMD has quite a customer base) and
mobile (where Samsung/Apple with ARM are in the lead). I haven't seen any
performant Intel or NVIDIA mobile SoC solution yet.

~~~
petecox
> mobile (where Samsung/Apple with ARM are in the lead). I haven't seen any
> performant Intel or NVIDIA mobile SoC solution yet.

Nvidia does have the Tegra, for the Android tablet market.

Interesting that they've ceded Windows on ARM (aka the rumoured Surface Phone)
to Qualcomm - I would have thought Denver's code-morphing architecture would
be better suited to emulating x86 instructions than a Snapdragon 835.

~~~
mschuster91
> Nvidia does have the Tegra, for the Android tablet market.

Which isn't used anywhere except a couple of tablets according to Wikipedia.
Samsung, inarguably leader of the high-class Android manufacturers, uses
either its own SoCs or Qualcomm. Many cheaper phones use Mediatek or Allwinner
chipsets.

~~~
dogma1138
It's used for it's automotive products.

It's not used because most likely NVIDIA never really wanted to push it, it
checked the water a bit with a few 1st and 3rd party tablets/android gaming
consoles but this always looked more like to recoup some investment on the
side while developing the SOC than an end goal.

The lessons learned from Tegra allowed NVIDIA to build the best (or at least
the most powerful) automotive integrated solution currently on the market and
at least as far as software compatibility and performance go they have no real
competition.

------
logicallee
[I've withdrawn this (joke) comment. It had read as below the line.]

\--

 _This is totally normal for companies to do. In the 1920s Nvidia made car
carborateurs. In the 1820s they made horsebuggy whips. In the 1700s they made
ships of the line, which they had made going back to the 15th century. It 's
little known fact that the Niña, the Pinta and Santa María (Columbus's three
ships) were powered by NVidia sails. yawn._

~~~
rgbrenner
? nvidia was founded in 1993

~~~
logicallee
it was a joke, and if it were upvoted I wanted to keep it but if it was
downvoted erase it. I guess I didn't do well but due to your reply I can't
erase it now.

nintendo really was a playing-card company foudned in the 19th century though.

~~~
jonahrd
And Nokia used to make rubber boots and elevator cables!

