
Intel Introduces Neuromorphic Test Chip - 40acres
https://newsroom.intel.com/editorials/intels-new-self-learning-chip-promises-accelerate-artificial-intelligence/
======
visarga
> This extremely energy efficient chip, which uses the data to learn and make
> inferences, gets smarter over time and does not need to be trained in the
> traditional way.

Does not need to be trained? The writer maybe wanted to say it does not need
to be _programmed_ in the traditional way.

If this was a news site, I'd understand - the same writers cover too many
topics, they can't be experts in all. But when it comes straight from Intel,
it's weird to see such mistakes from someone who signs as Dr. Michael
Mayberry.

> Researchers have demonstrated learning at a rate that is a 1 million times
> improvement compared with other typical spiking neural nets as measured by
> total operations to achieve a given accuracy when solving MNIST digit
> recognition problems.

MNIST is a joke already, why don't they show off the ImageNet numbers? And
spiking neural nets are less efficient compared to normal neural nets, so the
1 million speedup might not be as impressive as it appears at first hand. They
don't compare with GPU run TensorFlow or CNTK here, but with other spiking
neural nets which are seldom used, mostly as a proof of concept.

> A total of 130,000 neurons and 130 million synapses.

So the network size is medium - you can do a lot with 130 mil weights, but
there are models on GPU that go up to 1 billion weights. Even the "cat
detector" unsupervised neural net created by Google in 2011, had 1 billion
weights, and that's ages ago in AI time.

~~~
Seanny123
> Does not need to be trained? The writer maybe wanted to say it does not need
> to be programmed in the traditional way.

There are two ways to build spiking neural net architectures without training.
The [Neural Engineering Framework]([https://medium.com/@seanaubin/a-way-
around-the-coming-perfor...](https://medium.com/@seanaubin/a-way-around-the-
coming-performance-walls-neuromorphic-hardware-with-spiking-neurons-
facd4291b201)) and [Deneve's approach which I'm more uncertain
about]([https://forum.nengo.ai/t/robustness-of-deneves-
networks/320?...](https://forum.nengo.ai/t/robustness-of-deneves-
networks/320?u=seanny123)).

To be clear, when I say "training" in this context, I mean they don't require
iterative rounds of stochastic gradient descent. They just need to be given
the function that you want to approximate and then they're done.

~~~
visarga
Yes, maybe it uses something other than gradient descent, but why? Nothing
works nearly as well by comparison. In a chip we can do things we can't do in
the brain, such as sending gradients back on the same connections.

~~~
Seanny123
> Nothing works nearly as well by comparison.

Depends on what you mean by "well". It would be foolish of me to deny the
advances of deep learning. However, there are limitations and the NEF can act
as a bridge in those cases. For example, if you want:

1\. To approximate a dynamic system 2\. To use symbolic computation 3\. Use
the neural networks as an initial guess at a solution to a problem and then
want to use gradient descent to figure out the actual solution

I talk about using NEF as a bridge in [this blog
post]([https://medium.com/@seanaubin/deep-learning-is-almost-the-
br...](https://medium.com/@seanaubin/deep-learning-is-almost-the-
brain-3aaecd924f3d)), if you're curious. Unfortunately, I don't know of any
examples using the "initial guess" paradigm that I describe, but that's more
of a "not enough people using Nengo" problem.

------
meri_dian
>"Researchers have demonstrated learning at a rate that is a 1 million times
improvement compared with other typical spiking neural nets as measured by
total operations to achieve a given accuracy when solving MNIST digit
recognition problems."

>"Further, it is up to 1,000 times more energy-efficient than general purpose
computing required for typical training systems."

Wow. That is tremendous.

~~~
zitterbewegung
Comparing a system to MNIST isn't really an indicator that it provides an
actual improvement. Comparing it to VGG would be a better indicator.

~~~
p1esk
MNIST is a dataset, VGG is a software model (network architecture). You can
train/test VGG on MNIST, train/test this spiking network model on MNIST, and
compare the accuracy between the two.

If you want to compare energy efficiency, you look at the hardware these
models are running on (CPU, GPU, FPGA, ASIC), and see how much energy is spent
by each of these chips to reach a certain accuracy (preferably on ImageNet).

You can also compare the speed of processing (significantly affected by image
size): 1\. How fast can you get to a certain accuracy? 2\. How many images per
second can you classify with that accuracy?

~~~
0xbear
>> MNIST is a dataset, VGG is a software model

Strictly speaking, true. But one discovers early enough that you can screw up
a number of things when training a model on something as simple as MNIST and
the model will still work, because there's so much redundancy, the correct
layers will just train their way around the ones that don't really work.

VGG trained for general purpose image classification with a decent sized input
is a much better indicator of whether or not some ML hardware improvement is
useful.

~~~
p1esk
You're contradicting yourself. If you're afraid that redundancy will hide
hardware problems, then VGG is a bad choice, because it's pretty much the most
redundant NN model ever :)

------
tehsauce
This hardware is designed for running spiking neural network models? What is
the current state of the art in this field? I was under the impression that
training a spiking neural network was somewhat of an unsolved problem, because
backprop doesn't easily apply. Anyone have information on this?

~~~
aurelian15
There are several frameworks for the construction of spiking neural networks
given an abstract mathematical description of the desired function. The most
complete (note that I'm currently a student in the lab that developed this
method) is the Neural Engineering Framework [1]. There is a Python reference
implementation called Nengo [2] which can target several backends, including
neuromorphic hardware.

Training deep spiking neural networks is possible as well [3].

[1]
[http://compneuro.uwaterloo.ca/research/nef.html](http://compneuro.uwaterloo.ca/research/nef.html)
[2] [https://www.nengo.ai/](https://www.nengo.ai/) [3]
[https://arxiv.org/abs/1611.05141](https://arxiv.org/abs/1611.05141)

~~~
Seanny123
I'm from the lab that you're citing with those links and I'm kind of amazed
you're so up to date with our research. Why do you know so much about Nengo?
Are you actually one of my lab-mates, but I just can't identify you from your
username?

~~~
moh_maya
" (note that I'm currently a student in the lab that developed this method)".
Lab mate?

~~~
Seanny123
Huh, apparently. This will be an interesting conversation at the next lab
meeting.

------
bhouston
How does this compare with Google's proprietary neural network optimized chip?
That seems like a place that Intel could kick ass in and sell tons to Amazon
and Microsoft's cloud services for NN acceleration, especially if they
supported the main NN libraries.

And if Intel isn't pursuing the opportunity to compete for deep learning NN
against Google's chips, why aren't they? It seems like a well defined
opportunity, more so than whatever these chips are.

~~~
dnautics
I don't know for sure, but google's chips are advanced matrix multiplying
engines. the "neuromorphic" chips are (in a very stupid-fied explanation) kind
of like imagine a configurable fpgas with more-than-one-bit lines connecting
adjacent tiles.

~~~
copperx
> google's chips are advanced matrix multiplying engines

Excuse my ignorance, but aren't GPUs exactly that?

~~~
hedgehog
GPUs are generally much more flexible and can run something resembling a
normal C program. From the little public information available chips like
Google TPU or NVIDIA's Tensor Core (a functional unit on the Volta
architecture) are essentially hard-wired to do exactly low precision matrix
multiplication. Less flexible but more efficient when you know that's what
your workload is.

[https://devblogs.nvidia.com/parallelforall/inside-
volta/](https://devblogs.nvidia.com/parallelforall/inside-volta/)

[https://www.nextplatform.com/2017/04/05/first-depth-look-
goo...](https://www.nextplatform.com/2017/04/05/first-depth-look-googles-tpu-
architecture/)

------
lend000
Looking forward to when there is more information than just a press release
(the press release for Intel's 3D X-Point memory was over two years ago, and
we still aren't seeing those in production). Regardless, very exciting work,
and I look forward to getting my hands on the API/instruction set/manual.

~~~
profquail
3D XPoint has been available for several months now, just not in the DIMM form
factor. E.g. the Intel P4800X:

[https://www.anandtech.com/show/11209/intel-optane-ssd-
dc-p48...](https://www.anandtech.com/show/11209/intel-optane-ssd-
dc-p4800x-review-a-deep-dive-into-3d-xpoint-enterprise-performance)

------
CoffeeDregs
"It’s a future where first responders using image-recognition applications can
analyze streetlight camera images and quickly solve missing or abducted person
reports."

Erp. That's also known as "a massive surveillance network", right?

"It’s a future where [your government] using image-recognition applications
can analyze streetlight camera images and quickly solve [whether you're
involved in pre-crime]."

This stuff is going to happen (or is already here) so perhaps it's time to
switch to managing pervasive surveillance rather than preventing it.

------
m3kw9
So this thing only can be trained with a the built in algorithm?

~~~
visarga
Maybe there is a way to port regular neural nets onto spiking nets - you'd
train a regular network first using your algorithm of choice, then transfer-
learn to the spiking net.

------
Paradoxinabox
Skynet? Skynet.

~~~
kensai
Or Borg?! [http://tinyurl.com/borg29](http://tinyurl.com/borg29)

------
caqlar
Intel's countermove for Apple's Bionic chip. They are investing more and more
in human/environment identification.

