
Building a Silicon Brain - yigitdemirag
https://www.the-scientist.com/features/building-a-silicon-brain-65738
======
joe_the_user
Among the problems you get with the neuromorophic chips is that they are
choosing which hypothetical neural algorithm the chip should use and which
silicon implementation should be made of that algorithm.

Current AI is continuously modifying it's algorithms. Personally, I'd want to
see parallel chips with a programming interface at least as "general purpose"
as the GPU.

One might even boldly say something like "the only way this deep learning
explosion happened is through the consumer market making powerful parallel
chips ubiquitous". Or more cynically, "if chip maker had been looking at only
AI applications, they would have strangled the deep learning explosion by
their tendency to demand $10K+ per installation since they look at the well-
healed corporation that easily afford these and thus ignore the graduate
students who make the conceptual progress in their basements." Even Nvidia is
eager to divide their offerings between $500 game boards and $5000+ deep
learning boards. One assumes a neuromorophic chip would be way up this "high
end".

~~~
snrji
And why not the GPUs themselves? Which inherent limitations do they have?
Aren't they essentially general purpose parallel devices?

~~~
gbrown
No, GPUs are quite different than CPUs with more cores. They're optimized for
Singl-Instruction-Multiple-Data algorithms, in which the same operation is
done at the same time to many different inputs (think image or video
processing). They do very poorly in cases with complex branching logic.

~~~
snrji
I know, but isn't that what the OP was asking for? Isn't that an inherent
tradeoff? Or: isn't that what Neuromorphic hardware and deep learning are
about (ie. composing many relatively simple functions without complex
branching/logic)?

------
sowbug
Transmitting analog voltages in neuromorphic circuits reminds me of a cool
experiment I heard about a while ago. A team used an evolutionary algorithm to
adapt an FPGA to recognize sounds. When they got the behavior they wanted,
they tried copying the circuit to another FPGA and found that it didn't work
because the algorithm had evolved dependencies on specific quirks in the
original physical FPGA. I can't figure out how to link to the PDF directly,
but the original research is probably "An Evolved Circuit, Intrinsic in
Silicon, Entwined With Physics" by Adrian Thompson, which tells the story far
better than I have.

It's neat to think that our own brains might have adapted wiring that works
specifically for the actual molecules that make up our bodies. It's uniqueness
on a different level.

~~~
jcims
Lovely article about the paper, one of my favorite reads:

[https://www.damninteresting.com/on-the-origin-of-
circuits/](https://www.damninteresting.com/on-the-origin-of-circuits/)

~~~
havlenao
Thx. it's great

------
andrepd
On this topic, more specifically of whole-brain emulation, a very interesting
report I found a few months ago: "Whole Brain Emulation: A Roadmap". It's one
of a very rare breed: an evaluation of a _future_ technology that _does not_
enter into wild speculation or wishful thinking, but remains critical and
objective as far as possible.

[1]: [https://www.fhi.ox.ac.uk/brain-emulation-roadmap-
report.pdf](https://www.fhi.ox.ac.uk/brain-emulation-roadmap-report.pdf)

~~~
gwern
It's unfortunately quite old, predating the DL revolution. I think it would be
written much differently now: emphasizing more the possibility of brain-
_like_ architectures (perhaps using real brain data as rich 'supervision':
[https://www.reddit.com/r/reinforcementlearning/comments/9pwy...](https://www.reddit.com/r/reinforcementlearning/comments/9pwy2f/wbe_and_drl_a_middle_way_of_imitation_learning/)
) rather than direct brain emulation, and more skeptical about the extent to
which deep accurate simulation down to the molecular level is really necessary
for human-level intelligence.

I've suggested repeatedly to FHI & Sandberg that an update of the Roadmap
would be a very good project, but given how little progress there has been on
true brain emulation since then (especially compared to DL), I'm not surprised
that nothing's happened.

------
jo-m
Another interesting kind of hardware they have at INI (Giacomo Indiveri's
institute) are the dynamic vision sensors (initially developed by
Lichtsteiner, Posch, Delbruck, Berner, Kramer).

Those are cameras modeled after the human retina, and they overcome the
processing and frame rate bottleneck of traditional cameras by using an event
based architecture (event stream of pixel brightness changes -> no data if
nothing changes). This allows microsecond latency in machine vision systems,
without the high data rates you get with frame based cameras.

* Overview: [http://siliconretina.ini.uzh.ch/wiki/index.php](http://siliconretina.ini.uzh.ch/wiki/index.php)

* Company selling them: [https://inivation.com/dvs/](https://inivation.com/dvs/)

* Another company selling them split off by one of the PHDs: [https://www.prophesee.ai/](https://www.prophesee.ai/)

* Demo video: [https://youtu.be/jnzPuDUsP4w?t=14](https://youtu.be/jnzPuDUsP4w?t=14)

------
colinator
This neuromorphic stuff is super cool. I wish I could just buy some! I'm the
author of spikeflow (github.com/colinator/spikeflow). I'd love to support
neuromorphic hardware.

For the moment, biological neural networks (brains) are, I believe, many
orders of magnitude more powerful, and especially more efficient than, current
computers. But it's hard to say, because it's so hard to tease out the
important parts of what the brain does, because it does so much. There are so
many types of neurons, so many types of connectivity, so many interacting
neurotransmitters, so many support cells that affect computation... almost
like the brain is the result of a billion years of hack-upon-hack evolution...

~~~
yigitdemirag
This a very cool repository indeed! Have you compared the speed/accuracy of
spikeflow with other SNN libraries like Brian2 on decent GPUs (or even TPUs)?

~~~
tjungblut
agreed, very cool repo. Are you looking for help? I have some hours of my life
to devote for some cool ML stuff.

~~~
colinator
Yeah, totally! You can find me email from my github profile.

