
Superconducting Optoelectronic Neurons I: General Principles - indescions_2018
https://arxiv.org/abs/1805.01929
======
philipkglass
This is part one of five. Here are the other 4:

Superconducting Optoelectronic Neurons II: Receiver Circuits:
[https://arxiv.org/abs/1805.02599](https://arxiv.org/abs/1805.02599)

Superconducting Optoelectronic Neurons III: Synaptic Plasticity:
[https://arxiv.org/abs/1805.01937](https://arxiv.org/abs/1805.01937)

Superconducting Optoelectronic Neurons IV: Transmitter Circuits:
[https://arxiv.org/abs/1805.01941](https://arxiv.org/abs/1805.01941)

Superconducting Optoelectronic Neurons V: Networks and Scaling:
[https://arxiv.org/abs/1805.01942](https://arxiv.org/abs/1805.01942)

It's so ambitious and so questionable at the same time. I'd classify it as
hard science fiction, like a really good Orion's Arm entry, if the authors
weren't from NIST.

Final two discussion paragraphs from paper V:

 _While it may be difficult to build systems larger than 10 billion neurons in
the near term, such a system is not physically limited. Like the brain, such
limits will be incurred due to the velocity of signal propagation. From Fig.
6(c) we know that networks as large as data centers can sustain coherent
oscillations at 1 MHz. Such a facility would house 10^8 300 mm wafers if they
were stacked 100 deep. This would result in 100 trillion neurons per data
center across modules interconnected with another power law distribution._

 _Networks need not oscillate at 1 MHz, and if they supported system-wide
activity at 1 kHz—faster than any oscillation of the human brain—the neuronal
pool could occupy a significant fraction of the earth’s surface and employ
quintillions of neurons. We do not wish to cover earth in such devices, but
asteroids provide ample, uncontroversial real estate. The materials for this
hardware are abundant on M-type and S-type asteroids [76–80]. It appears
possible for an asteroid belt to form the nodes and light to form the edges of
a solar-system scale intelligent network. Asteroids can be separated by
billions of meters, so light-speed communication delays may be several seconds
or longer. For cognitive systems oscillating up to 20 MHz, such delays would
cause individual modules to operate as separate cognitive systems, much like a
society of humans._

Apart from the breathtaking scale of speculation -- which one could admittedly
also find to good effect in older papers about e.g. nuclear power -- there is
a more concrete question. What's the all-in energy cost-per-operation vs.
conventional hardware, CMOS devices operating above room temperature?
Operating at liquid helium temperatures dramatically shrinks the on-chip power
demand, and then the cryocooler dramatically re-inflates it. Lab scale
production of liquid helium takes ~570 watts of wall-plug power to produce 1
watt of cooling near 4.2 K [1]. At the wall plug, cryogenic cooling systems
included, how does this design compare to existing hardware on power and speed
for neural network training or inference? AFAICT, the authors do not attempt
to estimate such a figure of merit.

[1] Basics of low-temperature refrigeration:
[https://cds.cern.ch/record/1974048/files/arXiv:1501.07392.pd...](https://cds.cern.ch/record/1974048/files/arXiv:1501.07392.pdf)

~~~
philipkglass
I answered more of my questions about energy costs: in part II Appendix C the
authors explore the energy of a synaptic firing event. The on-chip energy
expended: 41 attojoules. (Though in part V the authors conclude that their
earlier Part II analysis was probably flawed, and the circuits probably need 4
times as much current.) That 41 attojoules on-chip translates to about 24
femtojoules of whole-system energy consumption. That in turn compares
favorably to the ~10 picojoules (10000 femtojoule) of energy required to
perform a single precision floating point arithmetic operation on CMOS
hardware. But this "neuromorphic" design is not easy to compare to
contemporary hardware on complete tasks, because it's more analog than
digital. And it's not clear that it can easily interoperate with existing
artificial neural networks built on digital logic, either for training or
inference. A low level "direct" comparison may be effectively impossible.
Instead we'd have to ask questions like "how many joules per face recognized?"
for complete facial recognition systems, admitting that GPU-based ANNs and
superconducting optoelectronic ANNs would have very different internal
structures.

Finally, I'll note that these designs rely on large numbers of Josephson
junctions. From my quick Wiki-skim it appears that nobody has ever built large
scale integrated circuits from Josephson junctions. It doesn't look like there
have even been serious Western attempts at it after the 1980s. That's not to
say that large scale fabrication of circuits incorporating Josephson junctions
is folly, but it looks like it requires a lot of basic R&D effort before
somebody can build one of these superconducting optoelectronic chips, much
less a whole wafer full of interconnected copies.

~~~
posterboy
I don't think a star-system sized computational network is supposed to be used
for the mere face recognition that my phone can already do. So the metric may
be comparing apples to oranges.

~~~
philipkglass
Perhaps. But this design needs to perform well on some relatively mundane
terrestrial applications before it scales up to stellar megaproject size.

