
Perceptrons from memristors - godelmachine
https://arxiv.org/abs/1807.04912
======
zgao
I don't mean to be too negative here, but this is hardly a new development, so
can someone clarify the novelty in this paper? Neural nets have been
extensively demonstrated in memristor-based architectures [1] and several
memristor-based training architectures have previously been proposed and
tested [2]. The abstract's claim that "no model for such a network has been
proposed so far" is prima facie blatantly false.

In any case, I have yet to see a conclusive, publicly explained solution to
the significant system-level problems with memristor-based neural
architectures, or indeed any analog neural architecture. The best claimed
digital architectures are around ~250 fJ per multiply-and-accumulate (MAC)
[Groq], and these generally involve 8-bit multiplication, which is extremely
expensive in the analog domain thanks to the exponential scaling of power with
precision levels. Even if you set aside the monstrous fabrication and device-
level variance issues with memristors, DAC and ADC consume tens of pJ per
sample in the realistic IP blocks that are commercially available. Although
only one pair of DAC and ADC operations is required per dot product, this is
still 40 fJ per MAC from DAC and ADC alone, assuming a 256x256 matrix
multiplication and not taking other system-level issues into account. This
limits memristors to a 5x over current digital architectures, and as nodes
shrink, by the time memristors come out, this will be around a 3x. While a 3x
is considerable, I don't think it justifies the moonshot-level deep tech risk
that memristors will continue to represent. Many hardware companies
[Tabula...] have failed attempting to reach something like a 3x in the main
figure-of-merit, only to find that system-level issues get them a 1x instead.
Besides, I'm sure digital architectures have more than 3x room for
improvement- plenty of tricks left for digital!

I'm hoping for a breakthrough, because I am fundamentally an optimist, but
memristors have been failing to deliver since 2008.

[1]
[http://www.cs.utah.edu/~rajeev/pubs/isca16-old.pdf](http://www.cs.utah.edu/~rajeev/pubs/isca16-old.pdf)
[2]
[https://ieeexplore.ieee.org/document/7010034/](https://ieeexplore.ieee.org/document/7010034/)

~~~
orbifold
Can you please give a reference for the ~250fj MAC?

~~~
zgao
You can go to Groq.com -- that startup claims to have 125fj per flop (and each
mac is two flops thanks to marketing logic). Started by 8 out of 10 founding
TPU team members.

