Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Yes, I'm not concerned with the computational cost. Reading the paper more carefully, I can't see any obvious weaknesses.

The main idea is based on [1]: a brain can learn exactly the same things as what can be learned with backprop. How come this is not more widely known? This sounds like a huge breakthrough to me.

https://www.mitpressjournals.org/doi/full/10.1162/NECO_a_009...



They don't address the fundamental issue of BPTT in an RNN model. To my mind this is a much larger block to biological plausibility than how to track the error locally. Besides this has been addressed in various forms in earlier work:

- Hochreiter and Schmidhuber present a truncated version of BPTT in the original LSTM paper that tracks part of the error locally and forward in time (https://www.bioinf.jku.at/publications/older/2604.pdf)

- Bellec et al. https://www.nature.com/articles/s41467-020-17236-y present a similar algorithm that tracks the error with something they call eligibility traces. Despite the title of the paper the method is also applicable to non-spiking RNNs.

- Bohnstigl et al. present another solution: https://arxiv.org/pdf/2007.12723.pdf




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: