No PDP book? It's old and weird but interesting and has a lot of original ideas, notwithstanding the actual original backprop being from before then. Nor the original backprop stuff?
The PDP book is the main textbook for a course I'm taking at CMU called... PDP. It's digestible, but man is it weird to see things like "this is an ongoing area of future research" where the future = after 1986.
I find it kind of hard to relate to for that reason — how do I know (besides asking my prof., whose PhD advisor was Hinton, or doing my Googling) what ideas ended up "sticking"? What areas of future research went nowhere vs spawned whole new subfields?
Is there a more modern textbook of the sort I could cross-reference?
Nobody really cares about this stuff anymore except weirdoes like me so you might actually be best off asking your prof.
There's also a lot of resurrection that goes on with weird ideas: FastFood cites a paper that cites a paper that cites Lecun's brain damage paper, for example
For those interested in PDP, one of Jay McClellands students is Professor Randy O'Reilly at CU. He develops the emergent neural network simulator and is using biologically plausible deep learning to learn how the mind works. He's been doing it all along. There was never a gap!
The PDP papers are in a bunch of volumes, in addition to the handbook. Chapter 3 of the handbook, I recall, has a little bit from Hinton's original BM work which is of interest to the cognitive scientist, as well as a pretty obstruse but very good description of the entire process. Very historical, very strange by modern ANN standards.
> Very historical, very strange by modern ANN standards.
That's what I gathered from a brief look at things, which is why I find it very interesting - both from a historical context, as well as the idea that maybe there are some nuggets of (possibly overlooked?) wisdom hidden within, that might be useful in today's world.
Then again, I am but an interested (and very amateur) hobbyist in this whole ML space, and not likely to come up with any breakthroughs - but you never know I suppose!
Well, if there's any overlooked nuggets of wisdom, Hinton would probably still know, and Sejnowski, and Smolensky, and McClelland, and all those other folks still doing DL stuff, as they have been doing continuously since Reagan was president.
