Not trying to "Schmidhuber" this or anything, but I think my 1999 NIPS paper gives a cleaner derivation and explanation for working on the Jacobian. In it, I derive a Jacobian operator that allows you to compute arbitrary products between the Jacobian and any vector, with complexity that is comparable to standard backprop.
This is so nice to randomly come across. Let know if there's something that I can do to help (I am the author of CBofN). You can reach me at <my first name>@<my last name>.org.
I don't think this comparison is fair to LeCun. He has literally been at the forefront of ML for over 40 years, starting with his PhD thesis and continuing to the present, and his public fame is entirely a byproduct of the awareness of the impact of his work. (Disclaimer: I've known Yann for decades and worked down the hall from him during his NEC years.)
To be clear, I’m not accusing LeCun of trying to “hoard credit”, rather I’m accusing others of having petty squabbles (often surrounding his published works) around who was “first” and thus “actually” deserves credit. To them, I say “Get over yourself” - progress doesn’t tend to be made when we’re busy trying to win imaginary races for “mindshare”. And LeCun has done more than enough to prove himself in the field (with all the hard work that entails) to have to deal with that sort of thing.
I recently learned that someone is seeking to impersonate me on IG. They've created a new IG profile w/ my name, photo, etc., and now have more followers and friends than I do. They've even started to friend people within my network, and have sent DMs acting as if they are me.
Meta / FB / IG have a reporting form for when identity theft is involved, and it requires that you send supporting documents like your driver's license and a picture of you holding the drivers license. I did all of that.
Here is the entirety of their response:
"Hi,
We have fewer people available to review your request due to the coronavirus (COVID-19) outbreak.
We’re only able to review requests for the most urgent cases. This means we can’t review your case right now. Please try again at a later date. Thank you for understanding.
Your second group represents the core "inner loop" of about a thousand revolutionary applications. Take the basic capability of translating image->text->speech (and the reverse), install it on a wearable device that can "see" an environment, and add domain-specific agents. From this setup, you're not too far away from having an AI that can whisper guidance into your ear like a co-pilot, enabling scenarios like:
1. step-by-step guidance for a blind person navigating the use of a public restroom.
2. an EMS AI helping you to save someone's life in an emergency.
3. an AI coach that can teach you a new sport or activity.
4. an omnipresent domain-expert that can show you how to make a gourmet meal, repair an engine, or perform a traditional tea ceremony.
5. a personal assistant that can anticipate your information need (what's that person's name? where's the exit? who's the most interesting person here? etc.) and whisper the answer in your ear just as you need it.
Now, add all of the above to an AR capability where you can now think or speak of something interesting and complex, and have it visualized right before your eyes. With this capability, I could augment my imagination with almost super-human capabilities that allow one to solve complex problems almost as if it was an internal mental monologue.
All of these scenarios are just a short hop from where were at now, so mark my words: we will have "borgs" like those described above long before we reach anything like general AI.
These are good examples of what we're getting close to, but I'd add that Copilot is already an extremely helpful tool for coding. I don't blindly trust its output, but its suggestions are what I want often enough to save a lot of typing.
I still have to do all the hard thinking, but once I figure out what I want written and start typing, Copilot will spit out a good portion of the contextually-obvious lines of code.
I was also one of Cynthia's Ph.D. advisors when she was a graduate student at Princeton, some twenty years ago. It was obvious to me then that she would go on to do great things, so it's delightful to read this news this morning.
My fondest memory of Cynthia, however, has nothing to do with science, and everything to do with just being a kind person. We were at the NEC Research Institute's company picnic where they had an inflatable dragon for the kids to jump around within its interior. Me, Cynthia, and my wife went inside without any kids and jumped around like idiots for a while. Cynthia and my wife got bored, so I stayed behind for One More Big Bounce. With the epic bounce, I also succeeded in cracking a vertebra, nearly passing out on the spot from the pain. Eventually, I would crawl out, an ambulance was called, and I was brought to the Princeton ER.
I would have a full recovery, but I was in the ER for several hours that night. Cynthia came with us to the ER, and when she saw how uncomfortable I was on the gurney, she went back to her dorm to retrieve her favorite blanket, so that I would have even a small comfort. I am not sure how long she stayed, but I know that she was there with me longer than anyone else except my wife.
Anyhow, she's a lovely human being and I am honored and proud to have known her and witnessed the origins of her career.
In my senior year at Duke, I took her ML class (her first semester at Duke). She was an excellent professor, one of the absolute best I had while there. She focused heavily on both implementation and theory, which I found to be rare.
Her class became so popular within the add/drop period that Duke added a second section and also doubled the attendance for each section. I'm pretty sure she went from being supposed to teach about 70 students to teaching 300. Nevertheless, her teaching was top notch, and I learned more there than pretty much any other CS class, and still rely on this knowledge today!
As a 14 year old in 1981, I bought a Timex Sinclair from Sears with money that I had been saving up for a over a year. Best. Purchase. Ever. What I learned on that $99 machine would change the course of my life.
Thank you, Sir Clive, for helping me to find my calling.
Hence, we've just mapped all rationals (with duplicates) to a single linear list. Since any linear list can always be mapped to the naturals (1, 2, 3, ...), they have the same cardinality.
[*] G.W. Flake & B.A. Pearlmutter, "Differentiating Functions of the Jacobian with Respect to the Weights," https://proceedings.neurips.cc/paper_files/paper/1999/file/b...