The key difference seems to be restructuring the course to include EBM (Energy Based Models). From the README:
I thought I was going to repropose the same practica I've used during NYU-DLSP20, last year edition, just in different order.
But I couldn't.
This year's students have LV-EBMs on their side. We told them about the cake and now I cannot pretend it doesn't exist and teach as if they were unaware of the elephant in the room. It would have been intellectually dishonest. Henceforth, I've redesigned my whole deck of slides.
...
Last year material is still valid. This year you have a different take. A more powerful one.
I was a contributing student in the course in Spring 2020. Seems that the main difference is in the last 3 weeks and a lot more detail in how to train and infer when using Latent Variables in EBMs (they covered this last time but that content was mostly verbal, whereas there is more time dedicated this time in the slides and lectures).
A lot more examples on SSL too.
Alfredo also went all in on the visuals again. Amazing job.
Alfredo Canziani is awesome and did a lot of the heavy lifting for this course. I agree with siddharthb_, it would be great to include his name in the title.
I've started doing the course, I was looking forward to learning from the legendary LeCun, but am finding him often difficult to understand—he has an unusual accent and talks fast and somewhat lazily. In everyday conversation that would be no problem, but not when frequently introducing new words, concepts, acronyms without warning. I've more than a few times been totally unable to decipher what he said, and the greatest challenge in the course so far has been understanding what words/acronyms he's saying, which isn't ideal. Alfredo on the other hand is a total pleasure.
Most of the content doesn't seem to be there. e.g. When I click on "Week 1" there are just three headers and nothing to click/read/watch/do. Am I missing something about how to use this site to get the course?
Here's a separate course titled "Introduction to causal inference (from a machine learning perspective)". It's from Brady Neal, who works in the group of Yoshua Bengio.
There are economists who are working on ML and causal inference. I know of no courses but I can refer you to some papers...
(These titles are from memory bc I am typing with one hand as I hold a sleeping infant with the other.)
- taddy, et al “deep IV”.
- Greg Lewis at Microsoft research has work here too... the title of that paper I cannot remember.
- Farrell, Liang and Misra, “neural networks for estimation and inference” (econometrica 2021)
- For non neural network based ml informed approaches to causal inference, victor chernozhukov, alexandre Belloni and Christian Hansen have a long series of papers going back to 2011 (sometimes with other coauthors) which are generally based on the LASSO in settings with many instrumental variables.
- chernozhukov, demirer, duflo, et al have a paper on “double machine learning” which is relevant. I think there is relevant work in biostat by Jamie Robins, but that is not my field.
- athey, imbens and wager have papers on random forest based approaches - at least one is in JASA. Might be called “causal forests” ?
- since economists frequently estimate in the GMM (generalIzed method of moments, not Gaussian mixture model) framework, there is some recent work by... Kallus (?) on formulating GMM as an adversarial game. Greg lewis and
V. Syrgkanis have also worked on this problem but the title of the paper escapes me.
I would like to teach this material myself so there will one day be a class (inshallah) but so far have not had time to organize it yet!
I know it's an introductory course, that's why I was asking if anybody knew any course that did. And even if it is introductory, you need some previous knowledge to get to this course.
I haven't found any CS curriculum that teaches causal inference in any way (maybe in optional courses, but that misses the point) and I think that it is a huge mistake. Causality is more important than simple prediction.
I don't understand why CS people escape causality in most courses, not even mentioning it.
The site itself appears to be additive (only new materials posted in later weeks) to the old 2020 site, which is here: https://atcold.github.io/pytorch-Deep-Learning/