
Stanford Class on Deep Multi-Task and Meta-Learning - hamsterbooster
https://cs330.stanford.edu/
======
317070
The field of meta-learning is still very immature though. I can see why you
would already want to start a scholarly discourse on the topic, but I am not
sure how useful these techniques are for the students involved. They are still
very ad hoc and often unprincipled.

This is a good article on the topic:
[https://arxiv.org/abs/1902.03477](https://arxiv.org/abs/1902.03477)

~~~
amcoastal
This is Stanford, they aren't teaching the next generation of applied ML
practicioners -- they are teaching the next generation of theorists. This is a
perfect class for that and getting their students ahead of everyone else. I'm
jealous of them.

~~~
frakt0x90
Assuming 330 is undergraduate level, they're teaching the full gambit of
practitioners, theorists, and future drop outs.

~~~
goliathDown
300 series are advanced graduate classes. As an undergrad the sight of a 300
is horrifying.

~~~
r00fus
It's surprising to me - 300+ level courses were part of my undergrad required
coursework - of course I wasn't studying at Stanford. Are course numbers
standardized across academia or unique to an institution?

~~~
copperx
Course numbers are not standardized, although there are common numbering
schemes. There are some uncommon ones such as MIT's, which uses a number and a
dot instead of a subject name. And I've never known what institution the
typical "CS 101" numbering scheme applies to.

------
panpanna
Asking as someone who does not work with AI but has taken a couple of courses
in ML:

What new things will I be able to do after this course?

(in a practical sense, the technical description on the course page I can read
myself)

~~~
Immortal333
At glance, It looks like ongoing research of Multi-Task and Meta-Learning will
be discussed. Some new tricks on the architecture of NN, Few new methods to
train, and some theoretical setting of the area. I will be looking forward to
seeing whole series. Maybe share some notes or summary of videos.

------
mark_l_watson
Good to see Chelsea Finn end up at Stanford. I had breakfast with her and her
parents in 2013 when she was an undergraduate at MIT and it was fun to hear
what options she was thinking about for her career.

I took a look at the course outline, and except for AutoML, it appears to be a
one stop shop for learning multi-task and meta-learning. I just bookmarked the
lectures on youtube.

------
ArtWomb
Humans observe an object once, such as a cup for drinking water, and we
immediately grasp its "cupness". We can identify infinite varieties of cups
despite variations in morphology, design, utility and context. Simply based on
a single learning instance. This absence of any neural theory of inference is
at the crux of the problem ;)

Shortcut Learning in Deep Neural Networks

[https://arxiv.org/abs/2004.07780](https://arxiv.org/abs/2004.07780)

~~~
hans1729
> _Humans observe an object once, such as a cup for drinking water, and we
> immediately grasp its "cupness_"

 _Adults_ do (i.e. the agents pretrained holistic model of its entire observed
physical context). By reducing the phenomenon to the single observation,
you're conveniently ignoring the early childhood phases spent exploring
shapes/3d-geometry that enable this very ability of inference. this isn't
fair, because regarding humans, the line between training-phase and trained
model is very blurry, whereas a statistical model is trained when the weights
are set and done.

Brute forcing through 2d-projections of 3d-objects (further denormalized
through camera-artifacts etc.) until something sticks in a convoluted (heh)
composition of arbitrarily initialized set of nodes and connections is
_obviously_ far different from the physical exploration kids do. Comparing the
models resulting from the latter with the former is, in a word, absurd.

Through exploration, humans develop a model of physics itself, from which the
nature of _cupness_ can be inferred (which is, in fact, the magic term).

Deep learning alone won't get us there, but it'll probably give us the
components that enable us to simulate this intricate process happening in kids
brains.

In fact, I'm pretty sure that that's what a lot of the smart people
researching general intelligence are working on (because that's what I would
do, excuse my hybris).

~~~
ArtWomb
Good discussion! I'll just respond here, but plenty of though-provoking points
all around ;)

I think what I was looking at was the result that has been often observed,
that progress in AI research roughly tracks with hardware developments.
Looking at AlphaGo to AlphaZero to MuZero. Training time for self-play
increases. But parallelism in the tensor units of the hardware is an order of
magnitude faster. It's great for problem domains like autonomous vehicles,
contactless payments in retail stores and fraud detection in the data center.
But what about generalizability? What about the black box communicating how it
has learned? Will it be suitable for next-gen applications like robots
designed to assist humans in space expansion?

I attended an event in NYC around the creative use of AI by a new breed of
emerging artists like Mario Kliegmann from Germany. ArtBreeder can train a GAN
on a single input sample and generate paintings in the style of Fragonard or
Picasso or Rothko. And someone made a remark along the lines of: "if this had
existed in the 1960s, we wouldn't have need Warhol to invent Pop Art!". But in
reality, Andy Warhol experimented with a wide variety of media and techniques.
From film to "oxidation art". And it struck me that was the truly creative
part of the process. One that arises from a place other than rational
optimization on a single task or even multiple known tasks.

------
arkadyark
This looks super cool! Prof. Finn has been doing a lot of interesting research
in RL and meta-learning for several years, it's great to get a chance to learn
this material directly from her.

