
Cutting Edge Deep Learning for Coders, Part 2 - mlboss
http://course.fast.ai/part2.html
======
jph00
Jeremy from fast.ai here. I've posted a summary of the course materials for
anyone who's interested :
[http://www.fast.ai/2018/05/07/part2-launch/](http://www.fast.ai/2018/05/07/part2-launch/)

Let me know if you have any questions about the material or approach. There's
also a discussion for the course here:
[http://forums.fast.ai/c/part2-v2](http://forums.fast.ai/c/part2-v2)

~~~
jonahx
Hey Jeremy, thanks for making these courses! I was curious if you could offer
any points of comparison between the fast.ai courses and the Udacity ML
nanodegree course (or any other MOOCs you have opinions about).

~~~
syncopate
I am not Jeremy but I have taken part in the current course and also took some
lessons by Siraj (who also does lessons at udacity), Andrew NG, and Karpathy.
I think all of them are awesome teachers. Like Siraj's lessons, the fastai
course is full of practical examples, whereas Ng/Karpathy's courses are more
theoretical. What makes the fast.ai course unique compared to Siraj (who
covers a lot of topics very briefly) is that Jeremy delves into depth, taking
a long time explaining things, so you have a lot of opportunities to
understand what's going on. In the 2018 course there is actually not that much
different content compared to the 2017 course, rather Jeremy took the time and
tried to make everything easier for everyone to understand and built a better
python library. Also another unique point about Jeremy is that he and Rachel
want not only to teach people but rather inspire a community of people working
together in teams. The fast.ai library will not be just a project for the
course, it will be something that a lot of people will use and contribute to
to make pytorch better.

~~~
bitL
Is Rachel the lady that always keeps interrupting Jeremy so that he goes out
of flow all the time, making lectures less enjoyable?

~~~
jph00
Dr Rachel Thomas, the co-founder of fast.ai, is the person who takes the
highest voted questions from our 600+ in-person students and passes them on to
me, as I asked her to do. At least 4 different people must have wanted the
question asked before she asks it.

This approach helps ensure that if I haven't explained something clearly
enough that I get another chance to do so, and, more importantly, keeps me
fresh and energized throughout each lesson (I get stale and boring without
some interaction).

~~~
bitL
I am a firm believer in candid feedback; I talked to multiple people taking
fast.ai and all of them took it as disturbing the flow of lectures. I really
like fast.ai, it's an excellent hands-on course, so please don't get offended.

~~~
ghufran_syed
Personally, I find the questions helpful, and for everyone who's watching on
video, you can skip whatever portion you like, can't you?

~~~
bitL
The thing is you can visibly see how it interrupts Jeremy; he is in the middle
of explaining something, then has the face of a surprised person, loses
context for a few seconds etc. Better IMO would be if he just finished a small
section in its entirety then did Q&A, instead of allowing himself being
interrupted all the time. And often those questions are missing the point,
which is to be expected with newbies, so the lecture loses efficiency. If I
didn't know most of the basics already, I'd have problem following them. I
found just doing the exercises better for learning.

~~~
krispin
I also find the questions very helpful. I understand for many people who
already have a good understanding, these questions might be interrupting the
flow. Many a times the questions his students raised were things that I had
not thought of, sometimes they even stumped Jeremy. Fortunately Jeremy in most
cases gives answers which vary in length, according to whether it's a concept
which he will clear later, or whether it is an important concept that needs to
clarified then itself.

So I think this more effective way of learning/teaching even though there is a
loss in efficiency.

------
source99
I can't say enough good things about the fast.ai courses. They may not be for
everyone but they are definitely for me. The teaching style is a perfect match
for my learning style. The style is to get some code working and then dive
deeper and deeper to understand it better and better.

Note this link is for part 2 of the course which is significantly more
advanced than part 1. Start with part 1 if if you are new.

------
apohn
Jeremy - First of all, I just want to say thanks to you and Rachel for the
course(s). I worked through part 1 (2017 version) last year and it was
fantastic.

I've got part 2 on my to-do list for this year. Looking at the 2018 part 1
course, quite a lot has changed (from Keras to Pytorch?) from a code
perspective in comparison to the 2017 part 1 course. If I want to start part
2, does it make sense to go-ahead with the 2018 version? Are there any 2018
part 1 lessons you recommend to bridge any gaps from the 2017 part 1 before I
start 2018 part 2?

Thanks!

~~~
gcmac
Not Jeremy (obviously) but having taken both versions of the course I would
definitely recommend jumping into the 2018 version. The core content is the
same (image recognition, recommender systems, NLP) but the 2018 version uses
more cutting edge tools and has a tighter feel (as a v2 of anything usually
does).

~~~
imnotadoctor999
The field is changing so quickly, but fast.ai is always ahead of the curve
teaching modern techniques! Hats off to them for being able to keep up, AND
share that with others.

------
doall
I watched all of the 2018 Part 1 videos and it was not my style.

One thing I liked is the positive attitudes of Jeremy and Rachel, but there
was not much theory and a lot of time was spent on questions and answers for
the participants of the lecture that I don't think necessary.

I wanted to see definitions and "why", but the course spends too much time on
"how". Sometimes I see "why" in the lecture, but many times it is just an
answer for the question from a random participant and the answers seemed not
well prepared, so it makes it hard to deeply understand the content.

I think fast.ai (at least Part 1) may be good after taking other deep learning
courses and before participating in the Kaggle competitions that when you are
already familiar with the theories.

~~~
jph00
Top-down vs bottom-up depends to some extent on what you enjoy, vs what you
need to be patient about. With our top-down approach you _do_ get to all of
the 'why', but only after understanding 'how'. So it's good for people who
want to get started doing stuff right away, and don't mind waiting a bit to
understand the details. It means you can start experimenting and building a
good intuition for training models, which I believe is the most important
skill for a practitioner.

On the other hand, with the bottom-up approach of Andrew Ng in
deeplearning.ai, you start with a lot of 'why', and later on get to 'how'
(although in less detail and fewer best practices than we show). So it's good
for people who want to understand the theory right away, and don't mind
waiting a bit to understand how to use it.

A lot of our students did Andrew's course after ours, and many did it in the
reverse order. All have reported finding the combination more helpful than
either on their own. When we describe 'why' it's mainly with code, whereas
with Andrew it's mainly with math - so which you prefer will also depend on
which notation and framework you're more comfortable with.

(But I promise - you do get all the 'how' with us, particularly in part 2! Our
students have gone on to OpenAI, Google Brain, and senior AI leadership
positions at well known startups, as well as writing and implementing new
papers. Here's an example of a student who just implemented a paper that was
released within the last month: [https://sgugger.github.io/deep-painterly-
harmonization.html#...](https://sgugger.github.io/deep-painterly-
harmonization.html#deep-painterly-harmonization) )

------
sampathweb
I have taken the classes and also did Udacity Nano Degree and Deeplearning.ai.
All of them help you in your journey of mastering DL. But Fast.AI part-2
really teaches you to cutting edge of DL through a number of practical
examples. Don't wait and join other fellow learners at forums.fast.ai

------
YeGoblynQueenne
>> Welcome to the new 2018 edition of fast.ai's second 7 week course, Cutting
Edge Deep Learning For Coders, Part 2, where you'll learn the latest
developments in deep learning, how to read and implement new academic papers,
and how to solve challenging end-to-end problems such as natural language
translation.

I would really like to know how to solve natural language translation. I think
everyone would. Many people have been trying to solve this devilishly hard
problem for several decades and failed. So I'm really curious how fast.ai has
finally manged to do it.

~~~
pixelHD
Well, I looked at the summary, and they're implementing a Seq2Seq model for
this. It is what I think of as an archetype for machine translation and chat
bot tasks.

Quite a few new network architectures in this space have been updates to this
model, which uses an RNN encoder and a decoder, along with attention between
them and a beam search for better results.

I wouldn't call this model a solution for natural language translation, nor
would anyone else. But I think fast.ai meant that they're going to explain and
go through this model, and how it's helped bring a new generation of models
with good performance in this particular space.

~~~
jph00
Yup it's multi-layer bidir seq2seq with attention, and a few tricks like
teacher forcing. Same as Google Translate. Their version takes a long time to
train on a lot of GPUs, so we simplify it by using less layers, and a smaller,
simplified corpus (it only contains questions, and limits them to 30 words
long).

By "solve end-to-end problems" I only mean that we show how to do the whole
process from beginning to end - I didn't mean to imply that the final model
would be human-equivalent or perfect or anything like that.

~~~
pixelHD
Oh, nice!

Yeah, I understood the intention behind that statement. Great work with the
course!

While you're here, what do you think about using temporal convolution for
sequence tasks? I've read a few articles, this particular one by my professor
comes to mind now [0], which say CNNs could work extremely well for the tasks
traditionally done with RNNs. A recent paper by the people at Google Brain [1]
mentioned that their CNN with attention network beats traditional RNN
approaches. More surprising is that the network is 130+ layers deep, and yet
trains faster than RNNs. Do you think we can potentially switch most machine
translation tasks to CNNs?

[0]: [https://towardsdatascience.com/the-fall-of-rnn-
lstm-2d1594c7...](https://towardsdatascience.com/the-fall-of-rnn-
lstm-2d1594c74ce0) [1]:
[https://twitter.com/lmthang/status/989261575482560513](https://twitter.com/lmthang/status/989261575482560513)

~~~
jph00
Yup potentially. The jury is still out on which will win, or whether both have
their place!

------
kriro
I have nothing to say other than that I loved DL1 and I'll dive into DL2 right
away. I really like the overall philosophy ("see DL is approachable and
doable, just do it") and I love that they encourage you to read papers even if
they seem hard and to translate math to code etc. etc. Fantastic job fast.ai
team.

It's pretty damn amazing that they build a library (and also the new
fastai.text) that goes one level of abstraction beyond PyTorch with the goal
of implementing interesting/helpful papers ASAP and because they wanted to
make their teaching even more efficient.

------
DrNuke
Hi Jeremy, Thanks again to you and Dr Rachel for these amazing courses and the
support to the community. My question is the following: are you thinking of an
organic fast.ai connected hub or foundry acting as a distillator, that is
disseminating & implementing in / with / through the fast.ai library the most
interesting research papers coming out day in day out from arxiv.org and
events? It is nearly impossible to follow the progress without clear state-of-
the-art perspective these days, which you have in abundance indeed.

~~~
jph00
We're certainly looking for ways to help develop the community. Obviously
[http://forums.fast.ai](http://forums.fast.ai) is a good starting point, and
is a very active and supportive community nowadays. But I'm interested in
finding ways to support face-to-face communities too. Some students are
already doing that - for instance AI Saturdays ([https://nurture.ai/ai-
saturdays](https://nurture.ai/ai-saturdays)) has study groups in 50 cities
now, and I'm told ~5000 people have been involved in them.

I'm always open for ideas in other ways we can help people get involved and
stay up to date. And I'm very interested in supporting students who are trying
to build these things in their local community.

------
tedsanders
Question: What is the job market like for folks who spend two months learning
deep learning, either through a class like fast.ai or a bootcamp like Insight
AI? What sorts of companies hire these people and how much do they pay? I know
it varies, but I'm just trying to get a sense to calibrate my expectations of
the current market. Thanks for any perspective anyone can offer.

~~~
DrNuke
As an outsider and / or away from the SV, where such an environment thrives
and opportuinities are a few dozens, your mileage may vary a lot. Personal
network is always better to refer to than shots in the dark though. In
general, you might need showcasing your knowledge through marketable case
studies in a sub-domain you are interested in, so that interviews will come
from a shared context.

~~~
jph00
Exactly right - it's all about your portfolio. We spend a lot of time
throughout the course talking about ways to build that portfolio, and folks on
the forums support each other with feedback and help on these projects.

Completing a MOOC isn't going to help you land an interview for a job,
generally speaking (although if it's any good, it should certainly help you
once you do get an interview!) However if you use your study time to, for
example, create useful projects that you make available on github, web apps
that you host on heroku, deep technical blog posts you post on medium, etc
then you should definitely get plenty of interview opportunities.

I've seen many many students go through this process, so I know it works. That
includes students that hadn't been getting interviews or job offers - until I
convinced them to spend time building a portfolio, and then they got multiple
offers from top companies.

(There are some companies - not too many thankfully - that strictly require a
PhD for DL applicants. I've noticed that such companies, that focus on
credentials over actual work output, seem to overlap a lot with companies that
turn out to have toxic cultures. So I'm not sure you should worry too much
about them...)

------
40four
Wow great share! I was unaware of these courses! Great work @jph00

------
ironfootnz
That's a really good upgrade.

