
Practical Deep Learning for Coders 2019 - jph00
https://www.fast.ai/2019/01/24/course-v3/
======
jph00
FYI, Google just completed adding all the pre-requisites for this course to
their Colab service, which provides a GPU-powered Jupyter Notebook for free.
There are no usage restrictions, other than that each session must be no
longer than 12 hours (after 12 hours, you can just create a new session). So
this means you can complete the whole course, and do your own projects, even
if you don't have a credit card.

Here's the setup guide:
[https://course.fast.ai/start_colab.html](https://course.fast.ai/start_colab.html)

~~~
lelima
That's great! I almost burn up my laptop running the NLP classes and wanted
something more scalable that paperspace(is still a great way to run fastai).

Thanks Jeremy!

------
ScoutOrgo
I've gone through each iteration of the fast.ai courses and must say that it
is worth it every time. The latest 2019 courses are again excellent. Even
though I've covered the concepts before, I always learn a ton from every
lesson.

Although it is recommended to dedicate ~10 hours per lesson, it is very
possible to go through each video, then circle back to focus on one of the
concepts/problems presented.

I recommend giving the first lesson a try. It is very rewarding feeling like
you can actually do or build something at the end of each lesson.

~~~
faitswulff
If you haven't done so already, would you recommend going through the old
lessons? Or is the current version good enough?

~~~
brtknr
The courses move with the fastai and Pytorch libraries so its best to do the
latest course so you don’t run into features that have been deprecated or
renamed or miss out on new features.

------
cr0sh
I don't have the time to immediately work on this course, but I might later
this year.

What I'd like to know, as someone who has completed (in the past) the
following MOOCs:

1\. ML Class (plus about half of AI Class) - 2011 2\. Udacity's CS373 Course -
2012 3\. Udacity's Self-Driving Car Engineer Nanodegree - 2016/17

...and gained a lot of knowledge about ML and DL thru them - would this
fast.ai course be worthwhile to pursue as an addition to my prior education on
the subjects? Would I find it just as challenging (from a cursory glance it
looks to be a "yes", but I wanted another opinion)?

I'm open to suggestions from everyone here, not just jph00/fast.ai ...

Also - and this I would also like opinions on - would my time be better spent
pursuing other areas I know I am not proficient in - mainly learning about
probability/stats, as well as basics in calculus...

For instance, in the prior courses, there was always mention of - for say
backprop - stuff about the "chain rule" and other "rules" for calculus, and I
feel I need to understand that more. That is, I am wanting to understand
things in ML and DL at a lower level; how the "black boxes" really work. So
would I be better off pursuing that, or just continuing with courses like this
one?

Or maybe both at the same time (if that is even possible)?

I've also considered that I might need to take those subjects
(probability/stats/calc) as actual courses via community college or similar
(or online MOOCs).

Any thoughts or suggestions are welcome!

~~~
drageth
Ive watched the first two episodes of fast ai and one thing i can say is that
the whole approach of this course is the opposite of what the courses you
listed are structured like. Fast Ai jumps right into "how to use the
script/tools" while slowly teaching the concepts behind why they work in
comparison to the typical ML/AI course that involves complex learning calculus
and statistics before doing anything practical.

------
montenegrohugo
This. I can tell anybody thinking about taking this course, do it! The way
Jeremy Howard explains things is so down to earth and clear, you'll be
surprised at how easy all this ML/DL nonsense is :)

But seriously, there's a lot to learn here, and if you are looking to make
your entry into the hype field of the decade I would strongly suggest starting
here first

(Alternatively, for the more math inclined, perhaps take Andrew Ng Coursera
course first: [https://www.coursera.org/learn/machine-
learning](https://www.coursera.org/learn/machine-learning))

~~~
cr0sh
I second the Andrew Ng Coursera course; I took it in 2011 when it was called
"ML Class" (pre-Coursera) - yep, I was one of the beta testers!

------
leesec
Just wanted to say, your first course made me, someone with out a degree,
extremely interested in the field, and empowered me to pursue a career in the
industry. You making ML "uncool" really made me believe I could do it, when
previously I had been quite self-conscious about my lack of academics.

I did realize however I was still going to be quite a hard sell for companies
without the credentials. But that has lead me towards obtaining a B.S. in Data
Management/Analytics from WGU.

Thank you for your incredible content and I look forward to completing this
course. Your enthusiasm and teaching style make the class a real joy.

------
integricho
Could someone recommend in which order should the fast.ai courses be taken for
someone new to the AI field completely? Should "Introduction to Machine
Learning for Coders" be the first? or I could dive into "Practical Deep
Learning" right now, followed up later perhaps by "Cutting Edge Deep Learning
for Coders" ? What is the difference between the two deep learning courses
also?

~~~
leesec
If you want to get to building ML models right away, take Practical Deep
Learning for Coders. Cutting Edge Deep Learning is a part 2 to that class,
with more advanced topics.

The Intro to Machine Learning, from my understanding, is more of a normal
class that will take a deeper dive into some of the concepts and theories that
aren't necessary for practical implementation, but will be helpful for
understanding the whole process.

I personally recommend starting with this newest course.

------
byt143
Can we have one in Julia please?

[https://julialang.org/blog/2019/01/fluxdiffeq](https://julialang.org/blog/2019/01/fluxdiffeq)

~~~
mark_l_watson
+1 for the idea

I use Python and TensorFlow at work but for at home side projects I am very
much enjoying Julia and Flux. I haven’t tried FluxDiffEq yet.

Jeremy, please :-)

------
2bitencryption
I'm so excited to dive into this.

One question: is there a reason you opted for PyTorch over Keras? I had the
impression that Keras was the go-to for "easy as ABC" neural networking.

EDIT: bonus question! I'm really fascinated with gameplay agents like AlphaGo
and more recently AlphaStar. If I finish this course, will I have what I need
to start work on a toy version of some gameplaying agent? If not, could you
recommend where I could go next to start exploring that area?

~~~
adamnemecek
The one advantage of pytorch is that you can make your graphs dynamic.

~~~
0101111101
Tensorflow also created Eager - their dynamic environment

------
cs702
In my view, this is the _best_ , most accessible course on the planet for
learning the _how to_ of deep learning as quickly and efficiently as possible.

Highly recommended!

------
lettergram
Well, very similar to one I just released (even in name)[1]. Specifically, I
targeted taking one problem through several network iterations. I think it can
be overwhelming to go through all those different networks. Once you get the
basics new networks don’t aren’t really too difficult. I haven’t taken their
class before (and wouldn’t want to spend money). I think that’s a huge
barrier, there’s not much of a reason to require setup on sagemaker.

[1] [https://austingwalters.com/neural-networks-to-production-
fro...](https://austingwalters.com/neural-networks-to-production-from-an-
engineer/)

~~~
jph00
It's free and there are no ads, and you can run it for free on colab or GCP.

I have no idea about where your sagemaker comment comes from.

------
jfrankamp
Thanks for posting this, I have started working up some cloudformation scripts
with the goal of getting spot instances into play for GPU nodes on AWS @
[https://github.com/frankamp/fast-ai-aws-
advanced](https://github.com/frankamp/fast-ai-aws-advanced) not quite there
yet, but using layering to get separation of concerns means a user can swap
out for a bigger gpu temporarily without losing anything. Cheers.

------
sigfubar
Does your definition of "coders" include those of us with at most middle
school mathematics knowledge? I've tried some machine learning courses before,
but couldn't make sense of the vast majority of information.

~~~
jph00
Yes it'll be a bit more work for you, but it should be OK. I provide links to
more math background where required in the course.

~~~
mywrathacademia
Can the exercises in the fast.ai deep learning course be done in another
language other than python? And how much time in weeks should it take to
complete the course? If my laptop doesn't have a GPU, can I still do all
exercises?

~~~
cr0sh
> If my laptop doesn't have a GPU, can I still do all exercises?

To answer this, while technically you could do the exercises, it is possible
that it will take longer (perhaps much longer) to train models and whatnot
without a GPU. Where you'll really have a hard time will be anything for deep
learning and large neural networks. What might take a few seconds or minutes
to train on a GPU might take hours with just a CPU with a few cores.

But not to worry - in the course it explains that you can use a variety of
"cloud based" GPU systems that are fairly cheap to access (provided you have a
credit card). So if you can't get access to a GPU, there are other ways that
won't put you in the poorhouse (just make absolutely sure to completely
shutdown and/or delete your VM instances you spin up).

Also - while you could use another language, it is best to stick with Python
for now, because it has become almost the defacto language for ML and Deep
Learning, at least as far as for education and such. Mainly because it has
such great libraries for scientific computing available, plus it is easily
approachable. It also has a great interface to Tensorflow (and ultimately CUDA
and NVidia GPUs - also, get used to the fact that if you do anything DL
related, you will be using NVidia almost exclusively; while AMD and others
have their own DL hardware, it isn't nearly as well supported - maybe it will
be better supported in the future).

That said - Tensorflow does have decent support in a number of languages,
especially C++ - so if you are more familiar with that language, you'll be set
to do some amazing things once you complete the course.

But Python, again, is really where it's at. Tensorflow is a great library that
abstracts away a lot of the pain in dealing with CUDA. But even it has pain -
and there are other libraries layered over it that make things even easier to
work with (ie - Keras). Seriously; in the training I've had with DL (mainly
thru Udacity), we had to learn how to build a simplified DL "library" of
methods for building simple neural networks.

Once you learn about Tensorflow, you will have an epiphany of "OMG so much
easier!". Then - when you learn about Keras, it's yet another epiphany - it
makes Tensorflow look "difficult". Of course, it's best to learn in this
manner if you can, so that you understand how the lower levels work - instead
of everything being a black-box.

Of course, at some level there will be "black-boxes"; for instance, you may or
may not want to learn how Tensorflow interfaces with CUDA. Or how CUDA
interfaces with the GPU. Or certain other methods in Tensorflow or some other
library that perhaps wraps certain methods and functionality. Honestly, things
are much nicer today than they were even 5 years ago. You can decide just what
level of depth you want to explore down to.

~~~
synaesthesisx
Why not simply run on a cloud solution (like Google Colab), which can connect
to GPU (and even TPU!) instances and is incredibly easy to work with out of
the box?

~~~
mcintyre1994
Jeremy mentioned elsewhere in the thread that you can do this :)

------
whytaka
As someone just wrapping up the FastAI Intro to Machine Learning course and
moving on to DL, I'm really glad this came out now.

Thanks for all the knowledge, Jeremy!

------
oarabbus_
Does anyone else find it potentially problematic we have all these "get coding
fast - for coders!" type resources available, when most programmers don't have
the mathematical foundations to truly understand ML/AI/DL?

It's funny, programmers scoff at non-technical folk using drag-and-drop tools
to replicate certain programming functionality but I see the same thing
happening here.

------
carbocation
The course is amazing. I was frustrated in late 2018 when they promoted their
code to "v1" but kept iterating with breaking changes on a literal daily
basis. I have stopped using their library because of this+insufficient time.
However, their work is so amazing that I will pick it up again when I have
more time. Yes, it's so good that I will deal with daily breaking changes to
use it.

~~~
jph00
The changes were just during the development phase. Should be quite settled
now.

~~~
carbocation
Wonderful. I should add that you and Sebastian were always extremely
responsive and helpful troubleshooting issues with that popped up due to the
breaking changes.

------
bigmit37
Any recommendations for getting up to date with Pytorch? I could pick some
cheap books from packt or maybe good blogs ?

------
anurag
It's worth noting Jeremy and team are making these invaluable resources
available for free, even though a lot of folks would be extremely happy to pay
for them.

It's amazing to me that learning materials of such exceptional quality and
deep practical application are available without a paywall, especially when
everyone else seems to be cashing in on the AI hype with a myriad of paid
nanodegrees and such.

Thank you, Jeremy and team, for everything you're doing!

------
hx2a
I finished the 2018 edition of the class two weeks ago but I am happy to go
through the latest version now. The class was so thought provoking. I came up
with a lot of good ideas for my thesis project while watching the videos.

------
hkhanna
Hey Jeremy, just FYI this release seems to have broken the links to the
Introduction to Machine Learning for Coders course. For example, I can no
longer access it from the fast.ai front page (I get a 404 error.)

~~~
jph00
Many thanks for letting me know - I've fixed that link now. FYI the ML course
is now at: [http://course18.fast.ai/ml](http://course18.fast.ai/ml)

~~~
ganeshkrishnan
Thank you for the link. It's currently broken here :
[https://www.fast.ai/2017/09/08/introducing-pytorch-for-
fasta...](https://www.fast.ai/2017/09/08/introducing-pytorch-for-fastai/) at
this line:

"As we developed our second course, Cutting-Edge Deep Learning for Coders, we
started to hit the limits of the libraries "

------
baddash
Hey, this course looks really interesting! Although I'm hesitating to take it
because I'm not sure what the difference is between Machine Learning and Deep
Learning. Can anyone explain?

~~~
alexbeloi
Machine Learning is a catch-all term for optimizing statistical models with
data.

Simplest example that you are very likely already familiar with is that of a
'best fit line' to some xy scatter plot. This starts by making an assumption
(model choice) that the relationship between `x` and `y` is linear, e.g. `y=m
_x + b`, then you can use data (xy points) to figure out the most likely
values for `m` and `b`. You can then make predictions for new `x_new` values
by plugging them into your known line to get `y_new`.

Machine learning often manifests in a two step process: first feature
extraction, and then fitting features to a desired output. Deep learning
combines these as an end-to-end process to eliminate 'human in the loop'
problems that occur from feature extraction.

Example: you want to predict who should win a chess game in a given board
state

_ Feature extraction (what information you think matters): what pieces does
white have, what pieces does black have, is white in check, is black in check,
how many valid squares can white king move to, how many valid squares can
black king move, etc...

* Fitting: make an assumption about the relationship between features and outcome (model choice), fit model using data (features, outcome)

The Deepblue 2 model that played Kasparov used around 8000 features (not sure
if this is the feature vector size or # of features). As you can imagine,
feature extraction is highly dependent on expert knowledge of the problem and
will often fail to cover unknown situations/cases.

Deep learning models aim is to avoid limitations of expert knowledge by using
raw data (e.g. occupancy of each square on a chess board) and extract features
implicitly rather than relying on explicit human formulas. It has also opened
up new possibilities for areas where expert knowledge has made little progress
in the past (e.g. there is not much an expert can say about what pixel
features are might indicate a dog/cat is contained in an image).

~~~
Tarean
Though this end-to-end nature also makes it harder to spot biases and errors.
There is some work that first trains with deep learning and then tries to
infer a simpler model that is understandable for humans.

------
patricklovesoj
This is amazing! :) I took portions of it before but motivates me to go back
and work on it again. Is it just me or the overview notes from Lesson 6 & 7
are the same?

~~~
patricklovesoj
Also wanted to add that as someone who hasn't professionally coded or have a
degree in CS, the learning curve was a bit steep (which is why I took a break
from taking the course).

Are there any resources or good ways to get up to speed on getting sufficient
coding experience/knowledge so that I can really digest the content?

~~~
jph00
There's a lot of Python learning resources here:
[https://forums.fast.ai/t/recommended-python-learning-
resourc...](https://forums.fast.ai/t/recommended-python-learning-
resources/26888)

My main recommendation is to try to do as many things as you can with code,
instead of manually or with domain-specific tools. Even if it's slower at
first, solving your own problems with code is the best way to become a
productive coder.

------
knicholes
So we should watch these newer ones instead of the older ones?

~~~
jph00
For sure. It's a giant step over last year's course.

~~~
amch
Has the classic ML course been deprecated? I noticed that following link no
longer works:
[https://course.fast.ai/lessonsml1/lesson1.html](https://course.fast.ai/lessonsml1/lesson1.html)

EDIT: looks like they've been moved to the following link instead:
[http://course18.fast.ai/ml.html](http://course18.fast.ai/ml.html)

------
mandeepj
The lessons are fairly brief compared to Coursera\Udacity

~~~
jph00
About 14 hours of lessons per part for fast.ai. A lot of Udacity courses I've
seen are shorter than that. I certainly wouldn't call it "brief"!

------
mlboss
The thing that separates fastai course from other courses is the use of fastai
library (built on top of pytorch). fastai library makes it dead simple to
train neural network faster and with more accuracy.

Write a neural network is easy. But tweaking hyper parameters takes lot of
time and knowledge. fastai library implements cyclical learning rates which
varies the learning rate during training along with some other defaults for
other hyperparameters.

[https://github.com/fastai/fastai](https://github.com/fastai/fastai)

------
infinitone
Great course, looking forward to checking it out. I skimmed thru the overview
and dismayed to see nothing on object detection still :/

~~~
jph00
Object detection is in part 2. You can use the 2018 part 2 for now. We had
hoped to move it to part 1 this year, but couldn't figure out a way to make it
easy enough yet.

------
kevintb
fast.ai courses are EXCELLENT. I've read/watched many interviews with both
creators and have been even more impressed by their drive to bring deep
learning to everyone, not just the AI specialists.

------
DTE
Congrats, Jeremy and crew!

~~~
jph00
Many thanks. I'm really happy with how it turned out. I hope you all like it!
:)

~~~
input_device
I'm really interested in learning the math basics for deep learning. Is there
an online guide that you can point us to if we want to learn the basics of
calculus, linear algebra, probability theory? In other words is there a
"fast.ai" version of "math for deep learning" out there?

~~~
joshvm
Yes for linear algebra, though it's probably best suited to people with at
least some exposure to matrix/vector math (for example you've used SVD but
have no idea how or why it works): [https://www.fast.ai/2017/07/17/num-lin-
alg/](https://www.fast.ai/2017/07/17/num-lin-alg/)

For probability, machine learning is more about statistics (the two are
related, but courses explicitly about probability will cover different
things), so I would lean towards that. An Introduction to Statistical Learning
in R (ISLR) is a frequently recommended book. You can ignore the R and do the
exercises in Python.

If you actually want to learn about probability, you can look at MIT's course:
[https://ocw.mit.edu/resources/res-6-012-introduction-to-
prob...](https://ocw.mit.edu/resources/res-6-012-introduction-to-probability-
spring-2018/)

EDIT: If you've never been exposed to calculus, many people swear by Khan
Academy's videos.

~~~
input_device
This is great. Thanks you!

~~~
Canadauni
I would also recommend the MIT calculus courses on edX. It has been a great
refresher for me since the last I looked at calc was 8 years ago.

~~~
input_device
I'll check it out, thanks!

