Here's the setup guide: https://course.fast.ai/start_colab.html
Although it is recommended to dedicate ~10 hours per lesson, it is very possible to go through each video, then circle back to focus on one of the concepts/problems presented.
I recommend giving the first lesson a try. It is very rewarding feeling like you can actually do or build something at the end of each lesson.
What I'd like to know, as someone who has completed (in the past) the following MOOCs:
1. ML Class (plus about half of AI Class) - 2011
2. Udacity's CS373 Course - 2012
3. Udacity's Self-Driving Car Engineer Nanodegree - 2016/17
...and gained a lot of knowledge about ML and DL thru them - would this fast.ai course be worthwhile to pursue as an addition to my prior education on the subjects? Would I find it just as challenging (from a cursory glance it looks to be a "yes", but I wanted another opinion)?
I'm open to suggestions from everyone here, not just jph00/fast.ai ...
Also - and this I would also like opinions on - would my time be better spent pursuing other areas I know I am not proficient in - mainly learning about probability/stats, as well as basics in calculus...
For instance, in the prior courses, there was always mention of - for say backprop - stuff about the "chain rule" and other "rules" for calculus, and I feel I need to understand that more. That is, I am wanting to understand things in ML and DL at a lower level; how the "black boxes" really work. So would I be better off pursuing that, or just continuing with courses like this one?
Or maybe both at the same time (if that is even possible)?
I've also considered that I might need to take those subjects (probability/stats/calc) as actual courses via community college or similar (or online MOOCs).
Any thoughts or suggestions are welcome!
But seriously, there's a lot to learn here, and if you are looking to make your entry into the hype field of the decade I would strongly suggest starting here first
(Alternatively, for the more math inclined, perhaps take Andrew Ng Coursera course first: https://www.coursera.org/learn/machine-learning)
I did realize however I was still going to be quite a hard sell for companies without the credentials. But that has lead me towards obtaining a B.S. in Data Management/Analytics from WGU.
Thank you for your incredible content and I look forward to completing this course. Your enthusiasm and teaching style make the class a real joy.
I use Python and TensorFlow at work but for at home side projects I am very much enjoying Julia and Flux. I haven’t tried FluxDiffEq yet.
Jeremy, please :-)
The Intro to Machine Learning, from my understanding, is more of a normal class that will take a deeper dive into some of the concepts and theories that aren't necessary for practical implementation, but will be helpful for understanding the whole process.
I personally recommend starting with this newest course.
One question: is there a reason you opted for PyTorch over Keras? I had the impression that Keras was the go-to for "easy as ABC" neural networking.
EDIT: bonus question! I'm really fascinated with gameplay agents like AlphaGo and more recently AlphaStar. If I finish this course, will I have what I need to start work on a toy version of some gameplaying agent? If not, could you recommend where I could go next to start exploring that area?
Also some info in the fastai release post: https://www.fast.ai/2018/10/02/fastai-ai/
I have no idea about where your sagemaker comment comes from.
It's funny, programmers scoff at non-technical folk using drag-and-drop tools to replicate certain programming functionality but I see the same thing happening here.
The live course was held over a time of 7 weeks. It should take around 70h to finish it in total but I really depends on how deep you want to dive in and how easy it is for you to understand.
To answer this, while technically you could do the exercises, it is possible that it will take longer (perhaps much longer) to train models and whatnot without a GPU. Where you'll really have a hard time will be anything for deep learning and large neural networks. What might take a few seconds or minutes to train on a GPU might take hours with just a CPU with a few cores.
But not to worry - in the course it explains that you can use a variety of "cloud based" GPU systems that are fairly cheap to access (provided you have a credit card). So if you can't get access to a GPU, there are other ways that won't put you in the poorhouse (just make absolutely sure to completely shutdown and/or delete your VM instances you spin up).
Also - while you could use another language, it is best to stick with Python for now, because it has become almost the defacto language for ML and Deep Learning, at least as far as for education and such. Mainly because it has such great libraries for scientific computing available, plus it is easily approachable. It also has a great interface to Tensorflow (and ultimately CUDA and NVidia GPUs - also, get used to the fact that if you do anything DL related, you will be using NVidia almost exclusively; while AMD and others have their own DL hardware, it isn't nearly as well supported - maybe it will be better supported in the future).
That said - Tensorflow does have decent support in a number of languages, especially C++ - so if you are more familiar with that language, you'll be set to do some amazing things once you complete the course.
But Python, again, is really where it's at. Tensorflow is a great library that abstracts away a lot of the pain in dealing with CUDA. But even it has pain - and there are other libraries layered over it that make things even easier to work with (ie - Keras). Seriously; in the training I've had with DL (mainly thru Udacity), we had to learn how to build a simplified DL "library" of methods for building simple neural networks.
Once you learn about Tensorflow, you will have an epiphany of "OMG so much easier!". Then - when you learn about Keras, it's yet another epiphany - it makes Tensorflow look "difficult". Of course, it's best to learn in this manner if you can, so that you understand how the lower levels work - instead of everything being a black-box.
Of course, at some level there will be "black-boxes"; for instance, you may or may not want to learn how Tensorflow interfaces with CUDA. Or how CUDA interfaces with the GPU. Or certain other methods in Tensorflow or some other library that perhaps wraps certain methods and functionality. Honestly, things are much nicer today than they were even 5 years ago. You can decide just what level of depth you want to explore down to.
Thanks for all the knowledge, Jeremy!
It's amazing to me that learning materials of such exceptional quality and deep practical application are available without a paywall, especially when everyone else seems to be cashing in on the AI hype with a myriad of paid nanodegrees and such.
Thank you, Jeremy and team, for everything you're doing!
"As we developed our second course, Cutting-Edge Deep Learning for Coders, we started to hit the limits of the libraries "
Simplest example that you are very likely already familiar with is that of a 'best fit line' to some xy scatter plot. This starts by making an assumption (model choice) that the relationship between `x` and `y` is linear, e.g. `y=mx + b`, then you can use data (xy points) to figure out the most likely values for `m` and `b`. You can then make predictions for new `x_new` values by plugging them into your known line to get `y_new`.
Machine learning often manifests in a two step process: first feature extraction, and then fitting features to a desired output. Deep learning combines these as an end-to-end process to eliminate 'human in the loop' problems that occur from feature extraction.
Example: you want to predict who should win a chess game in a given board state
Feature extraction (what information you think matters): what pieces does white have, what pieces does black have, is white in check, is black in check, how many valid squares can white king move to, how many valid squares can black king move, etc...
* Fitting: make an assumption about the relationship between features and outcome (model choice), fit model using data (features, outcome)
The Deepblue 2 model that played Kasparov used around 8000 features (not sure if this is the feature vector size or # of features). As you can imagine, feature extraction is highly dependent on expert knowledge of the problem and will often fail to cover unknown situations/cases.
Deep learning models aim is to avoid limitations of expert knowledge by using raw data (e.g. occupancy of each square on a chess board) and extract features implicitly rather than relying on explicit human formulas. It has also opened up new possibilities for areas where expert knowledge has made little progress in the past (e.g. there is not much an expert can say about what pixel features are might indicate a dog/cat is contained in an image).
Are there any resources or good ways to get up to speed on getting sufficient coding experience/knowledge so that I can really digest the content?
My main recommendation is to try to do as many things as you can with code, instead of manually or with domain-specific tools. Even if it's slower at first, solving your own problems with code is the best way to become a productive coder.
EDIT: looks like they've been moved to the following link instead: http://course18.fast.ai/ml.html
Jeremy, I know you're a fellow fan of J -- is it possible to do modern ML with J? It seems like numpy is just a worse version of the array paradigm from J/APL, but I'm guessing all the specialized ML packages are the real value of python here? Any idea of anyone has done similar work in J?
Write a neural network is easy. But tweaking hyper parameters takes lot of time and knowledge. fastai library implements cyclical learning rates which varies the learning rate during training along with some other defaults for other hyperparameters.
For probability, machine learning is more about statistics (the two are related, but courses explicitly about probability will cover different things), so I would lean towards that. An Introduction to Statistical Learning in R (ISLR) is a frequently recommended book. You can ignore the R and do the exercises in Python.
If you actually want to learn about probability, you can look at MIT's course: https://ocw.mit.edu/resources/res-6-012-introduction-to-prob...
EDIT: If you've never been exposed to calculus, many people swear by Khan Academy's videos.