For example, in the video I just watched he said "the natural way to compute the distance between two vectors is using cross entropy." And then he goes on to describe some unnatural features of cross entropy. The truly "natural" way to compute distances between vectors is the Euclidean distance, or at least any measure that has the properties of a metric.
I can understand this is a crash course and there isn't time to cover nuances, but I'd much rather the instructor say things like "one common/popular way to do X is..." rather than making blanket and misleading statements. Or else how can I trust his claims about deep learning?
For _probability_ vectors, there is a distance measure called cross-entropy. It's a standard error measure in classification problems. It has some important properties different from the Euclidean distance (especially for low probabilities) and there is information-theoretic interpretation.
More on it:
The link the OP posted is a (great) survey course dedicated to machine learning as a whole, which includes methods other than deep learning.
E.g. in the first question with code I had to reverse-engineer what they mean (including passing values in a format, which I consider non-standard (transpose!)). The first open-ended questions were entirely "ahh, you meant this aspect of the question".
Otherwise, the course (the general level, pace, overview) seems nice.
The IPython Notebook tasks (i.e. the core exercises) are nice.
On the other hand, some months ago I watched the ML course by Andrew Ng, and I still did not understand how to test a simple linear regression for myself, so I did not really understand it, and stopped watching the course.
It's more about understanding than "learning tools."
It comes with video, notes, all the math, cool ipython notebooks and will let you implement a deepish network from scratch. That includes doing backprop through the svn, softmax, max-pool, conv and ReLU layers.
After that you should be more than capable to build a 'real' net using your favourite lib (Tensorflow, theano etc).
In any case, I regret waiting so long for learning deep learning. (I thought that I needed to have many years of CUDA/C++ knowledge (I have none); but in fact, what I need to to know the chain rule, convolutions etc - things I've learnt long time ago.)
Udacity has a Linear Algebra review course, but I don't believe it is public for now. I had taken a linear algebra course before I took the GT ML class, but I wasn't a expert by any means. I don't believe you will need a deep understanding of linear algebra before taking this class. Singular value decomposition might come up. I think if you are familiar with everything in the following pdf you should be fine.
If you are motivated, you will do fine. Good luck!
One here: https://www.udacity.com/course/linear-algebra-refresher-cour...
And another here: https://www.udacity.com/course/intro-algebra-review--ma004
http://www.intmath.com/matrices-determinants/matrix-determin... - matrices
http://www.intmath.com/vectors/vectors-intro.php - vectors
Linear Algebra using Python:
https://www.coursera.org/course/matrix - Coursera: Coding the Matrix
http://codingthematrix.com/ - Website: Coding the Matrix
The simplest software for linear algebra would be GeoGebra, http://www.geogebra.org/. For instance, to enter a matrix just have the spreadsheet view open, enter the numbers, highlight the cells, then choose the option "Create Matrix". To enter a vector start writing "vec" in the input bar at the bottom and intellisense gives you the option to choose "Vector[<Start Point>, <End Point>]". Choose this. Fill it in, for example, "Vector[(-3, 4), (1, 2)]" (Hint: Use Tab to move between options in the input formula, here to move between "<Start Point>" and "<End Point>]".) Voila the vector is drawn! You can even draw a vector with just two clicks in Graphics view, if you first select the "Vectors" tab at the top (the symbol is a line with an arrowhead). The GeoGebra software is really incredible for learning/doing Linear Algebra, Calculus and Statistics. A real godsend.
Some visual matrix operations are here: http://setosa.io/ev/
If you want a beginner textbook, I recommend http://www.matrixanalysis.com/
Again, it is simple, so maybe you can even take course and look up Wikipedia when needed (but for me it is hard to guess you level, current knowledge, etc).
In any case, this ML course assumes some ML knowledge.
If the whole course is free are there more free courses on this site?
Thanks for the reply.
So free to take, money if you want degrees.
And if not natively (using docker/VMs), would they be able to use NVidia CUDA card on my system? And how much disk space would be needed.
This, I think, is more of a library demonstration than anything.
I am a data science freelancer and I mostly do projects for IT-dominated companies. First, I was surprised that such companies need some external help with relatively simple tasks; only later I discovered that top-notch performance in webdev (or even: algorithms) does not mean that someone is able to do simplest data analysis.
For data science / ML - I know a lot of openings in which they are looking for "data scientists", but what they mean is software engineers with at least a slight idea what is data analysis.
When it comes to deep learning in particular - I don't know.
I've been wanting to get into this field recently. Do you have more info about these openings, perhaps?
When 3 years ago I was looking for data science internships, most of interview were strictly in software engineering. (I got into a more data-analysis oriented.) Even when I applied to Google a year ago (and failed), all non-trivial questions where in software engineering (some with data-oriented paradigms, tough).
Look at https://medium.com/@rchang/my-two-year-journey-as-a-data-sci... - the taxonomy of "Type A Data Scientist" vs "Type B Data Scientist" is useful. You want to apply for the "B" or even - software engineer in a company which deals with data and is open to shifting roles.
Going back to the interviews: I see that the set of questions is entirely different. E.g. if the first question is "how to invert a binary table" or "how to test if a black-box number generator is fair". But sometimes it is not clear from the job opening.
If you are interested in my background: http://p.migdal.pl/2015/12/14/sci-to-data-sci.html
I personally took several ML courses from coursera/udacity/edX, and they helped me when I decided to move to another group that works on the ML-related projects.
Most other programmers wouldn't be able to contribute much value in a world that worked like this.
This perhaps mirrors how the chip market works, which similarly involves a limited number of researchers involved in advanced manufacturing techniques that are highly specialized and mostly a mystery to other people in the technology field.
I'm not sure that a deep learning course would be a good first course. But an intro course on linear regression and basic probability/statistics would be worth looking into.