I know there is a ton results for a google search, which is precisely my problem. SICP is a 4 letter word that everyone would understand here. Are there any such reputed courses for machine learning that jumps to your mind? Any help is much appreciated.
A great strategy that I've learned a lot from is to use a database like citeseer or ISI to find the author's of the most cited papers for a topic and then find their lectures on videolectures.net
Other's already mentioned Jordan, Bishop and Friedman -
these are all great
I really liked Thrun, Burgard, and Fox's text Probabilistic Robotics - they use a lot of ML like algorithms under very tough constraints (limited CPU and real-time performance)
I have lecture videos from this class. The man probably wouldn't like it if I posted them. If you really want them, i.e. will watch them, you can snail mail me 5GB in flash and I'll mail them back to you.
Chris Bishop's Pattern Recognition and Machine Learning is a good recent book aimed at 'advanced undergraduates'. It seems to be 'the book' in ML right now. Norvig and Russel's AIMA is a good general AI resource. The twice annual machine learning summer schools get videoed and put up on videolectures.net. Really good introductory lectures from the basics on up to various application areas. Even better if you can go to one and pick the presenters brains for a couple weeks, they are intended for both new graduate students and interested people from industry. After that there are specialized books in the (many) subfields.
I second videolectures as an excellent source. Just listen to the ones you are interested in! One tip I'd offer is to make sure you understand the math of each lecture before moving on. Skip over the maths enough and you'll find you haven't truly learned much.
However, in terms of books I would add Elements of Statistical Learning (Hastie, Tibshirani, and Friedman). It is an excellent text that covers a lot of ground. The down side of this of course is that it is written at the graduate level, so be prepared.
Read Norvig's introductory text, but don't feel you have to know it all - use it as a way to pick something you really are interested in (genetic algorithms are always popular) and read academic literature on the topic (citeseer is a good resource here). Use Wikipedia, Scholarpedia. Get an intuitive sense of dynamical systems, problem spaces and landscapes, and chaos theory. Get a grip on combinatorial explosion, and just how much it can suck for optimization problems. Learn theory of computation, and glance (more than once) at symbolic logic.
That will set you on your way! Good luck, it is fascinating stuff.
