Hacker News new | past | comments | ask | show | jobs | submit login
A Beginner’s Guide to Recurrent Networks and LSTMs (deeplearning4j.org)
211 points by vonnik on Sept 8, 2015 | hide | past | web | favorite | 10 comments

If this feels like jumping in the deep end, I highly recommend Stanford's online course on this subject:


I agree. Andrew Ng's course is a great place to start with ML. We've got a list of great online ML/DL courses here: http://deeplearning4j.org/deeplearningpapers.html

Machine Learning is in my bucket list of things to focus on at some point in my life, just like many of you.

I know it's a bit off topic, does anyone of you know any practical resource that teaches ML while creating some kind of robot with AI?

Thanks in advance.

To address what you and tgflynn are saying. Machine learning != robotics. Robotics is mostly probabilistic inference / search, control theory, etc. The Sebastian Thrun course actually looks great - for robotics. Even computer vision is mostly not related to machine learning. And when machine learning is used, it has usually been simpler, faster "dumber" algorithms.

Deep learning has only recently made advances in computer vision and robotics - it's still definitely an open area of research. Even in production it's only going to be a component of a larger system. I would bet the vast majority of the work in Google's self-driving car is related to probabilistic inference / mapping / search / feature-engineering, and mostly does not involve machine learning.

The future of both computer vision and robotics is in machine learning, that is where the state of the art will advance.

No autonomous vehicle that isn't integrally dependent on machine learning algorithms will ever hit the market, unless the entire road system is redesigned and human drivers are outlawed.

Maybe. Maybe not. Impossible to tell. People were saying this about SVMs and kernels 10 years ago (which, by the way, is why it's important to have a broad knowledge of probability and machine learning).

Sebastian Thrun teaches a course called AI for Robotics on Udacity: https://www.udacity.com/course/artificial-intelligence-for-r...

I took that class several years ago and it was definitely not about machine learning, mostly mapping/localization and route planning. It was a good class but the claim: "Learn how to program all the major systems of a robotic car" is vastly overblown. That would only be true if things like traffic sign recognition and pedestrian detection are not considered major systems of a robotic car.

I've been meaning to set aside more time to better understand ML as it applies to NLP. I've wanted for a while to understand how to, say, perform unsupervised training on my iMessage, Facebook and Slack history (which spans many years now) to tag messages, threads, and people with topics, sentiment, frequency, etc. I want to correlate that data with other time-series metrics that I either already have it can acquire easily enough, such as weather, medications taken, apps I have open on my desktop (which could signal context), to see if there are any interesting correlations. I have always thought that depression, anxiety, and AD/HD have had their own impacts on my life that are impossible to quantify without this sort of thing exercise, which could help me understand myself better, and help create a tool chain that would let others get the same kind of insight into their own data. I had always planned to make an open source kit of scripts or maybe Python notebooks out of the endeavor.

Things that I get stuck on, which then takes the wind out of my sails: tokenization (specifically handling the Unicode Emoji entities and ascribing meaning to them - Do I use them as tags/signals, or replace with synonyms?), lemmatizing (do I spend time going down the rabbit hole of simplifying all lines of chat to their most basic words?), grouping likes of dialog (if my reply was within ten minutes, consider it part of a "conversation" object), and how best to time stamp things (everything is individually stamped, but for some correlations, the time of day is the important bucket - for others, it may be the calendar day/season/busy work day).

It's such a huge domain, I keep spinning my wheels trying to feel like I'm going down a path that will lead to some form of success, no matter how small.

Another topic I've tried getting into is using ML to process my Hearthstone logs (live, not historical) to try my own approach on an unreleased project I recently read about that sought to predict opponent's cards. My thought was to create a series of dicts from popular "net decking" sites and compute cosine similarity between the cards an opponent has already played - the other project used game histories to predict the "next card", and I was seeking to predict which archetype my opponent is likely playing, since my own domain knowledge would figure out their likely "signature moves" once I had that clue. I'd maybe expand on it to predict how likely it is the opponent can go lethal on their next turn, given the cards they likely have and the ones on the board. With that topic, I've been trying to figure out state machines and various data structures in Python. Computing similarity I've figured out, with Counters seems to work, but the mechanics of doing so against potentially hundreds of "net decks" is challenging. Is a comprehension the way to go? A matrix function? I have no idea.

I guess I'm making this big dump of my own thoughts to see if anyone has any pointers, guidance, example projects, or general knowledge to share or direct me to that could help me figure this stuff out, because I'm excited to learn and I learn best when I have a pet project to which I can apply my newfound knowledge.

Anyone in the same boat? Or have anything to suggest?

You've got the data, so that's actually the hard part done. Now just pick some method and don't worry too much about the details. Go with the easiest route first, for example: each emoji is a word, no need to treat a smileyface differently in your model from a word like "happy", _until you see how the representation actually matters_, only then will you know what the right way is. If you pick a complicated method first, you won't know if it was worth it. If you pick a simple method first, you'll know if it doesn't work, but you won't have wasted time.

Re:lemmatisation, either grab an existing machine-readable dictionary [1] if you just want all lemmas with their ambiguity, or just do simple PoS-tagging with something like https://honnibal.wordpress.com/2013/09/11/a-good-part-of-spe... (I love that post for its no-nonsense approach)

[1] http://wiki.apertium.org/wiki/Using_an_lttoolbox_dictionary has some (and will tokenise for while analysing).

Guidelines | FAQ | Support | API | Security | Lists | Bookmarklet | Legal | Apply to YC | Contact