I implemented a slight variation on this CNN using Keras and TensorFlow for the third project in term 1 of Udacity's Self-Driving Car Engineer nanodegree course (not special in that regard - it was a commonly used implementation, as it works). Give it a shot yourself - take this paper, install TensorFlow, Keras, and Python, download a copy of Udacity's Unity3D car simulator (it was recently released on GitHub) - and have a shot at it!
Note: For training purposes, I highly recommend building a training/validation set using a steering wheel controller, and you'll want a labeled set of about 40K samples (though I have heard you can get by with much fewer, even unaugmented - my sample set actually used augmentation of about 8k real samples to boost it up to around 40k). You'll also want to use GPU and/or a generator or some other batch processing for training (otherwise, you'll run out of memory post-haste).
> More work is needed to improve the robustness of the network, to find methods to verify the robustness, and to improve visualization of the network-internal processing steps.
But it doesn't hint at how this would be approached.
Also, how they arrived at the particular network topology seems sort of a mystery.
> Machine learning is the science of credit assignment. The machine learning community itself profits from proper credit assignment to its members. The inventor of an important method should get credit for inventing it. She may not always be the one who popularizes it. Then the popularizer should get credit for popularizing it (but not for inventing it). Relatively young research areas such as machine learning should adopt the honor code of mature fields such as mathematics: if you have a new theorem, but use a proof technique similar to somebody else's, you must make this very clear. If you "re-invent" something that was already known, and only later become aware of this, you must at least make it clear later.
I feel like I should find a lot more info on this in the sentiment analysis literature but I don't really.
However, often there are no real shortcuts; in many projects the resource annotation takes much more work and more people than everything else together, it's not uncommon to see multiple man-years spent to do that properly.
What you say about the high level is just about all that can be said in general, everything else will depend on your particular problem. After you've fixed the bugs in your process, interannotator agreement is not really a description of your annotators but a measure of how subjective/objective your task is - and you can't really change that without meaningful changes to how exactly you define your task. Some tasks are well suited for crowdsourcing, and some need dedicated experts. Some annotation tasks are straightforward and annotation guidelines fit on one page; for others the annotation guidelines are a literal book, and one that needs revisions as a few years after you figure out that you need changes. It depends. Shallow sentiment analysis is generally on the trivial side of annotation (but highly subjective), but you can go far enough the rabbit hole to drag the whole surrounding issues of intent, degree of belief, degree of certainty, etc - then you hit the full complexity of deep semantic annotation.
Perhaps you just need to find the people who did the latest more interesting datasets in your domain and ask them directly. I don't handle sentiment, but http://alt.qcri.org/semeval2017/task5/ is one group of people that seems to do it seriously.
In a nutshell, its a deep learning model that given a word, predicts the other words around it, or alternatively, given some words in a sentence predicts the missing word. The idea is that similar words end up being assigned to similar vectors, all without knowing a ground truth.
Now that won't exactly answer your question, but then you can keep a couple of words related to your sentiment in a list and compare the words in the tweet to that list. If they are similar enough, you can write a rule to mark that tweet as matching your sentiment.
I've been reading a number of this papers but it's really tough to understand the nitty gritties of it.
I find it kind of hard to relate to for that reason — how do I know (besides asking my prof., whose PhD advisor was Hinton, or doing my Googling) what ideas ended up "sticking"? What areas of future research went nowhere vs spawned whole new subfields?
Is there a more modern textbook of the sort I could cross-reference?
edit: here's a link to the course website: http://www.cnbc.cmu.edu/~plaut/IntroPDP/
There's also a lot of resurrection that goes on with weird ideas: FastFood cites a paper that cites a paper that cites Lecun's brain damage paper, for example
The simulator: http://grey.colorado.edu/emergent
The textbook: https://grey.colorado.edu/CompCogNeuro
That's what I gathered from a brief look at things, which is why I find it very interesting - both from a historical context, as well as the idea that maybe there are some nuggets of (possibly overlooked?) wisdom hidden within, that might be useful in today's world.
Then again, I am but an interested (and very amateur) hobbyist in this whole ML space, and not likely to come up with any breakthroughs - but you never know I suppose!
Smolensky's hamonium (rbm)
the LeNet papers
Jaeger's ESN nature paper
Forget gate paper, whose author I forget (ironically)
 From one of my mentors: http://www.unofficialgoogledatascience.com/2016/10/practical...
Issue System log
issue_1 corresponding system log
issue_2 corresponding system log
issue_3 corresponding system log
issue_4 corresponding system log
issue_5 corresponding system log
Then again, if the log entry has a somewhat set length (or can be truncated), you could feed that in as the input to a CNN (one input node/neuron per character), and the output layer could consist of the issue labels. I'm not sure what if anything that could net you; perhaps an unknown log could be input on the trained network, and it could classify it to an existing issue?
I did some work using stack traces to predict duplicate bug reports, so I'm somewhat familiar with a similar problem.
Even a simple algorithm would be effective: the number of citations for each paper decayed by the age of the paper in years.
I want to buy a Raspberry Pi Zero, put it in a nice case, add to push buttons and turn it into a car music player (hook it into the USB charger and 3.5mm jack in my car). The two buttons will be "like" and "skip & dislike". I'll fill it with my music collection, write a python script that just finds a song, plays it, and waits for button clicks.
I want the "like" button to be positive reinforcement and the "skip & dislike" to be negative reinforcement.
Could someone point me in the right direction?
The way the system is evaluated is by building a model that will predict what rating the user will give a song even though the user has not rated it yet. Then the difference in predicted and actual rating will be computed as the testing error in the model. Some basic techniques for building a model are regression and matrix factorization using SVD (singular value decomposition).
Your usecase might be slightly different from this problem, because you wouldn't have to predict the rating other users give a song (only for yourself) and you want your model to change on the fly given a skip and dislike. A simple, but possibly effective solution might be to search the music dataset that contains the listening history of 1M users to find songs you haven't rated before and download it to listen.
My initial thought was just to keep a full list of songs and their liked, disliked, and ambivilant states. Make some function to score a song based on their distance & # likes from the played vector.
Doing this all by hand would be a LOT of work. Are there any frameworks that do this? Is there any way to interface with them that are strait forward? Do I need to get a PhD before this stuff will come easily to me?
So many questions, so little time.
The purpose of ML is that you can tell a machine to go through more data than any human can. By expanding the corpus of songs from songs you've heard of to all songs in the big dataset, you will be able to find songs you haven't heard based on others who liked the songs you liked and disliked the songs you disliked.
edit: Maybe I misunderstood the problem you're working on. I thought you wanted to get recommendations of songs to listen to, but now I see that you have a lot of songs you like already, so you just want a song recommended from that list based on some factors like mood, time of day, etc.
This project also has constraints. It needs to fit in a Raspberry Pi Zero and on a microsd card with no internet connection running off the power from a USB power supply in a car.
It needs to be fast-ish (I can run all of this math at the same time I'm playing the ~3 minute song) and it needs to be small. If I've got 10-20GB of music then songs+os+my data<=32GB. These days I think you can get a Rasbian install ~1-2GB so that leaves us with ~10GB of data if I don't expand my music collection ever because after I get it set up I can't be asked to update it ever again.
So yes for this I just want to start with my music folder. A folder of audio files. No lyrics, no nothing. Just things that can be infered from my like and dislike.
- Prefer songs from most liked genera/artist/album
- Favor liked song/genera/artist/album & avoid consistantly disliked song/genera/artist/album.
- If a specific song/genera/artist/album is liked more at a specific time, play it more often at that time.
- If a song is being played and specific song/genera/artist/album is always liked when it follows this song then favor those.
- If you are playing a song and there is a song that is always liked when it follows this song favor that song when choosing the next song.
- Corelate GPS location with song choices
- Corelate weather/humidity/tempature/month with song choices
- Do NLP sentiment analysis to score all of the songs on an emotional-tone scale to help group them well.
- Do some DSP on all of the audio to generate a intro and outro beat/rythm profile and attempt to best match the song that will lead to a good enterance/exit.
If you can sim a set of boxes, you can learn whats inside them.