What is the state of the art in sentiment analysis? I tried their demo page, and it seems completely useless (could not interpret the query "Not the greatest movie"). Are the algorithms used on Wall st. for predicting market behaviour based on twitter stream significantly more advanced?
imho that's a cool integration for Joe Average and some use cases do look useful, although Semantria has a standalone Excel plug-in geared towards researchers where you don't have to deal with any coding. For most needs that'd be the best use case, for some I'd use the API. It's interesting that this service uses Lexalytics' Salience Core that is one of the top-3 text analytics engines out there, much better (and, of course, way more expensive) than AlchemyAPI, however, it's PAYG vs. hosted.
Huh, our testing shows Alchemy beating out Salience in sentiment, named entities, and several other areas. I don't even see stuff like relations extraction in Salience. wht are you basing your eval on?
but this seems to have far more interesting and relevant emotional data. I wonder how well these systems can deal with convoluted / mismatched emotions?
We were really blown away by some of the use cases. One of my favorites: analyze support tickets in HelpScout, Zendesk or Desk.com and tag them based on sentiment (positive vs. negative).
The incredible thing is there is no code involved. It is crazy cool.
Sentiment mining is a hot topic, with many possible uses, but coders can usually do something decent without any crazy magic.
The problems start when any grammar processing happens - because algorithms are then dependant of the language
(What about this service? I didn't find many details about it.)
Meanwhile, if you stay at the dictionary level, you can get decent results - that's what things such as the General Inquirer let you do - and you won't need for an API to do basic stemming of words and summing up the sentiment values from a database - see http://www.wjh.harvard.edu/~inquirer/
I've played with it - that's more or less a 1 day project if you use the existing database.
Bonus points if you add simple things - such as replacing the stemming algorithm by a Levenshtein distance or a phonex-style algorithm (typos, etc), or use inverse term frequency to mitigate the influence of "very sentimental" words that are falling into common use, i.e. whose current sentimental ponderation differs from the general inquirer ponderation.
I once played with sentiment analysis of mailing lists, to automatically get red flags when sh*t happens :-)
That is really interesting (great link too!). As coders ourselves we've had the same experience: spinning up something simple is usually pretty straightforward. What we are a little more curious about is the ability to bring simple sentiment mining to non-coders (IE: your support or social media guy) via off the shelf tools.
Based on my experience the sensibility is good (ie it often gives a red flag when something happens) but the specificity is poor (ie there are many false positives).
My theory at the time was that some kind of semantic sliding was happening and led "sentiment loaded" words into common use, while removing their specific sentiment load (#)- which is why I tried TF-IDF, but even then the results where not that good and I was too lazy to dig into grammar analysis to correct for the other possible bias (negations and other complexities such as irony becoming more frequent)
(Are you part of the team behind this product? Would you have some comments about sensibility and specificity of your results?)
(#) : ex : teen talk ("OMG it's so crazy and fabulous!") is full of words that may have indicated a strong sentiment, but which that are devoid of it in the given "teen talk" context.
I am not a part of the Semantria team, instead I'm a Zapier cofounder (we offer some simple piping or glue that makes it easy to move data between APIs without code). Happy to put you in touch with the Semantria guys, they are very knowledgable.
