This kind of automated or rule-based analysis is not sufficiently smart to use as any kind of moderation tool. And it won't be until it can interpret semantic content as well as recognize patterns.
Consider the following exchange:
Person 1: Hello! I just wanted to chime in and make you aware of the fact that according to some very cool research <link>, people with sub-equatorial ancestry exhibit markedly lower test scores, in fact very similar to many of the great apes! Your mileage may vary, but in my humble opinion I would never hire someone or work with someone from that demographic.
Person 2: Shut up and go away, you racist prick.
This pattern plays out on twitter fairly regularly, and it's usually Person 2 who gets moderated, despite the fact that the content of their message is actually more appropriate and a net positive for the community (given the context.) Meanwhile, as long as it's polite, actual hate speech can make it through most of these filters.
I've heard this referred to as the "polite Nazi" problem, and it's quite real.
It sounds like what you want is not sentiment analysis but rather some sort of political-correctness analysis, which rather than identifying aggressive language identifies "wrongthink".
I suspect you're the sort of person who would have torn down the signs 4channers put up last year saying "It's OK to be white", because despite the obviously anodyne and correct content of the message, you would have interpreted it as "polite Nazi".
I thought his point was interesting and practical, and that he used a non-controversial example. The problem he's describing is probably impossible to solve because real examples aren't so obvious, but it seems like you jumped to the conclusion that he would be willing to accept the downside of such a system in practice (punishing people who are in fact acting in good faith, and not just hiding hatred behind apparently rational wording), and you seem to be personally attacking him. However, I can only say that by assuming from the way you wrote it that you believe the 4channers you referenced were by-and-large acting in good faith. You should be more clear if that's what you're saying, since that would be a surprising conclusion to me. From everything I saw and heard, that movement seemed to be dominated by the usual hatred, and "It's OK to be white", while ostensibly an addressal of very real problems, was really just a slogan that's unassailable outside of any other context and therefore easy to hide behind.
Implying that someone holds a racist ideology: unacceptable ad-homenim.
Actually holding a racist ideology: just fine, or at least we can't stop it without censoring free thought.
Also I don't know why you keep going on about 4chan. Nobody else is talking about that example (mostly because it was an obvious bad-faith troll and isn't worth talking about.)
You got it: that's exactly correct. Implying someone is racist when they've not said anything racist is unacceptable ad-hominem. I think any civilised person can agree on that.
And the second part is also correct. You can't stop people holding "racist ideologies" without censoring free thought, which short of Black Mirror style sci-fi machines is impossible. But even if such technology did exist you can't even define what "racist ideology" actually means with any level of consensus of precision. You're so far from even being able to think about doing anything here, that even attempting is worthless and will cause vast collateral damage. Therefore the correct solution is to do nothing.
This is fascinating. What other things fall into this intriguing category, where doing a Very Bad Thing is ok, but observing that someone seems to be in favor of a Very Bad Thing is downright terrible?
And how is a society to defend itself against Very Bad Things, when it's worse to talk about than to actually do?
The only reason I can think of why someone might hold such a logically contorted position, actually, is if they actually think Very Bad Thing isn't that bad a thing at all, but won't come right out and say so.
Yes, I do in fact think that blatant racism is "wrongthink" and has no place in civilized discourse. I definitely think it is a more serious issue than using curse words or expressing negative sentiments.
Also, if you think that that 4chan stunt was "anodyne" then it seems you have very little concept of how language or speech acts work, and are unqualified to weigh in on a discussion of them.
Consider the following exchange:
Person 1: Hello! I just wanted to chime in and make you aware of the fact that according to some very cool research <link>, people with sub-equatorial ancestry exhibit markedly lower test scores, in fact very similar to many of the great apes! Your mileage may vary, but in my humble opinion I would never hire someone or work with someone from that demographic.
Person 2: Shut up and go away, you racist prick.
This pattern plays out on twitter fairly regularly, and it's usually Person 2 who gets moderated, despite the fact that the content of their message is actually more appropriate and a net positive for the community (given the context.) Meanwhile, as long as it's polite, actual hate speech can make it through most of these filters.
I've heard this referred to as the "polite Nazi" problem, and it's quite real.