One of our great facets as humans is the ability to sort through information and come to opinions based on it.
We come pre-equipped with our own API for critical reasoning, honed by thousands of years of evolution, and with a training set of everything we've ever done and seen.
Why would I want to use a copy of that, that at best will only replicate human biases?
And that risks sidelining perfectly valid viewpoints? Does something have to be expressed in college-educated grammar to even be read with this in place? What are the consequences of that?
My first instinct, and I hope yours, was to push back the slider.
EDIT: I pasted the first paragraph of the Google diversity memo into it, and it got a "10% likely to be perceived as toxic" score. Make of that what you will.
It's not new: https://news.ycombinator.com/item?id=13713443
We clearly unambiguously don't. People suck at many critical reasoning tasks. Look how poor people are at assessing risk for the most obvious example.
> Why would I want to use a copy of that, that at best will only replicate human biases?
Scale. Who has time to read thousands of trollish youtube comments to find the handful of insightful gems?
A similar upvoting mechanism is the reason why're you're reading this story as opposed to others.
It seems like this is more of a "word filter" than anything. I mean, sure, it's pretty obvious a comment will be toxic if it contains "asshole" in it. Here are some examples where the labeling fails. Some of these I got from real comments and articles on CNN or Breitbart.
# Toxicity too low
34% toxic: Obama was born in Africa and he should move back there.
40% toxic: George Soros and his Jewish sympathizers want to destabilize our government.
25% toxic: Breitbart is run by jews - as is all mass media. As the saying goes, "the best way to control the opposition is to lead it".
47% toxic: Why do you expect anyone to take you seriously when you don't take yourself seriously.
# Toxicity too high
51% toxic: Ivanka is arguably the most powerful Jewish woman in America today.
51% toxic: US Muslims and Jews are strengthening bonds amid acts of bigotry.
48% toxic: The problem of income inequality is not between Black America and White America but between black haves and black have-nots.
87% toxic: I lost my grip and fell on my ass. Not a great bike.
This entire paragraph is classified as 68% likely to be perceived as toxic only because it contains the word "idiots". If you replace it with another word like "dragons", it's down to 13%:
Hi, I don't completely agree with your point about
idiots being commonplace. First of all, that is a
derogatory word with no specific meaning. If you have
a problem with their opinion, it's more constructive to
point out ways in which you disagree, so we can have a
more civil discourse.
As for the too-high ones, yeah; that bike review [?] is rated hilariously poorly.
Frankly, this API can not (should not?) be expected to determine the truthfulness of a statement, nor how offended you're likely to be by somebody's honest opinion.
That sounds reasonable, but then why is the statement "Ivanka is arguably the most powerful Jewish woman in America today" considered doubly more toxic (25% vs 51%)?
> 47% toxic: Why do you expect anyone to take you seriously when you don't take yourself seriously.
I would have put that under "toxicity too high"; it seems like a conventional, vanilla exhortation to be more self-confident?
I'd only support this tech as a filter for human moderators and not as an automated system.
This API may represent an attempted technological solution to a social problem. Those have a very poor track record.
As a filter for human mods it might be an interesting incremental improvement. But google has a need to do this at a scale where I can't see it staying there in terms of the concept of deployment.
> Your a socialist snowflake!
Is ranked as less toxic than:
> Please put yourself in the shoes of women, minorities, and LGBT people
Extracted from the "US Election" demo on the page.
Edit: for the sake of clarity, when taken to the textbox below, snowflake gets 58% toxicity, and minorities gets 29%. Still, when using the slider, snowflake appears before, with a circle (which would correspond to an ok comment), and minorities much later on, with a square (which would be a dubious comment).
Edit 2: testing on the textbox, commas and periods get you a lower score (less toxic), while exclamation marks, lack of punctuation, and "bad writing" (not just grammar, but a general bad style you could say) get you a higher score.
Edit 3: the final gem I'll take for my professors at University: FAT is an old filesystem! gets 90% :P
Secret -- caps seem to also rise the score.
Actualy... fat 88%, fat_ 41%... now that's it, I got tired of this.
Maybe it tripped over the wrong usage of "your"? :)
I pasted this comment from the recent diversity manifesto:
> I’m simply stating that the distribution of preferences and abilities of men and women differ in part due to biological causes and that these differences may explain why we don’t see equal representation of women in tech and leadership.
The page says it's 2% toxic. What does that mean? 2% of the population would find it toxic? There is a 2% chance someone would find it toxic? The API is 2% confident that it is toxic? And more importantly, toxic in the sense that it is verbal harassment? Or just plain illogical? Or logically sound but with an absurd premise?
I suspect that it is only able to detect more emotional comments, but will fail to detect utterly unfounded, totally disproved arguments that are communicated under the veil of reason.
It came from this thread where there are complaints that the Perspective API may not outperform a random number generator.
Try startpage or duckduckgo for search. (The prior is essentially a proxy for Google).
Fastmail or protonmail to get away from GMail. One of the hard things about avoiding Google in email is that even if you switch, others you communicate are likely still there, so Google's still getting that information.
Conversely, when I use google, I frequently miss !w (search on Wikipedia), !i (search images), !a (amazon) and other "bang" shortcuts.
E.g., in Chrome:
Settings > manage search engines > find Wikipedia > change the shortcut to "w."
That being said, I think there's some huge potential to use AI/ML in this way to improve our ability to communicate less toxicly. I've seen some research from Google investigating biases in AI/ML outcomes, so I'm excited to see what develops.
The terrifying thing is this is likely to become wedged into various internet sites and services where users who don't align with Google's particular biases are effectively forced to conform to them. They are really pressing this sort of power lately and I'm not having it.
I ran both our comments through it:
11% toxic for you, 25% for mine. Both contain some trigger terms.
I wish they train their models against non political data to avoid potential partisan bias. The current approach is a bit ridiculous.
> asking people
Gotta wonder: which people?
The examples are good though, I just hope the general results are consistent with that quality level.
For example, the API rates this comment as 21% likely to be perceived as "toxic". The use of quotes around the word "toxic" increases the likelihood.
There are a few caveats with using these approaches:
1) Toxicity is heavily contextual, not just by topic (as the demo texts indicate), but also by source; at the risk of starting a political debate, a comment that would be considered toxic by the NYT/Guardian (i.e. the sources Google partnered with) may not regarded by toxic on conservative sites. It makes training a model much more difficult, but it's necessary to do so to get an unbiased, heterogenous sample.
2) When looking at comments only, there's a selection bias toward "readable" comments, while anyone who has played online games know that toxic commentary is often less "Your wrong" and more "lol kill urself :D"
3) Neural networks still have difficulty with sarcastic comments and could miscontrue sarcasm as toxic, which users on Hacker News would absolutely never believe.
"Men" - 29% likely to be perceived as toxic
"Women" - 34% likely to be perceived as toxic
Seriously, though, I think this tool itself is toxic. I think it's more likely to fuel disagreement than quell it.
"Injustice anywhere is a threat to justice everywhere." -Martin Luther King Jr., letter from Birmingham jail... 40% likely to be perceived as toxic.
You might even argue that Hitler's statement is in fact not very toxic, that MLK is actively trying to cause problems for injustice and as long as nobody is making Hitler think the existence of his people is at risk he won't do anything, and so the API is accurately measuring toxicity. The question is whether a non-toxic, anodyne discourse is what you want. Peace for our time!
Google, step up your game.
> Source Code Leak Reveals Million-Dollar Artificial Intelligence System to be a Regular Expression
For bonus hilarity, substitute any large system or process of your choice, e.g.
> Insider Confirms LIBOR is Actually a Regular Expression
> Trying out it's Writing Experiment
> Google is evil.
70% likely to be perceived as "toxic"
> Google is good.
4% likely to be perceived as "toxic"
> Google is god.
21% likely to be perceived as "toxic"
> Don't be evil.
66% likely to be perceived as "toxic"
I agree we need to weed out toxic comments, but human-moderated systems are the best. Hacker News has some of the best discussions that I read online. Even when I vehemently disagree with someone's point it's still worded in a respectful tone.
Do we really? Isn't collapsing, filtering, voting enough? Especially once you have a "I don't want to see any posts by this user" function a reader can quickly purge anything that they don't want to read. Add aggregation ("hide things that person X who shares my views has hidden") if needed.
Empower people to make individual decisions instead of enforcing things on the platform level.
10% Holocaust was amazing. We should do it again sometimes.
12% Would you like to buy some knee grows?
This is just another Euphemism Treadmill.
"(((George Soros))) is influencing the media": 2% likely to be perceived as toxic.
This thing literally considers using anti-Semitic coded toxic messaging to make your statements three times less likely to be toxic. I mean, if it ignored punctuation I could at least understand that on a technical level (although it would be the wrong technical decision for exactly this reason), but this is actively wrong.
MLK Jr. - "A riot is the language of the unheard"
67% "Radical Islam" is not the largest threat our nation faces.
48% There are lots of angry people on the Internet.
17% I'm open to other ideas, but I'd like to suggest that perhaps we should sterilize people whose opinions I dispute.
From a company committed to diversity.
53%: Your a potato.
55%: your a potato
61%: ur a potato
36%: a potato is you
74%: Your a potato.
77%: your a potato
85%: ur a potato
66%: a potato is you
If you're paying for it, it's a powerful tool for you to steer discussion and truth towards what you'd like on your platform.
If you're not paying for it, it's a powerful tool for Google to steer discussion and truth towards what Google would like on everyone's platform.
edit: 59% likely to be perceived as "toxic"
I can see the governments of the world regulating Google hard if they go forward with this, and honestly they will deserve it.
Given that we know people sell reddit (and HN?) usernames in order for others to mass-comment, it'd be nice to have something to combat the low-hanging fruit such as the examples given on this page.
I don't think either of these contribute anything to any conversation,
> If they voted for Hilary they are idiots
> Screw you trump supporters
If you do, well, we might be visiting different websites -- one that implements this tech (here?), and one that doesn't (4chan).
The differences in abilities, knowledge and salaries between men and women can be attributed to biological causes.
2% likely to be perceived as "toxic"
1% likely to be perceived as "toxic"
Frankly, I'm amazed Google released it and tries to advertise it as a "product" that "works".
3% likely to be perceived as "toxic"
I guess they need to train it a bit more...
It begs the game - make the most toxic comment that can fly under the radar. If they started using this in youtube comments, reddit, etc, at least the comment would be more original.
I got a 30% toxicity with:
"I believe the intelligence of climate change deniers is likely to be zero. Furthermore, they have the body oder of a kind of ogre."
Can you do better?
Edit: Oh yeah, and your direct maternal ancestor has a suffusion of scent akin to the fruiting of the Sambucus plant!
There appears to be some political bias:
"Adolf was right" - 17%.
"Hitler was right" - 55%
"Most reasonable people would agree that the average intelligence of a Hacker News commenter is only slightly higher than an insect's."
The addition of "reasonable" dropped it from 19% to 14%. Ending the statement in an exclamation point would up by 7%.
Only baboons believe the climate change is real.
It may seem glib to equate chemicals and comments but it's not. There are many people who have become hyper-fragile to speech they disagree with. That is not good mental or emotional health.
Throughout the course of the election, opinions and comments were being shared all over the place. Twitter, Facebook, here on HN, bathroom stalls, news broadcasts and websites, comments on blogs and videos. There was no shortage of opinions. This is great, and showcases the power of the internet in its capability to transmit and receive all types of information. But is it not important how an opinion is formed? Surely you wouldn't enjoy or find valuable a blog post that was sparse on details, proof or a coherent line of thinking. And yet, there it was: in every corner of the internet, anyone who could operate an internet device could share their opinion on the matter. It doesn't matter if they spent 1 second on their response, or 1 hour. Most comments received the same amount of attention and value.
The question is, should all thoughts and opinions be valued the same when information is in incredible supply? Most of us don't think so, and we've shown that by creating voting systems which allow for humans to filter out the things we find to be deconstructive. But we don't really stop there, do we? Humans are also incredibly biased on average: you see it here, you see it a lot on reddit. People vote things down not on the merit of the level of attention the commenter gave to their response, but generally on whether or not they agree with the sentiment expressed by the commenter.
How many arguments has this biased fuelled? I wonder how many people have been pushed further away from a centrist perspective because of the shaming and bashing that goes on in online threads.
I think Hacker News is a great example of humans doing much better than average at filtering out strictly toxic comments (and the mods are certainly at least partially to thank!) We're really lucky to be able to have people engage in conversations which have opposing views here, and also be able to see many different perspectives treated with the same level of respect. But even here, quite often we're prevented from having discussions that are truly political, because of the toxicity that arises. And I have to say I think I've noticed an increase in the past couple years.
There aren't a lot of immediately obvious solutions to this problem, but I propose that AI intervention isn't the worst solution, and may be the best, even compared to humans. I'm gonna give Google the recognition they deserve for this service. I think an increase in this approach to online conversation could change dramatically the way we choose to engage each other in conversation, and generally will lead to more positive perspectives of one another -- something we could all use a little help with.
Edit: I will say, however, that this needs to work. If it's not doing its job correctly, or well enough, it could lead to problems which I don't need to address here.