
$10,000 Stack Overflow Toxicity Challenge - KC8ZKF
https://medium.com/dunder-data/10-000-stack-overflow-toxicity-challenge-75951a739993
======
KirinDave
This is a classic example of a publicity stunt based on the awful premise that
people should be _happy_ that Stack Overflow isn't more toxic. After all, on
some places on the internet people openly talk about raping or killing people,
so what's a little mean spirit and oppression on a tech help site?

What's more, the author has defined "toxic" in a way that guarantees a
negotiation process _after_ results are offered. They can simply sink the
discussion by demanding that, "diversity of thought means I need a new Nazi
and a hyper conservative anti-lgbt religious person on my panel. Surprise
surprise, no one can agree on anything."

I seldom reach for this categorization off the bat, but given that this is a
_classic_ anti-debate tactic that I see all the time, I'm going to call it.
This entire effort is disingenuous bullshit, and I flagged it hoping to avoid
having to see more of this kind of bad logic and bad posting on the Hacker
News main page.

~~~
AcerbicZero
"11\. An independent panel that both you and I agree upon will define what it
means to be “toxic”."

I mean, that's about as fair as you can get when it comes to negotiating an
agreement on something as subjective as "toxicity".

~~~
KirinDave
Having a panel whose members are open and that you could judge before
comitting hours of work: that would be fair.

Instead you have to come with evidence and then negotiate the panel allowing a
potentially disingenuous actor to counter pick your evidence. The way the
author worded this challenge demands we consider them an antagonist. Their
definition of unacceptable is rape and death threats. Given this fact, we must
assume they'd demand judges with similar views.

To engage in this challenge genuinely, we are forced to accept the author's
absurdly high threshold (10%?! That's a wildfire!) and their absurdly
permissive definition of harmless toxicity. I will do neither.

~~~
TedPetrou
Again this needs correcting. I am offering money. Why should I go out of my
way to make it easy for you. Try to build a case on your own if you think its
worthwhile (which you won't).

You seem hung up on the definition of toxicity being defined as rape and death
threats. Your reading comprehension skills are atrocious. Where did I mention
this?

You finally understand something, that the 10% threshold is high. I don't want
to lose my money. But you do realize that many people actually believe it to
be extremely high. YouTube comments might actually exceed that threshold.

~~~
KirinDave
No one wants your paltry prize money, Ted. What they want are environments
with less toxicity.

> You finally understand something, that the 10% threshold is high. I don't
> want to lose my money.

In one fell swoop you've admitted you have set an unreasonably high goal post,
that you don't expect it to be attainable, and that this was not actually
about the inquiry.

But sure, Ted, we should all be grateful to you for offering up money you don'
expect to pay. Why stop at 10%? I'm sure you can pay someone on a bus
somewhere to say 25%, and then you can peg it there. That wouldn't be
substantially less disingenuous than what you've done so far, so I say: live
your best life.

~~~
ThoAppelsin
> an unreasonably high goal post

I wouldn't say _unreasonably_ high for 10% for a single man offering that much
money. I would even say that it is rather very _risky_.

Besides, claiming a platform is "a toxic wasteland" puts that platform on par
with the ones like 4chan, and that does seem to require 20% toxicity to me,
maybe even higher. For it to get more strict like 3-5%, the claim should be
lighter, something like "slightly toxic". It is nice of him to empathise and
assume that the claims are probably exaggerated, and to set the threshold to
something in between.

> No one wants your paltry prize money, Ted.

Lastly, I definitely would enjoy an extra 10K, but I don't really think I can
beat that requirement, since I don't think Stack Overflow is even slightly
toxic.

~~~
KirinDave
It stuns me that you write this, in a public forum, and don't expect people to
conclude your entire effort was a self-centering publicity stunt.

Your goal is to discredit anyone who suggests your favorite crowdsourcing
website has any toxicity at all. So you set absurdly high thresholds and put
into place unfair processes to make sure the prize money can't be claimed.

------
minimaxir
This competition is a prosecutor's fallacy
([https://en.wikipedia.org/wiki/Prosecutor%27s_fallacy](https://en.wikipedia.org/wiki/Prosecutor%27s_fallacy)),
_especially_ since the definition of "toxicity" in this problem is 100%
arbitrary.

Also, about the required input fee:

> _I am offering a free $10,000 of my own money. If you really believe that
> you can win, then obviously $500 won’t deter you at all._

Since we're talking about statistical modeling, the rational argument would be
that a person would only participate if expected value of winning the
competition is greater than the input pot. A ~5% chance of winning, given the
time investment and opportunity cost involved, isn't compelling enough to risk
it. That's why Kaggle competitions are free.

~~~
IshKebab
He does address this:

> An independent panel that both you and I agree upon will define what it
> means to be “toxic”.

But the author is right so there's zero chance of anyone taking him up on this
offer.

~~~
KirinDave
That's a non-answer though. That process point alone means no prize need ever
be rewarded. If you actually wanted to get results from this, you'd have this
panel selected in advance so that people could just the fairness of the panel
_before_ engaging. Not allowing for post-engagement "Ha ha I demand Milo Y. is
on my panel and he thinks this is fine to say to a woman."

Of course that's just one of many red flags.

~~~
AcerbicZero
I believe this is addressed in the "Extraordinary claims require extraordinary
evidence" section. If Stack Overflow is anywhere near as toxic as some people
have suggested, this would seem to be a reasonable method of encouraging
people to prove it.

One of the often repeated comments of several popular conservative "speakers"
(Shapiro, although I think I've heard something similar from Peterson,
Crowder, etc.) is that if you present proof of a specific problem they would
be happy to work with anyone and everyone to resolve it. A generous
interpretation of the author could be that they want to see Stack Overflow
succeed, and by addressing this potential issue directly they can fix it, if
it does exist at all.

~~~
KirinDave
Who anywhere has suggested that Stack Overflow is 10% rape and murder threats?
This is an absurd standard that is being advanced.

Of course I can't prove that. I never wanted to. The author is using a straw
man so potent he can comfortably put a lot of money behind it.

~~~
TedPetrou
Why is your reading comprehension so bad?

~~~
KirinDave
Oh I'm sorry, did you NOT set a threshold of 10% and then say, and I quote,
"In an internet filled with people wishing you were dead or raped (see YouTube
comments sections), Stack Overflow is a saint."

Sorry for drawing the obvious conclusion: that that's where you draw the line.

Maybe, Ted, just maybe... no one needed your opinion, your shoddy poll, your
patronizing fake bounty, your bait and switch methodology, or your Dawkins-
style rant about how people should be grateful they aren't the direct victims
of violence. Maybe you had nothing positive to add, and you're feeling clap-
back because what you did add was so bad even Stack Overflow walked back from
you.

------
GVIrish
I don't know how one would define toxicity for the purpose of this challenge,
but I am certainly baffled as to how people accuse StackOverflow of being a
'toxic wasteland'.

Some people can be a bit curt or even rude with poorly worded or low-effort
questions but I can't say that's something I see there in even 1 out of 25-50
questions I see.

I suspect that some people get frustrated when they first try to ask questions
and get rebuffed because they didn't follow the rules of the site. If you hop
on there asking open-ended opinion questions or asking something that can
easily be found with search, then yeah it might get closed rather quickly.
Doesn't make SO a toxic community by any means.

------
sudosteph
I disagree with the tweet about SO being a toxic wasteland, but OP really
undermines his point by being so combative about it. Especially when he turns
around and says stuff like:

> If someone really believes that Stack Overflow is toxic then they should
> back it up with actual data and not hysterical and slanderous remarks.

It's poor form to call a woman's opinion "hysterical". There are other words
that can convey a similar meaning without the heaps of sexist baggage that
"hysteria" has historically implied. I suggest "inflammatory" here.

It's pretty telling that of the few comments on medium, two are actually
people from SO trying to distance themselves from OP's "challenge".

~~~
minimaxir
Yes, I recommend reading the comments on this article, they're interesting.
When a commenter argued that the fee is unfair toward minorites, OP replied:

> _What is it about minorities that makes you believe they have less
> propensity to make the donation? That’s blatantly racist. If you know you
> are going to win $10k, then it shouldn’t matter to you. I’ll be happy to
> lower this to $100 in 6 months if nobody makes a submission._

That attitude is arguably more toxic than the typical Stack Overflow comment.

~~~
xixi77
What is particularly toxic about this comment though?

I would actually agree that there won't be many poor/minority people
participating, but that has more to do with the challenge itself, not the fee.
Poor people buy a lot of lottery tickets, particularly when the chance of
winning is high. The issue here is that the chances of winning are not high,
so the only reason someone would try and go after the prize would be to prove
a point, and spending a lot of time and effort to prove a point like this one
is a luxury many people don't have.

------
kangnkodos
I can imagine an environment in which less than 10% of the posts are toxic,
yet it feels toxic to one person.

I’m at a conference between sessions, and there are small groups of people
standing around, discussing the latest developments at breakneck speed. Back
and forth. Talking a mile a minute. Saying half an idea out loud, and letting
others fill in the blanks. Tons of jargon and inside references I don’t
understand. I squeeze into one of the larger circles, and I would like to join
the conversation, at least to understand what they are talking about, and
perhaps add something to the conversation, if I can. I finally see an opening,
and ask a question. Each person in the group is suddenly silent and stares at
me for a few seconds. Finally one person says, “RTFM noob”. Everyone stifles a
laugh, and the conversation picks up where it left off.

Count the number of “posts” in this example. Hundreds, or maybe low thousands.
How many were toxic? One. Doing the math… Yeah. That’s much less than 10%.

Does this feel toxic to me? Oh yeah. Does it feel toxic to everyone else? Some
would say yes. Some would agree with you and say, “No. This does not feel like
a toxic wasteland to me. I’ll admit, there’s a small amount of toxic
discussion, but at least it’s much less toxic than other conferences I have
attended.”

My point is that you can have a forum where less than 10% of the comments are
toxic, and yet that forum can feel very toxic to some of the people. So your
10% challenge does not prove anything.

~~~
TedPetrou
I agree with nearly all of what you said but that wasn't the point of the
challenge. I completely agree that many individuals experience negative events
on Stack Overflow and would compassionately listen to them. The only thing I
am challenging is the notion, that on a whole, SO is a toxic wasteland. That
is it.

The reason 10% was set as a threshold is that many people actually believe it
to be higher than that. 10% is an absurdly high number of toxic posts and the
real number is going to be far lower. I value authentic statements with
accurate data to the highest degree.

I hope that we can get some real research on this topic with more accurate
data which can only help to improve outcomes.

~~~
kangnkodos
On reddit, it looks like someone had some data which calculated that the
number of toxic comments on SO is about 1%. That sounds about right to me.

But even 1% toxic comments correctly still makes it a toxic wasteland to the
people who have been on the receiving end of those toxic comments, and to
those who don't enjoy seeing others treated like that.

~~~
kangnkodos
I mean imagine that you had a job where you were constantly having short
conversations with people from all over your company.

I'm your boss, and I call you into a meeting to let you know that you are on
the edge of being fired because of your toxic comments. Some of your
conversations have resulted in coworkers crying and going home early. But I'm
willing to give you one more chance.

In the meeting you tell me, "Hey, relax. It's not true that 10% of my
conversations have resulted in coworkers going home early. I can prove that it
only happens in 1% of my conversations. And 1% is fine, so get off my back."

I would tell you to just pack your desk right now.

------
detaro
Response by StackOverflow's David Fullerton:
[https://medium.com/@dfullerton07/cto-of-stack-overflow-
here-...](https://medium.com/@dfullerton07/cto-of-stack-overflow-
here-27d84691d9a3)

~~~
valbaca
Maturely said.

~~~
TedPetrou
His comment is dishonest. It claims that "By point-blank denying the
experience of women, people of color, and others who are telling us that we
have a problem, we diminish them and communicate to them that they are not
valued and do not belong"

I am not denying anyone's personal experience or that there isn't a problem. I
am rejecting the claim that SO, as a whole, is a toxic wasteland.

------
XR0CSWV3h3kZWg
It would be nice if the author provided some examples of posts that they found
to be toxic.

------
knorker
Why have facts when you can just dump the term "toxic inhumane
wasteland[…]shaming[…]…excluding women and people of color" without a shred of
evidence and running away?

~~~
detaro
Have you looked at what she has posted, or did you see the screenshot of a
tweet in this post and believed that's all there is?

~~~
knorker
I saw her post, yes. One of her examples was the tragedy of SO automatically
removing "hello" from the start of posts.

Oh, the tragedy!

She completely ignores that it's _more_ disrespectful to waste the readers
time with small talk before a question. If she wrote an FAQ, would she write
an FAQ like this:

Q: Hello HSBC, I hope you're well this Friday afternoon. I was wondering what
your phone number is.

A: Hello dear customer. Happy Friday to you too! We would love for you to call
us. We always appreciate talking to customers about their banking. It's XXX-
XXXXXX.

Q: Hello HSBC, I hope you're well this Friday afternoon. I was wondering what
your phone opening hours are.

A: Hello dear customer. Happy Friday to you too! We would love for you to call
us. We always appreciate talking to customers about their banking. We'll be
ready for your call between XX and YY Mon-Sun.

So that's what a non-"toxic inhumane wasteland[…]shaming[…]…excluding women
and people of color" would look like?

------
prolepunk
It's really weird that even stack overflow as organization admits that they
have diversity problem and they are looking for ways to fix that.

Why this person needs to question that, it's already been accepted? Then he
includes so many caveats to this challenge that it looks that he's the only
person who needs to believe that stack overflow is a totally fair and unbiased
environment.

~~~
nmstoker
Your definition of toxicity may differ from mine, but a lack of diversity does
not make something intrinsically toxic to me (even though usually one would
typically want to try to reduce a lack of diversity)

For me toxicity generally implies an actively negative response.

Conflating toxicity with a lack of diversity would stereotype the dominant
group!

------
shkkmo
The issue isn't that the questions/answers are toxic. It is that the flagging
system is used toxically.

------
xixi77
Actually looking at other things the woman making the "toxic wasteland"
comment wrote, I certainly don't agree with her on everything, but I'd say she
actually makes a lot of good points. Particularly about how delivery style
matters.

And yet, her comment itself is clearly an exaggeration (let's be real, no one
is going to take up the $10K challenge), and is just as toxic as anything she
mentions. It just shows that it is always far easier to see problems and
possible fixes in other people's behavior, than in your own.

~~~
TedPetrou
Very well said and well done grasping exactly my issue - that calling SO a
toxic wasteland isn't correct nor helpful.

------
AcerbicZero
Stack Overflow's semi-hostile approach to asking questions you can look up
yourself was one of the most useful lessons I ever learned. It forced me to
explore all my options before asking a question, because getting called out
for not doing basic research was pretty embarrassing.

Maybe I'm an outlier here, and maybe some people don't appreciate that barrier
to entry, but it seems to work. I don't think I've ever run into any _really_
hostile or offensive comments while using Stack Overflow.

~~~
nmstoker
It's a matter of perspective about who's starting hostilities when a
questioner asks with no obvious investment of effort! But that way leads to
endless squabbles fairly soon.

Clearly it's good to deter such people, but where I think SO could improve a
little is if it had some more up front deterrence, and thus avoided more of
the naive (some would say lazy) questioners from proceeding, only to get
embarrassed/called out. They do try to identify similar questions but perhaps
a specific step right before a new user submits asking them to highlight which
part of their question covers what they've already tried/researched (with a
friendly option to add such a part if they've merely overlooked it)

Like you, I've never seen it particularly unfair or harsh, but smoothing
things along helps everyone.

------
latortuga
I'm sure it's going to go really well when you propose a contest in an attempt
to marginalize a woman's perspective by calling her hysterical.

------
gHosts
> All posts must be chosen at random, where each post has an equal chance of
> being included in the study (simple random sample)

Well there is a massive bias straight off.

Nobody arrives at a post at random, nor will the toxicity be uniformly
distributed.

When last did you go to Stack* and say "Show me a random comment please".

------
asdsa5325
What is the definition of "toxic"?

For the record, obviously not 10% are "toxic" (based off of my personal
definition of toxic).

But what if I considered comments that are off topic or slightly factually
wrong as "toxic". By that definition, 10% would be possible.

~~~
KirinDave
It seems like 10% is such an absurdly high minimum.

------
notananthem
This is just a publicity stunt, some no name guy who sells books and has
negligible twitter followers. Hence publishing on Medium.

Just because its on Medium it doesn't mean anything, its like linking your
personal Facebook page. He took a twitter poll of 20 people.

------
0x4f3759df
Toxic is one of those words like racism, someone accused of being a racist has
no accepted mechanism to prove otherwise.

