Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
[flagged] $10,000 Stack Overflow Toxicity Challenge (medium.com/dunder-data)
31 points by KC8ZKF on April 18, 2018 | hide | past | favorite | 61 comments


This is a classic example of a publicity stunt based on the awful premise that people should be happy that Stack Overflow isn't more toxic. After all, on some places on the internet people openly talk about raping or killing people, so what's a little mean spirit and oppression on a tech help site?

What's more, the author has defined "toxic" in a way that guarantees a negotiation process after results are offered. They can simply sink the discussion by demanding that, "diversity of thought means I need a new Nazi and a hyper conservative anti-lgbt religious person on my panel. Surprise surprise, no one can agree on anything."

I seldom reach for this categorization off the bat, but given that this is a classic anti-debate tactic that I see all the time, I'm going to call it. This entire effort is disingenuous bullshit, and I flagged it hoping to avoid having to see more of this kind of bad logic and bad posting on the Hacker News main page.


"11. An independent panel that both you and I agree upon will define what it means to be “toxic”."

I mean, that's about as fair as you can get when it comes to negotiating an agreement on something as subjective as "toxicity".


Having a panel whose members are open and that you could judge before comitting hours of work: that would be fair.

Instead you have to come with evidence and then negotiate the panel allowing a potentially disingenuous actor to counter pick your evidence. The way the author worded this challenge demands we consider them an antagonist. Their definition of unacceptable is rape and death threats. Given this fact, we must assume they'd demand judges with similar views.

To engage in this challenge genuinely, we are forced to accept the author's absurdly high threshold (10%?! That's a wildfire!) and their absurdly permissive definition of harmless toxicity. I will do neither.


Again this needs correcting. I am offering money. Why should I go out of my way to make it easy for you. Try to build a case on your own if you think its worthwhile (which you won't).

You seem hung up on the definition of toxicity being defined as rape and death threats. Your reading comprehension skills are atrocious. Where did I mention this?

You finally understand something, that the 10% threshold is high. I don't want to lose my money. But you do realize that many people actually believe it to be extremely high. YouTube comments might actually exceed that threshold.


No one wants your paltry prize money, Ted. What they want are environments with less toxicity.

> You finally understand something, that the 10% threshold is high. I don't want to lose my money.

In one fell swoop you've admitted you have set an unreasonably high goal post, that you don't expect it to be attainable, and that this was not actually about the inquiry.

But sure, Ted, we should all be grateful to you for offering up money you don' expect to pay. Why stop at 10%? I'm sure you can pay someone on a bus somewhere to say 25%, and then you can peg it there. That wouldn't be substantially less disingenuous than what you've done so far, so I say: live your best life.


> an unreasonably high goal post

I wouldn't say unreasonably high for 10% for a single man offering that much money. I would even say that it is rather very risky.

Besides, claiming a platform is "a toxic wasteland" puts that platform on par with the ones like 4chan, and that does seem to require 20% toxicity to me, maybe even higher. For it to get more strict like 3-5%, the claim should be lighter, something like "slightly toxic". It is nice of him to empathise and assume that the claims are probably exaggerated, and to set the threshold to something in between.

> No one wants your paltry prize money, Ted.

Lastly, I definitely would enjoy an extra 10K, but I don't really think I can beat that requirement, since I don't think Stack Overflow is even slightly toxic.


It stuns me that you write this, in a public forum, and don't expect people to conclude your entire effort was a self-centering publicity stunt.

Your goal is to discredit anyone who suggests your favorite crowdsourcing website has any toxicity at all. So you set absurdly high thresholds and put into place unfair processes to make sure the prize money can't be claimed.


I agree that having the panel decided in advance would be better, but I don't think that discredits the overall challenge, or the goal. It doesn't say you have to decide the panel make up after you start, so there is no reason to assume you couldn't negotiate that up front. If the author is making unreasonable demands during that negotiation, you've lost at worst an hour or two of time emailing back and forth, and you can write your own article fully discrediting this challenge.

I guess my point is that potential for unreasonable demands, doesn't make this challenge any less (or more) valid. Regardless, I appreciate your perspective on this, and your willingness to engage in a discussion of it.


I've listed at least 3 things that individually are problematic (predisposition, process, and the arbitrarily high threshold). Putting them together, I see no reason why we SHOULD extend a principle of charity.


I think SO has gotten what I would call more rules bound. When I started participating in 09 and there were fun code golf challenges some open discussions etc. Now its very strict questions must be directly answerable no discussion no challenges.

I don't understand what people find toxic about it. It just seems to be vague. Maybe I'm not getting something. Its fundamentally a site where people help you for internet points that seems on its basis to be the definition of non toxic.


I don't think it's so cut and dry. Reddit could be described as the same charter and I think we all have to agree that place got pretty toxic.


Maybe. I just assume any forum will have its bad actors. I don't know when that crosses the line to a toxic place.


A few things need correcting.

I did not claim that people should feel happy on Stack Overflow. My claim is that it is not a toxic wasteland. That is it. I understand people have bad experiences on the site and I hope I can offer some good suggestions to SO on how to improve.

Building a straw man around "raping or killing" isn't going to work. Toxicity has a lower threshold and nowhere did I insinuate this was where the threshold was.


If "raping and killing" were not your threshold why did you bring it up?


I believe it is just a manner of speech to talk about the extremes to claim the innocence of your defendant. Without such examples, declaring Stack Overflow as a saint would sound dull.


Having seen this exact line of reasoning used to reject varying levels of harassment in a variety of settings, I do not believe this.


This competition is a prosecutor's fallacy (https://en.wikipedia.org/wiki/Prosecutor%27s_fallacy), especially since the definition of "toxicity" in this problem is 100% arbitrary.

Also, about the required input fee:

> I am offering a free $10,000 of my own money. If you really believe that you can win, then obviously $500 won’t deter you at all.

Since we're talking about statistical modeling, the rational argument would be that a person would only participate if expected value of winning the competition is greater than the input pot. A ~5% chance of winning, given the time investment and opportunity cost involved, isn't compelling enough to risk it. That's why Kaggle competitions are free.


(Edit: When I wrote this comment, the parent comment only contained what is now the first paragraph. I didn't know about the entrance fee.)

And yet I agree with the author. I also hear about toxic posts and the terrible community, but I do not experience it myself. Having people try to put a finger on it would be good. When the $10k is to be paid is indeed unfortunately unclear, but I do like the competition, and I do hope that people that have experienced it will try to quantify it, even if they can't meet the 10% requirement.


He does address this:

> An independent panel that both you and I agree upon will define what it means to be “toxic”.

But the author is right so there's zero chance of anyone taking him up on this offer.


That's a non-answer though. That process point alone means no prize need ever be rewarded. If you actually wanted to get results from this, you'd have this panel selected in advance so that people could just the fairness of the panel before engaging. Not allowing for post-engagement "Ha ha I demand Milo Y. is on my panel and he thinks this is fine to say to a woman."

Of course that's just one of many red flags.


I believe this is addressed in the "Extraordinary claims require extraordinary evidence" section. If Stack Overflow is anywhere near as toxic as some people have suggested, this would seem to be a reasonable method of encouraging people to prove it.

One of the often repeated comments of several popular conservative "speakers" (Shapiro, although I think I've heard something similar from Peterson, Crowder, etc.) is that if you present proof of a specific problem they would be happy to work with anyone and everyone to resolve it. A generous interpretation of the author could be that they want to see Stack Overflow succeed, and by addressing this potential issue directly they can fix it, if it does exist at all.


Who anywhere has suggested that Stack Overflow is 10% rape and murder threats? This is an absurd standard that is being advanced.

Of course I can't prove that. I never wanted to. The author is using a straw man so potent he can comfortably put a lot of money behind it.


Why is your reading comprehension so bad?


Oh I'm sorry, did you NOT set a threshold of 10% and then say, and I quote, "In an internet filled with people wishing you were dead or raped (see YouTube comments sections), Stack Overflow is a saint."

Sorry for drawing the obvious conclusion: that that's where you draw the line.

Maybe, Ted, just maybe... no one needed your opinion, your shoddy poll, your patronizing fake bounty, your bait and switch methodology, or your Dawkins-style rant about how people should be grateful they aren't the direct victims of violence. Maybe you had nothing positive to add, and you're feeling clap-back because what you did add was so bad even Stack Overflow walked back from you.


5% chance of winning? Where does that number come from, 500/10,000?

That is not a 'chance of winning.'


Sorry, chance of winning assuming if winning was a independently random event, which in this case may not be accurate (although you could make an argument if the odds of winning are better or worse than 5%)


You must realise that the chance of winning is independent of the 'entry fee' for this game. You could argue of a ROI or something similar in terms of decision making for entry, but that still has no effect on your chances.


> "The following demonstrates the fallacy in the context of a prosecutor questioning an expert witness: “the odds of finding this evidence on an innocent man are so small that the jury can safely disregard the possibility that this defendant is innocent”.[1] The fallacy obscures that the odds of a defendant being innocent given said evidence in fact depends on the likely higher prior odds of the defendant being innocent, the explicitly lesser odds of the evidence in the case that he was innocent as mentioned, as well as the underlying cumulative odds of the evidence being on the defendant."

Care to explain how this fallacy is related to the above competition?


I lowered the donation to $50. I think I would be comfortable with quite a liberal definition of toxic as 10% is a high threshold.


I don't know how one would define toxicity for the purpose of this challenge, but I am certainly baffled as to how people accuse StackOverflow of being a 'toxic wasteland'.

Some people can be a bit curt or even rude with poorly worded or low-effort questions but I can't say that's something I see there in even 1 out of 25-50 questions I see.

I suspect that some people get frustrated when they first try to ask questions and get rebuffed because they didn't follow the rules of the site. If you hop on there asking open-ended opinion questions or asking something that can easily be found with search, then yeah it might get closed rather quickly. Doesn't make SO a toxic community by any means.


I disagree with the tweet about SO being a toxic wasteland, but OP really undermines his point by being so combative about it. Especially when he turns around and says stuff like:

> If someone really believes that Stack Overflow is toxic then they should back it up with actual data and not hysterical and slanderous remarks.

It's poor form to call a woman's opinion "hysterical". There are other words that can convey a similar meaning without the heaps of sexist baggage that "hysteria" has historically implied. I suggest "inflammatory" here.

It's pretty telling that of the few comments on medium, two are actually people from SO trying to distance themselves from OP's "challenge".


Yes, I recommend reading the comments on this article, they're interesting. When a commenter argued that the fee is unfair toward minorites, OP replied:

> What is it about minorities that makes you believe they have less propensity to make the donation? That’s blatantly racist. If you know you are going to win $10k, then it shouldn’t matter to you. I’ll be happy to lower this to $100 in 6 months if nobody makes a submission.

That attitude is arguably more toxic than the typical Stack Overflow comment.


What is particularly toxic about this comment though?

I would actually agree that there won't be many poor/minority people participating, but that has more to do with the challenge itself, not the fee. Poor people buy a lot of lottery tickets, particularly when the chance of winning is high. The issue here is that the chances of winning are not high, so the only reason someone would try and go after the prize would be to prove a point, and spending a lot of time and effort to prove a point like this one is a luxury many people don't have.


Thank you for the constructive feedback. I agree that I should have used 'inflammatory' and have just now changed it.

The comments from SO were, unfortunately, strawmen arguments. They apparently indulge in self-flagellation to appeal to the loudest.

I think it would be better if the SO team said something assertive along the lines of "No, we are not a toxic wasteland, but we have problems ..."


I can imagine an environment in which less than 10% of the posts are toxic, yet it feels toxic to one person.

I’m at a conference between sessions, and there are small groups of people standing around, discussing the latest developments at breakneck speed. Back and forth. Talking a mile a minute. Saying half an idea out loud, and letting others fill in the blanks. Tons of jargon and inside references I don’t understand. I squeeze into one of the larger circles, and I would like to join the conversation, at least to understand what they are talking about, and perhaps add something to the conversation, if I can. I finally see an opening, and ask a question. Each person in the group is suddenly silent and stares at me for a few seconds. Finally one person says, “RTFM noob”. Everyone stifles a laugh, and the conversation picks up where it left off.

Count the number of “posts” in this example. Hundreds, or maybe low thousands. How many were toxic? One. Doing the math… Yeah. That’s much less than 10%.

Does this feel toxic to me? Oh yeah. Does it feel toxic to everyone else? Some would say yes. Some would agree with you and say, “No. This does not feel like a toxic wasteland to me. I’ll admit, there’s a small amount of toxic discussion, but at least it’s much less toxic than other conferences I have attended.”

My point is that you can have a forum where less than 10% of the comments are toxic, and yet that forum can feel very toxic to some of the people. So your 10% challenge does not prove anything.


I agree with nearly all of what you said but that wasn't the point of the challenge. I completely agree that many individuals experience negative events on Stack Overflow and would compassionately listen to them. The only thing I am challenging is the notion, that on a whole, SO is a toxic wasteland. That is it.

The reason 10% was set as a threshold is that many people actually believe it to be higher than that. 10% is an absurdly high number of toxic posts and the real number is going to be far lower. I value authentic statements with accurate data to the highest degree.

I hope that we can get some real research on this topic with more accurate data which can only help to improve outcomes.


On reddit, it looks like someone had some data which calculated that the number of toxic comments on SO is about 1%. That sounds about right to me.

But even 1% toxic comments correctly still makes it a toxic wasteland to the people who have been on the receiving end of those toxic comments, and to those who don't enjoy seeing others treated like that.


I mean imagine that you had a job where you were constantly having short conversations with people from all over your company.

I'm your boss, and I call you into a meeting to let you know that you are on the edge of being fired because of your toxic comments. Some of your conversations have resulted in coworkers crying and going home early. But I'm willing to give you one more chance.

In the meeting you tell me, "Hey, relax. It's not true that 10% of my conversations have resulted in coworkers going home early. I can prove that it only happens in 1% of my conversations. And 1% is fine, so get off my back."

I would tell you to just pack your desk right now.


Response by StackOverflow's David Fullerton: https://medium.com/@dfullerton07/cto-of-stack-overflow-here-...


Maturely said.


His comment is dishonest. It claims that "By point-blank denying the experience of women, people of color, and others who are telling us that we have a problem, we diminish them and communicate to them that they are not valued and do not belong"

I am not denying anyone's personal experience or that there isn't a problem. I am rejecting the claim that SO, as a whole, is a toxic wasteland.


It would be nice if the author provided some examples of posts that they found to be toxic.


[flagged]


I think.. its a guy/cismale


April Wensel is a guy/cismale?


Why have facts when you can just dump the term "toxic inhumane wasteland[…]shaming[…]…excluding women and people of color" without a shred of evidence and running away?


Have you looked at what she has posted, or did you see the screenshot of a tweet in this post and believed that's all there is?


I saw her post, yes. One of her examples was the tragedy of SO automatically removing "hello" from the start of posts.

Oh, the tragedy!

She completely ignores that it's more disrespectful to waste the readers time with small talk before a question. If she wrote an FAQ, would she write an FAQ like this:

Q: Hello HSBC, I hope you're well this Friday afternoon. I was wondering what your phone number is.

A: Hello dear customer. Happy Friday to you too! We would love for you to call us. We always appreciate talking to customers about their banking. It's XXX-XXXXXX.

Q: Hello HSBC, I hope you're well this Friday afternoon. I was wondering what your phone opening hours are.

A: Hello dear customer. Happy Friday to you too! We would love for you to call us. We always appreciate talking to customers about their banking. We'll be ready for your call between XX and YY Mon-Sun.

So that's what a non-"toxic inhumane wasteland[…]shaming[…]…excluding women and people of color" would look like?


It's really weird that even stack overflow as organization admits that they have diversity problem and they are looking for ways to fix that.

Why this person needs to question that, it's already been accepted? Then he includes so many caveats to this challenge that it looks that he's the only person who needs to believe that stack overflow is a totally fair and unbiased environment.


Your definition of toxicity may differ from mine, but a lack of diversity does not make something intrinsically toxic to me (even though usually one would typically want to try to reduce a lack of diversity)

For me toxicity generally implies an actively negative response.

Conflating toxicity with a lack of diversity would stereotype the dominant group!


The issue isn't that the questions/answers are toxic. It is that the flagging system is used toxically.


Actually looking at other things the woman making the "toxic wasteland" comment wrote, I certainly don't agree with her on everything, but I'd say she actually makes a lot of good points. Particularly about how delivery style matters.

And yet, her comment itself is clearly an exaggeration (let's be real, no one is going to take up the $10K challenge), and is just as toxic as anything she mentions. It just shows that it is always far easier to see problems and possible fixes in other people's behavior, than in your own.


Very well said and well done grasping exactly my issue - that calling SO a toxic wasteland isn't correct nor helpful.


Stack Overflow's semi-hostile approach to asking questions you can look up yourself was one of the most useful lessons I ever learned. It forced me to explore all my options before asking a question, because getting called out for not doing basic research was pretty embarrassing.

Maybe I'm an outlier here, and maybe some people don't appreciate that barrier to entry, but it seems to work. I don't think I've ever run into any really hostile or offensive comments while using Stack Overflow.


It's a matter of perspective about who's starting hostilities when a questioner asks with no obvious investment of effort! But that way leads to endless squabbles fairly soon.

Clearly it's good to deter such people, but where I think SO could improve a little is if it had some more up front deterrence, and thus avoided more of the naive (some would say lazy) questioners from proceeding, only to get embarrassed/called out. They do try to identify similar questions but perhaps a specific step right before a new user submits asking them to highlight which part of their question covers what they've already tried/researched (with a friendly option to add such a part if they've merely overlooked it)

Like you, I've never seen it particularly unfair or harsh, but smoothing things along helps everyone.


Definitely. The best thing StackOverflow taught me is that I don't need StackOverflow.

Sure there are some exceptional cases, such as when things just Will. Not. Work. when getting started with a new language/framework/whatever, but I learned to make use of documentation, Google, and StackOverflow's search in place of asking the internet to be my personal debugger.

I only ask a question as a last resort, and sure sometimes it gets flagged (oh no, the horror!), or people in the comments assume I don't know anything about anything (maybe, but I tend to know at least slightly more than they take me for). But as a whole, it's a very healthy community and I tend to learn something new with each visit I make.


I'm sure it's going to go really well when you propose a contest in an attempt to marginalize a woman's perspective by calling her hysterical.


> All posts must be chosen at random, where each post has an equal chance of being included in the study (simple random sample)

Well there is a massive bias straight off.

Nobody arrives at a post at random, nor will the toxicity be uniformly distributed.

When last did you go to Stack* and say "Show me a random comment please".


What is the definition of "toxic"?

For the record, obviously not 10% are "toxic" (based off of my personal definition of toxic).

But what if I considered comments that are off topic or slightly factually wrong as "toxic". By that definition, 10% would be possible.


It seems like 10% is such an absurdly high minimum.


This is just a publicity stunt, some no name guy who sells books and has negligible twitter followers. Hence publishing on Medium.

Just because its on Medium it doesn't mean anything, its like linking your personal Facebook page. He took a twitter poll of 20 people.


Toxic is one of those words like racism, someone accused of being a racist has no accepted mechanism to prove otherwise.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: