
Does Transparency in Moderation Matter? [pdf] - wcerfgba
https://dl.acm.org/doi/pdf/10.1145/3359252
======
eganist
Relationship Advice lead-ish mod here.

I've posted this elsewhere on hackernews
([https://news.ycombinator.com/item?id=23259595](https://news.ycombinator.com/item?id=23259595)),
but I'll repost it here since it's relevant:

\---

We're (r/relationship_advice) rarely transparent with removal reasons. Our
templatized removal reasons generally look something like this:

> u/[user], please message the mods:

> 1\. to find out why this post was removed, and

> 2\. prior to posting any updates.

> Thanks.

or

> User was banned for this [submission/comment].

The reason is because we have a high population of people who:

1\. post too much information and expose themselves to doxxing risks, and

2\. post fantasies that never happened.

So in order to protect the people who inadvertently post too much information,
we tend to remove these posts using the same generic removal template.
However, if people know that the post was pulled for one of these two reasons,
the submitter may still end up on the receiving end of harassment as a result,
meaning we have to fuzz the possibility of the removal being for one of these
two reasons by much more broadly withholding removal reasons.

This is specific to r/relationship_advice. Other subreddits have other
procedures.

~~~
stronglikedan
Well, as long as you don't replace removal reasons with unwarranted insults
when withholding removal reasons, then you're already a step ahead of 90% of
reddit mods.

~~~
pmiller2
Indeed, I’ve been muted for daring to ask why a comment of mine was removed,
then subsequently banned for replying “yes, and here’s why,” including a link
to this very article when asked “Do you think you’re entitled to a reason?”

I’ve also been banned from a sub simply for posting in a completely different
sub.

------
tumetab1
Few years back Jeff Atwood on coding horror (which I can't find now) had a
"guide" that partially match this data. The poster, and everyone else, must
know see the original post and the reason it was moderated so the rule is
clearer for everyone.

I think in Reddit, like other communities, the problem still is that most
users still don't see moderated content because only upvoted/popular content
is seen.

Maybe that could be future analysis comparing communities where moderated
content is regularly visible by the overall community vs small set of the
community vs only the poster.

~~~
TulliusCicero
I had this debate with other mods on r/cscareerquestions a few years back. My
preference was for SomethingAwful-style modding transparency, where your bad
post (usually) stays up with a note that says "USER WAS BANNED FOR THIS POST",
and in fact there's a global listing of all the recent bans, including the
user and posts in question.

The other mods largely preferred invisible modding, that "stays out of the
way", that doesn't draw attention away from productive discussion.

I'm not sure either is obviously a better methodology, honestly.

~~~
tumetab1
My guess, is that both methodologies should be combined.

Let the focus be on productive discussions while also ask for the users
attention to the moderation done.

For example, every X time show to the user the forum space where all the
moderated content is.

------
CM30
As I'd have expected, it seems providing an explanation does indeed reduce the
likelihood the user will break the rules in future, and make them more likely
to contribute in future.

Surprised there wasn't much difference between a human provided reason and a
bot provided one though (and in fact, the latter performed slightly better).
Wonder what the reason behind this could be?

> s.Our results show that controlling for other factors,explanations provided
> by automated tools or bots are associated with higher odds of moderatedusers
> posting in the future

Either way, I always make sure to provide an explanation why a piece of
content was removed on any site I manage, with the sole exception of obvious
bot submissions (since the latter are literally incapable of understanding
feedback).

~~~
TulliusCicero
Sometimes, you don't want them to contribute anymore.

A large number of the people who get banned on the subs I moderate are just
belligerent assholes that we're better off without. Someone posting blatantly
racist or misogynistic comments isn't someone we want around.

~~~
naasking
> A large number of the people who get banned on the subs I moderate are just
> belligerent assholes that we're better off without.

Nobody is "just" anything.

~~~
TulliusCicero
I’m talking about their message board contributions, obviously.

That someone might have a normal life outside of screaming at women and
minorities online doesn’t have a ton of relevance on the forum where they’re
doing the screaming.

~~~
naasking
My point is that reducing even someone's comments to a simple label is a
dangerous mindset,particularly negative labels. Once you've applied that
label, there is no redemption. What would this person have to do to remove a
negative label you've associated with them?

~~~
TulliusCicero
I think it's more dangerous to allow them to stick around because of a
misguided sense of charity.

> What would this person have to do to remove a negative label you've
> associated with them?

First, let's stop right here. You're using weasely language to try to make it
sound like noticing someone has said bigoted things, and kicking them from the
community for it, is some kind of mistake, as if the bigotry was the fault of
the moderator, rather than the bigot.

Place blame where it belongs: if someone says something sexist, someone
noticing that they appear to be sexist is fully the responsibility of the
person who said the sexist things.

Let's reword what you said in a way that makes sense:

> What would the bigoted poster have to do to convince you they're no longer a
> bigot? (at least as far as internet posting goes)

There, much better.

Depends on what they said, of course. If they said something racist, I would
expect an earnest acknowledgement of that, that racism is bad, etc. As well as
evidence from posting elsewhere on Reddit that they've reformed.

~~~
naasking
> You're using weasely language to try to make it sound like noticing someone
> has said bigoted things, and kicking them from the community for it, is some
> kind of mistake, as if the bigotry was the fault of the moderator, rather
> than the bigot.

And you're trying to deflect from the problems by once again reducing a
person's behaviour to simple labels that justify any actions to stop "bad
people". Every mod has certainly banned people who are not bad actors, as but
one example. Maybe the mod had a bad day, maybe the poster had a bad day.

> Place blame where it belongs: if someone says something sexist, someone
> noticing that they appear to be sexist is fully the responsibility of the
> person who said the sexist things.

The use of blame is not the problem, the problems are the standards used to
assign blame, the processes by which correctness of blame assignment is
judged, the consequences to the accused should blame be found justified, the
path to redemption for the accused, and the transparency of the whole process
to the accused and the community so that everyone can see that justice is
being done and mod power is not being abused.

You know, the sort of thinking that took millennia to evolve into our modern
legal systems. Show me a modern nation where exile is a legitimate outcome for
a legal infraction. As flawed as modern justice is, at least we've evolved
past the tribal thinking inherent to punishments like exile.

If this laundry list may seem impossible to satisfy given current sites, maybe
it is. Maybe that also shouldn't be an excuse to not do it. I've written about
this briefly before with some ideas why the problem exists, and how some of
the issues might be mitigated [1,2].

> What would the bigoted poster have to do to convince you they're no longer a
> bigot? (at least as far as internet posting goes)

I'm sure that rephrasing is much more comforting to you. In reality, the
situation is:

> What would a poster whose comment I and a few others have _interpreted_ to
> be bigoted have to do to convince you they're not a bigot?

Regardless of the fact that no amount of circumstantial evidence would amount
to proof, you've basically just doubled down on your assertion that people
_you consider bigots_ don't belong in your community, even if they're polite.
That's exactly the problem.

> Depends on what they said, of course. If they said something racist, I would
> expect an earnest acknowledgement of that, that racism is bad, etc. As well
> as evidence from posting elsewhere on Reddit that they've reformed.

Few if any mods are going to do such detective work (due to time, patience,
whatever), nor would many even entertain a conversation with a person that was
banned for such reasons.

And as the set of mods changes over time, and the interpretation of policies
changes over time, and with no transparency of community checking of mod
power, they end up banning people for ever more specific infractions, with no
real appeals process, thus creating the perfect echo chambers.

Every mod and mod system starts out with good intentions, but particularly
with communities covering divisive issues, this is where naive moderation
inevitably ends up. I don't think examples of unjust moderation are in short
supply.

[1]
[https://news.ycombinator.com/item?id=23263181](https://news.ycombinator.com/item?id=23263181)

[2]
[https://news.ycombinator.com/item?id=23265160](https://news.ycombinator.com/item?id=23265160)

~~~
TulliusCicero
> And you're trying to deflect from the problems by once again reducing a
> person's behaviour to simple labels that justify any actions to stop "bad
> people".

The surface area someone has with a random internet message board is usually
very limited. They post shit, and thus others there think of them as shitty.
Pretty simple.

That they have some deeper inner life is irrelevant in the context. We're not
their parents. We're not their therapist. We're not their social worker. If
they can't behave themselves, then they shouldn't be allowed to ruin the space
for others who can behave.

> so that everyone can see that justice is being done and mod power is not
> being abused.

Please, we're talking about volunteers on internet message boards here, not
judges passing judgment over someone's right to basic freedom. The only power
I have is to stop someone from posting in a subreddit -- and even that's not
really accurate, since creating a new account is fairly trivial.

I can make further posting in a particular subreddit less convenient for them.
That's it.

> What would a poster whose comment I and a few others have interpreted to be
> bigoted have to do to convince you they're not a bigot?

All labels, all words are subjective somewhere, somehow. To belabor this point
is mere pedantry. Of course our judgments and rules are subjective, that's
true for all judgments and all rules throughout all history, so why bring it
up?

> Few if any mods are going to do such detective work (due to time, patience,
> whatever), nor would many even entertain a conversation with a person that
> was banned for such reasons.

It's never come up for people banned for bigotry; they're usually unceasingly
hostile. We have unbanned a few people who were more general assholes (e.g.
flaming) and came back later, based on exactly what I suggested. They
apologized for being aggressive and insulting previously, we checked their
post history, they seemed to be productive posters, so we unbanned them.

This doesn't involve very much work, actually, partially because it doesn't
come up very often, and partially because skimming through someone's post
history to see whether they're generally earnest/helpful doesn't take very
long.

> And as the set of mods changes over time, and the interpretation of policies
> changes over time, and with no transparency of community checking of mod
> power, they end up banning people for ever more specific infractions, with
> no real appeals process, thus creating the perfect echo chambers.

> Every mod and mod system starts out with good intentions, but particularly
> with communities covering divisive issues, this is where naive moderation
> inevitably ends up. I don't think examples of unjust moderation are in short
> supply.

I agree that issues of mod transparency and power abuse are an issue.
Certainly you can find plenty examples of terrible mods around.

Nevertheless, we're almost invariably talking about volunteers here. Holding
them to a standard comparable to real life legal procedures is ridiculous.
Ain't nobody got time for that.

For at least more popular subs, which applies to one of the ones I mod, I've
come to the conclusion that truly fixing the situation is essentially
infeasible. The total amount of content and activity outstrips what a
volunteer team can handle in a good, consistent way; the best you can aim for
is "passable, most of the time", imo. (And just increasing the number of
moderators linearly causes too many coordination problems)

------
CogentHedgehog
I'm a reddit moderator for one of the 50 most active subreddits. My experience
as a reddit moderator has been that users break down into 4 groups:

1\. Normal people that mostly follow the rules. This is BY FAR the largest
group, and they're usually great. Transparency benefits them most: once they
know what they did wrong, usually they won't do it again. Some even admit they
deserved penalties or short bans -- although a few will defend their actions.

2\. Jerks: people who contribute constructively sometimes but also cause a lot
of drama. If moderators don't show clearly where they broke a rule, usually
they're going to protest that they did nothing wrong. Providing transparency
saves moderator effort because it avoids some of this back-and-forth.
Sometimes. These are almost inevitably the people that claim mods were
"abusing their powers" when they get banned for obvious things (swearing at
people, posting self-promotional content, etc).

3\. Dedicated trolls who go out of their way to break rules and cause trouble.
They are relatively rare but as a moderator really the only thing you can do
is ban them. Transparency won't make a difference one way or another, but it
can be hard to tell a jerk from a dedicated troll at times so you're better
off citing specific violations.

4\. Spammers & obvious bots. With the right defenses (automoderator rules,
spam filter, etc) most of these can be handled with minimum human effort.
Without these defenses, spammers/bots can flood you pretty quickly.
Transparency is irrelevant here.

FWIW I _always_ provide a reason when issuing a ban (citing the rule broken
and where they did it).

------
benjaminjosephw
I wonder whether there's a membership size threshold where, after crossing
that tipping point, the community implicitly looses some degree of
transparency whatever the moderation rules are. At some point the increasing
impact each decisions have will skew the potential motivations of members and
moderators. Intent become very difficult to judge meaning the degree of
transparency decreases.

I've seen so many comments from people about the changing nature of some
online communities as they grow. Perhaps capping memberships could curb the
increasing politicisation of user behaviour and the weight of any individual
action would be limited as a result.

~~~
elliekelly
I wonder whether it’s even limited to online communities. I’ve had similar
experiences with a small office and in a small student organization that grew
rather large.

------
HPsquared
Too much transparency can result in people "gaming the system" if the full
workings are known.

~~~
naasking
If the rules are solid, 'gaming the system' means they're rule-abiding
members. If there's a problem with that, then improve the rules and don't
blame the users. Reducing transparency because you're relying on tricks is a
great way for users to begin hating your mods.

~~~
jfengel
Hating mods is not necessarily a problem. If you boot a user, you don't care
about their opinion of you. Mods select the community, and their goal is a
comfortable place for that community.

Booting easy cases should be easy. There will also be edge cases, and almost
by definition there's no easy way to make the rules crystal clear for edge
cases. You can't be consistent because consistency doesn't exist. The best you
can hope for is to encourage users to avoid getting close to edge cases. For
those who are uncomfortable with that, there are many unmoderated forums
elsewhere.

It's all a question of the the site owner's goals. Maximum traffic and maximum
content are not everyone's goals. Often they are seeking a community to deal
with that community's concerns, and to shut out the overall problems of the
world that attract the most attention.

