There are risks that people can understand and appreciate up front.
There are risks that simply are not fully conciously appreciated until they've been experienced directly.
There are risks whose cumulative effects only build with time, often taking years, even decades, to fully manifest.
And there are risks which fundamentally change the capacity of the affected to even recognise or admit their existence or severity.
PTSD is all of these.
Self-monitoring, encouragement to seek help, and provision of a nonclinical "wellness coach" is grossly insufficient.
The companies, and in this case, contractors (Accenture), providing such services, as well as outside oversight entities such as government regulators, physical and mental health services, unions, and insurers, know (or damned well should), can monitor, and impose regulations, limits, risk premiums, compensation and settlement systems, and collective bargaining powers, for such work.
Business is a risk-externalising engine. Some risks absolutely need to be fully internalised. This is a prime case.
NB: I've worked at various times as a moderator, on spam-detecting and reporting systems which entail seeing some unpleasantness. And on a large social media network where I was tasked with removing flagged images from the storage network. The flagging process hadn't been explained in any detail, and as I would be deleting millions of items of user content, I spot-checked a small handful to confirm the flagging was accurate.
After no more than a half dozen (and probably fewer) I simply didn't want to see any more -- the image from over a decade of a young girl still lodges in my mind, though I've never wiped, nuked, and shredded files on disk harder.
I'm satisfied with the fact that the scripts I created wiped all that content from our and our CDN's storage not in the weeks or months of the initial estimate (our CDN vendor apparently had never considered widespread deletions, or experienced them), but less than two days, with verification. Never heard any reports of unwarranted deletion either.
Somewhere else within the organisation, unknown to me, others had seen and verified those images. I think of who they might have been and the impacts on them.
I really don't understand how images or even videos are causing PTSD. Maybe it's because I grew up on 4chan in the age of goatse, but shocking content is a tiny fraction of how disturbing real life can be.
I understand that looking at disturbing content is going to have some impact on people, and that they may not be able to evaluate that risk properly, but it's nowhere near what every nurse, doctor, firefighter, police officer, soldier, and even teacher will live through over the course of their career.
I've been diagnosed with PTSD after seeing some deeply disturbing medical shit. I'm definitely impacted for life, but there were doctors and nurses there too, and they're impacted too.
Maybe I've been on the internet too long, but there's a massive delta, at least in my view, between seeing a disturbing video and living through real-life trauma. I'd take the video moderation job over being an ER nurse any day.
Forgive the bluntness, and speaking as someone who knows 4chan all too well, and is a paramedic:
goatse is one thing. We're more talking high definition videos of people being beheaded by Mexican drug cartels, people being held down while dogs eat their genitals. Videos of toddlers being raped.
You can't really compare that to a fairly mundane, if explicit, naked man showing his anus.
Also, as a paramedic, and speaking for many that I know (though not all, of course) - trauma is rarely (or a lot less) PTSD inducing. Gruesome, gory, sure, but in the end it's all blood and tissue. What gets to most of the people I know is the emotional violence - being called to child sexual assault cases, accidental deaths, things like that, that take the toll.
Thank you for your service to the community as a paramedic. I have no idea how you people do it.
As someone who has exposure to that world, how do you do deal with it? Is there some kind of training or protocol or therapy that's built in to your job that's different than that of the moderators? Does any of the mitigation even work?
For every one of those beheading videos, someone has to actually go collect the head. I would think that must be orders of magnitude more traumatizing than skipping through the video of it happening enough to flag it.
It's a very good question, because for the longest time, paramedics/EMTs/firefighters -were- expected to just "suck it up".
Now, with increases in the protection of our PPE (bunker gear, etc.) and other knowledge, there are less line of duty deaths due to accident or illness (typically cancer, though that's still a big one) - now the biggest cause is suicide, mostly as a result of PTSD.
There's a documentary that was funded in part by Denis Leary called "Burn: A Year in the Frontlines in the Battle to Save Detroit", talking about fire departments there. One of the veterans says "I wish my mind could forget what my eyes have seen".
Around here, the PNW, at least, there's a huge movement toward handling it proactively, access to counseling, therapy, hotlines, and as importantly as anything else, active efforts to remove the stigma associated with things.
We used to do CISDs (critical incident stress debriefings), which are now largely discredited - essentially "put everyone in a room and 'make' them talk about how they feel after a bad call, whether they want to or not", but now, more and more departments are hiring full-time mental health professionals. One near me has someone who specializes in PTSD, and another who works with sleep regulation (all those alarms in the middle of the night), and alcohol/drug use.
You don’t have to understand it for it to be real. The shit they see is way way worse than goatse. For example, a helpless screaming crying child (who may look like the moderator’s own child) getting raped by someone they trust. That and worse gets uploaded, and it’s their job to delete it.
Obviously being in the room with the victim is probably worse, but that doesn’t mean seeing this kind of thing regularly isn’t still enough to do some damage to some people.
Is there some expectation that they sit and watch the whole thing end-to-end with headphones on?
I'd think skipping around and finding a few frames would be more than enough to be like "nope, that's a bad one", flag it, and move on.
I would think the police and prosecutors that follow up would have to see and document that stuff in way more detail, and nobody's talking about their PTSD.
I'm not saying it's not bad, but I think the initial moderation flag has to be one of the least bad of all the people in the pipeline that will have to deal with those cases.
Dismissing one atrocity faster only means you get to the next one sooner.
You have no agency over the inflow, and there's little evidence that what you're doing has an effect -- the stream never stops. Metrics for performance and benefit of the effort as a whole are hard to construct and communicate.
As a moderator, it's just shitty shit often being done by shitty people, posted by other shitty people, again and again and again and again and again and again and again.
There's no diseased root to hack off. And if you do happen to find one, another grows in its place -- a literal Medusa's head.
> I would think the police and prosecutors that follow up would have to see and document that stuff in way more detail, and nobody's talking about their PTSD.
Uhm, it's actually super well known within law enforcement that law enforcement is emotionally exhausting, so much so that specific units are separated out and the officers rotated to avoid burnout. (Special victims Units, you may know them as)
> I would think the police and prosecutors that follow up would have to see and document that stuff in way more detail, and nobody's talking about their PTSD.
PTSD has long been associated with combat stress, and has gone by various names over time -- "shell shock", "combat fatigue", even "nostalgia" origially referred to a mental disability.
Burnout or occupational stress, now being recongised as a disorder by the WHO, seems to me strongly similar. And the first documented description of it comes not from those directly exposed to stress, but in trying to address it -- staff and volunteers at a free clinic for drug addicts, in 1974, by Henry Freudenberger.
For both PTSD and burnout, the failure of those not experiencing it to grasp and understand the severity, complexity, causes, and sheer insidiousness of the condition is a large part of the frustration of experiencing it.
The idea that burnout is a form of PTSD has occurred to me. This being the Internet, there are others who argue similarly, though I don't know that we're correct:
- You have agency over whether or not you're going to be exposed.
- You can chose to stop viewing, without consequences (generaly).
- You can block the sites entirely, in many cases, through various screening or filtering tools (PiHole, dansguardian, Squid Proxy, various parental control tools, etc.)
Content moderators are looking at what they're faced with, all day, every day, day in and day out, and their jobs, livelihood, rent/mortgage, food, and family depend on this.
That's a wholly different environment than occasionally seeing a gaped anus.
Typically people in emotionally traumatic jobs aren't going to be spending all day, every day, doing nothing but witnessing trauma. There's downtime - a police officer could go weeks to months at a time dealing with only relatively routine things.
The people doing moderation are spending all of their time, baring a few breaks, looking at stuff that other people have already flagged. That's going to take an emotional toll on you.
> Maybe it's because I grew up on 4chan in the age of goatse, but shocking content is a tiny fraction of how disturbing real life can be.
I'm surprised you can say that. I also frequent 4chan and there have been tons of times that I have to close a thread within the first few posts because I do not want to see the content in them. There have also been times when something I've seen in a thread has lingered in my head for hours after, making me uncomfortable and definitely impacting my mood.
> it's nowhere near what every nurse, doctor, firefighter, police officer, soldier, and even teacher will live through over the course of their career.
Blood, gore, and dead people are only some things that could trip people up. You have to understand that many of these flagged posts are actually engineered to cause outrage or to trigger the primitive, emotional parts of your brain. Even pure audio alone can scar you - if you're really interested, and I do not recommend this, and I'm going to put a large TRIGGER WARNING here so you can look away before I describe this, look up the dashcam audio of a family traveling down a highway when a 2x4 from a nearby pickup truck falls off the bed and flies through the window, killing the mother instantly. Listen to the screams of both children and the father driving as they realize their family has just been destroyed. It will haunt you more than any "simple death" can because it interacts directly with the emotional core that makes you human.
Honestly, I believe only sociopaths can do a job like this and make it out unscathed.
There are risks that simply are not fully conciously appreciated until they've been experienced directly.
There are risks whose cumulative effects only build with time, often taking years, even decades, to fully manifest.
And there are risks which fundamentally change the capacity of the affected to even recognise or admit their existence or severity.
PTSD is all of these.
Self-monitoring, encouragement to seek help, and provision of a nonclinical "wellness coach" is grossly insufficient.
The companies, and in this case, contractors (Accenture), providing such services, as well as outside oversight entities such as government regulators, physical and mental health services, unions, and insurers, know (or damned well should), can monitor, and impose regulations, limits, risk premiums, compensation and settlement systems, and collective bargaining powers, for such work.
Business is a risk-externalising engine. Some risks absolutely need to be fully internalised. This is a prime case.
NB: I've worked at various times as a moderator, on spam-detecting and reporting systems which entail seeing some unpleasantness. And on a large social media network where I was tasked with removing flagged images from the storage network. The flagging process hadn't been explained in any detail, and as I would be deleting millions of items of user content, I spot-checked a small handful to confirm the flagging was accurate.
After no more than a half dozen (and probably fewer) I simply didn't want to see any more -- the image from over a decade of a young girl still lodges in my mind, though I've never wiped, nuked, and shredded files on disk harder.
I'm satisfied with the fact that the scripts I created wiped all that content from our and our CDN's storage not in the weeks or months of the initial estimate (our CDN vendor apparently had never considered widespread deletions, or experienced them), but less than two days, with verification. Never heard any reports of unwarranted deletion either.
Somewhere else within the organisation, unknown to me, others had seen and verified those images. I think of who they might have been and the impacts on them.