
A machine learning approach to identifying thought markers of suicidal subjects - misnamed
http://onlinelibrary.wiley.com/doi/10.1111/sltb.12312/full
======
DINKDINK
A discussed version of the paper[1] says:

>A new study shows that computer technology known as machine learning is up to
93 percent accurate in correctly classifying a suicidal person

P(+|suicidal) = .93

I want to know P(suicidal|+) [2]

[1][http://neurosciencenews.com/suicide-machine-
learning-5448/](http://neurosciencenews.com/suicide-machine-learning-5448/)
[2][https://en.wikipedia.org/wiki/Bayes%27_theorem#Drug_testing](https://en.wikipedia.org/wiki/Bayes%27_theorem#Drug_testing)

~~~
d--b
Yes! That is what is scary as fuck with these "magic classification"
techniques. The more reliable they are, the more people dismiss false
positives. This, applied to criminality is totally 1984.

~~~
koube
For those that don't know, "this applied to criminality" already exists and
has some problems.

Propublica wrote a story criticizing it's use:
[https://www.propublica.org/article/machine-bias-risk-
assessm...](https://www.propublica.org/article/machine-bias-risk-assessments-
in-criminal-sentencing)

The company responded with a rebuttal:
[https://www.documentcloud.org/documents/2998391-ProPublica-C...](https://www.documentcloud.org/documents/2998391-ProPublica-
Commentary-Final-070616.html)

And Propublica has a counter-rebuttal:
[https://www.propublica.org/article/propublica-responds-to-
co...](https://www.propublica.org/article/propublica-responds-to-companys-
critique-of-machine-bias-story)

~~~
aub3bhat
Propublica has an "interesting" definition of "fairness". It turns out there
is a beautiful mathematical result that shows that the different definitions
of "fairness" are inherently incompatible with each other and cannot be
satisfied at the same time.

This is the best article that summarizes the controversy on this subject:

[https://www.washingtonpost.com/news/monkey-
cage/wp/2016/10/1...](https://www.washingtonpost.com/news/monkey-
cage/wp/2016/10/17/can-an-algorithm-be-racist-our-analysis-is-more-cautious-
than-propublicas/)

Paper by Jon Kleinberg and Sendhil M. and Manish Raghavan on this topic
[https://arxiv.org/abs/1609.05807](https://arxiv.org/abs/1609.05807)

------
oelmekki
I can appreciate the effort made in the research, but classifying possible
human behaviors is exactly a field where I don't want machine learning to go.

There is behind machine learning both a phantasm - artificial intelligences
that can guess what humans can't - and a reality - it's just statistical
models that are never 100% accurate because, well, that's an attribute of
statistics.

Those two elements combined and applied to behavior classification sounds like
a scary thing, not unlike the kind of errors eugenics made, over trusting
their science to apply it on social facts, totally discarding empathy and
individual context.

~~~
golergka
But humans already make those judgements, and when they make errors, it's
tragic too. If a psychologist erroneously decides that you're a menace to
yourself and should be institutionalized, would it make you feel better that
it was a human that made this decision? Even if AI would make the same error
with a much lower probability?

~~~
tomjen3
I can't speak for OP, but that is one reason I want to end all involuntary
treatments. Getting locked up in a psych facility is a nightmare to me.

But you can at least sue a doctor and get his license revoked. It is much
harder to sue an algorithm.

~~~
golergka
> I want to end all involuntary treatments

Are you sure you have thought through all implications? I mean, getting locked
up sounds pretty scary, but there are a lot of other scary stories that can
happen if you really ban the whole concept of involuntary treatments.

------
aub3bhat
This is subject specific journal reporting a "machine learning" study with
just ~400 odd patients dataset. I would not take these results seriously. The
accuracy number cited is meaningless, and other than being a bullet point on
someone's CV it does not have much of a value.

85% Accuracy or 0.8 AUC score is pointless, unless compared with current state
of art e.g. having psychologist give an opinion and comparing against the
correct population, e.g. all patients who get interviewed as opposed to a
balanced set.

------
nl
Based on [1], I think I can build a model that outperforms this:

    
    
        return False
    
    

[1] "An estimated 9.3 million adults (3.9% of the adult U.S. population)
reported having suicidal thoughts in the past year."
[https://www.cdc.gov/violenceprevention/pdf/suicide-
datasheet...](https://www.cdc.gov/violenceprevention/pdf/suicide-
datasheet-a.pdf)

~~~
didgeoridoo
But the patient pool was drawn largely from psych units, where your (very
efficient!) classifier might not work as well. The real headline should
probably be "identifying suicidality _in patients already admitted to
psychiatric institutions_ ".

------
rocqua
That 85% 'accuracy' is ambiguous.

I couldn't get into the article, but if that number includes false positives,
that's not really practical. Suicide is really rare, so it could just be that
it picks a 15% of the population that includes al suicidal people. That means
the _vast_ majority of those positives are false positives.

Basically, its the precision and recall that matters.

~~~
denzil_correa
I'm not even sure why people report ONLY accuracy as an evaluation metric. It
means nothing - appropriately weighted F scores, precision, recall give more
insight into the system.

~~~
dasboth
Completely agree. Although if your binary classes are perfectly balanced then
accuracy is a valid metric, but that's rarely the case.

I guess the problem is that an ROC curve wouldn't create flashy headlines.

~~~
denzil_correa
> I guess the problem is that an ROC curve wouldn't create flashy headlines.

You can, you just need to put in a little bit of work. We don't need to
specify the exact accuracy in the title.

------
thinkMOAR
"classify 379 subjects recruited from two academic medical centers and a rural
community hospital"

So they recruit people from hospitals and check if they ever thought about
suicide/self death? Hmm okay nice test group....

Everybody, healthy or not healthy, in life will think about how it would be or
how they would end themselves, it's human nature. Though people in hospitals,
potentially sick, generally older, for sure will have had those thoughts,
because they often had issues that makes them lets say less enthusiastic about
life?

~~~
thret
Surely it is more normal to fantasize about a Tom Sawyer funeral than actual
suicide?

How it would be/how to do it - completely different thought processes.

~~~
thinkMOAR
I don't know, i'm no expert, and certainly not a judge of what is normal and
what is not. I do know that 'normal' isn't the same for everybody.

But if you get a serious event in life (example, death of family member or
other loved one), that makes you doubt if you still enjoy life or whats the
use of 'finishing' it till your body gives up.. those thoughts will occur,
sometimes swiftly sometimes occupying your brain for longer. Do keep in mind,
having thoughts and actually acting upon those thoughts are two different
things.

Though pondering if tom sawyer has brass handles on his coffin.... Maybe if i
was reading tom sawyer, yes that would probably be a thought that came before
others. :) Have to admit, i never read that book, so not entirely sure why one
would fantasise about his funeral.

~~~
colomon
Tom Sawyer is sitting in the rafters watching his own funeral. There's more
than one sequel, and he survives all of them.

------
delegate
Suicide is a big taboo in the 'western' christian society, but there are (and
have been) lots of cultures were suicide is looked at differently - as
something honorable, good, etc.

Even our culture has different interpretations for it - eg suicide vs
sacrifice.

We morally condemn the former but elate the latter, even though the outcome is
the same - a person dies.

Metaphysically, the meaning of suicide is given by the perceived meaning of
death and the [lack of] belief in some sort of afterlife.

Is it a sin or not ? If there's an afterlife, will you be punished for
committing suicide? What about the loved ones ? They will judge..

But for example, if you knew that this life is actually a realistic VR
simulation that you've entered into, then suicide would be perceived as a sort
of 'ESC' key - a way out of the simulation. Like exiting a game.

If a person's circumstance in life is such that the person is bound to suffer
until death (eg. disease, mutilation or loss of everyone), then suicide might
be looked at as a sort of release - a good thing.. There's the whole
controversy regarding assisted suicide..

Then there's the sacrifice - going into battle screaming is a form of
attempted suicide combined with attempted murder. At the end of the day, the
battlefield participants are eventually split into killers and those who
committed suicide. From this perspective, going to war is collective
[attempted] suicide.

War is a form of temporarily suspending the moral rules we obey by (do not
kill others or self) and people gladly participate in both killing others and
themselves.

I guess my point is that tfa is looking at a very narrow spectrum of 'suicidal
people' \- suicide is a lot more prevalent than that and it's practiced not
just by people with mental illness..

~~~
Chris2048
> suicide is a lot more prevalent than

If you are talking about war, then no, that's not suicide. The aim is not to
die, even though it might be likely.

Even suicide bombers don't have death as their primary aim - if they achieve
the same result without dying, they would.

~~~
webmaven
_> Even suicide bombers don't have death as their primary aim - if they
achieve the same result without dying, they would."_

That depends on how you define "the same result". The assumed (in some
cultures) moral superiority of the suicide bomber regardless of the legitimacy
of the target can have considerable PR value. In a way that somewhat resembles
a ponzi scheme, those at the very top may reap higher rewards, and so choose
to elevate the _tactic_ into a _strategy_.

~~~
Chris2048
Consider "the result" to include religious goals based on a belief in the
afterlife. If they could obtain piety some other way they could - as opposed
to wanting to end life as it is no longer enjoyable.

~~~
webmaven
And you now know why some groups (religions, sects, etc.) use (or have used)
martyrdom (including the suicide bomb variety) as a tactic, and others have
not:

More than one goal exists for each group, many of these goals are not mutually
compatible, and different groups arrive at different mixes of tactics in
pursuit of their goals and to resolve (or ignore) the various contradictions
and incompatibilities in different ways, and these mixes change over time in
reaction to a changing environment, including the S&T employed by other
groups.

IOW, don't expect the strategies or tactics that groups use to settle on any
sort of Nash equilibrium.

------
belovedeagle
Great, now society has an excuse to put _scary_ people away with essentially
no due process thanks to the magical label of "suicidal". For those living
with mental illness, reasons to despair of safety and fair treatment are up,
up, up. Seeking help will never have seemed so dangerous.

~~~
0xcde4c3db
Not to mention that even if you're admitted voluntarily, you aren't
necessarily allowed to leave when you want, even beyond the oft-quoted 72-hour
hold. Legally you might be entitled to leave, but the facility can basically
force you to stay by denying the resources needed to fill out the discharge
paperwork.

------
jonplackett
So does this mean it misses 14 suicidal people for every one it finds. Or that
it classifies 14 people suicidal who aren't for each one it finds? Not super
great either way!

~~~
truth_sentinell
If it is what it claims, it's probably better than any human

~~~
denzil_correa
Is it?

~~~
DanBC
Suicide risk assessments are mostly worthless. They're done because you need
to show you've done something to try to identify people at risk of suicide and
to have tried to prevent that death.

------
lordnacho
And how accurate are traditional methods? Seems to be behind a paywall.

Exciting, whatever the answer is. It's awful when someone commits suicide, and
if there's some automated way to test it, that would help a lot. Also less
intrusive. Who knows how many people need help but are too shy to reach out.

~~~
sudhirj
This is also one of those perfect applications where it's quite okay to have a
false positive.

~~~
DINKDINK
"Hi this is an automated message from Facebook alerting you that the police
have been called to your husband's/friends/boss's location to prevent him from
killing himself"

"Actually just kidding, we had a little SNAFU. Thank you for being a loyal
Facebook user, we assume this had a neutral effect on your life"

~~~
aub3bhat
Actually Instagram does send a message to the user if enough people mark the
user in danger of being harmed. Similar with Google where if someone searches
for methods of committing suicide the user will get a number for helpline as
first result in bold letter.

[https://techcrunch.com/2016/10/19/instagram-tackles-self-
har...](https://techcrunch.com/2016/10/19/instagram-tackles-self-harm-and-
suicide-with-new-reporting-tools-support-options/)

------
B1FF_PSUVM
"The thought of suicide is a great consolation: by means of it one gets
through many a dark night."

\-- Friedrich Nietzsche

(Not exactly the most balanced bike in the shed, but some people like it that
way ...)

~~~
aangjie
I second this.. The idea that there's an escape route is a relief, when you
look at a big problem(that you have to solve) and your brain panics(it
definitely puts the problem in perspective). However, i'm still divided as if
having this is a good habit for self-growth or just an excuse to ignore the
problem and work around it.

~~~
pessimizer
It might be healthier to not hold on to suicide as a last escape from failure,
and just accept the possibility of life after failure. The thought process
that you're illustrating here seems like part of what's driving family
annihilators: the fathers who would rather kill you than fail you.

------
staticelf
My friend was suicidal and we tried to get help from the authorities but got
none. None at all. He later committed suicide for real and this makes me
wonder why it matters to discover it if you're not even helping people that is
asking for it?

~~~
vinay427
I don't understand your viewpoint. This work isn't being done by your local
authorities and isn't an alternative action to their providing help.

It provides clear potential benefits (if it performs as well as implied) for
numerous people at risk in the world, which is a substantial goal that can be
at least be applied in places that do have better forms of assistance. Perhaps
if your local authorities do eventually implement better protocols, this will
be of use for future suicidal residents there as well. I understand that
you're trying to raise awareness for the lack of help from authorities, but it
seems awfully dismissive of work that seemingly has nothing to do with the
first problem.

------
ClassyJacket
What do you test against to see if someone is indeed suicidal? Self reporting,
or do you wait around till they die and see if it was due to suicide or
something else?

~~~
Scarblac
You send them a suicide pill and see who takes it.

Just kidding, that wouldn't be scientific. Half of the pills are actually
placebos.

------
chestervonwinch
From Figure 1 (which illustrates some ROC curves):

> The gray line is the AROC curve for a baseline (random) classifier

The AROC is the area under the ROC curve, not the curve itself.

Also, the scale on Figure 1 for the x-axis (corresponding to the False
Positive Rate, or sensitivity) go above zero and below one, which doesn't make
any sense.

------
w_t_payne
Suicide is a _very_ serious health issue that affects an astonishing number of
people worldwide. Any advance that helps detection and treatment is to be
welcomed.

------
kristopolous
I'd love to take this test.

I don't really know the answer to if I am or not.

For instance, since everyone dies, I'd certainly like to die in a fantastic
and wonderfully improbable way. Or if a doctor told me I was going to die
soon, I'd accept it and start doing really irresponsible and dangerous
things...

I wouldn't be like "oh dearest me, I must clasp on to life longer." Is that
suicidal?

~~~
costcopizza
Not even close to suicidal.

------
sslayer
Thought conformity at our doorsteps.

