

Computer program can detect depression in blogs and online texts - mudil
http://www.medgadget.com/archives/2010/06/detecting_depression_in_blogs.html
Researchers have developed a program that detects depression in text without obvious terms like "depression" or "suicide". In a sample of 200 positively identified texts out of 300,000 which were screened by the program, there was a 78 percent agreement between the program and a panel of psychologists.
======
rada
_However, so far no efforts have been made to actually correlate the program's
judgment with the mental state of the writers._

So, it's never been tested? Say what?

~~~
raganwald
I read the post to mean that the computer program correlated highly with the
opinion of a panel of human experts when both are given the writing to study.

What hasn't been established is the correlation between the program and the
mental state of the authors as measured using more conventional methods.

~~~
rada
Yes, I saw that too, I am a big fan of reading before commenting :)

However, I don't think substituting one human error (that of the programmers)
by another human error (that of the psychologists) proves anything. You could
say that the second group is more qualified as domain experts but it still
doesn't make up for complete lack of real life testing. If it did, you
wouldn't have to conduct human trials for drugs - you'd just get a bunch of
doctors to say that they think the drug works.

The fact that the program purports to detect depression, something that
amounts to artificial intelligence, makes me especially skeptical. Not to
mention the software author's example of using combinations of the word
"black" with terms describing "sleep deprivation". By this definition, anyone
who blogs about, say, New York Fashion Week, is depressed.

~~~
samatman
This problem could be mitigated by testing the panel, rather than testing the
internet sources.

It's a non-trivial problem to get a representative sample of bloggers to
submit to a valid screening for depression. On the other hand, collecting a
data set of writings by depressed vs. non-depressed people is easier, such
data being (I suspect) readily available.

If the panel of experts can correlate between depressed writing and depressed
people, then there is validity to their correlations of the writing of unknown
people, without having to screen the unknowns separately for depression.

I suspect even a panel of non-experts could screen writing into depressed and
non-depressed. Humans are pretty good at detecting malaise in others.

------
jgrahamc
What's the false positive rate? Looks like the false negative rate is 22%.
Given the low rate of depression I'd be interested to know if this is in any
way useful as a diagnostic tool.

Oh. Actually in take that back. We have no idea what the FN rate is. They
haven't tested it at all.

------
chronomex
Some time ago, a friend of mine trained a Bayesian spam filter on the
Livejournal latest posts page, and bucketed posts into "angsty" and "not
angsty". It worked surprisingly well.

~~~
stcredzero
This might be a good way of auto-classifying teenagers into market-segment
cohorts. It seems to me that one should be able to recognize subculture as
well as socioeconomic level. Hmmm, this could actually be worth a heck of a
lot of money!

------
tsiki
I'm not sure how this is newsworthy - they're basically just detecting texts
with certain words in them. It sounds impressive, sure, but so does credit
score when you describe it as an algorithm that can predict if people will get
into financial trouble.

------
mikecane
How soon before there's software that can "discover" who is gay, who is bi,
who votes this way or that way, whose language is creditworthy, et al? I await
the day when all of this is discredited.

~~~
showerst
It's a relatively simple question whether a certain trait can be 'ascertained'
by these systems, and to what accuracy, in a given context.

There are a number NLP programs that can detect a writers gender from text at
a much greater accuracy than raw chance, especially in a limited domain. The
problem is that the world is a big, messy domain.

There's no reason in principal that other traits wouldn't be 'detectable'
(although I doubt sexuality), but the problem is getting a good set of
training/test data in context, and what your acceptable threshold for accuracy
is.

I'd be willing to bet that people whose text messages contain lots of U's and
L8R's probably have worse credit than those who text in proper English, but
there are much better ways of doing a credit check. If that's only accurate
to, say, 100 FICA score points it's probably not worth using it.

I wouldn't be too worried about big brother(s) using semantic analysis to read
your mind, but don't discredit the fact that language patterns encode certain
cultural information that is identifiable, or at least correlated with
behavior.

~~~
mikecane
>>>or at least correlated with behavior.

Oh really?

Briton Convicted for ‘Menacing’ Tweet Against Robin Hood Airport
[http://thelede.blogs.nytimes.com/2010/05/10/briton-
convicted...](http://thelede.blogs.nytimes.com/2010/05/10/briton-convicted-
for-menacing-tweet-against-robin-hood-airport/)

