
Computer personality judgments more accurate than those made by humans: study - seagullz
http://www.pnas.org/content/112/4/1036
======
Eridrus
This headline is a bit misleading.

They say their models are more accurate, but more accurate with what data? It
turns out models are better than humans at predicting how people rate
themselves on a personality model. So humans are rating their opinion, vs
algorithms that are predicting what people would say about themselves.

So I wouldn't take those numbers at face value.

The external validity comparisons are more compelling, but still, its telling
that self-reported data is better at predicting self-reported events.

It's entirely possible that computer based models are better at knowing us
than humans, particularly since algorithms can pay far closer attention to us,
but there's plenty of literature showing that we have the capacity for self-
deception, so I'm not sure how good a ground truth you can have here.

~~~
sova
Haha, this headline is not a little bit misleading, it borders on the absurd!
One might as well do a study on how "Computers are better at enjoying food
than humans" but it is interesting, to use machine learning to see what we can
gather purely from social media footprints versus what people see through one
anothers' eyes. It makes sense, their abstract, when it says that the computer
aided analysis is "more accurate" only insofar as many people will simply rely
on their echo chamber for value assessments, and will therefore miss out on a
key ~pseudo-objective viewpoint grounded in a more global image, which is what
the computers can account for, and what your close friends cannot.

------
kosei
"This study compares the accuracy of human and computer-based personality
judgments, using a sample of 86,220 volunteers... Computer predictions based
on a generic digital footprint (Facebook Likes) are more accurate (r = 0.56)
than those made by the participants’ Facebook friends using a personality
questionnaire (r = 0.49)"

The fact that this is even close (let alone higher for computers) is pretty
incredible. It is also quite scary for anyone with a public profile and speaks
to the potential power that an organization with bad intentions or ulterior
motives could have (ex: Cambridge Analytica).

~~~
21
If you think about the Foundations series, it was basically about this:
studying the psychology of masses of people and predicting (and influencing)
their behavior.

~~~
sova
Wow, that's really great to read. I did not get this idea from the first
Foundation much, although it was a little more prevalent in the second book. I
got something rather different from Foundation: the idea that one with
omniscience shares what is timely and relevant for the benefit of all.

------
panarky
This 2015 paper is in the news now because of the "information operations"
campaign that abused Facebook data to influence the 2016 election.

Authors of this paper communicated with the people behind the information
operations campaign [0, 1], but it's not clear to me how closely they
collaborated.

[0] [https://motherboard.vice.com/en_us/article/mg9vvn/how-our-
li...](https://motherboard.vice.com/en_us/article/mg9vvn/how-our-likes-helped-
trump-win)

[1] [https://www.theguardian.com/news/2018/mar/17/data-war-
whistl...](https://www.theguardian.com/news/2018/mar/17/data-war-
whistleblower-christopher-wylie-faceook-nix-bannon-trump)

------
rdtsc
> Our findings highlight that people’s personalities can be predicted
> automatically and without involving human social-cognitive skills.

I had noticed with with Netflix. After telling it enough movies I liked (over
say 100) it pretty much figured me out and it knew me better than I knew
myself. In other words, I'd see a movie description, watch the trailer and
think "Nah, I wouldn't like it". But Netflix was telling me I'd like it, quite
often after I watched it I did end up liking it.

On another note, I always tried to steer away from taking personality tests. I
have a strange belief that once I know the "official" results it would somehow
restrict my what I would attempt later in life. I'd think "Oh well your
personality officially doesn't match, you shouldn't try that".

~~~
gleenn
But Netflix has way more information than you do about the movie before you've
seen it, namely you get a 30 second trailer, and it gets how many million
people liked it. That doesn't show it is better than you, just that it has a
huge information advantage.

------
noobermin
For some reason, I feel like their control wasn't the best thing to compare
to. People filling out personality questionnaires might not compare to looking
at facebook likes. May be a better control would be letting humans look at the
facebook likes themselves and compare their prediction based on that to that
of computers looking at the set of facebook likes.

If they want to stick to the questionnaire, then train the computer model on
the questionnaire too and let's see who wins. Giving one side more powerful
data and giving the control just a questionnaire is like giving a robot a
machine gun and a human a knife and being surprised who wins.

------
cosmic_ape
Have a feeling that if one used something deeper than just lasso regression,
much better results could have been obtained. This is very similar to a
recommender system, or a text classification problem, and there much better
than lasso can be done. The data is sort of open, in principle, btw.

But nevermind what can be predicted from the likes. They say that coworkers
have r=0.27, while spouse has r=0.58. This looks like a serious problem with
the test more than anything else.

This may be interpreted as "cowokers don't know you (and don't care...)" kind
of stuff. But coworkers often have a comparable amount of contact with a
person vs spouse. Alternatively, one could also interpret this as there is no
"the one true personality", but different personalities in different
environments (at least when we define personality as a result of a test). And
so coworkers are not wrong, they just see a different picture.

And there might be other possibilities.

In fact, one should try to predict the coworker's (and spuse's, ect...) scores
of a person from person's likes, in addition to trying to predict person's own
assessment.

~~~
fastball

      Coworkers often have a comparable amount of contact with a person vs spouse.
    

You're not married, are you? The above statement is just so incredibly off-
base I honestly don't know where to begin criticizing it.

~~~
cosmic_ape
I understand what you mean. But the personality properties like agreeableness,
extroversion, openness, should reveal themselves through behavior over time.
Otherwise, what's the point in them? And, say, hi-tech work seems to provide
enough opportunities, say over a year, to reveal all of them. This is what I
meant by "contact".

Besides, have you not heard the "we do not spend enough time together" idea
from a spouse? Not to glorify it, nothing good about it, but it seems to be
the reality for more people than it should be.

------
vages
What worries me is not the unrealistic notion that a single algorithm will
ever replace human emotional judgment (it will not): It's the amount of
frustration that blind faith in such algorithms will cause before we settle on
supplementing them with common sense again.

> (...) the accuracy of the personality judgment depends on the availability
> and the amount of the relevant behavioral information, along with the
> judges’ ability to detect and use it correctly (1, 2, 5). Such
> conceptualization reveals a couple of major advantages that computers have
> over humans. First, computers have the capacity to store a tremendous amount
> of information, which is difficult for humans to retain and access. Second,
> the way computers use information—through statistical modeling—generates
> consistent algorithms that optimize the judgmental accuracy, whereas humans
> are affected by various motivational biases (27). Nevertheless, human
> perceptions have the advantage of being flexible and able to capture many
> subconscious cues unavailable to machines. Because the Big Five personality
> traits only represent some aspects of human personality, human judgments
> might still be better at describing other traits that require subtle
> cognition or that are less evident in digital behavior.

This last sentence is important: Looking at scores from quantitative
psychological models, it is easy to forget that although a model has some
predictive ability, the thought of a _complete_ model is an illusion. For
example, low and high IQ scores have some merit in predicting people's success
in life, but anyone who knows several people claiming to have high IQs will
observe that such people's ability to actually accomplish something with their
claimed ability varies greatly.

I don't doubt that psychological models can be an efficient way of filtering
through many potential candidates for a position, deciding what ad to show
someone or predicting the chances of someone becoming a drug addict. They
could even help us see past some of our cognitive biases when making a
decision. But I don't think that we will ever, at least in the next hundred
years, reach a point where most people let an algorithm trump their own
judgment in decisions where the emotional stakes are high. For example, in
choosing a life partner, dating services will probably use such algorithms to
filter the candidates, but the user will probably make the final call on who
to settle with.

After all, personality tests have been around for decades, but most employers
use a good old fashioned face-to-face interview to choose the person to fill
the position.

------
StanislavPetrov
Studies like this invariably raise the question, are computers getting very
good about making judgments or are humans just piss-poor at making judgements?

~~~
restuijs
My colleagues and I discussed this paper when it was published. Like lots of
AI research in this area, it picks a human strawman and is sort of misleading
in this regard. The human ratings are poorly designed and substandard in many
ways.

It's still interesting, but there are lots of issues being swept under the rug
in this area.

------
bryanrasmussen
How easy is it to game a personality judgement made by a computer vs. one by a
human?

------
neilwilson
And what about those who don't indulge in the Facebook Likes game?

~~~
PeterStuer
Hard to escape when each webpage you visit sends signals to the various
social/surveillance/marketing companies. There is some legal push-back [1],
but e.g. Facebook considers it 'industry standard' practice and tries to
appeal/wiggle out of these data collection restrictions where it can.

[1]
[https://www.theguardian.com/technology/2018/feb/16/facebook-...](https://www.theguardian.com/technology/2018/feb/16/facebook-
ordered-stop-collecting-user-data-fines-belgian-court)

