
Human psychology and behavioral studies overlook 85 percent of people - headalgorithm
https://www.sapiens.org/culture/weird-cultures-human-nature/
======
mettamage
I read the article. It barely scratches the surface of the issue, I was mad as
a psychology student that this was the state of our knowledge when I read it
in 2013 and didn't know about the publication crisis. The actual academic
publication is really interesting and I'd recommend it as a read on whatever
commute you're on.

Here is the link:

[https://www.ssoar.info/ssoar/bitstream/handle/document/42104...](https://www.ssoar.info/ssoar/bitstream/handle/document/42104/ssoar-2010-henrich_et_al-
The_weirdest_people_in_the.pdf?sequence=1)

Years after pondering this article, I felt I had figured it out: psychology is
secretly the study of particular subset of 18 to 21 year old American women.
The ones who study psychology because it's a female dominated study and all
those psych students need their credit and 'participating' in research is part
of it. Most of them are American because psychology is a bigger thing in the
US than in Europe, or at least it seemed to be regarding well-known theories,
so I presume most research happens there.

There is another big group. A lot of dead mice (neuroscience).

~~~
simple_phrases
> The ones who study psychology because it's a female dominated study and all
> those psych students need their credit and 'participating' in research is
> part of it.

Wait, does participating in a study as a _subject_ count for credit, or does
working on a study count?

Aren't studies explicit about the demographics of their participants? And
don't studies that make generalized claims usually control for things like
gender and age?

~~~
mettamage
I'm not sure whether this is a rant, or informing or both. Let me know if it's
a bit too much on the ranting side, I'll edit it.

> Wait, does participating in a study as a subject count for credit, or does
> working on a study count?

There was this course called "measurements and diagnostics" and for that you
had to pass the theory/tutorials of the course and also 10 hours of being a
participant in research. If you didn't do the 10 hours of participation, you
could do them at any moment you want and if you completed it, you immediately
passed the course.

My university has about 400 psychology students, so that is 4000 hours worth
of people participating per year. And I'm pretty sure that more universities
are doing this. In The Netherlands alone I estimate there to be about 40000
psychology students per year and probably about 5 hours of compulsory
'participation' in experimental studies. So a tiny country like The
Netherlands is skewing this particular bias about 20000 participant hours per
year.

Participating in experimental studies is quite fun (pro tip: go for the fMRI
and EEG studies ASAP! ;-) ), but I didn't appreciate how I was forced to do
it. I didn't learn too much from those experiments and I felt used. It
furthermore is a way to be lazy regarding participant recruitment. I'm pretty
sure that some startup could disrupt this particular problem. If you have
ideas about it, email me, I'm willing to brainstorm and help you to look for
ways to solve this problem.

> Aren't studies explicit about the demographics of their participants? And
> don't studies that make generalized claims usually control for things like
> gender and age?

Studies are explicit about the demographics, which is why the WEIRD article
could be written. And yes they do control for things like gender and age and
in general they are quite careful making those claims. The issue is though as
a research field psychology isn't the study of the human mind and human
behavior but of WEIRD people their mind and WEIRD people their behavior.

Like a lot of societal issues that are in popular media today, this is a
systemic issue. Though I could imagine psychology researchers being like:
"Hmm, I could get a representative group _or_ I could just get some psychology
students and churn my next paper out twice as fast!"

~~~
jackvalentine
Would it be fair to say that by being forced to participate in a study you
aren’t really giving your consent? Even though I presume you get to find/pick
the studies you participate in.

~~~
ACow_Adonis
Its not just fair, i'd say its accurate.

Here's another interesting phenomenon: a certain personality type (of which I
may or may not count my self among) will deliberately supply false data which
attempts to invalidate what we perceive as your study goal when forced to
participate against our wills in this way. Its a small rebellion against an
academic system that doesn't respect us or intellectual integrity, but is
focused on pumping out shoddy research on unwilling participants.

Turn academics into a game and thats how it'll be treated.

~~~
dwringer
I remember this practice of almost-compelled research participation even in
the introductory undergraduate Psychology 101 course, and in that class (which
included high school students and tons of non-majors) there was a lot of the
attitude you describe.

------
air7
I actually don't see this is as too problematic.

The entire field is murky because it's very hard to scientifically measure
human psychology and behavior, and the tools of the trade seem almost
laughably simplistic (like the aforementioned 5 point rating questionnaires).
So our body of knowledge doesn't even reliably describe WEIRD people. Rather
its a crude proxy, that just might contain some elements of truth warranting
further investigation.

But better crude tools than no tools, and better locally available subjects
than no subjects (because most studies don't have the budget to go to Zambia).
Similarly in other fields, research is done on rats pigs and monkeys with
conclusions drawn to humans. Obviously not perfect, but again it's at best a
starting point for later studies.

I think the real problem is the over-zealous interpretation of study results
as "truth".

~~~
mjburgess
I disagree.

If you're trying to measure a target 1cm across from 1 mile away with a ruler
and a squint _anything_ you say is not only likely wrong, but woefully
deceptive.

My view is that this shouldn't even be attempted because it just generates
superstitious theories. Psychology _is_ the skinner box pidegon.

There's a fallacy here that's hard to pin down clearly but roughly: to measure
inaccurately isnt to measure approximately. Its to measure totally in error.

The errors in social psychology are not just "second decimal place", they're
angels pushing stars.

~~~
slfnflctd
Most people I know believe and expect that some form of talk therapy is an
important part of a mental health safety net. Now, how should we train these
therapists? That's the utilitarian reason why we need something like a study
of psychology.

Unfortunately, much like with more physically oriented doctors, opinions and
practices will vary widely (including going against established research for
whatever reason)-- and those seem to be particularly diverse in this field.
The replication crisis clearly shows a strong need for reform at the research
level as well.

In my opinion, the way we approach these issues should be 99% focused on what
we know facilitates better outcomes (short and long term) for actual patients
in a cost effective way. Theorizing about fun new methods is exciting, but
what we need most these days is more disciplined and well trained pragmatists
in the field, who can put their world views aside for a while to provide basic
care for the (emotional equivalent of) bleeding gunshot wounds in front of
them. How we get there is a big question, but one part of it is always going
to be recruiting and properly equipping a certain percentage of incoming
students to do the work.

~~~
mjburgess
Talk therapy is a different case than research social psychology.

I take talk therapy to be a kind of practical skill in regulating the emotions
of another human being through narrative & interpersonal contact. That
requires a lot of practical training, and is in fact, mostly practical
training + high empathy.

Research social psychology does not inform this at all, being a totally
different area.

"Psychoanalytical research" is even more BS than social psychology, but that's
rather pre-advertised. And somewhat defensible as providing a "training
ground" for learning the practice of talk-therapy rather than any kind of
genuine explanatory framework.

~~~
threatofrain
But under this view how is talk therapy not more bullshit than research, or vs
more evidence based psychiatry?

~~~
mjburgess
Riding a bike isnt a theory of classical mechanics. Talking to another person
isnt a theory of human psychology.

Both require something to be true, but have no content on what that is.

------
wisty
That's not the only issue.

[https://en.wikipedia.org/wiki/Replication_crisis#Psychology_...](https://en.wikipedia.org/wiki/Replication_crisis#Psychology_replication_rates)

> A report by the Open Science Collaboration in August 2015 that was
> coordinated by Brian Nosek estimated the reproducibility of 100 studies in
> psychological science from three high-ranking psychology journals.[38]
> Overall, 36% of the replications yielded significant findings (p value below
> 0.05) compared to 97% of the original studies that had significant effects.
> The mean effect size in the replications was approximately half the
> magnitude of the effects reported in the original studies.

~~~
JoeAltmaier
That makes perfect sense. The published studies are a sample of 'study-space'
and that have outlier significance. Replicate published studies and their
significance likely returns to the norm.

Journals are filters to cherry-pick 'study space'. By the way they're
constituted, they publish new studies that have overstated significance.

------
rando14775
> [...] something like “I generally trust people.” Then participants are asked
> to choose one point along a five- or seven-point line ranging from strongly
> agree to strongly disagree. This numbered line is named a “Likert item”
> [...]

Oh god, having filled out a bunch of these for diagnosis and such I hate these
with a passion. I always wondered how well these actually work.

I've seen grammatical nonsense like, "Do you often do X? -- always, often,
sometimes, rarely, never". What, I often rarely do X? And what does often
mean, anyway? Like once a week? Every day?

Then, there are the abstract or vague questions that you then have to
interpret what concrete situation it could apply to. Hard to think up an
example off the top of my head, but how people reply to these surely depends
on what exactly they think it might mean.

Then you start losing patience after about 3 minutes of this shit, not to
mention 15 or 30 minutes, and just go through them barely reading the
questions, but for the first couple of questions you were pondering whether
you "agree" or "somewhat agree" for ages.

~~~
doubletgl
I didn't quite get why the article described these surveys as problematic. I
skimmed the linked article in that section but this didn't provide any answers
either. Assuming the participants fully understand the question, and the
survey is designed well, what cultural aspect is stopping people from
answering?

I grant that with a questions like "Do you often do X?", examples are
necessary to specify what "often" means.

From the article:

> Some people may refuse to answer. Others prefer to answer simply yes or no.
> Sometimes they respond with no difficulty.

That just sounds like some people boycott the Likert questions, but we don't
know why.

~~~
rando14775
I don't know what reasons other people have for refusing to answer. As a bit
of pedant I just wanted to explain why I personally don't like them, it has a
lot to do with being vague, abstract, unclear. Confusing really. You sit there
and are not sure you can even answer the question honestly.

In a normal situation I would probably refuse and ask them to clarify, or
challenge their assumptions or something. In this situation I just try to get
over my aversion to answer and try to interpret it as best as I can. Sometimes
I got no clue what it could mean and choose neutral or whatever. It's mentally
exhausting.

This doesn't invalidate the methodology. Just saying I don't like it. If
people refuse to answer or answer in a way that doesn't fit the schema and
they throw it out, that's obviously a problem though.

Edit: To maybe answer your question, I skimmed through the actual article and
they mention things like old people not culturally accepting a young person
administering questions, children thinking they should not speak in the
presence of elders. But also other weird things as mentioned that seem to me
as stemming from confusion. I mean, you get better at answering with practice,
so the whole concept might be totally baffling to some people which haven't
gone through a Western school, where you're also quite often expected to
answer unclear and confusing questions on tests.

More edit: The PhD students and professors designing these questions and the
college students used as test subjects are basically the most overschooled
people on the planet. They are the ones that, in school, excelled at answering
abstract, underspecified, questions with multiple choice answers. Many have
seen these before. They might even know how they are made and how the results
get processed. Of course they're going to have less trouble answering.

~~~
JoeAltmaier
Agreed. They are like 'which color represents your opinion on this complex
subject?' I don't know what to say - I agree with some of it, disagree with
another part, and have no opinion about how it applies to others anyway.

Then there are the 'Have you quit beating your wife?' questions, posed to make
it a dichotomy but I'm somewhere else entirely.

------
est31
The big problem with this is that the more diversity you have in your study
participant set (gender, age, skin color, handedness, etc), the bigger your
study has to be to correct for potential biases in results caused by
differences in the participants unrelated to what you're testing. And the
bigger the study, the more it'll cost, so there will be less studies overall.

~~~
taurath
Even among 18-21 year old college american women there are ridiculous amounts
of variables that you can't control for. Give up the idea that you can control
for biases in psychology. Do big mixed tests, then if there's positive
outcomes see if its correlated to any of those groups. Then you can do more
tests on the groups that didn't respond.

~~~
est31
Sorry, that's a sunken cost fallacy.

------
austincheney
This is a known problem. Nearly all experimentation-based psychology research
was in past decades conducted within the US and is heavily biased to US
culture. Cultural differences aren't limited to how people interact with each
other, but also include primitive qualities like how people perceive space and
use tools.

------
yyyymmddhhmmss
This isn’t even the worst problem with psychology.

The worst is it’s contextual loyalties against critiquing the relationsips
people have with one another. Power struggles, to be particular.

~~~
apatters
Can you elaborate on this?

~~~
yyyymmddhhmmss
By premeditating that diagnosis will be of the individual, the society is
assumed a constant. In science, we call that fraud.

Effectively, clinical psychology serves apology for societal and economic ills
by blaming the effects on the individuals themselves.

The process is strikingly transparent from an objective standpoint, but
ironically it’s precisely an objective standpoint that is undermined by the
institution. These days, you can visit one and tell them just about anything,
and they will diagnose your disposition as illness.

I don’t blame psychologists themselves, and who is to say they don’t help
people? Many surely do, but the institution is an absolute drag on our economy
and society; something like a collective excuse for avoiding self-critique.

Sociology theoretically steps in here, but it is merely a study, and some
branches of it would result in a totalitarian nightmare.

What’s needed is _humanist_ sociology and _humanist_ psychology. As we learned
in kindergarten class, there is nothing wrong with us. Each of us are unique
and beautiful in our own ways. The questions are about how to live with one
another.

Pyschology as a study is another thing, but as a medical malpractice it has
proven antithetical to systems design from the start.

~~~
SolaceQuantum
If you have scientific studies to show your objective viewpoint on mental
illness being merely a product of societal or economic ill, with definition of
'societal ill' and 'economic ill', I would be more than interested in seeing
the data you derived these conclusions from.

In the meantime, I would think the field of clinical psychology is a safer
bet.

~~~
50656E6973
>If you have scientific studies to show your objective viewpoint on mental
illness being merely a product of societal or economic ill

It's common sense, and there is no _objective proof_ to the contrary.

"Psychiatric diagnosis still relies exclusively on fallible subjective
judgments rather than objective biological tests"

-Alan Frances

[https://en.m.wikipedia.org/wiki/Allen_Frances](https://en.m.wikipedia.org/wiki/Allen_Frances)

Psychiatry/psychology also admits that social environment has substantial
effects on mental illness.

But asking for a "scientific" study of that is silly, because it is impossible
to have an indepedent variable or control group -- because _it is impossible
to separate an individual from societal influences._

~~~
SolaceQuantum
You are making the claim of an objective evaluation. It is your responsibility
to provide proof of its objectivity. I otherwise an more willing to take
something that at least has supporting evidence, dubious as it may be, to
claims that have no evidence at all beyond ‘common sense’. I think that’s a
pretty common sense position, but it’s clear we don’t agree.

~~~
50656E6973
There is no objective proof that mental illness even exists, so the burden of
proof is on those who claim it's just like real illnesses (psychiatrists and
psychologists). They're the one's claiming to be scientific.

I'm merely claiming they're not being scientific.

There is plenty of evidence that disfunctional social environments often lead
to disfunctional individuals.

------
hannob
Reading the intro I had a strange feeling about this text.

I don't have any issue with the general message. This surely is a legitimate
problem. But the intro reads a bit like "psychology is such a great endeavour,
if there wasn't this little issue."

Everyone following science news should be aware by now that psychology suffers
from a whole range of systemic methodological problems, notably publication
bias, widespread p-hacking and failed replications.

------
antwerpen
I remember our psychology 101 professor offering .1 GPA “extra credit” for
every psychology study you went to. There was no way to get a 4.0 in that
class unless you attended every class, answered every exam question correctly,
every quiz question, etc.

------
nabla9
The human behavioral and cultural adaptability to the environment is really
impressive and it makes many psychological findings local. With technological
and cultural changes some results may apply only to few generations before
they go away.

Think children growing in warzone or poor and violent environment. Their
behaviour as adults is often sexually more promiscuous, aggressive and their
impulse control seems to be less than 'the baseline'. They show trust issues.

How much of that is just damage and disorder as psychology seems to assume,
and how much is adaption to survive and procreate in an environment where
lifespans are short and life is uncertain. Maybe childhood stress and stress
hormones trigger survival strategies that work well in hard environment. They
are maladaptive only in the culture and safety of the developed world.

------
bellerose
Locus of control is another way of understanding how humans are not
necessarily universal to one another by how events are perceived. The missing
shape example might just be the children never encounter the square &
triangles in daily life and are thrown off it. I don't really consider that a
great example for showing how behavioural is different.

~~~
feanaro
The very fact that familiarity with a shape can influence performance on
pattern recognition tasks so dramatically is a significant finding on its own.

It would mean we are effectively blind(er) to unfamiliar shapes, even though
they are extremely simple, like a triangle or square are.

~~~
bellerose
I’m surprised that is a recent discovery. I would think if someone asked a
group of kids why someone would get the problem wrong. They would be like well
first time seeing shapes.

~~~
feanaro
Well, some certainly do, but as always, it is a game of sampling, so
individual errors are not as important as proportions. If we take the article
at face value, proportions differ drastically between very different cultures
so there must be something in the culture that biases the proportions towards
one outcome or the other systematically.

Of course, it's always possible any individual study contained unknown and
unaccounted biases. Funding in science is still relatively small, the number
of people participating few and science itself is hard.

------
SubiculumCode
Much of experimental design in psychology is an effort to attenuate the effect
of individual differences so that you can find the common (and probably core)
cognitive processes. Sometimes the risk to external validity is worth it in
order to reduce the complexity of data and models from confounding variables.
So we work with a system where most science is conducted with a relatively
homogeneous sample of college undergraduates who are convenient to obtain,
while well replicated or foundational findings are later tested across
cultures to determine degree of generalization. It seems an ok compromise to
me.

------
mrcactu5
a simple arithmetic computation shows that Europe has 750 million people, and
the world has 8 billion people. That's 1/10 of the population. We neglect
about 90% of people in... anything. South America has 500 million people, so
they should get a comparable share, and Africa has 1.2 billion.

So just by simple fractions we know something is off. A more careful study by
subject and region could be required.

------
taurath
Psychiatry is in the same boat. Anyone with multiple ongoing issues
(depression + anxiety, for instance) is disqualified from studies. Guess what
- those are the people that repeatedly have the worst outcomes or don't
respond to treatment. We don't even know the real effect of smoking with ADHD
people, because if they have anxiety they won't be included in the study.

------
vbuwivbiu
there could have been any number of blue triangles before that orange square -
we see 2, but what about the rest out of frame

------
anonu
75% of statistics are made up

------
ThomPete
Psychology and social studies, antropology (and economy and whats called
climate sciences) really shouldn't be called sciences. It's one of the rare
cases IMO where the word really obstructs the core meaning of the subject
matter.

They should be considered temporary interpretations of statistical data or in
short meta statistics cause that's really what they are.

Much damage is being done by treating these fields as science and the article
is only mentioning a few of those problems.

~~~
danans
> They should be considered temporary interpretations of statistical data or
> in short meta statistics cause that's really what they are.

This is what a lot of experimental sciences are, even physical ones, when the
systems being studied are complex.

There are very few areas of scientific study anymore that offer convenient,
deterministic results. That fruit was picked a long time ago.

Even at the cutting end of physics, researchers have to infer from statistical
results.

The difference is only that some of these fields have more reproducible
results than others, often because they are studying less complex phenomena,
whose causal factors and mechanisms can be more directly observed.

Psychology is at one end of that spectrum, because it is studying the output
of the mind, a biological information system whose mechanisms are among the
most complex and obscure that people have ever studied.

~~~
ThomPete
I understand that and I agree with how you think about it but the problem is
that it creates a very unhealthy public debate because one end of the spectrum
gets confused with the other.

