
A Test to Measure How Rational You Really Are - gojomo
http://ieet.org/index.php/IEET/more/dvorsky20130705
======
tunesmith
One thing I'm stuck regarding rationality vs instinct -

I'm partial to the argument that the reason we have heuristics or "rules of
thumbs" is because it saves our energy. If we invest fully rational mental
energy on every decision, then we wouldn't get as much done. Heuristics are
useful because they are useful, usually correct, and save time/energy.

However, they are also sometimes wrong. And I also know that cognitive biases
can lead us astray in the same way that a heuristic can be wrong. And that's
where rationality is useful, because it helps us make the correct decision in
those cases.

If both are true, then the optimized way of living would be to use heuristics
where they don't get you in trouble, and use rational examination when the
result would be counterintuitive. Here's where I'm stuck - how do you
recognize ahead of time when you should ignore your
instinct/heuristic/cognitive-bias, and instead use the more exhaustive
rational examination?

It's as if we need to develop an intuitive skill on when to recognize that a
reality is likely to be counterintuitive. "Oh, my gut tells me that this is
one of those cases where my gut will be wrong." Which seems a bit
contradictory.

~~~
StavrosK
It does seem contradictory, but it's not. You definitely learn to recognize
your biases, it's a skill. There have been many times when I've caught myself
thinking something, only to then say "wait a minute, I've just fallen victim
to bias X" and then reversed my decision or opinion.

I'm not great at it, but I'm very happy that it catches at least some of my
brain's bugs.

------
twoodfin
_I also discuss how many feel that George W. Bush was dysrationalic._

No, he outright makes that "diagnosis" of the former president, going so far
as to title the chapter 1 "teaser" of his book "Inside George W. Bush's Mind".
It's available here:

[http://yalepress.yale.edu/yupbooks/excerpts/stanovich_what.p...](http://yalepress.yale.edu/yupbooks/excerpts/stanovich_what.pdf)

Maybe he's being cagey about it because his "evidence" is incredibly weak
sauce. He expects most of the audience for his book to nod along when he
describes a guy who managed to achieve the highest political office in an
incredibly competitive environment as unable to make rational decisions. He
cites for his analysis of GWB's cognitive function a whole two people.

First, there's David Frum[1], who served as a speechwriter for 13 months from
early 2001 to early 2002. Before that he had "no connection to the campaign or
the Bush family". He's used that stint to great career effect, but seriously,
there are a lot of folks who worked more closely with Bush for a longer period
of time and have written about his thought and leadership style. Surely if
Stanovich wasn't just looking for a good "hook" to snare his liberal
readership, he could have found better sources, critical and otherwise[2]. Oh,
and Frum was a huge initial supporter of the Iraq invasion, and obviously
Stanovich or his imagined readership wouldn't dream of calling _that_ a
clearly irrational decision.

Second, there's a bit from a George Will column. The section in italics was
strangely omitted by Stanovich:

"He has neither the inclination nor the ability to make sophisticated
judgments about competing approaches to construing the Constitution. _Few
presidents acquire such abilities in the course of their pre-presidential
careers, and this president particularly is not disposed to such reflections._
"

So which U.S. presidents weren't "dysrationalic" by this standard?

This guy may be a terrific research psychologist, but he's obviously not above
slandering a president and pandering to his readership with bad scholarship
just to sell books.

[1]
[http://en.wikipedia.org/wiki/David_Frum](http://en.wikipedia.org/wiki/David_Frum)

[2]
[http://keithhennessey.com/2013/04/24/smarter/](http://keithhennessey.com/2013/04/24/smarter/)

~~~
x0x0
Yet George Bush's presidency is widely acknowledge to be, well, a fail parade.
Or perhaps we should go with by their works ye shall judge them. An (extremely
brief) overview of Bush's presidency:

* he waved off warnings of Bin Laden ("All right. You’ve covered your ass, now")

* he turned a highly effective -- under clinton -- national disaster agency into a joke, at least in part by putting a failed horse judge in charge of it, then oversaw an ineffective response to one of the largest national disasters to ever strike the US (katrina)

* he lost Bin Laden at Tora Bora

* he and his chosen vp and sec state back channeled incredibly wrong information into the public, in part via the nyt, in order to sell a war with, well, people who had no weapons of mass destruction and who were uninvolved in 9/11;

* by the by on the above, he fired the general (shinseki) who gave an accurate prediction of iraq war costs

* Bush hired wolfowitz, who predicted the iraq war would cost between $10 and $100B (what's $700B and counting between friends)

* Bush ignored, well, basically everyone who understood much of anything about the middle east in favor of a war which wildly destabilized a highly geopolitically important (oil) zone, while empowering iran

So your link, in which the author writes,

    
    
       [...] President Bush's thinking has several problematic aspects: lack of
       intellectual engagement, cognitive inflexibility, need for closure, belief
       perseverance, confirmation bias, overconfidence, and insensitivity to
       consistency.  These are all cognitive characteristics that have been studied
       by psychiatrists and that can be measured with at least some precision.
       However, they are all examples of thinking styles that are not tapped by IQ
       tests.  Thus, it is not surprising that someone could suffer from many of
       these cognitive deficiencies and still have a moderatively high IQ.
     
       Bush's cognitive deficiencies do not impair performance on intelligence tests,
       but they do impair rational decision making. [1]
    

is, in retrospect, incisive.

[1]
[http://yalepress.yale.edu/yupbooks/excerpts/stanovich_what.p...](http://yalepress.yale.edu/yupbooks/excerpts/stanovich_what.pdf)

~~~
twoodfin
Even if I grant you all of that, it doesn't change the fact that Stanovich's
claims re: the cognitive abilities of George W. Bush are based on extremely
flimsy evidence. If he had stronger evidence than the testimony of Frum and a
column from George Will (who, as far as I know, never attended a meeting in
which GWB made a decision), wouldn't he have used it?

Indeed, in contrast to your laundry list, Stanovich doesn't present a single
concrete example of an irrational decision by the former president and the
cognitive bias that led to it. He doesn't have to, because he wants to appeal
to a readership that is delighted to find that not only is the target of their
political animus wrong, but actually suffering from a mental disability!

It's intellectually lazy and unworthy of a serious scholar.

~~~
Retric
If this had been a dissertation I would completely agree with your point of
view. However, serious scholarship has little to do with writing a popular
book. If anything Malcolm Gladwell is the gold standard for popular,
interesting, and bulllshit books that sell like hot cakes.

Consider his religious affiliation suggests a level of irrational thinking,
but you can't say such things if you want a best seller. Yet, there are plenty
scholarly works focused on the connection between religion an irrationality
because scholars especially those with tenure can get away with such things.

------
tokenadult
The profiled psychologist doing the research discussed in this interesting
submission mentions his book What Intelligence Tests Miss,

[http://yalepress.yale.edu/book.asp?isbn=9780300164626](http://yalepress.yale.edu/book.asp?isbn=9780300164626)

and I have recommended that book repeatedly in Hacker News discussion over the
years. The book is readable, interesting, and surprising, and the bibliography
cites most of the best recent research on human cognition.

------
MikeCapone
Most here are probably already aware of it, but for those who have somehow
managed not to find this treasure trove:

A great place to study the art of human rationality is LessWrong.com. The
'sequences' are the place to start:

[http://wiki.lesswrong.com/wiki/Sequences](http://wiki.lesswrong.com/wiki/Sequences)

~~~
krichman
I have a hard time reading Yudkowsky. I'm not as in love with his writing as
he is. I think he makes good points. But if you're starting from the position
of evidence-based beliefs and Bayesian reasoning, the knowledge gained by
suffering through a longwinded essay seems too little. I have tried a few
times, because others seem to love Less Wrong, but each time I found I came
away with little or no new information.

~~~
MikeCapone
1) No writer can please everybody, so it's fine if he's not your style.

2) Have you read only a few things that he wrote about, for example, the
better known cognitive biases and such, or have you also read some of the more
advanced sequences?

Because if you're reading the advanced essays and feel you already know all
that stuff, you are a very rare breed -- good for you! I hope you're working
on some hard problem in an un-sexy field and not building another photo-
sharing app :)

To quote Dr. Aubrey de Grey:

>It has always appalled me that really bright scientists almost all work in
the most competitive fields, the ones in which they are making the least
difference. In other words, if they were hit by a truck, the same discovery
would be made by somebody else about 10 minutes later.

3) When someone says they don't like something that I like, I ask them what is
it that they like. I figure maybe they've found something even better and I'd
love to get my hands on it too, and it's also a good way to see if they're
just signalling superiority by disliking things that many others like. So what
would you recommend I read to learn more about human rationality? Anything
other than the usual suspects (Jayne, Kahneman, Tversky, Schelling, Hatie &
Dawes, etc)?

Thanks.

~~~
krichman
1) Agreed. I observe other people like him, so in this case it's probably all
me.

2) I've tried to read some of the major sequences, but end up quitting partway
through. Each essay seems so long for what it's trying to impart. I tried
reading HPMOR in hopes of getting the same or similar information with a
whimsical story instead, but didn't enjoy that either.

I doubt I'm a rationalist prodigy, so I'm more worried that I'm missing out on
something profound than failing humanity :)

3) Unfortunately, he's the only author I'm even aware of that writes on human
rationality and is well-regarded. I spend most of my reading time on other
topics.

------
realitygrill
Well, the people over at the Center for Applied Rationality have tried to work
on something like this, if only to attempt to better measure the effects of
their workshops. I suspect this is a very hard undertaking.

------
beefman
Define "rational". Psychologists like Kahneman often say they're using some
agreed-upon definition that exists outside their field. But applications of
Bernoulli and Bayes are an area of ongoing research, even narrowly [1][2].
More generally, no theory of general intelligence exists, and to assume you
can trick intelligent agents like humans with simplistic experiments is
unconvincing IMO.

[1]
[http://en.wikipedia.org/wiki/Proebsting%27s_paradox](http://en.wikipedia.org/wiki/Proebsting%27s_paradox)

[2]
[http://www.science20.com/hammock_physicist/statistical_physi...](http://www.science20.com/hammock_physicist/statistical_physics_attacks_st_petersburg_paradox_resolved-96549)

------
stiff
The notion of domain-independent rationality is the favourite way for some
nerds (so called "sceptics") to pump up their egos and then throw around
nonsensical judgements in domains they don't know the first thing about. But
hey, they know the Bayes theorem! So they will out-smart everyone on every
topic! In some sense this is an intellectual philosophers stone, the notion
(it is never said out loud but that is what seems to happen subconsciously)
that once you are "rational" enough you can judge things more effectively
without having expertise in a given field, medicine for example.

A good example is Nassim Taleb's last book, a collection of completely
ridiculous opinions on biology, medicine, computing and fitness...

------
jdpage
Depending on how rigorously the research on this is done, this is a test I
might actually take, especially since, as he says in the article, a lot of it
is stuff you can improve if you find yourself lacking.

On the other hand, I don't want to take an IQ test. This is because you end up
with a number which can now be used to place you on a scale relative to
others, which as far as I'm concerned is fairly useless[1]. I'd much rather be
judged based on what I've done rather on some number that purports to measure
my potential.

[1] In my (fairly limited, I grant) experience, IQ scores are mostly used by
insecure navel-gazers as a sort of bragging right.

------
D9u
The irony that a site which uses the word "ethics" in its title would host
such unethical content.

The bit about Bush should have been omitted.

But, that may just be my irrationality.

------
coldcode
Reading this makes me wonder if our brains are intelligent or rational enough
to understand our intelligence or rationality.

------
adambratt
Fellow readers, what's the best book you've read for improving your critical
thinking/rationality?

~~~
alok-g
This may be overwhelming but can help:

[https://en.wikipedia.org/wiki/List_of_cognitive_biases](https://en.wikipedia.org/wiki/List_of_cognitive_biases)

While I am not there yet myself with most of the biases, it seems that
memorizing the name of a bias once you understand it helps you spot it more
often in the daily life. [Sorry I do not know which cognitive bias would that
be! :-)]

~~~
adambratt
Yeah, I wish I could find a good tool/site for quizzing myself on these.

Something that would give me a paragraph and ask me to name the bias present
if any.

~~~
denzil
Anki has some decks like that: [http://ankisrs.net/](http://ankisrs.net/)

~~~
alok-g
Can you please point to the decks? I see only the software on that URL.

------
cdi
I think they will find eventually that rationality is a cultural/educational
thing.

~~~
oblique63
The best way to test this would be to approach it like Intelligence research
originally approached that hypothesis for IQ scores: by testing a reasonably
large set of twins separated at birth. That's such a tricky criteria to come
by, I can't imagine it'll be easy to do again, but it did help cement the idea
that IQ was likely _not_ significantly impacted cultural/environmental
factors, and thus possibly a result of subtle genetic traits. The interesting
thing about those studies though, was that the separated twins tended to have
a lot of other unexpectedly subtle quirks in common too (such as sense of
humor). So in that context, I actually wouldn't be surprised if rationality
falls under those similarities as well.

------
berntb
I used to naively assume that intelligence implied rationality. An old work
colleague taught me differently.

My old work friend was really intelligent. He had been considering a PhD in
philosophy and had problems to stop talking about Wittgenstein.

On one hand, this guy had a large interest -- and deep studies -- on the
subjects of language meaning, epistemology, logic, the scientific method, etc.

On the other hand, he believed in conspiracy theories like being a 9/11
truther, etc.

I tried to discuss this contradiction with him, but probably tried to reason
on a too low level. I argued that there is an infinite number of conspiracy
theories that would satisfy the available facts. It might have worked better
to simply ask: "What does your philosophy studies say about conspiracy
theories based on hand picked facts?"

Such a sad, sad waste of a good brain.

~~~
marcosdumay
> I argued that there is an infinite number of conspiracy theories that would
> satisfy the available facts.

Thus, you can't say it wasn't a conspiracy. Yeah, you can't say it was one
either. Both of you are jumping to conclusions, what's irrational by the
definition of the article.

~~~
berntb
I'm not going to discuss arguments about one of the beliefs of a non-rational
person. If you feel a pressing need to iron out the finer distinctions of such
stuff, I'm certain there are multiple suitable sub-reddits.

(But I'll add: There are also an infinite number of non-conspiracy theories
fitting the facts.)

Edit: coldtea, sorry I'm not going to discuss e.g. if an infinite number of
possible alien species could be responsible for 9/11\. (And no, not even if
the possibly existing alien species are enumerable :-) ) See above. I might,
if I had more time.

~~~
coldtea
> _There are also an infinite number of non-conspiracy theories fitting the
> facts._

That's wrong too. Social life and human action don't have infinite
possibilities.

Depending on what scale you draw the line for detail, there are is a finite
number of theories fitting any fact -- and less of them, if you include
possibilities (Ocamm's razor).

And the more facts you have, the smaller the number of theories becomes.

------
dakimov
I can measure how rational you really are.

The result is around zero.

Seriously, come on, people are not rationally thinking creatures at all,
nobody is. There is nothing to even measure.

If everybody became rational, the world would collapse instantly. Do you
realize how many people would cease their socially indispensable work if they
acted perfectly rational from an individual (meaning egotistical) perspective?

I clearly see that I am irrational, but I cannot stop being so.

I estimate that in order to become rationally thinking, I would have to
undergo an unbearably painful transformation of my entire mental entity. My
mind just works that way, it is hardwired irrationally, as any other human's
mind.

The article and the attempt itself are great though.

~~~
markdown
> Do you realize how many people would cease their socially indispensable work
> if they acted perfectly rational from an individual (meaning egotistical)
> perspective?

It is often in the interest of individuals to contribute to the group.

