Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Twelve virtues of rationality (yudkowsky.net)
28 points by Hexstream on Nov 15, 2008 | hide | past | favorite | 17 comments


"The best physicist in ancient Greece could not calculate the path of a falling apple. There is no guarantee that adequacy is possible given your hardest effort; therefore spare no thought for whether others are doing worse. If you compare yourself to others you will not see the biases that all humans share. To be human is to make ten thousand errors. No one in this world achieves perfection."

This seems almost to gesture toward a thought I've been tossing around my head lately. If you're basically a conventional person, with a conventional, average view of the world, conventional understanding of morality, and so forth, people who never knew you will look back at you from the future and ask, "How could you think such things? What was wrong with you?" We say it about people in the times when slavery was conventional, when segregation was conventional, when entrenched sexism was conventional, when Nazism was conventional. People will say it about conventions of our time as well, and, I must say, rightfully so.

The only way around this is to give serious and continual examination to the conventions of our time -- to quote John Taylor Gatto, "the great non-thought of received ideas." People of the future, with higher shoulders on which to stand, will probably still find places to criticize you. No one is perfect, and no one can escape entirely the influence of their time and place. But what shows one's commitment to the good is moving toward it -- which, in turn, helps others move toward it.

And the same applies to physics. Every physics major today knows about relativity. That does not make them better physicists than, say, Newton. That credit can only go to those who move conventions -- be they in moral philosophy, or physics, or medicine, or anything else -- not those who follow them, however advanced the conventions they inherit happen to be.


Are there virtues to irrationality too? It would be better to balance this article with one that presents the 12 virtues of irrationality...

To get started on this list:

1) Goodness - sometimes the rational choice is not the right choice.

2) Rapidity of execution - it can take hours or even a lifetime to come to a rational decision on some complicated, vaguely defined subjects (such as ethics). The irrational decision engine in your brain can make that decision in a split-second and be right most of the time.

3) Better resilience to incorrect or incomplete data - sometimes, you have incorrect or incomplete data, and no way to correct or complete it. Based on your rational analysis, you may decide to, for instance, refuse to meet someone. And yet, once you actually meet them, you may gather new information that changes your previous rational choice.

There's no doubt many more...


None of those represent virtues of irrationality, but rather common misunderstandings of rationality.

1) If goodness is your goal, rationality lights the path.

2) To engage in a lifetime of cogitation when a split-second decision would be good enough (given your goals) is not rational.

3) Rationality includes making decisions under incomplete information (including the correct assessment of risks). The argument you present here in fact presupposes rationality.


I don't see why you equate rationality to a brute-force search of the possibility space.


Some guy went to the Hacker News Coffee Palo Alto (http://news.ycombinator.com/item?id=285623) a few months back and handed out copies of this.

Does anyone know who this was? He looked strikingly similar to Yudkowsky...


I think that was Yudkowsky.


WTF man? Why didn't you say so? :)


Hm... isn't Yudkowsky's first name Eliezer? ;)



He listed 7 reasons his worldview was better than yours before covering humility. List ordering fail.

I deny that humility is a virtue and that humility corresponds to rationality.

Also, it's rational to be occasionally irrational, because that instability contributes to beneficial evolutionary changes in the structure of thought.


"It's rational to be occasionally irrational" is an irrational statement. It is rational to be open to new ideas, but that is not irrationality.


No I mean it's rational to go bat-shit "the lizard men would have gotten me if it weren't for the fairy tucking me into extra dimensions where my entire worldview fell apart like a glass chandelier shattering I feel like I'm dreaming but I'm wide awake get that guy some seizure meds" irrational every now and then.

It's most rational to be able to slide into any degree of irrationality that maximizes the placebo effect and evolutionary change in thought. Thinking is what you put into your brain to cocreate your reality. Irrational thinking can sometimes lead to the creation of a better reality. (It's probably irrational to feel optimistic about your business endeavor, but you're more likely to succeed if you think it anyway.) I mean a lot more then being open to new ideas. I mean having the mental flexibility to slide from completely insane to completely rational intentionally, and using that to create the best reality.


Not so sure I agree about the lizard men, but I agree in principle.

It's a bit like tai chi. You can say that chi is bs, and it's all about how you move your body. And you'd be right.

But, for most people at lest, without believing on some level that you're tossing energy around, you won't move your body right.

The key is to be able to believe that in the way that is useful, and yet, still have the clarity to look at your beliefs and recognize that some of them are not, in fact, accurate models of the universe. Outside of an epistemology essay, we can in fact believe and not believe the same thing at the same time, and quite often do. The trick is to do it intentionally.


The interaction between Tai Chi and thinking about energy and moving right tells us a lot about cognitive processing. It's an interesting thing. If Tai Chi relates to cognitive processing, they are just describing as energy a real phenomena we describe in cognitive processing terms. Kind of like when we make a model of something described as a wave, and another model of that thing described as a particle. And both contradict each other but are confirmable experimentally. Tai Chi is an excellent example of this kind of thing.

I would press on you one step farther here, though. You say "still have the clarity to look at your beliefs and recognize that some of them are not, in fact, accurate models of the universe." I would press one step farther and say it's not even possible to have an accurate model of the universe, and that anything you could possibly believe would not be an accurate model about the universe. All you can say is what happens when your nervous system bumps into reality.

The lesson from quantum physics is that there is no way to discharge your instruments (whether science tools or your brain) from what you are saying about your universe. Therefore, nothing you say is an accurate model of any part of the universe.


Just because some parts are wrong, that doesn't mean that no parts are right.

Models are expectation limiters, and experience predictors. Not all models are created equal; they vary in specificity and in accuracy.

The phrase "there are two cats on my bed", I'm pretty sure, is about as right as possible. This is my bed. I'm looking at two cats sleeping soundly on it. When I close my eyes, I can still feel them three. EVERY prediction that is limited by that model turns out to have been properly limited. EVERY prediction that model suggests is accurate. It is, in fact, as true as true gets.

On the other hand, the sentence "light is a wave" is saying a much more specific thing about the universe. Like the two cats on my bed statement, it rules out some expectations, and suggests some predictions. As it turns out, some of those expectations are correctly limited, and some are not; some of the predictions are validated, and some are not.

Therefor, we can say that, while it may be useful to consider light being a wave in some instances, it clearly is actually something other than a wave that just happens to be wave-like. If it was actually a wave, then every prediction would be validated, not just some of them.

We can go a step further, even, and in some cases talk about the degree of truth of a given model. The truth value only has to be predictably higher than 0% in order to be useful; but it has to be 100% to be considered actually true. Furthermore, specificity effects the value of truth. A statement about cats on my bed may be 100% true, but it's not nearly as risky or specific as a statement about the fundamental nature of light. The main usefulness of this 100% true statement is for me to know that my feet will be kept warm.

And, while I personally care deeply about the comfort of my feet, in the grand scheme of things, it's just not that big a deal.


I just learned I shouldn't be drinking when I read the sentence "the lizard men would have gotten me if it weren't for the fairy tucking me into extra dimensions where my entire worldview fell apart like a glass chandelier shattering I feel like I'm dreaming but I'm wide awake get that guy some seizure meds"


Admitting that you need "evolutionary changes in the structure of thought" IS humility.

He's not talking about Mother Theresa stuff here, but good old fashioned intellectual honesty and curiosity.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: