

Newcomb's paradox - monort
https://en.wikipedia.org/wiki/Newcomb%27s_paradox

======
vilhelm_s
Worth mentioning Scott Aaronson's blog post
([http://www.scottaaronson.com/blog/?p=30](http://www.scottaaronson.com/blog/?p=30)),
and a _lot_ of discussion on Less Wrong over the years
([http://wiki.lesswrong.com/wiki/Newcomb%27s_problem](http://wiki.lesswrong.com/wiki/Newcomb%27s_problem)).

------
greggyb
For the risk-averse:

Make a bet with someone for $500,000.00 that you can prove The Predictor is
fallible. Take only box B. If box B contains $1,000,000.00, then you have lost
the bet and you are left with only $500,000.00. If box B contains no money,
you have wone the bet and are left with $500,000.00.

~~~
baddox
Who would take the other side of that bet?

~~~
lobe
Since it is a rule of the game that the predictor is infallible, anyone with
$500,000 would take that

~~~
empthought
It's not a rule of the game that the predictor is infallible, only that it's
very likely to be correct.

~~~
greggyb
It seems there are multiple versions of the paradox. It does seem reasonable
that you can find someone to take the bet for <$999K if it is known that The
Predictor is "very likely to be correct". Any bet amount <$999K is
qualitatively the same as my original $500K suggestion, you increase your
guaranteed minimum by decreasing your potential maximum without having to
resolve the paradox.

~~~
empthought
I think any version of the paradox that _defines_ the predictor as infallible
misses the whole point of the paradox. That's just defining the outcome of the
game as the will of God. "Has always been observed to be correct" is the
appropriate construction.

However, I think someone could resuscitate the original intent of the paradox
by having the Predictor's actions also hinge on whether or not it predicts
that you would make such a bet, and leave box B empty if it does predict that.
Essentially defining itself to be correct in the situation where, without this
addendum, it would have been incorrect and you would have won the bet.

~~~
greggyb
I mean, now you're just creating a moving target.

I think you are intent upon maintaining the paradox, whereas I was just
pointing out a loophole that allows a better outcome without resolving it.
Gordian knot and all.

~~~
empthought
The paradox is supposed to be maintained. It is supposed to illustrate the
incommensurability of the two decision methods that it illustrates. It's not
supposed to be "solved" or "cut."

~~~
greggyb
The paradox exists. I did not solve it., I worked within the confines of it to
find a potentially better outcome.

You have changed the definition of the paradox. Both can be valuable avenues
of thought. I tend to view paradoxes as learning opportunities or a way to
practice critical and logical thinking skills. Both of us have achieved that,
so yay, but there's definitely not a single way to approach a paradox when
presented with one.

------
mietek
_> When formulated using Bayesian networks, two standard decision algorithms
(Evidential Decision Theory and Causal Decision Theory) can be shown to fail
systematically when faced with aspects of the prisoner’s dilemma and so-called
“Newcomblike” problems. We describe a new form of decision algorithm, called
Timeless Decision Theory, which consistently wins on these problems._

— Alex Altair, MIRI, “A Comparison of Decision Algorithms on Newcomblike
Problems”

[https://intelligence.org/files/Comparison.pdf](https://intelligence.org/files/Comparison.pdf)

------
cwp
I always find this kind of philosophical thought experiment unsatisfying.

Super-accurate predictions of human behaviour are just not possible. If I
could do it, I'd be a gazillionaire philanthropist/playboy dating supermodels
and advising heads of state because I can't be bothered to rule the world
directly. As it is, I can't do better than a draw against a 5-year-old at
rock-paper-scissors.

So this paradox tells us more about psychology than philosophy. Folks who
think "A and B" is the right answer basically ignore the bit about the
predictor never (or almost never) being wrong and go with a strategy that is
great for fallible human predictors.

And well they should. The only thing more ridiculous than an infallible
predictor is one that wastes his time playing shell games where the best he
can do is break even.

~~~
davidrusu
I can be more interesting reframed as such:

The player is the AI, and the predictor is the AI programmer.

The AI programmer can look into the innards of the AI ie. the source code and
can thus predict with high accuracy what the AI will do.

What is a winning strategy for the AI?

Or taken another way, you can have AI's that have access to each other's
source code and are competing for some scarce resource, how do you design an
AI that 'wins' when it's behaviour is known it's opponent?

~~~
bluepnume
Sure you can. The AIs behaviour could be as simple as "Always pick box B".

The difficult bit would be designing an AI which, given perfect knowledge of
its logic, would pick both boxes despite appearing to be more likely to pick
only B.

In that case, you could simply have an AI with a 0.499999 chance of picking
both and a 0.5000001 chance of picking B. The expected winnings would be
$1,000,500.

But then, once it comes down to probability, the predictor is no longer a
'perfect predictor' any more.

~~~
atopal
That would probably be counted as a random choice and as the rules state: > if
the Predictor predicts that the player will choose randomly, then box B will
contain nothing.

------
altrego99
Not sure why it is a paradox - assuming the predictor is superintelligent, you
don't try to fool it. By definition its intelligence can predict what you will
do in the very last moment, so the fact that it doesn't get to change anything
once prediction is made, is immaterial.

~~~
davidrusu
Let's reverse the numbers and make both boxes transparent.

If The Predictor suspects you will choose just box A, it'll put 1 million in
box B and 1 thousand in box A, if it suspects you will take both A and B, it
will put nothing anything in box B and 1 thousand in box A.

So now you standing in front of the two transparent boxes. You see that there
is $1 million in box B, yet you still take just box A?

~~~
altrego99
In that scenario, the predictor will always "predict" that you will take both
boxes.

The lack of information of box B, is exactly what makes the other case
different. Then you need to only rely on thinking and you do not know if box B
has a million dollars till you open it. If you risk taking two boxes, you will
lose it.

\---

Let me give another example to make the original scenario transparent.

Imagine I will write a function "decide(double content_of_A)" to decide
whether B or both will be opened, given content of A.

Imagine you can examine the function beforehand, and you are super-intelligent
compared to me, so any attempt to obfuscate the code in my part will be
utterly useless and easily seen through.

And you are honest - in putting the $1M in box B if your analysis suggests
that the decision function will only take B.

Note that my function gets called after you have placed the money, just as in
the original scenario.

Would I not write the function to choose B? I would.

------
fiatmoney
The standard "solution" (that isn't really a solution, but an explanation) is
that the predictor has decided to reward the kind of person who one-boxes, and
is extremely good at predicting whether you are that kind of person (perhaps
even better than you yourself are).

So, if you can "decide to be the kind of person" who one-boxes (and perhaps by
induction "decide to be the kind of person who decides to be a certain kind of
person"), you can make out pretty well.

------
_greim_
I think there's a weak echo of this concept that plays out in an election.
Except swap in the inconvenience of voting for giving up the $1,000 box.

On the one hand, why bother voting? It's a pain in the ass and my single vote
has such a negligible effect. On the other hand, if everyone like me has that
same attitude, then I and others like me lose our voice in the election. So
should I vote, or not?

~~~
baddox
That's a fairly simple one. If all you value is the benefits to you of your
influence on the election, do not bother to vote (and especially don't bother
spending the resources required to educate yourselves so as to vote
responsibly). It's very clearly not worth it except perhaps for the smallest
local elections or closest large elections.

If you value other things related to voting, like feeling as if you've done
your civic duty, or feeling like a member of a group (like a political party),
then by all means vote if the hassle of physically voting is less than that
benefit.

This seems like a pretty clear description of rational behavior, and if you
expect people to behave largely rationally, this description seems to explain
some of the problems often attributed to elections, like low voter
participation and low voter knowledge of the candidates and issues.

~~~
SilasX
But then if people only care about the benefits to them, then they'll be
overtaken by "lizards" who exploit them but never get voted out because no one
thinks voting passes a CBA. Populations who vote "despite" its wastefulness
systematically win against those who don't.

Arguably, the only reason any population isn't overrun by lizards is because
it's people are mostly "wasteful" in this sense.

One intermediate solution is to force everyone to vote so that it's no longer
costly. This is arguably what is accomplished when people promote voting out
of civic duty, etc.

~~~
baddox
But I don't think people are wasteful in that sense. Voter participation and
voter awareness tends to be low, at least in the USA. Besides, if your
government only works if people act in a specific irrational manner, I don't
have high hopes for it, especially considering that one usually cited
fundamental role of government is to fix problems where individual rationality
does not lead to group rationality.

Forcing people to vote doesn't incentivize people to educate themselves on the
issues, which is required to "vote responsibly" according to the usual Western
civics class description of how democracy is supposed to work.

~~~
SilasX
Voter participation is very high, and voter decisions very wise, relative to
the lizard scenario; and it's not clear that "not being overlorded by lizards"
_is_ a kind of irrationality.

Remember, the lizard scenario is something like "500 lizards outvote 300
million and put in 99% tax rates on non-lizards, to be spent entirely on
lizards, all because none of the 300 million want to vote, reasoning that
their vote doesn't affect the outcome."

Mandatory voting would definitely be an improvement over that for much the
same reasons I gave before.

------
olalonde
I like Scott Aaronson's solution
([http://www.scottaaronson.com/blog/?p=30](http://www.scottaaronson.com/blog/?p=30)).
Basically, assuming:

1) completely infallible and incapable of error

2) to get a perfect prediction, the predictor must simulate you perfectly to
the extent that the simulated you won't have any way to know it is the
simulation

Therefore, at the time of the decision, "you" might actually be the "simulated
you" and any decision you make will indeed affect the content of the boxes for
the "real you". Which makes single boxing the rational decision.

------
fitter
I guess this boils down to whether you believe in determinism.

If you do, then the predictor will always be right, and you have essentially
zero chance of fooling it by "changing your mind" later for the extra money.

If you don't believe in such a thing, then theoretically some nondeterministic
volition of yours could allow you to change your mind in a way that the
predictor could not have deterministically forseen.

~~~
empthought
It's not about changing your mind for the money, it's about the fact that
causality doesn't (shouldn't) run backwards in time.

Consider this: when you are faced with the choice, the allotted money is
_already_ under the boxes. How could what you choose _now_ affect this
outcome? It can't. You must always take both boxes to get the maximum amount
of money possible. Either the $1,000,000 is under the single box, or it is
not. If you take both boxes, you will get $1000 regardless of anything else,
and possibly $1,001,000. If you take the single box, you will either get $0 or
$1,000,000, but _your action can 't possibly change that_, unless you believe
somehow that causality runs backwards in time.

~~~
SilasX
Under a generalization of this problem, you can do transparent boxes and get
basically the same paradox: in that case, omega never even presents you with
this choice while putting $1 million in a box _unless_ you're "the type of
person" who would one-box even then.

Still no causality violation: omega simulates everyone,[2] and only offers the
filled box to one-boxers, but leaves it empty for two-boxers. [1]

But you don't even have to conser these esoteric, hypothetical situations to
get a newcomblike paradox: even "merchants vs shoplifters" has a similar
dynamic: you will only be in the position of being able to trivially shoplift
merchandise if you're in a neighborhood that draws from the set of people who
usually don't. Merchants (Omega) are accurate enough in their predictions to
be profitable.

[1] See counterfactual mugging for a similar dynamic.

[2] With a thorough enough simulator, it may not be possible to tell whether
"you" are in the simulator or doing the real thing.

~~~
empthought
That generalization is no longer a paradox; it's just a situation. The paradox
is about choice theory and you have eliminated any element of choice.

People who choose (or act, if you don't care for free will) to take only one
box _are always leaving money on the table_ , full stop. The point of the game
is to maximize winnings.

The alternative approach is that people who have chosen one box _have always
received more money than those who chose both_. Explanations about how or why
are distractions and are inconsequential; the paradox is about these two --
both generally considered to be valid -- approaches yielding such different
results.

In my opinion, the resolution of the paradox is that's an impossible
situation. Either someone is lying about the mechanisms (in which case take
one box like everyone else because it's a magic trick of some kind) or not (in
which case the "predictor" can be wrong and the boxes are already set, so take
both boxes to eliminate the risk of receiving nothing and to maximize your
winnings).

~~~
SilasX
>That generalization is no longer a paradox; it's just a situation. The
paradox is about choice theory and you have eliminated any element of choice.

You're still choosing whether to use a decision procedure that results in how
many boxes to take when offered this choice, which then determines whether you
get this offer at all.

And I don't know what you're trying to say with the paradox/situation
distinction; "Newcomb's problem with transparent boxes" is a paradox and a
situation, just like the original: how are people ending up better off by
"leaving money on the table"? (whatever that would mean)

>People who choose (or act, if you don't care for free will) to take only one
box are always leaving money on the table, full stop. The point of the game is
to maximize winnings.

But once you pin down what "leaving money on the table" means, it's not at all
clear that the concept coincides with something you want to avoid. If the
people "leaving money on the table" have more money, then "I don't want to be
right", as the saying goes.

>In my opinion, the resolution of the paradox is that's an impossible
situation.

I disagree. At the very least, you can play as omega against an algorithm,
with varying degrees of scrutability. How should that kind of algorithm be
written so that it gets more money (in transparent boxes, how to get omegas to
offer you filled boxes in the first place)? Your answer would require
addressing the same issues that arise here for humans in that situation.

There are also statistical versions of the paradox, like merchants vs
shoplifters. Obviously, they aren't perfect predictors, but they do well
enough for the sort of "acausal" effects in the paradox to happen, ie people
not shoplifting, even when they could get away with it. Here are some more
real life examples:

[http://lesswrong.com/lw/4yn/realworld_newcomblike_problems/](http://lesswrong.com/lw/4yn/realworld_newcomblike_problems/)

To be sure, people aren't predictable enough _now_ to get the kind of scenario
described in the problem. But they are predictable enough for the
uncomfortable implications: even an accuracy slightly better than chance gets
you situations were one-boxing is statistically superior.

(I do agree that _in pracitce_ , whenever you see this kind of situation, you
should assume there's some trick until overwhelming evidence comes in to the
contrary.)

~~~
empthought
> But once you pin down what "leaving money on the table" means, it's not at
> all clear that the concept coincides with something you want to avoid.

In this case (which I have to imagine is deliberate on the part of Nozick or
Newcomb), "leaving money on the table" means literally _leaving money on the
table_. Taking one box always, _always_ results in less money than the total
amount of money available in the boxes to people who take both boxes. (Of
course, the evidence to date is that people who choose both boxes always have
less money available to them in the first place.)

But the equally justifiable decision-making method is to perform the action
that has yielded the best observed results in the past for others, despite
there being no way that one's actions _now_ can possibly have affected the
_past_ (choice or determinism doesn't matter).

The nature-of-the-predictor stuff is just irrelevant nonsense in either
approach to the problem, which is a happy coincidence because it is, in fact,
irrelevant and impossible nonsense. :)

Edit: "there's no way that one's actions now can possibly have affected the
past" is given in the original problem. Wikipedia's article quotes it as "what
you actually decide to do is not part of the explanation of why he made the
prediction he made."

------
samatman
Via Turing, we know that the Predictor would be able to decide programs, and
since it can't, it can't be a Predictor.

The isomorphism between the program which defeats decidability and the two-box
Newcomb problem is left as an exercise.

~~~
jcoffland
You assume that a human cannot be described with something less powerful than
a Turing machine. There is no reason to believe that knowing what a human will
do in this instance is equal to decidability of computer programs.

~~~
Dylan16807
I almost objected, but you're right. The human has finite time with which to
calculate. The predictor can easily be assumed to have more time. There's no
problem calculating if a program will halt in finite time.

------
sixQuarks
What I don't understand is why would anyone risk taking both A & B just to
gain a measly $1,000? Just take the million and be happy with it.

------
eli_gottlieb
It's not a paradox to those who understand _compatibilist_ free will, and how
basic information-sharing means that events at two different points in time
and space can be isomorphic. There's no impossibility being committed, and no
retrocausality either.

