The Volokh Conspiracy

In first problem, there's a 50% chance there is $50 in the box and 50% chance there is $200 - making the EV $125 (.5*50+.5*200). EV for keeping first box is $100, so take a chance on the second box.

The principle holds for all solutions (using actual cash). The second box has an EV of 0.5*0.5X + 0.5*2X.

You should always switch. There is a 50-50 chance of gaining or losing, but the amount of gain and loss is not the same. If you gain your money goes up by 100% (or in the specific example $100); if you lose it only goes down by 50% or $50.

Assuming you are a "profit" maximizer (you want to keep as much of your money as possible -- though why you would play this particular game in that case is beyond me), and you were able to see (or already knew) what was in the box I chose, then your willingness to exchange would tend to indicate that the box you are left with has less. Nothing said you were bound to offer an exchange after my choice.
However, knowing I would be suspicious of your willingness to trade, perhaps you are engaging in reverse psychology. But, again, you don't have to offer, so this would be pointless, unless you just like playing head games and don't mind losing the $100. Answer -- don't take the trade.

If you are blinded, then perhaps it is always better to take the exchange. I mean, $100 is not that much, and there is no upper bound on the contents of the boxes. There is a lower bound ($0), so I have more room to move up than room to move down. Giving the limitations on the money supply, the upper bound is not quite infinite, and perhaps you are not that rich, but surely you have a lot more than $100 to offer, and I can afford to risk the $100.

Having not seen the "related problem" yet...

Take the other box.

When you randomly select, your expected return is 1.5x, where x is the amount of money in lesser box. (The total amount of money is x+2x; you get, on average, 50% of that, or 1.5x.)

Once you see the money, you don't know if you've received x or 2x, but it doesn't matter, because the frequency of getting either is .5. So you'll get, on average, .5x or x, or .75x.

If you switch, though, you may get 2x or you may get .5x. On average, it's 1.25x. So, you gain .5x by switching.

After the $100 is revealed, you know that x=100 or x=50. You also know that your expected return is 200+50= $250, divided by two, or $125 if you switch and $100 if you don't.

Oh, hmm, I did it the hard way :)

Expected Value thinkers, consider for a moment. You seem to agree that blinded or not you should take the choice (EV=125). Now what if the guy (blinded, not trying to "cheat" you) offers you the choice again? Does EV go up again?? Ad infinitum? You'll be switching so often that the transaction costs (time)will bury you. Get a grip...

I was just thinking what CB was thinking.

You make the first switch, because EV=$125, right? But then the other guy asks you to switch again. But you don't have any information different that before the first switch! So a the second switch has an EV=1.25*($125)!

And so on...

So there's got to be something wrong with the EV calculation.

I should have done the calculation with the "minor variation" - where the amount in the first box is unknown (X). So the EV of the other box is 1.25X. Then the EV of another switch is 1.25x(1.25X). And so on.

The issue (highlighted by the additional scenario) is that seeing the $100 in the current box doesn't add any information. You don't know whether you currently have x or 2 * x.

This is different from the Monty Hall problem, where you get new information when he opens the door you didn't pick.

The EV analysis made sense to me for the first round of swapping (regardless of whether you knew the cash amount of your box), but the analysis breaks down, as A.S. and CB noted, once the second round of swapping begins. If it always makes sense to swap, and there are only two boxes, why would you ever stop? If the EV analysis can't tell you when to stop swapping, honestly what good is it?

This dilemma was memorably played out in the Princess Bride when the villain asked the hero to choose which glass of wine to drink when of the two glasses had been poisoned. The hero's only safe bet: secretly poison both glasses and spend the previous five years slowly building up an immunity to poison. I like that answer better than the EV analysis.

Speaking of being risk averse, the difference between $100 and $50 is small enough for me that I would take a gamble on $200. But if the first box had $1,000,000, I'd keep it.

If you know before hand that you can make the switch, then it makes no rational difference which box you should take since it is just as likely as before you opened the box that the other has twice the value. But because we make decisions based on emotions, I would probably take the second box anyways.

The only time you should take the choice for sure is if your box had only one penny in it when you opened it because that is the smallest amount of positive value possible and the other box would have to have two pennies.

Cullen,
The princess bride scenario was a true "game". The initiator knows which has the iocaine powder, they both have everything to lose (and supposedly much to gain, although she was not that hot), and the initiator essentially has no active role except the requirement to drink at the end. Of course they can play head games with each other...
This is much simpler. The initiator has the option to make the offer to trade with the chooser. If he wants to win, and knows the contents of the boxes, then he will only offer if it is of benefit to him. If he is blinded, it is again a very different game from the Princess Bride scenario...

This is very different from the Monty Hall problem.

This is very different from the Monty Hall problem.

I only meant it was like the Monty Hall problem in the sense that the "uniform distribution" is a bad way to think about it. Should have been more explicit about that.

Aaron Bergman's got it right.

Huh? The original problem specifies that we are NOT told the distribution. Thus, the original problem asks what we should do, given that we DON'T know the distribution. It cannot possibly be a correct answer to say that what we should do depends upon information that the problem says we don't get.

Consider -- What if there problem had stated: "I have 2 boxes, and they are (with equal probability) either $50 and $100, or $100 and $200, but I'm not going to tell you which. You take a box and it has $100 in it. Should you switch?"

The answer to this question is NOT "It depends upon whether the other box has $50 or $200." That is part of the information that, by the call of the question, we are not given. For the same reason, the answer to the original question is not "The answer depends on the particular distribution you choose." The answer might reasonably be: "The question does not give us sufficient information to provide a conclusive answer," but the answer to the question actually asked (in which it is given that we are not told the distribution) is not "the answer depends upon the distribution." The correct answer may be (and in my opinion is) "There is not enough information to conclude with confidence which choice leads to a higher expectation value, for reasons that would take a while to explain" but that is different from "The answer depends on the particular distribution you choose."

Indeed, the claim "The answer depends on the particular distribution you choose" is LESS right than "The answer depends upon what is in the other box." If you are allowed to say that the answer is "based on" information you do not have, then surely the fact of what is in the other box is a more important fact (leads to correct choice 100% of the time) than the fact of what the original distribution is (leads to correct choice 50% of the time).

Bottom line -- if the solution is allowed to be expressed in terms of "The answer depends on [some fact that is withheld by the nature of the question]," then why is "The answer depends on the particular distribution you choose" a better answer than "The answer depends upon what is in the other box."?

[NOTE -- This is one of my all-time favorite puzzles as well. I can't let the "answer" be a one- or two-liner.]

Jeez,

You guys are hopeless.
The correct answer is I tell you if you don't show me how much money is in the second box I'll hit you so hard in the mouth it will wake up your dentist. I then decide with certainty.
Come on.

You're right, C.B., that the Princess Bride poison powder dilemma started out as a true "game", with the initiator knowing which glass had the deadly iocaine powder (excellent recall!). By the time the contestants sat down to actually choose and drink, though, neither contestant knew which glass had been poisoned. To that extent, the iocaine powder and money box examples are, I think, analogous.

That said, your point about the stakes being different in the two "games" is spot on. Which leads me to think that Slipperman's comment is the one that ultimately ought to be dispositive here - if someone gives you a box with $100, take it.

There may be a genuine mathematical way to reason this out, but I'm not sure it's been put persuasively here yet. It's out of my bailiwick, so I defer to the better brains in the e-room, or as the Dread Pirate Roberts might've said, "Aaas Yooouuuu Wiiiiish."

You guys are right, in that the EV analysis is off. There's no real solution (except for the clever odd-even point, above; and risk-averse accountings, e.g., there reaches a point--above $100 for me--at which you wouldn't switch).

I think you need to look at economics instead of mathematics.

The assumption that you should trade depends on constant marginal utility of money (almost always false) and on no risk-aversion (rare).

David,
The EV analysis even works in the blinded case. It doesn't matter what is in your box, and whether you know what is in your box. Or can you show a difference in the EV calculation between the two cases?

David,
Actually I'm starting to think you are right and I am wrong. The blinded case is still "x", and the EV of the trade is 1.25x. So no trades beyond the first. Perhaps that is true. I'll think about it some more...
But my original answer is still undoubtedly correct: in the non-blinded case (at least not blinded with respect to the box offerer), the offerer has absolutely nothing to gain by letting you exchange if he knows he has the box with the greater value (2x). Therefore you should not exchange.

If everyone is blinded (or even just the offerer/initiator), then maybe the change is worthwhile...

A.S. must be right--there is something wrong with an approach to this problem that says you always want the other box. My take on it is that the expected value theory is based on the statistical assumption that you could repeat the experiment multiple times. That is, you are given a sum of money and have an option to keep it or go through the 50/50 chance of doubling or halving it. If you can do this multiple times, always picking the option, you will eventually enrich yourself. The expected value theory applied to choice in an individual situation says you should do whatever would, if you faced the exact situation and information many times, would be the best decision over the long run. However, if you repeat the experiment, you will start to learn additional information based on the dollar amounts you find under the boxes. To restrict this information as part of the thought game's rules is artificial.

Expected value makes sense in a business environment where you are doing more or less the same kind of thing over and over with the goal of amassing money. However, when you are doing something only once, I would think that an individual's own current wealth, risk aversion, etc., would do a lot more work than the expected value.

Over multiple iterations, this double-or-half game is incomplete as a model for human behavior also because it assumes a generous game provider. It is somewhat akin to a false dichotomy, in that it sets up an apparent logical conundrum that could not exist in nature. There is nothing in this game for the person providing the money under the boxes. Once we assume that the person providing the game wants to make money on it (after all, what point is expected value theory if we discount the most fundamental economic assumption), then that person has to either charge a fee or change the payout structure. Let's say they don't change the game, but charge a flat fee for each iteration ($100). With the knowledge that all people desire to make a profit over time, if the provider is in the business of doing this game, then we will be able to deduce a range of likely dollar amounts under the boxes. For instance, if we find $10,000 under the first box, we know that this outcome could only occur rarely if the flat fee is $100, and we will ignore the statement that "half of the time, the other box contains 1/2 the amoutn and half of the time it has 2X the amount" because even if strictly true as a mean average, it discounts the additional information that the dollar amount provides in the instant transaction.

It feels like I am still missing something, but that is my attempt to explain away the apparent logic-loop.

Except of course that the EV analysis never works, at least in the standard way, because you're assuming a uniform prior distribution. Of course the answer depends on your prior distribution (and also on your utility function and risk function). Luckily I'm Bayesian enough to not be overly concerned by the amusing puzzle.

It's an important example of the importance of priors. The "trick" is that people, given the statement of no knowledge of the distribution, assume that the distribution must be uniform.

What if the question was: in one box there is one million times the amount in the other. I open the box and find 10 thousand dollars. Now, there may be 10 billion dollars in the other box (hope its big), but the likelihood is that it is a penny.

You are being asked to make a choice which is more likely 50 and 100 or 100 and 200. That judgement depends on the wealth of the person asking the question and the amount of money available. I'd switch. Kevin seems kind of loose with the money.

For this reason, there's no way that you can say that there's a 50% chance of there being $200 dollars in the other box and a 50% chance of there being $50 dollars in the other box.

OK, Symmetry, so what are the probabilities? 75/25? 60/40? We know it has to be one or the other.

It cannot possibly be a correct answer to say that what we should do depends upon information that the problem says we don't get.

Silly frequentist. ;)

As I've made clear, put this PhD probabilist down with Mr. Bergman.

Random,
The "distribution" plays out like Mike Keenan said:

You are being asked to make a choice which is more likely 50 and 100 or 100 and 200.

And you can't assume that the two scenarios are 50/50. So that is why you can't do the EV...

Aaron Bergman is still right. If you're not happy with his answer because it contains "jargon," then replace the jargon with some measure theory:

http://mathworld.wolfram.com/ProbabilityMeasure.html
http://mathworld.wolfram.com/Measure.html

Theorem: There does not exist a uniform probability measure over the measurable space of natural numbers.

Proof: Assume there exists a uniform probability measure over the set of natural numbers N.

A uniform probability measure has the property that m(X) = k for all X such that X is a member of N. By the definition of a measure, k is a non-negative real number.

If k > 0, then there exists an integer j such that jk > 1. Define Y to be the set of the first j integers. Then, m(Y) = jk (union of disjoint subsets). m(N) >= m(Y) > 1. Since m(N) = 1, we conclude that k cannot be greater than zero.

If k = 0, then m(N) = 0 (union of disjoint subsets). Since m(N) = 1, we conclude that k cannot be zero.

Therefore, we have a contradiction, and we conclude that no such distribution exists.

It's the same problem as asking whether the sentence "this sentence is false" is false. You're given a logically impossible premise, so there is no answer.

What about sheer curiosity? In the first problem, I'd switch boxes just to find out what the other box had.

If the game is honest, it doesn't matter if you switch.

Here's why the EV analysis doesn't work:

Suppose you pick the first box. You know that there's money in it, but not how much. Then if the EV analysis were true, the expected value of the second box would be 1.25 the value of the first box REGARDLESS OF WHETHER YOU OPEN IT. But the same would be true if you chose the second box.

If the game is dishonest, you should keep the $100, because it's more likely you chose the right box. If you chose the wrong box, why would he ask you to switch.

"I have two boxes."

Box X and Box Y

Only two boxes I note.

"Each has some positive amount of money in it, but I will give you no information about the possible dollar amounts other than the fact that one box has exactly twice the amount of money in it as the other."

X and Y values are positive integers

X=2Y or X=1/2Y

"You randomly select one of the two boxes, open it, and find $100 inside."

X=100

Solve for Y

Y=200 or Y=50

"I now give you the option of keeping the $100 or switching boxes with me and keeping whatever's inside the other box. Which should you choose?"

Expected Value of Y:

1. number of possible values for Y - 2
2. probability for each value is thus %50
3. Solve for expected value:

E(Y)= 1/2*200 + 1/2*50
E(Y)= 125

Now - the value of Y is not ever going to be $125 - however this Expected value thingy is telling me to take the second box.

I think opening the first box DOES give extra information -- the amount inside is an even number.

If the first box you opened had an odd number in it (say, $5.01), then you know it can't be the box that contains the 2x higher amount (since an odd number can't be 2x anything.) Since the box you opened has an even number, keep it. It has a higher probability of being the "2x" value, since it is even.

The second example does not give this information, since you haven't opened the box.

The odd/even angle is the key.

Both boxes cannot hold odd amounts (to the penny or the dollar) because one box holds "exactly" twice as much as the first). This reduces the number of possibilities so that knowing one box hold $100 does affect your choice.

Before the first box is opened, the possibilities are [box 1/box 2]:

odd/even, and box 2 holds twice the amount in box 1
even/odd, and box 1 holds twice the amount in box 2
even/even, and box 1 holds twice the amount in box 2
even/even, and box 2 holds twice the amount in box 1

When box 1 is revealed to hold an even amount, we are left with these three possibilities [box 1/box 2]:

even/odd, and box 1 holds twice the amount in box 2
even/even, and box 1 holds twice the amount in box 2
even/even, and box 2 holds twice the amount in box 1

It is twice as likely that box 1 holds the greater amount. Keep the $100.

I can see how, assuming uniform distribution, the EV of the second box is $125. I cannot, however, see how the EV of the first box, after choosing the second box, could be anything but $100.

Bruce,
I don't have a PhD in statistics, but I'm fairly comfortable saying that "Probabilities are just a lack of information" is just not true. Known probabilities are actually a wealth of information. Uncertainty is a completely different art...

Can someone with a PhD in statistics explain further?

I think part of what's feeding the battling intuitions here is that, in fact, 100/200 is *not* equally likely as 50/100, because in general in life higher amounts are less frequent than lower amounts.

But I want to go back to the point that, if you really have *no information* about the likelihood of two choices, then each choice is 50% likely. We're not talking about the spin of quarks here, so a priori the odds that each box contains a certain dollar amount are either 1 or 0. The probability we assign just reflects our level of uncertainty about what the boxes contain. But like CB, if someone can give a theoretical explanation of why I'm wrong, I'd love to see it.

www.math.hmc.edu/funfacts/ffiles/20001.6-8.shtml

www.j-bradford-delong.net/movable_type/archives/001395.html

My previous proof is wrong.

Independent of the initial box chosen, a switch would be made!

Barely passed, also barely passed Econometrics -- but luckily this problem is not at all econometric!

Regardless of "distributions", if a switch is called for even when you don't know what you have, then the original choice is moot. If offered the choice again, you would switch again. This can't be helpful.

So if the expected value analysis doesn't hold up, then what does this tell us about the probabilities to start with.....

The problem I see with traditional EV is that there's nothing to suggest that the person running the puzzle is as likely to put $200 in a box as he is to put $50 in a box.

Symmetry: fine, I see your point, it's the same as Spieler's, who made it crystal clear in one sentence while lots of other people here were running around with lemmas and distributions:

The problem I see with traditional EV is that there's nothing to suggest that the person running the puzzle is as likely to put $200 in a box as he is to put $50 in a box.

Suppose I specify that they are equally probable. EV still can't be right. What's the mathematical reason for that?

As a lawyer with more common sense than math skills, I'm enjoying this fascinating discussion. Here's my question to those saying you ought to switch boxes based on the expected value.

Going in, you have a 50% chance of choosing the box with the higher value, and 50% chance of picking the lower value. I presume that if you played this game 1000 times, 50% of the first picks chose the higher value box and 50% chose the lower value box. If that is so, then switching 100% of the time will result in ending up with the higher value box 50% of the time and the lower value box 50% of the time. If that is right, it shouldn't matter whether you switch or not. What am I missing here?

Simple Guy:
Nice point, very elegant. You are not as simple as you would like to admit...

And Box Keeper wins it!
Discussion of Expected Values is now concluded.

Sheesh. The answer is in the rec.puzzles FAQ.

http://brainyplanet.com/index.php/Envelope%20Solution

Unfortunately this seems to be on a slow server that often returns blank or truncated pages.

I believe that the Expected Value analysis as outlined above (say by Alceste and many others) is incorrect because you are failing to take into account the initial value of the pick.

The two boxes have a smaller amount 'X' and a larger amount '2X' in them. Assuming your choice is random, your expected value on choosing (and though it doesn't effect the analysis opening) is (X+2X)/2=1.5X

If you have chosen the box with the smaller amount you get X. If you have chosen the box with the larger amount you get 2X.

Now the switching comes in. If you initially had chosen the box with the smaller amount you will now receive 2X. If you initially had chosen the box with the larger amount you will now receive X. The expected value post-switch is 1.5X which is the same as the pre-switch scenario.

Therefore, on an expected value analysis you have no reason to switch.

On a marginal value analysis you may have a good reason to switch depending on different values of X. (For a $1 revealed amount I always switch because $2 in hand is worth more to me than the 50 cent loss even though my EV is the same in both cases. For a $1 million revealed amount I might not switch because the $500,000 'loss' would make a big difference to me. For a $100 million revealed amount I would probablly switch because the $50 million loss wouldn't feel as much to me as the gain I could get by getting $100 million. But in all cases the EV of switching or not switching remains the same.)

I can't believe this has gone on for this long since the first comment was dead on correct.

Given that the first statement was essentially "You can't decide" disguised in language which says "I'm smarter than you," it can hardly be a shock that the discussion went on.

All the yammering about uniform distributions and natural numbers and whatnot is basically people who know too much missing the point. Sort of like what would happen if you asked a law professor "Can I legally do X?" and got back a 180 page law review article discussion deontological vs. welfarist thories of justice.

My (pre-law) training was in physics, and I can assure you that if Choset puts up a puzzle about the uncertainty principle, my respone will not include the words "commutation relations" however relevant they may be.

How about, "shoot Kevan and take both boxes?"

So the question Howie Kevan poses us is,

Deal... or no deal?

Given that the first statement was essentially "You can't decide" disguised in language which says "I'm smarter than you," it can hardly be a shock that the discussion went on.

It wasn't disguised in language to me at all. It was the most parsimonious language possible, and completely accurate.

Sort of like what would happen if you asked a law professor "Can I legally do X?" and got back a 180 page law review article discussion deontological vs. welfarist thories of justice.

Except it was hardly 180 pages. More like asking a law professor some question and he responds succinctly by using the precise Latin legalese for the situation, which unfortunately does not always illuminate without a longer response. Ask a math question, get a math answer.

Naturally, of course, to spell out the answer in more detail requires using more words. But there you go.

Another attempt, till there's more explanation:

So at first, I have no idea whatsoever how much money I'm going to get. It could be one dollar or it could be a trillion dollars. Ok, probably not a trillion (but again, if the first box is half a trillion, why not?).

Then I open my box. Blam! It's a million dollars. I'm thinking, "Sweet! That's huge! I'm getting at least 500k here." So now I have the question of whether to switch.


So how do I possibly not switch? Not knowing anything else, I don't see how it can be anything other than 50/50 for 500k or 2 mil.

Maybe the problem is that it wouldn't actually be 100 dollars. If there were truly no limits, the number would be something with like a million zeros on it, right? If the number were truly picked randomly.

If there are no limits, though, I still say you have to switch, if you're given a specific number.

This is one of my favorite puzzles too -- or should I call it, not a "puzzle" but a philosophical problem, because the answers I come up with never make much sense.

For the reasons discussed by some prior commenters, if I open Box 1 and it contains $100, I will think to myself "if I change to Box 2 I will either gain $100 or lose $50. Being risk-neutral, the expected gain outweighs the expected loss. Therefore, I will change to Box 2."

BUT ... nothing in that reasoning depends on the specific value of $100. I would have reasoned "this box contains X, changing to the other box will lead to either a gain of X or a loss of 1/2 X, and therefore I will change," no matter what X was.

Nor does anything in this reasoning depend on it being YOU who chooses the first box. Suppose the situation were that I (the player) gets to choose one of the two boxes, open it, note the amount inside, then keep that box or change to the other one.

Based on the above reasoning, my optimal strategy would always be "pick one box, open it, and then, whatever is in it, reject it in favor of the other box." But if I'd opened the other box to begin with, by identical reasoning, I would have changed that box for the first one.

In any event, that seems to be the reasoning I would go through ... except it doesn't seem to make a whole lot of sense.

Robert Lyman,

I actually find it fairly condescending when someone assumes that I won't understand them, and I expect that a lot of people feel the same way.

If you don't like rigorous math, skip over my posts, and don't take them so personally.

More like asking a law professor some question and he responds succinctly by using the precise Latin legalese for the situation, which unfortunately does not always illuminate without a longer response.

Yes, that is a better analogy. But if the answer is "you can't decide without more information" then there is no need to dress it up in Latin or anything else.

Nor does it make sense to stand on what lawyers would call "nice" distictions like the distribution when the average layman is just going to assume 50/50 and go from there. The interesting question is what happens in the 50/50 case, and why are our intuitions so contradictory?

I'm not opposed to parsimonious languge in and of itself, having used it in many discussions with physicists and lawyers, but it should hardly surprise anyone that non-specialists don't accept the parsimonious explanation only accessible to specialists. To return to the uncertainty principle example, I would hardly expect this sentence to end all discussion here at VC: "The paradox is resolved because you cannot have simultaneous eigenstates of non-commuting observables."

Bruce,

The point of all the math types here is the following: there is no way to completely speicify the process by which amounts are chosen for the boxes are such that the answer to the question "what is the probability that the other box is twice the amount I have now?" is always 50/50. For instance, if the process is to draw from a uniform distribution between $0 and $100, put that amount in one box and put twice that amount in the other box, then if you open a box and it has over $100 in it, the probability that the other box has $200 in it is zero, not 50%.

Just because you weren't told the procedure by which amounts were put in the boxes (other than one has twice the other) doesn't imply you have no beliefs about the matter. You must have some belief about where these amounts came from. And whatever these beliefs are, the probability of the other box having twice the amount of the box you opened is not 50%.

Robert Lyman,

It is not correct to say "you can't decide without more information". Like I state in my previous post, you can't help but have beliefs regarding the process which determines how much money is put in the lower amount box. The answer is not "you can't decide" but you whether you should switch depends on what these beliefs are.

My answers are that in either case (problem 1 or problem 2) it doesn't matter whether you switch or not. I'm assuming that you know absolutely nothing about the prior distribution (I agree it can't be uniform), so any factors about the wealth of the person running the game (or even the wealth of the universe, for that matter) should be ignored. If you like, instead of thinking of it as a dollar value, you can just think of it as a number written on a piece of paper. It can be arbitrarily large.

As for the reasoning behind my answers, I'll wait a bit, since the comments keep flowing.

Kevan, is my 3:20 reasoning extensive enough or did I miss something?

Here's another variation on the problem that gets away from the equal distribution issue (I think). You're given a box and you open it. It contains $100. Now Kevan presents you with two more boxes. One box contains $50, and one box contains $200. You can stick with the box you have, or you can choose one of the 2 new boxes. What should you do?

Now replace my dollar values with variables. You've switched, and Kevan shuffles the 2 boxes (including the one you originally had) 3-card-monte-like so that you don't know which is which. You had x, and now you have either x/2 or 2x. Should you switch again? The EV of switching again is .25(x+x/2) + .25(x+2x) = 9x/8, compared to the 5x/4 you now have. (Alternatively, you are given the option to switch back to the box you originally had. Should you? No.) So you should switch once, then stay pat.

I don't see how this problem is any different than the one above, and it doesn't require assuming some sort of uniform probability distribution among all integers, just among 2 or 3 boxes.

CB - the discussion's proceeded far enough along toward actually solving the problem, that whether the iocaine powder game is really analogous to the switched box game is probably no longer a germane question. That said, in the interests of the completeness of the record, I give you:

Vizzini: I'm afraid so. I can't compete with you physically, and you're no match for my brains.
Man in black: You're that smart?
Vizzini: Let me put it this way: Have you ever heard of Plato, Aristotle, Socrates?
Man in black: Yes.
Vizzini: Morons!
Man in black: Really! In that case, I challenge you to a battle of wits.
Vizzini: For the princess? [Man in black nods] To the death? [Man in black nods again] I accept!
Man in black: Good, then pour the wine. [Vizzini pours the wine] Inhale this but do not touch.
Vizzini: [taking a vial from the man in black] I smell nothing.
Man in black: What you do not smell is Iocaine powder. It is odorless, tasteless, and dissolves instantly in liquid and is among the more deadly poisons known to man.
Vizzini: [shrugs with laughter] Hmmm.
Man in black: [turning his back, and adding the poison to one of the goblets] Alright, where is the poison? The battle of wits has begun. It ends when you decide and we both drink - and find out who is right, and who is dead.
Vizzini: But it's so simple. All I have to do is divine it from what I know of you. Are you the sort of man who would put the poison into his own goblet or his enemy's? Now, a clever man would put the poison into his own goblet because he would know that only a great fool would reach for what he was given. I am not a great fool so I can clearly not choose the wine in front of you...But you must have known I was not a great fool; you would have counted on it, so I can clearly not choose the wine in front of me.
Man in black: You've made your decision then?
Vizzini: [happily] Not remotely! Because Iocaine comes from Australia. As everyone knows, Australia is entirely peopled with criminals. And criminals are used to having people not trust them, as you are not trusted by me. So, I can clearly not choose the wine in front of you.
Man in black: Truly, you have a dizzying intellect.
Vizzini: Wait 'till I get going!! ...where was I?
Man in black: Australia.
Vizzini: Yes! Australia! And you must have suspected I would have known the powder's origin, so I can clearly not choose the wine in front of me.
Man in black: You're just stalling now.
Vizzini: You'd like to think that, wouldn't you! You've beaten my giant, which means you're exceptionally strong...so you could have put the poison in your own goblet trusting on your strength to save you, so I can clearly not choose the wine in front of you. But, you've also bested my Spaniard, which means you must have studied...and in studying you must have learned that man is mortal so you would have put the poison as far from yourself as possible, so I can clearly not choose the wine in front of me!
Man in black: You're trying to trick me into giving away something. It won't work.
Vizzini: It has worked! You've given everything away! I know where the poison is!
Man in black: Then make your choice.
Vizzini: I will, and I choose...[pointing behind the man in black] What in the world can that be?
Man in black: [turning around, while Vizzini switches goblets] What?! Where?! I don't see anything.
Vizzini: Oh, well, I...I could have sworn I saw something. No matter. [Vizzini laughs]
Man in black: What's so funny?
Vizzini: I...I'll tell you in a minute. First, lets drink, me from my glass and you from yours.
[They both drink]
Man in black: You guessed wrong.
Vizzini: You only think I guessed wrong! That's what's so funny! I switched glasses when your back was turned! Ha ha, you fool!! You fell victim to one of the classic blunders. The most famous is never get involved in a land war in Asia; and only slightly less well known is this: Never go in against a Sicilian, when death is on the line!
[Vizzini continues to laugh hysterically. Suddenly, he stops and falls right over. The Man in black removes the blindfold from the princess.]
Buttercup: Who are you?
Man in black: I'm no one to be trifled with. That is all you'll ever need know.
Buttercup: And to think, all that time it was your cup that was poisoned.
Man in black: They were both poisoned. I spent the last few years building up immunity to iocaine powder.

The flaw in most reasoning (including Kevan's) here is that the choice is not between the boxes but between the scenarios: 50 and 100 or 100 and 200. Those are not equally probably. Now, if we ignore the finite amount on money in the universe, maybe they are the same, but that changes the problem.

To clarify, the problem could read: Has Kevan put 50 and 100 into the boxes or has Kevan put 100 and 200 into the boxes.

I think that the double/half point misleads some people. Let's strike it out: " I have two boxes. Each has some positive amount of money in it, but I will give you no information about the possible dollar amounts other than the fact that one box has exactly twice the amount of money in it as the other. You randomly select one of the two boxes, open it, and find $100 inside. I now give you the option of keeping the $100 or switching boxes with me and keeping whatever's inside the other box. Which should you choose?".

The puzzle remains the same.

The puzzle remains the same.

How does the puzzle remain the same? In the original you know something about the contents of the second box. In your version you know literally nothing.

Dan

I think my altered version is equivalent to Dan's. In both cases, it seems you should switch once and hold.

I don't think you switch once and hold under my version. Each time you have a box, there is a 50% chance the other box contains double and a 50% chance the other box contains half, because it was determined with a coin flip. Why isn't the expected value analysis that leads to infinite switching valid here?

Dan

Perhaps I'm dense, but it seems like a straightforward EV problem, if you know beforehand that you get the option to switch. Transaction costs seem pretty minimal for one switch.

If the guy gives you that option only after you've chosen, though, and it is his money, it seems to me like you're in the Princess Bride situation.

The possibility of additional switching is not part of the problem and can't be reasonably analyzed unless you restate the problem with the necessary info. If the third box is created only after you have chosen the first two, it inevitably allows for experimenter manipulation and the logical answer would be to stop at two. If there are two "third boxes" each placed before you depending on which second box you got, I think it's back to the EV problem.

Try approaching the problem using a proability tree.
For the first pick: p (high) =.5
p (low) =.5

For the second pick p (high given high) =0
p (low given high) =.5
p (high given low) =.5
p (low given high) =0

So the odds of getting the high box on a swtich are the same as getting on your first pick .5. The amount discovered in the box is irrelevant.

for calrification high given high = .5(p of high at first pick) * 0 (p of high at second pick given taht you picked the hight the first time) = 0

similary low given low = .5 * 1 (p of getting high first time, times p of getting high on switch given that you picked low first time)= .5

high given low = .5 * 1 = .5

low gien low = .5 * 0

I meant "no that's my point, the odds are the same no matter what box you pick, no matter whether you switch or not"

How does the puzzle remain the same? In the original you know something about the contents of the second box. In your version you know literally nothing.

The double/half part of the puzzle does not give you any useful information to keep the original box or switch it either.

I think the answer is to not switch in Dan's version.

Dan says he puts X in Box A and flips a coin to put either X/2 or 2X in Box B. He then hands you a Box. Care to switch Boxes?

If you knew you had Box A, you would switch because your expectation goes from X to 1.25X. If you knew you had Box B, you would not switch because your expectation goes from 1.25X to X.

But, you don't know which Box you have - so you have no reason to switch.

Dan earlier said:

I don't think you switch once and hold under my version. Each time you have a box, there is a 50% chance the other box contains double and a 50% chance the other box contains half

That's true, but if you hold Box A you are doubling or halving from a constant amount (X). If you hold Box B, you are halving from a higher amount (2X) than you are doubling from (X/2). So, Dan's conclusion that you always increase your expectation doesn't hold.

Wow, who knew math people were as weasily about answering questions as lawyers?

Simple question: One box contains $X, the other contains $2X. You pick one, and it contains $100. Should you switch?

First math-people answer: "Well, assuming an infinite amount of money in the universe, you can't tell."

Fine, we limit the amount of money in the problem. Second math-people answer: "if the limit is under $200, definitely don't switch because then you know you have $2X."

OK, screw limits. We specifiy a distribution: 50/50, decided by flipping a coin. Should you switch?

Third math-people answer: "I leave that as an exercise to the reader."

It's nice that you've solved the trivial cases for us. That's why we have experts: to dodge the question!

It's clear that there's some problem with straightforward EV. Mev may have explained it, and if so I'm sorry that I don't know enough Baysian probability to make sense of his explanation. But to me what's really interesting here is to understand when one can, and cannot, use EV in other, less artificial problems. Given how common EV is in the real world, I'd love to know that but so far I'm pretty much stumped.

I submit that Dan's altered problem - with the intervening coin toss - has the same solution as Problem 1, with Kevan's clarifications.

1. We agree that the likelihood of 50 or 200 in the second box is the same in Dan's scenario, due to the coin toss. That is 50/50. And thus, switching does not matter.

2. Assume, for the sake of argument, that in Problem 1, there is some difference between the likelihood of 50 and 100 or 100 and 200. How would you know this as a rational player? You couldn't know. Hence you have no basis to decide which scenario is more likely, even if in objective fact one might be. Thus, you will not benefit yourself by switching.

As an aside, has someone created a simulator of this game and run it through iterations to test these hypotheses with some empirical data?

Mr. Lyman,

In my experience, when business people applied EV in the weighted average fashion (not the only way mind you, just the easiest one to explain), it was used as an enhanced way to predict value.

In other words, a businessman says, well there's a 30% chance I make $50, 40% chance I make $75, and a 30% chance I make $100, so I should overall expect this - and then he computes a weighted average. Of course this is all guessing in the first place.

But, it was seen as preferrable to just sticking your finger up in the air and saying "this will make $80."

I'm not a statistician - so I'm not sure how they use it. It's clear that a lot of people hear probably learned EV in b classes in undergrad or law school, based on their assumption that you can straightaway apply to probabilty problems. Nope.

from

The Solution. Analogously, in the Two Envelope Paradox, the expected value of X in the "2X" part of the formula (where Envelope A is the envelope with less) is less than the expected value in the "1/2 X" part of the formula (where Envelope A is the envelope with more). You would expect less in Envelope A if you knew that it was the envelope with less than you would if you knew it was the envelope with more. Allowing X to have different expectations in different parts of the formula in this way is like comparing apples and oranges. The "X" in the "2X" just isn't the same as the "X" in the "1/2 X" part.

The better calculation in the Two Envelope Paradox involves setting X to the amount in the envelope with less and calculating the expected value of Envelope B as (.5)X + (.5)2X = 3/2 X -- and the expected value Envelope A likewise as (.5)X + (.5)2X = 3/2 X. In these calculations the expectation of X in the first term of each equation is identical to its expectation in the second term: The expected amount of money in the envelope with less does not change depending on whether Envelope A is the envelope with more or Envelope B is.

Dan is right. And now Aaron appears to agree. I agree (not that you care), but I believe that Aaron's most recent post shows that the distribution P(x) is irrelevant to the solution.

Dan's coin-flip variation and also Bruce's 3-box variation are both true "double or half" gambles. Any rational person should accept such a gamble. The original formulation of the two-box puzzle (whether you see the contents of the box or not) is entirely different. You are not being being offered a double-or-half gamble based on what is in the first box.

To see why, imagine that there are two players simultaneously looking into the two boxes, under each of the two possible scenarios that follow from the presence of a $100 bill in one box. Each player might be inclined to believe (erroneously) that the chance to switch would amount to a double-or-half gamble, but of course that's not possible.

If the actual amounts are $50 and $100, the person looking at the $50 will think "it could be $100, or it could be $25" (incorrect EV of $62.5) of while the person looking at the $100 will think "it could be $200 or it could be $50." (incorrect EV of $125). These EV calculations total much more than what is in the two boxes combined. The reason they are collectively skewed upward is because they contain a figure (here $200) that is not contained in any box, and that upward skew is not fully compensated for by the fact that there is also a downside figure that does not appear in any box (here, $25). The fact that only the two middle figures are possibilities, and not the outliers, is why neither player is actually justified in viewing the gamble as "double or half."

It works the same way if the actual amounts are $100 and $200. Then the person looking at the $100 will think "it could be $50 or it could be $200" while the person looking at the $200 will think "it could be $100 or it could be $400." Again, these ruminations are flawed, because the total set of figures under contemplation contains an upside figure not contained in any box (here, $400) that is not mathematically made up for by the presence of the low figure not contained in any box (here, $50).

Going from Box A to Box B improves your expectation. Going from Box B to Box A reduces your expectation by a like amount.

I think that's incorrect. If I go from Box A to Box B I improve my expected value. But I don't know which box I have when I am holding Box B. Box B is either twice Box A or half of Box A. Similarly Box A is either twice Box B or half Box B. If I go from Box B to Box A, then my expected value is .5(2B)+.5(B/2)which equals B + .25 B which is 1.25 times B, so I should switch back. That leads to perpetual switching.

Dan

Box Keeper -

I don't see how my version is materially different than the classic version. The only difference is that I have specified the probability distribution of the second box. It's exactly the same problem otherwise. However, specifying the probability distribution eliminates what seems to be the main objection to the problem.

Dan

The problem with straightforward expected value calculation is that, by necessity, you need to make some assumption about probabilities to do the calculation. The obvious one, that the value you observe when you open the box is irrelevant for the computation, turns out to be impossible.

Aaron -

How does that hold in my coin flip version? It does seem that the value you observe when you open the box is irrelevant.

Dan

Dan:

It is not true that your expectation value changes when you switch. The problem is when you start out and say that, without looking, assume that the value in the box you have is B. There is a probability distribution for that value that you have to take into account for the rest of the calculation.

The paradox with the original formulation is that, when you open the box, no matter what the value, you should switch which would imply that you should switch even before opening the box. This contradicts the symmetry of the situation.

The resolution, as I said above, is that there does not exist a distribution of probabilities with finite mean that leads to the situation where you should always switch. Switching is irrelevant before you open the box and, after the box is opened, the decision depends on the probability distribution for the values in the box.

"Without the actual distribution, the only correct answer is "it depends on the distribution and what you saw in the first box. Give me the distribution and I will tell you how."

That is wrong. The distribution between the two boxes is known. It is x and 2x. What is not known is whether or not you have opened the X-box (heh) or the 2X-box.

When you initally select the box, before you open it your expected value for a random selection between the two boxes is [(50% times X)+(50% times 2X]/2. Therefore your expected value (EV) after the initial selection is 1.5X.

If you choose to switch you are switching from the bigger value to the smaller value half of the time. The other half of the time you are switching from the smaller value to the bigger value. You end up with the bigger one half of the time and the smaller one half of the time. That is [(50% times 2X)+(50% times X)/2. Therefore your expected value after the switch is 1.5X.

Since this is the same as your expected value before the switch, there is no (expected value) reason to switch.

ok ok, without getting into a debate about the nature of disunctions and how it is possible to pick 50 or 200, i think you seem what i am saying, only one of those values is possible, the other should bot be used in calculating ev.

Dan,
The difference between the original version and the coin flip goes to what makes the original puzzle puzzling. People make the mistake of treating the choice whether to switch in the standard puzzle as if it were identical to a double-or-half gamble (that is, as if it were actually true that there's a 50% chance of getting half of, and a 50% chance of doubling, whatever you spy in the box). But they are wrong to do so, as I explained above. That is, they are wrong to do so unless someone has come along and helpfully specified that the actual value in the second box is either exactly half what you have or exactly double what you have, and that there's a 50% chance of each outcome. If someone does specify that, there's no puzzle at all -- everyone should just take that gamble. With it unspecified (as it is in the original puzzle), the boxes are identical in EV terms.

Robert:

"Sebastian and Box Keeper seem to have gotten it right but I can't say exactly why)."

the various arguments above to the effect that "you don't know any more after opening the first box than before" present the logical, or shall we say, "intuitive" answer. the arguments given by S and BK are abbreviated versions of the following formal calculation. note the importance of defining random variables explicitly and that it's a conditional expectation that's being calculated:

Consider boxes A &B. one contains an amount of money D, the other 2D, where D is finite. define random variables X and Y which are the amounts of money in boxes A and B respectively. Choose a box, say A, and open it to find an amount of dollars, necessarily either D or 2D. what is the conditional expectation of the amount of money in box B (Y) given the amount in box A (X).

E{Y|X}=

DxP{Y=D|X=D}xP{X=D} +
2DxP{Y=2D|X=D}xP{X=D} +
DxP{Y=D|X=2D}xP{X=2D} +
2DxP{Y=2D|X=2D}xP{X=2D}

= Dx0x0.5 + 2Dx1x0.5 + Dx1x0.5 + 2Dx0x0.5
= 0 + D + .5D + 0 = 1.5D = E{X}

Obviously Kevin intended us to use an RFID scanner to read the secret New World Order metal strips in the bills. Otherwise there would be no answer and Kevin couldn't possibly give us a broken puzzle.

Ok, so in additional to the coin flip I also specify that the initial amount was selected at random, so all initial amounts are equally likely. Why does the paradox not arise then?

Because that's an impossible specification. You cannot select a random initial amount with equal probabilities for all natural numbers (or real numbers if you prefer.)

(Gotta go now...)

Dan, your example doesn't help/change things; the answer is still that a switch will not yield a higher expected value.

Having chosen a box with x amount ($100), we STILL DON'T KNOW whether the other is x/2 or 2x. In your scenario, this means we don't know whether or box 'x' was your first box (random amount) or the second (determined by coin toss).

My initial critique of the expected value analysis still holds. For the "pure" version of the problem (no limit on money, no risk aversion, etc), it doesn't matter what is in the box, the relative value of the boxes, or whether you look or not. If it is worth switching once, then it should be worth switching back. Therefore it is not worth switching. If you play this many times, the switch or not-switch strategies will be equal.

Box Keeper (at 3:04 PM) has the most straightforward explanation of the probability tree that shows this.

Aaron, you keep fighting the hypothetical to dodge the question. If it's unbounded, you say the problem is "impossible." If it's bounded, you bring up the lame example of an amount in the box near the bound.

For heaven's sake, how about the case of a bounded problem where the amount is nowhere near the bound???? That's the only interesting case.

Kevan,
Would you just declare Box Keeper the winner already?! (As I did 4 hours ago.)

If we take Dan's formulation of (a slightly modified) Problem 1, we still conclude it makes no difference whether or not you switch.

The problem: We put an amount X in Box A. We flip a coin and half the time put 2X in Box B and half the time put X/2 in Box B. We pick a Box (don't know which one) at random, open it up and see there are Z dollars. However, Z doesn't represent any particular amount but rather the complete set of possible values of Z in the likelihood that they will occur. Should we switch?

Note that we can't answer the problem for any particular value of Z because we need to know the distribution of X. Trivial example: for Z=100 we would never switch if X always equaled 50, but always switch if X always equaled 200. But, if we integrate over all possible values of Z, we can solve the problem.

Example: We always put 50 in Box A. Z=50 with Prob=1/2 (we chose Box A). Z=100 with Prob=1/4 (we chose Box B and it had 2X). Z=25 with Prob=1/4 (we chose Box B and it had X/2). When we see 50 and switch, our expected value goes up by 12.5. When we see 100 and switch, our expected value drops 50. When we see 25 and switch, our expected value increases by 25. Overall change in expected value by switching = 1/2*12.5 - 1/4*50 + 1/4*25 = 0.

Answer: it doesn't matter when we integrate over all the possible values of Z. The paradox appears only when we consider a specific value of Z (and goes away when we integrate over all Z).

First, let me join the mathematicians in agreeing that the seeming paradox exists because the distribution isn't specified, and a uniform distribution over an unbounded domain is impossible.

However, I think Robert Lyman has an understandable grievence with the mathematicians so far, why can't we give some usable guidance. Well, with the caveat that there are all sorts of bizare distributions which will invalidate these guidelines, here goes.

We must have some a priori assumptions about the distribution -- at the very least it is bounded by all the money in the world. (at the very least there would be significant problems with inflation and/or security at very high values which would invalidate the linearity of the value of a dollar) There are two assumptions which lead to easy decision rules.

a) If the amount in your envelope can not be halved, then it must be the smaller amount so you should switch. Given many likely specific scenarios (e.g. the envelopes contain legal tender)this will apply.

b) If the amount is more than half the maximum you believe possible, then it must be the larger, so don't switch.

c) Under any 'economic' distribution (i.e. excluding those where the probability increases with the amount over a large region) if the amount is significantly below the maximum, then the probability of it being the smaller is no more than, and likely less than, one half. Thus you should be at least as likely to switch as in the pure half or double 50/50 case. For instance, in the case of a uniform, continuous bounded distribution, for X < 1/2 the maximum the probability that X is the smaller amount is 2/3. Thus the EV of switching is 1.5X (1/3 * .5 X + 2/3 * 2X)

So for the given case, as long as I belive that there is a good chance that Kevin would put at least $200 in an envelop, then I should switch.

We must have some a priori assumptions about the distribution

Yes, if the problem is should you switch when you see exactly $100 in your box. No, if the problem is should you switch when you see X dollars in your box, where X is a random variable and we integrate over all X. In this latter case, your expectation will not change by switching not matter what the disribution of X is.

And the specific-value-of-100 versus all-possible-values-of-X distinction is at the heart of the paradox.

Christ, would people ignore the fact that there's a finite amount of money in the world. This is a probability problem, not an economics one. Look, imagine the lower of the two boxes is picked to be any positive number x with equal likelihood (normal distribution over an infinite range for x > 0), and the other box is just twice that. The expected gain is 3/2 x, no matter what method you use.

The 'paradox' arises because everyone wants to express their gain in terms of the first box's amount, which could be high or low. If you instead use the more sensible definition of x as the lower box, period, you can easily calculate the gains and show they have equal expectation values. Doing otherwise muddles things by introducing more scenarios ($50 and $100, or $100 and $200?) which themselves do not have equal expectations, though they have equal likelihoods.

My apologies for not reading the comments closely enough to find commenters that got the correct answer earlier.

Bruce,

Actually you do. y=x-2 in your first example and y=x+2 in your second example no matter which way the quark spins.

Looking back a few comments Sebastian Holclaw got it, using the same idea of defining x as the lower of the two boxes. That's really the best way to do it. It's just that we want to define an answer in terms of the amount in the box we just opened (since that's an amount we actually have seen), which is trickier.

The bottom line is that finding $100 does not provide you with any information that you didn't already have (you already knew you were going to find some amount of money in the box you chose, whether it be $50, $100, or $1 trillion). Therefore, it doesn't matter if you switch.

jrose: The best choice, in a given iteration, will of course depend on the distribution and on which box you pick, but the assumption is that any total amount is equally likely and that you don't know what that is, so that's not information you can use.

I went in circles like most people trying to do things in terms of the first box you pick, until I realized it's simple if you just do things in terms of the lower box, whatever that may be. Maybe it's $50, maybe it's $100, but the point is, there is a lower box, and either choice on average gets you 3/2 that amount. This way of formulating things, unfortunately, does not tell you what that amount actually is in terms of the box you know, but it does answer the puzzle.

It's late. I just reread my own post, and I'm not sure I understand it.

My point was that EV analysis requires random variables. Random variables must have distributions. If all values of x are equally likely, then x is not a random number, and EV analysis doesn't apply.

Trying to use EV analysis on situations that aren't random according to a strict definition of random is like trying to divide by zero.

The rules that define what random means are designed to prevent you from reaching absurd conclusions, just like the rules that define when you can divide a number by another prevent you from proving that 0 = 1.

Have a good night, everyone.

Dan Schmutter - Truly you have a dizzying intellect.

Brian said: You were never told the distribution of the amounts. You were not told it was random.

Well, no, but in the absence of information as to the distribution, this is the only assumption that makes the problem interesting. I mean, the question's pointless otherwise. In these sorts of problems, that's a safe assumption to make. I mean, when someone uses a coin flip analogy to refer to a 50/50 situation you're not that guy who says, "That's only valid if the coin isn't improperly weighted, and only if it's tossed with many rotations," are you? If someone asks, "How many presidents were named George?", you don't respond, "That question is flawed because there are many other nations, and corporate leaders, with the title president, and you have to count them too."

All I can say is, if someone put Scarlett Johannson behind Door Number 1 and asked me whether I'll switch, I'm not going to spend 10 minutes asking questions like, "She doesn't have VD does she? You have to tell me if she does. A giant block isn't going to fall on me just as I walk through the door, is it?" I'm just going through Door Number 1.

Anyway, I'm done with this. Fun puzzle! Beats all those 'list of names' puzzles.

Patrick McKenzie writes:

Suppose we rephrase the game like this: I have two two boxes. The first box holds money, the second box holds a ball. The ball is either white or black. The game rules are as follows: you first receive the box with the money in it (*shake shake* yep, it has money). You are then offered the option to open the ball box. If the ball is white, I take half the money from the money box and keep it. If the ball is black, I double the money in the money box, and you get it all.

(skipped a little)
Here's my claim: this problem is *exactly equivalent* to the one in the original post.

Patrick,

This claim is incorrect. The problem you posed is constructed to make it always the case that the amount of money you observe has no value in helping you decide whether opening the other box will double or halve your money. But that can never be the case with the original problem. In the original problem, it is impossible to come up with an example which specifies how the amount in the lower box is determined where the amount you observe never gives you information to help you decide whether opening the other box will double or halve your money.

And Kevan,

This has nothing at all to do with whether we put an upper bound on the amount of money that can be in the lower box. All that is required is that whatever process determines this, the expected amount put in the lower box is finite.

chrisnm,

Mulling it over, I think what you say makes sense. At least, I haven't found a flaw. They key point is knowing that half the high boxes are over $1 million in my hypo, and all the low boxes are below it. So if you didn't draw a box over $1 million, the probability that you got the low box is greater than 1/2 because there are a million low boxes below $1 million but only 500,000 high boxes down there. Do the other math whizzes concur?

But returning to the problem as stated, with no info on bounds or any such thing: I stand by my first post. Peeking or not, switching or not, you had a 50/50 chance of getting it right and switching doesn't change that.

Final post of the night:

Reconsidering chrisnm's solution, I still think it correct. But my opinion that all the math talk and brackets and natural numbers whatnot are completely irrelevant nonsense and comparable to the application of Rawlsian political theory to the legal question "Is this will valid?" has been reinforced.

Kevan's problem can be restated as follows:

Here are two boxes. They have unequal amounts of money in them. Box A has $100. Would you prefer Box B?

There is no need to resort to mathematical jargon or detailed proofs to say: who knows?

I really wish I had thought of that hours ago.

One more way of seeing this. Go back to the lower box having $50 or $100 in it, 50/50. Order the events ($50 in low and you pick low, $50 in low and you pick high, $100 in low and you pick low, $100 in high and you pick high). If you follow a policy of never switching no matter what you see, your expected payoff is 1/4 $50 + 1/4 $100 + 1/4 $100 + 1/4 $200 = $112.50. If you follow a policy of always switching no matter what you see, your expected payoff is 1/4 $100 + 1/4 $50 + 1/4 $200 + 1/4 $100 = $112.50. If you follow a policy of switching unless you see $200, your expected payoff is 1/4 $100 + 1/4 $50 + 1/4 $200 + 1/4 $200 = $137.50.

So there must be valuable information in seeing what's in the box you open, at least for this example. The key is that there is always valuable info in seeing what's in the box you open.

"my opinion that all the math talk and brackets and natural numbers whatnot are completely irrelevant nonsense ... has been reinforced."

altho you are correct that much of the "math talk" has been nonsense or irrelevant (distributions over natural numbers, measure theory, martingales, et al), "brackets" (ie, symbols) are part of the language of probability theory. your not knowing that language doesn't make it nonsensical any more than not knowing any other language makes it nonsensical. given that your question was answered at every level from the layman's "intuitive" to the professional's "formal", I conclude that your boorish response was born of frustration at your own ignorance.

Kevan's problem can be restated as follows:

Here are two boxes. They have unequal amounts of money in them. Box A has $100. Would you prefer Box B?

There is no need to resort to mathematical jargon or detailed proofs to say: who knows?

I really wish I had thought of that hours ago.

This was exactly my restatement some posts ago.

zarevitz has it right, I think.

Knowing that one box is 2x the other is not enough information to calculate the EV of switching. The EV calculation requires seven things:

* value of switching if the other box is 2x = x
* value of switching if the other box is 0.5x = -0.5x
* value of keeping if the other box is 2x = 0
* value of keeping if the other box is 0.5x = 0
* probability that you opened the bigger box given that there is $x in it = 50%
* probability that there is $x/$2x in the box = ??%
* probability that there is $x/$0.5x in the box = ??%

Without all seven pieces of information, you can't evaluate Bayes' formula, and you can't calculate EV.

In zarevitz's problem, you're missing the first four pieces of information, but it doesn't matter because you either have all seven or you don't.

If there is a probability that there is $x in the box, however, then a strategy of always switching has the same EV (knowing that probability distribution, which we don't) as the strategy of never switching.

Again, it doesn't matter, said the five hundredth way.

chrismn's example (1:33 AM message), demonstrates the following:

1) If you knew the distribution of the money in the lower-valued box (random variable X), and you saw a specific amount of money in the box you picked (random variable Z), then you could calculate whether it is better to switch or not. In his example, you switch if Z equals $50 or $100 and don't switch if Z equals $200.

2) More importantly, when integrating over all possibilities of Z, chrismn found it didn't matter whether you switched or not - the expectation was the same. He would have found the same result for any assumed distribution of X.

The paradox results because Problem 1 gave us a specific value of Z (Z=100), while Problem 2 is equivalent to integrating over all Z. In Problem 1, we can't know the answer without knowing X's distribution. In Problem 2, we know switching makes no difference no matter what X's distribution is.

Again, it doesn't matter, said the five hundredth way

Mev, I went back just now and hastily re-read all of your posts.

This may be the 500th way you've said "it doesn't matter" but it is the first time you've used the words "It doesn't matter."

Hey, Kevan and all you "It doesn't matter" people, I have a
game I would like to play with you. It goes like this:

1. Dealer (me) takes two boxes and randomly puts X dollars
in one box and 2X dollars in the other box.

2. Picker (you) opens one of the two boxes and takes out
P dollars. ($100 in the original problem.)

3. (Key) Picker now pays Dealer 1.25P dollars for the
opportunity to play the game. ($125 in the original
problem.)

4. Picker may now keep the money found in the first box
or open the second box and keep the money found in
that one.

We can play this game all day long and switching off
between Dealer and Picker. Want to play?

My overnight thoughts:

1. I don't see what uniform distribution has to do with it. In the first scenario, you have a finite number of possibilities, and the assumption, one has to think, is that they are equally likely. How is that impossible? Seems possible to me.

Then, in the second scenario, the distribution seems completely irrelevant; the real problem is that the second pick is entirely an illusion. The fact is that the second scenario only allows one pick; Regis simply asks if that's your final answer. Changing, thus, is obviously meaningless. Distributions is a subject for another day.

2. Another way of stating the paradox with the first scenario, though, is that it still seems absurd that Box2 would somehow be more profitable than Box1. The only thing that makes it Box2 is the fact that you chose it second. This can't make it worth more. Thus, it seems, switching really can't matter.

3. How about this: The problem is with the math that proves doubling is somehow more good than halving is bad. Isn't that false? If you end up with 50 dollars, you can say you halved your money. But you could also say you'd have doubled your money by keeping the box. I can't turn this into an equation, but my suspicion is that the 1.25 figure isn't the whole story. E.g., when you're down to 50 dollars, that 100 dollars looks a lot better than it did; now you have to double your money just to break even!

I can't do the math myself, but I think we're understating the catastrophe of halving your money, and this is what makes it look like we should switch.

Robert Lyman,

I agree that philosophically, it's obvious that there's no reason to prefer switching or not switching. That's not the paradox.

If you asked someone with no mathematical background, they'd tell you it's crazy to open a box and always pick the other one, and they'd be correct.

The paradox is that this problem *looks* mathematical, but the intuitive way of applying mathematics (using expected value) leads us to an absurd conclusion.

The part of the problem that I find interesting is understanding why the same mathematical tools that are normally so helpful fail us in this particular case, and my posts have addressed this topic.

Kevan gave us two problems, an explicit one (what do we do?) and an implicit one (why doesn't the math work?). Simply asserting that the answer to the first problem is that "it doesn't matter" doesn't explain why, and it doesn't give any insight into the second problem.

I've tried to explore the reasons that you can't express this situation as an expected value problem, and these reasons are interesting and require precise mathematical language.

The problem here is that some of us want to understand why the math doesn't work. It's more interesting to us than the stated problem, the stated problem is only not obvious when you start trying to apply mathematics incorrectly to find the answer.

Mev said earlier, "My point was that EV analysis requires random variables. Random variables must have distributions. If all values of x are equally likely, then x is not a random number, and EV analysis doesn't apply.

Trying to use EV analysis on situations that aren't random according to a strict definition of random is like trying to divide by zero."

This is a point Mev and other math-types have said repeatedly, right from the very first comment, so I take it it's true. But my difficulty with this is, randomness is not (often) an inherent feature of the universe. It's just our lack of knowledge as to causes. So if you don't know the probability of whether you're in a 50/100 game or a 100/200 game after opening the first box, why isn't it correct to say it's 50-50? Sure, it's not *really* 50-50; it's either 1 or 0. Mev says that's like assessing the probability that circles are square or true statements are false. But I'm missing the internal contradiction. I understand that there's a proof above that you can't have a uniform distribution of probabilities over all integers or all real numbers. But we're not dealing with the ex ante situation of all integers, we're dealing with 2 sets of 2 integers each. Why is the probability that your integer is part of one set as opposed to the second, and only other, set indeterminate?

If anyone wants to take a last go at clearing that up for me, I'd appreciate it. Otherwise, thanks all, interesting problem.

The original question, after the explanatory set-up, was:
"Which should you choose?"

The operative word is "should," is it not?
For that reason, I see this not as a mathematical question, but as a philisophical one.

If your only motivation is to increase the amount of money you had (before the money box offer came your way) by some indeterminate amount, then there is no compelling reason why you "should" switch, since you don't care whether you increase your money by $50, $100, or $200; only that you increase your money.
In other words, you've got nothing to lose, but have assigned no relative value to gaining $50, $100 or $200, so why switch?

However, if you have a predetermined minimum amount that you want to acquire through the money box offer, that amount will guide your decision to switch. In this case, you may need to start doing some of that complex math seen above in this thread, to determine the probability of meeting your preset goal.

The money box scenario involves no stakes -- the game is played for free. One only thinks there are stakes because one thinks they may "lose" part of their initial gain. But $50, $100, and $200 are all greater than the $0 you started the game with, and you paid nothing to play, so it is all net profit regardless.

Personally, I am not a gambler. If someone gave me $100 I would say "Thanks, see ya!" and the game would be over. Having not set a predetermined minimum gain for myself, I am satisfied with the $100 and see no need to try to double it at the perceived "risk" of halving it. I could also quickly see, with the simplest of arithmetic, that given a presumably equal chance of doubling or halving my $100 (all math theory aside), I am already holding the 2nd best of the three (theoretical) alternatives, which is good enough for me.

Returning to the motivational aspect of the question, that is, "should" I switch:
If I halve my $100, I still come out ahead of where I was to begin with, but is that important to me? How important? How important is it to me that I double the $100? My own personal circumstances and my own valuation of money guides that answer.

So, the answer is that it is completely subjective.

There may be a mathematical way to determine the liklihood of the various outcomes (or not, depending upon which arguments in this thread you agree with), but in the end the math is irrelevant when placed into the real-world context of the initial question's "should" I switch.

Should I switch? Should you switch?
There's probably two different, equally correct answers to those questions, and they have little or nothing to do with math or probability, and lots to do with how you and I respectively value money, view gambling, or which bills we need to pay this week.

If we throw out the idea that this is a philosophical question, and look at it purely from a math standpoint, then the cue to the apparent paradox is definitely found in the fact that doubling/halving does some weird things to numbers.

For instance, if the numbers in a set continue to double, the numbers in the set form a non-linear graph. In fact, they plot out asymptotically. Surely that curve has an effect on both the probability function and the intuitive value of doubling vs. halving??

I really don't know. That's why I prefer the philosphical approach! :)

The problem here is that some of us want to understand why the math doesn't work.

I'm one of those people, as my posts show. I asked repeatedly why EV doesn't work. But it turns out you're wrong: the answer doesn't require precise language or formal proofs. The answer is: we don't know the probabilities of each of the two possible outcomes because we know nothing about how he picked the amounts in the boxes (coin flip? Whatever was in his wallet at the time? Oil prices? Nuclear decay?). Thus, we can't apply the EV forumula because we're missing two key numbers. Ta-da! Done, with only sloppy layman's language (some people got there way, way before me, of course.)

Our tools, in other words, don't actually fail us at all, they're just missing a key ingredient. You don't need to be an expert on electrical engineering to figure out that your TV doesn't work if it isn't plugged in. And it would be pretty weird--and perhaps a tad frustrating--to encounter someone citing Maxwell's equations (and insisting on their absolute indispensibility as the "language of electricity") when the answer is "plug it in."

Again, I didn't mean to be "boorish," so I'm sorry if that's how it comes across.

In fairness Mev, I'm the one complaining about jargon.

then we want to think that the probability that we have the bigger box is 50%.

Now you have well and truly lost and confused me. I am offered two boxes. One of them has more money than the other. I pick one. The odds that I have the bigger box isn't 50%? It's actually undefined? Is that a typo?

I can believe that, in Kevan's problem, the odds that the other box contains $200 vs $50 is not 50/50 (and is in fact undefined), because that depends on his selection method, which might involve random distributions or might involve mugging Eugene Volokh and using the money he gets from selling his wedding ring.

But I'm pretty sure if I just grab a box, that's 50/50, bigger/smaller. Right? If not, then I give up.

To solve this puzzle, you need to ask yourself "What is the
value of this game?" Before opening the first box, there
is obviously no way to compute the value of the game. After
opening the box with $100, you can compute the value of the game.

If you think that it doesn't matter if you switch to the
other box, you are in effect saying "The value of this game
is $100" because switching to the other box will result
in no net change in value. If, like me, you think that the
value of the game after opening the first box is $125, you
must take the money in the other box.

The key point is that opening the first box DOES give you
information. It tells you what the value of the game is.

Bruce,

I'm no Mev, but let me take a crack at the "indeterminate probability" problem:

What is the probability that my first daughter will be named Alice?

How could you begin to calculate that? If I gave you some information about my selection method (say, I intend to name her after one of her grandmothers, but I'll flip a coin as to which) you could figure it out (in this case, zero, since neither grandmother is named Alice). Or, you might adopt some reasonable assumption (I am the "average person" and I'll pick her name with the same probabilities as are reflected in all the girl babies born in the world, or in the US, or in some particular state).

But I think without any further information or assumptions, the probability that my first daughter will be "Alice" must be undefined.

Mev can tell you if that's right or not.

"... 'boorish', although I think that word a little strong. For that, I apologize."

it was, so I'll see your apology and raise you one.

when I first encountered the monty hall problem, I used "verbal reasoning" instead of the formal analysis (of which I am quite capable despite decades of removal from any serious application) and got the wrong answer. you drew your conclusions from the numerous "hand-waving" posts, I drew mine, viz, the formal analysis is always best if you want to get the right answer. it forces you to accurately model the problem, helps organize your steps, and the results are explicit so critics can challenge the model, assumptions, formulas, calculations, etc.

for example, "you can't say for sure that the chances of $50 and $200 are equal, so you can't take the weighted average)." I'm not entirely sure what this means, but I think it is, if you'll pardon the expression, "nonsense". as I tried to point out earlier, the reason the early EV analysis failed was because it computed the wrong quantity, the right one being a conditional EV rather than an unconditional EV (sorry for the jargon, but even without knowing the exact meaning I think one can get gist of the concept from my earlier comment). the actual numbers ($50, $200) are irrelevant and an EV, conditional or not, is a weighted average, which I "computed" with no trouble whatsoever in the comment.

IMO, spending a relatively modest amount of time trying to understand my formal (loosely speaking, of course)formulation would have been much more profitable than participating in another batch of hand-waving exchanges. the majority of the issues raised in the comments are addressed in the four lines it took to write out the (very simple) equation. the remaining ones were mostly about initial distributions on D, a red herring (unless I'm missing a deep subtlety buried in piles of obfuscation).

Robert's correct. You may think, "Robert's equally likely to select any name," but there are an infinite number of names, most of which are ridiculous.

If you say they're ridiculous and restrict yourself to English names used in the past centery, it's an easy problem (once you know how many names there are).

If you really allow any name (just like if you really allow any amount, like a $10^100), then the probability is undefined. The probability of any one name must be zero, by symmetry, but how can an event with probability zero happen?

It can't, and the reason it seems to occur is because not all outcomes are equally likely in reality, and some outcomes have non-zero probability (making them more likely than others).

ctw,

I re-read your first comment, and I'll take your word for it that it solves the whole problem in 4 lines. I surely understand the need for formal analysis, but there is also a need for sound non-formal understanding. Have you ever encountered an earnest undergrad who does a series of careful calculations and then comes up with a completely ridiculous answer because of a minor error? If not, have a look at the early EV calculations, which are impecable as to their arithmatic but deficient in their underlying understanding.

I've not only encountered such people, I've been one on enough occasions.

Here's what I meant when I said "you can't say for sure that the chances of $50 and $200 are equal, so you can't take the weighted average":

Expected value of second box= Pa*(200) + Pb*(50).

Where Pa and Pb are probabilities.

You don't know what Pa and Pb are. They total 1, but that's all you know. So you can't use the EV formula. Period. Mev says Pa =! Pb, but that's surely not true Kevan tells you put $100 in the first box and then flipped a coin.

Clealy, my sentence was inartfully phrased. I should have said "you don't know what the probability of each outcome is, so you can't apply EV."

Now, given some information (like the distribution, or Kevan's method, whatever it might be), you could calculate those probabilities. Without them, however, your EV equation is correct, but impotent.

Your TV is not plugged in.

Also, I didn't mean that Pa != Pb in general. I just meant that you can't find any way for Kevan to pick values to put into the boxes where Pa = Pb for every a and b.

I think I've got the EV thing (meaning I didn't get it until I read all these responses.)

Kevan gives me a box. It contains a goat worth $1. He points at another box and says "This other box contains either 50 cents or a million dollars, would you like to switch?" And if I say yes I get a box that contains 2 quarters and a lottery ticket. And I congratulate myself for buying the lottery ticket with an EV of $1,000,000 / 4,000,000, or 25 cents, for 50 cents (one goat less 50 cents change) instead of the dollar I usually waste on it?

(And my kids say "But DAAAD! We'd really like a goat as a pet!")

I know the million-dollar lottery ticket is worth 25 cents (or whatever the odds tell me), and therefore the ticket plus 50 cents is worth 75 cents, and under those rules I should keep the one-dollar goat.

Now he hands me a box with $50. He says there is a chance p (p in 0 to 1 inclusive, being a probability) that this other box has $50 in it, and a chance 1-p that this other box has $200 in it. I quickly conclude that for any value p, p * 50 + (1-p) * 200 = 250 - 200p >= 50, and I switch. I can't lose and I might win.

Finally he hands me a box with $100 and offers to trade it for the same kind of other box. I think "For some values of p, 250 - 200p < 100, and for other values > 100. If p < 0.75, the EV of the other box is > 100, and if p > 0.75, EV < 100. Do I have any information about the value of p?

I think "I got here by flipping a coin, q(heads) = 0.5", and I wrack my brain trying to find the connection between p and q.

Here are a couple of variations that helped me understand this problem:

Variation 1

The planet Marsiter is made up entirely of only 2 kinds of rocks: red rocks and blue rocks. Blue rocks are worth $1,000. Red rocks are worth $1. In my hand, I have a closed bag containing one rock, selected randomly from all the rocks on Marsiter. I don’t know what color rock is in the bag. I offer to sell you the bag for $400. Do you accept my offer?

Variation 2

Same as above, except I now tell you that 99% of the rocks on Marsiter are red. Do you accept my offer?

Kevan’s problem is like Variation 1. The people doing expected value calculations falsely assumed Kevan’s problem is like Variation 2.

My brother - who is a very knowledgable statistician and applied mathematician- tells me that this is actually a much deeper problem than has yet been made clear. It is apparently possible to calculate posterior probabilities using something called Dirichlet processes. Sometime in the past 25 years or so Diaconis and Friedman apparently wrote a paper exploring the intricacies of related problems. So, like everything else in life, everyone's a little wrong and everyone's a little right.

I will attempt to resolve the seeming parodox of Kevan's original problem with as little math as possibe. You open the first box and see $100.

Seeming Paradox:

A) If you switch, you have a 50% chance of $50 and a 50% chance of $200. This averages to $125 (1.25 x $100).

B) Therefore, you should should switch.

C) Since the logic in statement A) had nothing to do with my actually seeing $100 (as opposed to $1 or $1000 dollars), I should therefore switch regardless of what I see.

D) That means I should switch even if I haven't opened the first box yet.

E) That also means I should switch back again if offered to so and so on and so on.

Where is the fault in this sequence?

The problem is that it is impossible to come up with any method whatsover for determining how much is put into the lower box where both statements A) and C) are true. For some methods of determining how much is put in the lower box, when you see $100 you should switch precisely because doing so raises your expected payoff to $125 precisely because, conditional on your having seen $100 in the box you opened, you have a 50% of the other box having $50 in it and a 50% chance of the other box having $200 in it. That is, it is easy to posit a method for determining how much is put in the lower box where statement A) is true.

But in any such example statement C) is false. For any method of determining how much is put into the lower box, if you want to switch when seeing $100, there will be other amounts you could have seen which would make it so you shouldn't switch. That is, that you want to switch does indeed depend on your seeing $100.

Thus the line of reasoning is false. No paradox.

But in any such example statement C) is false. For any method of determining how much is put into the lower box, if you want to switch when seeing $100, there will be other amounts you could have seen which would make it so you shouldn't switch. That is, that you want to switch does indeed depend on your seeing $100.

Are you saying "If you know the other box contains either $50 or $200, and you're given the choice of switching or keeping a box that contains $100 $500, of course you would keep the $500?

Yeahbut, since I know the rules of the game that I am playing, all I have to know is that there is some amount of money in my box, call it N.

(And yes I understand the symmetry argument, I'm just trying to understand why the EV analysis doesn't work. I thought I understood it when I stopped reading these comments yesterday afternoon, but then I couldn't reconstruct it this morning.)

Going back, chrismn wrote:
Suppose a dog has 4 puppies and each puppy is a 50/50 independent coin flip of being a boy or girl. Is it most
likely that the litter will be 2 boys and 2 girls?

Her answer would be, "No, it is most likely that the litter will be 3 of one sex, 1 of the other."

This is both correct and incorrect. If the event space is (all four same sex, 3 of one 1 of the other, 2 and 2) then she is correct since the probabilities of these events are, respectively, (1/8, 4/8, 3/8) and 4/8 is the highest of this list.

On the other hand, if the event space is (4 male, 3 male 1 female, 2 and 2, 1 male 3 female, 4 female) then the probabilities are (1/16, 4/16, 6/16, 4/16, 1/16) and thus
the most likely is 2 and 2.

Of course it is not likely to be 2 boys and 2 girls, since young male puppies and young female puppies aren't called by those names. But to restate it, "Most likely" isn't meaningful without knowing "among what" (that is, what you call correctly "event space".) Related favorite epigram: "The vast majority of people have more than the average number of legs" -- where "average" means "arithmetic mean", which would be something like 1.99, or, if we define legs so that the answer is integral (a partial amputation is either a full leg or no leg, by some definition) "Nobody has the arithmetic mean number of legs".

And in fact some people think Mendel cooked his books, because the chances of his peas coming out exactly the way genetics predict is much smaller than the chances of his peas coming out in any other possible distribution, even if you can say with certainty that for N peas it is 65% likely that the number of white flowers will be between N * 0.5 - s and N * 0.5 + s.

But that's because I was searching for "boy" to review Mike Keenan's
To change the topoic slightly, here is a similar paradox. I tell you that at least one of my two children is a boy. What are the odds that both are a boy? Now, I tell you that the older one is a boy. The odds have changed because of the change in information.

In the second choice, the possibilities are (always listed older,younger): boy,girl and boy,boy and always assuming the chances for any given child is 1/2 the answer is 1/2.
In the first choice, out of the entire space of two-child sets, namely (b,b),(b,g),(g,b) and (g,g), the probability of each being 1/4, [and the probability of one-of-each is p(b,g) + p(g,b) or 1/2], we know, having paid attention to Bayesian stuff more than the first time we heard this, that the equally likely possibilities are (b,b),(b,g) and (g,b) and in only 1/3 of those is it b,b. But it still feels wierd.

There is a similar partitioning-of-the-possible-outcomes paradox involving 3 coins, so that the answer to "what are the chances that they are all the same" appears different if you flip 2 and look and then flip the 3rd versus flipping all 3 together. (I don't remember the wording.)

That doesn't change your expected value in terms of X.

So EV = 1.25X, and I don't know what X is, even though the only possible values of X are 50 and 100?

Repeating, hoping to believe it, I don't know what X is...

You know the set of options for X now, but the point is that on the second pick, your EV is still 1.25X. (Indeed, the fourth comment way at the top made this point. The error there was not considering that the EV was 1.25X the first time around too.)

I have two children. I have written the sex of one of them on a piece of paper in my pocket. What is the chance that they are both boys?

I have two children. I have written the sex of the older on a piece of paper in my pocket. What is the chance that they are both boys?

David,

1/4 to both your questions.

Robert Lyman,

Let's face it. We both need paradox discussion therapy. :-)

AppSocRes,

!! ALERT -- SEVERE MATH JARGON AHEAD -- ALERT !!

A Dirichlet process is a process that generates a random probability distribution, and the distribution is randomly generated from all possible distributions according to a separate probability distribution.

I warned you. It gets worse.

You can use this model to evaluate the expected value of switching in the absence of a known distribution for what Kevan's putting in the boxes.

The problem is that you need a base distribution for how Kevan fills the boxes to get started, and you have no reason to pick any one distribution over any other (mathematically, though not perhaps philosophically). The first time, therefore, you still can't evaluate the EV equation for switching without making some assumption about the distribution Kevan is using.

The second time, though, you can improve your understanding of the random distribution that Kevan is using because you know it's less likely that he's using a distribution that make it unlikely for you to have seen what you saw in the first round.

You can continue this process, and hopefully, you'll converge on Kevan's actual distribution by making your Dirichlet process more and more likely to generate a distribution that would have produced what you've seen so far.

Mev, see my comment of yesterday at 3:20. I use the notation you use in your 12:47 response to one of my other comments. The game is set up such that we know the boxes are x and 2x. The inital choice has an EV of 1.5x (which is [.5x+.5(2x)]/2).

At the time of the choice to switch you presently have either the x box or the 2x box. If you choose to switch, 50% of the time you will switch from x to 2x. 50% of the time you will swith from 2x to x. So after the switch you have a 50% of having the x box and a 50% chance of having the 2x box. This has an EV of 1.5x.

The EV is the same for switching and not switching. Therefore there is no reason to switch and no reason not to switch. The two choices have an equal EV.

The fundamental problem with the EV = $125 calculation is that it ignores how making one choice (the one Kevan made, of which box to hand to you) changes the probabilities of the outcomes of a second choice. More precisely, it changes the probabilities of each possible outcome of each choice you can make to either 0 or 1.

First, I'll define a couple of scenarios more precisely, so we do in fact have 50-50 underlying probabilities:

I. Kevan gets $3x. He puts 1x in box 1. He flips a coin, and depending on the result, he puts either 0.5x or 2x in box 2. He gives you box 1, with the option to swap it for box 2.

In this case, the EV of box 2 is indeed 0.5*(0.5x)+0.5*(2x) = $1.25x. You should swap.

(There are exceptions where nonmathematical considerations override this. If I find $1,000,000 in the box, that's enough to retire on right now and spend the rest of my life commenting on blogs with none of those pesky interruptions by my employer. Half of it isn't enough. Doubling that wouldn't give me enough additional enjoyment to be worth the risk.)

II. Kevan sets up the boxes as above. He flips a coin, and depending on the result, he gives you either box 1 or box 2.

Let's call the box he gives you box A, the other one box B, and the lower of the values of the two boxes 'y'. You know box A has $100, but you have no idea whether this is y or 2y. Then the expected value of box A is 1.5y, and so is the expected value of box B. Play this game a million times, and whether you hold or swap it will come out the same.

But what about the EV calculation? The random events in this scenario happened before you arrived. Either x=y or x=2y, and box A is either y or 2y, and these two events were independent. The results of choosing to swap or not are not independent of the previous events, so you have to evaluate conditional probabilities (which are either 0 or 1) for four equally likely possible cases crossed with your two possible choices.
1) x = y, and box A is box 1: Holding gives you x, swapping gives you 2x.
2) x = 2y and box A is box 1: Holding gives you x, swapping gives you y = 0.5x.
3) x = y and box A is box 2: Holding gives you 2x, swapping gives you x.
4) x = 2y and box A is box 2: Holding gives you y = 0.5x, swapping gives you x.

EV for holding is (1+1+0.5+2)x/4 = 1.25x
EV for swapping is (2+0.5+1+1)x/4 = 1.25x

Which means you did get that 25% increase in expected value - but it came from Kevan's coin flips, not from you making a choice afterwards.

Anybody but me wish that Kevan would post the explanation he believes is correct?

markm: exactly.

Robert: but on the information presented to you, you have one of two possible options, and no information about the process which went into determining them. All I'm saying is that for purposes of practical reason, it makes sense to treat that as random.

Right. I'm now going to make like Bush should and get out of this quagmire. :-)

Paul,

Make whatever assumptions you want. Your result (switch) is perfectly correct for your assumptions (that it's 50/50, 200/50).

Just know that Kevan didn't say that, and as you no doubt know, not saying something (like "imminent threat") can matter as much as what you do say.

I did a simulation in C++. I tried several different algorithms for when to change you mind. It can be configured to allow a range of value from 1 to a large unsigned integer. I also made it so I could limit the values to just two values for the smaller number, like $50 and $100. I implemented all the suggested algorithms.

The only thing that made any major difference was picking again on odd numbers and when you had picked one cent.

So empirically changing your mind doesn't help.

Gob - way to work your magic.

Steve Holt!

isn't it rational to assume that an unknown decision method which selected among two possible options gives a .5 probability to each

No. If that were the case, then for any and every amount of money in our box, we would still assume that half the time we picked the bigger-valued box - i.e., knowing the value of money in our box has given us no information. How could that possibly be? Only if all values of money have the same probability of being the amount in our box. But, the only way that could happen is if the uniform distribution is defined over an infinite interval - which it is not.

Therefore, we have gained information when we see any specific amount of money in our box - and the assumption that we have picked the correct box with Prob=50% no longer holds.

Gob Bluth states: "switching is irrelevant. There is an
information gap." Yes, there is an information gap, but
it does not follow that switching is irrelevant. If you
are to participate in this puzzle, you MUST choose. You
must choose between (A) keeping the $100 or (B) taking
the money in the other box. In order to choose between
(A) and (B) you must put values on these options. We
know (A) is worth $100, but what is the value of (B)? We
can calculate the value of (B) if we know the relative
probability of the boxes being 50-100 or 100-200. If the
probability of the boxes being 50-100 is greater than
2/3, then we should keep the $100. If the probability of
the boxes being 50-100 is less than 2/3, we should switch
to the other box. In order to choose between (A) and (B),
we MUST estimate or guess what that probability is. Many
people have assumed a 50%-50% probability and this seems
like a reasonable guess, absent any other information. If
you say that switching is irrelvant, you are in effect
saying the probability of 50-100 boxes is EXACTLY 2/3
because that is the only situation where switching or not
does not matter. I contend there is absolute no basis
for claiming the original probably is 2/3, therefore
claiming "switching is irrelevant" is wrong.

Box Keeper writes:

There's no way to tell which is which, but it is obviously
wrong to treat them as "equally likely" and average them.
Because of the mathematics of doubling, the high-side phantom
games will systematically distort more than the low-side phantom
games, yielding wrongheaded EV calculations.

I can agree with this. However, since $100 a relatively small
amount that I am sure Kevan can afford, I think it is reasonable
to say they are "approximately equally likely". In order to
not switch, the probability of 50-100 vs. 100-200 would need
to exceed 67%. I do not think that is a reasonable conclusion,
therefore switching is best.

Kevan - why are you MIA? You're shirking your Puzzleblogger duties.

I can't begin to understand why people think the way the lower number in the boxes is initially determined matters (we all, I hope, agree that once that number - call it D - is determined, calculating 2D is straightforward and deterministic). if anyone else who is foolish enough to still be reading this also thinks that, please explain how - having determined D, put that randomly into either box A or box B, and then put 2D into the other box - the probability that D is in box A can be other than 0.5.

ron -

not sure your comment was a response to mine. if so, it doesn't address my question. yes, the expected gain is zero if the probabilities that D is in box A and box B are both 0.5, which they are. but numerous commenters have argued that they aren't, and I'm just curious how they reach that conclusion.

Typo (the arghh effect as you hit the POST button)

> The paradox comes from getting you to think that the D/2 : D case is the general one, where there must always be a cutoff case that kills the reasoning in fact.

D/2 : 2D case, it should be.

ctw,

Trivial example. Half the time Kevan puts $100 in one box and $50 in the other. Half the time he puts $200 in one box and $100 in the other.

For D=50 or 100, your expectation goes up by switching. For D=200, it goes down. When weighted across all D, the expectation stays the same.

Sorry, that should be "50% (the dealer's choice of game)"

Mike Keenan: "I tell you that at least one of my two children is a boy. What are the odds that both are a boy? Now, I tell you that the older one is a boy. The odds have changed because of the change in information."
I think this is relevant because the mistake many posters seem to make in the boxes problem is analogous to, although less obvious, than the mistake rank amateurs to probability make in analyzing the boy problem: they assume there are three cases (2 boys, 1 of each, 3 girls) when there are really 4 equally likely cases (boy-boy, boy-girl, girl-boy, girl-girl). Knowing that 1 child is a boy eliminates the girl-girl case, so the chance that the other child is a boy is 1/3. Knowing that the oldest child is a boy eliminates two cases, so the chance the second child is a boy is 1/2.

Here is a mathy argument that there is not enough information to answer (a la Aaron Bergman's reasoning):

The distribution of the money placed in the box does indeed affect whether you should switch. If the probability distribution happens to have a density function d(x) on a domain containing 50 and 200, then the probability that the other box is the bigger one is d(200)/(d(200)+d(50)) (application of Bayes' theorem). This means that for the most "obvious" choice of distribution, the uniform distribution supported on some domain (0 to 500, say), the other box is equally likely to contain $50 as $200, because d(x) = 1 everywhere in the support. But if the money is distributed, say, with a standard normal, the probability you have the lesser box is different, which might affect how you should choose (whether you should switch in this case is left as an exercise to the reader).

You can arguably say that given no information on the distribution, you would have to assume the other box has 200 with a prob of 50%. But this choice is biased and maybe even arbitrary. Let's play the game again, where I give you two boxes, I tell you no info on any relationship between the quantities in each, you open one and find $100. Do you switch? If only you knew the mean! Declaring the mean to be 1000 and basing your decision to switch on this assumption is definitely flawed, and similarly in the original problem, because you base your decision on some assumption about the distribution.