Notes and Solutions #6 Meeting of 21 October 2008

Transcription

1 HAVERFORD COLLEGE PROBLEM SOLVING GROUP Notes and Solutions #6 Meeting of October 008 The Two Envelope Problem ( Box Problem ) An extremely wealthy intergalactic charitable institution has awarded you a genius prize. During the prize ceremony you are offered two sealed envelopes. One of them you don t know which has twice as much money in it as the other. To collect your prize, you will open one envelope and count the cash, and then you may decide whether to keep the money or to switch and take what s in the unopened envelope. What should you do? Or doesn t it matter? Comment You might think that switching and not switching would be equally acceptable strategies, because you are equally uncertain as to which envelope contains the larger amount. The paradox is that switching seems to be the superior option, because (letting x be the amount of money in the envelope you opened) switching offers alternatives between payoffs of x and x/, whose expected value of ½(x) + ½(x/) = 5x/4 exceeds the value in the opened envelope. Hint This one is difficult to simulate with a computer program. Why? Solution Many approaches to this problem start by assuming there is a probability distribution the prior describing the amounts of money you think likely. For example, your prior distribution might be uniform in the range from $0 to $0 6. This means that before opening the first envelope, you figure the probability that the smaller amount is between a and b (and therefore the larger is between a and b) equals (b-a)/(0 6-0) whenever a and b are between 0 and 0 6 and a < b. Alternatively, being a bit of a pessimist, you might suppose that the probability equals e -a e -b no matter what a and b are (provided they aren t negative). This is an interesting assumption because (a) it really is a probability distribution and (b) it assigns some probability, although only a tiny bit, to arbitrarily large amounts. Let s call the prior distribution f, whatever it is. Choosing a prior lets you carry out the calculation of Bayes Theorem. This Theorem is mathematically trivial: starting with the axiomatic relationships P[ B A] P[ A] = P[ BA] = P[ AB] = P[ A B] P [ B] () among the probabilities and conditional probabilities associated with any two events A and B, it is immediate that Oct 8, 008 W:\Haverford\Problems\008-9\Solutions 6.doc

2 [ B A] [ A] P[ A B] = P P P[ B] provided B has nonzero probability of occurring. Observe that () merely gives four ways of expressing the probability that both A and B occur: from left to right, A occurs and then B occurs (given that A has), B and A occur, A and B occur, and B occurs and then A occurs (given that B has). In the present application, we want to know the probability that the unopened envelope contains x given that we saw x in the opened one. The experiment contains two elements of randomness that we have to consider: () A random number ω is drawn from your prior distribution and then () Independently of that, one value is drawn at random from the set {ω, ω}. The sample space, or set of all possible experimental outcomes, can be described as the set of all ordered pairs giving the results of () and (): {( ω, x) ω 0, x ωorx ω} Ω= = =. Some relevant events (sets of outcomes) to think about include { } A(ω) = the random number drawn is ω = (, ),(, ) ωω ω ω, { } B(x) = opened envelope contains x = (, ),( /, ) xx x x, and C = the smaller amount is in the opened envelope = ( ) {, 0} ωω ω. Note for later use that C is statistically independent of A(ω) because it is the result of a physically independent process (() versus ()). Because it is equally likely for either envelope to contain the smaller amount, no matter what value was drawn, the probability of C is P [ C ] = and the probability of its Note, too to keep your sanity when you think about conditional probabilities that a phrase like A occurs given that B has does not mean that B has to occur in time before A did! All it means is that we are limiting our calculations to the experimental outcomes in set B and we intend to compute the relative probability (among all outcomes in B) that the outcome is in A. For example, if B has probability ½ and A has probability /3, then A has /3 / (/) = /3 of all the probability in B. This is the conditional probability of A given B. It is customary to use the lower case Greek omega ω to refer to outcomes, reserving lower case Latin letters x, y, etc. for numerical values associated with those outcomes. This convention reminds us that any experimental procedure usually produces non-numerical results (ω) but that to analyze them mathematically we have to associate numbers (such as x) to those results. As a mathematician, you will immediately recognize that the systematic assignment of numbers to the elements of a set is a function. This is the formal definition of a random variable: a real-valued function on a probability space. Page /7

3 complement, C' = Ω C, is P [ C '] = =. P [ A( ω)] is determined by your prior distribution, so there s nothing more specific we can say about it now; let s just call it f(ω). We can compute the probability of B(x) by recognizing the two ways it can occur: x is drawn and C is true or else x/ is drawn and C is false: P[ Bx ( )] = P[ AxC ( ) ] + P [ Ax ( /) C'] = f( x) + f( x/). Two probability rules justify this calculation: because the event B(x) has been broken into mutually exclusive events A(x)C and A(x/)C', we add their probabilities; because A(x) is (statistically) independent of C (and therefore A(x/) is independent of C'), we can multiply their probabilities. The random choice of envelope to open implies that P [ Bx ( ) Ax ( )] =. (In words: the chance that the opened envelope contains x, given that the amounts x and x were placed in the envelopes, is ½.) Therefore P[ Bx ( ) Ax ( )] P[ Ax ( )] f( x) f( x) P[ Ax ( ) Bx ( )] = = =. () P[ Bx ( )] f( x) + f( x/) f( x) + f( x/) This is the probability that the unopened envelope contains x given that the opened one contains x. The complement of this (conditional) event is that the unopened envelope contains x/, so we obtain its probability by subtracting () from.0. This maneuver is the crux of Bayes theorem: we can easily compute one conditional probability, but what we really need is the conditional probability with the two events A(x) and B(x) reversed. The theorem implies those two probabilities are not usually the same and therein lies the basis of much bad intuition and confusion. At this point we can escape many complications discussed at length in the literature by noting there are only two values of the prior distribution f that need to be considered: f(x) and f(x/). The rest of f is irrelevant! Let s look at some examples before proceeding. Example Suppose you assign nonzero probability only to the possibilities ω in {$, $4, $8}. Consider the prior f($) = 7/0, f($4) = /0, f($8) = /0. We worked out the probabilities of each outcome in Ω by using a probability tree: Page 3/7

4 { } ω: f(ω) x P ( ω x) [, ] $ 7/0 $: 0.7 $4: 0. $8: 0. $4 7/0 $4 /0 $8 /0 $8 /0 $6 /0 There are only four possible values of x to observe. We tabulated them along with the conditional probabilities of ω and the potential gains to be made by switching envelopes: x ω P [ A( ω) Bx ( )] Amount in unopened envelope $ $ $4 $4 $ 7/9 $ $4 $4 /9 $8 $8 $4 /3 $4 $8 $8 /3 $6 $6 $8 $8 Stop for a minute and compute some of these conditional probabilities for yourself. Example Express the amounts in millions of dollars and assume (for the prior distribution) that all values in the interval [0, ] are equally likely. Thus, f(ω) = for ω between zero and one million dollars and f(ω) = 0 otherwise. By our conventions, this means we assume the envelopes can contain between zero and two million dollars, but no more. (i) (ii) If you observe, say, x = 0.6, the value of () is /(+) = ½. You conclude there are equal chances that the unopened envelope contains. or 0.3. If you observe x =., the value of () is 0/(0+) = 0. You conclude there is no chance the unopened envelope contains.4; it is certain to contain 0.6. Page 4/7

5 Example 3 This time, take the prior distribution to be the exponential f(ω) = e -ω. The expression () simplifies to x e P [ Ax ( ) Bx ( )] = =. x x/ x/ e + e + e (i) (ii) If you observe x = 0.6, the probability that the unopened envelope contains the larger amount is /( + e 0.6/ ) The chance of getting. by switching is and the chance of getting 0.3 by switching is = If you observe x =., the probability is /( + e./ ) The chance of getting.4 by switching is and the chance of getting 0.6 by switching is = (End of examples.) One method of choosing a strategy is to convert money into utility which we might as well assume has been done at the outset and to select a strategy that maximizes the expected utility of the choice 3. The utility of keeping the amount in the opened envelope obviously is x. The expected utility of switching is the probability-weighted value of the two possible amounts: f( x) f( x) x+ x/ f( x) + f( x/) f( x) + f( x/). = ( f ( x ) + f ( x /)/) x. (3) f( x) + f( x/) Thus, the Bayesian who really believes in their prior will want to switch when the coefficient of x is greater than unity. This is algebraically equivalent to f( x) > f( x/). Examples, continued Plot the coefficient of x in the utility formula (3): this is the relative utility of switching. 3 We re ignoring the question of risk aversion here. A risk-averse decision maker will prefer to make a choice that has a more certain expectation, all other things being equal. Page 5/7

6 Relative Expectation of Switching Prior distribution is Uniform[0,] Relative Expectation of Switching Prior distribution is f(x )=e -x Switching is the thing to do wherever the graph is above. In the first example (left graph), our strategy should be to switch whenever x and stay pat otherwise. This makes intuitive good sense: when x is small, we switch because we have even chances of getting x and x/, but when x is greater than, our prior says there s no chance of doubling the money. In the second example, we should switch only when x ln(). This, too, agrees with the pessimistic nature of the prior: it has greatest probability density at x = 0 and decreases as x increases. Eventually, as x increases, the prior views the chance of doubling the money as being so low it no longer makes sense to switch. Obviously, the optimal strategy depends on your prior. There is an entirely different route to a solution, an approach that does not depend on choosing a prior. As far as I know, this has not been published. The idea is to make a randomized decision. You will decide to switch envelopes by spinning a dial after observing one envelope s contents x. The dial has a chance δ(x) of telling you not to switch and a chance δ(x) of switching. Reconsider the original experiment: the charity chooses by any means 4 a number y. It puts y and y into the envelopes. You will choose one of them at random. If you choose the one with y in it, your strategy calls for you to spin a dial with δ(y) chance of not switching. That nets you a prize of y with probability δ(y) and a prize of y with probability - δ(y), for an expected amount of yδ(y) + y(-δ(y)) = ( - δ(y))y. This occurs with a chance of ½. There is also a chance of ½ that you choose the envelope with y in it. You will then spin the dial with δ(y) chance of not switching, gaining y with probability δ(y) and y with probability δ(y), for an expected amount of 4 It doesn t have to be random in the sense of being characterized by a prior distribution. An example we mentioned is for the charity to choose either y = n or (/) n with equal probability on the occasion of donating its n th genius prize. There is no long run probability that can describe the values actually appearing in the envelopes! Page 6/7

7 yδ(y) + y( - δ(y)) = y(½ + ½δ(y)). Given that y, although it is a value unknown to you, has definitely been chosen by the charity, your expected winnings are ( yδ( y) + y( δ( y)) ) + ( yδ( y) + y( δ( y)) ) 3 + ( δ( y) δ( y) ) = y. (4) The value of y is not revealed until after all is over, but this procedure guarantees these expected winnings regardless of the value of y. How does this compare with any Bayesian solution? When the envelope with y is opened, the Bayesian will switch provided f( y) > f( y/). When the envelope with y is opened, the switch occurs provided f( y) > f( y). When the Bayesian is good at guessing the likely value of y, she will use the correct strategy in both cases and the expectation of the result is y. Otherwise the expectation is 3y/ (which is no better than always switching or always standing pat) and, in some cases, could be just y. This will happen with any y for which both switching rules are bad: that is, whenever, f( y) > f( y) f( y/)/. (5) Are there any prior distributions with no bad values of y? Define g(x) = log (f( x )) The relations (5) are equivalent to for all x; that is, + g(x+) > g(x) g(x-) g(x) < g(x+) + for all x and g(x) g(x+) + for all x. Obviously this cannot happen. If the Bayesian makes a poor guess of the charity s behavior, then, she could be using the worst possible strategy and consistently choose the smallest amount of money! In contrast to this, the user of a carefully crafted spinner can always do better than always switching or always standing pat, and will never do as badly as the Bayesian might: just let δ be any function that strictly increases and has a range in the interval [0, ]. This will assure that the multiple of y in equation (4) is always strictly greater than 3/. (Actually, we only need that δ(y) > δ(y) for all y > 0.) In particular, any strategy that switches or not independently of the observed amount x is inadmissible, because its expectation is (3/)y for all y: any of the delta-strategies is always better, regardless of what amount is in the envelopes. PS Jake opened an envelope with $5, chose to switch, and won an IOU for $0 that he split with everybody. He used psychological reasoning, not mathematical, and almost regretted it when the second envelope was first opened and no bill was evident! Bill Huber, October 008 Page 7/7