1 Statement of the Game

Transcription

1 ANALYSIS OF THE CHOW-ROBBINS GAME JON LU May 10, 2016 Abstract Flip a coi repeatedly ad stop wheever you wat. Your payoff is the proportio of heads ad you wish to maximize this payoff i expectatio. I this paper, we will derive upper ad lower bouds for the expected value of this game. We will also examie stoppig coditios for several differet strategies ad expected payoffs for these strategies. 1 Statemet of the Game Cosider the followig game: We flip a fair coi repeatedly util we wish to stop. Our payoff for the game is the proportio of heads we ve observed up to that poit. This game is a variatio of the Robbis s problem of optimal stoppig, or Secretary Problem, which was first itroduced by Yua-Shih Chow ad Herbert Robbis [2] i For the rest of this paper, we will aalyze strategies by their expected payoff. Occasioally we will also use the term expected payoff whe referrig to a state of the game, i which case we are referrig to the expected payoff uder optimal play give we are at that state. I the followig sectios, we will establish upper ad lower bouds o the optimal payoff of this game. 2 Recursive Relatio ad Lower Boud Let h, represet our positio, where is the total umber of times we have flipped the coi ad h is the umber of heads we have observed. 1

2 From h,, we have two choices: stop or cotiue. If we stop, we receive h/ i payoff. If we cotiue, with 1/2 probability we ed up at h, + 1 ad with 1/2 probability we ed up at h + 1, + 1. Thus, if we let V h, deote our expected payoff from h,, we have the followig recursive relatio: h V h, V h + 1, + 1 V h, = max,. 1 2 At first glace this may seem like a stadard recursio but upo closer ispectio we otice somethig strage: each V h, depeds o terms with successively larger, so the base case for this recursio is V h,, which we ca t compute! To get aroud this problem, we will choose a horizo N such that we fid lower bouds for all V h, N ad the recursively apply 1 to get lower bouds for all V h,, < N. We ca thik of N as the cutoff poit where we cease to use the recursive formula 1 ad as a result do t compute ay of the terms V h,, > N. To obtai a lower boud, otice that at positio h,, we ca guaratee h/ by stoppig. Moreover, if h/ < 1/2, we ca wait util the proportio of heads is at least 1/2 to stop which occurs with probability 1. 1 Thus, we attai the followig simple lower boud: h V h, max, While this lower boud holds i geeral, for smaller values of it is ot very tight ad so we will avoid usig it whe possible. Choosig a larger horizo N to apply the lower boud from 2 o gives a tighter boud o each of the V h, s after recursig. As N approaches ifiity, the lower boud we obtai for each V h, approaches the true value. Usig N = 10 7, Häggström ad Wästlud [1] compute the followig lower boud: V 0, 0 > Upper Boud o V h, The followig sectio is motivated by Häggström ad Wästlud [1]. Let Ṽ h, be the expected payoff from positio h, uder ifiite clairvoyace, 1 For a rigorous proof, see Theorem

3 i.e., we have complete kowledge of the results of all future coi flips ad Ṽ h, is the largest proportio of heads ever attaied. Sice we kow that V h, Ṽ h,, ay upper boud o Ṽ h, is ecessarily a upper bod o V h,. I order to establish a upper boud o Ṽ h,, we first prove the followig: Let P h,, p be the probability that m, m such that the proportio of heads i the first m flips exceeds p. P h,, p ca also be thought of as the probability that startig from h,, at some poit ow or i the future the overall proportio of heads exceeds p. Let k = kh,, p be the miimum umber of coi flips from h, for which it is possible to achieve a overall proportio of heads exceedig p i k more flips. For example, k1, 3, 0.5 = 2 ad k2, 3, 0.5 = 0. Lemma 3.1. P h,, p 1 2p k Proof. Oe ca check that if p maxa/, 1/2, the statemet holds. If p < a/ or p 1/2, the RHS 1/2p k 1. The case of p = a/ is left as a exercise for the reader. Cosider Prext k flips are heads proportio p will evetually be exceeded. O oe had, P rext k flips are heads proportio p evetually exceeded 4 P rext k flips are heads proportio p evetually exceeded = proportio p evetually exceeded = P rext k flips are heads proportio p evetually exceeded = 1/2k P h,, p, where the secod equality holds because the ext k flips beig heads guaratees that a proportio p of heads will evetually be exceeded. However, we ca also establish a lower boud for this probability. I 4, we coditio o the existece of a m for which the proportio of heads i the first m 3 5

4 flips exceeds p. Sice we assumed p > 1/2, we kow by the Weak Law of Large Numbers that with probability 1 there exists a maximal m = M for which the first m coi flips have a proportio greater tha p of heads. Because we are operatig uder ifiite clairvoyace, we kow the value of M ad so we also kow the exact umber of heads H i flips + 1, + 2,..., M. Furthermore, we kow that all permutatios of those M flips cotaiig H heads are equally likely by symmetry. Cosider flip + 1. If k > 1, the proportio of heads up to this poit is less tha or equal to p but at flip M exceeds p so the proportio of heads q 1 amog flips + 1,..., M ecessarily exceeds p. Sice flip + 1 is just as likely as ay of flips + 2,..., M of beig amog the q 1 > p that are heads, the probability that flip +1 is heads is q 1 > p. Similarly, cosider flip + 2. Coditioed o flip + 1 beig heads, the proportio of heads up to this poit is still at most p but exceeds p at flip M. If we let q 2 be the proportio of heads amog flips + 2,..., M, we kow q 2 > p ad cosequetly the probability of flip + 2 beig heads give flip + 1 is heads is greater tha p. Cotiuig this argumet for i = 3,..., k, we have that flip x+i has a probability greater tha p of beig heads give flips x + 1,..., x + i 1 were heads. Thus, the coditioal probability of flips x + 1,..., x + k beig heads P rext k flips are heads proportio p evetually exceeded p k. 6 Combiig 5 ad 6 ad rearragig gives the desired result. Theorem 3.2. h Ṽ h, max, 1 1 π + mi 2 4, h Proof. The followig derivatio is reproduced from Häggström ad Wästlud [1]. From the iverse-cdf defiitio of expected value, we have: By defiitio, k satisfies which reduces to 1 Ṽ h, = P h,, pdp 0 h = max, P h,, pdp 2 max h, 2 1 h + k 1 + k 1 p k 1 + p h 1 p. 8 4

5 Furthermore, k is the largest iteger that satisfies 8 so we have: Substitutig 9 ito Lemma 3.1. yields: k p h 1 p. 9 P h,, p 1 2p p h 1 p It follows that h Ṽ h, max, max h, 1 2 dp 2p p h 1 p Substitutig p = 1 + t/2 ad usig the well-kow iequality. 10 log1 + t 1 t t, 10 simplifies to h Ṽ h, max, h = max, h max, max 2h 1,0 dt exp max 2h,0 1 Usig the substitutio u = t, we obtai: h Ṽ h, max, 1 2 h max, t 1+t 2h 1 t 1 + t 2h log1 + t 1 t exp 1 + tt + 2htdt max 2h, exp max 2h,0 exp max 2h,0 Usig the substitutio w = u 2h /, exp u 2 + 2h u du = 2h From 11 ad 12, we ca check that h Ṽ h, max, h 5 0 dt u 2 + 2h u du u 2 + 2h u du 11 exp w 2 2h w dw 12 exp u 2 2h u du. 13

6 Furthermore, by discardig terms from the itegral i 13, we have 1 2 exp u 2 2h u du 1 2h 2 exp u 2 du 2h = 1 π 4 ad 1 2 exp u 2 2h u du 1 2h 2 1 = 2h 2h exp u du h. 15 Combiig 14 ad 15, we have 1 2 exp u 2 2h u 2h du mi 1 π 4, 1 2 2h 16 Substitutig 16 ito 13 gives the desired result. To compute a upper boud o Ṽ 0, 0, we tur to the a similar strategy to the oe we used to calculate a lower boud. I particular, we will use Theorem 3.2 to compute upper bouds o terms of the form Ṽ h, N for some large N, from which we also have upper bouds o the V h, Ns. We will the apply our fudametal recursive relatio from 1 to compute upper bouds o all V h,, < N. Usig N = 10 7 gives the upper boud [1]: V 0, 0 < Lastly, combiig the lower boud 3 we derived earlier with this upper boud 17 gives us fairly tight bouds o the expected value of the game uder optimal play: < V 0, 0 < Strategies for Playig the Game Now that we ve derived bouds o the expected payoff of this game, it s worth examiig some differet strategies ad to see how they fare i compariso. We will fid the followig results helpful i our aalysis of these strategies: 6

7 Lemma 4.1. If we flip a fair coi forever, the probability of at some poit obtaiig k more heads tha tails or k more tails tha heads is 1 for all values of k. Proof. Let h be the umber of heads ad t be the umber of tails we observe i the first flips respectively. Let d = h t be the heads-tails differetial after flips. We are tryig to show that with probability 1, d = k for some. The key realizatio we make is that if d N < k, flippig 2k heads or 2k tails i a row causes d N+2k > k, which meas for some value of {N + 1,..., N + 2k 1}, d = k. Thus, i order for the coditio of d < k to hold for all, we ca ever flip 2k heads or 2k tails cosecutively. We ow cosider blocks of 2k flips at a time. The probability that we avoid flippig 2k cosecutive heads or 2k cosecutive tails withi the first 2k flips is simply 2k 2. 2 k Likewise, this is also true for the ext 2k flips, the 2k flips after that, etc. Thus, the probability that i the first b blocks of 2k flips each, o idividual block cotais all heads or all tails is b 2 2k k As b goes to ifiity, this expressio goes to 0, so the probability that some block cotais 2k heads or 2k tails is 1, completig the proof. Theorem 4.2. [Simple Radom Walks] If we flip a fair coi forever, the probability of at some poit obtaiig k more heads tha tails is 1 for all values of k. Proof. Reusig our otatio from Lemma 4.1, we seek to show that with probability 1, d = k for some. From Lemma 4.1, we kow that with probability 1, d = k for some. By symmetry, d is equally likely to reach k or k first. Thus, with probability 1/2, d reaches k first ad we are doe. With probability 1/2, d reaches k first. However, reapplyig this argumet, the heads-tails differetial startig from the + 1-th flip is equally likely to reach 2k or 2k. With probability 1/2, it reaches 2k first, resultig i d = k +2k = k, i which case we are doe. With probability 1/2, it reaches 2k first, causig d = k 2k = 3k, i which case we repeat the argumet agai. Thus, the probability of d = k at some poit is: This completes the proof = 1. 7

8 The first strategy we will cosider is a very simple strategy: Strategy If the first flip is heads, stop. of heads equals the umber of tails. If it is tails, flip util the umber Aalysis. With probability 1/2, we flip heads ad our payoff is 1 ad with probability 1/2 we flip tails ad evetually receive a payoff of 1/2 we ca guaratee termiatio because of Theorem 4.2. Thus the expected payoff of this strategy is just E = = Notice that this simple strategy actually does reasoably well compared to the boud obtaied i 18. We ow cosider a more iterestig strategy: Strategy Stop whe the umber of heads first exceeds the umber of tails. Aalysis. We first ote that termiatio is guarateed sice this correspods to k = 1 from Theorem 4.2. We also ote that termiatio will always occur o a odd umbered flip. With this i mid, the expected payoff of this strategy is the E = + 1 p , =0 where p 2+1 is the probability we termiate o the th flip ad + 1/2 + 1 is our payoff from termiatig o the th flip. We ow try to compute p 2+1 : p 2+1 = # of ways to termiate o th flip # of possibilities for first flips 20 Clearly, the deomiator of 20 is just To compute the umerator, otice that the termiatig flip will always be a head sice if it were tails we would have termiated sooer. Thus, computig the umerator reduces to fidig the umber of sequeces X 1,..., X 2 cotaiig exactly heads ad tails such that for o i {1,..., 2} does X 1,..., X i cotai more heads tha tails this last coditio prevets early termiatio. 8

9 However, this is equivalet to the umber of Dyck paths 2 of legth 2, which is kow to be the -th Catala umber C. Thus, we have: + 1 E = p =0 C = =0 = =0 = 1 2 arcsi1 = π 4. Note that π/ , so this is remarkably close to the bouds obtaied i 18. Commet: Because of Theorem 4.2, all strategies of the form stop whe the umber of heads first exceeds the umber of by tails by k are valid, i.e., termiate with probability 1. It turs out that the strategy above k = 1 has the highest expected value out of all strategies i this family. Questio to the reader: Ituitively why does this make sese? 5 Further Exploratio As of today, o kow strategy has a expected payoff withi the bouds we derived 18 ad so a optimal strategy has ot yet bee discovered. To my kowledge, the tightest kow bouds for the expected value of this game are give by Julia Wisema [3], who used a larger horizo of N = Possible areas for research would be improvig these bouds either through icreased computatioal power or by usig a more computatioally efficiet boudig strategy. Aother possible extesio of this problem which was suggested by a classmate was to repeat the same aalysis we performed but for biased cois with probability p 1/2 of ladig heads. 2 Sequeces cotaiig equal umbers of up ad dow steps i which every prefix has at least as may up steps as dow steps. 9

10 Refereces [1] Olle Häggtröm ad Joha Wästlud, Rigorous computer aalysis of the Chow- Robbis game, The America Mathematical Mothly 12010: [2] Yua-Shih Chow ad Herbert Robbis, O optimal stoppig rules for s /, Ill. J. Math., 9: , [3] Julia D. A. Wisema, The Chow & Robbis Problem: Stop at h=5, t=3, web page, robbis.html 10