Keywords: Last-Success-Problem; Odds-Theorem; Optimal stopping; Optimal threshold AMS 2010 Mathematics Subject Classification 60G40, 62L15

Transcription

1 CONCERNING AN ADVERSARIAL VERSION OF THE LAST-SUCCESS-PROBLEM arxiv:8.0538v [math.pr] 3 Dec 08 J.M. GRAU RIBAS Abstract. There are idepedet Beroulli radom variables with parameters p i that are observed sequetially. Two players, A ad B, act i turs startig with playera.each playerhas the possibilityohistur, whe I k =, to choose whether to cotiue with his tur or to pass his tur o to his oppoet for observatio of the variable I k+. If I k = 0, the player must ecessarily to cotiue with his tur. After observig the last variable, the player whose tur it is wis if I =, ad loses otherwise. We determie the optimal strategy for the player whose tur it is ad establish the ecessary ad sufficiet coditio for player A to have a greater probability of wiig tha player B. We fid that, i the case of Beroulli radom variables with parameters /, the probability of player A wiig is decreasig with towards its limit e = We also study the game whe the parameters are the results of uiform radom variables, U[0, ]. Keywords: Last-Success-Problem; Odds-Theorem; Optimal stoppig; Optimal threshold AMS 00 Mathematics Subject Classificatio 60G40, 6L5. Itroductio The Last-Success-Problem LSP) is the problem of maximizig the probability of stoppig o the last success i a fiite sequece of Beroulli trials. There are Beroulli radom variables which are observed sequetially. The problem is to fid a stoppig rule to maximize the probability of stoppig at the last. This problem has bee studied by Hill ad Kregel [4] ad Hsiau ad Yag [5] for the case i which the radom variables are idepedet ad was simply ad elegatly solved by T.F. Bruss i [] with the followig famous result. Theorem. Odds-Theorem, T.F. Bruss 000). Let I,I,...,I be idepedet Beroulli radom variables with kow. We deote by i =,...,) p i the parameter of I i ; i.e. p i = PI i = )). Let q i = p i ad r i = p i /q i. We defie the idex { max{ k : j=k s = r j }, if i= r i ;, otherwise To maximize the probability of stoppig o the last i the sequece, it is optimal to stop o the first that we ecouteramog the variables I s,i s+,...,i. The optimal wi probability is give by ) Vp,...,p ) := r i j=s q j i=s

2 J.M. GRAU RIBAS We propose the followig adversarial versio of the problem i this paper. There are idepedet Beroulli radom variables I i with parameters p i that are observed sequetially. Two players, A ad B, act i turs startig with A. After observig the value of I k, if I k =, the the player whose tur it is may pass his tur to his oppoet or use it ad observe the variable I k+. Whe the last evet is reached, if the result is success I = ), the player whose tur it is wis, ad loses otherwise. Specifically, if I i = 0 for all i, player A loses. This is remiiscet of the hot potato game i which the goal is ot to be holdig the hot potato at the ed of the game, with the rule of beig able to pass it o if oe so wishes) to oe s oppoet whe I k =. Let us deote by V k the probability of the player whose tur it is wiig whe we are about to observe the variable I k. I particular, the probability of player A wiig is V ; hece the probability of player B wiig is V. Likewise, o observig the last radom variable, the player whose tur it is will wi with probability p k, i.e. V = p. The dyamic program to fid the optimal strategy is straightforward. After observig the variable I k, if I k = 0, which occurs with probability p k, the player the irrevocably goes o to observe the variable I k+ without givig up his tur. If I k =, the optimal strategy of the player whose tur it is will cosist i passig his tur to his oppoet if V k+ < / ad i cotiuig with his tur if V k+ /. We shall the have the followig recurrece. V k = p k max{v k+, V k+ }+ p k )V k+ ;V = p. Optimal strategy We shall see that the optimal strategy is extremely simple ad that it is also very easy to determie which of the two players has the greatest probability of wiig. Aother matter altogether is the exact calculatio of this probability, which geerally requires the computatio of recurrece or calculatios of the equivalet cost. Propositio. If for all k [r,], p k < /, the for all k [r,] the followig is fulfilled: p = V < V k+ < V k <. Proof. It is evidet that V = p < /. We proceed by backward iductio. We assume that the propositio is true for all i [k+,] ad shall prove that it also holds for i = k V k = p k max{v k+, V k+ }+ p k )V k+ From the iductio hypothesis, V k+ < /, therefore V k+ > / > V k+, ad hece V k = p k V k+ )+ p k )V k+ > p k V k+ + p k )V k+ = V k+ V k = p k V k+ )+ p k )V k+ < p k V k+ )+ p k ) V k+ ) = V k+ <.

3 CONCERNING AN ADVERSARIAL VERSION OF THE LAST-SUCCESS-PROBLEM 3 Propositio. Let Ω r := {k [r,] : p k /} ad cosiderig { maxω, if Ω r ; u r := r, if Ω r =. The optimal strategy for the player whose tur it is whe observig the variable I r is ot to give up his tur before stage u r ad to do so whe he may startig from u r. I additio, the followig is true. If Ω r =, the V r <. If Ω r ad p ur = /, the V r =. If Ω r ad p ur > /, the V r >. Proof. If Ω r =, from Propositio we have that V k+ < V k < V r < / for all k [r +,]. Thus, it is always preferable for the player to give up his tur after ay stage k r tha to cotiue with his tur. If the player cotiues with his tur, the wi probability is V k+ < /, while if he gives up his tur, it is greater. If Ω r, we have that p ur ad p k < / for all k [u r +,]. Hece, similar reasoig as above may be applied. If p ur = /, the V r = /maxv r+, V r+ )+/V r+ If p ur > /, the V r = / V r+ )+/V r+ = V r = p ur maxv r+, V r+ )+ p ur )V r+ V r = p ur V r+ )+ p ur )V r+ < Let us deote by u the optimal threshold of the first player i his first tur, u := u the last Beroulli evet with parameter /. The optimal strategy of the first player cosists i cotiuig with his tur util reachig the u-th evet ad thereafter givig up his tur wheever possible. Obviously, player B will do the same i his optimal game because, whe his tur comes, he will be i the same situatio as player A. I short, we have the followig result. Theorem. The optimal strategy for both players is to give up their tur whe ad oly whe) there are o radom variables left to observe whose parameter is greater tha or equal to /. I fact, whe played optimally by both players, the game ca be see as a game of solitaire played by player A assumig his oppoet uses the optimal strategy. Thus, the probability of player A wiig, the optimal threshold beig u, is the probability that the umber of s startig from the u resultig from the radom variats is odd; that is to say: Hece the followig result: ) V = P I i = odd i=u

4 4 J.M. GRAU RIBAS Propositio 3. The probability of player A wiig, whe both players use the optimal strategy, is ) V = p i ) i+ p +i i. +i i=u The above propositio allows establishig a somewhat coarse) lower boud for the probability of player A wiig. Bear i mid that the wi probability i this game is greater tha the probability of wiig i the LSP. Propositio 4. If p i i= p i, the V > e. Proof. It suffices to keep i mid that the probability of wiig i the LSP uder these coditios is greater tha /e see []). 3. All the radom variables have the same parameter I this sectio, we study the particular case that all the Beroulli radom variables have the same parameter. Propositio 5. If p i = p for all i =,...,, the the probability of player A wiig is strictly icreasig with always below its limit as teds to ifiity V ) = p) Proof. We take Propositio 3 ito cosideratio. If is eve, we have <. V ) = i=0 Similarly, if is odd, we have V ) = i=0 ) p) i+ p +i = p). +i ) p) i+ p +i = p). +i Propositio 6. If we have Beroulli radom variables with p i =, the probability of player A wiig is decreasig ad is always greater that its limit as teds to ifiity, amely e = Proof. If = the V =. If = the V =. If > 3, the optimal strategy for both players is to give up their tur wheever possible. Hece, player A will wi if the umber of s resultig from the radom variables is odd. If is eve V = i=0 ) i+ ) +i ) = +i

5 CONCERNING AN ADVERSARIAL VERSION OF THE LAST-SUCCESS-PROBLEM 5 = Similarly, if is odd V = i=0 = + ) + + ) + + ) ) ) i+ ) +i ) = +i + ) + + ) + + ) ) lim V = e = Propositio 7. If we have Beroulli radom variables with p i, the probability of player A wiig is greater tha e = Proof. If p i / for some i, the V. Otherwise, thik of the auxiliar game with all the parameters equal to / i which the probability is greater tha e. Now, there is o more to cosiderig successivemodificatios of this game, as i Lemma, with which the wi probability icreases, util reachig the game cosidered. Propositio 8. If we have Beroulli radom variables, of which there are m with p i m, the probability of player A wiig is greater tha e = Proof. It ca easily be see that if p i < / for all i, the the probability of player A wiig is greater tha the probability that he would have i the game resultig from excludig some radom variable. Cosequetly, it suffices to observe that the value, e, is exceeded i the auxiliary game resultig from excludig some radom variables. I fact, if we have m variables with p i < /m for all of these variables, cosiderig the game i which the other variables are excluded, the we are able to use the previous propositio. Lemma. Let us cosider the game with parameters p i < ad deote by V i the probability of a player wiig whe it is his tur after observig the variable I i i the resultig auxiliary problem whe chagig p k to p k > p k. Hece, For all i [k +,],V i = V i For all i [,k],v i > V i I other words, if we icrease the value of the parameter of oe of the Beroulli radom variables i a game, the the player s probability of wiig o his tur icreases.

6 6 J.M. GRAU RIBAS Proof. Let us recall that V i ad V i respectively deote the player s probability of wiig o his tur at stage i i the origial game ad i the auxiliary game. For all i > k, it is evidet that V i = V i as we are i a subsequet stage to the modified variable ad the process has o memory ad therefore does ot affect. For all i k, we will proceed by iductio backwards. Let us first see that it is true for i = k. V k = p k V k+ )+ p k )V k+ V k = p k V k+ )+ p k )V k+ V k V k = p k p k ) V k+ ) p k p k )V k+ = p k p k ) V k+ ) > 0 We ow assume that the proposal is fulfilled for i+ ad shall prove that it is fulfilled for i V i = p i V i+ )+ p i )V i+ V i = p i V i+ )+ p i )V i+ V i V i = p i V i+ V i+ ) p i )V i+ V i+ ) = p i )V i+ V i+ ) > A variat: If there have bee o s, the game is repeated. We have see that the game is advatageous for player A if ad oly if some parameter is greater tha /. The reaso that the game is disadvatageous for player A is related to the fact that he ca lose because the results of all the radom variables are 0. I fact, if ay of the variables is worth, the the probability of the player wiig by givig up his tur is greater tha /. The followig result shows that, if the rule of repeatig the game is itroduced ad if there have bee o s, the the game is very advatageous for player A. Propositio 9. If p i = / for all i ad cosiderig the rule that the game is repeated i the case of I i = 0 for all i, the probability of player A wiig is V) := Besides, we have that V) is icreasig ad ) +) +) + +). +e = V) < V) < 3 e = Proof. Obviously, the optimal strategy with this rule is the same as i the game i its origial versio. The differece lies oly i the probability of wiig, which is coditioed by i= I i > 0. Thus, bearig i mid that ) P I i > 0 = P I = I =... = I = 0) = ) i=

7 CONCERNING AN ADVERSARIAL VERSION OF THE LAST-SUCCESS-PROBLEM 7 we have that V) = + ) + + ) + + Moreover, V) is icreasig ad its limit is +e e. ) ) ) 4. Radom parameters for the Beroulli variables We fially determie the probability of player A wiig mea probability) whe the parameters are the results of uiform radom variables, U[0, ]. That is to say, before holdig the competitio, the parameters of the Beroulli variables are draw via radom trials of a uiform radom variable U[0, ] ad these parameters are revealed to the players. Lemma. If the last k variables {I i } i= k+ have parameters p i, which are the results of the uiform radom variables U[0, /], the the wi probability of the player whose tur it is at stage k + is V k+ = k + k) I particular, if k = V = + ) Proof. We deote by X i the wi probability of player whose tur it is at stage i +. Bearig i mid that t = p k+ is the result of a uiform radom variable, U[0, /] X i = EtX i + t) X i )) X i = tx i + t) X i ))dt = 0 4 +X i ) ad solvig with X 0 = 0 gives X k = k + k) Lemma 3. If we have k ed variables with parameters p i that are the result of uiform radom variables, U[0,/], ad p k is the result of a uiform radom variable, U[/, ], the the probability of player A wiig is J k := + k Proof. We deote by J k the wi probability of the player whose tur it is after k +. Reasoig similar to above J k = = / J k = EtX k + t) X k ) tx k + t) X k ))dt k + k +4x ) dt = + k )

8 8 J.M. GRAU RIBAS Lemma 4. If all the parameters p i are the result of uiform radom variables, U[0, /], the player A s probability of wiig is J k := + k Propositio 0. If we have a game with variables whose parameters are the result of the uiform radom variables, U[0, ], player A s probability of wiig is 4 ). 3 Proof. The probability that the last parameter greater tha or equal to / will be the k-th is +k ad the probability that all the parameters are less tha / will be. E) = + ) + k= J k k = 4 ) 3 5. Coclusios ad future challeges The proposed adversarial versio of the Last-Success-Problem has a very simple optimal game strategy that does ot require ay calculatio. It oly requires idetifyig the last variable whose parameter is greater tha / ad, as from that poit o, always givig up oe s tur to oe s oppoet. It seems iterestig to pose the problem with o-idepedet radom variables. The Last-Success-Problem with depedet Beroulli radom variables was addressed by Tamaki i [8], who cosidered that I,...,I costitute a Markov chai with trasitio probabilities α j = PI j+ = I j = 0) β j = PI j+ = 0 I j = ) ad established a optimal stoppig rule with a Markov versio of the oddstheorem. We predict that the adversarial versio with depedet variables will also be simple ad the optimal strategy will most likely cosist i adoptig, at each k- th stage, the optimal strategy while assumig that the remaiig variables are idepedet Îk+,...,Î with parameters computable by recurrece) I short, we cojecture the followig. p i = PI i = I k = ) Cojecture. Let I,I,...,I be depedet Beroulli radom variables. Let p i,k := PI i = I k = ). The, the optimal strategy for the player whose tur it is after observig the variable I k = is to give up his tur to his oppoet if ad oly if p i,k < / for all i > k. It may also be iterestig to pose the game with more tha players, i which case differet types of paymet could be cosidered. For ay versio, it is ormal to cosider the loser to be the player whose tur it is after the last Beroulli trial, but several types of paymet may be cosidered for the other players. If we cosider that the players who do ot lose each receive the same paymet, we have the simplest

9 CONCERNING AN ADVERSARIAL VERSION OF THE LAST-SUCCESS-PROBLEM 9 versio. I this respect, we coclude by posig the challege to determie the limit with m players, whe teds to ifiity, of the loss probability of each player, cosiderig idepedet Beroulli radom variables with parameters /. I fact, for 3 players, it is o loger a trivial problem, as oly the limit for the probability of the first player losig is exactly calculable i a relatively straightforward way. Not without difficulty ad usig the Mathematica symbolic calculatio package, we obtaied the followig limit for the probability of the first player losig: loss = 3 + cos 3 ) 3e 3 = However, it is o loger viable to fid the exact limit of the probability of the other two players losig via this path. Computig for large values of allows a approximatio, but oly that. Specifically, we have that: loss ad loss I all the above calculatios, we have assumed that the optimal strategy for both players is to give up their tur wheever possible. Of course, this will udoubtedly be true i this case. I geeral, however, there will be a optimal strategy that does ot always cosist i passig oe s tur to the followig player. Refereces [] Bruss, F.T. 000) Sum the odds to oe ad stop. A. Probab. 8, o. 3, [] Bruss, F.T. 003) A ote o bouds for the odds theorem of optimal stoppig. A. Probab. 3, o. 4, [3] Dedievel, Rémi. 03) New developmets of the odds-theorem. Mathematical Scietist. Vol. 38 Issue, -3. [4] Hill T. P. ad Kregel, U. 99). A prophet iequality related to the secretary problem. Cotemp. Math [5] Hsiau, S. R. ad Yag, J. R. 000). A atural variatio of the stadard secretary problem. Statist. Siica [6] Katsuori Ao, Hideo Kakiuma ad Naoto Miyoshi 00) Odds theorem with multiple selectio chaces. Joural of Applied Probability, vol. 47, o. pp [7] Tamaki, M. 00) Sum the multiplicative odds to oe ad Stop. Joural of Applied Probability, vol. 47, o. 3 pp [8] Tamaki, M. 006) Markov versio of Bruss odds-theorem the developmet of iformatio ad decisio processes). RIMS Kokyuroku, 504: Departameto de Matemáticas, Uiversidad de Oviedo, Avda. Calvo Sotelo s/, Oviedo, Spai address: grau@uiovi.es