When p = 1, the solution is indeterminate, but we get the correct answer in the limit.

Transcription

1 The Mathematia Journal Gambler s Ruin and First Passage Time Jan Vrbik We investigate the lassial problem of a gambler repeatedly betting $1 on the flip of a potentially biased oin until he either loses all his money or wins the money of his opponent. This is then extended to the ase of his adversary (a asino) having pratially unlimited resoures and used to derive the inverse Gaussian distribution of the first passage time. Probability of Winning the Game We assume that the probability of winning a single round of this game is p, whih is usually lose, but not neessarily equal, to 1. Thus, for example, when betting $1 on blak in the game of roulette, with 18 blak, 18 red, and 1 green, all with equally likely outomes, p is equal to 18 = We also assume that our player and his adversary start with a 37 dollars and b dollars, respetively, and that they are both determined to ontinue playing until one of them goes broke. Our first objetive is to find the probability that our player wins the game, ending up with a + b dollars. To do this, we have to imagine that the game has been going on for some time, and the player has reahed the point of having exatly i dollars in his poket, so that his opponent has a + b - i. Given that, we denote our player s probability of winning the game by w i. If one more round is played, one an see that w i = p w i+ 1 + H1 - p w i-1, depending on whether the player wins (the first term) or loses (the seond one) the next round [1]. The end onditions are w a+b = 1 (the game is already won) and w 0 = 0 (the game is lost). We thus have a + b - 1 linear equations for the orresponding w i probabilities, where 0 < i < a + b. The equations are of a rather speial type, as eah of them involves only three onseutive values of w i, so rather than using the usual ineö arsolve proedure, it is more appropriate to swith to RSolve. (1)

2 Jan Vrbik sol = RSolve@8w@iD ã p w@i + 1D + H1 - p w@i - 1D, w@0d ã 0, w@a + bd ã 1<, w@id, id@@1dd :w@id Ø -1 + J 1-p p Ni > -1 + J 1-p p Na+b When p = 1, the solution is indeterminate, but we get the orret answer in the limit. imitbw@id ê. sol, p Ø 1 F i a + b All we have to do now is to evaluate w i using i = a. To this end, we build the following funtion. Pw@a_, b_, p_d := I1 - HH1 - p ê p a M ë H1 - HH1 - p ê p^ha + b Pw@a_, b_, 1 ê D := a ê Ha + b We apply the funtion to the ases disussed so far. Pw@10, 10, 1 ê D êê N Pw@10, 10, 18 ê 37D êê N Pw@100, 100, 18 ê 37D êê N From these examples we an see that when both players start with the same amount of money and the oin is fair, eah of them an win with the same probability of 50%. But as soon as the oin is slightly biased, the probability of the disadvantaged player winning dereases; this disparity beomes more pronouned as the initial apital of both players inreases. In pratie, things are even worse than that: the asino starts with pratially unlimited resoures, whih implies that a disadvantaged player does not stand any hane of winning

3 Gambler s Ruin and First Passage Time 3 whenever p 1. Should p be slightly higher than 1 though, his hanes to ontinue winning indefinitely are omputed by taking the limit of w a as b Ø, resulting in p p a. () Distribution of the Gameʼs Duration To find the distribution of the random number of rounds it takes to omplete a game with two players, eah with finite resoures, given that urrently the first player has i dollars in his poket, we introdue the orresponding probability generating funtion H i Hz. Similarly to how we solved for w i, we an now set up the equations H i Hz = z Hp H i+1 Hz + H1 - p H i-1 Hz, where the multipliation by z is neessary to aount for the extra round played. We also know that H 0 Hz = H a+b Hz = 1 (a probability generating funtion of 1 indiates that there are no more rounds to be played). We now solve these equations. RSolve@8H@iD ã z p H@i + 1D + z H1 - p H@i - 1D, H@0D ã 1, H@a + bd ã 1<, H@iD, id@@1dd êê FullSimplify (3) :H@iD Ø -i H-1 + p i - a+b H-1 + p a+b H-1 + p i a+b H-1 + p a+b ì H-1 + p a+b H-1 + p a+b >

4 4 Jan Vrbik Substituting a for i yields the orresponding result. H@a_, b_, p_d := ModuleB:r = p H1 - p z, s = p H1 - p z >, r a I1 - s a+b M - s a I1 - r a+b M F r a+b - s a+b Using this funtion, we an now easily find and display the orresponding distribution of the game s duration. aux = Series@N@H@10, 10, 18 ê 37D, 36D, 8z, 0, 400<D; istplot@table@8i, Coeffiient@aux, z, id<, 8i, 10, 400, <D, AspetRatio Ø.3, ImageSize Ø 700D One an see that these games an easily last hundreds of rounds (things get worse when p approahes 1 ). Sometimes it is suffiient to know only the mean and variane of this distribution. m@a_, b_, p_d := a - Ha + b Pw@a, b, pd 1 - p Var@a_, b_, p_d := a + b 1 - p 4 J 1-p p Na m@a, b, pd J 1-p p Na+b Ha + b Pw@a, b, pd H1 - Pw@a, b, pd p + 4 p H1 - p m@a, b, pd H1 - p

5 Gambler s Ruin and First Passage Time 5 The orretness of these formulas an be verified by realling that the mean m of a distribution is given by H ' Hz evaluated at z = 1, and its variane similarly by H '' Hz = 1 + m - m. ID@H@a, b, pd, zd ê. z Ø 1 ê. H1-4 H1 - p p q_ ß H1 - p q M - m@a, b, pd êê Simplify 0 ID@H@a, b, pd, 8z, <D ê. z Ø 1 ê. H1-4 H1 - p p q_ ß H1 - p q M + 0 m@a, b, pd - m@a, b, pd - Var@a, b, pd êê Simplify The two formulas an be extended to the ase of p = 1 by taking the orresponding limit. imit@m@a, b, pd, p Ø 1 ê D a b imit@var@a, b, pd, p Ø 1 ê D 1 3 a b I- + a + b M These impose the following. m@a_, b_, 1 ê D := a b Var@a_, b_, 1 ê D := a b Ia + b - M 3 The Case of an Infinitely Rih Adversary By analyzing the loal variables r and s in the definition of H, we an see that r > 1 and 0 < s < 1, for any 0 z < 1. As b Ø, we thus get H a Hz = s = p H1 - p z a. (4)

6 6 Jan Vrbik When p > 1 1-p, the probability that the game finishes in finite time is J p Na, aording to (). The onditional probability generating funtion of the total number of rounds, given that the game does not ontinue indefinitely, is thus p H1 - p z H1 - p z a. The more interesting ase is when p 1, whih implies that the game annot ontinue indefinitely. Expanding (4) in powers of z would enable us to plot the orresponding distribution, as was done at the beginning of this setion; we leave this as an exerise. Here is the mean and variane of the total number of rounds. (5) aux = p H1 - p z a ; D@aux, zd ê. z Ø 1 ê. H1-4 H1 - p p q_ ß H1 - p q êê Simplify a 1 - p ID@aux, 8z, <D ê. z Ø 1 ê. H1-4 H1 - p p q_ ß H1 - p q M + % - % êê Simplify 4 a H-1 + p p H-1 + p 3 Brownian Motion In this last setion, we explore yet another interesting limit of the gambler s ruin problem. First we assume that the game is happening in real time t and that n rounds are played during eah hour (or any other unit of time). We also assume that p = 1 + d, ÿ n where d and > 0 are two onstants, and that n (6) dollars are bet in eah round instead of the original $1. In the limit as n Ø, this results in a new proess alled Brownian

7 Gambler s Ruin and First Passage Time 7 motion that runs in real (i.e. ontinuous) time and whose values onstitute a ontinuous, even though nowhere differentiable, funtion of t []. To visualize its possible ourse, we take d = -0.35, =, and n = 0 and display a random realization of the proess; that is, we display its urrent values the amount of money in the gambler s poket as a funtion of t during the next 50 hours, taking the initial value to be $100. ModuleB8n = 0, =, d = -0.35, t = 50, A = 100, aux<, aux = RandomVariateBBinomialDistributionB1, 1 + d n F, t nf - 1 n ; istplot@table@8i, A + Apply@Plus, Take@aux, idd<, 8i, t n<d, AspetRatio Ø.3, ImageSize Ø 700DF Assuming the gambler an borrow money and ontinue playing even when the value of the proess (his urrent worth) is negative, it is easy to see that this value at time t is a normally distributed random variable with mean A + d t and variane t, where A is the initial value, and d and (introdued earlier) are the so-alled drift and diffusion oeffiients, respetively. This assertion an be proved by realizing that the moment-generating funtion of the net win (equal to 1 with probability p and to -1 with probability 1 - p) in one round of the original game is MHu = p e u + H1 - p e -u. To adjust this to the new game (winning or losing by u n n dollars, instead of $1), replae eah u of the last expression. To get the moment-generating funtion of the total win (or loss, when negative) aumulated during a time interval of length t, we have to add the results of n t independent rounds, whih orresponds to raising MHu to the power of n t. Finally, we need to take the limit as n Ø.

8 8 Jan Vrbik imitb 1 + d n ExpBu n F d n ExpB-u n F n t, n Ø F 1 t u H d+ u This is the moment-generating funtion of a normal distribution with expeted value d t (the exponent s oeffiient of u) and variane of t (the oeffiient of u ). Sine it represents only the total net winnings up to time t, we have to add the initial value of A to get the distribution of the gambler s net worth at time t; this will inrease the expeted value to A + d t. Inverse Gaussian Distribution We now establish the distribution of T, the first passage time through the value of zero, whih is the time at whih the gambler has to stop playing, sine he has lost all his money. We assume that he starts with the amount A and that d < 0; reahing the value of zero in finite time is thus guaranteed. We proeed similarly to deriving the distribution of his net worth: we first find the moment-generating funtion of T in terms of the old game, with n rounds played every hour, and then take the limit as n Ø. To get the moment-generating funtion of the number of rounds of the old game needed to go broke, we simply replae z in (4) by e u, and a by A n, the initial amount expressed in the new monetary units. To onvert the result into hours, eah onsisting of n rounds, we need to replae u by u n. imitb J1 - d N Exp@ u ê nd n K1 + d n O Exp@u ê nd A n, n Ø F A 1 - d - d - u

9 Gambler s Ruin and First Passage Time 9 The resulting moment-generating funtion an be rewritten in the following simple form, using only two parameters: = - d A and t 0 = - A (sine d < 0, both are positive): d 1-1- t 0 u e. The orresponding probability density funtion is (7) f Ht = ÿ t 0 Ht - t0 exp - 3 p t t t 0 (8) for t > 0 and f Ht = 0 otherwise. To prove this, we ompute the moment-generating funtion of (8), whih agrees with (7). IntegrateB t0 Ht - t0 ExpB- p t 3 t t0 + t uf, 8t, 0, <, Assumptions Ø 8Re@t0 D > 0, Re@uD < Re@ ê t0d<f êê PowerExpand - t0 - u+ t0 The orresponding distribution is alled the inverse Gaussian; it has mean t 0 and variane t 0. ModuleB:M = ExpB J1-1 - t0 u ê NF, m>, m = D@M, ud ê. u Ø 0; 9m, HD@M, 8u, <D ê. u Ø 0 - m =F :t0, t0 > Its distribution funtion of umulative probabilities an be expressed in terms of the distribution funtion of the standard normal distribution (denoted F) as: F t ÿ t 0 ÿ Ht - t 0 + e ÿ F - t ÿ t 0 ÿ Ht + t 0. (9)

10 10 Jan Vrbik We verify this, getting an expression that agrees with (8). 1D, t t0 Ht - t0f + Exp@ D CDFBNormalDistribution@0, 1D, - tf êê Simplify t t0 Ht + t0f, - Ht-t0 t t0 t0 t t0 p t Applying the inverse Gaussian distribution to our previous example ( =, d = -0.35, and A = 100), we now display the distribution of time it takes the orresponding Brownian motion to reah zero for the first time. ModuleB8 =, d = -0.35, A = 100,, t0<, = -d A ê ; t0 = -A ê d; PlotB t0 Ht - t0 ExpB- F, 8t, 0, 600<FF p t 3 t t

11 Gambler s Ruin and First Passage Time 11 One last issue: to find the probability density funtion of T when p = 1, we need to take the limit of (8) as d Ø 0. imitb d A ê Ht + A ê d ExpB- F, d Ø 0F êê p t 3 t A ê d A PowerExpand A - A t p t 3ê This distribution has an infinite mean and variane. Conlusion The study of first passage times and their distributions is an important area of researh. The author hopes that this artile has provided a useful introdution to the topi. Referenes [1] W. Feller, An Introdution to Probability Theory and Its Appliations, Vol. I, 3rd ed., New York: Wiley, [] M. S. Bartlett, An Introdution to Stohasti Proesses, with Speial Referene to Methods and Appliations, nd ed., Cambridge: Cambridge University Press, J. Vrbik, Gamblerʼs Ruin and First Passage Time, The Mathematia Journal, 01. dx.doi.org/doi: /tmj.14-7 About the Author Jan Vrbik Department of Mathematis, Brok University 500 Glenridge Ave., St. Catharines Ontario, Canada, S 3A1 jvrbik@broku.a