1 Gambler s Ruin Problem

Transcription

1 Coyright c 2017 by Karl Sigman 1 Gambler s Ruin Problem Let N 2 be an integer and let 1 i N 1. Consider a gambler who starts with an initial fortune of $i and then on each successive gamble either wins $1 or loses $1 indeendent of the ast with robabilities and q = 1 resectively. Let X n denote the total fortune after the n th gamble. The gambler s objective is to kee successively gambling so as to reach a total fortune of $N, without first getting ruined (running out of money). If the gambler succeeds, then the gambler is said to win the game. In any case, the gambler stos laying after winning or getting ruined, whichever haens first. In essence, we can view each successive gamble as an indeendent coin fli which lands heads (H) with robability and lands tails (T) with robability q = 1. Each time the coin is flied, the gambler wins 1 dollar if it lands H and loses 1 dollar if it lands T. As soon as total earnings hits N or 0, the gambler stos fliing; the game ends. While the game roceeds before stoing, {X n : n 0} forms a simle random walk X n = i n, n 1, X 0 = i, where { n } forms an indeendent and identically distributed (i.i.d.) sequence of r.v.s. distributed as P ( = 1) =, P ( = 1) = q = 1, and reresents the earnings on the successive gambles. E( ) = q = 2 1. If > 1/2, equivalently > q, then E( ) > 0; while if < 1/2, equivalently < q, then E( ) < 0. Finally, if = 1/2, equivalently = q, then E( ) = 0. Thus, when > 1/2, the gambling is (on average) in favor of the gambler, when < 1/2, the gambling is (on average) in favor of the casino. Only when = 1/2 is the game fair. Recursively, this can be re-written as X n+1 = X n + n+1, n 0. Since the game stos when either X n = 0 or X n = N, let τ i = min{n 0 : X n {0, N} X 0 = i}, denote the time at which the game stos when X 0 = i; that is the time until either 0 or N are hit (for the first time). If X τi = N, then the gambler wins, if X τi = 0, then the gambler is ruined. Let P i (N) = P (X τi = N) denote the robability that the gambler wins when X 0 = i. P i (N) denotes the robability that the gambler, starting initially with $i, reaches a total fortune of N before ruin; 1 P i (N) is thus the corresonding robably of ruin Clearly P 0 (N) = 0 and P N (N) = 1 by definition, and we next roceed to comute P i (N), 1 i N 1. Proosition 1.1 (Gambler s Ruin Problem Solution) 1 ( q )i P i (N) = 1 ( q, if q; )N (1) i N, if = q =

2 Proof : For our derivation, we let P i = P i (N), that is, we suress the deendence on N for ease of notation. The key idea is to condition on the outcome of the first gamble, 1 = 1 or 1 = 1, yielding P i = P i+1 + qp i 1. (2) The derivation of this recursion is as follows: If 1 = 1, then the gambler s total fortune increases to X 1 = i + 1 and what haens after this is indeendent of the ast; the gambler will now win with robability P i+1. Similarly, if 1 = 1, then the gambler s fortune decreases to X 1 = i 1 and so the gambler will now win with robability P i 1. The robabilities corresonding to the two outcomes are and q yielding (2). Since + q = 1, (2) can be re-written as P i + qp i = P i+1 + qp i 1, yielding P i+1 P i = q (P i P i 1 ). In articular, P 2 P 1 = (q/)(p 1 P 0 ) = (q/)p 1 (since P 0 = 0), so that P 3 P 2 = (q/)(p 2 P 1 ) = (q/) 2 P 1, and more generally Thus P i+1 P i = ( q )i P 1, 0 < i < N. P i+1 P 1 = = i (P k+1 P k ) k=1 i k=1 ( q )k P 1, yielding i P i+1 = P 1 + P 1 ( q i )k = P 1 ( q )k = k=1 k=0 1 ( q P )i ( q ), if q; P 1 (i + 1), if = q = 0.5. (3) (Here we are using the geometric series equation i n=0 a n = 1 ai+1 1 a, for any number a 1 and any integer i 1, and of course i n=0 1 n = 1 + i.) Choosing i = N 1 and using the fact that P N = 1 yields 1 ( q 1 = P N = P )N 1 1 ( q, if q; ) P 1 N, if = q = 0.5, from which we conclude that P 1 = 1 q 1 ( q )N, if q; 1 N, if = q = 0.5, 2

3 thus obtaining from (3) (after algebra) the solution 1 ( q )i P i = 1 ( q, if q; )N (4) i N, if = q = 0.5. Remark 1.1 {X n : n 0} yields a Markov chain (MC) on the state sace S = {0, 1,..., N}. def The transition robabilities, P i,j = P (X n+1 = j X n = i), are given by P i,i+1 =, P i,i 1 = q, 0 < i < N, and both 0 and N are absorbing states, P 00 = P NN = 1. 1 For examle, when N = 4 the transition matrix P = (P i,j ) is given by q P = 0 q q Becoming infinitely rich or getting ruined In the formula (1), it is of interest to see what haens as N ; denote this by P i ( ) = lim N P i (N). This limiting quantity denotes the robability that the gambler, if allowed to lay forever unless ruined, will in fact never get ruined and instead will obtain an infinitely large fortune. Formally, this is true because of the basic result in robability that if {A n : n 1} is either an increasing or decreasing sequence of events, then lim n P (A n ) = P (lim n A n ). Increasing means that A n A n+1, n 1, in which case lim A n def = n A n, n=1 while decreasing means that A n A n+1, n 1 in which case lim A n def = A n, n In our alication, A N = {Gambler reaches N before 0 given that X 0 = i} is decreasing in N, and P i (N) = P (A N ), and n=1 lim A N = {Gambler reaches before 0 given that X 0 = i}. N Proosition 1.2 Define P i ( ) = lim N P i (N). If > 0.5, then If 0.50, then P i ( ) = 1 ( q )i > 0. (5) P i ( ) = 0. (6) Thus, unless the gambles are strictly better than fair ( > 0.5), ruin is certain. 1 There are three communication classes: C 1 = {0}, C 2 = {1,..., N 1}, C 3 = {N}. C 1 and C 3 are recurrent whereas C 2 is transient. 3

4 Proof : If > 0.5, then q < 1; hence in the denominator of (1), ( q )N 0 yielding the result. If < 0.50, then q > 1; hence in the denominator of (1), ( q )N yielding the result. Finally, if = 0.5, then i (N) = i/n 0. Examles 1. John starts with $2, and = 0.6: What is the robability that John obtains a fortune of N = 4 without going broke? i = 2, N = 4 and q = 1 = 0.4, so q/ = 2/3, and we want P 2 (4) = 1 (2/3)2 1 (2/3) 4 = What is the robability that John will become infinitely rich? P 2 ( ) = 1 (2/3) 2 = 5/9 = If John instead started with i = $1, what is the robability that he would go broke? The robability he becomes infinitely rich is P 1 ( ) = 1 (q/) = 1/3, so the robability of ruin is 1 P 1 ( ) = 2/ Alications Risk insurance business Consider an insurance comany that earns $1 er day (from interest), but on each day, indeendent of the ast, might suffer a claim against it for the amount $2 with robability q = 1. Whenever such a claim is suffered, $2 is removed from its reserve of money. Thus on the n th day, the net income for that day is exactly n as in the gamblers ruin roblem: 1 with robability, 1 with robability q. If the insurance comany starts off initially with a reserve of $i 1, then what is the robability it will eventually get ruined (run out of money)? The answer is given by (5) and (6): If > 0.5 then the robability is given by ( q )i > 0, whereas if 0.5 ruin will always ocurr. This makes intuitive sense because if > 0.5, then the average net income er day is E( ) = q > 0, whereas if 0.5, then the average net income er day is E( ) = q 0. So the comany can not exect to stay in business unless earning (on average) more than is taken away by claims. In later lectures, we will learn how to solve for the ruin robabilities for the much more general model in which the claims arrive according to a Poisson rocess, money comes in at a constant rate c and the claim sizes have a general distribution G: The classic Cramér-Lundberg Model. If the insurance business starts out with a reserve of x > 0, what is the robability it will get ruined? The answer turns out to be deely related to a queueing model in stationarity. 4

5 1.3 Random walk hitting robabilities Let R n = n, n 1, R 0 = 0 denote a simle random walk initially at the origin, the { n } are iid with P ( = 1) =, P ( = 1) = q = 1. Note that E( ) = q = 2 1 and σ 2 = V ar( ) = E( 2 ) E 2 ( ) = 1 (2 1) 2 = 4(1 ). Let a > 0 and b > 0 be integers and define (a) = P ({R n } hits level a before hitting level b). In words this is the robability that the random walk goes u by the amount a before going down by the amount b. By letting i = b, and N = a + b, we can equivalently imagine a gambler who starts with i = b and wishes to reach N = a+b before going broke (e.g. go u by the amount a before going down by the amount b ). So we can comute (a) by casting the roblem into the framework of the gambler s ruin roblem: (a) = P i (N) where N = a + b, i = b. Thus Examles 1 ( q )b (a) = 1 ( q, if q; )a+b (7) b a+b, if = q = Ellen bought a share of stock for $10, and it is believed that the stock rice moves (day by day) as a simle random walk with = What is the robability that Ellen s stock reaches the high value of $15 before the low value of $5? We want the robability that the stock goes u by 5 before going down by 5. This is equivalent to starting the random walk at 0 with a = 5 and b = 5, and comuting (a). (a) = 1 ( q )b 1 (0.82)5 1 ( q = = 0.73 )a+b 1 (0.82) What is the robability that Ellen will become infinitely rich (without ever hitting 0 first)? Here we equivalently want to know the robability that a gambler starting with i = 10 becomes infinitely rich before going broke. Just like Examle 2 on Page 3: 1 (q/) i = 1 (0.82) = Maximums and minimums of the simle random walk Formula (7) can immediately be used for comuting the robability that the simle random walk {R n }, starting initially at R 0 = 0, will ever hit level a, for any given ositive integer a 1: Kee a fixed while taking the limit as b in (7). The result deends on wether < 0.50 or A little thought reveals that we can equivalently state this roblem as comuting the tail P (M a), a 0, where M def = max{r n : n 0} is the all-time maximum of the 5

6 random walk; a non-negative random variable, because the corresonding two events are equal; {M a} = {R n = a, for some n 1}: On the one hand, if R n at some time n ever hits as high as a, it might hit even higher at a later time, hence M a. On the other hand, if M a, then R n must have hit a for some n. The increasing/decreasing sequence of events framework/results that we used in justifying Proosition 1.2 is used here as well: the events A b = {R n hits a before b } are increasing in b, and P (A b ) = (a) and thus lim b P (A b ) = P (lim b A b ) = {R n = a, for some n 1} = {M a}. Proosition 1.3 Let M def = max{r n : n 0} for the simle random walk starting initially at the origin (R 0 = 0). 1. When < 0.50, P (M a) = (/q) a, a 0; M has a geometric distribution with success robability 1 (/q): P (M = k) = (/q) k (1 (/q)), k 0. In this case, the random walk drifts down to, w1, but before doing so reaches the finite maximum M. 2. If 0.50, then P (M a) = 1, a 0: P (M = ) = 1; the random walk will, with robability 1, reach any ositive integer a no matter how large. Proof : Taking the limit in (7) as b yields the result by considering the two cases < 0.5 or 0.5: If < 0.5, then (q/) > 1 and so both (q/) b and (q/) a+b tend to as b. But before taking the limit, multily both numerator and denominator by (q/) b = (/q) b, yielding (a) = (/q)b 1 (/q) b (q/) a. Since (/q) b 0 as b, the result follows. If > 0.5, then (q/) < 1 and so both (q/) b and (q/) a+b tend to 0 as b yielding the limit in (7) as 1. If = 0.5, then (a) = b/(b + a) 1 as b. If < 0.5, then E( ) < 0, and if > 0.5, then E( ) > 0; so Proosition 1.3 is consistent with the fact that any random walk with E( ) < 0 (called the negative drift case) satisfies lim n R n =, w1, and any random walk with E( ) > 0 ( called the ositive drift case) satisfies lim n R n = +, w1. 2 But furthermore we learn that when < 0.5, although w1 the chain drifts off to, it first reaches a finite maximum M before doing so, and this rv M has a geometric distribution. Finally Proosition 1.3 also offers us a roof that when = 0.5, the symmetric case, the random walk will w1 hit any ositive value, P (M a) = 1. By symmetry, we also obtain analogous results for the minimum, : 2 R From the strong law of large numbers, lim n n n + if E( ) > 0. = E( ), w1, so R n ne( ) if E( ) < 0 and 6

7 Corollary 1.1 Let m def = min{r n : n 0} for the simle random walk starting initially at the origin (R 0 = 0). 1. If > 0.5, then P (m b) = (q/) b, b 0. In this case, the random walk drifts u to +, but before doing so dros down to a finite minimum m 0. Taking absolute values of m makes it non-negative and so we can exress this result as P ( m b) = (q/) b, b 0; m has a geometric distribution with success robability 1 (q/): P ( m = k) = (q/) k (1 (q/)), k If 0.50, then P (m b) = 1, b 0: P (m = ) = 1; the random walk will, with robability 1, reach any negative integer a no matter how small. Note that when < 0.5, P (M = 0) = 1 (/q) > 0. This is because it is ossible that the random walk will never enter the ositive axis before drifting off to ; with ositive robability it will remain forever 0: P (R n 0, n 0) = P (M = 0) = 1 (/q). Similarly, if > 0.5, then P (m = 0) = 1 (q/) > 0; with ositive robability the random walk will never enter the negative axis before drifting off to +. Recurrence of the simle symmetric random walk Combining the results for both M and m in the revious section when = 0.5, we have Proosition 1.4 The simle symmetric ( = 0.50) random walk, starting at the origin, will w1 eventually hit any integer a, ositive or negative. In fact it will hit any given integer a infinitely often, always returning yet again after leaving; it is a recurrent Markov chain. Proof : The first statement follows directly from Proosition 1.3 and Corollary 1.1; P (M = ) = 1 = P (m = ) = 1. For the second statement we argue as follows: Using the first statement, we know that the simle symmetric random walk starting at 0 will hit 1 eventually. But when it does, we can use the same result to conclude that it will go back and hit 0 eventually after that, because that is stochastically equivalent to starting at R 0 = 0 and waiting for the chain to hit 1, which also will haen eventually. But then yet again it must hit 1 again and so on (back and forth infinitely often), all by the same logic. We conclude that the chain will, over and over again, return to state 0 w1; it will do so infinitely often; 0 is a recurrent state for the simle symmetric random walk. Thus (since the chain is irreducible) all states are recurrent. Let τ = min{n 1 : R n = 0 R 0 = 0}, the so-called return time to state 0. We just argued that τ is a roer random variable, that is, P (τ < ) = 1. This means that if the chain starts in state 0, then, if we wait long enough, we will (w1) see it return to state 0. What we will rove later is that E(τ) = ; meaning that on average our wait is infinite. This imlies that the simle symmetric random walk forms a null recurrent Markov chain. 7