Radom Models Tusheg Zhag February 14, 013 1 Radom Walks Let me describe the model. Radom walks are used to describe the motio of a movig particle (object). Suppose that a particle (object) moves alog the real lie. At each step it moves either oe uit to the right with probability p or moves oe uit to the left with probability q 1 p. The move of differet steps are idepedet of each other. Let S deote the positio of the particle after steps (at time ). The {S, 0} is called a simple radom walk. Mathematically we have the followig Defiitio 1.1 A simple radom walk startig from a is a sequece {S, 0} of radom variables ( called a stochastic process) such that S 0 a ad S S 1 + Y, 1,,... where Y 1, Y,...Y,... are idepedet, idetically distributed radom variables with P (Y 1) p, P (Y 1) 1 p q If p q 1, it is called a simple symmetric radom walk. Remark 1. 1. Y is the move of the particle durig the th step. If Y 1, the the particle moves upwards at the th step. If Y 1, the the particle moves dowwards at the th step.. S 1 S 0 +Y 1 a+y 1, S S 1 +Y a+y 1 +Y,..., S a+ i1 Y i... 3. We will write P a (A) for the probability of the evet A for radom walk startig at a at time 0. A radom walk ca also be used to model the fortue of a gambler. A gambler G plays the followig game at a casio. The crupier tosses a ( possibly biased) coi repeatedly; each time heads appears, he gives the gamler G oe poud, ad each time tails appears he takes oe poud from G. Writig S for G s fortue after tosses of the coi, S, 0 is a radom walk such that S S 1 + Y where P (Y 1) p, P (Y 1) 1 p q. What ca we do about the radom walk? We ca compute may iterestig probabilities cocerig the radom walk. 1
Propositio 1.3 Let I 1, I,..., I be idepedet, idetically distributed Beroulli radom variables with P (I r 1) p, P (I 1 0) q. Set N I 1 + I +... + I The N has a biomial distributio B(, p), i.e., ( ) P (X k) p k (1 p) k, k k 0,..., Proof. We kow that if X B(, p), the G X (s) (ps + q). Therefore, to prove N B(, p), it is eough to show that G N (s) (ps + q). Ideed, by the idepedece, G N (s) G I1 +I +...+I (s) G I1 (s) G I (s)...g I (s) This completes the proof. (ps + q) The followig propositio provides us the probability distributio of the radom walk. Propositio 1.4 Let S 0 0. For ay 0, we have S N, where N is a radom variable havig biomial distributio B(, p). Cosequetly, P (S r) P (N r) P (N + r ) ( ) +r p +r (1 p) +r if r + is eve, P (S r) 0 if r + is odd. Proof. Recall S r1 Y r. Write I r Y r + 1, r 1,,..., The I r is a Beroulli radom variable. P (I r 1) P (Y r 1) p, P (I r 0) P (Y r 1) 1 p I r, r 1,,... are idepedet. 1 (S + ) 1 (Y r + 1) r1 I r N Bi(, p) r1
So S + N. From ow o, we assume a 0 for most of the discussios. Otherwise, we have P a (S k) P (a + Y r k) P ( Y r k a) P 0 (S k a) r1 This meas that the probabilities ca be writte i terms of the probabilities for the case the radom walk starts from 0. The probabilities of evets ivolvig the positios of the r.w. at a fiite umber of times are give i the followig Propositio. r1 Propositio 1.5 For 1 1 3 <... < k, P (S 1 r 1, S r,..., S k r k ) P (S 1 r 1 )P (S 1 r r 1 )...P (S k k 1 r k r k 1 ) Proof. We give a proof for the case k. P (S 1 r 1, S r ) P (S 1 r 1, S S 1 r r 1 ) P (S 1 r 1, P (S 1 r 1 )P ( i 1 +1 i 1 +1 Y i r r 1 ) Y i r r 1 ) Sice i 1 +1 Y i ad S 1 1 i1 Y i have the same probability law, we coclude that P (S 1 r 1, S r ) P (S 1 r 1 )P (S 1 r r 1 ) Example. P (S 4, S 7 1) P (S 4 )P (S 7 4 1 ) ( ) ( ) 4 3 P (S 4 )P (S 3 3) p 3 (1 p) (1 p) 3 3 0 Spatial homogeeity ad time homogeeity Observe that P (S r S 1 r 1 ) P (S r, S 1 r 1 ) P (S 1 r 1 ) P (S 1 r 1 )P (S 1 r r 1 ) P (S 1 r 1 ) P (S 1 r r 1 ) 3
This meas that P (S r S 1 r 1 ) depeds oly o the distace of the positios ad differece of the times (steps). This is the so called spatial homogeeity ad time homogeeity. Markov property For ay positios r 1, r,...r k, r k+1 ad ay time poits 1 < <... < k < k+1, it holds that P (S k+1 r k+1 S 1 r 1, S r,..., S k r k ) P (S k+1 r k+1 S k r k ) Proof. Let A {S k+1 r k+1 }, B {S k r k } ad C {S 1 r 1, S r,..., S k 1 r k 1 }. So Left P (A B C) P (A B C) P (B C) P (S 1 r 1, S r,..., S k r k, S k+1 r k+1 ) P (S 1 r 1, S r,..., S k r k ) P (S 1 r 1 )...P (S k k 1 r k r k 1 )P (S k+1 k r k+1 r k ) P (S 1 r 1 )...P (S k k 1 r k r k 1 ) P (S k+1 k r k+1 r k ) P (S k+1 r k+1 S k r k ) What does the above fact tell us? If we thik of k as the curret time (preset), A is a future evet, B is a preset evet, ad C is a past evet, so the above fact is sayig that P (Future Past ad Preset) P (Future Preset) This meas that oce we kow the curret situatio of the process, the past is irrelevat to its future. The past has o ifluece o its future give its preset. This o-memory property is called the Markov Property. Geeral radom walk If i the defiitio of a simple radom walk, we allow the distributio of Y s to be ay distributio cocetrated o itegers, the at each step the radom walk ca jump from oe positio ito ay other positio. This is called a geeral radom walk. We ca compute may iterestig probabilities about the radom walk. First Retur Probabilities We will fid the probability that the r.w. returs to its startig positio (poit) for the first time at time. There is o loss of geerality i assumig 4
that the startig positio a 0 because of the spatial homogeeity. Write for 1, f P (S 1 0, S 0,..., S 1 0, S 0), which is the probability that the r.w. returs to its startig positio (poit) for the first time at time. Defie f 0 0 sice we do t wat to cout time zero as a retur. We are goig to compute f by usig geeratig fuctios. Defie u P (S 0), 0. The u 0 1 ad u is the probability that the r.w. returs to its startig positio (poit) at time, but might ot be the first time the r.w. returs to 0. By Propositio 3.4, we have u 0 if is odd, ad u ( The relatio betwee f k ad u k is give by ) p (1 p), 1,,... Propositio 1.6 The First Retur Idetity: For 1, we have u f r u r r0 f 1 u 1 + f u +... + f 1 u 1 + f Proof. Splittig the evet A {S 0} accordig to the time at which the first retur to 0 occurs, we get where A r1(a B r ) B r {S 1 0, S 0,..., S r 1 0, S r 0} is the evet that the first retur to 0 occurs at the r th step. Sice the evets A B r, r 1,,..., are disjoit, we have u P (A) P (A B r ) r1 Now, P (B r ) f r ad by Markov property P (B r )P (A B r ) P (A B r ) P (S 0 S 1 0, S 0,..., S r 1 0, S r 0) P (S 0 S r 0) r1 P (S r 0) u r by the spatial homogeeity. Therefore we obtai u f r u r r0 5
Cosider geeratig fuctios of (f, 0) ad {u, 0}, F (s) U(s) f s 0 f s 1 u s 1 + 0 u s Let us ow fid the relatio betwee F (s) ad U(s). Multiply the above fuctios together to get where 1 F (s)u(s) ( f s )( u s ) k0 k0 0 0 f k s k u l s l l0 f k u l s k+l l0 c l+k ( f k u l )s 0 c s, 0 f k u l l+k f r u r By the First Retur Idetity, c u whe 1. Note that c 0 f 0 u 0 0. Therefore, which gives F (s)u(s) c 0 + c s 1 r0 u s U(s) 1, 1 F (s) 1 1 U(s) Before itroducig ext theorem, we recall some otatios. For ay real umber a, defie ( ) a a(a 1)(a )...(a + 1), 1,,...! ( ) a 1 0 With this defiitio, the followig expasio holds ( ) a (1 + x) a x, 1 x < 1 0 6
Oe ca check that ( 1) ( 1 ) ( ) Above formula ca be easily verified for 3. Theorem 1.7 For a simple radom walk, we have F (s) 1 1 4pqs where f 0 if is odd, ad f s, 0 f k ( k k ) p k (1 p) k, k 1,,... k 1 Proof of Theorem. From earlier discussio, we kow that ( ) u P (S 0) p (1 p), u +1 0. It follows that Hece, 0 ( 0 U(s) u s 0 ) p (1 p) s ( 1 u s 0 ( 0 ) (4s p(1 p)) ) ( 4s p(1 p)) (1 4pqs ) 1 F (s) 1 1 U(s) 1 (1 4pqs ) 1 Usig the biomial expasio of (1 + x) 1 F (s) 1 1 0 ( 1 with x 4pqs, we see that ) ( 4s pq) ( 1) +1 ( 1 ) s p q Agai, usig the defiitio of the biomial coefficiet, oe ca check that ( 1 ) ( ) ( 1) +1 1 1 7
So it follows that F (s) 1 ( ) 1 1 p q s Matchig the coefficiets, we obtai that f +1 0 ad f ( ) 1 1 p q, 1 Example. Let p q 1. The ( ) 4 1 f 4 4 1 (1 )4 ( 1 )3, while u 4 P (S 4 0) 3( 1 )3 The recurrece probability. Defie the recurrece probability as the probability that the r.w. ever returs to the startig positio. So f recurrece probability {the r.w. ever returs to 0} {S k 0, for some k 1} 1{a first retur to zero occurs at } P ({a first retur to zero occurs at }) 1 Corollary 1.8 For a simple radom walk we have f 1 p q. Thus, f 1 if p q 1, whereas f < 1 if p q. Proof. Recall F (s) f s 1 1 4pqs 1 So f F (1) 1 1 4pq. But Hece, f 1 p q. 1 4pq (p + q) 4pq (p q). 8 1 f
Remark 1.9 The above discussio shows that if p q 1, the it is certai that the r.w. will retur to the startig positio. I this case, we say that the r.w. is recurret. If p q, with the positive probability 1 f the r.w. ever returs to the startig positio. We say that the r.w. is trasiet. Total umber of returs. Let R deote the total umber of returs. R k meas that the r.w. returs to its startig positio exactly k times. R is the case the r.w. returs to its startig positio ifiitely ofte. Agai we assume S 0 0. Propositio 1.10 1.If p q the P (R k) (1 f )(f ) k ad E(R) f 1 f.. If p q 1, the P (R ) 1. Proof. We first prove P (R m) (f ) m, m 1, by iductio. For m 1, we have P (R 1) P (r.w. ever returs to 0) f Assume the claim holds for m. We prove it for m + 1. Split the evet {R m + 1} accordig to the time at which a first retur occurs to get P (R m + 1) P (S 1 0,..., S 1 0, S 0, R m + 1) 1 P (S 1 0,..., S 1 0, S 0, at least m returs after time ) 1 By the Markov property, P (S 1 0,..., S 1 0, S 0)P (at least m returs after time S 0) 1 By the time homogeeity, f P (R m) (f ) m 1 1 The proof of the claim is complete. Cosequetly, f (f ) m+1 P (R k) P (R k) P (R k + 1) (f ) k (f ) k+1 (1 f )(f ) k Sice R is a radom variable takig o-egative iteger-values, we have E(R) P (R > k) k0 9 P (R k + 1) k0
(f ) k+1 f 1 f k0 Method : Calculate the geeratig fuctio G R (s) P (R 0) + P (R k)s k k1 P (R 0) + (1 f 1 )[ 1 f s 1] E[R] G (1) f 1 f If p q 1, the f 1 ad P (R m) 1 for all m 1. Let m to get P (R + ) 1. I this case, we say that the r.w. is recurret because it keeps returig to its startig positio. Whe p q, we say that the r.w. is trasiet because the r.w. will ot returs to the startig positio ay more after a certai time. Probabilities of visitig particular states (positios) Let gk deote the probability that the r.w.(startig from 0) ever visits the positio k. Our aim is to fid a explicit expressio for gk. Theorem 1.11 1. If p q 1, the g k 1 for all k.. If p > q, the gk 1 for all k 1, ad g k ( q p ) k for all k 1. 3. If p < q, the gk 1 for all k 1, ad g k ( p q )k for all k 1. Proof. We prove 3. Other cases are similar. If p < q, the the probability that the r.w. moves to the left is greater tha the probability that the r.w. moves to the right. Ituitively, the r.w. evetually moves to the left. The strog law of large umbers cofirms this. By the law of large umbers, S Y 1 + Y +... + Y where µ E(Y 1 ) p q < 0. Thus we coclude S µ µ, This meas that the r.w. visits every egative state k 1 sice it caot get to without doig so. To obtai the formula for gk, we first claim that gk (g1) k, k 1 Deote by g a,a+1 the probability that the r.w. startig from a ever visits a + 1. The g a,a+1 g 1 by the spatial homogeeity. To reach k the r.w. 10
must visit 1, ad the startig from 1 it must visits,..., ad fially startig from k 1 it has visit k. By the Markov property, we have To fid g 1, itroduce ad write This gives that g k g 0,1g 1,...g k,k+1 (g 1) k B {r.w. ever visits 1} B (B {S 1 1}) (B {S 1 1}) g 1 P (B) P (S 1 1)P (B S 1 1) + P (S 1 1)P (B S 1 1) p 1 + q g 1,1 p + g q p + (g 1) q I other words, g 1 is the root of the equatio: x p + qx Notice that the above equatio has two roots: x 1 1 ad x p. If p < q, q we must have g1 p. Otherwise, if it were true that q g 1 1, the f P (the r.w. ever returs to 0) g 0, 1g 1,0 1 g 1 1 which cotradicts the fact that f < 1. So summarizig what we got we arrive at g k (g 1) k ( p q )k for k 1. Example 1.1 A gambler who has iitial capital a uits bets over ad over agaist a ifiitely rich oppoet o a game i which he wis 1 uit with probability p ad loses 1 uit with probability q. 1. For what values of p is he certai to become bakrupt?. If a 100, for what values of p is the probability that he ever becomes bakrupt at least 1? 3. If p, for what values of a is the probability that he ever becomes 3 bakrupt at least 15? 16 Solutio. Let S be the fortue of the gambler after th bet. {S, 1} forms a r.w. with S 0 a. The followig is obviously true: {The gambler becomes bakrupt} (S 0 for some 1) 11
By the spatial homogeeity, P a {The gambler becomes bakrupt} P a (S 0 for some 1) P 0 (S a for some 1) g a 1. If p q 1 or p < q(p < 1 ), by Theorem 3.11 g a 1, i.e., he is certai to become bakrupt.. The probability that the gambler ever becomes bakrupt is 1 g a 1 ( q p )a I order that there is a positive probability that the gambler ever becomes bakrupt, accordig to the theorem, oe must require that p > q. Assume a 100. the 1 ( q p )100 1 if ad oly if ( q p )100 1. This happes if ad oly if ( 1/100 (1 p)) 100 p 100. This is equivalet to ( 1/100 (1 p)) p, which gives 3. Assume p 3 so that p 1/100 1 + 1/100. 1 g a 1 ( q p )a 1 ( 1 )a So the probability that he ever becomes bakrupt at least 15 16 to 1 ( 1 )a 15 16 so that is equivalet ( 1 )a 1 16 which yields a 4. The gambler s rui problem Assume that a gambler s iitial capital is k uits. He makes a simple bet, over ad over agai, ad o each bet he has probability p of wiig 1 uit from the bak, ad probability q 1 p of losig 1 uit to the bak. Assume that the bak s iitial capital is m uits. So the total capital is l k + m. The gambler cotiues bettig till (i) he is ruied (he loses all his moey); or (ii) the bak is ruied (the gambler s capital icreases to l uits ). Problem: Fid the probability that the gambler ruis the bak. Let S deote the fortue of the gambler after the th bet. As we kow, {S } forms a radom walk with S 0 k. The gambler is ruied is equivalet to that the r.w. S hits 0. The bak is ruied is the same as the r.w. hits the total capital l. The evet that the gambler ruis the bak is 1
the same as the evet that the r.w. hits l before hittig 0. So the problem is to fid r k P k (s hits l before hittig o) Theorem 1.13 Suppose 0 k l. (i). If p q 1, the r k k l. (ii). if p q, the Proof. Let r k 1 ( q p )k 1 ( q p )l A { the r.w. hits l before hittig 0} Clearly r 0 0 ad r l 1. By cosiderig the result of the first step, we get r k P k (A {S 1 k + 1}) + P k (A {S 1 k 1}) P k (S 1 k+1)p k (A S 1 k+1)+p k (S 1 k 1)P k (A S 1 k 1) pr k+1 +qr k 1 This is a differece equatio which we will solve. Write a q. The above p equatio is (p + q)r k pr k+1 + qr k 1 or p(r k+1 r k ) q(r k r k 1 ) Namely, (r k+1 r k ) a(r k r k 1 ). Particularly, let k 1 to get r r 1 a(r 1 r 0 ) ar 1 Thus, Lettig k we obtai r (1 + a)r 1 r 3 r a(r r 1 ) a r 1 So r 3 r + a r 1 (1 + a + a )r 1 Repeatig this argumet, we arrive at r k (1 + a + a +... + a k 1 )r 1 Cosequetly, r k 1 ak r 1 a 1 if a 1, ad r k kr 1 if a 1. Usig r l 1 we get that r 1 1 a if a 1,ad r 1 a l 1 1 if a 1. We completes the proof of l the theorem by substitutig these ito the above formula. 13
Corollary 1.14 Let m 0 ad k 0. We have (1) P 0 ( the r.w. hits m before hittig k) { k if p q, m+k 1 ( q p )k if p q. 1 ( q p )m+k () Proof. (1) follows from () follows from P k ( the r.w. hits 0 before hittig l) { l k if p q, l 1 ( p q )l k if p q. 1 ( p q )l P 0 ( the r.w. hitsmbefore hittig k) P k ( the r.w. hitsm + kbefore hittig 0) P k ( the r.w. hits0before hittig l) 1 P k ( the r.w. hitslbefore hittig 0) Example 1.15 Suppose 0 < m < k < l. Fid (i) P k ( the r.w. hits l before hittig 0). (ii) P k ( the r.w. hits m before hittig l). Solutio. (i) P k ( the r.w. hits l before hittig 0) (ii) P l k ( the r.w. hits 0 before hittig l) P k ( the r.w. hits m before hittig l) P l k ( the r.w. hits l m before hittig 0) Example 1.16 A gambler makes repeated bets of 1 uit of the capital o black o a roulette wheel. O each bet, he has probability 18 of wiig oe 37 uit ad probability 19 of losig oe uit. The bak s iitial capital is 10 37 uits, ad the gambler cotiues bettig till either he or the bak has lost all their moey. (i). What is the probability that the gambler ruis the bak if his iitial capital is k uits? (ii). For what values of k is the probability that the gambler ruis the bak at least α 0 1( 18 3 19 )10? 14
Solutio. The capital S of the gambler after the -th bet forms a radom walk. Take 1000 as 1 uit. Note that q 19. p 18 (i). Here l m + k 15. So r k 1 ( 19 18 )k 1 ( 19. 18 )10+k (ii). We eed to choose k such that r k α 0, that is, This is equivalet to Collectig terms we have Take l o both sides to get 1 ( 19 18 )k 1 ( 19 1 18 )10+k 3 (18 19 )10 ( 19 18 )k 1 (( 19 18 )10+k 1) 1 3 (18 19 )10 ( 19 18 )k 3 (1 1 3 (18 19 )10 ) Thus, k(l( 19 18 )) L(3 (1 1 3 (18 19 )10 )) k L( 3(1 1( 18 3 19 )10 )) (L( 19)). 18 Example 1.17 The probability of the thrower wiig i the dice game called craps is p 0.49. Suppose Player A is the thrower ad begis the game with 5 ad Player B, his oppoet, begis with 10. Assume that the bet is 1 per roud. What is the probability that Player A goes bakrupt before Player B? What is the probability that Player A ruis Player B? Solutio. Let S deote the fortue of Player A after the -th bet. {S } is a radom walk with S 0 5. (1). Here k 5, l 10 + 5 15. Let T deote the probability that Player A goes bakrupt before Player B. The T is the probability the r.w. hits 0 before hittig l. So (). T 1 ( q p )l k 1 ( q p )l r 5 15