MATH 529 The Boudary Problem The drukard s walk (or boudary problem) is oe of the most famous problems i the theory of radom walks. Oe versio of the problem is described as follows: Suppose a particle begis at a positive iteger height o the y -axis. O each uit step time iterval, the particle moves up 1 uit with positive probability p, dow 1 uit with positive probability q, or remais at the same height with probability r = 1 p q, where r < 1. We shall call this process a simple radom walk. The height after k steps ca be writte as h k = + X 1 +... + X k, where X i =1 Up 1 Dow are i.i.d. (a) For m < <, what is the probability that a particle ever reaches oe of the boudary heights of or m? (b) What is the probability that a particle reaches the upper boudary before the lower boudary m? (c) What is the average umber of steps eeded to reach either boudary of or m? After we solve these problems, other results ca be obtaied by takig limits as or as m, ad also by geeralizig to processes that ump more tha 1 uit at a time. 1. The Probability of Hittig a Boudary We iitially assume that the particle begis at a iteger height, where 0. The by vertical traslatio, we ca obtai results for m. We first aim to show that a radom particle will reach a boudary of 0 or with probability 1. But this result does ot mea that every sigle path reaches a boudary. Ideed, a path that starts at height 5 ad goes up, dow, up, dow, up, dow, ad ifiitum, will ever reach a boudary of 0 or 10, but istead will always oscillate betwee heights 5 ad 6. However, the probability of this sigle path occurrig is (p q) = 0. I fact, eve =1 though there are a ucoutable umber of paths that always remai iside the boudaries, the probability of this collectio of paths is equal to 0. But rather tha show this result directly, we shall show that the probability of hittig a boudary equals 1 by usig the techique of differece equatios. We ow derive the probability of hittig a boudary. We let B 0 deote the set of paths that cause a simple radom walk to reach a boudary of 0 or whe startig at height, ad let e = P( B 0 ) for 0. The 0 e 1. Moreover, if a walk begis at height 0 or at height, the it automatically reaches a boudary without eve takig a step, thus we have the boudary coditios e 0 = 1 = e (1.1)
For 0 < <, ay walk that hits a boudary must either go up, go dow, or remai costat o the first step; thus, we ca apply the Law of Total Probability to obtai: P( B 0 ) = P X 1 =1 ( ) P B 0 ( X 1 =1) + P X 1 = 1 ( ) +P( X 1 = 0) P B 0 ω 1 = 0 = p P +1 B 0 ( ) P B 0 ( ) + q P ( 1B 0 ) + r P ( B 0 ). ( X 1 = 1) That is, for 0 < <, e = p e +1 + q e 1 + r e. (1.2) Because e = (p + q + r)e, we ca substitute i (1.2) to obtai (p + q + r)e = p e +1 + q e 1 + r e, or p (e e +1 ) = q (e 1 e ) (1.3) for 0 < <. (Coveietly, the terms ivolvig r have dropped out.) Note that if p = 0 ad q > 0, the e 1 = e for 0 < <. Thus, 1 = e 0 = e 1 = e 2 =... = e. We see that P( B 0 ) = 1 for all 0. Likewise, P( B 0 ) = 1 for all if q = 0 ad p > 0. So hereby we ca assume that p 0 ad q 0. Next we defie the forward differece f by f = e e +1 for 0 <. The Equatio (1.3) becomes p f = q f 1, or f = q p f 1 (1.4) But also f 1 = (q / p) f 2. By recursio, we ca write f i terms of f 0 as f = q f p 0 (1.5) We ow ca evaluate the sum of the forward differeces to solve for f 0 :
0 = e 0 e = (e 0 e 1 ) + (e 1 e 2 ) +...+(e 1 e ) 1 = f i = i=0 q f p 0 1 i= 0 f 0 = 1 (q / p) f 0 1 q / p i if p q (1.6) Hece, f 0 must equal 0. The by Equatio (1.5), 0 = f = e e +1, ad thus e = e +1, for 0 <. Thus, 1 = e 0 = e 1 = e 2 =... = e 1 = e. We therefore state: Theorem 1. Let 0. There is probability 1 that a simple radom walk begiig at height will reach a boudary of 0 or. A simple radom walk that begis at height ad eds at boudaries m ad is equivalet to a simple radom walk that begis at height m ad eds at boudaries 0 or m. That is, B m = m B 0 m for m. Applyig Theorem 1 with iitial height m ad upper boudary m, we ca state: Corollary 1. Let m. There is probability 1 that a simple radom walk begiig at height will reach a boudary of m or. 2. The Boudary Problem We ow kow that simple radom walks reach a boudary with probability 1. We ext wish to determie the probability P m of reachig the top boudary before the lower boudary m. So we ow let U m be the set of paths that cause a simple radom walk to reach height before height m whe startig at height ad discrete time 0. We ow derive the probability of hittig the upper boudary before the lower boudary. To derive the result for m = 0, we eed oly modify the boudary coditios of our differece equatios argumet. Here we let U 0 be the set of paths that cause a simple radom walk to reach height before height 0 whe startig at height, ad let e = P 0 = P( U 0 ). The 0 e 1. Now if a walk begis at height, the it reaches first with probability 1; but if it begis at height 0, the it reaches first with probability 0. Thus, e 0 = 0 ad e = 1. (2.1) Sice a walk that hits first, for 0 < <, must either go up, go dow, or remai costat o the first step, we may apply the Law of Total Probability to obtai:
( ) + P X 1 = 1 ( ) P( U 0 ) = P ( X 1 =1 ) P U 0 ω 1 =1 +P( X 1 = 0) P U 0 ω 1 = 0 ( ) + q P ( 1U 0 ) + r P ( U 0 ) = p P +1 U 0 = p e +1 + q e 1 + r e. The as with Equatio (1.3), we obtai, for 0 < <, ( ) ( ) P U 0 ω 1 = 1 p (e e +1 ) = q (e 1 e ), (2.2) Agai, if p = 0 ad q > 0, the e 1 = e for 0 < <. Thus, 0 = e 0 = e 1 = e 2 =... e 1 = 0. We see that P 0 = 0, for all 0 <, ad P 0 = e = 1. However if q = 0 ad p > 0, the e = e +1 for 0 < <. Thus, e 1 = e 2 =... = e 1 = e = 1. We see that P 0 = 1, for all 1, ad 0 P 0 = e0 = 0. I other words, if p = 0, the there is o chace of reachig the top boudary first uless we start at the top boudary. Ad if q = 0, the we are certai to reach the top boudary first uless we start at the bottom boudary. (Sice we still are assumig that r < 1, the p ad q caot both be 0.) Sice these trivial cases are resolved, we ow may assume that p 0 ad q 0. We agai let f = e e +1 be the forward differece for 0 <. Equatio (2.2) ad recursio, we have The by f = q f p 0. (2.3) Evaluatig the sum of these forward differeces we obtai: 1 = e 0 e = (e 0 e 1 )+ (e 1 e 2 )+...+(e 1 e ) 1 1 q = f i = i f 0 p i=0 i=0 f 0 = 1 (q / p) f 0 1 q / p if p q. (2.4)
Thus, f 0 = 1 1 q / p 1 (q / p) if p q. (2.5) Now by evaluatig the partial sum of the forward differeces, we obtai 1 e = (e 0 e ) = f i i=0 f 0 = 1 (q / p) f 0 1 q / p 1 q = i f 0 p i=0 / = 1 (q / p) if p q 1 (q / p) if p q. (2.6) Moreover, the simple radom walks that begis at height ad reach height before height m are equivalet to the simple radom walks that begis at height m ad reach height m before height 0. Thus, the probability of reachig the upper boudary first remais the same after subtractig m from each height. That is, P m = mp 0 m. Makig these substitutios ito Equatio (2.6), we ca state: Theorem 2. For simple radom walks begiig at height, with m, the probability of reachig height before height m is give by m m P m = 1 (q / p) m 1 (q / p) m if p q. The probability of reachig height m before height is give by Q m =1 P m = m (q / p) m (q / p) m 1 (q / p) m if p q. But if p = 0, the P m = 0 for m < ad P m = 1. A closed-form formula is ot kow for the geeral case whe the walks do ot move up ad dow with the same heights. For that case, we will use the matrix methods to fid umerical values for P m.
3. Average Number of Steps Because we kow that simple radom walks hit a boudary with probability 1, we also ca obtai the average of the umber of steps S m eeded to hit a boudary. Because ( ) = P m = 1, the average umber of steps the ca be expressed by the usual P S m < defiitio of the mea of a discrete radom variable: E S m = ( ) k P S m = k, k =k 0 where k 0 = Mi{ m, }. But rather tha compute this sum directly to fid the average, we istead ca use the differece equatios argumet with a variatio of the boudary coditios. Let S m be the umber of steps eeded for a simple radom walk to reach a boudary of height or m whe startig at height. I particular, let S 0 be the umber of steps eeded for a simple radom walk to reach either height or height 0 whe startig at height, with 0, ad let e = E S 0 [ ]. The 0 e, ad if a walk begis at height or at height 0, the it reaches a boudary i 0 steps; thus, e 0 = 0 = e. Now suppose that 0 < <. Assume p = 0 so that paths ever go upward. Sice we assume that r < 1, the q > 0 ad the paths evetually will drift dowward ad reach the lower boudary. Sice it will take dowward steps to reach 0, the actual umber of steps required to do so forms a egative biomial radom variable. I this case, the average umber of steps e equals the egative biomial average of /q. Similarly, if q = 0, the the average umber of steps eeded to reach the (top) boudary is e = ( ) / p. Heceforth, we may assume that either p or q equals 0. So ow for 0 < <, we ca express E[ S 0 ] as E[ S 0 ] = 1 + p e +1 + q e 1 + r e. (Oe step must be take, the we proceed from there.) Applyig a similar differece equatios argumet, we obtai E S 0 = ( ) 2p 1 (q / p) p q 1 (q / p) p q if p q. (3.1)
Now if a walk begis at height ad eds at boudaries of m or, with m, the we ca subtract m from each height to obtai a equivalet walk that begis at height m ad eds at boudaries of 0 or m. The umber of steps eeded to reach a upper boudary remais the same alog each path, thus so does the average umber m of steps. That is, E[ S m ] = E[ m S 0 ]. Makig this substitutio ito (3.1), we have Theorem 3. For simple radom walks begiig at height, with m, the average umber of steps eeded to reach a boudary of height or height m is give by E S m = ( )( m) 2p m p q 1 q / p 1 q / p ( ) m ( ) m m p q if p q. But if p = 0, the E[ S m ] = ( m) / q for m < ad E[ S m ] = 0. As with the probability of hittig the top boudary first, a geeral formula is ot kow for the average umber of steps i the case whe the walks do ot move up ad dow with the same heights. For that case, we agai will use matrix methods. Disclaimer: The differece equatios argumet to derive (3.1) assumes a priori that the average is fiite i order to subtract averages ad create the forward differeces. If we make this assumptio (see Addedum), the we ca also apply Wald s Idetity to obtai the expected value of T = S 0. The fial height is either 0 or. Thus, P 0 = E[hT ] = + E[X 1 +... + X T ] = + E[X 1 ] E[T ] = + (p q)e[t ]. (3.2) I this case of p = q, Wald s Idetity does ot help for (3.2) simply becomes =. But for p q, (3.1) becomes 1 (q / p) 1 (q / p) = P 0 = + (p q)e[t ], which gives E[T ] = p q 1 (q / p) 1 (q / p) p q.
4. Geeral Average Edig Height Usig the formula for P m, we easily ca fid the average height of the simple radom walk upo stoppig at a boudary of or m. Here we let F m deote the fial height upo stoppig. Sice F m is either or m, the average edig height is give by Simplifyig this expressio, we obtai: E[ F m ] = P m + m 1 ( P m ). Theorem 4. For simple radom walks begiig at height, with m, the average height upo stoppig at boudaries of or m is give by E F m = ( m)(q / p) m m(q / p) m 1 (q / p) m if p q. But if p = 0, the E[ F m ] = m for m <, ad E[ F m ] =. Although our formulas have bee derived for simple radom walks that go up or dow 1 uit at a time, they apply wheever the walks go up ad dow the same amout a o each step. Ideed, we eed oly divide by this value of a ad the apply the formulas. I this case, the average fial height will be a E F m [ ], where,, ad m are the scaled-dow iitial coditios. Example. You start with $5000 ad proceed to bet $50 o red at the roulette wheel util you reach $6000 or drop to $3000. (a) What is the probability of reachig $6000 before droppig to $3000? (b) What is the average umber of bets that will be made i this situatio? (c) What is the average amout of moey that remais i this situatio? Solutio. By dividig by $50, the problem is equivalet to startig with = $100 ad bettig $1 at a time util you reach = $120 or drop to m = $60. Usig p = 18/38 ad q = 20/38, we have 120 1 (q / p) 40 (a) 100 P 60 = 60 0.12, ad 1 (q / p) 120 (b) E[ 100 S 60 ] = 60 p q 1 (q / p)40 40 60 1 (q / p) p q 623.2.
So you have about a 12% chace of reachig the goal of $6000 before droppig to $3000. O the other had, you probably ca play for a log time sice the average umber of bets would be over 623. (c) The average edig fortue is 120 $6000 100 P 60 + $3000 120 100 Q 60 $3359.99. About 12% of the time we reach $6000 ad quit, ad about 88% of the time we drop to $3000 ad quit. O average, we are left with $3359.99. Addedum I order to use Wald s Idetity or differece equatios to derive the average time eeded to hit a boudary, we first eed to kow that the average time is fiite. So assume that q > p. Now igore costat steps ad cosider oly those paths that move up or dow with probabilities!p = p / (p + q) ad!q = q / (p + q). The!q >!p. I this case, we have previously see that the probability of hittig height 0 from height for the first time i 2i + (o-costat) steps is give by P( T! 2i + = 2i + ) =!p i!q i+. 2i + i Thus, for q > p, the average umber of (o-costat) steps eeded to drop to height 0 is E[!T ] = (2i + )P(T = 2i + ) =!q (2i + )! (!p!q) i. (1) i! (i + )! i=0 i=0 We ote that the maximum value of!p!q is 1/4 whe!p =!q =1/ 2, ad!p!q <1/ 4 whe q > p. Usig the Ratio Test, we ca show that the sum (1) is fiite as follows: (2i + 2 + )! (!p!q) i+1 (i +1)! (i +1+ )! i! (i + )! (2i + + 2)(2i + +1) = i (2i + )! (!p!q) (i +1)(i +1+ ) (!p!q) 4!p!q <1. i Now let N i ~ geo(p + q) be the umber of steps required to obtai the ith ocostat step. The total umber of steps required to drop to 0 is the N 1 +... + N!T ; thus, E[N 1 +... + N!T ] = E[N 1 ] E[! T ] = 1 p + q E[! T ] <. Thus, for q > p, the average umber of steps eeded to reach a height of 0 or must be less tha or equal to the average umber of steps eeded to reach height 0, which is fiite. Next suppose that p > q. By symmetry, the average umber of steps eeded to go up to height must be fiite. Thus, the average umber of steps eeded to reach a height of 0 or must also be fiite.
Exercises 1. Cosider a oe-dimesioal radom walk that begis at height ad goes up or dow oe uit at a time. You wish to reach = + 3 before droppig to height 0 with probability 0.95. (a) Derive the miimum height at which you ca begi. For this with p = q ad r = 0.2, what is the average umber of steps eeded to reach a boudary of 0 or = + 3, ad what is the average edig height? (b) Derive the miimum height at which you ca begi if p = 0.66 ad r = 0.12. For this, what is the average umber of steps eeded to reach a boudary ad what is the average edig height? (c) If p = 0.25 ad r = 0, fid the maximum possible probability of reachig = + 3 before droppig to 0 (regardless of iitial height ). (d) Regardless of startig height, what is the maximum possible average umber of steps eeded to reach 0 or = + 3 whe (i) p = q = 0.4 ad whe (ii) p = 0.66, r = 0.12? (i.e., i these cases, what happes to the average as?) 2. Complete the differece equatios argumet to derive the formula for E[ S 0 ] i the case of p = q. 3. Let!p = p / (p + q),!q = q / (p + q) ad let! S0 be the umber of o-costat steps required to hit a boudary. Use (3.1) with!p ad!q to show that E! S0 = (p + q)e S 0. 4. Cosider a reflective lower boudary for which paths must move upward 1 uit o the subsequet step after hittig the lower boudary of 0. For 0 < <, derive the average umber of steps eeded to reach a upper boudary of whe startig at height. Bous Problem: I the case of p = q, argue idepedetly that the average umber of steps eeded to hit a boudary must be fiite.