Lecture 4: The Simple Radom Walk 1 of 9 Course: M36K Itro to Stochastic Processes Term: Fall 014 Istructor: Gorda Zitkovic Lecture 4 The Simple Radom Walk We have defied ad costructed a radom walk {X } N0 i the previous lecture. Our ext task is to study some of its mathematical properties. Let us give a defiitio of a slightly more geeral creature. Defiitio 4.1. A sequece {X } N0 of radom variables is called a simple radom walk (with parameter p (0, 1)) if 1. X 0 = 0,. X +1 X is idepedet of (X 0, X 1,..., X ) for all N, ad 3. the radom variable X +1 X has the followig distributio ( ) 1 1 q p where, as usual, q = 1 p. If p = 1, the radom walk is called symmetric. The adjective simple comes from the fact that the size of each step is fixed (equal to 1) ad it is oly the directio that is radom 1. Propositio 4.. Let {X } N0 be a simple radom walk with parameter p. For 1 the distributio of the radom variable X is discrete with support {, +,...,, }, ad probabilities p l = P[X = l] give by ( ) p l = l+ p (+l)/ q ( l)/, (4.1) for l =, +,...,,. 1 Oe ca study more geeral radom walks where each step comes from a arbitrary prescribed probability distributio. Proof. X is composed of idepedet steps ξ k = X k X k 1, k = 1,...,, each of which goes either up or dow. I order to reach level l i those steps, the umber u of up-steps ad the umber d of dowsteps must satisfy u d = l (ad u + d = ). Therefore, u = +l ad d = l. The umber of ways we ca choose these u up-steps Last Updated: March 3, 016
Lecture 4: The Simple Radom Walk of 9 from the total of is ( +l ), which, with the fact the probability of ay trajectory with exactly u up-steps is p u q u, gives the probability (4.1) above. Equivaletly, we could have oticed that the radom variable +X has the biomial b(, p)-distributio. 4 The proof of Propositio 4. uses the simple idea already hited at i the previous lecture: view the radom walk as a radom trajectory i some space of trajectories, ad, compute the required probability by simply coutig the umber of trajectories i the subset (evet) you are iterested i, ad addig them all together, weighted by their probabilities. To prepare the groud for the future results, let C be the set of all possible trajectories: C = {(x 0, x 1,..., x ) : x 0 = 0, x k+1 x k = ±1, k 1}. You ca thik of the first steps of a radom walk simply as a probability distributio o the state-space C. Now we kow how to compute the probabilities related to the positio of the radom walk {X } N0 at a fixed future time. A mathematically more iterestig questio ca be posed about the maximum of the radom walk o {0, 1,..., }. More precisely, for N 0 we defie its ruig maximum process {M } N0 by M = max(x 0,..., X ), for N 0. A ice expressio for the pmf of M is available for the case of symmetric simple radom walks. 4 1 3 4 Propositio 4.3. Let {X } N0 be a symmetric simple radom walk. The M 0 = 0 ad for 1, the support of M is {0, 1,..., } ad its probability mass fuctio is give by p l = P[M = l] = P[X = l] + P[X = l + 1] ( ) = +l+1, for l = 0,...,. Figure 1: The superpositio of all trajectories i C for = 4 ad a particular oe - (0, 1, 0, 1, ) - i red. Proof. Let us first pick a level l {0, 1,..., } ad compute the auxilliary probability q l = P[M l] by coutig the umber of trajectories whose maximal level reached is at least l. Ideed, the symmetry assumptio esures that all trajectories are equally likely. More precisely, let A l C 0 () be give by A l = {(x 0, x 1,..., x ) C : max x k l} k=0,..., = {(x 0, x 1,..., x ) C : x k l, for at least oe k {0,..., }}. The P[M l] = 1 #A l, where #A deotes the umber of elemets i the set A. Whe l = 0, we clearly have P[M 0] = 1, sice X 0 = 0. Last Updated: March 3, 016
Lecture 4: The Simple Radom Walk 3 of 9 To cout the umber of elemets i A l, we use the followig clever observatio (kow as the reflectio priciple): Claim 4.4. For l N, we have #A l = #{(x 0, x 1,..., x ) : x > l} + #{(x 0, x 1,..., x ) : x = l}. Proof Claim 4.4. We start by defiig a bijective trasformatio which maps trajectories ito trajectories. For a trajectory (x 0, x 1,..., x ) A l, let k(l) = k(l, (x 0, x 1,..., x )) be the smallest value of the idex k such that x k l. I the stochastic-process-theory parlace, k(l) is the first hittig time of the set {l, l + 1,... }. We kow that k(l) is well-defied (sice we are oly applyig it to trajectories i A l ) ad that it takes values i the set {1,..., }. With k(l) at our disposal, let (y 0, y 1,..., y ) C be a trajectory obtaied from (x 0, x 1,..., x ) by the followig procedure: 1. do othig util you get to k(l): y 0 = x 0, y 1 = x 1,... y k(l) = x k(l).. use the flipped values for the coi-tosses from k(l) owards: 6 y k(l)+1 y k(l) = (x k(l)+1 x k(l) ), y k(l)+ y k(l)+1 = (x k(l)+ x k(l)+1 ),... 4 y y 1 = (x x 1 ). The picture o the right shows two trajectories: a blue oe ad its reflectio i red, with = 15, l = 4 ad k(l) = 8. Graphically, (y 0,..., y ) looks like (x 0,..., x ) util it hits the level l, ad the follows its reflectio aroud the level l so that y k l = l x k, for k k(l). If k(l) =, the (x 0, x 1,..., x ) = (y 0, y 1,..., y ). It is clear that (y 0, y 1,..., y ) is i C. Let us deote this trasformatio by 4 6 8 10 1 14 Figure : A trajectory (blue) ad its image (red) uder Φ. Φ : A l C, Φ(x 0, x 1,..., x ) = (y 0, y 1,..., y ) ad call it the reflectio map. The first importat property of the reflexio map is that it is its ow iverse: apply Φ to ay (y 0, y 1,..., y ) i A l, ad you will get the origial (x 0, x 1,..., x ). I other words Φ Φ = Id, i.e. Φ is a ivolutio. It follows immediately that Φ is a bijectio from A l oto A l. To get to the secod importat property of Φ, let us split the set A l ito three parts accordig to the value of x : 1. A > l = {(x 0, x 1,..., x ) A l : x > l}, Last Updated: March 3, 016
Lecture 4: The Simple Radom Walk 4 of 9. A = l = {(x 0, x 1,..., x ) A l : x = l}, ad 3. A < l = {(x 0, x 1,..., x ) A l : x < l}, So that Φ(A > l ) = A < l, Φ(A < l ) = A > l, ad Φ(A = l ) = A = l. We should ote that, i the defiitio of A > l ad A = l, the a priori stipulatio that (x 0, x 1,..., x ) A l is ucessary. Ideed, if x l, you must already be i A l. Therefore, by the bijectivity of Φ, we have ad so #A < l = #A > l = #{(x 0, x 1,..., x ) : x > l}, #A l = #{(x 0, x 1,..., x ) : x > l} + #{(x 0, x 1,..., x ) : x = l}, just as we claimed. Now that we kow that Claim 4.4 holds, we ca easily rewrite it as follows: P[M l] = P[X = l] + P[X = j] j>l = P[X = j] + P[X = j]. j>l j l Fially, we subtract P[M l + 1] from P[M l] to get the expressio for P[M = l]: P[M = l] = P[X = l + 1] + P[X = l]. It remais to ote that oly oe of the probabilities P[X = l + 1] ad P[X = l] is o-zero, the first oe if ad l have differet parity ad the secod oe otherwise. I either case the o-zero probability is give by ( ) +l+1. Let us use the reflectio priciple to solve a classical problem i combiatorics. Example 4.5 (The Ballot Problem). Suppose that two cadidates, Daisy ad Oscar, are ruig for office, ad N voters cast their ballots. Votes are couted by the same official, oe by oe, util all of them have bee processed. After each ballot is opeed, the official records the umber of votes each cadidate has received so far. At the ed, the official aouces that Daisy has wo by a margi of m > 0 votes, i.e., that Daisy got ( + m)/ votes ad Oscar the remaiig ( m)/ like i the old days. Last Updated: March 3, 016
Lecture 4: The Simple Radom Walk 5 of 9 votes. What is the probability that at o time durig the coutig has Oscar bee i the lead? We assume that the order i which the official couts the votes is completely idepedet of the actual votes, ad that each voter chooses Daisy with probability p (0, 1) ad Oscar with probability q = 1 p. For k, let X k be the umber of votes received by Daisy mius the umber of votes received by Oscar i the first k ballots. Whe the k + 1-st vote is couted, X k either icreases by 1 (if the vote was for Daisy), or decreases by 1 otherwise. The votes are idepedet of each other ad X 0 = 0, so X k, 0 k is (the begiig of) a simple radom walk. The probability of a up-step is p (0, 1), so this radom walk is ot ecessarily symmetric. The ballot problem ca ow be restated as follows: What is the probability that X k 0 for all k {0,..., }, give that X = m? The first step towards uderstadig the solutio is the realizatio that the exact value of p does ot matter. Ideed, we are iterested i the coditioal probability P[F G] = P[F G]/P[G], where F deotes the family of all trajectories that always stay o-egative ad G the family of those that reach m at time. Each trajectory i G has ( + m)/ upsteps ad ( m)/ dow-steps, so its probability weight is always equal to p (+m)/ q ( m)/. Therefore, P[F G] = P[F G] P[G] = #(F G) p(+m)/ q ( m)/ #(F G) = #G p (+m)/ q ( m)/ #G. (4.) We already kow how to cout the umber of paths i G - it is equal to ((+m)/) - so all that remais to be doe is to cout the umber of paths i G F. The paths i G F form a portio of all the paths i G which do t hit the level l = 1, so that #(G F) = #G #H, where H is the set of all paths which fiish at m, but cross (or, at least, touch) the level l = 1 i the process. Ca we use the reflectio priciple to fid #H? Yes, we do. I fact, you ca covice yourself that the reflectio of ay path i H aroud the level l = 1 after its first hittig time of that level poduces a path that starts at 0 ad eds at m. Coversely, the same procedure applied to such a path yields a path i H. The umber of paths from 0 to m is easy to cout - it is equal to ( ). Puttig everythig together, we get (+m)/+1 P[F G] = ( k ) ( k+1 ) ( k ) = k + 1 k + 1, where k = + m. The last equality follows from the defiitio of biomial coefficiets ( k ) =! k!( k)!. Last Updated: March 3, 016
Lecture 4: The Simple Radom Walk 6 of 9 The Ballot problem has a log history (goig back to at least 1887) ad has spurred a lot of research i combiatorics ad probability. I fact, people still write research papers o some of its geeralizatios. Whe posed outside the cotext of probability, it is ofte phrased as i how may ways ca the coutig be performed... (the differece beig oly i the ormalizig factor ( k ) appearig i (4.) above). A special case m = 0 seems to be eve more popular - the umber of -step paths from 0 to 0 ever goig below zero is called the Catala umber 3 ad equals to C = 1 ( ). (4.3) + 1 3 See Problem 4.6 for more iformatio about Catala umbers. PROBLEMS Problem 4.1. Let {X } N0 be a symmetric simple radom walk. Compute the followig 1. P[X = 0], N 0,. P[X = X ], N 0, 3. P[ X 1 X X 3 = ], 4. P[X 7 + X 1 = X 1 + X 16 ]. Solutio: Let ξ k = X k X k 1, k N. 1. P[X = 0] = ( ). 0, is odd,. P[X = X ] = P[ξ +1 + ξ + + + ξ = 0] = / ( ), is eve. 3. X 1 X X 3 = oly i the followig two cases: X 1 = 1, X =, X 3 = 1 or X 1 = 1, X =, X 3 = 1. The probability of each trajectory is 1 8, so P[ X 1X X 3 = ] = 1 4. 4. We write the evet {X 1 + X 1 = X 7 + X 16 } i terms of ξs: {X 7 + X 1 = X 1 + X 16 } = {ξ + ξ 3 + + ξ 7 = ξ 13 + ξ 14 + ξ 15 + ξ 16 }. All ξs are idepedet, have the same distributio ad ξ k has the same distributio as ξ k. Thus, P[ξ + ξ 3 + + ξ 7 = ξ 13 + ξ 14 + ξ 15 + ξ 16 ] / = P[ξ + ξ 3 + + ξ 7 ξ 13 ξ 14 ξ 15 ξ 16 = 0] ( ) 10 = P[ξ 1 + ξ + + ξ 10 = 0] = 10. 5 Last Updated: March 3, 016
Lecture 4: The Simple Radom Walk 7 of 9 Problem 4.. Let {X } N0 be a simple symmetric radom walk. Which of the followig processes are simple radom walks? 1. {X } N0?. {X } N0? 3. { X } N0? 4. {Y } N0, where Y = X 5+ X 5? How about the case p = 1? Solutio: 1. No - the distributio of X 1 has support {, } ad ot { 1, 1}.. No - X1 = 1, ad ot ±1 with equal probabilities. 3. Yes - check the defiitio. 4. Yes - check the defiitio. The aswers are the same if p = 1, but, i 3., X comes with probability 1 p. Problem 4.3. Let {X } N0 be a simple symmetric radom walk. Give N, what is the probability that X does ot visit 0 durig the time iterval 1,...,. Solutio: Let us deote the required probability by p, i.e., p = P[X 1 = 0, X = 0,..., X = 0]. For = 1, p 1 = 1, sice X 1 is either 1 or 1. For > 1, let ξ 1 be the first icremet ξ 1 = X 1 X 0 = X 1. If ξ 1 = 1, we eed to compute that probability that a radom walk of legth 1, startig at 1, does ot hit 0. This probability is, i tur, the same as the probability that a radom walk of legth 1, startig from 0, ever hits 1. By the symmetry of the icremets, the same reasoig works for the case ξ 1 = 1. Therefore, p = P[X 1 0, X 0,..., X 1 0] = P[M 1 = 0], where M = max{x 0,..., X }. By Propositio 4.3, this probability is give by ( ) 1 p = +1. / Last Updated: March 3, 016
Lecture 4: The Simple Radom Walk 8 of 9 Problem 4.4. (30pts) A fair coi is tossed repeatedly ad the record of the outcomes is kept. Tossig stops the momet the total umber of heads obtaied so far exceeds the total umber of tails by 3. For example, a possible sequece of tosses could look like HHTTTHHTHHTHH. What is the probability that the legth of such a sequece is at most 10? Solutio: Let X, N 0 be the umber of heads mius the umber of tails obtaied so far. The, {X } N0 is a simple symmetric radom walk, ad we stop tossig the coi whe X hits 3 for the first time. This will happe durig the first 10 tosses, if ad oly if M 10 3, where M deotes the (ruig) maximum of X. Accordig to the reflectio priciple, P[M 10 3] = P[X 10 3] + P[X 10 4] = (P[X 10 = 4] + P[X 10 = 6] + P[X 10 = 8] + P[X 10 = 10]) [( ) ( ) ( ) ( )] 10 10 10 10 = 9 [ + + + = 11 ]. 3 3 1 0 Problem 4.5. Let {X } N0 be a simple radom walk with P[X 1 = 1] = p (0, 1). Defie Y = 1 X k, for N. Compute E[Y ] ad Var[Y ], for N. Hit: You ca use the followig formulas: j = j=1 ( + 1), j ( + 1)( + 1) = 6 j=1 without proof. Solutio: Let us first represet Y i terms of the sequece {ξ } N : Y = 1 = 1 X k = 1 ( k + 1)ξ k ( ) ξ 1 + (ξ 1 + ξ ) + + (ξ 1 + ξ + + ξ ) Rememberig that E[ξ k ] = p q ad that Var[ξ k ] = 1 (p 1) = 4pq, we have E[Y ] = 1 = p q ( k + 1)E[ξ j ] = p q k = (p q) + 1. ( k + 1) Last Updated: March 3, 016
Lecture 4: The Simple Radom Walk 9 of 9 By the idepedece of {ξ } N we have Var[Y ] = 1 Var[( k + 1)ξ k ] = 1 k + 1) ( Var[ξ k ] = 1 k Var[ξ k ] = 4pq ( + 1)( + 1) 6 = ( + 1)( + 1) pq. 3 Problem 4.6 (Optioal). Let C deote the -th Catala umber (defied i (4.3)). 1. Use the reflectio priciple to show that C is the umber of paths (x 0,..., x ) C such that x k 0, for all k {0, 1,..., } ad x = 0.. Prove the Seger s recurrece formula C +1 = i=0 C ic i. Hit: Do t compute - just thik about paths. 3. Show that C is the umber of ways that the vertices of a regular go ca be paired so that the lie segmets joiig paired vertices do ot itersect. 4. Prove that C = ( ) ( ), + 1 both algebraically (usig the formula for the biomial coefficiet) ad combiatorially (by coutig). Last Updated: March 3, 016