A Detailed Look at a Discrete Randomw Walk with Spatially Dependent Moments and Its Continuum Limit

A Detailed Look at a Discrete Randomw Walk with Spatially Dependent Moments and Its Continuum Limit David Vener Department of Mathematics, MIT May 5, 3 Introduction In 8.366, we discussed the relationship between random walks and diffusion equations at length. To a large extent, we used the continuum diffusion limit to approximate the behavior of the discrete random walk because the mathematics of calculus is often easier than the combinatorics required to consider the exact distribution of the random process. Problem of Problem Set exemplifies this attitude towards the calculation; we considered a discrete random walk on a one-dimensional lattice with spatially dependent transition probabilities. In the continuum limit, which will be discussed in more detail, we found that the probability distribution to leading order satisfies the Fokker- Planck equation and then solved this equation. In this project, I will further develop this calculation, discuss approaches to the discrete problem, and compare the continuum approximation to results derived from the discrete problem. The Discrete Problem Let M >> be a large integer, and let X n be the position of a random walker on the integers after n steps with the following transition probabilities for all n : i M, j = i +, M i M i Prob {X n = j X n = i} = +, j = i, M i M M, otherwise Since the transition probabilities only depend on the current position of the walker, this random walk is said to satisfy the Markov property. Furthermore, since the walker can only reach M i M, the walk can be described by a finite Markov Chain. Let P ij be a M + by M + matrix defined by P ij = Prob {X = j X = i} i i = ν i,j + + ν i,j+ M M

D. Vener 8.366 Random Walks and Diffusion Spring 3 Course Project for M i, j M. Now since the Markov property holds, we can write Prob {X = j X = i} = M Prob {X = k X = i} Prob {X = j X = k} k= M M = P ik P kj k= M Similarly, we can use mathematical induction to prove that = P. 3 ij P n ij = Prob {X n = j X = i}. 4 Therefore, given an initial position, i, for the random walker, we can calculate the probability that the walker is at position j after n steps, simply by calculating the elements of the i th row of P n. One might consider the discrete problem finished now since given any finite number of steps and any initial position we have an algorithm for computing the exact probability that the walker is at any position. However, for large M which, in fact, is a necessary component of the continuum limit discussed next the calculation may be very difficult by hand and may require too much memory on a computer. Therefore, we may also wish to ask whether or not P n approaches a limit in some sense, and, if so, how quickly. We will return to this problem in Section 4. 3 The Continuum Approximation of the Discrete Problem 3. The Continuum Limit Now suppose that the discrete random walk described above occurs on a lattice with spacing a << and that the spacing between time steps is δ <<. Then we can associate for each M i M we associate the position x = ai and for each n a time t = nδ. Now define the random variable X t; a, δ, M such that X nδ; a, δ, M = X n. Rewriting the transition probabilities from the discrete walk in terms of X t and x, we have { Prob X n + δ; a, δ, M = x X nδ; a, δ, M = x [ ] x = ν x x a + x + ν x x + a. 5 Ma Ma So by defining X t; a, δ, M X t + δ; a, δ, M X t; a, δ, M, 6 we can now calculate the spatially dependent moments, m l x; a, δ, M, by m l x; a, δ, M < [ X t; a, δ, M] l X t; a, δ, M = x > al x a l x = + + Ma Ma { al = M x, l odd l 7 a, l even.

D. Vener 8.366 Random Walks and Diffusion Spring 3 Course Project 3 Now let s define D l x; a, δ, M m l x;a,,m l!, and let us choose a, δ, and M, so that D and D are both O as δ. For example, we could choose M and a δ to get D = x and D =. Given a sufficient choice, let D = πx and D = D. Then we have al l!m D x, l odd l x; a, δ, M = a l l!, l even, l / l! D l / πx δ l /, l odd = 8 l/ D l/ δ l/, l even. l! This means that if D and π are O quantities, then D l = O δ l+/, which is different from the standard case where D l is assumed to be O δ l /, for l 3. In the limit of δ, a, and M, holding π and D fixed, X t; a, δ, M approaches a random variable, X t, which, for each t, can take values x from a continuous set,. Now let χ x, t x, be the probability density function PDF of X t given X = x. In Problem of Problem Set, we found a partial differential equation for the dynamics of a more general PDF; adapting that equation to the case of moments with no explicit time dependence, we can write χ [ x, t x, = πx + δ ] πx χ + [ D δ ] π x + δπd χ t x [ ] x 3 δ 4 [ δ ] x 3 3 πdx χ x 4 3 D χ + O δ. 9 This equation combined with the initial condition that χ x, x, = ν x x, allows us to solve for χ x, t x, up to O δ. This is the continuous analog for calculating the matrix P n ij, where n = t/δ as discussed in Section. 3. A Regular Perturbation Series for ρ Let us non-dimensionalize the PDE 8 by supposing that the problem has a characteristic time T and a characteristic length L. Defining dimensionless variables t = t/t and x = x/l, Equation 9 becomes [ ] χ t x, [ δ ] DT δ π δπdt T t x, = πt x + πt x χ + x L x + L χ x [ ] [ ] 3 δ πdt 4 δ D T x χ 4 3 L 4 χ + O δ. x 3 3 L x Choosing πt = DT/L =, we take T = /π and L = D/π to give us Equation 9 in the following dimensionless form. { [ ] [ ]} χ δ x x, t t x, = [ χ + xχ χ x xχ] + x T x x { [ ] [ ]} δ 3 4 δ = xχ + χ + O. T 3 x 3 4 x 3 T

D. Vener 8.366 Random Walks and Diffusion Spring 3 Course Project 4 Recall that t = δ represents the number of steps taken by the random walker. Therefore, taking γ δ /T << represents looking at the position of the walker after many steps. In this limit, we can expand χ in a regular perturbation series by taking χ x, t x, = χ x, t x, + γχ x, t x, + O γ, 3 where χ x, t x, satisfies subject to the constraint χ t x, t x, = [ x xχ ] + x χ, 4 We then force χ x, t x, to satisfy χ t subject to the constraint χ x, x, = ν x x. 5 x, t x, = x [ xχ ] + x χ + δ { [ ] [ ]} T x xχ x x χ δ { 3 [ ] 4 [ ]} = T x 3 3 xχ + x 4 3 χ, 6 χ x, x, =. 7 χ represents the leading order behavior of the PDF and will now be calculated. We will calculate χ, the leading order correction, in Section 3.4. 3.3 The Leading Order Behavior This problem was solved in the solutions to Problem Set ; therefore we will not go into too much detail here. In order to solve Equation 4, we make the following change of variables: ρ = xe t ξ = t. With this change, Equation 4 becomes, Taking the Fourier transform in the ρ variable, gives which has the solution χ ξ χ = χ + e. 8 ξ ρ χˆ k, ξ = e ξ k χˆ k, ξ, 9 ξ χˆ k, ξ = C k exp ξ e ξ k,

D. Vener 8.366 Random Walks and Diffusion Spring 3 Course Project 5 for an arbitrary C k which must be chosen to satisfy the initial condition χ ρ, = ν ρ x /L. Taking the Fourier transform of Equation allows us to write χˆ k, = e ikx /L = C k exp k which, in turn, implies that, upon defining x = x /L, { } ik x ξ χˆ k, ξ = e ξ e exp k e. 3 Now that we have the Fourier transform of χ, we can invert it to calculate χ ρ, ξ itself. We have { } x ξ χ ρ, ξ = e ξ dk e ikξ e ik exp k e λ { } = e ξ dk exp k e ξ + ik ρ x λ } [ ] } xi ρ x dk ξ x = e exp e ξ λ exp e k i ρ e ξ } ρ x =. 4 λ e ξ exp e ξ Finally, if we wish to express χ, in terms of the original variables, we have upon the necessary re-normalization } π π x x e γt χ x, t x, = λd e γt exp. 5 D e γt As a brief aside, we note here that for any x, π { π } lim χ x, t x, = exp x, 6 t λd D which shows that the leading order term approaches a steady solution which is independent of the initial starting position. We will comment on this again in Section 4. 3.4 The First Correction to Leading Order Now we have solved Equation 4 for χ, we can plug this into Equation 6 to solve for χ. In terms of ρ and ξ as defined in Section 3.3, Equation 6 is { [ ] [ ] [ ] } χ e 4ξ 4 χ = χ + e ξ χ ρ + ξ 3 ρχ ρχ. 7 ξ ρ ρ ρ e χ e ξ ρ 3 3 3 ρ 4

D. Vener 8.366 Random Walks and Diffusion Spring 3 Course Project 6 Now, when taking the Fourier transform of this equation, recall that ξ ik, ρ i ξ. Therefore, the transform of Equation 7 gives χˆ = k e ξ χˆ χˆ k + k χˆ + k e ξ χˆ + k 3 e ξ χˆ + k 4 e 4ξ χˆ 8 ξ k k 3 k 3 To solve this equation, note that from Equation 3, we compute that { } = exp ξ + k e ξ + ik x ξ χˆ ξ ξ = + k e. 9 χˆ Therefore, by dividing Equation 8 by ˆχ, we recognize that it can be rewritten to read ˆχ = ξ ˆχ ˆχ k ˆχ k + k ˆχ k + k e ξ ˆχ + 3 k3 e ξ ˆχ k + 3 k4 e 4ξ ˆχ = k ik x + k4 ik 3 x k x + 6 k4 e 4ξ 3 k4 e ξ + 3 ik3 x e ξ. 3 We may now solve for ˆχ, finding 4ξ ξ ξ χˆ = χˆ k ξ ikˆ x ξ + k 4 ξ ik 3 xˆ ξ k x ˆξ + k 4 e k 4 e + ik 3 xˆ e + B k, 4 6 6 3 where B k must be chosen to satisfy Equation 7, i.e. the initial condition for χ. Since χ x, x, =, χˆ ρ, = 3 and χˆ k, ξ [ ] = χˆ ρ, ξ ξ k ik k 4 x ik 3 x k + x [ ] 4ξ ξ ξ χˆ k 4 e k 4 e + ik 3 x e. 33 6 3 3 Recall that, when taking the inverse Fourier transform, k m ξ m, so that i m m χ ξ χ ξ 4 χ 3 χ ξ χ χ ρ, ξ x, = ξ x x ξ x ρ 3 ρ + ρ ρ 4 ρ e4ξ 4 χ ξ ξ e 4 χ e 3 χ + 4 ρ 4 + 6 ρ 4 x 6 ρ 3 ξ χ ξ x x + ξ χ e 3 χ = + ξ ρ ρ x 6 ρ 3 eξ e 4ξ ξ 4 χ +. 34 6 4 ρ 4

D. Vener 8.366 Random Walks and Diffusion Spring 3 Course Project 7 However, this can be simplified by noting that } m ρ x m m χ ρ m = m ρ m λ e ξ exp e ξ ρ x = H m χ, 35 e ξ n/ e ξ where H m z is the m th order Hermite Polynomial. In terms of the Hermite Polynomials, we find that the first order correction is [ ] [ ] x x ξe ξ ξe ξ ρ x ρ x χ ρ, ξ = H χ e H ξ χ + e ξ e ξ e ξ [ ] ξe 3ξ e ξ ρ x + x H 3 χ e [ ξ 3/ 6 e ξ e ξ ] e ξ + e ξ ξe 4ξ ρ x + H 4 χ. 36 6 e ξ 4 e ξ e ξ e ξ 4 The Many-step Limit of the Discrete Process Recall that in Section 3.3, we found that π { π } lim χ x, t x, = exp x, 37 t λd D a independent of x. Since π = M and D =, we can rewrite Equation 37 in terms of the parameters of the original discrete problem. That is { } { } dx dx x lim Prob x X t < x + X = x = exp. 38 t a Mλ M a Recalling that x = aj, where j is the position on the lattice, we see even more explicitly that the leading order continuum approximation predicts that for the discrete process { } { } lim Prob j X n < j + = i = exp M j. 39 n X Mλ From this prediction we might expect the discrete process has a stationary limit which is also independent of the initial starting position. In this section we will see to what extent that is the case. First we introduce a theorem which will be used later. 4. Doeblin s Theorem Suppose Q = Q ij i,j K is probability transition matrix on a finite number of states. Further suppose that there exists an γ > such that Q ij e for all i and j. Then Doeblin s Theorem states that there exists a unique vector µ such that µq = µ, µ i for all i, and K i= µ i =. Furthermore, given an vector v with v i for all i, and K i= v i =, K Note that v i= v i. vp n µ γn n. 4

D. Vener 8.366 Random Walks and Diffusion Spring 3 Course Project 8 4. A Stationary State for the Discrete Process Let ξ be a by M + row vector with ξ i, for all i, and M i= M ξ i =. We shall interpret ξ to be the state of the system after some number of steps since we note that by choosing ξ i = Prob {X n = i} and allowing P as defined in Section to act on ξ by right multiplication, we have M ξp j = ξ i P ij i= M M = Prob {X n+ = j X n = i} Prob {X n = i} i= M = Prob {X n+ = j}. 4 That is, if ξ contains the probability that the random walker is at each of the points on the lattice after n steps, then ξp is a row vector that contains the probability that the random walker is at each of the points on the lattice after n + steps. If lim n Prob {X n = j}, then there must exist a state vector such that it satisfies all of the properties of the ξ above and = P. To compute, we first note that it is an eigenvector of P corresponding to eigenvalue λ =. Therefore, i i = i P ij ν ij = i ν i,j + + i ν i,j+ i ν ij M M M + j = j + M + + j j+ j. 4 M M This implies that in addition to the conditions that of must satisfy the following recurrence relation. M i= M i = and i for all i, the elements M + + j j+ = M j M + j j. 43 This recursion can be solved with generating functions; however, we will just verify that j = M is the solution. First note that M M +j M M M M = M +j M j M M + j M M + j j= M j= M = M + M =. 44

D. Vener 8.366 Random Walks and Diffusion Spring 3 Course Project 9 Also, M M M + j M M j M + j j = M M + j M + j M! M! = M M M + j! M j! M + j! M j! [ ] M! M = M M + j! M j! M + j [ ] M! M j = M M + j! M j! M + j = M! M M + j! M j! = M + j + j+. 45 4.3 Interpretation of the Stationary State In Section 4., we found that the discrete random walk introduced in Section has a steady-state solution which is consistent with analysis of the continuum equation. However, one may ask the question, Does P n ij tend to j for any initial position i? We shall see that the answer to this question is both no and yes. Strictly speaking, lim n P n ij does not exist. This is easily seen, since the walker must either take exactly one step either to the left of the right. Therefore, if the walker is on an even position after n steps, he is guaranteed to be on an odd position after n + steps. Similarly, if the walker is at an odd position after n steps, he will be on a even position after n + steps. Therefore, P n ij is either for all even n or all odd n. But since j is non zero for all j, j cannot be the desired limit. Thus, we have lost some information about the system in the continuum limit which cannot be retrieved. However, we will now show that the walker does reach the stationary state in the following weaker sense. We will prove that for all i and j [ ] lim P n ij + P n+ = j, 46 n i.e. that the average state is the stationary state. To do this, let us first consider the elements of P. We have from matrix multiplication ij P i + i i + = + ν i,j + + i ν ij i,j 4 M M M M i i + + i + ν i,j+ 47 4 M M Therefore, if, instead of considering the random walker taking one step at a time, we only look at his position after an even number of steps, he will always be on a position with the same parity as his original position. Thus we can separate the even and odd points in the lattice into two separate classes with the property that transitions generated from P stay within the same class, so that the classes can be considered separately. To this end, let E be the M M + by + matrix representing the transition prob- M abilities between the even states of P, and let O be the M by matrix representing i,j

D. Vener 8.366 Random Walks and Diffusion Spring 3 Course Project the transition probabilities between between the odd states of P. Since even a random walker who starts at the far end has a positive probability of reaching the other end after M total steps. Therefore, E M >, for all i, j. i,j >, O M i+,j+ Thus, by Doeblin s Theorem for finite state Markov chains, both lim n E n i,j and lim n O n i+,j+ exist and are independent of i, for all i and j. In fact, a similar calculation to the one used to calculate the j above verifies that M lim E n i,j = = M j, 48 n M + j and that M lim O n i+,j+ = = M j+. 49 M + j + n Now let s reconsider the original Markov chain on all of the M + points on the lattice. If i and j have the same parity, we have already argued that P k+ = for all k. From our ij consideration of E and O and the fact that exactly one of n and n + is even, we have shown that [ ] lim P n ij + P n+ = [ j] = j. 5 n i,j If, however, i and j have different parity, we have already shown that P k ij Furthermore, = for all k. M P k+ = P ii P k ij i = M i j i i = P k + + P k. 5 M i+,j M i,j This, since i ± and j must have the same parity, we have [ ] [ ] i i lim n P n ij + P n+ = lim P k i,j n + + P k M i+,j M i,j i i = lim P k + + lim P k 4 M k i+,j 4 M k i,j i i = j + + j = j. 5 4 M 4 M We have thus proved our claim that the average position of the random walker after n steps approaches a steady distribution that is independent of the starting position. 5 A Comparison of the Discrete and Continuous Solutions For the purpose of comparison, let us choose M =,δ =.5, and a =. In this case, we have π = = and D = a =, so that the non-dimensional equations hold. M

D. Vener 8.366 Random Walks and Diffusion Spring 3 Course Project x 4.5.5 error.5.5.5 5 5 5 5 x position Figure : Error in leading order term Let us consider the symmetric case where the random walker begins at x =. Then for n = 5, i.e. t = 5 the error at each x made by the continuous approximation of the timeaveeraged probabilities can be seen in Figure. This picture seems to match an even fourth order polynomial times a Gaussian, which is the long time behavior exhibited by the correction term derived in Section 3.4. In fact, the error made by the continuous approximation when we include the first two terms is shown in Figure. This error appears to be a higher order polynomial times a Gaussian, as might be expected from the calculations above. Furthermore, for all x within a central region the results are not qualitatively different for t 5. 6 Conclusions In this project, we have calculated a better continuous approximation to a discrete process by finding the leading order correction to the continuum limit. This correction appears to be very accurate within the central region. We have also demonstrated that the exact limiting distribution for the discrete process can be calculated given any initial position, and we then used that calculation to see that some information about the exact distribution is lost and cannot be recovered in the continuum limit. That is, even the most accurate continuous approximation cannot predict that the discrete process is periodic and is not a good approximation for a particular time. References [] S. Karlin and H. M. Taylor, A First Course in Stochastic Processes, nd ed., Academic Press, New York, NY 997. [] H. Risken, The Fokker-Planck Equation: Methods of Solution and Applications, nd ed., Springer Verlag, Berlin 996.

D. Vener 8.366 Random Walks and Diffusion Spring 3 Course Project 3 x 7 error 3 5 5 5 5 x position Figure : Error in first and second order terms