HW 2 Solutions. The variance of the random walk is explosive (lim n Var (X n ) = ).

Size: px

Start display at page:

Download "HW 2 Solutions. The variance of the random walk is explosive (lim n Var (X n ) = )."

Rosemary Scott
5 years ago
Views:

1 Stochastic Processews Prof Olivier Scaillet TA Adrien Treccani HW 2 Solutions Exercise. The process {X n, n } is a random walk starting at a (cf. definition in the course. [ n ] E [X n ] = a + E Z i = a + n E [Z i ] i= i= = a + ne [Z ] = ( a + n (p ( p = a + n (2p. n Var (X n = Var Z i = n Var (Z i, by independence ( i= i= = nvar (Z = n (p + ( p (2p 2 = 4np ( p. The variance of the random walk is explosive (lim n Var (X n =. 2. Let A = {X atteint b avant d atteindre }. Then P b (a = P(A Z = P(Z = + P(A Z = P(Z =, that is, P b (a = P b (a + p + P b (a q, ( by the total probability formula. This formula allows to compute P b (a. The boundary conditions are P b (b = and P b ( =. We make an educated guess that P b (a has the form P b (a = x a where x is to be determined. By substitution in (, px 2 x + q =, or x = ± 4pq 2p. We have the following cases: p = q = /2. In this case, x = is the only solution of the quadratic equation. A general solution of ( is given by P b (a = α + βa, where α and β are constants to be determined from the boundary conditions. We get α = and β = /b, and finally, P b (a = a/b. p /2. Let ω = ( 4pq. A general solution of ( is P b (a = a ( a +ω α 2p + β ω 2p. Using the boundary conditions, we get α = β, [ ( b ( ] b +ω α = 2p ω 2p, and finally [( a ( a ] [ ( b ( ] b + ω ω + ω ω P b (a =. 2p 2p 2p 2p 3. Let Y > and dµ Y (y the law of Y. If EY <, we have sp(y > s s y dµ Y (y,

2 as s. If Y has a density f, we have d dy P (Y > s = f(s and the integration by parts gives EY = sf(s ds = sp (Y > s + P (Y > s ds = P (Y > s ds. In the general case, we can apply Fubini s theorem. If Y takes its values in N, the expression for the expectation becomes EY = P(Y > k. 4. Let T be the first time b is reached. Then, n {T < n} = {a + Z = b} {a + Z + Z 2 = b}... {a + Z k = b}, which shows that {T n} = {T < n} c σ(z,..., Z n. 5. We have b a = X T a = n Z n {T n}. Suppose that ET is finite. Then, using Fubini s theorem, we can swap the expectation and the sum, and we get b a = E Z n {T n} = E ( 4. Z n {T n} = E(Z n P(T n = 3. EZ ET. n n n 6. If p /2, we have EZ = 2p. Also T > and b a >. If T is integrable, then we get a contradiction with 5. We see that the average time to reach le level b starting from a < b is infinite if p /2. Exercise 2. S n+ = S n +X n+. As X n+ is independent from the past of {S k, n k }, the process {S n, n } is a Markov chain. 2. Y n+ = X n+ + X n = X n+ + Y n X n n = X n+ + ( n k Y k + ( n X k= This shows that the whole past of the process is necessary to compute P (Y n+ = i Y n = j, Y n = y n,..., and that this probability is different from P (Y n+ = i Y n = j. Thus Y n is not a Markov chain. 3. Z n+ = Z n +S n+. Here, S n+ is not independent from the past of {Z k, n k }. We have: Z n+ = Z n + S n + X n+ = 2Z n Z n + X n+. The second lag (n thus gives some information on Z n+ and thus the process {Z n, n } is thus a Markov chain. Exercise 3 P is stochastic if all its entries are non-negative and if P. =, where = (,,...,. This implies that P h = P h P = P h =... = P =, and thus P h is also a stochastic matrix. 2

3 Exercise 4 Advice: Draw the transition diagrams corresponding to the following transition matrices.. For each of these three states, we have p ii (3 = and for all other times, which shows that each of these states has period For the state, we have p (2 + 2k > for all k >, and otherwise. Thus, state is periodic and has period 2. For the states 2, 3 and 4, p ii (2k > for all k >, and otherwise. Thus, these states are periodic with period 2. Exercise 5. We have that f ( = and f (k = for all k >. Thus f =, which shows that state is recurrent. 2. We have f 22 ( =.5 and otherwise, thus f 22 =.5 and state 2 is transitory. 3. The mean time of recurrence is m = k= kf (k =. 4. p ii (k > for k, i =, 2. The two states are thus aperiodic (d i =, i =, 2. Exercise 6. Trivial. 2. En remarquant que f ( = p, on a, f = p + n=2 = p + p ( p p n 2 p p k p. 3. If p <, we can show that state is recurrent: f = ( p + p p p =. Equivalently, f =. Exercise 7. P = q p q p 3

4 2. We aim to compute f and f 3. f (2k + = (pq k q for all k, and otherwise. Equivalently f 3 (2k = p 2 (pq k for all k >, and otherwise. Thus et f = f 3 = = f (2k + (pq k q = q (pq k = q pq, p 2 (pq k = p 2 k= (pq k = p 2 pq. 3. The mean time of ruin starting the game with one franc is: m = (2k + (pq k q ( ( = q 2 k (pq k + (pq k 2pq = q ( pq 2 + pq = q ( + pq ( pq 2. Reminder: For λ <, kλk = λ ( λ 2. Exercise 8 2. Drawing the graph corresponding to P (we label states from to 6, we observe that f = =, thus state is recurrent. As state and state 5 intercommunicate, state 5 is also recurrent. f 22 = + 2 ( k 2 2 = 4 = 2 <, thus state 2 is transitory. As 2 state 2 and state 6 intercommunicate, state 6 is also transitory. Finally, f 44 = = 5 9 <, thus state 4 is transitory. As state 4 and state 3 intercommunicate, state 3 is also transitory. 3. is a recurrent state. m = = 5 3. m 4 and m 6 are infinite as states 4 and 6 are transitory. Remark: It is possible to compute k= kf 44 (k and k= kf 66 (k. These sums are finite, but they do not stand for mean times of recurrence as states 4 and 6 are transitory. By convention, we define the mean time of recurrence of transitory states to be infinite. 4. We swap states 2 and 5, which gives the matrix P, used in the next question. 5. The vector π = [π π 2 ] is a stationary distribution of the Markov chain associated to P, if it satisfies the following conditions: π, π 2 y = π + π 2 = πp, = π The system has a unique solution that corresponds to the stationary distribution of the chain: π = [ ]. 4

5 6. We know that for a Markov chain with a unique stationary distribution, the mean time of recurrence m i of state i corresponds to the inverse of the corresponding element of π. Thus, m = 5 3 and m 2 =

MATH 564/STAT 555 Applied Stochastic Processes Homework 2, September 18, 2015 Due September 30, 2015

MATH 564/STAT 555 Applied Stochastic Processes Homework 2, September 18, 2015 Due September 30, 2015 ID NAME SCORE MATH 56/STAT 555 Applied Stochastic Processes Homework 2, September 8, 205 Due September 30, 205 The generating function of a sequence a n n 0 is defined as As : a ns n for all s 0 for which