Zdzis law Brzeźniak and Tomasz Zastawniak

Transcription

1 Basic Stochastic Processes by Zdzis law Brzeźniak and Tomasz Zastawniak Springer-Verlag, London 1999 Corrections in the 2nd printing Version: 21 May 2005 Page and line numbers refer to the 2nd printing of the book. A list of corrections in the 1st printing is also available. Please see To make a comment or report further mistakes please contact the authors at bz506@york.ac.uk or tz506@york.ac.uk Your feedback will be greatly appreciated! viii insert at the end of Preface: As this book is going into its 3rd printing, we would like to thank our students and readers for their support and feedback. In particular, we wish to express our gratitude to Iaonnis Emmanouil of the University of Athens and to Brett T. Reynolds and Chris N. Reynolds of the University of Wales in Swansea for their extensive and meticulous lists of remarks and valuable suggestions, which helped us to improve the current version of Basic Stochastic Processes. We would greatly appreciate further feedback from our readers, who are invited to visit the Web Page for more information and to check the latest corrections in the book. Zdzis law Brzeźniak and Tomasz Zastawniak Kingston upon Hull, June Replace the text of Exercise 1.6 by: Show that if ξ has discrete distribution with values x 1, x 2,..., then F ξ is constant on each interval (s, t] not containing any of the x i s and has jumps of size P {ξ = x i } at each x i Replace Solution 1.6 by: If s < t are real numbers such that x i / (s, t] for any i, then F ξ (t) F ξ (s) = P {ξ t} P {ξ s} = P {ξ (s, t]} = 0, i.e. F ξ (s) = F ξ (t). Because F ξ is non-decreasing, this means that F ξ is constant on (s, t]. To show that F ξ has a jump of size P {ξ = x i } at each x i, we compute lim F ξ (t) lim F ξ (s) = lim P {ξ t} lim P {ξ s} t x i s x i t xi s x i = P {ξ x i } P {ξ < x i } = P {ξ = x i }. 1

2 21 11 B E(ξ) dp B ξ dp η = 1 2x 1 η(x) = 1 2x B = Ω A = Ω the sentence Indeed, if A is such a set, then A = {η 2A 1} σ(a). Indeed, if A is such a set, then A = {η B}, where B = {2x : x A [0, 1 2 ]} is a Borel set, so A σ(η) F (y) F : R R 37 2 F (y) F : R R ( ) 59 5 P {τ > 2Kn} P {τ > 2Kn} n= replace 2) by 1) replace the last 10 lines on page 72 by: for any a < b. Since the set of all pairs of rational numbers a < b is countable, the event { } A = lim U n[a, b] < (4.3) n a<b rational n=1 has probability 1. (The intersection of countably many events has probability 1 if each of these events has probability 1.) We claim that the sequence ξ n converges a.s. to a limit ξ. Consider the set B = {lim inf ξ n < lim sup ξ n } Ω n n on which the sequence ξ n fails to converge. Then for any ω B there are rational numbers a, b such that lim inf n ξ n (ω) < a < b < lim sup ξ n (ω), n implying that lim n U n [a, b](ω) =. This means that B and the event A in (4.3) are disjoint, so P (B) = 0, since P (A) = 1, which proves the claim the sentence Because this σ-field is generated by the family of sets F 1 F 2 it follows that ξ = 1 A a.s. The family G consisting of all sets B F such that B ξ dp = B 1 A dp is a σ-field containing F 1 F 2. As a result, G contains the σ-field F generated by the family F 1 F 2. By Lemma 2.1 it follows that ξ = 1 A a.s. 2

3 Solution 4.5 : In Example 3.4 it was verified that ξ n = E(ξ F n ) is a martingale. Let ε > 0. By Lemma 4.2 there is a δ > 0 such that P (A) < δ = ξ dp < ε. By Jensen s inequality ξ n E( ξ F n ) a.s., so E( ξ ) E( ξ n ) ξ n dp MP { ξ n > M}. If we take M > E( ξ )/δ, then { ξ n M} A P { ξ n > M} < δ. Since { ξ n > M} F n, it follows that ξ n dp E( ξ F n ) dp = { ξ n >M} { ξ n >M} { ξ n >M} proving that ξ n = E(ξ F n ) is a uniformly integrable sequence P (ξ n+1 = 0 ξ n = 1) = P (ξ n+1 = 0, ξ n = 1) P (ξ n = 1) P (ξ n+1 = 1 ξ n = 0) = P (ξ n+1 = 1, ξ n = 0) P (ξ n = 0) 93 5 P (η 1 = 0) = q = 1 p P (η 1 = 1) = q = 1 p 93 9 η 0 = 0 ξ 0 = η 0 = i ξ 0 = i 94 4 p n = P (ξ n = 0) p 0 (n) = P (ξ n = 0) 96 4 j i 0 j, i 0 ξ dp < ε, replace Propostion 5.6 by: The probability of survival in Exercise 5.12, part 2) equals 0 if λ 1, and 1 ˆr k if λ > 1, where k is the initial Vugiel population and ˆr (0, 1) is a solution to r = e (r 1)λ. (5.28) probability of extinction is certain probability of extinction is λ > 0 λ > 1 3

4 (n + 1)th (n + 1)st ± F jj (x) n f n(j j) = 1 F jj (x) n f n(j j) = 1 as x (5.73) (5.43) ε := p(j i)p(i j) ε := p m (j i)p n (i j) r,s S p m(i s)p k (s r)p n (r i) r,s S p n(i s)p k (s r)p m (r i) These lines the following: To prove 5) it is enough to show that d(i) d(j). Using the first inequality derived above, we have p n+k+m (i i) εp k (j j) for all k N. From this inequality we can draw two conclusions: (a) d(i) n + m, since by taking k = 0 we get p n+m (i i) > 0; (b) if p k (j j) > 0, then p n+k+m (i i) > 0. From (a) and (b) we can see that d(i) k provided that p k (j j) > 0. This proves what is required p(j i) > 0 for all i, j C for all i, j C there exists an n N such that p n (j i) > R = S \ C R = S \ T If i j, the both If i j, then both the line where F ji (x) is defined in (5.39); where F ji (1) is the probability that the chain will ever visit state j if it starts at i, see (5.39), and where m j is the mean recurrence time of state j, see (5.44); if each i S is ergodic. if each i S is ergodic, i.e. each state i S is positive, recurrent and aperiodic ergodic Markov chain ergodic irreducible Markov chain 4

5 113 5 replace the hint by: Use Theorem 5.5. You may assume as a known fact that if j is recurrent and i j, then F ji (1) = i i replace Exercise 5.36 by: Prove that if there exists an invariant measure for the Markov chain in Exercise 5.14, then λ := j=0 jq j < 1. Assuming that the converse is also true, conclude that the chain is ergodic if and only if λ < 1. Show that if such an invariant measure exists, then it is unique replace the hint by: Suppose that π = j=0 π jδ j is an invariant measure. Write down an infinite system of linear equations for π j. If you don t know how to follow, look at the solution p kn0 (j i) = q n (j i) p kn0 (j i) = q k (j i) max k p n (j k) min k p n (j k) min i p n (j i) min i p n+1 (j i) min k p n (j k) max k p n (j k) max i p n (j i) max i p n+1 (j i) m j (n) m n (j) ε s S p n(j s)εp n (s j) ε s S p n(j s)p n (s j) Chapman-Kolmogorov equations by the Chapman-Kolmogorov equations p k (s i) ε p(s i) ε m n (j) s S (1 ε) (1 ε)m n(j) m n+1 m n+1 (j) m n m n (j) equals ±1 or 0 equals 1 or j i 1 i 1, j i the sentences On the other hand, P (ξ n+1 = j ξ n = 0) = λj j! e λ if j i = 0. Finally, if j = i 1, i 1, then P (ξ n+1 = i 1 ξ n = i) = pe λ. On the other hand, if j 0, then P (ξ n+1 = j ξ n = 0) = λj j! e λ. 5

6 124 5 formula (5.71) q j, if i = 0, j N, p(j i) = q j i+1, if i 1, j i 1, 0, otherwise, (5.71) formula (5.72) q k = { pqk, if k = 0, (1 p)q k 1 + pq k, if k 1. (5.72) P 00 (x) = n=0 f n(0 0)x n P 00 (x) = n=0 p n(0 0)x n i S p n(j i)µ i µ i p n (j i)µ i µ i Π(x) = j=0 π jx i Π(x) = j=0 π jx j Q(x) = j=0 q jx i Q(x) = j=0 q jx j G(x) = j=0 q j xi G(x) = j=0 q j xj these lines : 1) if λ 1+λ λ 1, then Π(x) π 0 1 λ ; 2) if λ = 1, then Π(x). In case 1) 1+λ λ λ 1 λ 1, since π 0 1. It follows that λ < 1. Moreover, in this case 1 λ 0, and so ( q p π 0 = 1 λ 1 λ + λ. The above argument shows that if there exists an invariant measure, then it is unique. Now suppose that an invariant measure exists (and then it is unique). ) ix ( q p) i x x = φ(i) φ(0) x = φ(1) φ(0) insert = before ( q p )i 1 ( q p )a {(x, y) : x y A} {(x, y) : y x A} {y : x y A} {y : y x A} p(t s, x, x u) p(t s, x, x + u) P {η 1 < s} P {η 1 < s 1 } t V (t) = W (t + T ) t V (t) = W (t + T ) W (T ) 6

7 t in the denominator 2t t in the denominator t p(t s, W (t), y) p(t s, W (s), y) p(t s, W (t), y) p(t s, W (s), y) the approximations should consist only of random variables adapted to the approximations of the integrand should consist only of processes adapted to depend only by what has happened up to time t, not on any future events. depend only on what has happened up to time t, but not on any future events form from Exercise 7.4 Exercise insert the following sentence at the end of the hint: You may also need the following identity: ( a 2 b 2) 2 = (a b) (a b) 3 b + 4 (a b) 2 b f n (t) f(t) 2 2C 2 f n (t) f(t) 2 4C T C 2 4T C n 1 i=0 ( n i W )2 t n 1 i=0 ( n i W )2 T nk 1 i=0 ( n k i W ) 2 t n k 1 i=0 ( n k i W ) 2 T x [ n 1, n + 1] x / [ n 1, n + 1] will replaced will be replaced where a belongs to M 2 t and b to L 1 t where a belongs to L 1 t and b to M 2 t form from To obtain a solution Once we have shown that such a ξ M 2 T exists, to obtain a solution are strict contractions. are strict contractions with contracting constants α 1 and α 2 such that α 1 + α 2 < 1. 7

8 206 2 C2 λ ξ ζ λ C 2 λ ξ ζ 2 λ λ > C 2 λ > C 2 /ε Φ 2 is a strict contraction. Φ 2 is a strict contraction with contracting constant ε E(η j ζ j 2 j ) E(η jζ j j W 2 ) E( 2 j ) E( jw 2 ) the line in L 2 as n, since by the Cauchy-Schwartz inequality in L 2 as n. Indeed, by the classical inequality n n 1 i=0 a i 2 and by the Cauchy-Schwartz inequality ( ) E(W t W s ) 2 Ws 2 4E ( ) (W t W s ) 2 Ws 2 n 1 i=0 a i F (t, W (t)) = nw (t) n 1 F x(t, W (t)) = nw (t) n F (t, W (t)) = nw (t) n 1 F x(t, W (t)) = nw (t) n σε αt F x(t, x) σe αt F x(t, x) σε αt F x(t, x) σe αt F x(t, x) 8