Convergence in Mean Square Tidsserieanalys SF2945 Timo Koski

Transcription

1 Avd. Matematisk statistik Convergence in Mean Square Tidsserieanalys SF2945 Timo Koski MEAN SQUARE CONVERGENCE. MEAN SQUARE CONVERGENCE OF SUMS. CAUSAL LINEAR PROCESSES

2 1 Definition of convergence in mean square Definition 1.1 A random sequence {X n } n=1 with E [X2 n ] < is said to converge in mean square to a random variable X if as n. E [ X n X 2] 0 (1.1) In Swedish this is called konvergens i kvadratiskt medel. We write also X = l.i.m. n X n. This definition is silent about convergence of individual sample paths X n (s). By Chebysjev s inequality we see that convergence in mean square implies convergence in probability. 2 Mean Ergodic Theorem Although the definition of converge in mean square encompasses convergence to a random variable, in many applications we shall encounter convergence to a degenerate random variable, i.e., a constant. Theorem 2.1 The random sequence {X n } n=1 WN(µ, σ2 ) Then µ = l.i.m. n 1 n n X n. Proof: Let us set S n = 1 n n j=1 X n. We have E [S n ] = µ and V ar [S n ] = 1 n σ2, since the variables are non-correlated. For the claimed mean square convergence we need to consider j=1 E [ S n µ 2] = E [ (S n E [S n ]) 2] = V ar [S n ] = 1 n σ2 so that E [ S n µ 2] = 1 n σ2 0 as n, as was claimed. Since convergence in mean square implies convergence in probability, we have the weak law of large numbers:

3 Theorem 2.2 The random sequence {X n } n=1 is WN(µ, σ2 ). Then in probability. 1 n n X n µ j=1 3 Cauchy-Schwartz and Triangle Inequalities Lemma 3.1 E [ XY ] E [ X 2 ] E [ Y 2 ]. (3.2) E [ X ± Y 2 ] E [ X 2 ] + E [ Y 2 ]. (3.3) The inequality (3.2) is known as the Cauchy- Schwartz inequality, and the inequality (3.3) is known as the triangle inequality. 4 Properties of mean square convergence Theorem 4.1 The random sequences {X n } n=1 and {Y n} n=1 are defined in the same probability space and E [Xn] 2 < and E [Yn 2 ] <. Let Then it holds that X = l.i.m. n X n, Y = l.i.m. n Y n. (a) (b) (c) E [XY ] = lim n E [X n Y n ] E [X] = lim n E [X n ] E [ X 2] = lim n E [ X n 2]

4 (d) if E [Z 2 ] <. E [X Z] = lim n E [X n Z] Proof: All the rest of the claims follow easily, if we can prove (a). First, we see that E [X n Y n ] < and E [XY ] < in view of Cauchy - Schwartz and the other assumptions. In order to prove (a) we consider E [X n Y n ] E [XY ] E [(X n X)Y n + X (Y n Y ) ] since E [Z] E [ Z ]. Now we can use the ordinary triangle inequality for real numbers and obtain: E [(X n X)Y n + X (Y n Y ) ] E [ (X n X)Y n ] + E [ X (Y n Y ) ]. But Cauchy-Schwartz entails now and E [ (X n X)Y n ] E [ X n X 2 ] E [ Y n 2 ] E [ (Y n Y )X ] E [ Y n Y 2 ] E [ X 2 ]. But by assumption E [ X n X 2 ] 0 and E [ Y n Y 2 ] 0, and thus the assertion (a) is proved. We shall often need Cauchy s criterion for mean square convergence, which is the next theorem. Theorem 4.2 Consider the random sequence {X n } n=1 with E [X2 n ] < for every n. Then E [ X n X m 2] 0 (4.4) as min(m, n) if and only if there exists a random variable X such that X = l.i.m. n X n. This is left without a proof. A useful form of Cauchy s criterion is known as Loève s criterion: Theorem 4.3 E [ X n X m 2] 0 E [X n X m ] C. (4.5) as min(m, n), where the constant C is finite and independent of the way m, n.

5 Proof: Proof of =: We assume that E [X n X m ] C. Thus E [ X n X m 2] = E [X n X n + X m X m 2X n X m ] C + C 2C = 0. Proof of = : We assume that E [ X n X m 2 ] 0. Then Here E [X n X m ] = E [(X n X)X m ] + E [XX m ]. E [(X n X) X m ] E [l.i.m (X n X) l.i.mx m ] = 0, by theorem 4.1 (a), since X = l.i.m. n X n according to Cauchy s criterion. Also E [XX m ] E [Xl.i.mX n ] by theorem 4.1 (d). Hence E [X n X m ] 0 + C = C. 5 Applications Consider a random sequence {X n } n=0 WN(µ, σ2 ). We wish to find conditions such that we may regard an infinite linear combination of random variables as a mean square convergent sum i.e. a i X i = l.i.m. n n a i X i The Cauchy criterion in theorem 4.2 gives for Y n = n a ix i and n < m that E [ [ ] Y n Y m 2] m m ( m ) 2 = E a i X i 2 = σ 2 a 2 i + µ 2 a i, i=n+1 i=n+1 i=n+1 since EZ 2 = V ar(z)+(e [Z]) 2 for any random variable. Hence we see that E [ Y n Y m 2 ] converges to zero if and only if a2 i < and a i <

6 in case µ 0 and in case µ = 0. We note here that a 2 i < a i < a 2 i <. (5.6) 6 Causal Linear Processes 6.1 A Notation for White Noise We say that {Z t, t T Z} is WN(0, σ 2 ) if E [Z t ] = 0 for all t T and { σ 2 h = 0 γ Z (h) = (6.1) 0 h 0 Next, Kronecker 1 delta, denoted by δ i,j, is defined for integers i and j as δ i,j = { 1 i = j 0 i j. (6.2) Then we can write the ACVF in (6.1) above as γ Z (h) = σ 2 δ 0,h. (6.3) and even as γ Z (r, s) = σ 2 δ r,s. (6.4) 6.2 Causal Linear Processes Let {X t, t T Z} be given by X t = where {Z t, t T Z} is WN(0, σ 2 ). ψ j Z t j (6.5) j=0 1 Leopold Kronecker, a German mathematician, , has also deeper contributions to mathematics, see history/mathematicians/kronecker.html

7 Definition 6.1 If ψ j < (6.6) j=0 then we say that (6.5) is a causal linear process. The condition (6.6) guarantees (c.f. (5.6)) that the infinite sum in (6.5) converges in mean square. By causality we mean that the current value X t is influenced only by values of the white noise in the past, i.e., Z t 1, Z t 2,..., and its current value Z t, but not by values in the future. Alternatively, X t = ψ j Z t j (6.7) j= is causal if ψ j = 0 for j < 0. Then we can also write X t = t j= ψ t j Z j. (6.8) Now we compute that ACVF of any causal linear process. We use the convergences in theorem 4.1 above. Assume h > 0. γ X (t, t + h) = E [X t X t+h ] = ψ k ψ l E [Z t k Z t+h l ] = l=0 ψ k ψ l σ 2 δ t k,t+h l l=0 = σ 2 ψ k ψ h+k, (6.9) where we applied (6.4). Hence for h > 0 γ X (h) = σ 2 ψ k ψ h+k. (6.10) One finds that γ X (h) = γ X ( h). In fact we have as above that if h < 0 γ X (t, t h) = E [X t X t h ] = ψ k ψ l E [Z t k Z t h l ] l=0 = σ 2 ψ k ψ k h. (6.11)

8 Now we have to recall that causality requires ψ k h for k h < 0, i.e., k < h. Thus σ 2 ψ k ψ k h = σ 2 ψ k ψ k h. By change of variable s = k h we get k=h σ 2 ψ k ψ k h = σ 2 ψ s+h ψ s = γ X (h). k=h Hence we have shown the following. Proposition 6.1 If j=0 ψ j <, then a causal linear process X t = is (weakly) stationary and we have The process variance is thus t s=0 j= ψ t j Z j. (6.12) γ X (h) = σ 2 ψ k ψ k+ h. (6.13) γ Y (0) = σ 2 ψk 2. (6.14)