Part III Example Sheet 1 - Solutions YC/Lent 2015 Comments and corrections should be ed to

Transcription

1 TIME SERIES Part III Example Sheet 1 - Solutions YC/Lent 2015 Comments and corrections should be ed to Y.Chen@statslab.cam.ac.uk. 1. Let {X t } be a weakly stationary process with mean zero and let a and b be constants. (a) If Y t = a + bt + s t + X t where s t is a seasonal component with period 12, show that 12 Y t = (1 B)(1 B 12 )Y t is weakly stationary. (b) If Y t = (a + bt)s t + X t where s t is again a seasonal component with period 12, show that 2 12Y t = (1 B 12 )(1 B 12 )Y t is weakly stationary. (a) We have 12 Y t = (1 B B 12 + B 13 )Y t = b(t t + 1 t t 13) Therefore, + (s t s t 12 ) (s t 1 s t 13 ) + X t X t 1 X t 12 + X t 13 = X t X t 1 X t 12 + X t 13. E( 12 Y t ) = 0, E( 12 Y t )( 12 Y t+h ) = E(X t X t 1 X t 12 + X t 13 )(X t+h X t+h 1 X t+h 12 + X t+h 13 ). Since these values do not depend on t, 12 Y t is weakly stationary. (b) We have and Hence 2 12Y t is stationary. R t := (1 B 12 )Y t = 12bs t 12 + X t X t Y t = (1 B 12 )R t = X t 2X t 12 + X t Let Y t = a + bt + X t, where {X t, t = 0, ±1,...} is an independent and identically distributed sequence of random variables with mean 0 and variance σ 2, and a and b are constants. Define W t = (2q + 1) 1 q Y t+j. Compute its mean. Show that although {W t } is not stationary, it has an autocovariance function γ h = cov(w t, W t+h ) that does not depend on t. Briefly discuss your findings in relation to time series smoothing. j= q We have E(W t ) = 1 2q + 1 E q j= q Y t+j = 1 2q + 1 q [a + b(t + j)] = a + bt j= q 1

2 and q q cov(w t, W t+h ) = (2q + 1) 2 cov X t+j, j= q j= q X t+h+j σ 2 /(2q + 1), h = 0 = σ 2 (2q + 1 h )/(2q + 1) 2, 0 < h 2q 0, h > 2q, which does not depend on t. This filter is a low-pass (or smoothing) filter since it takes the data {Y t } and removes from it the rapidly fluctuating (or high frequency) component {X t }, to leave the slowly varying estimated trend a + bt. 3. Which, if any, of the following functions defined on the integers is the autocovariance function of a weakly stationary time series? (a) (b) f(h) = { 1 if h = 0, 1/h if h 0. 1 if h = 0, 0.8 if h = ±1, f(h) = 0.6 if h = ±2, 0 otherwise. (a) No. An autocovariance function should be even, while f(h) is not. (b) No. An autocovariance function should be semi-positive (or non-negative) definite. It is easy to see that the following matrix is not semi-positive definite. 4. Let ω (0, π) be given, U Uniform[ π, π], and define X t = cos(ωt + U). Prove that the process is weakly stationary. In addition, show that X t = (2 cos ω)x t 1 X t 2, so we can perform perfect prediction using a linear function of past observations. Note that E(X t ) = 1 π cos(ωt + u)du = 0 2π π 2

3 and π γ h = cov(x t, X t+h ) = 1 cos(ωt + u) cos(ωt + ωh + u)du 2π π = 1 π 1 {cos(ωh) + cos(2ωt + ωh + 2u)}du 2π π 2 = 1 {2π cos(ωh) + 0} 4π = cos(ωh)/2. Thus {X t } is weakly stationary. As a side remark, γ h does not go to zero as h. Now because X t + X t 2 = cos(ωt + U) + cos(ωt 2ω + U) = 2 cos(ω) cos(ωt ω + U) = 2 cos(ω)x t 1, we have that X t = (2 cos ω)x t 1 X t For a given weakly stationary series {X t } and a pre-determined series {a 0, a 1, a 2,...} with i=0 a i <. Consider the sequence Y m t = m a j X t j j=0 for m = 0, 1, 2,.... Argue that Yt m has a mean square limit, that is, there exists a random variable Y t with EYt 2 < such that E Y t Yt m 2 0, as m. Explain why this result is useful when we write an ARMA model in its causal (i.e. MA( )) or invertible (i.e. AR( )) representation. It suffices to prove that {Yt m, m N} is a Cauchy sequence, i.e., E Yt n n > m > 0. This follows from E Y n t Y m t 2 = E = m<j n m<j n m<k n a j X t j 2 m<j n m<k n m<j n m<k n ( = var(x 1 ) m<j n a j a k E(X t j X t k ) a j a k E(X t j X t k ) a j a k (EX 2 t j) 1/2 (EX 2 t k) 1/2 a j ) 2 0, Yt m 2 0 as m, n for where we used the Cauchy Schwarz inequality in the second last line and the stationarity of {X t } in the last line. This result allows us to represent the present observation from ARMA as an infinite sum of past noise (i.e. MA( )), or represent the present noise as an infinite sum of past observations (i.e. AR( )). We now know that these representations need to be understood in the sense of mean squares convergence. 3

4 6. Prove that if an MA(q) process is invertible, then the roots of the equation Θ(z) = 0 all lie outside the unit circle. Let Θ(B) = 1 + θ 1 B + θ 2 B θ q B q and write the MA(q) process as X t = Θ(B)ɛ t. By the invertibility of the process, we can find Π(B) = π j B j, with j=0 π j < such that ɛ t = Π(B)X t. Therefore, Θ(B)ɛ t = Θ(B)Π(B)X t = Θ(B)Π(B)Θ(B)ɛ t, or equivalently, j=0 {Θ(B) Θ(B)Π(B)Θ(B)}ɛ t = 0. Note that Θ(z) Θ(z)Π(z)Θ(z) is a polynomial (though might be of degree ). Multiplying both sides of the above equation by ɛ t h for any integer h 0 and taking the expectation allows us to see that the coefficient of z h in {Θ(z) Θ(z)Π(z)Θ(z)} is zero. Furthermore, because j=0 π j <, Θ(z) Θ(z)Π(z)Θ(z) is well-defined for any z 1 (as the limit of a polynomial of degree ). Consequently, Θ(z) Θ(z)Π(z)Θ(z) = 0 for any z 1. For any z 1 such that Θ(z) 0, this implies that Π(z)Θ(z) = 1. Because Π(z)Θ(z) is a continuous function, and the set {z : Θ(z) = 0} can contain at most q points, we conclude that Π(z)Θ(z) = 1 for any z 1. If there is a complex number in the unit circle, say z 0 1, for which Θ(z 0 ) = 0, then it would imply Π(z 0 )Θ(z 0 ) = 0, which would lead to a contradiction. Therefore, all the roots of Θ(z) are outside the unit circle. Moreover, the series {π j } can be determined by solving Π(z) = j=0 π j z j = 1, z 1. Θ(z) 7. For any MA(2) process, if η 1, η 2 C are the roots of the MA polynomial Θ(z), show that they can be replace by 1/η 1 and 1/η 2 without changing the ACF. That is, MA processes with Θ(z) = (1 η1 1 z)(1 η 1 2 z) and Θ (z) = (1 η 1 z)(1 η 2 z) have the same ACF. Briefly comment on how you would generalise this result to MA(q) with q > 2. For an MA(2) process, the ACF cuts off after lag 2, so we only need to concentrate on ρ 1 and ρ 2. Suppose that the MA polynomial is Θ(z) = 1 + θ 1 z + θ 2 z 2, then ρ 1 = θ 1 + θ 1 θ θ1 2 +, ρ 2 = θ2 2 θ θ θ2 2 4

5 By plugging in the following two sets of equations (also note that η 1 and η 2 are conjugate), { θ 1 = η1 1 + η2 1 θ 2 = η1 1 η 1 2 we can verify that the ACFs are the same. { θ 1 = η 1 + η 2 θ 2 = η 1 η 2, This result can be generalised to MA(q), where if η 1,..., η q C are the roots of the MA polynomial Θ(z), some (or all) of the roots η i can be replace by 1/η i (1 i q) without changing the ACF. Spectral analysis turns out to be extremely useful in proving this generalisation. 8. Show that in order for an AR(2) process with autoregressive polynomial Φ(z) = 1 φ 1 z φ 2 z 2 to be causal, the parameters (φ 1, φ 2 ) must lie in the triangular region determined by the intersection of the three regions, φ 1 + φ 2 < 1, φ 2 φ 1 < 1 and φ 2 < 1. The process can be written as Φ(B)x t = ɛ t, where B is the backward shift operator. {X t } is causal if all of the (complex) roots of Φ(z) = 0 are outside the unit disc in C. Note that Φ is a quadratic polynomial with real coefficients, so either we have two different real roots, a repeated real root or a complex conjugate pair. Suppose that Φ can be factorised as Φ(z) = (1 az)(1 bz) with a, b < 1. Then by comparing coefficients of z and z 2, we see that φ 1 = a + b, φ 2 = ab. (a) Repeated root: a = b, thus (φ 1, φ 2 ) must lie on the parametric curve f(a) = (2a, a 2 ), a < 1. (b) Distinct real roots: say a = c + d, b = c d for some c, d with c < 1, d < 1 c. Then, φ 1 = 2c, φ 2 = c 2 + d 2, which are bounded by the curve from the first case, and the lines φ 1 + φ 2 = 1 and φ 2 φ 1 = 1. (c) Complex conjugate pair: let a = c + id, b = c id with c 2 + d 2 < 1. Then, φ 1 = 2c, φ 2 = (c 2 + d 2 ) which are bounded by the curve and the line φ 2 = 1. The combination of all these cases is the interior of the given region in the (φ 1, φ 2 ) plane. We conclude that the stipulated conditions are equivalent to causality. 9. Let ɛ t i.i.d. N(0, σ 2 ) and let φ < 1 be a constant. Consider the process X 1 = ɛ 1 and X t = φx t 1 + ɛ t, for t = 2, 3,.... (a) Find the mean and the variance of the series. Is {X t } stationary? (b) Find corr(x t, X t+h ) for h N. Argue that for large t, var(x t ) σ 2 /(1 φ 2 ) and corr(x t, X t+h ) φ h. So in a sense, {X t } is asymptotically stationary. 5

6 (c) Now suppose X 1 = ɛ 1 / 1 φ 2. Is this new process stationary? Comment on how you could use these findings to simulate n observations from a stationary AR(1). (a) Write X t as X t = ɛ t + φɛ t 1 + φ 2 ɛ t 2 + φ t 2 ɛ 2 + φ t 1 ɛ 1. So EX t = 0 and EX 2 t = σ 2 (1 φ 2t )/(1 φ 2 ). Therefore, {X t } is not (either weakly or strongly) stationary. (b) For h N, It follows that as t. cov(x t, X t+h ) = σ 2 (φ h + φ h φ 2t+h 2 ) = σ 2 φ h var(x t ). var(x t ) = σ 2 (1 φ 2t )/(1 φ 2 ) σ 2 /(1 φ 2 ), [ ] 1/2 cov(x t, X t+h ) corr(x t, X t+h ) = var(xt )var(x t+h ) = var(xt ) φh = φ h 1 φ 2t var(x t+h ) 1 φ 2(t+h) φh, (c) Now EX t = 0 and var(x t ) = σ 2 {1 + φ 2 + φ φ 2(t 2) + φ 2(t 1) /(1 φ 2 )} = σ 2 /(1 φ 2 ). Moreover, for h N, cov(x t, X t+h ) = σ 2 {φ h + φ h φ 2t+h 4 + φ 2t+h 2 /(1 φ 2 )} = σ 2 1 φ 2, which does not depend on t. Therefore, the new series is (both weakly and strongly) stationary. When simulating an AR(1), we can either discard the initial chunk of data, or simulate the first observation X 1 from its stationary distribution. 10. Identify the following models as ARMA(p, q) models, and determine whether they are causal and/or invertible: (a) X t = 0.4X t X t 2 + ɛ t + ɛ t ɛ t 2. (b) X t = X t 1 0.5X t 2 + ɛ t ɛ t 1. (a) Rewrite the process as (1 0.9B)( B)X t = ( B) 2 ɛ t. Since ( z) is the common factor in both the AR polynomial and the MA polynomial, there is an issue of parameter redundancy. This process is actually an ARMA(1,1) process, which can be written as (1 0.9B)X t = ( B)ɛ t. Since both the root of 1 0.9z = 0 and the root of z = 0 lie outside the unit circle, we conclude that this process is both causal and invertible. (b) Rewrite the process as (1 B + 0.5B 2 )X t = (1 B)ɛ t. This is an ARMA(2,1) process. Since the roots of the AR polynomial, 1 ± i, lie outside the unit circle, the process is causal. However, it is not invertible as the MA polynomial has a unit root. φ h 6

7 11. For an AR(2) model given by X t = 0.8X t 2 + ɛ t, determine and sketch its ACF and PACF. X t = ɛ t 0.8X t 2 = ɛ t 0.8ɛ t X t 4 = = ( 0.8) j ɛ t 2j It is easy to see that ρ h = 0 when h is odd. When h is even, we let Y t = X 2t. Therefore, j=0 ρ h = corr(x t, X t+h ) = corr(y t, Y t+h/2 ) = ( 0.8) h /2, where the last equality follows from the fact that {Y t } is an AR(1) process with φ = 0.8. For PACF, because {X t } is an AR(2) process, φ hh = 0 for any h > 2. By definition, φ 11 = ρ 1 = 0. Lastly, since β = ρ 1 = 0, we have φ 22 = corr(x t βx t+1, X t+2 βx t+1 ) = 0.8. This example demonstrates that both ACF and PACF are not necessarily decreasing functions (even in absolute values)! 12. ( ) Suppose that {X t } and {Y t } are two zero-mean weakly stationary processes with the same autocovariance function and that {Y t } is an ARMA(p, q) process. Show that {X t } must also be an ARMA(p, q) process 1. First, we state and prove the following lemma: Lemma. If {X t } is a non-zero stationary process with autocovariance function γ such that γ h = 0 for h > q and γ q 0, then {X t } is an MA(q) process, i.e., there exists a white noise process {Z t } such that X t = Z t + θ 1 Z t θ q Z t q. (1) Proof of Lemma. For each t, define the subspace M t = sp{x s, < s t} of L 2 and set Z t = X t P Mt 1 X t. (2) Here sp{ } is the closure of a linear span and P M ( ) represents the projection onto a subspace M. Clearly Z t M t, and by definition of P Mt 1, Z t M t 1, where M is the orthogonal complement of M. Thus if s < t, Z s M s M t 1 and hence EZ s Z t = 0. Moreover, P sp{xs,s=t n,...,t 1}X t m.s. P Mt 1 X t, n, where m.s. means convergence in mean squares. By weak stationarity and the continuity of the l 2 -norm, Z t+1 = X t+1 P Mt X t+1 = lim n X t+1 P sp{xs,s=t+1 n,...,t}x t+1 = lim n X t P sp{xs,s=t n,...,t 1}X t = X t P Mt 1 X t = Z t. 1 In this question, ARMA(p, q) processes should be interpreted in the broad sense, where the errors are white noise but not necessarily Gaussian. 7

8 Defining σ 2 = Z t 2, we conclude that {Z t } WN(0, σ 2 ). Now by (2), it follows that M t 1 = sp{x s, s < t 1, Z t 1 } = sp{x s, s < t q, Z t q,..., Z t 1 } and consequently M t 1 can be decomposed into the two orthogonal subspaces, sp{z t q,..., Z t 1 } and M t q 1. Since γ h = 0 for h > q, it follows that X t M t q 1 and P Mt 1 X t = P Mt q 1 X t + P sp{zt q,...,z t 1}X t = θ 1 Z t θ q Z t q where θ j := σ 2 E(X t Z t j ), which by stationarity is independent of t for j = 1,..., q. Substituting for P Mt 1 X t in (2) gives (1). Let {W t } be time series satisfying W t := X t φ 1 X t 1 φ p X t p = Φ(B)X t, where Φ(z) = 1 φ 1 z φ p z p and φ 1,..., φ p are the AR coefficients for {Y t }. The autocovariance function of {W t } thus satisfies γ W h = cov { Φ(B)X t, Φ(B)X t+h } = cov { Φ(B)Yt, Φ(B)Y t+h }. Since {Φ(B)Y t } is an MA(q) process by assumption, we have that γh W = 0 for h > q. It follows from the above lemma, that {W t } is also an MA(q) process and thus {X t } is an ARMA(p, q) process. 8