LECTURE 10 LINEAR PROCESSES II: SPECTRAL DENSITY, LAG OPERATOR, ARMA. In this lecture, we continue to discuss covariance stationary processes.

Transcription

1 MAY, 0 LECTURE 0 LINEAR PROCESSES II: SPECTRAL DENSITY, LAG OPERATOR, ARMA In this lecture, we continue to discuss covariance stationary processes. Spectral density Gourieroux and Monfort 990), Ch. 5; Hamilton 994), Ch. 6) A convenient way to represent the sequence of autocovariances {γj) : j 0,,...} of a covariance stationary process is by the means of the spectral density or spectrum. The spectral density is defined as follows. f λ) γj)e iλj, j where i. Notice that, since γ j) γ j), it follows that the spectral density is real valued. f λ) γ 0) + γj)e iλj + γj)e iλj j j γ 0) + γ j)e iλj + γj)e iλj j j γ0) + γj) e iλj + e iλj). Next, e iλj cos λj) + i sin λj), e iλj cos λj) i sin λj), and, therefore, j e iλj + e iλj cos λj). Hence, f λ) γ0) + j γj) cos λj) j γj) cos λj). Since cos λj) cos λj), the spectral density is symmetric around zero. Furthermore, since cos is a periodic function with the period, the range of values of the spectral density is determined by the values of f λ) for 0 λ π. The autocovariance function and spectral density are equivalent, as it follows from the result below. Theorem Suppose that h γh) <. Then γj) π f λ) eiλj dλ. Proof. f λ) e iλj dλ h h γh) γh)e iλh ) e iλj dλ e iλj h) dλ,

2 where summation and integration can be interchanged because h γh) <. Next, For j h, e iλj h) dλ if j h. e iλj h) dλ cos λ j h)) + i sin λ j h))) dλ π π sin λ j h)) cos λ j h)) j h i j h sin π j h)) sin j h)) j h cos π j h)) cos j h)) i. j h However, since cos and sin are periodic with the period, cos π j h)) cos j h) + j h)) cos j h)). Therefore, e iλj h) dλ 0 if j h. The result of Theorem implies in particular that γ 0) f λ) dλ. Thus, the area under the spectral density function of X t between and π gives the variance of X t. The argument λ of f λ) is called the frequency. Notice that if {X t } is covariance stationary with absolutely summable autocovariances, the long-run variance is determined by the spectral density at the zero frequency. n ) ω X lim V ar n / X t n h f 0). γ h) Next, we discuss how linear MA) transformations of a covariance stationary process affect the spectral density and long-run variance. Theorem Let {X t } be a covariance stationary process with the autocovariance function γ X such that j γ X j) <. Define Y t c jx t j, where c j <. Then {Y t} is covariance stationary and its spectral density is given by f Y λ) c je iλj fx λ), where f X is the spectral density of {X t }. t

3 Proof. Cov Y t, Y t h ) Cov c j X t j, c j X t h j k0 c j c k Cov X t j, X t h k ) k0 c j c k γ X h + k j). Hence, Cov Y t, Y t h ) is independent of t. Furthermore, by the same argument as on pages - of Lecture 9, c j c k γ X h + k j) γ X h) k0 Therefore, {Y t } is covariance stationary. Next, f Y λ) h <. Cov Y t, Y t h ) e iλh h k0 c j h0 c j c k γ X h + k j) e iλh c j e iλj c k e iλk γ X h + k j) e iλh+k j) k0 h c j e iλj c j e iλj f X λ) c j e iλj f X λ). The last equality follows because c j e iλj j j j c j cos λj) i sin λj)) c j cos λj) i j c j sin λj). Its complex conjugate is c j cos λj) + i j j c j sin λj) j j c j cos λj) + i sin λj)) c j e iλj, 3

4 and hence c j e iλj c j cos λj) + c j sin λj) c j e iλj c j e iλj. In the above theorem, the spectral density at the zero frequency and, as a result, the long-run variance is finite if c j <. However, absolute summability, c j <, is a stronger assumption than square summability, c j <, as we show next. Suppose c j <. First, c j < implies that c j 0 as j. Therefore, the sequence {c j } is uniformly bounded. Next, c j sup j c j c j <. Suppose that {X t } is covariance stationary and purely indeterministic. Then it has the MA ) representation X t a j ε t j, ) where {ε t } is a WN, and a j <. Let V ar ε t) σ. Since the spectrum of a WN process is flat: Theorem implies that and the long-run variance of {X t } is f λ) σ for all λ, f X λ) σ a j e iλj, ω X f X 0) σ If we take ) as the generating mechanism, the condition a j < ensures that {X t} is covariance stationary. However, the sufficient condition for the long-run variance to be finite is a j <. If the last condition fails, we can have that j γ X j). Such a process is called long memory. If a j < holds for a long memory process, then its autocovariance function converges to zero, however, at the rate that is too slow for the long-run variance to be finite. Let {Y t } be as defined in Theorem. Then its spectral density and long-run variance are given by f Y λ) σ a j e iλj c j e iλj, ω Y σ. a j a j c j. 4

5 Lag operator Gourieroux and Monfort 990), Ch. 5; Hamilton 994), Ch. ) The lag operator L transforms the process {X t } into itself such that LX t X t, L X t LLX t LX t X t,... L h X t X t h. The lag polynomial C L) c jl j transforms {X t } into another process {Y t } such that and Y t C L) X t c j L j X t c j X t j. Let A L) a jl j and B L) b jl j. Then We have that AL)BL) AL) + BL) a j + b j ) L j, a j L j b j L j h0 a j b h L j+h a 0 b 0 + a 0 b + b 0 a ) L + a b + a 0 b + a b 0 ) L AL) + BL) BL) + AL), AL)BL) BL)AL). Under certain conditions, a lag polynomial can be inverted. The inverse of a lag polynomial C L) is another lag polynomial, say B L) such that so we can write C L) B L), ) C L) B L). Inversion of lag polynomials is important for the following reason. Consider the following autoregressive process of order AR)): X t cx t + ε t. 3) This process is generated recursively given some exogenous white noise process {ε t }, a starting value X 0 a random variable with the variance equal to V ar X t ) to be determined later), and the coefficient c: X cx 0 + ε, X cx + ε, 5

6 and etc. Thus, the process {X t } is an endogenous solution to the difference equation 3). This difference equation can be also written as X t cx t ε t or C L) X t ε t, C L) cl. If C L) can be inverted, then the solution can be written as X t C L) ε t. Thus, it is important to determine under what conditions a polynomial in lag operator can be inverted and how to compute the coefficients of its inverse. Consider first a polynomial of order. Without loss of generality, we can set the coefficient associated with L 0 as c 0 : C L) cl. Suppose that c <. Then, we can define the inverse of cl as follows. This definition satisfies ) since cl) + cl + c L ) cl) c j L j. The definition 4) is not the only solution that satisfies ). Adding to it the term V c t, where V is some random variable, satisfies it as well, since cl) V c t V c t V clc t V c t V cc t 0. However, if we take cl) cj L j + V c t, then cl) ε t cj ε t j + V c t ε t and is not a covariance stationary process. Therefore, we restrict V 0. Next, consider a lag polynomial of order : We can factor the polynomial as where C L) c L c L. c L c L λ L) λ L) 5) c λ + λ, c λ λ. Another way to find λ and λ is as follows. Let z and z be the solutions possibly complex) to We can write 6) as and by comparing this with 5), we obtain c z c z 0. 6) c z c z z z) z z), λ z and λ z. This is due to the Wold decomposition result. Alternatively, this can be viewed as a normalization: multiplying all coefficients by some constant c 0 0 affects only the variance of the process. 6

7 The polynomial in 5) can be inverted provided that λ < and λ <. These conditions are equivalent to the condition that all the roots of the polynomial in 6) are outside the unit circle: z >. If this condition is satisfied then c L c L ) λ L) λ L) λ j Lj. λ j Lj Alternatively, write Then, λ L λ L λ λ c L c L ) λ λ λ λ λ λ L λ ). λ L λ λ j Lj λ λ j Lj) The result can be extended to a lag polynomial of order p, since we can factor it as where λ j s satisfy C L) c L... c p L p, c L... c p L p c z... c p z p ) ) λ j+ λ j+ L j. p λ j L), j p λ j z). The polynomial can be inverted provided that the roots of c z... c p z p are outside the unite circle. ARMA c L... c p L p ) j p λ j L). 7) Let {ε t } be a WN with V ar ε t ) σ. An MA q) process, say {X t } is generated as where j X t ε t + θ ε t θ q ε t q Θ L) ε t, By Theorem, MA q) has the spectral density Θ L) + θ L θ q L q. f X λ) σ Θ e iλ ) σ + θ e iλ θ q e iλq, 7

8 and the long-run variance ω X f X 0) σ Θ ) σ + θ θ q ). Notice that for certain values of θ j s, the long-run variance can be zero. For 0 j q, the effect of a shock in period t on X after j periods is θ j X t+j ε t, and the shocks have no effect after more than q periods. Definition Autoregression) A process {X t } is said to be autoregressive of order p or autoregression), denoted as AR p), if it is generated according to where {ε t } is a WN. Let Then, ARp) can be written as X t φ X t + φ X t φ p X t p + ε t, Φ L) φ L... φ p L p. Φ L) X t ε t. Provided that all the roots of Φ z) lie outside the unit circle, Φ L) can be inverted, and the process has the following M A ) representation Wold decomposition). X t Φ L) ε t for some Ψ L). Suppose that p. Then, and, according to 4), Ψ L) ε t ψ j ε t j, 8) φ L) X t ε t, X t φ L) ε t φ j Lj ε t φ j ε t j. Hence, in the case of the AR ) process, the coefficients of Ψ L) in 8) are given by ψ j φ j. To check the square-summability condition of Theorem, ψj φ j φ <, 8

9 provided that φ <. Then, Theorem implies that AR ) is covariance stationary, and its spectral density and long-run variance of the AR ) process are f X λ) σ ω X f X 0) φ j e iλj σ φ ). σ φ e iλ, The long-run variance of a stationary AR ) process is finite as well. Hence, in this case we actually have that the coefficients in MA ) for a stationary AR ) process are absolutely summable: ψ j φ j φ <. This is because ψ j φ j 0 as j at the exponential rate. In the case of AR p), p >, due to 7), ψ j < provided that all the roots of Φ z) lie outside the unit circle. Then AR p) is covariance stationary with the spectral density and long-run variance f X λ) σ Φ e iλ ), ω X σ Φ ). Definition ARMA) A process {X t } is ARMA p, q), if it is generated according to where {ε t } is a WN. Φ L) X t Θ L) ε t, When the roots of Φ z) lie outside the unit circle, ARMA p, q) has an MA ) representation X t Φ L) Θ L) ε t. Its spectral density and long-run variance are f X λ) σ Θ e iλ) Φ e iλ ), Θ ) ω X σ Φ ). When the roots of Θ z) lie outside the unit circle, ARMA p, q) has an AR ) representation Θ L) Φ L) X t ε t. Thus, in practice, any covariance stationary ARM A p, q) or M A ) process can be approximated by AR m n ) model, with m n increasing with n, however, at the slower rate. An ARMA process with the nonzero mean µ can be written as Φ L) X t µ) Θ L) ε t. 9

10 Vector case Suppose that the vector process {X t } is covariance stationary and EX t 0 for all t. Let Γ j) EX t X t j. Notice that, in order for {X t } to be covariance stationary, Γ j) does not need to be symmetric for j 0, however, Γ j) Γ j). Consider a sequence of k-matrices {C j }, and define The variance of Y t is given by EY t Y t E Let A tr A A). We have i0 i0 Y t C i X t i ) C j X t j. C j X t j C i Γ j i) C j C j Γ 0) C j + h Cj Γ h) C j+h + C j+h Γ h) C j ). EY t Y t C i Γ j i) C j i0 C i C j Γ j i) i0 Γ h) C j, h0 where the last inequality is by the same argument as on pages - of Lecture 9. Hence, EY t Y t is finite provided that C j <, 9) and h Γ h) <. 0) Definition 3 A k-vector process {ε t } is a vector WN if Eε t 0 and Eε t ε t Σ, a positive definite matrix, for all t, and Eε t ε s 0 for t s. If {X t } is purely indeterministic mean zero covariance stationary process such that 0) holds, similarly to the scalar case, it has the MA ) representation X t C j ε t j, 0

11 where {ε t } is a vector WN and C j s satisfy 9), and C 0 I k. Again, as in the scalar case, ε t s are the linear one step ahead prediction errors. The spectral density of a vector process is defined similarly to the scalar case: f λ) j Γ 0) + Γ j) e iλj Γ j) e iλj + Γ j) e iλj), j Again, the long-run variance-covariance matrix is given by In order for the long-run variance to be finite, Ω f 0). C j <. For the vector WN process {ε t }, the spectral density is flat: f ε λ) Σ. Let {X t } be a covariance stationary process with the spectral density f X. Define Y t C j X t j. Then, the spectral density of {Y t } is given by f Y λ) C j e iλj f X λ) C je iλj. Let {C j } be a sequence of k-matrices. The lag polynomial C L) in the vector case is defined as C L) I k + C L + C L +..., where we set C 0 I k according to the Wold decomposition. We say if B L) C L) B L) C L) I k. The polynomial C L) is invertible provided that the roots of det C z)) 0 lie outside the unit circle. For example, if C L) is of order p, det I + C z C p z p ) 0 has to satisfy that all the roots are greater than one in absolute value.