E 4101/5101 Lecture 6: Spectral analysis

Transcription

1 E 4101/5101 Lecture 6: Spectral analysis Ragnar Nymoen 3 March 2011

2 References to this lecture Hamilton Ch 6 Lecture note (on web page)

3 For stationary variables/processes there is a close correspondence between analysis in the time domain and analysis in the frequency domain Both approaches aid the understanding of time series properties. The correspondence applies to: Representation of series properties (ACF and power spectral density function, PSD). Estimation, empirical ACF/regression in the time domain and empirical periodogram cross-periodogram. In this lecture the main emphasis is on the use of spectral analysis to aid the interpretation of the properties of ARMA and ARIMA series

4 For estimation purposes the time domain is usually most practical and we will not cover the technical aspects estimation in the frequency domain, but the lecture note gives the concepts.

5 Weak stationarity I Let {Y t ; t = 0, ±1, ±2, ±3,...} represent a time series each Y t is stochastic variable in the usual sense. Stationarity is defined in terms of linear properties: Expectation and covariance. The (theoretical) Auto-Correlation Function, ACF, {ρ 1, ρ 2,...}, is where ρ j,t = Corr{Y t, Y t j } = Cov[Y t, Y t j ] Var[Y t ] γ j,t = E[(Y t µ)(y t j µ)], j = 0, 1, 2,..., µ t = E[Y t ] = y t f Yt (y t )dy t for an unconditional probability density function f Yt (y t ). = γ j,t γ 0,t, (1)

6 Weak stationarity II Definitions (Stationarity) If µ t, γ j,t ( j = 0, 1, 2,...) do not depend on t, the Y t process is weakly stationary (covariance stationary): E[Y t ] = µ, for all t E[(Y t µ)(y t j µ)] = γ j for all t and j. Note also that if Y t is stationary, γ j = γ j. For an actual, finite sample, time series {y t ; t = 1, 2, 3,...T } we use the empirical autocorrelations ˆγ j = 1/T T (y t ȳ)(y t j ȳ), j = 0, 1, 2,..., T 1 (2) t=j+1

7 Weak stationarity III where ȳ j = 1/T T t=1 y t, and the empirical ACF is ˆρ j = ˆγ j ˆγ 0, (3) If {Y t ; t = 0, ±1, ±2, ±3,...} is stationary, the ACF can be estimated consistently from the empirical ACF. This provides the theoretical foundation for estimation of other parameters, notably the coefficients in dynamic equations (with OLS and GMM).

8 White-noise A process ε t is white-noise if A white-noise process is stationary. If, in addition to (4)-(6), ε t is Gaussian white-noise. E[ε t ] = 0, (4) Var[ε t ] = E [ε 2 t] = σ 2, (5) Cov[ε t, ε t j ] = γ j = 0. (6) ε t IIN(0, σ 2 ) (7)

9 Maintaining stationarity through linear filtering I A linear filter is a linear combination of L j, j = ±1, ±2, ±3,.... A linear filter can be of finite or infinite order. The linear filter ψ(l) = j= ψ j L j is well defined if j ψ j <. Theorem If j= ψ j < (alternatively j= ψ 2 j < ), a linear filtering of a stationary process will produce a new process which is also stationary. We have seen that if Y t is generated by a stable stochastic difference equation of order p, the solution defines Y t as a well defined linear filter of ε t.

10 Maintaining stationarity through linear filtering II ε t white-noise, Y t is stationary. ε t MA(q), Y t is also stationary. From now on we write Y t ARMA(p, q) (8) for the case when Y t is a stationary solution of Y t = φ 0 + φ 1 Y t φ p Y t p + ε t + θ 1 ε t φ p ε t q (9) Theorem If Y t ARMA(p, q), Y t is a stationary variable.

11 Maintaining stationarity through linear filtering III Theorem If Y t ARMA(p, q) and causal, Y t is a stationary variable given by a one-sided linear filter of ε t. Y t µ = ψ(l)ε t = ψ j ε t j, E[Y t ] = µ j=0 For the AR(p) process Y t φ 1 Y t 1.. φ p Y t p = φ 0 + ε t, with φ 0 = µ(1 φ 1... φ p )

12 Maintaining stationarity through linear filtering IV the autocovariances are given by γ j φ 1 γ j 1.. φ p γ j p = 0 (10) and the autocorrelations are given by the Yule-Walker equations: ρ j φ 1 ρ j 1.. φ p ρ j p = 0. (11) Hence the ACF follow the same dynamics as the Y t process itself. In particular: γ j = b 1 λ j 1 + b 2λ j b pλ j p, where the b i s are numbers and λ i are the roots of the characteristic equation associated with the solution of the homogenous part of the difference equation. Note that p i=1 b i = Var[Y t ].

13 Maintaining stationarity through linear filtering V The results that the ACF has the same dynamics as the Y t itself carries over to the case with a MA part. For ARMA(p, q) low order γ j are influenced by the MA part, but for for higher orders the same qualitative result holds: The ACFs are dominated by the autoregressive part of the generating equation.

14 Period and frequency I A periodic function with amplitude A and phase ϕ : f (t) = A cos(λt ϕ) = a cos(λt) + b sin(λt) (12) where a = A cos(ϕ) and b = A sin(ϕ). λ is frequency in radians. The period, C, is defined the length in time of one full cycle C = ϕ + 2π λ ϕ λ = 2π λ. If C = 2 years, the number of cycles per year is 1/2. We define frequency as the number of cycles per unit of time, hence v = C 1.

15 Period and frequency II For reference: the relationship between the two frequency variables λ = 2πv Later we shall plot a function (power spectral density, PSD) which is symmetric between 1/2 and 1/2 when v is the frequency. v Radians Freq in OxMetrics /4 π/2 1/2 1/2 π 1 Might be different in other software.

16 Discrete Fourier Transform I Heuristically we want to approximate a time series {x t } as closely as possible by a linear combination of cosine functions, as in x t = a 0 + P j=1 {a j cos(λ j t) + b j sin(λ j t)} + rest This problem has a solution in Fourier analysis. The Discrete Fourier Transform (DFT) of the time series {x t }: X (k) = X C (k) ix S (k), (13) where v k = k/t ; k = 0, 1, 2,..T 1, and X C (k) and X S (k) are called the cosine and sine transformations of {x t }:

17 Discrete Fourier Transform II and X C (k) = T X S (k) = T T 1 1/2 x t cos(2πv k t) (14) t=0 T 1 1/2 x t sin(2πv k t) (15) t=0 The lecture note shows that X (k) can be written as: X (k) = X C (k) ix s (k) = T T 1 1/2 x t exp{ 2πv k it} (16) t=0 which defines X (k) as complex numbers associated with the frequencies v k.

18 Discrete Fourier Transform III The DFT in (16), also have the inverse x t = T 1/2 T 1 k=0 X (k) exp{2πv kit} = T 1/2 T 1 k=0 X (k){cos(2πv kt) + i sin(2πv k t)} (17) showing that the DFT gives what we hoped for: A decomposition of {x t } in terms of cosine waves, with X (k) as weights for the different frequencies.

19 Periodogram Since X (k) is complex, the real numbered contribution from each frequency is defined as P x (k): P x (k) = X (k)x (k) = X (k) 2 = X C (k) 2 + X S (k) 2, (18) where X (k) is the conjugate and X (k) is the norm of X (k). P x (k) is real and P x (k) is proportional to the amplitude to the cosine function with frequency υ k. The plot of P x (k) against v k is called the periodogram.

20 Periodogram 0.75 LC Periodogram D LC P x (k) for quarterly Norwegian private consumption ln(cp t )and ln(cp t )

21 Infinite Fourier transform I The DFT has properties that are very similar to Fourier transformations of more general functions a t defined over, t = 0, ±1, ±2,... If s= a s <, the Infinite Fourier transformation, IFT, of {a s } is defined as with the inverse: A(v) = a t = 1/2 a t exp( 2πivt) (19) t= 1/2 A(v) exp(2πivt)dv. (20) v is without a subscript, since v is a continuous frequency here.

22 Population power spectrum density I A direct application if the IFT gives the spectral representation of the ACF R x (m) for a stationary time series x t : R x (m) = E [(x t+m µ)(x t µ)] (21) where µ = E [x t ]. Stationarity means that From (19), the IFT of R x (m) is f x (v) = R x (m) <. (22) m= R x (m) exp( 2πivm) (23) m=

23 Population power spectrum density II and R x (m) = 1/2 1/2 f x (v) exp(2πivm)dv. (24) where f x (v) is called the population power spectrum density, PSD The PSD f x (v) is unique and real if x t is real. f x (v) is also positive and symmetric. We therefore have for x t real f x (v) = f x ( v) R x (m) = 2 1/2 0 f x (v) exp(2πivm)dv (25) saying that both R x (m) and f x (v) are completely described by the frequencies in the interval 0 v 1/2.

24 Population power spectrum density III Note that, by setting m = 0 in (24) we have that the variance of x t can be written as Var[x t ] = R x (0) = 1/2 1/2 f x (v)dv (26) showing that f x (v)dv is the contribution to the variance from each frequency. If x t is white-noise, we have: { σ R x (m) = 2, m = 0 0, m = ±1, ±2... (23) gives f x (v) = σ 2, 1/2 v 1/2 the (PSD) is constant and equal at all frequencies.

25 The PSD of ARMA processes I The lecture note shows that the power spectral density for {y t } is: when f y (v) = A(v) 2 f x (v) = a(exp( 2πiv) 2 f x (v). (27) y t = s= a s x t s = a(l)x t s, (28) s= where a s <, is a filter a(l) = s= s= a s L s and {x t } is weakly stationary. is called the frequency response function. A(υ) = a(exp( 2πiv)) (29)

26 The PSD of ARMA processes II The relationship f y (v) = A(v) 2 f x (v) Shows that the power spectrum of the input series is changed by filtering, and that the effect of the change is described as a multiplication by the squared magnitude of the frequency response function

27 The PSD of ARMA processes III Armed with (27) it is possible to find the PSD of any y t ARMA(p, q): φ(l)y t = θ(l)ε t, ε t UIN(0, σ 2 ), (30) with φ(l) = 1 φ 1 L φ 2 L 2... φ p L p and θ(l) = 1 + θ 1 L + θ 2 L θ p L q. The lecture note shows f y,arma(p,q)] (v) = θ(exp( 2πiv) 2 φ(exp( 2πiv) 2 σ2 (31)

28 PSD of AR(1) The lecture note shows f y,arma(1,0) = σ 2 1 2φ 1 cos(2πv) + φ1 2. (32) v = min[1 2φ 1 cos(2πv) + φ1 2] = 0 when φ 1 > 0 and v 0 v 1/2. The PSD has a peak in v = 0 and declines with increasing v until v = 1/2. If φ 1 < 0, v = 1/2 and the spectral density is increasing in v,

29 PSD of ARMA(2,1) y t ARMA(2, 1) has the PSD: f y,arma(2,1) (v) = y t φ 1 y t 1 φ 2 y t 2 = ε t + θ 1 ε t 1, σ 2 (1 + θ 1 2 cos(2πv) + θ 2 1 ) 1 + φ φ2 2 φ 1(1 φ 2 )2 cos(2πv) φ 2 2 cos(4πv).

30 Power-shift and phase-shift of a filter I Let x t denote a stationary time series and let y t = s= a s x t s, where s= The IFT for the filter {a s } is, as we have seen A(υ) = s= a s <. s= a s exp( 2πivs). s= A(v) is the frequency response function and the filter a s (s = 0, ±1, ±2,...) is often called the impulse-response function in the literature.

31 Power-shift and phase-shift of a filter II Since A(v) often is complex, it is useful to write A(v) in polar-coordinate form: A(v) = A(v) exp(iκ(v)) A(v) is called the power-shift and κ(v) is called phase-shift. It can be shown that symmetric filters have no phase-shifting effect but that one-sided filters a s (s = 0, 1, 2,...) have such an effect. We concentrate on power-shifts.

32 Power-shift and phase-shift of a filter III From (27) we have f y (v) = A(v) 2 f x (v) (33) Filters can amplify or weaken certain frequencies in the input series x t. Filters are often classified as low-pass or high pass depending on whether high or low frequencies are amplified by the filter.

33 Effect of differencing I Let y t = x t which implies the filter a 0 = 1, a 1 = 1, a s = 0 for other values of s. A(v) 2 = 2(1 cos(2πv)) A plot starts in zero and increases in v. If x t has a root close to 1 a the v = 0, this root will be removed from the filtered series The differenced series will be more stationary than the level series itself.

34 The spectrum of ARIMA-series I We have found the PSD of an AR(1) process, cf. equation (32). When φ 1 = 1 that PSD becomes f y,rw (v) = σ 2 2(1 cos(2πv)) (34) The which is infinite near the zero-frequency and declines sharply with increasing frequency v. The simple random-walk is dominated by the long waves. The lecture note shows that if y t ARMA(p, q) so that y t ARIMA(p, 1, q)

35 The spectrum of ARIMA-series II we have f y,arima[p,1,q] (v) = f y,rw (v) f y,arma[p,q] (v) (35) Since f y,arma[p,q] (v) is finite for all frequencies, the PSD of ARIMA[p, q] will be dominated by the random-walk component f y,rw (v) which is infinite at the zero frequency. This defines what Granger called the typical spectral shape for economic time series.

36 Seasonally integrated series I Let y t be generated by: y t = y t 1 y t 2 y t 3 + ε t, ε t UIN(0, σ 2 ). (36) which we can write as S(L)y t = ε t (37) where S(L) : S(L) = 1 + L + L 2 + L 3. We can interpret S(L) as a filter.

37 Seasonally integrated series II The lecture note shows that the PSD of y t in this case is: σ2 f y (v) = A(v) 2 (38) = σ cos(2πv) + 4 cos(4πv) + 2 cos(6πv) which is infinite at v = {0, 25, 0, 5} and almost-zero elsewhere.

38 Seasonal random walk I If the generating equation is: y t = y t 4 + ε t (39) the power spectrum is: f y (v) = σ 2 2(1 cos(8πv)) (40) The graph which infinite at v = 0, 1/4, 1/2. f 4 y (v) flat in this case (white-noise by construction), as a result of the power-shift of the filter 1 L 4, which is zero at the same frequency.

39 Seasonal random walk II If we denote a unit-root by z j and the corresponding frequency by v j we have for (39): {z j, v j } = {1, 0; i, 1/2; 1, 1/2; i, 3/4}. The seasonally integrated (36) has roots {z j,v j } = { i, 1/2; 1, 1/2; i, 3/4} since S(L) = (1 + L)(1 + L 2 ).

40 Estimation I Non-parametric: The periodogram P x (v k ) can be used directly to estimate the power spectrum density, but crude relative to best practice estimation of PSD. Parametric: Estimate well specified ARIMA model and use to estimate to calculate empirical PSD analytically.