SF2943: TIME SERIES ANALYSIS COMMENTS ON SPECTRAL DENSITIES This document is meant as a complement to Chapter 4 in the textbook, the aim being to get a basic understanding of spectral densities through specific examples; this is not a replacement for the material in the Brockwell and Davis. Some of the time series we consider in this note are (1) (2) (3) (4) X t 0.7X t 1 = Z t + 0.8Z t 1, X t + 0.7X t 1 + 0.8X t 2 = Z t, X t + 0.7X t 1 = Z t, X t 0.7X t 1 = Z t, where {Z t } is a sequence of i.i.d. Gaussian random variables. The associated AR-polynomials all have roots located outside the unit circle, thus (1)-(4) are all examples of causal processes. As a preliminary investigation, we study how the linear systems defined above transform the deterministic input (z 1, z 2,... ) = (1, 0,... ) to output (x 1, x 2,... ). The result is shown in Figures 1-4. Notice for example the periodic behaviors of the AR(1) process (3) and the AR(2) process (2). 1.0 0.0 1.0 0 1 2 3 4 5 6 7 8 9 10 12 14 15 16 18 20 Figure 1. X t defined in (1) with deterministic input. Next, we use the spectral densities of (1)-(4) to (partially) understand the behaviors observed in Figures 1-4. Recall that a zero-mean stationary process with absolutely summable 1
2 SF2943: TIME SERIES ANALYSIS COMMENTS ON SPECTRAL DENSITIES 0.5 0.0 0.5 1.0 0 1 2 3 4 5 6 7 8 9 10 12 14 15 16 18 20 Figure 2. X t defined in (2) with deterministic input. 1.0 0.0 1.0 0 1 2 3 4 5 6 7 8 9 10 12 14 15 16 18 20 Figure 3. X t defined in (3) with deterministic input. acvf γ has spectral density (5) f(λ) = 1 2π h= e ihλ γ(h). Because of both sine and cosine have period 2π it suffices to consider frequencies λ ( π, π]. Moreover, we showed in class that the spectral density is a real-valued, nonnegative even function - f(λ) = f( λ) - and so it is sufficient consider λ [0, π] Using a result for time-invariant linear filters (see Section 4.3 in the textbook) we showed (Proposition 4.4.1) that a causal ARMA(p, q) process has spectral density f(λ) = σ2 θ(e iλ ) 2 2π φ(e iλ ) 2, λ [ π, π],
SF2943: TIME SERIES ANALYSIS COMMENTS ON SPECTRAL DENSITIES 3 1.0 0.0 1.0 0 1 2 3 4 5 6 7 8 9 10 12 14 15 16 18 20 Figure 4. X t defined in (4) with deterministic input. where σ 2 is the variance of the underlying noise sequence. Using this we can compute, for example, the spectral density of a causal AR(1) process: f(λ) = σ2 2π 1 1 φ 1 e iλ 2 = σ2 2π 1 (1 φ 1 cos(λ)) 2 + (φ 1 sin(λ)) 2 = σ2 2π 1 1 2φ 1 cos(λ) + φ 2. 1 We could of course also compute the spectral density from the original definition, using that the acvf is γ(h) = σ 2 φ h 1 /(1 φ2 1 ). Now, consider the two AR(1) processes (3) and (4). Both processes have φ = 0.7 and the difference is the sign; for a φ < 0 there is a periodic behavior (Figure 3) whereas for φ > 0 there is not. By studying the spectral density, we see that for φ 1 > 0 (the process (4)) the denominator in the expression for f is minimized for λ close to zero. Hence f is maximized for such low frequencies, and similarly minimized for high frequencies (λ close to π); the spectral density is shown in Figure 5. By the same arguments, for the process (3) the spectral density is large for high-frequency components (λ close to π) and small for low frequencies, see Figure 6. This suggests a period of 2π/π = 2, which is consistent with Figure 3. The spectral density for a causal AR(2) process with coefficients (φ 1, φ 2 ) is obtained completely analogous to the AR(1) case. Moreover, it is straightforward to check that the denominator of the spectral density has an extreme point in λ such that cos λ = φ 1(φ 2 1) 4φ 2.
4 SF2943: TIME SERIES ANALYSIS COMMENTS ON SPECTRAL DENSITIES 0.0 0.5 1.0 1.5 Figure 5. Spectral density f for (4). 0.0 0.5 1.0 1.5 Figure 6. Spectral density f for (3). One can then investigate either a plot of the spectral density or, more rigorous, the second derivative of the denominator to conclude whether the extreme of the denominator corresponds to a maximum or a minimum of the spectral density. The AR(2) process in (2) has φ 1 = 0.7 and φ 2 = 0.8, which amounts to a maximizing frequency λ arccos( 0.39) 1.98. This in turn corresponds to a period of 2π/λ 3, consistent with what we see in Figure 2; the spectral density is shown in Figure 7. Lastly, the spectral density of an ARMA(1, 1) process is easily found to be f(λ) = σ2 (1 + θ 2 + 2θ cos(λ)) 1 + φ 2. 2φ cos(λ)
SF2943: TIME SERIES ANALYSIS COMMENTS ON SPECTRAL DENSITIES 5 0 1 2 3 4 Figure 7. Spectral density f for (2). The spectral density for the ARMA(1, 1) process defined by (1) is shown in Figure 8. Similar to the AR(1) process (2) the spectral density is large for low frequencies, which is consistent with the lack of any periodic behavior in Figure 1. 0 1 2 3 4 5 Figure 8. Spectral density f for (1). To further illustrate the information contained in the spectral density, consider the AR(12) process (6) X t φ 12 X t 12 = Z t, where as before Z t has variance 1. A realization of this process, with φ 12 = 0.2, is shown in Figure 9. The spectral density takes the form f(λ) = σ 2 2π(1 2φ 12 cos(12λ) + φ 2 12 ),
6 SF2943: TIME SERIES ANALYSIS COMMENTS ON SPECTRAL DENSITIES 3 2 1 0 1 2 3 0 50 100 150 200 Figure 9. A realization of the process (1). 0.12 0.14 0.16 0.18 0.20 0.22 0.24 Figure 10. A realization of the process (6). which is periodic with period 2π/12 0.52; see Figure 10. We end this note by computing estimating the spectral density for the processes (1)-(4). Figures 11-14 show the raw, i.e., no smoothing, periodogram I n and the true spectral density; the periodogram, based on a sample {x 1,..., x n }, is defined as I n (λ) = 1 n 2 x t e itλ n t=1
SF2943: TIME SERIES ANALYSIS COMMENTS ON SPECTRAL DENSITIES 7 Recall that I n is not a consistent estimator of the 2πf, but we can obtain consistent estimators by using so-called discrete spectral average estimators (see Section 4.2). These are simply a particular class of smoothed versions of the periodogram and for this note we are satisfied with the raw periodogram. Series: x Raw Periodogram spectrum 1e 03 1e 01 1e+01 0.0 0.1 0.2 0.3 0.4 0.5 frequency bandwidth = 0.00144 Figure 11. Raw periodogram and spectral density for (1). spectrum 0.01 0.10 1.00 50.00 Series: x Raw Periodogram 0.0 0.1 0.2 0.3 0.4 0.5 frequency bandwidth = 0.00144 Figure 12. Raw periodogram and spectral density for (2).
8 SF2943: TIME SERIES ANALYSIS COMMENTS ON SPECTRAL DENSITIES spectrum 0.02 0.20 2.00 20.00 Series: x Raw Periodogram 0.0 0.1 0.2 0.3 0.4 0.5 frequency bandwidth = 0.00144 Figure 13. Raw periodogram and spectral density for (3). Series: x Raw Periodogram spectrum 0.01 0.10 1.00 10.00 0.0 0.1 0.2 0.3 0.4 0.5 frequency bandwidth = 0.00144 Figure 14. Raw periodogram and spectral density for (4).