Introduction to Time Series Analysis. Lecture 18.

Transcription

1 Introduction to Time Series Analysis. Lecture Review: Spectral density, rational spectra, linear filters. 2. Frequency response of linear filters. 3. Spectral estimation 4. Sample autocovariance 5. Discrete Fourier transform and the periodogram 1

2 Review: Spectral density If a time series {X t } has autocovariance γ satisfying h= γ(h) <, then we define its spectral density as f(ν) = h= γ(h)e 2πiνh for < ν <. We have γ(h) = 1/2 1/2 e 2πiνh f(ν)dν. 2

3 Review: Rational spectra For a linear time series with MA( ) polynomial ψ, f(ν) = σ 2 w ψ ( e 2πiν) 2. If it is an ARMA(p,q), we have f(ν) = σw 2 θ(e 2πiν 2 ) φ(e 2πiν ) θ 2 q = σw 2 q j=1 e 2πiν z j 2 p j=1 e 2πiν p j 2, φ 2 p where z 1,...,z q are the zeros (roots of θ(z)) and p 1,...,p p are the poles (roots of φ(z)). 3

4 Review: Time-invariant linear filters A filter is an operator; given a time series {X t }, it maps to a time series {Y t }. A linear filter satisfies time-invariant: a t,t j = ψ j : Y t = j= a t,j X j. causal: j < 0 impliesψ j = 0. Y t = j= ψ j X t j. Y t = ψ j X t j. j=0 4

5 Time-invariant linear filters The operation ψ j X t j j= is called the convolution ofx withψ. 5

6 Time-invariant linear filters The sequence ψ is also called the impulse response, since the output{y t } of the linear filter in response to a unit impulse, 1 if t = 0, X t = 0 otherwise, is Y t = ψ(b)x t = j= ψ j X t j = ψ t. 6

7 Introduction to Time Series Analysis. Lecture Review: Spectral density, rational spectra, linear filters. 2. Frequency response of linear filters. 3. Spectral estimation 4. Sample autocovariance 5. Discrete Fourier transform and the periodogram 7

8 Frequency response of a time-invariant linear filter Suppose that {X t } has spectral density f x (ν) and ψ is stable, that is, j= ψ j <. Then Y t = ψ(b)x t has spectral density f y (ν) = ψ ( e 2πiν) 2 f x (ν). The function ν ψ(e 2πiν ) (the polynomial ψ(z) evaluated on the unit circle) is known as the frequency response or transfer function of the linear filter. The squared modulus,ν ψ(e 2πiν ) 2 is known as the power transfer function of the filter. 8

9 Frequency response of a time-invariant linear filter For stable ψ, Y t = ψ(b)x t has spectral density f y (ν) = ψ ( e 2πiν ) 2 f x (ν). We have seen that a linear process, Y t = ψ(b)w t, is a special case, since f y (ν) = ψ(e 2πiν ) 2 σ 2 w = ψ(e 2πiν ) 2 f w (ν). When we pass a time series {X t } through a linear filter, the spectral density is multiplied, frequency-by-frequency, by the squared modulus of the frequency response ν ψ(e 2πiν ) 2. This is a version of the equality Var(aX) = a 2 Var(X), but the equality is true for the component of the variance at every frequency. This is also the origin of the name filter. 9

10 Frequency response of a filter: Details Why isf y (ν) = ψ ( e 2πiν) 2 f x (ν)? First, γ y (h) = E ψ j X t j ψ k X t+h k = = j= j= j= ψ j ψ j k= k= k= ψ k E[X t+h k X t j ] ψ k γ x (h+j k) = j= ψ j l= ψ h+j l γ x (l). It is easy to check that j= ψ j < and h= γ x(h) < imply that h= γ y(h) <. Thus, the spectral density of y is defined. 10

11 Frequency response of a filter: Details f y (ν) = = = h= h= j= j= γ(h)e 2πiνh ψ j ψ h+j l γ x (l)e 2πiνh l= ψ j e 2πiνj γ x (l)e 2πiνl = ψ(e 2πiνj )f x (ν) l= h= = ψ(e 2πiνj ) 2 f x (ν). ψ h e 2πiνh h= ψ h+j l e 2πiν(h+j l) 11

12 Frequency response: Examples For a linear process Y t = ψ(b)w t, f y (ν) = ψ ( e 2πiν) 2 σ 2 w. For an ARMA model,ψ(b) = θ(b)/φ(b), so{y t } has the rational spectrum f y (ν) = σw 2 θ(e 2πiν 2 ) φ(e 2πiν ) θ 2 q = σw 2 q j=1 e 2πiν z j 2 p j=1 e 2πiν p j 2, where p j andz j are the poles and zeros of the rational function z θ(z)/φ(z). φ 2 p 12

13 Frequency response: Examples Consider the moving average Y t = 1 2k +1 k j= k X t j. This is a time invariant linear filter (but it is not causal). Its transfer function is the Dirichlet kernel ψ(e 2πiν ) = D k (2πν) = 1 k e 2πijν 2k +1 j= k 1 if ν = 0, = otherwise. sin(2π(k+1/2)ν) (2k+1) sin(πν) 13

14 Example: Moving average 1 Transfer function of moving average (k=5) ν 14

15 Example: Moving average 1 Squared modulus of transfer function of moving average (k=5) ν This is a low-pass filter: It preserves low frequencies and diminishes high frequencies. It is often used to estimate a monotonic trend component of a series. 15

16 Example: Differencing Consider the first difference Y t = (1 B)X t. This is a time invariant, causal, linear filter. Its transfer function is ψ(e 2πiν ) = 1 e 2πiν, so ψ(e 2πiν ) 2 = 2(1 cos(2πν)). 16

17 Example: Differencing 4 Transfer function of first difference ν This is a high-pass filter: It preserves high frequencies and diminishes low frequencies. It is often used to eliminate a trend component of a series. 17

18 Introduction to Time Series Analysis. Lecture Review: Spectral density, rational spectra, linear filters. 2. Frequency response of linear filters. 3. Spectral estimation 4. Sample autocovariance 5. Discrete Fourier transform and the periodogram 18

19 Estimating the Spectrum: Outline We have seen that the spectral density gives an alternative view of stationary time series. Given a realization x 1,...,x n of a time series, how can we estimate the spectral density? One approach: replace γ( ) in the definition f(ν) = h= γ(h)e 2πiνh, with the sample autocovariance ˆγ( ). Another approach, called the periodogram: compute I(ν), the squared modulus of the discrete Fourier transform (at frequencies ν = k/n). 19

20 Estimating the spectrum: Outline These two approaches are identical at the Fourier frequencies ν = k/n. The asymptotic expectation of the periodogram I(ν) isf(ν). We can derive some asymptotic properties, and hence do hypothesis testing. Unfortunately, the asymptotic variance ofi(ν) is constant. It is not a consistent estimator of f(ν). We can reduce the variance by smoothing the periodogram averaging over adjacent frequencies. If we average over a narrower range as n, we can obtain a consistent estimator of the spectral density. 20

21 Estimating the spectrum: Sample autocovariance Idea: use the sample autocovariance ˆγ( ), defined by ˆγ(h) = 1 n n h t=1 (x t+ h x)(x t x), for n < h < n, as an estimate of the autocovariance γ( ), and then use a sample version of f(ν) = γ(h)e 2πiνh, h= That is, for 1/2 ν 1/2, estimate f(ν) with ˆf(ν) = n 1 h= n+1 ˆγ(h)e 2πiνh. 21

22 Estimating the spectrum: Periodogram Another approach to estimating the spectrum is called the periodogram. It was proposed in 1897 by Arthur Schuster (at Owens College, which later became part of the University of Manchester), who used it to investigate periodicity in the occurrence of earthquakes, and in sunspot activity. Arthur Schuster, On Lunar and Solar Periodicities of Earthquakes, Proceedings of the Royal Society of London, Vol. 61 (1897), pp To define the periodogram, we need to introduce the discrete Fourier transform of a finite sequence x 1,...,x n. 22

23 Introduction to Time Series Analysis. Lecture Review: Spectral density, rational spectra, linear filters. 2. Frequency response of linear filters. 3. Spectral estimation 4. Sample autocovariance 5. Discrete Fourier transform and the periodogram 23

24 Discrete Fourier transform For a sequence (x 1,...,x n ), define the discrete Fourier transform (DFT) as (X(ν 0 ),X(ν 1 ),...,X(ν n 1 )), where X(ν k ) = 1 n n t=1 x t e 2πiν kt, and ν k = k/n (for k = 0,1,...,n 1) are called the Fourier frequencies. (Think of{ν k : k = 0,...,n 1} as the discrete version of the frequency range ν [0,1].) First, let s show that we can view the DFT as a representation ofxin a different basis, the Fourier basis. 24

25 Discrete Fourier transform Consider the space C n of vectors ofncomplex numbers, with inner product a,b = a b, where a is the complex conjugate transpose of the vector a C n. Suppose that a set {φ j : j = 0,1,...,n 1} of n vectors inc n are orthonormal: 1 ifj = k, φ j,φ k = 0 otherwise. Then these {φ j } span the vector space C n, and so for any vector x, we can write x in terms of this new orthonormal basis, x = n 1 φ j,x φ j. (picture) j=0 25

26 Discrete Fourier transform Consider the following set ofnvectors inc n : { e j = 1 ( e 2πiν j,e 2πi2ν j,...,e 2πinν ) } j : j = 0,...,n 1. n It is easy to check that these vectors are orthonormal: e j,e k = 1 n e 2πit(ν k ν j ) = 1 n (e 2πi(k j)/n) t n n t=1 t=1 1 ifj = k, = 1 n e2πi(k j)/n 1 (e 2πi(k j)/n ) n otherwise 1 e 2πi(k j)/n 1 ifj = k, = 0 otherwise, 26

27 Discrete Fourier transform where we have used the fact that S n = n t=1 αt satisfies αs n = S n +α n+1 αand sos n = α(1 α n )/(1 α) for α 1. So we can represent the real vector x = (x 1,...,x n ) C n in terms of this orthonormal basis, x = n 1 e j,x e j = j=0 n 1 j=0 X(ν j )e j. That is, the vector of discrete Fourier transform coefficients (X(ν 0 ),...,X(ν n 1 )) is the representation ofxin the Fourier basis. 27

28 Discrete Fourier transform An alternative way to represent the DFT is by separately considering the real and imaginary parts, X(ν j ) = e j,x = 1 n n t=1 = 1 n n t=1 e 2πitν j x t = X c (ν j ) ix s (ν j ), cos(2πtν j )x t i 1 n n t=1 sin(2πtν j )x t where this defines the sine and cosine transforms, X s andx c, of x. 28

29 Introduction to Time Series Analysis. Lecture Review: Spectral density, rational spectra, linear filters. 2. Frequency response of linear filters. 3. Spectral estimation 4. Sample autocovariance 5. Discrete Fourier transform and the periodogram 29