Introduction to Time Series Analysis. Lecture 19.

Transcription

1 Introduction to Time Series Analysis. Lecture Review: Spectral density estimation, sample autocovariance. 2. The periodogram and sample autocovariance. 3. Asymptotics of the periodogram. 1

2 Estimating the Spectrum: Outline We have seen that the spectral density gives an alternative view of stationary time series. Given a realization x 1,...,x n of a time series, how can we estimate the spectral density? One approach: replace γ( ) in the definition f(ν) = h= γ(h)e 2πiνh, with the sample autocovariance ˆγ( ). Another approach, called the periodogram: compute I(ν), the squared modulus of the discrete Fourier transform (at frequencies ν = k/n). 2

3 Estimating the spectrum: Outline These two approaches are identical at the Fourier frequencies ν = k/n. The asymptotic expectation of the periodogram I(ν) isf(ν). We can derive some asymptotic properties, and hence do hypothesis testing. Unfortunately, the asymptotic variance ofi(ν) is constant. It is not a consistent estimator of f(ν). 3

4 Review: Spectral density estimation If a time series {X t } has autocovariance γ satisfying h= γ(h) <, then we define its spectral density as f(ν) = h= γ(h)e 2πiνh for < ν <. 4

5 Review: Sample autocovariance Idea: use the sample autocovariance ˆγ( ), defined by ˆγ(h) = 1 n n h (x t+ h x)(x t x), for n < h < n, as an estimate of the autocovariance γ( ), and then use ˆf(ν) = n 1 h= n+1 ˆγ(h)e 2πiνh for 1/2 ν 1/2. 5

6 Discrete Fourier transform For a sequence (x 1,...,x n ), define the discrete Fourier transform (DFT) as (X(ν 0 ),X(ν 1 ),...,X(ν n 1 )), where X(ν k ) = 1 n n x t e 2πiν kt, and ν k = k/n (for k = 0,1,...,n 1) are called the Fourier frequencies. (Think of{ν k : k = 0,...,n 1} as the discrete version of the frequency range ν [0,1].) First, let s show that we can view the DFT as a representation ofxin a different basis, the Fourier basis. 6

7 Discrete Fourier transform Consider the space C n of vectors ofncomplex numbers, with inner product a,b = a b, where a is the complex conjugate transpose of the vector a C n. Suppose that a set {φ j : j = 0,1,...,n 1} of n vectors inc n are orthonormal: 1 ifj = k, φ j,φ k = 0 otherwise. Then these {φ j } span the vector space C n, and so for any vector x, we can write x in terms of this new orthonormal basis, x = n 1 φ j,x φ j. (picture) j=0 7

8 Discrete Fourier transform Consider the following set ofnvectors inc n : { e j = 1 ( e 2πiν j,e 2πi2ν j,...,e 2πinν ) } j : j = 0,...,n 1. n It is easy to check that these vectors are orthonormal: e j,e k = 1 n e 2πit(ν k ν j ) = 1 n (e 2πi(k j)/n) t n n 1 ifj = k, = 1 n e2πi(k j)/n 1 (e 2πi(k j)/n ) n otherwise 1 e 2πi(k j)/n 1 ifj = k, = 0 otherwise, 8

9 Discrete Fourier transform where we have used the fact that S n = n αt satisfies αs n = S n +α n+1 αand sos n = α(1 α n )/(1 α) for α 1. So we can represent the real vector x = (x 1,...,x n ) C n in terms of this orthonormal basis, x = n 1 e j,x e j = j=0 n 1 j=0 X(ν j )e j. That is, the vector of discrete Fourier transform coefficients (X(ν 0 ),...,X(ν n 1 )) is the representation ofxin the Fourier basis. 9

10 Discrete Fourier transform An alternative way to represent the DFT is by separately considering the real and imaginary parts, X(ν j ) = e j,x = 1 n n = 1 n n e 2πitν j x t = X c (ν j ) ix s (ν j ), cos(2πtν j )x t i 1 n n sin(2πtν j )x t where this defines the sine and cosine transforms, X s andx c, of x. 10

11 Periodogram The periodogram is defined as I(ν j ) = X(ν j ) 2 = 1 n n 2 e 2πitν j x t = X 2 c(ν j )+X 2 s(ν j ). X c (ν j ) = 1 n n X s (ν j ) = 1 n n cos(2πtν j )x t, sin(2πtν j )x t. 11

12 Periodogram SinceI(ν j ) = X(ν j ) 2 for one of the Fourier frequencies ν j = j/n (for j = 0,1,...,n 1), the orthonormality of thee j implies that we can write n 1 n 1 x x = X(ν j )e j X(ν j )e j j=0 j=0 = n 1 j=0 X(ν j ) 2 = n 1 j=0 I(ν j ). For x = 0, we can write this as ˆσ 2 x = 1 n n x 2 t = 1 n n 1 j=0 I(ν j ). 12

13 Periodogram This is the discrete analog of the identity σ 2 x = γ x (0) = 1/2 1/2 f x (ν)dν. (Think ofi(ν j ) as the discrete version off(ν) at the frequency ν j = j/n, and think of(1/n) ν j as the discrete version of ν dν.) 13

14 Estimating the spectrum: Periodogram Why is the periodogram at a Fourier frequency (that is,ν = ν j ) the same as computingf(ν) from the sample autocovariance? Almost the same they are not the same at ν 0 = 0 when x 0. But if either x = 0, or we consider a Fourier frequency ν j with j {1,...,n 1},... 14

15 Estimating the spectrum: Periodogram I(ν j ) = 1 n = 1 n = 1 n n 2 e 2πitν j x t = 1 2 n e 2πitν j (x n t x) ( n )( n ) e 2πitν j (x t x) e 2πitν j (x t x) e 2πi(s t)ν j (x s x)(x t x) = s,t n 1 h= n+1 ˆγ(h)e 2πihν j, where the fact that ν j 0 implies n e 2πitν j = 0 (we showed this when we were verifying the orthonormality of the Fourier basis) has allowed us to subtract the sample mean in that case. 15

16 Asymptotic properties of the periodogram We want to understand the asymptotic behavior of the periodogram I(ν) at a particular frequency ν, as n increases. We ll see that its expectation converges tof(ν). We ll start with a simple example: Suppose that X 1,...,X n are i.i.d.n(0,σ 2 ) (Gaussian white noise). From the definitions, X c (ν j ) = 1 n n cos(2πtν j )x t, X s (ν j ) = 1 n n we have that X c (ν j ) and X s (ν j ) are normal, with EX c (ν j ) = EX s (ν j ) = 0. sin(2πtν j )x t, 16

17 Asymptotic properties of the periodogram Also, Var(X c (ν j )) = σ2 n = σ2 2n n cos 2 (2πtν j ) n (cos(4πtν j )+1) = σ2 2. Similarly, Var(X s (ν j )) = σ 2 /2. 17

18 Asymptotic properties of the periodogram Also, Cov(X c (ν j ),X s (ν j )) = σ2 n = σ2 2n Cov(X c (ν j ),X c (ν k )) = 0 Cov(X s (ν j ),X s (ν k )) = 0 Cov(X c (ν j ),X s (ν k )) = 0. n cos(2πtν j )sin(2πtν j ) n sin(4πtν j ) = 0, for any j k. 18

19 Asymptotic properties of the periodogram That is, if X 1,...,X n are i.i.d.n(0,σ 2 ) (Gaussian white noise; f(ν) = σ 2 ), then thex c (ν j ) and X s (ν j ) are all i.i.d.n(0,σ 2 /2). Thus, 2 σ 2I(ν j) = 2 σ 2 ( X 2 c (ν j )+X 2 s(ν j ) ) χ 2 2. So for the case of Gaussian white noise, the periodogram has a chi-squared distribution that depends on the variance σ 2 (which, in this case, is the spectral density). 19

20 Asymptotic properties of the periodogram Under more general conditions (e.g., normal {X t }, or linear process {X t } with rapidly decaying ACF), thex c (ν j ),X s (ν j ) are all asymptotically independent andn(0,f(ν j )/2). Consider a frequency ν. For a given value of n, let ˆν (n) be the closest Fourier frequency (that is, ˆν (n) = j/n for a value ofj that minimizes ν j/n ). Asnincreases, ˆν (n) ν, and (under the same conditions that ensure the asymptotic normality and independence of the sine/cosine transforms), f(ˆν (n) ) f(ν). (picture) In that case, we have 2 f(ν) I(ˆν(n) ) = 2 f(ν) ( X 2 c(ˆν (n) )+X 2 s(ˆν (n) )) d χ

21 Asymptotic properties of the periodogram Thus, EI(ˆν (n) ) = f(ν) ( 2 2 E f(ν) f(ν) 2 E(Z2 1 +Z 2 2) = f(ν), ( ) ) Xc(ˆν 2 (n) )+Xs(ˆν 2 (n) ) where Z 1,Z 2 are independent N(0,1). Thus, the periodogram is asymptotically unbiased. 21

22 Introduction to Time Series Analysis. Lecture Review: Spectral density estimation, sample autocovariance. 2. The periodogram and sample autocovariance. 3. Asymptotics of the periodogram. 22