STAD57 Time Series Analysis Lecture 23 1
Spectral Representation Spectral representation of stationary {X t } is: 12 i2t Xt e du 12 1/2 1/2 for U( ) a stochastic process with independent increments du(ω)= U(ω+dω) U(ω), such that: Process U(ω) has ( ) U( ) 12 12 E du( ) 0 & Var du( ) f ( ) d 12 Var U( ) f ( v) dv F( ) 2
Frequency Domain Representation Spectral Representation Theorem: For any stationary process {X t } w/ autocovariance γ(h) there is a spectral density f(ω) such that: 1 2 i 1 2 2 h i 2 t 1 2 t 1 2 ( h) e f ( ) d & X e du ( ) where U(ω) is stochastic process w/ independent 0-mean increments & Var U( ) F( ) f ( v) dv 12 Moreover, spectral density f(ω) is given by: i2 h ( ) ( ), 1 2 1 2 f h e h 3
Frequency Domain Representation Spectral density f (ω) uniquely determines autocovariance γ (h) & vice versa Duality between time & frequency domain For real series (X t R) the spectral density f(ω) is symmetric around 0 & measures variance (i.e. strength) at frequency ω in the series f ( ) F( ) F(1/ 2) (0) 12 12 12 12 4
Example Find spectral density of 2 { Wt} ~ WN(0, w) 5
Spectral Representation of ARMA Process For causal & invertible ARMA(p,q) process 2 ( B) Xt ( B) Wt, Wt ~ WN(0, w) can show that its spectral density is given by f p ( z) 1 j 1 jz where p ( e ) ( z) 1 j 1 jz i2 2 ( e ) 2 ( ) w, i2 2 k k ratio of squared complex moduli AR & MA polynomials 6
Example Spectral density of MA(1): Xt Wt Wt 1 7
0.5 1.0 1.5 2.0 Example X W W f t t t1 w f ( ) 2.5, 1 ( ) 1.25 cos(2 ) In R: arma.spec( ma=.5, var.noise=1, log='no') MA part σ 2 w don t plot f(ω) on log-scale 8
Example Spectral density of AR(1): X X W t t 1 t 9
0.5 1.5 2.5 3.5 Example X X W f 2.5, 1 ( ) t t1 t w 1 1.25 cos(2 ) f ( ) 10
0.5 1.0 1.5 2.0 2.5 3.0 0 1 2 3 4 5 6 0 1 2 3 4 5 Example AR(2) : Xt.9 Xt 1.5Xt2Wt MA(2) : Xt Wt.6 Wt 1.6W t2 f ( ) ARMA(2,2) : X.9 X.5X t t1 t2 W.6 W.6W t t1 t2 11
-0.4 0.0 0.4-0.4 0.0 0.4 Example Signals from earthquake (eq) & explosion (ex) (eq) (ex) 0 500 1000 1500 2000 Want automatic way to distinguish the 2 phenomena based on series characteristics 0 500 1000 1500 2000 12
-0.5 0.0 0.5 1.0-0.5 0.0 0.5-0.5 0.0 0.5 1.0-0.5 0.0 0.5 1.0 Example (cont d) ACF PACF (eq) 0 20 40 60 80 100 0 20 40 60 80 100 (ex) 0 20 40 60 80 100 0 20 40 60 80 100 13
Example (cont d) Can try to use ARMA, but fitted models are hard to interpret in term of signal behavior (eq) ARIMA(3,0,4) with zero mean ar1 ar2 ar3 ma1 ma2 ma3 ma4 2.3782-1.8504 0.4543 0.3872-0.5436-0.4937-0.2900 s.e. 0.0695 0.1357 0.0688 0.0718 0.0497 0.0632 0.0567 sigma^2 estimated as 0.000144: log likelihood=1504.99 (ex) ARIMA(3,0,4) with zero mean ar1 ar2 ar3 ma1 ma2 ma3 ma4 1.9941-1.5476 0.3741-0.9165-0.6511 0.6297 0.0915 s.e. 0.1269 0.2086 0.1222 0.1326 0.1154 0.0965 0.0970 sigma^2 estimated as 0.002096: log likelihood=831.22 14
0.0 0.4 0.8 0.0 0.4 0.8 Example (cont d) Look at corresp. spectral densities instead (eq) f ( ) (ex) 15
1e-07 1e-03 1e-07 1e-03 Example (cont d) Or, more commonly, use the log-spectrum (eq) log-scale on y-axis log f ( ) (ex) 16
Parametric Spectral Estimation Spectral densities in previous example were derived from fitted ARMA model, i.e. from 2 estimated parameters ˆ,, ˆ, ˆ,, ˆ, ˆ 1 p 1 q w Called parametric spectral estimation In practice, it is often preferable to use AR instead of ARMA model AR is easier/faster to fit (w/ Yule-Walker) There is always an AR model that approximates f(ω) pretty well, but w/ possibly large order Can use AIC/BIC to select AR model order p 17
1e-07 1e-03 1e-07 1e-03 Example AR vs ARMA parametric estimates of f (ω) (eq) log f ( ) (ex) AR(12) - - ARMA(3,4) AR(26) - - ARMA(3,4) Parametric f (ω) estimation w/ AR better suited for spectra w/ lots of peaks R function: spectrum(eq, method="ar") 18
Nonparametric Spectral Estimation It is possible to avoid fitting ARMA model & estimate f (ω) directly from data instead Method is called nonparametric spectral estimation and uses the periodogram For data (x 1,,x n ) & frequencies ω j =j/n, j=0,,n 1 ( ) Discrete Fourier Transform (DFT): Periodogram: 1/2 n i2 jt d( j) n x t 1 te I( ) d( ) j j 2 j 0,..., n 1 ( ) ω j =j/n, j=0,,n 1 called Fourier or fundamental frequencies 19
Periodogram Periodogram is like sample version of f(ω) n1 ˆ i2 j h i2h j h( n1) h I( ) ( h) e & f ( ) ( h) e In particular, as n, we have: i2h [ ( )] ( ) ( ) h E I h e f I.e. periodogram I(ω j ) is unbiased estimate of spectral density f(ω), where ω j is closest fundamental frequency to ω (for #n of data) 20
1e-07 1e-03 1e-07 1e-03 Example Raw I(ω) - - ARMA f (ω) (eq) (ex) Raw I(ω) - - ARMA f (ω) R function: spec.pgram(eq,taper=0) 21
1e-07 1e-05 1e-03 1e-01 Smoothed Periodogram Raw periodogram tends to be noisy (choppy) So, typically look at smoothed periodogram To smooth I(ω j ) use kernel smoother, i.e. a weighted moving average Average I(ω j ) within each rolling window with certain weights 22
1e-07 1e-03 1e-07 1e-03 Example (eq) Smooth I(ω) - - ARMA f (ω) (ex) Smooth I(ω) - - ARMA f (ω) R function: spectrum(eq, spans = c(13,3)) smoothing params 23
Final Exam Monday Apr 29, 9am-12pm @ IC212 3hrs total, ~8 questions Aids allowed: scientific calculator one 2-sided, standard letter-sized (8½ 11) aid sheet with your own notes Will hold office hours before exam see announcements on Bb 24
Final Exam Material covered: everything in lectures 1-23 In terms of textbook: Chapter 1: sections 1.1-1.6 Chapter 2: sections 2.1-2.3 (2.2 is Regression review; just read it through) Chapter 3: sections 3.1-3.9 (for 3.3 just study the examples, not solutions to difference equations) Chapter 4: sections 4.1-4.3 Chapter 5: sections 5.2 and 5.4 25
Final Exam No R programming questions in final But you should know how to interpret R output To prepare: go over problem sets, term tests & assignments Can also look at past exams from library, but I don t have sols Note: there will be no remarks for term tests & assignments after Mon, April 22 Contact me earlier if you suspect marking error 26