STAD57 Time Series Analysis Lecture 8 1
ARMA Model Will be using ARMA models to describe times series dynamics: ( B) X ( B) W X X X W W W t 1 t1 p t p t 1 t1 q tq Model must be causal (i.e. stationary) & invertible Equivalently, all roots of φ(z)=0 & θ(z)=0 have z >1 Next, want to know the dependence structure (i.e. ACF) that we get from an ARMA model Want to match that to sample ACF from data t t 2
ACF of ARMA Model Since ARMA model is invertible, could use its causal representation to find ACF For causal process X ( B) W W, ACF is given by: t t j 0 j t j 2 j0 j0 1 Find ψ-weights from: j ( B) B ( B) ( B) If model has MA part only finite # of ψ-weights which are given by θ(β): ( B) ( B) j j If model has AR part infinite # of ψ-weights In practice, since weights are absolutely summable (go to 0 exponentially fast), can approximate ρ(h) using only a large 3 but finite number of ψ s j0 j ( h ) j j h j
Finding ψ-weights For general ARMA(p,q): ( B) ( B) ( B) q (1 B B )( B B ) (1 B B ) p 2 1 p 0 1 2 1 Solve by matching coefficients of powers of B: 0 1 1 1 0 1 2 1 1 2 0 2 First p equations have different # of terms q p1 1 p2 p1 0 p1 p 1 p1 p 0 p j 0, for jq j 1 j1 p jp 0 Remaining equations have same # of terms 4
Finding ψ-weights Rewrite last equations more succinctly as: j j, for 0 max(, 1) 1 k k jk j j p q p j 0, for max(, 1) k 1 k jk j p q where 0for j p & 0for j q j Equations can be solved by substitution: 0 1 1 1 0 1 1 1 1 ( ) 2 1 1 2 0 2 2 1 1 1 2 2 j 5
-0.5 0.0 0.5 1.0 Finding ψ-weights in R R function ARMAtoMA()calculates ψ-weights E.g. for X.7 X.3 X W.5 W.2W t t1 t2 t t1 t2 list of φ s list of θ s # of ψ s ARMAtoMA(ar=c(.7,-.3), ma=c(.5,-.2), lag.max=10) Resulting ψ-weights: (first 10 lags) 6 1 2 3 4 5 6 7 8 9 10
ACF of ARMA Model There is an easier way to find ACF of ARMA model, without having to calculate all the ψ s & then using them to find causal model s ACF Trick is to find & solve a recurrence equation (like the one for ψ s) directly in terms of ρ(h) Begin with the simpler case of a pure AR process, and then move to general ARMA 7
ACF of AR Model Consider AR(p) model: X X X W t 1 t1 p tp t Take equation for X t+h & multiply both sides by X t Now take expectations & divide both sides by γ(0)=σ 2 t to get: ( h) ( h 1) ( h p) 1 p 8
ACF of AR Model Want to solve recurrence (or difference) equation: ( h) ( h 1) ( h p), for h 1 1 Get first p values {ρ(0),...,ρ(p 1)} (initial conditions) & use substitution into equation for h=p,p+1, To get initial conditions, use ρ(0)=1 & the recurrence equations for h=1,,p 1, together with the fact that ρ( h)=ρ(h) for any h 1 This gives set of #p linear eqn s with #p unknowns, which can be solved to get {ρ(0),...,ρ(p 1)} p 9
Example Find ρ(h) for h=1,2,3 of AR: X.5 X.3X W t t1 t2 t 10
ACF of ARMA Model For general ARMA recurrence equation for γ(h) is given by: ( h) ( h 1) ( h p), for h max( p, q 1) 1 with initial conditions: Initial conditions define system of linear equations, which can be solved to find {γ(0),...,γ(max(p 1,q))} Once we know γ(0), we can find ρ(h)=γ(h)/γ(0) We only need to know ψ j for 0 j q (ψ 0 =1) p 2 ( h ) p ( h j ) q, for 0 h max( p, q 1) j1 j w jh j j h p q t j 1 j tj j 0 j tj 0 X X W ( 1) 11
ACF of ARMA Model Derivation of recurrence equation 12
13
Example Find ρ(h) for general ARMA(1,1) model: X X W W t t1 t t1 14
15
-0.5 0.0 0.5 1.0 Finding ACF in R R function ARMAacf()calculates ρ(h) E.g. for X.7 X.3 X W.5 W.2W t t1 t2 t t1 t2 list of φ s list of θ s # of ρ s ARMAacf(ar=c(.7,-.3), ma=c(.5,-.2), lag.max=20) Resulting ACF: (first 20 lags) 16
Partial ACF For pure MA(q) model, ρ(h)=0 for h>q. For both AR(p) & ARMA(p,q) models, however, ρ(h) tails off to 0 as h increases Can we distinguish AR from ARMA, as we can do with MA from ARMA based on ACF? Partial ACF (PACF) is another dependence measure that can expose differences between pure AR and ARMA models 17
Partial ACF Problem with ACF is that, even for simple AR(1) model, the correlation between X t and its past carries over to infinite lags: E.g. For X t =φχ t 1 +W t, we have (2) Cov( X, X ) Cov( X W, X ) t t2 t1 t t2 Cov( X W W, X ) (0) 2 2 t2 t1 t t2 Ideally, would like to find the relation of X t with X t+h, after removing the effect of their in between variables X t+1,,x t+h 1 18
Partial ACF For lag h, define the best linear predictors of X t+h & X t, based on {X t+1,,x t+h 1 } Xˆ X X X th 1 th1 2 th2 h1 t1 Xˆ X X X t 1 t1 2 t2 h1 th1 Predictors (i.e. β s) are best in that they minimize 2 the Mean Squared Error (MSE) E[( X Xˆ ) ] To remove effect of intermediate variables, look at correlation of ( X Xˆ ) and ( X Xˆ ) th th th th t t 19
Partial ACF The partial autocorrelation function (PACF) of a stationary time series {X t }, denoted by φ hh for h=1,2,, is defined as: Cor( X, X ) (1), and 11 t1 t Cor[( X Xˆ ),( X Xˆ )], h 2 hh th th t t We will see how to calculate the PACF later For now, we will just look at its overall behavior for AR and MA models 20
Partial ACF For pure AR(p) model X X X W t 1 t1 p tp t the PACF has the following properties: hh, for h hh pp p hh not necessarily 0, for 1h 0, for h p p For MA / ARMA model, the PACF is not necessarily 0, but tails off to 0 as h increases Heuristic proof: write MA / ARMA as AR( ) from invertibility, and use properties above p 21
-0.6-0.4-0.2 0.0 0.2 0.4 Finding ACF in R R function ARMAacf() can also calculate φ hh E.g. for X.7 X.3 X W.5 W.2W t t1 t2 t t1 t2 list of φ s list of θ s # of φ hh s option for PACF ARMAacf(ar=c(.7,-.3), ma=c(.5,-.2), 20, pacf=true) Resulting PACF: (first 20 lags) 22
Model Identification Can use properties of (sample) ACF & PACF to identify the order of an ARMA model The following table describes their behavior AR(p) MA(q) ARMA(p,q) ACF Tails off Cuts off (for h>q) Tails off PACF Cuts off (for h>p) Tails off Tails off 23
-1.0-0.5 0.0 0.5 1.0-0.8-0.6-0.4-0.2 0.0 Example Try to identify order of ARMA model with: ACF PACF 0 5 10 15 20 5 10 15 20 24
-0.2 0.0 0.2 0.4 0.6 0.8 1.0-0.2 0.0 0.2 0.4 0.6 Example Try to identify order of ARMA model with: ACF PACF 0 5 10 15 20 5 10 15 20 25
-0.2 0.0 0.2 0.4 0.6 0.8 1.0-0.2-0.1 0.0 0.1 Example Try to identify order of ARMA model with: ACF PACF 0 5 10 15 20 5 10 15 20 26