Module 4 Stationary Time Series Models Part 1 MA Models and Their Properties Class notes for Statistics 451: Applied Time Series Iowa State University Copyright 2015 W. Q. Meeker. February 14, 2016 20h 58min 4-1
Module 4 Segment 1 ARMA Notation, Conventions, Expectations, and other Preliminaries 4-2
Deviations from the Sample Mean The sample mean is computed as µ = z = nt=1 z t n We use the centered realization z t z in the computation of many statistics. For example σ Z = γ 0 = S Z = nt=1 (z t z) 2 n 1 Also used in computing sample autocovariances and autocorrelations. Subtracting out the mean ( z) does not change the sample variance ( γ 0 ), sample autocovariances ( γ k ), or sample autocorrelations ( ρ k ) of a realization. 4-3
Change in Business Inventories 1955-1969 and Centered Change in Business Inventories 1955-1969 Change in Inventories Billions of Dollars -5 0 5 10 1955 1957 1959 1961 1963 1965 1967 1969 Year Centered Change in Inventories Billions of Dollars -10 0 5 10 1955 1957 1959 1961 1963 1965 1967 1969 Year 4-4
Deviations from the Process Mean The stochastic process is Z 1,Z 2,... The process mean is µ = E(Z t ). Some derivations are simplified by using Ż t = Z t µ to compute model properties (only the mean has changed). In particular, Similarly, E(Ż t ) = E(Z t µ) = E(Z t ) E(µ) = µ µ = 0 γ 0 = Var(Ż t ) = Var(Z t µ) = Var(Z t )+Var(µ) = Var(Z t ) = σ 2 Z and γ k = Cov(Ż t,ż t+k ) = Cov(Z t µ,z t+k µ) = Cov(Z t,z t+k ) ρ k = γ k /γ 0 4-5
Backshift Operator Notation Backshift operators: BZ t = Z t 1 B 2 Z t = BBZ t = BZ t 1 = Z t 2 B 3 Z t = Z t 3, etc. If C is a constant, then BC = C Differencing operators: (1 B)Z t = Z t BZ t = Z t Z t 1 (1 B) 2 Z t = (1 2B+B 2 )Z t = Z t 2Z t 1 +Z t 2 Seasonal differencing operators: (1 B 4 )Z t = Z t B 4 Z t = Z t Z t 4 (1 B 12 )Z t = Z t B 12 Z t = Z t Z t 12 4-6
Polynomial Operator Notation φ(b) = φ p (B) (1 φ 1 B φ 2 B 2 φ p B p ) is useful for expressing the AR(p) model. For example, with p = 2, an AR(2) model. or Other polynomial operators: φ 2 (B)Ż t = a t (1 φ 1 B φ 2 B 2 )Ż t = a t Ż t φ 1 Ż t 1 φ 2 Ż t 2 = a t Ż t = φ 1 Ż t 1 +φ 2 Ż t 2 +a t θ(b) = θ q (B) (1 θ 1 B θ 2 B 2 θ q B q ) [as in Ż t = θ q (B)a t ] ψ(b) = ψ (B) (1+ψ 1 B+ψ 2 B 2 + ) [as in Ż t = ψ(b)a t ] π(b) = π (B) (1 π 1 B π 2 B 2 ) [as in π(b)ż t = a t ] 4-7
General ARMA(p,q) Model in Terms of Ż t Ż t Z t µ φ p (B)Ż t = θ q (B)a t (1 φ 1 B φ 2 B 2 φ p B p )Ż t = (1 θ 1 B θ 2 B 2 θ q B q )a t Ż t φ 1 Ż t 1 φ 2 Ż t 2 φ p Ż t p = a t θ 1 a t 1 θ 2 a t 2 θ q a t q or Ż t = φ 1 Ż t 1 +φ 2 Ż t 2 + +φ p Ż t p θ 1 a t 1 θ 2 a t 2 θ q a t q +a t 4-8
Mean (µ) and Constant Term (θ 0 ) for an ARMA(2,2) Model Replace Ż t with Ż t = Z t µ and solve for Z t =. For example, using p = 2 and q = 2, giving an ARMA(2,2) model: and Ż t = φ 1 Ż t 1 +φ 2 Ż t 2 θ 1 a t 1 θ 2 a t 2 +a t Z t µ = φ 1 (Z t 1 µ)+φ 2 (Z t 2 µ) θ 1 a t 1 θ 2 a t 2 +a t Z t = µ φ 1 µ φ 2 µ+φ 1 Z t 1 +φ 2 Z t 2 θ 1 a t 1 θ 2 a t 2 +a t Z t = θ 0 +φ 1 Z t 1 +φ 2 Z t 2 θ 1 a t 1 θ 2 a t 2 +a t where θ 0 = µ φ 1 µ φ 2 µ = µ(1 φ 1 φ 2 ) is the ARMA model constant term. Also, E(Z t ) = µ = θ 0 /(1 φ 1 φ 2 ). Note that E(Z t ) = µ = θ 0 for an MA(q) model. 4-9
Module 4 Segment 2 Properties of the MA(1) Model and the PACF 4-10
Simulated MA(1) Data with θ 1 = 0.9,σ a = 1 plot(arima.sim(n=250, model=list(ma=0.90)), ylab= ) 4 2 0 2 0 50 100 150 200 250 Time 4-11
Simulated MA(1) Data with θ 1 = 0.9,σ a = 1 plot(arima.sim(n=250, model=list(ma=-0.90)), ylab= ) 4 2 0 2 4 0 50 100 150 200 250 Time 4-12
Mean and Variance of the MA(1) Model Model: Z t = θ 0 θ 1 a t 1 +a t, a t nid(0,σ 2 a) Mean: µ Z E(Z t ) = E(θ 0 θ 1 a t 1 +a t ) = θ 0 +0+0 = θ 0. Variance: γ 0 σz 2 t Var(Z t ) E[(Z t µ Z ) 2 ] = E(Żt 2) γ 0 = E(Żt 2) = E[( θ 1 a t 1 +a t ) 2 ] = E[(θ1 2a2 t 1 2θ 1 a t 1 a t + a 2 t )] = θ1 2E(a2 t 1 ) 2θ 1E(a t 1 a t ) + E(a 2 t ) = θ1 2σ2 a 0 + σa 2 = (θ1 2 +1)σ2 a 4-13
Autocovariance and Autocorrelation Functions for the MA(1) Model Autocovariance: γ k Cov(Z t,z t+k ) = E[(Z t µ Z )(Z t+k µ Z )] = E(Ż t Ż t+k ) γ 1 E(Ż t Ż t+1 ) = E[( θ 1 a t 1 +a t )( θ 1 a t +a t+1 )] = E(θ 2 1 a t 1a t θ 1 a t 1 a t+1 θ 1 a 2 t + a t a t+1 ) = 0 0 θ 1 σ 2 a + 0 = θ 1 σ 2 a Thus ρ 1 = γ 1 γ 0 = θ 1 (1+θ 2 1 ). Using similar operations, it is easy to show that γ 2 E(Ż t Ż t+2 ) = 0 and thus ρ 2 = γ 2 /γ 0 = 0. In general, for MA(1), ρ k = γ k /γ 0 = 0 for k > 1. 4-14
Partial Autocorrelation Function For any model, the process PACF φ kk can be computed from the process ACF from ρ 1,ρ 2,... using the recursive formula from Durbin (1960): where φ 1,1 = ρ 1 φ kk = ρ k 1 k 1 j=1 k 1 j=1 φ k 1, j ρ k j φ k 1, j ρ j, k = 2,3,... φ kj = φ k 1,j φ kk φ k 1,k j (k = 3,4,...; j = 1,2,...,k 1) This formula is based on the solution of the Yule-Walker equations (covered later). Also, the sample PACF can be computed from the sample ACF. That is, φ kk is a function of ρ 1, ρ 2,... 4-15
Hints For Using Durbin s formula φ 11 = ρ 1 depends on ρ 1 φ 22 = ρ 2 φ 11 ρ 1 1 φ 11 ρ 1 depends on ρ 1,ρ 2 and φ 11 φ 33 depends on ρ 1,ρ 2,ρ 3,φ 21, and φ 22 φ 44 depends on ρ 1,ρ 2,ρ 3,ρ 4,φ 31,φ 32, and φ 33 φ 55 depends on ρ 1,ρ 2,ρ 3,ρ 4,ρ 5,φ 41,φ 42,φ 43, and φ 44 and so on. 4-16
True ACF and PACF for MA(1) Model with θ 1 = 0.95 True ACF ma: 0.95 True ACF -1.0 0.0 1.0 True PACF ma: 0.95 True PACF -1.0 0.0 1.0 4-17
True ACF and PACF for MA(1) Model with θ 1 = 0.95 True ACF ma: -0.95 True ACF -1.0 0.0 1.0 True PACF ma: -0.95 True PACF -1.0 0.0 1.0 4-18
Simulated Realization (MA(1),θ 1 = 0.95,n = 75) Graphical Output from Function iden Simulated data w= Data Range-Mean Plot w -3-2 -1 0 1 2 0 10 20 30 40 50 60 70 time ACF Range 3.5 4.0 4.5 5.0 ACF Partial ACF -1.0 0.0 0.5 1.0-1.0 0.0 0.5 1.0 PACF -0.8-0.6-0.4-0.2 0.0 0.2 Mean 4-19
Simulated Realization (MA(1),θ 1 = 0.95,n = 300) Graphical Output from Function iden Simulated data w= Data Range-Mean Plot w -2 0 2 0 50 100 150 200 250 300 time ACF Range 3 4 5 6 ACF Partial ACF -1.0 0.0 0.5 1.0-1.0 0.0 0.5 1.0 PACF -0.1 0.0 0.1 0.2 Mean 4-20
Simulated Realization (MA(1),θ 1 = 0.95,n = 75) Graphical Output from Function iden Simulated data w= Data Range-Mean Plot w -2 0 2 4 0 10 20 30 40 50 60 70 time ACF Range 1 2 3 4 5 6 ACF Partial ACF -1.0 0.0 0.5 1.0-1.0 0.0 0.5 1.0 PACF -0.6-0.4-0.2 0.0 0.2 0.4 0.6 Mean 4-21
Simulated Realization (MA(1),θ 1 = 0.95,n = 300) Graphical Output from Function iden Simulated data w= Data Range-Mean Plot w -4-2 0 2 4 0 50 100 150 200 250 300 time ACF Range 2 3 4 5 6 ACF Partial ACF -1.0 0.0 0.5 1.0-1.0 0.0 0.5 1.0 PACF -0.5 0.0 0.5 1.0 Mean 4-22
Module 4 Segment 3 Inverting the MA(1) Model and MA(q) Properties 4-23
Geometric Power Series 1 (1 θ 1 B) = (1 θ 1B) 1 = (1+θ 1 B+θ 2 1 B2 + ) This expansion provides a convenient method for re-expressing parts of some ARMA models. When θ 1 < 1 the series converges. Similarly, 1 (1 φ 1 B) =(1 φ 1B) 1 = (1+φ 1 B+φ 2 1 B2 + ) Again, when φ 1 < 1 the series converges. 4-24
Using the Geometric Power Series to Re-express the MA(1) Model as an Infinite AR Ż t = (1 θ 1 B)a t a t = (1 θ 1 B) 1 Ż t a t = (1+θ 1 B+θ1 2 B2 + )Ż t Ż t = θ 1 Ż t 1 θ1żt 2 2 +a t Thus the MA(1) can be expressed as an infinite AR model. If 1 < θ 1 < 1, then the weight on the old observations is decreasing with age (having more practical meaning). This is the condition of invertibility for an MA(1) model. 4-25
Using the Back-substitution to Re-express the MA(1) Model as an Infinite AR Ż t = θ 1 a t 1 +a t a t 1 = Ż t 1 +θ 1 a t 2 a t 2 = Ż t 2 +θ 1 a t 3 a t 3 = Ż t 3 +θ 1 a t 4 Substituting, successively, a t 1,a t 2,a t 3..., shows that Ż t = θ 1 Ż t 1 θ 2 1Żt 2 +a t This method works, more generally, for higher-order MA(q) models, but the algebra is tedious. 4-26
Notes on the MA(1) Model Given ρ 1 = γ 1 γ = θ 1 0 (1+θ1 2 we can see ) 0.5 ρ 1 0.5 Solving the ρ 1 = quadratic equation for θ 1 gives 1 θ 1 = 2ρ ± 1 1 (2ρ 1 ) 2 1 ρ 1 0 0 ρ 1 = 0 The two solutions are related by θ 1 = 1 θ 1 For any 0.5 ρ 1 0.5, the solution with 1 < θ 1 < 1 is the invertible parameter value. Substituting ρ 1 for ρ 1 provides an estimator for θ 1. 4-27
Mean, Variance, and Covariance of the MA(q) Model Model: Z t = θ 0 +θ q (B)a t = θ 0 θ 1 a t 1 θ q a t q +a t Mean: µ Z E(Z t ) = E(θ 0 θ 1 a t 1 θ q a t q +a t ) = θ 0. Variance: γ 0 Var(Z t ) E[(Z t µ Z ) 2 ] = E(Ż 2 ) Centered MA(q): Ż t = θ q (B)a t = θ 1 a t 1 θ q a t q +a t Var(Z t ) γ 0 = E(Ż 2 t ) = (1+θ 2 1 + +θ2 q)σ 2 a Cov(Z t,z t+k ) = γ k = E(Ż t Ż t+k ) = ρ k = γ k γ 0 4-28
Re-expressing the MA(q) Model as an Infinite AR More generally, any MA(q) model can be expressed as Ż t = θ q (B)a t = (1 θ 1 B θ 2 B 2 θ q B)a t 1 θ q (B)Żt = π(b)ż t = (1 π 1 B π 2 B 2...)Ż t = a t Ż t = π 1 Ż t 1 +π 2 Ż t 2 + +a t = k=1 π k Ż t k +a t Values of π 1,π 2,... depend on θ 1,...,θ q. For the model to have practical meaning, the π j values should not remain large as j gets large. Formally, the invertibility condition is met if i=1 π j < Invertibility of an MA(q) model can be checked by finding the roots of the polynomial θ q (B) (1 θ 1 B θ 2 B 2 θ q B q ) = 0. All q roots must lie outside of the unit-circle. 4-29
Module 4 Segment 4 Checking Invertibility and Polynomial Roots 4-30
Checking the Invertibility of an MA(2) Model Invertibility of an MA(2) model can be checked by finding the roots of the polynomial θ 2 (B) = (1 θ 1 B θ 2 B 2 ) = 0. Both roots must lie outside of the unit-circle. Roots have the form B = θ 1 ± θ 2 1 +4θ 2 2θ 2 z = x+iy where i = 1. A root is outside of the unit circle if z = x 2 +y 2 > 1 This method works for any q, but for q > 2 it is best to use numerical methods to find the q roots. 4-31
Roots of (1 1.5B 0.4B 2 ) = 0 arma.root(c(1.5, 0.4)) Polynomial Function versus B Coefficients= 1.5, 0.4 Roots and the Unit Circle Function -20-15 -10-5 0 Imaginary Part -4-2 0 2 4-8 -6-4 -2 0 B -4-2 0 2 4 Real Part 4-32
Roots of the Quadratic (1 0.5B+0.9B 2 ) = 0 and the Unit Circle Unit Circle Coefficients= 0.5, 0.9 Imaginary Part 1.5 1.0 0.5 0.0 0.5 1.0 1.5 1.5 1.0 0.5 0.0 0.5 1.0 1.5 Real Part 4-33
Roots of (1 0.5B+0.9B 2 ) = 0 arma.root(c(0.5, 0.9)) Polynomial Function versus B Coefficients= 0.5, -0.9 Roots and the Unit Circle Function 0.94 0.96 0.98 1.00 1.02 Imaginary Part -1.5-0.5 0.5 1.5 0.0 0.2 0.4 0.6-1.5-0.5 0.5 1.5 B Real Part 4-34
Roots of (1 0.5B+0.9B 2 0.1B 3 0.5B 4 ) = 0 arma.root(c(0.5, 0.9, 0.1, 0.5)) Polynomial Function versus B Coefficients= 0.5, -0.9, 0.1, 0.5 Roots and the Unit Circle Function -100-80 -60-40 -20 0 Imaginary Part -1 0 1-4 -2 0 2 B -1 0 1 Real Part 4-35
Checking the Invertibility of an MA(2) Model Simple Rule Both roots lying outside of the unit-circle implies θ 2 +θ 1 < 1 θ 2 < 1 θ 1 θ 2 θ 1 < 1 θ 2 < 1+θ 1 1 < θ 2 < 1 1 < θ 2 < 1 and the inequalities define the MA(2) triangle. 4-36
Module 4 Segment 5 MA(2) Model Properties 4-37
Autocovariance and Autocorrelation Functions for the MA(2) Model γ 1 Cov(Z t,z t+1 ) = E[(Z t µ Z )(Z t+1 µ Z )] = E(Ż t Ż t+1 ) Thus = E[( θ 1 a t 1 θ 2 a t 2 +a t )( θ 1 a t θ 2 a t 1 +a t+1 )] = E(θ 1 θ 2 a 2 t 1 θ 1a 2 t)+0+0+0+0+0+0+0 = θ 1 θ 2 E(a 2 t 1 ) θ 1E(a 2 t) = (θ 1 θ 2 θ 1 )σ 2 a ρ 1 = γ 1 = θ 1θ 2 θ 1 γ 0 1+θ1 2. +θ2 2 Using similar operations, it is easy to show that ρ 2 = γ 2 θ = 2 γ 0 1+θ1 2 +θ2 2 and that, in general, for MA(2), ρ k = γ k /γ 0 = 0 for k > 2. 4-38
True ACF and PACF for MA(2) Model with θ 1 = 0.78,θ 2 = 0.2 True ACF ma: 0.78, 0.2 True ACF -1.0 0.0 1.0 True PACF ma: 0.78, 0.2 True PACF -1.0 0.0 1.0 4-39
True ACF and PACF for MA(2) Model with θ 1 = 1,θ 2 = 0.95 True ACF ma: 1, -0.95 True ACF -1.0 0.0 1.0 True PACF ma: 1, -0.95 True PACF -1.0 0.0 1.0 4-40
Simulated Realization (MA(2),θ 1 = 0.78,θ 2 = 0.2,n = 75) Graphical Output from Function iden Simulated data w= Data Range-Mean Plot w -3-2 -1 0 1 2 0 10 20 30 40 50 60 70 time ACF Range 3.8 4.0 4.2 4.4 4.6 4.8 ACF Partial ACF -1.0 0.0 0.5 1.0-1.0 0.0 0.5 1.0 PACF -0.04-0.02 0.0 0.02 0.04 0.06 0.08 Mean 4-41
Simulated Realization (MA(2),θ 1 = 0.78,θ 2 = 0.2,n = 300) Graphical Output from Function iden Simulated data w= Data Range-Mean Plot w -2 0 2 4 0 50 100 150 200 250 300 time ACF Range 3 4 5 6 7 ACF Partial ACF -1.0 0.0 0.5 1.0-1.0 0.0 0.5 1.0 PACF -0.2-0.1 0.0 0.1 0.2 Mean 4-42
Simulated Realization (MA(2),θ 1 = 1,θ 2 = 0.95,n = 75) Graphical Output from Function iden Simulated data w= Data Range-Mean Plot w -2 0 2 4 0 10 20 30 40 50 60 70 time ACF Range 4 5 6 7 ACF Partial ACF -1.0 0.0 0.5 1.0-1.0 0.0 0.5 1.0 PACF -0.2 0.0 0.2 0.4 Mean 4-43
Simulated Realization (MA(2),θ 1 = 1,θ 2 = 0.95,n = 300) Graphical Output from Function iden Simulated data w= Data Range-Mean Plot w -6-4 -2 0 2 4 0 50 100 150 200 250 300 time ACF Range 4 6 8 10 ACF Partial ACF -1.0 0.0 0.5 1.0-1.0 0.0 0.5 1.0 PACF -0.6-0.4-0.2 0.0 0.2 0.4 0.6 Mean 4-44
MA(2) Triangle Relating ρ 1 and ρ 2 to θ 1 and θ 2 4-45
MA(2) Triangle Showing ACF and PACF Functions 4-46