Stationary Time Series Models Part 1 MA Models and Their Properties Class notes for Statistics 41: Applied Time Series Ioa State University Copyright 1 W. Q. Meeker. Segment 1 ARMA Notation, Conventions, Expectations, and other Preliminaries February 14, 16 h 8min 4-1 4- Deviations from the Sample The sample mean is computed as Change in Business Inventories 19-1969 and Centered Change in Business Inventories 19-1969 Change in Inventories nt=1 z t µ = z = n We use the centered realization z t z in the computation of many statistics. For example nt=1 (z t z) σ Z = γ = S Z = n 1 Billions of Dollars - 1 19 197 199 1961 1963 196 1967 1969 Year Centered Change in Inventories Also used in computing sample autocovariances and autocorrelations. Subtracting out the mean ( z) does not change the sample variance ( γ ), sample autocovariances ( γ k ), or sample autocorrelations ( ρ k ) of a realization. Billions of Dollars -1 1 19 197 199 1961 1963 196 1967 1969 Year 4-3 4-4 Deviations from the Process The stochastic process is Z 1,Z,... The process mean is µ = E(Z t ). Some derivations are simplified by using Ż t = Z t µ to compute model properties (only the mean has changed). Backshift Operator Notation Backshift operators: BZ t = Z t 1 B Z t = BBZ t = BZ t 1 = Z t B 3 Z t = Z t 3, etc. In particular, E(Ż t ) = E(Z t µ) = E(Z t ) E(µ) = µ µ = Similarly, γ = Var(Ż t ) = Var(Z t µ) = Var(Z t )+Var(µ) = Var(Z t ) = σz and γ k = Cov(Ż t,ż t+k ) = Cov(Z t µ,z t+k µ) = Cov(Z t,z t+k ) ρ k = γ k /γ If C is a constant, then BC = C Differencing operators: (1 B)Z t = Z t BZ t = Z t Z t 1 (1 B) Z t = (1 B+B )Z t = Z t Z t 1 +Z t Seasonal differencing operators: (1 B 4 )Z t = Z t B 4 Z t = Z t Z t 4 (1 B 1 )Z t = Z t B 1 Z t = Z t Z t 1 4-4-6
Polynomial Operator Notation φ(b) = φ p(b) (1 φ 1 B φ B φ pb p ) is useful for expressing the AR(p) model. For example, ith p =, an AR() model. φ (B)Ż t = a t (1 φ 1 B φ B )Ż t = a t General ARMA(p,q) Model in Terms of Ż t Ż t Z t µ φ p(b)ż t = θ q(b)a t (1 φ 1 B φ B φ pb p )Ż t = or Other polynomial operators: Ż t φ 1 Ż t 1 φ Ż t = a t Ż t = φ 1 Ż t 1 +φ Ż t +a t θ(b) = θ q(b) (1 θ 1 B θ B θ qb q ) [as in Ż t = θ q(b)a t ] ψ(b) = ψ (B) (1+ψ 1 B+ψ B + ) [as in Ż t = ψ(b)a t ] π(b) = π (B) (1 π 1 B π B ) [as in π(b)ż t = a t ] (1 θ 1 B θ B θ qb q )a t Ż t φ 1 Ż t 1 φ Ż t φ p Ż t p = a t θ 1 a t 1 θ a t θ qa t q or Ż t = φ 1 Ż t 1 +φ Ż t + +φ p Ż t p θ 1 a t 1 θ a t θ qa t q +a t 4-7 4-8 (µ) and Constant Term (θ ) for an ARMA(,) Model Replace Ż t ith Ż t = Z t µ and solve for Z t =. For example, using p = and q =, giving an ARMA(,) model: and Ż t = φ 1 Ż t 1 +φ Ż t θ 1 a t 1 θ a t +a t Z t µ = φ 1 (Z t 1 µ)+φ (Z t µ) θ 1 a t 1 θ a t +a t Segment Properties of the MA(1) Model and the P Z t = µ φ 1 µ φ µ+φ 1 Z t 1 +φ Z t θ 1 a t 1 θ a t +a t Z t = θ +φ 1 Z t 1 +φ Z t θ 1 a t 1 θ a t +a t here θ = µ φ 1 µ φ µ = µ(1 φ 1 φ ) is the ARMA model constant term. Also, E(Z t ) = µ = θ /(1 φ 1 φ ). Note that E(Z t ) = µ = θ for an MA(q) model. 4-9 4-1 Simulated MA(1) Data ith θ 1 =.9,σ a = 1 plot(arima.sim(n=, model=list(ma=.9)), ylab= ) Simulated MA(1) Data ith θ 1 =.9,σ a = 1 plot(arima.sim(n=, model=list(ma=-.9)), ylab= ) 4 4 4 1 1 Time 1 1 Time 4-11 4-1
and Variance of the MA(1) Model Model: Z t = θ θ 1 a t 1 +a t, a t nid(,σa) : µ Z E(Z t ) = E(θ θ 1 a t 1 +a t ) = θ ++ = θ. Variance: γ σz t Var(Z t ) E[(Z t µ Z ) ] = E(Żt ) γ = E(Żt ) = E[( θ 1 a t 1 +a t ) ] = E[(θ1 a t 1 θ 1 a t 1 a t + a t )] = θ1 E(a t 1 ) θ 1E(a t 1 a t ) + E(a t ) = θ1 σ a + σa = (θ1 +1)σ a 4-13 Autocovariance and Autocorrelation Functions for the MA(1) Model Autocovariance: γ k Cov(Z t,z t+k ) = E[(Z t µ Z )(Z t+k µ Z )] = E(Ż t Ż t+k ) γ 1 E(Ż t Ż t+1 ) = E[( θ 1 a t 1 +a t )( θ 1 a t +a t+1 )] = E(θ 1 a t 1a t θ 1 a t 1 a t+1 θ 1 a t + a t a t+1 ) = θ 1 σ a + = θ 1 σ a Thus ρ 1 = γ 1 γ = θ 1 (1+θ 1 ). Using similar operations, it is easy to sho that γ E(Ż t Ż t+ ) = and thus ρ = γ /γ =. In general, for MA(1), ρ k = γ k /γ = for k > 1. 4-14 Partial Autocorrelation Function For any model, the process P φ kk can be computed from the process from ρ 1,ρ,... using the recursive formula from Durbin (196): here φ 1,1 = ρ 1 k 1 ρ k φ k 1, j ρ k j j=1 φ kk =, k =,3,... k 1 1 φ k 1, j ρ j j=1 φ kj = φ k 1,j φ kk φ k 1,k j (k = 3,4,...; j = 1,,...,k 1) Hints For Using Durbin s formula φ 11 = ρ 1 depends on ρ 1 φ = ρ φ 11 ρ 1 1 φ 11 ρ 1 depends on ρ 1,ρ and φ 11 φ 33 depends on ρ 1,ρ,ρ 3,φ 1, and φ φ 44 depends on ρ 1,ρ,ρ 3,ρ 4,φ 31,φ 3, and φ 33 This formula is based on the solution of the Yule-Walker equations (covered later). Also, the sample P can be computed from the sample. That is, φ kk is a function of ρ 1, ρ,... 4-1 φ depends on ρ 1,ρ,ρ 3,ρ 4,ρ,φ 41,φ 4,φ 43, and φ 44 and so on. 4-16 True and P for MA(1) Model ith θ 1 =.9 True and P for MA(1) Model ith θ 1 =.9 True ma:.9 True ma: -.9 True True P ma:.9 True P True True P ma: -.9 True P 4-17 4-18
-3 - -1 1 Simulated Realization (MA(1),θ 1 =.9,n = 7) 1 3 4 6 7 - Plot - Simulated Realization (MA(1),θ 1 =.9,n = 3) - Plot 1 1 3 3. 4. 4.. Partial P 3 4 6 Partial P -.8 -.6 -.4 -... -.1..1. 4-19 4- Simulated Realization (MA(1),θ 1 =.9,n = 7) Simulated Realization (MA(1),θ 1 =.9,n = 3) - 4-4 - 4 1 3 4 6 7 1 1 3 - Plot - Plot 1 3 4 6 Partial P 3 4 6 Partial P -.6 -.4 -....4.6 -... 1. 4-1 4- Geometric Poer Series Segment 3 Inverting the MA(1) Model and MA(q) Properties 1 (1 θ 1 B) = (1 θ 1B) 1 = (1+θ 1 B+θ 1 B + ) This expansion provides a convenient method for re-expressing parts of some ARMA models. When θ 1 < 1 the series converges. Similarly, 1 (1 φ 1 B) =(1 φ 1B) 1 = (1+φ 1 B+φ 1 B + ) Again, hen φ 1 < 1 the series converges. 4-3 4-4
Using the Geometric Poer Series to Re-express the MA(1) Model as an Infinite AR Using the Back-substitution to Re-express the MA(1) Model as an Infinite AR Ż t = (1 θ 1 B)a t a t = (1 θ 1 B) 1 Ż t a t = (1+θ 1 B+θ 1 B + )Ż t Ż t = θ 1 Ż t 1 θ 1Żt +a t Thus the MA(1) can be expressed as an infinite AR model. If 1 < θ 1 < 1, then the eight on the old observations is decreasing ith age (having more practical meaning). This is the condition of invertibility for an MA(1) model. Ż t = θ 1 a t 1 +a t a t 1 = Ż t 1 +θ 1 a t a t = Ż t +θ 1 a t 3 a t 3 = Ż t 3 +θ 1 a t 4 Substituting, successively, a t 1,a t,a t 3..., shos that Ż t = θ 1 Ż t 1 θ 1Żt +a t This method orks, more generally, for higher-order MA(q) models, but the algebra is tedious. 4-4-6 Notes on the MA(1) Model Given ρ 1 = γ 1 γ = θ 1 (1+θ1 e can see ), Variance, and Covariance of the MA(q) Model Model: Z t = θ +θ q(b)a t = θ θ 1 a t 1 θ qa t q +a t. ρ 1. Solving the ρ 1 = quadratic equation for θ 1 gives 1 θ 1 = ρ ± 1 1 (ρ 1 ) 1 ρ 1 ρ 1 = The to solutions are related by θ 1 = 1 θ 1 For any. ρ 1., the solution ith 1 < θ 1 < 1 is the invertible parameter value. Substituting ρ 1 for ρ 1 provides an estimator for θ 1. : µ Z E(Z t ) = E(θ θ 1 a t 1 θ qa t q +a t ) = θ. Variance: γ Var(Z t ) E[(Z t µ Z ) ] = E(Ż ) Centered MA(q): Ż t = θ q(b)a t = θ 1 a t 1 θ qa t q +a t Var(Z t ) γ = E(Ż t ) = (1+θ 1 + +θ q)σ a Cov(Z t,z t+k ) = γ k = E(Ż t Ż t+k ) = ρ k = γ k γ 4-7 4-8 Re-expressing the MA(q) Model as an Infinite AR More generally, any MA(q) model can be expressed as Ż t = θ q(b)a t = (1 θ 1 B θ B θ qb)a t 1 θ q(b)żt = π(b)ż t = (1 π 1 B π B...)Ż t = a t Ż t = π 1 Ż t 1 +π Ż t + +a t = π k Ż t k +a t k=1 Values of π 1,π,... depend on θ 1,...,θ q. For the model to have practical meaning, the π j values should not remain large as j gets large. Formally, the invertibility condition is met if π j < i=1 Invertibility of an MA(q) model can be checked by finding the roots of the polynomial θ q(b) (1 θ 1 B θ B θ qb q ) =. All q roots must lie outside of the unit-circle. 4-9 Segment 4 Checking Invertibility and Polynomial Roots 4-3
Checking the Invertibility of an MA() Model Invertibility of an MA() model can be checked by finding the roots of the polynomial θ (B) = (1 θ 1 B θ B ) =. Both roots must lie outside of the unit-circle. Roots of (1 1.B.4B ) = arma.root(c(1.,.4)) Polynomial Function versus B Coefficients= 1.,.4 Roots and the Unit Circle Roots have the form B = θ 1 ± θ1 +4θ θ z = x+iy here i = 1. A root is outside of the unit circle if z = x +y > 1 This method orks for any q, but for q > it is best to use numerical methods to find the q roots. Function - -1-1 - -8-6 -4 - B Imaginary Part -4-4 -4-4 Real Part 4-31 4-3 Roots of the Quadratic (1.B+.9B ) = and the Unit Circle Roots of (1.B+.9B ) = arma.root(c(.,.9)) Imaginary Part 1. 1.... 1. 1. Unit Circle Coefficients=.,.9 Function.94.96.98 1. 1. Polynomial Function versus B...4.6 B Coefficients=., -.9 Imaginary Part -1. -.. 1. Roots and the Unit Circle -1. -.. 1. Real Part 1. 1.... 1. 1. Real Part 4-33 4-34 Roots of (1.B+.9B.1B 3.B 4 ) = arma.root(c(.,.9,.1,.)) Polynomial Function versus B Coefficients=., -.9,.1,. Roots and the Unit Circle Checking the Invertibility of an MA() Model Simple Rule Function -1-8 -6-4 - -4 - Imaginary Part -1 1-1 1 Both roots lying outside of the unit-circle implies θ +θ 1 < 1 θ < 1 θ 1 θ θ 1 < 1 θ < 1+θ 1 1 < θ < 1 1 < θ < 1 and the inequalities define the MA() triangle. B Real Part 4-3 4-36
Segment MA() Model Properties Autocovariance and Autocorrelation Functions for the MA() Model γ 1 Cov(Z t,z t+1 ) = E[(Z t µ Z )(Z t+1 µ Z )] = E(Ż t Ż t+1 ) Thus = E[( θ 1 a t 1 θ a t +a t )( θ 1 a t θ a t 1 +a t+1 )] = E(θ 1 θ a t 1 θ 1a t)+++++++ = θ 1 θ E(a t 1 ) θ 1E(a t) = (θ 1 θ θ 1 )σ a ρ 1 = γ 1 = θ 1θ θ 1 γ 1+θ1. +θ Using similar operations, it is easy to sho that ρ = γ θ = γ 1+θ1 +θ and that, in general, for MA(), ρ k = γ k /γ = for k >. 4-37 4-38 True and P for MA() Model ith θ 1 =.78,θ =. True and P for MA() Model ith θ 1 = 1,θ =.9 True ma:.78,. True ma: 1, -.9 True True P ma:.78,. True P True True P ma: 1, -.9 True P 4-39 4-4 Simulated Realization (MA(),θ 1 =.78,θ =.,n = 7) Simulated Realization (MA(),θ 1 =.78,θ =.,n = 3) -3 - -1 1-4 1 3 4 6 7 1 1 3 - Plot 3.8 4. 4. 4.4 4.6 4.8 -.4 -....4.6.8 Partial P 3 4 6 7 - Plot Partial 1 1 P -. -.1..1. 4-41 4-4
1 3.4 4 6 1 P 1 1 1 7 4 - -.6 -.4 - Plot.. 1.4 1.6 1 P 1 3 1 4-44 1 Simulated Realization (MA(), θ1 = 1, θ =.9, n = 3) -. 4-46 MA() Triangle Shoing and P Functions 1.... - Plot. 4 Simulated Realization (MA(), θ1 = 1, θ =.9, n = 7) -. 4-43 MA() Triangle Relating ρ1 and ρ to θ1 and θ 4-4 Partial 1. -4-6 1 8 6 4-1. 1... -1. Partial.. -1. 1... -1. - 7 6 4