14 Autoregressive Moving Average Models

Transcription

1 14 Auoregressive Moving Average Models In his chaper an imporan parameric family of saionary ime series is inroduced, he family of he auoregressive moving average, or ARMA, processes. For a large class of auocovariance funcions γ( ) i is possible o find an ARMA process {X } wih ACVF γ X ( ) such ha γ( ) is well approximaed by γ X ( ). In paricular, for any posiive ineger K, here exiss an ARMA process {X } such ha γ X (h) = γ(h) for h = 0, 1,..., K. For his ( oher) reasons, he family of ARMA processes plays a key role in he modeling of ime series daa. The linear srucure of ARMA processes also leads o a subsanial simplificaion of he general mehods for linear predicion (see Chaper 15). Example. Figure 14.1 shows differen ARMA processes wih he corresponding auocorrelaion funcion parial auocorrelaion funcion (see Secion 14.3) ARMA(1, 1) Processes We sar wih an ARMA(1, 1) process o inroduce some key properies of he auoregressive moving average processes (ARMA processes). Definiion The ime series {X } is an ARMA(1, 1) process if i is saionary for every saisfies X φx 1 = Z + θz 1, where {Z } WN(0, σ 2 ) φ + θ 0. Proposiion A saionary soluion of he ARMA(1, 1) equaion exiss if only if φ ±1. ˆ If φ < 1, hen he unique saionary soluion is given by he MA( ) process X = Z + (φ + θ) φ j 1 Z j. In his case {X } is called causal (or fuure-independen) or a causal funcion of {Z }, since X can be expressed in erms of he curren pas values Z s, s. ˆ If φ > 1, hen he unique saionary soluion is X = θφ 1 Z (φ + θ) φ j 1 Z +j. The soluion is noncausal, since {X } is a funcion of Z s, s. 14-1

2 Simulaion ARMA(0.9 / 0.5) Simulaion ARMA(-0.6 / -0.3) Simulaion ARMA(0.9,-0.4 / 0.5,-0.3) Simulaion ARMA(-0.6,-0.4 / -0.5,0.9) Simulaion ARMA(0.9,-0.6,0.4 / 0.5,-0.3) Simulaion ARMA(0.9,-0.6 / 0.5,-0.3,0.9) Figure 14.1: Simulaions of differen ARMA processes. The lef column shows an excerp (m = 96) of he whole ime series (n = 480). 14-2

3 Example. The processes X φx 1 = Z wih φ > 1 are called explosive, because he values of he ime series quickly become large in magniude. ˆ However, i is possible o modify his ime series o obain a saionary process as follows. Wrie X +1 = φx + Z +1, in which case X = φ 1 X +1 φ 1 Z +1 = φ 1 (φ 1 X +2 φ 1 Z +2 ) φ 1 Z +1. = φ k X +k k φ j Z +j by ieraing forward k seps. Because φ 1 < 1, his resul suggess he saionary fuure dependen AR(1) model X = φ j Z +j. Unforunaely, his model is useless because i requires us o know he fuure o be able o predic he fuure, i.e. X is noncausal. ˆ Neverheless, excluding explosive models from consideraion is no a problem because he models have causal counerpars. For example he wo processes X φx 1 = Z, wih φ > 1 {Z } IID N(0, σ 2 Z), Y φ 1 Y 1 = W, wih {W } IID N(0, σ 2 Zφ 2 ) are sochasically equal, i.e. all finie disribuions of he processes are he same. For example, if X 2X 1 = Z wih σ 2 Z = 1, hen Y 1 2 Y 1 = W, wih σ 2 W = 1 4, is an equivalen causal process. Jus as causaliy means ha X is expressible in erms of Z s, s, he dual concep of inveribiliy means ha Z is expressible in erms X s, s. Proposiion The ARMA(1, 1) process is ˆ inverible if θ < 1, Z is expressed in erms of X s, s, by Z = X (φ + θ) ( θ) j 1 X j, ˆ noninverible if θ > 1, Z is expressed in erms of X s, s, by Z = φθ 1 X + (φ + θ) ( θ) j 1 X +j. 14-3

4 14.2 ARMA(p, q) Processes Definiion The ime series {X } is an ARMA(p, q) process if i is saionary if for every i saisfies X φ 1 X 1... φ p X p = Z + θ 1 Z θ q Z q, (14.1) where {Z } WN(0, σ 2 ) he polynomials have no common facors. (1 φ 1 z... φ p z p ) (1 + θ 1 z θ q z q ) Example. Consider he model X φx 1 = Z φz 1 which looks like an ARMA(1, 1) process can also be wrien as (1 φb)x = (1 φb)z. Apply he operaor (1 φb) 1 o boh sides o obain X = Z. Therefore X is simply a whie noise process. The reason for his redundancy is he common facor in he polynomials (1 φz) (1 + θz). Remark. I is convenien o use he form φ(b)x = θ(b)z, where φ( ) θ( ) are he ph he qh-degree polynomials B is he backward shif operaor. φ(z) = 1 φ 1 z... φ p z p θ(z) = 1 + θ 1 z θ q z q, Definiion The process {X } is said o be an ˆ ARMA(p, q) process wih mean µ if {X µ} is an ARMA(p, q) process, ˆ AR(p) process if θ(z) 1 ˆ MA(q) process if φ(z) 1. An imporan par of Definiion is he requiremen ha {X } be saionary. For he ARMA(1, 1) we showed in Proposiion , ha a saionary soluion exiss is unique if only if φ ±1. The analogous condiion for he general ARMA(p, q) process is φ(z) = 1 φ 1 z... φ p z p 0 for all complex z wih z = 1. Complex z is used here, since he zeros of a polynomial of degree p > 1 may be eiher real or complex. The region defined by he se of complex z such ha z = 1 is referred o as he uni circle. 14-4

5 Example. Consider he ARMA(2, 1) process X 3X X 16 2 = Z + 5Z 4 1 wih {Z } WN(0, σ 2 ). The polynomial φ(z) = 1 3z z2 has zeros a z 1,2 = 2(1±i 3)/3 which lie ouside he uni circle. The process herefore is causal. On he oher h, he polynomial θ(z) = 1 + 5z has a zero a z = 4, hence he process {X 4 5 } is no inverible. Proposiion (Exisence uniqueness). A saionary soluion {X } of (14.1) exiss ( is also he unique saionary soluion) if only if φ(z) = 1 φ 1 z... φ p z p 0 for all z = 1. Proposiion (Causaliy or fuure-independence). An ARMA(p, q) process {X } is causal if here exis consans {ψ j } such ha ψ j < X = Causaliy is equivalen o he condiion ψ j Z j for all. (14.2) φ(z) = 1 φ 1 z... φ p z p 0 for all z 1. Remark. The sequence {ψ j } in (14.2) is deermined by he relaion or equivalenly by ψ j ψ(z) = ψ j z j = θ(z)/φ(z) p φ k ψ j k = θ j, j = 0, 1,..., k=1 where θ 0 := 1, θ j := 0 for j > q, ψ j := 0 for j < 0. Proposiion (Inveribiliy). An ARMA(p, q) process {X } is inverible if here exis consans {π j } such ha π j < Z = Inveribiliy is equivalen o he condiion π j X j for all. (14.3) θ(z) = 1 + θ 1 z θ q z q 0 for all z 1. Remark. The sequence {π j } in (14.3) is deermined by he equaions π j + q θ k π j k = φ j, j = 0, 1,..., k=1 where φ 0 := 1, φ j := 0 for j > p, π j := 0 for j <

6 Example. Consider he process X 4 10 X X 2 = Z + Z Z 2 or, in operaor form, (1 410 B 920 ) B2 X = (1 + B + 14 ) B2 Z. A firs, X appears o be an ARMA(2, 2) process. Bu, he associaed polynomials φ(z) = ( z)(1 0.9z) θ(z) = ( z) 2 have a common facor ha can be canceled. So he model is an ARMA(1, 1) process (1 0.9B)X = ( B)Z. I is causal because (1 0.9z) = 0 when z = 10 which is ouside he uni circle also 9 inverible because ( z) = 0 when z = 2 which is also ouside he uni circle. The causal represenaion is X = Z j 1 Z j, he inverible one is X 1.4 ( 0.5) j 1 X j = Z. Proposiion Le {X } be he ARMA(p, q) process saisfying he equaions φ(b)x = θ(b)z, {Z } WN(0, σ 2 ), where φ(z) 0 θ(z) 0 for all z = 1. Then here exis polynomials, φ(z) θ(z), nonzero for z 1, of degree p q respecively, a whie noise sequence {Z } such ha {X } saisfies he causal inverible equaion φ(b)x = θ(b)z. Proof. Define φ(z) = φ(z) r<j p θ(z) = θ(z) s<j q 1 a j z 1 a 1 j z 1 b j z 1 b 1 j z, 14-6

7 where a r+1,..., a p b s+1,..., b q are he zeros of φ(z) θ(z) which lie inside he uni circle. Since φ(z) 0 θ(z) 0 for all z 1, i suffices o show ha he process defined by Z = φ(z) θ(z) X is whie noise, i.e., ( ) ( ) {Z } WN(0, σ 2 a j 2 b k 2. r<j p r<j p Example. The ARMA process X 2X 1 = Z + 4Z 1, {Z } WN(0, σ 2 ), is neiher causal nor inverible. Inroducing φ(z) = 1 0.5z θ(z) = z, we see ha {X } has he causal inverible represenaion X 0.5X 1 = Z Z 1, {Z } WN(0, 4σ 2 ) Auocorrelaion Parial Auocorrelaion Funcion of ARMA(p, q) Processes Firs we calculae he auocovariance auocorrelaion funcion of a causal ARMA(p, q) process {X }. Secondly we define he parial auocorrelaion funcion (P) Calculaion of he Auocovariance Funcion Le φ(b)x = θ(b)z, {Z } WN(0, σ 2 ), be a causal ARMA(p, q) process. The causaliy assumpion implies ha X = ψ j Z j, (14.4) where ψ j z j = θ(z)/φ(z), z 1. (14.5) From Proposiion (14.4) we obain γ(h) = E(X +h X ) = σ 2 ψ j ψ j+ h. 14-7

8 Example. The auocovariance funcion of an ARMA(1, 1) process wih φ < 1 is given by X φx 1 = Z + θz 1 γ X (0) = σ 2 γ X (1) = σ 2 ψj 2 = σ 2 [ 1 + ] (θ + φ)2, 1 φ 2 ψ j+1 ψ j [ = σ 2 θ + φ + (θ + ] φ)2 φ, 1 φ 2 γ X (h) = φ h 1 γ(1), h 2. The calculaion of he auocorrelaion funcion is sraighforward ρ X (h) := γ(h) γ(0) Parial Auocorrelaion Funcion The parial auocorrelaion funcion, like he auocorrelaion funcion, conveys informaion regarding he dependence srucure of a saionary process. The parial auocorrelaion α(k), k 2, is he correlaion of he wo residuals obained afer regressing X k+1 X 1 on he inermediae observaions X 2,..., X k. Example. To moivae he idea of parial auocorrelaion funcion consider he causal AR(1) model, X φx 1 = Z. Then, γ X (2) = Cov(X, X 2 ) = Cov(φ 2 X 2 + φz 1 + Z, X 2 ) = φ 2 γ(0). Suppose we break his chain of dependence by removing he effec X 1. Tha is, we consider he correlaion beween X φx 1 X 2 φx 1, because i is he correlaion beween X X 2 wih he liner dependence of each on X 1 removed. In his way, we have broken he dependence chain beween X X 2. In fac, Cov(X φx 1, X 2 φx 1 ) = Cov(Z, X 2 φx 1 ) = 0. Hence, he ool we need is parial auocorrelaion, which is he correlaion beween X X s wih he linear effec of everyhing in he middle removed. 14-8

9 To formally define he parial auocorrelaion funcion for a mean-zero saionary ime series, le ˆX +h, for h 2, denoe he regression of X +h on {X +h 1, X +h 2,..., X +1 }, which we wrie as ˆX +h = β 1 X +h 1 + β 2 X +h β h 1 X +1. (14.6) No inercep is needed because he mean of X is zero. In addiion, le ˆX denoe he regression of X on {X +1, X +2,..., X +h 1 }, hen ˆX = β 1 X +1 + β 2 X β h 1 X +h 1. (14.7) Because of saionariy, he coefficiens β 1,..., β h 1 are he same in (14.6) (14.7). Definiion The parial auocorrelaion funcion α( ) of a saionary ime series is defined by α(1) = Cor(X 2, X 1 ) = ρ(1), α(k) = Cor(X k+1 ˆX k+1, X 1 ˆX 1 ), k 2. Proposiion An equivalen definiion of he parial auocorrelaion funcion on an ARMA process {X } is he funcion α( ) defined by where φ hh is he las componen of α(0) = 1 α(h) = φ hh, h 1, γ h (1) = (γ(1),..., γ(h)) Γ h = [γ(i j)] h i,. φ h = Γ 1 h γ h, (14.8) Example. For MA(1) processes, i can be shown from (14.8) ha he parial auocorrelaion funcion a lag h is α(h) = φ hh = Le lag h = 2. Recall from (13.6) ha I follows ha ( γ(0) Γ 2 = γ(1) Then ( θ) h (1 + θ θ 2h ). γ(0) = σ 2 (1 + θ 2 ) γ(1) = σ 2 θ γ(2) = 0. ) γ(1) γ(0) Γ 1 2 = φ 2 = Γ 1 2 γ 2 α(2) = 14-9 ( 1 γ 2 (0) γ 2 (1) θ θ 2 + θ 4. γ(0) γ(1) ) γ(1) γ(0)

10 Example. For causal AR(p) processes he bes linear predicor of X h+1 in erms of 1, X 1,..., X h is ˆX h+1 = φ 1 X h + φ 2 X h φ p X h+1 p. Since he coefficien φ hh of X 1 is φ p if h = p 0 if h > p, we conclude ha he parial auocorrelaion funcion α( ) of he process {X } has he properies α(p) = φ p α(h) = 0, for h > p. For h < p he values of α(h) can easily be compued from (14.8)