Calculation of ACVF for ARMA Process: I consider causal ARMA(p, q) defined by

Transcription

1 Calculation of ACVF for ARMA Process: I consider causal ARMA(p, q) defined by φ(b)x t = θ(b)z t, {Z t } WN(0, σ 2 ) want to determine ACVF {γ(h)} for this process, which can be done using four complementary methods 1st method is based on MA( ) representation X t = ψ j Z t j = ψ(b)z t, j=0 where ψ(b) = φ 1 (B)θ(B) have noted (overhead VII 8) that ACVF can be expressed as γ(h) = σ 2 ψ j ψ j+ h j=0 BD: 78, CC: 56, SS: 25 IX 1

2 Calculation of ACVF for ARMA Process: II can use recursive scheme to compute ψ j s (overhead VIII 16): p ψ j = φ k ψ j k + θ j, j = 0, 1, 2,..., k=1 but in general need ψ j s for infinite number of integers j since ψ j 0 as j, could compute ψ j s out to, say, j = J + h and use γ(h) = σ 2 J ψ j ψ j+ h σ 2 ψ j ψ j+ h, j=0 j=0 with approximation getting better with increasing J if we have a manageable expression for ψ j s (true for some processes), can get analytic expression for γ(h) BD: 78, CC: 79, SS: 93 IX 2

3 Example ARMA(1,1) Process: I for an ARMA(1,1) process, overhead VII 25 says that X t = Z t + (φ + θ) φ j 1 def Z t j = ψ j Z t j, j=1 so ψ 0 = 1 and ψ j = (φ + θ)φ j 1 for j 1 armed with j=0 x j = 1 1 x γ(0) σ 2 = ψj 2 = 1 + (φ + θ)2 j=0 j=0 (valid for x < 1), away we go: j=1 φ 2j 2 = 1 + (φ + θ) 2 φ 2j = 1 + j=0 (φ + θ)2 1 φ 2 BD: 78, CC: 78, SS: 96 IX 3

4 Example ARMA(1,1) Process: II for h > 0, with ψ j = (φ + θ)φ j 1 for j 1, have γ(h) σ 2 = ψ j ψ j+h = ψ h + ψ j ψ j+h j=0 j=1 = (φ + θ)φ h 1 + (φ + θ) 2 j=1 = (φ + θ)φ h 1 + φh (φ + θ) 2 = φ h 1 ( note: γ(h) = φγ(h 1) for h 2 φ + θ + 1 φ 2 φ(φ + θ)2 1 φ 2 ) φ 2j+h 2 BD: 78, CC: 78, SS: 96 IX 4

5 Example ARMA(1,1) Process: III following overheads show ACVFs (circles) for ARMA(1,1) processes {X t } with σ 2 = 1 AR parameter φ = 0.9 MA parameter θ ranging from 0.99 down to 0.99 have γ X (h) = φγ X (h 1) for h 2, but not for h = 1 casual AR(1) process {Y t } has ACVF γ Y (h) = φ h γ Y (0) have γ Y (h) = φγ Y (h 1) for h 1 overheads also show ACVFs (asterisks) for {Y t } with φ = 0.9 and with γ Y (0) set such that γ Y (1) = γ X (1) note: for some θ, not possible to do! IX 5

6 ACVF for ARMA(1,1) Process, φ = 0.9, θ = 0.99 ACVF h (lag) IX 6

7 ACVF for ARMA(1,1) Process, φ = 0.9, θ = 0.5 ACVF h (lag) IX 7

8 ACVF for ARMA(1,1) Process, φ = 0.9, θ = 0 ACVF h (lag) IX 8

9 ACVF for ARMA(1,1) Process, φ = 0.9, θ = 0.25 ACVF h (lag) IX 9

10 ACVF for ARMA(1,1) Process, φ = 0.9, θ = 0.5 ACVF h (lag) IX 10

11 ACVF for ARMA(1,1) Process, φ = 0.9, θ = 0.75 ACVF h (lag) IX 11

12 ACVF for ARMA(1,1) Process, φ = 0.9, θ = 0.9 ACVF h (lag) IX 12

13 ACVF for ARMA(1,1) Process, φ = 0.9, θ = 0.99 ACVF h (lag) IX 13

14 Example ARMA(1,1) Process: IV for three cases (θ = 0.25, 0.5 and 0.75), ARMA(1,1) process {X t } has an ACVF γ X (h) that can be expressed as { γ γ X (h) = Y (h) + C, h = 0; γ Y (h), h 0, where C > 0, and γ Y (h) is ACVF for AR(1) process called a nugget effect in geological literature for θ = 0.99 and 0.5, AR(1) ACVF emerges at lags h 1, while setting θ = 0 reduces ARMA(1,1) process to AR(1) θ = 0.9 reduces ARMA(1,1) to white noise θ = 0.99 causes slow decay of γ X (h) < 0 toward zero different from AR(1) ACVF when φ = γ Y (1) < 0: has γ Y (h) alternating between positive & negative as h increases IX 14

15 for an MA(q) process, have Example MA(q) Process X t = Z t + θ 1 Z t 1 + θ q Z t q = ψ j Z t j, j=0 so ψ 0 = 1, ψ j = θ j for 1 j q, and ψ j = 0 for j > q letting θ 0 = 1, have already noted (overhead VII 7) that { γ(h) = σ 2 σ 2 q h ψ j ψ j+ h = j=0 θ j θ j+ h, h q; 0, h > q j=0 consider q = 12 with θ 1 = = θ 12 = 1 (13-point sums) BD: 79, CC: 65, SS: 94 IX 15

16 ACVF for MA(12) Process, θ 1 = = θ 12 = 1 ACVF h (lag) IX 16

17 Roots of θ(z) y * * * * * * * * * * * * x IX 17

18 Realization of MA(12) Process x t t IX 18

19 Calculation of ACVF for ARMA Process: III 2nd method (of interest when p 1): multiply both sides of X t φ 1 X t 1 φ p X t p = Z t + θ 1 Z t θ q Z t q by X t k for k 0 and take expectations: γ(k) φ 1 γ(k 1) φ p γ(k p) = E{Z t X t k } + θ 1 E{Z t 1 X t k } + + θ q E{Z t q X t k } X t = since ψ j Z t j, get E{Z t l X t k } = j=0 ψ j E{Z t l Z t k j } = σ 2 ψ l k j=0 (recall that ψ l k def = 0 when l k < 0), yielding γ(k) φ 1 γ(k 1) φ p γ(k p) = σ 2 q θ l ψ l k l=0 BD: 79, SS: 95 IX 19

20 Calculation of ACVF for ARMA Process: IV since ψ j = 0 for j < 0, right-hand side of γ(k) φ 1 γ(k 1) φ p γ(k p) = σ 2 q θ l ψ l k l=0 will be 0 when k > l for all l = 0,..., q, i.e., when k q + 1 if k p so that k p 0, then γ(k), γ(k 1),..., γ(k p) on left-hand side will involve p + 1 distinct elements of {γ(h)} thus, letting m = max{q + 1, p}, we have γ(k) φ 1 γ(k 1) φ p γ(k p) = 0, k = m, m + 1,... theory of homogeneous linear difference equations says that, if p roots z j of φ(z) = 0 are distinct, then γ(h) = α 1 z h 1 + α 2 z h α p z h p, h m p BD: 79, SS: 95 IX 20

21 Calculation of ACVF for ARMA Process: V to determine α j s for given z j s, plug γ(h) s expressible as γ(h) = α 1 z1 h + α 2 z2 h + + α p zp h into q γ(k) φ 1 γ(k 1) φ p γ(k p) = σ 2 θ l ψ l k, 0 k < m, yielding a system of m linear equations to be solved for m unknowns, where the unknowns are either l=0 α 1,..., α p when m = p or α 1,..., α p, γ(0),..., γ(m p 1) when m = q + 1 > p (recall that this method is only of interest when p 1) note: need to recall recursive scheme for computing ψ j s given φ j s and θ j s (see overhead VIII 16) BD: 79, SS: 95 IX 21

22 Example ARMA(1,1) Process: V for ARMA(1,1) process X t φx t 1 = Z t + θz t 1, q γ(k) φ 1 γ(k 1) φ p γ(k p) = σ 2 θ l ψ l k l=0 becomes γ(k) φγ(k 1) = σ 2 (ψ k + θψ 1 k ), k = 0, 1,... overhead VII 24: ψ j = 0, j < 0; ψ 0 = 1 & ψ 1 = φ + θ, so γ(0) φγ(1) = σ 2 (1 + θψ 1 ) = σ 2 (1 + θ[φ + θ]) (1) γ(1) φγ(0) = σ 2 θ (2) γ(k) φγ(k 1) = 0, k = 2, 3,... (3) root z of φ(z) = 1 φz is 1/φ, so γ(h) = αz h = αφ h, h 1 using γ(1) = αφ in (1) and (2) yields two linear equations to be solved to get unknowns α and γ(0) BD: 79, 80, SS: IX 22

23 Example ARMA(1,1) Process: VI two equations are thus matrix formulation [ is ] [ ] 1 φ 2 γ(0) φ φ α γ(0) αφ 2 = σ 2 (1 + θ[φ + θ]) αφ φγ(0) = σ 2 θ = σ 2 = [ σ 2 (1 + θ[φ + θ]) σ 2 θ assuming φ 0 (i.e., ARMA(1,1) is not an MA(1)), have [ ] [ ] [ γ(0) 1 φ φ 2 σ = 2 ] (1 + θ[φ + θ]) α φ(1 φ 2 ) φ 1 σ 2 θ 1 + (φ+θ)2 1 φ 2 1+θ(φ+θ) 1 φ 2 + θ φ(1 φ 2 ) ] BD: 79, 80, SS: IX 23

24 Example ARMA(1,1) Process: VII thus γ(0) (φ + θ)2 = 1 + σ2 1 φ 2 in agreement with expression obtained by 1st method (cf. overhead IX 3) hurray!!! substituting value for α into γ(h) = αφ h yields ( ) γ(h) σ 2 = φ h 1 + θ(φ + θ) θ 1 φ 2 + φ(1 φ 2, ) which, after some algebra, ( becomes ) γ(h) σ 2 = φ h 1 φ(φ + θ)2 φ + θ + 1 φ 2, the same expression we got using 1st method (see overhead IX 4) we re really on a roll!!! BD: 79, 80, SS: IX 24

25 Example AR(2) Process: I for causal AR(2) process X t φ 1 X t 1 φ 2 X t 1 = Z t, q γ(k) φ 1 γ(k 1) φ p γ(k p) = σ 2 θ l ψ l k becomes γ(k) φ 1 γ(k 1) φ 2 γ(k 2) = σ 2 ψ k, k = 0, 1,... since ψ 0 = 1 while ψ j = 0 when j < 0, get l=0 γ(0) φ 1 γ(1) φ 2 γ(2) = σ 2 γ(k) φ 1 γ(k 1) φ 2 γ(k 2) = 0, k = 1, 2,... assuming roots z 1 & z 2 of φ(z) = 0 are such that z 1 z 2, solution takes form γ(h) = α 1 z h 1 + α 2 z h 2, h 0 BD: 80 81, CC: 72, SS: IX 25

26 Example AR(2) Process: II substituting γ(h) = α 1 z h 1 + α 2 z h 2 into γ(0) φ 1 γ(1) φ 2 γ(2) = σ 2 γ(1) φ 1 γ(0) φ 2 γ(1) = 0 leads to ( ) ( ) α 1 + α 2 φ 1 α 1 z1 1 + α 2 z2 1 φ 2 α 1 z1 2 + α 2 z2 2 ( ) α 1 z1 1 + α 2 z2 1 φ 1 (α 1 + α 2 ) φ 2 α 1 z1 1 + α 2 z2 1 collecting terms gives ( ) ( ) α 1 1 φ 1 z1 1 φ 2 z1 2 + α 2 1 φ 1 z2 1 φ 2 z2 2 ( ) ( ) α 1 z1 1 φ 1 φ 2 z1 1 + α 2 z2 1 φ 1 φ 2 z2 1 = σ 2 = 0 = σ 2 = 0 BD: 80 81, CC: 72, SS: IX 26

27 Example AR(2) Process: III in matrix form, we have [ 1 φ 1 z1 1 φ 2 z1 2 1 φ 1 z2 1 φ 2 z2 2 z1 1 φ 1 φ 2 z1 1 z2 1 φ 1 φ 2 z2 1 now ] [ α1 α 2 ] = [ σ 2 φ(z) = 1 φ 1 z φ 2 z 2 = (1 z )(1 z ) ( 1 = )z+ z2 z 1 z 2 z 1 z 2 z 1 z 2 tells us that φ 1 = z1 1 + z2 1 and φ 2 = z1 1 z 1 2, which yields ] [ 1 z1 2 z1 1 z z1 3 z z2 2 z1 1 z z1 1 ] z 3 2 α1 z2 1 + z1 2 z 1 2 z1 1 + z1 1 = z 2 α 2 2 solving above for α 1 and α 2 yields solutions in terms of σ 2 and roots z 1 and z 2 0 ] [ σ 2 0 ] BD: 80 81, CC: 72, SS: IX 27

28 Example AR(2) Process: IV plugging solutions for α 1 and α 2 into γ(h) = α 1 z1 h + α 2 z2 h yields (after a considerable amount of reduction!) σ 2 z 2 [ ] γ(h) = 1 z2 2 z 1 h 1 (z 1 z 2 1)(z 2 z 1 ) z1 2 1 z1 h 2 z2 2 1 for complex conjugate roots z 1 = re iω and z 2 = z 1, have γ(h)/γ(0) = r h sin(hω + ψ)/ sin(ψ), where σ 2 (r 6 + r 4 ) γ(0) = (r 2 1)(r 4 2r 2 cos(2ω) + 1) and tan(ψ) = r2 + 1 r 2 1 tan(ω) note: r > 1 (roots are assumed to be outside unit circle) ( ) is damped sinusoid with period 2π ω, with damping slow when r 1, i.e., when roots are close to unit circle ( ) BD: 80 81, CC: 72, SS: IX 28

29 ACVF for AR(2) Process with z 1 = 2 & z 2 = 5 ACVF h (lag) BD: 80 IX 29

30 Reciprocal Roots Plot for z 1 = 2 & z 2 = 5 y x IX 30

31 Realization of AR(2) Process x t t IX 31

32 ACVF for AR(2) Process with z 1 = 10 9 & z 2 = 2 ACVF h (lag) BD: 81 IX 32

33 Reciprocal Roots Plot for z 1 = 10 9 & z 2 = 2 y x IX 33

34 Realization of AR(2) Process x t t IX 34

35 ACVF for AR(2) Process with z 1 = 10 9 & z 2 = 2 ACVF h (lag) BD: 81 IX 35

36 Reciprocal Roots Plot for z 1 = 10 9 & z 2 = 2 y x IX 36

37 Realization of AR(2) Process t x t IX 37

38 ACVF for AR(2) Process with z 1 = i 2 3 & z 2 = z 1 ACVF h (lag) BD: 82 IX 38

39 Reciprocal Roots Plot for z 1 = i 2 3 & z 2 = z 1 y x IX 39

40 Realization of AR(2) Process t x t IX 40

41 ACVF for AR(2) Process, z 1. = i & z2 = z 1 ACVF h (lag) IX 41

42 Reciprocal Roots Plot for z 1. = i & z2 = z 1 y x IX 42

43 Realization of AR(2) Process t x t IX 43

44 Calculation of ACVF for ARMA Process: VI 3rd method uses equations derived for 2nd method (IX 19): q γ(k) φ 1 γ(k 1) φ p γ(k p) = σ 2 θ l ψ l k, k 0 ( ) l=0 letting k = 0, 1,..., p gives following system of equations: q γ(0) φ 1 γ(1) φ p γ(p) = σ 2 def θ l ψ l = c 0 γ(1) φ 1 γ(0) φ p γ(p 1) = σ 2 γ(p) φ 1 γ(p 1) φ p γ(0) = σ 2. l=0 q l=0 q l=0 θ l ψ l 1 def = c 1 θ l ψ l p def = c p BD: 81 IX 44

45 Calculation of ACVF for ARMA Process: VII leads to following matrix equation: 1 φ 1 φ 2 φ p 1 φ p φ 1 1 φ 2 φ 3 φ p 0 φ 2 φ 1 φ 3 1 φ φ p 1 φ p 2 φ p φ p φ p φ p 1 φ p 2 φ 1 1 solving above gives ACVF for lags 0 to p γ(0) γ(1) γ(2). γ(p 1) γ(p) lags k p + 1 can be gotten recursively by rearranging ( ): q γ(k) = φ 1 γ(k 1) + + φ p γ(k p) + σ 2 θ l ψ l k l=0 = c 0 c 1 c 2. c p 1 c p BD: 81 IX 45

46 Example ARMA(1,1) Process: VIII as noted before, for ARMA(1,1) process, γ(k) φ 1 γ(k 1) φ p γ(k p) = σ 2 q θ l ψ l k l=0 becomes γ(k) φ 1 γ(k 1) = σ 2 (ψ k + θψ 1 k ), yielding γ(0) φγ(1) = σ 2 (1 + θψ 1 ) = σ 2 (1 + θ[φ + θ]) γ(1) φγ(0) = σ 2 θ γ(k) φγ(k 1) = 0, k = 2, 3,... can get γ(0) & γ(1) by solving [ ] [ ] 1 φ γ(0) = φ 1 γ(1) [ σ 2 (1 + θ[φ + θ]) σ 2 θ remaining values gotten from γ(k) = φγ(k 1), k = 2, 3,... ] BD: IX 46

47 Calculation of ACVF for ARMA Process: VIII 4th method uses the fact that an ARMA(p, q) process can be created by filtering an AR(p) process starting with the representation φ(b)x t = θ(b)z t, note that causality allows us to write X t = φ 1 (B)θ(B)Z t = θ(b)φ 1 (B)Z t = θ(b)y t, where Y t def = φ 1 (B)Z t since we can also write φ(b)y t = Z t, it follows that {Y t } is an AR(p) process, from which we can get {X t } by subjecting {Y t } to the MA filter θ(b) = 1 + θ 1 B + + θ q B q IX 47

48 Calculation of ACVF for ARMA Process: IX recall that, if {Y t } is a stationary process with mean 0 and ACVF {γ Y (h)}, then q def X t = θ j Y t j, where, as usual, θ 0 = 1, j=0 is stationary with mean 0 and ACVF q q γ X (h) = θ j θ k γ Y (h + k j) j=0 k=0 (the above follows readily from overhead VII 4) hence, if we can get ACVF for AR(p) process, we can readily compute ACVF for ARMA(p, q) process IX 48

49 Calculation of ACVF for ARMA Process: X reconsider equations derived for 2nd method (IX 19), namely, q γ(k) φ 1 γ(k 1) φ p γ(k p) = σ 2 θ l ψ l k, k 0 l=0 and specialize them for AR(p) case (i.e., q = 0): γ(k) φ 1 γ(k 1) φ p γ(k p) = σ 2 ψ k, k 0, where ψ 0 = 1, while ψ k = 0 for all k > 0 IX 49

50 Calculation of ACVF for ARMA Process: XI leads to following matrix equation (special case of 3rd method): 1 φ 1 φ 2 φ p 1 φ p γ Y (0) φ 1 1 φ 2 φ 3 φ p 0 γ Y (1) φ 2 φ 1 φ 3 1 φ γ Y (2) = φ p 1 φ p 2 φ p φ p γ Y (p 1) φ p φ p 1 φ p 2 φ 1 1 γ Y (p) after solving above to get γ Y (k) for lags 0 to p, can get it for lags k p + 1 recursively from γ Y (k) = φ 1 γ Y (k 1) + + φ p γ Y (k p) exercise: use this approach to get ARMA(1,1) γ X (h) note: after discussion of Levinson Durbin recursions, can formulate a 5th method that is a variation on the 4th σ IX 50

51 Calculation of ACVF for ARMA Process Summary 1st method can lead to analytic expression based directly on γ(h) = σ 2 ψ j ψ j+ h j=0 (if not, gives easy way to calculate γ(h) approximately) 2nd method based on γ(h) φ 1 γ(h 1) φ p γ(h p) = σ 2 q θ l ψ l h l=0 (gives analytic expression and/or exact calculation of γ(h)) 3rd method is variation on 2nd (starts with same equations) 4th method gets γ(h) via two-stage procedure using idea that ARMA process is result of filtering AR process with MA filter IX 51