Ch 4. Models For Stationary Time Series. Time Series Analysis

Transcription

1

2 This chapter discusses the basic concept of a broad class of stationary parametric time series models the autoregressive moving average (ARMA) models. Let {Y t } denote the observed time series, and {e t } represent an unobserved white noise series (i.i.d. r.v.s with zero mean.) Assumptions for the models: 1 {Y t } is stationary and zero mean. (If {Y t } has a nonzero mean µ, we may replace {Y t } by {Y t µ} to get a zero mean series. e.g., Y t µ = (Y t 1 µ) 0.24(Y t 2 µ) + e t. ) 2 e t is independent to Y t k (thus E(e t Y t k ) = 0) for k = 1, 2, 3,. One important issue is the pattern of the autocorrelation function {ρ k }, which will be estimated by the sample autocorrelation function {r k } to build appropriate models in later chapters. The {ρ k } could be solved recursively by the Yule-Walker equations.

3 4.1 General Linear Processes Def. A general linear process, {Y t }, is one that can be represented as a weighted linear combination of (finite or infinite) present and past white noise terms as Y t = e t + Ψ 1 e t 1 + Ψ 2 e t 2 +. Let Ψ 0 = 1. We assume that Var (Y t ) = i=0 Ψ2 i < to make the process meaningful.

4 Ex. An important example is Ψ i = φ i for a given φ ( 1, 1). So Y t = e t + φe t 1 + φ 2 e t 2 + We have = e t + φ(e t 1 + φe t 2 + ) = e t + φy t 1 (an AR(1) ). E(Y t ) = 0, ( ) ( ) Var (Y t ) = Var φ i e t i = φ 2i Var (e t i ) = σe 2 φ 2i i=0 Since Y t = φy t 1 + e t, and Y t 1 and e t are independent, we get i=0 Cov (Y t, Y t 1 ) = Cov (φy t 1 + e t, Y t 1 ) = φvar (Y t 1 ) = φσ2 e 1 φ 2, [ ] [ ] φσ 2 Corr (Y t, Y t 1 ) = e σ 2 1 φ 2 / e 1 φ 2 = φ. Similarly, we get i=0 Cov (Y t, Y t k ) = φk σ 2 e 1 φ 2, Corr (Y t, Y t k ) = φ k. Clearly, {Y t } is stationary. = σ2 e 1 φ 2

5 In general, Every general linear process (with finite Y t variance) Y t = e t + Ψ 1 e t 1 + Ψ 2 e t 2 + is stationary, with E(Y t ) = 0, ( ) γ k = Cov (Y t, Y t k ) = σe 2 Ψ i Ψ i+k, k 0. i=0

6 4.2 Moving Average Processes Def. A general linear process with only finite nonzero terms Y t = e t θ 1 e t 1 θ 2 e t 2 θ q e t q is called a moving average process of order q (abbr. MA(q).) Remarks. 1 We change notation from Ψ s to θ s in MA processes. 2 The R software uses + signs before the θ s.

7 MA(1) Process Y t = e t θe t 1 By direct computation, E(Y t ) = 0, Var (Y t ) = σe(1 2 + θ 2 ), Cov (Y t, Y t 1 ) = Cov (e t θe t 1, e t 1 θe t 2 ) = θσe, 2 Cov (Y t, Y t k ) = Cov (e t θe t 1, e t k θe t k 1 ) = 0, for k 2. Important fact: the MA(1) process has no correlation beyond lag 1. Theorem 1 For a MA(1) model Y t = e t θe t 1, E(Y t ) = 0, γ 0 = σe(1 2 + θ 2 ), γ 1 = θσe, 2 ρ 1 = θ 1 + θ 2, γ k = ρ k = 0 for k 2.

8 Ex. Exhibit 4.1 displays a graph of the lag 1 autocorrelation values for θ ranging from -1 to +1. rho=function(theta){-theta/(1+theta 2)} # Define rho as a function with variable theta plot(rho,xlim=c(-1,1),ylab=expression(rho[1]), xlab=expression(theta), main=expression(paste( Lag 1 Autocorrelation of an MA(1) Process for Different, theta))) R code explanations: 1 ylab=expression(rho[1]) specifies that the y label is ρ 1. Similarly for the expression commands in xlab and main. 2 Try?legend or?plotmath for more about typesetting or plotting a formula. See TS-ch4.R

9 Ex. An MA(1) series with MA coefficient equal to θ 1 = 0.9 and of length n = 100 can be simulated as follow (see TS-ch4.R): set.seed(12345) # initializes the seed of the random number generator to reproduce a simulation. y=arima.sim(model=list(ma=-c(-0.9)),n=100) # simulate a realization of size 100 of an MA(1) model with θ 1 = 0.9 R code explanations: 1 The arima.sim function simulates a time series from a given ARIMA model passed into the function as a list that contains the AR and MA parameters as vectors. 2 R uses a plus convention in parameterizing the MA part, so we have to add a minus sign before the vector of MA values to agree with our parameterization. 3 A list object consists of a list of components, each of which contains data with possibly different data structures. The elements of a list are ordered according to the order they are entered. The list is the most flexible data structure in R.

10 The plot shows a moderately strong positive correlation at lag 1.

11 The plot shows a moderately strong upward trend.

12

13 The dataset ma1.1.s is simulated by a MA(1) process with θ = 0.9. We compute that ρ 1 = The plot shows a moderately strong negative correlation at lag 1.

14

15

16 4.2.2 MA(2) Process Y t = e t θ 1 e t 1 θ 2 e t 2 We compute that γ 0 = Var (Y t ) = Var (e t θ 1 e t 1 θ 2 e t 2 ) = (1 + θ1 2 + θ2)σ 2 e, 2 γ 1 = Cov (Y t, Y t 1 ) = ( θ 1 + θ 1 θ 2 )σe, 2 γ 2 = Cov (Y t, Y t 2 ) = θ 2 σe, 2 γ k = 0 for k > 2. Theorem 2 For the MA(2) model Y t = e t θ 1 e t 1 θ 2 e t 2, ρ 1 = θ 1 + θ 1 θ θ1 2 +, ρ 2 = θ2 2 and ρ k = 0 for k = 3, 4, θ θ1 2 +, θ2 2

17 Ex 4.8. (Time Plot of an MA(2) Process with θ 1 = 1 and θ 2 = 0.6) The dataset ma2.s is simulated from a MA(2) process with θ 1 = 1 and θ 2 = 0.6, that is, Y t = e t e t e t 2. We have ρ 1 = and ρ 2 =

18 The scatterplot apparently reflects the negative autocorrelation at lag 1.

19 The plot shows a weak positive autocorrelation at lag 2.

20 The plot suggest the lack of autocorrelation at lag 3.

21 4.2.3 The General MA(q) Process Theorem 3 For the MA(q) process Y t = e t θ 1 e t 1 θ 2 e t 2 θ q e t q, we have γ 0 = (1 + θ θ θ 2 q)σ 2 e, ρ k = { θk +θ 1 θ k+1 +θ 2 θ k+2 + +θ q k θ q 1+θ 2 1 +θ θ2 q, for k = 1, 2,, q 0, for k > q where the numerator of ρ q is just θ q.

22 4.3 Autoregressive Processes Def. (AR(p)) A pth-order autoregressive process {Y t } satisfies the equation Y t = φ 1 Y t 1 + φ 2 Y t φ p Y t p + e t For every t, we assume that e t is independent of Y t 1, Y t 2, Y t 3, Autoregressive models are based on the idea that the current value Y t of the series can be explained as a function of p most recent past values plus an innovation term e t that incorporates everything new in the series at time t.

23 4.3.1 The AR(1) Process Y t = φy t 1 + e t Assume stationarity and zero mean on {Y t }. We take variances of both sides of Y t = φy t 1 + e t and obtain γ 0 = φ 2 γ 0 + σ 2 e = γ 0 = σ2 e 1 φ 2. By γ 0 > 0, we must have φ < 1. For any k > 0, we multiply both sides of Y t = φy t 1 + e t by Y t k and take expected values: E(Y t Y t k ) = φe(y t 1 Y t k ) + E(e t Y t k ) = γ k = φγ k 1, k = 1, 2,

24 Theorem 4 For the AR(1) process Y t = φy t 1 + e t, we have σ 2 e γ k = φ k 1 φ 2, ρ k = φ k, k = 0, 1, 2, Since φ < 1, the magnitude of the autocorrelation function decreases exponentially as the number of lags k increases. 1 If 0 < φ < 1, all correlations are positive. 2 if 1 < φ < 0, the lag 1 autocorrelation ρ 1 = φ is negative, and the signs of successive autocorrelations alternate with their magnitudes decreasing exponentially.

25 In R, the theoretical ACF of a stationary ARMA process can be computed by the ARMAacf function. The ar (resp. ma) parameter vector, if present, is to be passed into the function via the ar (resp. ma) argument. The maximum lag may be specified by the lag.max argument. Type?ARMAacf for more options (e.g. pacf). We define a function AR1acf to plot the autocorrelation functions for AR(1) models with different φ (Type?plotmath for more about displaying math symbols):

26 See TS-ch4.R. The file also shows simulated AR(1) series of size n = 300 and φ = 0.4, together with the plot of its sample acf.

27 Ex 4.13 The dataset ar1.s is simulated from a AR(1) process with φ = 0.9. The smoothness of the plot shows a strong autocorrelation at lag 1.

28 The plot shows a strong autocorrelation at lag 1.

29 The plot shows a strong autocorrelation at lag 2.

30 The plot shows a high autocorrelation at lag 3.

31 The AR(1) Model may be represented as a general linear process: Y t = e t + φy t 1 = e t + φ(e t 1 + φy t 2 ) = e t + φe t 1 + φ 2 Y t 2 = e t + φe t 1 + φ 2 (e t 2 + φy t 3 ) = e t + φe t 1 + φ 2 e t 2 + φ 3 Y t 3 = = e t + φe t 1 + φ 2 e t φ k 1 e t k+1 + φ k Y t k Y t = e t + φe t 1 + φ 2 e t 2 + φ 3 e t 3 + The stationarity condition for the AR(1) process Y t = φy t 1 + e t is φ < 1.

32 The AR(2) Process Y t = φ 1 Y t 1 + φ 2 Y t 2 + e t We assume that e t is independent of Y t 1, Y t 2,. The process is equivalent to e t = Y t φ 1 Y t 1 φ 2 Y t 2. Def. For the AR(2) process Y t = φ 1 Y t 1 + φ 2 Y t 2 + e t, the AR characteristic polynomial is φ(x) = 1 φ 1 x φ 2 x 2 and the AR characteristic equation is 1 φ 1 x φ 2 x 2 = 0 The equation has two (possibly complex) roots z 1, z 2 = φ 1 ± φ φ 2. 2φ 2

33 The stationarity of an AR process is determined by the roots of its characteristic equation. Theorem 5 A stationary process Y t = φ 1 Y t 1 + φ 2 Y t 2 + e t exists iff both roots of the AR characteristic equation has modulus exceed 1, iff it meets the stationarity conditions for the AR(2) model: φ 1 + φ 2 < 1, φ 2 φ 1 < 1, φ 2 < 1.

34

35 To derive the autocorrelation function for the AR(2) process, we multiply both sides of Y t = φ 1 Y t 1 + φ 2 Y t 2 + e t by Y t k and take expected value. Assuming stationarity, zero means, and that e t is independent of Y t k, we get dividing through by γ 0, γ k = φ 1 γ k 1 + φ 2 γ k 2, k = 1, 2, 3, (1) ρ k = φ 1 ρ k 1 + φ 2 ρ k 2, k = 1, 2, 3, (2) Equations (1) and (2) are called the Yule-Walker Equations, esp. the set of equations for k = 1 and k = 2: { ρ 1 = φ 1 + φ 2 ρ 1 ρ 2 = φ 1 ρ 1 + φ 2 (3) The ρ 1 and ρ 2 can be solved above, and successive ρ k may be calculated by the Yule-Walker Equations (2).

36 The variance γ 0 of the AR(2) process may be solved by the joint equations of 1 taking variances on both sides of Y t = φ 1 Y t 1 + φ 2 Y t 2 + e t, and 2 the Yule-Walker equation (1) for k = 1. ( ) 1 φ2 σe 2 γ 0 = 1 + φ 2 (1 φ 2 ) 2 φ 2. 1

37

38 Clearly G 1, G 2 < 1 for stationary processes. 1 If the roots are real and distinct, then ρ k and γ k are linear combinations of G1 k and G 2 k. Similarly if the roots are identical. The curves dies out exponentially. 2 If the roots are complex, then ρ k and γ k are linear combinations of R k sin(θk) and R k cos(θk), where R = φ 2 and cos Θ = φ 1 2 φ 2. The curves displays a damped sine wave behavior.

39

40

41 The smoothness of the plot shows the strong correlations in successive points.

42 The Ψ-coefficients for the AR(2) model as a general linear process may be obtained by substituting the general linear process representations of Y t, Y t 1 and Y t 2, Y t = e t + Ψ 1 e t 1 + Ψ 2 e t 2 +, to Y t = φ 1 Y t 1 + φ 2 Y t 2 + e t, then equating coefficients of e k and get the recursive relationships: Ψ 0 = 1, Ψ 1 φ 1 Ψ 0 = 0, Ψ k φ 1 Ψ k 1 φ 2 Ψ k 2 = 0, for k = 2, 3, The Ψ k can be solved recursively. The sequence {Ψ k } has similar pattern as that of {ρ k } and {γ k } (determined by whether the roots of characteristic equation are real or complex).

43 4.3.3 The General Autoregressive Process Def. The pth-order autoregressive model AR(p): Y t = φ 1 Y t 1 + φ 2 Y t φ p Y t p + e t (4) has AR characteristic polynomial φ(x) = 1 φ 1 x φ 2 x 2 φ p x p, and AR characteristic equation 1 φ 1 x φ 2 x 2 φ p x p = 0. Theorem 6 (Stationarity) The AR(p) process is stationary iff the p roots of the characteristic equation each exceeds 1 in modulus.

44 Assuming stationarity and zero means, we may multiply Equation (4) by Y t k, take expectations, divide by γ 0, and obtain the important recursive relationship ρ k = φ 1 ρ k 1 + φ 2 ρ k 2 + φ 3 ρ k φ p ρ k p for k 1. (5) Putting k = 1, 2,, p into Equation (5) and using ρ 0 = 1 and ρ k = ρ k, we get the general Yule-Walker equations ρ 1 = φ 1 + φ 2 ρ 1 + φ 3 ρ φ p ρ p 1 ρ 2 ρ 3 ρ p = φ 1 ρ 1 + φ 2 + φ 3 ρ φ p ρ p 2 = φ 1 ρ 2 + φ 2 ρ 1 + φ φ p ρ p 3. = φ 1 ρ p 1 + φ 2 ρ p 2 + φ 3 ρ p φ p (6) Given φ 1,, φ p, we can solve ρ 1,, ρ k by the Yule-Walker equations, and solve the other ρ s by (5).

45 Multiply (4) by e t and take expectation. We get E(e t Y t ) = σ 2 e. Multiply (4) by Y t and take expectation. We get Use ρ k = γ k /γ 0. We get γ 0 = φ 1 γ 1 + φ 2 γ φ p γ p + σ 2 e γ 0 = and solve the other γ s. σ 2 e 1 φ 1 ρ 1 φ 2 ρ 2 φ p ρ p Facts: each of the ρ k, γ k, and Ψ k (in general linear process representation) is a linear combination of exponentially decaying terms and damped sine wave terms corresponding to the roots of the characteristic equation.

46 4.4 The Mixed Autoregressive Moving Average Model Def. A process {Y t } is called a mixed autoregressive moving average process of orders p and q (abbr. ARMA(p, q)), if Y t = φ 1 Y t 1 +φ 2 Y t 2 + +φ p Y t p +e t θ 1 e t 1 θ 2 e t 2 θ q e t q (7) Remark. We assume that there are no common factors in the autoregressive and moving average polynomials: 1 φ 1 x φ 2 x 2 φ p x p and 1 θ 1 x θ 2 x 2 θ q x q. If there were, we could cancel them and the model would reduce to an ARMA model of lower order.

47 The ARMA(1,1) Model: The defining equation is To derive the Yule-Walker type equation: Y t = φy t 1 + e t θe t 1. (8) E(e t Y t ) = E[e t (φy t 1 + e t θe t 1 )] = σ 2 e E(e t 1 Y t ) = E[e t 1 (φy t 1 + e t θe t 1 )] = φσe 2 E(Y t k Y t ) = E[Y t k (φy t 1 + e t θe t 1 )] The last equation for k = 0, 1, 2, 3, yields γ 0 = φγ 1 + [1 θ(φ θ)]σe 2 γ 1 = φγ 0 θσe 2 γ k = φγ k 1 for k 2. (9)

48 We can solve that γ 0 = (1 2φθ + θ2 ) 1 φ 2 σe 2 (10) (1 θφ)(φ θ) ρ k = 1 2θφ + θ 2 φk 1 for k 1. (11) The ARMA(1,1) autocorrelation function decays exponentially with the damping factor φ as the lag k increases. The decay starts from initial value ρ 1. In contrast, the AR(1) autocorrelation function decays with the damping factor φ and from initial value ρ 0 = 1. (See Exercises 4.19 & 4.20) The general linear process form of ARMA(1,1) is: Y t = e t + (φ θ) φ j 1 e t j. j=1

49 The ARMA(p,q) model: Theorem 7 (Stationarity) A stationary solution to the model Y t = φ 1 Y t 1 +φ 2 Y t 2 + +φ p Y t p +e t θ 1 e t 1 θ 2 e t 2 θ q e t q exists iff all the roots of the AR characteristic equation φ(x) = 0 exceed 1 in modulus. The autocorrelation function can be shown to satisfy ρ k = φ 1 ρ k 1 + φ 2 ρ k φ p ρ k p for k > q. (12) Similarly equations can be developed for k = 1, 2,, q that involve θ 1, θ 2,, θ q. The autocorrelation function can be computed by ARMAacf in R. The ARMA(p,q) model can be written as a general linear process with the Ψ coefficients determined by similar equations as those of the AR(p) model, except that θ 1,, θ q are involved.

50 4.5 Invertibility The MA(1) model with θ has the same autocorrelation function as the MA(1) model with 1/θ. Similar nonuniqueness of MA(q) models for given autocorrelation function exist in every q. It can be resolved by assuming invertibility of the MA(q) model. An AR process can always be expressed as a general linear process (an infinite-order MA process). Conversely, how and when can an MA process be expressed as an infinite-order AR process?

51 Ex. Consider the MA(1) model: Y t = e t θe t 1. We get e t = Y t + θe t 1 = Y t + θ (Y t 1 + θe t 2 ) = Y t + θy t 1 + θ 2 e t 2 = = Y t + θy t 1 + θ 2 Y t 2 + When θ < 1, we succeed in converting the MA(1) process to an AR process: Y t = θy t 1 θ 2 Y t 2 + e t So the MA(1) model is invertible iff θ < 1.

52 Def. For a general MA(q) or ARMA(p,q) model, we define the MA characteristic polynomial as θ(x) = 1 θ 1 x θ 2 x 2 θ q x q and the MA characteristic equation Theorem 8 (Invertibility) 1 θ 1 x θ 2 x 2 θ q x q = 0. The MA(q) or ARMA(p,q) model is invertible; that is, there are coefficients π j such that Y t = π 1 Y t 1 + π 2 Y t 2 + π 3 Y t e t iff the roots of the MA characteristic equation exceed 1 in modulus. Theorem 9 Given a suitable autocorrelation function, there is only one set of parameter values that yield an invertible MA process. From now on, we require both stationarity and invertibility for an ARMA(p,q) model.

53