Lecture 2: ARMA(p,q) models (part 2)

Transcription

1 Lecture 2: ARMA(p,q) models (part 2) Florian Pelgrin University of Lausanne, École des HEC Department of mathematics (IMEA-Nice) Sept Jan Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

2 Motivation Introduction Characterize the main properties of MA(q) models. Estimation of MA(q) models Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

3 Road map Introduction 1 Introduction 2 MA(1) model 3 Application of a counterfactual MA(1) 4 Moving average model of order q, MA(q) 5 Application of a MA(q) model Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

4 MA(1) model Moving average models 2.1. Moving average model of order 1, MA(1) Definition A stochastic process (X t ) t Z is said to be a moving average model of order 1 if it satisfies the following equation : X t = µ + ɛ t θɛ t 1 t where θ 0, µ is a constant term, (ɛ t ) t Z is a weak white noise process with expectation zero and variance σ 2 ɛ (ɛ t WN(0, σ 2 ɛ )). Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

5 MA(1) model Remarks : 1. In lag notation, one has : X t = µ + Θ(L)ɛ t µ + (1 θl)ɛ t 2. The previous process can be written in mean-deviation as follows : where X t = ɛ t θɛ t 1 X t = X t µ. Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

6 MA(1) model Remarks (cont d) : 3. The properties of (X t ) only depend on those of the weak white noise process (ɛ t ). To some extent, the behavior of (X t ) is more noisy relative to an AR(1) process Iterating on the past infinite (and with some regularity conditions), the infinite autoregressive representation writes : X t = µ 1 θ + θ k X t k + ɛ t. k=1 5. The infinite autoregressive representation illustrates the fact that a certain form of persistence is captured by a moving average model, especially when θ is close to one. Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

7 MA(1) model Simulation of a moving average process of order 1 (θ = 0.9) Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

8 MA(1) model Scatter plots of a moving average process of order 1: Left panel (Xt-1 versus Xt) and right panel (Xt-2 versus Xt) X _(t-1) X_t X _ (t-2) X_t Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

9 MA(1) model Stationarity and invertibility conditions Since (ɛ t ) is a weak noise process, (X t ) is weakly stationary (by definition). The invertibility condition is the counterpart of the stability (stationary) condition of an AR(1) process : 1 If θ < 1, then (X t ) is invertible. 2 If θ = 1, then (X t ) is non invertible. 3 If θ > 1, there exists a non-causal invertible representation of (X t ) that we rule out. Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

10 MA(1) model Alternatively, if θ < 1, then : (1 θl) 1 = and θ k L k k=0 ɛ t = (1 θl) 1 (X t µ) i.e. X t = µ 1 θ + θ k X t k + ɛ t k=1 1 This is the infinite autoregressive representation of a MA(1) process. 2 The MA(1) representation is then called the fundamental or causal representation. Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

11 MA(1) model More formally... Definition The representation of the moving average process of order one defined by : X t = µ + ɛ t θɛ t 1, is said to be causal or fundamental (ɛ t ) is the innovation process if the root of the characteristic equation zθ(z 1 ) = 0 z θ = 0 lies outside the unit circle : z < 1 θ < 1. Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

12 MA(1) model Remark : One can also use the inverse characteristic equation to find the invertibility condition : Θ(z) = 0 1 θz = 0. The condition writes (for a MA(1) process) : z > 1 θ < 1. Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

13 Moments of a MA(1) MA(1) model Definition Let (X t ) be a stationary stochastic process that satisfies a (fundamental) MA(1) representation, X t = µ + ɛ t θɛ t 1. Then : E [X t ] = µ V [X t ] = (1 + θ 2 )σ 2 ɛ γ X (1) = θσɛ 2 γ X (h) = 0 for all h > 1 Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

14 MA(1) model Definition Let (X t ) be a stationary stochastic process that satisfies a (fundamental) MA(1) representation, X t = µ + ɛ t θɛ t 1. Then, the autocorrelation function is given by : 1 if h = 0 ρ X (h) = θ if h = ±1 1+θ 2 0 otherwise. Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

15 MA(1) model The autocorrelation function of a moving average process of order 1, MA(1), is always zero for orders higher than 1 ( h > 1) : MA(1) process has no memory beyond 1 period (see Scatter plots and autocorrelograms). This property generalizes to MA(p) processes. Partial autocorrelations : Nothing special, with the exception that it should decrease (possibly, with damped oscillations)! The partial autocorrelation function cannot help for characterizing a MA(1). Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

16 MA(1) model Correlograms of a moving average process of order one (θ = 0.9, 0.5, and 0.2) Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

17 MA(1) model 0.8 Correlogram of an AR(1) with phi=0.8 1 Correlogram of an AR(1) with phi= Correlogram of an MA(1) with theta=0.8 Correlogram of an MA(2) with theta=(0.4;0.3) Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

18 Estimation MA(1) model Estimation is more difficult since the ɛ t terms are not observed! Different techniques : 1 Conditional nonlinear least squares estimator 2 Maximum likelihood estimator 3 Generalized method of moments estimator. Without loss of generality, the constant term is omitted and the model is written as : X t = ɛ t + θɛ t 1. Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

19 MA(1) model Nonlinear conditional least squares estimator The objective function of the ordinary least squares estimator is : ˆθ mco = argmin θ T (x t θɛ t 1 ) 2 t=2 Conditionally on ɛ 0, one has (backcasting procedure) : t 2 ɛ t 1 = ( θ) j x t 1 j + ( θ) t 1 ɛ 0 j=0 Suppose that ɛ 0 = 0, the nonlinear objective function (with respect to θ) writes : 2 T t 2 x t θ ( θ) j x t 1 j t=2 j=0 Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

20 MA(1) model The conditional nonlinear least squares estimator of θ is defined by : ˆθ cnls = argmin θ T t 2 x t θ ( θ) j x t 1 j t=2 j=0 The asymptotic distribution is given by : T (ˆθ cnls θ) a.d. N (0, 1 θ 2 ) The effect of ɛ 0 = 0 dies out if T is sufficiently large. An alternative is to consider ɛ 0 as an unknown parameter. An estimator of σ 2 ɛ is : ˆσ 2 ɛ = 1 T 1 T (x t ˆθ cnls ɛ t 1 ) 2. t=2 2 Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

21 MA(1) model Maximum likelihood estimator Two estimators : the conditional maximum likelihood estimator and the exact maximum likelihood estimator The conditional maximum likelihood estimator proceeds in the same way as the conditional nonlinear least squares estimator (backcasting procedure) : Suppose that ɛ t is a Gaussian White noise process For t = 1 : For t > 1 : ɛ 1 = x 1 θɛ 0 t 1 ɛ t = ( θ) j x t j + ( θ) t ɛ 0 j=0 Write the conditional likelihood function (with ɛ 0 = 0) and maximize with respect to θ and σ 2 ɛ. Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

22 MA(1) model The exact maximum likelihood estimator can be calculated by two convenient algorithms : 1 The Kalman filter 2 The triangular factorization of the variance-covariance matrix of a MA(1) process In contrast to the conditional maximum likelihood estimator, the exact maximum likelihood estimator does not require that the moving average representation is invertible. Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

23 MA(1) model The (Generalized) method of moments estimator A simple method of moments estimator... Consider the first two moments of a MA(1) : E [ Xt 2 ] = (1 + θ 2 )σɛ 2 and E [X t X t 1 ] = θσɛ 2 Using the empirical counterpart of these two moments conditions yields : T ( g T (x; θ, σɛ 2 ) = T 1 x 2 t σɛ 2 (1 + θ 2 ) ) x t x t 1 σ 2 t=1 ɛ θ ( T = 1 ) T t=1 x t 2 σɛ 2 (1 + θ 2 ) T 1 T t=2 x tx t 1 σɛ 2 θ Solving the exactly (just-) identified equation g T (x; ˆθ, ˆσ 2 ɛ ) = for ˆθ and ˆσ 2 ɛ (with sone regularity conditions...) gives the method of moment estimator. Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

24 MA(1) model Estimation of moving average processes of order 1 Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

25 Application of a counterfactual MA(1) 3. Application of a counterfactual MA(1) Effective Fed fund rate : 1970 : :01 (monthly observations) As to be expected from the (partial) autocorrelogram function (and thus theory!), a moving average model of order 1 is not probably the most appropriate model... However, it is interesting to compare it with the AR(1) specification of the effective Fed fund rate. The estimate of the constant term (respectively, θ) is (respectively, 0.941). Both estimates are statistically significant. Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

26 Application of a counterfactual MA(1) Effective Fed fund rate---ma(1) model Residual Actual Fitted Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

27 Application of a counterfactual MA(1) Effective Fed fund rate---diagnostics of a MA(1) model 1.0 Autocorrelation Actual Theoretical Partial autocorrelation Actual Theoretical Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

28 Application of a counterfactual MA(1) Effective Fed fund rate---impulse response function of the estimated MA(1) model 2.5 Impulse Response ± 2 S.E Accumulated Response ± 2 S.E Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

29 Moving average model of order q, MA(q) Moving average models 4. Moving average model of order q, MA(q) Definition A stochastic process (X t ) t Z is said to be a moving average model of order q if it satisfies the following equation : X t = µ + ɛ t + θ 1 ɛ t θ q ɛ t q t = µ + Θ(L)ɛ t where θ q 0, µ is a constant term, (ɛ t ) t Z is a weak white noise process with expectation zero and variance σ 2 ɛ (ɛ t WN(0, σ 2 ɛ )), and Θ(L) = 1 + θ 1 L + + θ q L q. Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

30 Moving average model of order q, MA(q) Remark : One can also use the notation X t = µ + ɛ t θ1ɛ t 1 θqɛ t q t = µ + Θ (L)ɛ t where θj = θ j for j = 1,, q θq 0 Θ (L) = 1 θ1l θql q ɛ t WN(0, σɛ 2 ). Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

31 Moving average model of order q, MA(q) Stationarity and invertibility conditions A MA(q) process is always weakly stationary irrespective of the moving average part A MA(q) process is invertible if all the roots of the characteristic equation z q Θ(z 1 ) = 0 are of modulus less than one : for i = 1,, q. z q + θ 1 z q 1 + θ 2 λ θ q = 0 z i < 1 Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

32 Moving average model of order q, MA(q) Definition The representation of the moving average process of order q defined by : X t = µ + ɛ t + θ 1 ɛ t φ q ɛ t q, is said to be causal or fundamental (ɛ t ) is the innovation process all the roots of the characteristic equation z q Θ(z 1 ) = 0 are of modulus less than one : for i = 1,, q. z q + θ 1 z q 1 + θ 2 λ θ q = 0 z i < 1 Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

33 Moving average model of order q, MA(q) Moments of stationary MA(q) Mean and autocovariances Using the insights of a MA(1) model, one gets : E(X t ) = µ γ X (h) = σ 2 ɛ σ 2 ɛ ( ) q 1 + θi 2 if h = 0 ( i=1 ) q θ h + θ i θ i h if 1 h < q i=h+1 θ q σ 2 ɛ if h = q 0 if h > q Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

34 Moving average model of order q, MA(q) Autocorrelations ρ X (h) = 1 if h = 0 θ h + q i=h+1 θ i θ i h q 1+ θi 2 i=1 θ q q 1+ θi 2 i=1 0 if h > q if h = q if 1 h < q The autocorrelation function of a moving average process of order q, MA(q), is always zero for orders high than q ( h > q) : MA(q) process has no memory beyond q periods. Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

35 Moving average model of order q, MA(q) Partial autocorrelations : Nothing special! The theoretical partial autocorrelation of an MA(q) dies out as h but is not exactly zero as an AR(p) process for h > p. Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

36 5. Application Application of a MA(q) model Effective Fed fund rate : 1970 : :01 (monthly observations) As to be expected from the (partial) autocorrelogram function (and thus theory!), a moving average model of order q is probably not the most appropriate model... However, it is interesting for comparison purposes. In particular, increasing the number of lags augments the overall persistence at the expense of over-parameterizing the model... Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

37 Application of a MA(q) model ML estimation of the effective Fed fund rate : MA(4) and MA(8) Coefficients Estimates Std. Error P-value µ θ θ µ θ θ θ All parameters are statistically significant at 1%, with the exception of θ 8...All the roots are outside the unit circle. Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

38 Application of a MA(q) model Effective Fed fund rate---ma(4) model Residual Actual Fitted Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

39 Application of a MA(q) model MA(4) Effective Fed fund rate: diagnostics MA(8) A u t o c o r r e l a t i o n A u t o c o r r e la t io n Actual Theoretical Actual Theoretical P a r tia l a u to c o r r e la tio n P a r t i a l a u t o c o r r e l a t i o n Actual Theoretical Actual Theoretical Florian Pelgrin (HEC) Univariate time series Sept Jan / 40

40 Application of a MA(q) model Effective Fed fund rate---impulse response functions MA(4) MA(8) 2.0 Impulse Response ± 2 S.E. 1.6 Impulse Response ± 2 S.E Accumulated Response ± 2 S.E. 10 Accumulated Response ± 2 S.E Florian Pelgrin (HEC) Univariate time series Sept Jan / 40