Lecture 1: Stationary Time Series Analysis

Transcription

1 Syllabus Stationarity ARMA AR MA Model Selection Estimation Lecture 1: Stationary Time Series Analysis : Time Series Econometrics Spring 2018 Jacek Suda

2 Syllabus Stationarity ARMA AR MA Model Selection Estimation Website Description Prerequisites Requirements Readings Outline Website Class website: Office hours: by appointments

3 Syllabus Stationarity ARMA AR MA Model Selection Estimation Website Description Prerequisites Requirements Readings Outline Description The purpose of this course is to familiarize students with current techniques used in macroeconomic time series models with applications in macroeconomics, international finance, and finance; with the ultimate aim of providing necessary tools to conduct original research in the area. The focus is on implementation of the models presented in the course. Topics include ARMA models, VARs and impulse response functions; unit roots, and structural breaks; spurious regressions; cointegration and VECM; ARCH models of volatility, and trend/cycle decomposition methods, including Kalman filtering. We will mostly work with the classical framework in the time domain but will touch upon Bayesian and frequency domain frameworks.

4 Syllabus Stationarity ARMA AR MA Model Selection Estimation Website Description Prerequisites Requirements Readings Outline Prerequisites I will assume some familiarity with matrix algebra and introductory statistics and econometrics. The course is the continuation of Econometrics course but we will also review the univariate time series analysis in this course.

5 Syllabus Stationarity ARMA AR MA Model Selection Estimation Website Description Prerequisites Requirements Readings Outline Requirements Attending lectures and participating in classroom discussions is essential to the learning process. There will be two or three homework assignments. The class ends either in-class 2 hour long exam. The assignments and exam will require the use of econometrics software. The weights in determining your grade are given as follows: Class participation 10% Homework assignments 40% Final exam 50%

6 Syllabus Stationarity ARMA AR MA Model Selection Estimation Website Description Prerequisites Requirements Readings Outline Readings The book that covers most of the material is Time Series Analysis by James D. Hamilton, Princeton University Press, Other texts are State-Space Models with Regime Switching by Chiang-Jin Kim and Charles R. Nelson, MIT Press, New Introduction to Multiple Time Series Analysis by Helmut Lütkepohl, Springer-Verlag, Introduction to Bayesian Econometrics by Edward Greenberg, Cambridge University Press, Time Series and Panel Data Econometrics by M. Hashem Pesaran, Oxford University Press, Applied Econometric Time Series by Walter Enders, Wiley, The readings include journal articles and chapters from the above books.

7 Syllabus Stationarity ARMA AR MA Model Selection Estimation Website Description Prerequisites Requirements Readings Outline Outline 1 Stationary Time Series Analys Overview of ARMA models State-Space Representation Kalman Filter 2 Structural Analysis Granger Causality, VAR, IRFs, Estimation, Variance decomposition Reduced-form and structural VAR models Distributed lag models 3 Unit Roots and Structural Breaks Unit root tests Structural break tests Trend/cycle decomposition Cointegration VEC models 4 Nonlinearity (time permitting) ARCH/GARCH models Markov switching Time-varying parameters Gibbs sampling Threshold models

8 Syllabus Stationarity ARMA AR MA Model Selection Estimation Website Description Prerequisites Requirements Readings Outline Today Outline: 1 Covariance-Stationary Processes 2 Wold Decomposition Theorem 3 ARMA Models 1 Auto-Correlation Function (ACF) 2 Partial Auto-Correlation Function (PACF) 3 Model Selection 4 Estimation 5 Forecasting

9 Syllabus Stationarity ARMA AR MA Model Selection Estimation Stochastic Process Cov-Stationarity Strict Stationary Nonstationary Wold STATIONARITY

10 Syllabus Stationarity ARMA AR MA Model Selection Estimation Stochastic Process Cov-Stationarity Strict Stationary Nonstationary Wold Stochastic Process Stochastic process: a collection of random variables {..., Y 1, Y 0, Y 1, Y 2,..., Y T,...} = {Y t }. Observed series {y 1, y 2,..., y T } realizations of a stochastic process. We want a model for {Y t } to explain observed realizations {y t } T 1.

11 Syllabus Stationarity ARMA AR MA Model Selection Estimation Stochastic Process Cov-Stationarity Strict Stationary Nonstationary Wold Covariance-Stationary Definition {Y t } is covariance-stationary (weak stationary) if (i) E[Y t] = µ t (ii) Cov(Y t, Y t j) = E[(Y t µ)(y t j µ)] = γ j, t, j Note: mean is time-invariant

12 Syllabus Stationarity ARMA AR MA Model Selection Estimation Stochastic Process Cov-Stationarity Strict Stationary Nonstationary Wold Covariance-Stationary Definition {Y t } is covariance-stationary (weak stationary) if (i) E[Y t] = µ t (ii) Cov(Y t, Y t j) = E[(Y t µ)(y t j µ)] = γ j, t, j Note: covariance doesn t depend on t

13 Syllabus Stationarity ARMA AR MA Model Selection Estimation Stochastic Process Cov-Stationarity Strict Stationary Nonstationary Wold Covariance-Stationary Definition {Y t } is covariance-stationary (weak stationary) if (i) E[Y t] = µ t (ii) Cov(Y t, Y t j) = E[(Y t µ)(y t j µ)] = γ j, t, j Note: Var(Y t ) = γ 0 variance is also constant.

14 Syllabus Stationarity ARMA AR MA Model Selection Estimation Stochastic Process Cov-Stationarity Strict Stationary Nonstationary Wold Covariance-Stationary Definition {Y t } is covariance-stationary (weak stationary) if (i) E[Y t] = µ t (ii) Cov(Y t, Y t j) = E[(Y t µ)(y t j µ)] = γ j, t, j It is weak stationarity because it only relates to the first two moments. Higher moments can be time-variant. Examples 1 {Y t} with E(Y t) = 0, Var(Y t) = σ 2, and E(Y ty τ ) = 0 white noise (WN). 2 Y t iid(0, σ 2 ) independent white noise. 3 Y t iidn(0, σ 2 ) Gaussian white noise.

15 Syllabus Stationarity ARMA AR MA Model Selection Estimation Stochastic Process Cov-Stationarity Strict Stationary Nonstationary Wold Strict (Strong) Stationary Definition {Y t } is (strictly/strongly) stationary if for any values of j 1, j 2,..., j n the joint distribution of (Y t, Y t+j1, Y t+j2,..., Y t+jn ) depends only on the intervals separating the dates (j 1, j 2,..., j n ) and not on a date itself (t). For all τ, t 1, t 2,..., t n : F Y (y t1, y t2,..., y tn ) = F Y (y t1+τ, y t2+τ,..., y tn+τ ) If a process is strictly stationary with a finite second moment it is also covariance-stationary. Normality strong stationarity: whole distribution depends on the first two moments.

16 Syllabus Stationarity ARMA AR MA Model Selection Estimation Stochastic Process Cov-Stationarity Strict Stationary Nonstationary Wold Nonstationary Processes Examples: 1 Y t = β t + ε t, ε t WN

17 Syllabus Stationarity ARMA AR MA Model Selection Estimation Stochastic Process Cov-Stationarity Strict Stationary Nonstationary Wold Nonstationary Processes Examples: 1 Y t = β t + ε t, ε t WN t - time dummy

18 Syllabus Stationarity ARMA AR MA Model Selection Estimation Stochastic Process Cov-Stationarity Strict Stationary Nonstationary Wold Nonstationary Processes Examples: 1 Y t = β t + ε t, ε t WN deterministic part

19 Syllabus Stationarity ARMA AR MA Model Selection Estimation Stochastic Process Cov-Stationarity Strict Stationary Nonstationary Wold Nonstationary Processes Examples: 1 Y t = β t + ε t, ε t WN stochastic component

20 Syllabus Stationarity ARMA AR MA Model Selection Estimation Stochastic Process Cov-Stationarity Strict Stationary Nonstationary Wold Nonstationary Processes Examples: 1 Y t = β t + ε t, ε t WN E[Y t] = β t depends on t. But X t = Y t β t is covariance stationary.

21 Syllabus Stationarity ARMA AR MA Model Selection Estimation Stochastic Process Cov-Stationarity Strict Stationary Nonstationary Wold Nonstationary Processes Examples: 1 Y t = β t + ε t, ε t WN E[Y t] = β t depends on t. But X t = Y t β t is covariance stationary. 2 Y t = Y t 1 + ε t, ε t WN, Y 0 constant Random walk.

22 Syllabus Stationarity ARMA AR MA Model Selection Estimation Stochastic Process Cov-Stationarity Strict Stationary Nonstationary Wold Nonstationary Processes Examples: 1 Y t = β t + ε t, ε t WN E[Y t] = β t depends on t. But X t = Y t β t is covariance stationary. 2 Y t = Y t 1 + ε t, ε t WN, Y 0 constant Random walk. Solving recursively : Y t = t ε j + Y 0. j=1 E[Y t] = Y 0, time-invariant mean. But Var(Y t) = t σ 2 depends on t. X t = Y t Y t 1 is covariance stationary.

23 Syllabus Stationarity ARMA AR MA Model Selection Estimation Stochastic Process Cov-Stationarity Strict Stationary Nonstationary Wold Wold s Decomposition Theorem Any covariance stationary {Y t } has infinite order, moving-average representation: Y t = ψ j ε t j + κ t, j=0 ψ 0 = 1, ε t WN. ψ 0 = 1, j=0 ψ2 j <, ε t WN(0, σ 2 ), E(ε tκ s) = 0 t,s, κ t deterministic term (perfectly forecastable). Example: κ t = µ, constant mean. κ = a cos(λt) + b sin(λt), λ fixed number, Ea = Eb = Eab = 0.

24 Syllabus Stationarity ARMA AR MA Model Selection Estimation Stochastic Process Cov-Stationarity Strict Stationary Nonstationary Wold Wold s Decomposition Theorem Any covariance stationary {Y t } has infinite order, moving-average representation: Y t = ψ j ε t j + κ t, j=0 ψ 0 = 1, ε t WN. ψ 0 = 1, j=0 ψ2 j <, ε t WN(0, σ 2 ), E(ε tκ s) = 0 t,s, κ t deterministic term (perfectly forecastable). Linear combination of ε s (innovations over time), ε t = Y t E(Y t Y t 1, Y t 2,...), Weights does not depend on time t, they only depend on j, i.e. how long ago the shock ε occurred.

25 Syllabus Stationarity ARMA AR MA Model Selection Estimation Stochastic Process Cov-Stationarity Strict Stationary Nonstationary Wold Wold s Decomposition Theorem - Illustration Let X t = Y t κ t (purely indeterministic process). Then, E[X t] = ψ j E[ε t j ] = 0, j=0 E[Xt 2 ] = ψj 2 E[ε 2 t j] = σ 2 ψj 2 <, j=0 j=0 as ε t are uncorrelated; we have constant finite variance. E[X t X t j ] = E[(ε t + ψ 1 ε t 1 + ψ 2 ε t )(ε t j + ψ 1 ε t j 1 + ψ 2 ε t j )] = σ 2 (ψ j + ψ j+1 ψ 1 + ψ j+2 ψ ) = σ 2 ψ k ψ k+j, depends on j not t. k=0 So we have a covariance stationary process in mean and variance.

26 Syllabus Stationarity ARMA AR MA Model Selection Estimation ARMA Models Lag Operator Dynamic Multiplier IRF Cumulative impact ARMA

27 Syllabus Stationarity ARMA AR MA Model Selection Estimation ARMA Models Lag Operator Dynamic Multiplier IRF Cumulative impact ARMA Models Approximate Wold form with finite number of parameters. Wold Form: Y t µ = ψ j ε t j, ε t WN, j=0 ARMA(p,q): Y t µ = φ 1 (Y t 1 µ)+...+φ p (Y t p µ)+ε t +θ 1 ε t θ q ε t q.

28 Syllabus Stationarity ARMA AR MA Model Selection Estimation ARMA Models Lag Operator Dynamic Multiplier IRF Cumulative impact Lag Operator Define the operator L as LX t X t 1, L 2 X t = L LX t = X t 2. In general, If c is a constant, Also, L k X t = X t k. Lc = c. L 1 X t X t+1, X t = (1 L)X t = X t X t 1.

29 Syllabus Stationarity ARMA AR MA Model Selection Estimation ARMA Models Lag Operator Dynamic Multiplier IRF Cumulative impact Lag Operator It satisfies and, when φ < 1, L(αX t + βy t ) = αx t 1 + βy t 1 (a L + b L 2 ) X t = ax t 1 + bx t 2, lim (1 + φl + j φ2 L φ j L j ) = (1 φl) 1

30 Syllabus Stationarity ARMA AR MA Model Selection Estimation ARMA Models Lag Operator Dynamic Multiplier IRF Cumulative impact ARMA Models in Lag Notaion ARMA(p, q): Y t µ = φ 1 (Y t 1 µ)+...+φ p (Y t p µ)+ε t +θ 1 ε t θ q ε t q, or Y t µ φ 1 (Y t 1 µ)... φ p (Y t p µ) = ε t +θ 1 ε t θ q ε t q, With lag operator: φ(l)(y t µ) = θ(l)ε t, where φ(l) = 1 φ 1 L φ 2 L 2... φ p L p, θ(l) = 1 + θ 1 L + θ 2 L θ q L q.

31 Syllabus Stationarity ARMA AR MA Model Selection Estimation ARMA Models Lag Operator Dynamic Multiplier IRF Cumulative impact Stochastic Difference Equation (SDE) Representation Let X t = Y t µ and w t = θ(l)ε t. Then φ(l)x t = w t, or X t = φ 1 X t φ p X t p + w t, is a pth-order stochastic difference equation.

32 Syllabus Stationarity ARMA AR MA Model Selection Estimation ARMA Models Lag Operator Dynamic Multiplier IRF Cumulative impact SDE Representation - AR(1) Example Example: First-order SDE (AR(1)): X t = φx t 1 + ε t, ε t WN Solve for Wold Form (recursive substitution) X t = φ t+1 X 1 + φ t ε 0 + φ t 1 ε φε t 1 + ε t = φ t+1 X 1 + ψ iε t i. i=0 where X 1 is an initial condition and ψ i = φ i. We approximated Wold form with 1 parameter form for AR(1).

33 Syllabus Stationarity ARMA AR MA Model Selection Estimation ARMA Models Lag Operator Dynamic Multiplier IRF Cumulative impact Dynamic Multiplier The dynamic multiplier measurers the effect of ε t on subsequent values of X τ : X t+j ε t For the X t being AR(1) process = X j ε 0 = ψ j. X t+j ε t = ψ j = φ j. The dynamic multiplier for any linear difference equation depends only on the length of time j, not on time t.

34 Syllabus Stationarity ARMA AR MA Model Selection Estimation ARMA Models Lag Operator Dynamic Multiplier IRF Cumulative impact Impulse Response Function The impulse-response function is a sequence of dynamic multipliers as a function of time from the one time impulse to ε t ,s IRF s, Time

35 Syllabus Stationarity ARMA AR MA Model Selection Estimation ARMA Models Lag Operator Dynamic Multiplier IRF Cumulative impact Cumulative impact Permanent increase in ε at time t, i.e. ε t = 1, ε t+1 = 1, ε t+2 = 1,... or X t+j ε t + X t+j ε t+1 + X t+j ε t X t+j ε t+j = ψ j + ψ j ψ + 1 X t ε t + Xt+1 ε t In the limit, as j + Xt+2 ε t Xt+j ε t = ψ j + ψ j ψ + 1 [ Xt+j lim + X t+j X ] t+j = j ε t ε t+1 ε t+j ψ j = ψ(1), j=0 where ψ(1) = ψ(l = 1) = 1 + ψ 1 + ψ

36 Syllabus Stationarity ARMA AR MA Model Selection Estimation AR(1) ACF AR(p) Stability Yule-Walker Equations AUTOREGRESSIVE PROCESS

37 Syllabus Stationarity ARMA AR MA Model Selection Estimation AR(1) ACF AR(p) Stability Yule-Walker Equations AR(1) Model Recall X t = φx t 1 + ε t, ε t WN, = φ j ε t j = ψ j ε t j. Wold coefficients j=0 j=0 ψ j = φψ j 1, ψ j = Y t+j ε t If φ < 1 X t is a stable and stationary solution to first-order SDE. If φ = 1 then ψ j = 1 j, X t = X 1 + t j=0 εj is neither stationary nor stable solution, and ψ(1) is infinite.

38 Syllabus Stationarity ARMA AR MA Model Selection Estimation AR(1) ACF AR(p) Stability Yule-Walker Equations AR(1): Lag notation AR(1): X t = φx t 1 + ε t, ε t WN, (1 φl)x t = ε t Multiply both sides by (1 φl) 1 : X t = (1 φl) 1 ε t = (1 + φl + φ 2 L 2 + φ 3 L )ε t = φ j ε t j = ψ j ε t j, j=0 j=0 ψ(l) = (1 φl) 1.

39 Syllabus Stationarity ARMA AR MA Model Selection Estimation AR(1) ACF AR(p) Stability Yule-Walker Equations AR(1): Long-run effects For AR(1), if φ < 1 the permanent increase in ε t equals X t+j ε t and as j + Xt+j ε t Xt+j ε t+j = 1 + φ + φ 2 + φ φ j. ψ(1) = 1 + φ + φ = 1/(1 φ) the cumulative consequences for X of a one-time change in ε, j=0 X t+j ε t = 1/(1 φ)

40 Syllabus Stationarity ARMA AR MA Model Selection Estimation AR(1) ACF AR(p) Stability Yule-Walker Equations AR(1): Mean Intercept representation for Y t X t + µ Mean Y t = c + φy t 1 + ε t. E[Y t] = c + φe[y t 1] + E[ε t], Since we have covariance stationary process, E[Y t] = E[Y t 1] and E[Y t] = c µ, c = µ(1 φ). 1 φ

41 Syllabus Stationarity ARMA AR MA Model Selection Estimation AR(1) ACF AR(p) Stability Yule-Walker Equations AR(1): Variance Variance Var(Y t ) = E[(Y t µ) 2 ] = E[(φ(Y t 1 µ) + ε t ) 2 ] = φ 2 E[(Y t 1 µ) 2 ] + 2φE[(Y t 1 µ)ε t ] + E[ε 2 t ]. Since Y t is covariance stationary and ε t is independently distributed, Var(Y t ) = Var(Y t 1 ) and so E[Y t 1 ε t ] = 0, Var(Y t ) = σ2 1 φ 2 γ 0.

42 Syllabus Stationarity ARMA AR MA Model Selection Estimation AR(1) ACF AR(p) Stability Yule-Walker Equations AR(1): Covariance Covariance Cov(Y t, Y t j ) = E[(Y t µ)(y t j µ)] = φe[(y t 1 µ)(y t j µ)] + E[ε t (Y t j µ)], Cov(Y t, Y t j ) γ j = φγ j 1.

43 Syllabus Stationarity ARMA AR MA Model Selection Estimation AR(1) ACF AR(p) Stability Yule-Walker Equations AR(1): Moments Summing up E[Y t ] = c 1 φ µ, Var(Y t ) = σ 2 1 φ 2 γ 0, Cov(Y t, Y t j ) γ j = φγ j 1.

44 Syllabus Stationarity ARMA AR MA Model Selection Estimation AR(1) ACF AR(p) Stability Yule-Walker Equations Auto-correlation Function (ACF) Define ρ j γ j γ 0 j th autocorrelation corr(y t, Y t j ) For AR(1), ρ j = φρ j 1. For AR(1) ACF and IRF are the same. In general it not true. ACF < 1, 1 >.

45 Syllabus Stationarity ARMA AR MA Model Selection Estimation AR(1) ACF AR(p) Stability Yule-Walker Equations AR(1): Auto-correlation Function (ACF) 0 < φ < ACFAR 1, < φ < ACF AR 1, f= Out[142]=

46 Syllabus Stationarity ARMA AR MA Model Selection Estimation AR(1) ACF AR(p) Stability Yule-Walker Equations Pth-order SDE: AR(p) An AR(p) process Y t µ = φ 1 (Y t 1 µ) + φ 2 (Y t 2 µ) φ p (Y t p µ) + v t, can be rewritten as a 1 st order system Y t µ φ 1 φ 2... φ p 1 φ p Y t 1 µ Y t 2 µ =. Y t p+1 µ In the state-space, companion form notation β t = F β t 1 + ε t and we are back in 1 st order system. Y t 1 µ Y t 2 µ Y t 3 µ. Y t p µ + v t

47 Syllabus Stationarity ARMA AR MA Model Selection Estimation AR(1) ACF AR(p) Stability Yule-Walker Equations Pth-order SDE: AR(p) An AR(p) process Y t µ = φ 1 (Y t 1 µ) + φ 2 (Y t 2 µ) φ p (Y t p µ) + v t, can be rewritten as a 1 st order system Y t µ φ 1 φ 2... φ p 1 φ p Y t 1 µ Y t 2 µ =. Y t p+1 µ In the state-space, companion form notation β t = F β t 1 + ε t and we are back in 1 st order system. Y t 1 µ Y t 2 µ Y t 3 µ. Y t p µ + v t

48 Syllabus Stationarity ARMA AR MA Model Selection Estimation AR(1) ACF AR(p) Stability Yule-Walker Equations Pth-order SDE: AR(p) An AR(p) process Y t µ = φ 1 (Y t 1 µ) + φ 2 (Y t 2 µ) φ p (Y t p µ) + v t, can be rewritten as a 1 st order system Y t µ φ 1 φ 2... φ p 1 φ p Y t 1 µ Y t 2 µ =. Y t p+1 µ In the state-space, companion form notation β t = F β t 1 + ε t and we are back in 1 st order system. Y t 1 µ Y t 2 µ Y t 3 µ. Y t p µ + v t

49 Syllabus Stationarity ARMA AR MA Model Selection Estimation AR(1) ACF AR(p) Stability Yule-Walker Equations Pth-order SDE: AR(p) An AR(p) process Y t µ = φ 1 (Y t 1 µ) + φ 2 (Y t 2 µ) φ p (Y t p µ) + v t, can be rewritten as a 1 st order system Y t µ φ 1 φ 2... φ p 1 φ p Y t 1 µ Y t 2 µ =. Y t p+1 µ In the state-space, companion form notation β t = F β t 1 + ε t and we are back in 1 st order system. Y t 1 µ Y t 2 µ Y t 3 µ. Y t p µ + v t

50 Syllabus Stationarity ARMA AR MA Model Selection Estimation AR(1) ACF AR(p) Stability Yule-Walker Equations Pth-order SDE: AR(p) An AR(p) process Y t µ = φ 1 (Y t 1 µ) + φ 2 (Y t 2 µ) φ p (Y t p µ) + v t, can be rewritten as a 1 st order system Y t µ Y t 1 µ Y t 2 µ =. Y t p+1 µ φ 1 φ 2... φ p 1 φ p Y t 1 µ Y t 2 µ Y t 3 µ. Y t p µ + v t In the state-space, companion form notation β t = F β t 1 + ε t and we are back in 1 st order system.

51 Syllabus Stationarity ARMA AR MA Model Selection Estimation AR(1) ACF AR(p) Stability Yule-Walker Equations Pth-order SDE: AR(p) An AR(p) process Y t µ = φ 1 (Y t 1 µ) + φ 2 (Y t 2 µ) φ p (Y t p µ) + v t, can be rewritten as a 1 st order system Y t µ Y t 1 µ Y t 2 µ. Y t p+1 µ = φ 1 φ 2... φ p 1 φ p Y t 1 µ Y t 2 µ Y t 3 µ. Y t p µ + v t In the state-space, companion form notation β t = F β t 1 + ε t and we are back in 1 st order system.

52 Syllabus Stationarity ARMA AR MA Model Selection Estimation AR(1) ACF AR(p) Stability Yule-Walker Equations AR(p): Stability Consider a state space form β t+j = F j+1 β t 1 + F j ε t Fε t+j 1 + ε t+j. AR(p) is stable and stationary if, lim j Fj = 0 i.e. when eigenvalues of F are inside unit circle (have modulus < 1). Shocks die out.

53 Syllabus Stationarity ARMA AR MA Model Selection Estimation AR(1) ACF AR(p) Stability Yule-Walker Equations Eigenvalues Consider equation Fx = λx. x is eigenvector and λ is a corresponding eigenvalue. To compute the eigenvalue, write it as If x is a non-zero vector then (F λi)x = 0. F λi is singular det(f λi) = 0.

54 Syllabus Stationarity ARMA AR MA Model Selection Estimation AR(1) ACF AR(p) Stability Yule-Walker Equations Example Consider the AR(2) Then, in a matrix notation where Y t µ = φ 1 (Y t 1 µ) + φ 2 (Y t 2 µ) + v t β t = [ Yt µ Y t 1 µ β t = Fβ t 1 + ε t, ] [ φ1 φ, F = Eigenvalues λ of the matrix F solve ( ) φ1 λ φ det 2 = λ 2 λφ 1 λ 1 φ 2 = 0, λ i = φ 1 ± φ φ 2 2 If λ i <1 for i = 1, 2, the AR(2) is stable. ]

55 Syllabus Stationarity ARMA AR MA Model Selection Estimation AR(1) ACF AR(p) Stability Yule-Walker Equations AR(p): Stability For pth-order SDE, solve λ p φ 1 λ p 1... φ p 1 λ φ p = 0 In general, the solution involves complex and real roots. The AR(p) system is stable if all eigenvalues are inside the unit circle. Note: complex eigenvalues imply periodic behavior. AR 2, f1=+0.4 f2=

56 Syllabus Stationarity ARMA AR MA Model Selection Estimation AR(1) ACF AR(p) Stability Yule-Walker Equations Characteristic Equation Recall that we could write down AR(p) process Y t µ = φ 1 (Y t 1 µ) + φ 2 (Y t 2 µ) φ p (Y t p µ) + ε t, in the lag polynomial form φ(l)(y t µ) = ε t, where φ(l) = 1 φ 1 L φ 2 L 2... φ p L p. The stability can be then studied through the characteristic equation φ(z) = 1 φ 1 z φ 2 z 2... φ p z p = 0.

57 Syllabus Stationarity ARMA AR MA Model Selection Estimation AR(1) ACF AR(p) Stability Yule-Walker Equations Roots of Characteristic Equation The roots (z s) of characteristic equation, φ(z) = 0, are inverse of eigenvalues (λ s) of companion matrix F: as z = 1/λ. 1 φ 1 z φ 2 z 2... φ p z p = (1 λ 1 z)(1 λ 2 z) (1 λ p z) = z p (1/z λ 1 )(1/z λ 2 ) (1/z λ p ) For z > 1 stochastic difference equation is stable and stationary.

58 Syllabus Stationarity ARMA AR MA Model Selection Estimation AR(1) ACF AR(p) Stability Yule-Walker Equations Stability φ(l) can be decomposed into (1 φ 1 L φ 2 L 2... φ p L p ) = (1 λ 1 L)(1 λ 2 L)... (1 λ p L) Factor the polynomial into AR(1) elements. Since i λ i < 1 then (1 λ i L) 1 exists and so φ(l) 1 exists = ψ(l) = φ(l) 1 for AR(p).

59 Syllabus Stationarity ARMA AR MA Model Selection Estimation AR(1) ACF AR(p) Stability Yule-Walker Equations Yule-Walker Equations Variance, covariances, autocorrelations, and dynamic multipliers have the same p th -order form: Var(Y t ) = γ 0 = φ 1 γ 1 + φ 2 γ φ p γ p + σ 2 Cov(Y t, Y t j ) = γ j = φ 1 γ j 1 + φ 2 γ j φ p γ j p corr(y t, Y t j ) = ρ j = φ 1 ρ j 1 + φ 2 ρ j φ p ρ j p ψ j = φ 1 ψ j 1 + φ 2 ψ j φ p ψ j p

60 Syllabus Stationarity ARMA AR MA Model Selection Estimation AR(1) ACF AR(p) Stability Yule-Walker Equations Yule-Walker Equations: AR(2) Example AR(2) Example: φ(l) 1 = ψ(l) φ(l)ψ(l) = 1 (1 φ 1 L φ 2 L 2 )(1 + ψ 1 L + ψ 2 L ) = (ψ 1 φ 1 )L + (ψ 2 φ 1 ψ 1 + φ 2 )L = 1 Then ψ 1 = φ 1, ψ 2 = φ 1 ψ 1 + φ 2,. ψ j = φ 1 ψ j 1 + φ 2 ψ j 2.

61 Syllabus Stationarity ARMA AR MA Model Selection Estimation AR(1) ACF AR(p) Stability Yule-Walker Equations Example: AR(2) Moments AR(2) Process: Y t = c + φ 1 Y t 1 + φ 2 Y t 2 + ε t Mean Covariance µ = c/(1 φ 1 φ 2 ) γ j = E[(Y t µ)(y t j µ)] = E[(φ 1 (Y t 1 µ) + φ 2 (Y t 2 µ) + ε t )(Y t j µ)] = φ 1 γ j 1 + φ 2 γ j 2 γ 1 = φ 1 γ 0 + φ 2 γ 1 = γ 0 φ 1 1 φ 2 γ 2 = φ 1 γ 1 + φ 2 γ 0 = γ 0 ( φ φ 2 + φ 2 )

62 Syllabus Stationarity ARMA AR MA Model Selection Estimation AR(1) ACF AR(p) Stability Yule-Walker Equations Example: AR(2) Moments Variance γ 0 = E[(Y t µ)(φ 1 (Y t 1 µ) + φ 2 (Y t 2 µ) + ε t )] = φ 1 γ 1 + φ 2 γ 2 + σ 2 [ φ 2 = 1 + φ 2φ 2 ] 1 + φ 2 2 γ 0 + σ 2 1 φ 2 1 φ 2 1 φ 2 γ 0 = (1 + φ 2 ) [ ] (1 φ 2 ) 2 φ 2 1 Autocorrelation ρ j = φ 1 ρ j 1 + φ 2 ρ j 2 ρ 1 = φ 1 + φ 2 ρ 1 = φ 1 /(1 φ 2 ) ρ 2 = φ 1 ρ 1 + φ 2

63 Syllabus Stationarity ARMA AR MA Model Selection Estimation MA(1) Invertibility PACF MOVING AVERAGE (MA) PROCESSES

64 Syllabus Stationarity ARMA AR MA Model Selection Estimation MA(1) Invertibility PACF Moving Average (MA) Processes Recall Wold Form: Y t = µ + ψ j ε t j, j=0 ε t WN, MA(q) process : truncated Wold form. Y t = µ + ε t + θ 1 ε t 1 + θ 2 ε t θ q ε t q Y t = µ + θ(l)ε t, θ(l) = 1 + θ 1 L + θ 2 L θ q L q. Moving average as Y t is constructed from a weighed sum to the q most recent values of ε

65 Syllabus Stationarity ARMA AR MA Model Selection Estimation MA(1) Invertibility PACF MA(1) Example MA(1) Moments Mean Variance Y t = µ + ε t + θε t 1 = µ + (1 + θl)ε t E[Y t] = µ, Var(Y t) = γ 0 = E[(Y t µ) 2 ]

66 Syllabus Stationarity ARMA AR MA Model Selection Estimation MA(1) Invertibility PACF MA(1) Example MA(1) Moments Mean Variance Y t = µ + ε t + θε t 1 = µ + (1 + θl)ε t E[Y t] = µ, Var(Y t) = γ 0 = E[(Y t µ) 2 ] = E[(ε t + θε t 1) 2 ]

67 Syllabus Stationarity ARMA AR MA Model Selection Estimation MA(1) Invertibility PACF MA(1) Example MA(1) Moments Mean Variance Y t = µ + ε t + θε t 1 = µ + (1 + θl)ε t E[Y t] = µ, Var(Y t) = γ 0 = E[(Y t µ) 2 ] = E[(ε t + θε t 1) 2 ] = E[ε 2 t + 2θε tε t 1 + θ 2 ε 2 t 1]

68 Syllabus Stationarity ARMA AR MA Model Selection Estimation MA(1) Invertibility PACF MA(1) Example MA(1) Moments Mean Variance Y t = µ + ε t + θε t 1 = µ + (1 + θl)ε t E[Y t] = µ, Var(Y t) = γ 0 = E[(Y t µ) 2 ] = E[(ε t + θε t 1) 2 ] = E[ε 2 t + 2θε tε t 1 + θ 2 ε 2 t 1] = σ θ 2 σ 2

69 Syllabus Stationarity ARMA AR MA Model Selection Estimation MA(1) Invertibility PACF MA(1) Example MA(1) Moments Mean Variance Y t = µ + ε t + θε t 1 = µ + (1 + θl)ε t E[Y t] = µ, Var(Y t) = γ 0 = E[(Y t µ) 2 ] = E[(ε t + θε t 1) 2 ] = E[ε 2 t + 2θε tε t 1 + θ 2 ε 2 t 1] = σ θ 2 σ 2 = (1 + θ 2 )σ 2,

70 Syllabus Stationarity ARMA AR MA Model Selection Estimation MA(1) Invertibility PACF MA(1) Example MA(1) Moments Covariance Y t = µ + ε t + θε t 1 = µ + (1 + θl)ε t Cov(Y t, Y t 1) = γ 1 = E[(Y t µ)(y t 1 µ)]

71 Syllabus Stationarity ARMA AR MA Model Selection Estimation MA(1) Invertibility PACF MA(1) Example MA(1) Moments Covariance Y t = µ + ε t + θε t 1 = µ + (1 + θl)ε t Cov(Y t, Y t 1) = γ 1 = E[(Y t µ)(y t 1 µ)] = E[(ε t + θε t 1)(ε t 1 + θε t 2)]

72 Syllabus Stationarity ARMA AR MA Model Selection Estimation MA(1) Invertibility PACF MA(1) Example MA(1) Moments Covariance Y t = µ + ε t + θε t 1 = µ + (1 + θl)ε t Cov(Y t, Y t 1) = γ 1 = E[(Y t µ)(y t 1 µ)] = E[(ε t + θε t 1)(ε t 1 + θε t 2)] = θσ 2,

73 Syllabus Stationarity ARMA AR MA Model Selection Estimation MA(1) Invertibility PACF MA(1) Example MA(1) Moments Covariance Y t = µ + ε t + θε t 1 = µ + (1 + θl)ε t Cov(Y t, Y t 1) = γ 1 = E[(Y t µ)(y t 1 µ)] = E[(ε t + θε t 1)(ε t 1 + θε t 2)] = θσ 2, Cov(Y t, Y t j) = γ j = 0 j > 1. Autocorrelation Function ρ 1 = γ1 θσ 2 = γ 0 (1 + θ 2 )σ = θ θ 2 ρ j = 0, j > 1.

74 Syllabus Stationarity ARMA AR MA Model Selection Estimation MA(1) Invertibility PACF Invertibility of MA Processes Consider MA(1) Y t = µ + (1 + θl)ε t θ and 1/θ give the same ACF ρ 1 = γ1 = θ γ θ = 1/θ θ 2 = 1 θ 1 = θ 1 + θ 2 θ 2 θ 2 +1 Normalize θ by using invertible MA representation. Example: Both Y t = ε t + 0.5ε t 1 and Y t = ε t + 2ε t 1 have the same autocorrelation function: ρ 1 = = (0.5) 2 = 0.4

75 Syllabus Stationarity ARMA AR MA Model Selection Estimation MA(1) Invertibility PACF Invertibility of MA Processes But for MA(1) with θ < 1, there exists -order AR representation: Y t µ = (1 + θl)ε t, θ < 1, = (1 θ L)ε t, for θ = θ It can rewritten as (1 θ L) 1 (Y t µ) = ε t (1 + θ L + θ 2 L )(Y t µ) = ε t Y t µ = θ(y t 1 µ) θ 2 (Y t 2 µ) + θ 3 (Y t 3 µ)... + ε t i.e. AR( ) with φ(l) = 1 + θ θ 2 + θ 3 θ θ < 1 is not a stability requirement, MA system is always stable. It allows invertibility and AR( ) representation.

76 Syllabus Stationarity ARMA AR MA Model Selection Estimation MA(1) Invertibility PACF Partial Autocorrelation Function (PACF) Definition k th -order partial autocorrelation is the regression coefficient (for the population) φ kk in k th -order autoregression Y t = c + φ k1 Y t 1 + φ k2 Y t φ kk Y t k + ε t It measures how important is the last lag.

77 Syllabus Stationarity ARMA AR MA Model Selection Estimation Box-Jenkins GDP ACF PACF Box-Ljung Selection Residual MODEL SELECTION

78 Syllabus Stationarity ARMA AR MA Model Selection Estimation Box-Jenkins GDP ACF PACF Box-Ljung Selection Residual Box-Jenkins Approach Matching model with actual data Transform data to appear covariance stationary We may have data that is not covariance stationary, e.g. GDP. Box-Jenkins approach: maybe GDP is not covariance-stationary but some transformation of it is and we can get forecast of it from transformed series. take logs (natural) differences detrend

79 Syllabus Stationarity ARMA AR MA Model Selection Estimation Box-Jenkins GDP ACF PACF Box-Ljung Selection Residual GDP 16000,0 GDP 14000, , ,0 8000,0 6000,0 GDP 4000,0 2000,0 0, We can t use ARMA model as the graph shows it s not covariance stationary series. Also, not linear.

80 Syllabus Stationarity ARMA AR MA Model Selection Estimation Box-Jenkins GDP ACF PACF Box-Ljung Selection Residual Log of GDP Take natural logarithm. 12 LN(GDP) LN(GDP)

81 Syllabus Stationarity ARMA AR MA Model Selection Estimation Box-Jenkins GDP ACF PACF Box-Ljung Selection Residual Difference in log of GDP = Growth rate If we take first difference of the log series, we get growth rate of level series. 0,07 Diff ln(gdp) 0,06 0,05 0,04 0,03 0,02 Diff ln(gdp) 0,01 0-0, ,02-0,03 It looks much more like AR or MA process. Just looking at graph may not be enough.

82 Syllabus Stationarity ARMA AR MA Model Selection Estimation Box-Jenkins GDP ACF PACF Box-Ljung Selection Residual AR(1) y:ar AR 1, time 2

83 Syllabus Stationarity ARMA AR MA Model Selection Estimation Box-Jenkins GDP ACF PACF Box-Ljung Selection Residual AR(1) y:ar 1 AR 1, time 2 3

84 Syllabus Stationarity ARMA AR MA Model Selection Estimation Box-Jenkins GDP ACF PACF Box-Ljung Selection Residual AR(1) y:ar 1 AR 1, time 2

85 Syllabus Stationarity ARMA AR MA Model Selection Estimation Box-Jenkins GDP ACF PACF Box-Ljung Selection Residual AR(1) y:ar 1 25 AR 1, time

86 Syllabus Stationarity ARMA AR MA Model Selection Estimation Box-Jenkins GDP ACF PACF Box-Ljung Selection Residual AR(1) y:ar 1 AR 1, time 2 3

87 Syllabus Stationarity ARMA AR MA Model Selection Estimation Box-Jenkins GDP ACF PACF Box-Ljung Selection Residual AR(2) y:ar 2 AR 2, time 2

88 Syllabus Stationarity ARMA AR MA Model Selection Estimation Box-Jenkins GDP ACF PACF Box-Ljung Selection Residual MA(1) y:ma 1 MA 1, time 2

89 Syllabus Stationarity ARMA AR MA Model Selection Estimation Box-Jenkins GDP ACF PACF Box-Ljung Selection Residual MA(1) y:ma 1 MA 1, time 2

90 Syllabus Stationarity ARMA AR MA Model Selection Estimation Box-Jenkins GDP ACF PACF Box-Ljung Selection Residual MA(2) y:ma 2 MA 2, time 2

91 Syllabus Stationarity ARMA AR MA Model Selection Estimation Box-Jenkins GDP ACF PACF Box-Ljung Selection Residual AR(1) It s not that bad, though: y:ar 1 AR 1, time 1 2

92 Syllabus Stationarity ARMA AR MA Model Selection Estimation Box-Jenkins GDP ACF PACF Box-Ljung Selection Residual AR(1) It s not that bad, though: y:ar 1 AR 1, time 2

93 Syllabus Stationarity ARMA AR MA Model Selection Estimation Box-Jenkins GDP ACF PACF Box-Ljung Selection Residual Box-Jenkins Approach Matching model with actual data Transform data to appear covariance stationary take logs (natural) differences detrend Examine the sample ACF and PACF We know that there is a 1-1 mapping between ACF and series

94 Syllabus Stationarity ARMA AR MA Model Selection Estimation Box-Jenkins GDP ACF PACF Box-Ljung Selection Residual ACF: AR(1) 1.0 ACFAR 1,

95 Syllabus Stationarity ARMA AR MA Model Selection Estimation Box-Jenkins GDP ACF PACF Box-Ljung Selection Residual ACF: AR(2) ACFAR 2,

96 Syllabus Stationarity ARMA AR MA Model Selection Estimation Box-Jenkins GDP ACF PACF Box-Ljung Selection Residual ACF: MA(1) 1.0 ACFMA 1,

97 Syllabus Stationarity ARMA AR MA Model Selection Estimation Box-Jenkins GDP ACF PACF Box-Ljung Selection Residual ACF: MA(2) 1.0 ACFMA 2,

98 Syllabus Stationarity ARMA AR MA Model Selection Estimation Box-Jenkins GDP ACF PACF Box-Ljung Selection Residual Sample ACF ȳ = 1 T ˆγ j = 1 T T y t, t=1 T (y t ȳ)(y t j ȳ), sample auto covariance estimate t=j+1 ρ j = ˆγ j ˆγ 0 sample auto correlation

99 Syllabus Stationarity ARMA AR MA Model Selection Estimation Box-Jenkins GDP ACF PACF Box-Ljung Selection Residual Sample PACF Use OLS y t = c + φ 1k y t φ kk y t k + ε t, k = 1, 2,... If y t iid(µ, σ 2 ), ρ j = 0, j 0. (no correlation across observation) ˆ var(ˆρ j ) 1 T and ˆ var( ˆφ kk ) 1 T This values are used when reporting bounds in ACF and PACF: 5%-95%: ±1.96var(ˆρ ˆ j ) or ±1.96var( ˆ ˆφ kk ).

100 Syllabus Stationarity ARMA AR MA Model Selection Estimation Box-Jenkins GDP ACF PACF Box-Ljung Selection Residual Model selection Assume we can reject iid, i.e. PACF and ACF show some significant lags. How to determine which model is correct? Process ACF PACF AR(p) Exponential or oscillatory decay φ kk = 0, k > p MA(q) ρ k = 0, k > q Exponential or oscillatory decay ARMA(p,q) decay begins at lag q decay begins at lag p But we have many models we can t really discriminate between just by looking.

101 Syllabus Stationarity ARMA AR MA Model Selection Estimation Box-Jenkins GDP ACF PACF Box-Ljung Selection Residual Box-Jenkins Matching model with actual data Transform data to appear covariance stationary take logs (natural) differences detrend Examine the sample ACF and PACF Estimate ARMA models Perform diagnostic analysis to confirm that model is consistent with data

102 Syllabus Stationarity ARMA AR MA Model Selection Estimation Box-Jenkins GDP ACF PACF Box-Ljung Selection Residual Box-Ljung Statistics Use modified Box-Ljung Statistics (Q-Stat) to test joint statistical significance of ˆρ 1,..., ˆρ k : Q (k) = T(T + 2) k i=1 Under H 0 : Y t iid(µ, σ 2 ), Q (k) A χ 2 (k). 1 T i ˆρ2 i. For significance level α, the critical region for rejection of the hypothesis of randomness (i.e. reject ρ 1 =... = ρ k = 0) is Q (k) > χ 2 1 α(k). For smaller sample, empirical distribution has fatter tail and one needs to move the critical point to the right.

103 Syllabus Stationarity ARMA AR MA Model Selection Estimation Box-Jenkins GDP ACF PACF Box-Ljung Selection Residual Akaike Information Criterion (AIC) Akaike Information Criterion (AIC): AIC(p, q) = ln(ˆσ 2 ) + 1 2(p + q), T where the first term is responsible for the fit of the model and the second one is a penalty for number of parameters.

104 Syllabus Stationarity ARMA AR MA Model Selection Estimation Box-Jenkins GDP ACF PACF Box-Ljung Selection Residual Schwarz (Bayesian) Information Criterion Schwarz (Bayesian) Information Criterion (BIC): BIC(p, q) = ln(ˆσ 2 ) + 1 ln(t)2(p + q), T where now penalty is related to sample size.

105 Syllabus Stationarity ARMA AR MA Model Selection Estimation Box-Jenkins GDP ACF PACF Box-Ljung Selection Residual Selection: AIC vs BIC Calculate selection criterium for a set of estimated models and try to find the one that minimize the AIC or BIC. Result: If p, q considered are larger than true orders (i) BIC picks true ARMA(p,q) model as T. (ii) AIC picks overparameterized model as T. For smaller sample it might be a a problem with choice BIC: Trade-off between efficiency and consistency: Better to overparameterize than to have incorrect result.

106 Syllabus Stationarity ARMA AR MA Model Selection Estimation Box-Jenkins GDP ACF PACF Box-Ljung Selection Residual Residual Diagnostic Use sample ACF and PACF of sample residuals. Compute Box-Ljung Statistics for sample of residuals: Q (k) A χ 2 (k (p + q)), where (p+q) is a degree of freedom adjustment Use LM test (it has higher power) T R 2 χ 2 (k), with R 2 computed from the regression ˆε 2 t = c + α 1ˆε t α k ˆε t k + v t.

107 Syllabus Stationarity ARMA AR MA Model Selection Estimation ARMA OLS MLE AR(1) CMLE Exact MLE MA(1) ESTIMATION

108 Syllabus Stationarity ARMA AR MA Model Selection Estimation ARMA OLS MLE AR(1) CMLE Exact MLE MA(1) ARMA Wold representation: Y t = κ t + ψ j ε t j ε t WN(0, σ 2 ), j=0 ψj 2 < j=0 AR(p): Y t µ = φ 1 (Y t 1 µ)+φ 2 (Y t 2 µ)+...+φ p (Y t p µ)+ε t, ε t WN MA(q): Y t µ = ε t + θ 1 ε t 1 + θ 2 ε t θ q ε t q, ε t WN ARMA(p,q): Y t µ = φ 1 (Y t 1 µ) φ p (Y t p µ) + ε t + θ 1 ε t θ q ε t q

109 Syllabus Stationarity ARMA AR MA Model Selection Estimation ARMA OLS MLE AR(1) CMLE Exact MLE MA(1) Estimation OLS MLE AR(1) MA(1)

110 Syllabus Stationarity ARMA AR MA Model Selection Estimation ARMA OLS MLE AR(1) CMLE Exact MLE MA(1) OLS For AR(p), OLS is equivalent to Conditional MLE Model: y t = c + φ 1 y t φ p y t p + ε t, ε t WN(0, σ 2 ). = x tβ + ε t, β = (c, φ 1, φ 2,..., φ p ), x t = (1, y t 1, y t 2,..., y t OLS: ˆβ = ˆσ 2 = ( T ) 1 T x t x t x t y t, t=1 1 T (p + 1) t=1 T (y t x t ˆβ) 2. t=1

111 Syllabus Stationarity ARMA AR MA Model Selection Estimation ARMA OLS MLE AR(1) CMLE Exact MLE MA(1) Properties Note: E[ ˆβ] β because x t is random and E(ε Y) 0. But, if z > 1 for φ(z) = 0 (or λ < 1 for F) then, ˆβ p β, ˆσ 2 p σ 2. Estimator might be biased but consistent, it converges in probability. Illustration: φ = 0.7 n = Median =

112 Syllabus Stationarity ARMA AR MA Model Selection Estimation ARMA OLS MLE AR(1) CMLE Exact MLE MA(1) Properties Note: E[ ˆβ] β because x t is random and E(ε Y) 0. But, if z > 1 for φ(z) = 0 (or λ < 1 for F) then, ˆβ p β, ˆσ 2 p σ 2. Estimator might be biased but consistent, it converges in probability. Illustration: φ = 0.7 n = Median =

113 Syllabus Stationarity ARMA AR MA Model Selection Estimation ARMA OLS MLE AR(1) CMLE Exact MLE MA(1) Properties Note: E[ ˆβ] β because x t is random and E(ε Y) 0. But, if z > 1 for φ(z) = 0 (or λ < 1 for F) then, ˆβ p β, ˆσ 2 p σ 2. Estimator might be biased but consistent, it converges in probability. Illustration: φ = 0.7 n = Median =

114 Syllabus Stationarity ARMA AR MA Model Selection Estimation ARMA OLS MLE AR(1) CMLE Exact MLE MA(1) Properties Note: E[ ˆβ] β because x t is random and E(ε Y) 0. But, if z > 1 for φ(z) = 0 (or λ < 1 for F) then, ˆβ p β, ˆσ 2 p σ 2. Estimator might be biased but consistent, it converges in probability. Illustration: φ = 0.7 n = Median =

115 Syllabus Stationarity ARMA AR MA Model Selection Estimation ARMA OLS MLE AR(1) CMLE Exact MLE MA(1) Properties Note: E[ ˆβ] β because x t is random and E(ε Y) 0. But, if z > 1 for φ(z) = 0 (or λ < 1 for F) then, ˆβ p β, ˆσ 2 p σ 2. Estimator might be biased but consistent, it converges in probability. Illustration: φ = 0.7 n = Median =

116 Syllabus Stationarity ARMA AR MA Model Selection Estimation ARMA OLS MLE AR(1) CMLE Exact MLE MA(1) Properties Note: E[ ˆβ] β because x t is random and E(ε Y) 0. But, if z > 1 for φ(z) = 0 (or λ < 1 for F) then, ˆβ p β, ˆσ 2 p σ 2. Estimator might be biased but consistent, it converges in probability. Illustration: φ = 0.7 n = Median =

117 Syllabus Stationarity ARMA AR MA Model Selection Estimation ARMA OLS MLE AR(1) CMLE Exact MLE MA(1) Hypothesis testing Hypothesis testing: T( ˆβ β) d N(0, σ 2 V 1 1 ), V = plim T T T x t x t. If we have enough data (T ) then t-student will converge to Normal distribution. t=1 t β=β0 = ˆβ β 0 t student(t 1) N(0, 1). SE( ˆ ˆβ) T See Hamilton for a downward bias in a finite sample, i.e. E[ ˆβ] < β.

118 Syllabus Stationarity ARMA AR MA Model Selection Estimation ARMA OLS MLE AR(1) CMLE Exact MLE MA(1) MLE If Y t iid, joint likelihood is the product of marginal likelihoods (pdf) L( θ y 1,..., y T ) = = T L( θ y t ), t=1 T f (y t θ), t=1 Different parameters have different probability of generating {y 1,..., y T }.

119 Syllabus Stationarity ARMA AR MA Model Selection Estimation ARMA OLS MLE AR(1) CMLE Exact MLE MA(1) MLE If Y t iid, joint likelihood is the product of marginal likelihoods (pdf) L( θ y 1,..., y T ) = = T L( θ y t ), t=1 T f (y t θ), t=1 Different parameters have different probability of generating {y 1,..., y T }. But if Y t is not independent, factorization is invalid. Instead, factor into conditional distributions. f (Y 1, Y 2 θ) = f (Y 2 Y 1, θ)f (Y 1 θ), f (Y 1, Y 2, Y 3 θ) = f (Y 3 Y 2, Y 1, θ)f (Y 2, Y 1 θ), L( θ y 1,..., y T ) = f (Y 1,..., Y T θ) = T f (Y t Y t 1,..., Y 2, Y 1, θ)f (Y 1 θ) t=2

120 Syllabus Stationarity ARMA AR MA Model Selection Estimation ARMA OLS MLE AR(1) CMLE Exact MLE MA(1) MLE Conditional MLE assumes Y 1 fixed (not random). It s just a simplifying assumption. L c ( θ y 1,..., y T ) = T f (Y t Y t 1,..., Y 1, θ). t=2 Given normality, first order conditions of maximization L c are linear in θ. For AR models ˆφ CMLE OLS. Conditional MLE is consistent. (It s not so efficient as we ignore randomness of Y 1.) Exact MLE requires non-linear optimization. Often we don t know distribution of the the data. One thing that is assumed in OLS is the ε WN. Assuming normality in CMLE is not a bad assumption as we still get consistent estimator: quasi MLE. (see Davidson and MacKinnon.) We don t get unbiasness in any of MLE. What can be done is to compute what the bias is and correct for it.

121 Syllabus Stationarity ARMA AR MA Model Selection Estimation ARMA OLS MLE AR(1) CMLE Exact MLE MA(1) Estimation AR(1) Recall: AR(1) Y t = c + φy t 1 + ε t, ε iidn(0, σ 2 ), φ < 1. If we don t believe in normality of ε t, we have quasi-mle. If we do: MLE. If ε N so is Y t Y t 1 N: Y t Y t 1 N(c + φy t 1, σ 2 ).

122 Syllabus Stationarity ARMA AR MA Model Selection Estimation ARMA OLS MLE AR(1) CMLE Exact MLE MA(1) Conditional MLE If we know Y t 1 the only term that is random is ε t. f (y t y t 1, θ) = 1 2πσ 2 e 1 2σ 2 (yt c φyt 1)2, where θ = (c, φ, σ 2 ). What about Y 1? Y 1 N(E[Y t ], var(y t )) N(µ, σ 2 1 φ 2 ), µ = c 1 φ. If φ = 1 no unconditional mean or variance exist.

123 Syllabus Stationarity ARMA AR MA Model Selection Estimation ARMA OLS MLE AR(1) CMLE Exact MLE MA(1) Exact MLE Maximum likelihood i.e. L( θ y 1,..., y T ) = f (y 1 θ) 1 L( θ y 1,..., y T ) = 2π( σ2 1 φ ) 2 T f (y t y t 1, θ), t=2 e 1 2σ 2 /(1 φ 2 (y1 c ) 1 φ )2 T t=2 1 2πσ 2 e 1 2σ 2 (yt c φyt 1)2. Need to solve non-linear optimization problem.

124 Syllabus Stationarity ARMA AR MA Model Selection Estimation ARMA OLS MLE AR(1) CMLE Exact MLE MA(1) MA(1) Recall Y t = µ + ε t + θε t 1, ε t iidn(0, σ 2 ), θ < 1. θ < 1 is assumed for invertible representation only. Figure: Bi-model likelihood function for MA process

125 Syllabus Stationarity ARMA AR MA Model Selection Estimation ARMA OLS MLE AR(1) CMLE Exact MLE MA(1) Estimation MA(1) Y t ε t 1 N(µ + θε t 1, σ 2 ), f (y t ε t 1, θ) = 1 1 2πσ 2 e 2σ 2 (yt µ θεt 1)2, θ = (µ, θ, σ 2 ). Problem: without knowing ε t 2 we don t observe ε t 1. Need to know ε t 2 to know ε t 1 = y t µ θε t 2. But ε t 2 unobservable. Assume ε 0 = 0. Make it non-random, just fix it with number 0. The trick works with any number.

126 Syllabus Stationarity ARMA AR MA Model Selection Estimation ARMA OLS MLE AR(1) CMLE Exact MLE MA(1) Estimation MA(1) Y 1 ε 0 N(µ, σ 2 ), Y 1 = µ + ε 1 ε 1 = Y 1 µ, Y 2 = µ + ε 2 + θε 1 ε 2 = Y 2 µ θ(y 1 µ), ε t = Y t µ θ(y t 1 µ) ( 1) t 1 θ t 1 (Y 1 µ). Conditional likelihood (ε 0 = 0).: L( θ y 1,..., y T, ε 0 = 0) = T t=1 1 2πσ 2 e 1 2σ 2 ε2 t. If θ < 1 (much less), ε 0 doesn t matter, CMLE is consistent. Exact MLE requires Kalman Filter.