= k xt-k+ j. If k is large and <1, then the first part of this expression is negligible, hence. t j

Transcription

1 Let x be a zero-mean stochastic process. A roghly linear relationship between sccessive observations may be described by a simple atoregressive model of the form = + where is zero-mean white noise. A stationary process x satisfying this first-order difference eqation is called a first-order atoregressive process (AR() process). Sbstitting xt-+t- for xt-, xt-3+t- for xt-, gives xt = xt-+t = ( xt-+t-)+t = xt-+t-+t = ( xt-3+t-) +t-+t = 3 xt-3+ t-+t-+t k = k xt-k+. 0 If k is large and <, then the first part of this expression is negligible, hence M t 0 x. t t IR

2 Sppose that <. Then the soltion of the difference eqation is stationary becase E( xt) E t = x t x t xt Var(xt)= ( ) Var(t-)= 0 and Cov(xt,xt-k)= = do not depend on t. k E( 0 t t ) t ) = 0 0=0, ) ( = ( = 0, For example, the atocovariance at lag k= is given by Cov(xt,xt-) = E(t+ t-+ t-+...)( t-+ t-+ t-3+...) = E(t+ t-+ t-+...) = E(t t + t- = Ett-+ t Et-t-+ t + t- Et-t-+ t = ( )+( + 0+ )+( )+ = = 0 (( ) + ( ) + ( ) + ) σ = + ). IM

3 Using the lag operator L we can write the eqation as and the eqation as xt xt- = t t = xt xt- = L 0 (xt) L (xt) = (L 0 L)(xt) = ( L)(xt) xt = 0 xt = 0 t = ( 0 L ( ) L t )(t) = ( ( L) )(t). 0 A comparison of the eqations and sggests that the operator is the inverse operator of ( L)(xt) = t xt = ( ( L) )(t) 0 0 ( L) L and, more generally, that the lag operator follows the sal algebraic rles. We may therefore write or (L) - = ( L) 0 L = 0 ( L). 3

4 A stationary process x satisfying the p th-order difference eqation xt=xt-+ +pxt-p+t, where is zero-mean white noise and p0, is called an atoregressive process of order p (AR(p) process). A stationary process X satisfying xt=xt-+ +pxt-p+t +t-+ +qt-q, where is zero-mean white noise, p0, and q0, is called an atoregressive moving average process of order (p,q) (ARMA(p,q) process). The eqation xt=xt-+ +pxt-p+t +t-+ +qt-q can also be written as xtxt- pxt-p=t +t-+ +qt-q or as (L pl p )(xt)=(+l+ +ql q )(t). An ARMA(p,0) process is an AR(p) process. An ARMA(0,q) is called a moving average process of order q (MA(q) process). A process x is called an atoregressive integrated moving average process of order (p,d,q) (ARIMA(p,d,q) process) if its d th difference is an ARMA(p,q) process, i.e., (L pl p )( d xt)=(+l+ +ql q )(t). ARIMA(p,d,q) processes may be generalized by permitting the degree of differencing, d, to take fractional vales. The fractional differencing operator d =(-L) d is defined as a power series expansion in integer powers of L: (L) d =dl+d(d) L d(d)(d) L 3 3! + A process is called an atoregressive fractionally integrated moving average process (ARFIMA(p,d,q) process) if the fractionally differenced process is an ARMA process. 4

5 We say that an ARMA(p,q) representation (L pl p )(xt)=(+l+ +ql q )(t), is casal if xt can be expressed in terms of present and past shocks, i.e., xt= 0. t It is called invertible if t has a representation of the form t= Casality is eqivalent to 0 ξ. x t (z)=z pz p 0 z and invertibility is eqivalent to (z)=+z+ +qz q 0 z. For example, if the only zero / of the polynomial (z)=z satisfies />, then < and xt= 0 t, hence the AR() representation (-L)(xt)=t is casal. 5

6 Exercise: Determine which of the following ARMA representations are casal and which are invertible. (i) xt0.7xt-=t M (ii) xt+0.4xt-.5xt-=t (iii) xt=t0.t- (iv) xt=t+0.8t-+.6t- M (v) xt=t t-+t- (vi) xt=t+t- (vii) xtxt-=t0.3t- M3 (viii) xt0.xt-=t+0.4t-.6t- (ix) xt.xt-+0.xt-=t+0.3t-.t- Exercise: Show that the MA() representation of the casal ARMA(,) process xt xt-=t+t- is given by xt=t+(+) t. Soltion : We have or, eqivalently, z z = 0 z (+z)=(z)( 0+z+z + ). Eqating coefficients of z we obtain: =0: =0, =: =0 =+0=+, M Soltion : We have xt = L Lt = L t t+l L = 0 t = t+ +L t 0 t + t. ME 6

7 Exercise: Show that the atocovariances (k)=cov(xt,xt-k) of a casal ARMA(,) process satisfy xt xt-=t+t- (k)= (k), if k>. Soltion: Mltiplying the difference eqation by xt-k and taking expectations we obtain for k> xt xt-k xt- xt-k =t xt-k+t- xt-k, Ext xt-k Ext- xt-k =Et xt-k+ Et- xt-k, (k) (k)=0. Exercise: Find the stationary soltion of the noncasal nivariate AR() eqation xt= xt-+t, >. Soltion: xt= xt-+t xt-= xt t xt = xt+ t+ = ( xt+ t+) t+ = = xt+ M φ t t+ t+ MN 7

8 Exercise: Show that the stationary soltion xt= φ t of the noncasal nivariate AR() eqation xt= xt-+t, >, var(t)= also satisfies the casal nivariate eqation for a sitable white noise v. Soltion: If vt = xt xt- = xt-+xt = = xt= xt-+vt φ t φ t t+( 3 )t++( 4 )t++, then the mean of vt is zero 0 and its variance is given by ( 4 = ( 4 =. +( 3 ) +( 4 +( Moreover, if h>0, then Evtvt+h = (( h σ = h σ = h = 0. (( + )( 4 h ) ) 0 (( 4 ) + ) + 6 +( h3 +( 3 )+( )+( h )) )( 3 )+ ) )( 3 )+ ) + )+( )+ ) Ths vis white noise. MC This exercise shows that the stationary soltion of the noncasal eqation also satisfies a casal eqation. Nothing will therefore be lost if we consider only casal eqations. 8

9 Exercise: Show that the invertible MA() model xt=t+ t-, var(t)=4 implies the same atocovariance fnction as the noninvertible MA() model xt=t+t-, var(t)=. MO Remark: Since we can only observe xt and not t, we cannot distingish between the two models. They are therefore called observationally eqivalent. Exercise: Fit an ARMA(,) model to the growth rates of the qarterly GDP. r.dm <- r-mean(r) # r is demeaned p <- ; q <- arima(r.dm,order=c(p,0,q),inclde.mean=false, transform.pars=true) If transform.pars=true, the AR parameters are checked and, if necessary, transformed to ensre casality. We obtain the following estimates of,,, and : Coefficients: ar ma ma sigma^ estimated as 7.734e-05 9

10 Exercise: Use the model-selection criteria AIC and BIC to find the best ARMA(p,q) model with p3, q3 for the (demeaned) growth rates of the qarterly GDP. Hint: Trying to strike a balance between the goodness of fit (measred by the log likelihood L) and the complexity of the model (measred by the total nmber p+q+ of model parameters,,p,,,q, ), these criteria select that model which minimizes and respectively. AIC(p,q) = L + (p+q+) BIC(p,q) = L + (p+q+) log(n), AIC <- BIC <- matrix(nrow=4,ncol=4) for (p in 0:3) for (q in 0:3) { h <- arima(r.dm,order=c(p,0,q),inclde.mean=f) AIC[p+,q+] <- -*h$loglik+*(p+q+) BIC[p+,q+] <- -*h$loglik+(p+q+)*log(n) } 0:3; cbind(0:3,aic); 0:3; cbind(0:3,bic) AIC selects the ARMA(3,) model and BIC selects the ARMA(,0) model. Unfortnately, AIC and BIC rarely select the same model. And to make matters worse, there are a lot more modelselection criteria that cold be sed. So it seems that atomatic model selection with a model-selection criterion still contains a sbective element becase we have to select the model-selection criterion first. 0

11 Exercise: Plot the (demeaned) growth rates of the qarterly GDP, the residals from the ARMA(,0) model, which is the best model for this time series according to BIC, and the fitted vales. par(mfrow=c(3,),mar=c(,,,)) p <- ; q <- 0 h <- arima(r.dm,order=c(p,0,q),inclde.mean=false, transform.pars=true) plot(d[:n],r.dm,type="l",ylim=c(-0.04,0.04)) plot(d[:n],h$residals,type="l",ylim=c(-0.04,0.04)) plot(d[:n],r.dm-h$residals,type="l",ylim=c(-0.04,0.04)) This ARMA model explains very little. The residals are almost of the same size as the data.

12 The oint probability density of a sample x,,xn from a Gassian AR() process = + with i.i.d. N(0, ) innovations t is given by,, =,,, = = = exp exp. The maximm likelihood (ML) estimates and the conditional ML estimates of the parameters and are obtained by maximization of the log likelihood fnction and the log likelihood fnction conditioned on the first observation, respectively, with respect to and. We have and log,, ;," = log & ' log log,, = log. Exercise: Find the conditional ML estimates of the parameters and.. ML MM