Linear Model Under General Variance Structure: Autocorrelation

Transcription

1 Linear Model Under General Variance Structure: Autocorrelation A Definition of Autocorrelation In this section, we consider another special case of the model Y = X β + e, or y t = x t β + e t, t = 1,..,. where we relax assumption A3, while maintaining assumption A4. We assume that E(e t e t* ) = V(e t ) = σ e if t = t*, = Cov(e t, e t ) 0 if t t*, t = 1,,. While maintaining a constant variance across e t s (as implied by assumption A4), this allows for non-zero covariances across observations. his is particularly relevant in analyzing time series data, where the random variables have some memory, implying that the error terms are correlated over time. his is called autocorrelation. With autocorrelation we have: V(e) = V(e 1,, e ) = is a non-diagonal (full) ( x ) matrix. σ e Cov( e1, e). Cov( e1, e ) Cov( e1, e) σ e. Cov( e, e ).... Cov( e1, e ) Cov( e1, e). σ e ypically, some parametric structure is imposed on Cov(e t, e t ). A common structure is to assume that an autoregressive process generates the error terms. hat is: the random variable e t follows a p th autoregressive process (denoted by AR(p)) if

2 e t = p Σi=1r i e t-i + v t, where (ρ 1,..., ρ p ) are autoregressive parameters, and v t is a random variable satisfying E(v t ) = 0, t = 1,,, E(v t v t ) = Cov(v t, v t* ) = 0 for all t t*, V(v t ) = σ v, t = 1,..., Τ. For simplicity, we consider only a first-order autoregressive process AR(1), implying p = 1: e t = r e t-1 + v t. he First-Order Autoregressive Process Under AR(1), after successive substitutions we have: e t = ρ e t-1 + v t = ρ [ρ e t- + v t-1 ] + v t = ρ [ρ e t-3 + v t- ] + ρ v t-1 + v t = = ρ i e t-i + Σ j=0 ρ j v t-j = Σ i=0 ρ i v t-i. Note that ρ i 0 as i if ρ < 1, ρ i as i if ρ > 1. his means that if ρ < 1, then the distant past (v t-i ) has a small and declining influence on the present (e t ). his is called stationarity. Alternatively, if ρ > 1, then the distant past (v t-i ) has a large and growing influence on the present (e t ). his is called non-stationarity.

3 3 hus, the AR(1) process Is stationary if ρ < 1 Is non-stationary if ρ > 1 Has a unit root if ρ = 1 (this is the border-line case between stationarity and non-stationarity). For practical purpose, we will be interested only in stationary processes, so that we do not need to observe the infinite past to understand and model the present. As a result, we will limit our analysis to the stationary case, and assume that r < 1. Under stationarity ( ρ < 1), we have E(e t ) = Σ i=0 ρ i E(v t-i ) = 0 since E(v t-i ) = 0. V(e t ) = σ e = V(ρ e t-1 + v t ) his implies: = ρ V(e t-1 ) + V(v t ) + ρ Cov(e t-1, v t ) = ρ σ e + σ v + ρ Cov( Σ i=0 ρ i v t-1-i, v t ) = ρ σ e + σ v, since E(v t v t* ) = Cov(v t, v t* ) = 0 for all t t*. (1 - ρ ) σ e = σ v, or s e = s v /(1 - r ). We also know that: Cov(e t, e t-i ) = E(e t e t-i ) = E[(ρ i e t-i + = ρ i E(e t-i ) + Σ j=0 ρ j v t-j ) e t-i ] S j=0 ρ j E(v t-j e t-i ) = ρ i σ e + j=0 Σ ρ j E[v t-j ( Σ m=0 ρ i v t-i-m )] with e t-i = Σm=0 ρ i v t-i-m, = ρ i σ e since E(v t v t* ) = 0 for all t t*. = σ v ρ i /(1 - ρ ), since σ e = σ v /(1 - ρ ).

4 4 hus, under a stationary AR(1) process for e t, we have V(e) = V(e 1,, e ) = σ v 1 ρ 1 ρ ρ. ρ ρ 1 ρ. ρ ρ ρ 1. ρ ρ ρ ρ Using our earlier notation (where V(e) = σ ψ), let σ = σ v. We have: ψ = 1 1 ρ 1 ρ ρ. ρ ρ 1 ρ. ρ ρ ρ 1. ρ ρ ρ ρ It can be shown that 1 ρ ρ 1 + ρ ρ. 0 0 ψ = ρ + ρ, and ρ ρ ρ 1.. P = 1 ρ ρ ρ , where P' P = ψ ρ 1

5 5 Estimation of Model Parameters Under an AR(1) Error Structure Obtain the consistent least squares estimator of β, b s = (X'X) -1 X'Y and the consistent estimator of e, e s = Y X b s. Given e s = e e s1 s, regress e st on e s,t-1 to obtain the least squares t= estimator of ρ, r s = ( Σ e s,t-1 ) -1 ( Σ t= e st e s,t-1 ). ρ s is a consistent estimator of ρ. Let ψ e be the matrix ψ evaluated at r = r s. hus, ψ e is a consistent estimator of ψ. Evaluate the FGLS estimator of β, b fg = [X y -1 X] -1 X y -1 Y. he β fg is a consistent, and asymptotically efficient estimator of β, satisfying b fg» N[b, V(b fg )] as fi. he variance of β fg, V(β fg ), can be consistently estimated: V(b fg ) = s vu [X'(y e ) -1 X], where σ vu is a consistent estimator of σ v s vu = (Y X b fg )'(y e ) -1 (Y - b fg )/( K). Estimation of Model Parameters Under an AR(1) Error Structure: An Alternative Approach We have considered the model specification y t = x t β + e t where e t = ρ e t-1 + v t, t = 1,, (Model A) his implies that: y t-1 = ρ x t-1 β + ρ e t-1, t =,,. Subtracting this expression from [y t = x t β + e t ] gives y t - ρ y t-1 = x t β - x t-1 ρβ + e t - ρ e t-1, t =,,. Recognizing that e t - ρ e t-1 = v t, we obtain the following:

6 6 y t = ρ y t-1 + x t β - x t-1 ρβ + v t, t =,, (Model A*) Note that, except for the omission of the first observation in model A * ), models A and A* are equivalent. However, note that the error term e t in model A exhibits autocorrelation (thus does not satisfy assumption A3), while the error term v t in model A* does not exhibit autocorrelation and satisfies assumption A3. On the other hand, model A is linear in the parameters (and satisfies assumption A1), while model A * is non-linear in the parameters (since rb appears as the coefficient of x t-1 in A * ) and thus does not satisfy assumption A1. his suggests that the parameters (β, ρ) can be alternatively estimated by non-linear least squares applied under assumption A3 to model A*. (See the chapter on non-linear regression). Prediction Under the AR(1) Error Structure Consider the time series model, y t = x t β + e t, e t = ρ e t-1 + v t, t = 1,,. We want to know how to predict y +1 (i.e. one period ahead) given the sample observations. Under AR(1) for e t, and using our earlier discussion of prediction, it can be shown that P 3-1 P = ρ. hus, assuming that r is known, the best linear unbiased predictor of y +1 is x t+1 b g + r [y x b g ]. Alternatively, if r is unknown and needs to be estimated, then a consistent predictor of y +1 is x t+1 b fg + r e [y x b fg ]. Hypothesis esting About the Value of r Consider the hypothesis: H 0 : r = 0 (no autocorrelation) H 1 : r 0 (the e t s follow an autoregressive process of order one, AR(1)).

7 7 An Asymptotic est It can be shown that ρ s = ( Σ e s,t-1 ) -1 ( Σ t= t= estimator of ρ, with asymptotic distribution r s» N[r, (1-r )/] as fi. e st e s,t-1 ) and is a consistent It follows that z = (r s - r)/[(1-r )/] 1/ fi N(0, 1) as fi. hus, under H 0, (where ρ = 0), z = () 1/ r s fi N(0, 1) as fi. his suggests the following asymptotic test procedure: Choose the significance level α Find z 0 such that α/ = P[z > z 0 z N(0, 1)] = P[z < -z 0 z N(0, 1)]. Do not reject H 0 if z 0 () 1/ ρ s z 0, Reject H 0 if () 1/ ρ s > z 0, or () 1/ ρ s < -z 0. he Durbin-Watson est Consider the Durbin-Watson statistic t=1 t= d = ( Σ e st ) -1 [ Σ (e st - e s,t-1 ) ]. In general, it can be shown that 0 < d < 4, and that d under H 0. his suggests not rejecting H 0 when d is around, and rejecting H 0 when d is either much less than or much greater than. As expected, making this procedure practical requires knowing the distribution of the Durbin-Watson statistic d under the null hypothesis H 0. Unfortunately, assuming that the v t are normally distributed, Durbin and Watson (who proposed this test in 1950) were not able to establish the distribution of d under H 0. However, they were able to approximate it. hey proposed a test procedure based on this approximation, procedure that is commonly used in testing for autocorrelation.