7. GENERALIZED LEAST SQUARES (GLS)

Transcription

1 7. GENERALIZED LEAST SQUARES (GLS) [1] ASSUMPTIONS: Assume SIC except that Cov(ε) = E(εε ) = σ Ω where Ω I T. Assume that E(ε) = 0 T 1, and that X Ω -1 X and X ΩX are all positive definite. Examples: Autocorrelation: The ε t are serially correlated. (Ω is not diagonal.) Heteroskedasticity: Ω is diagonal, but diagonals are not identical. [] PROPERTIES OF OLS ˆ β is unbiased (and consistent). ˆ = + ( XX ) X E( ˆ β ) β β ε = β. ˆ Cov( β) = ( X X ) X σ Ω X ( X X ). ( )( ) ( ) ˆ Cov( β) = E ˆ ˆ β β β β = E ( X X ) X εε X ( X X ) = εε = σ ( XX) XE( ) X( XX) ( XX) X ΩX( XX). Comment: All the usual t and F tests are invalid. This is because s (X X) -1 is no longer an unbiased estimator of Cov( ˆ β ). GLS-1

2 [3] GLS ESTIMATOR (3.1) CASE I: Ω is known Assume that Ω is positive definite. Then, there exist a T T nonsingular matrix V, such that V V = Ω -1. Comment: For GLS, it is sufficient to find V such that V V = aω -1, where a is some positive constant. VΩV = I T VΩV = V(V V) -1 V = VV -1 (V ) -1 V = I T I T = I T. Assume that X Ω -1 X is positive definite. Then, Vy = VXβ + Vε satisfies ideal conditions. EV ( ε ) = VE( ε ) = 0 T 1. Cov( Vε) = VCov( ε) V = Vσ Ω V =σ IT. GLS-

3 (Aitken) The BLUE of β is the GLS estimator β = ( X Ω X) X Ω y. Since Vy = VXβ + Vε (***) satisfies ideal conditions, the BLUE must be OLS on (***). But, (X V VX) -1 X V Vy = (X Ω -1 X) -1 X Ω -1 y = β. Comment: β is unbiased (consistent) and BLUE. It is also efficient (asymptotically efficient) if ε is normal. 1 ( β) = σ ( Ω ) 1 Cov X X ( ) 1 = + Ω Ω. 1 β β X X X ε ( )( ) Cov( β) = E β β β β. ( ( ) 1 εε ( 1 ) 1 ) = E X Ω X X Ω Ω X X Ω X = Ω Ω Ω Ω ( X X) X E( εε ) X( X X) = ( X Ω X) X Ω σ ΩΩ X( X Ω X) = σ ( X Ω X). GLS-3

4 β is efficient relative to ˆ β. ˆ Cov( β) = σ ( X X ) X ΩX ( X X ). Cov( β) = σ ( X Ω X ). It is enough to show that (X X) -1 X ΩX(X X) -1 - (X Ω -1 X) -1 is positive semidefinite. But showing this is equivalent to showing that X Ω -1 X - (X X)(X ΩX) -1 (X X) is positive semidefinite. Define P = X Ω -1 - (X X)(X ΩX) -1 X. Then, it can be shown that X Ω -1 X - (X X)(X ΩX) -1 (X X) = PΩP, which is positive semidefinite. Let ε be the residual vector from OLS on Vy = VXβ + Vε. Then, σ = εε /( T k) is an unbiased and consistent estimator of σ Note that Vy = VXβ + Vε satisfies ideal conditions. Therefore, the unbiased and efficient estimator of σ is given by s from OLS on Vy = VXβ + Vε. That is, SSE /( T k) = ( Vy VX β) ( Vy VX β) /( T k) = εε /( T k).. GLS-4

5 Note: 1) All usual tests can be done directly to Vy = VXβ + Vε. ; ) β ~ N( β, σ ( X Ω X) ) and β and ( T k) σ σ ~ χ ( T k) σ are stochastically independent. 3) Even if ε is not normal, ) holds if T is large. ; (3.) Ω is not known Assumption: Let Ω (T T) depend on a p 1 vector, θ (p < T): Ω = Ω(θ). Examples: 1) AR(1): ε t = ρε t-1 + v t, v t iid with N(0,σ ). Ω depends on ρ. ) ARCH: Autoregressive Conditional Heteroskedasticity..1) Let Ω t-1 be the set of information available at time t-1..) ε t ~ N(0,h t ), where h t = var(ε t Ω t-1 ) and, h t = ω + α 1 ε t-1 + α ε t α p ε t-p. Called ARCH(p) model. Ω depends on ω and α 1,..., α p. Ω=Ω ˆ ( ˆ θ ) is consistent for Ω if ˆ θ is consistent for θ. GLS-5

6 Definition: A feasible GLS (FGLS) is given by = ( Ωˆ ) Ωˆ. β f X X X y Comments: 1) No reason to believe that FGLS and GLS are always asymptotically equivalent even if T is large. [For example, See Schmidt.] ) But, often if X is nonstochastic. [4] Efficiency of GLS Maximum-Likelihood Estimator (MLE) Assume that ε ~ N(0 T 1, σ Ω(θ)). Then, log-likelihood function is: l T (β,σ,θ) = constant - (T/) ln(σ ) - (1/)ln[det(Ω(θ))] - {1/(σ )}(y-xβ) Ω(θ) -1 (y-xβ). MLE of β, σ and θ are obtained by maximizing l T (β,σ,θ). These MLEs are efficient when T is large. Almost β β ˆ β, where T is large and X is nonstochastic (strictly f MLE exogenous). [See Schmidt for a counterexample for this almost theorem.] GLS-6

7 Comments: When y = Xβ + ε satisfies SIC other than Ω I T, Vy = VXβ + Vε satisfies all of SIC. When y = Xβ + ε satisfies WIC other than Ω I T, Vy = VXβ + Vε might violate WIC. It might be the case that 1 1 plimt XVV ε = plimt X Ω ε 0k 1. T T Definition: We say that the regressors x t are weakly exogenous with respect to the ε t if E( ε t xt, xt 1,,..., x1 ) = 0 for any t. We say that the regressors x t are strictly exogenous with respect to the ε t if E( ε t xt, xt 1,,..., x1 ) = 0 for any t. Note that WIC only requires weakly exogenous regressors. For cross-section data, the regressors are most likely to be strictly exogenous. But, strictly exogenous regressors are rare in time-series data models. When regressors are weakly exogenous, GLS may be inconsistent. Even when GLS and FGLS are consistent, the asymptotic distributions of GLS and FGLS can be different for some cases. If we strengthen WIC with the assumption of strictly exogenous regressors, Vy = VXβ + Vε satisfies WIC. GLS-7