Asymptotic theory for linear regression and IV estimation

Size: px

Start display at page:

Download "Asymptotic theory for linear regression and IV estimation"

Erick Holt
6 years ago
Views:

1 Asymtotic theory for linear regression and IV estimation Jean-Marie Dufour McGill University First version: November 20 Revised: December 20 his version: December 20 Comiled: December 3, 20, : his work was suorted by the William Dow Chair in Political Economy (McGill University), the Bank of Canada (Research Fellowshi), a Guggenheim Fellowshi, a Konrad-Adenauer Fellowshi (Alexander-von-Humboldt Foundation, Germany), the Canadian Network of Centres of Excellence [rogram on Mathematics of Information echnology and Comlex Systems (MIACS)], the Natural Sciences and Engineering Research Council of Canada, the Social Sciences and Humanities Research Council of Canada, and the Fonds de recherche sur la société et la culture (Québec). William Dow Professor of Economics, McGill University, Centre interuniversitaire de recherche en analyse des organisations (CIRANO), and Centre interuniversitaire de recherche en économie quantitative (CIREQ). Mailing address: Deartment of Economics, McGill University, Leacock Building, Room 59, 855 Sherbrooke Street West, Montréal, Québec H3A 27, Canada. EL: () ; FA: () ; jean-marie.dufour@mcgill.ca. Web age: htt://

2 Contents. Estimator consistency 2. Consistency of least squares in linear regression 2 3. Instrumental variables 4 i

3 . Estimator consistency Let y, y 2,...be a sequence of observations and ˆθ ˆθ (y, y 2,...,y ) (.) an estimator for a k arameter vector θ. We say that ˆθ is consistent (or weakly consistent) for θ when ˆθ θ. (.2) his is also written: his means that lim ˆθ θ. (.3) lim P[ ˆθ θ > ε ] 0, ε > 0 (.4) where reresents the Euclidean distance. We say that ˆθ is strongly consistent consistent (or weakly consistent) for θ when i.e., when It is easy to see that strong consistency entails weak consistency. We say that ˆθ is asymtotically unbiased for θ when ˆθ a.s. θ, (.5) [ ] P lim ˆθ θ. (.6) lim E( ˆθ ) θ. (.7) In general, a consistent estimator is not necessarily asymtotically unbiased, for examle when the estimator does not have a finite mean. Similarly an asymtotically unbiased estimator may not be consistent, for examle if it unbiased but not consistent. In the following roosition, we give a general condition under which asymtotic unbiasedness entails consistency.. Proosition If the estimator ˆθ satisfies lim E( ˆθ ) θ (.8) and then ˆθ θ. lim V( ˆθ ) 0, (.9)

4 2. Consistency of least squares in linear regression Let us now consider a linear regression model of the form y y β + ε (2.) where β is a fixed k arameter vector, y and ε are vectors, is a k matrix, y ε y 2 ε 2. y [x,x 2,..., x k ], ε. ε x x 2 x k x 2 x 22 x 2k... x x 2 x k, (2.2) Instead of the finite-samle assumtions of the classical linear model, we make the following asymtotic assumtions:. is nonsingular with robability one for all k (2.3) hen, we have: Σ where det(σ ) 0, (2.4) ε 0, (2.5) ε ε > 0. (2.6) ˆβ ( ) y β +( ) ε (2.7) ( ) β + ε β + Σ 0 β (2.8) and the least squares estimator is (weakly) consistent. Similarly, the unbiased least squares estimator of σ 2, s 2 k ˆε ˆε (2.9) 2

5 where ˆε M()ε [I ( ) ]ε, satisfies s 2 k ε M()ε k ε [ I ( ) ] ε [ ε ε ε ( ) ] ε k k [ ( ) ( ) ε ε ε ε ] (2.0) where ε ε σ 2, (2.) ( ) ( ) ( ) ε ε 0 Σ 0 0, (2.2) so that In other words, s 2 is a consistent estimator of σ 2. If furthermore, ε satisfies a central limit theorem, namely we have, using (2.7), s 2 σ 2 (2.3) L ε N[ 0, σ 2 ] Σ, (2.4) [ ˆβ β] ( ) ε ( ) ε L N[ 0, σ 2 Σ ]. (2.5) In other words, the distribution of [ ˆβ β] is aroximately normal for large enough. his entails that the distributions of the t and F statistics can be aroximated by the distributions obtained under the assumtions of the Gaussian classical linear model. [he details of the arguments to establish asymtotic distributions are not resented in this course.] 3

6 3. Instrumental variables If and ε are asymtotically correlated, i.e. we have ε σ ε 0, (3.) ) ˆβ β +( ε β + Σ σ ε β (3.2) and the least squares estimator is not consistent for β. Alternative estimation methods are tyically required to deal with this roblem. he instrumental variables (IV) method is the simlest alternative to least squares when exlanatory variables and disturbances are asymtotically correlated. Instrumental variables can be defined as variables which are (asymtotically) uncorrelated with the disturbance term but still correlated with the variables in. More recisely, suose with a l matrix Z of variables with the following roerties: Z ε 0, (3.3) Z Z and Z are full rank matrices with robability one for all, (3.4) Z Z Σ Z where det(σ Z ) 0, (3.5) Z Σ Z where rank(σ Z ) k. (3.6) Assumtion (3.3) means that Z and ε are (asymtotically) uncorrelated (instrument validity). Assumtion (3.4) means that Z Z and Z are full rank matrices, Assumtion (3.5) means they are not (asymtotically) collinear, while Assumtion (2.4) means the variables in Z contain information about all the variables in (asymtotically) Consider now equation (2.) and multily both sides by Z : If we then multily by, we get: Z y Z β + Z ε. (3.7) Z y Z β + Z ε. (3.8) Consider first the case where the number of instruments is equal to the number of exlanatory variables (l k), so that Z is a square invertible matrix. In view of assumtion (3.3), we exect Z ε to be close to zero for large enough. his suggests to estimate β by solving the equation Z y Z β, (3.9) 4

7 which leads to the estimator: β (Z ) Z y. (3.0) his estimator is called the IV estimator of β based on the instrument Z (in the case where l k). It is easy to see that β is consistent for β : β β +(Z ) Z ε ( ) β + Z Z ε β + Σ Z 0 β (3.) It is interesting to note the least squares estimator ˆβ can be viewed as a secial case of the IV estimator obtained by taking Z. Of course, ˆβ will be consistent only if the orthogonality condition (2.5) holds. Similarly, if we allow the number of instruments to be larger than the number of exlanatory variables (l k), suose temorarily that Z is fixed. hen the covariance matrix of the error term Z ε in (3.7) is: V ( Z ε ) E [ Z εε Z ] Z E(εε )Z σ 2 Z Z. (3.2) his suggests to consider the following generalized least squares estimator: If l k, Z is a square invertible matrix, so that β [ Z(Z Z) Z ] Z(Z Z) Z y. (3.3) β (Z ) (Z Z)( Z) Z(Z Z) Z y (Z ) Z y (3.4) reduces to the estimator in (3.0). So β is also called the IV estimator of β based on the instrument Z (in the general case where l k). Again, it is easy to see that β is consistent for β : β + [ ( )( Z Z β [ Z(Z Z) Z ] Z(Z Z) Z y β +[ Z(Z Z) Z ] Z(Z Z) Z ε ) ( ) ] ( Z )( Z Z ) ( ) Z ε β +[Σ ZΣ Z Σ Z] Σ ZΣZ 0 β. (3.5) If furthermore, Z ε satisfies a central limit theorem, namely Z L ε N[ 0, σ 2 ] Σ Z, (3.6) 5

8 we find [ β β] [ ( )( ) ( Z Z ) ] ( )( ) ( ) Z Z Z ε Z [ ( )( ) ( ) ] ( )( ) ( ) Z Z Z Z Z Z ε L N[ 0, σ 2 [Σ ZΣ Z Σ Z] ]. (3.7) ests based on this distribution can also be derived. [he details are not resented in this course.] 6

Statistical models and likelihood functions

Statistical models and likelihood functions Jean-Marie Dufour McGill University First version: February 1998 This version: February 11, 2008, 12:19pm This work was supported by the William Dow Chair in