Multiple Linear Regression

Transcription

1 Multiple Linear Regression Asymptotics Asymptotics

2 Multiple Linear Regression: Assumptions Assumption MLR. (Linearity in parameters) Assumption MLR. (Random Sampling from the population) We have a random sample: satisfying the equation above Assumption MLR.3 (No perfect Collinearity) In the sample, none of the independent variables is a linear combination of the others. Assumption MLR.4 (Zero Conditional Mean) this implies Assumption MLR.5 (Homoskedasticity) Var( u x σ, x,..., x k ) These are the Gauss-Markov assumptions. Under MLR. to MLR.5 OLS is BLUE Asymptotics

3 Consistency Under the Gauss-Markov assumptions OLS is BLUE (Best Linear Unbiased Estimator) With only MLR. to MLR.4 we have shown that OLS is unbiased Under weaker assumptions, we cannot prove unbiasedness In those cases, we may be happy with estimators that are consistent, meaning that as n, the distribution of the estimator collapses to the parameter value (or: the probability that the estimator is really close to the true parameter converges to ) Asymptotics 3

4 Sampling Distributions as n n 3 n < n < n 3 n n Asymptotics 4

5 Consistency of OLS Under MLR. to MLR.4, the OLS estimator is consistent besides being unbiased For unbiasedness, assumption MLR.4 was crucial: E(u x, x,,x k ) 0 Consistency can be proved with the weaker assumption MLR.4 : E(u) 0 and Cov(x,u) 0, for,,, k Will need to take probability limits (plim) to establish consistency Asymptotics 5

6 Rough proof of Consistency For the simple linear regression, using assumptions MLR. to MLR.4 : ˆ plimˆ ( ) i ( ) u n x x ( ( x x ) y ) ( x x ) + i because Cov ( n ( x x ) ) ( ) + ( x, u) 0 i Cov i ( x, u) Var( x ) i E(u) 0 and Cov(x,u) 0, for,,, k is a crucial assumption Without this assumption, OLS will be inconsistent Asymptotics 6 i

7 Inconsistency with omitted variables Just as we analysed the omitted variable bias earlier, now we can think about inconsistency, or asymptotic bias True model: You think : u x whereδ + v y y and, Cov ( x, x ) Var( x ) x ~ plim x + + x + v + u, so that Asymptotic bias depends on the population variance and covariance, while bias uses the sample counterparts δ Asymptotics 7

8 Large Sample Inference With assumption MLR.6 (which makes MLR.4 and MLR.5 redundant), the sampling distributions of the estimators are normal, so we could derive t and F distributions for testing Assumption MLR.6 (Normality) The distribution of the population error u is independent of x, x,,x k and u is normally distributed with mean 0 and variance σ We write: u ~ Normal (0,σ ) What if normality is an unreasonable assumption? (remember wage regression) It turns out that only with MLR. through MLR.5, the distribution of the estimators previously derived is still valid asymptotically Asymptotics 8

9 Asymptotic Normality Under assumptions MLR. through MLR.5: (i) where a x n ( ) a ˆ ~ Normal( 0, σ a ) plim ( n rˆ ) and rˆ on all the other regressors; σ i i, are the residuals from regressing a is called the asymptotic variance (ii) σˆ is a consistent estimator of σ (iii) ( ) ( ) a ˆ se ˆ ~ Normal( 0,) Asymptotics 9

10 Asymptotic Standard Errors With assumptions MLR. through MLR.5, we have thus asymptotic normality of the OLS estimators If we cannot guarantee that normality holds (MLR.6) we refer to the standard errors as asymptotic standard errors Formulas for standard errors are exactly the same as before, but the derived distribution of the estimators is not exact, it is only asymptotic se ( ˆ ) σˆ SST ( ), R Asymptotics 0

11 Asymptotic Standard Errors se se and ( ˆ ) ( ˆ ) SST c σˆ SST ( R ) nvar( x ), n, since R converges to some constant Standard errors are expected to decrease at a rate equal to the inverse of the square root of the sample size Can perform hypothesis tests on individual coefficients ust as before, using the Normal distribution (although t-distribution could also be used). Can also perform F tests since the F statistics will also have approximate F distributions in large samples Asymptotics

12 Lagrange Multiplier statistic With only assumptions MLR. through MLR.5 we rely on large sample distributions for inference. But we can use more than t and F statistics The Lagrange multiplier or LM statistic is an alternative for testing multiple exclusion restrictions Asymptotics

13 LM Statistic (cont) Suppose we have a standard model, y 0 + x + x +... k x k + u and our null hypothesis is H 0 : k-q+ 0,..., k 0; the number of restrictions is q First, we ust run the restricted model ~ ~ ~ y ~ 0+ x k qxk q+ u Now take the residuals, u~, and regress u~ on x, x,..., x (i.e. all the variables) LM nr u, k where R u is the R - squared from this regression Asymptotics 3

14 LM Statistic (cont) LM a ~ χ q value, c, from a under the null, so can choose a critical level or calculate the p - value χ q distribution, given a significance With a large sample, the results from an F test and from an LM test should be similar Asymptotics 4

15 LM Statistic (Example) Dependent variable: Log of monthly wages, n064 Test wether Labor market experience (and its square) have an effect on wages, α0.05 q (number of restrictions) LM obs >5.99 so we reect the null Asymptotics 5

16 Asymptotic Efficiency Other estimators besides OLS will be consistent However, under the Gauss-Markov assumptions, the OLS estimators will have the smallest asymptotic variances (check theorem 5.3 in the textbook) We say that OLS is asymptotically efficient Asymptotics 6