Testing Linear Restrictions: cont. The F-statistic is closely connected with the R of the regression. In fact, if we are testing q linear restriction, can write the F-stastic as F = (R u R r)=q ( R u)=(n k) ; where R u and R r are the R from the unrestricted and restricted model, respectively. So far we have seen how to write down the unrestricted (large) model and the restricted (smaller) model and test whether the restrictions are "true" or not, via an F-test. The construction of the F-test requires the estimation of both models. There is an alternative test, known as Wald test, which allows to test for multiple linear restrictions via the estimation of the large model only. Indeed, later in the course, we ll see other advantages (in terms of exibility and robustness) of the Wald test over the F-test. Consider, y = X + u where y is n ; X is n k; is k and u is n : Specify restrictions using matrix R and vector r, in R = r where each row of R is a q k vector, where q < k denotes the number of restrictions and so r is q :. Suppose we want to test H : = vs H : 6= : equivalent to = vs 6= R = : : : ; r =. Suppose we want to test H : = 3 = ::: = k = vs H : i 6= for at least one i = ; :::; k: In this case we test 3 : : : R = 6 4.... 7 5 ; r = 6 4 where R is (k ) k and r is (k ).. 3 7 5
3. Suppose we want to test H : + 3 = vs H : + 3 6= : In this case we test R = : : : ; r = 4. Suppose we want to test H : 3 4 = and = vs H : 3 4 6= and/or 6= : In this case we test R has two rows, one for =, and one for 3 4 = : : : : R = ; : : : r = Wald tests We want to test As we do not know ; we replace it with : b Intuitively, r is close to zero under the null, and far away from zero under the alternative. Now, assume that A.MLR-A.MLR6 hold. (recall, that A.MLR6 requires ujx ' N(; I n u)): We know that, ^ = + (X X) X u and so R ^ = R (X X) X u; and regardless which hypothesis is true, E jx = R h var i R jx = E R ^ R ^ jx = RE ^ ^ jx R = ur (X X) R and so rjx ' N R r; ur (X X) R Thus under the null, Using the result that if rjx ' N ; R (X X) R Y N k [; ]
then: it follows that under the null, = (Y ) (Y ) (k) ; r h ur (X X) R i r (q) where q, the number of restrictions, is the row dimension of R. The problem is that we do not know u and we need to replace it with its estimator b u = u u=(n k): De ne the Wald statistic as: W = r hb ur (X X) R i r For n large, b u is very close to u; and so for n large, we (do not) reject H, at a 5% signi cance level, if W is (smaller) larger than the 95% critical value of a (q) : For small n; instead W=q is distributed as F (q; n k): In fact as n ; qf (g; n k) approaches a (q): LM tests: testing based on the restricted regression We have seen that the F test required estimation of both the unrestricted and restricted model, while the Wald test requires only the estimation of the unrestricted model. Finally, the LM (Lagrange Multiplier) test requires only the estimation of the restricted model. Consider the unrestricted model: y i = + x ;i + 3 x 3;i + 4 x 4;i + 5 x 5;i + u i and consider the restricted model y i = + x ;i + 5 x 5;i + u i We want to test H : 3 = 4 = versus H : 3 6= and/or 4 6= : Estimate the restricted model, take the residuals bu i = y i b b x ;i b 5 x 5;i : Now, run a OLS regression with bu i as a dependent variable and x 3;i and x 4;i as regressors. Compute the R from that regression. Under the null, and as n ; nr is distributed as a () : Intuitively, under the null, x 4;i and x 5;i do not explain y i ; thus we expect an R very close to zero. 3
Large Sample Properties of OLS So far we have considered properties of OLS which hold for any sample size n; for n > k: For example, under A.MLR-A.MLR5 b is BLUE, i.e. best unbiased estimator, this holds for any n > k: Also, under A.MLR6, ujx ' N(; I n u)) we have seen that t-statistics have exactly a Student-t distribution with n k degrees of freedom, F-statistics have exactly a Fisher-F distribution with g and n k degrees of freedom, where g denotes the number of restrictions. However, A.MLR6 is quite strong, often the error has a distribution di erent from the normal. Basically, we want to be able to do valid inference even when A.MLR6 does not hold. This is possible, as n gets large. How large is large? It depends on the problems, as for OLS properties, say that n > is reasonably large. We want to show that (i) b is consistent for and (ii) n = b is asymptotically normal. From (i) and (ii), we ll be able to show that t-statistics, under the null, have a standard normal distribution as n gets large, and that F-statistics (multiplied for the degree of freedom at the numerator), under the null, have a chi-squared limiting distribution. Therefore, when n is large, can perform hypothesis tests even without assuming that the error has a normal distribution. Consistency Let W n be an estimator of a parameter ; based on a sample (Y ; Y ; :::; Y n ): We say that W n is consistent for ; if for any " > (" small), Pr (jw n j > ") as n () that is as n the probability that W n and are far away for more than " is going to zero. Otherwise, W n is inconsistent for : When () holds, we also say that is the probability limit (plim) of W n ; or p limw n = Properties of plim. Suppose that p limt n = and p limu n = : We have: (P) p lim (T n + U n ) = + ; i.e. the plim of the sum is the sum of the plims. (P) p lim (T n U n ) = ; i.e. the plim of the product is the product of the plims. (P3) p lim (T n =U n ) = =; provided 6= ; i.e. the plim of the ratio is the ratio of the plims. (P4) for any continuous function f; p limf(t n ) = fp lim(t n ) = f() How can we check that an estimator is consistent? Typically, estimators are function of sample means, and then consistency is checked via the Law of Large Numbers. 4
Law of Large Numbers Let (Y ; Y ; :::; Y n ) be identically and independently (iid) distributed, with mean Y. Then, for " > ; Pr n Y i Y > " as n or p limn P n Y i = Y : Thus, for the law of large numbers, the sample mean is a consistent estimator of the mean. Asymptotic Normality Let fz n ; n = ; ; :::g be a sequence of random variables. We say that Z n is asymptotically standard normal, if for any number z; Pr (Z n < z) (z) for n ; where (z) is the CDF (cumulative distribution function) of a standard normal. In other words, Z n is asymptotically normal if its CDF converges to that of a standard normal as n : How can we check whether a sequence of random variables is asymptotically normal? Via the Central Limit Theorem. Central Limit Theorem Let (Y ; Y ; :::; Y n ) be identically and independently (iid) distributed, with mean Y and variance Y : Then, Pr n = n = Y i Y Y Y i Y Y IMPORTANT: Suppose that b Y n = Y i b Y Y is asymptotically normal < z = (z) for n is a consistent estimator of Y : Then, is asymptotically normal In other words, if we use a consistent estimator of the standard deviation, instead of the standard deviation itself, Central Limit Theory still applies. Consistency of OLS Let ^ = (X X) X y and let A.ML (linearity), A.MLR ((y i ; X i ) identically and independently distributed, A.MLR4 (no perfect collinearity), and replace A.MLR3 (E(ujX) =E(u) = ) with A.MLR3 : E(u) = and E (X u) = (i.e. u is uncorrelated with X): Note that A.MLR3 implies A.MLR3 but not the other way round. Result LS-OLS-: Let A.MLR, A.MLR, A.MLR3, A.MLR4 hold. Then, p lim b = ; 5
i.e. b is consistent for : Note that in nite sample b may be biased, but as n ; b gets closer and closer to : Sketch of Proof: ^ = (X X=n) X u=n By Properties P-P4 of the plim, p lim ^ = (p lim (X X=n)) p lim (X u=n) Given A.MLR, by the law of large number p lim (X u=n) = Cov(X u) and Cov(X u) = by A.MLR3. Computing Inconsistency of OLS We now see that in the case of omitted relevant variable, OLS is not only biased but also inconsistent. Back to our old example. True model y i = + x ;i + 3 x 3;i + u i () with E(u i ) = : Cov(u i x i ) = ; x i = (x ;i ; x 3;i ): We estimate: y i = + x ;i + e i (3) Note that, e i = u i + 3 x 3;i : Now, if x ;i is correlated with x 3;i then E(e i jx 3;i ) 6= E(e i ) and is not zero. Thus, b is biased. b = Now, by the law of large number P n x ;i b x yi b y P n x ;i b x = + 3 n P n x ;i b x x3;i b x3 n P n x ;i b x + n P n x ;i b x ui n P n x ;i b x p lim n n X = Cov(x ; x 3 ) x ;i b x x3;i b x3 p lim n n X = V ar(x ) x ;i b x 6
p lim n n X = Cov(x ; u) = x ;i b x ui p lim b = + 3 Cov(x ; x 3 ) V ar(x ) 7