1 General linear Model Continued..

Transcription

1 Geeral liear Model Cotiued.. We have We kow y = X + u X o radom u v N(0; I ) b = (X 0 X) X 0 y E( b ) = V ar( b ) = (X 0 X) We saw that b = (X 0 X) X 0 u so b is a liear fuctio of a ormally distributed radom vector u ad so is itself ormal. b v N(; (X 0 X) ) From before b bu 0 bu = k = u0 Nu k where N = I X(X 0 X) X 0 Now we ca show that for ay symmetric idempotet matrix with " v N(0; I ) "Q" Sice N is symmetric ad idempotet v trace(q) Now u 0 Nu ( k)b = v k b = (X 0 X) X 0 u as M is idempotet. So =) X( b ) = X(X 0 X) X 0 u = Mu =) ( b ) 0 X 0 X( b ) = u 0 M u = u 0 Mu ( b ) 0 X 0 X( b ) = u0 Mu v k

2 As MN = 0 these two results are idepedet That is u 0 Mu=k u 0 Nu= k v k k v F (k; k) ( b ) 0 X 0 X( b )= (y X b ) 0 (y X b )= k v F (k; k) which ca be used to costruct sigi cace tests o. replace the value of give by the ull ad compute the ratio. More usually we dot test the hypothesis o the whole parameter space. Cosider the liear restrictio Wat to test R = d H 0 : R = d H : R 6= d R is a rxk matrix of costats with Rak(R) = r ad d is a rx vector. May ecoomic hypotheses ca be put o this form. To test y i = 0 + x i + x i + 3 x 3i + u i = 0 : R = (000) : d = 0 = 3 : R = (00) : d = = : R = (0) : d = I each case there is oe restrictio ad Rak(r) =. For R = 0 = 0; = ; = : d = ad the Rak(R) = 3. I all of these cases the restricted model ca be estimated by imposig the restrictios o the data used to estimate the model y i = 0 + x i + x i + 3 x 3i + u i y i = 0 + x i + x i + 3 x 3i + u i whe = y i x i = 0 + (x i x 3i ) + u i whe = 3 y i x i = (x i x 3i ) + u i whe 0 = 0 3 5

3 Note that i each case after you impose r restrictios oly estimate k-r parameters. So each restrictio reduces the umber of parameters by oe. What we are doig is miimisig bu 0 bu subject to the restrictios R = d As a Lagragia L = bu 0 bu + (R d) = (y X) 0 (y X) (R d) The restricted estimator ca be writte = b (X 0 X) R So de e urestricted ad restricted residual sum of squares URSS = (y X b ) 0 (y X b ) RRSS = (y X ) 0 (y X ) The if the ull is true (RRSS U RSS)=r URSS= k v F (r; k) which is the familiar F test. This measures whether a icrease i RSS per degree of freedom from imposig the restrictios is large relative to the variace. If it is large reject the restrictio, if it is small accept the restrictio. What is large depeds o the sigi cace level chose.note that it is distributed F oly if the ull is true. If R b 6= d the will be a log way from b. So a alterative way of lookig at this is that if the restrictios are ot true R b d will be far from zero. To measure how far we could stadardise by the estimated variace of R b d amely V b ( R b d). The with R r b 0 d bv (R b d) (R b d) v F (r; k) bv (R b d) = R(X 0 X) R 0 A importat special case is where the hypothesis to be tested is that a particular equals a particular value, eg = The R = [000:::] : d = R b d = b with b V (R b d) the estimated variace of b So b V ar( b ) v F (; k) 3

4 Now if w v F (; k) =) w v t k so which is the commoly used t test b se( b ) v t k These are small sample tests ad require the model be liear ad the errors ormal. It these dot hold the have to use asymptotic tests. The 3 widely asymptotic tests are: The likelihood ratio test: uses restricted ad urestricted The Wald test: uses oly the urestricted The Lagrage Multiplier (LM) test: uses oly restricted These are asymptotically equivalet ad are each distributed r (where r is the umber of restrictios) if the ull hypothesis is true. All ca be writte as measures of distace stadardised by a variace covariace matrix, each di erig by which distace is measured. The ML estimates are those which maximise LL(), i.e. the b ; = S( b ) = 0 where S( b ) is the score vector, the derivatives of the LL with respect to each of the k elemets of the vector evaluated at the values, b ; which make S() = 0: We will call these the urestricted estimates ad the value of the Log-likelihood at b, LL( b ). Suppose theory suggests m k prior restrictios of the form R() = 0: If m = k, theory speci es all the parameters ad there are oe to estimate. The restricted estimates maximise L = LL() 0 R() where is a m vector of Lagrage Multipliers. The @ = 0 gives the restricted estimator ; we ca write this S( ) F ( ) = 0 where S( ) is the k Score vector evaluated at the restricted estimates ad F ( ) is the k m matrix of the derivatives of the restrictios with respect to the parameters evaluated at the restricted estimates. Notice that at the derivative of the Log-likelihood fuctio with respect to the parameters is ot equal to zero but to F ( ) : The value of the Log-likelihood at is LL( ) which is less tha or equal to LL( b ): If the hypotheses (restrictios) are true: 4

5 . the two log-likelihoods should be similar, i.e. LL( b ) LL( ) should be close to zero;. the urestricted estimates that satisfy the restrictios R( b ) should be close to zero (ote R( ) is exactly zero by costructio); 3. the restricted score, S( ), should be close to zero (ote S( b ) is exactly zero by costructio) or equivaletly the Lagrage Multipliers should be close to zero, the restrictios should ot be bidig. ILLUSTRATE WITH DIAGRAM FROM KENNEDY These implicatios are used as the basis for three types of test procedures. The issue is how to judge close to zero? To judge this we use the asymptotic equivalets of the liear distributioal results used above i the discussio of the properties of the Liear Regressio Model. Asymptotically the ML estimator is ormal b N(; I() ) asymptotically the scalar quadratic form is chi-squared ad asymptotically R( b ) is also ormal ( b ) 0 I()( b ) (k): R( b ) N(R(); F () 0 I() F ()) This gives us three procedures for geeratig asymptotic test statistics for the m restictios H 0 : R() = 0; each of which are distributed (m), whe the ull hypothesis is true:. Likelihood Ratio Tests LR = (LL( b ) LL( )) (m). Wald Tests W = R( b ) 0 [F () 0 I() F ()] R( b ) (m) 3. Lagrage Multiplier (or E ciet Score) Tests LM = S( ) 0 I( ) S( ) (m): The Likelihood ratio test is straightforward to calculate whe both the restricted ad urestricted models have bee estimated. The Wald test oly requires the urestricted estimates. 5

6 The Lagrage Multiplier test oly requires the restricted estimates. For the liear regressio model, the iequality W>LR>LM holds, so you are more likely to reject usig W. I the LRM, the LM test is usually calculated usig regressio residuals. The Wald test is ot ivariat to how you write o-liear restrictios. Suppose m =, ad R() is 3 = 0. This could also be writte 3 = = 0 ad these would give di eret values of the test statistic. The former form, usig multiplicatio rather tha divisio, is usually better. These formulas are ot used to compute the test. The LR test is usually h log = LL( b i ; y) LL( ; y) v r while the LM ad W are ormally calculated from tests o the regressio residuals.. Istrumetal Variables We kow that if the regressors are ot idepedet of disturbaces the OLS estimates are biased ad icosistet Assume that y = X + u =) b = + (X 0 X) X 0 u =) p lim b = + p lim X0 X X 0 X p lim = XX a positive de ite matrix of full rak ad X 0 u p lim = Xu 6= 0 the p lim b = + XX Xu p lim X0 u So the correlatio of the disturbace term with oe or more of the regressors will make OLS icosistet. Such correlatios ca be caused by measuremet error i oe or more regressors, but there are other possibilities: 6

7 lagged depedet variables autoregressive disturbace simultaeity Ca get cosistet estimator by istrumetal variables. Cosider: y = X + u with var(u) = I but X 0 u p lim 6= 0 Suppose we ca d a data matrix Z of order xl where l > k. variables i Z correlated with variables i X Z 0 X p lim = ZX a ite matrix of full rak Variables i Z are i the limit ucorrelated with u Z 0 u p lim = 0 the take premultiply both sides y = X + u Z 0 y = Z 0 X + Z 0 u var(z 0 u) = Z 0 Z If we use GLS b GLS = b IV = (X 0 Z(Z 0 Z) Z 0 X) X 0 Z(Z 0 Z) Z 0 y = (X 0 P X) X 0 P y with P = Z(Z 0 Z) Z 0 the V ar biv = (X 0 P X) b = (y b IV ) 0 (y b IV )= 7

8 usig Now k or does ot matter asymptotically). b IV = + X0 P X X0 P u X0 P X = X0 Z Z0 Z Z0 X Assume middle term has plim ZZ so p lim X0 P X = XZ ZZ ZX which will be ite o sigular matrix. Similarly p lim Z0 P u = XZ ZZ Zu = 0 as istrumets z are assumed ucorrelated with u i the limit. So the IV estimator is cosistet If l = k so that Z has the same umber of colums as X the X 0 Z is kxk ad osigular. The all simpli es to if l < k the sigular b IV = (Z 0 X) Z 0 y var( b IV ) = (Z 0 X) (Z 0 Z)(X 0 Z) Two Stage Least Squares applicatio of OLS A form of IV ca be see as result of double. Regress each of the variables i the X matrix o Z to give X b bx = Z(Z 0 Z) Z 0 X = P X. Regress y o X b to obtai b T SLS = ( X b 0 X) b X b 0 y = (X 0 P X) (X 0 P y) = b IV As before var( b IV ) = (X 0 P X) b = (y X b IV ) 0 (y X b IV )= 8

9 Choice of istrumets ca use variables from X ay variable thought exogeous ad idepedet of disturbaces lagged values Whe some of the X are used eed to partitio: X = [X X ] ad Z = [X Z ] X is xr X is x(k r) with r < k Z is x(p r) ca show i bx = hx X b where bx = Z(Z 0 Z) Z 0 X So variables i X serve as istrumets for themselves ad the remaiig secod stage regressors are the tted values of X obtaied form regressig X o the full set of istrumets Z Miimum umber of istrumets is k icludig ay variables that serve as their ow istrumets. Asymptotic e ciecy icreases with the umber of istrumets, but so does ite sample bias if select istrumets the P = I ad get the biased ad icosistet OLS estimates. Multicolliearity Have assumed i the past that the explaatory variables were liearly idepedet. This meas that (X 0 X) exists. Perfect multicollieairty implies that (X 0 X) will be a sigular matrix with rak less tha k. Which implies do ot have uique solutios to the ormal equatios As b = (X 0 X) X 0 y the if (X 0 X) is sigular b caot be estimated. I fact it meas that ot all regressio parameters (the whole vector of parameters) are estimable, oly certai liear fuctios of b i. 9