Heretoskedasticity: Cont.

Transcription

1 Heretoskedasticity: Cot. We have see last time how to costruct heteroskedasticity robust t-test, i.e. b i 1= bi = (W hite) se bi 1= (W hite)se bi We ow see how to costruct heteroskedastic robust Wald test ad LM test. That is we costruct test for multiple liear restrictios which are asymptotically valid eve i the presece of coditioal heteroskedasticity. Heteroskedastic robust Wald tests. H 0 : R = r vs H 1 : R 6= r where R is q k; with q be the umber of restrictios. The usual Wald test is costructed as 0 W = R^ r hb ur (X 0 X) 1 R 0i 1 R^ r Recall that whe performig a Wald test we estimate oly the urestricted model. If E (uu 0 jx) 6= ui ; the the usual Wald test has o loger a (q) limitig distributio uder the ull. This is because, we are usig a estimator of the variace which is ot cosistet for the true variace. Though, we ca have robust Wald test if we use a heteroskedastic robust covariace matrix. How to implemet this? We eed to replace (b ur(x 0 X) 1 R 0 ) 1 with (R(X 0 X) 1 (X 0 uu 0 X)(X 0 X) 1 R 0 ) 1 : Thus, the heteroskedastic robust Wald test is give by: WR = R^ 0 r hr(x 0 X) 1 (X 0 uu 0 X)(X 0 X) 1 R 0i 1 R^ r Uder the ull, WR is asymptotically (q): Thus, we ca simply compare the value we obtai for WR with the 90% or 95% critical values of a (q): F-tests Recall that a easy way of performig tests for liear restrictio is the use of F test, where F = (SSRr SSR u)=q ; SSR u =( k) where SSR u ad SSR r are respectively the sum of the square errors from the urestricted ad the restricted models. We have see that i the case of iid ormal error ad coditioal homoskedasticity, uder the ull, F F (q; k): If we drop the assumptio of ormal errors (but keep all the others), it is o 1

2 loger true that uder the ull F F (q; k), however it is true that uder the ull qf! d (q): Thus, ca still perform tests for liear restrictio based o the simple F statistic. However, if the assumptio of coditioal homoskedasticity is violated, the it is o loger true that uder the ull qf! d q; this bacause the F statistics implicitely use the usual variace estimator (i.e. as the usual/ o robust Wald test). Thus iferece based o the chi-square with q degree of freedom is ot valid eve i large sample. For example the probability of type I error we commit by rejectig the ull if we get a value larger tha the 95% critical value of a chi-square with q degree of freedom is ot 5%; but it may be larger or smaller tha 0:05: There is o a atural way of makig a heteroskedastic robust F-test. Heteroskedastic Robust LM tests. Urestricted model: y i = 1 + x ;i + 3 x 3;i + 4 x 4;i + 5 x 5;i + u i Restricted model: y i = 1 + x ;i + 3 x 3;i + u i We estimate ONLY the restricted model, take the residuals bu i;r : Now, regress bu i;r all regressors (or oly o the excluded), ad take the R from this regressio. If coditioal homoskedasticity hold, the uder the ull, R is asymptotically distributed as () (we have two restrictios!!). Ufortuately, i the presece of coditioal heteroskedasticity, the R test does o loger provide valid iferece (e.g. is o loger asymptotically distributed as () whe the restrictios hold). A robust versio of LM test ca be costructed. Step 1: As before compute the residuals from the restricted model, bu i;r Step : Regress x 4;i o x ;i ; x 3;i, take residuals bu 4;i : Also, regress x 5;i o x ;i ad x 3;i ; take the residuals bu 5;i : I geeral, each of the excluded regressors is regressed o the icluded oes ad the residuals are take. Step 3: Costruct the products bu i;r bu 4;i ad bu i;r bu 5;i : I geeral we takes the products of residuals from restricted models ad residuals from the regressio of excluded variables o all icluded variables; there are as may products as the umber of excluded variables. Step 4: Regress 1 o bu i;r bu 4;i ad bu i;r bu 5;i (yes, the depedet variable is a vector of all oes!). Costruct SSR 1 ; where SSR 1 is the sum of squared residuals from the last regressio. Uder the ull, SSR 1 is asymptotically distributed as () (as here we have two restrictios!). Tests for the Null of (Coditioal) Heteroskedasticity Wat to test: H 0 : E(u i jx) = u versus H A : E(u i jx) is fuctio of X Thus we wat to have a test for the ull of coditioal homoskedasticity versus the alterative of a coditioal variace which is a fuctio of the regressors. (ii) Breusch ad Paga test Let bu i be the residuals from the regressio of y i o costat, x ;i ; :::; x k;i : Regress bu i o costat, x ;i ; :::; x k;i ; take the R from this regressio, Uder

3 the ull, R is asymptotically distributed as (k 1): Note that the degree of freedom are give by the umber of variables (i additio to itercept) o which we have regressed u i : Logic is simple, uder the ull, E(u i jx) is costat, ad thus u i is ucorrelated with ay fuctio of the X: As we do ot kow u i ; we replace it with bu i ; ad make use of the fact that, as the OLS estimator for are cosistet, tests based o bu i are asymptotically equivalet to tests based o u i : (i) White Test Very similar to Breusch ad Paga. Though, istead of regressig bu i o costat, x ;i ; :::; x k;i we regress bu i o costat, x ;i; :::; x k;i ; x ;i ; :::x k;i ; x ;ix 3;i ; :::; x ;i x k;i ; :::; x k 1;i x k;i : I other words, we regress the squared residuals o all x; o all their squared ad o all the possible cross products amog the x; ad take the R : Uder the ull, the R is a (q); where q = (k 1) + (k 1)(k )=; i.e. q is the umber o variables (i additio to itercept) o which we regress bu i : The logic uderlyig White s test, is that it may be possible that u i is ot correlated with x i ; but is istead correlated with x i or some oliear fuctio of the x: Basically, the White test has power agaist a large spectrum of alteratives, though it has less power versus the alterative that u i is a liear fuctio of the x: (iii) Modi ed White tes Regress bu i o by i ad by i ; where by i = b 1 + P k b i= k x i;k : Take the R : Uder the ull of coditioal homoskedsticity, R is asymptotically distributed as (): Importat: The test for coditioal heteroskedasticity outlied above have a well de ed chi-squared distributio ad so ca be used to perform valid (large sample) iferece, uder the assumptio of coditioal homokurtosis, i.e. if E(u 4 i jx i) = 4 ; where 4 is a costat. I priciple, ca costruct test for coditioal homoskedasticity robust to coditioal heterokurtosis etc...gets too complicated! Example d price = 1:77(9:5)+0:001(:00064)lotsize+:13(:013)sqrf t+13:85(9:01)bdrms ote egative itecept (bad!). = 88, R = :67: "Usual" stadard errors i brackets. We do ot kow whether there is coditioal heteroskedsticity or ot... Do a Breusch-Paga test. Hit: if we reject with BP test, o eed to perform a White test; if we fail to reject with BP, better to perform also White ad modi ed White test. Regress bu i o costat, lotsize; sqrft; bdrm; get a R = :16: Thus, R = 88 :16 = 14:9: A sample of 88 is ot that large, though little bit of cautio i applyig asymptotic results. We kow that uder the ull of coditioal homoskedasticity, R d! (3): The 95% critical value of (3) is 7:8; thus we reject the ull at 5%: Also, ote that Pr ( (3) > 14:9) = :008; very tiy P-values, ca reject at ay level! Very ofte, by usig a log-liear model istead of a liear model, we "reduce" 3

4 heteroskedasticity. Try... d log(price) = 5:61(:65) + :168(:038) log(lotsize) +:7(:093) log(sqrf t) + :037(:08) log(bdrms) = 88; R = :643: Take squared residuals from the log regressio, bu i ad regress o costat, log(lotsize); log(sqrft); log(brms); get a R = :048: Now, R = 88 0:048 = 4:: Now, 4: < 7:8; caot reject at 5%: Look at P value; Pr ( (3) > 4:) = :39; well ca oly reject at sigi cat level above 4%! As we do ot reject, we also try the modi ed White test. By regressig bu i o costat, log(price); d log(price) d we get a R = 0:039; smaller tha above. Thus, do ot reject. Icidetally, it is quite commo that the level model display coditioal heteroskedasticity while the logged model does ot! Weighted Least Squares We have see: how to costruct t-test, Wald-test, ad LM which are robust to coditioal heteroskedasticity. O the other had, better to forget the F- test, as we caot robustify it. Also, we have see how to test for the ull of coditioal homoskedasticity (i.e. NO coditioal heteroskedasticity). I practice, rst we test for coditioal homoskedasticity. If we fail to reject, good...ca proceed usig OLS ad perform t, F, Wald. Still, it remais the issue that our estimates are o loger e ciet. For the time beig, suppose we kow the "true" coditioal variace. That is suppose that, E(u i jx) = E(u i jx ;i ; :::; x k;i ) = uh (x ;i ; :::; x k;i ) ad suppose that we kow the fuctioal form of h: Note that h is mappig from R k 1 to R + : De e yi = y i =h 1= (x ;i ; :::; x k;i ) ; x j;i = x j;i =h 1= (x ;i ; :::; x k;i ) for j = ; :::; k: Cosider, y = X + u where y = (y 1; :::; y ) 0 ad X is a k matrix with geeric elemet x j;i for j = ; :::; k ad i = 1; :::; ; x 1;i = 1=h1= (x ;i ; :::; x k;i ) ; ad u i = u i=h 1= (x ;i ; :::; x k;i ) : This is simple...cosider y i = 1 + x ;i + u i where E(u i jx ;i) = uh(x ;i ): Divide both left ad right had sides by h 1= (x ;i ); we have: y i = 1 + x ;i + u i 4

5 where E(u i jx ;i ) = E u i h(x jx ;i) ;i = 1 h(x E ;i) u i jx ;i = 1 h(x h(x ;i) ;i) u = u: Thus, we are back to the case of coditioal homoskedasticity!!! Now, we ru a OLS regressio of y i 1=h1= (x ;i ; :::; x k;i ), x j;i =h 1= (x ;i ; :::; x k;i ) j = ; :::; k ad we compute. b wls = (X X 0 ) 1 X y Result WLS-1: Let A.MLR1-3, A.MLR4 hold. Also, assume that E(u i jx) = uh (x ;i ; :::; x k;i ) : The, i ite sample b wls is BLUE ad i large sample, 1= b wls d! N 0; up lim (X X 0 =) 1 Note that we have required A.MLR3, i.e. E(u i jx) = 0; istead A.MLR3, i.e. E(X 0 u=) = 0: Sketch of proof: For simplicity, assume the case of the simple liear model, y i = 1 + x ;i + u i ; where E(u i jx ;i) = uh(x ;i ): We regress y i =h 1= (x ;i ) o 1=h 1= (x ;i ) ad x ;i =h 1= (x ;i ); ad we obtai b ;wls = P x ;i b x y i P x ;i b x b y where b x = 1 P x ;i = 1 P x ;i =h 1= (x ;i ) ad b y = 1 P yi = 1 P y i =h 1= (x ;i ): Note that yi b y = x ;i b x + (u i b u ) ad that P x ;i b x b u = 0; where b u = 1 P u i : Thus, 1= P 1= x ;i b x ;wls b = P 1 x ;i b x u i Now, E x ;i b x = E x ;i b x = 0 u i 1 h 1= (x ;i ) E(u ijx ;i ) 5

6 Recallig that x ;i ad u i are iid becase y i; x ;i ad h(x ;i ) are iid, V ar 1= X = 1 X! x ;i b x u i E x ;i b x u i = E E x ;i b x u i jx ;i 1 = E x ;i b x h(x ;i ) E u i jx ;i = E x ;i b x u = var(x ;i ) u as E u i jx ;i = u h(x ;i ): Thus, because of the cetral limit theorem, 1= X x ;i b x u i d! N(0; uvar(x ;i )) ad so 1= b ;wls d! N(0; u =var(x ;i )) Example Cosider rst the simple savig fuctio, s i = 1 + ic i + u i where s i ad ic i are savig ad icome of household i: We have data o 100 households i We rst estimate the model above by usig OLS, ad the by usig WLS assumig that E(u i jic i) = uic i : Thus, estimatio by WLS etails regressig s i =ic 1= i o 1=ic 1= i ad o ic i =ic 1= i : We obtai: for OLS, b 1 = 14(655); b = :147(0:58); R = :06: For WLS, b 1;wls = 14:1(480); b ;wls = :17(:057); R = :085: Note that, MPS (margial propesity for savig) is substatially higher for WLS. This is ot surprisig at all, give that we have imposed a give form of heteroskedasticity. 6

7 Weighted Least Squares: cot. We have see that if we kew the true coditioal variace, the we ca implemet Weighted Least Squares ad get e ciet or asymptotically e ciet estimators. Though, it is somewhat uusual to kow (up to a multiplicative factor) the covariace matrix. There is oe case, i which ideed we kow the fuctioal form of the covariace. This is the case i which we have data o averages over di eret groups ad, at the idividual level, we ca rely o the homoskedasticity assumptio. Suppose that we wat to estimate the cotributio of a idividual to her/his pesio pla as a fuctio of how much the employer cotributes (this has clear importat implicatios for policy, social security issues etc.). Ideally we have data o sigle employees at di eret rms, i.e. the equatio at idividual level is cotr i;e = 1 + ear i;e + 3 age i;e + 4 mrate i + u i;e where cotr i;e ; ear i;e ; age i;e are aaul cotributio to pesio pla, aaual earigs ad age of employee e at rm i; while mrate i is the amout rm i put i frot of every dollar put i the pesio pla by oe of its employees. Suppose that E(u i;e jx) = u; i.e. at idividual level coditioal homoskedasticity holds. Though, we do ot observe idividual data, but oly averages for each rm, i.e. we ca oly estimate ave cotr i = 1 + ave ear i + 3 ave age i + 4 mrate i + eu i where ave cotr i = m 1 P m e=1 cotrib i;e; ave age i ; ave ear i are de ed aalogously. Thus, ote that eu i = m 1 P mi i e=1 u i;e: If E(u i;e ) = u; the E(eu i ) =E m 1 P mi i e=1 u i;e = u=m i : Thus, we ca implemet weighted least squares by moltiplyig all data by m 1= i ; i.e. i this case h(x ;i ; :::; x k;i ) = 1=m i : Thus, a straightforward case of applicatio of WLS is whe we have averages of idividuals for groups. However, i geeral we do ot kow the fuctioal form of h: Ofte, we postulate the fuctioal form of h; up to some parameters to be estimated. A frequetly used model is the followig. E(u i jx) = u exp ( 1 + x ;i + ::: + k x k;i ) ; (1) that is h(x :i ; :::; x k;i ) = exp ( 1 + x ;i + ::: + k x k;i ) : The idea is that this model is exible eough to capture di eret forms of coditioal heteroskedasticity. Basically, we ca estimate the ; ad the scale the data by 1= b h 1= i = 1= exp b1 + b x ;i + ::: + b 1= k x k;i : 7

8 How to proceed? Step 1: Estimate the model for the coditioal mea, e.g. regress y i o itercept, x ;i ; :::; x k;i : Call b 1 ; b ; :::; b k the estimated coe ciet o itercept, x ;i ; :::; x k;i : Step : Take the residual bu i ; form log(bu i ) ad regress log(bu i ) o costat x ;i ; :::; x k;i : Deote b 1 ; b ; :::; b k the estimated coe ciet, ad de e bg i the resultig predicted value, i.e. bg i = d log(bu i ) = b 1 + b x ;i + ::: + b k x k;i Step 3: Form b h i = exp(bg i ); ad costruct y + i = y i = b h 1= i ; x + j;i = x j;i= b h 1= i ; j = ; :::; k; as Step 4: Regress y + i o 1= b h 1= i ; x + ;i ; :::; x+ k;i ad form b fwls = X + X +0 1 X + y + where b fwls is called feasible weighted least square estimator. Result WLS-: Let A.MLR1-A.MLR4 hold. Suppose that: E(u i jx) = u exp ( 1 + x ;i + ::: + k x k;i ) : If b = b1 ; b ; :::; b k 0 is cosistet for ; the: 1= b fwls d! N 0; up lim (X X 0 =) 1 that is 1= bfwls has the same limitig distributio as 1= bwls ad so it is asymptotically e ciet. Remark: i ite sample b fwls is o loger BLUE. ; Sketch of Proof: Note that y + i = yi hi b : Give that p lim b = ; p lim hi hi b = 1: hi Thus, i large sample, regressig y + i o 1= b h 1= i ; x + ;i ; :::; x+ k;i ad regressig y i o 1=h 1= i ; x ;i ; :::; x k;i is equivalet. From the result above, we see that if we have a cosistet estimator for the true coditioal variace, the Weighted Least Squares ad Feasible Weighted Least Squares have the same limitig distributio. The issue is that ofte we do ot kow the fuctioal form of h; ad have o particular ideas o that. I that case, we use a so called oparametric estimators of the coditioal variace. These estimators have the advatage of beig able to approximate ay fuctio (subject to some regularity coditios, such as twice cotiuous di eretiability). I this sese they are very exible. The price we pay is that they coverge very very slow. Need a lot of data to have them workig. A well kow type of estimators is the so called kerel estimators. 8

9 If we do ot kow the fuctioal form of h; the idea is to use a kerel estimator of the coditioal variace, i order to get a estimator able to approximate every h fuctio. Suppose that h i = h(x ;i ; x 3;i ): De e, = b hnp (x ;i ; x 3;i ) P 1 j=1 bu j K xj x ;i K P j=1 K xj x ;i K 1 where for example xj x ;i K = 0:75 1 if x j x ;i < 1: x3j x 3;i x3j x 3;i ;! xj x ;i ad! 0 as! 1 but rather slow (e.g. = 1=6 ): Note that K is called the kerel ad the badwidht. Now, provide the true h fuctio is twice cotiuously di eretiable, for ay poit x ;i ; x 3;i p lim bhnp (x ;i ; x 3;i ) h(x ;i ; x 3;i ) = 0 Thus, eve if we do ot kow h; ca always use a oparametric estimator ad costruct b fwls by regressig y i = b h 1= NP (x ;i; x 3;i ) o 1= b h 1= NP (x ;i; x 3;i ); x ;i = b h 1= NP (x ;i; x 3;i ) ad x 3;i = b h 1= NP (x ;i; x 3;i ): The issue is that b h NP (x ;i ; x 3;i ) coverges to the true h(x ;i ; x 3;i ); very slowly. Ad, the slower the higher is the umber of variables o which h i deped. Thus, for small sample b fwls ca behave quite bad. Heuristically, it is a good idea doig weighted least squares with oparametric estimators, if we have sample of about observatios or more. What happes if we compute (feasible) WLS usig the wrog weightig matrix? I other words, suppose that E(u i jx) = ug(x): However, we do WLS usig h(x) (or feasible WLS usig a estimator cosistet for h(x))? Well, cosistecy is preserved, though the estimator is o loger e ciet. I this case, still eed to use White stadard errors, usig X or X + ad the residual from the weighted regressio. Example: Demad for cigarettes. cigs i ; ic i ; educ i ; age i are umber of cigarettes per day, year icome, educatio ad age of idividual i; pcigs i price of cigarettes for the state where idividual i lives (US data!), ad rest i dummy equal to 1 if smokig is prohibited i restaurats i the states where idividual i lives. We have: dcigs i = 3:64(4) + :88(:7) log(ic i ) :75(5:7) log(pcigs i ) 0:5(:167)educ i + :77(:16)age i :009(:0017)age i :83(1:1)rest i 9

10 = 807: We costruct a Breusch-Paga test for heteroskedasticity. We regress bu i o costat, log(ic i); educ i ; age i ; age i ; ad log(pcigs i); rest i : Get R = :04; thus R = 8070:04 = 3; which is larger tha the 95% critical values of (6): Reject the ull of coditioal heteroskedasticity. We use the model i (1) ad follow Step1-Step4 above to costruct feasible weighted least square estimators, which are reported below. dcigs i = 5:64(18) + 1:3(:44) log(ic i ) ::94(4:5) log(pcigs i ) 0:46(:1)educ i + :48(:09)age i :006(:0009)age i 3:46(0:8)rest i The rst thig we ote is that (feasible)wls estimators are quite di eret from OLS. For example, coe ciet o icome, is more tha twice tha before, also coe ciet o age ad restaurat are quite di eret... What should we coclude? We kow that both OLS ad (feasible)wls are cosistet, thus i large sample (ad 800 is large) should be quite close each other. Oe explaatio is that the uderlyig model is misspeci ed, so that OLS ad (feasible)wls coverge to two di eret probability limit. We ll come back to that whe we ll do Hausma Test! 10