Chapter 11. Heteroskedasticity The Nature of Heteroskedasticity. In Chapter 3 we introduced the linear model (11.1.1)

Transcription

1 Chaper 11 Heeroskedasiciy 11.1 The Naure of Heeroskedasiciy In Chaper 3 we inroduced he linear model y = β+β x (11.1.1) 1 o explain household expendiure on food (y) as a funcion of household income (x). We begin his secion by asking wheher a funcion such as y = β 1 + β x is beer a explaining expendiure on food for low-income households han i is for high-income households. Income is less imporan as an explanaory variable for food expendiure of highincome families. I is harder o guess heir food expendiure. Undergraduae Economerics, nd Ediion-Chaper 11 Slide 11.1

2 This ype of effec can be capured by a saisical model ha exhibis heeroskedasiciy. y = β+β x + e 1 (11.1.) We assumed he e were uncorrelaed random error erms wih mean zero and consan variance σ. Tha is, Ee ( ) = 0 var( ) e = σ cov( i, j) 0 e e = (11.1.3) Including he sandard errors for b 1 and b, he esimaed mean funcion was y ˆ = x (11.1.4) (.139)(0.0305) A graph of his esimaed funcion, along wih all he observed expendiure-income poins ( y, x ), appears in Figure Undergraduae Economerics, nd Ediion-Chaper 11 Slide 11.

3 Noice ha, as income (x ) grows, he observed daa poins ( y, x ) deviae more and more from he esimaed mean funcion. have a endency o The leas squares residuals, defined by e = y b b x ˆ 1 (11.1.5) increase in absolue value as income grows. [Figure 11.1 here] The observable leas squares residuals ( ˆ ) are proxies for he unobservable errors ( ) ha are given by e e e = y β β x 1 (11.1.6) The informaion in Figure 11.1 suggess ha he unobservable errors also increase in absolue value as income (x ) increases. Is his ype of behavior consisen wih he assumpions of our model? Undergraduae Economerics, nd Ediion-Chaper 11 Slide 11.3

4 The parameer ha conrols he spread of y around he mean funcion, and measures he uncerainy in he regression model, is he variance σ. If he scaer of y around he mean funcion increases as x increases, hen he uncerainy abou y increases as x increases, and we have evidence o sugges ha he variance is no consan. Thus, we are quesioning he consan variance assumpion var( y ) var( ) = e =σ (11.1.7) The mos general way o relax his assumpion is o add a subscrip o σ, recognizing ha he variance can be differen for differen observaions. We hen have var( y ) = var( e ) =σ (11.1.8) In his case, when he variances for all observaions are no he same, we say ha heeroskedasiciy exiss. Alernaively, we say he random variable y and he random error are heeroskedasic. e Undergraduae Economerics, nd Ediion-Chaper 11 Slide 11.4

5 Conversely, if (11.1.7) holds we say ha homoskedasiciy exiss, and y and e are homoskedasic. The heeroskedasic assumpion is illusraed in Figure 11.. [Figure 11. here] The exisence of differen variances, or heeroskedasiciy, is ofen encounered when using cross-secional daa. 11. The Consequences of Heeroskedasiciy for he Leas Squares Esimaor If we have a linear regression model wih heeroskedasiciy and we use he leas squares esimaor o esimae he unknown coefficiens, hen: 1. The leas squares esimaor is sill a linear and unbiased esimaor, bu i is no longer bes. I is no longer B.L.U.E. Undergraduae Economerics, nd Ediion-Chaper 11 Slide 11.5

6 . The sandard errors usually compued for he leas squares esimaor are incorrec. Confidence inervals and hypohesis ess ha use hese sandard errors may be misleading. Consider he model y = β+β x + e 1 (11..1) where Ee ( ) = 0 var( e) =σ cov( e, e) = 0 (i j) i j In Chaper 4, equaion 4..1, we wroe he leas squares esimaor for β as where b = β + we (11..) w = x x ( x x) Undergraduae Economerics, nd Ediion-Chaper 11 Slide 11.6

7 The firs propery ha we esablish is ha of unbiasedness. ( ) = ( β ) + ( ) = β + we ( e) =β E b E E we (11..4) The nex resul is ha he leas squares esimaor is no longer bes. The way we ackle his quesion is o derive an alernaive esimaor which is he bes linear unbiased esimaor. This new esimaor is considered in Secions 10.3 and To show ha he usual formulas for he leas squares sandard errors are incorrec under heeroskedasiciy, we reurn o he derivaion of var(b ) in (4..11). From ha equaion, and using (11..), we have Undergraduae Economerics, nd Ediion-Chaper 11 Slide 11.7

8 ( b) = ( β ) + ( we ) = var ( we ) = w var ( e) + ww i jcov ( ei, ej) var var var = w σ i j ( x x) σ = ( x x) (11..5) Noe from he las line in (11..5) ha var( b ) σ ( x x ) (11..6) Undergraduae Economerics, nd Ediion-Chaper 11 Slide 11.8

9 Noe ha sandard compuer sofware for leas squares regression will compue he esimaed variance for b based on (11..6), unless old oherwise Whie's Approximae Esimaor for he Variance of he Leas Squares Esimaor Halber Whie, an economerician, has suggesed an esimaor for he variances and covariances of he leas squares coefficien esimaors when heeroskedasiciy exiss. In he conex of he simple regression model, his esimaor for var(b ) is obained by replacing σ by he squares of he leas squares residuals e, in (11..5). Large variances are likely o lead o large values of he squared residuals. Because he squared residuals are used o approximae he variances, Whie's esimaor is sricly appropriae only in large samples. ˆ If we apply Whie's esimaor o he food expendiure-income daa, we obain Undergraduae Economerics, nd Ediion-Chaper 11 Slide 11.9

10 We could wrie our esimaed equaion as var( ˆ b ) = var( ˆ b ) = y ˆ = x (3.704) (0.038) (Whie) (.139) (0.0305) (incorrec) In his case, ignoring heeroskedasiciy and using incorrec sandard errors ends o oversae he precision of esimaion; we end o ge confidence inervals ha are narrower han hey should be. We can consruc wo corresponding 95% confidence inervals for β. Whie: b ± cse( b ) Incorrec: b ± cse( b ) = ±.04(0.038) = [0.051, 0.06] = ±.04(0.0305) = [0.067, 0.190] Undergraduae Economerics, nd Ediion-Chaper 11 Slide 11.10

11 11.3 Proporional Heeroskedasiciy Reurn o he example where weekly food expendiure (y ) is relaed o weekly income (x ) hrough he equaion y = β+β x + e (11.3.1) 1 We make he following assumpions: ( ) = 0 var( e ) =σ E e cov( e, e ) = 0 (i j) i j By iself, he assumpion var(e ) = σ is no adequae for developing a beer procedure for esimaing β 1 and β. Undergraduae Economerics, nd Ediion-Chaper 11 Slide 11.11

12 We overcome his problem by making a furher assumpion abou he σ. Our earlier inspecion of he leas squares residuals suggesed ha he error variance increases as income increases. A reasonable model for such a variance relaionship is ( ) var x e = σ =σ (11.3.) The assumpion of heeroskedasic errors in (11.3.) is a reasonable one for he expendiure model. Under heeroskedasiciy he leas squares esimaor is no he bes linear unbiased esimaor. One way of overcoming his dilemma is o change or ransform our saisical model ino one wih homoskedasic errors. Leaving he basic srucure of he model inac, i is possible o urn he heeroskedasic error model ino a homoskedasic error model. Once his ransformaion has been carried ou, applicaion of leas squares o he ransformed model gives a bes linear unbiased esimaor. Begin by dividing boh sides of he original equaion in (11.3.1) by x Undergraduae Economerics, nd Ediion-Chaper 11 Slide 11.1

13 y 1 x e =β 1 +β + x x x x (11.3.3) Define he ransformed variables y = y * x x * 1 = 1 x x * = x x e e = (11.3.4) x * (11.3.3) can be rewrien as y = β x +β x + e (11.3.5) 1 1 The beauy of his ransformed model is ha he new ransformed error erm e is homoskedasic. The proof of his resul is: e 1 1 var( e ) = var = var( e) = σ x x x x =σ (11.3.6) Undergraduae Economerics, nd Ediion-Chaper 11 Slide 11.13

14 The ransformed error erm will reain he properies Ee ( ) = 0 and zero correlaion beween differen observaions, cov( e, e ) = 0 for i j. i j As a consequence, we can apply leas squares o he ransformed variables,, y x 1 and x o obain he bes linear unbiased esimaor for β 1 and β. The ransformed model is linear in he unknown parameers β 1 and β. These are he original parameers ha we are ineresed in esimaing. The ransformed model saisfies he condiions of he Gauss-Markov Theorem, and he leas squares esimaors defined in erms of he ransformed variables are B.L.U.E. The esimaor obained in his way is called a generalized leas squares esimaor. One way of viewing he generalized leas squares esimaor is as a weighed leas squares esimaor. Recall ha he leas squares esimaor is hose values of β 1 and β ha minimize he sum of squared errors. In his case, we are minimizing he sum of squared ransformed errors ha are given by Undergraduae Economerics, nd Ediion-Chaper 11 Slide 11.14

15 e T T * e = = 1 = 1 x The errors are weighed by he reciprocal of x. When x is small, he daa conain more informaion abou he regression funcion and he observaions are weighed heavily. When x is large, he daa conain less informaion and he observaions are weighed lighly. In his way we ake advanage of he heeroskedasiciy o improve parameer esimaion. Undergraduae Economerics, nd Ediion-Chaper 11 Slide 11.15

16 Remark: In he ransformed model x 1 1. Tha is, he variable associaed wih he inercep parameer is no longer equal o 1. Since leas squares sofware usually auomaically insers a 1 for he inercep, when dealing wih ransformed variables you will need o learn how o urn his opion off. If you use a weighed or generalized leas squares opion on your sofware, he compuer will do boh he ransforming and he esimaing. In his case suppressing he consan will no be necessary. Applying he generalized (weighed) leas squares procedure o our household expendiure daa yields he following esimaes: y ˆ = x (11.3.7) (17.986)(0.070) Undergraduae Economerics, nd Ediion-Chaper 11 Slide 11.16

17 I is imporan o recognize ha he inerpreaions for β 1 and β are he same in he ransformed model in (11.3.5) as hey are in he unransformed model in (11.3.1). The sandard errors in (11.3.8), namely se( ˆβ 1 ) = and se( ˆβ ) = are boh lower han heir leas squares counerpars ha were calculaed from Whie's esimaor, namely se(b 1 ) = and se(b ) = Since generalized leas squares is a beer esimaion procedure han leas squares, we do expec he generalized leas squares sandard errors o be lower. Remark: Remember ha sandard errors are square roos of esimaed variances; in a single sample he relaive magniudes of variances may no always be refleced by heir corresponding variance esimaes. Thus, lower sandard errors do no always mean beer esimaion. Undergraduae Economerics, nd Ediion-Chaper 11 Slide 11.17

18 The smaller sandard errors have he advanage of producing narrower more informaive confidence inervals. For example, using he generalized leas squares resuls, a 95% confidence inerval for β is given by β 垐 ± se( β ) c = ±.04(0.070) = [0.086, 0.196] The leas squares confidence inerval compued using Whie's sandard errors was [0.051, 0.06]. Undergraduae Economerics, nd Ediion-Chaper 11 Slide 11.18

19 11.4 Deecing Heeroskedasiciy Residual Plos One way of invesigaing he exisence of heeroskedasiciy is o esimae your model using leas squares and o plo he leas squares residuals. If he errors are homoskedasic, here should be no paerns of any sor in he residuals. If he errors are heeroskedasic, hey may end o exhibi greaer variaion in some sysemaic way The Goldfeld-Quand Tes A formal es for heeroskedasiciy is he Goldfeld-Quand es. I involves he following seps: 1. Spli he sample ino wo approximaely equal subsamples. If heeroskedasicy exiss, some observaions will have large variances and ohers will have small variances. Undergraduae Economerics, nd Ediion-Chaper 11 Slide 11.19

20 Divide he sample such ha he observaions wih poenially high variances are in one subsample and hose wih poenially low variances are in he oher subsample.. Compue esimaed error variances ˆσ 1 and ˆσ for each of he subsamples. Le ˆσ 1 be he esimae from he subsample wih poenially large variances and le ˆσ be he esimae from he subsample wih poenially small variances. If a null hypohesis of equal variances is no rue, we expec 3. Compue where c 垐 1 σ垐 1 σ o be large. GQ =σ σ and rejec he null hypohesis of equal variances if GQ > Fc F is a criical value form he F-disribuion wih ( T1 K) and (T K) degrees of freedom. The values and T are he numbers of observaions in each of he T1 subsamples; if he sample is spli exacly in half, T1 = T = T. Applying his es procedure o he household food expendiure model, we se up he hypoheses H 0 : σ =σ 1 : σ x H =σ (11.4.1) Undergraduae Economerics, nd Ediion-Chaper 11 Slide 11.0

21 Afer ordering he daa according o decreasing values of x, and using a pariion of 0 observaions in each subse of daa, we find σ ˆ = 85.9 and σ ˆ = Hence, he value of he Goldfeld-Quand saisic is 85.9 GQ = = The 5 percen criical value for (18, 18) degrees of freedom is F c =.. Thus, because GQ = 3.35 > F c =., we rejec H 0 and conclude ha heeroskedasiciy does exis; he error variance does depend on he level of income. 1 Undergraduae Economerics, nd Ediion-Chaper 11 Slide 11.1

22 REMARK: The above es is a one-sided es because he alernaive hypohesis suggesed which sample pariion will have he larger variance. If we suspec ha wo sample pariions could have differen variances, bu we do no know which variance is poenially larger, 11.5 A Sample Wih a Heeroskedasic Pariion Economic Model Consider modeling he supply of whea in a paricular whea growing area in Ausralia. In he supply funcion he quaniy of whea supplied will ypically depend upon he producion echnology of he firm, on he price of whea or expecaions abou he price of whea, and on weaher condiions. Undergraduae Economerics, nd Ediion-Chaper 11 Slide 11.

23 We can depic his supply funcion as Quaniy = f (Price, Technology, Weaher) (11.5.1) The daa we have available from he Ausralian whea growing disric consis of 6 years of aggregae ime-series daa on quaniy supplied and price. Because here is no obvious index of producion echnology, some kind of proxy needs o be used for his variable. We use a simple linear ime-rend, a variable ha akes he value 1 in year 1, in year, and so on, up o 6 in year 6. An obvious weaher variable is also unavailable; hus, in our saisical model, weaher effecs will form par of he random error erm. Using hese consideraions, we specify he linear supply funcion q = β+β p +β + e = 1,,...,6 (11.5.) 1 3 Undergraduae Economerics, nd Ediion-Chaper 11 Slide 11.3

24 q is he quaniy of whea produced in year, p is he price of whea guaraneed for year, =1,,...,6 is a rend variable inroduced o capure changes in producion e echnology, and is a random error erm ha includes, among oher hings, he influence of weaher. To complee he economeric model in (11.5.) some saisical assumpions for he random error erm e are needed. In his case, however, we have addiional informaion ha makes an alernaive assumpion more realisic. Afer he 13h year, new whea varieies whose yields are less suscepible o variaions in weaher condiions were inroduced. These new varieies do no have an average yield ha is higher han ha of he old varieies, bu he variance of heir yields is lower because yield is less dependen on weaher condiions. Undergraduae Economerics, nd Ediion-Chaper 11 Slide 11.4

25 Since he weaher effec is a major componen of he random error erm e, we can model he reduced weaher effec of he las 13 years by assuming he error variance in hose years is differen from he error variance in he firs 13 years. Thus, we assume ha ( ) ( e ) ( e ) E e = 0 var =σ = 1, K,13 1 var =σ = 14, K,6 From he above argumen, we expec ha (11.5.3) σ <σ Generalized Leas Squares Through Model Transformaion Wrie he model corresponding o he wo subses of observaions as ( ) ( ) q =β +β p +β + e var e =σ = 1, K, q =β +β p +β + e var e =σ = 14, K,6 1 3 (11.5.4) Undergraduae Economerics, nd Ediion-Chaper 11 Slide 11.5

26 Dividing each variable by σ 1 for he firs 13 observaions and by σ for he las 13 observaions yields q 1 p e =β +β +β + = 1, K,13 σ σ σ σ σ q 1 p e =β +β +β + = 14, K,6 σ σ σ σ σ 1 3 (11.5.5) This ransformaion yields ransformed error erms ha have he same variance for all observaions. Specifically, he ransformed error variances are all equal o one because e 1 σ1 var = var ( e ) = = 1 = 1, K,13 σ1 σ1 σ1 e 1 σ var = var ( e ) = = 1 = 14, K,6 σ σ σ Undergraduae Economerics, nd Ediion-Chaper 11 Slide 11.6

27 Providing and σ are known, he ransformed model in (11.5.5) provides a se of σ1 new ransformed variables o which we can apply he leas squares principle o obain he bes linear unbiased esimaor for (β1, β, β 3 ). The ransformed variables are q 1 p σ σ σ σ i i i i (11.5.6) where is eiher or σi σ1 σ, depending on which half of he observaions are being considered. Like before, he complee process of ransforming variables, hen applying leas squares o he ransformed variables, is called generalized leas squares. Undergraduae Economerics, nd Ediion-Chaper 11 Slide 11.7

28 Implemening Generalized Leas Squares The ransformed variables in (11.5.6) depend on he unknown variance parameers and σ. Thus, as hey sand, he ransformed variables canno be calculaed. To overcome his difficuly, we use esimaes of as if he esimaes were he rue variances. σ 1 and σ 1 σ and ransform he variables I makes sense o spli he sample ino wo, applying leas squares o he firs half o esimae σ 1 and applying leas squares o he second half o esimae hese esimaes for he rue values causes no difficulies in large samples. For he whea supply example we obain σ. Subsiuing ˆσ 1 = ˆσ = (R11.7) Using hese esimaes o calculae observaions on he ransformed variables in (11.5.6), and hen applying leas squares o he complee sample defined in (11.5.5) yields he esimaed equaion: Undergraduae Economerics, nd Ediion-Chaper 11 Slide 11.8

29 qˆ = p+3.83 (R11.8) (1.7) (8.81) (0.81) Undergraduae Economerics, nd Ediion-Chaper 11 Slide 11.9

30 Remark: A word of warning abou calculaion of he sandard errors is necessary. As demonsraed below (11.5.5), he ransformed errors in (11.5.5) have a variance equal o one. However, when you ransform your variables using ˆσ 1 and ˆσ, and apply leas squares o he ransformed variables for he complee sample, your compuer program will auomaically esimae a variance for he ransformed errors. This esimae will no be exacly equal o one. The sandard errors in (R11.8) were calculaed by forcing he compuer o use one as he variance of he ransformed errors. Mos sofware packages will have opions ha le you do his, bu i is no crucial if your package does no; he variance esimae will usually be close o one anyway. Undergraduae Economerics, nd Ediion-Chaper 11 Slide 11.30

31 Tesing he Variance Assumpion To use a residual plo o check wheher he whea-supply error variance has decreased over ime, i is sensible o plo he leas-squares residuals agains ime. See Figure The dramaic drop in he variaion of he residuals afer year 13 suppors our belief ha he variance has decreased. For he Goldfeld-Quand es he sample is already spli ino wo naural subsamples. Thus, we se up he hypoheses H : σ =σ 0 1 H : σ <σ1 (11.5.9) 1 The compued value of he Goldfeld-Quand saisic is GQ ˆ = σ = = σˆ Undergraduae Economerics, nd Ediion-Chaper 11 Slide 11.31

32 T = T = and K = 3; hus, if is rue, is an observed value from an F disribuion wih (10, 10) degrees of freedom. The corresponding 5 percen criical value is F c =.98. H 0 Since GQ = > F c =.98, we rejec H 0 and conclude ha he observed difference beween ˆσ 1 and ˆσ could no reasonably be aribuable o chance. There is evidence o sugges he new varieies have reduced he variance in he supply of whea. Undergraduae Economerics, nd Ediion-Chaper 11 Slide 11.3