Wednesday, November 7 Handout: Heteroskedasticity

Transcription

1 Amhers College Deparmen of Economics Economics 360 Fall 202 Wednesday, November 7 Handou: Heeroskedasiciy Preview Review o Regression Model o Sandard Ordinary Leas Squares (OLS) Premises o Esimaion Procedures Embedded wihin he Ordinary Leas Squares (OLS) Esimaion Procedure Wha Is Heeroskedasiciy? Consequences of Heeroskedasiciy o Our Suspicions o Confirming Our Suspicions: A Simulaion Accouning for Heeroskedasiciy: An Example Jusifying he Generalized Leas Squares (GLS) Esimaion Procedure Robus Sandard Errors: An Alernaive Approach Regression Model y = β Cons + β x x + e y = Dependen variable x = Explanaory variable e = Error erm =, 2,, = Sample size he error erm is a random variable ha represens random influences: Mean[e ] = 0 Sandard Ordinary Leas Squares (OLS) Premises Error erm Equal Variance Premise: he variance of he error erm s probabiliy disribuion for each observaion is he same; all he variances equal Var[e]: Var[e ] = Var[e 2 ] = = Var[e ] = Var[e] Error erm/error erm Independence Premise: he error erms are independen: Cov[e i, e j ] = 0. Knowing he value of he error erm from one observaion does no help you predic he value of he error erm for any oher observaion. Explanaory Variable/Error erm Independence Premise: he explanaory variables, he x s, and he error erms, he e s, are no correlaed. Knowing he value of an observaion s explanaory variable does no help you predic he value of ha observaion s error erm. Ordinary Leas Squares (OLS) Esimaion Procedure Several esimaion procedures are embedded wih he ordinary leas squares esimaion procedure. For our purposes, he hree mos imporan are a procedure o esimae he Σ = (y y )(x x ) value of he coefficien, β x : b x = Σ = (x x ) 2 variance of he error erm s probabiliy disribuion, Var[e]: EsVar[e] = SSR Degrees of Freedom EsVar[e] variance of he coefficien esimae s probabiliy disribuion, Var[b x ]: EsVar[b x ] = Σ = (x x ) 2

2 2 Summary: When he sandard OLS regression premises are saisfied: Each of he esimaion procedures embedded wihin he ordinary leas squares (OLS) esimaion procedure is ; he ordinary leas squares (OLS) esimaion procedure for he coefficien value is he. Criical Poin: When he ordinary leas squares (OLS) esimaion procedure performs is calculaions i implicily assumes ha he hree sandard ordinary leas squares (OLS) premises are saisfied. Sraegy: We shall now consider each of he sandard OLS regression premises in urn o invesigae wha happens when a premise is violaed. Error erm Equal Variance Premise: he variance of he error erm s probabiliy disribuion for each observaion is he same; all he variances equal Var[e]: Var[e ] = Var[e 2 ] = = Var[e ] = Var[e] Recall Professor Lord s hree sudens who ake a quiz every uesday morning. Suden Suden 2 Suden 3 y = β Cons + β x x + e y 2 = β Cons + β x x 2 + e 2 y 3 = β Cons + β x x 3 + e 3 y = Suden s score y 2 = Suden 2 s score y 3 = Suden 3 s score x = Suden s sudying x 2 = Suden 2 s sudying x 3 = Suden 3 s sudying e = Suden s error erm e 2 = Suden 2 s error erm e 3 = Suden 3 s error erm Error erm Equal Variance Premise: he variance of he error erm s probabiliy disribuion for each suden is he same; ha is, he spread of each suden s error erm probabiliy disribuion is idenical. We can use our Economerics Lab Error erm Simulaion o illusrae his premise. Wha Is Heeroskedasiciy? Heer = 0: No Heeroskedasiciy he firs sandard premise assumes ha he variances of he error erms are idenical. he simulaion illusraes his when Heer is specified as 0. Repeiion Err Var Click Sar and hen Coninue for a few imes o see how he simulaion Sar works. Noe ha he disribuion of each suden s error erms is illusraed as a hisogram; also, he mean and variance of each suden s error erms are compued. Nex, uncheck he Pause checkbox and click Coninue. Afer many, many repeiions of he experimen click Sop Pause Heer

3 3 Suden Suden 2 Suden 3 Mean: Variance: Mean: Variance: Mean: Variance: Afer many, many repeiions: he mean of each suden s error erm is indicaing ha. he variance of he error erm s probabiliy disribuion for each suden is. Heer = Nex, change he value of he Heer from 0 o. Since a posiive value is specified, he disribuion spread increases as we move from Suden o Suden 2 o Suden 3. Now, click Sar and hen afer many, many repeiions of he experimen click Sop. Suden Suden 2 Suden 3 Mean: Variance: Mean: Variance: Mean: Variance: Afer many, many repeiions: he mean of each suden s error erm is indicaing ha. he variance of he error erm s probabiliy disribuion for each suden. Quesion: Is he error erm equal variance premise violaed? Consequences of Heeroskedasiciy: Are he Ordinary Leas Squares Esimaion Procedures Sill Unbiased? Quesion: How does he presence of heeroskedasiciy affec he ordinary leas squares (OLS) esimaion procedures for he Σ = (y y )(x x ) value of he coefficien, β x : b x = Σ = (x x ) 2 variance of he error erm s probabiliy disribuion, Var[e]: EsVar[e] = SSR Degrees of Freedom EsVar[e] variance of he coefficien esimae s probabiliy disribuion: EsVar[b x ] = Σ = (x x ) 2 More specifically, are he hree esimaion procedures embedded in he ordinary leas squares (OLS) esimaion procedure sill unbiased in he presence of heeroskedasiciy? Noe ha when you do his, he ile of he variance lis changes o Mid Err Var. his occurs because heeroskedasiciy is now presen and he variances differ from suden o suden. he lis specifies he middle suden s, Suden 2 s, variance.

4 4 Firs, we consider he ordinary leas squares (OLS) esimaion procedure for he value of he coefficien. Esimaion Procedure for he Value of he Coefficien Quesion: In he presence of heeroskedasiciy, is he OLS esimaion procedure for he value of he coefficien unbiased? ha is, does Mean[b x ] sill equal β x? Review: Arihmeic of Means Mean of a consan plus a variable: Mean[c + x] = c + Mean[x] Mean of a consan imes a variable: Mean[cx] = c Mean[x] Mean of he sum of wo variables: Mean[x + y] = Mean[x] + Mean[y] Review he Deviaion of he Mean of he Coefficien Esimae s Probabiliy Disribuion Mean[b x ] = Mean[β x + (x x )e + (x 2 x )e 2 + (x 3 x )e 3 (x x ) 2 ] = β x + Mean[ (x x )e + (x 2 x )e 2 + (x 3 x )e 3 (x x ) 2 ] Applying Mean[c + x] = c + Mean[x] Rewriing he fracion as a produc = β x + Mean[( (x x )((x ) 2 x )e + (x 2 x )e 2 + (x 3 x )e 3 )] Applying Mean[cx] = c Mean[x] = β x + (x x [Mean[(x ) 2 x )e + (x 2 x )e 2 + (x 3 x )e 3 ]] Applying Mean[x + y] = Mean[x] + Mean[y] = β x + (x x ) 2 [Mean[(x x )e ] + Mean[(x 2 x )e 2 ] + Mean[(x 3 x )e 3 ]] Applying Mean[cx] = c Mean[x] = β x + [ (x x (x ) 2 x )Mean[e ] + (x 2 x )Mean[e 2 ] + (x 3 x )Mean[e 3 ]] Error erm has no sysemaic effec on y Mean[e ] = Mean[e 2 ] = Mean[e 3 ] = 0 = β x + = β x (x x ) 2 [ (x x ) 0 + (x 2 x ) 0 + (x 3 x ) 0 ] Simplifying Quesion: Have we relied on he error erm equal variance premise o show ha he ordinary leas squares (OLS) esimaion procedure for he coefficien value is unbiased (Mean[b x ] = β x )? Quesion: In he presence of heeroskedasiciy, should we expec he ordinary leas squares (OLS) esimaion procedure for he coefficien value sill o be unbiased?

5 5 OLS Esimaion Procedure for he Variance of he Coefficien Esimae s Probabiliy Disribuion Quesion: In he presence of heeroskedasiciy, is he ordinary leas squares (OLS) esimaion procedure for he variance of he coefficien esimae s probabiliy disribuion unbiased? o address his quesion, review he sraegy behind he esimaion procedure: Sep : Esimae he variance of he error erm s probabiliy disribuion from he available informaion; ha is, informaion from he firs quiz SSR EsVar[e] = Degrees of Freedom Sep 2: Apply he relaionship beween he variances of coefficien esimae s and error erm s probabiliy disribuions Var[e] Var[b x ] = Σ = (x x ) 2 EsVar[e] EsVar[b x ] = Σ = (x x ) 2 Firs, consider he equaion we derived an equaion for he variance of he coefficien Var[e] esimae s probabiliy disribuion in erms, Var[b x ] = Σ = (x x. ) 2 Quesion: Wha does Var[e] equal? Answer: Var[e ] = Var[e 2 ] = = Var[e ] = Var[e] SSR Second, when we use he equaion EsVar[e] = Degrees of Freedom o esimae he variance of he error erm s probabiliy disribuion, we are esimaing a single Var[e]. Sraegy: he sraegy used by he ordinary leas squares (OLS) esimaion procedure is based on he premise ha here is a single Var[e]. Quesions: In he presence of heeroskedasiciy: Suspicions Is here a single Var[e]? Should we expec he ordinary leas squares (OLS) esimaion procedure for he variance of he coefficien esimae s probabiliy disribuion sill o be unbiased? So, where do we sand? We suspec ha when heeroskedasiciy is presen he ordinary leas squares OLS esimaion procedure for esimaing he coefficien value is unbiased. variance of he coefficien esimae s probabiliy disribuion may be flawed because is calculaions are based on a flawed premise. We shall use a simulaion o confirm our suspicions.

6 6 Unbiased esimaion procedure: Afer many, many repeiions of he experimen he average (mean) of he esimaes equals he acual value. Esimaed coefficien value from his repeiion: Σ = (y y )(x x ) b x = Σ = (x x ) 2 EsVar[e] = Ac Coef SSR Degrees of Freedom EsVar[e] EsVar[b x ] = Σ = (x x ) 2 Esimae of he variance for he coefficien esimae s probabiliy disribuion calculaed from his repeiion Repeiion Coef Value Es Mean Var Sum Sqr XDev SSR Coef Var Es Mean Is he esimaion procedure for he coefficien s value unbiased? Mid Err Var Mean (average) of he esimaed coefficien values from all repeiions. Acual Variance of Middle Error erm s Probabiliy Disribuion Variance of he esimaed coefficien values from all repeiions. Is he esimaion procedure for he variance of he coefficien esimae s probabiliy disribuion unbiased? Average of he variance esimaes from all repeiions. Afer many, many repeiions: Is OLS esimaion procedure for he coefficien s value unbiased? Is OLS esimaion procedure for he variance of he of he coefficien esimae s probabiliy disribuion unbiased? Acual Esimae of Variance of he Esimae of he variance coefficien coefficien esimaed coefficien for coefficien esimae s value value values probabiliy disribuion Mean (Average) Variance of he Average of Acual of he Esimaed Esimaed Coefficien Esimaed Variances, Heer Value Values, b x, from Values, b x, from EsVar[b x ], from Facor of β x All Repeiions All Repeiions All Repeiions

7 7 Simulaions: When heeroskedasiciy is presen Good news: he OLS procedure for esimaing he coefficien value is unbiased. Bad news: he OLS procedure for esimaing he variance of he coefficien esimae s probabiliy disribuion is flawed because i is based on a false premise. Consequenly, all calculaions based on he variance of he coefficien esimae s probabiliy disribuion will be flawed: he sandard errors, -saisics, and ail probabiliies appearing on he OLS regression prinou are flawed. Accouning for Heeroskedasiciy Sraegy Sep : Apply he Ordinary Leas Squares (OLS) Esimaion Procedure. o Esimae he model s parameers wih he ordinary leas squares (OLS) esimaion procedure. Sep 2: Consider he Possibiliy of Heeroskedasiciy. o Ask wheher here is reason o suspec ha heeroskedasiciy may be presen. o Use he ordinary leas squares (OLS) regression resuls o ge a sense of wheher heereoskedasiciy is a problem by examining he residuals. o If he presence of heereoskedasiciy is suspeced, formulae a model o explain i. o Use he Breusch-Pagan-Godfrey approach by esimaing an arificial regression o es for he presence of heeroskedasiciy. Sep 3: Apply he Generalized Leas Squares (GLS) Esimaion Procedure. o Apply he model of heeroskedasiciy and algebraically manipulae he original model o derive a new, weaked model in which he error erms do no suffer from heeroskedasiciy. o Use he ordinary leas squares (OLS) esimaion procedure o esimae he parameers of he weaked model. An Example: GDP and Inerne Use heory: Higher per capia GDP increases Inerne use. Model: LogUsersInerne = β Cons + β GDP GdpPC + e where LogUsersInerne = Log of Inerne users per,000 people in naion GdpPC = Per capia GDP (,000 s of real inernaional dollars) in naion heory: β GDP > 0.

8 8 Sep : Apply he Ordinary Leas Squares (OLS) Esimaion Procedure. 992 Inerne Daa: Cross secion daa of Inerne use and gross domesic produc for 29 counries in 992. Dependen Variable: LogUsersInerne Explanaory Variable: GdpPC Dependen Variable: LOGUSERSINERNE Mehod: Leas Squares Sample: 29 Included observaions: 29 Coefficien Sd. Error -Saisic Prob. GDPPC C Esimaed Equaion: LogUsersInerne = Inerpreaion: We esimae ha a $,000 increase in real per capia GDP resuls in a in Inerne users. Criical Resul: he GdpPC coefficien esimae equals. he sign of he coefficien esimae suggess ha higher per capia GDP Inerne use. his evidence he heory. Probabiliy Disribuion Suden -disribuion Mean = SE = DF = H 0 : β GDP = 0 Per capia GDP does no affec Inerne use H : β GDP > 0 Higher per capia GDP increases Inerne use b Un GDP Using he ails probabiliy: Prob[Resuls IF H 0 rue] =. Quesion: Could here be a poenial problem here? Quesion: Wha do we know abou EsVar[b GDP ] when heeroskedasiciy is presen? Answer: I is based on a premise. EsVar[b GDP ] could be. Quesion: Wha is he SE[b GDP ]? Answer: he ails probabiliy is based on SE[b GDP ]. Consequenly, he sandard error and he ails probabiliy could be. Our calculaion of Prob[Resuls IF H 0 rue] could be.

9 9 Sep 2: Consider he Possibiliy of Heeroskedasiciy. Is here reason o suspec ha heroskedasiciy may be presen? Use OLS resuls: We can hink of he residual as he esimaed errors: he error erms, he e s, he residual, he Res s, are unobservable are observable y = β Cons + β x x + e Res = y Esy e = y (β Cons + β x x ) Res = y (b Cons + b x x ) since Esy = b Cons + b x x Formulae a Model: Heeroskedasiciy Model: (e Mean[e ]) 2 = α Cons + α GDP GdpPC + v heory: α GDP > 0 Since Mean[e ] = 0. Breusch-Pagan-Godfrey Dependen Variable: ResSqr Explanaory Variable: GdpPC e 2 ResSqr = α Cons + α GDP GdpPC + v We can hink of he residuals as he esimaed errors. = α Cons + α GDP GdpPC + v Dependen Variable: RESSQR Mehod: Leas Squares Sample: 29 Included observaions: 29 Coefficien Sd. Error -Saisic Prob. GDPPC C Criical Resul: he GdpPC coefficien esimae equals. he sign of he coefficien esimae suggess ha higher per capia GDP he squared deviaion of he error erm from is mean. his evidence he view ha heeroskedasiciy is presen. H 0 : α GDP = 0 H : α GDP > 0 Per capia GDP does no affec he squared deviaion of he residual Higher per capia GDP increases he squared deviaion of he residual Prob[Resuls IF H 0 rue] = Heeroskedasiciy Model: Var[e ] = V GdpPC where V equals a consan.

10 0 Geing Sared in EViews Run he ordinary leas squares (OLS) regression. In he equaion window, click View, Residual Diagnosics, and Heeroskedasiciy ess he Breusch-Pagan-Godfrey es is he defaul Afer checking he explanaory variables (he regressors), click OK. Sep 3: Apply he Generalized Leas Squares (GLS) Esimaion Procedure. Apply he model of heeroskedasiciy and algebraically manipulae he original model o derive a new, weaked model in which he error erms do no suffer from heeroskedasiciy. We begin wih he original model: Original Model: LogUsersInerne = β Cons + β GDP GdpPC + e Divide boh sides of he model by he square roo of he populaion, GdpPC : weaked Model: LogUsersInerne GdpPC = β Cons GdpPC = β Cons GdpPC GdpPC + β GDP GdpPC + + β GDP GdpPC + e GdpPC e GdpPC Wha is he variance of he new error erm s probabiliy disribuion, Var[ e GdpPC ] = GdpPC Var[e ] e GdpPC? Arihmeic of variances: Var[cx] = c 2 Var[x] Heeroskedasiciy Model: Var[e ] = V GdpPC where V equals a consan = GdpPC V GdpPC = V In he weaked model is he equal error variance premise saisfied? weaked Dependen Variable: AdjLogUsersInerne = LogUsersInerne GdpPC weaked Explanaory Variables: AdjCons = GdpPC AdjGdpPC = GdpPC Use he ordinary leas squares esimaion procedure o esimae he weaked model: Dependen variable: AdjLogUsersInerne Explanaory variables: AdjGdpPC and AdjConsan (NB: here is no consan.) Dependen Variable: ADJLOGUSERSINERNE Mehod: Leas Squares Dae: 02/04/0 ime: 7:20 Sample: 29 Included observaions: 29 Coefficien Sd. Error -Saisic Prob. ADJGDPPC ADJCONS

11 Comparing he Ordinary Leas Squares (OLS) and Generalized Leas Squares (GLS) Esimaes β GDP Coefficien Sandard ails Esimae Error -Saisic Probabiliy Ordinary Leas Squares (OLS) Generalized Leas Squares (GLS) Jusifying he Generalized Leas Squares (GLS) Esimaion Procedure We shall now use a simulaion o illusrae ha he generalized leas squares (GLS) esimaion procedure for he value of he coefficien esimae is unbiased. variance of he coefficien esimae s probabiliy disribuion is unbiased. value of he coefficien esimae is he bes linear unbiased esimaion procedure (BLUE). Afer many, many repeiions: Is esimaion procedure for he coefficien s value unbiased? Is esimaion procedure for he variance of he of he coefficien esimae s probabiliy disribuion unbiased? Acual Esimae of Variance of Esimae of he variance coefficien coefficien esimaed coefficien for coefficien esimae s value value values probabiliy disribuion Average of Variance of Average of Acual he Esimaed he Esimaed Esimaed Variances, Heer Esim Value Values, b x, from Values, b x, from EsVar[b x ], from Facor Proc of β x All Repeiions All Repeiions All Repeiions 0 OLS OLS GLS 2.0 Summary: Is he esimaion procedure Sd Premises Heeroskedasiciy an unbiased esimaion procedure for he OLS OLS GLS o coefficien value? o variance of he coefficien esimae s probabiliy disribuion? for he coefficien value he bes linear unbiased esimaion procedure (BLUE)?

12 2 Robus Sandard Errors: An Alernaive Approach wo issues emerge wih he ordinary leas squares (OLS) esimaion procedure when heeroskedasiciy is presen: he sandard error calculaions made by he ordinary leas squares (OLS) esimaion procedure are flawed. While he ordinary leas squares (OLS) for he coefficien value is unbiased, i is no he bes linear unbiased esimaion procedure (BLUE). Robus sandard errors address he firs issue and are paricularly appropriae when he sample size is large. Whie robus sandard errors consiue one such approach. Geing Sared in EViews Run he ordinary leas squares (OLS) regression. In he equaion window, click Esimae and Opions In he Coefficien covariance marix box selec Whie from he drop down lis. Click OK. Dependen Variable: LogUsersInerne Explanaory Variable: GdpPC Whie robus sandard errors: Dependen Variable: LOGUSERSINERNE Mehod: Leas Squares Included observaions: 29 Whie heeroskedasiciy-consisen sandard errors & covariance Variable Coefficien Sd. Error -Saisic Prob. GDPPC C Sandard errors based on he equal error erm variance premise: Dependen Variable: LOGUSERSINERNE Mehod: Leas Squares Sample: 29 Included observaions: 29 Coefficien Sd. Error -Saisic Prob. GDPPC C