Chaper 6: Ordnary Leas Squares Esmaon Procedure he Properes Chaper 6 Oulne Cln s Assgnmen: Assess he Effec of Sudyng on Quz Scores Revew o Regresson Model o Ordnary Leas Squares () Esmaon Procedure o he Esmaes, b Cons and b, Are Random Varables Sraegy: General Properes and a Specfc Applcaon o Revew: Assessng Cln s Opnon Poll Resuls o Prevew: Assessng Professor Lord s Quz Resuls Sandard Ordnary Leas Squares () Premses New Equaon for he Ordnary Leas Squares () Coeffcen Esmae General Properes: Descrbng he Coeffcen Esmae s Probably Dsrbuon o Mean (Cener) of he Coeffcen Esmae s Probably Dsrbuon o Varance (Spread) of he Coeffcen Esmae s Probably Dsrbuon Esmaon Procedures and he Esmae s Probably Dsrbuon o Imporance of he Mean (Cener) o Imporance of he Varance (Spread) Relably of he Coeffcen Esmae o Varance (Spread) of he Error erm s Probably Dsrbuon o Sample Sze: Number of Observaons o Range of he Eplanaory Varable o Relably Summary Bes Lnear Unbased Esmaon Procedure (BLUE) Chaper 6 Prep Quesons. Run he Dsrbuon of Coeffcen Esmaes smulaon n he Economercs Lab by clckng he followng lnk: [Lnk o MI-Lab 6P. goes here.] Afer compleng he lab, fll n he followng blanks: Numercal Value Of Your Coeffcen Calculaons Smulaon s Calculaons
Repeon Esmae Mean Varance Mean Varance 3 NB: You mus clck he Ne Problem buon o ge o he smulaon s problem.. Revew he arhmec of means: a. Mean of a consan mes a varable: Mean[c] b. Mean of a consan plus a varable: Mean[c + ] c. Mean of he sum of wo varables: Mean[ + y] 3. Revew he arhmec of varances: a. Varance of a consan mes a varable: Var[c] b. Varance of he sum of a varable and a consan: Var[c + ] c. Varance of he sum of wo varables: Var[ + y] d. Varance of he sum of wo ndependen varables: Var[ + y] 4. Consder an esmae s probably dsrbuon: a. Why s he mean (cener) of he probably dsrbuon mporan? Eplan. b. Why s he varance (spread) of he probably dsrbuon mporan? Eplan. Cln s Assgnmen: Assess he Effec of Sudyng on Quz Scores Cln s assgnmen s o assess he heory ha addonal sudyng ncreases quz scores. o do so he mus use daa from Professor Lord s frs quz, he number of mnues suded and he quz score for each of he hree sudens n he course: Suden Mnues Suded () Quz Score (y) 5 66 5 87 3 5 90 able 6.: Frs Quz Resuls Projec: Use daa from Professor Lord s frs quz o assess he effec of sudyng on quz scores. Revew Regresson Model Cln uses he followng regresson model o complee hs assgnmen:
3 y β Cons + β + e where y quz score of suden mnues suded by suden e error erm for suden he Parameers: β Cons and β β Cons and β are he parameers of he model. Le us revew her nerpreaon: β Cons reflecs he number of pons Professor Lord gves sudens jus for showng up. β reflecs he number of addonal pons earned for each addonal mnue of sudyng. he Error erm he error erm, e, plays a crucal role n he model. he error erm represens random nfluences. he mean of he error erm s probably dsrbuon for each suden equals 0: Mean[e ] 0 Mean[e ] 0 Mean[e 3 ] 0 Consequenly, he error erms have no sysemac on affec quz scores. Somemes he error erm wll be posve and somemes wll be negave, bu afer many, many quzzes each suden s error erms wll average ou o 0. When he probably dsrbuon of he error erm s symmerc, he chances ha a suden wll score beer han usual on one quz equal he chances ha he suden wll do worse han usual. Ordnary Leas Squares () Esmaon Procedure As a consequence of he error erms (random nfluences), we can never deermne he acual values of β Cons and β ; consequenly, Cln has no choce bu o esmae he values. he ordnary leas squares () esmaon procedure s he mos commonly used procedure for dong hs: ( y y)( ) b b y b Cons ( ) Usng he resuls of he frs quz, Cln esmaes he values of he coeffcen and consan: Frs Quz Daa Ordnary Leas Squares () Esmaes: Esy 63 +. Suden y 5 66 5 87 3 5 90 b Cons Esmaed pons for showng up 63 b Esmaed pons for each mnue suded.
4 + + 5 + 5 + 5 45 66 87 90 43 5 y + y + y + + y 8 3 3 3 3 3 3 3 3 Suden y y y y 66 8 5 5 5 0 87 8 6 5 5 0 3 90 8 9 5 5 0 Suden ( y y)( ) ( ) ( 5)( 0) 50 (-0) 00 (6)(0) 0 (0) 0 3 (9)(0) 90 (0) 00 Sum 40 Sum 00 ( y y)( ) 40 ( ) 00 ( y y)( ) 40 6 6 b. b y b 8 5 63 00 5 5 Cons ( ) he Esmaes, b Cons and b, Are Random Varables In he prevous chaper, we used he Economercs Lab o show ha he esmaes for he consan and coeffcen, b Cons and b, are random varables. As a consequence of he error erms (random nfluences), we could no deermne he numercal value of he esmaes for he consan and coeffcen, b Cons and b, before we conduc he epermen, even f we knew he acual values of he consan and coeffcen, β Cons and β. Furhermore, we can never epec he esmaes o equal he acual values. Consequenly, we mus assess he relably of he esmaes. We shall focus on he coeffcen esmae: Esmae Relably: How relable s he coeffcen esmae calculaed from he resuls of he frs quz? ha s, how confden can Cln be ha he coeffcen esmae,., wll be close o he acual value of he coeffcen? Sraegy: General Properes and a Specfc Applcaon Revew: Assessng Cln s Opnon Poll Resuls Cln faced a smlar problem when he polled a sample of he suden populaon o esmae he fracon of sudens supporng hm. of he 6 randomly seleced sudens polled, 75 percen, suppored Cln hereby suggesng ha he was leadng. Bu we hen observed ha was possble for hs resul o occur even f
5 he elecon was acually a ossup. In vew of hs, we asked how confden Cln should be n he resuls of hs sngle poll. o address hs ssue, we urned o he general properes of pollng procedures o assess he relably of he esmae Cln obaned from hs sngle poll: General Properes versus One Specfc Applcaon Cln s Esmaon Procedure: Calculae he fracon of he 6 randomly seleced sudens supporng Cln Apply he pollng procedure once o Cln s sample of he 6 randomly seleced sudens: v + v + + v6 EsFrac 6 Before Poll v f for Cln Afer Poll 0 f no for Cln Random Varable: Esmae: Numercal Value Probably Dsrbuon 3 EsFrac.75 6 4 How relable s EsFrac? Mean[ EsFrac] p AcFrac Acual fracon of he populaon supporng Cln p( p) p( p) Var[ EsFrac] where SampleSze 6 Mean and varance descrbe he cener and spread of he esmae s probably dsrbuon Whle we could no deermne he numercal value of he esmaed fracon, EsFrac, before he poll was conduced, we could descrbe s probably dsrbuon. Usng algebra, we derved he general equaons for he mean and varance of he esmaed fracon s, EsFrac s, probably dsrbuon. hen, we checked our algebra wh a smulaon by eplong he relave frequency nerpreaon of probably: afer many, many repeons, he dsrbuon of he numercal values mrrors he probably dsrbuon for one repeon. Wha can we deduce before he poll s conduced? General properes of he pollng procedure descrbed by EsFrac s probably dsrbuon.
6 Probably dsrbuon s descrbed by s mean (cener) and varance (spread). Use algebra o derve he equaons for he probably dsrbuon s mean and varance. Mean[ EsFrac] p p( p) Var[ EsFrac] Check he algebra wh a smulaon by eplong he relave frequency nerpreaon of probably. he esmaed fracon s probably dsrbuon allowed us o assess he relably of Cln s poll. Prevew: Assessng Professor Lord s Quz Resuls Usng he ordnary leas squares () esmaon procedure we esmaed he value of he coeffcen o be.. hs esmae s based on a sngle quz. he fac ha he coeffcen esmae s posve suggess ha addonal sudyng ncreases quz scores. Bu how confden can we be ha he coeffcen esmae s close o he acual value? o address he relably ssue we shall focus on he general properes of he ordnary leas squares () esmaon procedure: General Properes versus One Specfc Applcaon Esmaon Procedure: Esmae β Cons and β by fndng he b Cons and b ha mnmze he sum of squared resduals Before epermen Random Varable: Probably Dsrbuon Mean[b ]? Var[b ]? Model: y β Cons + β + e b equaons: Cons ( y y)( ) ( ) b y b Apply he esmaon procedure once o he frs quz s daa: Afer epermen Esmae: Numercal Value 40 6 b. 00 5 6 b Cons 8 5 63 5
7 Mean and varance descrbe he cener and spread of he esmae s probably dsrbuon Whle we canno deermne he numercal value of he coeffcen esmae before he quz s gven, we can descrbe s probably dsrbuon. he probably dsrbuon ells us how lkely s for he coeffcen esmae based on a sngle quz o equal each of he possble values. Usng algebra, we shall derve he general equaons for he mean and varance of he coeffcen esmae s probably dsrbuon. hen, we wll check our algebra wh a smulaon by eplong he relave frequency nerpreaon of probably: afer many, many repeons, he dsrbuon of he numercal values mrrors he probably dsrbuon for one repeon. Wha can we deduce before he poll s conduced? General properes of he esmaon procedure descrbed by he coeffcen esmae s probably dsrbuon. Probably dsrbuon s descrbed by s mean (cener) and varance (spread). Use algebra o derve he equaons for he probably dsrbuon s mean and varance. Check he algebra wh a smulaon by eplong he relave frequency nerpreaon of probably. he coeffcen esmae s probably dsrbuon wll allow us o assess he relably of he coeffcen esmae calculaed from Professor Lord s quz. Sandard Ordnary Leas Squares () Regresson Premses o derve he equaons for he mean and varance of he coeffcen esmae s probably dsrbuon, we shall apply he sandard ordnary leas squares () regresson premses. As we menoned Chaper 5, hese premses make he analyss as sraghforward as possble. In laer chapers, we wll rela hese premses o sudy more general cases. In oher words, we shall sar wh he mos sraghforward case and hen move on o more comple ones laer. Error erm Equal Varance Premse: he varance of he error erm s probably dsrbuon for each observaon s he same; all he varances equal Var[e]: Var[e ] Var[e ] Var[e ] Var[e]
8 Error erm/error erm Independence Premse: he error erms are ndependen: Cov[e, e j ] 0. Knowng he value of he error erm from one observaon does no help us predc he value of he error erm for any oher observaon. Eplanaory Varable/Error erm Independence Premse: he eplanaory varables, he s, and he error erms, he e s, are no correlaed. Knowng he value of an observaon s eplanaory varable does no help us predc he value of ha observaon s error erm. o keep he algebra manageable, we shall assume ha he eplanaory varables are consans n he dervaons ha follow. hs assumpon allows us o apply he arhmec of means and varances easly. Whle hs smplfes our algebrac manpulaons, does no affec he valdy of our conclusons. New Equaon for he Ordnary Leas Squares () Coeffcen Esmae In Chaper 5, we derved an equaon ha epressed he coeffcen esmae n erms of he s and y s: b ( y y)( ) ( ) I s advanageous o use a dfferen equaon o derve he equaons for he mean and varance of he coeffcen esmae s probably dsrbuon, however; we shall use an equvalen equaon ha epresses he coeffcen esmae n erms of he s, e s, and β raher han n erms of he s and y s: b β + ( ) e ( ) o keep he noaon as sraghforward as possble, we shall focus on he 3 observaon case. he logc for he general case s dencal o he logc for he 3 observaon case: b ( e ) + ( e ) + ( e ) β + ( ) ( ) ( ) 3 3 + + 3 General Properes: Descrbng he Coeffcen Esmae s Probably Dsrbuon Mean (Cener) of he Coeffcen Esmae s Probably Dsrbuon
9 o calculae he mean of b s probably dsrbuon, revew he arhmec of means: Mean of a consan mes a varable: Mean[c] c Mean[]; Mean of a consan plus a varable: Mean[c + ] c + Mean[]; Mean of he sum of wo varables: Mean[ + y] Mean[] + Mean[y]; and recall ha he error erm represens random nfluences: he mean of each error erm s probably dsrbuon s 0: Mean[e ] Mean[e ] Mean[e 3 ] 0. Now, we apply algebra o he new equaon for he coeffcen esmae, b : ( ) e+ ( ) e + ( 3 ) e3 Mean[ b ] Mean[ β + ] ( ) + ( ) + ( 3 ) Applyng Mean[c + ] c + Mean[] ( ) e+ ( ) e + ( 3 ) e3 β + Mean[ ] ( ) + ( ) + ( 3 ) Rewrng he fracon as a produc β + Mean ( ) e + ( ) e + ( ) e ( ) + ( ) + ( ) [( )( 3 3) ] 3 Applyng Mean[c] cmean[] β + Mean ( ) + ( ) + ( ) ( ) + ( ) + ( ) 3 [( e e 3 e3) ] Applyng Mean[ + y] Mean[] + Mean[y] [ Mean[( ) e ] Mean[( ) e] Mean[( 3 ) e3] ] ( ) + ( ) + ( 3 ) β + + + Applyng Mean[c] cmean[] [ ( )Mean[ e ] ( )Mean[ e] ( 3 )Mean[ e3] ] ( ) + ( ) + ( 3 ) β + + + β So, we have shown ha Mean[b ] β Snce Mean[e ] Mean[e ] Mean[e 3 ] 0
0 Probably Dsrbuon b Mean[b ] β Fgure 6.: Probably Dsrbuon of Coeffcen Esmaes Consequenly, he ordnary leas squares () esmaon procedure for he value of he coeffcen s unbased. In any one repeon of he epermen, he mean (cener) of he probably dsrbuon equals he acual value of he coeffcen. he esmaon procedure does no sysemacally overesmae or underesmae he acual coeffcen value, β. If he probably dsrbuon s symmerc, he chances ha he esmae calculaed from one quz wll be oo hgh equal he chances ha wll be oo low. Economercs Lab 6.: Checkng he Equaon for he Mean We can use he Dsrbuon of Coeffcen Esmaes smulaon n our Economercs Lab o replcae he quz many, many mes. Bu n realy, Cln only has nformaon from one quz, he frs quz. How hen can a smulaon be useful? he relave frequency nerpreaon of probably provdes he answer. he relave frequency nerpreaon of probably ells us ha he dsrbuon of he numercal values afer many, many repeons of he epermens mrrors he probably dsrbuon of one repeon. Consequenly, repeang he epermen many, many mes reveals he probably dsrbuon for he one quz: Dsrbuon of he Numercal Values Afer many, many repeons Probably Dsrbuon We can use he smulaon o check he algebra we used o derve he equaon for he mean of he coeffcen esmae s probably dsrbuon:
Mean[b ] β If our algebra s correc, he mean (average) of he esmaed coeffcen values should equal he acual value of he coeffcen, β, afer many, many repeons. Fgure 6.: Dsrbuon of Coeffcen Esmaes Smulaon [Lnk o MI-Lab 6. goes here.] Recall ha a smulaon allows us o do somehng ha we canno do n he real world. In he smulaon, we can specfy he acual values of he consan and coeffcen, β Cons and β. he defaul seng for he acual coeffcen value s. Be ceran ha he Pause checkbo s checked. Clck Sar. Record he numercal value of he coeffcen esmae for he frs repeon. Clck Connue o smulae he second quz. Record he value of he coeffcen esmae for he second repeon and calculae he mean and varance of he numercal esmaes for he frs wo repeons. Noe ha your calculaons agree wh hose provded by he smulaon. Clck Connue agan o smulae he hrd quz. Calculae he mean and varance of he numercal esmaes for he frs hree repeons. Once agan, noe ha your calculaons and he smulaon s calculaons agree. Connue o clck Connue unl you are convnced ha he smulaon s calculang he mean and varance of he numercal values for he coeffcen esmaes correcly. Now, clear he Pause checkbo and clck Connue. he smulaon no longer pauses afer each repeon. Afer many, many repeons, clck Sop.
Probably Dsrbuon Fgure 6.3: Hsogram of Coeffcen Value Esmaes b Queson: Wha does he mean (average) of he coeffcen esmaes equal? Answer: I equals abou.0. hs lends suppor o he equaon for he mean of he coeffcen esmae s probably dsrbuon ha we jus derved. Now, change he acual coeffcen value from o 4. Clck Sar and hen afer many, many repeons, clck Sop. Wha does he mean (average) of he esmaes equal? Ne, change he acual coeffcen value o 6 and repea he process. Equaon: Smulaon: Mean of Mean (Average) of Coef Esmae Esmaed Coef Acual Prob Ds Smulaon Values, b, from β Mean[b ] Repeons he Epermens >,000,000.0 4 4 >,000,000 4.0 6 6 >,000,000 6.0 able 6.: Dsrbuon of Coeffcen Esmae Smulaon Resuls Concluson: In all cases, he mean (average) of he esmaes for he coeffcen value equals he acual value of he coeffcen afer many, many repeons. he smulaons confrm our algebra. he esmaon procedure does no sysemacally underesmae or overesmae he acual value of he coeffcen. he ordnary leas squares () esmaon procedure for he coeffcen value s unbased.
3 Varance (Spread) of he Coeffcen Esmae s Probably Dsrbuon Now, we urn our aenon o he varance of he coeffcen esmae s probably dsrbuon. o derve he equaon for he varance, begn by revewng he arhmec of varances: Varance of a consan mes a varable: Var[c] c Var[]. Varance of he sum of a varable and a consan: Var[c + ] Var[]. Varance of he sum of wo ndependen varables: Var[ + y] Var[] + Var[y]. Focus on he frs wo sandard ordnary leas squares () premses: Error erm Equal Varance Premse: Var[e ] Var[e ] Var[e 3 ] Var[e]. Error erm/error erm Independence Premse: he error erms are ndependen; ha s, Cov[e, e j ] 0. Recall he new equaon for b : ( e ) + ( e ) + ( 3 e ) 3 b β + ( ) + ( ) + ( 3 ) herefore, ( ) e+ ( ) e + ( 3 ) e3 Var[ b ] Var[ β + ] ( ) + ( ) + ( 3 ) Applyng Var[c + ] Var[] ( ) ( ) ( 3 ) 3 Var [ e + e + e ] ( ) + ( ) + ( 3 ) Rewrng he fracon as a produc Var [( )(( ) e+ ( ) e + ( 3 ) e 3) ] ( ) + ( ) + ( 3 ) Applyng Var[c] c Var[] Var [(( ) e+ ( ) e + ( 3 ) e 3) ] [( ) + ( ) + ( 3 ) ] Error erm/error erm Independence Premse: Var[ + y] Var[] + Var[y] Var[( ) ] Var[( ) ] Var[( 3 ) 3] ( ) ( ) ( ) e + e + + + e [ ] [ ] 3 Applyng Var[c] c Var[] ( ) Var[ ] ( ) Var[ ] ( 3 ) Var[ 3] [( ) ( ) ( 3 ) ] [ ] e e + + e + + Error erm Equal Varance Premse: Var[e ] Var[e ] Var[e 3 ] Var[e]
4 ( ) Var[ ] ( ) Var[ ] ( 3 ) Var[ ] [( ) ( ) ( 3 ) ] [ ] e e + + e + + Facorng ou he Var[e] 3 [( ) ( ) ( 3 ) ] [ ] + + ( ) + ( ) + ( ) Var[ e ] Smplfyng Var[ e] ( ) + ( ) + ( 3 ) Var[ e] We can generalze hs: Var[ b ] ( ) he varance of he coeffcen esmae s probably dsrbuon equals he varance of he error erm s probably dsrbuon dvded by he sum of squared devaons. Economercs Lab 6.: Checkng he Equaon for he Varance We shall now use he Dsrbuon of Coeffcen Esmaes smulaon o check he equaon ha we jus derved for he varance of he coeffcen esmae s probably dsrbuon. Fgure 6.4: Dsrbuon of Coeffcen Esmaes Smulaon [Lnk o MI-Lab 6. goes here.] he smulaon auomacally spreads he values unformly beween 0 and 30. We shall connue o consder hree observaons; accordngly, he values are 5,
5 5, and 5. o convnce yourself of hs, be ceran ha he Pause checkbo s checked. Clck Sar and hen Connue a few mes o observe ha he values of are always 5, 5, and 5. Ne, recall he equaon we jus derved for he varance of he coeffcen esmae s probably dsrbuon: Var[ e] Var[ e] Var[ b ] ( ) + ( ) + ( 3 ) ( ) By defaul, he varance of he error erm probably dsrbuon s 500; herefore, he numeraor equals 500. Le us urn our aenon o he denomnaor, he sum of squared devaons. We have jus observed ha he values are 5, 5, and 5. her mean s 5 and her sum of squared devaons from he mean s 00: + + 3 5+ 5+ 5 45 3 3 3 5 Suden ( ) 5 5 0 ( 0) 00 5 5 0 (0) 0 3 5 5 0 (0) 00 Sum 00 ha s, ( ) + ( ) + ( ) 00 3 ( ) 00 When he varance of he error erm s probably dsrbuon equals 500 and he sum of squared devaons equals 00, he varance of he coeffcen esmae s probably dsrbuon equals.50: Var[ e] Var[ e] 500 Var[ b ].50 ( ) + ( ) + ( 3 ) 00 ( ) o show ha he smulaon confrms hs, be ceran ha he Pause checkbo s cleared and clck Connue. Afer many, many repeons clck Sop. Indeed, afer many, many repeons of he epermen he varance of he numercal values s abou.50. he smulaon confrms he equaon we derved for he varance of he coeffcen esmae s probably dsrbuon. Esmaon Procedures and he Esmae s Probably Dsrbuon: Imporance of he Mean (Cener) and Varance (Spread) Le us revew wha we learned abou esmaon procedures when we suded Cln s opnon poll n Chaper 3:
6 Imporance of he Probably Dsrbuon s Mean: Formally, an esmaon procedure s unbased whenever he mean (cener) of he esmae s probably dsrbuon equals he acual value. he relave frequency nerpreaon of probably provdes nuon: If he epermen were repeaed many, many mes he average of he numercal values of he esmaes wll equal he acual value. An unbased esmaon procedure does no sysemacally underesmae or overesmae he acual value. If he probably dsrbuon s symmerc, he chances ha he esmae calculaed from one repeon of he epermen wll be oo hgh equal he chances he esmae wll be oo low. Acual Value Esmae Fgure 6.5: Probably Dsrbuon of Esmaes Imporance of he Mean Imporance of he Probably Dsrbuon s Varance: When he esmaon procedure s unbased, he varance of he esmae s probably dsrbuon s varance (spread) reveals he esmae s relably; he varance ells us how lkely s ha he numercal value of he esmae calculaed from one repeon of he epermen wll be close o he acual value:
7 Fgure 6.6: Probably Dsrbuon of Esmaes Imporance of he Varance When he esmaon procedure s unbased, he varance of he esmae s probably dsrbuon deermnes relably. o As he varance decreases, he probably dsrbuon becomes more ghly cropped around he acual value makng more lkely for he esmae o be close o he acual value. o On he oher hand, as he varance ncreases, he probably dsrbuon becomes less ghly cropped around he acual value makng less lkely for he esmae o be close o he acual value. Relably of he Coeffcen Esmae We shall focus on he varance of he coeffcen esmae s probably dsrbuon o eplan wha nfluences s relably. We shall consder hree facors: Varance of he error erm s probably dsrbuon. Sample sze. Range of he s. Esmae Relably and he Varance of he Error erm s Probably Dsrbuon Wha s our nuon here? he error erm represens he random nfluences. I s he error erm ha nroduces uncerany no he m. As he varance of he error erm s probably dsrbuon ncreases, uncerany ncreases; consequenly, he avalable nformaon becomes less relable. As he varance of he error erm s probably dsrbuon ncreases, we would epec he coeffcen esmae o become less relable. On he oher hand, as he varance of he error erm s
8 probably dsrbuon decreases, he avalable nformaon becomes more relable, and we would epec he coeffcen esmae o become more relable. o jusfy hs nuon, recall he equaon for he varance of he coeffcen esmae s probably dsrbuon: Var[ e] Var[ e] Var[ b ] ( ) + ( ) + ( 3 ) ( ) he varance of he coeffcen esmae s probably dsrbuon s drecly proporonal o he varance of he error erm s probably dsrbuon. Economercs Lab 6.3: Varance of he Error erm s Probably Dsrbuon [Lnk o MI-Lab 6.3 goes here.] We shall use he Dsrbuon of Coeffcen Esmaes smulaon o confrm he role played by he varance of he error erm s probably dsrbuon. o do so, check he From-o checkbo. wo lss now appear: a From ls and a o ls. Inally,.0 s seleced n he From ls and 3.0 n he o ls. Consequenly, he smulaon wll repor he percen of repeons n whch he coeffcen esmae falls beween.0 and 3.0. Snce he defaul value for he acual coeffcen, β, equals.0, he smulaon repors on he percen of repeons n whch he coeffcen esmae falls whn.0 of he acual value. he smulaon repors he percen of repeons n whch he coeffcen esmae s close o he acual value where close o s consdered o be whn.0. By defaul, he varance of he error erm s probably dsrbuon equals 500 and he sample sze equals 3. Recall ha he sum of he squared devaons equals 00 and herefore he varance of he coeffcen esmae s probably dsrbuon equals.50: Var[ e] Var[ e] 500 Var[ b ].50 ( ) + ( ) + ( 3 ) 00 ( ) Be ceran ha he Pause checkbo s cleared. Clck Sar and hen afer many, many repeons, clck Sop. As able 6.3 repors, he coeffcen esmae les whn.0 of he acual coeffcen value n 47.3 percen of he repeons. Now, reduce he varance of he error erm s probably dsrbuon from 500 o 50. he varance of he coeffcen esmae s probably dsrbuon now equals.5: Var[ e] Var[ e] 50 Var[ b ].5 ( ) + ( ) + ( 3 ) 00 4 ( )
9 Clck Sar and hen afer many, many repeons, clck Sop. he hsogram of he coeffcen esmaes s now more closely cropped around he acual value,.0. he percen of repeons n whch he coeffcen esmae les whn.0 of he acual coeffcen value rses from 47.3 percen o 95.5 percen. Smulaons: Probably Esmaed Coeffcen Values, b Acual Dsrbuon Percen Values Sample Equaons: Mean Beween β Var[e] Sze Mn Ma Mean[b ] Var[b ] (Average) Varance.0 and 3.0 500 3 0 30.0.50.0.50 47.3% 50 3 0 30.0.5.0.5 95.5% able 6.3: Dsrbuon of Coeffcen Esmae Smulaon Relably Resuls Why s hs mporan? he varance measures he spread of he probably dsrbuon. hs s mporan when he esmaon procedure s unbased. As he varance decreases, he probably dsrbuon becomes more closely cropped around he acual coeffcen value and he chances ha he coeffcen esmae obaned from one quz wll le close o he acual value ncreases. he smulaon confrms hs; afer many, many repeons he percen of repeons n whch he coeffcen esmae les beween.0 and 3.0 ncreases from 47.3 percen o 95.5 percen. Consequenly, as he error erm s varance decreases, we can epec he esmae from one quz o be more relable. As he varance of he error erm s probably dsrbuon decreases, he esmae s more lkely o be close o he acual value. hs s conssen wh our nuon, s no? Esmae Relably and he Sample Sze Ne, we shall nvesgae he effec of he sample sze, he number of observaons, used o calculae he esmae. Increase he sample sze from 3 o 5. Wha does our nuon sugges? As we ncrease he number of observaons, we wll have more nformaon. Wh more nformaon he esmae should become more relable; ha s, wh more nformaon he varance of he coeffcen esmae s probably dsrbuon should decrease. Usng he equaon, le us now calculae he varance of he coeffcen esmae s probably dsrbuon when here are 5 observaons. Wh 5 observaons he values are spread unformly a 3, 9, 5,, and 7; he mean (average) of he s,, equals 5 and he sum of he squared devaons equals 360: + + 3+ 4 + 5 3 + 9 + 5 + + 7 75 5 5 3 5 Suden ( )
0 3 5 ( ) 44 9 5 6 ( 6) 36 3 5 5 0 (0) 0 4 5 6 (6) 36 5 7 5 () 44 Sum 360 ( ) 360 Applyng he equaon for he value of he coeffcen esmae s probably dsrbuon: Var[ b ] Var[ e] Var[ e] ( ) + ( ) + ( 3 ) + ( 4 ) + ( 5 ) ( ) 50 (3 5) + (9 5) + (5 5) + ( 5) + (7 5) 50 50 50.388.4 ( ) + (6) + (0) + (6) + () 44 + 36 + 0 + 36 + 44 360 he varance of he coeffcen esmae s probably dsrbuon falls from.5 o.4. he smaller varance suggess ha he coeffcen esmae wll be more relable. Economercs Lab 6.4: Sample Sze Are our nuon and calculaons suppored by he smulaon? In fac, he answer s yes. [Lnk o MI-Lab 6.4 goes here.] Noe ha he sample sze has ncreased from 3 o 5. Clck Sar and hen afer many, many repeons clck Sop: Smulaons: Probably Esmaed Coeffcen Values, b Acual Dsrbuon Percen Values Sample Equaons: Mean Beween (Average) 500 3 0 30.0.50.0.50 47.3% 50 3 0 30.0.5.0.5 95.5% 50 5 0 30.0.4.0.4 99.3%
able 6.4: Dsrbuon of Coeffcen Esmae Smulaon Relably Resuls Afer many, many repeons he percen of repeons n whch he coeffcen esmae les beween.0 and 3.0 ncreases from 95.5 percen o 99.3 percen. As he sample sze ncreases, we can epec he esmae from one quz o be more relable. As he sample sze ncreases, he esmae s more lkely o be close o he acual value. Esmae Relably and he Range of s Le us agan begn by appealng o our nuon. As he range of s becomes smaller, we are basng our esmaes on less varaon n he s, less dversy; accordngly, we are basng our esmaes on less nformaon. As he range becomes smaller, he esmae should become less relable and consequenly, he varance of he coeffcen esmae s probably dsrbuon should ncrease. o confrm hs, ncrease he mnmum value of from 0 o 0 and decrease he mamum value from 30 o 0. he fve values are now spread unformly beween 0 and 0 a, 3, 5, 7, and 9; he mean (average) of he s,, equals 5 and he sum of he squared devaons equals 40: + + 3+ 4 + 5 + 3+ 5 + 7 + 9 75 5 5 3 5 Suden ( ) 5 4 ( 4) 6 3 5 ( ) 4 3 5 5 0 (0) 0 4 7 5 () 4 5 9 5 4 (6) 6 Sum 40 ( ) 40 Applyng he equaon for he value of he coeffcen esmae s probably dsrbuon: Var[ e] Var[ e] Var[ b ] ( ) + ( ) + ( 3 ) + ( 4 ) + ( 5 ) ( ) 50 ( 5) + (3 5) + (5 5) + (7 5) + (9 5) 50 50 50 5.5 ( 4) + () + (0) + () + (4) 6+ 4+ 0+ 4+ 6 40 4
he varance of he coeffcen esmae s probably dsrbuon ncreases from abou.4 o.5. Economercs Lab 6.5: Range of s Our ne lab confrms our nuon. [Lnk o MI-Lab 6.5 goes here.] Afer changng he mnmum value of o 0 and he mamum value o 0, clck he Sar and hen afer many, many repeons clck Sop. Smulaons: Probably Esmaed Coeffcen Values, b Acual Dsrbuon Percen Values Sample Equaons: Mean Beween (Average) 500 3 0 30.0.50.0.50 47.3% 50 3 0 30.0.5.0.5 95.5% 50 5 0 30.0.4.0.4 99.3% 50 5 0 0.0.5.0.5 6.8% able 6.5: Dsrbuon of Coeffcen Esmae Smulaon Relably Resuls Afer many, many repeons he percen of repeons n whch he coeffcen esmae les beween.0 and 3.0 decreases from 99.3 percen o 6.8 percen. An esmae from one repeon wll be less relable. As he range of he s decreases, he esmae s less lkely o be close o he acual value. Relably Summary Our smulaon resuls llusrae relaonshps beween nformaon, he varance of he coeffcen esmae s probably dsrbuon, and he relably of an esmae: More and/or more relable Less and/or less relable nformaon. nformaon. Varance of coeffcen esmae s probably dsrbuon smaller. Esmae more relable; more lkely he esmae s close o he acual value. Bes Lnear Unbased Esmaon Procedure (BLUE) Varance of coeffcen esmae s probably dsrbuon larger. Esmae less relable; less lkely he esmae s close o he acual value.
In Chaper 5 we nroduced he mechancs of he ordnary leas squares () esmaon procedure and n hs chaper we analyzed he procedure s properes. Why have we devoed so much aenon o hs parcular esmaon procedure? he reason s sraghforward. When he sandard ordnary leas squares () premses are sasfed, no oher lnear esmaon procedure produces more relable esmaes. In oher words, he ordnary leas squares () esmaon procedure s he bes lnear unbased esmaon procedure (BLUE). Le us now eplan hs more carefully. If an esmaon procedure s he bes lnear unbased esmaon procedure (BLUE), mus ehb hree properes: he esmae mus be a lnear funcon of he dependen varable, he y s. he esmaon procedure mus be unbased; ha s, he mean of he esmae s probably dsrbuon mus equal he acual value. No oher lnear unbased esmaon procedure can be more relable; ha s, he varance of he esmae s probably dsrbuon when usng any oher lnear unbased esmaon procedure canno be less han he varance when he bes lnear unbased esmaon procedure s used. he Gauss-Markov heorem proves ha he ordnary leas squares () esmaon procedure s he bes lnear unbased esmaon procedure. We shall llusrae he heorem by descrbng wo oher lnear unbased esmaon procedures ha whle unbased, are no as relable as he ordnary leas squares () esmaon procedure. Please noe ha whle we would never use eher of hese esmaon procedures o do serous analyss, hey are useful pedagogcal ools. hey allow us o llusrae wha we mean by he bes lnear unbased esmaon procedure. wo New Esmaon Procedures We shall now consder he Any wo and he Mn-Ma esmaon procedures: Any wo Esmaon Procedure: Choose any wo pons on he scaer dagram; draw a sragh lne hrough he pons. he coeffcen esmae equals he slope of hs lne. 3
4 y Any wo 3 9 5 7 Fgure 6.7: Any wo Esmaon Procedure Mn-Ma Esmaon Procedure: Choose wo specfc pons on he scaer dagram; he pon wh he smalles value of and he pon wh he larges value of ; draw a sragh lne hrough he wo pons. he coeffcen esmae equals he slope of hs lne. y Mn-Ma 3 9 5 7 Fgure 6.8: Mn-Ma Esmaon Procedure Economercs Lab 6.6: Comparng he Ordnary Leas Squares (), Any wo, and Mn-Ma Esmaon Procedures We shall now use he BLUE smulaon n our Economercs Lab o jusfy our emphass on he ordnary leas squares () esmaon procedure. [Lnk o MI-Lab 6.6 goes here.]
5 By defaul, he sample sze equals 5 and he varance of he error erm s probably dsrbuon equals 500. he From-o values are specfed as.0 and 3.0: Sample Sze 5 Smulaons: Acual Esmaed Coeffcen Values, b Esmaon Values Mean Percen Beween Procedure β Var[e] (Average) Varance.0 and 3.0.0 500.0.4 60.4% Any wo.0 50.0 4.0 9.0% Mn-Ma.0 50.0.7 55.% able 6.6: BLUE Smulaon Resuls Inally, he ordnary leas squares () esmaon procedure s specfed. Be ceran ha he Pause checkbo s cleared. Clck Sar and hen afer many, many repeons clck Sop. For he esmaon procedure, he average of he esmaed coeffcen values equals abou.0 and he varance.4. 60.4 percen of he esmaes le wh.0 of he acual value. Ne, selec he Any wo esmaon procedure nsead of. Clck Sar and hen afer many, many repeons clck Sop. For he Any wo esmaon procedure, he average of he esmaed coeffcen values equals abou.0 and he varance 4.0; 9.0 percen of he esmaes le whn.0 of he acual value. Repea he process one las me afer selecng he Mn-Ma esmaon procedure; he average equals abou.0 and he varance.7; 55. percen of he esmaes le wh.0 of he acual value. Le us summarze: In all hree cases, he average of he coeffcen esmaes equal.0, he acual value; afer many, many repeons he mean (average) of he esmaes equals he acual value. Consequenly, all hree esmaon procedures for he coeffcen value appear o be unbased. he varance of he coeffcen esmae s probably dsrbuon s smalles when he ordnary leas squares () esmaon procedure s used. Consequenly, he ordnary leas squares () esmaon procedure produces he mos relable esmaes. Wha we have jus observed can be generalzed. When he sandard ordnary leas squares () regresson premses are me, he ordnary leas squares () esmaon procedure s he bes lnear unbased esmaon procedure because no oher lnear unbased esmaon procedure produces esmaes ha are more relable.
6 Append 6.: New Equaon for he Coeffcen Esmae Begn by recallng he epresson for b ha we derved prevously n Chaper 5: b ( y y)( ) ( ) b s epressed n erms of he s and y s. We wsh o epress b n erms of he s, e s, and β. Sraegy: Focus on he numeraor of he epresson for b and subsue for he y s o epress he numeraor n erms of he s, e s, and β. As we shall shorly show, once we do hs, our goal wll be acheved. We begn wh he numeraor, for y : Cons ( y y)( ), and subsue β Cons + β + e ( y y)( ) ( β + β + e y)( ) Rearrangng erms. ( β y + β + e )( ) Cons Addng and subracng β. ( β + β y+ β β + e )( ) Cons Smplfyng. [( β + β y) + β ( ) + e )]( ) Cons Splng he summaon no hree pars. ( βcons β y)( ) β ( ) ( ) e + + + Smplfyng he frs and second erms. ( βcons β y) ( ) β( ) ( ) e + + +
7 Now, focus on he frs erm, ( βcons + β y) ( ). Wha does equal? ( ) Replacng wh. Snce. Smplfyng. 0 Ne, reurn o he epresson for he numeraor, ( y y)( ) : ( ) ( y y)( ) ( βcons + β y) ( ) + β( ) + ( ) e ( ) 0 0 β( ) ( ) e + + herefore, ( y y)( ) β( ) + ( ) e Las, apply hs o he equaon we derved for b n Chaper 5:
8 b ( y y)( ) ( ) Subsung for he numeraor. β( ) + ( ) e ( ) ( ) ( ) e ( ) ( ) β + + β ( ) e ( ) Splng he sngle fracon no wo. Smplfyng he frs erm. We have now epressed b n erms of he s, e s, and β.
9 Append 6.: Gauss-Markov heorem Gauss-Markov heorem: When he sandard ordnary leas squares () premses are sasfed, he ordnary leas squared () esmaon procedure s he bes lnear unbased esmaon procedure. Proof: Le b Ordnary leas squares () esmae Frs, noe ha b s a lnear funcon of he y s: 3 b ( y y)( ) ( ) Le w equal he ordnary leas squares () lnear weghs ; more specfcally, ( ) b w ( y y) wherew ( ) Now, le us derve wo properes of w 0 w ( ) Frs, w 0: w ( ) ( ) ( ) ( ) w : Placng he summaon n he numeraor. Splng he summaons n he numeraor.
30 ( ) Snce here are erms. ( ) Snce. ( ) Smplfyng. ( ) Snce he numeraor equals 0. 0 Second, 0 w : ( ) ( ) ( ) ( ) w Smplfyng. ( ) ( ) Placng he summaon n he numeraor.
3 ( ) ( ) Snce he numeraor and denomnaor are equal. Ne, consder a new lnear esmaon procedure whose weghs are Only when each squares () esmaon procedure. Le calculaed usng hs new lnear esmaon procedure: ( + ) b w w y w + w. w equals 0 wll hs procedure o dencal o he ordnary leas Now, le us perform a lle algebra: ( + ) b w w y ( w w)( βcons β e) + + + b equal he coeffcen esmae Snce y β Cons + β + e Mulplyng hrough. ( w w ) βcons ( w w ) β ( w w ) e + + + + + Facorng ou β Cons from he frs erm and β from he second. Cons( w w) β( w w) ( w w) e β + + + + + Agan, smplfyng he frs wo erms. Consw Consw w w ( w w ) e β + β + β + β + + Snce w 0 and w.
3 0 βconsw β βw ( w w ) e + + + + + herefore, βcons + β + β + ( + ) b w w w w e Now, calculae he mean of he new esmae s probably dsrbuon, Mean[b ]: [ e] Mean[ b ] Mean βconsw + β + βw + ( w + w ) Snce Mean[c + ] c + Mean[] [ ] Cons w w w w β + β + β + Mean ( + ) e Focusng on he las erm, Mean[c] cmean[] β w + β + β w + ( w + w)mean[ e ] Cons Focusng on he las erm, snce he error erms represens random nfluences, Mean[e ] 0 Consw w β + β + β he new lnear esmaon procedure mus be unbased: Mean[ b] β herefore, w w 0 and 0 Ne, calculae he varance of b : [ e] Var[ b ] Var βconsw + β + βw + ( w + w ) [ e ] Var ( w w) + Snce Var[c + ] Var[].
33 [ e ] Var ( w w) + Snce he error erms are ndependen, covarances equal 0: Var[ + y] Var[] + Var[y]. Snce Var[c] c Var[]. ( w + w) Var[ e ] ( w w) Var[] e + Var[ e ] ( w + w ) Var[ e ] ( w ) + w w + ( w) he varance of each error erm s probably dsrbuon s dencal, Var[e]. Facorng ou Var[e]. Epandng he squared erms. Splng up he summaon. Var[ e ] ( ) + + ( ) Now, focus on he cross produc erms, w w w ( ) ( ) w w w w w w : ( ) ( w ) ( ) ( w w) ( ) Placng he summaon n he numeraor. Splng he summaons n he numeraor.
34 herefore, 0 w w ( ) 0 0 ( ) Facor ou from he second erm n he numeraor. Snce w 0 and w 0. Snce he numeraor equals 0. ( ) w w w w Snce w w 0 w w Var[ b ] Var[ e ] ( ) + + ( ) ( ) Var[ e ] ( ) + ( ). he varance of he esmae s probably dsrbuon s mnmzed whenever each w equals 0, whenever he esmaon procedure s he ordnary leas squares () esmaon procedure. Append 6. appearng a he end of hs chaper shows how we can derve he second equaon for he coeffcen esmae, b, from he frs. he proof appears a he end of hs chaper n Append 6.. 3 o reduce poenal confuson, he summaon nde n he denomnaor has been changed from o.