Deprtmet f Ecmc Uvert f Trt ECO48F M.A. Ecmetrc Lecture Nte Smple Regre Mdel
Smple Regre Mdel I the frt lecture we lked t fttg le t ctter f pt. I th chpter we eme regre methd f eplrg the prbbltc tructure f the dt dd rdm dturbce t the mdel. ε depedet ble (rdm ble) epltr r depedet ble (fed-tchtc) ε rdm errr term Wh de rdm errr term re?. Mdel mplfct f relt. Meuremet errr ble Fr ever vlue f Χ there et prbblt dtrbut f ε d hece prbblt dtrbut f the Υ ' INSERT GRAPH Imprtt umpt uderlg tw ble regre mdel Remrk: ) The relthp betwee Υ d Χ ler ) Χ tchtc ble whe vlue fed 3) The errr h zer epected vlue Ε( ε ) 4) The errr term h ctt ce fr ll bervt Ε ε ε 5) The rdm bleε re tttcll depedet ll d j 6) Errr term rmll dtrbuted Ε fr ε ε j 3 - mde fr cveece 4 - therefre me ce hmkedtc hwever f the ce t ctt the hve heterkedtct (re fte cr-ect) E.g. INSERT GRAPH
5 - me tht errr tht re rdm d d t hve crrelt betwee them INSERT GRAPH Crrlr t umpt d 3 tht Χ depedet f the redul Ε Χ ε Χ Ε ε C l tte umpt term f Υ 3 - The rdm ble Υ h epected vlue α Ε( Υ ) Ε( α Χ ε ) α Χ Ε( ε ) α Χ 4 - The rdm ble Υ h ctt ce 5 - The rdm ble Υ re depedet Regre: Ordr Let Squre Regre methd re pprch tht lk t the crrelt betwee tw (r mre) ble. Wth regre mdel; teret fcue lkg t the cllect f dtrbut whch ll hve the me ce d hve me le lg trght le. Te prblem t etmte the le (lpe d tercept). Regre Mdel ε ε ~ Ν - prmeter f mdel (eed t be etmted) the epltr bledepedet ble ( fed t rdm) depedet ble (epled b ) ε rdm errr term gve uepled prt f regre The bve cdt mpl tht rdm ble gve wth. Ε ( ) ( ) rml 3
d d Χ re ll ctt ther ce re ll zer d cce wth ε re zer jut leveε Gve bervt Χ d Υ hw d we etmte the prmeter f the mdel d gve... d... Ν Ν The rdrl let qure (OLS) prceed b fdg etmte f d b mmzg ( N.B. pck tht errr ε re mll If jut ue ( ) lrge errr mpl lmted frmt but ue rule tht rule ut lrge errr (e.g. qurg) ( )( ) The lut f the OLS prblem wll be d ( ) Nte the frmul fr mple tht the etmted regre le wll ru thrugh the mple me f d The etmt fr gve b ( ) Hve me reult the ptmlt f the etmte f Gu Mrkv Therem: OLS BLUE The rdr let qure etmtr Therem: ce) ler (ler B Q U E... d f d ) ubed etmtr re the bet (mmum 4
The etmtr f the bet (mmum) ce qudrtc (qudrtc ) ubed etmtr... Smplg dtrbut fr the etmtr d Therem: The mplg dtrbut f d Ν ~.e. hve bte rml dtrbut wth the gve me d cce mtr Wht th me the fllwg d cv. Sce ukw we replce t wth prctce. Therem: Smplg dtrbut f ~ Χ Ad depedetl f d 5
We c ue the ce f t cduct tet the tercept d lpe prmeter. We eed the etmte f the cce betwee the lpe d tercept term f we re tereted cductg tet me ler cmbt f the lpe r tercept. Oce we kw the mplg dtrbut f d we c d hpthe tetg. TEST ON t-tet ~ t TEST ON ~ t TESTS ON ARBITRARY LINEAR COMBINATION OF AND e.g. Recll - cv ~ t Predct Itervl Ctruct the α tervl whch ct the et bervt gve α α α Pr t p t b 6
p Vr Iterpretg OLS Ceffcet e Wht? Hw Iterpreted? I mplet ce lpe f regre le. repreet the cree whch wuld ccur f were creed b ut. the mrgl cree whch wuld ccur f were creed. Smetme reult re preeted term f eltcte. Recll eltct defed : η Or: η The let qure etmte f etmte f ll we eed t d frm etmte f uull t the me t etmte the eltct ~ η 7
Gde f Ft Attempt t meure ft betwee etmted regre le d the le Gd etmte f the regre le epl lrge prt f the ble f Υ Lrge redul pr ft; mll redul gd ft Drwbck: Th deped ut f meure f depedt ble Need ut free meure Defe the t f Υ but t me Vrt Υ Υ Υ Gl t dvde t f Υ t tw prt:. Accuted fr regre equt. The uepled cmpet If the lpe f the regre kw t be zer ft regre b etmtg l tercept.e. me f Υ The bet predct fr mple me f Υ Υ α Χ α Υ Υ cted fr Χ the gve b the Whe the lpe -zer c mprve ur predct b ccutg fr Υ beg depedt Χ Υ α Χ The ddtl frmt prt f the t Υ T hw th dd zer clever w Υ Υ Υ Υ Υ Υ Χ c help reduce the uepled ADD SLOPEGRAPH 8
T meure: Squre bth de d tke um ver Ν bervt ( Υ Υ) ( Υ Υ ) ( Υ Υ) ( Υ Υ )( Υ Υ) C hw ( Υ )( Υ Υ) Υ Left wth: Υ Υ Υ Υ Υ Υ Ttl Sum Redul Vrt Regre Sum f Squre (Errr Sum f f Squre Squre) TSS ESS RSS Dvde bth de b TSS ESS RSS TSS TSS ESS RSS R TSS TSS R the ttl prprt f the ttl t Υ epled b regre f Υ Χ Sce Errr Sum f Squre le betwee zer d TSS d R wll le betwee zer Th l true whe there tercept. Wthut tercept c get R > r < INSERT GRAPH N ft jut rdm pt D hgh vlue f N t rell! R mpl gd ft d lw vlue f R pr ft? Hgh vlue f R re cmm tme ere dt lw vlue re cmm cr-ectl dt 9
Nte pecl relthp fry α X ε.e. mple regre R jut qured crrelt ceffcet betwee Υ d Χ Tetg the Regre Equt Dvdg t Υ t tw cmpet ugget tttcl tet fr the etece f ler relthp betwee Υ d Χ EpledVrce F Ν U epledvrce RSS ESS Ν I geerl th tet tttc wll hve F dtrbut Ν F-tttc zer whe epled t zer Tetg the Regre Equt: Subdvdg the t Υ t tw cmpet ugget tttcl tet fr the etece f relthp betwee Υ d Χ EpledVrce F Ν U epledvrce RSS ESS ( Ν ) Epect trg tttcl relthp betwee Χ d Υ t reult lrge rt f epled t uepled ce tet wll hve F- dtrbut wth d Ν degree f freedm F wll be l whe the epled t the regre Fr ler relthp betwee Χ d Υ eed bg tet tttc Al te tht F t qured vlue f t-tttc fr Χ tw Ν Ν ble regre mdel