ECOOMICS 35* -- OTE 9 ECO 35* -- OTE 9 F-Tess and Analyss of Varance (AOVA n he Smple Lnear Regresson Model Inroducon The smple lnear regresson model s gven by he followng populaon regresson equaon, or PRE: Y + + u where u s d as (, σ (,, ( Ths s he smple Classcal ormal Lnear Regresson Model (CLRM OLS esmaon of he PRE ( yelds he followng OLS sample regresson equaon (or OLS-SRE: Y + + û Ŷ + û (,, ( where he OLS esmaor of he nercep coeffcen ; he OLS esmaor of he slope coeffcen ; Ŷ + s he -h esmaed (or predced value of he dependen varable from he OLS regresson, called he OLS sample regresson funcon (or OLS-SRF; û Y Ŷ Y s he -h OLS resdual ECO 35* -- oe 9: AOVA n he Smple CLRM Page of pages
ECOOMICS 35* -- OTE 9 Formulas: ( Y (Y ( y unbased esmaor of ; Y unbased esmaor of ; û σ unbased esmaor of σ ; ( Vâr( Σ σ Σ σ unbased esmaor of ; ( Var Vâr( ê( s σ unbased esmaor of ; ( se ( Vâr( σ σ unbased esmaor of ; ( Var Vâr( ê( s σ unbased esmaor of ( se ECO 35* -- oe 9: AOVA n he Smple CLRM Page of pages
ECOOMICS 35* -- OTE 9 F-Tess of Indvdual Coeffcen Equaly Resrcons oe 6 derved boh he -sasc and he F-sasc for oe 8 eplaned how o use he -sasc for o perform ess of equaly resrcons on ndvdual regresson coeffcens Ths secon oulnes how he F-sasc for can be used o perform wo-al ess of equaly resrcons on ndvdual regresson coeffcens The F- and -sascs for (, ( F( ~ F[, ] ; ( ~ [ ] Vâr( sê( Vâr( Relaonshp beween he es sascs F( and ( : ( ( F( or equvalenly ( F( The F-sasc for he square of he -sasc for The -sasc for he square roo of he F-sasc for Relaonshp beween he and F dsrbuons: The square of a -dsrbuon wh degrees of freedom has he F- dsrbuon wh numeraor degree of freedom and denomnaor degrees of freedom ( [ ] has he [, ] dsrbuon: F ( [ ] ~ F[, ] Implcaons: Le α be he chosen sgnfcance level for a wo-al hypohess es The crcal values of he F[, ] and [] dsrbuons are relaed as follows: ( [ Fα [, ] or [ ] Fα [, ] α ] α ECO 35* -- oe 9: AOVA n he Smple CLRM Page 3 of pages
ECOOMICS 35* -- OTE 9 Two-al F-ess of equaly resrcons on ndvdual regresson coeffcens Lke he -sasc (, he F-sasc F( can be used o perform wo-al ess of equaly resrcons on ndvdual regresson coeffcens ull and alernave hypoheses: H : b or b where b s a specfed consan H : b or b a wo-sded alernave hypohess a wo-al es A feasble es sasc for s eher he F-sasc or he -sasc for : ( ~ F[, ] F( or ( ~ [ ] Vâr( sê( Calculae he sample value of he F-sasc F( or he -sasc ( under he null hypohess H In he epresson for F( or (, se equal o b, whch s he value of specfed by H The resulng sample values of he es sascs F( and ( under H are: ( b F ( and Vâr( b ( sê( The null dsrbuons of F ( and ( : If he null hypohess H: b s rue, hen ( b ~ F[, ] F ( under H ; Vâr( b ( ~ [ ] under H sê( ECO 35* -- oe 9: AOVA n he Smple CLRM Page 4 of pages
ECOOMICS 35* -- OTE 9 Decson Rule -- Formulaon Le α he chosen sgnfcance level for he es F [, α ] he α-level crcal value of he F [, ] dsrbuon; α [ ] he α/-level crcal value of he [ ] dsrbuon A sgnfcance level α: Rean H : b f F ( Fα [, ] ; f ( α [ ] Reec H : b f F ( > Fα [, ] ; f ( > α [ ] Decson Rule -- Formulaon : he p-value approach The p-value for he calculaed F-sasc F ( Pr( F > F The wo-al p-value for he calculaed -sasc ( Pr > A sgnfcance level α: ( Rean H : b f p-value for F ( α ; f wo-al p-value for ( α Reec H : b f p-value for F ( < α f wo-al p-value for ( < α ECO 35* -- oe 9: AOVA n he Smple CLRM Page 5 of pages
ECOOMICS 35* -- OTE 9 Equvalence of he F-es and -es of H : b versus H : b follows from wo facs: ( The sample values of he wo es sascs are relaed accordng o he equaly F ( Under he null hypohess H : b, he calculaed sample value of he general F-sasc equals he square of : ( b b b F ( Vâr( Vâr( sê( ( The null dsrbuons of he wo es sascs are relaed accordng o a smlar equaly: ( [ ] ~ F[, ] e, he square of a [ ] dsrbuon has he F[, ] dsrbuon Implcaon: The square of he α crcal value of he [ ] dsrbuon equals he α-level crcal value of he F[, ] dsrbuon; e, ( [ ] F [, ] α α Usage of -sascs and F-sascs for esng ndvdual coeffcen equaly resrcons -sascs can be used o perform boh wo-al and one-al ess of equaly resrcons on ndvdual regresson coeffcens F-sascs can be used only for wo-al ess of equaly resrcons on ndvdual regresson coeffcens ECO 35* -- oe 9: AOVA n he Smple CLRM Page 6 of pages
ECOOMICS 35* -- OTE 9 3 The AOVA Table for he OLS SRE The OLS Decomposon Equaon: The Analyss-of-Varance (AOVA able for an OLS SRE such as ( s based on he OLS decomposon equaon y ŷ + û (3 Each of he hree erms n equaon (3 are defned as follows: ( y ( Y Y TSS he Toal Sum of Squares he oal sample varaon of he observed Y values ( y$ ( Y$ Y ESS he Eplaned Sum of Squares he sum of squares eplaned by he sample regresson funcon, e, by he regressor ( (3 u$ Y Y$ RSS he Resdual Sum of Squares he uneplaned varaon of he observed sample values Y of he regressand Y around he sample regresson lne ECO 35* -- oe 9: AOVA n he Smple CLRM Page 7 of pages
ECOOMICS 35* -- OTE 9 The General AOVA Table: The Analyss-of-Varance (AOVA able for an OLS SRE akes he followng general form Source of varaon SS df MSS SS/df The regresson funcon ESS y (eplaned $ ESS K ŷ K K The resduals RSS u (uneplaned $ RSS û K K K Toal sample varaon TSS y of Y Defnons: K he oal number of esmaed regresson coeffcens n he OLS- SRE Thus, K he number of esmaed slope coeffcens n he OLS-SRE The AOVA Table for a Smple OLS-SRE: The Analyss-of-Varance (AOVA able for a smple OLS-SRE such as equaon ( akes he followng form, where K and hence K Y + + û Ŷ + û (,, Source of varaon SS df MSS SS/df The regresson ESS funcon (eplaned ŷ ESS ŷ The resduals RSS u (uneplaned $ RSS û Toal sample TSS y varaon of Y oe: ESS ŷ where (,, ECO 35* -- oe 9: AOVA n he Smple CLRM Page 8 of pages
ECOOMICS 35* -- OTE 9 4 The F-Sasc for he AOVA Table: Smple Regresson Form of he AOVA F-Sasc: For a smple lnear regresson model such as equaon ( and s correspondng OLS-SRE ( Y + + û Ŷ + û (,,, he AOVA F-sasc s defned as he rao of ( he MSS for he regresson funcon o ( he MSS for he resduals: ( he MSS for he OLS sample regresson funcon ESS K ESS ( he MSS for he OLS resduals RSS K RSS The rao of ( o ( s he AOVA F-sasc For he general lnear regresson model wh K regresson coeffcens: ESS K AOVA F RSS K For he smple lnear regresson model wh K regresson coeffcens: ESS AOVA F RSS ECO 35* -- oe 9: AOVA n he Smple CLRM Page 9 of pages
ECOOMICS 35* -- OTE 9 Alernave Formula for he AOVA F-Sasc: For a smple lnear regresson model such as equaon ( and s correspondng OLS-SRE (, he AOVA F-sasc can be wren alernavely n erms of he OLS slope coeffcen esmaor and s esmaed varance âr( : V AOVAF ESS RSS ( σ û ŷ ŷ ( snce û ( σ σ σ Vâr( snce dvdng by snce σ ŷ Vâr( ECO 35* -- oe 9: AOVA n he Smple CLRM Page of pages
ECOOMICS 35* -- OTE 9 Dsrbuon of he AOVA F-Sasc: Under he null hypohess H : ha s f he null hypohess H : s rue he AOVA F - sasc has he F-dsrbuon wh numeraor degrees-of-freedom and denomnaor degrees-of-freedom ( Tha s, f he null hypohess H : s rue, ŷ ŷ ESS AOVAF ~ F[, ] RSS ( û ( σ AOVA F-Tes of H : agans H : : H : H : a wo-sded alernave hypohess The calculaed sample value of he AOVA F-sasc under he null hypohess H : s: ŷ ŷ ESS AOVA F RSS ( û ( σ Decson Rule Le F [, ] α he α-level crcal value of he F[, ] dsrbuon A sgnfcance level α: Rean H : f AOVA F F α[, ] ; f p-value for AOVAF α Reec H : f AOVA F > F α[, ] ; f p-value for AOVAF < α ECO 35* -- oe 9: AOVA n he Smple CLRM Page of pages
ECOOMICS 35* -- OTE 9 Equvalence of AOVA F-Tes and General F-Tes of H : agans H : : Apples only n he case of he smple lnear regresson model ha has us one slope coeffcen and one regressor The general F-sasc for akes he form ( ~ F[, ] F( Vâr( Under he null hypohess H :, he sample value of he general F- sasc ( s: F F ( ~ F[, ] under H : Vâr( For he smple lnear regresson model, he AOVA F-sasc equals he general F-sasc for under he null hypohess H: Tha s, only for he smple lnear regresson model s rue ha ESS AOVA F F ( : RSS ( Vâr( ECO 35* -- oe 9: AOVA n he Smple CLRM Page of pages