Fdametals of Regresso Aalyss
Regresso aalyss s cocered wth the stdy of the depedece of oe varable, the depedet varable, o oe or more other varables, the explaatory varables, wth a vew of estmatg ad/or predctg the poplato mea or average vales of the former terms of the kow or fxed repeated samplg vales of the latter. Dataset
Depedet Varable xplaed varable Predctad Regressad Respose dogeos Otcome Cotrolled varable Idepedet varable Idepedet varable Predctor Regressor Stmls xogeos Covarate Cotrol varable 3
Icome 8 4 6 8 4 6 Cosmpto 55 65 79 8 35 37 5 6 7 84 93 7 5 36 37 45 5 65 74 9 95 4 4 55 75 7 8 94 3 6 3 44 5 65 78 75 85 98 8 8 35 45 57 75 8 88 3 5 4 6 89 85 5 6 9 Total 35 46 445 77 678 75 685 43 966 Codtoal mea 65 77 89 3 5 37 49 6 73 4
Weekly Cosmpto 9 7 5 3 9 7 5 Codtoal expected vales 6 8 4 6 8 4 6 8 Weekly Icome Poplato Regresso Crve A poplato regresso crve s smply the locs of the codtoal meas of the depedet varable for the fxed vales of the explaatory varables. Dataset 5
f Codtoal xpectato Fcto CF Poplato Regresso Poplato Regresso Fcto PRF Lear Poplato Regresso Fcto Regresso Coeffcets 6
7 3 3 4 3 e Lear parameter fctos No-lear parameter fcto
8 Stochastc error term Systematc compoet Nosystematc compoet ] [ ] Dataset
Weekly Cosmpto SRF vs SRF 9 7 5 3 9 7 5 SRF y =,576x + 7,7 SRF y =,59x + 4,455 6 6 6 Weekly Icome Dataset 9
PRF SRF stmate
ṷ A
Learty. The relatoshp betwee depedet ad depedet varable s lear. Fll Rak. There s o exact relatoshp amog ay depedet varables. xogeety of depedet varables. The error term of the regresso s ot a fcto of depedet varables. Homoscedaststy ad o Atocorrelato. rror term of the regresso s depedetly ad ormally dstrbted wth zero meas ad costat varace. Normalty of rror term
3, f
4
5, Cov Var Var, Cov
Lear Regresso Model s assmed to be ostochastc. Zero mea vales of dstrbace
Homoscedastcty or eqal varace of f var [ ]
f Heteroscedastct y var
No atocorrelato betwee the dstrbaces } ] }{[ ] {[,, cov j j j j j j xogeety. Zero covarace betwee ad, cov
The mber of observatos shold be greater tha the mber of parameters to be estmated k. Varablty vales The regresso model s correctly specfed. There s o perfect mltcollearty.
W W W stmator Tre vale j j W W W W W W W Note that W
W W Accordg to or xogety assmpto. rror term s depedet from varable. Ths, OLS estmator s based estmator.
j j j j j j WW W WW W W W Var W Accordg to Homoscedastcty ad o ato-correlato assmptos.
j Var W Accordg to Homoscedastcty ad o ato-correlato assmptos.
STDV STDV var var, cov
Samplg dstrbto of
TSS = SS + RSS TSS RSS TSS SS R
y y y y TSS = SS + RSS TSS RSS TSS SS y y y y R
TSS = SS + RSS R R R SS TSS R RSS TSS RSS / N TSS / N RSS TSS The R sqared creases f a regressor s added to the model. Why? Ht: Cosder sm of sqared resdals. N N k k
t se / t se / OLS estmates have t-dstrbto wth -k df, where k s the mber of parameters. Pr[ t / se t / se ]
I ths lectre we wll cosder two cases. Hypotheses volvg coeffcet t-test. Hypotheses volvg or more coeffcets F-test
. Hypotheses volvg coeffcet xamples: For regresso coeffcet the hypothess ca be whch s a two-sded test, or t ca be whch s a oe-sded test Iterpret these tests for b = b H b H a : : b H b H a : :
Test statstcs t whch has t-dstrbto wth df -k t k b se If the Nll hypothess s ot tre, the t-statstc s lkely to have a large absolte vale. If the absolte vale of t-statstc s greater tha ts crtcal vale we reject the Nll hypothess, otherwse we caot. The crtcal vale ca be looked p from the table for t-dstrbto s crtcal vales. k For two-sded test t s For oe-sded test t s t / t
Hypothess volvg or more coeffcets The eed for these kds of tests arse whe we estmate mltple regresso models... For example, the wage fcto k k wage edc 3geder 4 exp rego
Hypothess volvg or more coeffcets H, : 3 4 Ha : They are ot both Techcally we eed to compare two models wage edc exp 3geder 4 whch says that geder ad rego are mportat factors of wage rego ad wage edc exp whch says that they are ot
Hypothess volvg or more coeffcets RSS Deote as the sm of sqared errors for the frst model f the Nll hypothess s tre RSS Deote as the sm of sqared errors for the secod model f the Nll hypothess s ot tre F RSS RSS RSS / N / m k Ths F statstc has F-dstrbto wth m df for the merator ad N-k df of deomator, where m s the mber of restrctos the dfferece the mber of depedet varables If F F c the reject yor Nll hypothess
xample. Sppose yo wat to test that the margal propesty to cosme s less tha.65 Based o 3 observatos yo estmated the smple regresso model by OLS ad obtaed the reslts Cos 97.5.6Ic Let the stadard error of the slope be se.
xample coted or hypothess test s set p as t-statstc s eqal to t H H a : :.65.65 Compare that wth.5 wth df = 8 The coclso s that we reject the ll..6.65.5. t.7
xample : Sppose that yo wat to test that geder ad age do ot affect wage Based o 5 observatos yo estmated the followg wage fcto or hypothess s set p as Ha : Both are ot eqal to zero wage 5.85 36.8geder.47edc 8.54age.39 exp 45.rego H : 3, 4
xample coted Sppose that RSS of the restrcted model Nll s ot tre s 33.85 ad RSS of the restrcted model Nll s tre s 4.5, the F statstc s F 4.5 33.85 33.85/ 44 / 5.64 The crtcal vale for F statstc wth df ad 44 ad 5% sgfcace level s 3.34 Sce or F statstc s greater tha the crtcal vale the we reject the ll.