Economics 326 Methods of Empirical Research in Economics. Lecture 8: Multiple regression model

Transcription

1 Ecoomics 326 Methods of Empirical Research i Ecoomics Lecture 8: Multiple regressio model Hiro Kasahara Uiversity of British Columbia December 24, 2014

2 Why we eed a multiple regressio model I There are may factors a ectig the outcome variable Y. I If we wat to estimate the margial e ect of oe of the factors (regressors), we eed to cotrol for other factors. I Suppose that we are iterested i the e ect of X 1 o Y, but Y is a ected by both X 1 ad X 2 : Y i = β 0 + β 1 X 1,i + β 2 X 2,i + U i. I Suppose we regress Y oly agaist X 1 : ˆβ 1 = (X 1,i X 1 ) Y i (X 1,i X 1 ) 2. 1/17

3 Omitted variable bias Sice Y depeds o X 2 : Y i = β 0 + β 1 X 1,i + β 2 X 2,i + U i, I We have: ˆβ 1 = (X 1,i X 1 ) (β 0 + β 1 X 1,i + β 2 X 2,i + U i ) (X 1,i X 1 ) 2 (X 1,i X 1 ) X 2,i = β 1 + β 2 (X 1,i X 1 ) 2 + (X 1,i X 1 ) U i (X 1,i X 1 ) 2. I Assume that E (U i jx 1,i, X 2,i ) = 0. Now, coditioal o X s: E ˆβ 1 = β1 + β 2 (1/) (X 1,i X 1 ) (X 2,i X 2 ) (1/) (X 1,i X 1 ) 2 {z } t Cov(X 1, X 2 )/Var(X 1 ) 2/17

4 Omitted variable bias I Whe the true model is but we regress oly o X 1, Y i = β 0 + β 1 X 1,i + β 2 X 2,i + U i, Y i = β 0 + β 1 X 1,i + V i, where V i is the ew error term V i = β 2 X 2,i + U i. I If X 1 ad X 2 are related, E (V i jx 1,i ) = E (β 2 X 2,i + U i.jx 1,i ) = β 2 E (X 2,i jx 1,i ) 6= 0. I Whe X 1 chages, X 2 chages as well, which cotamiates estimatio of the e ect of X 1 o Y. I As a result, ˆβ 1 from the regressio of Y o X 1 aloe is biased. 3/17

5 Example 3.1: Determiatios of College GPA I Data: A sample of 141 studets from a large uiversity. colgpa I Y i = colgpa i : the college GPA I X 1,i = hsgpa i : high school GPA I X 2,i = ACT i : achievemet test score I The OLS regressio lie: \ colgpa = hsgpa ACT I The simple model without hsgpa: \ colgpa = ACT Cov(ACT i, V i ) = β 1 Cov(ACT i, hsgpa i ) > 0 ) Positive Bias 4/17

6 Multiple liear regressio model I The ecoometricia observes the data: f(y i, X 1,i, X 2,i,..., X k,i ) : i = 1,..., g. I The model: Y i = β 0 + β 1 X 1,i + β 2 X 2,i β k X k,i + U i, E (U i jx 1,i, X 2,i,..., X k,i ) = 0. I We also assume o multicolliearity: Noe of the regressors are costat ad there are o exact liear relatioships amog the regressors. 5/17

7 Iterpretatio of the coe ciets Y i = β 0 + β 1 X 1,i + β 2 X 2,i β k X k,i + U i. I β j is a partial (margial) e ect of X j o Y : β j = Y i X j,i. I For example, β 1 is the e ect of X 1 o Y while holdig the other regressors costat (or cotrollig for X 2,..., X k ) Y = β 0 + β 1 X 1 + β 2 X β k X k + U. I I data, the values of all regressors usually chage from observatio to observatio. If we do ot cotrol for other factors, we caot idetify the e ect of X 1. 6/17

8 Chagig more tha oe regressor simultaeously I There are cases whe we wat to chage more tha oe regressor at the same time to d a e ect o Y. I Example 3.2: the results from 526 observatios o workers \ log(wage) = edu exper +.022teure I The e ect of stayig oe more year at the same rm: icreasig both exper ad teure. I Holdig edu xed, log(wage) \ = 0041 exper teure = = /17

9 OLS estimatio I The OLS estimators ˆβ 0, ˆβ 1,..., ˆβ k are the values that miimize the least squares fuctio: mi Q (b 0, b 1,..., b k ), where b 0,b 1,...,b k Q (b 0, b 1,..., b k ) = I The partial derivative with respect to b 0 is Q (b 0, b 1,..., b k ) = 2 b 0 (Y i b 0 b 1 X 1,i... b k X k,i ) 2. (Y i b 0 b 1 X 1,i... b k X k,i ). I The partial derivative with respect to b j, j = 1,..., k is Q (b 0, b 1,..., b k ) = 2 b j (Y i b 0 b 1 X 1,i... b k X k,i ) X j,i. 8/17

10 Normal equatios ( rst-order coditios for OLS) I The OLS estimators ˆβ 0, ˆβ 1,..., ˆβ k are obtaied by solvig the followig system of ormal equatios: Y i ˆβ 0 ˆβ 1 X 1,i... ˆβ k X k,i = 0, Y i ˆβ 0 ˆβ 1 X 1,i... ˆβ k X k,i X1,i = 0,. =. Y i ˆβ 0 ˆβ 1 X 1,i... ˆβ k X k,i Xk,i = 0. 9/17

11 Normal equatios ( rst-order coditios for OLS) I Sice the tted residuals are Û i = Y i ˆβ 0 ˆβ 1 X 1,i... ˆβ k X k,i, the ormal equatios ca be writte as Û i = 0, Û i X 1,i = 0,. =. Û i X k,i = 0. I We choose ˆβ 0, ˆβ 1,..., ˆβ k so that Û s ad regressors are orthogoal (ucorrelated i the sample). 10/17

12 Partitioed regressio I A represetatio for idividual ˆβ s ca be obtaied through the partitioed regressio result. Suppose we wat to obtai a expressio for ˆβ 1. I Cosider rst regressig X 1,i agaist other regressors ad a costat: X 1,i = ˆγ 0 + ˆγ 2 X 2,i ˆγ k X k,i + X 1,i, where ˆγ 0, ˆγ 2,..., ˆγ k are the OLS coe ciets, ad X 1,i is the tted OLS residual: X 1,i = 0, ad i =1 X 1,i X j,i = 0 for j = 2,..., k. i =1 I The ˆβ 1 ca be writte as ˆβ 1 = i =1 X 1,i Y i i =1 X 2 1,i. 11/17

13 Proof of the partitioed regressio result I We ca write Y i = ˆβ 0 + ˆβ 1 X 1,i + ˆβ 2 X 2,i ˆβ k X k,i + Û i, where Ûi = Ûi X 1,i =... = Ûi X k,i = 0. I Now, X 1,i Y i X 2 1,i = = X 1,i ˆβ 0 + ˆβ 1 X 1,i + ˆβ 2 X 2,i ˆβ k X k,i + Û i X 2 1,i = ˆβ 0 X 1,i X 2 1,i + ˆβ 1 X 1,i X 1,i X 2 1,i + + ˆβ 2 X 1,i X 2,i X 2 1,i ˆβ k X 1,i X k,i X 2 1,i + X 1,i Û i X 2 1,i 12/17

14 Proof of the partitioed regressio result X 1,i Y i X 2 1,i = ˆβ 0 X 1,i X 2 1,i + ˆβ 1 X 1,i X 1,i X 2 1,i + + ˆβ 2 X 1,i X 2,i X 2 1,i ˆβ k X 1,i X k,i X 2 1,i + X 1,i Û i X 2 1,i. We will show that: 1. X 1,i = X 1,i X 2,i =... = X 1,i X k,i = X 1,i X 1,i = X 1,i X 1,i Û i = 0. The X 1,i Y i X 2 1,i = ˆβ 1. 13/17

15 Proof of the partitioed regressio result (steps 1-2) I X 1,i is the tted OLS residual: X 1,i = ˆγ 0 + ˆγ 2 X 2,i ˆγ k X k,i + X 1,i, where ˆγ 0, ˆγ 2,..., ˆγ k are the OLS coe ciets. I The ormal equatios for this regressio are: X 1,i = 0, X 1,i X 2,i = 0,. =. X 1,i X k,i = 0. 14/17

16 Proof of the partitioed regressio result (step 3) Agai, because X 1,i are the OLS residuals ( tted) from the regressio of X 1 agaist X 2,..., X 4 : = X 1,i X 1,i X 1,i = ˆγ 0 ˆγ 0 + ˆγ 2 X 2,i ˆγ k X k,i + X 1,i X 1,i + ˆγ 2 X 1,i X 2,i ˆγ k = ˆγ ˆγ ˆγ k 0 + X 2 1,i = X 1,i X k,i + X 2 1,i (Because of the ormal equatios for the X 1 regressio.) X 1,i X 1,i 15/17

17 Proof of the partitioed regressio result (step 4) Lastly, because Û are the tted residuals from the regressio of Y agaist all X s: Û i = Û i X 1,i =... = Û i X k,i = 0. = = X 1,i Û i (X 1,i ˆγ 0 ˆγ 2 X 2,i... ˆγ k X k,i ) Û i X 1,i Û i ˆγ 0 Û i ˆγ 2 X 2,i Û i... ˆγ k X k,i Û i = 0 ˆγ 0 0 ˆγ ˆγ k 0 = 0. 16/17

18 "Partiallig out" ˆβ 1 = X 1,i Y i X 2 1,i 1. First, we regress X 1 agaist the rest of the regressors (ad a costat) ad keep X 1 which is the "part" of X 1 that is ucorrelated. 2. The, to obtai ˆβ 1, we regress Y agaist X 1 which is "clea" from correlatio with other regressors (o itercept). ˆβ 1 measures the e ect of X 1 after the e ects of X 2,..., X k have bee partialled out or etted out. 17/17