ACE 564 Spring Lecture 7. Extensions of The Multiple Regression Model: Dummy Independent Variables. by Professor Scott H.

Transcription

1 ACE 564 Spring 2006 Lecure 7 Exensions of The Muliple Regression Model: Dumm Independen Variables b Professor Sco H. Irwin Readings: Griffihs, Hill and Judge. "Dumm Variables and Varing Coefficien Models Chaper 12 in Learning and Pracicing Economerics Kenned. Dumm Variables, Chaper 14 in A Guide o Economerics ACE 564, Universi of Illinois a Urbana-Champaign 7-1

2 Inroducion Independen variables in regression models someimes are qualiaive in naure Male vs. female Warime vs. peaceime Farmer vs. non-farmer Region 1 vs. Region 2 Corn vs. sobeans The qualiaive naure of hese variables means some pe of prox mus be consruced Definiion: A dumm variable is an arificial variable consruced o ake on he value of one when he qualiaive phenomenon i represens occurs, and zero oherwise Once creaed, dumm variables can be used in he muliple linear regression model jus like oher variables However, will change inerpreaion of model! ACE 564, Universi of Illinois a Urbana-Champaign 7-2

3 ACE 564, Universi of Illinois a Urbana-Champaign 7-3

4 Dumm Variables and Income of Professionals Daa on income of professionals as shown on previous page Assume an individual s income depends on heir profession, so = α D + α D + α D + e 1 1, 2 2, 3 3, where is an individual s annual income D1, is a dumm variable aking on a value of one if a person is a docor, zero oherwise D 2, is a dumm variable aking on a value of one if a person is a professor, zero oherwise 3, D is a dumm variable aking on a value of one if a person is a lawer, zero oherwise ACE 564, Universi of Illinois a Urbana-Champaign 7-4

5 Now, le s consider he form of he model when we limi i o a paricular profession Docor: = α 1+ α 0+ α 0+ e = α + e 1 Professor: = α 0+ α 1+ α 0+ e = α + e 2 Lawer: = α 0+ α 0+ α 1+ e = α + e 3 Look familiar? Jus he consan mean model we sudied in ACE 362 ACE 564, Universi of Illinois a Urbana-Champaign 7-5

6 Hpoheical Daa Se for Professional Incomes D1, D2, D3, $50, $55, $75, $35, $38, $30, $45, $47, $41, ACE 564, Universi of Illinois a Urbana-Champaign 7-6

7 Esimae consan mean b sample average for each profession Docors $60, Professors $34, Lawers $44, x Turns ou we ge exacl he same parameer esimaes if we esimae he full model wih leas squares! ˆ = 60,000D + 34,333.33D + 44,333.33D 1, 2, 3, (12.71) (7.27) (9.39) ( -sa.) Dumm variable parameer esimaes in his regression show he average, or expeced, income for each of he professions ACE 564, Universi of Illinois a Urbana-Champaign 7-7

8 Noice ha an inercep in he usual sense is no included in he regression model This is done o avoid he dumm variable rap The inercep ( β 1 ) in a regression model is implicil represened b a column of ones B definiion, 1 = D1+ D2 + D3 Hence, if an inercep is included in he model an exac linear relaionship exiss beween he dumm variables and he implici inercep variable of he regression model Impossible o esimae model parameers wih leas squares ACE 564, Universi of Illinois a Urbana-Champaign 7-8

9 I is pical o include an inercep in regression models wih dumm variables Bu, one of he dummies has o be dropped in order o avoid he dumm variable rap Changes he inerpreaion of he dumm parameer esimaes! Le s consider his approach b dropping he dumm variable for docors from he model and changing noaion o keep he differen dumm variable models sraigh = β + β D + β D + e 1 2 2, 3 3, In his formulaion: The income of he omied caegor (docors) is given b he inercep The income of he included caegories (professors and lawers) is given b he sum of he inercep and relevan slope ACE 564, Universi of Illinois a Urbana-Champaign 7-9

10 Docor: = β + β 0+ β 0+ e = β + e 1 Professor: = β + β 1+ β 0+ e = β + β + e 1 2 Lawer: = β + β 0+ β 1+ e = β + β + e 1 3 ACE 564, Universi of Illinois a Urbana-Champaign 7-10

11 If we esimae his formulaion wih leas squares, he resuls are ˆ = 60,000 25,666.77D 15,666.77D 2, 3, (12.71) ( 3.84) ( 2.35) ( -sa.) The esimaed expeced incomes for he hree professions are, Docor: Professor: ˆ = $60,000 Lawer: = 60,000 25, = $34, ˆ = 60,000 15, = $44, ˆ ACE 564, Universi of Illinois a Urbana-Champaign 7-11

12 We can now see ha he wo formulaions of he dumm variable regression model provide he same informaion regarding expeced incomes of professionals, wih proper inerpreaion Wih dumm variables for all hree professions and no inercep, slopes direcl esimae expeced incomes of he differen professions Wih one dumm variable omied, he omied profession becomes he benchmark o which he ohers are compared More formall, he equivalence beween he parameers of he wo regression models is, α = β 1 1 α2 = β1+ β2 α3 = β1+ β3 Mos researchers prefer he inercep version, as i direcl shows wheher he caegorizaion makes a difference and b how much ACE 564, Universi of Illinois a Urbana-Champaign 7-12

13 ACE 564, Universi of Illinois a Urbana-Champaign 7-13

14 ACE 564, Universi of Illinois a Urbana-Champaign 7-14

15 To esimae he impac of gender on expeced income, we add one more dumm variable o he regression model = β + β D + β D + β D + e 1 2 2, 3 3, 4 4, where is an individual s annual income D 2, is a dumm variable aking on a value of one if a person is a professor, zero oherwise D 3, is a dumm variable aking on a value of one if a person is a lawer, zero oherwise D 4, is a dumm variable aking on a value of one if a person is female, zero if a person is male Noe ha onl one dumm variable is added for gender If we creaed one dumm for males and one for females, hen we would be back in he dumm variable rap because DM + DF equals one, he same value as he implici inercep variable ACE 564, Universi of Illinois a Urbana-Champaign 7-15

16 We can now work ou he model b profession and gender Male Docor: = β + β 0+ β 0+ β 0+ e = β + e 1 Female Docor: = β + β 0+ β 0+ β 1+ e = β + β + e 1 4 Male Professor: = β + β 1+ β 0+ β 0+ e = β + β + e 1 2 Female Professor = β + β 1+ β 0+ β 1+ e = β + β + β + e ACE 564, Universi of Illinois a Urbana-Champaign 7-16

17 Male Lawer: = β + β 0+ β 1+ β 0+ e = β + β + e 1 3 Female Lawer: = β + β 0+ β 1+ β 1+ e = β + β + β + e ACE 564, Universi of Illinois a Urbana-Champaign 7-17

18 Hpoheical Daa Se for Professional Incomes D2, D3, D4, $50, $55, $75, $35, $38, $30, $45, $47, $41, ACE 564, Universi of Illinois a Urbana-Champaign 7-18

19 If we esimae his expanded model wih leas squares, he resuls are ˆ = 67, ,666.67D 19,333.33D 11,000.00D 2, 3, 4, (16.00) ( 5.57) ( 4.03) (-2.76) ( -sa.) The esimaed expeced incomes for he professions and gender are, Male Docor: Female Docor: ˆ = $67, = 67, , = $56, ˆ Male Professor: = 67, , = $41, ˆ Female Professor: = 67, , , = $30, ˆ ACE 564, Universi of Illinois a Urbana-Champaign 7-19

20 Male Lawer: = 67, , = $48, ˆ Female Lawer: = 67, , , = $37, ˆ Noe ha he regression income esimae for a professional group does no equal he corresponding sample average; his onl occurs in he special case illusraed earlier in his secion Also noe ha he formulaion in his secion forces he gender income differenial o be he same for he differen professions We could specif a more flexible model ha allowed he gender differenial o var b profession Allow ineracion effecs beween profession and gender Need o specif separae dumm variables for each caegor of profession and gender (Kenned, pp ) ACE 564, Universi of Illinois a Urbana-Champaign 7-20

21 Dumm Variables, Income of Professionals, and Years of Experience The previous example is unrealisic, in ha all of he independen variables are dumm variables In mos applicaions involving dumm variables, he model will also include oher quaniaive independen variables This can be illusraed b expanding he income of professionals model o include ears of professional work experience Gender no included o keep he model simple The expanded model is, = β + β D + β D + β x + e 1 2 2, 3 3, 4 4, where he oher variables are he same, excep x 4, represens he number of ears of professional experience This model, in essence, expresses income as a linear funcion of experience, wih a differen inercep for each profession ACE 564, Universi of Illinois a Urbana-Champaign 7-21

22 More specificall, he incomes for he differen professions are now given as, Docor: = β + β 0+ β 0+ β x + e , = β + β x + e 1 4 4, Professor: = β + β 1+ β 0+ β x + e , = β + β + β x + e , Lawer: = β + β 0+ β 1+ β x + e , = β + β + β x + e , ACE 564, Universi of Illinois a Urbana-Champaign 7-22

23 Hpoheical Daa Se for Professional Incomes D2, D3, x4, $50, $55, $75, $35, $38, $30, $45, $47, $41, ACE 564, Universi of Illinois a Urbana-Champaign 7-23

24 If we esimae his formulaion wih leas squares, he resuls are ˆ = 53, ,480.04D 14,853.29D x 2 3 4, (12.54) ( 5.39) ( 3.02) (2.48) ( - sa.) The esimaed expeced income funcions for he hree professions are, Docor: ˆ = 53, x 4, Professor: = 53, , x = 27, x ˆ 4, 4, Lawer: = 53, , x = 38, x ˆ 4, 4, ACE 564, Universi of Illinois a Urbana-Champaign 7-24

25 Esimaed Regression Models for Professional Income Wih Inercep Dumm Variables $80,000 $70,000 Docors $60,000 Annual Income $50,000 Lawers Professors $40,000 $30,000 $20, Years of Professional Experience ACE 564, Universi of Illinois a Urbana-Champaign 7-25

26 Tesing for Individual Qualiaive Effecs The poin esimaes for he regression model indicae ha, Holding experience consan, docors earn $26,480 more per ear han professors Holding experience consan, docors earn $14,853 more per ear han lawers We would like o es wheher hese income differences are significanl differen from a saisical sandpoin We sar b re-saing he esimaion resuls, ˆ = 53, , D 14,853.29D x 2 3 4, (12.54) ( 5.39) ( 3.02) (2.48) ( - sa.) ACE 564, Universi of Illinois a Urbana-Champaign 7-26

27 Problem I: Tesing Significance of Difference in he Income of Docors and Professors 1. Hpoheses H H : β = : β Tes saisic 2 * b2 β2 b2 0 b2 26, = = = = = 5.39 se( b ) se( b ) se( b ) 4, Rejecion Region For α = 0.05, we wan o find he criical - value where P [ 2 c ] = 0.05 Since his is a wo-ailed es, he rejecion region is he wo-ailed region deermined as, α/2, T K 2 α/2, T K 0.05/ 2, / 2,9 4 2 ACE 564, Universi of Illinois a Urbana-Champaign 7-27

28 4. Decision Since ( 2 = 5.39) < ( c = 2.571) we rejec he null hpohesis and conclude ha he alernaive hpohesis is more consisen wih he sample daa Sample evidence suppors he proposiion of a saisicall significan difference in he annual income of docors and professors Problem II: Tesing Significance of Difference in he Income of Docors and Lawers 1. Hpoheses H H : β = 0 : β Tes saisic 2 * b3 β3 b3 0 b3 14, = = = = = 3.02 se( b ) se( b ) se( b ) 4, ACE 564, Universi of Illinois a Urbana-Champaign 7-28

29 3. Rejecion Region For α = 0.05, we wan o find he criical - value where P [ 2 c ] = 0.05 Since his is a wo-ailed es, he rejecion region is he wo-ailed region deermined as, α/2, T K 2 α/2, T K 0.05/ 2, / 2, Decision Since ( 2 = 3.02) < ( c = 2.571) we rejec he null hpohesis and conclude ha he alernaive hpohesis is more consisen wih he sample daa Sample evidence suppors he proposiion of a saisicall significan difference in he annual income of docors and lawers ACE 564, Universi of Illinois a Urbana-Champaign 7-29

30 Tesing Joinl for Qualiaive Effecs Now we wan o es wheher he income differences are joinl significanl differen This es is jus anoher applicaion of he F-es mehodolog presened in Lecure 6 The regression model wih a full se of inercep of dumm variables is specified as he unresriced regression model, = β + β D + β D + β x + e 1 2 2, 3 3, 4 4, When esimaed, his model will have sum of squared errors SSE U The null hpohesis of ineres is, H : β = β = ACE 564, Universi of Illinois a Urbana-Champaign 7-30

31 Now, impose he null hpohesis resricions on he unresriced regression model o obain he following resriced model, = β + β x + e 1 4 4, When esimaed, his model will have sum of squared errors SSE R We can hen compue he es saisic F, F = ( SSER SSEU)/ J SSE /( T K) U Finall, compare he calculaed F o he criical value from an F-disribuion able, F ( JT, K) α ACE 564, Universi of Illinois a Urbana-Champaign 7-31

32 Unresriced model esimaes: ˆ = 53, , D 14,853.29D x SSE U 2 3 4, (12.54) ( 5.39) ( 3.02) (2.48) ( - sa.) = 180,366,932.8 Resriced model esimaes: ˆ = 45, x (25.44) ( 5.78) SSE = 1,404,714,992 R 4, (-sa.) F (1,404,714, ,366,932.8) / 2 = = ,366,932.8/(9 4) Since F = > F 0.05(2,5) = 5.79, rejec null hpohesis Sample daa are no consisen wih he hpohesis ha he inercep parameers are he same for all hree professions Inappropriae o resric he inercep o be he same for docors, professors and lawers ACE 564, Universi of Illinois a Urbana-Champaign 7-32

33 Inercep and Slope Dumm Variables, Income of Professionals and Years of Experience We are no limied o regression models where onl he inercep is allowed o shif I is possible o allow he slope on non-dumm independen variables o shif as well The seup for his pe of regression model is, = β + β D + β D + β x + β ( D x ) + β ( D x ) + e 1 2 2, 3 3, 4 4, 5 2, 4, 6 3, 4, where wo new variables, ( D2, x4, ) and ( D3, x 4, ), are included o represen he ineracion of profession and ears of professional experience on income Experience ma be more valuable in some professions β 5 is he difference beween slope for docors and professors 6 β is he difference beween slope for docors and lawers ACE 564, Universi of Illinois a Urbana-Champaign 7-33

34 More specificall, he incomes for he differen professions are now given as, Docor: = β + β 0+ β 0 + β x + β (0 x ) + β (0 x ) + e , 5 4, 6 4, = β + β x + e 1 4 4, Professor: = β + β 1+ β 0 + β x + β (1 x ) + β (0 x ) + e , 5 4, 6 4, = β + β + ( β + β ) x + e , Lawer: = β + β 0+ β 1 + β x + β (0 x ) + β (1 x ) + e , 5 4, 6 4, = β + β + ( β + β ) x + e , ACE 564, Universi of Illinois a Urbana-Champaign 7-34

35 Hpoheical Daa Se for Professional Incomes D2, D3, x4, D2, x4, D3, x 4, $50, $55, $75, $35, $38, $30, $45, $47, $41, ACE 564, Universi of Illinois a Urbana-Champaign 7-35

36 If we esimae his formulaion wih leas squares, he resuls are ˆ = 44, ,024.57D2 3,760.10D3 + (25.44) ( 5.78) ( 1.47) x4, 1,695.99( D2, x4, ) 1,477.50( D3, x4, ) (10.99) ( 7.63) ( 4.87) ( -sa.) The esimaed expeced income funcions for he hree professions are, Docor: ˆ = 44, ,056.45x 4, Professor: ˆ = 44, , (2, ,695.99) x = 31, x 4, 4, Lawer: ˆ = 44, , (2, ,477.50) x = 40, x 4, 4, ACE 564, Universi of Illinois a Urbana-Champaign 7-36

37 Esimaed Regression Models for Professional Income Wih Boh Inercep and Slope Dumm Variables $80,000 Docors $70,000 $60,000 Annual Income $50,000 $40,000 Lawers Professors $30,000 $20, Years of Professional Experience ACE 564, Universi of Illinois a Urbana-Champaign 7-37

38 The Chow Tes We would like o formall es wheher, Inerceps and slopes are differen for each of he hree groups of professionals vs. Inerceps and slopes are he same for each of he hree groups of professionals This es is jus anoher applicaion of he F-es mehodolog presened in Lecure 6 Named afer economerician Gregor Chow, who firs proposed he es Also, known as a es of he equivalence of regressions, a es of pooling daa or a es of srucural change ACE 564, Universi of Illinois a Urbana-Champaign 7-38

39 The regression model wih a full se of inercep and slope dumm variables is specified as he unresriced regression model, = β + β D + β D + β x + β ( D x ) + β ( D x ) + e 1 2 2, 3 3, 4 4, 5 2, 4, 6 3, 4, When esimaed, his model will have sum of squared errors SSE U The null hpohesis of ineres is, H : β = β = β = β = Now, impose he null hpohesis resricions on he unresriced regression model o obain he following resriced model, = β + β x + e 1 4 4, When esimaed, his model will have sum of squared errors SSE R ACE 564, Universi of Illinois a Urbana-Champaign 7-39

40 We can hen compue he es saisic F, F = ( SSER SSEU)/ J SSE /( T K) U Finall, compare he calculaed F o he criical value from an F-disribuion able, F ( JT, K) Unresriced model esimaes: α ˆ = 44, ,024.57D 3,760.10D (25.44) ( 5.78) ( 1.47) x 1,695.99( D x ) 1,477.50( D x ) 4, 2, 4, 3, 4, (10.99) ( 7.63) ( 4.87) ( -sa.) SSE = 8,680, U Resriced model esimaes: ˆ = 45, x (25.44) ( 5.78) SSE = 1,404,714,992 R 4, (-sa.) F (1,404,714,992 8,680, ) / 4 = = ,680, /(9 6) ACE 564, Universi of Illinois a Urbana-Champaign 7-40

41 Since F = > F 0.05(4,3) = 9.13, rejec null hpohesis Sample daa are no consisen wih he hpohesis ha he inercep and slope parameers are he same for all hree professions Inappropriae o pool he sample daa for docors, professors and lawers and esimae one regression model While i is clearl inappropriae o esimae one regression for all hree professions, we do no e know wheher we can resric he inerceps or slopes (bu no boh) o be he same for he differen professions These ess are no Chow ess for pooling bu are so close in procedure ha his is a good place o go over such esing ACE 564, Universi of Illinois a Urbana-Champaign 7-41

42 Tesing for Equivalence of Slopes Once again, he regression model wih a full se of inercep and slope dumm variables is specified as he unresriced regression model, = β + β D + β D + β x + β ( D x ) + β ( D x ) + e 1 2 2, 3 3, 4 4, 5 2, 4, 6 3, 4, When esimaed, his model will have sum of squared errors SSE U The null hpohesis of ineres is, H : β = β = Now, impose he null hpohesis resricions on he unresriced regression model o obain he following resriced model, = β + β D + β D + β x + e 1 2 2, 3 3, 4 4, When esimaed, his model will have sum of squared errors SSE R ACE 564, Universi of Illinois a Urbana-Champaign 7-42

43 We can hen compue he es saisic F, F = ( SSER SSEU)/ J SSE /( T K) U Finall, compare he calculaed F o he criical value from an F-disribuion able, F ( JT, K) Unresriced model esimaes: α ˆ = 44, ,024.57D 3,760.10D (25.44) ( 5.78) ( 1.47) x 1,695.99( D x ) 1,477.50( D x ) 4, 2, 4, 3, 4, (10.99) ( 7.63) ( 4.87) ( -sa.) SSE = 8,680, U Resriced model esimaes: ˆ = 53, , D 14,853.29D x SSE R 2 3 4, (12.54) ( 5.39) ( 3.02) (2.48) ( - sa.) = 180,366,932.8 F (180,366, ,680, ) / 2 = = ,680, /(9 6) ACE 564, Universi of Illinois a Urbana-Champaign 7-43

44 Since F = > F 0.05(2,3) = 9.55, rejec null hpohesis Sample daa are no consisen wih he hpohesis ha he slope parameers are he same for all hree professions Inappropriae o resric he slopes o be he same for docors, professors and lawers ACE 564, Universi of Illinois a Urbana-Champaign 7-44

45 Tesing for Equivalence of Inerceps Once again, he regression model wih a full se of inercep and slope dumm variables is specified as he unresriced regression model, = β + β D + β D + β x + β ( D x ) + β ( D x ) + e 1 2 2, 3 3, 4 4, 5 2, 4, 6 3, 4, When esimaed, his model will have sum of squared errors SSE U The null hpohesis of ineres is, H : β = β = Now, impose he null hpohesis resricions on he unresriced regression model o obain he following resriced model, = β + β x + β ( D x ) + β ( D x ) + e 1 4 4, 5 2, 4, 6 3, 4, When esimaed, his model will have sum of squared errors SSE R ACE 564, Universi of Illinois a Urbana-Champaign 7-45

46 We can hen compue he es saisic F, F = ( SSER SSEU)/ J SSE /( T K) U Finall, compare he calculaed F o he criical value from an F-disribuion able, F ( JT, K) Unresriced model esimaes: α ˆ = 44, ,024.57D 3,760.10D (25.44) ( 5.78) ( 1.47) x 1,695.99( D x ) 1,477.50( D x ) 4, 2, 4, 3, 4, (10.99) ( 7.63) ( 4.87) ( -sa.) SSE = 8,680, U Resriced model esimaes: ˆ = ,654.03x 2,677.79( D x ) 1,752.36( D x ) 4, 2, 4, 3, 4, SSE R (13.96) (6.95) ( 6.94) (-3.76) ( - sa.) = 115,326,714.6 F (115,326, ,680, ) / 2 = = ,680, /(9 6) ACE 564, Universi of Illinois a Urbana-Champaign 7-46

47 Since F = > F 0.05(2,3) = 9.55, rejec null hpohesis Sample daa are no consisen wih he hpohesis ha he inercep parameers are he same for all hree professions Inappropriae o resric he inercep o be he same for docors, professors and lawers I is ineresing o noe ha hpohesis esing suggess he unresriced model, wih a full se of inercep and slope shifers, is consisen wih he sample daa Wih one excepion, his is he same as saing ha hree separae regressions should be esimaed for he differen professions Unresriced model assumes variance of regression is he same for all hree professions Three separae regressions allows he variance of regression o differ across he professions ACE 564, Universi of Illinois a Urbana-Champaign 7-47

48 Srucural Change and he Consumpion-Income Relaionship in he US The income of professionals example is used o examine cross-secional qualiaive effecs I is useful o presen an example of ime-series qualiaive effecs The Problem: Esimaing he relaionship beween consumpion and income in he US over Time period spans he Grea Depression and several wars Consumpion was paricularl resriced relaive o income during WWII Ma no be appropriae o assume he same relaionship over he enire sample period ACE 564, Universi of Illinois a Urbana-Champaign 7-48

49 ACE 564, Universi of Illinois a Urbana-Champaign 7-49

50 Begin wih he following saisical model, = β + β x + e 1 3 where is real per capia consumpion in he US x is real per capia income in he US Nex, define a dumm variable o reflec he period in WWII when consumpion was resriced hrough war-ime raioning D 1 if = 1941,...,1946 = 0 oherwise ACE 564, Universi of Illinois a Urbana-Champaign 7-50

51 Now, add he dumm variable o he saisical model, = β + β D + β x + e The model for differen ime periods is, , : = β + β 0 + β x + e = β + β x + e : = β + β 1+ β x + e = β + β + β x + e Esimaion Resuls: ˆ = D x (3.98) ( 10.91) (58.73) ( - sa.) ACE 564, Universi of Illinois a Urbana-Champaign 7-51

52 ACE 564, Universi of Illinois a Urbana-Champaign 7-52