Statstcs for Managers Usng Mcrosoft Excel/SPSS Chapter 14 Multple Regresson Models 1999 Prentce-Hall, Inc. Chap. 14-1
Chapter Topcs The Multple Regresson Model Contrbuton of Indvdual Independent Varables Coeffcent of Determnaton Categorcal Explanatory Varables Transformaton of Varables Model Buldng 1999 Prentce-Hall, Inc. Chap. 14-2
Populaton Y-ntercept The Multple Regresson Model Relatonshp between 1 dependent & 2 or more ndependent varables s a lnear functon Populaton slopes Random Error Y X X 0 1 1 2 2 p X p Ŷ b b X b X 0 1 1 2 2 b p X p e Dependent (Response) varable for sample Independent (Explanatory) varables for sample model 1999 Prentce-Hall, Inc. Chap. 14-3
Sample Multple Regresson Model Y Y b0 b1 X1 b2x 2 b p X p e e X 2 X 1 Ŷ b b X b X 0 1 1 2 2 b p X p 1999 Prentce-Hall, Inc. Chap. 14-4
Multple Regresson Model: Example Develop a model for estmatng heatng ol used for a sngle famly home n the month of January based on average temperature and amount of nsulaton n nches. O l (G a l) T e m p ( 0 F) In su la to n 275.30 40 3 363.80 27 3 164.30 40 10 40.80 73 6 94.30 64 6 230.90 34 6 366.70 9 6 300.60 8 10 237.80 23 10 121.40 63 3 31.40 65 10 203.50 41 6 441.10 21 3 323.00 38 3 52.50 58 10 1999 Prentce-Hall, Inc. Chap. 14-5
Sample Regresson Model: Example Ŷ b b X b X 0 1 1 2 2 Excel Output b p X C o e ffc e n ts I n te r c e p t 5 6 2. 1 5 1 0 0 9 2 X V a r a b l e 1-5. 4 3 6 5 8 0 5 8 8 X V a r a b l e 2-2 0. 0 1 2 3 2 0 6 7 p Ŷ 562. 151 5. 437X 20. 012X 1 2 For each degree ncrease n temperature, the average amount of For each ncrease n one nch heatng ol used s decreased by of nsulaton, the use of heatng 5.437 gallons, holdng nsulaton ol s decreased by 20.012 constant. gallons, holdng temperature constant. 1999 Prentce-Hall, Inc. Chap. 14-6
Usng The Model to Make Predctons Estmate the average amount of heatng ol used for a home f the average temperature s 30 0 and the nsulaton s 6 nches. Ŷ 562. 151 5. 437X 20. 012X 1 2 562. 151 5. 437 30 20. 012 6 278. 969 The estmated heatng ol used s 278.97 gallons 1999 Prentce-Hall, Inc. Chap. 14-7
Coeffcent of Multple Determnaton Excel Output R eg resso n S tatstcs M u lt p le R 0. 9 8 2 6 5 4 7 5 7 R S q u a re 0. 9 6 5 6 1 0 3 7 1 A d ju s t e d R S q u a re 0. 9 5 9 8 7 8 7 6 6 S t a n d a rd E rro r 2 6. 0 1 3 7 8 3 2 3 O b s e rva t o n s 15 2 r Y, 12 Adjusted r 2 SSR SST reflects the number of explanatory varables and sample sze s smaller than r 2 1999 Prentce-Hall, Inc. Chap. 14-8
Resdual Plots Resduals Vs Y May need to transform Y varable Resduals Vs X 1 May need to transform X 1 varable Resduals Vs X 2 May need to transform X 2 varable Resduals Vs Tme May have autocorrelaton 1999 Prentce-Hall, Inc. Chap. 14-9
Resduals Resdual Plots: Example 60 Tem perature R esdual P lot Excel Output 40 20 Insulaton R esdual P lot 0-2 0 0 20 40 60 80-4 0-6 0 No Dscernable Pattern 0 2 4 6 8 10 12 1999 Prentce-Hall, Inc. Chap. 14-10
Testng for Overall Sgnfcance Shows f there s a lnear relatonshp between all of the X varables together and Y Use F test Statstc Hypotheses: H 0 : 1 = 2 = = p = 0 (No lnear relatonshp) H 1 : At least one 0 ( At least one ndependent varable affects Y) 1999 Prentce-Hall, Inc. Chap. 14-11
Test for Overall Sgnfcance Excel Output: Example A N O V A df SS MS F S gnfcance F R e g re sso n 2 2 2 8 0 1 4.6 1 1 4 0 0 7.3 1 6 8.4 7 1 2 0 2 8 1.6 5 4 1 1 E -0 9 R e sd u a l 12 8 1 2 0.6 0 3 6 7 6.7 1 6 9 T o ta l 14 2 3 6 1 3 5.2 p = 2, the number of explanatory varables n - 1 p value MRS MSE = F Test Statstc 1999 Prentce-Hall, Inc. Chap. 14-12
Test for Overall Sgnfcance Example Soluton H 0 : 1 = 2 = = p = 0 H 1 : At least one I 0 a =.05 df = 2 and 12 Crtcal Value(s): a = 0.05 0 3.89 F Test Statstc: F 168.47 (Excel Output) Decson: Reject at a = 0.05 Concluson: There s evdence that At least one ndependent varable affects Y 1999 Prentce-Hall, Inc. Chap. 14-13
Test for Sgnfcance: Indvdual Varables Shows f there s a lnear relatonshp between the varable X and Y Use t test Statstc Hypotheses: H 0 : = 0 (No lnear relatonshp) H 1 : 0 (Lnear relatonshp between X and Y) 1999 Prentce-Hall, Inc. Chap. 14-14
t Test Statstc Excel Output: Example C o e ffc e n ts S ta n d a rd E rro r t Test Statstc for X 1 (Temperature) t S ta t I n te r c e p t 5 6 2. 1 5 1 0 0 9 2 1. 0 9 3 1 0 4 3 3 2 6. 6 5 0 9 4 X V a r a b l e 1-5. 4 3 6 5 8 0 6 0. 3 3 6 2 1 6 1 6 7-1 6. 1 6 9 9 X V a r a b l e 2-2 0. 0 1 2 3 2 1 2. 3 4 2 5 0 5 2 2 7-8. 5 4 3 1 3 t Test Statstc for X 2 (Insulaton) 1999 Prentce-Hall, Inc. Chap. 14-15
t Test : Example Soluton Does temperature have a sgnfcant effect on monthly consumpton of heatng ol? Test at a = 0.05. H 0 : 1 = 0 H 1 : 1 0 df = 12 Crtcal Value(s): Reject H 0 Reject H 0.025.025-2.1788 0 2.1788 Z Test Statstc: t Test Statstc = -16.1699 Decson: Reject H 0 at a = 0.05 Concluson: There s evdence of a sgnfcant effect of temperature on ol consumpton. 1999 Prentce-Hall, Inc. Chap. 14-16
Confdence Interval Estmate For The Slope Provde the 95% confdence nterval for the populaton slope 1 (the effect of temperature on ol consumpton). b t n S 1 p1 b 1 Coeffcents Lower 95% Upper 95% Inte rce pt 562.151009 516.1930837 608.108935 X V a ra ble 1-5.4365806-6.169132673-4.7040285 X V a ra ble 2-20.012321-25.11620102-14.90844-6.169 1-4.704 The average consumpton of ol s reduced by between 4.7 gallons to 6.17 gallons per each ncrease of 1 0 F. 1999 Prentce-Hall, Inc. Chap. 14-17
Testng Portons of Model Contrbuton of One X to Model (holdng all others constant) Denote by SSR(X all varables except ) 2 r Y 1.2 = Coeffcent of partal determnaton of X 1 wth Y holdng X 2 constant Evaluate Separate Models Useful n Selectng Independent Varables 1999 Prentce-Hall, Inc. Chap. 14-18
Testng Portons of Model: SSR Contrbuton of X 1 gven X 2 has been ncluded: SSR(X 1 X 2 ) = SSR(X 1 and X 2 ) - SSR(X 2 ) From ANOVA secton of regresson for Ŷ b0 b1 X1 b2x 2 From ANOVA secton of regresson for Ŷ b0 b2x 2 1999 Prentce-Hall, Inc. Chap. 14-19
Partal F Test For Contrbuton of X Hypotheses: H 0 : Varable X does not sgnfcantly mprove the model gven all others ncluded H 1 : Varable X sgnfcantly mproves the model gven all others ncluded Test Statstc: F = Wth df = 1 and (n - p -1) SSR( X all others ) MSE 1999 Prentce-Hall, Inc. Chap. 14-20
Coeffcent of Partal Determnaton 2 SSR( X1 X 2 ) Y 1 2 SST SSR( X1 and X 2 ) SSR( X1 X 2 r. ) From ANOVA secton of regresson for Ŷ b0 b1 X1 b2x 2 From ANOVA secton of regresson for Ŷ b0 b2x 2 1999 Prentce-Hall, Inc. Chap. 14-21
Testng Portons of Model: Example Test at the a =.05 level to determne f the varable of average temperature sgnfcantly mproves the model gven that nsulaton s ncluded. 1999 Prentce-Hall, Inc. Chap. 14-22
Testng Portons of Model: Example H 0 : X 1 does not mprove model (X 2 ncluded) H 1 : X 1 does mprove model A N O V A F SSR( a =.05, df = 1 and 12 Crtcal Value = 4.75 (For X 1 and X 2 ) A N O V A (For X 2 ) SS SS X 1 MSE X 2 ) MS R e g re sso n 2 2 8 0 1 4.6 2 6 3 1 1 4 0 0 7.3 1 3 R e sd u a l 8 1 2 0.6 0 3 0 1 6 6 7 6.7 1 6 9 1 8 T o ta l 2 3 6 1 3 5.2 2 9 3 R e g r e s s o n 5 1 0 7 6. 4 7 R e s d u a l 1 8 5 0 5 8. 8 T o t a l 2 3 6 1 3 5. 2 228, 015 51, 076 676, 717 Concluson: Reject H 0. X 1 does mprove model = 261.47 1999 Prentce-Hall, Inc. Chap. 14-23
Curvlnear Regresson Model Relatonshp between 1 response varable and 2 or more explanatory varable s a polynomal functon Useful when scatter dagram ndcates non-lnear relatonshp Curvlnear model: Y 0 X 1 The second explanatory varable s the square of the 1st. 1999 Prentce-Hall, Inc. Chap. 14-24 1 X 2 2 1
Curvlnear Regresson Model Curvlnear models may be consdered when scatter dagram takes on the followng shapes: Y Y Y Y X 1 X 1 2 > 0 2 > 0 2 < 0 2 < 0 X 1 X 1 2 = the coeffcent of the quadratc term 1999 Prentce-Hall, Inc. Chap. 14-25
Testng for Sgnfcance: Curvlnear Model Testng for Overall Relatonshp Smlar to test for lnear model F test statstc = MSR MSE Testng the Curvlnear Effect Compare curvlnear model 2 Y X X 0 1 wth the lnear model Y X 1 0 1 1 2 1 1999 Prentce-Hall, Inc. Chap. 14-26
Dummy-Varable Models Categorcal Varable Involved (dummy varable) wth 2 Levels: yes or no, on or off, male or female, Coded 0 or 1 Intercepts Dfferent Assumes Equal Slopes Regresson Model has Same Form: Y X X X 0 1 1 2 2 p p 1999 Prentce-Hall, Inc. Chap. 14-27
Dummy-Varable Models Assumpton Gven: Ŷ b0 b1x 1 b2x 2 Y = Assessed Value of House X 1 = Square footage of House X 2 = Desrablty of Neghborhood = Desrable (X 2 = 1) Ŷ b0 b1x 1 b2( 1) ( b0 b2 ) b1x 1 Undesrable (X 2 = 0) Ŷ b0 b1x 1 b2( 0) b0 b1x 1 0 f undesrable 1 f desrable Same slopes 1999 Prentce-Hall, Inc. Chap. 14-28
Dummy-Varable Models Assumpton Y (Assessed Value) Intercepts dfferent b 0 + b 2 b 0 Same slopes X 1 (Square footage) 1999 Prentce-Hall, Inc. Chap. 14-29
Evaluatng Presence of Interacton Hypothesze Interacton Between Pars of Independent Varables Contans 2-way Product Terms Y 0 1X1 2X 2 3X1 X 2 Hypotheses: H 0 : 3 = 0 (No nteracton between X 1 and X 2 H 1 : 3 0 (X 1 nteracts wth X 2 ) 1999 Prentce-Hall, Inc. Chap. 14-30
Usng Transformatons For Non-lnear Models that Volate Lnear Regresson Assumptons Determne Type of Transformaton From Scatter Dagram Requres Data Transformaton Ether or Both Independent and Dependent Varables May be Transformed 1999 Prentce-Hall, Inc. Chap. 14-31
Square Root Transformaton Y X X 0 1 1 2 2 Y 1 > 0 1 < 0 X 1 Smlarly for X 2 Transforms one of above model to one that appears lnear. Often used to overcome heteroscedastcty. 1999 Prentce-Hall, Inc. Chap. 14-32
Logarthmc Transformaton Y ln( X ) 0 1 1 2 ln( X 2 ) Y 1 > 0 Smlarly for X 2 1 < 0 X 1 Transformed from an orgnal multplcatve model 1999 Prentce-Hall, Inc. Chap. 14-33
Exponental Transformaton Orgnal Model Y e 0 1X 1 2X 2 Y 1 > 0 Smlarly for X 2 1 < 0 X 1 Transformed nto: lny 0 1X1 2X 2 ln 1 1999 Prentce-Hall, Inc. Chap. 14-34
Collnearty Hgh Correlaton Between Explanatory Varables Coeffcents Measure Combned Effect No New Informaton Provded Leads to Unstable Coeffcents Dependng on the explanatory varables VIF Used to Measure Collnearty VIF 1 j 1 R 2 j, 1999 Prentce-Hall, Inc. Chap. 14-35 2 j R = Coeffcent of Multple Determnaton of X j wth all the others
Model Buldng Goal s to Develop Model wth Fewest Explanatory Varables Easer to nterpret Lower probablty of collnearty Stepwse Regresson Procedure Provde lmted evaluaton of alternatve models Best-Subset Approach Uses the C p Statstc Selects model wth small C p near p+1 1999 Prentce-Hall, Inc. Chap. 14-36
Model Buldng Flowchart Choose X 1,X 2, X k Run Regresson to fnd VIFs Any VIF>5? No Run Subsets Regresson to Obtan best models n terms of C p Remove Varable wth Hghest VIF Yes Yes More than One? No Remove ths X Do Complete Analyss Add Curvlnear Term and/or Transform Varables as Indcated Perform Predctons 1999 Prentce-Hall, Inc. Chap. 14-37
Chapter Summary Presented The Multple Regresson Model Consdered Contrbuton of Indvdual Independent Varables Dscussed Coeffcent of Determnaton Addressed Categorcal Explanatory Varables Consdered Transformaton of Varables Dscussed Model Buldng 1999 Prentce-Hall, Inc. Chap. 14-38