Chapter 3 Studet Lecture Notes 3- Basc Busess Statstcs (9 th Edto) Chapter 3 Smple Lear Regresso 4 Pretce-Hall, Ic. Chap 3- Chapter Topcs Types of Regresso Models Determg the Smple Lear Regresso Equato Measures of Varato Assumptos of Regresso ad Correlato Resdual Aalyss Measurg Autocorrelato Ifereces about the Slope 4 Pretce-Hall, Ic. Chap 3- Chapter Topcs (cotued) Correlato - Measurg the Stregth of the Assocato Estmato of Mea Values ad Predcto of Idvdual Values Ptfalls Regresso ad Ethcal Issues 4 Pretce-Hall, Ic. Chap 3-3 Statstcs for Maagers Usg Mcrosoft Excel, /e 999 Pretce-Hall, Ic.
Chapter 3 Studet Lecture Notes 3- Purpose of Regresso Aalyss Regresso Aalyss s Used Prmarly to Model Causalty ad Provde Predcto Predct the values of a depedet (respose) varable based o values of at least oe depedet (explaatory) varable Expla the effect of the depedet varables o the depedet varable 4 Pretce-Hall, Ic. Chap 3-4 Types of Regresso Models Postve Lear Relatoshp Relatoshp NOT Lear Negatve Lear Relatoshp No Relatoshp 4 Pretce-Hall, Ic. Chap 3-5 Smple Lear Regresso Model Relatoshp betwee Varables s Descrbed by a Lear Fucto The Chage of Oe Varable Causes the Other Varable to Chage A Depedecy of Oe Varable o the Other 4 Pretce-Hall, Ic. Chap 3-6 Statstcs for Maagers Usg Mcrosoft Excel, /e 999 Pretce-Hall, Ic.
Chapter 3 Studet Lecture Notes 3-3 Smple Lear Regresso Model Populato regresso le s a straght le that descrbes the depedece of the average value (codtoal mea) of oe varable o the other Populato Itercept Depedet (Respose) Varable Populato Regresso µ Le (Codtoal Mea) Populato Slope Coeffcet = β + β + ε Radom Error Idepedet (Explaatory) Varable 4 Pretce-Hall, Ic. Chap 3-7 (cotued) Smple Lear Regresso Model (cotued) (Observed Value of ) = = β + β + ε ε = Radom Error β β Observed Value of µ = β + β (Codtoal Mea) 4 Pretce-Hall, Ic. Chap 3-8 Lear Regresso Equato Sample regresso le provdes a estmate of the populato regresso le as well as a predcted value of Sample Itercept = b + b + e ˆ = b + b = Sample Slope Coeffcet Resdual Smple Regresso Equato (Ftted Regresso Le, Predcted Value) 4 Pretce-Hall, Ic. Chap 3-9 Statstcs for Maagers Usg Mcrosoft Excel, /e 999 Pretce-Hall, Ic.
Chapter 3 Studet Lecture Notes 3-4 Lear Regresso Equato b b ad are obtaed by fdg the values of b ad that mmze the sum of the b squared resduals ( ) ˆ = e = = b β b β provdes a estmate of provdes a estmate of (cotued) 4 Pretce-Hall, Ic. Chap 3- β Lear Regresso Equato (cotued) = β + β + ε = b + b + e b e ε Observed Value µ = β + β = b + b 4 Pretce-Hall, Ic. Chap 3- ˆ b β Iterpretato of the Slope ad Itercept β = µ = s the average value of whe the value of s zero chage µ β = chage measures the chage the average value of as a result of a oeut chage 4 Pretce-Hall, Ic. Chap 3- Statstcs for Maagers Usg Mcrosoft Excel, /e 999 Pretce-Hall, Ic.
Chapter 3 Studet Lecture Notes 3-5 Iterpretato of the Slope ad Itercept ( ) (cotued) b = ˆ = s the estmated average value of whe the value of s zero chage ˆ b = s the estmated chage chage the average value of as a result of a oeut chage 4 Pretce-Hall, Ic. Chap 3-3 Smple Lear Regresso: Example ou wsh to exame the lear depedecy of the aual sales of produce stores o ther szes square footage. Sample data for 7 stores were obtaed. Fd the equato of the straght le that fts the data best. Aual Store Square Sales Feet ($),76 3,68,54 3,395 3,86 6,653 4 5,555 9,543 5,9 3,38 6,8 5,563 7,33 3,76 4 Pretce-Hall, Ic. Chap 3-4 Scatter Dagram: Example Aual Sales ($) 8 6 4 3 4 5 6 Square Feet Excel Output 4 Pretce-Hall, Ic. Chap 3-5 Statstcs for Maagers Usg Mcrosoft Excel, /e 999 Pretce-Hall, Ic.
Chapter 3 Studet Lecture Notes 3-6 Smple Lear Regresso Equato: Example ˆ = b+ b = 636.45+.487 From Excel Prtout: Coeffcets Itercept 636.4476 Varable.486633657 4 Pretce-Hall, Ic. Chap 3-6 Graph of the Smple Lear Regresso Equato: Example Aual Sales ($) 8 6 4 = 636.45 +.487 3 4 5 6 Square Feet 4 Pretce-Hall, Ic. Chap 3-7 Iterpretato of Results: Example ˆ = 636.45+.487 The slope of.487 meas that for each crease of oe ut, we predct the average of to crease by a estmated.487 uts. The equato estmates that for each crease of square foot the sze of the store, the expected aual sales are predcted to crease by $487. 4 Pretce-Hall, Ic. Chap 3-8 Statstcs for Maagers Usg Mcrosoft Excel, /e 999 Pretce-Hall, Ic.
Chapter 3 Studet Lecture Notes 3-7 Smple Lear Regresso PHStat I Excel, use PHStat Regresso Smple Lear Regresso Excel Spreadsheet of Regresso Sales o Footage Mcrosoft Excel Worksheet 4 Pretce-Hall, Ic. Chap 3-9 Measures of Varato: The Sum of Squares SST = SSR + SSE Total = Sample Explaed Varablty Varablty + Uexplaed Varablty 4 Pretce-Hall, Ic. Chap 3- Measures of Varato: The Sum of Squares (cotued) SST = Total Sum of Squares Measures the varato of the values aroud ther mea, SSR = Regresso Sum of Squares Explaed varato attrbutable to the relatoshp betwee ad SSE = Error Sum of Squares Varato attrbutable to factors other tha the relatoshp betwee ad 4 Pretce-Hall, Ic. Chap 3- Statstcs for Maagers Usg Mcrosoft Excel, /e 999 Pretce-Hall, Ic.
Chapter 3 Studet Lecture Notes 3-8 Measures of Varato: The Sum of Squares (cotued) SSE = ( - _ ) SST = ( -) _ SSR = ( -) _ 4 Pretce-Hall, Ic. Chap 3- Ve Dagrams ad Explaatory Power of Regresso Varatos store Szes ot used explag varato Sales Szes Sales Varatos Sales explaed by the error term or uexplaed by Szes SSE ( ) Varatos Sales explaed by Szes or varatos Szes used explag varato Sales SSR ( ) 4 Pretce-Hall, Ic. Chap 3-3 The ANOVA Table Excel ANOVA df SS MS F Sgfcace F Regresso Error k -k- SSR SSE MSR =SSR/k MSE =SSE/(-k-) MSR/MSE P-value of the F Test Total - SST 4 Pretce-Hall, Ic. Chap 3-4 Statstcs for Maagers Usg Mcrosoft Excel, /e 999 Pretce-Hall, Ic.
Chapter 3 Studet Lecture Notes 3-9 Measures of Varato The Sum of Squares: Example Excel Output for Produce Stores Degrees of freedom ANOVA df SS MS F Sgfcace F Regresso 338456. 338456. 8.799.8 Error 5 8799.595 37439.99 Total 6 35655.7 Regresso (explaed) df Error (uexplaed) df Total df SST SSE SSR 4 Pretce-Hall, Ic. Chap 3-5 The Coeffcet of Determato SSR Regresso Sum of Squares r = = SST Total Sum of Squares Measures the proporto of varato that s explaed by the depedet varable the regresso model 4 Pretce-Hall, Ic. Chap 3-6 Ve Dagrams ad Explaatory Power of Regresso Sales r = Szes SSR = SSR + SSE 4 Pretce-Hall, Ic. Chap 3-7 Statstcs for Maagers Usg Mcrosoft Excel, /e 999 Pretce-Hall, Ic.
Chapter 3 Studet Lecture Notes 3- Coeffcets of Determato (r ) ad Correlato (r) r =, r = + r =, r = - ^ ^ = b + b = b + b r =.8, r = +.9 r =, r = ^ = b + b ^ = b + b 4 Pretce-Hall, Ic. Chap 3-8 Stadard Error of Estmate S ( ˆ ) SSE = = = Measures the stadard devato (varato) of the values aroud the regresso equato 4 Pretce-Hall, Ic. Chap 3-9 r =.94 Measures of Varato: Produce Store Example Excel Output for Produce Stores Regresso Statstcs Multple R.97557 R Square.94989 Adjusted R Square.9337754 Stadard Error 6.7557 Observatos 7 94% of the varato aual sales ca be explaed by the varablty the sze of the store as measured by square footage. S yx 4 Pretce-Hall, Ic. Chap 3-3 Statstcs for Maagers Usg Mcrosoft Excel, /e 999 Pretce-Hall, Ic.
Chapter 3 Studet Lecture Notes 3- Lear Regresso Assumptos Normalty values are ormally dstrbuted for each Probablty dstrbuto of error s ormal Homoscedastcty (Costat Varace) Idepedece of Errors 4 Pretce-Hall, Ic. Chap 3-3 Cosequeces of Volato of the Assumptos Volato of the Assumptos No-ormalty (error ot ormally dstrbuted) Heteroscedastcty (varace ot costat) Usually happes cross-sectoal data Autocorrelato (errors are ot depedet) Usually happes tme-seres data Cosequeces of Ay Volato of the Assumptos Predctos ad estmatos obtaed from the sample regresso le wll ot be accurate Hypothess testg results wll ot be relable It s Importat to Verfy the Assumptos 4 Pretce-Hall, Ic. Chap 3-3 Varato of Errors Aroud the Regresso Le f(e) values are ormally dstrbuted aroud the regresso le. For each value, the spread or varace aroud the regresso le s the same. Sample Regresso Le 4 Pretce-Hall, Ic. Chap 3-33 Statstcs for Maagers Usg Mcrosoft Excel, /e 999 Pretce-Hall, Ic.
Chapter 3 Studet Lecture Notes 3- Resdual Aalyss Purposes Exame learty Evaluate volatos of assumptos Graphcal Aalyss of Resduals Plot resduals vs. ad tme 4 Pretce-Hall, Ic. Chap 3-34 Resdual Aalyss for Learty e e Not Lear Lear 4 Pretce-Hall, Ic. Chap 3-35 Resdual Aalyss for Homoscedastcty SR SR Heteroscedastcty Homoscedastcty 4 Pretce-Hall, Ic. Chap 3-36 Statstcs for Maagers Usg Mcrosoft Excel, /e 999 Pretce-Hall, Ic.
Chapter 3 Studet Lecture Notes 3-3 Resdual Aalyss: Excel Output for Produce Stores Example Excel Output Resdual Plot Observato Predcted Resduals 4.34447-5.344473 398.8384-533.83845 3 58.7753 83.4897 4 9894.664688-35.664688 5 3557.454-39.4543 6 498.984 644.9863 7 3588.36477 7.63589 3 4 5 6 Square Feet 4 Pretce-Hall, Ic. Chap 3-37 Resdual Aalyss for Idepedece The Durb-Watso Statstc Used whe data are collected over tme to detect autocorrelato (resduals oe tme perod are related to resduals aother perod) Measures volato of depedece assumpto ( e e ) Should be close to. = D = If ot, exame the model e for autocorrelato. = 4 Pretce-Hall, Ic. Chap 3-38 Durb-Watso Statstc PHStat PHStat Regresso Smple Lear Regresso Check the box for Durb-Watso Statstc 4 Pretce-Hall, Ic. Chap 3-39 Statstcs for Maagers Usg Mcrosoft Excel, /e 999 Pretce-Hall, Ic.
Chapter 3 Studet Lecture Notes 3-4 Obtag the Crtcal Values of Durb-Watso Statstc Table 3.4 Fdg Crtcal Values of Durb-Watso Statstc α =. 5 k= k= d L d U d L d U 5.8.36.95.54 6..37.98.54 4 Pretce-Hall, Ic. Chap 3-4 Usg the Durb-Watso Statstc H : No autocorrelato (error terms are depedet) H : There s autocorrelato (error terms are ot depedet) Reject H (postve autocorrelato) Icoclusve Reject H Do ot reject H (o autocorrelato) (egatve autocorrelato) d L d U 4-d U 4-d L 4 4 Pretce-Hall, Ic. Chap 3-4 Resdual Aalyss for Idepedece Not Idepedet e Graphcal Approach Tme e Idepedet Tme Cyclcal Patter No Partcular Patter Resdual s Plotted Agast Tme to Detect Ay Autocorrelato 4 Pretce-Hall, Ic. Chap 3-4 Statstcs for Maagers Usg Mcrosoft Excel, /e 999 Pretce-Hall, Ic.
Chapter 3 Studet Lecture Notes 3-5 Iferece about the Slope: t Test t Test for a Populato Slope Is there a lear relatoshp betwee ad? Null ad Alteratve Hypotheses H : β = (o lear relatoshp) H : β (lear relatoshp) Test Statstc b β t = where Sb = Sb df.. = = ( ) 4 Pretce-Hall, Ic. Chap 3-43 S Example: Produce Store Data for 7 Stores: Aual Store Square Sales Feet ($),76 3,68,54 3,395 3,86 6,653 4 5,555 9,543 5,9 3,38 6,8 5,563 7,33 3,76 Estmated Regresso Equato: ˆ = 636.45 +.487 The slope of ths model s.487. Are square footage ad aual sales learly related? 4 Pretce-Hall, Ic. Chap 3-44 H : β = H : β α =.5 df = 7 - = 5 Crtcal Value(s): Reject.5 -.576 Ifereces about the Slope: t Test Example Reject.576 Test Statstc: From Excel Prtout S Coeffcets Stadard Error t Stat P-value Itercept 636.447 45.4953 3.644.55 Footage.4866.65 9.99.8.5 t b b Decso: Reject H. p-value Cocluso: There s evdece that square footage s learly related to aual sales. 4 Pretce-Hall, Ic. Chap 3-45 t Statstcs for Maagers Usg Mcrosoft Excel, /e 999 Pretce-Hall, Ic.
Chapter 3 Studet Lecture Notes 3-6 Ifereces about the Slope: Cofdece Iterval Example Cofdece Iterval Estmate of the Slope: b ± t S Excel Prtout for Produce Stores Lower 95% Upper 95% Itercept 475.896 797.853 Footage.64937.977694 At 95% level of cofdece, the cofdece terval for the slope s (.6,.9). Does ot clude. Cocluso: There s a sgfcat lear relatoshp betwee aual sales ad the sze of the store. 4 Pretce-Hall, Ic. Chap 3-46 b Ifereces about the Slope: F Test F Test for a Populato Slope Is there a lear relatoshp betwee ad? Null ad Alteratve Hypotheses H : β = (o lear relatoshp) H : β (lear relatoshp) Test Statstc SSR F = SSE ( ) Numerator d.f.=, deomator d.f.=- 4 Pretce-Hall, Ic. Chap 3-47 Relatoshp betwee a t Test ad a F Test Null ad Alteratve Hypotheses H : β = (o lear relatoshp) H : β (lear relatoshp) ( t ) = F, The p value of a t Test ad the p value of a F Test are Exactly the Same The Rejecto Rego of a F Test s Always the Upper Tal 4 Pretce-Hall, Ic. Chap 3-48 Statstcs for Maagers Usg Mcrosoft Excel, /e 999 Pretce-Hall, Ic.
Chapter 3 Studet Lecture Notes 3-7 H : β = H : β α =.5 umerator df = deomator df = 7 - = 5 6.6 Ifereces about the Slope: F Test Example Test Statstc: From Excel Prtout ANOVA df SS MS F Sgfcace F Regresso 338456. 338456. 8.79.8 Resdual 5 8799.595 37439.99 Total 6 35655.7 Reject α =.5 F, p-value Decso: Reject H. Cocluso: There s evdece that square footage s learly related to aual sales. 4 Pretce-Hall, Ic. Chap 3-49 Purpose of Correlato Aalyss Correlato Aalyss s Used to Measure Stregth of Assocato (Lear Relatoshp) Betwee Numercal Varables Oly stregth of the relatoshp s cocered No causal effect s mpled 4 Pretce-Hall, Ic. Chap 3-5 Purpose of Correlato Aalyss (cotued) Populato Correlato Coeffcet ρ (Rho) s Used to Measure the Stregth betwee the Varables 4 Pretce-Hall, Ic. Chap 3-5 Statstcs for Maagers Usg Mcrosoft Excel, /e 999 Pretce-Hall, Ic.
Chapter 3 Studet Lecture Notes 3-8 Purpose of Correlato Aalyss (cotued) Sample Correlato Coeffcet r s a Estmate of ρ ad s Used to Measure the Stregth of the Lear Relatoshp the Sample Observatos r = ( )( ) = ( ) ( ) = = 4 Pretce-Hall, Ic. Chap 3-5 Sample Observatos from Varous r Values r = - r = -.6 r = r =.6 r = 4 Pretce-Hall, Ic. Chap 3-53 Ut Free Features of ρ ad r Rage betwee - ad The Closer to -, the Stroger the Negatve Lear Relatoshp The Closer to, the Stroger the Postve Lear Relatoshp The Closer to, the Weaker the Lear Relatoshp 4 Pretce-Hall, Ic. Chap 3-54 Statstcs for Maagers Usg Mcrosoft Excel, /e 999 Pretce-Hall, Ic.
Chapter 3 Studet Lecture Notes 3-9 Hypotheses H : ρ = (o correlato) H : ρ (correlato) Test Statstc t r ρ where = r r= r = t Test for Correlato = ( )( ) ( ) ( ) 4 Pretce-Hall, Ic. = = Chap 3-55 Example: Produce Stores Is there ay evdece of lear relatoshp betwee aual sales of a store ad ts square footage at.5 level of sgfcace? From Excel Prtout Regresso Statstcs Multple R.97557 R Square.94989 Adjusted R Square.9337754 Stadard Error 6.7557 Observatos 7 H : ρ = (o assocato) H : ρ (assocato) α =.5 df = 7 - = 5 4 Pretce-Hall, Ic. Chap 3-56 r Example: Produce Stores Soluto r ρ.976 t = = = 9.99 r.94 5 Crtcal Value(s): Reject.5 -.576 Reject.5.576 Decso: Reject H. Cocluso: There s evdece of a lear relatoshp at 5% level of sgfcace. The value of the t statstc s exactly the same as the t statstc value for test o the slope coeffcet. 4 Pretce-Hall, Ic. Chap 3-57 Statstcs for Maagers Usg Mcrosoft Excel, /e 999 Pretce-Hall, Ic.
Chapter 3 Studet Lecture Notes 3- Stadard error of the estmate Estmato of Mea Values Cofdece Iterval Estmate for : The Mea of Gve a Partcular t value from table wth df=- ˆ ± t S + Sze of terval vares accordg to dstace away from mea, ( ) = ( ) µ = 4 Pretce-Hall, Ic. Chap 3-58 Predcto of Idvdual Values Predcto Iterval for Idvdual Respose at a Partcular Addto of creases wdth of terval from that for the mea of ˆ ± t S + + ( ) = 4 Pretce-Hall, Ic. Chap 3-59 ( ) Iterval Estmates for Dfferet Values of Predcto Iterval for a Idvdual Cofdece Iterval for the Mea of = b + b a gve 4 Pretce-Hall, Ic. Chap 3-6 Statstcs for Maagers Usg Mcrosoft Excel, /e 999 Pretce-Hall, Ic.
Chapter 3 Studet Lecture Notes 3- Example: Produce Stores Data for 7 Stores: Aual Store Square Sales Feet ($),76 3,68,54 3,395 3,86 6,653 4 5,555 9,543 5,9 3,38 6,8 5,563 7,33 3,76 Cosder a store wth square feet. Regresso Model Obtaed: = 636.45 +.487 4 Pretce-Hall, Ic. Chap 3-6 Estmato of Mea Values: Example Cofdece Iterval Estmate for Fd the 95% cofdece terval for the average aual sales for stores of, square feet. Predcted Sales = 636.45 +.487 = 46.45 ( $) = 35.9 S = 6.75 t - = t 5 =.576 ˆ ( ) ± t S + = 46.45± 6.66 ( ) = 3997. < < 5.34 µ = µ = 4 Pretce-Hall, Ic. Chap 3-6 Predcto Iterval for : Example Predcto Iterval for Idvdual = Fd the 95% predcto terval for aual sales of oe partcular store of, square feet. Predcted Sales = 636.45 +.487 = 46.45 ( $) = 35.9 S = 6.75 t - = t 5 =.576 ˆ ( ) ± t S + + = 46.45± 687.68 ( ) = = 9. < < 697.37 4 Pretce-Hall, Ic. Chap 3-63 Statstcs for Maagers Usg Mcrosoft Excel, /e 999 Pretce-Hall, Ic.
Chapter 3 Studet Lecture Notes 3- Estmato of Mea Values ad Predcto of Idvdual Values PHStat I Excel, use PHStat Regresso Smple Lear Regresso Check the Cofdece ad Predcto Iterval for = box Excel Spreadsheet of Regresso Sales o Footage Mcrosoft Excel Worksheet 4 Pretce-Hall, Ic. Chap 3-64 Ptfalls of Regresso Aalyss Lackg a Awareess of the Assumptos Uderlg Least-Squares Regresso Not Kowg How to Evaluate the Assumptos Not Kowg What the Alteratves to Least- Squares Regresso are f a Partcular Assumpto s Volated Usg a Regresso Model Wthout Kowledge of the Subject Matter 4 Pretce-Hall, Ic. Chap 3-65 Strategy for Avodg the Ptfalls of Regresso Start wth a scatter plot to observe possble relatoshp betwee o Perform resdual aalyss to check the assumptos Use a hstogram, stem-ad-leaf dsplay, boxad-whsker plot, or ormal probablty plot of the resduals to ucover possble oormalty 4 Pretce-Hall, Ic. Chap 3-66 Statstcs for Maagers Usg Mcrosoft Excel, /e 999 Pretce-Hall, Ic.
Chapter 3 Studet Lecture Notes 3-3 Strategy for Avodg the Ptfalls of Regresso (cotued) If there s volato of ay assumpto, use alteratve methods (e.g., least absolute devato regresso or least meda of squares regresso) to least-squares regresso or alteratve least-squares models (e.g., curvlear or multple regresso) If there s o evdece of assumpto volato, the test for the sgfcace of the regresso coeffcets ad costruct cofdece tervals ad predcto tervals 4 Pretce-Hall, Ic. Chap 3-67 Chapter Summary Itroduced Types of Regresso Models Dscussed Determg the Smple Lear Regresso Equato Descrbed Measures of Varato Addressed Assumptos of Regresso ad Correlato Dscussed Resdual Aalyss Addressed Measurg Autocorrelato 4 Pretce-Hall, Ic. Chap 3-68 Chapter Summary (cotued) Descrbed Iferece about the Slope Dscussed Correlato - Measurg the Stregth of the Assocato Addressed Estmato of Mea Values ad Predcto of Idvdual Values Dscussed Ptfalls Regresso ad Ethcal Issues 4 Pretce-Hall, Ic. Chap 3-69 Statstcs for Maagers Usg Mcrosoft Excel, /e 999 Pretce-Hall, Ic.