Chapter 14 Student Lecture Notes Department of Quantitative Methods & Information Systems. Business Statistics. Chapter 14 Multiple Regression

Transcription

1 Chapter 14 Student Lecture Notes 14-1 Department of Quantitative Methods & Information Systems Business Statistics Chapter 14 Multiple Regression QMIS 0 Dr. Mohammad Zainal Chapter Goals After completing this chapter, you should be able to: Apply multiple regression analysis to business decisionmaking situations Analyze and interpret the computer output for a multiple regression model Perform a hypothesis test for all regression coefficients Chap 14-

2 Chapter 14 Student Lecture Notes 14- The Multiple Regression Model Idea: Examine the linear relationship between 1 dependent (Y) & or more independent variables (X i ) Multiple Regression Model with k Independent Variables: Y-intercept Population slopes Random Error Y β0 β1x1 βx βkxk ε Chap 14-3 Multiple Regression Equation The coefficients of the multiple regression model are estimated using sample data Multiple regression equation with k independent variables: Estimated (or predicted) value of y yˆ i b 0 Estimated intercept b x 1 1i Estimated slope coefficients b x i b k x ki Chap 14-4

3 Chapter 14 Student Lecture Notes 14-3 Multiple Regression Equation Two variable model y yˆ b 0 b1x1 bx x x 1 Chap 14-5 Multiple Regression Model Two variable model y i y Sample observation yˆ b 0 b1x1 bx y i < e i = (y i y i ) < x i x x 1i The best fit equation, y, < x 1 is found by minimizing the sum of squared errors, e Chap 14-6

4 Chapter 14 Student Lecture Notes 14-4 Standard Multiple Regression Assumptions The values x i and the error terms ε i are independent The error terms are random variables with mean 0 and a constant variance,. E[ε ] 0 i and E[ε ] σ i for (i 1,,n) (The constant variance property is called homoscedasticity) Chap 14-7 Standard Multiple Regression Assumptions The random error terms, ε i, are not correlated with one another, so that E[ε i ε j ] 0 for all i j It is not possible to find a set of numbers, c 0, c 1,..., c k, such that c0 c1x1i cxi ckxki 0 (This is the property of no linear relation for the X j s) Chap 14-8

5 Chapter 14 Student Lecture Notes 14-5 Example: Independent Variables A distributor of frozen desert pies wants to evaluate factors thought to influence demand Dependent variable: Pie sales (units per week) Independent variables: Price (in $) Data are collected for 15 weeks Advertising ($100 s) Chap 14-9 Pie Sales Example Week Pie Sales Price ($) Advertising ($100s) Multiple regression equation: Sales = b 0 + b 1 (Price) + b (Advertising) QMIS 0, 15 by Dr. M. 300 Zainal Chap 14-10

6 Chapter 14 Student Lecture Notes 14-6 Estimating a Multiple Linear Regression Equation Excel will be used to generate the coefficients and measures of goodness of fit for multiple regression Data / Data Analysis / Regression Chap Multiple Regression Output Regression Statistics Multiple R R Square Adjusted R Square Standard Error Observations 15 ANOVA df SS MS F Significance F Regression Residual Total Coefficients Standard Error t Stat P-value Lower 95% Upper 95% Intercept Price Advertising Chap 14-1

7 Chapter 14 Student Lecture Notes 14-7 The Multiple Regression Equation Sales (Price) (Advertising) where Sales is in number of pies per week Price is in $ Advertising is in $100 s. b 1 = : sales will decrease, on average, by pies per week for each $1 increase in selling price, net of the effects of changes due to advertising b = : sales will increase, on average, by pies per week for each $100 increase in advertising, net of the effects of changes due to price Chap Coefficient of Determination, R Reports the proportion of total variation in y explained by all x variables taken together R SSR SST regressionsumof squares total sumof squares This is the ratio of the explained variability to total sample variability Chap 14-14

8 Chapter 14 Student Lecture Notes 14-8 Coefficient of Determination, R Regression Statistics Multiple R R Square Adjusted R Square Standard Error Observations 15 ANOVA df SS MS F Significance F Regression Residual Total Coefficients Standard Error t Stat P-value Lower 95% Upper 95% Intercept Price Advertising Chap Estimation of Error Variance Consider the population regression model Y β i 0 β x 1 1i β The unbiased estimate of the variance of the errors is x i β K x Ki ε i where e i s e i n ei i 1 SSE n K 1 n K 1 y yˆ i The square root of the variance, s e, is called the standard error of the estimate Chap 14-16

9 Chapter 14 Student Lecture Notes 14-9 Standard Error, s e Regression Statistics Multiple R R Square Adjusted R Square Standard Error Observations 15 ANOVA df SS MS F Significance F Regression Residual Total Coefficients Standard Error t Stat P-value Lower 95% Upper 95% Intercept Price Advertising Chap Adjusted Coefficient of Determination, R never decreases when a new X variable is added to the model, even if the new variable is not an important predictor variable This can be a disadvantage when comparing models What is the net effect of adding a new variable? We lose a degree of freedom when a new X variable is added Did the new X variable add enough explanatory power to offset the loss of one degree of freedom? R Chap 14-18

10 Chapter 14 Student Lecture Notes Adjusted Coefficient of Determination, Used to correct for the fact that adding non-relevant independent variables will still reduce the error sum of squares SSE /(n K 1) R 1 SST /(n 1) (where n = sample size, K = number of independent variables) Adjusted R provides a better comparison between multiple regression models with different numbers of independent variables Penalize excessive use of unimportant independent variables Smaller than R Chap R R Regression Statistics Multiple R R Square Adjusted R Square Standard Error Observations 15 ANOVA df SS MS F Significance F Regression Residual Total Coefficients Standard Error t Stat P-value Lower 95% Upper 95% Intercept Price Advertising Chap 14-0

11 Chapter 14 Student Lecture Notes Evaluating Individual Regression Coefficients Use t-tests for individual coefficients Shows if a specific independent variable is conditionally important Hypotheses: H 0 : β j = 0 (no linear relationship) H 1 : β j 0 (linear relationship does exist between x j and y) Chap 14-1 Evaluating Individual Regression Coefficients H 0 : β j = 0 (no linear relationship) H 1 : β j 0 (linear relationship does exist between x i and y) Test Statistic: t b j S 0 b j (df = n k 1) Chap 14-

12 Chapter 14 Student Lecture Notes 14-1 Evaluating Individual Regression Coefficients Regression Statistics Multiple R R Square Adjusted R Square Standard Error Observations 15 ANOVA df SS MS F Significance F Regression Residual Total Coefficients Standard Error t Stat P-value Lower 95% Upper 95% Intercept Price Advertising Chap 14-3 Example: Evaluating Individual Regression Coefficients H 0 : β j = 0 H 1 : β j 0 From Excel output: Coefficients Standard Error t Stat P-value Price Advertising a/=.05 a/=.05 Decision: Conclusion: Reject H 0 Do not reject H -t 0 α/ t α/ 0 Reject H 0 Chap 14-4

13 Chapter 14 Student Lecture Notes Confidence Interval Estimate for the Slope Confidence interval limits for the population slope β j b j t n K 1,α/ S b j where t has (n K 1) d.f. Coefficients Standard Error Intercept Price Advertising Here, t has (15 1) = 1 d.f. Example: Form a 95% confidence interval for the effect of changes in price (x 1 ) on pie sales: ± (.1788)(10.83) So the interval is < β 1 < Chap 14-5 Confidence Interval Estimate for the Slope Confidence interval for the population slope β i Coefficients Standard Error Lower 95% Upper 95% Intercept Price Advertising Example: Excel output also reports these interval endpoints: Weekly sales are estimated to be reduced by between 1.37 to pies for each increase of $1 in the selling price Chap 14-6

14 Chapter 14 Student Lecture Notes Test on All Coefficients F-Test for Overall Significance of the Model Shows if there is a linear relationship between all of the X variables considered together and Y Use F test statistic Hypotheses: H 0 : β 1 = β = = β k = 0 (no linear relationship) H 1 : at least one β i 0 (at least one independent variable affects Y) Chap 14-7 F-Test for Overall Significance Test statistic: MSR s F e SSR/K SSE/(n K 1) where F has k (numerator) and (n K 1) (denominator) degrees of freedom The decision rule is Reject H if F 0 F k,n K 1,α Chap 14-8

15 Chapter 14 Student Lecture Notes F-Test for Overall Significance Regression Statistics Multiple R R Square Adjusted R Square Standard Error Observations 15 ANOVA df SS MS F Significance F Regression Residual Total Coefficients Standard Error t Stat P-value Lower 95% Upper 95% Intercept Price Advertising Chap 14-9 H 0 : β 1 = β = 0 H 1 : β 1 and β not both zero a =.05 df 1 = df = 1 F-Test for Overall Significance Test Statistic: MSR F MSE Decision: 0 a =.05 F Conclusion: Chap 14-30

16 Chapter 14 Student Lecture Notes Copyright The materials of this presentation were mostly taken from the PowerPoint files accompanied Business Statistics: A Decision-Making Approach, 7e 008 Prentice-Hall, Inc.