Chapter 13 Student Lecture Notes Department of Quantitative Methods & Information Systems. Business Statistics

Transcription

1 Chapter 13 Student Lecture Notes 13-1 Department of Quantitative Methods & Information Sstems Business Statistics Chapter 14 Introduction to Linear Regression and Correlation Analsis QMIS 0 Dr. Mohammad Zainal Chapter Goals After completing this chapter, ou should be able to: Calculate and interpret the simple correlation between two variables Determine whether the correlation is significant Calculate and interpret the simple linear regression equation for a set of data Understand the assumptions behind regression analsis Determine whether a regression model is significant Chap 13-

2 Chapter 13 Student Lecture Notes 13- Chapter Goals After completing this chapter, ou should be able to: Calculate and interpret confidence intervals for the regression coefficients Recognize regression analsis applications for purposes of prediction and description Recognize some potential problems if regression analsis is used incorrectl Recognize nonlinear relationships between two variables (continued) Chap 13-3 Scatter Plots and Correlation A scatter plot (or scatter diagram) is used to show the relationship between two variables Correlation analsis is used to measure strength of the association (linear relationship) between two variables Onl concerned with strength of the relationship No causal effect is implied Chap 13-4

3 Chapter 13 Student Lecture Notes 13-3 Scatter Plot Eamples Linear relationships Curvilinear relationships Chap 13-5 Scatter Plot Eamples Strong relationships (continued) Weak relationships Chap 13-6

4 Chapter 13 Student Lecture Notes 13-4 Scatter Plot Eamples No relationship (continued) Chap 13-7 Correlation Coefficient Correlation measures the strength of the linear association between two variables The sample correlation coefficient r is a measure of the strength of the linear relationship between two variables, based on sample observations (continued) Chap 13-8

5 Chapter 13 Student Lecture Notes 13-5 Features of r Unit free Range between -1 and 1 The closer to -1, the stronger the negative linear relationship The closer to 1, the stronger the positive linear relationship The closer to 0, the weaker the linear relationship Chap 13-9 Eamples of Approimate r Values r = -1 r = -.6 r = 0 r = +.3 r = +1 Chap 13-10

6 Chapter 13 Student Lecture Notes 13-6 Calculating the Correlation Coefficient Sample correlation coefficient: r or the algebraic equivalent: [ ( ( )( ) ) ][ n r [n( ) ( ) ][n( ) where: r = Sample correlation coefficient n = Sample size = Value of the independent variable = Value of the dependent variable ( ) ] ( Chap ) ] Calculation Eample Tree Height Trunk Diameter =31 =73 =314 =14111 =713 Chap 13-1

7 Chapter 13 Student Lecture Notes 13-7 Tree Height, 70 Calculation Eample r (continued) n [n( ) ( ) ][n( ) ( ) ] (314) (73)(31) [8(713) (73) ][8(14111) (31) ] Trunk Diameter, Chap Ecel Output Ecel Correlation Output Tools / data analsis / correlation Tree Height Trunk Diameter Tree Height 1 Trunk Diameter Chap 13-14

8 Chapter 13 Student Lecture Notes 13-8 Significance Test for Correlation Hpotheses H 0 : ρ = 0 H A : ρ 0 (no correlation) (correlation eists) Test statistic t r 1 r n The Greek letter ρ (rho) represents the population correlation coefficient (with n degrees of freedom) Chap Eample: Produce Stores Is there evidence of a linear relationship between tree height and trunk diameter at the.05 level of significance? H 0 : ρ = 0 H 1 : ρ 0 (No correlation) (correlation eists) =.05, df = 8 - = 6 t r 1 r n Chap 13-16

9 Chapter 13 Student Lecture Notes 13-9 Eample: Test Solution r t 1 r n d.f. = 8- = 6 /= /=.05 Decision: Reject H 0 Conclusion: There is sufficient evidence of a linear relationship at the 5% level of significance Reject H 0 -t α/ Do not reject H 0 0 Reject H t 0 α/ Chap Introduction to Regression Analsis Regression analsis is used to: Predict the value of a dependent variable based on the value of at least one independent variable Eplain the impact of changes in an independent variable on the dependent variable Dependent variable: the variable we wish to eplain Independent variable: the variable used to eplain the dependent variable Chap 13-18

10 Chapter 13 Student Lecture Notes Simple Linear Regression Model Onl one independent variable, Relationship between and is described b a linear function Changes in are assumed to be caused b changes in Chap Tpes of Regression Models Positive Linear Relationship Relationship NOT Linear Negative Linear Relationship No Relationship Chap 13-0

11 Chapter 13 Student Lecture Notes Population Linear Regression The population regression model: Dependent Variable Population intercept Population Slope Coefficient 1 Independent Variable β0 β ε Random Error term, or residual Linear component Random Error component Chap 13-1 Linear Regression Assumptions Error values (ε) are statisticall independent Error values are normall distributed for an given value of The probabilit distribution of the errors is normal The distributions of possible ε values have equal variances for all values of The underling relationship between the variable and the variable is linear Chap 13-

12 Chapter 13 Student Lecture Notes 13-1 Population Linear Regression Observed Value of for i 1 β0 β ε (continued) Predicted Value of for i ε i Random Error for this value Slope = β 1 Intercept = β 0 i Chap 13-3 Estimated Regression Model The sample regression line provides an estimate of the population regression line Estimated (or predicted) value Estimate of the regression intercept Estimate of the regression slope ŷi b0 b1 Independent variable The individual random error terms e i have a mean of zero Chap 13-4

13 Chapter 13 Student Lecture Notes Least Squares Criterion b 0 and b 1 are obtained b finding the values of b 0 and b 1 that minimize the sum of the squared residuals e ( ŷ) ( (b 0 b 1 )) Chap 13-5 The Least Squares Equation The formulas for b 1 and b 0 are: b ( )( ) 1 and ( ) b0 b1 algebraic equivalent for b 1 : b n 1 ( ) n Chap 13-6

14 Chapter 13 Student Lecture Notes Interpretation of the Slope and the Intercept b 0 is the estimated average value of when the value of is zero b 1 is the estimated change in the average value of as a result of a one-unit change in Chap 13-7 Finding the Least Squares Equation The coefficients b 0 and b 1 will usuall be found using computer software, such as Ecel or Minitab Other regression measures will also be computed as part of computer-based regression analsis Chap 13-8

15 Chapter 13 Student Lecture Notes Simple Linear Regression Eample A real estate agent wishes to eamine the relationship between the selling price of a home and its size (measured in square feet) A random sample of 10 houses is selected Dependent variable () = house price in $1000s Independent variable () = square feet Chap 13-9 Sample Data for House Price Model House Price in $1000s () Square Feet () Chap 13-30

16 Chapter 13 Student Lecture Notes Regression Using Ecel Data / Data Analsis / Regression Chap Ecel Output Regression Statistics Multiple R R Square Adjusted R Square Standard Error Observations 10 The regression equation is: ANOVA df SS MS F Significance F Regression Residual Total Coefficients Standard Error t Stat P-value Lower 95% Upper 95% Intercept Square Feet Chap 13-3

17 House Price ($1000s) Chapter 13 Student Lecture Notes Graphical Presentation House price model: scatter plot and regression line Square Feet houseprice (squarefeet) Chap Interpretation of the Intercept, b 0 houseprice (squarefeet) b 0 is the estimated average value of Y when the value of X is zero (if = 0 is in the range of observed values) Here, no houses had 0 square feet, so b 0 = just indicates that, for houses within the range of sizes observed, $98,48.33 is the portion of the house price not eplained b square feet Chap 13-34

18 Chapter 13 Student Lecture Notes houseprice Interpretation of the Slope Coefficient, b (squarefeet) b 1 measures the estimated change in the average value of Y as a result of a oneunit change in X Here, b 1 = tells us that the average value of a house increases b.10977($1000) = $109.77, on average, for each additional one square foot of size Chap Least Squares Regression Properties The sum of the residuals from the least squares regression line is 0 ( ( ) ˆ 0 ) The sum of the squared residuals is a minimum (minimized ( ) ˆ ) The simple regression line alwas passes through the mean of the variable and the mean of the variable The least squares coefficients are unbiased estimates of β 0 and β 1 Chap 13-36

19 Chapter 13 Student Lecture Notes Eplained and Uneplained Variation Total variation is made up of two parts: SST SSE SSR Total sum of Squares Sum of Squares Error Sum of Squares Regression SST ( ) SSE ( ŷ) where: = Average value of the dependent variable = Observed values of the dependent variable ŷ = Estimated value of for the given value SSR (ŷ ) Chap Eplained and Uneplained Variation SST = total sum of squares Measures the variation of the i values around their mean SSE = error sum of squares Variation attributable to factors other than the relationship between and SSR = regression sum of squares Eplained variation attributable to the relationship between and (continued) Chap 13-38

20 Chapter 13 Student Lecture Notes 13-0 i _ Eplained and Uneplained Variation _ SST = ( i - ) SSE = ( i - i ) _ SSR = ( i - ) (continued) _ X i Chap Coefficient of Determination, R The coefficient of determination is the portion of the total variation in the dependent variable that is eplained b variation in the independent variable The coefficient of determination is also called R-squared and is denoted as R R SSR SST where 0 R 1 Chap 13-40

21 Chapter 13 Student Lecture Notes 13-1 R Coefficient of Determination, R Coefficient of determination SSR SST (continued) sumof squareseplainedb regression total sumof squares Note: In the single independent variable case, the coefficient of determination is where: R r R = Coefficient of determination r = Simple correlation coefficient Chap Eamples of Approimate R Values R = 1 R = 1 Perfect linear relationship between and : 100% of the variation in is eplained b variation in R = +1 Chap 13-4

22 Chapter 13 Student Lecture Notes 13- Eamples of Approimate R Values (continued) 0 < R < 1 Weaker linear relationship between and : Some but not all of the variation in is eplained b variation in Chap Eamples of Approimate R Values (continued) R = 0 No linear relationship between and : R = 0 The value of Y does not depend on. (None of the variation in is eplained b variation in ) Chap 13-44

23 Chapter 13 Student Lecture Notes 13-3 Ecel Output Regression Statistics Multiple R R Square Adjusted R Square Standard Error Observations 10 ANOVA df SS MS F Significance F Regression Residual Total Coefficients Standard Error t Stat P-value Lower 95% Upper 95% Intercept Square Feet Chap Test for Significance of Coefficient of Determination Hpotheses H 0 : ρ = 0 H A : ρ 0 H 0 : The independent variable does not eplain a significant portion of the variation in the dependent variable H A : The independent variable does eplain a significant portion of the variation in the dependent variable Test statistic F SSR/1 SSE/(n ) (with D 1 = 1 and D = n degrees of freedom) Chap 13-46

24 Chapter 13 Student Lecture Notes 13-4 Ecel Output Regression Statistics Multiple R R Square Adjusted R Square Standard Error Observations 10 ANOVA df SS MS F Significance F Regression Residual Total Coefficients Standard Error t Stat P-value Lower 95% Upper 95% Intercept Square Feet Chap Standard Error of Estimate The standard deviation of the variation of observations around the simple regression line is estimated b s ε SSE n Where SSE = Sum of squares error n = Sample size Chap 13-48

25 Chapter 13 Student Lecture Notes 13-5 The Standard Deviation of the Regression Slope The standard error of the regression slope coefficient (b 1 ) is estimated b s where: s b1 s ε b 1 s ε ( ) s ε ( = Estimate of the standard error of the least squares slope SSE n = Sample standard error of the estimate Chap n ) Ecel Output Regression Statistics Multiple R R Square Adjusted R Square Standard Error Observations 10 ANOVA df SS MS F Significance F Regression Residual Total Coefficients Standard Error t Stat P-value Lower 95% Upper 95% Intercept Square Feet Chap 13-50

26 Chapter 13 Student Lecture Notes 13-6 Comparing Standard Errors Variation of observed values from the regression line Variation in the slope of regression lines from different possible samples smalls smalls b1 larges large s b1 Chap Inference about the Slope: t Test t test for a population slope Is there a linear relationship between and? Null and alternative hpotheses H 0 : β 1 = 0 H A : β 1 0 Test statistic t b1 β s b 1 1 (no linear relationship) (linear relationship does eist) where: b 1 = Sample regression slope coefficient β 1 = Hpothesized slope s b1 = Estimator of the standard error of the slope d.f. n Chap 13-5

27 Chapter 13 Student Lecture Notes 13-7 House Price in $1000s () Square Feet () Inference about the Slope: t Test (continued) Estimated Regression Equation: houseprice (sq.ft.) The slope of this model is Does square footage of the house affect its sales price? Chap Inferences about the Slope: t Test Eample H 0 : β 1 = 0 H A : β 1 0 d.f. = 10- = 8 From Ecel output: Coefficients Standard Error t Stat P-value Intercept Square Feet Decision: /=.05 /=.05 Conclusion: Reject H 0 Do not reject H -t 0 α/ t α/ Reject H 0 Chap 13-54

28 Chapter 13 Student Lecture Notes 13-8 Regression Analsis for Description Confidence Interval Estimate of the Slope: b1 t /s d.f. = n - b 1 Ecel Printout for House Prices: Coefficients Standard Error t Stat P-value Lower 95% Upper 95% Intercept Square Feet Chap Regression Analsis for Description Coefficients Standard Error t Stat P-value Lower 95% Upper 95% Intercept Square Feet Since the units of the house price variable is $1000s, we are 95% confident that the average impact on sales price is between $33.70 and $ per square foot of house size This 95% confidence interval does not include 0. Conclusion: There is a significant relationship between house price and square feet at the.05 level of significance Chap 13-56

29 Chapter 13 Student Lecture Notes 13-9 Confidence Interval for the Average, Given Confidence interval estimate for the mean of given a particular p ŷ Size of interval varies according to distance awa from mean, 1 (p ) n ( ) t /sε Chap Confidence Interval for an Individual, Given Confidence interval estimate for an Individual value of given a particular p ŷ 1 (p ) 1 n ( ) t /sε Chap 13-58

30 Chapter 13 Student Lecture Notes Eample: House Prices House Price in $1000s () Square Feet () Estimated Regression Equation: houseprice (sq.ft.) Predict the price for a house with 000 square feet Chap Eample: House Prices Predict the price for a house with 000 square feet: (continued) houseprice (sq.ft.) (000) The predicted price for a house with 000 square feet is ($1,000s) = $317,850 Chap 13-60

31 residuals residuals Chapter 13 Student Lecture Notes Residual Analsis Purposes Eamine for linearit assumption Eamine for constant variance for all levels of Evaluate normal distribution assumption Graphical Analsis of Residuals Can plot residuals vs. Can create histogram of residuals to check for normalit Chap Residual Analsis for Linearit Not Linear Linear Chap 13-6

32 Residuals residuals residuals Chapter 13 Student Lecture Notes 13-3 Residual Analsis for Constant Variance Non-constant variance Constant variance Chap Ecel Output RESIDUAL OUTPUT Predicted House Price Residuals House Price Model Residual Plot Square Feet Chap 13-64

33 Chapter 13 Student Lecture Notes Chapter Summar Introduced correlation analsis Discussed correlation to measure the strength of a linear association Introduced simple linear regression analsis Calculated the coefficients for the simple linear regression equation Described measures of variation (R and s ε ) Addressed assumptions of regression and correlation Chap Chapter Summar Described inference about the slope Addressed estimation of mean values and prediction of individual values Discussed residual analsis (continued) Chap 13-66

34 Chapter 13 Student Lecture Notes Problems A random sample of eight drivers insured with a compan and having similar auto insurance policies was selected. The following table lists their driving eperiences (in ears) and monthl auto insurance premiums. Driving Eperience (ears) Monthl Auto Insurance Premium Chap Problems a. Does the insurance premium depend on the driving eperience or does the driving eperience depend on the insurance premium? Do ou epect a positive or a negative relationship between these two variables? Chap 13-68

35 Chapter 13 Student Lecture Notes Problems b. Compute SS, SS, and SS. Chap Problems c. Find the least squares regression line b choosing appropriate dependent and independent variables based on our answer in part a. Chap 13-70

36 Chapter 13 Student Lecture Notes Problems d. Interpret the meaning of the values of a and b calculated in part c Chap Problems e. Plot the scatter diagram and the regression line Chap 13-7

37 Chapter 13 Student Lecture Notes Problems f. Calculate r and r and eplain what the mean Chap Problems g. Predict the monthl auto insurance premium for a driver with 10 ears of driving eperience Chap 13-74

38 Chapter 13 Student Lecture Notes Problems h. Compute the standard deviation of errors Chap Problems i. Construct a 90% confidence interval for B. Chap 13-76

39 Chapter 13 Student Lecture Notes Problems j. Test at the 5% significance level whether B is negative. Chap Problems k. Using α =.05, test whether ρ is different from zero. Chap 13-78

40 Chapter 13 Student Lecture Notes Copright The materials of this presentation were mostl taken from the PowerPoint files accompanied Business Statistics: A Decision-Making Approach, 7e 008 Prentice-Hall, Inc. Chap 10-79