MULTIPLE REGRESSION ANALYSIS AND OTHER ISSUES. Business Statistics

Transcription

1 MULTIPLE REGRESSION ANALYSIS AND OTHER ISSUES Business Statistics

2 CONTENTS Multiple regression Dummy regressors Assumptions of regression analysis Predicting with regression analysis Old exam question Further study

3 MULTIPLE REGRESSION The regression model so far is for one dependent variable (Y) and one independent (explanatory) variable (X) There are many cases where several explanatory variables might play a role... might explain the dependent variable Y Example: house prices depend on floor area ground area (first floor + garden) number of rooms age of the house etc.

4 MULTIPLE REGRESSION Generalize simple regression model from Y = β 0 + β 1 X 1 + ε to Y = β 0 + β 1 X 1 + β 2 X 2 + ε or even to Y = β 0 + β 1 X 1 + β 2 X β k X k + ε Multiple regression a quite obvious extension we can reuse much of the theory of simple regression still based on OLS, R 2, F-test, and t-test Now, you ll understand why we used a subscript 0 for the constant in β 0...

5 MULTIPLE REGRESSION SPSS output Estimated model: Y = X X 2

6 MULTIPLE REGRESSION Step 0 (statistical model): Y = β 0 + β 1 X 1 + β 2 X 2 + ε, with ε~n 0, σ 2 Step 1: H 0 : β 1 = β 2 = 0; H 1 : at least one of these not 0 Step 2: Sample statistic: F = MSR MSE ; reject for too large values Step 3: Under H 0 : F~F 2,n 3 ; assumption: see model (step 0) Step 4: F calc = ; F crit = F 2,n 3;α Step 5: reject/not reject H 0 mind that the null hypothesis does not include the constant (intercept) β 0 with k regressors: df 1 = k df 2 = n k 1

7 MULTIPLE REGRESSION Rejecting the F-test in multiple regressions means: at least one of the slope coefficients differs from 0 not β 1 = β 2 = 0 which one differs (or differ) from 0 must be investigated by separate t-tests So, while in simple regression the overall F-test and the t-test for β 1 do exactly the same thing the two tests have a complimentary role in multiple regression first look at overall F, then go to the individual ts

8 MULTIPLE REGRESSION First, overall model test, using F-test Next, test each slope coefficient, using k times a t-test not interesting

9 EXERCISE 1 What does it mean when in multiple regression a. the overall F-test yields a significant result? b. a t-test of an individual coefficient β 3 yields a significant result?

10 MULTIPLE REGRESSION Example: overall F-test: highly significant both regression slopes: highly significant coefficient of determination (R 2 ): very high (90%) a very useful model in fact: better than the simple regression model with R 2 = 82%

11 MULTIPLE REGRESSION Observe: including more explanatory variables will in general improve the model R 2 will increase, even if we include non-sense variables (e.g., street number of the house) 2 R adj ( R-square-adjusted ) penalizes for including too many regressors 2 = 1 SSE/n k 1 SST/n 1 R adj while R2 = 1 SSE SST

12 DUMMY REGRESSORS House prices (numerical) depend on: numerical variables (floor area, ground area, etc.) binary categorical variables (with/without garage, etc.) other categorical variables (no/free/paid parking, etc.) However: regression for numerical X and numerical Y ANOVA for categorical X and numerical Y So, how to combine numerical X 1 and categorical X 2? Solution: dummy variables for categorical variable dummy regressors/dummy regression

13 DUMMY REGRESSORS We can include dummy variables in multiple regression Splitting binary in several binary original variable: garage = no/yes garage: 0=no; 1=yes Splitting non-binary in several binary original variable: parking = no/free/paid free_parking: 0=no; 1=yes paid_parking: 0=no; 1=yes Dummy variables only for independent (X) variables never for dependent (Y) variable Y must be numerical (think about ε~n) Omitted variable: no_garage (redundant): garage=0 Omitted variable: no_parking (redundant): free=0, paid=0

14 DUMMY REGRESSORS Example House price (Y) as a function of floor area (X 1 ) dummy for garden (X 2 ; 0=No, 1=Yes) Price = FloorArea Garden meaning extra when there is a garden (whatever the size)

15 DUMMY REGRESSORS Use dummy variables only for the independent (explanatory) variable not for the dependent variable.(logistic regression, not in this course!) It is quite common to indicate dummy explanatory variables with a D instead of an X for instance: Y = β 0 + β 1 X 1 + β 2 D 2 + β 3 D 3 + ε

16 EXERCISE 2 We want to explain car prices in terms of 1) engine power 2) number of seats 3) gas/diesel/electric. What is the theoretical model?

17 ASSUMPTIONS OF REGRESSION ANALYSIS The OLS equations always find coefficients b 0, b 1, that minimize the residual sum of squares (SSE) so no assumptions required for that part But when testing the model (and when testing the coefficients β 1, β 2, ) we need to assume a statistical model with ε~n 0, σ 2 : the residual terms should be normally distributed the residual terms should come from a distribution with constant variance the residual terms should be independent of each other there should be a linear relationship between the X-variable(s) and Y

18 ASSUMPTIONS OF REGRESSION ANALYSIS A final word on the residual ε~n 0, σ 2 Theoretical regression model Y = β 0 + β 1 X 1 + β 2 X β k X k + ε Estimated regression model Y = b 0 + b 1 X 1 + b 2 X β k X k Observations Y i = b 0 + b 1 X 1,i + b 2 X 2,i + + β k X k,i + e i And the standard deviation of the residual term σ = σ 2 is estimated by s = SSE = n k 1 MSE is known as the standard error of the regression or standard error of the estimate

19 PREDICTION WITH REGRESSION ANALYSIS Given a sample of data x 1i, x 2i,, y i with i = 1,, n we can use OLS to estimate the regression model Y = b 0 + b 1 X 1 + b 2 X 2 + subsequently, given the floor area, we can estimate the price of the house Now, a new incomplete observations arrives for instance, a new house with known floor area (x n+1 ), but with unknown price (no y n+1 ) We can use the regression model to estimate the house price so to predict y n+1

20 PREDICTION WITH REGRESSION ANALYSIS Example: Y = X a house with floor area x = 85 m2 has an estimated price y = = ( )

21 PREDICTION WITH REGRESSION ANALYSIS So, we can predict a value of y for a given x (or x 1, x 2, ) and given estimated regression coefficients (b 0, b 1, ) The quality of this estimate depends obviously on the quality of the regression model try to find a confidence interval for the estimated y-value two types: the confidence interval for the average price of a house of 85 m2 the confidence interval for a particular house of 85 m2

22 PREDICTION WITH REGRESSION ANALYSIS Point prediction: Case 1: confidence interval (95%) for prediction of mean price , Case 2: confidence interval (95%) for individual prediction 96372, Individual predictions are always less accurate wider confidence interval (this one even includes 0) Price (Y) unknown, area (X) known

23 OLD EXAM QUESTION 26 March 2015, Q3a

24 FURTHER STUDY Doane & Seward 5/E 12.7, Tutorial exercises week 4 multiple regression dummy regression prediction interval