Prepared by: Prof. Dr Bahaman Abu Samah Department of Professional Development and Continuing Education Faculty of Educational Studies Universiti

Transcription

1 Prepared by: Prof Dr Bahaman Abu Samah Department of Professional Development and Continuing Education Faculty of Educational Studies Universiti Putra Malaysia Serdang

2 M L Regression is an extension to Simple Linear Regression Assess relationship between several IV s and one DV Predict the value of DV based on given values of set of IVs Prediction/regression equation is in the form: Y ˆ b b X b X b p X p where: Y X s Ŷ Next b 0 b i p Dependent variable Independent variables predicted value of Y Y-intercept Regression coefficients Number of IVs

3 Multiple Linear Regression is applied to: Derive prediction equation which can be used to predict the value of the dependent variable (Ŷ) Assess the overall model fit Test the individual contribution of individual predictors on the criterion/dependent variable Describe the nature of relationship between predictors and dependent variable (R and R 2 and adj-r 2 ) Next

4 Requirements: DV Interval/Ratio IV Interval/Ratio Multiple Linear Regression requires: Dependent variable MUST be Interval or ratio scale Predictors MUST also be interval or ratio scale Categorical independent variables can also be entered into this analysis provided that they be transformed into dummy variables The number of dummy variables to be created equals one less than the number of categories (k 1) Next

5 Berry (1993) identifies the following assumptions for regression analysis: 1 Variable types quantitative or categorical (with two categories) 2 Independent it is assumed that all of the values of the outcome variable are independent 3 No perfect multicollinearity Assumptions: 1 Matric variable 2 Scores independent 3 Multicollinearity 4 Independence of errors 5 Normality 6 Linearity 7 Homoscadesticity Berry, W D (1993) Understanding regression assumptions Sage university paper series on quantitative applications in the social sciences, Newbury Park, CA: Sage Next

6 4 Independent errors for any two observations, the residual terms should be uncorrelated (or independent) 5 Normally distributed errors 6 Linearity a linear relationship between IV s and DV 7 Homoscedasticity the residuals at each level of the predictor(s) should have the same variance Assumptions: 1 Matric variable 2 Scores independent 3 Multicollinearity 4 Independence of errors 5 Normality 6 Linearity 7 Homoscadesticity Next

7 Descriptive Regression/prediction equation R R 2 In addition: Adjusted R 2 Test for influential cases Test for assumptions Hypothesis Tests Regression model Individual Slope Inferential Next

8 1 Derive Prediction Equation Model Summary: R, R 2 and Adj R 2 Hypothesis Test: Regression Model Hypothesis Test: Slope Influential Cases 6 Testing for Assumptions Next

9

10 Unstandardized Prediction Equation Yˆ Tenure 668 Attitude 513 Stress Standardized Prediction Equation Yˆ 191Tenure 510 Attitude 426 Stress

11

12 R = 809 R 2 = 654 Adj R 2 = 607

13

14 1 State H O and H A 2 3 Set Confidence Interval (α) Run Analysis Report F and sig-f 5 4 Decision Conclusion

15 H H O A : Y : Y 0 0 i X 1 1 X 2 2 p X p i Generally set α = 05 Report F and sig-f

16 Reject H O : sig-f α Fail to Reject H O : sig-f > α Reject H O : Regression model fits the data Fail to Reject H O : Regression model does not fit data

17

18 1 State H O and H A Set Confidence Interval (α) Run Analysis Report t and sig-t Decision Conclusion 5

19 H H O A : i : i 0 0 Generally set α = 05 Report t and sig-t

20 Reject H O : sig-t α Fail to Reject H O : sig-t > α Reject H O : X i contributes sig towards Y Fail to Reject H O : X i does not contribute sig towards Y

21

22 Examine whether certain cases exert undue influence over the parameters of a model This procedure will also unveil outliers Residual statistics: Cook s distance Mahalanobis distance Leverage (hat values)

23 A measure of the overall influence of a case on the model Cook and Weisberg (1982) have suggested that cases with Cook s distance > 1 are potential as outliers Look for cases with the highest values However it is not easy to establish a cut-off point Sample of 100 with 3 predictors, a conservative cutoff-point is > 15

24

25 Also known as hat values; the influence of the observed value of the outcome variable over the predicted value Leverage ( p 1) n where p = # IV s Leverage ranges between 0 and 1 0 = case has no influence Hoaglin and Welsch (1978) recommend to investigate cases with values > 2 Leverages Steven (1992) suggests > 3 Leverages

26 Cook and Weisberg (1982) have suggested that values (Cook s distance) greater than 1 may be cause of concern No cases has undue influence on the model Mahalanobis distance Sample of 100 and three predictors, values > 15 are problematic; a conservative cut-off Leverage = (p + 1)/n = 4/26 = 1538 Hoaglin and Welsch (1978) values > 2 Leverages (3076) Steven (1992) suggests > 3 Leverages

27

28 1 Multicollinearity Independence of errors Multivariate Normality Linearity 5 Homoscedasticity

29 Multicollinearity is problem associated with high correlations between IVs It has substantial effect on the results of regression: limits the size of R 2 effects contribution of each IV How to identify collinearity? High correlation between IV s (r 90) Tolerance Variance Inflation Factor (VIF)

30 The amount of variability of an IV not explained by other IVs Small values of tolerance indicate high collinearity Cutoff threshold: tolerance 10 VIF is the inverse of tolerance value High VIF values indicate collinearity Cutoff threshold: VIF 10 VIF 1 tolerance Hair et al (2010) and Pallant (2010)

31 Use Durbin Watson to test this assumption Value closer to 2 indicates congruent to the assumption Value < 1 or > 3 violates the assumption

32 Regression analysis offers an assessment of the three assumptions thru analysis of residuals (scatterplot) Plot standardized predicted value of DV (ZPRED) against standardized residuals (ZRESID) X-axis - ZPRED Y-axis - ZRESID

33 Interpretations: (a) All assumptions met, (b) Failure of normality (c) Nonlinear (d) Heteroscedasticity (a) (b) (c) (d)

34

35

36 1 3 2 Enter Stepwise Hierarchical

37 Use as confirmatory, specify a set of IVs as possible factors/predictors Predictors are entered into the regression model simultaneously This method relies on good theoretical reasons for including the selected predictors Determine the significance of a linear combination of IVs and each individual IV on a DV

38 Results of Variable b SE Beta t p Intercept Tenure Attitude Stress F = R = 809 sig-f = 000 R 2 = 654

39 Predictors are entered in sequence one at a time The first predictor to be entered into the regression model will be based on the highest zero-order correlation Each subsequence predictor to be entered into the model will be based on the highest partial correlation This method should ensure that you end up with the smallest possible set of predictor variables included in a regression model

40 Results of Variable b Beta R R 2 R 2 Intercept 1816 Attitude Stress

41 Enter the predictors into the model in a specified order or sequence The order specified should reflect some theoretical consideration or previous findings Don t use this method if you have no reason to believe that one variable is likely to be more important than another As each variable is entered into the model, its contribution will be assessed A variable is dropped if it does not significantly increase the predictive power of the model

42 Sequence of Variable(s): 1 Attitude 2 Stress and Tenure

43 Step 1: Attitude Step 2: Stress and Tenure

44 Results of Variable R R 2 R 2 F p Attitude Stress Tenure