22S39: Class Notes / November 14, 2000 back to start 1

Transcription

1 Model diagnostics Interpretation of fitted regression model 22S39: Class Notes / November 14, 2000 back to start 1

2 Model diagnostics 22S39: Class Notes / November 14, 2000 back to start 2

3 Model diagnostics Before we interpret a fitted (linear) regression model, we need to check whether or not the fitted regression model provide good fit to the data. In other words, we need to check whether or not the regression model is a plausible data mechanism for the data on hand. 22S39: Class Notes / November 14, 2000 back to start 2

4 Model diagnostics Before we interpret a fitted (linear) regression model, we need to check whether or not the fitted regression model provide good fit to the data. In other words, we need to check whether or not the regression model is a plausible data mechanism for the data on hand. This requires checking whether or not the residuals fulfill the asumptions of (1) having no pattern (2) constant variance (3) normally distributed and (4) independent. 22S39: Class Notes / November 14, 2000 back to start 2

5 Model diagnostics Before we interpret a fitted (linear) regression model, we need to check whether or not the fitted regression model provide good fit to the data. In other words, we need to check whether or not the regression model is a plausible data mechanism for the data on hand. This requires checking whether or not the residuals fulfill the asumptions of (1) having no pattern (2) constant variance (3) normally distributed and (4) independent. The residuals may display a trend if the mean response is not linear in X. This can be more readily detected by plotting the residuals against the fitted values, or plotting the residuals against the X variable. Either plots convey the same information. Why? 22S39: Class Notes / November 14, 2000 back to start 2

6 For example: below are data simulated from the model Y = X X 2 + e where the errors are normally distributed and the X s are N(0, 5 2 ) 22S39: Class Notes / November 14, 2000 back to start 3

7 y residuals x x residuals normal quantiles fitted residual quantiles Figure 1: 22S39: Class Notes / November 14, 2000 back to start 4

8 22S39: Class Notes / November 14, 2000 back to start 5

9 The scatter diagram of the X and Y shows slight curvature which is much clearer in the residual vs fitted values plot. 22S39: Class Notes / November 14, 2000 back to start 5

10 The scatter diagram of the X and Y shows slight curvature which is much clearer in the residual vs fitted values plot. Also, the q-q plot of the residuals show non-normality mainly due to the curvature in the residuals. 22S39: Class Notes / November 14, 2000 back to start 5

11 The scatter diagram of the X and Y shows slight curvature which is much clearer in the residual vs fitted values plot. Also, the q-q plot of the residuals show non-normality mainly due to the curvature in the residuals. When the residuals show curvature, we may fit a quadratic model Y = α 0 + β 1 (X X) + β 2 (X X) 2 + e or cubic models, or polynomial models of higher degrees. 22S39: Class Notes / November 14, 2000 back to start 5

12 The scatter diagram of the X and Y shows slight curvature which is much clearer in the residual vs fitted values plot. Also, the q-q plot of the residuals show non-normality mainly due to the curvature in the residuals. When the residuals show curvature, we may fit a quadratic model Y = α 0 + β 1 (X X) + β 2 (X X) 2 + e or cubic models, or polynomial models of higher degrees. Sometimes, the mean response only bears a linear relationship with X after a suitable transformation. 22S39: Class Notes / November 14, 2000 back to start 5

13 The scatter diagram of the X and Y shows slight curvature which is much clearer in the residual vs fitted values plot. Also, the q-q plot of the residuals show non-normality mainly due to the curvature in the residuals. When the residuals show curvature, we may fit a quadratic model Y = α 0 + β 1 (X X) + β 2 (X X) 2 + e or cubic models, or polynomial models of higher degrees. Sometimes, the mean response only bears a linear relationship with X after a suitable transformation. For example, we have simulated data from the model: Y = exp(0.6 X + e)) where e N(0, ), i.e., Y grows exponentially with X on the average. If we transform Y to Z = log(y ), we have Z = 0.6 X + e, a linear regression model. 22S39: Class Notes / November 14, 2000 back to start 5

14 The following figures show the diagnostics plots when we fit a simple linear regression model of Y on X 22S39: Class Notes / November 14, 2000 back to start 6

15 y residuals x fitted residuals normal quantiles x data quantiles Figure 2: 22S39: Class Notes / November 14, 2000 back to start 7

16 22S39: Class Notes / November 14, 2000 back to start 8

17 We see that the residuals has a curved pattern, and also its variance increases with the magnitude of the fitted value. That is the residuals appear to have non-constant variance, known as heteroscedasticity. Do you expect these two phenomena? 22S39: Class Notes / November 14, 2000 back to start 8

18 We see that the residuals has a curved pattern, and also its variance increases with the magnitude of the fitted value. That is the residuals appear to have non-constant variance, known as heteroscedasticity. Do you expect these two phenomena? If the residuals show curvature and heteroscedascity, it calls for transforming Y. 22S39: Class Notes / November 14, 2000 back to start 8

19 We see that the residuals has a curved pattern, and also its variance increases with the magnitude of the fitted value. That is the residuals appear to have non-constant variance, known as heteroscedasticity. Do you expect these two phenomena? If the residuals show curvature and heteroscedascity, it calls for transforming Y. Common transformations that one may try include 22S39: Class Notes / November 14, 2000 back to start 8

20 We see that the residuals has a curved pattern, and also its variance increases with the magnitude of the fitted value. That is the residuals appear to have non-constant variance, known as heteroscedasticity. Do you expect these two phenomena? If the residuals show curvature and heteroscedascity, it calls for transforming Y. Common transformations that one may try include 1. Z = Y (good to try for responses that are counts so that variance of residuals increases with the fitted value), 22S39: Class Notes / November 14, 2000 back to start 8

21 We see that the residuals has a curved pattern, and also its variance increases with the magnitude of the fitted value. That is the residuals appear to have non-constant variance, known as heteroscedasticity. Do you expect these two phenomena? If the residuals show curvature and heteroscedascity, it calls for transforming Y. Common transformations that one may try include 1. Z = Y (good to try for responses that are counts so that variance of residuals increases with the fitted value), 2. Z = log(y ) (good to try if you suspect that the response should increase by a certain percent per unit change in X, or if the that variance of residuals increases with the squared fitted value), 22S39: Class Notes / November 14, 2000 back to start 8

22 3. and Z = 1/Y (good to try if the recipropocal may make sense, e.g., automobile efficieny can be measured by miles per gallon, or gallons per mile). 22S39: Class Notes / November 14, 2000 back to start 9

23 residuals (sqrt(y)) residuals (log(y)) fitted fitted residuals (1/y)) fitted Figure 3: 22S39: Class Notes / November 14, 2000 back to start 10

24 22S39: Class Notes / November 14, 2000 back to start 11

25 Based on the residual plots, we find that the log transformation is appropriate. 22S39: Class Notes / November 14, 2000 back to start 11

26 Based on the residual plots, we find that the log transformation is appropriate. Other common problems encountered in regression analysis are outliers and influential data. 22S39: Class Notes / November 14, 2000 back to start 11

27 Based on the residual plots, we find that the log transformation is appropriate. Other common problems encountered in regression analysis are outliers and influential data. Influential cases refer to cases with extreme X values that may influence the fitted model unduly. 22S39: Class Notes / November 14, 2000 back to start 11

28 y x Figure 4: Note that if we lower the Y-value of the point with X = 10, the fitted line is changed a lot. 22S39: Class Notes / November 14, 2000 back to start 12

29 Errors need not be independent, for example, if there is a learning effect, then the error variance of the later runs (of experiments) than the earlier ones. Plotting the residuals against the order of the experiments were done may point out some such problems. 22S39: Class Notes / November 14, 2000 back to start 13

30 Errors need not be independent, for example, if there is a learning effect, then the error variance of the later runs (of experiments) than the earlier ones. Plotting the residuals against the order of the experiments were done may point out some such problems. Outliers refer to cases where the residuals are unusual, e.g., with magnitude larger than 3 MSE. Here is the scatter plot of the Presidential Election data from Florida 22S39: Class Notes / November 14, 2000 back to start 13

31 Errors need not be independent, for example, if there is a learning effect, then the error variance of the later runs (of experiments) than the earlier ones. Plotting the residuals against the order of the experiments were done may point out some such problems. Outliers refer to cases where the residuals are unusual, e.g., with magnitude larger than 3 MSE. Here is the scatter plot of the Presidential Election data from Florida Clearly, Palm Beach is an outlier. 22S39: Class Notes / November 14, 2000 back to start 13

32 Errors need not be independent, for example, if there is a learning effect, then the error variance of the later runs (of experiments) than the earlier ones. Plotting the residuals against the order of the experiments were done may point out some such problems. Outliers refer to cases where the residuals are unusual, e.g., with magnitude larger than 3 MSE. Here is the scatter plot of the Presidential Election data from Florida Clearly, Palm Beach is an outlier. Both outliers and influential cases may unduly affect the model fit, and they may be dropped from the analysis in many cases. However, the bottom line is that regression analysis can point out unusual cases which may tell us something interesting. 22S39: Class Notes / November 14, 2000 back to start 13

33 Interpretation of fitted regression model Here we check the model fit of the regression model of Price on (polishing) Time: Coef SE Coef T p TIME (constant) s= 20.5 R-Sq: 84.47%,R-sq(adj): 84.2% Analysis of Variance Source Df SS MS F p Regression Residual Error S39: Class Notes / November 14, 2000 back to start 14

34 Total S39: Class Notes / November 14, 2000 back to start 15

35 Total We shall consider this model fit later on. Accepting the model fit for the moment, we note that the intercept is not significant(ly different from zero), which makes sense. The slope is significantly different from 0, indicating that TIME is a useful predictor for PRICE. In particular, each unit increase in polishing time leads to an increase of about 2.5 unit price, on the average. 22S39: Class Notes / November 14, 2000 back to start 15

36 Total We shall consider this model fit later on. Accepting the model fit for the moment, we note that the intercept is not significant(ly different from zero), which makes sense. The slope is significantly different from 0, indicating that TIME is a useful predictor for PRICE. In particular, each unit increase in polishing time leads to an increase of about 2.5 unit price, on the average. Furthermore, TIME explains about 84.5% of the variation in in PRICE. 22S39: Class Notes / November 14, 2000 back to start 15

37 Total We shall consider this model fit later on. Accepting the model fit for the moment, we note that the intercept is not significant(ly different from zero), which makes sense. The slope is significantly different from 0, indicating that TIME is a useful predictor for PRICE. In particular, each unit increase in polishing time leads to an increase of about 2.5 unit price, on the average. Furthermore, TIME explains about 84.5% of the variation in in PRICE. Now, let s stand back to see if the above model provides good fit to the data. 22S39: Class Notes / November 14, 2000 back to start 15

38 Total We shall consider this model fit later on. Accepting the model fit for the moment, we note that the intercept is not significant(ly different from zero), which makes sense. The slope is significantly different from 0, indicating that TIME is a useful predictor for PRICE. In particular, each unit increase in polishing time leads to an increase of about 2.5 unit price, on the average. Furthermore, TIME explains about 84.5% of the variation in in PRICE. Now, let s stand back to see if the above model provides good fit to the data. Note that we shall plot the standardized residuals, defined as residuals divided by the square root of RMS, so that the standardized residuals are approximately N(0, 1) if the fitted model is appropriate for the data. 22S39: Class Notes / November 14, 2000 back to start 15

39 PRICE standardized residuals TIME fitted standardized residuals normal quantiles diameter data quantiles Figure 5: 22S39: Class Notes / November 14, 2000 back to start 16

40 22S39: Class Notes / November 14, 2000 back to start 17

41 We note that there seems to be an outlier, as indicated by the case whose standardized residual has magnitude larger than 3. 22S39: Class Notes / November 14, 2000 back to start 17

42 We note that there seems to be an outlier, as indicated by the case whose standardized residual has magnitude larger than 3. Also, the residuals vs fitted plot appears to show no particular pattern. However, the residuals vs the diameter plot suggests a linear relationship. That is, we can predict the residual values based on the diameter. 22S39: Class Notes / November 14, 2000 back to start 17

43 We note that there seems to be an outlier, as indicated by the case whose standardized residual has magnitude larger than 3. Also, the residuals vs fitted plot appears to show no particular pattern. However, the residuals vs the diameter plot suggests a linear relationship. That is, we can predict the residual values based on the diameter. The above points to the fact that besides TIME, DIAM, diameter of the tableware, is also an important predictor. 22S39: Class Notes / November 14, 2000 back to start 17