Stat 501, F. Chiaromonte Lecture #8 Data set: BEARS.MTW In the minitab example data sets (for description, get into the help option and search for "Data Set Description"). Wild bears were anesthetized, and their bodies were measured and weighed. One goal of the study was to make a table (or perhaps a set of tables) for hunters, so they could estimate the weight of a bear based on other measurements (this would be used because in the forest it is easier to measure the length of a bear, for example, than it is to weigh it). Variable Count (N) Description Age 143 Bear's age, in months (can be derived). Sex 143 1 = male 2 = female Head.L 143 Length of the head, in inches Head.W 143 Width of the head, in inches Neck.G 143 Girth (distance around) the neck, in inches Length 143 Body length, in inches Chest.G 143 Girth (distance around) the chest, in inches Weight 143 Weight of the bear, in pounds Unit=observation and measurement of a beard (but some animals were re-captured and re-measured several times ) (this is not all the data set) 1
Well suited example for a regression application: 1. Construct and estimate a regression model for Y = Weight on one or more of the measurements. 2. Use the model to estimate the mean-weight, or predict a new weight, given the measurement(s) --point estimate, confidence interval, prediction interval. Descriptive Statistics Variable N Mean Median TrMean StDev Age 143 43.43 32.00 40.17 34.02 Head.L 143 13.416 13.500 13.419 1.921 Head.W 143 6.331 6.000 6.293 1.314 Neck.G 143 21.327 20.000 21.254 5.069 Length 143 61.283 61.000 61.488 9.352 Chest.G 143 36.313 35.000 36.134 8.235 Weight 143 192.16 154.00 186.03 110.54 Variable SE Mean Minimum Maximum Q1 Q3 Age 3.73 8.00 177.00 19.0 58.0 Head.L 0.161 9.000 18.500 12.0 15.0 Head.W 0.110 4.000 10.000 5.5 7.0 Neck.G 0.424 10.000 33.000 18.0 25.5 Length 0.782 36.000 83.000 57.0 67.5 Chest.G 0.689 19.000 55.000 30.5 42.0 Weight 9.24 26.00 514.00 116.0 250.0 2
Correlations (Pearson) Ag Hd.L Hd.W Nk.G Lgt Ct.G We Ag Hd.L 0.687 Hd.W 0.669 0.744 Nk.G 0.734 0.862 0.805 Lgt 0.691 0.895 0.736 0.873 Ct.G 0.734 0.854 0.756 0.940 0.889 We 0.774 0.833 0.756 0.943 0.875 0.966 134.75 50.25 Age 16.125 11.375 Head.L 8.5 5.5 Head.W 27.25 15.75 Neck.G 71.25 47.75 Length 46 28 Chest.G 392 148 Weight 50.25 134.75 11.375 16.125 5.5 8.5 15.75 27.25 47.75 71.25 28 46 148 392 3
As to be expected, measurements on a body are very much related to one another, with strong linear components in the relationships (although, as the scatter plot matrix shows, the relationships are not necessarily purely or mainly linear ) Since we are still doing simple regression, we need to costruct a model that uses only one measurement. As a try, let us use the one that has the strongest linear relationship with Weight, i.e. Chest.G Regression Analysis The regression equation is Weight = - 279 + 13.0 Chest.G Predictor Coef StDev T P Constant -278.75 10.88-25.61 0.000 Chest.G 12.9680 0.2923 44.36 0.000 S = 28.68 R-Sq = 93.3% R-Sq(adj) = 93.3% Analysis of Variance Source DF SS MS F P Regression 1 1619241 1619241 1968.0 0.000 Residual Error 141 116012 823 Total 142 1735253 4
The regression line parameters estimates can be derived from the descriptive statistics in the previous outputs: b 1 = (s Y /s X ) r XY = b 0 = Av(Y) - b 1 Av(X) = Note that b 0 is negative, and anyway Chest.G=0 is not "reasonable". In order to give an interpretation of the "location" of our line (given the slope), we could interpret: A Weight value: The fitted value on min(chest.g)=19 A Chest.G value: The Chest.G value at which the estimated line crosses the abscisa axis 5
Regression Plot Y = -278.749 + 12.9680X R-Sq = 93.3 % 500 400 Weight 300 200 100 0 20 30 40 50 Chest.G 6
Take Chest.G = 40. From the regression output, the descriptive statistics, and tables for the t-distribution, we can calculate Confidence int. for the mean-weight (Chest.G=40) Prediction int. for a new weight (Chest.G=40) This is very close to the sample mean of Chest.G, and well within the sample range. Now do the same at Chest.G = 80 (far out) Confidence int. for the mean-weight (Chest.G=80) Prediction int. for a new weight (Chest.G=80) With Minitab we can get the bands about the estimated regression line (in "Fitted line plot", check out the "Options") 7
Regression Plot Y = -278.749 + 12.9680X R-Sq = 93.3 % 500 400 300 Weight 200 100 0-100 Regression 95% CI 95% PI 20 30 40 50 Chest.G 8
This could be a good summary to give to a hunter: Measure the Chest.G, and you have A point-estimate of the corresponding mean-weight An interval about it, which is guaranteed to contain the unknown meanweight with probability.95 Another interval about it (wider), which is guaranteed to contain a new weight observation with probability.95 But we are in for a surprise! Although the estimated regression line captured a lot of the Weight-variability, maybe a line in Chest.G is not the most appropriate regression model to use! 9
100 RESI1 0-100 20 30 40 50 Chest.G Normal Probability Plot for RESI1 ML Estimates Percent 99 95 90 80 70 60 50 40 30 20 10 5 1 Mean: StDev: 0.0000000 28.4829-50 0 50 Data 10
The residuals show quite a bit of curvature along the predictor. The coefficient of determination is quite high (the linear component of the relationship strong) But the simple regression is not performing well because it leaves residuals containing a clear patern along the predictor (a curve here) (If we take the residuals as the sample-equivalent of the errors, we have graphical evidence against the assumption: E(ε) = 0 at every level of X) We will get into diagnostics next time; for now, intuitively: This ough to suggest that the mean-relationship between Y and X is not linear Maybe we should change our model and consider a parabolic relationship (second order polynomial): Weight = β 0 + β 1 Chest.G + β 2 Chest.G 2 + ε with ε indep. X, E(ε) = 0, var(ε) = σ 2 This model, although it still refers to one variable, contains two terms (linear in the parameters) so it is technically a multiple regression. (create the variable (Chest.G)^2 in Minitab through "Calc", and then regress Weight on both Chest.G and (Chest.G)^2) 11
Regression Analysis The regression equation is Weight = 12.7-3.17 Chest.G + 0.212 (Chest.G)^2 Predictor Coef StDev T P Constant 12.65 35.19 0.36 0.720 Chest.G -3.166 1.901-1.67 0.098 (Chest.G)^2 0.21247 0.02484 8.56 0.000 S = 23.33 R-Sq = 95.6% R-Sq(adj) = 95.5% Analysis of Variance Source DF SS MS F P Regression 2 1659068 829534 1524.38 0.000 Residual Error 140 76185 544 Total 142 1735253 Regression Plot Y = 12.6535-3.16627X + 0.212473X**2 R-Sq = 95.6 % 500 400 Weight 300 200 100 0 Regression 95% CI 95% PI 20 30 40 50 Chest.G 12
100 50 RESI2 0-50 20 30 40 50 Chest.G Normal Probability Plot for RESI2 ML Estimates Percent 99 95 90 80 70 60 50 40 30 20 10 5 Mean: StDev: -0.0000000 23.0816 1-50 0 50 Data 13
Now we have eliminated the curved pattern in the residual plot quite efficiently But now we can see a clear non-constant variance: Residuals seem to be larger at larger predictor levels. (we have graphical evidence against the assumption: var(ε) = σ 2 at every level of X) One way to eliminate or weaken this kind of phenomenon is to transform the response via a variance stabilizing transformation. A typical one is the log(.). Maybe we should change our model again and consider: log(weight) = β 0 + β 1 Chest.G + β 2 Chest.G 2 + ε with ε indep. X, E(ε) = 0, var(ε) = σ 2 (again, create the variable log(weight) in Minitab through "Calc") Why should it "stabilize" the variability? It tends to "blow up" very small values, and "shrink" very large ones --in comparison. 14
Regression Analysis The regression equation is log(weight)= 0.309 + 0.0745 Chest.G - 0.000580 (Chest.G)^2 Predictor Coef StDev T P Constant 0.30873 0.09496 3.25 0.001 Chest.G 0.074508 0.005129 14.53 0.000 Chest.G^2-0.00057992 0.00006702-8.65 0.000 S = 0.06295 R-Sq = 94.3% R-Sq(adj) = 94.3% Analysis of Variance Source DF SS MS F P Regression 2 9.2369 4.6184 1165.61 0.000 Residual Error 140 0.5547 0.0040 Total 142 9.7916 Regression Plot Y = 0.308728 + 7.45E-02X - 5.80E-04X**2 R-Sq = 94.3 % 2.5 log(weight) 2.0 Regression 1.5 95% CI 95% PI 20 30 40 50 Chest.G 15
0.2 0.1 RESI3 0.0-0.1-0.2-0.3 20 30 40 50 Chest.G Normal Probability Plot for RESI3 ML Estimates Percent 99 95 90 80 70 60 50 40 30 20 10 5 Mean: StDev: 0.0000000 0.0622827 1-0.3-0.2-0.1 0.0 0.1 0.2 Data 16
Sign of the quadratic term in the parabola has changed passing to log of the Weight. Coeff. of determination has decreased slightly, but the non-constant variance in the residual plot has almost completely vanished. Notice there seems to be an outlier (a point far out from the bulk of the observations, that seems to "come from another population" and not to satisfy the relationship between response and predictor that is suggested by the other points). We could repeat the analysis eliminating the far out point. Also, we could separate females and males: We have not used the qualitative variable "sex" yet. Construct and estimate two separate regression models for males and females: If one obtains very similar models, based on the sample data the relationship between Weigt and Chest.G appears to be the same in the two sex-subpopulations. Might as well have one common model for both.. If one obtains very different models, based on the sample data the relationship between Weight ad Chest.G is sex-specific. Two separate models will fit the data much better than a common one. (in other words: is the gain in explanatory power and precision large enough to justify the fact that the hunter will carry along two tables/models, one for male and one for female bears?) Notice we are reasoning on how to introduce a qualitative variable in a (multiple) regression. 17