School of Mathematical Sciences MTH5120 Statistical Modelling I Practical 8 and Assignment 7 Solutions Question 1 Figure 1: The residual plots do not contradict the model assumptions of normality, constant variance and linearity. Regression Analysis: Y versus X1, X2, X3, X4, X5 Y = - 2.35 + 1.73 X1-0.00887 X2 + 0.180 X3 + 0.0140 X4 + 0.0620 X5 Constant -2.3521 0.7732-3.04 0.005 X1 1.7256 0.4325 3.99 0.000 X2-0.008866 0.002882-3.08 0.004 X3 0.1799 0.1151 1.56 0.128 X4 0.013982 0.007252 1.93 0.063 X5 0.06203 0.05176 1.20 0.240 S = 0.366788 R-Sq = 91.3% R-Sq(adj) = 89.9% Regression 5 45.1395 9.0279 67.11 0.000 Residual Error 32 4.3051 0.1345 Total 37 49.4446 X1 1 42.4219 X2 1 1.7170 X3 1 0.5000 X4 1 0.3073 X5 1 0.1932 1
Comments: T-tests: The hypotheses are H 0 : β i = 0, that is, that the coefficient of x i is zero given all other explanatory variables are in the model. The p-values for testing β 1 and β 2 are smaller than 0.01, hence we can reject the null hypothesis for each of these two parameters. The intercept is also highly significant. The p-value for testing β 4 is 0.063, hence we can reject the null hypothesis on the significance level α = 0.1 but not at α = 0.05. There is weak evidence against the null hypothesis. The p-values for testing β 3 and β 5 are larger than 0.1, hence there is no evidence against the null hypotheses, neither for β 3 nor for β 5. H 0 : β 1 = β 2 = β 3 = β 4 = β 5 versus H 1 : H 0 can be rejected at α = 0.001. That is, the overall regression is highly significant. It means that some of the explanatory variables explain a significant part of the variability of the response variable. The sequential sums of squares show that SS(β 1 ) is much larger than SS(β 2 β 1 ), that is, adding X 2 to the model given that X 1 is already in the model does not increase the regression sum of squares very much. Also SS(β 3 β 1, β 2 ) is very small and so adding X 3 to the model given that X 1 and X 2 are already there will not increase the SS R by a large amount. Similarly, SS(β 4 β 1, β 2, β 3 ) and SS(β 5 β 1, β 2, β 3, β 4 ) are very small. However, we do need to check their statistical significance. Figure 2: The matrix plot shows that there is a clear relationship between Y and each of X 1 - X 4, however not X 5. Apart from X 5, which seems not to be related to any of the variables, we can see that all explanatory variables show some dependencies among themselves. This may make the model fitting more difficult.. 2
Figure 3: The residual plots do not indicate any contradiction to the assumptions of normality, constant variance and linearity. Regression Analysis: Y versus X1, X2, X4 Y = - 1.33 + 2.09 X1-0.00781 X2 + 0.0105 X4 Constant -1.3313 0.4538-2.93 0.006 X1 2.0900 0.3488 5.99 0.000 X2-0.007812 0.002256-3.46 0.001 X4 0.010453 0.005898 1.77 0.085 S = 0.377956 R-Sq = 90.2% R-Sq(adj) = 89.3% Regression 3 44.588 14.863 104.04 0.000 Residual Error 34 4.857 0.143 Total 37 49.445 X1 1 42.422 X2 1 1.717 X4 1 0.449 Comments: We have three predictors in the model. The p-values for testing β 1 and β 2 are smaller than 0.01, hence we can reject the null hypothesis for each of these parameters. The intercept is also highly significant. However, the p-value for β 4 shows that this parameter is weakly significant. H 0 : β 1 = β 2 = β 4 = 0 versus H 1 : H 0 can be rejected at α = 0.001, that is, the overall regression is highly significant. This means that at least one of the three predictors explains a significant part of the variability of the response variable. As before, the sequential sums of squares show that SS(β 1 ) is much larger than SS(β 2 β 1 ), that is, adding X 2 to the model given that X 1 is already in the model, does not increase the regression sum of squares very much. SS(β 4 β 1, β 2 ) is smaller than SS(β 2 β 1 ). Although the statistical significance of the parameters needs to be tested. 3
Figure 4: The residual plots do not contradict the model assumptions of normality, constant variance and linearity. Regression Analysis: Y versus X1, X2 Y = - 1.30 + 2.45 X1-0.00782 X2 Constant -1.2961 0.4670-2.78 0.009 X1 2.4497 0.2922 8.38 0.000 X2-0.007821 0.002324-3.37 0.002 S = 0.389345 R-Sq = 89.3% R-Sq(adj) = 88.7% Regression 2 44.139 22.069 145.59 0.000 Residual Error 35 5.306 0.152 Total 37 49.445 X1 1 42.422 X2 1 1.717 Comments: The p-values for testing β 1 and β 2 are smaller than 0.01, hence we can reject the null hypothesis for each of these parameters. The intercept is also highly significant. H 0 : β 1 = β 2 = 0 versus H 1 : H 0 can be rejected at α = 0.001, that is, the overall regression is highly significant. This means that at least one of the two explanatory variables explains a significant part of the variability of the response variable. As before, the sequential sums of squares show that SS(β 1 ) is much larger than SS(β 2 β 1 ), that is, adding X 2 to the model given that X 1 is already in the model, does not increase the regression sum of squares very much, but again, the statistical significance of the parameters needs to be tested. 4
Question 2 The question is: can we reduce the set of regressors X 1, X 2,..., X p 1 to, say, X 1, X 2,..., X q 1 (renumbering if necessary) where q < p, by omitting X q, X q+1,..., X p 1? This can be done by testing the hypothesis versus The test function is H 0 : β q = β q+1 =... = β p 1 = 0 H 1 : H 0. F = SS extra/(p q) MS E H0 F p q,n p, where SS extra = SS R SSR red; SS R, MS E are the sums of squares obtained in the full model, SSR red is the regression sum of squares obtained in the reduced model. (a) Here we compare two models: the full model including all explanatory variables X 1, X 2, X 3, X 4, X 5 with the reduced model including only X 1, X 2 and X 4. The number of parameters in the reduced model is q = 4 and in the full model is p = 6, that is, we have p q = 2. The null and alternative hypotheses are H 0 : β 3 = β 5 = 0 versus H 1 : H 0. To obtain the value of the test statistic we calculate: SS extra = SS R SS red R = 45.1395 44.588 = 0.5515 F obs = 0.5515/2 0.1345 = 2.05019. Comparing this value with the critical value of F α;p q,n p = F 0.1;2,32 = 2.47651 we see that there is no evidence to reject the null hypothesis. (b) Here we compare two models: the full model including all explanatory variables X 1, X 2, X 3, X 4, X 5 with the reduced model including only X 1 and X 2. Here q = 3 and p = 6, that is, p q = 3. The null and alternative hypotheses are H 0 : β 3 = β 4 = β 5 = 0 versus H 1 : H 0. To obtain the value of the test statistic we calculate: SS extra = SS R SS red R = 45.1395 44.139 = 1.005 F obs = 1.005/3 0.1345 = 2.47955. We have F 0.1;3,32 = 2.26345 and F 0.05;3,32 = 2.90112. Hence we would reject the null hypothesis at the significance level α = 0.1 but not at α = 0.05. There is weak evidence against the null hypothesis. It is not very clear which model would be better for fitting the petrol consumption. The X 4 predictor is not highly significant when X 1 and X 2 are in the model, also there is only weak evidence against the null hypothesis H 0 : β 3 = β 4 = β 5 = 0 while there is no evidence against H 0 : β 3 = β 5 = 0. The values of R 2 adjusted are not much different, both are high. However, the model with X 1 and X 2 only is more parsimonious. 5