Coefficient of Determination

Transcription

1 Coefficient of Determination ST 430/514 The coefficient of determination, R 2, is defined as before: R 2 = 1 SS E (yi ŷ i ) = 1 2 SS yy (yi ȳ) 2 The interpretation of R 2 is still the fraction of variance explained by the regression model. It measures the correlation between the dependent variable Y and the independent variables jointly; R 2 is also the (square of the) multiple correlation and is sometimes called the multiple R 2. 1 / 15 Multiple Linear Regression Coefficients of Determination

2 Adjusted Coefficient of Determination Because the regression model is adapted to the sample data, it tends to explain more variance in the sample data than it will in new data. Rewrite: 1 R 2 = SS E SS yy = 1 n 1 n (yi ŷ i ) 2 (yi ȳ) 2 Numerator and denominator are biased estimators of variance. 2 / 15 Multiple Linear Regression Coefficients of Determination

3 Replace 1 n with the multipliers that give unbiased variance estimators: 1 n p 1 n 1 (yi ŷ i ) 2 (yi ȳ), 2 where as before p = k + 1, the number of estimated βs. This defines the adjusted coefficient of determination: 1 (yi ŷ Ra 2 n p i ) 2 = 1 1 (yi ȳ) n 1 2 = 1 n 1 n p (yi ŷ i ) 2 (yi ȳ). 2 R 2 a < R 2, and for a poorly fitting model you may even find R 2 a < 0! 3 / 15 Multiple Linear Regression Coefficients of Determination

4 Looking Ahead... To assess how well a model will predict new data, you can use deleted residuals (see Section 8.6, The Jackknife): Delete one observation, say y i ; Refit the model, and use it to predict the deleted observation as ŷ (i) ; The deleted residual (or prediction residual) is d i = y i ŷ (i). More R 2 (see Section 5.11, External Model Validation): R 2 jackknife = 1 ( ) 2 ( ) 2 yi ŷ (i) (yi ȳ) 2, yi ŷ (i) P2 = 1 ( ) 2. yi ȳ (i) 4 / 15 Multiple Linear Regression Coefficients of Determination

5 A useful R function: PRESS <- function(l) { r <- residuals(l) sse <- sum(r^2) d <- r / (1 - hatvalues(l)) press <- sum(d^2) sst <- sse / (1 - summary(l)$r.squared) n <- length(r) ssti <- sst * (n / (n - 1))^2 c(stat = press, pred.rmse = sqrt(press / n), pred.r.square = 1 - press / sst, P.square = 1 - press / ssti) } 5 / 15 Multiple Linear Regression Coefficients of Determination

6 Estimation and Prediction ST 430/514 The multiple regression model may be used to make statements about the response that would be observed under a new set of conditions x new = (x 1,new, x 2,new,..., x k,new ). As before, the statement may be about: E(Y x = x new ), the expected value of Y under the new conditions; a single new observation of Y under the new conditions. 6 / 15 Multiple Linear Regression Estimation and Prediction

7 The point estimate of E(Y x = x new ) and the point prediction of Y when x = x new are both ŷ = ˆβ 0 + ˆβ 1 x 1,new + + ˆβ k x k,new. The standard errors are different because, as always, Y = E(Y ) + ɛ. 7 / 15 Multiple Linear Regression Estimation and Prediction

8 Example: prices of grandfather clocks Get the data and plot them: clocks = read.table("text/exercises&examples/gfclocks.txt", header = pairs(clocks[, c("price", "AGE", "NUMBIDS")]) Fit the first-order model and summarize it: clockslm = lm(price ~ AGE + NUMBIDS, clocks) summary(clockslm) PRESS(clocksLm) Check the residuals: plot(clockslm) 8 / 15 Multiple Linear Regression Estimation and Prediction

9 Predictions are for an auction of a 150-year old clock with 10 bidders. 95% confidence interval for E(Y AGE = 150, NUMBIDS = 10): predict(clockslm, newdata = data.frame(age = 150, NUMBIDS = 10), interval = "confidence", level =.95) 95% prediction interval for Y when AGE = 150, NUMBIDS = 10: predict(clockslm, newdata = data.frame(age = 150, NUMBIDS = 10), interval = "prediction", level =.95) 9 / 15 Multiple Linear Regression Estimation and Prediction

10 Interaction Models One property of the first-order model E(Y ) = β 0 + β 1 x 1 + β 2 x β k x k is that β i is the change in E(Y ) as x i increases by 1 with all the other independent variables held fixed, and is the same regardless of the values of those other variables. Not all real-world situations work like that. When the magnitude of the effect of one variable is affected by the level of another, we say that they interact. 10 / 15 Multiple Linear Regression Interaction Model

11 A simple model for two factors that interact is E(Y ) = β 0 + β 1 x 1 + β 2 x 2 + β 3 x 1 x 2. Rewrite this in two ways: E(Y ) = β 0 + (β 1 + β 3 x 2 )x 1 + β 2 x 2 = β 0 + β 1 x 1 + (β 2 + β 3 x 1 )x 2. Holding x 2 fixed, the slope of E(Y ) against x 1 is β 1 + β 3 x 2. Holding x 1 fixed, the slope of E(Y ) against x 2 is β 2 + β 3 x / 15 Multiple Linear Regression Interaction Model

12 We can fit this using k = 3: if we set x 3 = x 1 x 2. E(Y ) = β 0 + β 1 x 1 + β 2 x 2 + β 3 x 3 Example: grandfather clocks again. clockslm2 <- lm(price ~ AGE * NUMBIDS, clocks) summary(clockslm2) Note: the formula PRICE AGE * NUMBIDS specifies the interaction model, which includes the separate effects of AGE and NUMBIDS, together with their product, which will be labeled AGE:NUMBIDS. 12 / 15 Multiple Linear Regression Interaction Model

13 Output Call: lm(formula = PRICE ~ AGE * NUMBIDS, data = clocks) Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) AGE NUMBIDS ** AGE:NUMBIDS e-06 *** --- Signif. codes: 0 *** ** 0.01 * Residual standard error: on 28 degrees of freedom Multiple R-squared: , Adjusted R-squared: F-statistic: 193 on 3 and 28 DF, p-value: < 2.2e / 15 Multiple Linear Regression Interaction Model

14 Note that the t-value on the AGE:NUMBIDS line is highly significant. That is, we strongly reject the null hypothesis H 0 : β 3 = 0. Effectively, this test is a comparison of the interaction model with the original non-interactive (additive) model. The other two t-statistics are usually irrelevant: if AGE:NUMBIDS is important, then both AGE and NUMBIDS should be included in the model; do not test the corresponding null hypotheses. 14 / 15 Multiple Linear Regression Interaction Model

15 We can write the fitted model as E(PRICE) = ( NUMBIDS) AGE 93 NUMBIDS meaning that the effect of age increases with the number of bidders. Check the model: plot(clockslm2) More satisfactory than the additive model. 15 / 15 Multiple Linear Regression Interaction Model