lm statistics Chris Parrish

Transcription

1 lm statistics Chris Parrish Contents s e and R 2 1 experiment experiment experiment experiment experiment experiment conclusions s e and R 2 Regression problems are framed by imagining two numerical population variables x and y related to each other by an equation of the form y = β 0 + β 1 x + ɛ. Here β 0 and β 1 are the y-intercept and slope of the regression line and ɛ Normal(0, σ 2 ) expresses the fact that there is a random component to the values of y. Linear models calculated on random samples from the population, y = b 0 + b 1 x, produce statistics b 0 and b 1 which capture information about the parameters β 0 and β 1, and s e and R 2 which measure how well the data in the sample matches the model. The e in s e stands for errors, or residuals, and is an estimator of σ, e i = y i ŷ i, s e = e 2 i n 2 s e = ˆσ R 2 is the proportion of the variation in y that is explained by the linear model (EPS, p.529). We would like to perform some experiments illustrating the meaning of s e and R 2. 1

2 experiment1 Start with a horizontal line. Load package. library(ggplot2) Assemble the data. xs <- seq(from = 0, to = 10, by = 0.01) beta0 <- 0 beta1 <- 0 sigma <- 1 data <- data.frame(x = xs, y = rnorm(1001, 0, sigma)) illustration ggplot(data, aes(x, y)) + geom_point(shape = 20, color = "darkred") + geom_smooth(method = "lm") 2 0 y 2 Statistics x options(show.signif.stars = FALSE) lm1 <- lm(y ~ x, data = data) summary(lm1) 2

3 Call: lm(formula = y ~ x, data = data) Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) x Residual standard error: on 999 degrees of freedom Multiple R-squared: , Adjusted R-squared: F-statistic: on 1 and 999 DF, p-value: observations What values do you expect to see for b 0 and b 1? Why? What values do you actually see for b 0 and b 1? data.frame(lm = 1, b0 = as.numeric(lm1$coefficients[1]), b1 = as.numeric(lm1$coefficients[2])) lm b0 b What values do you expect to see for s e and R 2? Why? What do you actually see for s e and R 2? data.frame(lm = 1, s.e = summary(lm1)$sigma, R.sq = summary(lm1)$r.squared) lm s.e R.sq experiment2 Design and run two more experiments in which the data is just as for experiment 1 except that epsilon is set to 2 and then to 3. Comment on s e and R 2. beta0 <- 0 beta1 <- 0 sigma <- 2 data <- data.frame(x = xs, y = rnorm(1001, 0, sigma)) 3

4 illustration ggplot(data, aes(x, y)) + geom_point(shape = 20, color = "darkred") + geom_smooth(method = "lm") y Statistics x lm2 <- lm(y ~ x, data = data) summary(lm2) Call: lm(formula = y ~ x, data = data) Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) x Residual standard error: on 999 degrees of freedom Multiple R-squared: , Adjusted R-squared: F-statistic: on 1 and 999 DF, p-value:

5 observations What values do you expect to see for s e and R 2? Why? What do you actually see for s e and R 2? data.frame(lm = 2, s.e = summary(lm2)$sigma, R.sq = summary(lm2)$r.squared) lm s.e R.sq experiment3 beta0 <- 0 beta1 <- 0 sigma <- 3 data <- data.frame(x = xs, y = rnorm(1001, 0, sigma)) illustration ggplot(data, aes(x, y)) + geom_point(shape = 20, color = "darkred") + geom_smooth(method = "lm") 5

6 10 5 y Statistics x lm3 <- lm(y ~ x, data = data) summary(lm3) Call: lm(formula = y ~ x, data = data) Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) x Residual standard error: on 999 degrees of freedom Multiple R-squared: 6.353e-05, Adjusted R-squared: F-statistic: on 1 and 999 DF, p-value: observations What values do you expect to see for s e and R 2? Why? What do you actually see for s e and R 2? 6

7 data.frame(lm = 3, s.e = summary(lm3)$sigma, R.sq = summary(lm3)$r.squared) lm s.e R.sq e-05 experiment4 In experiment 4, we set β 0 = 0 and β 1 = 1 and we reset ɛ = 1. beta0 <- 0 beta1 <- 1 sigma <- 1 data <- data.frame(x = xs, y = xs + rnorm(1001, 0, sigma)) illustration ggplot(data, aes(x, y)) + geom_point(shape = 20, color = "darkred") + geom_smooth(method = "lm") 12 8 y 4 0 Statistics x 7

8 lm4 <- lm(y ~ x, data = data) summary(lm4) Call: lm(formula = y ~ x, data = data) Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) x <2e-16 Residual standard error: on 999 degrees of freedom Multiple R-squared: 0.891, Adjusted R-squared: F-statistic: 8167 on 1 and 999 DF, p-value: < 2.2e-16 observations What values do you expect to see for b 0 and b 1? Why? What values do you actually see for b 0 and b 1? data.frame(lm = 4, b0 = as.numeric(lm4$coefficients[1]), b1 = as.numeric(lm4$coefficients[2])) lm b0 b What values do you expect to see for s e and R 2? Why? What do you actually see for s e and R 2? data.frame(lm = 4, s.e = summary(lm4)$sigma, R.sq = summary(lm4)$r.squared) lm s.e R.sq experiment5 Design and run two more experiments in which the data is just as for experiment 4 except that epsilon is set to 2 and then to 3. beta0 <- 0 beta1 <- 1 sigma <- 2 data <- data.frame(x = xs, y = xs + rnorm(1001, 0, sigma)) 8

9 illustration ggplot(data, aes(x, y)) + geom_point(shape = 20, color = "darkred") + geom_smooth(method = "lm") 10 y Statistics x lm5 <- lm(y ~ x, data = data) summary(lm5) Call: lm(formula = y ~ x, data = data) Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) x <2e-16 Residual standard error: on 999 degrees of freedom Multiple R-squared: , Adjusted R-squared: F-statistic: 2042 on 1 and 999 DF, p-value: < 2.2e-16 9

10 observations What values do you expect to see for s e and R 2? Why? What do you actually see for s e and R 2? data.frame(lm = 5, s.e = summary(lm5)$sigma, R.sq = summary(lm5)$r.squared) lm s.e R.sq experiment6 beta0 <- 0 beta1 <- 1 sigma <- 3 data <- data.frame(x = xs, y = xs + rnorm(1001, 0, sigma)) illustration ggplot(data, aes(x, y)) + geom_point(shape = 20, color = "darkred") + geom_smooth(method = "lm") 10

11 10 y 0 Statistics x lm6 <- lm(y ~ x, data = data) summary(lm6) Call: lm(formula = y ~ x, data = data) Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) x <2e-16 Residual standard error: on 999 degrees of freedom Multiple R-squared: , Adjusted R-squared: F-statistic: on 1 and 999 DF, p-value: < 2.2e-16 observations What values do you expect to see for s e and R 2? Why? What do you actually see for s e and R 2? 11

12 data.frame(lm = 6, s.e = summary(lm6)$sigma, R.sq = summary(lm6)$r.squared) lm s.e R.sq conclusions Summarize these experiments by defining s e and R 2 in your own words. 12