Regression Examples in R

Transcription

1 Eric F. Lock UMN Division of Biostatistics, SPH 09/06/2018

2 Gifted Children Study An investigator is interested in understanding the relationship, if any, between the analytical skills of young gifted children and age in month when the child first counted to 10 successfully. The analytical skills are evaluated using a standard testing procedure, and the score on this test is used as the response variable. Data were collected from schools in a large city on a set of thirty-six children who were identified as gifted children soon after they reached the age of four. Original Source: Graybill, F.A. & Iyer, H.K., (1994) Regression Analysis: Concepts and Applications, Duxbur, p library(openintro) data(gifted)

3 Scatterplot of Gifted Child Study plot(gifted$count, gifted$score, pch = 19, xlab = "Age (months) when Child Counted to 10", ylab = "Score in Test of Analytical Skills") Score in Test of Analytical Skills Age (months) when Child Counted to 10

4 Some Notation and a Trend Line Let Y i be the score on the test of analytical skills test for i th subject Let X i be the age when the i th child first started counting to 10 Suppose we want to characterize the relationship between age at which a gifted child starting counting to 10 and their score on the skills test. We might assume that Y i = β 0 + β 1 X i + i (1) The goal is to ESTIMATE β 0 and β 1 and make inferences on those parameters

5 Key Assumptions To ESTIMATE β 0 and β 1 and make inferences on those parameters we must make some assumptions. Note that these assumptions imply a statistical model. L I N E

6 Output from R model1 <- lm(score count, data = gifted) summary(model1) Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) < 2e-16 *** count *** --- Signif. codes: 0 *** ** 0.01 * Residual standard error: on 34 degrees of freedom Multiple R-squared: , Adjusted R-squared: F-statistic: on 1 and 34 DF, p-value:

7 Interpretation of Model Parameters How would you interpret for the count variable the estimate, std. error, p-value? What is the interpretation of R 2? What did the software do to get the output in the previous slide?... How can we assess the stated assumptions?

8 Checking Assumptions par(mfrow=c(2,2)) plot(model1, ask=f) Residuals vs Fitted Normal Q Q Residuals Standardized residuals Fitted values Theoretical Quantiles Standardized residuals Scale Location Standardized residuals Residuals vs Leverage Cook's distance Fitted values Leverage

9 What if certain assumptions are not satisfied? For example, consider the following dataset which has data on all flights departing NYC in library(nycflights13) data(flights) Here we focus on Delta flights out of LGA in August. library(dplyr) flights_a <- filter(flights, origin == "LGA", month == 8, carrier == "DL", year == 2013) We want to relate the expected departure delay to the departure time. flights_a$sched_dep_hour <- floor(flights_a$sched_dep_time/100)

10 Plot of data boxplot(dep_delay sched_dep_hour, data=flights_a, ylim=c(-20,50),xlab="scheduled Hour of Departure", ylab= "Delay") Delay Scheduled Hour of Departure

11 Plot of data mean_delay <- tapply(flights_a$dep_delay, flights_a$sched_dep_hour, mean) points(1:16, mean_delay, col="red", pch=19) Delay Scheduled Hour of Departure

12 Violation of Assumptions What are possible violations to each of the assumptions in LINE? How could we change our statistical model to accommodate this?

13 Gifted Children Study Adjustment for Additional Covariates Suppose we are interested in the effect of nurture average number of hours per week the child s mother or father reads to them average number of hours per week the child watched an educational program on TV during the past three months average number of hours per week the child watched cartoons on TV during the past three months on the score for analytical skills but want to adjust for nature covariates including mother IQ and father IQ Within a regression setting how can we adjust for other covariates? How can we graphically assess the relationship between covariates and outcome?

14 Scatterplot Between Covariates and Outcome plot(gifted[,c("score","motheriq","fatheriq", "read","edutv","cartoons")]) score motheriq fatheriq read edutv cartoons

15 Gifted Children Study Proposed Model One possible model to relate the nurture variables to the score for analytical skills is the following: Y i = β 0 +β 1 Read i +β 2 edutv i +β 3 cartoons i +β 4 miq i +β 5 fiq i + i This of course does not fully specify a statistical model Again what assumptions do we typically make to fully specify the statistical model?

16 Gifted Children Study Output from R model2 <- lm(score motheriq + fatheriq + read + edutv + cartoons, data = gifted) summary(model2) Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) ** motheriq e-06 *** fatheriq read e-06 *** edutv * cartoons Residual standard error: 2.58 on 30 degrees of freedom Multiple R-squared: , Adjusted R-squared: F-statistic: on 5 and 30 DF, p-value: 7.669e-08

17 Gifted Children Study Interpretation of Model Parameters How would you interpret for the read variable row the estimate, std. error, p-value? What is the interpretation of R 2? Is there anyway to determine the relative importance of each covariate? What did the software do to get the output in the previous slide?... How can we assess the stated assumptions?

18 Gifted Children Study Checking Assumptions par(mfrow=c(2,2)) plot(model2, which=c(1:3,5), ask=f) Residuals vs Fitted Normal Q Q Residuals Standardized residuals Fitted values Theoretical Quantiles Standardized residuals Scale Location Standardized residuals Residuals vs Leverage Cook's distance Fitted values Leverage

19 Spam Study: Non continuous outcomes Many statistical tests to relate binary covariates to binary outcomes (most well-known is the Pearson Chi-Squared test) One way to relate a continuous covariate to a binary outcome while adjusting for other covariates is through the use of logistic regression A motivating example that we will consider is to predict the probability that an is spam based on a dataset of s. library(openintro) data( )

20 Spam Study: Covariates included in the Model to multiple Indicator for whether the was addressed to more than one recipient. winneryes: Indicates whether winner appeared in the . format: Indicates whether the was written using HTML. re subj: Whether the subject started with Re:, RE:, re:, or re: exclaim subj: Whether there was an exclamation point in the subject. cc: Indicator for whether anyone was CCed. attach: Indicator for attached files. dollar: Indicator for dollar sign or the word dollar appeared in the . inherit: Indicator for whether inherit (or an extension, such as inheritance ) appeared in the . password: Indicator for whether password appeared in the .

21 Spam Study: A natural extension of the previous linear models? Define Y i to be the indicator for whether message i is spam Define X i to be a covariate on the i th message We could consider the same model as before Y i = β 0 + β 1 X i + i What are some limitations of this approach?

22 Spam Study: A model for the odds Instead we model the log odds as a linear combination of the covariates What are odds? What is the possible values of log odds? If we only had one covariate the model would be log(odds) = β 0 + β 1 X i What happened to the outcome Y i? What happened to the error term?

23 Spam Study: Code for R Convert predictors to 0-1 values: e <- e$cc <- ifelse( $cc > 0, 1, 0) e$attach <- ifelse( $attach > 0, 1, 0) e$dollar <- ifelse( $dollar > 0, 1, 0) e$inherit <- ifelse( $inherit > 0, 1, 0) e$password <- ifelse( $password > 0, 1, 0) Fit model: g <- glm(spam to_multiple + winner + format + re_subj + exclaim_subj + cc + attach + dollar + inherit + password, data=e, family=binomial) summary(g)

24 Spam Study: Model Output from R ## Estimate Std. Error z value Pr(> z ) ## (Intercept) e-18 ## to_multiple e-20 ## winneryes e-07 ## format e-37 ## re_subj e-15 ## exclaim_subj e-01 ## cc e-01 ## attach e-06 ## dollar e-01 ## inherit e-01 ## password e-03

25 Spam Study: Interpretation of Model Parameters How would you interpret the coefficient (e.g. estimate) for the to multiple covariate? Should you transform it? What is the interpretation of the p-value for this covariate? How could you construct at 95% confidence interval for the transformation of β 1? Is there anyway to determine the relative importance of each covariate? What did the software do to get the output in the previous slide?... How can we assess the stated assumptions?...