Simple Linear Regression: Introduction to Diagnostics

Transcription

1 Simple Linear Regression: Introduction to Diagnostics Anscombe Data Frame (1.973) A data frame proposed by Anscombe in and composed by 4 sets of 11 (x,y) points. All of them provide the same estimates for the intercept and the slope in the least squared line. Goodness of fit in terms of the coefficient of determination (R2) provided for the 4 sets is the same common value Data set are clearly different between them, a simple regression line is well-suited for data set A, but it is not suitable for sets B, C and D. Observations that are identified by outliers in its residual (ordinary residual or easier studentized residuals rstudent(model)), high leverage observations (hatvalues(model)) and influent data identified by atypical values in Cook s distance (cooks.distance(model)) are diagnostic indicators that are easily understood in simple regression and are very useful in the diagnosis of general multiple regression models estimated by least squares. XA YA XB YB XC YC XD YD 10 8, , ,46 8 6,58 8 6,95 8 8,14 8 6,77 8 5, , , ,74 8 7,71 9 8,81 9 8,77 9 7,11 8 8, , , ,81 8 8, , , ,84 8 7,04 6 7,24 6 6,13 6 6,08 8 5,25 4 4,26 4 3,10 4 5, , , , ,15 8 5,56 7 4,82 7 7,26 7 6,42 8 7,91 5 5,68 5 4,74 5 5,73 8 6,89 1 For every data set A to D plot and discuss results in the follow items: 1. Simple linear regression Y vs X. Write the equation. Plot residuals (studentized), leverages, Cook s distance (cooks.distance(model)). 2. Identify pattern in residuals. 3. Check the presence of outliers in residuals (boxplot or statistical distribution). 4. Check the presence of observations with high leverages. 5. Check the presence of observations that are influent data. 6. Copy scatterplots of (x,y) data and residuals vs fitted values. Draw the estimated regression line.

2 Y vs. X Residuals vs. Fitted values r = R 2 =. (recta ) = Y vs. X Residuals vs. Fitted values r = R 2 =. (recta ) =

3 Y vs. X Residuals vs. Fitted values r = R 2 =. (recta ) = Y vs. X Residuals vs. Fitted values r = R 2 =. (recta ) =

4 Some pieces of the script For example for data set C in the simple regression example of Anscombe73 data, the R script might be: # Joc C cor(anscombe$xc, anscombe$yc) # Calcula el coeficient de correlació lineal par(mfrow=c(1,1)) plot(anscombe$xc, anscombe$yc) # Diagrama bivariant, dades originals anscombe.lmc <- lm(anscombe$yc ~ anscombe$xc, data=anscombe) summary(anscombe.lmc) # Càlcul del model lineal simple : resultats de l ajust lines(anscombe$xc,anscombe.lmc$fitted.values) text(x=anscombe$xc,y=anscombe$yc,labels=row.names(anscombe), adj=1) # Sobrepasar al Diago. Bivariant Y vs X, la recta ajustada, # tot identificant les observacions pel seu id. par(mfrow=c(2,2)) plot(anscombe.lmc) # Gràfics de diagnosi standard par(mfrow=c(1,1)) levc <- hatvalues(anscombe.lmc) cooc <- cooks.distance(anscombe.lmc) tresc <- rstudent(anscombe.lmc) anscombe <- data.frame( anscombe, levc, cooc, tresc ) # Calcular: factor d anclatge, dist.cook i resid Student pel model C # guardant les columnas en el dataframe anscombe. A continuació venen #els plots estándar de tresid vs hii, tresid vs cooki tresid vs fitts # attributes(anscombe) plot(anscombe$levc,anscombe$tresc) text(x=anscombe$levc,y=anscombe$tresc,labels=row.names(anscombe), adj=1) plot(anscombe$cooc,anscombe$tresc) text(x=anscombe$cooc,y=anscombe$tresc,labels=row.names(anscombe), adj=1) plot(anscombe.lmc$fitted.values,anscombe$tresc) text(x=anscombe.lmc$fitted.values,y=anscombe$tresc,labels=row.names(anscombe ), adj=1) 4

5 Multiple regression: Yi = β 1+ β 2 X 2, i + + β p X p, i + ε i on ε i N 0, σ 2 ( ) independents Example: Duncan data on prestige of professions or weight vs height in Davis Study correlations between numeric variables appearing in the work space. Explicative variables are income and education. Response variables is prestige and we have to propose a multiple regression model to explain the prestige of jobs. 5 Suggested steps Correlation matrix in R, cor(duncan1, use="pairwise.complete.obs" ) Matrix of 2 by 2 scatterplots. Forward regression from the nul model with a direction forward option in method step(). > duncan1.lm0 <- lm( prestige ~1, data=duncan1)

6 > summary(duncan1.lm0) > step(duncan1.lm0, ~income+education, direction= forward, data=duncan1) Backward regression from the model with INCOME+EDUCATION in backward direction option in method step(). > duncan1.lm2 <- lm( prestige ~ income+education, data=duncan1) > summary(duncan1.lm2) > step(duncan1.lm2, direction= backward,data=duncan1) Use method step(.) in R from the nul model to the maximal model with direction specification both (it is the default) > duncan1.lml <- lm( prestige ~income+education, data=duncan1) > summary(duncan1.lm1) > duncan1.lm<- step(duncan1.lm1, ~income+education, data=duncan1) Linear correlation between a response variable and explicative variables might not be significative once some of the explicative variables are already included in the model. A touch on diagnostics: Check outliers in residuals and influent data in the selected model. Compute histogram of studentized residuals (rstudent(model)), leverage (hatvalues(model)) and Cook s distance (cooks.distance(model)). 1. R 2 and global regression test H : β = β Residual analysis: Detection of outliers. 0 2 = p = 6 Scatterplot of studentitzed residual vs. Y. Scatterplot of studentitzed residual vs. Y vs. X i. Detection of a priori and a posterior influent data. Scatterplot of studentitzed residual vs. leverage. Scatterplot of studentitzed residual vs. Cook s distance.

7 Example: weight vs height in Davis The Davis data frame has 200 rows and 5 columns. The subjects were men and women engaged in regular exercise. There are some missing data. This data frame contains the following columns: sex: A factor with levels: F, female; M, male. weight:measured weight in kg. height: Measured height in cm. r_weight : Reported weight in kg. r_height : Reported height in cm. 7 Firstly, we examine the relationship between the reported weight and the actual weight in order to assess how data behaves. Pay attention to outliers. Secondly, we focus on the classical relationship between weight (Y) and height (X): does a quadratic fit hold? Why?

8 Suggested steps Correlation matrix in R, cor(davis, use="pairwise.complete.obs" ) Matrix of 2 by 2 scatterplots. Multiple regression weight (Y) vs r_weight (Y). Interpret the regression equation and quality of the fit Multiple regression weight (Y) vs height (X). Interpret the regression equation and quality of the fit Multiple regression weight (Y) vs poly(height,2) (X). Can you Interpret the regression equation and quality of the fit? 8