Regression. Bret Larget. Department of Statistics. University of Wisconsin - Madison. Statistics 371, Fall Correlation Plots

Transcription

1 Correlation Plots r = 0.97 Regression Bret Larget Department of Statistics Statistics 371, Fall Universit of Wisconsin - Madison Correlation Plots r = 0.21 Correlation December 2, 2004 The correlation coefficient r is measure of the strength of the linear relationship between two variables. r = 1 n ( ) i ( ) i ȳ n 1 i=1 s s (i )( = i ȳ) (i ) 2 ( i ȳ) 2 Notice that the correlation is not affected b linear transformations of the data (such as changing the scale of measurement). Statistics 371, Fall 2004 Statistics 371, Fall Statistics 371, Fall

2 Correlation Plots Correlation Plots r = 1 r = Statistics 371, Fall Statistics 371, Fall Correlation Plots Correlation Plots r = 0 r = Statistics 371, Fall Statistics 371, Fall

3 Summar of Correlation Correlation Plots The correlation coefficient r measures the strength of the linear relationship between two quantitative variables, on a scale from 1 to 1. The correlation coefficient is 1 or 1 onl when the data lies perfectl on a line with negative or positive slope, respectivel. If the correlation coefficient is near one, this means that the data is tightl clustered around a line with a positive slope. Correlation coefficients near 0 indicate weak linear relationships. However, r does not measure the strength of nonlinear relationships. If r = 0, rather than X and Y being unrelated, it can be the case that the have a strong nonlinear relationshsip. If r is close to 1, it ma still be the case that a nonlinear relationship is a better description of the data than a linear relationship. Statistics 371, Fall r = Statistics 371, Fall Simple Linear Regression Correlation Plots Simplelinearregression isthestatisticalprocedurefordescribing the relationship between an quantitative eplanator variable X and a quantitative response variable Y with a straight line. In simple linear regression, the regression line is the line that minimizes the sum of the squared residuals r = Statistics 371, Fall Statistics 371, Fall

4 Rile Rile Rile Larget is m son. Below is a plot of his height versus his age, from birth to 8 ears. Height (inches) Age (months) Height (inches) Age (months) Statistics 371, Fall Statistics 371, Fall Finding a best linear fit An line we can use to predict Y from X will have the form Y = b 0 + b 1 X where b 0 is the intercept and b 1 will be the slope. The value ŷ = b 0 + b 1 is the predicted value of Y if the eplanator variable X =. In simple linear regression, the predicted values form a line. (In more advanced forms of regression, we can fit curves or fit functions of multiple eplanator variables.) For each data point ( i, i ), the residual is the difference between the observed value and the predicted value, i ŷ i. Graphicall, each residual is the positive or negative vertical distance from the point to the line. Simple linear regression identifies the line that minimizes the n residual sum of squares, ( i ŷ i ) 2. i=1 Rile The plot indicates that it is not reasonable to model the relationshipbetweenageandheight as linearovertheentireage range, but it is fairl linear from age 2 ears to 8 ears (24 96 months). Statistics 371, Fall Statistics 371, Fall

5 The General Case Let X = + zs, so X is z standard deviations above the mean. ŷ = b 0 + b 1 ( + zs ) = (ȳ b 1 ) + b 1 + b 1 zs ( = ȳ + r s ) zs s = ȳ + (rz) s Least Squares Regression We won t derive them, but there are simple formulas for the slope and intercept of the least squares line as a function of the sample means, standard deviations, and the correlation coefficient. Y = b 0 + b 1 X b 1 = r s s b 0 = ȳ b 1 Notice that if X is z SDs above the mean, we predict Y to be onl rz SDs above the mean. In the tpical situation, r < 1, so we predict the value of Y to beclosertothemean (instandardunits) than X. This iscalled the regression effect. Statistics 371, Fall Statistics 371, Fall Rile (cont.) > n = length(age2) > m = mean(age2) > s = sd(age2) > m = mean(height2) > s = sd(height2) > r = cor(age2, height2) > print(c(m, s, m, s, r, n)) [1] > b1 = r * s/s > b0 = m - b1 * m > print(c(b0, b1)) [1] (Rile s predicted height in inches) = (Rile s age in months) A Special Case Consider the predicted value of an observation X =. ŷ = b 0 + b 1 = (ȳ b 1 ) + b 1 = ȳ So, the regression line alwas goes through the point (, ȳ). Statistics 371, Fall Statistics 371, Fall

6 Rile A Residual Plot > plot(fitted(fit2), residuals(fit2)) > abline(h = 0) Using R > age2 = age[(age > 23) & (age < 97)] > height2 = height[(age > 23) & (age < 97)] > fit2 = lm(height2 ~ age2) > summar(fit2) Call: lm(formula = height2 ~ age2) residuals(fit2) Residuals: Min 1Q Median 3Q Ma Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) <2e-16 *** age <2e-16 *** --- Signif. codes: 0 *** ** 0.01 * fitted(fit2) Residual standard error: on 14 degrees of freedom Multiple R-Squared: , Adjusted R-squared: F-statistic: 7627 on 1 and 14 DF, p-value: < 2.2e-16 Statistics 371, Fall Statistics 371, Fall Rile Interpretation Rile Plot of Data We can interpret the slope to mean that from age 2 to 8 ears, Rile grew an average of about 0.25 inches per ear month, or about 3 inches per ear. The intercept is the predicted value when X = 0, or Rile s height (length) at birth. This interpretation ma not be reasonable if 0 is out of the range of the data. Height (inches) Age (months) Statistics 371, Fall Statistics 371, Fall

7 Residual Standard Deviation Rile Etrapolation The residual standard deviation, s Y X, is a measure of a tpical size of a residual. Its formula is s Y X = SS(resid) n 2 Notice that in simple linear regression, there are n 2 degrees of freedom, in contrast to our formula n 1. Height (inches) Thereasonisthat ourmodel for themean usestwo parameters it takes two points to determine a line Age (months) Statistics 371, Fall Statistics 371, Fall Frogs Rile Etrapolation In a stud on ooctes (developing egg cells) from the frog Xenopus laevis, a biologist injects individual ooctes from the same female with radioactive leucine, to measure the amount of leucine incorporated into protein as a function of time. (See Eercise 12.3 on page 536.) Height (inches) Age (months) Statistics 371, Fall Statistics 371, Fall

8 R Fitting a model > fit = lm(leucine ~ time) > summar(fit) Call: lm(formula = leucine ~ time) Residuals: Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) time e-06 *** --- Signif. codes: 0 *** ** 0.01 * Residual standard error: on 5 degrees of freedom Multiple R-Squared: , Adjusted R-squared: F-statistic: 340 on 1 and 5 DF, p-value: 8.628e-06 R Entering the data Here is some R code to create a data frame frog with the time and leucine variables. > frog = data.frame(time = seq(0, 60, b = 10), leucine = c(0.02, , 0.54, 0.69, 1.07, 1.5, 1.74)) > attach(frog) > frog time leucine Statistics 371, Fall Statistics 371, Fall R Verifing tetbook formulae > n = nrow(frog) > m = mean(time) > s = sd(time) > m = mean(leucine) > s = sd(leucine) > r = cor(time, leucine) > c(m, s, m, s, r) [1] > b1 = r * s/s > b0 = m - b1 * m > s = sqrt(sum(residuals(fit)^2)/(n - 2)) > seb1 = s/s/sqrt(n - 1) > ts = b1/seb1 > c(b0, b1, s, seb1, ts) [1] R Plotting the data > plot(time, leucine) leucine time Statistics 371, Fall Statistics 371, Fall

9 Fitting a Quadratic Model We add an additional parameters to fit a curve. Fitting a quadratic curve is a reasonable choice (unless there was a scientific reason to fit an eponential curve, for eample). R A Residual Plot > plot(fitted(fit), residuals(fit), lab = "Fitted Values", lab = "Residuals") > abline(h = 0) Here is a quadratic model. (leucine) = β 0 + β 1 (time) + β 2 (time) 2 The residual sum of squares will cannot be larger when fitting a quadratic model instead of a linear model lines are special cases of parabolas. But, we want to be careful not to overfit. Residuals Fitted Values There might be some nonlinearit.... Statistics 371, Fall Statistics 371, Fall R Fitting a Curve > fit2 = lm(leucine ~ time + I(time^2)) > summar(fit2) Call: lm(formula = leucine ~ time + I(time^2)) R Plotting the regression line > plot(time, leucine, pch = 16) > abline(fit) Residuals: Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) 2.452e e time 2.061e e * I(time^2) 1.441e e Signif. codes: 0 *** ** 0.01 * Residual standard error: on 4 degrees of freedom Multiple R-Squared: , Adjusted R-squared: F-statistic: on 2 and 4 DF, p-value: 5.359e-05 leucine time Statistics 371, Fall Statistics 371, Fall

10 Tree Eample We want to be able to estimate the age of trees in the Amazon b their diameter. Use the carbon dating as a pro for the actual age. (See eercise on page 575.) R Plotting the Curve > plot(time, leucine, pch = 16) > abline(fit) > new = predict(fit2, data.frame(time = 0:60)) > lines(0:60, new, lt = 2) leucine time Statistics 371, Fall Statistics 371, Fall Tree Eample > amazon = read.table("amazon.tt", header = T) > attach(amazon) > amazon diameter age Statistics 371, Fall R Residual Plot for Quadratic Model > plot(fitted(fit2), residuals(fit2), lab = "Fitted Values", lab = "Residuals") > abline(h = 0) Residuals Fitted Values Statistics 371, Fall

11 Amazon Residual Plot Amazon Plot > plot(fitted(fit), residuals(fit), lab = "Fitted Values", lab = "Residuals") > abline(h = 0) > plot(diameter, age, pch = 16) Residuals age Fitted Values diameter Statistics 371, Fall Statistics 371, Fall Amazon Normal Probabilit Plot Amazon Fitting a Line > qqnorm(residuals(fit)) Normal Q Q Plot > fit = lm(age ~ diameter) > summar(fit) Call: lm(formula = age ~ diameter) Sample Quantiles Residuals: Min 1Q Median 3Q Ma Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) diameter * --- Signif. codes: 0 *** ** 0.01 * Residual standard error: on 18 degrees of freedom Multiple R-Squared: , Adjusted R-squared: F-statistic: on 1 and 18 DF, p-value: Theoretical Quantiles Statistics 371, Fall Statistics 371, Fall

12 Amazon Normal Probabilit Plot > qqnorm(residuals(fit2)) Normal Q Q Plot Amazon Log Transformation > fit2 = lm(log(age) ~ diameter) > summar(fit2) Call: lm(formula = log(age) ~ diameter) Sample Quantiles Residuals: Min 1Q Median 3Q Ma Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) e-09 *** diameter * --- Signif. codes: 0 *** ** 0.01 * Theoretical Quantiles Residual standard error: on 18 degrees of freedom Multiple R-Squared: , Adjusted R-squared: F-statistic: on 1 and 18 DF, p-value: Statistics 371, Fall Statistics 371, Fall One Last Eample FEV (forced epirator volume) is a measure of lung capacit and strength. It increases as children grow larger. Amazon Residual Plot > plot(fitted(fit2), residuals(fit2), lab = "Fitted Values", lab = "Residuals") > abline(h = 0) The net page has two separate residual plots, one of the residuals versus height from a fit of FEV versus height and one of the residuals versus height from a fit of log(fev) versus height. Notice that the residuals in the first plot have a fan shape but those in the second are more evenl spread out. Also, the first plot shows a nonlinear trend that is not present in the second plot. Residuals Fitted Values Statistics 371, Fall Statistics 371, Fall

13 One Last Eample Original Data Transformed Data Residuals Residuals Fitted Values Fitted Values Statistics 371, Fall