Inference for Regression Inference about the Regression Model and Using the Regression Line, with Details. Section 10.1, 2, 3

Transcription

1 Inference for Regression Inference about the Regression Model and Using the Regression Line, with Details Section 10.1, 2, 3

2 Basic components of regression setup Target of inference: linear dependency of a response variable on one or more explanatory variables One explanatory variable simple linear regression (SLR) One or more multiple regression The least-squares regression line describes this dependency in the data. A population regression line describes its underlying idealization Involved in describing the probabilities of observing certain values in the sample measurements.

3 Brief review of least-squares regression The least-squares regression line makes the sum of squared-prediction errors as small as possible. The slope is and the intercept is Predictions are made by plugging in values of x The residuals describe the leftover variation in y after fitting the least-squares regression line The coefficient of determination, r 2, measures the proportion of variability in y that is explained by x

4 Formal setup for inference in regression The data arise as n pairs of measurements, (x 1, y 1 ),, (x n, y n ) (x i, y i ) are measurements on the i th individual The statistical model is y i = β 0 + β 1 x i + ε i µ y = β 0 + β 1 x i is the mean response when x = x i The ε i are independent, and each ε i is N(0, σ) The least-squares regression line is Sample estimate of µ y = β 0 + β 1 x i

5 Example: Wages and experience Do wages rise with experience? In a study of employment trends, wage (y, in $/week) and length of service (LOS = x, in months) measurements were obtained from n = 59 workers in similar customer-service positions. Wages LOS Wages LOS Wages LOS Wages LOS Wages LOS Wages LOS

6 Example: Wages and experience (continued) Summary statistics: Least-squares regression line and scatterplot:

7 Sampling framework Idea: Each value of x defines a subpopulation Multiple, independent SRSs: Each SRS is drawn from a distinct subpopulation y = response variable = measurement of interest x = explanatory variable = subpopulation and sample labels One SRS, with multiple measurements: measure (x i, y i ) on the i th individual but treat the x i as fixed quantities Model describes the conditional distribution of y given its associated subpopulation

8 Comments on the statistical model y i = β 0 + β 1 x i + ε i with independent ε i, each N(0, σ) Data = Fit + Residual Linearity: µ y = β 0 + β 1 x connects subpopulation means Constant spread: σ does not depend on x Normality: response measurements are bell-shaped within each subpopulation

9 Residuals and residual standard deviation Unknown population quantities: The random variables ε i are residual deviations The parameter σ is the residual standard deviation Analogous quantities calculated from the sample: The i th (sample) residual is e i = y i ŷ i The regression standard error is

10 Properties of the slope estimate Suppose (x 1, y 1 ),, (x n, y n ) satisfy the assumptions of the statistical model for SLR Mean: Standard deviation: Standard error:

11 Some computational formulas Regression standard error: Standard error for slope:

12 Example: Wages and experience (continued) Regression standard error: Standard error for slope:

13 The t test and CI for slope in SLR Assumptions: The statistical model for SLR Hypotheses: H 0 : β 1 = 0 versus a one- or two-sided H a Test statistic: P-value: P(T -t) for H a : β 1 < 0 P(T t) for H a : β 1 > 0 2P(T t ) for H a : β 1 0, where T is t(n 2) CI: For confidence level C, the interval is where t* is such that P(T t*) = (1 C)/2

14 Example: Wages and experience (continued) Hypotheses: H 0 : β 1 = 0 versus H a : β 1 > 0 Summary statistics: b 1 = 0.59, s = 82.2, and SE b1 = 0.21 Test statistic: t = b 1 / SE b1 = 0.59 / 0.21 = 2.85 P-value: P(T 2.85) = 0.003, with k = n 2 = 57 d.f. Decision: Reject H 0 at significance level α = 0.05, and conclude that wages rise with experience

15 Example: Wages and experience (continued) How much do wages rise with experience? 95% CI: P(T 2.00) = 0.025, using k = n 2 = 57 d.f. t* = 2.00, and the interval is b 1 ± t*se b1 = 0.59 ± (2.00)(0.21) = 0.59 ± 0.41 = (0.18, 1.00) Conclude an increase in weekly salary between $0.18 and $1.00 per month of service, on average

16 Robustness A moderate lack of Normality may be tolerated Better for large n Outliers or influential observations may be problematic Basic tool: residual plots Example: Wages and experience (continued)

17 Connections to correlation One SRS, with multiple measurements: (x i, y i ) are paired measurements from one SRS Idea: Treat x as random and work with correlation r is the sample correlation ρ is the population correlation A test of H 0 : ρ = 0 may be carried out with identical calculations as a test of H 0 : β 1 = 0 but CI formulas for ρ and β 1 are very different Different interpretations: Correlation is for two-way relationships; regression is for one-way relationships

18 Uncertainty in predicted values Plugging x into ŷ = b 0 + b 1 x provides a prediction of the response. Two possible interpretations: ŷ is an estimate of the subpopulation mean µ y = β 0 + β 1 x ŷ is a prediction of an unobserved response, y, from a subpopulation with mean µ y = β 0 + β 1 x Note: There is more uncertainty in the second interpretation since the target of inference is random

19 Confidence interval for µ y Suppose ŷ is to be an estimate of µ y = β 0 + β 1 x Standard error: CI: For confidence level C, the interval is where t* is such that P(T t*) = (1 C)/2

20 Prediction interval for y Suppose ŷ is to be a prediction of y from a subpopulation with mean µ y = β 0 + β 1 x Standard error: PI: For confidence level C, the interval is where t* is such that P(T t*) = (1 C)/2

21 Example: Wages and experience (continued) What is the mean of the subpopulation of workers who s length of service is x = 125? Estimate of µ y : ŷ = b 0 + b 1 x = (0.59)(125) = Standard error: 95% CI:

22 Example: Wages and experience (continued) Suppose the length of service of some interesting worker is x = 125. What is his or her weekly wage? Prediction of y: ŷ = b 0 + b 1 x = Standard error: 95% PI:

23 Confidence and prediction bands Observe: PIs are less precise than CIs Reflects greater uncertainty of the prediction problem

24 Decomposition of variation Analysis of variance (ANOVA) equation: Total variation in y (= 0 if all y i are equal) Variation about the line (= 0 if all y i = ŷ i ) Variation along the line (= 0 if b 1 = 0)

25 ANOVA setup Total variation in y: Variation along the line: Variation about the line: Note: Total d.f. = Regression d.f. + Residual d.f.

26 Related calculations Coefficient of determination: proportion of total variation accounted for by the regression line Mean squares: Alternative formula (which generalizes to multiple regression)

27 Example: Wages and experience (continued) Relevant summary statistics: Mean square statistics: Sums of square statistics (MS d.f.):

28 Testing in ANOVA The ANOVA F statistic is May be used to test H 0 : β 1 = 0 versus H a : β 1 0 and its generalization to multiple regression Large values of F provide evidence against H 0 : β 1 = 0 In SLR case, t = F P-value is 2P(T F) where T is t(n 2)