13: SIMPLE LINEAR REGRESSION V

Transcription

1 13: SIMPLE LINEAR REGRESSION V Using The Regression Equation Point Estimators and Predictors Once we are convinced that the model is reason- able, we can use the fitted regression equation ŷ = b + b x 0 1 to estimate E (y x ) and to predict a future value of y for a given x. Eg: Consider the Team Batting Average (x ) and Team Winning Percentage (y ) for the 14 teams in the American League in 1986.

2 -2- The scatterplot shows some indication of a positive linear association, although some of the teams with high batting averages have surprisingly low winning percentages. These teams are Cleveland, Milwaukee, Toronto, and Minnesota (the most extreme case). The residual plot confirms that the linear model is far from perfect. The points which were "surpris- ingly low" in the scatterplot now show up as strongly negative residuals, indicating that for these teams, their winning percentages fall short of what would be predicted by a linear regression model. Another problem is that the residuals indicate an overall upward trend. This is a sign that the outliers

3 -3- have "dragged down" the fitted line. The above problems are warning signals which should not be ignored in a real business application, but since this is only baseball (!!) we will continue. The fitted model is ŷ = x. The p -value for β is.070, and R 2 1 is.248, indicat- ing a weak to moderate linear association. Incidentally, if we delete the outlier teams, the p - value goes down to.000 and R 2 goes up to.821. So the linear relationship is strong for the remain- ing 10 teams.

4 -4- Now, consider a (hypothetical) team with a batting average of.260. Their winning percentage is ran- dom, since it cannot be predicted with certainty, even if we know the batting average. The mean (i.e., expected) winning percentage for such a team is E (y x ) =β +β (.260), 0 1 which we can estimate by the fitted value ŷ = b + b (.260) = (.260) = If, instead of estimating the mean winning percenactually obtained by a single team with a.260 tage, we wanted to predict the winning percentage bat-

5 -5- ting average, we would use the same value, ŷ =.494. Thus, the point estimator of E (y x ) and the point forecast of a future y are numerically identical (both are simply ŷ ) but conceptually different, since the "targets" (E (y x ) and the future y ) are different. When we ask for interval estimators and predic- tions, however, a numerical difference emerges. Interval Estimators And Predictors If we simply want an interval estimator for E (y x ), the mean value of y for a given x, we

6 -6- should use the confidence interval for E[Y], ŷ ± t α/2 ; (n 2) s n 2 x (x 1 0 x) + (n 1)s 2 1/2, where ŷ = b + b x, and x is the given value of x. Later, we will show that this provides an (exact) level 1 α confidence interval for E [Y X = x ] if the 0 u i are normal. Even if the u i are not normal, the interval will be asymptotically valid. We can obtain this confidence interval from the Minitab output. The interval is denoted by "CI".

7 -7- Eg: For the baseball example, before running the regression, we select "Options". We enter the value. 260 in the box marked "Prediction interval for new observations:". (This will generate a confidence interval and a prediction interval). Minitab gives a fitted value of.494, and a 95% confidence interval for E (y x ) of (.453,.536). If we want to predict a future value of y given a specific value of x, we use the prediction interval. Jobson uses the unfortunate name, "Confidence interval for a particular Y". But it s not a confidence interval, it s a prediction interval. The

8 -8- formula is ŷ ± t α/2 ; (n 2) s 2 x (x 0 x) + n (n 1)s 2 1/2. The prediction interval is denoted "PI" by Minitab. Eg: For the baseball example, the 95% prediction interval for the winning percentage of a team with a batting average of.260 is (.336,.653). Interpretation of a Prediction Interval: If we repeat the experiment of obtaining a regression data set many times, each time forming a level 1 α prediction interval at X = x, and waiting to see 0 what the future value of Y is at X = x, the 0 n

9 -9- roughly (1 α)100% of the prediction intervals will contain the corresponding actual future value of Y. This future value of Y will be given by β + β x + u, where u is independent of u,...,u, and also of x,...,x 1 n 1 n. If we declare that the future value will lie within the given prediction interval, then statements of this kind will be correct (1 α)100% of the time, in the long run. Both the confidence interval and the prediction interval require that the u be normally distributed i in order to be valid for small samples.

10 -10- A prediction interval is similar in spirit to a confidence interval, except that the prediction interval is designed to cover a "moving target", the ran- dom future value of y. The confidence interval seeks to cover the f ixed target, E (y x ). Although both are centered at ŷ, the prediction interval is wider than the confidence interval, for given x and α. This makes sense, since the predic- tion interval must take account of the tendency of y to fluctuate from its mean value, E (y x ). The confidence interval simply needs to account for the uncertainty in estimating E (y x ).

11 -11- For a given data set, the error in estimating E (Y X = x ) grows as x moves away from x. 0 0 Thus, the further x 0 is from x, the wider the confidence intervals and prediction intervals will be. Why does this make sense? If any of the assumptions underlying the model are violated, then the confidence intervals and predreason why it is so important to check the assump- iction intervals may be invalid as well. This is one tions by examining residuals, etc.