Simple Linear Regression: A Model for the Mean. Chap 7

Transcription

1 Simple Linear Regression: A Model for the Mean Chap 7

2 An Intermediate Model (if the groups are defined by values of a numeric variable) Separate Means Model Means fall on a straight line function of the group values Equal Means Model 2

3 Meaning of the Word Regression statistical meaning not related to the usual English definition of regression misnomer, but there s a historical reason refers to finding the best fitting (straight line) relationship between the mean of Y and values of the variable defining the groups (X) 3

4 Galton s data 75 height of son height of father 4

5 Case Study Hubble data (from 92 s) observational data each point represents a nebula Y distance of the nebula X recession velocity of the nebula 5

6 Case Study don t worry about the geometry, but Big Bang theory implies that Y = const. * X (i.e. a straight line relationship between Y and X, with intercept ) furthermore, const. should be the age of the universe 6

7 Case Study 2 DISTANCE 5 VELOCITY 7

8 Case Study Regression methods can help answer big cosmological questions: Is the Big Bang theory correct? is the intercept of the straight-line relationship? How old is the universe? what is the slope of the straight-line relationship? 8

9 Case Study 2 designed experiment response variable (Y) ph of steer carcass explanatory variable (X) log time (hours) after slaughter what is the approximate ph of a particular steer carcass 3 hours after slaughter? about how long do you have to wait after slaughter for the ph to reach 6.? 9

10 Case Study PH PH TIME log(time) 2

11 Regression Analysis statistical methods based on describing the distribution of values of one variable (Y the response variable) as a function of the other variable (X the explanatory variable) simple linear regression: the function is a straight line function

12 Regression Analysis specifically, the mean of Y is a straight line function of X µ{y X} = β + β X where β is the intercept and β is the slope the standard deviation of Y is constant σ{y X} = σ the distribution of values of Y for any value of X is normal 2

13 Simple Linear Regression Model LINEARITY The mean response, µ, has a straight-line relationship with X, (µ{y X}= β + β X). ASSUMPTIONS NORMALITY For any given X, Y is normally distributed around µ{y X}. y x µ y = β + β x EQUAL STANDARD DEVIATION The standard deviation(σ) of Y is the same for all values of X. INDEPENDENCE Observations are independent of each other. 3

14 Simple Linear Regression Model Response Explanatory µ { Y X} = β + β X Regression coefficents 4

15 Simple Linear Regression Model Response Regressor µ { Y X} = β + β X Intercept Slope 5

16 Interpretation of coefficients slope: for a -unit change in X, Y changes by β units on average intercept: when X is, the mean value of Y is β 6

17 Simple Linear Regression Model advantage relative to separate means model interpolation: making inferences about the distribution of Y for X values not actually included in the data set (but within the range of the observed X values e.g. estimating the mean ph of carcasses 2.5 hours after slaughter) danger relative to separate means model extrapolation: making inferences about the distribution of Y for X values outside the range of the observed X values (e.g. estimating the mean ph of carcasses 2 hours after slaughter) 7

18 Estimating β, β and σ we can t just calculate means to estimate β and β (like we did in ANOVA) strategy: use least squares to find best fitting line; then use slope and intercept of best fitting line as estimates true slope and intercept 8

19 Least squares find the straight line that minimizes Σ(res i ) 2 µ{y X} = β +β X fit i Y i res i = Y i -fit i 9

20 Formulas for estimators the least squares principle is just for background 2 ) ˆ ˆ ˆ ) ( ) )( ( ˆ 2 2 = = = = = n res X Y X X Y Y X X i n i i n i i i ( σ β β β 2

21 Degrees of Freedom n-2 here n-i in one way ANOVA n- in one-sample t tools in general, df = (number of observations) (number of parameters in model for the means) 2

22 Standard Errors ) ( ) ( ˆ ) ˆ ( ) ( ˆ ) ˆ ( = + = = n X X s where s n X n SE s n SE i X X X σ β σ β 22

23 Inferences with these estimators and standard errors, you can test hypotheses about and compute confidence intervals for β and β in the usual way e.g. est. +/- t (df)(-α/2) SE(est.) based on the t-distribution with n-2 degrees of freedom 23

24 Regression Software in practice you never need to hand-calculate these estimators nor their standard errors but you might have to hand-calculate tests or confidence intervals MINITAB and S-Plus Hubble data: test the null hypothesis that β = estimate β 24

25 25

26 26

27 Hubble data Regression Analysis: DISTANCE versus VELOCITY ˆ ˆβ ( ˆ β σˆ SE β ) SE( ˆ β ) The regression equation is DISTANCE = VELOCITY Predictor Coef SE Coef T P Constant VELOCITY S =.45 R-Sq = 62.4% R-Sq(adj) = 6.6% 27

28 Hubble data Regression Analysis: DISTANCE versus VELOCITY The regression equation is DISTANCE = VELOCITY Predictor Coef SE Coef T P Constant VELOCITY S =.45 R-Sq = 62.4% R-Sq(adj) = 6.6% Since p=.3, the simple Big Bang theory is not correct 28

29 Hubble data Regression Analysis: DISTANCE versus VELOCITY The regression equation is DISTANCE = VELOCITY ˆβ SE( ˆ β ) Predictor Coef SE Coef T P Constant VELOCITY S =.45 R-Sq = 62.4% R-Sq(adj) = 6.6% 29

30 Hubble data hand calculate confidence interval for β /- 2.7 (.2274) = (.98,.844) or roughly (.89,.8) billion years 3

31 ph data Regression Analysis: PH versus log(time) The regression equation is PH = log(time) Predictor Coef SE Coef T P Constant log(time S =.8226 R-Sq = 98.2% R-Sq(adj) = 98.% (but these statistics don t answer the questions of interest to us) 3

32 Three New Kinds of Inference. Estimating the mean value of Y at some specified value X=X (e.g. What is the mean ph of steer carcasses 3 hours after slaughter?) ˆ{ µ Y X } = ˆ β + ˆ β X SE( ˆ{ µ Y X }) = σˆ n + ( X ( n X ) ) s 2 x 2 32

33 Three New Kinds of Inference 2. Predicting the value of Y for a specific individual at some specified value X=X (e.g. What is the ph of a particular steer s Garth s -- carcass 3 hours after slaughter?) pred{ Y X } = ˆ β + ˆ β X SE( pred{ Y X }) = ˆ σ + n + ( X ( n X ) ) s 2 x 2 = ˆ σ 2 + SE( ˆ{ µ Y X }) 2 33

34 Three New Kinds of Inference 3. Estimating the value of X that results in Y=Y (inverse prediction or calibration) (e.g. About how long do you have to wait after slaughter for the mean ph to reach 6.?) Xˆ = Y ˆ β ˆ β SE( Xˆ ) = SE( ˆ{ µ Y ˆ β Xˆ }) or SE( pred{ Y ˆ β Xˆ }) 34

35 Software for first two inferences MINITAB easy SPlus a little harder (for me) 35

36 36

37 37

38 ˆ µ { Y X } SE ( ˆ{ µ Y X }) ˆ{ Y X } ± t( n 2)(.975) SE( ˆ{ µ Y X }) µ Predicted Values for New Observations New Obs Fit SE Fit 95.% CI 95.% PI ( 6.257, ) ( 5.987, ) Values of Predictors for New Observations New Obs log(time. pred{ Y X } ± t( n 2)(.975) SE( pred{ Y X }) X = ln(3.)=.99 38

39 Hand calculations for inverse estimation for Y =6 Xˆ = Y ˆ β ˆ β = =.35 exp(.35) = 3.86 hours SE( Xˆ ) = SE( ˆ{ µ Y ˆ β Xˆ }) = =.37 39

40 Related ideas computer centering trick confidence and prediction bands multiple comparison issues 4

41 Regression Plot 7 PH 6 Regression 95% CI 5 95% PI log(time) 2 4

42 Standard Errors We have discussed six standard errors SE of the slope SE of the intercept SE of the estimated mean of Y at X SE of the predicted value of Y (new individual) at X SE of inverse estimate of X for a mean Y SE of inverse estimate of X for an individual Y 42

43 Correlation Correlation is a measure of linear association between two variables Formula on page 94 between - and + zero correlation implies no linear relation + implies on points fall exactly on a straight line with positive slope 43