Multiple Regression an Introduction. Stat 511 Chap 9

Transcription

1 Multiple Regression an Introduction Stat 511 Chap 9 1

2 case studies meadowfoam flowers brain size of mammals 2

3 case study 1: meadowfoam flowering designed experiment carried out in a growth chamber general goal of research: understand the biology of meadowfoam in order to elevate production response variable: number of flowers per plant 3

4 two explanatory variables, one numeric and one categorical: light intensity timing of light six different light intensities two different times to start the light treatments 4

5 80 LATE EARLY FLOWERS PER PLANT LIGHT INTENSITY 5

6 case study 2: brain size of mammals observational study goal of research: biologists derive evolutionary conclusions from investigating which physiological characteristics are related to brain size response variable: brain weight 3 explanatory variables: body weight, gestation period, litter size 6

7 logbrain logbody loggestation loglitter 7

8 multiple linear regression the culmination of our discussion this semester multiple linear regression methods include as special cases 1-sample t-tools 2-sample t-tools one-way ANOVA tools (separate means model) simple linear regression methods tools for many, many more models 8

9 multiple linear regression specifies the mean of the response variable as an equation involving several (more than one) explanatory variables (e.g. µ{y X 1,X 2 }) usually unwise to think that there is some exact, discoverable regression equation often a few approximate but adequate regression equations that can be used to answer research questions 9

10 multiple linear regression the adjective linear in multiple linear regression does not mean that the models are restricted to things like straight lines means that the equation for the mean involves a linear combination of unknown regression coefficients, e.g. 0 1 [ ] [ ] [ ] µ { Y...} = β + β + β + β

11 11 a few examples of multiple linear regression models ) log( ) log( }, { }, { } { }, { X X X X Y X X X X X X Y X X X Y X X X X Y β β β µ β β β β µ β β β µ β β β µ + + = = + + = + + = tilted flat plane in 3D space parabola (special curved line) in 2D space twisted tilted plane in 3D space curved surface in 3D space

12 multiple linear regression additional assumptions of the multiple linear regression model constant variance or equal standard deviation σ{y X 1, X 2 } = σ normality distribution of Y values for specified values of the explanatory variables is the normal curve independence 12

13 multiple regression analysis in general, multiple regression analysis involves formulating a good model for the mean of Y estimating regression model parameters from the data formulating research questions in terms of regression model parameters employing appropriate inferential tools for answering research questions (tests, estimates, confidence intervals, predictions) 13

14 multiple linear regression easy to do on MINITAB or SPlus simply specify more than one explanatory variable in the appropriate window e.g. brain size of mammals 14

15 15

16 Regression Analysis: logbrain versus logbody, loggest, loglitt The regression equation is logbrain = logbody loggest loglitt Predictor Coef SE Coef T P Constant logbody loggest loglitt S = R-Sq = 95.4% R-Sq(adj) = 95.2% Analysis of Variance Source DF SS MS F P Regression Residual Error Total

17 interpretation of regression coefficients have to understand the meaning of regression coefficients in order to use multiple regression models effectively interpretation of regression coefficients is a little more complicated in multiple linear regression models than in simple linear regression consider a regression model such as µ{flowers light, time} = β 0 + β 1 light + β 2 time β 1 can be interpreted as the effect of a 1-unit increase in light on the mean of flowers, while holding time fixed 17

18 interpretation of regression coefficients note that because the interpretation involves the idea of holding the pother variables fixed, the interpretation depends on what other variables are in the model e.g. the interpretation of β 1 in the model µ{flowers light, time} = β 0 + β 1 light + β 2 time is different than the interpretation of β 1 in the model µ{flowers light, time} = β 0 + β 1 light 18

19 specially constructed explanatory variables squared variable for curvature indicator variable to distinguish between two groups set of indicator variables for categorical explanatory variables with more than two groups product of two variables for interaction 19

20 squared term for curvature e.g. silkworm data 2 cocoons age 15 20

21 squared term for curvature model: µ{cocoons age} = β 0 + β 1 age + β 2 age 2 linear in β i s useful tool for incorporating slight curvature difficult and usually unnecessary to interpret coefficients the point is that the straight line model is inadequate, but a parabola seems to be adequate higher order polynomial terms (like age 3 ) can also be added for more complicated curves 21

22 squared term for curvature Regression Analysis: cocoons versus age, agesq The regression equation is cocoons = age agesq Predictor Coef SE Coef T P Constant age agesq S = R-Sq = 83.0% R-Sq(adj) = 78.8% 22

23 squared term for curvature multiple linear regression with age and age 2 as explanatory variables 2 cocoons age

24 indicator variable for two groups sometimes we have a categorical explanatory variable with 2 groups e.g. onset times in meadowfoam study indicator variable takes on the value 0 for one of the categories and the value 1 for the other category easily created on MINITAB, a little more work on SPlus 24

25 indicator variable for two groups model: µ{flowers age, early} = β 0 + β 1 age + β 2 early where early is an indicator variable set up to equal 1 for the early timing and 0 for the late timing (opposite assignment of 1 and 0 would be fine as well) if early=0, the model becomes µ{flowers age, early=0} = β 0 + β 1 age if early=1, the model becomes µ{flowers age, early=1} = β 0 + β 1 age + β 2 = (β 0 + β 2 ) + β 1 age we are actually fitting two lines with same slope but different intercepts (parallel lines) 25

26 indicator variable for two groups Regression Analysis: FLOWERS versus INTENS, EARLY The regression equation is FLOWERS = INTENS EARLY Predictor Coef SE Coef T P Constant INTENS EARLY S = R-Sq = 79.9% R-Sq(adj) = 78.0% 26

27 indicator variable for two groups Late Early FLOWERS INTENSITY 27

28 set of indicator variables for more than 2 groups alternate terminology categorical variable in regression = factor categories of the variable = levels of the factor if factor has k levels, we need k-1 indicator (0-1) variables in the multiple linear regression model form k indicator variables for the k levels, then leave one (any one) out call the level whose indicator is left out the reference level 28

29 set of indicator variables for more than 2 groups example: handicap study use 4 indicator variables to represent the 5 levels of the handicap factor leave out one of the possible 5 indicator variables; using none as the reference level is a sensible choice in this study the handicap factor is the only explanatory variable, but there could be numeric explanatory variables as well 29

30 set of indicator variables for more than 2 groups The regression equation is SCORE = amputee crutches hearing wheelchair Predictor Coef SE Coef T P Constant amputee crutches hearing wheelcha S = R-Sq = 15.0% R-Sq(adj) = 9.7% Analysis of Variance Source DF SS MS F P Regression Residual Error Total

31 compare to one-way ANOVA output One-way ANOVA: SCORE versus HANDICAP Analysis of Variance for SCORE Source DF SS MS F P HANDICAP Error Total

32 set of indicator variables for more than 2 groups in practice, you don t have to create the indicator variables for factors in multiple linear regression most (all?) statistical software packages include general linear model (glm) programs these are just multiple linear regression programs except that they have the ability to generate indicator variables for factors with any number of levels the only problem is that they choose their own reference level or set up the indicator variables in a different way than we have discussed 32

33 product of two variables for interaction 2 variables interact if the effect of one of them on the mean of the response variable depends on the value of the other variable to allow for interaction in multiple linear regression, simply use the product of the two variables as an additional explanatory variable e. g. meadowfoam data using the model: µ{flowers age, early} = β 0 + β 1 age + β 2 early + β 3 age*early 33

34 product of two variables for interaction Regression Analysis: FLOWERS versus INTENS, EARLY, early*intens The regression equation is FLOWERS = INTENS EARLY EARLY*INT Predictor Coef SE Coef T P Constant INTENS EARLY EARLY*INT S = R-Sq = 79.9% R-Sq(adj) = 76.9% 34

35 product of two variables for interaction Late Early FLOWERS INTENSITY

36 shorthand notation for model leave out the β s e.g. write µ{flowers age, early} = β 0 + β 1 age + β 2 early + β 3 age*early as µ{flowers age, early} = age + early + age*early most regression programs accept the model description in this form 36

37 data analysis strategy for multiple linear regression Preliminary: define the questions of interest, review design and scope of inference 1. explore data (graphical tools, fit tentative model, consider transformations, check outliers) 2. formulate (tentative) model to be used for inferences; formulate questions of interest in terms of model parameters 3. check model (fit richer model, test for curvature and interactions, examine residuals for nonconstant variance and outliers); if problems, go to step 2 and reformulate model 4. Infer answers to questions of interest using appropriate inferential tools (tests, confidence intervals, predictions) Communicate results in subject matter language 37

38 graphical tools for multiple regression matrix plots coded scatterplots jittered scatterplots trellis graphs 38