Operators and the Formula Argument in lm

Transcription

1 Operators and the Formula Argument in lm Recall that the first argument of lm (the formula argument) took the form y. or y x (recall that the term on the left of the told lm what the response variable is and the term on the right tells lm what the predictor variable(s) is(are)). To frame the following discussion, please load the the fuel economy dataset into R and save it as a data frame named cars (see Module 2 if you don t remember how to do this!). Exercises: 1. What model would you expect to be fit with the following formula argument? Fuel. Economy Weight + Horsepower + Seating Does the model include a predictor which is numerically equal to the following vector? c a r s $Weight + c a r s $Horsepower + c a r s $ Seating 2. What model do you think is fit when the formula argument is as below? Fuel. Economy. Seating Cylinders Run lm with this formula and see if the result matches your intuition. 3. How many terms are included in the model when the formula argument is as below? Fuel. Economy Weight Fuel. Capacity Is this what you had expected? 4. What models are being fit by the following code? How does model1 compare with model2? Is this what you expected? > model1 < lm( Fuel. Economy Weight, data = c a r s ) > model2 < lm( Fuel. Economy Weight ˆ2, data=c a r s ) 5. Based on your answer to Exercise 4, do you expect model3 and model4 to be different? > model3 < lm( Fuel. Economy ( Weight + Horsepower ), data = c a r s ) > model4 < lm( Fuel. Economy ( Weight + Horsepower )ˆ2, data=c a r s ) Be sure to run lm to check your intuition! 1

2 In the above examples and exercises, we have seen several operators which don t behave as they normally do outside of lm. For instance, the + tells lm to include Weight, Horsepower, and Seating as separate predictors in the model rather than first computing the sum of the three variables and then using that as a single predictor in a simple regression model. Similarly, the * tells lm to include an interaction between Weight and Fuel.Capacity in addition to Weight and Fuel Capacity themselves (the interaction term is numerically equal to Weight Fuel.Capacity). This interaction term is denoted Weight:Fuel.Capacity in the summary of the model fit. > model2 < lm( Fuel. Economy Weight Fuel. Capacity, data = c a r s ) > summary( model2 ) Call : lm( formula = Fuel. Economy Weight Fuel. Capacity, data = c a r s ) Residuals : Min 1Q Median 3Q Max C o e f f i c i e n t s : Estimate Std. Error t value Pr( > t ) ( I n t e r c e p t ) < 2e 16 Weight e 15 Fuel. Capacity < 2e 16 Weight : Fuel. Capacity e 12 S i g n i f. codes : Residual standard e r r o r : 3.39 on 226 degrees o f freedom Multiple R squared : , Adjusted R squared : F s t a t i s t i c : on 3 and 226 DF, p value : < 2. 2 e 16 We also observe that ˆ did not act like the usual exponentiation operator. We ve now seen a couple of examples of operators like +,,, ˆ which do not behave in the usual arithmetic sense when they are used in the the formula argument of lm 1 Table 1 summarize the meaning of these operators in lm 1 This phenomenon is known as operator overloading. 2

3 Operator Meaning + include the following predictor in the model - exclude the following predictor in the model * introduce these predictors along with interaction ˆ include predictors and higher-order interactions. include all predictors Table 1: Meaning of operators in lm The ˆ operator tells lm to include all interaction terms up to a specified order. As we saw in model4 from Exercise 5 above, when we had (Weight + Horsepower)ˆ2 in the formula, lm fit a model Weight, Horsepower, and an interaction term between them. This interaction term, because it involves two terms, is known as a second-order interaction. In general, if we wrote ˆk in the formula, where k is a positive integer, lm will include terms for all possible interactions of k different predictors. Note that when there are fewer than k predictors, lm will include interactions up to the largest possible order. To see this, consider the two models below. new. model1 < lm( Fuel. Economy. ˆ 3, data=c a r s ) new. model2 < lm( Fuel. Economy ( Weight + Horsepower )ˆ3, data=c a r s ) In the first, we have included all possible two-way and three-way interactions and in the second we have only included two-way interactions (since we are only passing two different predictors to lm). 3

4 Transformations As we saw in Table 1, many standard arithmetic operators have alternate meanings within lm. In particular, if we decided, for whatever reason, that we wanted to predict fuel economy using the following variables (Weight + Horsepower) 2 and (Height Width Length), it would appear that we re in a bit of a bind. In order to prevent operator overloading (e.g. when we really want + to mean plus ) we use I(expr) as follows: > model < lm( Fuel. Economy I ( ( Weight+Horsepower )ˆ2)+ I ( Length Weight Height ), data=c a r s ) The syntax I(expr) tells lm that it should evaluate the expression expr normally and use the result as a predictor in the linear model that is fit. Note, for other transformations like log or square-root you don t need to use I (...) : > model < lm( log ( Fuel. Economy) log ( Weight + 1) + sqrt ( Fuel. Capacity ), data = c a r s ) 4

5 Categorical Variables Formally, there is no new syntax required to actually fit a model with categorical predictors. However, as you should recall from other statistics classes, when including categorical predictors we need to include a reference level 2. So if you have a categorical predictor that has k values, R will include k 1 dummy variables in the regression model (and so it will only estimate k 1 coefficients for the categorical predictor). To demonstrate how we can include categorical predictors in linear regression models, we will use the updated fuel economy dataset includes the manufacturer, type of transmission, and number of transmission speeds for each car in the original fuel economy dataset. If we load this data into R and look at summaries of these three new variables, we see that there are 121 cars with automatic transmission and 109 with manual transmission. > c a r s 2 < read. table ( f i l e = c a r s 2. csv, header = TRUE) > summary( c a r s 2 [, c ( Transmission. Speeds, Transmission, Manufacturer ) ] ) Transmission. Speeds Transmission Manufacturer Min. : Automatic :121 Toyota : 15 1 s t Qu. : Manual :109 Chevrolet : 14 Median : Ford : 14 Mean : Mazda : 9 3 rd Qu. : Mercedes Benz : 9 Max. : Nissan : 9 ( Other ) :160 As we see in the above, when we look at the summary of a categorical variable, R reports the counts for each level. By default, R treats the first level as the reference level and to determine which is the first levels, we can use the levels function: levels ( c a r s 2 $Manufacturer ) [ 1 ] Acura Aston Martin Audi BMW [ 5 ] Buick C a d i l l a c Chevrolet Chrysler [ 9 ] Dodge F e r r a r r i Ford GMC [ 1 3 ] Honda Hyundai I n f i n i t i Isuzu [ 1 7 ] Jaguar Jeep Kia Lamborghini [ 2 1 ] Lexus Lincoln Lotus Maserati [ 2 5 ] Mazda Mercedes Benz Mercury MINI [ 2 9 ] Mitsubishi Nissan Oldsmobile Panoz [ 3 3 ] Pontiac Porsche Rolls Royce Saab [ 3 7 ] Saturn Subaru Suzuki Toyota [ 4 1 ] Volkswagen Volvo 2 This is so that the model is identifiable 5

6 As we see from above the reference manufacturer is Acura and in general, when it reads in a categorical variable, R will sort the levels alphabetically. If we want to change the reference level, we can use the relevel function: > c a r s 2 $Manufacturer < relevel ( c a r s 2 $Manufacturer, r e f= Rolls Royce ) In the context of modeling fuel economy, one might argue that we should be treating the number of transmission speeds as a categorical variable. However, as we saw above, R treats this variable as a scalar. To convert it to a categorical variable, we use as.factor > c a r s 2 $Transmission. Speeds < as. factor ( c a r s 2 $Transmission. Speeds ) > summary( c a r s 2 $Transmission. Speeds ) We see that there are 69 4-speed cars in our dataset and this is the reference category. 6

7 Logistic Regression While linear regression is a very useful tool, we note that it is not always the most appropriate tool to use. Returning to the fuel economy example, suppose we with to predict only whether or not a car will get more than 25 miles per gallon using the available predictors. We can create a new dummy variable in our data frame cars2 which is equal to 1 if a car gets more than 25 miles per gallon and zero otherwise as follows: > c a r s 2 $newy < as. numeric( c a r s 2 $Fuel. Economy>= 25) Now, we are dealing with a binary outcome and a linear regression model is not necessarily the most useful model. Instead, we ought to consider a logistic regression model, which can be fit in R using glm as follows: > l o g i s t i c 1 < glm (..., family = binomial, data =... ) The first argument of glm is the formula argument, which is formatted exactly like the formula in lm and the data argument is also analogous to that of lm. The only new syntax in glm is the family argument (for more information on this argument, please see the help page for glm). Exercises 1. Fit a logistic regression model with outcome variable cars2$newy to predict the logodds of a car having a fuel economy greater than 25 MPG using its weight, type of transmission, and manufacturer. 2. Fit a logistic regression model to include all two-way interactions between the available predictors. Note that you should not include Fuel.Economy as a predictor. 7