Regression With a Categorical Independent Variable

Transcription

1 Regression With a Categorical Independent Variable Lecture 15 March 17, 2005 Applied Regression Analysis Lecture #15-3/17/2005 Slide 1 of 29

2 Today s Lecture» Today s Lecture» Midterm Note» Example Regression with a single categorical independent variable. Coding procedures for analysis. Effect coding. How to accomplish these tasks in SPSS. Lecture #15-3/17/2005 Slide 2 of 29

3 Midterm Note» Today s Lecture» Midterm Note» Example Please deposit your midterm by 4pm Friday in one of my mailboxes (either in Psychology or Education). I will only be accepting paper copies of the midterm (please do not me your midterm). Lecture #15-3/17/2005 Slide 3 of 29

4 Example Variable: Two Categories» Today s Lecture» Midterm Note» Example From Pedhazur (1997; p. 343): Assume that the data reported [below] were obtained in an experiment in which E represents an experimental group and C represents a control group. E C Y Ȳ (Y Ȳ) 2 = y Lecture #15-3/17/2005 Slide 4 of 29

5 » Example: Effect Coded» Example 1» Fixed Effects Linear Model Effect coding is the less straight-forward method of coding categorical variables when compared with dummy coding. In effect coding, one (again) creates a set of column vectors that represent the membership of an observation to a given category level. Like dummy coding, the total number of column vectors for a categorical variable are equal to one less than the total number of category levels. Lecture #15-3/17/2005 Slide 5 of 29

6 » Example: Effect Coded» Example 1» Fixed Effects Linear Model If an observation is a member of a specific category level, they are given a value of 1 in that category level s column vector. If an observation is not a member of a specific category and is not a member of the omitted category, they are given a value of 0 in that category level s column vector. If an observation is a member of the omitted category, they are given a value of -1 in every category level s column vector. Lecture #15-3/17/2005 Slide 6 of 29

7 » Example: Effect Coded» Example 1» Fixed Effects Linear Model For each observation, a no more that a single 1 will appear in the set of column vectors for that variable. The column vectors represent the predictor variables in a regression analysis, where the dependent variable is modeled as a function of these columns. Because all observations at a given category level have the same value across the set of predictors, the predicted value of the dependent variable, Y, will be identical for all observations within a category. The set of category vectors (and a vector for an intercept) are now used as input into a regression model. Lecture #15-3/17/2005 Slide 7 of 29

8 Effect Coded Regression Example» Example: Effect Coded» Example 1» Fixed Effects Linear Model Y X 1 X 2 Group E E E E E C C C C C Mean Lecture #15-3/17/2005 Slide 8 of 29

9 Effect Coded Regression» Example: Effect Coded» Example 1» Fixed Effects Linear Model The General Linear Model states that the estimated regression parameters are given by: b = (X X) 1 X y Lecture #15-3/17/2005 Slide 9 of 29

10 Effect Coded Regression - X 1 and X 2» Example: Effect Coded» Example 1» Fixed Effects Linear Model For our second example analysis, consider the regression of Y on X 1 and X 2. a = 15 b 2 = 2 Y = a + b 2 X 2 + e y 2 = 100 SS res = X X = 60 SS reg = = 40 R 2 = = 0.4 Lecture #15-3/17/2005 Slide 10 of 29

11 Effect Coded Regression - X 1 and X 2» Example: Effect Coded» Example 1» Fixed Effects Linear Model a = 15 is the overall mean of the dependent variable across all categories. b 2 = 2 is the called the effect of the experimental group. This effect represents the difference between the experimental group mean and the overall mean. For members of the E category: Y = a + b 2 X 2 = (1) = 17 For members of the C category: Y = a + b 2 X 2 = ( 1) = 13 The fit of the model is the same as was found in the dummy coding from the previous class. Lecture #15-3/17/2005 Slide 11 of 29

12 The Fixed Effects Linear Model» Example: Effect Coded» Example 1» Fixed Effects Linear Model Effect coding is built to estimate the fixed linear effects model. Y ij = µ + β j + ǫ ij Y ij is the value of the dependent variable of individual i in group/treatment/category j. µ is the population (grand) mean. β j is the effect of group/treatment/category j. ǫ ij is the error associated with the score of individual i in group/treatment/category j. Lecture #15-3/17/2005 Slide 12 of 29

13 The Fixed Effects Linear Model» Example: Effect Coded» Example 1» Fixed Effects Linear Model The fixed effects linear model states that a predicted score for an observation is a composite of the grand mean and the treatment effect of the group to which the observation belongs. Y ij = µ + β j + ǫ ij For all category levels (total represented by G), the model has the following constraint: G β g = 0 g=1 Lecture #15-3/17/2005 Slide 13 of 29

14 The Fixed Effects Linear Model» Example: Effect Coded» Example 1» Fixed Effects Linear Model This constraint means that the effect for the omitted category level (o) is equal to: β o = β g = β 1 β 2... g =o From the example, the effect for the control group is equal to: β C = β E = 2 Just to verify: β E + β C = 2 + ( 2) = 0 Lecture #15-3/17/2005 Slide 14 of 29

15 Hypothesis Test of the Regression Coefficient» Example: Effect Coded» Example 1» Fixed Effects Linear Model Because each model had the same value for R 2 and the same number of degrees of freedom for the regression (1), all hypothesis tests of the model parameters will result in the same value of the test statistic. F = R 2 /k (1 R 2 )/(N k 1) = 0.4/1 (1 0.4)/(10 1 1) = 5.33 From Excel ( =fdist(5.33,1,8) ), p = If we used a Type-I error rate of 0.05, we would reject the null hypothesis, and conclude the regression coefficient for each analysis would be significantly different from zero. Lecture #15-3/17/2005 Slide 15 of 29

16 Example Data Set» Hungry?» Breakfast Cereal Example» Can You Guess?» Predicted Values Neter (1996, p. 676). The Kenton Food Company wished to test four different package designs for a new breakfast cereal. Twenty stores, with approximately equal sales volumes, were selected as the experimental units. Each store was randomly assigned one of the package designs, with each package design assigned to five stores. The stores were chosen to be comparable in location and sales volume. Other relevant conditions that could affect sales, such as price, amount and location of shelf space, and special promotional efforts, were kept the same for all of the stores in the experiment. Lecture #15-3/17/2005 Slide 16 of 29

17 Cereal» Hungry?» Breakfast Cereal Example» Can You Guess?» Predicted Values Lecture #15-3/17/2005 Slide 17 of 29

18 » Hungry?» Breakfast Cereal Example» Can You Guess?» Predicted Values Generalizing the concept of effect coding, we revisit the cereal experiment data. Recall that there were four different types of cereal boxes. A effect coding scheme would involve creation of three new column vectors, each representing observations from each box type. The choice of omitted category level is arbitrary. Any level can be omitted and you will get the same results...this is due to the equivalence of linear models under effect coding. Lecture #15-3/17/2005 Slide 18 of 29

19 One-Way Analysis of Variance» Hungry?» Breakfast Cereal Example» Can You Guess?» Predicted Values Just as was the case for the example with two categories, a multiple category regression model with a single categorical independent variable has a direct link to a statistical test you may be familiar with. The regression model tests for mean differences across all pairings of category levels simultaneously. Testing for a difference between multiple groups (> 2) equates to a one-way ANOVA model (for a model with a single categorical independent variable). Lecture #15-3/17/2005 Slide 19 of 29

20 Y X 1 X 2 X 3 X 4 Type Mean

21 Breakfast Cereal Example» Hungry?» Breakfast Cereal Example» Can You Guess?» Predicted Values Group means: Group Mean We will omit the final category from our analysis. Y ij = µ + β j + ǫ ij Lecture #15-3/17/2005 Slide 20 of 29

22 It s the Guess the Parameter Game» Hungry?» Breakfast Cereal Example» Can You Guess?» Predicted Values µ = β 1 = β 2 = β 3 = β 4 = Group means: Group Mean Grand mean Lecture #15-3/17/2005 Slide 21 of 29

23 It s the Guess the Parameter Game» Hungry?» Breakfast Cereal Example» Can You Guess?» Predicted Values Group means: Group Mean Grand mean µ = (the grand mean). β 1 = = β 2 = = β 3 = = 0.75 β 4 = -(-4.05) - (-5.25) = = 8.55 Lecture #15-3/17/2005 Slide 22 of 29

24 Breakfast Cereal Example Therefore:» Hungry?» Breakfast Cereal Example» Can You Guess?» Predicted Values Ȳ A = Y A = µ + β 1 = = 14.6 Ȳ B = Y B = µ + β 1 = = 13.4 Ȳ C = Y C = µ + β 1 = = 19.4 R 2 = Ȳ D = Y D = µ + β 1 = = 27.2 Lecture #15-3/17/2005 Slide 23 of 29

25 Hypothesis Test» Hungry?» Breakfast Cereal Example» Can You Guess?» Predicted Values To test that all means are equal to each other (H 0 : µ 1 = µ 2 =... = µ k ) against the hypothesis that at least one mean differs (H 1 : At least one µ = µ ), called an omnibus test, the same hypothesis test from before can be used: y 2 = SS res = SS reg = = R 2 = / = Lecture #15-3/17/2005 Slide 24 of 29

26 Hypothesis Tests» Hungry?» Breakfast Cereal Example» Can You Guess?» Predicted Values F = R 2 /k (1 R 2 )/(N k 1) = 0.788/3 ( )/(20 3 1) = From Excel ( =fdist(19.803,3,16) ), p = If we used a Type-I error rate of 0.05, we would reject the null hypothesis, and conclude that at least one regression coefficient for this analysis would be significantly different from zero. Having a regression coefficient of zero means having zero difference between the mean of one category and the grand mean. Having all regression coefficients of zero means absolutely no difference between any of the means (all means are equal to the grand mean). Lecture #15-3/17/2005 Slide 25 of 29

27 Categorical Variable Regression in SPSS» GLM in SPSS Estimating categorical regression in SPSS can be accomplished in one of two ways: 1. Creating dummy or effect coded variables and running the regression through the familiar Analyze...Regression...Linear. 2. Using the General Linear Model subroutine (via Analyze...General Linear Model...Univariate). The GLM subroutine does not require coded variables -it will encode them for you and run the regression. This subroutine uses dummy coding (but you will not notice). Lecture #15-3/17/2005 Slide 26 of 29

28 GLM in SPSS» GLM in SPSS Lecture #15-3/17/2005 Slide 27 of 29

29 Final Thought» Final Thought» Next Class Effect coding will lead to regression parameters representing the effects of being in a given category level or treatment group. This model leads to such generalizations as ANCOVA and Mixed Effects models. You do not normally need to encode your variables to run an analysis. Green beer doesn t taste much different than other beers. Lecture #15-3/17/2005 Slide 28 of 29

30 Next Time Who cares? It s spring break... See you when the basketball team is in the final four.» Final Thought» Next Class Lecture #15-3/17/2005 Slide 29 of 29