5:1LEC - BETWEEN-S FACTORIAL ANOVA

Transcription

1 5:1LEC - BETWEEN-S FACTORIAL ANOVA The single-factor Between-S design described in previous classes is only appropriate when there is just one independent variable or factor in the study. Often, however, researchers want to examine the effect of two or more factors at the same time. Such studies generally involve factorial designs in which subjects are randomly or otherwise assigned to all combinations of the factors. For example, a social psychology study involving attitude change about exercise might expose 60 subjects to attitude change messages in one of six conditions defined by two levels of Expertise of the source of information (Expert vs. Nonexpert) and three levels of Threat (Low vs. Medium vs. High). The 60 subjects would ideally be assigned equally to the 2 x 3 = 6 following conditions (or cells) defined by the two factors, with 10 subjects in each cell. Expertise Threat Low Med High Expert EL EM EH Nonexpert NL NM NH The factorial design has a number of benefits over single factor designs. First, it is an efficient use of subjects. In this example, 10 subjects per cell would allow comparisons between Expert and Nonexpert (the main effect of Expertise) to be based on 30 subjects in each condition. Similarly, it would allow comparisons between Low, Medium, and High levels of threat (the main effect of Threat) to be based on 20 subjects per cell. Testing each of these effects in separate studies would require 2 x 60 = 120 subjects to have the same degree of power for both comparisons. More importantly, however, the factorial design allows researchers to study interactions between factors. An interaction has a precise meaning in statistics; it means that the effect of one factor varies or differs across the levels of the other factor. In the present study, Threat might work differently for Expert and Nonexpert presentations. For example, Threat might increase intention to exercise in the Expert condition but have no effect or even decrease intention to exercise in the Nonexpert condition. That is, the effect of Threat (how the Ms vary across the levels of Threat) would differ for the different levels (Expert vs. Nonexpert) of the Expertise factor. Interactions are often very important in psychological research, both for theoretical and practical reasons. Factorial designs vary with respect to whether each factor is Between-Subjects or Within- Subjects. The persuasion study could be done completely Within-S if 10 subjects heard 6 different messages targeting a different behaviour and using all combinations of conditions, or if 60 subjects were tested on something relevant to the dependent variable (e.g., gullibility?), then formed into 10 blocks of 6 subjects each matched on gullibility (i.e., everyone in the block had similar scores), and finally, one of each subject in a block was assigned randomly to each cell in the study. Alternatively, one factor could be Within-S and one Between-S. Here we consider only designs with both factors Between-S. Factorial designs can involve more than two factors; the persuasion study, for example, could include Gender, with half the participants Female and half Male. The study would now involve 2 x 3 x 2 = 12 cells or conditions. This unit addresses only two-factor designs. A final observation although ANOVA is often associated with true experiments in which people are randomly assigned to conditions, factors can in fact be either experimental (e.g., Expertise and Threat) or non-experimental (e.g., Gender). The statistical analysis is the same; however, causal inferences are stronger for well-designed and executed experiments than for non-experimental studies. Poorly designed experiments are no better than non-experiments and sometimes even weaker sources for drawing valid causal inferences. The formulas for two-factor Between-S designs are numbered 34 to 39 on our formula sheet.

2 We will also be using some formula from the Between-S single factor design with appropriate modifications. The basic notation for factorial ANOVA is explained on pages 8.15 to 8.18 of the text. Briefly, the letters A and B stand for the two factors and for the number of levels of each factor. The lowercase letters a and b are variables that index the different levels of A and B; that is, a = 1, 2,..., A and b = 1, 2,..., B. To illustrate, M 23 would denote the mean for the 2 nd level of A (a = 2) and the 3 rd level of B (b = 3), and n 14 would be the sample size for the 1 st level of A and the 4 th level of B. And if factor A has 3 levels and factor B has 4 levels, the number of cells = A x B = 3 x 4 = 12. Calculations for Factorial ANOVA The default factorial ANOVA partitions SS Total as follows (A x B is the interaction between A and B): SS Total = SS A + SS B + SS AxB + SS Error Each of these SSs has a certain df associated with it and can be used to calculate four MSs, one of which is MS Error, to be used as the denominator for three F tests. The remaining three MSs are each associated with specific hypotheses about the means and will be used to calculate three numerators, one associated with each hypothesis. SS A and SS B will be used to test the effects of factors A and B, respectively, averaged over the levels of the other factor. These are called the Main Effects. SS AxB will be used to test the significance of the interaction. The calculations will be illustrated for a 2 x 4 factorial study of mistakes in a selective attention task with various distracting sounds. The sounds were of two Types: random Noise or Speech, and were played at four levels of Volume: Subthreshold, Audible, Normal Speech, and Shouting. A total of 24 subjects participated in the experiment, with 3 subjects assigned to each of the 2 x 4 = 8 cells of the experiment. The results appear in the following table with some calculations to be explained next. Volume (B) Type (A) 1. Subthr 2. Audible 3. Speech 4. Shout M a n a Noise M ab M M M M n ab n 11 3 n 12 3 n 13 3 n 14 3 M n Speech M ab M M M M n ab n 21 3 n 22 3 n 23 3 n 24 3 M n M b M M M M M G 5.0 n b n.1 6 n.2 6 n.3 6 n.4 6 N 24 SD G SS Total. To calculate SS Total, we simply ignore all of the levels of both factors, and think of the dataset as consisting of 24 individual observations with M G = 6.0. We would subtract M G from each of the 24 ys, square the resulting deviations, and sum them up over all of the observations within a group, and then over all levels of the volume and type factors, represented by below. That is,

3 SS Total = (y abi - M G ) 2 = (1-5.0) 2 + (4-5.0) 2 + (7-5.0) (3-5.0) 2 + (5-5.0) 2 + (4-5.0) 2 + (3-5.0) 2 + (2-5.0) 2 + (1-5.0) (6-5.0) 2 + (10-5.0) 2 + (8-5.0) 2 = = df = N - 1 = 24-1 = 23 If the overall standard deviation of the entire set of data is known, as above, then: SS Total = (N - 1)s G 2 = (24-1) = This represents the total variability in the 24 scores, which we want to partition into error, main effect of Type, main effect of Volume, and the interaction between Type and Volume. The df = N - 1 because we subtract one M G from N observations. SS Error. As in the single factor Between-S design, error is the variation of scores within each unique condition around the mean for the condition. In the factorial design, each unique condition is defined by a level of factor A (Type) and a level of factor B (Volume). Thus, there are 8 unique conditions, each with its own M ab and the error is the squared deviations of the three observations about M ab, summed across the three observations, and then across the four levels of B and the two levels of A (i.e., over all 8 conditions). That is, SS Error = (y abi - M ab ) 2 = (1-4.0) 2 + (4-4.0) 2 + (7-4.0) (3-4.0) 2 + (5-4.0) 2 + (4-4.0) 2 + (3-2.0) 2 + (2-2.0) 2 + (1-2.0) (6-8.0) 2 + (10-8.0) 2 + (8-8.0) 2 = = 60.0 = SS ab = SS 11 + SS SS 24 = = (n ab - 1)s ab 2 df = (n ab - 1) = (n 11-1) + (n 12-1) (n 24-1) = (3-1) +... = 16 = N - AxB = 24-2x4 = 24-8 = 16 SS Main Effects. To calculate SS for the main effects of A and B, we treat the study as though the other factor does not exist. Averaged across the Volume factor, Type (or A) produces two means, each based on 12 observations; these are denoted in general as M a. For the first level of A (i.e., a = 1), M 1. = 4.0, and for the second level of A (i.e., a = 2), M 2. = 6.0. The reason for the period (.) in the subscript is to make clear that the 1 and 2 subscripts refer to the levels for A, rather than B. Similarly, averaged across the Type factor, Volume (or B) produces four means, each based on 6 observations; these are denoted in general as M b, giving M.1 = 3.0, M.2 = 4.0, M.3 = 7.0, and M.4 = 6.0. The period in the first position indicates that the subscripts denote the levels of factor B. We now calculate SS A and SS B essentially in the same manner as the single-factor SS for treatment. That is, SS A = n a (M a - M G ) 2 = 12( ) ( ) 2 = 12x x1.0 2 = 24.0 df A = A - 1 = 2-1 = 1 SS B = n b (M b - M G ) 2 = 6( ) 2 + 6( ) 2 + 6( ) 2 + 6( ) 2 = 6x x x x1.0 2 = 60.0 df B = B - 1 = 4-1 = 3

4 SS Interaction. We later consider several ways to compute SS AxB independently, but for now we simply calculate it by subtraction. SS AxB is any variability left over from Total after Error and Main Effects of A and B are calculated; that is, SS AxB = SS Total - SS Error - SS A - SS B = = 36.0 df AxB = df Total - df Error - df A - df B = = 3 We now have everything needed to carry out ANOVA for the Between-S two-factor design, as shown below. The results correspond exactly with those produced by SPSS Source SS df MS F F.05 Type Rej H0 Volume Rej H0 T x V ?? Error Total SPSS Analyses for Between-S Factorial Design The standard way to enter data for the Between-S Two-Factor design is to enter three values for each case: one value represents the level of factor A, another value represents the level of factor B, and the final value represents the dependent variable in the study. The following syntax creates a data set with 24 rows and 3 values for each row or case, which is analyzed by subsequent ANOVAs. DATA LIST FREE / typ vol mis. BEGIN DATA END DATA. GLM err BY typ vol /PRINT = DESCR /PLOT = PROFILE(vol BY typ). typ vol Mean Std. Deviation N Total Total Total Total

5 Tests of Between-Subjects Effects Corrected Model (a) SS A + SS B + SS AxB Intercept x (5.0-0) 2 typ vol typ * vol Error Total SS Total Corrected Total SS Total a R Squared =.667 (Adjusted R Squared =.521) R 2 = 120.0/180.0 The primary GLM results agree with our earlier calculations of SSs and dfs. Dividing SS by df produces the four means squares required to test our three hypotheses. MS Error = 60.0 / 16 = 3.75, which serves as the denominator for the three critical tests. The notes above explain some of the secondary quantities calculated by GLM. The main effect tests, lead to the following conclusions: Reject H 0 : Noise = Speech and Reject H 0 : Vol-1 = Vol-2 = Vol-3 = Vol-4 In both cases, we accept the H a that one or more equality is false. The conclusion about the interaction is somewhat ambiguous; the F is very close to significant, p =.052, and the pattern shown in the graph indicates an interaction. Specifically, the effect of Volume level is quite different for the Noise and Speech distractors, having a much more marked effect for Speech. Equivalently, the difference between Speech and Noise is more marked for some levels of Volume (i.e., 3 and 4) than for other levels of Volume (1 and 2). Moreover, closer examination of interaction analyses will reveal that the standard F test for the interaction can be insensitive for many observed interactions. The following MANOVA analysis duplicates the various quantities calculated earlier and produced by GLM. MANOVA mis BY typ(1 2) vol(1 4) /PRINT = CELL. FACTOR CODE Mean Std. Dev. N 95 percent ConfInt typ 1 vol vol vol vol typ 2 vol vol vol vol For entire sample Source of Variation SS DF MS F Sig of F WITHIN CELLS typ vol typ BY vol (Model) (Total) R-Squared =.667 Adjusted R-Squared =.521

6 Using SPSS to Compute Various Sss There are various ways to get SPSS to help with calculating SSs for the Between-S factorial. One informative way is to use GLM s ability to compute and save predicted (and residual) values. The next few pages illustrate the process. The various scores being produced are shown after the final analysis. The actual ANOVAs are not shown. GLM mis /SAVE PRED(mg). Saves M G as mg... COMPUTE tot = mis - mg. y - M G COMPUTE tot2 = tot**2. DESCR tot tot2 /STAT = SUM. N Sum tot tot = SS Total GLM mis BY typ vol /SAVE PRED(mab). Saves M ab as mab... COMPUTE err = mis - mab. y - M ab COMPUTE err2 = err**2. DESCR err err2 /STAT = SUM. N Sum err err = SS Error GLM mis BY typ /SAVE PRED(ma). Saves M a as ma... COMPUTE amain = ma - mg. M a - M G COMPUTE amain2 = amain**2. DESCR amain amain2 /STAT = SUM. N Sum amain amain = SS A = SS Type GLM mis BY vol /SAVE PRED(mb). Saves M b as mb... COMPUTE bmain = mb - mg. M b - M G COMPUTE bmain2 = bmain**2. DESCR bmain bmain2 /STAT = SUM. N Sum bmain bmain = SS B = SS Volume

7 The following table shows the primary variables created by the preceding commands. FORMAT mg tot mab err ma amain mb bmain (F5.1). LIST typ vol mis mg tot mab err ma amain mb bmain. y M G y-m g M ab y-m ab M a M a -M G M b M b -M G typ vol mis mg tot mab err ma amain mb bmain =SS Total 2 =SS Error 2 =SS Type 2 =SS Volume Computing SS for the Interaction Calculating the interaction by subtraction demonstrates that the interaction is variability that cannot be accounted for by main effects or error, which is technically correct but perhaps has limited intuitive meaning. But it is possible to calculate the interaction in ways that reveals somewhat more clearly what SS Interaction represents. Akin to the subtraction of SSs, for example, we could calculate interaction by determining if there is any variability left in the cell means (the M ab s) when the main effects are removed. Alternatively, we could determine whether the observed cell means differ from the cell means expected if there were only main effects and no interaction. Both approaches lead to the same conclusion. Below are the 8 cell means (M ab s) calculated earlier, along with the row (M a s) and column (M b s) means, and the grand mean (M G ). Also, the effect of A for a = 1 and a = 2 is shown in the column headed M a - M G, and the effect of B for b = 1 to b = 4 is shown in the row labelled M b - M G. These are the effects that need to be subtracted from the cell means to remove main effects. The subsequent two tables show the results of subtracting these main effects. The left hand table shows the result from subtracting the main effects of factor B. The two original means for Volume 1 (i.e., b = 1) are adjusted by subtracting the B effect of Volume 1, which is - 2.0; to illustrate, M 11 = 4.0, and M 11 '= = 6.0, and M 21 = 2.0, and M 21 '= = 4.0. The cell means for Volumes 2, 3, and 4 are adjusted by subtracting their Volume effects: -1.0, +2.0, and +1.0, respectively. The resulting cell means are denoted M ab to indicate one effect has been adjusted. Note that when the main effect means for B are recalculated using these adjusted cell means, all the M b s = 5.0 = M G ; that is, there is no longer any main effect of B. The table to the right removes the main effects for A from the M ab s in a similar manner. Now all the row and column means (i.e., the main effect means) equal M G. If there were no interaction, then the 8 adjusted cell means would also equal M G = 5.0. But they do not, indicating the presence of

8 interaction in the data. That is, there is variability due to the unique combination of specific levels of Factor A and specific levels of Factor B. M ab s Volume (B) Type M a M a -M G (A) M b M G 5.0 M b -M G M ab = M ab s-(m b -M G ) Vol (B) M ab = M ab s-(m a -M G )Vol(B) Type M a M a -M G Type M a (A) (A) M b M G 5.0 M b M G 5.0 The final step in the calculation of SS Interaction is to calculate the deviation of each M ab from M G, square the deviations, multiply by the number of observations in each cell (i.e., n ab ), and sum to get a total. The resulting deviations are shown in the next table. We discuss shortly what these deviations represent. M ab - M G Volume Type Therefore, SS Interaction = 3( ) = 36.0, the same value we obtained earlier by subtraction of SSs. The above operations are represented by the second formula 38 on the formula sheet. The second approach to SS Interaction is to calculate what the cell means would have been if there were only main effects and no interaction, and then determine how much the observed cell means (the M ab s) deviate from the no interaction cell means. These deviations, which will equal those we just computed, will be squared, multiplied by n ab, and summed to obtain SS Interaction. What this approach makes clear is that SS Interaction represents how much the observed cell means deviate from those expected if there were no interaction. To determine the expected cell means given no interaction, we simply add the main effects of A and B to the grand mean, as illustrated in the left table below. The deviations of observed cell means (M ab ) from these predicted cell means if no interaction are shown in the right table and agree with those above. M ab = M G + (M a - M G ) + (M b - M G ) M ab - M ab Volume (B) Volume (B) Type M a M a -M G Type (A) (A) M b M G 5.0 M b -M G

9 The critical thing to note about the predicted cell means above is that the main effects of A are exactly the same at every level of B. That is, the difference between Noise and Speech is exactly 2.0 units for Volumes 1, 2, 3, and 4, indicating 0 interaction. This could also be stated as the main effect deviations for the two levels of A (i.e., -1 and +1) are exactly the same at every level of Volume. Equivalently, the main effects of B are exactly the same at every level of A. That is, the main effect deviations for Volume (-2, -1, +2, +1) are duplicated exactly for the Noise and Speech conditions. For the noise condition, the deviations of 2, 3, 6, and 5 from the row mean of 4.0 are -2, -1, +2, and +1. For the speech condition, the deviations of 4, 5, 8, and 7 from the row mean of 6.0 are -2, -1, +2, and +1, exactly the same as for the Noise condition and the main effect of Volume. Another way of saying this is that a graph of the M ab s above would produce perfectly parallel lines, as we ll see shortly, indicating no interaction. The interaction deviations in the right table and the earlier table indicate how far away the observed cell means are from perfectly parallel lines (i.e., from 0 interaction). The above operations are given as the first formula 38 on the formula sheet. Getting SPSS to Compute SS Interaction We earlier did much of the work necessary for GLM to compute SS Interaction when we computed the main effects and error using GLM. Reviewing the earlier commands will show that we had GLM save predicted scores for M G, M ab, M a, and M b and that we further calculated M a - M G (called amain) and M b - M G (called bmain). These are all the elements needed to compute the interaction according to the two procedures described above, first removing main effects and second generating predicted cell means for only main effects. Two ways are shown to obtain predicted cell means with only main effects, one using COMPUTE statements and a second method using the /DESIGN option on GLM. The default factorial GLM tests main effects and interaction and corresponds to /DESIGN typ vol typ BY vol. Below, an explicit /DESIGN typ vol instructs GLM to calculate main effects only and use only those main effects to generate predicted scores. COMPUTE mabsubmain = mab - amain - bmain. subtract main effects from cell means COMPUTE intone = mabsubmain - mg. COMPUTE intone2 = intone**2. DESCR intone2 /STAT = SUM. N Sum intone COMPUTE mabmain = mg + amain + bmain. COMPUTE inttwo = mab - mabmain. COMPUTE inttwo2 = inttwo**2. add main effects to grand mean DESCR inttwo2 /STAT = SUM. N Sum inttwo *alternative way to get expected cell means based on just main effects. GLM mis BY typ vol /SAVE PRED(mabmaintwo) /DESIGN typ vol. Corrected Model (a) Intercept typ vol Error Total Corrected Total

10 FORMAT typ vol (F1.0) mis (F2.0) mg TO mabmaintwo (F4.1). LIST. typ vol mis mg mab ma amain mb bmain mabsubmain intone intone2 mabmain inttwo inttwo2 mabmaintwo =SS Interaction 2 =SS Interaction Graphs and Interaction There are several ways to graph the results of factorial ANOVA, and graphs are often the best way to show interaction effects. The basic factorial graph for a two-factor study uses the horizontal axis for one of the factors (usually the one with the most levels, here the vol factor with four levels), and separate lines + markers for the other factors (here typ, with two levels). The graph presented earlier was created by the following GLM command, specifically the /PLOT = PROFILE(effects) option. The effects can be either main effects (e.g., PROFILE(vol)) or interactions, as shown here. GLM mis BY typ vol /PRINT = DESCR /PLOT = PROFILE(vol BY typ). This same graph could have been produced from menus by Graph Line Multiple Define, which brings up the dialogue box shown to the right. Users then specify which factor goes on the Category Axis (vol has been inserted here) and which factor is used to Define Lines (typ here). Users also have the option of what to plot; here we want to plot the means for mis, our dependent variable. Clicking Ok will create the basic graph, which would generally be edited further in the chart editor. Understanding of the factorial ANOVA, especially the interaction, can benefit from plotting some of the other quantities that we have produced, namely the

11 expected cell means with just main effects and no interaction, and the adjusted cell means with main effects removed (i.e., pure interaction). These graphs are shown below. The earlier graph of the means is reproduced for comparison. The original (left) and main effects only (right) graphs illustrate the logic of the interaction deviations. In essence each deviation represents how far its cell mean is from where it would be if there were only main effects and no interaction, as represented in the right hand graph. M 11 = 2.0 (i.e., speech, vol 1), for example, is 2 units lower than it should be (4.0) if there were no interaction. The sum of all these squared deviations represents the total evidence for an interaction between volume and type of distractor in this study. The third graph shows the adjusted cell means with main effects removed. This graph represents pure interaction.

12 Examples of Factorial ANOVA Outcomes The following analyses illustrate some of the possible outcomes for a factorial 2 x 3 design. GLM y1 BY a b /PRINT = DESCR Total Total Total Total Total Corrected Model.000(a) a b a * b Total Corrected Total a R Squared =.000 (Adjusted R Squared = -.208) GLM y2 BY a b /PRINT = DESCR Total Total Total Total Total Source Type III Sum of df Mean Sq uare F Sig. Corrected Model (a) a b a * b Total Corrected Total a R Squared =.250 (Adjusted R Squared =.094)

13 GLM y3 BY a b /PRINT = DESCR Total Total Corrected Model (a) a b a * b Total Corrected Total a R Squared =.333 (Adjusted R Squared =.194) GLM y4 BY a b /PRINT = DESCR Total Total Total Total Corrected Model (a) a b a * b Total Corrected Total a R Squared =.455 (Adjusted R Squared =.341)

14 GLM y5 BY a b /PRINT = DESCR Total Total Total Corrected Model (a) a b a * b Total Corrected Total a R Squared =.250 (Adjusted R Squared =.094) GLM y6 BY a b /PRINT = DESCR Total Total Total Total Total Corrected Model (a) a b a * b Total Corrected Total a R Squared =.400 (Adjusted R Squared =.275)

15 GLM y7 BY a b /PRINT = DESCR Total Total Total Total Corrected Model (a) a b a * b Total Corrected Total a R Squared =.455 (Adjusted R Squared =.341) GLM y8 BY a b /PRINT = DESCR Total Total Total Total Corrected Model (a) a b a * b Total Corrected Total a R Squared =.538 (Adjusted R Squared =.442)