4:3 LEC - PLANNED COMPARISONS AND REGRESSION ANALYSES

Transcription

1 4:3 LEC - PLANNED COMPARISONS AND REGRESSION ANALYSES FOR SINGLE FACTOR BETWEEN-S DESIGNS Planned or A Priori Comparisons We previously showed various ways to test all possible pairwise comparisons for a study in which there were no expectations about the outcome. Post hoc tests do not always lead to clean conclusions. In the psychopathy study, for example, the Tukey and Bonferroni tests led to the conclusion that groups 2, 1, and 3 did not differ from one another, that groups 3 and 4 did not differ, but groups 2 and 1 differed from group 4. The fact that group 3 is not significantly different from 2 and 1 or from 4 creates a problem for drawing a tidy conclusion. In contrast, the LSD and SNK procedures did in that particular study lead to the tidy conclusion that group 4 students (i.e., business students) obtained significantly higher psychopathy scores than all three groups from other faculties. A much better approach to this and most studies (i.e., more certain of giving a sensible pattern) is to make predictions about the expected pattern of results and perform contrasts that correspond to those expectations. An added benefit of such planned comparisons (called contrasts) is that the omnibus F need not be significant to carry out these follow-up analyses, a benefit since it is possible for the omnibus F to be non-significant while specific contrasts are significant. There are also costs and risks associated with planned comparisons, however. One cost is that researchers generally are restricted to a smaller number of comparisons than with post hoc procedures; one guideline for smaller is to use the df for the factor to limit the comparisons (i.e., number of comparisons = k - 1). A risk is that predictions might be wrong, leading to analyses that do not in fact correspond well to the observed data. The availability of prior studies and well-founded theories will determine how great this risk is. Planned comparisons make use of contrasts (also called linear contrasts), which essentially are sets of coefficients (numbers), one for each of the k groups, that sum to 0. These coefficients (denoted by c j = c 1, c 2, c k ) define patterns expected in the data and can be tested for significance. With k = 4, possible patterns include the following (labels in the left column are explained later): Group c j C12v Clinear Ccurve C123v C12v C1v Tests with contrasts essentially ask whether these patterns correlate well enough with the pattern observed in the cell means to reject the null hypothesis of no relationship. Some contrasts can be interpreted as differences between M j s, as in C12v34, C123v4, C12v3, and C1v2 above. Specifically, M j s for groups with negative numbers are being contrasted with M j s for groups with positive numbers. Other contrasts define more complex patterns or differences. The coefficients for Clinear increase in a linear fashion from group 1 to group 4. A systematic increase or decrease in means would correlate positively or negatively with these coefficients. Ccurve defines a curvilinear or nonlinear pattern. If the means showed a U shaped or inverted U shape pattern, they would correlate with Ccurve s coefficients. Whether the correlation is positive or negative is incidental to the statistical test, although important for interpretation of the results. Although not strictly required, we will limit our analyses to k - 1 contrasts defined this way with the additional restriction that the contrasts are orthogonal to one another; orthogonal simply means

2 independent or uncorrelated and can be determined easily for any set of contrasts. Specifically, two sets of contrast coefficients, c j and c j, are orthogonal if c j c j = 0, that is, if the cross products of the coefficients sum to 0. The lack of correlation given this condition occurs because SCP = 0 if c j c j = 0 for contrasts (recall the computational formula for SCP = xy - (xy)/n and that for contrasts x = 0 and y = 0). The test of orthogonality is illustrated below for several pairs shown above. c j c j C12v Clinear C12v34 x Clinear = 9 NOT orthogonal Clinear Ccurve Clinear x Ccurve Orthogonal C123v C12v Orthogonal C12v C1v Orthogonal C123v C1v Orthogonal Given this test of independence (orthogonality), the tricky part is often how to generate the k - 1 orthogonal contrasts, although some common patterns will emerge with practice. The third pair, for example, is orthogonal because C12v3 compares groups 1 and 2 with 3, and these three groups were all coded -1 (i.e., the same) in the preceding contrast, C123v4. The last three contrasts are mutually orthogonal. Moreover, note that they actually correspond to meaningful comparisons for the psychopathy study. C123v4 would compare the non-business students with business students (i.e., the group we predicted to score higher on psychopathy). This is our primary prediction. C12v3 compares humanities and social science students with natural science students, and C1v2 compares humanities students with social science students. These latter two hypotheses complete the required k - 1 contrasts, but are of less theoretical interest than C123v4, and in some cases of no interest. Once contrasts are specified, the next step is to calculate a linear contrast score (L) for each contrast. This is the sum of the cross products of the contrast coefficients and the corresponding means. L can be tested for significance using a t test or else converted to a SS and tested for significance using an equivalent F test. Here are relevant formula and calculations, where MSE is MS Error from the omnibus ANOVA :

3 1-Hum 2-SS 3-NS 4-Bus M j L SS L C123v = 5 x / 12 C12v C1v SS L = 70.0 = SS Major To illustrate the above calculations for the first contrast, C123v4, L = -1x x x x8.0 = +12.0, and SS L = 5 x / 12 = Each L and SS L can be tested for significance but first note that the sum of the SS L s is equal to the SS Between from the omnibus ANOVA. Selecting k - 1 orthogonal contrasts has resulted in SS Between being partitioned into three components, one associated with each of our contrasts. Of particular note, most of the 70.0 units of variability loaded on our first contrast, as predicted, whereas the omnibus ANOVA divided 70.0 into k - 1 equal units. The contrasts can be tested for significance using either a t-test (df = N - k) or an F-test (df = 1, N - k). For the first contrast, These contrasts correspond to differences between means and we can therefore show that the t and F statistics just calculated using L agree with earlier t and F tests that we have studied for differences between means. To illustrate with contrast C123v4, given M 123 = ( ) = 4.0, t 123v4 = ( ) / SQRT(4.0(1/15 + 1/5)) = 4.0 / = = t L1. And SS 123v4 = 15( ) 2 + 5( ) 2 = 15 x x = 60.0 = SS L1. The strength of the contrast approach is that it will work as well for contrasts that do not represent simple differences between means (e.g., Linear patterns). Planned Contrasts in SPSS ONEWAY, GLM, and MANOVA all provide ways to conduct planned contrasts. ONEWAY produces results shown below. Two ts are reported, one assuming equal variances and the other not requiring that assumption. We consider only the first t. Note the various correspondences with our preceding analysis: L = Value of Contrast, SE, t, and df. The contrast is significant, even by a nondirectional test, although planned contrasts would generally involve directional predictions. ONEWAY does not provide a partitioning of the SS Between. ONEWAY psypath BY major /CONTRAST = Sum of Squares df Mean Square F Sig. Between Groups Within Groups Total Contrast Value of Std. Error t df Sig. (2-tailed) Contrast psypath Assume equal variances Does not assume equal variances The following analyses illustrate various aspects of the GLM procedure; the factor name must be

4 specified in the /CONTRAST command, which is followed by values that specify the contrast, either numerical values with the SPECIAL option, or a keyword option for some built-in contrasts. The first analysis shows a single contrast producing values for the t (i.e., numerator, denominator, and significance) but not the actual t, as well as an ANOVA summary table for the contrast, including SS L, F L, and significance. The results agree with our earlier calculations. GLM psypath BY major /CONTRAST(major) = SPECIAL( ). Custom Hypothesis Tests major Special Contrast Dependent Variable psypath L1 Contrast Estimate Hypothesized Value 0 Std. Error Sig..001 Contrast The next analysis shows all three contrasts specified within a single contrast option. The SPECIAL contrast is followed by k - 1 sets of k coefficients, corresponding to our three contrasts. Other built-in options for contrasts appear in the menu system for GLM. GLM produces results for three tests of significance, again providing basic quantities for the t statistic, but only a single ANOVA summary table, aggregating together the SSs for the three contrasts. This ANOVA corresponds to our original omnibus ANOVA and adds little here. Only the first contrast, that comparing business students to the three other groups, is significant. GLM psypath BY major /CONTRAST(major) = SPECIAL( ). Custom Hypothesis Tests L1 Contrast Estimate Std. Error Sig..001 L2 Contrast Estimate Std. Error Sig..190 L3 Contrast Estimate Std. Error Sig..441 Contrast In the following GLM analysis, three separate CONTRAST subcommands were specified, one for each contrast. This produces a separate ANOVA for each contrast, illustrating that SS Treatment has been partitioned into three independent components. Note that the p value for contrast 3 corresponds to the corresponding LSD result because this is a pairwise comparison and no adjustment has been made.

5 GLM psypath BY major /CONTRAST(major) = SPECIAL( ) /CONTRAST(major) = SPECIAL( ) /CONTRAST(major) = SPEC( ). Custom Hypothesis Tests #1 L1 Contrast Estimate Std. Error Sig..001 Contrast Custom Hypothesis Tests #2 L1 Contrast Estimate Std. Error Sig..190 Contrast Custom Hypothesis Tests #3 L1 Contrast Estimate Std. Error Sig..441 Contrast A final way to perform contrasts with GLM is to use the LMATRIX option, which is particularly handy for certain analyses of factorial designs. For single factor designs, the results are identical to those obtained with the CONTRAST option. GLM psypath BY major /LMATRIX major Custom Hypothesis Tests L1 Contrast Estimate Std. Error Sig..001 Contrast MANOVA has many options for how results of planned contrasts are reported. One unique feature of MANOVA is that it requires that k - 1 orthogonal contrasts be specified; that is, it is not possible to request only some of the contrasts. Moreover, if the specified contrasts are not orthogonal, MANOVA will change them to make them orthogonal; so some caution is required. The first MANOVA shows how to specify contrasts. The format is similar to that for GLM with one exception; in addition to the k - 1 orthogonal sets of k coefficients within the SPECIAL brackets, it is necessary to enter k 1s first (these correspond to the grand mean). The default analysis results in t- tests following the omnibus ANOVA. The results agree with those observed previously. These t tests are also reported in other forms of MANOVA, but are deleted as they are redundant with those shown here.

6 MANOVA psypath BY major(1 4) /CONTRAST(major) = SPECIAL( ). Estimates for psypath --- Individual univariate.9500 confidence intervals Parameter Coeff. Std. Err. t-value Sig. t The next MANOVA includes a subcommand that requests SPSS to partition every Between-S effect with df > 1 (i.e., for factors with k > 2) into single df effects, using the single df contrasts specified in the CONTRAST subcommand (or default contrasts if none are specified). The result is that the overall Major effect is partitioned as shown earlier. This analysis, also makes clear that the omnibus F for a Between-S factor is the average of the single df Fs; that is, ( )/3 = = F Major. The fact that large and small single df Fs are averaged together to produce the omnibus F means that the omnibus F could be not significant and a specific planned contrast significant if it captures enough of the variability among the means. MANOVA psypath BY major(1 4) /PRINT = SIGNIFICANCE(SINGLEDF) /CONTRAST(major) = SPECIAL( ). Source of Variation SS DF MS F Sig of F WITHIN CELLS major ST Parameter ND Parameter RD Parameter (Model) (Total) The final MANOVA illustrates how to request single df F tests using the DESIGN option. Instead of an overall major effect with df = 3, the default DESIGN for MANOVA, the following DESIGN statement asks for three separate components of the major effect, with major(1) referring to the first contrast, major(2) to the second contrast, and major(3) to the third contrast. Note that the numbers in parentheses represent the k - 1 = 3 contrasts and not the k = 4 levels for major. MANOVA psypath BY major(1 4) /CONTRAST(major) = SPECIAL( ) /DESIGN major(1) major(2) major(3). Source of Variation SS DF MS F Sig of F WITHIN+RESIDUAL MAJOR(1) MAJOR(2) MAJOR(3) (Model) (Total) Regression Analyses for the Between-S Single Factor Design Recall from last term that multiple regression can be used to conduct ANOVA for differences between groups. The basic principle was that p, the number of predictors, must equal k - 1. That is, for two groups, p = 1; for three groups, p = 2; and so on. At the time, however, we glossed over how these predictors might be created and interpreted. One answer to that question is the p predictors may correspond to k - 1 orthogonal contrasts. The following analysis illustrates the appropriate analysis for

7 the psychopathy study using our earlier set of contrasts. The first step is to create three predictors, one corresponding to each contrast; these predictors are created using RECODE statements and appear in a later listing of the data file. Then the dependent variable psypath is regressed on these three predictors. RECODE major (1 2 3 = -1) (4 = 3) INTO c123v4. RECODE major (1 2 = -1) (3 = 2) (4 = 0) INTO c12v3. RECODE major (1 = -1) (2 = 1) (3 4 = 0) INTO c1v2. REGRESS /DESCR /DEP = psypath /ENTER c12v3 c1v2 /ENTER c123v4 /SAVE PRED(prdp.m) RESI(resp.m). Mean Std. Deviation N psypath c123v Predictors are contrasts, Ms = 0 c12v c1v psypath c123v4 c12v3 c1v2 c123v Predictors are orthogonal rs = 0 c12v c1v The preliminary descriptive statistics reveal some interesting aspects of this analysis. First, note that rs = 0 for all three correlations between the predictors. This demonstrates that our three contrasts are orthogonal to one another. Also note that each of the contrasts has a non-zero r with the dependent variable, although r = 0 is possible for contrasts that are independent of the pattern in the means. Squaring these rs and multiplying by SS Total gives the same results as our earlier calculations for the SS L s. To illustrate, x = = SS for the C123v4 contrast. The remainder of the analysis captures all aspects of our earlier ANOVA analyses. The overall ANOVA table for Model 2 with all three predictors corresponds to the omnibus F in all details, essentially because the predicted values correspond to the group means (see listing below) and the residual values correspond to y - M j, the source of MSE in ANOVA. REGRESS /STAT = DEFAU CHANGE /DESCR /DEP = psypath /ENTER c12v3 c1v2 /ENTER c123v4 /SAVE PRED(prdp.m) RESI(resp.m). Model R R Adjusted Std. Error of Change Statistics Square R Square the Estimate R Square Change F Change df1 df2 Sig. F Change 1.273(a) (b) Model Sum of Squares df Mean Square F Sig. 1 Regression (a) Residual Total Regression (b) Residual Total Model Unstandardized Standardized t Sig. Coefficients Coefficients B Std. Error Beta 2 (Constant) c12v c1v c123v With respect to individual predictors, the results correspond exactly with our earlier calculations and output from ONEWAY, GLM, and MANOVA, including: the ts and p values for the three contrasts,

8 and the F L and F Change for c123v4. Also, note that SS Change = = 60.0 = SS L for c123v4. VARIABLE LABEL prdp.m '' resp.m ''. LIST. major psypath c123v4 c12v3 c1v2 prdp.m resp.m One advantage of multiple regression is that predictors need not be orthogonal, because regression is powerful enough to determine the unique contribution of each predictor. It is therefore possible to create predictors that correspond to some (but not all six) of the pairwise comparisons tested by the post hoc procedures. The following analysis compares business students (group 4) to each of the other groups in turn. The overall ANOVA remains the same, and tests for individual predictors agree with the corresponding pairwise comparisons from earlier. Note in the descriptive statistics that the predictors do not correspond to contrasts (Ms 0) and are not orthogonal to one another (rs = -.333). RECODE major (4 2 3 = 0) (1 = 1) INTO c4v1. RECODE major (4 1 3 = 0) (2 = 1) INTO c4v2. RECODE major (4 1 2 = 0) (3 = 1) INTO c4v3. REGRESS /DESCR /DEP = psypath /ENTER c4v1 c4v2 c4v3 /SAVE PRED(prdp.m2) RESI(resp.m2). Mean Std. Deviation N psypath c4v Predictors not contrasts, Ms 0 c4v c4v psypath c4v1 c4v2 c4v3 c4v Predictors not orthogonal, rs 0 c4v c4v Model R R Square Adjusted R Std. Error of Square the Estimate 1.723(a) Model Sum of Squares df Mean Square F Sig. 1 Regression (a) Residual Total

9 Model Unstandardized Standardized t Sig. Coefficients Coefficients B Std. Error Beta 1 (Constant) c4v c4v c4v VARIABLE LABEL prdp.m2 '' resp.m2 ''. LIST major psypath c4v1 c4v2 c4v3 prdp.m2 resp.m2. major psypath c4v1 c4v2 c4v3 prdp.m2 resp.m SPSS Menu System and Alternative Types of Contrasts Although some contrasts correspond to differences between means, that is not the case for all contrasts. Polynomial contrasts, for example, are orthogonal contrasts that partition SS Treatment into linear, quadratic, cubic, and so on components, with each component representing a linear or nonlinear pattern. For k = 3, the linear coefficients are and the quadratic coefficients are The former capture a consistent increase or decrease in the means, whereas the latter capture any curvilinear pattern, either U-shaped or inverted U-shaped. For k = 4, the linear coefficients are , the quadratic are , and the cubic are Note the increasing number of changes in direction (bends) necessary to capture all the possible variability in means as k increases. These and polynomial coefficients for larger values of k can be found in Appendix T-5 of the text. It is not always necessary for users to specify the actual coefficients to use for a contrast. Both MANOVA and GLM have built-in coefficients for certain commonly used contrasts, including contrasts presented earlier (called Difference contrasts) and Polynomial contrasts. The MANOVA analysis on the next page requests polynomial contrasts for our current study. The linear and quadratic components are significant, indicating that psychopathy increases significantly from Humanities to Social Science to Natural Science to Business majors, but that there are significant deviations from a purely linear relationship. Note that the linear contrast, the most significant, accounts for only 49.0 units of variability, whereas our earlier first contrast ( ) accounted for 60.0 units. That is, the earlier contrast provided a better single df fit to the data. Box 1 shows the selection of contrasts from the GLM Menu. After specifying the overall design for the study, the Contrasts option is selected and brings up the top option screen. For each factor in the design, it is possible to select one of the available contrast types and click on Change to insert it after the factor name. The default contrast for each factor is None. It is also possible to get a (somewhat cryptic) description of each of the factor types available in GLM. The description for Difference contrasts

10 corresponds to our initial contrasts for this study. Note that in the final syntax only the label for the contrast type is specified and that all k - 1 contrasts are carried out. One warning about SPSS contrasts. GLM and MANOVA may sometimes use different numerical values for contrast coefficients that demonstrate the same pattern as the integer values we use. In these cases, the final statistics (e.g., t, F, SS) will agree, but not some intermediate values. Manual calculations for the linear contrast shown below, for example, produces L = -3x x x5.0 +3x8.0 = 14.0 (vs below), which gives SS Linear = 5x /20 = 49.0, the value shown below. The linear coefficients used by SPSS were normalized: , , , Squared normalized coefficients sum to 1.0 (i.e., c j 2 = 1.0, when coefficients are normalized). MANOVA psypath BY major(1 4) /PRINT = SIGNIF(SINGLE) /CONTRAST (major) = POLYNOMIAL. Source of Variation SS DF MS F Sig of F WITHIN CELLS major ST Parameter ND Parameter RD Parameter (Model) (Total) Estimates for psypath --- Individual univariate.9500 confidence intervals Parameter Coeff. Std. Err. t-value Sig. t Lower -95% CL- Upper Box 1. GLM Menu and Contrasts: Selecting, Showing Definition, and Subsequent Syntax.