Chapter 14. One-Way Analysis of Variance for Independent Samples Part 2

Transcription

1 Tuesday, December 12, 2000 One-Way ANOVA: Independent Samples: II Page: 1 Richard Lowry, All rights reserved. Chapter 14. One-Way Analysis of Variance for Independent Samples Part 2 For the items covered in Part 2 of this chapter, you will need access to the following summary information from the illustrative analysis performed in Part 1. (Click here if you wish to see full array of data on which this analysis was performed.) =28.86 M b =25.04 M c =22.50 M d Source SS df MS F P between groups ("effect") within groups ("error") < TOTAL Post-ANOVA Comparisons: the Tukey HSD Test A significant F -ratio tells you only that the aggregate difference among the means of the several samples is significantly greater than zero. It does not tell you whether any particular sample mean significantly differs from any particular other. For some research purposes this might be entirely sufficient. Since the investigators in the present example regard their experiment with laboratory rats as only a first step in testing the medication, we can imagine they might be content simply with the global conclusion suggested by the graph of their data: namely, that the curve of "pull" (presumably a reflection of the effect of the medication) slopes downward from A to B to C, then levels off between C and D. There are, however, many situations in which the investigator might wish to

2 Tuesday, December 12, 2000 One-Way ANOVA: Independent Samples: II Page: 2 determine specifically whether significantly differs from M b, or M b from M c, and so on. As noted toward the beginning of Chapter 13, this comparison of sample means two at a time cannot be done by way of simple t-tests, because it potentially involves 3 or more comparisons, depending on the number of samples, k, involved in the original analysis. With k=3, there would be 3 potential comparisons: A B, A C, B C With k=4, as in the present case, there would be 6 potential comparisons: A B, A C, A D, B C, B D, C D With k=5, there would be 10: A B, A C, A D, A E, B C, B D, B E, C D, C E, D E and so forth. The performance of any one or several of these pair-wise comparisons requires a procedure that takes the full range of potential comparisons into account. The subject of post-anova comparisons is a rather complex one, and most of it lies beyond the scope of an introductory presentation. I will describe here only one of the available procedures, which I think will serve the beginning student well enough for most practical purposes. It goes under the name of the Tukey HSD test, the "HSD" being an acronym for the forthright phrase "honestly significant difference." The Tukey test revolves around a measure known as the Studentized range statistic, which we will abbreviate as Q. For any particular pair of means among the k groups, let us designate the larger and smaller as M L and M S, respectively. The Studentized range statistic can then be calculated for any particular pair as Q = M L M S sqrt[ms wg / where MS wg is the within-groups MS obtained in the original analysis and is the number of values of X i per sample ("p/s"=per sample). For the present example, MS wg =7.27 and =5. If the k samples are of different sizes, the value of can be set as equal to the harmonic mean of the sample sizes. For k=3, this would be 3 = (1/N a )+(1/N b )+(1/N c )

3 Tuesday, December 12, 2000 One-Way ANOVA: Independent Samples: II Page: 3 For k=4, 4 = (1/N a )+(1/N b )+(1/N c )+(1/N d ) And so on for k=5, k=6, etc. If the k samples are of different sizes, the value of can be set as equal to the harmonic mean of the sample sizes. For k=3, this would be As it happens, you do not really need to worry about calculating Q, because there is a simpler way of applying the Tukey test. However, I will pause to calculate one instance of it, just to give you an idea of what it looks like. For the present example, =28.86, M b =25.04, MS wg =7.27, and =5. Thus, for the comparison between and M b the Studentized range statistic would be Q = sqrt[7.27 / 5 = 3.16 And similarly for any of the other pair-wise comparisons one might wish to make among the means of this particular set of 4 groups. In any particular case, this Studentized range statistic belongs to a sampling distribution defined by two parameters: the first is k, the number of samples in the original analysis; and the second is df wg, the number of degrees of freedom associated with the denominator of the F-ratio in the original analysis. Within any particular one of these sampling distributions you can define the value of Q required for significance at any particular level. The following three scrollable windows list the critical values of Q at the.05 and.01 levels of significance for three values of K and various values of df wg. These do not constitute the full table of critical values of Q, though they should be sufficient for most situations that the beginning student is likely to encounter. Partial Table of Critical Values of the Studentized Range Statistic, Q

4 Tuesday, December 12, 2000 One-Way ANOVA: Independent Samples: II Page: 4 [If the desired value of df wg does not appear in the table, refer to the next smaller value. Example: for df wg =33, use the tabled critical values for df wg =30. The values listed in these three tables have been calculated by the author using Microsoft Excel. For a complete table of critical values of Q, see E. Pearson and H. Hartley, Biometrika Tables for Statisticians, 3rd edition. New York, Cambridge University Press, In each table, the first column is df wg ; the second and third columns are the critical values of Q for the.05 and.01 levels of significance, respectively. [Interactive tables omitted from this print file. For each of the listings in these tables we will use the designations Q.05 and Q.01 to refer to the critical values for the.05 and.01 levels. For the present example, k=4 and df wg =16. Scroll down the window for k=4 to the point where df wg =16, and you will find Q.05 =4.05 and Q.01 =5.19. The Tukey HSD test then uses these critical values of Q to determine how large the difference between the means of any two particular groups must be in order to be regarded as significant. The other participants in this determination, MS wg and, are the same items you saw in the earlier formula for Q. The following two "HSD" formulas are simply algebraic jugglings of the original formula, in which the value of Q is set to one or the other of the two critical values, Q.05 and Q.01. For the.05 level: HSD.05 = Q.05 x sqrt [ MS wg = 4.05 x sqrt [ = 4.88 And for the.01 level: HSD.01 = Q.01 x sqrt [ MS wg = 5.19 x sqrt [ = 6.26 That is: In order to be considered significant at or beyond the.05 level, the difference between any two particular group means (larger smaller) must be equal to or greater than That is: In order to be considered significant at or beyond the.01 level, the difference between any two particular group means (larger smaller) must be equal to or greater than 6.26.

5 Tuesday, December 12, 2000 One-Way ANOVA: Independent Samples: II Page: 5 The blue entries in the following table show the differences between each pair of group means in our example. As you can see, two of the comparisons (A C and A D) are significant beyond the.01 level, while all the others fail to achieve significance even at the basic.05 level. A B =28.86 M b = HSD.05 = 4.88 A C =28.86 M c = HSD.01 = 6.26 A D =28.86 M d 6.56 B C M b =25.04 M c = B D M b =25.04 M d 2.74 C D M c =22.50 M d 0.20 Our investigators would therefore be able to conclude that 2 units and 3 units of the experimental medication each produced significantly lower mean levels of "pull" than was found in the zero-unit control group. They would not be able to conclude that the effect of 2 units or 3 units was significantly greater than the effect of 1 unit, nor that the mean "pull" of the 1-unit group was significantly smaller that that of the zero-unit control group. Please note carefully, however, that failing to find a significant difference between and M b would not entail that 1 unit of the medication has no effect at all. It merely means that the Tukey HSD test does not detect a significant difference between the two in this particular situation. If the investigators had found approximately the same array of group means with samples of twice the size (10 per group, rather than 5), they would very likely have found all of the pair-wise comparisons to be significant, except for the one between M c and M d.

6 Tuesday, December 12, 2000 One-Way ANOVA: Independent Samples: II Page: 6 One-Way ANOVA and Correlation Here yet again is Figure 14.1, which you have now seen several times over. It will be fairly obvious to the naked eye that the two variables, dosage and pull, are correlated in the sense that variations in the one are associated with variations in the other. It will be equally obvious that the relationship is not of the rectilinear (straight-line) sort described in Chapter 3. It is better described by a curved line, hence "curvilinear." Within the context of a one-way analysis of variance for independent samples, a useful measure of the strength of a curvilinear relationship between the independent and dependent variable is given by a quantity known as as eta-square ("eta" to rhyme with "beta"), which is simply the ratio of SS bg to SS T. For the medication experiment it comes out as SS bg eta 2 = = SS T = 0.55 The essential meaning of "eta 2 =0.55" is this: Of all the variability that exists within the dependent variable "pull," 55% is associated with variability in the independent variable "dosage level." A moment's reflection of what we observed in Chapter 3 will remind you that this is also the essential meaning of the coefficient of determination, r 2. The only intrinsic difference between the two is that r 2 can measure the strength of a correlation only insofar as it is linear (can be described by a straight line), while eta 2 provides a measure of the strength of correlation irrespective of whether it is linear or curvilinear. If the relationship is linear fully describable by a straight line then the values of r 2 and eta 2 will be the same. In the degree that a curved line describes the relationship better than a straight line, then eta 2 will be greater than r 2. This point is illustrated by the two panels of Figure 14.3, which show the data for the N T =20 individual subjects of the experiment layed out in the form of a scatter plot. Applying the procedures of linear correlation to this set of bivariate data will yield the straight regression line shown in the panel on the left, along with r 2 =0.48. The panel on the right shows the same data with a curvilinear line of best fit, corresponding to our calculated value of eta 2 =0.55. Figure Linear and Curvilinear Correlation

7 Tuesday, December 12, 2000 One-Way ANOVA: Independent Samples: II Page: 7 Please note, however, that it is meaningful to speak of eta 2 as analogous to r 2 only when the levels of the independent variable are quantitative and linear, as in the present example where zero units, 1 unit, 2 units, and 3 units of the medication represent points along an equal-interval scale. If the levels of the independent variable are only categorical (several different types of medication, several different types of music, etc.), the meaning of eta 2 reverts back to a version of the more general statement given above: Of all the variability that exists within the dependent variable, such-and-such percent is associated with the differences among the levels of the independent variable. Note that this chapter includes a subchapter on the Kruskal-Wallis Test, which is a non-parametric alternative to the one-way ANOVA for independent samples. End of Chapter 14. Return to Top of Chapter 14, Part 2 Go to Subchapter 14a [The Kruskal-Wallis Test Go to Chapter 15 [One-Way Analysis of Variance for Correlated Samples Home Click this link only if the present page does not appear in a frameset headed by the logo Concepts and Applications of Inferential Statistics