Descriptions of post-hoc tests

Transcription

1 Experimental Statistics II Page 81 Descriptions of post-hoc tests Post-hoc or Post-ANOVA tests! Once you have found out some treatment(s) are different, how do you determine which one(s) are different? For the moment we will be concerned only with examining for differences among the treatment levels. We will assume that we have already detected a significant difference among treatments levels with ANOVA. So, having rejected the Null hypothesis we wish to determine how the treatment levels interrelate. This is the post-anova part of the analysis. These tests fall into two general categories. Post hoc tests (LSD, Tukey, Scheffé, Duncan's, Dunnett's, etc.) A priori tests or pre-planned comparisons (contrasts) A priori tests are better. These are tests that the researcher plans on doing before they gather data, and if we dedicate 1 d.f. to each one we generally feel comfortable doing each at some specified level of alpha. However, since multiple tests do entail risks of higher experiment wide error rates, it would not be unreasonable to apply some technique, like Bonferroni's adjustment, to insure an experimentwise error rate of the desired level of alpha ( ). So how might we do these post hoc tests? The simplest approach would be to do pairwise test of the treatments using something like the two-sample t-test. If you are interested in testing between treatment level means, then you probably have fixed effects. If the levels were randomly selected from a large number of possible choices we would probably not be interested in the individual levels chosen. This tests examines the null hypothesis H 0 : or H 0 : 0, against the alternative H 0 : 0 or H 0 : 0 or H 0 : 0. Recall two things about the two-sample t-test. First, in a t-test we had to determine if the variance was equal for the two populations tested. We tested H : with an F test to determine if this was the case Second, the variance of the test (variance of the difference between 1 and 2) was equal to 2 2. This is a variance for the linear combination from our null hypothesis, that is, the n n variance of is ( 1), if the variables are independent. If the variance are equal (as they are often assume to be for ANOVA) then the variance is We estimate 2 with the mean square error (MSE). n n

2 Experimental Statistics II Page 82 Y1 Y2 So, we would test each pair of means using the two sample t-test as t S n n p( ). For ( Y1 Y2 ) ANOVA, using the MSE as our variance estimate, we have t. If the design 1 1 MSE n n ( Y1 Y2 ) is balanced this simplifies this to t. 2MSE n Notice that if the calculated value of t is greater than the tabular value of t, we would reject the null hypothesis. To the contrary, if the calculated value of t is less than the tabular value we would fail to reject. ( Y Call the tabular value t*, and write the case for rejection of H 0 as 1 Y2 ) t. 2MSE n ( Y So we would reject H 0 if 1 Y2 ) t ( Y 2 1 Y2 ) t MSE n. or if t 2 MSE 2MSE ( Y Y n ) n or So, for any difference ( Y1 Y 2) that is greater than t 2MSE n we find the difference between the means to be statistically significant (reject H 0 ), and for any value less than this value we find the difference to be consistent with the null hypothesis. Right? This value of t 2MSE n is what R. A. Fisher called the Least Significant Difference, commonly called the LSD (not to be confused with the Latin Square Design = LSD). 1 1 LSD tcritical MSE( n n ) or LSD tcriticals Y1 Y2 This value is the exact width of an interval Y1 Y 2 which would give a t-test equal to t critical. Any larger values would be significant and any smaller values would not. This is called the Least Significant Difference. LSD t S critical Y1 Y2 We calculate this value for each pair of differences and if the observed difference is less, the treatments are not significantly different. If greater they are significantly different. One last detail. I have used the simpler version of the variance assuming that n 1 = n 2. If the experiment is unbalanced (i.e. there are unequal numbers of observations in the treatment 1 1 levels) then the value is MSE. n n

3 Experimental Statistics II Page 83 The property of balance is nice because all of the pairwise tests have the same sample sizes and the same standard error. However, balance is not necessary. For an 1 1 unbalanced design we must calculate the standard error ( MSE ) for each n n pairwise test because they will be different. This is the first of our post ANOVA tests, it is called the LSD. But hey, wait a minute! Didn't Fisher invent ANOVA in the first place to avoid doing a bunch of separate t-tests? So, now we are doing a bunch of separate t-tests. What is wrong with this picture? So, this is Fishers solution. When we do a bunch of separate t-tests, we don't know if there are any real differences at the level. After we do the ANOVA test we know that there are some differences. So we only do the LSD if the ANOVA says that there are actually differences, otherwise, don't do the LSD. This is called Fisher's Protected LSD: we use the LSD ONLY if the ANOVA shows differences, otherwise we are NOT justified in using the LSD. Makes sense. But there were still a lot of nervous statisticians looking for something a little better. As a result there are MANY alternative calculations. We will discuss the classic solutions. This least significant difference calculation can be used to either do pairwise tests on observed differences or to place a confidence interval on observed differences. The LSD can be done in SAS in one of two ways. The MEANS statement produces a range test (LINES option) or confidence intervals (CLDIFF option), while the LSMEANS statement gives pairwise comparisons. Other Post ANOVA tests Basically, we calculate the LSD with our chosen value of. We then do our mean comparisons. Each test has a pairwise error rate of. We have already seen one alternative, the Bonferroni adjustment. If we do 5 tests, or 10 tests, our error rate is no more than 5( /2) or 10( /2). Generally, for g tests our error rate is no more than g( /2). To keep an experiment wide error rate of, we simply do each comparison using a t value for an a equal to /2g. For two tailed tests (which are the most common) we do each test at /2 and the Bonferroni test would use a t for an error rate of Dunnetts test discussed below.. One tailed tests are possible, but usually only done with 2g The Bonferroni adjustment is fine if we are only doing a few tests. However, it is an upper boundary of the error, the highest that the error can be. The real probability of error is actually less, perhaps much less. So if we are doing very many tests, Bonferroni gets very conservative, giving us an actual error rate much lower than the we really want. So we seek alternatives. The major applications are Tukey's and Scheffé's. We will also consider Dunnett's and Duncan's since they are fairly commonly. Each of the tests is discussed below.

4 Experimental Statistics II Page 84 The LSD has an probability of error on each and every test or for each comparison. It is called to as a comparisonwise error rate. The whole idea of ANOVA is to give a probability of error that is for the whole experiment, so, much work in statistics has been dedicated to this problem. Some of the most common and popular alternatives are discussed below. Most of these are also discussed in your textbook. The LSD is the LEAST conservative of those discussed, meaning it is the one most likely to detect a difference and it is also the one most likely to make a Type I error when it finds a difference. However, since it is unlikely to miss a difference that is real, it is also the most powerful. The probability distribution used to produce the LSD is the t distribution. Bonferroni's adjustment. Bonferroni pointed out that in doing k tests, each at a probability of Type I error equal to, the overall experimentwise probability of Type I error will be NO MORE than k*, where k is the number of tests. Therefore, if we do 7 tests, each at =0.05, the overall rate of error will be NO MORE than =0.35, or 35%. So, if we want to do 7 tests and keep an error rate of 5% overall, we can do each individual test at a rate of /k = 0.05/7 = For the 7 tests we have an overall rate of 7* = The probability distribution used to produce the LSD is the t distribution. Duncan's multiple range test. This test is intended to give groupings of means that are not significantly different among themselves. The error rate is for each group, and has sometimes been called a familywise error rate. This is done in a manner similar to Bonferroni, except the calculation used to calculate the error rate is [1(1 ) r1 ] instead of the sum of. For comparing two means that are r steps apart, where for adjacent means r=2. Two means separated by 3 other means would have r = 5, and the error rate would be [1(1 ) r1 ] = [1(1 0.05) 4 ] = The value of a needed to keep an error rate of is the reverse of this calculation, [1(10.05) 1/4 ] = The Student-Neuman-Keuls test The value of is calculated as similar test to Duncans, controlling the familywise error rate. 1 1 r /2. It is Tukey's adjustment. This test seems to be most appropriate in most cases since it keeps an error rate for all possible pairwise tests for the whole experiment, which is often what an investigator wants to do. This test basically allows for all pairwise tests and keeps an experimentwise error rate of for all pairwise tests. The Tukey adjustment allows for, Tukey developed his own tables (see Appendix table A.7 in your book, Percentage points of the studentized range). For t treatments and a given error degrees of freedom the table will provide 5% and 1% error rates that give an experimentwise rate of Type I error. Note SAS puts HSD by Tukey's. This stands for Honest Significant Difference. Scheffé's adjustment This test is the most conservative. It allows the investigator to do all possible tests, and still maintain an experimentwise error rate of. All possible tests includes not only all pairwise tests, but comparisons of all possible combinations of treatments with other combinations of treatments (e.g. H 0 : 2 3, CONTRASTS will be covered later). The calculation is based on a square root of the F distribution, and can be used for range type tests or confidence intervals. The test is more general than the others mentioned, for the special case of pairwise comparisons.

5 Experimental Statistics II Page 85 The critical value for Scheffés test is based on the F distribution. The statistic is given by t 1 *F t 1, n(t 1) for a balanced design with t treatments and n observations per treatment. This test is appropriate for data dredging. Place the post-hoc tests above in order from the one most likely to detect a difference (and the one most likely to be wrong) to the one least likely to detect a difference (and the one least likely to be wrong). LSD is first, followed by Duncan's test, Tukey's and finally Scheffé's. Dunnett's is a special test that is similar to Tukey's, but for a specific purpose, so it does not fit well in the ranking. The Bonferroni approach produces an upper bound on the error rate, so it is conservative for a given number of tests. It is a useful approach if you want to do a few tests, fewer than allowed by one of the others (e.g. you may want to do just a few and not all possible pairwise). In this case, the Bonferroni may be better. Note that if you want to do a couple of pairwise tests you can calculate Bonferroni and compare the critical value to Tukey's. Tukey's is for all pairwise tests and would be conservative for fewer than all pairwise tests. Bonferroni may be overly conservative because it is a bound. For other sets of tests including some that are not pairwise, compare Bonferroni to Scheffé. Post ANOVA test comparison Comparisonwise error rate: LSD Experimentwise error rate: Tukey (all pairwise), Bonferroni (selected tests), Scheffé (all possible contrasts). When doing pairwise tests, the LSD is the test most likely to find differences, and the one most likely to be wrong when it finds a difference. However, power is the ability to find differences, so although error prone in the type I error sense, the LSD is the most powerful of the tests. Scheffé is the test least likely to find a difference, and least likely to be wrong with respect to type I error. Other tests that are used in particular circumstances. We will mention only Dunnett's, which is used to compare one treatment (usually a control) to all other treatments. This is the only post hoc test in SAS that has one-tailed tests (e.g. DUNNETTL and DUNNETTU). Applying Post ANOVA test comparisons All of these tests can be expressed in one of two ways. If the analysis is BALANCED, then there is a popular expression of pairwise tests that starts with ranked means. Suppose we calculate a value of the LSD equal to 8, and we have sorted the means of treatment levels and have 5, 14, 17, 23, 29, and 38. Treatment Level Mean Groups If the critical value of the LSD = 8 then means below that differ by less than 8 do not differ statistically. This is represented by giving them a common letter so they share a letter.

6 Experimental Statistics II Page 86 For an LSD critical value of 8 Same means compared with a Tukey adjusted critical value of 10 Same means compared with a Scheffé adjusted critical value of 15 Treatment Level Mean Groups 3 38 A 9 B 6 23 B C 5 17 D C 2 14 D 4 5 E Treatment Level Mean Groups 3 38 A 9 A B 6 23 B C 5 17 C 2 14 C D 4 5 D Treatment Level Mean Groups 3 38 A 9 A B 6 23 A B 5 17 B C 2 14 B C 4 5 C SAS Example (Appendix 12a) Note the test of homogeneity of variance (random or repeated statement). Test the effects of TREATMENTS. Post hoc tests : They can be done from MIXED using the LSMeans statement. In GLM either the MEANS or LSMeans statement can be used. SAS statements results to compare (post ANOVA or post hoc tests ) Results with the LSD Results with Tukey's. Results with Scheffé's. Results with Dunnett's. NOTE that normally only one post-anova examination would be done. We have done several here in order to compare. Note the use of a macro to get sorted and labeled means to indicate significant differences.

7 ANOVA & Post ANOVA tests CRD and RBD examples Page dm'log;clear;output;clear'; 2 options ps=512 ls=99 nocenter nodate nonumber; 3 TITLE1 'Appendix12: Comparing Cuckoo egg sizes'; 4 5 ODS HTML style=minimal body='c:\sas\appendix12 Cuckoo.HTML' ; NOTE: Writing HTML Body file: C:\SAS\Appendix12 Cuckoo.HTML 6 *ODS rtf style=minimal body='c:\sas\appendix12 Cuckoo.RTF' ; 7 *ODS PDF style=minimal body='c:\sas\appendix12 Cuckoo.PDF' ; 8 filename input1 'C:\SAS\Appendix12 Cuckoo.dat'; 9 FILENAME OUT1'C:\SAS\Appendix12 Cuckoo01.CGM'; 10 FILENAME OUT2'C:\SAS\Appendix12 Cuckoo02.CGM'; 11 FILENAME OUT3'C:\SAS\Appendix12 Cuckoo03.CGM'; ****************************************************************************; 14 *** Source: L.H.C. Tippett, The Methods of Statistics, 4th Edition, ****; 15 *** John Wiley and Sons, Inc., 1952, p ****; 16 *** Description: L.H.C. Tippett ( ) was one of the pioneers ****; 17 *** in the field of statistical quality control, This data on the ****; 18 *** lengths of cuckoo eggs found in the nests of other birds (drawn ****; 19 *** from the work of O.M. Latter in 1902) is used by Tippett in his ****; 20 *** fundamental text. Cuckoo are knows to lay their eggs in the nests ****; 21 *** of other (host) birds. The eggs are then adopted and hatched by ****; 22 *** the host birds. ****; 23 ****************************************************************************; 24 *** That cuckoo eggs were peculiar to the locality where found was ****; 25 *** already known in A study by E.B. Chance in 1940 called The ****; 26 *** Truth About the Cuckoo demonstrated that cuckoos return year after ****; 27 *** year to the same territory and lay their eggs in the nests of a ****; 28 *** particular host species. Further, cuckoos appear to mate only within****; 29 *** their territory. Therefore, geographical sub-species are developed, ****; 30 *** each with a dominant foster-parent species, and natural selection ****; 31 *** has ensured the survival of cuckoos most fitted to lay eggs that ****; 32 *** would be adopted by a particular foster-parent species. ****; 33 ****************************************************************************; data cuckoo; infile input1 missover; 37 Input S1 S2 S3 S4 S5 S6; 38 HostSpecies='Meadow Pipit'; EggLt=s1; if egglt ne. then output; 39 HostSpecies='Tree Pipit'; EggLt=s2; if egglt ne. then output; 40 HostSpecies='Hedge Sparrow'; EggLt=s3; if egglt ne. then output; 41 HostSpecies='Robin'; EggLt=s4; if egglt ne. then output; 42 HostSpecies='Pied Wagtail'; EggLt=s5; if egglt ne. then output; 43 HostSpecies='Wren'; EggLt=s6; if egglt ne. then output; 44 keep HostSpecies EggLt; 45 datalines; NOTE: The infile INPUT1 is: File Name=C:\SAS\Appendix12 Cuckoo.dat, RECFM=V,LRECL=256 NOTE: 45 records were read from the infile INPUT1. The minimum record length was 45. The maximum record length was 45. NOTE: The data set WORK.CUCKOO has 120 observations and 2 variables. NOTE: DATA statement used (Total process time): 0.01 seconds 0.00 seconds 45! run; 46 ; proc sort data=cuckoo; by HostSpecies; run; NOTE: There were 120 observations read from the data set WORK.CUCKOO. NOTE: The data set WORK.CUCKOO has 120 observations and 2 variables. NOTE: PROCEDURE SORT used (Total process time): 0.00 seconds 0.00 seconds

8 ANOVA & Post ANOVA tests CRD and RBD examples Page proc print data=cuckoo; run; NOTE: There were 120 observations read from the data set WORK.CUCKOO. NOTE: The PROCEDURE PRINT printed page 1. NOTE: PROCEDURE PRINT used (Total process time): 0.09 seconds 0.01 seconds 50 TITLE2 'RAW DATA LISTING'; RUN; Appendix12: Comparing Cuckoo egg sizes Obs hostspecies EggLt 1 Hedge Sparrow Hedge Sparrow Hedge Sparrow Hedge Sparrow Hedge Sparrow Hedge Sparrow Hedge Sparrow Hedge Sparrow Hedge Sparrow Hedge Sparrow Hedge Sparrow Hedge Sparrow Hedge Sparrow Hedge Sparrow Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Meadow Pipit Pied Wagtail Pied Wagtail Pied Wagtail Pied Wagtail Pied Wagtail Pied Wagtail Pied Wagtail Pied Wagtail Pied Wagtail Pied Wagtail Pied Wagtail Pied Wagtail Pied Wagtail Pied Wagtail Pied Wagtail Robin Robin Robin Robin Robin Robin Robin Robin Robin Robin Robin Robin Robin Robin Robin Robin Tree Pipit Tree Pipit Tree Pipit Tree Pipit Tree Pipit Tree Pipit Tree Pipit Tree Pipit Tree Pipit Tree Pipit Tree Pipit Tree Pipit Tree Pipit Tree Pipit Tree Pipit Wren Wren Wren Wren Wren Wren Wren Wren Wren Wren Wren Wren Wren Wren Wren goptions reset=all device=cgmlt97l gsfname=out2 gsfmode=replace 53 ftext='swissx' ftitle='swissx'; GOPTIONS GSFNAME=OUT2; 56 proc boxplot data = cuckoo; plot EggLt * HostSpecies; run ; NOTE: Processing beginning for PLOT statement number 1. NOTE: 172 RECORDS WRITTEN TO C:\SAS\Appendix12 Cuckoo02.CGM NOTE: 59 RECORDS WRITTEN TO C:\SAS\sasgraph\boxplot.gif NOTE: There were 120 observations read from the data set WORK.CUCKOO. NOTE: PROCEDURE BOXPLOT used (Total process time): 0.48 seconds 0.14 seconds

9 ANOVA & Post ANOVA tests CRD and RBD examples Page Egg length Hedge Sparrow Meadow Pipit Pied Wagtail Robin Host species Tree Pipit Wren 58 PROC MIXED DATA=Cuckoo cl covtest; CLASSES HostSpecies; 59 TITLE2 'Analysis of Variance with PROC MIXED'; 60 TITLE3 'This model tests for non-homogeniety'; 61 MODEL EggLt = HostSpecies / htype=3 DDFM=Satterthwaite; 62 repeated / group=hostspecies; 63 LSMEANS HostSpecies / ADJUST=Dunnett pdiff; 64 run; NOTE:Convergence criteria met. NOTE: The PROCEDURE MIXED printed page 2. NOTE: PROCEDURE MIXED used (Total process time): 0.12 seconds 0.04 seconds Analysis of Variance with PROC MIXED This model tests for non-homogeniety The Mixed Procedure Model Information Data Set Dependent Variable Covariance Structure Group Effect Estimation Method Residual Variance Method Fixed Effects SE Method Degrees of Freedom Method WORK.CUCKOO EggLt Variance Components hostspecies REML None Model-Based Satterthwaite Class Level Information Class Levels Values hostspecies 6 Hedge Sparrow Meadow Pipit Pied Wagtail Robin Tree Pipit Wren Dimensions Covariance Parameters 6 Columns in X 7 Columns in Z 0 Subjects 120 Max Obs Per Subject 1

10 ANOVA & Post ANOVA tests CRD and RBD examples Page 243 Number of Observations Number of Observations Read 120 Number of Observations Used 120 Number of Observations Not Used 0 Iteration History Iteration Evaluations -2 Res Log Like Criterion Convergence criteria met. Covariance Parameter Estimates Z Cov Parm Group Estimate Error Value Pr Z Alpha Lower Upper Residual hostspecies Hedge Sparrow Residual hostspecies Meadow Pipit < Residual hostspecies Pied Wagtail Residual hostspecies Robin Residual hostspecies Tree Pipit Residual hostspecies Wren FitStatistics -2 Res Log Likelihood AIC (smaller is better) AICC (smaller is better) BIC (smaller is better) Null Model Likelihood Ratio Test DF Chi-Square Pr > ChiSq Type 3 Tests of Fixed Effects Num Den Effect DF DF F Value Pr > F hostspecies <.0001 Testing for a difference between full and reduced models. Full model -2LL = Reduced model -2LL = d.f. difference = 5 (i.e. 6 variance model versus 1 variance model) Chi sq = = 4.6 P > Chi Sq = Least Squares Means Effect hostspecies Estimate Error DF t Value Pr > t hostspecies Hedge Sparrow <.0001 hostspecies Meadow Pipit <.0001 hostspecies Pied Wagtail <.0001 hostspecies Robin <.0001 hostspecies Tree Pipit <.0001 hostspecies Wren <.0001 Differences of Least Squares Means Effect hostspecies _hostspecies Estimate Error DF t Value Pr > t Adjustment Adj P hostspecies Meadow Pipit Hedge Sparrow Dunnett hostspecies Pied Wagtail Hedge Sparrow Dunnett hostspecies Robin Hedge Sparrow Dunnett hostspecies Tree Pipit Hedge Sparrow Dunnett hostspecies Wren Hedge Sparrow <.0001 Dunnett < PROC MIXED DATA=Cuckoo cl covtest; CLASSES HostSpecies; 67 TITLE2 'Analysis of Variance with PROC MIXED'; 68 TITLE3 'This model assumes homogeniety'; 69 MODEL EggLt = HostSpecies / htype=3 DDFM=Satterthwaite outp=resids; 70 LSMeans HostSpecies / pdiff adjust=tukey CL; 71 LSMEANS HostSpecies / ADJUST=Scheffe pdiff; 72 LSMEANS HostSpecies / ADJUST=Bon pdiff; 73 *Order: Hedge_Sparrow Meadow_Pipit Pied_Wagtail Robin Tree_Pipit Wren; 74 Contrast 'Pipits v others' HostSpecies ; 75 Contrast 'Wren sized v others' HostSpecies ; 76 ods output diffs=ppp lsmeans=mmm;

11 ANOVA & Post ANOVA tests CRD and RBD examples Page ods listing exclude diffs;* lsmeans; 78 run; NOTE: The data set WORK.MMM has 18 observations and 10 variables. NOTE: The data set WORK.PPP has 45 observations and 15 variables. NOTE: The data set WORK.RESIDS has 120 observations and 9 variables. NOTE: The PROCEDURE MIXED printed page 3. NOTE: PROCEDURE MIXED used (Total process time): 0.17 seconds 0.07 seconds TITLE4 'Post hoc adjustment with macro by Arnold Saxton'; 81 * SAS Macro by Arnold Saxton: Saxton, A.M A macro for ; 82 * converting mean separation output to letter groupings in Proc Mixed. ; 83 * In Proc. 23rd SAS Users Group Intl., SAS Institute, Cary, NC, pp ; 84 %include 'C:\SAS\pdmix800.sas'; 712 %pdmix800(ppp,mmm,alpha=0.05,sort=yes); RUN; PDMIX processing Tukey-Kramer values for hostspecies are (avg) (min) (max). 713 %include 'C:\pdmix800.sas'; Analysis of Variance with PROC MIXED This model assumes homogeniety The Mixed Procedure Model Information Data Set Dependent Variable Covariance Structure Estimation Method Residual Variance Method Fixed Effects SE Method Degrees of Freedom Method WORK.CUCKOO EggLt Diagonal REML Profile Model-Based Residual Class Level Information Class Levels Values hostspecies 6 Hedge Sparrow Meadow Pipit ied Wagtail Robin Tree Pipit Wren Dimensions Covariance Parameters 1 Columns in X 7 Columns in Z 0 Subjects 1 Max Obs Per Subject 120 Number of Observations Number of Observations Read 120 Number of Observations Used 120 Number of Observations Not Used 0 Covariance Parameter Estimates Z Cov Parm Estimate Error Value Pr Z Alpha Lower Upper Residual <

12 ANOVA & Post ANOVA tests CRD and RBD examples Page 245 Fit Statistics -2 Res Log Likelihood AIC (smaller is better) AICC (smaller is better) BIC (smaller is better) Type 3 Tests of Fixed Effects Num Den Effect DF DF F Value Pr > F hostspecies <.0001 Contrasts Num Den Label DF DF F Value Pr > F Pipits v others Wren sized v others <.0001 Least Squares Means Effect hostspecies Estimate Error DF t Value Pr > t Alpha Lower Upper hostspecies Hedge Sparrow < hostspecies Meadow Pipit < hostspecies Pied Wagtail < hostspecies Robin < hostspecies Tree Pipit < hostspecies Wren < hostspecies Hedge Sparrow < hostspecies Meadow Pipit < hostspecies Pied Wagtail < hostspecies Robin < hostspecies Tree Pipit < hostspecies Wren < hostspecies Hedge Sparrow < hostspecies Meadow Pipit < hostspecies Pied Wagtail < hostspecies Robin < hostspecies Tree Pipit < hostspecies Wren < Effect=hostspecies Method=Bonferroni(P<0.05) Set=3 Letter Obs hostspecies Estimate Error Alpha Lower Upper Group 1 Hedge Sparrow A 2 Tree Pipit A 3 Pied Wagtail A 4 Robin A 5 Meadow Pipit A 6 Wren B Effect=hostspecies Method=Scheffe(P<0.05) Set=2 Letter Obs hostspecies Estimate Error Alpha Lower Upper Group 7 Hedge Sparrow A 8 Tree Pipit A 9 Pied Wagtail A 10 Robin A 11 Meadow Pipit A 12 Wren B Effect=hostspecies Method=Tukey-Kramer(P<0.05) Set=1 Letter Obs hostspecies Estimate Error Alpha Lower Upper Group 13 Hedge Sparrow A 14 Tree Pipit A 15 Pied Wagtail AB 16 Robin AB 17 Meadow Pipit B 18 Wren C

13 ANOVA & Post ANOVA tests CRD and RBD examples Page options ps=512 ls=132; 761 PROC UNIVARIATE DATA=Resids PLOT NORMAL; VAR resid; 762 ods exclude basicmeasures extremeobs quantiles testsforlocation; 763 TITLE2 'Analysis of residuals from PROC MIXED'; 764 RUN; NOTE: SAS set option OBS=0 and will continue to check statements. This may cause NOTE: No observations in data set. NOTE: PROCEDURE UNIVARIATE used (Total process time): 0.00 seconds 0.00 seconds Analysis of residuals from PROC MIXED The UNIVARIATE Procedure Variable: Resid (Residual) Moments N 120 Sum Weights 120 Mean 0 Sum Observations 0 Std Deviation Variance Skewness Kurtosis Uncorrected SS Corrected SS Coeff Variation. Std Error Mean Tests for Normality Test --Statistic p Value Shapiro-Wilk W Pr < W Kolmogorov-Smirnov D Pr > D < Cramer-von Mises W-Sq Pr > W-Sq Anderson-Darling A-Sq Pr > A-Sq Stem Leaf # Boxplot Normal Probability Plot * **+* * ** *** ***** ** *** *** *** *-----* **** **** *** ** ** *** * ** * * * * * * Multiply Stem.Leaf by 10**