The Effects of Topological Inaccuracy in Evolutionary Trees on the Phylogenetic Comparative Method of Independent Contrasts

Transcription

1 Syst. Biol. 51(4): , 2002 DOI: / The Effects of Topological Inaccuracy in Evolutionary Trees on the Phylogenetic Comparative Method of Independent Contrasts MATTHEW R. E. SYMONDS University Museum of Zoology, Downing Street, Cambridge CB2 3EJ, U.K.; Current address: Department of Zoology, University of Melbourne, Victoria 3010, Australia; Abstract. Computer simulations were used to test the effect of increasing phylogenetic topological inaccuracy on the results obtained from correlation tests of independent contrasts. Predictably, increasing the number of disruptions in the tree increases the likelihood of signi cant error in the r values produced and in the statistical conclusions drawn from the analysis. However, the position of the disruption in the tree is important: Disruptions closer to the tips of the tree have a greater effect than do disruptions that are close to the root of the tree. Independent contrasts derived from inaccurate topologies are more likely to lead to erroneous conclusions when there is a true signi cant relationship between the variables being tested (i.e., they tend to be conservative). The results also suggest that random phylogenies perform no better than nonphylogenetic analyses and, under certain conditions, may perform even worse than analyses using raw species data. Therefore, the use of random phylogenies is not bene cial in the absence of knowledge of the true phylogeny. [Comparative method; computer simulation; independent contrasts; phylogenetic error; random phylogenies; topological inaccuracy.] Perhaps the most important assumption associated with all phylogenetic comparative methods and with independent contrasts (Felsenstein, 1985) in particular is that the true phylogeny of the organisms under study is known completely and without error. However, in reality we are never able to know if this assumption is correct. Clearly, if the phylogeny is wrong, then so will be the results of comparative analyses that use that phylogeny. Harvey and Pagel (1991:121) recognized this problem and warned that until more work is done in this area, comparative biologists should be aware that their results may depend on a particular reconstruction of phylogeny. The problem is particularly important because some authors have questioned the use of phylogenetic comparative methods per se on the grounds of their restrictive and unrealistic assumptions (Björklund, 1997; Price, 1997), whereas others have maintained that it is still better to use such methods than to use nonphylogenetic approaches (Martins, 2000). Errors can manifest themselves in phylogenies in a number of related ways, including errors in the estimation of branch lengths and in the assumptions about the model of evolution used. The effects on comparative analyses of both of these types of error have already been investigated using computer simulation. D õ az-uriarte and Garland (1998) found that when branch lengths are suitably transformed (see Garland et al., 1992) branch length error does not seem to affect dramatically the results of independent contrasts analyses, almost never producing a type I error rate >10%. D õ az-uriarte and Garland (1996) also tested the effect of deviations from the Brownian motion model of Felsenstein s independent contrasts and drew similar conclusions. The most obvious form of phylogenetic error is that where the actual pattern of evolutionary relationships is wrong (topological inaccuracy). A phylogeny can have two types of topological inaccuracy (identi ed by Purvis and Garland, 1993). First, the topology may be unresolved, with the relationships between taxa depicted as polytomies. Many researchers have investigated the effects of this problem and ways to overcome it. Grafen (1989), Pagel (1992), and Purvis and Garland (1993) have all suggested methods by which polytomies may be accounted for in phylogenetic comparative analyses. Computer simulations (Grafen, 1989; Purvis et al., 1994) have shown that independent contrasts analyses are actually fairly robust to some bushiness in the phylogeny. What happens if the phylogeny is totally unknown? Losos (1994) and Martins (1996) have both suggested techniques involving the use of randomly generated evolutionary trees as the basis for comparative analyses. Losos (1994) suggested using the most common result from such an analysis (i.e., the correlation coef cient or regression slope value 541

2 542 SYSTEMATIC BIOLOGY VOL. 51 or range that is obtained most frequently). The smaller the range of results, the more con dence can be placed on the nal result. Martins (1996) developed the technique further by using the mean of the results obtained using the different random trees and provided a method by which con dence limits on the results could be estimated. However, Abouheif (1998) cautioned against the use of random trees suggesting that the overall effect is the same as using a completely unresolved phylogeny (which will, in effect, be the consensus phylogeny of all the random phylogenies). Therefore, Abouheif argued, the parameter estimates obtained with Martins (1996) method are the same as those obtained using raw data. Although these simulation studies allude to topological error and other researchers have examined the effects of using competing phylogenetic hypotheses (e.g., Björklund in Harvey, 1991; Bauwens and D õ az-uriarte, 1997), there has been a lack of speci c focus on the question of how outright topological error affects phylogenetic comparative analyses even though this error is the one most likely to be of concern to comparative biologists. Purvis and Garland (1993: 569) pointed out that topological inaccuracy violate[s] the assumptions of Felsenstein s method to an unknown and possibly very serious degree. Frequently, one is actually confronted by the problem of having several possible phylogenies for a group, in which case the questions that arise are which tree do I use? and what happens if the tree I use is wrong? I investigated the effect on independent contrasts of using inaccurate topologies. I used computer simulations to generate known true phylogenies (which would not be available for real data sets) and continuous data. Other trees were produced that were increasingly different from the true phylogeny. The continuous data were analyzed raw and then using independent contrasts with the true phylogeny and the increasingly erroneous phylogenies. Perhaps previous researchers have avoided evaluating the effects of inaccurate topologies because of the statistical problems involved. Developing a full stochastic model of the process described here presents dif culties. Tree topology is conventionally treated as a parameter in such a model, but there are problems with this interpretation (Yang et al., 1995). Accordingly, rather than employ tree comparison metrics, I have used a simpler approach and assumed that the amount of error in the tree is proportional to the number of disruptions put into the tree (i.e., the number of times that a species or group of species is removed from the tree and placed at random elsewhere in the tree). Furthermore, based on the tree topology, I have grouped the species in the tree into hypothetical genera. In this way disruptions can be introduced at the tips of the tree (by moving around individual species) and deeper in the tree (moving around whole genera). This approach simpli ed the analysis and the interpretation of results, and the results should have greater meaning to comparative biologists. Using this framework, I also explored how the taxonomic level of the disruptions in the tree affects the results obtained in the comparative analysis. I also address here the issue of using random trees in comparative analyses. METHODS The general protocol can be summarized as follows: 1. Generate for 50 hypothetical species a fully resolved and known phylogeny and two sets of simulated pairs of continuous variables. In one set, the two variables are independent of each other, and in the other set one variable is partly determined by (i.e., is dependent on) the other. 2. Create a series of phylogenies that are increasingly different from the true phylogeny. 3. Analyze the data with independent contrasts using the true phylogeny and the increasingly erroneous phylogenies to assess the effect on the results of comparative analysis. In these analyses, I attempted to keep the simulations as basic as possible. Therefore, I used equal untransformed branch lengths in all simulations. The continuous data were then generated using a xed likelihood of random change per unit branch length to reduce the possible confounding effect of having heterogenous branch lengths in the analysis, leaving topology as the main factor affecting results. The resulting trees and continuous data are not very biologically

3 2002 SYMONDS TOPOLOGICAL INACCURACY AND INDEPENDENT CONTRASTS 543 realistic (not least because the endpoint species are different distances from the root of the tree), but the results should show the effects of topological inaccuracy less ambiguously. Generating Trees Ten completely resolved, dichotomously branching phylogenies of 50 hypothetical species were created using a specially written computer simulation program called Evolve (written in CC, available upon request). The program creates the phylogenies using a random Markovian branching process (as used by Martins [1996], who also discussed other methods for creating random phylogenies). This process starts with an ancestral species (or node) dichotomously branching to produce two daughter lineages. One of these two lineages is then chosen at random to split dichotomously again, producing three species in total. One of these three lineages is then chosen at random to branch, and so on until the required number of species is produced. Generating Continuous Data Continuous data were generated for the 50 species along each of the 10 true phylogenies using another speci cally written program entitled Branch (available upon request). Branch works by assigning a random amount of character change to each branch of the phylogeny and then summing these changes along the phylogeny to provide values of the continuous characters for the endpoint species. This procedure is the same as that used in previous simulation studies (e.g., Purvis et al., 1994). To simulate character change, Branch adapts the Brownian motion model (as in Felsenstein, 1985). Thus, per unit time character change can occur in any direction by a random amount. Because all branch lengths are assumed to be equal, the values for change over each branch length are drawn from the same Normal distribution. The model used in these simulations is therefore essentially the same as Martins and Garland s (1991) FL1P speciational model. Three continuous variables were generated, x, y, and z. Of these, x and y are independent of each other, whereas z is partially dependent on (i.e., correlated with) x. Character change along each branch for these three variables was generated in the following way: for x, a random number from a normal distribution of x D 0 and SD D 1; for y, a separate random number derived in the same manner as that for x; and for z, a number that is partially dependent on x and given by the relation z D rx C (1 r)w where w is a random number drawn from a Normal distribution of x D 0 and SD D 1, and r represents the input correlation. This formula was used both by Martins and Garland (1991) and Purvis et al. (1994) in their simulations of comparative data. Martins and Garland (1991) initially suggest using r D 0:5 to make type II errors easier to detect. In pilot studies, I found that adopting this value of r resulted in x and z being too highly correlated to demonstrate any effect of phylogenetic inaccuracy (at high levels of correlation the effect of phylogeny becomes less; see Klingenberg [1996] for illustration of this effect). Therefore, r was given the value of 0.28 because on average this value gives a correlation of x to z that just attains signi cance at the 95% level for a sample of 50. To increase the number of tests, I altered the Branch program so that it actually simulated the character change in ve sets of y and z for every set of changes in x. Ten sets of values for x were generated along each hypothetical phylogeny. In this way, 500 sets of simulated data were generated in total (5 sets of y and z for each of 10 sets of x, making 50 sets of continuous data for each of the 10 simulated trees). A sample tree with the continuous data generated is shown in Figure 1. Generating Increasingly Inaccurate Trees Each of the 10 true hypothetical phylogenies generated by Evolve was taken as the starting point for generating nine types of increasingly erroneous tree. The nine categories, and how each were generated, are described as follows. 1. Five species wrongly placed locally (5sl). 2. Ten species wrongly placed locally (10sl). Using another speci cally written C+ program, Mix, 1 of the 50 species was chosen at random, removed from the phylogeny, and repositioned as the sister taxon to a species that was between two and four tips away from it in the true tree. This local

4 544 SYSTEMATIC BIOLOGY VOL. 51 FIGURE 1. equal. Sample simulated topology with continuous data generated for 50 species. All branch lengths are

5 2002 SYMONDS TOPOLOGICAL INACCURACY AND INDEPENDENT CONTRASTS 545 repositioning was repeated 5 or 10 times accordingly with replacement (i.e., the same species could be moved twice). The overall intended effect is one of having a generally correct topology with minor errors at the tips of the tree, which might be equivalent to the real situation of having errors in the relationships within genera but still having the correct pattern of relationships between genera. 3. Five species wrongly placed anywhere (5sa). 4. Ten species wrongly placed anywhere (10sa). Using Mix, the same procedure was performed except without any restrictions on the repositioning of the chosen random species (i.e., the chosen species could be replaced as the sister taxon to any of the tips in the tree). 5. One genus wrongly placed anywhere (1g). 6. Three genera wrongly placed anywhere (3g). Each of the 10 true phylogenies was examined and split by hand into eight monophyletic groupings (equivalent to and thus called genera ). As far as possible the number of species in each of the genera was made equal. Ideally, with 50 species in eight genera the size of the genus should be close to the average of 8.25 species. In reality, the actual topology of the individual phylogenies placed restrictions on the ability to achieve equality. Because of its arbitrary nature, this method for assigning genera is unusual but was chosen to decrease the likelihood of species from small genera having an overly strong effect on results. The program Mix was used to select a genus at random and reposition it elsewhere in the tree. This process was performed one or three times in an attempt to simulate a situation where the errors were occurring at deeper nodes in the tree, equivalent to having the correct interspeci c relationships but the wrong intergeneric relationships. 7. Random relationships within genera, correct relationships between genera (rs). 8. Random relationships between genera, correct relationships within genera (rg). 9. Totally random phylogeny (rand). The program Evolve was again used but modi ed such that it created a random tree (via a Markovian branching process) for a given number of taxa, randomly allocating the names of taxa to each of the tips of the newly created phylogeny. In this way, eight new intrageneric phylogenies were created. These random intrageneric phylogenies could then be inserted into the true intergeneric phylogenetic framework (step 7). Next, the true intrageneric phylogenies were used again but a new random tree was created for the relationship between the genera (step 8). Finally, a completely new tree was created by Evolve with the names of each species put on the tips at random (step 9). This nal step essentially represents a completely perturbed phylogeny for the species, whereas the previous two steps represent cases where the relationships between the species are known completely at one taxonomic level but are potentially completely wrong at another. All of these steps were repeated 10 times using each true phylogeny as the starting point. Because 50 sets of continuous data were simulated for each true phylogeny, the 10 sets of wrong phylogenies were mixed around ve times arbitrarily to produce 50 sets of wrong phylogenies for each true phylogeny. Data Analysis All statistical tests were performed using the MINITAB statistical computing package, (release 10 Xtra, 1995). The sets of continuous data were tested for correlation between x and y and between x and z by calculating standard (i.e., nonphylogenetic) Pearson product moment correlation coef cients. The continuous data were then analyzed using standardized independent contrasts in the manner described by Felsenstein (1985). The computer package CAIC (Purvis and Rambaut, 1995) was used to perform all calculations. In all analyses, x was held as the independent variable. All branch lengths were assigned the same value (2) and were not transformed. Standardized independent contrasts were produced for x, y, and z using the appropriate true phylogenies for the data and all the appropriate increasingly erroneous phylogenies. Correlation coef cients were then calculated for the relationships between the contrasts of x and y and of x and z. These correlations were forced through the origin (see Garland et al., 1992). To test the hypothesis that the variance

6 546 SYSTEMATIC BIOLOGY VOL. 51 of the correlation coef cients obtained increases with increasing error in the phylogeny, the F ratio (the ratio of variances) was also calculated for the correlation coef cients obtained using each of the erroneous phylogenies. For each type of phylogeny, all the correlation coef cients were tested for significant difference from the correlation obtained when using the true phylogeny (Z-tests; Zar, 1996: 313). All statistical tests were two-tailed (unless stated). RESULTS Analysis of variance of the correlation coef- cients produced by data from each of the 10 true phylogenies indicated that there was no signi cant effect of individual tree topology on the correlation coef cients themselves or on the error rates in the subsequent analyses. Independent Variables x and y Table 1 shows the range, mean, and SD of the correlation coef cients obtained using the different analyses for the relationship between x and y. Also shown are the F ratios and associated probabilities of the comparison between the variances of the correlation coef cients (r) for the treatment and those obtained by using the true phylogeny. Kolmogorov Smirnov tests showed that none of the distributions of correlation coef cients signi cantly differed from a normal distribution. These results show that the range and SD of the r values increase with TABLE 1. Properties of the 500 correlation coef cients obtained for the relationship between x and y, when raw data were used (i.e., no phylogeny) and when independent contrasts were used with true and different inaccurate phylogenies. F ratios and associated signi cance levels are for the comparison of variances of the correlations obtained with these phylogenies versus the true phylogeny. Phylogeny Range Mean SD F a True to None to sl to sl to sa to sa to g to g to rs to rg to rand to a P < 0:001: the number of disruptions in the phylogenies. Likewise, for disruptions at the tips of the tree, there is greater deviation in the r values when the disruptions are more extreme (i.e., when the species are replaced anywhere in the tree as oppose to just locally). Disruptions at the tips also seem to have a greater effect than disruptions at the base of the tree (i.e., at the generic level). At the generic level, the F ratios show that the variances of the r values are not signi cantly different from those obtained with the true phylogeny, even when the relationships between the genera are random. When the relationships between the species are random but the correct intergeneric relationships are used, then the SD (and F ratio) is still less than when there are disruptions that place the species in the completely wrong part of the tree. Table 2 summarizes the differences between the r values obtained with the different inaccurate phylogenies and those obtained with the true phylogeny. These differences are rst expressed qualitatively as the number of times where the conclusion (i.e., whether the correlation achieves signi cance or not) differs from the conclusion drawn from using the true phylogeny. I distinguish between when the working phylogeny yields a signi cant result and the true phylogeny yields a nonsigni cant result, and when the working phylogeny yields a nonsigni cant result and the true phylogeny yields a signi cant result. I have called these two types of error type Axy error and type Bxy error, respectively. As a whole, the results show that use of no phylogeny or of a completely random phylogeny gives the wrong answer over one-third of the time (34.4% and 37.6%, respectively). The results again show that disruptions that occur closer to the root of the tree (i.e., in the intergeneric phylogenies) have less effect, with wrong results being obtained in <1 of 25 tests. Disruptions that are closer to the tips of the tree seem to have a greater effect, although in this case increasing the number of disruptions does not appear to profoundly affect the conclusions. However, mistaken conclusions appear to be almost twice as likely when disruptions at the tips are profound (i.e., the species are replaced anywhere in the tree, not locally). Using random species within the correct generic phylogeny again yields results approximately equivalent to those obtained

7 2002 SYMONDS TOPOLOGICAL INACCURACY AND INDEPENDENT CONTRASTS 547 TABLE 2. Number of qualitative errors and quantitative differences in the estimation of the correlation between x and y when producing independent contrasts using raw data (i.e., no phylogeny) and phylogenies that are increasingly erroneous. Qualitative errors Phylogeny Axy Bxy Total wrong (%) Mean differences in r Quantitative differences No. of signi cant differences (%) Mean jzj None (34.4) (14.8) sl (11.8) (0.2) sl (14.8) (1) sa (23) (3.8) sa (22.6) (7.8) g (2.6) g (3.2) rs (16.2) (0.6) rg (4) (0.2) rand (37.6) (15.2) with phylogenies that have 10 local disruptions at the tips of the tree. Table 2 also shows the mean difference in values of r derived from the incorrect phylogenies versus the true phylogeny and the number of quantitatively signi cant differences between the correlation coef cients, together with the average modular value of Z, which provides an indicator of the amount of difference between the correlation coef- cients. Apart from sharing the same basic features of the data as described above, as a whole the Z-test shows relatively less significant differences between the results of the analysis using the true phylogeny and those obtained using inaccurate phylogenies. The only major exception to this trend is when either no phylogeny or a random phylogeny is used (and in these cases only in approximately 15% of all the results). The results do indicate the effect of large-scale disruptions (the results for phylogenies 5sa and 10sa are considerably higher than those for 5sl and 10sl). Linked Variables x and z Tables 3 and 4 show results equivalent to those in Tables 1 and 2, respectively, except for the relationship between x and z instead of that between x and y. In general, the pattern is the same as that for x and y, with a few notable differences. The SDs of the correlation coef cients are slightly less for the relationship between x and z (Table 3), and the F ratios for the results obtained from the phylogenies 3g and rg almost reach signi cance (P D 0:1 and 0.06, respectively). In Table 4, type Bxz error is considerably greater than the type Axz error (as expected because x and z are signi cantly correlated on the true phylogeny in 66.8% of the tests). In comparison with Table 2, this table also indicates that the number of wrong conclusions drawn using incorrect generic relationships is somewhat higher ( % for x and z; 2.6 4% for x and y). These results also indicate even more strongly that profound errors at the tips (i.e., where the species may be placed in the completely wrong part of the tree) have a much more dramatic effect on results than smaller local errors. The number of signi cant differences (as calculated by a Z-test) is considerably higher with phylogenies 5sa and 10sa than with phylogenies 5sl, 10sl, and even rs. However, the mean modular Z values are almost (with the exception of that of 5sl) the same as those obtained from TABLE 3. Properties of the 500 correlation coef cients obtained for the relationship between x and z, when raw data were used (i.e., no phylogeny) and when independent contrasts were used with true and different inaccurate phylogenies. F ratios and associated signi cance levels are for the comparison of variances of the correlations obtained with these phylogenies versus the true phylogeny. Phylogeny Range Mean SD F a True to None to sl to sl to sa to sa to g to g to rs to rg to rand to a P < 0:001:

8 548 SYSTEMATIC BIOLOGY VOL. 51 TABLE 4. Number of qualitative errors and quantitative differences in the estimation of the correlation between x and z when producing independent contrasts using raw data (i.e., no phylogeny) and phylogenies that are increasingly erroneous. Qualitative errors Phylogeny Axy Bxy Total wrong (%) Mean differences in r Quantitative differences No. of signi cant differences (%) Mean jzj None (30.4) (17.6) sl (20) sl (19.8) (0.2) sa (25.4) (4.6) sa (27.2) (6.4) g (4.6) g (8) rs (21.2) (1.4) rg (8.8) rand (32.4) (15.6) the tests using the independent variables x and y (see Table 2). Random Phylogenies At rst sight, Tables 1 4 appear to indicate that using a completely random phylogeny gives approximately the same results as using no phylogeny at all. The F values, modular Z values, and percentage of wrong answers are all considerably worse than for any of the other inaccurate phylogenies. The results from using the phylogenies rs and rg indicate that having any knowledge of the phylogenetic relationships substantially increases the chances of obtaining something near to the correct result. Qualitatively, however, there is some indication that the results obtained with raw data and a random phylogeny may be different; in fact, using a completely random phylogeny may be worse than using no phylogeny at all. For x and y, in 285 of 500 cases the correlation coef cient obtained using a random phylogeny is more different from the truth than that obtained using no phylogeny. The expected value, were the two approaches equivalent, would be 250. This difference is statistically signi cant (two-tailed binomial test: P D 0:002). However, for the relationship between x and z, the results do almost equal the expected value, with 251 of 500 cases (two-tailed binomial test: P D 0:964) showing the same pattern. DISCUSSION Number and Taxonomic Level of Disruptions As the number of disruptions in a tree topology increases, the more different the results of comparative analyses become from that obtained using the true phylogeny and the greater the chance of serious error in the estimation of correlation coef cients. This nding might also explain why disruptions at the tips of the tree appear to be more signi cant than those at the base of the tree. In these tests only 1 or 3 disruptions were made at the generic level, whereas 5 or 10 were made at the tips. But the number of disruptions explanation cannot account for why the rg phylogeny, with potentially seven contrast errors (the number of genera minus 1), performed far better than did the 5sl and 5sa phylogenies. The results from phylogenies with disruptions at the generic level approximate those obtained using the true phylogeny, which implies that disruptions that occur closer to the root of the phylogeny are less important than disruptions that occur closer to the tips. In other words, it is not crucial to get the higher level relationships between the organisms in the phylogeny correct, but it could be essential that the species relationships be correct. There are two possible reasons why this may be the case. First, Felsenstein s (1985) technique, as implemeted by CAIC (Purvis and Rambaut, 1995), calculates contrast values for variables by working downwards through the tree, from the species values at the tips to the root of the phylogeny. Clearly, a disruption at the tips of the tree will then affect the calculation of all the contrasts below it (i.e., closer to the root) in the phylogeny. A disruption that occurs closer to the root of the tree (i.e., a disruption at a higher taxonomic level) will affect far fewer contrast values.

9 2002 SYMONDS TOPOLOGICAL INACCURACY AND INDEPENDENT CONTRASTS 549 Second, to account for error in the calculation of basal contrasts, CAIC increases branch lengths closer to the root of the tree, thereby increasing variances and decreasing the size of the higher taxonomic standardized contrasts (Felsenstein, 1985; Purvis and Rambaut, 1995). In other words, less weight is placed on the contrasts at higher taxonomic levels anyway, which again may explain why disruptions at this level seem to be less profound. It could be argued that by comparing generic level errors with errors at the tips I am not creating a true gradient of phylogenetic inaccuracy because there are fewer ways in which a genus, compared with a species, can be placed wrongly. To fully elucidate the effect of taxonomic level, it would be necessary to construct much larger phylogenies where generic level differences are relatively closer to the tips of the tree. However, the explanations given above would still hold in this situation and would yield similar results. The strength of the effect as observed supports this assertion. Where some caution is warranted in the interpretation of these results is in their applicability to the real world, because the equal branch lengths evolutionary model employed here is not the most biologically realistic. In reality, there is likely to be large phenotypic differences between higher taxonomic levels, which will increase the size and importance of the contrasts generated at these levels and may make it more crucial that higher level relationships are known accurately. Position of Disruptions When species are moved in the phylogeny, the detrimental effect on estimating the true correlation coef cient is considerably greater if they are replaced in completely the wrong part of the tree. The r values obtained using the 5sa and 10sa phylogenies are further from the truth than those obtained using the 5sl and 10sl phylogenies. Indeed, they are even worse than those obtained using the rs phylogenies, where the interspeci c relationships are random but at least all species are in the correct part of the tree. Clearly, if you are going to make an error, you should make sure that it is not a large one. A species placed in a wrong genus will generate a greater number of erroneous contrasts than one placed wrongly in the correct genus (because contrasts from two genera will be affected instead of one). Congeners will also tend to be phenotypically relatively similar because they share more common ancestry. Therefore, making a mistake within the same genus will be less profound because the erroneous contrasts produced are still likely to be reasonably correct. If a species is placed in the wrong genus there is likely to be a greater difference in the phenotype (and hence trait values) for that species and other members of the taxon to which it is now mistakenly said to be related. Therefore, the contrast value is likely to be erroneously large. Large contrasts will have a stronger effect on the estimation of r values. In phylogenies 5sl and 10sl, the differences from the truth are higher than they would be in reality. These two treatments are meant to represent the case where species level errors are made but are within the same genus. The way that the program Mix works is simply to move species to nearby in the phylogeny. Because of the properties of individual topologies, this movement could still result in a species being placed in a wrong genus. Therefore, in reality, when disruptions at the tips are con ned to the correct genus the effects are actually likely to be less than those estimated using phylogenies 5sl and 10sl. Types of Error With the two independent variables x and y, we should expect levels of type Bxy error (where the true phylogeny gives a signi cant relationship but the inaccurate topologies do not) to be very low. However, in the phylogenies with disruptions at the higher taxonomic levels (i.e., 1g, 3g, and rg), although type Axy errors are considerably reduced compared with other inaccurate phylogenies the amount of type Bxy error (what there is) remains much the same (Table 2). Thus, an evolutionary correlation may be more likely to be masked by slight phylogenetic inaccuracy than is the case where no correlation is wrongly interpreted as signi cant correlation because of inaccuracy. There may be a generally greater danger with inaccurate topologies of underestimating a true evolutionary correlation (i.e., what I term type B error but traditionally might represent type II error). This distinction is noteworthy because researchers that discuss

10 550 SYSTEMATIC BIOLOGY VOL. 51 the need for phylogeny to be incorporated into comparative analyses (e.g., Felsenstein, 1985) nearly all talk about type I error as the main problem. Huey (1987) is an exception, and his demonstration of missing a true evolutionary correlation when using raw species data may reveal a greater danger with inaccurate topologies. Abouheif (1998) made a similar point for completely random topologies. Further Remarks on the Danger of Using Random Trees Martins (1996) proposed that in the absence of any knowledge of the true phylogeny comparative analyses on a data set should be performed using a number (say 1,000) of random trees, and the results should then be combined. Although I have not speci cally tested that method here, the results from my use of random phylogenies (where the 500 data sets were analyzed using one different random phylogeny each) provide an almost equivalent test. The results indicate that random phylogenies do not provide good estimates of the true evolutionary correlation, either quantitatively or qualitatively. In this respect, my results suggest an interpretation similar to that of Abouheif (1998), who argued that using random phylogenies is equivalent to using an unresolved bush phylogeny, which is the situation assumed by a standard raw data analysis. Both Martins (1996) and Abouheif (1998) used real biological data in their analyses; therefore, they could not know the true phylogeny of the organisms under study. There is some indication here that using random phylogenies is actually worse than using no phylogeny at all, which goes against using Martins (1996) method and against Abouheif s (1998) argument. Intuitively, Abouheif s (1998) explanation does make sense. The consensus phylogeny of all the possible phylogenies for a group of organisms is an unresolved bush, so sampling at random from the full range of possible phylogenies might be expected to produce a mean result that is the same as that produced from the consensus. However, when this approach is taken, some random relationships between species will appear more often than others by chance. These relationships are most likely to be wrong and will produce misleading contrasts that will result in the calculated correlation coef cient being even further from the truth than that obtained from the raw data. Further support for this argument can be obtained from Abouheif s (1998) article. For his hypothetical example (his Figure 5), he used the relationship between three taxa. Using each of the three possible phylogenies, he calculated that the correlation coef cients obtained for the relationship between two hypothetical variables (r D 0:67, 0.67, and 0.12). He then showed that the mean of these values is equal to the correlation coef cient obtained from the raw data (r D 0:5). However, suppose just two of these trees were taken as a sample of random trees (as per Martins method). Clearly, no matter which two trees are chosen the mean of the correlation coef cients is not going to equal that obtained using raw data. Scaling this problem up, with just 10 species in a phylogeny, there are 34,459,425 possible topologies; thus, even 1,000 random trees would be a mere fraction of those possible. The poor performance of random phylogenies compared with raw data only seems to apply, however, to the results of the analysis of the relationship between the two independent variables, that is, x and y. When the relationship between two linked variables, x and z, was analyzed, the random phylogenies performed as well as the raw data (no phylogeny at all), in agreement with Abouheif (1998). The most obvious explanation for this nding is that as variables become more highly correlated the effect of phylogeny ceases to be important. The SD of the r values obtained from using random phylogenies is less for the xz relationship than for the xy relationship (Tables 1 and 3). However, the results with x and z indicate that using no phylogeny or random phylogenies still produces considerably wrong results when compared to the true phylogeny. The reason behind this difference in results with random phylogenies therefore remains puzzling. Other Phylogenetic Comparative Methods Independent contrasts are usually considered the most robust of all the phylogenetic comparative methods. Purvis et al. (1994) found that under most conditions independent contrasts dealt better with poorly resolved phylogenies than did phylogenetic

11 2002 SYMONDS TOPOLOGICAL INACCURACY AND INDEPENDENT CONTRASTS 551 autocorrelation (Cheverud et al., 1985). Similarly, Oakley and Cunningham (2000) showed that independent contrasts more accurately estimated evolutionary correlations than did methods based on ancestor reconstruction (e.g., Schluter et al., 1997). Maximum likelihood methods (e.g., Lynch, 1991) have been less explicitly tested but are considered by some to be unreliable at the present time (Ridley and Grafen, 1996). Given the heavy reliance of all the above methods on an accurate, fully resolved phylogeny, it seems unlikely that these methods would perform any better than independent contrasts when used with an inaccurate topology. Methods based on ancestor reconstruction would probably perform worse (because of the attempt to estimate the states of ancestors that never existed). The only comparative phylogenetic method that might be less affected than independent contrasts is that of pairwise comparisons (as used by Møller and Birkhead, 1994), where a fully resolved topology is less essential (although still important). However, because the degrees of freedom are heavily reduced using this method, the usefulness of pairwise comparisons remains limited, except for large data sets. CONCLUSIONS How wrong can a topology be before the results of independent contrasts analyses are invalidated? The answer depends on a number of factors, including the nature of the true relationship between the variables being tested and the level and position of the phylogenetic disruptions. In these analyses I have attempted to provide several measures, both qualitative and quantitative, to help estimate how wrong is wrong. However, the answer to the question could be different depending on which measure is being evaluted. If a subjective decision is made that a 10% level of error is the maximum allowable to be able to accept con dently the conclusions of the analysis, then inaccuracy can be accepted at the intergeneric level but not at the interspeci c level. However, if only actual signi cant differences in the r values obtained are examined (Tables 2 and 4), then a fairly high amount of topological inaccuracy, even at the tips of the tree, might be acceptable provided that there is a high level of con dence that a species has not been placed in a completely wrong part of the tree. Broadly speaking, a phylogeny with minor errors at the tips con ned to the correct genus might produce incorrect conclusions up to 20% of the time, whereas a phylogeny with major errors at the tips would be more likely to yield wrong conclusions up to 30% of the time. In reality, the rst estimate is probably too high given the usual tendency for congeners to be phenotypically similar. As disruptions occur deeper in the tree, error rates drop to <10%, although this value this will be dependent on the extent of phenotypic differences among the higher taxonomic levels in the sample. From these results, I propose three guidelines. First, the use of random phylogenies is not helpful in a situation where the true phylogeny is unknown. It is unlikely that these phylogenies would provide anywhere near the correct estimate of r values, being qualitatively wrong between 30% and 40% of the time (equivalent to using raw data). The only case I can see for agreeing to their use would be that envisaged by Losos (1994), where if all the random phylogenies yielded the same answer (e.g., that a relationship was signi cant), then one probably could assume that the true phylogeny would also yield the same result. Using even a little information (e.g., a classi cation) to help construct a phylogeny is preferable to a completely random approach (clearly the more resolution the tree has the more accurate the comparative analysis will be). Martins (1996) also emphasized this point and only proposed using her method when absolutely no phylogenetic information is available. However, I am tempted to agree with Harvey et al. (1995); in the absence of any information on evolutionary relationships, a phylogenetic comparative analysis is not possible at all. My second guideline is that if there are grave doubts over the positions of certain species within a part of the tree, then these species should be omitted from the data set. The effect of placing a species into a wrong genus or family can be great enough to in- uence results detrimentally. In the simulations described here, between 20% and 30% of the results were qualitatively wrong when species were placed in the wrong genus. The third guideline is that raw data should not be used in a cross-species analysis to estimate an evolutionary correlation or

12 552 SYSTEMATIC BIOLOGY VOL. 51 regression. The results described here suggest that use of no phylogeny may give qualitatively misleading results at least 30% of the time. This conclusion is of course somewhat biased in that the simulations were set up to give this conclusion anyway. However, the need to incorporate phylogeny will probably increase in importance at the recent evolutionary level, such as in studies of species within a genus or populations within a species. Using a phylogeny may not be so important in an analysis at higher taxonomic levels because the groups involved are more distant from their common ancestor and will have experienced considerably more independent evolution. These three guidelines may be used to answer in part the question of how wrong is wrong, but what of the real world? The problem here is that if one is uncertain of the true phylogeny, one cannot know how wrong a particular phylogeny is. In reality, one is not going to know whether a phylogeny has ve errors at the tips or not. Is using what is believed to be an approximately correct phylogeny acceptable? Again, the answer is: It depends. Studies should incorporate a manageable number of possible phylogenies. Examination of the raw data and the independent contrasts produced from them (in particular the distribution of the latter) should at least provide some clue as to the con dence that can be placed in the conclusions drawn. Concern is warranted if either of two conditions exist: 1. The distribution of the r values obtained is particularly broad. As the amount of topological inaccuracy in the phylogeny increases so does the variance of the r values obtained. 2. The correlation or regression values obtained are strongly dependent on one or two outlying contrasts. These contrasts are the ones most likely to affect results if they are the result of wrong relationships. Although I do not advocate the automatic removal of such contrasts, they should be checked to see by which sister taxa they are produced. If these contrasts result from a relationship that is controversial or doubtful, then the results and conclusion should be treated with caution. The conclusions drawn about the evolutionary relationship between two variables depend on the phylogeny used in the analysis, as Harvey and Pagel (1991) explicitly warned. A exploration of the robustness of these conclusions given known topological disruptions has revealed that the number, position, and level of the disruptions each play an important role. Three things must be avoided in comparative analysis: too much uncertainty at the tips (the interspeci c relationships are the most crucial of all), the placement of species into wrong higher taxa, and using completely random phylogenies. All of these factors result in high levels of error in the results. Correct interspeci c relationships may be more important than correct higher taxon relationships, which is surprising. Intuitively, the opposite might be expected, that the relationships among orders and families is most important and that a few errors at the species level do not really matter. In fact, the present analyses show that independent contrasts may be robust to even a fairly high amount of topological inaccuracy at higher taxonomic levels but that even a few mistakes at the tips cause problems, especially if species are placed in the wrong part of the tree. These analyses also show that any knowledge of the phylogeny (even a classi cation) can substantially improve results and that random phylogenies are unlikely to yield meaningful results. ACKNOWLEDGMENTS Ehab Abouheif, Adrian Friday, and Chris Klingenberg read various early drafts of this article. Their advice helped resolve some muddy thinking and improved the manuscript greatly. I also thank Andy Purvis and Emilia Martins for their constructive criticism. This work was supported by a BBSRC Research Studentship. REFERENCES ABOUHEIF, E Random trees and the comparative method: A cautionary tale. Evolution 52: BAUWENS, D., AND R. D õ AZ-URIARTE Covariation of life-history traits in lacertid lizards: A comparative study. Am. Nat. 149: BJÖRKLUND, M Are comparative methods always necessary? Oikos 80: CHEVERUD, J. M., M. M. DOW, AND W. LEUTENEGGER The quantitative assessment of phylogenetic constraints in comparative analyses: Sexual dimorphism in body weight among primates. Evolution 39: DÍAZ-URIARTE, R., AND T. GARLAND, JR Testing hypotheses of correlated evolution using phylogenetically independent contrasts: Sensitivity to deviations from Brownian motion. Syst. Biol. 45:27 47.

13 2002 SYMONDS TOPOLOGICAL INACCURACY AND INDEPENDENT CONTRASTS 553 DÍAZ-URIARTE, R., AND T. GARLAND, JR Effects of branch length errors on the performance of phylogenetically independent contrasts. Syst. Biol. 47: FELSENSTEIN, J Phylogenies and the comparative method. Am. Nat. 125:1 15. GARLAND, T., JR., P. H. HARVEY, AND A. R. IVES Procedures for the analyses of comparative data using phylogenetically independent contrasts. Syst. Biol. 41: GRAFEN, A The phylogenetic regression. Philos. Trans. R. Soc. Lond. B 326: HARVEY, P. H Comparing uncertain relationships: The Swedes in revolt. Trends Ecol. Evol. 6: HARVEY, P. H., AND M. D. PAGEL The comparative method in evolutionary biology. Oxford Univ. Press, Oxford, U.K. HARVEY, P. H., A. F. READ, AND S. NEE Further remarks on the role of phylogeny in comparative ecology. J. Ecol. 83: HUEY, R. B Phylogeny, history and the comparative method. Pages in New directions in ecological physiology (M. E. Feder, A. F. Bennett, W. W. Burggren, and R. B. Huey, eds.). Cambridge Univ. Press, Cambridge, U.K. KLINGENBERG, C. P Multivariate allometry. Pages in Advances in morphometrics (L. F. Marcus, M. Corti, A. Loy, G. J. P. Naylor, and D. E. Slice, eds.). Plenum, New York. LOSOS, J. B An approach to the analysis of comparative data when a phylogeny is unavailable or incomplete. Syst. Biol. 43: LYNCH, M Methods for the analysis of comparative data in evolutionary biology. Evolution 45: MARTINS, E. P Conducting phylogenetic comparative studies when the phylogeny is not known. Evolution 50: MARTINS, E. P Adaptation and the comparative method. Trends Ecol. Evol. 15: MARTINS, E. P., AND T. GARLAND, JR Phylogenetic analyses of the correlated evolution of continuous characters: A simulation study. Evolution 45: MØLLER, A. P., AND T. R. BIRKHEAD A pairwise comparative method as illustrated by copulation frequency in birds. Am. Nat. 138: OAKLEY, T. H., AND C. W. CUNNIGHAM Independent contrasts succeed where ancestor reconstruction fails in a known bacteriophage phylogeny. Evolution 54: PAGEL, M. D A method for the analysis of comparative data. J. Theor. Biol. 156: PRICE, T Correlated evolution and independent contrasts. Philos. Trans. R. Soc. Lond. B 352: PURVIS, A., AND T. GARLAND, JR Polytomies in comparative analyses of continuous characters. Syst. Biol. 42: PURVIS, A., J. L. GITTLEMAN, AND H.-K. LUH Truth or consequences: Effects of phylogenetic accuracy on two comparative methods. J. Theor. Biol. 167: PURVIS, A., AND A. RAMBAUT Comparative analysis by independent contrasts (CAIC): An Apple Macintosh application for analysing comparative data. Comp. Appl. Biosci. 11: RIDLEY, M., AND A. GRAFEN How to study discrete comparative methods. Pages in Phylogenies and the comparative method in animal behavior (E. P. Martins, ed.). Oxford Univ. Press, New York. SCHLUTER, D., T. PRICE, A. Ø. MOOERS, AND D. LUDWIG Likelihood of ancestor states in adaptive radiation. Evolution 51: YANG, Z., N. GOLDMAN, AND A. FRIDAY Maximum likelihood trees from DNA sequences: A peculiar statistical estimation problem. Syst. Biol. 44: ZAR, J. H Biostatistical analysis, 3rd edition. Prentice-Hall, Upper Saddle River, New Jersey. First submitted 9 May 2001; reviews returned 11 November 2001; nal acceptance 13 March 2002 Associate Editor: Rod Page