Bayesian Model Adequacy and Choice in Phylogenetics

Transcription

1 Bayesian Moel Aequay an Choie in Phylogenetis Jonathan P. Bollbak Department of Biology, University of Rohester Bayesian inferene is beoming a ommon statistial approah to phylogeneti estimation beause, among other reasons, it allows for rapi analysis of large ata sets with omplex evolutionary moels. Conveniently, Bayesian phylogeneti methos use urrently available stohasti moels of sequene evolution. However, as with other moel-base approahes, the results of Bayesian inferene are onitional on the assume moel of evolution: inaequate moels (moels that poorly fit the ata) may result in erroneous inferenes. In this artile, I present a Bayesian phylogeneti metho that evaluates the aequay of evolutionary moels using posterior preitive istributions. By evaluating a moel s posterior preitive performane, an aequate moel an be selete for a Bayesian phylogeneti stuy. Although I present a single test statisti that assesses the overall (global) performane of a phylogeneti moel, a variety of test statistis an be tailore to evaluate speifi features (loal performane) of evolutionary moels to ientify soures failure. The metho presente here, unlike the likelihoo-ratio test an parametri bootstrap, aounts for unertainty in the phylogeny an moel parameters. Introution The results of any phylogeneti analysis are onitional on the hosen moel. Moels that fit the ata poorly an lea to erroneous or onsistently biase inferenes of phylogeny (Felsenstein 1978; Huelsenbek an Hillis 1993; Gaut an Lewis 1995; Sullivan an Swoffor 1997; Bruno an Halpern 1999). For example, a moel that assumes equal rates aross sites (rate homogeneity) may result in inonsistent inferenes even if all other parameters of the moel are orret (Gaut an Lewis 1995). The tremenous inrease in omputational power over the last few years has resulte in the evelopment of a bewilering assortment of moels of sequene evolution for researhers to hoose from (for a review see Swoffor et al. 1996; Huelsenbek an Bollbak 01). Despite the potentially severe effets of poor moel fit, inaequate moels were use in four out of five reent artiles in a primary systematis journal (Posaa an Cranall 01). The parameters of a phylogeneti moel esribe the unerlying proess of sequene evolution. The maximum likelihoo an Bayesian methos of statistial inferene both estimate these parameters (inluing the topology) using the likelihoo funtion, a quantity hereafter referre to as p(x) (whih shoul be rea as the probability of the ata, X, onitione on a speifi ombination of moel parameters, ; more formally, the likelihoo is proportional to the probability of observing the ata). In maximum likelihoo, inferenes are base on fining the topology relating the speies, branh lengths, an parameter estimates of the phylogeneti moel that maximize the probability of observing the ata. Bayesian inferenes, on the other han, are base on the posterior probability of the topology, branh lengths, an parameters of the phylogeneti moel on- Key wors: phylogenetis, Bayesian inferene, moel etermination, moel seletion, moel aequay, posterior preitive ensities, posterior preitive simulations. Aress for orresponene an reprints: Department of Biology, University of Rohester, Rohester, New York bollbak@brahms.biology.rohester.eu. Mol. Biol. Evol. 19(7): by the Soiety for Moleular Biology an Evolution. ISSN: itione on the ata. Posterior probabilities an be alulate using Bayes s theorem. Determining whih moel is best suite to the ata an be ivie into two istint riteria moel aequay (or assessment) an moel hoie (or seletion). Moel aequay is an absolute measure of how well a moel uner srutiny fits the ata. Moel hoie, on the other han, is a relative measure: the best fitting moel from those available is hosen. Although a moel may be the best hoie, it may be, by absolute stanars, inaequate. The likelihoo-ratio test (LRT) an Bayes fators are moel hoie tests: they measure relative merits of ompeting moels but reveal little about their overall aequay. (Although formally, the LRT evaluates the aequay of a moel [Golman 1993], in pratie it is use as a moel hoie strategy.) Although moel aequay an hoie are istint but relate riteria, they are often evaluate simultaneously by omparing neste moels whih iffer by a single parameter (see Golman 1993). Ieally, we woul use only aequate moels for a phylogeneti analysis, but in pratie we often settle for the best available moel. In fat, most moels appear to be poor esriptions of sequene evolution (Golman 1993). How oes one hoose an aequate phylogeneti moel? Traitional maximum likelihoo approahes to moel seletion employ the LRT (for hierarhially neste moels) or the parametri bootstrap (for nonneste moels) (Golman 1993). Both methos epen on a partiular topology, often generate by a relatively fast metho suh as parsimony or neighbor-joining. (See Posaa an Cranall [01] for an analysis of the effets of topology hoie on moel seletion using the LRT, the Akaike information riterion [AIC; Akaike 1974], an the Bayesian information riterion [BIC; Shwarz 1974].) The LRT evaluates the merits of one moel against another by fining the ratio of their maximum likelihoos. For neste moels, the LRT statisti is a- symptotially 2 -istribute with q egrees of freeom (Wilks 1938), permitting omparison with stanar 2 tables to etermine signifiane. Unfortunately, signifiane annot be evaluate in this way when moels are not neste, or the null fixes parameters of the alternative 1171

2 1172 Bollbak moel at the bounary of the parameter spae, beause the regularity onitions of the 2 are not satisfie. The parametri bootstrap, alternatively, is not onstraine by regularity onitions allowing omparison of nonneste moels but is time-intensive an may require researhers to write omputer simulations to approximate the null istribution. Unfortunately, this omputationally expensive approah, the AIC, an the BIC remain the only urrent methos (apart from simple inspetion of the log likelihoo sores) to ompare nonneste likelihoo moels. The results of the LRT an the parametri bootstrap are onitional on the topology an moel parameters hosen to onut the test. The assume topology may be hosen using a fast metho, suh as parsimony, known to be inonsistent uner ertain onitions (Felsenstein 1978). The branh lengths an moel parameters (suh as transition/transversion bias) are generally maximum likelihoo point estimates onitional on the assume topology. Ieally, a statistial metho shoul minimize the number of assumptions mae. Bayesian methos offer an effiient means of reuing this reliane on assumptions. These methos an aommoate unertainty in topology, branh lengths, an moel parameters. For example, Suhar, Weiss, an Sinsheimer (01) reently evelope a Bayesian metho of moel seletion that uses reversible jump Markov hain Monte Carlo (MCMC) an employs Bayes fators for omparing moels. This approah is a Bayesian analog of the LRT: the Bayes fator iniates relative superiority of ompeting moels by evaluating the ratio of their marginal likelihoos. In this approah, prior probability istributions of the moels must be proper but allowably vague. If the information ontaine in the ata about moel aequay is small, then the priors will etermine the outome of the test. In this situation, most of the posterior will be plae on the more ompliate moel (Carlin an Chib 1995). Although the metho of Suhar, Weiss, an Sinsheimer (01) allows omparison of moels without strit epenene on a partiular set of assumptions, like traitional likelihoo approahes, it oes not expliitly evaluate the absolute merits of a moel. The hosen moel may well be severely inaequate. Here, I present a Bayesian metho using posterior preitive istributions to expliitly evaluate the overall aequay of D moels of sequene evolution. The approah I use, posterior preitive hek by simulation (Rubin 1984; Gelman, Dey, an Chang 1992; Gelman et al. 1995; Gamerman 1997), is a Bayesian analog of lassial frequentist methos suh as the parametri bootstrap or ranomization tests (Rubin 1984). A similar approah has been use reently to test moleular evolution hypotheses (Huelsenbek et al. 01; Nielsen an Huelsenbek 01; Nielsen 02). The rationale motivating this approah is that an aequate moel shoul perform well in preiting future observations. In the absene of future observations, preite observations are simulate from the posterior istribution, uner the moel in question. These preite ata are then ompare with the original ata using a test statisti that summarizes the ifferenes between them. Careful evaluation of the moel parameters permits enhanement (aition of parameters) or simplifiation (elimination of irrelevant parameters) of the moel to improve its overall fit to the ata. Here, I use the multinomial test statisti to evaluate overall aequay of phylogeneti moels. Materials an Methos Moels of Sequene Evolution Moels of sequene evolution use in phylogenetis moel nuleotie substitutions as a stohasti proess, most of whih are time-homogenous, time-reversible Markov proesses. Reversibility of a moel is satisfie when the rate of forwar an reverse hanges are equal, suh that i q ij j q ji. However, general, nonreversible substitution moels have also been evelope an explore in a variety of phylogeneti ontexts (Yang 1994; Huelsenbek, Bollbak, an Levine 02). For the sake of brevity, this stuy restrits itself to reversible moels, but the metho is easily extene to nonreversible, time-heterogeneous, or other lasses of moels. Four moels will be use in this stuy (1) the generaltime-reversible moel (; Tavaré 1986), (2) Hasegawa-Kishino-Yano moel (HKY85; Hasegawa, Kishino, an Yano 1985), (3) Kimura s two-parameter moel (; Kimura 1980), an (4) Jukes-Cantor moel (; Jukes an Cantor 1969). These represent the most ommonly implemente moels in the phylogeneti literature. The first three moels are speial ases of the. The is the most general moel of D sequene evolution allowing for ifferent rates for eah substitution lass an aommoating unequal base frequenies. The instantaneous rate matrix, Q, for this moel is: ac bg T a A G et Q {q ij}. (1) ba C f T e f A C G The iagonals of the matrix are set suh that the rows eah sum to 0. When the rates in the aforementione matrix are onstraine suh that b e, an a f 1, the moel ollapses into the HKY85 moel with the following rate matrix: C G T A G T A C T Q {q } (2) ij A C G where is a rate parameter esribing a transition-transversion bias. If the HKY85 moel is onstraine suh that the base frequenies are equal ( A C G T 0.25), this moel ollapses into the moel with the following rate matrix:

3 Moel Determination Q {q }. (3) ij 1 1 Finally, if the moel is onstraine suh that 1, it ollapses into the moel with equal rates between all substitution lasses. Using instantaneous rates, substitution probabilities for a hange from nuleotie i to j over a branh of length v an be alulate as P {P ij } e Qv. In the ase of the,, an HKY85 moels, lose-form analytial solutions for the substitution probabilities are available (Swoffor et al. 1996). For the moel, lose-form solutions o not exist, an stanar numerial linear algebra approahes are employe to exponentiate the term Qv (Swoffor et al. 1996). With a matrix of substitution probabilities available, alulation of the likelihoo is straightforwar using Felsenstein s (1981) pruning algorithm. Posterior Preitive Simulations In evaluating a moel s aequay, we woul like to know how well it esribes the unerlying proess that generate the D sequene ata in han. Therefore, an ieal moel shoul perform well in preiting future observations of the ata. In pratie, future observations are unavailable to researhers at the time of ata analysis. However, surrogate future observations uner the moel being teste an be simulate by sampling from the joint posterior ensity of trees an moel parameters (hene, posterior preitive simulations; Rubin 1984). Beause of the omplexity of the phylogeny problem the large number of possible ombinations of topology, branh lengths, an moel parameters the posterior ensity annot be evaluate analytially. Lukily, we an use numerial methos to obtain an approximation of this ensity (pˆ[x]) using the MCMC tehnique (Li 1996; Mau 1996; Mau an Newton 1997; Yang an Rannala 1997; Larget an Simon 1999; Mau, Newton, an Larget 1999; Newton, Mau, an Larget 1999; Huelsenbek an Ronquist 01). Moel assessment using this approah requires approximating the following preitive ensity: p(x X obs) B(s) k p(x, v,, X )p(, v, X ) v. k1 vk k obs k k obs k (4) Trees are labele 1, 2,..., B(s), where B(s) (2s 5)!/2 s3 (s 3)! is the number of unroote trees for s speies. For all unroote topologies, B(s), we integrate over branh lengths (v k ) an parameters of the moel (). Evaluation of this ensity requires knowlege of the joint posterior ensity, but one an approximation of the joint posterior ensity of moel parameters an topologies, pˆ(, v, X), has been obtaine, the posterior preitive ensity (eq. 4) an be approximate numerially by Monte Carlo simulation in the following way (1) Make a ranom raw from the joint posterior istribution of trees an moel parameters, uner the moel being teste. (In pratie, this an be aomplishe by sampling the posterior output of a program that approximates posterior istributions, suh as MrBayes [Huelsenbek an Ronquist 01]). (2) Using these ranom raws (whih inlue values for the parameters of the substitution proess, topology, an branh lengths) an the moel being teste, simulate a ata set, X 1,ofthe same size as the original ata set. (3) Repeat steps 1 an 2 N times to reate a olletion of ata sets, X 1, X 2,..., X N. (4) These simulate ata sets are a numerial Monte Carlo approximation of the posterior preitive ensity (eq. 4): N N 1 obs j k k obs N k1 j1 pˆ (X X ) p(x, v,, X ). (5) Test Statistis We now have an approximation of the posterior preitive ensity of the ata, simulate uner the phylogeneti moel being srutinize. But we are still left with the following problem: how an we use this posterior preitive istribution to assess the phylogeneti moel s aequay? This requires a esriptive test statisti (or isrepany variable; Gelfan an Meng 1996) that quantifies isrepanies between the observe ata an the posterior preitive istribution. The test statisti is referre to as a realize value when summarizing the observe ata. An appropriate test statisti an be efine to measure any aspet of the preitive performane of a moel (Gelman et al. 1995). I use the general notation T( ), where refers to the variable being teste. To use this statisti, alulate T( ) (an example of the propose statisti will be shown later), for the posterior preitive ata sets to arrive at an approximation of the preitive istribution of this test quantity. This istribution an then be ompare with the realize test statisti, whih is alulate from the original ata. To asses how well a phylogeneti moel is able to preit future nuleotie observations (overall aequay), a test statisti that quantifies the frequeny of site patterns is appropriate. Here I use the multinomial test statisti to summarize the ifferene between the observe an posterior preitive frequenies of site patterns (Golman 1993). A minor limitation of the multinomial is its assumption of inepenene among sites, restriting its appliation to phylogeneti moels that assume inepenene. Deviations in the posterior preitive frequeny of site patterns from the observe our beause the phylogeneti moel is an imperfet esription of the evolutionary proess. If the evolutionary proess that generate the ata exhibits a GC bias, for instane, then site patterns ontaining a preominane of these bases will be overrepresente. An aequate moel shoul be able to preit this eviation, given the information ontaine in the original sequene ata. The multinomial test statisti of the ata, T(X), is alulate in the following way. Let (i) be the ith unique

4 1174 Bollbak observe site pattern an N (i) the number of instanes this pattern is observe. For a total number of N sites, S 4 k possible site patterns, an n unique site patterns observe, the multinomial test statisti (T[X]) an be alulate as follows: n N (i) N(i) i1 N T(X) ln. (6) This is the natural log ensity of the maximum likelihoo estimator of the multinomial. Alternatively, for ease of omputation, equation 6 an be rewritten as: n S T(X) N ln(n ) N ln(n). (7) To illustrate the multinomial test statisti, let us fin the realize T(X) for k 4 sequenes with N sites from the following hypothetial aligne matrix of D sequenes: AAATCCAGGG AAACCCAACA X. AATCGGTTCA AATCGGTATT There are seven unique site patterns in the matrix. Site patterns x 1,2 {AAAA}, x 3,7 {AATT}, an x 5,6 {CCGG} are observe twie; the four remaining site patterns are observe only one eah. The realize test statisti for this ata, using equation 7 is then: T(X) 3 2 ln(2) 4 1 ln(1) ln() Numerous test statistis an be formulate, but to be useful these test statistis shoul represent a relevant summary of the moel parameters an ata. Preitive P Values Classial frequeny statistis rely on tail-area probabilities to assign statistial signifiane; values that lie in the extremes of the null istribution of the test quantity are onsiere signifiant. Uner lassial statistis, the istributions are onitione on point estimates for moel parameters. Preitive ensities, on the other han, are not. Beause values are sample from the posterior istribution of moel parameters an trees, they are sample in proportion to their marginal probabilities. This sampling sheme allows them to be treate as nuisane parameters values not of iret interest an to be integrate out. The preitive istribution of the test statisti allows us to evaluate the posterior preitive probability of the moel. The posterior preitive P value for the test statisti is: N 1 P I(T(X ) T(X)), (8) T N i1 where I is an iniator funtion that takes on the value 1 when the equality is satisfie an 0 otherwise, T(X i ) the multinomial test statisti for the ith simulate ata set, an T(X) the realize test statisti. Probabilities less i than the ritial threshol, say 0.05, suggest that the moel uner examination is inaequate an shoul be rejete or refine. Preitive P values are interprete as the probability that the moel woul proue with as extreme a test value as that observe for the ata (Gelman et al. 1995). For an aequate moel, the preitive istribution of T(X) shoul be entere aroun the realize test statisti (i.e., P T 0.5). This approah evaluates the pratial fit of the moel to a ata set; inlusion of aitional taxa or new sequenes requires a revaluation of the moel an its fit. Simulations To etermine the utility an power of this approah, I simulate 300 ata sets uner a variety of moels an parameter values (see table 1 for a esription of the speifis for eah analysis). For all ata sets the true (moel ata was simulate uner) an the moels are examine. Briefly, I performe three sets of simulations to examine (1) the overall moel aequay, (2) the effets of sequene ivergene, an (3) the moel sensitivity. I isuss eah of these in turn subsequently. To test overall moel aequay, I simulate ata sets of 500,, an 4,000 sites uner the moel. The parameters of the moel, for eah ata set, were assigne in the following way (1) instantaneous rates were ranomly hosen from the uniform interval, U(0.0, ], (2) values for the base frequenies were rawn from a Dirihlet istribution with parameters ( A, C, G, T ) ranomly hosen from the interval U[, 4.0], an (3) the overall substitution rate (m) was fixe at 0.5. Trees were simulate uner the birth-eath proess as esribe by Rannala an Yang (1996). Speiation (), extintion (), an taxon sampling () rates were fixe at 2.0,, an 0.75, respetively. To test the effets of sequene ivergene, I simulate ata sets of sites uner the moel. Parameters of the moel were hosen as in the test of overall aequay. For all ata sets the tree in figure 1 was use. The overall substitution rate was varie from low (m 0.1) to high (m 0.75) ivergene. Finally, for the moel sensitivity analyses, ata sets of an sites were simulate uner the moel. Violation of the moel s assumptions varie from none ( 1) to extreme ( 12). The tree in figure 1, with an intermeiate substitution rate (m 0.5), was use to simulate ata. Power Analysis Uner the posterior preitive simulation approah the null hypothesis is that the moel is an aequate fit to the ata. A moel is rejete if the realize test statisti is less than the ritial value ( 0.05). Otherwise the moel was aepte. The fration of times the null moel is aepte falsely is an estimate of Type II error rate, the omplement (1 ) is the power of a test. The power of the multinomial test statisti to rejet a false moel is etermine by the analysis of all the ata sets esribe previously using the moel.

5 Moel Determination 1175 Table 1 Simulation Conitions Test True Moel True Moel Parameters True Tree Number Taxa Number Charaters Substitution Rate (m) Moels Teste Repliates Overall aequay... Sequene ivergene.. Moel sensitivity , , a Rate parameters of the moel (,,, e, f) are hosen for eah repliate by rawing a uniform ranom number from the interval, U(0.0, ]. b Base frequenies of the moel ( A C G T ) are rawn from a Dirihlet istribution using parameters rawn ranomly from the interval, U[1,0, 4.0]. For eah repliate a birth-eath tree was simulate as esribe in Materials an Methos. Topology was kept fixe for these simulations (fig. 1). Branh lengths were multiplie by the overall rate of substitution (m). Analysis of the -Globin Pseuogene To illustrate the metho of moel etermination using posterior preitive istributions, a D sequene ata set was analyze uner the, HKY85, an moels. The ata set is the primate -globin pseuogene (Koop et al. 1986; Golman 1993) with the aition of one speies the pygmy himpanzee. This ata set onsists of seven speies human beings (Homo sapiens), himpanzee (Pan trogloytes), pygmy himpanzee (Pan panisus), gorilla (Gorilla gorilla), orangutan (Pongo pygmaeus), rhesus monkey (Maaa mulatta), an owl monkey (Aotus trivirgatus). The original D ata matrix was 2,5 sites. Inels ( 183 sites) were exlue from the analyses, yieling a matrix of 2,022 sites. Programs MrBayes v2.0 was use to approximate the posterior istribution of a moel s parameters an trees (Huelsenbek an Ronquist 01). The Metropolis-ouple MCMC algorithm was use with four hains (Huelsenbek an Ronquist 01). The Markov hains were run for 0,000 generations an sample every 0th generation. The first,000 generations were isare as burn-in to ensure sampling of the hain at stationarity. Convergene of the Markov hains was verifie by plotting the log probability of the hain as a funtion of generation to verify that they ha plateaue. A program that reas the posterior output of MrBayes, simulates preitive ata sets, an evaluates the multinomial test statisti was written in the C language. The oe is available upon request. Results an Disussion Overall Moel Aequay The overall aequay of an evolutionary moel was explore by simulating nuleotie ata sets of a variety of sequene lengths, on a birth-eath tree, uner the moel (see table 1). The birth-eath proess was use to explore the effets of ifferent branh lengths an branhing orer. A omparison of the moel with ata sets simulate uner the, beause of the

6 1176 Bollbak FIG. 1. Tree use in simulations, testing sequene ivergene, an sensitivity to moel violations. The branh lengths were multiplie by the overall rate of substitution, m. Values for m an be foun in table 1. large ifferene in the number of parameters (eight), represents a onservative estimate of power. An illustration of the preitive istribution of the multinomial test statisti for three ata sets of 500,, an 4,000 sites is presente in figure 2. As expete the true () moel enters the simulate istributions aroun the realize test statisti (fig. 2A, C, an E). The false () moel performe poorly (P T ; fig. 2B, D, an F). Inreasing the number of sites in the ata set inrease the power of the test to rejet the moel: the preitive istributions uner the moel move farther from the realize test statisti. This is beause of the higher number of unique site patterns: inreasing the number of sites inreases the probability of observing rare patterns. The effet of an inreasing number of sites was measure in two ways: (1) using the mean posterior preitive P value (P T; table 2), an (2) using the power of the test (table 3). The first measure, mean P value, ereases below the ritial value as the number of sites inreases. For the true () moel, the mean P value was lose to the expete value of 0.5 for ata sets of all sizes. For the false () moel, the mean P value erease as expete, as the number of sites inrease, ropping below The seon measure, the power of the test, inreases as the number of sites inreases. The true () moel was aepte 0% of the time for ata sets of all sizes. Interestingly, the false () moel was often aepte for small ata sets (table 3). The low power (or high Type II error rate) of the multinomial test statisti to rejet a moel, with small amounts of ata, oul be attribute to a number of auses. First, the FIG. 2. Illustration of the metho omparing versus. Data sets of 500 (A, B), (C, D), an 4,000 (E, F) sites were simulate uner the moel. Preitive istributions were simulate uner the (A, C, E) an (B, D, F) moels. Arrows iniate the values for the realize statisti from the original ata. In all ases the moel, as expete, proue an aequate fit to the ata, whereas the i not (P T ). small number simulations performe result in fairly large 95% onfiene intervals (CI) for the Type II error rate (24% 68%). Seon, the test statisti might not represent a omplete summary of the unerlying proess of sequene evolution. Thir, the approximation of the joint posterior istribution from small amounts of ata may result in a large amount of unertainty an inrease Type II error rates. Sequene Divergene The effet of an inrease in sequene ivergene on power was explore by varying the overall rate of substitution aross the tree shown in figure 1 (m 0., 0.25,, an 0.75). Test ata sets were simulate uner the moel (table 1). The results of sequene ivergene are shown in tables 2 an 3. As previously one, two measurements to evaluate the metho are pre-

7 Moel Determination 1177 Table 2 Mean Posterior Preitive P Values for Simulations Test True Moel Number Charaters Substitution Rate (m) Kappa () Moel Teste P Ta Stanar Deviation Overall aequay... Sequene ivergene... Moel sensitivity , , a Values in bol are signifiant at the 0.05 level. sente: the mean preitive P value (P T) an the power of the test. Using the first, the moel performe well, approahing a mean preitive P value of 0.5 as m inrease. An inrease in the stanar eviation of the preitive P value, from to 0.137, was observe with an inrease in ivergene. This may be the result of a erease in the iversity of site patterns as sites experiene multiple hits an states begin to onverge. The moel, on the other han, performe poorly at all values of m. At ivergene levels of m 0.25, the mean posterior preitive P value (P T; table 2) was below the ritial level of Using the seon measurement, the moel again performe well it was aepte 0% of the time at all levels of ivergene. The moel performe poorly at low levels of sequene ivergene (m Table 3 Power of Test Statisti for Simulations Test True Moel Number Charaters Substitution Rate (m) Kappa () Moel Teste Power (1 ) (%) Overall aequay... Sequene ivergene... Moel sensitivity ,

8 1178 Bollbak 0.); the power of the test was relatively low (50) but rapily inrease to 0 at larger ivergenes (m 0.75). For m, the power of the test was onsierably lower than in ata sets with iential simulation onitions in the test of overall moel aequay (95%; see Overall Moel Aequay). This reution in power may be the result of a number of fators. The first, an most likely, explanation is sampling error; the small number of repliates leas to large onfiene intervals (CI) aroun the Type II error rate (95% CI, 14% 53%). Seon, the moel may be robust to minor violations of its assumptions. For example, in repliates for whih the moel was aepte, assumptions were not severely violate. Analyses of moel sensitivity support this explanation (see later). Thir, the simulation tree for these analyses ha a smaller sum of branh lengths than in the analysis of overall aequay branh lengths are in terms of the expete number of substitutions per site. When m 0.5, figure 1 has a tree length of 2.266, whereas the mean tree length in the aequay analysis was (15 of overall aequay repliates ha longer tree lengths, some as muh as 36% longer). Therefore, the effets of ivergene on power shoul be interprete as a funtion of the total number of expete substitutions per site aross the phylogeny not simply the rate from the root to the tips of the tree (m). Finally, the statisti may be sensitive to the shape of the topology or variations in branh lengths aross the tree. Sensitivity to Moel Violations The sensitivity of the multinomial test statisti to rejet inaequate moels was explore by simulating ata sets uner the moel, varying from 1 to 12, followe by analysis with both the (true) an moels. When 1, the moel ollapses into the moel. Uner these onitions, the moel is not violate an is expete to perform as well as the moel. As inreases, refleting an inrease in the transition-transversion bias, the moel beomes more severely violate an is expete to perform more poorly. The effets of moel violations were explore on ata sets of two sizes:, an sites (tables 2 an 3). Both the an moels performe well for ata sets of sites simulate with a value of 1. The mean posterior preitive P values for the an moels were an 0.426, respetively. Both moels were aepte in 0% of the repliates. For the moel, as inrease the mean P values eline, whereas the moel ontinue to perform well. The probability of aepting the moel was 0% for all repliates exept one ( 12, 95%). The moel performe well at 3 (95% aepte), but as the moel beame inreasingly violate, the power inrease to 95% an 0%, at values of 6 an 12, respetively. A fivefol inrease in the number of sites move the mean posterior preitive P values for the moel towar 0.5 (table 2), an all repliates analyze uner the moel were aepte 0% of the time. As the number of sites inrease from to, the isriminating power of the test statisti inrease, as shown by the rapi eline in the mean preitive P values with inreasing (table 2) an by the inrease power to rejet the moel (table 3). For example, there was a nearly -fol rop in the mean preitive P value between an sites uner the moel with moerate violation for 3 the mean P value erease from to (table 2). In aition, the variane aross the repliate ata sets erease markely. The moel was aepte 0% of the time when was 1 but eline with an inrease in ompare with the site ata sets. This pattern is most ramatially emonstrate in a omparison of ata sets simulate with 3. For sites there was a Type II error rate of 95% as ompare with a Type II error rate of % with sites uner the moel. Analysis of the -Globin Pseuogene The primate -globin pseuogene ata set was analyze uner the, HKY85, an moels. Pseuogenes are nonfuntional opies in whih mutations are not onstraine by seletion, an thus substitution biases shoul reflet mutational biases. Biases in the mutational spetrum will give rise to biases in the observe frequeny of site patterns. The analysis of the mean base frequenies for the -globin pseuogene iniates an AT bias ( A 0.296, C 0.190, G 0.238, T 0.277). Consequently, moels that assume equal base frequenies (i.e., ) are not expete to perform as well as moels that allow for unequal frequenies (i.e., HKY85 an ). The HKY85 an moels are aequate summaries of the true unerlying proess (, P T 0.199; HKY85, P T 0.303), although the HKY85 represents a better fit to the ata the HKY85 moel was better able to enter the preitive istribution of the test statisti aroun the realize value (fig. 3). This ifferene may be beause of a better moel fit or stohasti error. The moel represents a poor fit to the ata (fig. 3, P T 0.053), even though it annot be expliitly rejete at the 0.05 level. Interestingly, Golman (1993), using the parametri bootstrap, rejete the moel for a similar ata set that exlue the pygmy himpanzee. The moel performe less poorly with the metho presente here. What an we attribute this apparent isrepany to? One possible explanation is that with small numbers of taxa, an subsequently a smaller number of possible site patterns, there is low power assuming that the moel is inaequate, whih, of ourse, may not be the ase. Removal of the pygmy himpanzee sequene an reanalysis of the moel results in an inrease in the preitive P value (, P T 0.123). For the six-speies ata set, the moel performs better at preiting the ata than in the seven-speies ata set. This is not surprising beause with fewer taxa there are fewer possible site patterns an the moel, even with minor violations, shoul perform well. Another explanation is that aommoating unertainty in the topol-

9 Moel Determination 1179 FIG. 3. Analysis of the -globin pseuogene uner the (A), HKY85 (B) an (C) moels. The ata set onsiste of seven taxa an 2,022 nuleoties (see Materials an Methos). The MCMC analysis, performe using MrBayes v2.0, was use to approximate the joint posterior istribution of moel parameters an topologies. The hain was run for 0,000 generations, sampling every 0th generation. The first,000 generations were isare as burn-in. A total of samples was ranomly rawn from the joint posterior istribution of moel parameters, an topologies an simulate ata sets of 2,022 sites were generate uner eah of the moels. The arrows above the istributions are the realize test statisti for the original ata set (4,651.32). Posterior preitive P values for (A), HKY85 (B), an (C) are P T 0.053, P T 0.303, an P T 0.199, respetively. ogy, using the present metho, more aurately esribes moel variane. The moel performe less poorly at preiting the observations than the parametri bootstrap when unertainty was aommoate. The parametri bootstrap, by not aounting for unertainty, may be more liberal in rejeting moels. Analysis of the posterior istribution of trees for this ata set suggests a high egree of unertainty in the relationships between humans, gorillas, an himpanzees there was equal posterior support for the three possible subtrees of these speies. This unertainty was reognize in Golman s (1993) analysis as a polytomy. Therefore, aounting for unertainty in the topology, branh lengths, an moel parameters appears to be important in etermining moel aequay. When we are onfronte with two moels that appear to perform equally well, how o we proee in hoosing between them? One approah woul be to simply hoose the less omplex moel, thus favoring a reution in the number of free parameters to be estimate. Another alternative woul be to use the metho presente here, with a test statisti that summarizes loal features of the moels. In this way, ientifiation of partiular features of a moel that o not ontribute explanatory power an be ientifie an eliminate. Conversely, testing the aition of new parameters to a simpler moel oul lea to a better fit to the ata using an expane moel. In these ways, we an ientify the best moel an arrive at a soun statistial hoie. Conlusions The metho I present here permits expliit evaluation of a phylogeneti moel s aequay using posterior preitive simulations. An aequate moel shoul perform well in preiting future observations of the ata; in the absene of suh observations, simulations from the posterior istribution are use as surrogate observations. This approah iffers, most importantly, from the traitional likelihoo-base approahes by taking into aount unertainty in topology, branh lengths, an moel parameters. Therefore, moel hoie has been free from onitioning on these parameters an results in a more aurate estimate of moel variane. The multinomial test statisti is presente to evaluate the global (or overall) performane of a moel through the posterior preitive istribution. The power of the multinomial test statisti was explore uner a wie range of onitions. A number of fators have been shown here to inrease power (1) inreasing the number of sites, (2) inreasing sequene ivergene (expete number of substitutions per site), an (3) the egree of violation to a moel s assumptions. An appealing aspet of posterior preitive istributions, when use for moel heking, is that a wie variety of test statistis an be formulate to hek various aspets of phylogeneti moels. For example, posterior preitive istributions an be use to etet variation in rates aross ata partitions, allowing moels to be expane to aommoate rate heterogeneity. The generality of the posterior preitive approah, an the evelopment of new test statistis, will permit further exploration an evelopment of more omplex an realisti phylogeneti moels. Aknowlegments This work was supporte by grants from the NSF to John Huelsenbek (MCB an DEB ), to whom I am grateful for his support throughout this projet. Also, I woul like to express my eep thanks to Anrea Betanourt, John Huelsenbek, Kelly Dyer, Rasmus Nielsen, an Freerik Ronquist for taking the time to rea early versions of the manusript. Eah an every one of them provie invaluable omments, that ultimately mae the manusript better. John Huelsenbek, Bret Larget, Rasmus Nielsen, Ken Karol, an Anrea Betanourt patiently listene to me rone on about this projet, an offere insightful omments that benefite this work, an for this they have my eepest gratitue. An finally, I woul like to thank two anonymous reviewers who gave ritial attention to the manusript an provie valuable omments.

10 1180 Bollbak LITERATURE CITED AKAIKE, H A new look at statistial moel ientifiation. IEEE Trans. Autom. Contr. 19: BRUNO, W. J., an A. L. HALPERN Topologial bias an inonsisteny of maximum likelihoo using wrong moels. Mol. Biol. Evol. 16: CARLIN, B. P., an CHIB, S Bayesian moel hoie via Markov hain Monte Carlo methos. J. R. Stat. So. B 57: FELSENSTEIN, J Cases in whih parsimony or ompatibility methos will be positively misleaing. Syst. Zool. 27: Evolutionary trees from D sequenes: a maximum likelihoo approah. J. Mol. Evol. 17: GAMERMAN, D Markov Chain Monte Carlo: stohasti simulation for Bayesian Inferene. Chapman an Hall, New York. GAUT, B., an P. LEWIS Suess of maximum likelihoo in the four taxon ase. Mol. Biol. Evol. 12: GELFAND, A. E., an X.-L. MENG Moel heking an moel improvement. Pp in W. R. GILKS, S. RICH- ARDSON, an D. J. SPIEGELHALTER, es. Markov hain Monte Carlo in pratie. Chapman an Hall, New York. GELMAN, A., J. B. CARLIN, H.S.STERN, an D. B. RUBIN Bayesian ata analysis. Chapman an Hall, New York. GELMAN, A. E., D. K. DEY, an H. CHANG Moel etermination using preitive istributions with implementation via sampling-base methos. Pp in J. M. BERRDO, J. O. BERGER, A. P. DAWID, an A. F. M. SMITH, es. Bayesian statistis 4. Oxfor University Press, New York. GOLDMAN, N Statistial tests of moels of D substitution. J. Mol. Evol. 36: HASEGAWA, M., H. KISHINO, an T. YANO Dating the human-ape split by a moleular lok of mitohonrial D. J. Mol. Evol. 22: HUELSENBECK, J. P., an J. P. BOLLBACK. 01. Appliation of the likelihoo funtion in phylogeneti analysis. Chap. 15, pp in D. J. BALDING, M.BISHOP, an C. CAN- NINGS, es. Hanbook of statistial genetis. John Wiley an Sons In., New York. HUELSENBECK, J. P., J. P. BOLLBACK, an A. LEVINE. 02. Inferring the root of a phylogeneti tree. Syst. Biol. 51: HUELSENBECK, J. P., an D. M. HILLIS Suess of phylogeneti methos in the four taxon ase. Syst. Biol. 42: HUELSENBECK, J. P., an F. RONQUIST. 01. MRBAYES: Bayesian inferene of phylogeneti trees. Bioinformat. Appl. Note 17: HUELSENBECK, J. P., F. RONQUIST, R. NIELSEN, an J. P. BOLL- BACK. 01. Bayesian inferene of phylogeny an its impat on evolutionary biology. Siene 294: JUKES, T., an C. CANTOR Evolution of protein moleules. Pp in H. MUNRO, e. Mammalian protein metabolism. Aaemi Press, New York. KIMURA, M A simple metho for estimating evolutionary rate of base substitutions through omparative stuies of nuleotie sequenes. J. Mol. Evol. 16: KOOP, B. F., M. GOODMAN, P.XU, K.CHAN, an J. L. SLIGH- TOM, Primate eta-globin D sequenes an man s plae among the great apes. Nature 319: LARGET, B., an D. SIMON Markov hain Monte Carlo algorithms for the Bayesian analysis of phylogeneti trees. Mol. Biol. Evol. 16: LI, S Phylogeneti tree onstrution using Markov hain Monte Carlo. Dotoral issertation, Ohio State University, Columbus. MAU, B Bayesian phylogeneti inferene via Markov hain Monte Carlo methos. Dotoral issertation, University of Wisonsin, Maison. MAU, B., an M. NEWTON Phylogeneti inferene for binary ata on enrograms using Markov hain Monte Carlo. J. Comput. Graph. Stat. 6: MAU, B., M. NEWTON, an B. LARGET Bayesian phylogeneti inferene via Markov hain Monte Carlo methos. Biometris 55:1 12. NEWTON, M., B. MAU, an B. LARGET Markov hain Monte Carlo for the Bayesian analysis of evolutionary trees from aligne moleular sequenes. In F. SEILLER-MOSEI- WITCH, T. P. SPEED, an M. WATERMAN, es. Statistis in moleular biology. Monograph series of the Institute of Mathematial Stuies. NIELSEN, R. 02. Mapping mutations on phylogenies. Syst. Biol. (in press). NIELSEN, R., an J. P. HUELSENBECK. 01. Deteting positively selete amino ais sites using posterior preitive p-values. Pp in R. B. ALTMAN, A. K. DUNKER, L. HUNTER, K. LAUDERDALE, an T. E. KLEIN, es. Paifi symposium on bioomputing. Worl Sientifi, New Jersey. POSADA D., an K. A. CRANDALL. 01. Seleting the best-fit moel of nuleotie substitution. Syst. Biol. 50: RANLA, B., an Z. YANG Probability istribution of moleular evolutionary trees: a new metho of phylogeneti inferene. J. Mol. Evol. 43: RUBIN, D. B Bayesianly justifiable an relevant frequeny alulations for the applie statistiian. Ann. Stat. 12: SCHWARZ, G Estimating the imension of a moel. Ann. Stat. 6: SUCHARD, M. A., R. E. WEISS, an J. S. SINSHEIMER. 01. Bayesian seletion of ontinuous-time Markov hain evolutionary moels. Mol. Biol. Evol. 18:1 13. SULLIVAN, J., an D. L. SWOFFORD Are guinea pigs roents? The importane of aequate moels in moleular phylogenies. J. Mammal. Evol. 4: SWOFFORD, D., G. OLSEN, P. WADDELL, an D. M. HILLIS Phylogeneti inferene. Pp in D. HILLIS, C. MORITZ, an B. MABLE, es. Moleular systematis. 2n eition. Sinauer, Sunerlan, Mass. TAVARÉ, S Some probabilisti an statistial problems on the analysis of D sequenes. Pp in Letures in mathematis in the life sienes. Vol. 17[Please provie the publisher name an loation]. YANG, Z Maximum likelihoo estimation of phylogeny from D sequenes when substitution rates iffer over sites. Mol. Biol. Evol. : Maximum likelihoo phylogeneti estimation from D sequenes with variable rates over sites: approximate methos. J. Mol. Evol. 39: YANG, Z., an B. RANLA Bayesian phylogeneti inferene using D sequenes: a Markov hain Monte Carlo metho. Mol. Biol. Evol. 14: WILKS, S The large-sample istribution of the likelihoo ratio for testing omposite hypotheses. Ann. Math. Stat. 9: BRANDON GAUT, reviewing eitor Aepte Marh 25, 02