Effects of the ordering of natural selection and population regulation mechanisms on Wright-Fisher models

Transcription

1 G3: Genes Genomes Genetcs Early Onlne, publshed on May 12, 2017 as do: /g Effects of the orderng of natural selecton and populaton regulaton mechansms on Wrght-Fsher models Zhangy He a,, Mark Beaumont b, Feng Yu a a School of Mathematcs, Unversty of Brstol, Brstol BS8 1TW, Unted Kngdom b School of Bologcal Scences, Unversty of Brstol, Brstol BS8 1TQ, Unted Kngdom Abstract We explore the effect of dfferent mechansms of natural selecton on the evoluton of populatons for one- and two-locus systems. We compare the effect of vablty and fecundty selecton n the context of the Wrght-Fsher model wth selecton under the assumpton of multplcatve ftness. We show that these two modes of natural selecton correspond to dfferent orderngs of the processes of populaton regulaton and natural selecton n the Wrght-Fsher model. We fnd that under the Wrght-Fsher model these two dfferent orderngs can affect the dstrbuton of trajectores of haplotype frequences evolvng wth genetc recombnaton. However, the dfference n the dstrbuton of trajectores s only apprecable when the populaton s n sgnfcant lnkage dsequlbrum. We fnd that as lnkage dsequlbrum decays the trajectores for the two dfferent models rapdly become ndstngushable. We dscuss the sgnfcance of these fndngs n terms of bologcal examples of vablty and fecundty selecton, and speculate that the effect may be sgnfcant when factors such as gene mgraton mantan a degree of lnkage dsequlbrum. Keywords: Wrght-Fsher model, Vablty selecton, Fecundty selecton, Lnkage dsequlbrum 1 1. Introducton In populaton genetcs, one studes the genetc composton of bologcal populatons and the changes n genetc composton that result from the operaton of varous factors, ncludng natural selecton. Bürger (2000), Ewens (2004) and Durrett (2008) provded an excellent theoretcal ntroducton to ths feld. The most basc, but also most mportant, model n populaton Correspondng author at: School of Mathematcs, Unversty of Brstol, Brstol BS8 1TW, Unted Kngdom Emal address: zh0131@brstol.ac.uk (Zhangy He) Preprnt submtted to G3: Genes, Genomes, Genetcs May 12, 2017 The Author(s) Publshed by the Genetcs Socety of Amerca.

2 genetcs s the Wrght-Fsher model, developed by Fsher (1922) and Wrght (1931), whch forms the bass for most theoretcal and appled research n populaton genetcs to date, ncludng Kngman s coalescent (Kngman, 1982), Ewens samplng formula (Ewens, 1972), Kmura s work on fxaton probabltes (Kmura, 1955) and technques for nferrng demographc and genetc propertes of bologcal populatons (see Tataru et al., 2016, and references theren). Such wdespread applcaton s manly due to not only the strong unversalty results for the Wrght-Fsher model, e.g., Möhle s work on the Cannngs model (Möhle, 2001), but also the fact that the Wrght-Fsher model captures the essence of the bology nvolved and provdes an elegant mathematcal framework for characterzng the dynamcs of gene frequences, even n complex evolutonary scenaros. The smplest verson of the Wrght-Fsher model (Fsher, 1922; Wrght, 1931) s concerned wth a fnte random-matng populaton of fxed populaton sze evolvng n dscrete and nonoverlappng generatons at a sngle ballelc locus, whch can be regarded as a smplfed verson of the lfe cycle where the next generaton s randomly sampled wth replacement from an effectvely nfnte gene pool bult from equal contrbutons of all ndvduals n the current generaton. The Wrght-Fsher model can be generalzed to ncorporate other evolutonary forces such as natural selecton (see, for example, Etherdge, 2011). Natural selecton s the dfferental survval and reproducton of ndvduals due to dfferences 24 n phenotype, whch has long been a topc of nterest n populaton genetcs. Accordng to Chrstansen (1984), natural selecton can be classfed accordng to the stage of an organsm s lfe cycle at whch t acts: vablty selecton (or survval selecton) whch acts to mprove the rate of zygote survval and fecundty selecton (or reproducton selecton) whch acts to mprove the rate of gamete reproducton, as shown n Fgure 1. Nagylak (1997) provded dfferent dervatons of multnomal-samplng models for genetc drft at a sngle multallelc locus n a monoecous or doecous dplod populaton for dfferent orders of the evolutonary forces (.e., mutaton, natural selecton and genetc drft) n the lfe cycle. Prugnolle et al. (2005) found that gene mgraton occurrng before or after asexual reproducton n the lfe cycle of monoecous trematodes can have dfferent effects on a fnte sland model, dependng on values of the other genetc parameters. A natural queston that arses here s whether dfferent stages of an organsm s lfe cycle at whch natural selecton acts can 2

3 Adults (N < ) populaton regulaton meoss vablty selecton Gametes ( ) Zygotes ( ) matng (a) The lfe cycle stage at whch vablty selecton acts. Adults (N < ) fecundty selecton populaton regulaton meoss Gametes ( ) Zygotes ( ) matng (b) The lfe cycle stage at whch fecundty selecton acts. Fgure 1: Lfe cycles of a dplod populaton ncorporated wth dfferent types of natural selecton cause dfferent populaton behavors under the Wrght-Fsher model. It s an nevtable choce that we have to make n current statstcal nferental frameworks based on the Wrght-Fsher model for nferrng demographc and genetc propertes of bologcal populatons, especally for natural selecton, whch also affects the performance of these statstcal nference methods. 3

4 In the present work, we are concerned wth a fnte random-matng dplod populaton of fxed populaton sze N (.e., a populaton of 2N chromosomes) evolvng wth dscrete and nonoverlappng generatons under natural selecton wthn the framework of the Wrght-Fsher model, especally for the evoluton of one- and two-locus systems under natural selecton. We carry out dffuson analyss of Wrght-Fsher models wth selecton and use extensve Monte Carlo smulaton studes to address the queston of whether dfferent types of natural selecton can cause dfferent populaton behavors under the Wrght-Fsher model, especally when natural selecton takes the form of vablty or fecundty selecton. Our man fndng s that the dstrbuton of the trajectores of haplotype frequences for two recombnng loc depends on whether vablty or fecundty selecton s operatng. However, ths dfference s only apprecable when the haplotype frequences are n sgnfcant lnkage dsequlbrum, and arses as a consequence of the nterplay between genetc recombnaton and natural selecton. Once lnkage dsequlbrum dsappears, the dstrbutons of trajectores under the Wrght-Fsher model wth ether vablty or fecundty selecton become almost dentcal farly quckly Materals and Methods In ths secton, we provde detaled formulatons for a fnte random-matng dplod populaton of fxed populaton sze evolvng wth dscrete and nonoverlappng generatons under natural selecton (.e., vablty and fecundty selecton, respectvely) wthn the framework of the Wrght-Fsher model and ther dffuson approxmatons. We also ntroduce the Hellnger dstance to measure the dfference n the behavor of the Wrght-Fsher model between vablty and fecundty selecton Wrght-Fsher models wth selecton Consder a monoecous populaton of N randomly matng dplod ndvduals evolvng wth dscrete and nonoverlappng generatons under natural selecton. We assume that there are two alleles segregatng at each autosomal locus, and the populaton sze N s fxed. Let X (N) (k) be the frequency of haplotype n N adults of generaton k N and X (N) (k) denote the vector wth frequences of all possble haplotypes. We can study the populaton evolvng under natural selecton n terms of the changes n haplotype frequences from generaton to generaton. 4

5 To determne the transton of haplotype frequences from one generaton to the next, we need to nvestgate how the mechansms of evoluton (e.g., natural selecton) alter the genotype frequences at ntermedate stages of the lfe cycle. Let Y (N) j (k) be the frequency of the ordered genotype made up of haplotypes and j n N adults of generaton k and Y (N) (k) desgnate the vector wth frequences of all possble genotypes. Note that genotypes are regarded as ordered here only for smplcty of notaton. Under the assumpton of random matng, the genotype frequency s equal to the product of the correspondng haplotype frequences (Edwards, 2000), Y (N) j (k) = X (N) (k)x (N) j (k) = X (N) (k)x (N) j (k) = Y (N) j (k). (1) As llustrated n Fgure 1, natural selecton takes the form of vablty selecton and the lfe cycle moves through a loop of populaton regulaton, meoss, random matng, vablty selecton, populaton regulaton, and so forth (see Fgure 1a), or natural selecton takes the form of fecundty selecton and the lfe cycle moves through a loop of populaton regulaton, fecundty selecton, meoss, random matng, populaton regulaton, and so forth (see Fgure 1b). The entrance of the lfe cycle here s assumed to be wth the populaton reduced to N adults rght after a round of populaton regulaton. In the lfe cycles shown n Fgure 1, there are four potental mechansms of evolutonary change, natural selecton, meoss, random matng 83 and populaton regulaton. We assume that natural selecton, meoss and random matng occur n an effectvely nfnte populaton so can be treated determnstcally (Hamlton, 2011). Suppose that populaton regulaton (.e., genetc drft) acts n a smlar manner to the Wrght- Fsher samplng ntroduced by Fsher (1922) and Wrght (1931). In other words, populaton regulaton corresponds to randomly drawng N zygotes wth replacement from an effectvely nfnte populaton to become new adults n the next generaton, consequently completng the lfe cycle shown n Fgure 1. Thus, gven the genotype frequences Y (N) (k) = y, the genotype frequences n the next generaton satsfy Y (N) (k + 1) Y (N) (k) = y 1 N Multnomal (N, q), (2) where q s a functon of the genotype frequences y denotng the vector wth frequences of all possble genotypes of an effectvely nfnte populaton after the possble mechansms of 5

6 evolutonary change (except populaton regulaton) at ntermedate stages of the lfe cycle such as natural selecton, meoss and random matng wthn generaton k. The explct expresson of the samplng probabltes q wll be gven n the followng two sectons for the evoluton of one- and two-locus systems under natural selecton, respectvely. To smplfy notaton, we ntroduce a functon ρ uv of three varables u, v and, defned as ρ uv = 1 2 (δ u + δ v ), (3) where δ u and δ v denote the Kronecker delta functons. We can then express the frequency of haplotype n terms of the correspondng genotype frequences as X (N) (k) = ρ uvy uv (N) (k), (4) u,v and the transton probabltes of the haplotype frequences from one generaton to the next can be easly obtaned from Eqs. (2)-(4). We refer to the process X (N) = {X (N) (k) : k N} as the Wrght-Fsher model wth selecton, whose frst two condtonal moments satsfy ( ) E (k,x) X (N) (k + 1) = p (5) ( ) E (k,x) X (N) (k + 1)X (N) j (k + 1) = p (δ j p j ) + p p j, 2N (6) where ( ) E (k,x) ( ) = E X (N) (k) = x s the short-hand notaton for the condtonal expectaton of a random varable gven the populaton of the haplotype frequences x n generaton k and p = u,v ρ uvq uv s the frequency of haplotype of an effectvely nfnte populaton after the possble mechansms of evolutonary change (except populaton regulaton) at ntermedate stages of the lfe cycle such as natural selecton, meoss and random matng wthn generaton k, whch obvously can be expressed n terms of the haplotype frequences x. 6

7 One-locus Wrght-Fsher models wth selecton Let us consder a monoecous populaton of N randomly matng dplod ndvduals at a sngle autosomal locus A, segregatng nto two alleles, A 1 and A 2, evolvng under natural selecton n dscrete and nonoverlappng generatons. We call the two possble haplotypes A 1 and A 2 haplotypes 1 and 2, respectvely. As we have stated above, we need to formulate the samplng probabltes q for the evoluton of one-locus systems under natural selecton, whch s the vector of frequences of all possble genotypes of an effectvely nfnte populaton after a sngle generaton of natural selecton, meoss (wthout genetc recombnaton) and random matng due to the absence of genetc recombnaton at meoss n the evoluton of one-locus systems. Wth a sngle ballelc locus, there are four possble zygotes (.e., four ordered genotypes) that result from the random unon of two gametes. We denote the ftness of the genotype formed by haplotypes and j by w j for, j = 1, 2. In the lfe cycles llustrated n Fgure 1, the number of adults s regulated to be of sze N and the gene frequences n the subsequent generaton can be descrbed by multnomal samplng of the normalzed frequences n an effectvely nfnte pool of zygotes. When natural selecton takes dfferent forms, genotype frequences n the zygotes can be modeled n dfferent ways. In the case of what can be called vablty selecton shown n Fgure 1a, the N adults have an equal chance of formng gametes, whch unte at random to form zygotes. Ths s followed by vablty selecton on genotypes of zygotes, leadng to modfed genotype frequences. The subsequent genotype frequences are obtaned by multnomal samplng. We can express the frequency of the genotype formed by haplotypes and j of an effectvely nfnte populaton after a sngle generaton of meoss, random matng and vablty selecton as q (v) j = w j w 2 ρ uvy uv 2 ρ j uvy uv, (7) 127 where 2 w = 2 w j ρ uvy uv,j=1 2 ρ j uvy uv. (8) Alternatvely, n what can be termed fecundty selecton llustrated n Fgure 1b, the N adults have an unequal chance of formng gametes, dependng on ther genotypes; the gametes 7

8 130 unte at random and the subsequent genotype frequences reman unchanged untl multnomal 131 samplng. We can express the frequency of the genotype formed by haplotypes and j of 132 an effectvely nfnte populaton after a sngle generaton of fecundty selecton, meoss and random matng as where q (f) j = 2 ρ w uv uv w y uv w = 2 2 ρ j w uv uv w y uv, (9) w uv y uv. (10) From Eqs. (7)-(8) and (9)-(10), we see that the transton probabltes of the genotype frequences from one generaton to the next depend only on the genotype frequences n the current generaton for both vablty and fecundty selecton, but take on dfferent forms. Combnng 135 wth Eqs. (2)-(4), we fnd that the process X (N) s a tme-homogeneous Markov process wth respect to the fltraton F (N) = {F (N) k state space Ω X (N) = { x : k N} generated by the process Y (N) evolvng n the { } 1 2 0, 2N,..., 1 : whch we call the one-locus Wrght-Fsher model wth selecton. } 2 x = 1, = Two-locus Wrght-Fsher models wth selecton Now we turn to the study of a monoecous populaton of N randomly matng dplod ndvduals at two lnked autosomal loc named A and B, each segregatng nto two alleles, A 1, A 2 and B 1, B 2, evolvng under natural selecton wth dscrete and nonoverlappng generatons. We call the four possble haplotypes A 1 B 1, A 1 B 2, A 2 B 1 and A 2 B 2 haplotypes 1, 2, 3 and 4, respectvely. As we have stated above, we need to formulate the samplng probabltes q for the evoluton of two-locus systems under natural selecton, whch s the vector wth frequences of all possble genotypes of an effectvely nfnte populaton after a sngle generaton of natural selecton, meoss (wth genetc recombnaton) and random matng due to the presence of genetc recombnaton at meoss n the evoluton of two-locus systems. Wth two ballelc loc, there are 16 possble zygotes (.e., 16 ordered genotypes) that result 147 from the random unon of four gametes. We denote the ftness of the genotype formed by 148 haplotypes and j by w j for, j = 1, 2, 3, 4, and desgnate the recombnaton rate between the 8

9 two loc by r (.e., the rate at whch a recombnant gamete s produced at meoss). To smplfy notaton, we ntroduce a vector of auxlary varables η = (η 1, η 2, η 3, η 4 ), where η 1 = η 4 = 1 and η 2 = η 3 = 1. In the lfe cycles shown n Fgure 1, the number of adults s regulated to be of sze N and the gene frequences n the subsequent generaton can be descrbed by multnomal samplng of the normalzed frequences n an effectvely nfnte pool of zygotes. When natural selecton takes dfferent forms, genotype frequences n the zygotes can be modeled n dfferent ways. Followng smlar reasonng as n the one-locus case, we can express the frequency of the genotype formed by haplotypes and j of an effectvely nfnte populaton after a sngle generaton of meoss, random matng and vablty selecton as q (v) j = w j w ρ uvy uv + η rd ρ j uvy uv + η j rd, (11) where w =,j=1 D = w j ρ uvy uv + η rd ρ 1 uvy uv ρ 4 uvy uv ρ j uvy uv + η j rd (12) ρ 2 uvy uv ρ 3 uvy uv. (13) Smlarly, the frequency of the genotype formed by haplotypes and j of an effectvely nfnte populaton after a sngle generaton of fecundty selecton, meoss and random matng 161 s where q (f) j = ρ w uv uv w y uv + η rd ρ j w uv uv w y uv + η j rd, (14) w = D = w uv y uv ρ 1 w uv uv w y uv ρ 4 w uv uv w y uv ρ 2 w uv uv w y uv ρ 3 w uv uv w y uv. (15) (16) 9

10 From Eqs. (11)-(13) and (14)-(16), we see that the transton probabltes of the genotype frequences from one generaton to the next depend only on the genotype frequences n the 162 current generaton for both vablty and fecundty selecton, but take on dfferent forms. Combnng wth Eqs. (2)-(4), we show that the process X (N) s a tme-homogeneous Markov process wth respect to the fltraton F (N) = {F (N) k the state space Ω X (N) = { x : k N} generated by the process Y (N) evolvng n { } 1 4 0, 2N,..., 1 : whch we call the two-locus Wrght-Fsher model wth selecton. } x = 1, = Dffuson approxmatons Due to the nterplay of stochastc and determnstc forces, the Wrght-Fsher model wth selecton presents analytcal challenges beyond the standard Wrght-Fsher model for neutral populatons. The analyss of the Wrght-Fsher model wth selecton today s greatly facltated by ts dffuson approxmaton, commonly known as the Wrght-Fsher dffuson wth selecton, whch can be traced back to Kmura (1964) and has already been successfully appled n the statstcal nference of natural selecton (see Malaspnas, 2016, and references theren). Here we only present the formulaton of the Wrght-Fsher dffuson wth selecton and refer to Durrett (2008) for a rgorous proof, especally for one- and two-locus systems. The Wrght-Fsher dffuson wth selecton s a lmtng process of the Wrght-Fsher model wth selecton characterzng the changes n haplotype frequences over tme n an extremely large populaton evolvng under extremely weak natural selecton. More specfcally, the se- lecton coeffcents (and the recombnaton rate f there s more than one locus) are assumed to be of order 1/(2N) and the process runs tme at rate 2N,.e., t = k/(2n). The selecton coeffcent mentoned here represents the dfference n ftness between a gven genotype and the genotype wth the hghest ftness. For example, a common category of ftness values for a dplod populaton at a sngle locus can be presented as follows: genotypes A 1 A 1, A 1 A 2 and A 2 A 2 at a gven locus A have ftness values 1, 1 h A s A and 1 s A, respectvely, where s A s the selecton coeffcent and h A s the domnance parameter (see Hamlton, 2011, for other categores of ftness values presented n terms of selecton coeffcents). Let X (N) (k) denote the change n the frequency of haplotype from generaton k to the 10

11 next. Usng Eqs. (5) and (6), we can obtan ts frst two condtonal moments E (k,x) ( X (N) E (k,x) ( X (N) (k) X (N) j (k) ) (k) = p x ) = p (δ j p j ) 2N + (p x )(p j x j ). Consderng the lmts as the populaton sze N goes to nfnty, we can formulate the nfntesmal mean vector µ(t, x) as ( ) µ (t, x) = lm N 2NE([2Nt],x) X (N) ([2Nt]) = lm N 2N(p x ) (17) and the nfntesmal covarance matrx Σ(t, x) as ( ) Σ j (t, x) = lm N 2NE([2Nt],x) X (N) ([2Nt]) X (N) j ([2Nt]) = lm N p (δ j p j ) + 2N(p x )(p j x j ), (18) where [ ] s used to denote the nteger part of the value n the brackets, accordng to standard technques of dffuson theory (see, for example, Karln & Taylor, 1981). The process X (N) thereby converges to a dffuson process, denoted by X = {X(t), t 0}, satsfyng the stochastc dfferental equaton of the form dx(t) = µ (t, X(t)) dt + σ (t, X(t)) dw (t), where the dffuson coeffcent matrx σ(x) satsfes the relaton that σ(t, x)σ T (t, x) = Σ(t, x) and W (t) s a mult-dmensonal standard Brownan moton. We refer to the process X as the Wrght-Fsher dffuson wth selecton, whch we wll use to nvestgate the dfference n the behavor of the Wrght-Fsher model between vablty and fecundty selecton n Secton

12 Statstcal dstances Gven that the Wrght-Fsher model wth selecton s a Markov process, the evoluton s completely determned by ts transton probabltes. We can, therefore, study the dfference n the behavor of the Wrght-Fsher model between vablty and fecundty selecton n terms of ther transton probabltes. We defne the condtonal probablty dstrbuton functon of the Wrght-Fsher model wth selecton X (N) evolvng from the populaton of the ntal haplotype frequences x 0 over k generatons (.e., k-step transton probabltes) as π(x 0, x k ) = P (X (N) (k) = x k X (N) (0) = x 0 ). (19) We use a statstcal dstance to quantfy the dfference between two probablty dstrbutons for the Wrght-Fsher model wth vablty and fecundty selecton (.e., the dfference n the behavor of the Wrght-Fsher model between vablty and fecundty selecton). Rachev et al. (2013) provded an excellent ntroducton of statstcal dstances, and here we employ the Hellnger dstance, ntroduced by Hellnger (1909), to quantfy the dfference n the behavor of the Wrght- Fsher model between vablty and fecundty selecton, defned as H(π (v), π (f) )(x 0, k) = 1 2 x k ( π (v) (x 0, x k ) π (f) (x 0, x k ) ) 2, (20) where π (v) s the probablty dstrbuton for the Wrght-Fsher model wth vablty selecton and π (f) s the probablty dstrbuton for the Wrght-Fsher model wth fecundty selecton, both of whch are gven by Eq. (19) combned wth the Wrght-Fsher model wth the correspondng type of natural selecton. Gven the dffcultes n analytcally formulatng the probablty dstrbuton π, especally for the populaton evolvng over a long-tme perod, we resort to Monte Carlo smulaton here, whch enables us to get an emprcal probablty dstrbuton functon assocated to the probablty dstrbuton π, defned as ˆπ(x 0, x k ) = 1 M M m=1 I {ξ (m) k =x k }, (21) where I s the ndcator functon, namely I {ξk =x k } s one f ξ k = x k s true and zero otherwse, 12

13 ξ (m) k s the m-th realzaton of the haplotype frequences smulated under the Wrght-Fsher model wth selecton from the populaton of the ntal haplotype frequences x 0 over k generatons, and M s the total number of ndependent realzatons n Monte Carlo smulaton. Combng wth Eqs. (20) and (21), we can formulate the Monte Carlo approxmaton for the Hellnger dstance H(π (v), π (f) ) as Ĥ(π (v), π (f) )(x 0, k) = H(ˆπ (v), ˆπ (f) )(x 0, k) Accordng to Van der Vaart (2000), the rate of convergence for the emprcal probablty dstrbuton ˆπ to the probablty dstrbuton π wth respect to the Hellnger dstance s of order C 1 ( Ω X 1) 1/2 /M 1/2, where C 1 s a constant and Ω X s total number of possble states n the state space Ω X. Combnng wth Le Cam & Yang (2000), we fnd that the rate of convergence for the Hellnger dstance approxmated by Monte Carlo smulaton, Ĥ(π (v), π (f) ), to the Hellnger dstance H(π (v), π (f) ) s of order C 2 ( Ω X 1) 1/2 /M 1/2, where C 2 s a constant. Therefore, n theory, the Monte Carlo approxmaton for the Hellnger dstance Ĥ(π(v), π (f) ) can be used nstead of the Hellnger dstance H(π (v), π (f) ) as long as we ncrease the total number of ndependent realzatons M, whch we wll apply to measure the dfference n the behavor of the Wrght-Fsher model between vablty and fecundty selecton n Secton Data avalablty The authors state that all data necessary for confrmng the conclusons presented n the artcle are represented fully wthn the artcle. Code used to smulate the Wrght-Fsher model wth selecton, ncludng both vablty and fecundty selecton, and compute the results s provded n Supplemental Materal, Fle S Results In ths secton, we use dffuson analyss of Wrght-Fsher models wth selecton and extensve Monte Carlo smulaton studes to nvestgate whether dfferent types of natural selecton can cause dfferent populaton behavors under the Wrght-Fsher model, especally when natural selecton takes the form of vablty or fecundty selecton. 13

14 We employ the category of ftness values presented n terms of selecton coeffcents gven n Secton 2.2 and consder the smple case of drectonal selecton wth 0 s A 1, whch mples that the A 1 allele s the type favored by natural selecton. The domnance parameter s assumed to be n the range 0 h A 1,.e., general domnance. Suppose that ftness values of two-locus genotypes are determned multplcatvely from ftness values at ndvdual loc, e.g., the ftness value of genotype A 1 B 2 /A 2 B 2 s (1 h A s A )(1 s B ), whch means that there s no poston effect,.e., couplng and repulson double heterozygotes have the same ftness, w 14 = w 23 = (1 h A s A )(1 h B s B ). Moreover, the recombnaton rate defned n Secton s n the range 0 r Dffuson analyss of the Wrght-Fsher model Now we use dffuson analyss of Wrght-Fsher models wth selecton to address the queston of whether vablty and fecundty selecton can cause dfferent populaton behavor under the Wrght-Fsher model, especally for the evoluton of one- and two-locus systems under natural 243 selecton. From Secton 2.2, we see that the dffuson approxmaton for the Wrght-Fsher model wth selecton s fully determned from ts nfntesmal mean vector µ(t, x) n Eq. (17) and the nfntesmal covarance matrx Σ(t, x) n Eq. (18), whch mples that we can carry out dffuson analyss of Wrght-Fsher models wth selecton to nvestgate the dfference n the behavor of the Wrght-Fsher model between vablty and fecundty selecton by comparng the dfference n the haplotype frequences of an effectvely nfnte populaton after a sngle generaton of natural selecton, meoss and random matng mentoned n Eqs. (17) and (18) between vablty and fecundty selecton. Under the set of assumptons on ftness values gven above and usng Taylor expansons wth respect to the selecton coeffcent s A, we fnd that for one-locus systems there s no dfference n the haplotype frequences of an effectvely nfnte populaton after a sngle generaton of natural selecton, meoss and random matng between vablty and fecundty selecton,.e., p (v) p (f) = 0, for = 1, 2. For two-locus systems wth the selecton coeffcents s A and s B and the recombna- 14

15 ton rate r, Supplemental Materal, Fle S2, shows that p (v) p (f) = O ( ) 1 N 2, for = 1, 2, 3, 4. Combnng wth Eqs. (17) and (18), we have µ (v) (t, x) = µ (f) (t, x) Σ (v) (t, x) = Σ (f) (t, x), whch leads to the same stochastc dfferental equaton representaton of the Wrght-Fsher dffuson wth vablty and fecundty selecton for the evoluton of one- and two-locus systems under natural selecton, respectvely (see Supplemental Materal, Fle S2, for detaled calculatons). Therefore, we can conclude that vablty and fecundty selecton brng about the same populaton behavor under the Wrght-Fsher dffuson for the evoluton of one- and two-locus systems under natural selecton, whch mples that there s almost no dfference n populaton behavors under the Wrght-Fsher model between vablty and fecundty selecton for the evolutonary scenaro of an extremely large populaton evolvng under extremely weak natural selecton (and genetc recombnaton f there s more than one locus) Smulaton analyss of the Wrght-Fsher model Dffuson analyss of Wrght-Fsher models wth selecton requre assumptons on genetc parameters for tractablty,.e., t s only guaranteed to be a good approxmaton of the underlyng Wrght-Fsher model n the case of an extremely large populaton evolvng under extremely weak natural selecton (and genetc recombnaton f there s more than one locus). In ths secton, we use extensve Monte Carlo smulaton studes to nvestgate the dfference n the behavor of the Wrght-Fsher model between vablty and fecundty selecton for other evolutonary scenaros such as a small populaton evolvng under strong natural selecton. We llustrate how dfferent types of natural selecton affect the behavor of the Wrght-Fsher model wth haplotype 1 n detal n the followng and expect other haplotypes to behave n a smlar manner. Let us desgnate the margnal probablty dstrbuton of the frequency of haplotype 1 by π 1 and smulate the dynamcs of the Hellnger dstance H(π (v) 1, π(f) 1 ) over tme for dfferent 15

16 evolutonary scenaros to nvestgate whether vablty and fecundty selecton can cause dfferent populaton behavors under the Wrght-Fsher model usng Eqs. (20) and (21) n Eq. (19). M=10000 M= selecton coeffcent generaton generaton (a) We generate M ndependent realzatons from smulatng the one-locus Wrght-Fsher model wth selecton, where we adopt N = 500, h A = 0.5 and x 0 = (0.3, 0.7). M=10000 M= selecton coeffcent generaton generaton (b) We generate M ndependent realzatons from smulatng the two-locus Wrght-Fsher model wth selecton, where we adopt N = 500, s B = 5, h A = 0.5, h B = 0.5, r = 5 and x 0 = (0.3,,, 0.1). Fgure 2: Dynamcs of the Hellnger dstance H(π (v) 1, π(f) 1 ) smulated wth the varyng selecton coeffcent sa over 500 generatons under the one- and two-locus Wrght-Fsher models wth selecton In Fgure 2, we show the dynamcs of the Hellnger dstance H(π (v) 1, π(f) 1 ) wth the varyng selecton coeffcent s A over 500 generatons under the one- and two-locus Wrght-Fsher models wth selecton, respectvely, n whch the Hellnger dstance H(π (v) 1, π(f) 1 ) s clearly not always close to zero. Ths may result from the dfferent forms natural selecton takes or the statstcal nose produced n Monte Carlo smulaton. Comparng the left columns of Fgures 2a and 2b 16

17 wth ther rght columns, we fnd that the Hellnger dstance H(π (v) 1, π(f) 1 ) decreases as the total number of ndependent realzatons M n Monte Carlo smulaton ncreases, especally when the selecton coeffcent s A s close to zero. We, therefore, beleve that the dscrepancy should be caused manly by the statstcal nose produced n Monte Carlo smulaton rather than the dfferent forms natural selecton takes when the selecton coeffcent s close to zero, whch otherwse leads to a contradcton to what we have already acheved n Secton 3.1. On the contrary, when the selecton coeffcent s not close to zero, the dscrepancy should be ndeed caused by the dfferent forms natural selecton takes rather than the statstcal nose n Monte Carlo smulaton. We wll confrm and dscuss ths pont n more detal n Secton 4.1. Fgure 2 shows that the dfference n the behavor of the Wrght-Fsher model between vablty and fecundty selecton does exst, whch becomes more sgnfcant when the effect of natural selecton on the populaton evolvng over tme ncreases. For the populaton evolvng under natural selecton at a sngle locus, the dfference n the behavor of the Wrght-Fsher model between vablty and fecundty selecton s almost neglgble. However, for the populaton evolvng under natural selecton at two lnked loc, the dfference n the behavor of the Wrght- Fsher model between vablty and fecundty selecton s no longer neglgble, especally when 296 natural selecton s not extremely weak. Therefore, we assert that vablty and fecundty selecton can cause dfferent populaton behavors under the Wrght-Fsher model, especally for the populaton evolvng under natural selecton at two lnked loc. For further nvestgaton nto the dfference n the populaton evolvng over tme at two lnked loc under the Wrght-Fsher model between vablty and fecundty selecton, we ntroduce the coeffcent of lnkage dsequlbrum, proposed by Lewontn & Kojma (1960), to quantfy the level of lnkage dsequlbrum between the two loc, whch s defned as D (N) (k) = X (N) 1 (k)x (N) 4 (k) X (N) 2 (k)x (N) 3 (k) We smulate the dynamcs of the Hellnger dstance H(π (v) 1, π(f) 1 ) wth the varyng selecton coeffcent s A and recombnaton rate r over 500 generatons under the two-locus Wrght-Fsher model wth selecton for the populaton ntally at dfferent levels of lnkage dsequlbrum n Fgure 3. Comparng the mddle column of Fgure 3 wth ts left and rght columns, we fnd that the 17

18 generaton r=5 and D=5 200 selecton coeffcent 100 r=5 and D=5 r=5 and D= r=5 and D= r=5 and D=0 selecton coeffcent r=5 and D= generaton (v) r=5 and D=5 r=5 and D=0 selecton coeffcent r=5 and D= generaton (f ) Fgure 3: Dynamcs of the Hellnger dstance H(π1, π1 ) smulated wth the varyng selecton coeffcent sa and recombnaton rate r over 500 generatons under the two-locus Wrght-Fsher model wth selecton for the populaton ntally at dfferent levels of lnkage dsequlbrum. We generate M = 105 ndependent realzatons n Monte Carlo smulaton and adopt N = 500, sb = 5, ha = 0.5 and hb = dfference n the behavor of the two-locus Wrght-Fsher model between vablty and fecundty 305 selecton becomes neglgble when the populaton s ntally n lnkage equlbrum (see the 306 mddle column of Fgure 3), whch mples that whether dfferent types of natural selecton can 307 cause dfferent populaton behavors under the Wrght-Fsher model at two lnked loc depends 308 sgnfcantly on the level of lnkage dsequlbrum. We thereby consder the dynamcs of lnkage 309 dsequlbrum over tme under the two-locus Wrght-Fsher model wth vablty and fecundty 18

19 310 selecton, respectvely, for the populaton ntally at dfferent levels of lnkage dsequlbrum. k=0 and D= 5 k=3 and D= 5 k=6 and D= 5 k=9 and D= 5 k=12 and D= 5 k=15 and D= 5 probablty dstrbuton vablty fecundty probablty dstrbuton k=0 and D= k=3 and D= k=6 and D= k=9 and D= k=12 and D= k=15 and D= probablty dstrbuton k=0 and D= lnkage dsequlbrum k=3 and D= lnkage dsequlbrum k=6 and D= lnkage dsequlbrum k=9 and D= lnkage dsequlbrum k=12 and D= lnkage dsequlbrum k=15 and D= lnkage dsequlbrum Fgure 4: Dynamcs of the probablty dstrbutons π (v) D and π(f) D smulated over 15 generatons under the twolocus Wrght-Fsher model wth selecton for the populaton ntally at dfferent levels of lnkage dsequlbrum. We generate M = 10 5 ndependent realzatons n Monte Carlo smulaton and adopt N = 500, s A = 0.95, s B = 5, h A = 0.5, h B = 0.5 and r = 5. We defne the condtonal probablty dstrbuton functon of lnkage dsequlbrum D (N) evolvng under the Wrght-Fsher model from the populaton of the ntal haplotype frequences x 0 over k generatons as π D (D 0, D k ) = P (D (N) (k) = D k D (N) (0) = D 0 ), where D 0 = x 0,1 x 0,4 x 0,2 x 0, We smulate the dynamcs of the probablty dstrbuton π D over tme for the populaton ntally at dfferent levels of lnkage dsequlbrum n Fgure 4. We observe that the probablty dstrbuton π D becomes concentrated on a narrower and narrower range of possble values centered around 0 from generaton to generaton untl fxng at 0. The dynamcs of the probablty dstrbuton π D over tme seems not to be assocated wth whether vablty or fecundty selecton s occurrng. From Rdley (2004), there are three potental mechansms of evolutonary change, genetc recombnaton, natural selecton and populaton regulaton (genetc drft), n the lfe cycle, 19

20 319 whch may affect the dynamcs of lnkage dsequlbrum over tme. Genetc recombnaton always works towards lnkage equlbrum due to genetc recombnaton generatng new gamete types to break down non-random genetc assocatons (Rdley, 2004). Natural selecton alone can not move the populaton far away from lnkage equlbrum under the assumpton that ftness values of two-locus genotypes are multplcatve (Slatkn, 2008). Genetc drft can destroy lnkage equlbrum and create many genetc assocatons snce genetc drft leads to the random change n haplotype frequences, whch, however, can cause persstent lnkage dsequlbrum only n small enough populatons (Rdley, 2004). So once lnkage equlbrum has been reached, populatons usually wll not move far away from lnkage equlbrum. From Eqs. (11)-(13) and (14)-(16), provded that the populaton evolvng over tme s always close to lnkage equlbrum, the amount of the change n haplotype frequences caused by genetc recombnaton s neglgble. The two-locus Wrght-Fsher model where each locus segregates nto two alleles thereby becomes smlar to the one-locus Wrght-Fsher model for a sngle locus segregatng four alleles, where each gamete s analogous to a sngle allele. Therefore, once lnkage equlbrum has been reached, populatons evolvng under natural selecton at two lnked loc wthn the framework of the Wrght-Fsher model would no longer depend on whether vablty or fecundty selecton s occurrng. That s, vablty and fecundty selecton can cause dfferent populaton behavors under the Wrght-Fsher model only when the populaton s far away from lnkage equlbrum, whch s confrmed by Fgure 3. Now we dscuss why vablty and fecundty selecton lead to dfferent populaton behavors under the two-locus Wrght-Fsher model wth selecton. We smulate the dynamcs of the probablty dstrbutons π (v) 1 and π (f) 1 over tme for the populaton ntally at dfferent levels of lnkage dsequlbrum n Fgure 5, from whch t s clear that the Wrght-Fsher models wth dfferent types of natural selecton have dfferent rates of change n the frequency of haplotype 1 approachng fxaton when the populaton s ntally far away from lnkage equlbrum, whch leads to the dfference n the behavor of the Wrght-Fsher model between vablty and fecundty selecton, as llustrated n the left and rght columns of Fgure 3. More specfcally, as shown n Fgure 5a, when the populaton s n negatve lnkage dse- qulbrum, the Wrght-Fsher model wth selecton of two lnked loc drves haplotype 1 more rapdly toward fxaton than the Wrght-Fsher model wth selecton of two completely lnked 20

21 probablty dstrbuton k=0 vablty wth r=5 vablty wth r= k= k= k= k= k=15 k=0 k=3 k=6 k=9 k=12 k=15 probablty dstrbuton vablty wth r=5 fecundty wth r= probablty dstrbuton probablty dstrbuton k=0 k=3 k=6 k=9 k=12 k=15 fecundty wth r=5 fecundty wth r=0 haplotype frequency haplotype frequency haplotype frequency haplotype frequency haplotype frequency (a) The populaton s ntally n negatve lnkage dsequlbrum (D 0 = 5). k=0 k=3 k=6 k=9 vablty wth r=5 vablty wth r= haplotype frequency k=12 k= probablty dstrbuton k=0 vablty wth r=5 fecundty wth r= k= k= k= k= k=15 probablty dstrbuton k=0 fecundty wth r=5 fecundty wth r=0 haplotype frequency k=3 haplotype frequency k=6 haplotype frequency k=9 haplotype frequency k=12 haplotype frequency (b) The populaton s ntally n postve lnkage dsequlbrum (D 0 = 5) k=15 haplotype frequency Fgure 5: Dynamcs of the probablty dstrbutons π (v) 1 and π (f) 1 smulated over 15 generatons under the twolocus Wrght-Fsher model wth selecton for the populaton ntally at dfferent levels of lnkage dsequlbrum. We generate M = 10 5 ndependent realzatons n Monte Carlo smulaton and adopt N = 500, s A = 0.95, s B = 5, h A = 0.5, h B = 0.5 and r = loc (.e., r = 0). Ths s due to the fact that genetc recombnaton renforces the change n haplotype frequences caused by natural selecton for negatve lnkage dsequlbrum (see Hamlton, 2011). The mddle row of Fgure 5a shows that the Wrght-Fsher model wth vablty selecton drves haplotype 1 more rapdly toward fxaton than the Wrght-Fsher model wth fecundty selecton when the populaton s n negatve lnkage dsequlbrum, whch m- 21

22 ples that genetc recombnaton affectng natural selecton at two lnked loc s more sgnfcant n the Wrght-Fsher model wth vablty selecton than that n the Wrght-Fsher model wth fecundty selecton for negatve lnkage dsequlbrum. These results are also confrmed wth Fgure 5b for the populaton n postve lnkage dsequlbrum. Therefore, the dfference n the behavor of the two-locus Wrght-Fsher model between vablty and fecundty selecton results from the dfferent effect of genetc recombnaton on dfferent types of natural selecton wthn the framework of the Wrght-Fsher model,.e., genetc recombnaton has a more sgnfcant effect on natural selecton at two lnked loc when natural selecton takes the form of vablty selecton than fecundty selecton. Notce that here we employ the Hellnger dstance to measure the dfference n the behavor of the Wrght-Fsher model between vablty and fecundty selecton, and only smulate the Hellnger dstance between two probablty dstrbutons for the Wrght-Fsher model wth vablty and fecundty selecton n lmted evolutonary scenaros such as completely addtve gene acton (h = 0.5). In Supplemental Materal, we smulate the Hellnger dstance between two probablty dstrbutons for the Wrght-Fsher model wth vablty and fecundty selecton for completely domnant gene acton (h = 0) and completely recessve gene acton (h = 1), receptvely (see Supplemental Materal, Fle S3). We also provde the dynamcs of the dfference n the behavor of the Wrght-Fsher model between vablty and fecundty selecton for dfferent populaton szes (see Supplemental Materal, Fle S4). Moreover, we smulate the total varaton dstance between two probablty dstrbutons for the Wrght-Fsher model wth vablty and fecundty selecton for dfferent evolutonary scenaros (see Supplemental Materal, Fle S5). All these smulated results gven n Supplemental Materal confrm what we have acheved above Dscusson In ths secton, we dscuss the robustness of Monte Carlo smulaton studes carred out n Secton 3. Moreover, we summarze our man results and dscuss the sgnfcance of these fndngs n terms of bologcal examples of fecundty and vablty selecton Robustness of Monte Carlo smulaton studes We perform a robustness analyss of Monte Carlo smulaton studes carred out n Secton 3. Ths s manly desgned to demonstrate that the dscrepancy n Monte Carlo smulaton studes 22

23 383 results from whether vablty selecton or fecundty selecton s occurrng s not due to the 384 statstcal nose produced n Monte Carlo smulaton and s ndeed caused by the dfferent forms 385 natural selecton takes. N=500 and M= selecton coeffcent N=500 and M= N=5000 and M= selecton coeffcent N=5000 and M= generaton generaton 500 (a) Natural selecton takes the form of vablty selecton. N=500 and M= selecton coeffcent N=500 and M= N=5000 and M= selecton coeffcent N=5000 and M= generaton generaton 500 (b) Natural selecton takes the form of fecundty selecton. ( ) ( ) Fgure 6: Dynamcs of the Hellnger dstance H(π 1, π 1 ) smulated wth the varyng selecton coeffcent sa over 500 generatons under the two-locus Wrght-Fsher model wth selecton for natural selecton takng dfferent forms. We adopt sb = 5, ha = 0.5, hb = 0.5, r = 5 and x0 = (0.3,,, 0.1). 23

24 We smulate the dynamcs of the Hellnger dstance of the two emprcal probablty dstrbutons for the two runs of the two-locus Wrght-Fsher model wth the same type of natural selecton here as an llustraton, the two-locus Wrght-Fsher model wth vablty selecton n Fgure 6a and the two-locus Wrght-Fsher model wth fecundty selecton n Fgure 6b, respectvely. Fgure 6 shows that the statstcal nose n Monte Carlo smulaton s ncreasng n the populaton sze N but s deceasng n the total number of ndependent realzatons M. Ths s confrmed by the rate of convergence for the Hellnger dstance approxmated by Monte Carlo smulaton we have stated n Secton 2.3. For the populaton of N = 500 ndvduals n Monte Carlo smulaton studes, generatng M = 10 5 ndependent realzatons n Monte Carlo smulaton s large enough to remove most of the statstcal nose. In comparson wth Fgures 6a and 6b, the dscrepancy n Fgure 2b shares the same pattern except for large selecton coeffcents, whch mples that the dfference n the behavor of the Wrght-Fsher model between vablty and fecundty selecton ndeed results from the dfferent forms natural selecton rather than the statstcal nose produced n Monte Carlo smulaton when natural selecton s not extremely weak Summary and further perspectves Our dffuson analyss of Wrght-Fsher models wth selecton and extensve Monte Carlo smulaton studes show that wth genetc recombnaton, the populaton evolvng under natural selecton wthn the framework of the Wrght-Fsher model depends sgnfcantly on whether vablty or fecundty selecton s occurrng when the populaton s far away from lnkage equlbrum. Ths s manly caused by dfferent effects of genetc recombnaton on dfferent types of natural selecton,.e., genetc recombnaton has a more sgnfcant effect on natural selecton at lnked loc when natural selecton takes the form of vablty selecton than fecundty selecton. The dfference n the behavor of the Wrght-Fsher model becomes more sgnfcant as the effect of natural selecton and genetc recombnaton ncreases. However, after the populaton reaches lnkage equlbrum, populaton behavors under the Wrght-Fsher model wth vablty and fecundty selecton become almost dentcal farly quckly. We have shown that wth genetc recombnaton, the populaton evolvng under natural selecton wthn the framework of the Wrght-Fsher model depends sgnfcantly on whether vablty or fecundty selecton s occurrng when the populaton s far away from lnkage equlbrum. Ac- 24

25 cordng to Rdley (2004), most natural populatons are probably near lnkage equlbrum, whch mples that the dfferent behavor of the Wrght-Fsher model caused by vablty or fecundty selecton may not be sgnfcant. However, one evolutonary scenaro n whch t may have sgnfcant consequences s that of admxture between two populatons. In ths case, there s lkely to be sgnfcant lnkage dsequlbrum due to the allele frequency dfferences between populatons (Prtchard & Rosenberg, 1999), and also, there s lkely to be local natural selecton n whch alleles that are favored n one populaton are selected aganst n the other (Charlesworth et al., 1997), and vce-versa. Therefore, the results presented here may lead to testable predctons about the role of lfe-hstory on the natural selecton dynamcs n populatons. For example, we predct that genetc recombnaton wll have a less sgnfcant effect on the natural selecton dynamcs when natural selecton acts through fecundty dfference between genotypes. It should be noted that n our formulaton the fecundty selecton s assocated wth a dfference among monoecous dplod genotypes n ther overall gamete producton, and should not be confused wth gametc selecton, for example natural selecton on sperm bndng alleles (Palumb, 1999). Moreover, we do not dstngush between sperm and egg producton (.e., the two haplotype frequences X and X j n Eq. (1) are exchangeable). The most relevant organsms that may be expected to conform to ths assumpton may be monoecous plant speces, where correlatons between pollen and seed producton may be nduced through genetc varaton n flower sze (e.g., Galen, 2000). Gven that we have developed a Monte Carlo framework for smulatng the Wrght-Fsher model under ether scenaro (.e., vablty or fecundty selecton), future developments could nclude the addton of populaton structure, and t should then be straghtforward to develop a statstcal nferental framework to allow us to dstngush between vablty selecton or fecundty selecton at canddate loc such as the approxmate Bayesan computaton (ABC) framework (Lepe et al., 2014), whch enables us to compute the posteror probabltes of the Wrght-Fsher models wth vablty and fecundty selecton. Once the posteror probabltes of canddate models have been estmated, we can make full use of the technques of Bayesan 443 model comparson. Furthermore, n the present work, we nvestgate how dfferent types of natural selecton affect the populaton evolvng under the Wrght-Fsher model manly through Monte Carlo smulaton studes, whch can only demonstrate the qualtatve dfference n the 25

26 behavor of the Wrght-Fsher model. It would be far more challengng to carry out the analyss of the effect of dfferent types of natural selecton on the Wrght-Fsher model that can lead to more quanttatve comparsons. 449 Acknowledgments 450 We would lke to thank Dr. Danel Lawson for crtcal readng and suggeston and the two anonymous revewers for ther helpful comments. Ths work was funded n part by the Engneerng and Physcal Scences Research Councl (EPSRC) Grant EP/I028498/1 to F.Y. 453 References Bürger, R. (2000). The Mathematcal Theory of Selecton, Recombnaton, and Mutaton. Chch- ester: Wley. 456 Charlesworth, B., Nordborg, M., & Charlesworth, D. (1997). The effects of local selecton, balanced polymorphsm and background selecton on equlbrum patterns of genetc dversty n subdvded populatons. Genetcal Research, 70, Chrstansen, F. (1984). Defnton and measurement of ftness. In B. Shorrocks (Ed.), Evolu- tonary ecology: the 23rd Symposum of the Brtsh Ecologcal Socety, Leeds, Oxford: Blackwell Scence Ltd Durrett, R. (2008). Probablty Models for DNA Sequence Evoluton. New York: Sprnger- Verlag Edwards, A. W. (2000). Foundatons of Mathematcal Genetcs. Cambrdge: Cambrdge Un- versty Press Etherdge, A. (2011). Some Mathematcal Models from Populaton Genetcs: École Probabltés de Sant-Flour XXXIX Berln: Sprnger-Verlag. D Été de Ewens, W. J. (1972). The samplng theory of selectvely neutral alleles. Theoretcal Populaton Bology, 3, Ewens, W. J. (2004). Mathematcal Populaton Genetcs 1: Theoretcal Introducton. New York: Sprnger-Verlag. 26