A N-branching random walk with random selection

ALEA, Lat. Am. J. Probab. Math. Stat. 14, 117 137 217 A -branching random walk with random selection Aser Cortines and Bastien Mallein Faculty of Industrial Engineering and Management, Technion - Israel Institute of Technology, Technion City, Haifa 32, Israel. E-mail address: cortines@technion.ac.il URL: https://web.iem.technion.ac.il/en/people/userprofile/cortines.html Institut für Mathematik, Universität Zürich Winterthurstrasse 19, CH-857, Zrich, Swiss. E-mail address: bastien.mallein@math.uzh.ch URL: http://www.math.ens.fr/~mallein Abstract. We consider an exactly solvable model of branching random walk with random selection, which describes the evolution of a population with individuals on the real line. At each time step, every individual reproduces independently, and its offspring are positioned around its current locations. Among all children, individuals are sampled at random without replacement to form the next generation, such that an individual at position x is chosen with probability proportional to e βx. We compute the asymptotic speed and the genealogical behavior of the system. 1. Introduction In a general sense, a branching-selection particle system is a Markovian process of particles on the real line evolving through the repeated application of the two steps: Branching step: every individual currently alive in the system splits into new particles, with positions with respect to their birth place given by independent copies of a point process. Selection step: some of the new-born individuals are selected to reproduce at the next branching step, while the other particles are killed. We will often see the particles as individuals and their positions as their fitness, that is, their score of adaptation to the environment. From a biological perspective, Received by the editors May 13, 216; accepted January 4, 217. 21 Mathematics Subect Classification. 6J8, 6K35. Key words and phrases. Branching random walk, Coalescent process, Selection, Poisson- Dirichlet partition. Research partially supported by the Israeli Science Foundation grant 1723/14. 117

118 A. Cortines and B. Mallein branching-selection particle systems model the competition between individuals in an environment with limited resources. t t t + 1 t + 1 a Branching step b Selection step Figure 1.1. One time step of a branching-selection particle system These models are of physical interest Brunet and Derrida, 1997; Brunet et al., 27 and can be related to reaction-diffusion phenomena and the F-KPP equation. Different methods can be used to select the individuals. For example, one can consider an absorbing barrier, below which particles are killed Aïdékon and Jaffuel, 211; Berestycki et al., 213; Mallein, 215a; Pain, 216. Another example is the case where only the rightmost individuals are chosen to survive Brunet and Derrida, 1997; Brunet et al., 27; Bérard and Gouéré, 21; Durrett and Remenik, 211, the so-called -branching random walk. In this paper, we introduce a new selection mechanism, in which the individuals are randomly selected with probability depending on their positions. Based on numerical simulations Brunet and Derrida, 1997 and the study of solvable models Brunet et al., 27, it has been predicted that the dynamical and structural aspects of many branching selection particle systems satisfy universal properties. For example, the cloud of particles travels at speed v, which converges to a limit v as the size of the population diverges. It has been conectured in Brunet et al. 27 that v v ϕ log + 3 log log + olog log 2 as, for an explicit constant ϕ depending on the law of reproduction. Some of these conectures have been recently proved for the -branching random walk Bérard and Maillard, 214; Bérard and Gouéré, 21; Couronné and Gerin, 214; Durrett and Remenik, 211; Maillard, 216; Mallein, 215b. Bérard and Gouéré 21 prove that v v behaves like ϕlog 2. evertheless, several conectures about this process remain open, such as the asymptotic behavior of the genealogy or the second-order expansion of the speed. Other examples in which the finite-size correction to the speed of a branching-selection particle system is explicitly computed can be found in Bérard and Maillard 214; Comets and Cortines 215; Couronné and Gerin 214; Durrett and Remenik 211; Mallein 215b; Mueller et al. 211. To study the genealogical structure of such models we define the ancestral partition process Π n t of a population choosing n individuals from a given generation T and tracing back their genealogical linages. That is, Π n t is a process in P n the set of partitions or equivalence classes of [n] : {1,..., n} such that i and belong to the same equivalence class if the individuals i and have a common ancestor t generations backwards in time. otice that the direction of time

A -branching random walk with random selection 119 is the opposite of the direction of time for the natural evolution of the population, that is, t is the current generation, t 1 brings us one generation backward in time and so on. It has also been conectured in Brunet et al. 27 that the genealogical trees of branching selection particle systems converge to those of a Bolthausen-Sznitman coalescent and that the average coalescence times scale like a power of the logarithm of the population size. These conectures contrast with classical results in neutral population models, such as Wright-Fisher and Moran s models, that lay in the Kingman coalescent universality class Möhle and Sagitov, 21. Mathematically, these conectures are difficult to be verified and they have only been proved for some particular models, see Berestycki et al. 213; Cortines 216. We define in this article a solvable model of branching selection particle system evolving in discrete time, and compute its asymptotic speed as well as its genealogical structure. Given and β > 1, it consists in a population with a fixed number of individuals. At each time step, the individuals die giving birth to offspring that are positioned according to independent Poisson point processes with intensity e x dx that we write PPPe x dx for short. Then, individuals are sampled without replacement to form the next generation, such that a child at the position x is sampled with probability proportional to e βx. We introduce the following notation. Let X 1,..., X R be the initial position of the particles and {P t,, t } be a family of i.i.d. PPPe x dx. Given t 1 and X t 11,..., X t 1 the positions at time t 1, we define the new positions as follows: i. Each individual X t 1 gives birth to infinitely many children that are positioned according to the point process X t 1+P t. Let t : t k; k be the sequence obtained by all positions ranked decreasingly, that is t k, k Rank { X t 1 + p; p P t, }. ii. We sample successively individuals X t 1,..., X t composing the tth generation from { t 1, t 2,...} such that for all i {1, 2,..., }: P X t i t t, X t 1,..., X t i 1 eβ t 1 { t {Xt 1,...X t i 1}} + k1 eβ tk. 1.1 i 1 k1 eβx t k To keep track of the genealogy of the process we define A t i if X t i { X t 1 + p, p P t }, 1.2 that is, A n i if Xt i is an offspring of Xt 1. We call this system the, β-branching random walk or, β-brw for short. It can be checked that the sum in the denominator of 1.1 is finite if β 1, and that it diverges as β 1 see Proposition 1.3 below, thus the model is only defined for β 1,. otice that as β the sum in the denominator is dominated by the high values of t. Precisely, it can be checked that the following limits hold a.s. lim β e β t1 + k1 eβ tk 1, lim β e β t2 + k2 eβ tk 1, and so on. Therefore, the case β is the exponential model from Brunet and Derrida 1997, 212; Brunet et al. 27, in which the rightmost individuals

12 A. Cortines and B. Mallein are selected to form the next generation. In contrast with the examples already treated in the literature, when β < one does not necessarily select the rightmost offspring. In this paper, we will take interest in the dynamical and genealogical aspects of the, β-brw, showing that it travels at a deterministic speed and that its genealogical trees converge in distribution. The next result concerns the speed of the, β-brw. Theorem 1.1. For all and β 1, ], there exists v,β such that max Xt min Xt lim lim v,β t + t n + t a.s. 1.3 moreover, v,β log log + o1 as. The main result of the paper is the following theorem concerning the convergence in law of the ancestral partition process Π n t; t of the, β-brw. Theorem 1.2. For all and β 1, ], let c be the probability that two individuals uniformly chosen at random have a common ancestor one generation backwards in time. Then, we have lim c log 1 and the rescaled coalescent process Π t/c, t converges in distribution toward the Bolthausen- Sznitman coalescent. The Bolthausen-Sznitman coalescent in Theorem 1.2 can be roughly explained by an individual going far ahead of the rest of the population, so that its offspring are more likely to be selected and overrun the next generation. Based on precise asymptotic of the coalescence time, Brunet et al. 27 argue that the genealogical trees of the exponential model converge to the Bolthausen-Sznitman coalescent and conecture that this behavior should be expected for a large class of models. The, β-brw can be though as a finite temperature version of the exponential model from Brunet et al. 27. In this sense, Theorem 1.2 attests for the robustness of their conectures showing that even under weaker selection constrains this convergence occurs. It indicates that whenever the rightmost particles are likely to be selected, then the Bolthausen-Sznitman coalescent is to be expected. Different coalescent behavior should be expected when the selection mechanism does not favor the rightmost particles, the classical example being the Wright-Fisher model. Another example can be obtained modifying the selection mechanism of the, β-brw. It can be checked using the techniques developed in this paper see for example Theorem 3.3 that if we systematically eliminate the first individual sampled X 1 t, so that it does not reproduce in the next generation, then this new branching-selection particle system lays in the Kingman s coalescent universality class. otice that this new selection procedure no longer favors the rightmost particles in this case, the rightmost particle, which ustifies this change of behavior. otation. In this article, we write fx gx as x a if lim x a fx gx 1; fx ogx as x a if fx lim x a gx ; and fx Ogx as x a if lim sup x a fx gx < +.

A -branching random walk with random selection 121 1.1. Preliminary results. In this section, we prove that the, β-brw is well defined and provide elementary properties such as the existence of the speed v,β. Proposition 1.3. The, β-brw is well-defined for all and β 1, ]. Moreover, setting Xt eq : log 1 ex t, the sequence δ k t+1 Xt eq : t k is an i.i.d. family of Poisson point processes with intensity measure e x dx. Proof : With and β fixed, assume that the process has been constructed up to time t with X t 1,..., X t denoting the positions of the particles. Thanks to the invariance of superposition of independent PPP, { X t +p; p P t, } is also a PPP with intensity measure i1 e x Xti dx e x X t eq dx. Therefore, with probability one: all points have multiplicity one and the sequence k t + 1; k is uniquely defined. Since there are finitely many points k t + 1 that are positive and E e β kt+1 1 { k t+1<} < we have that e β k t+1 < a.s. As a consequence, the selection step is well-defined, proving the first claim. Moreover, k t+1 X t eq; k is a PPPe x dx independent from the t first steps of the, β-brw, proving the second claim. Remark 1.4. It is convenient to think Xt eq as an equivalent position of the front at time t, in the sense that the particles positions in the t+1th generation are distributed as if they were generated by a unique individual positioned at Xt eq. We use Proposition 1.3 to prove the existence of the speed v,β, the study of its asymptotic behavior is postponed to Section 4. Lemma 1.5. With the notation of the previous proposition, 1.3 in Theorem 1.1 holds with v,β : EX 1 eq X eq. Proof : By Proposition 1.3, X t+1eq X t eq t eq : t are i.i.d. random variables X with finite mean, therefore lim t t + v,β a.s. by the law of large numbers. otice that both max Xt Xt 1eq t and min Xt Xt 1eq t are sequences of i.i.d. random variables with finite mean, which yields 1.3. In a similar way, we are able to obtain a simple structure for the genealogy of the process, and describe its law conditionally on the position of the particles. Lemma 1.6. The sequence A t t defined in 1.2 is i.i.d. Moreover, it remains independent conditionally on H σxt,, t, with the conditional probabilities P A t+1 k H θ t k 1...θ t k, where k k 1,..., k {1,..., } ; and θ t k : e X t k i1 ex t i. 1.4

122 A. Cortines and B. Mallein Proof : To each point x δ t+1k Xt eq we associate the mark i if it is a point coming from Xt i+p t i. The invariance under superposition of independent PPP yields P x Xt i + P t i H e x X t i ext i 1 e x X t. By definition of 1 ex t A t i, it is precisely the mark of Xt+1i, which yields 1.4. The independence between the A t can be easily checked using Proposition 1.3. Organization of the paper. In Section 2, we obtain some technical lemmas concerning the Poisson-Dirichlet distributions. We focus in Section 3 on a class of coalescent processes generated by Poisson Dirichlet distributions and we prove a convergence criterion. Finally, in Section 4, we provide an alternative construction of the, β-brw in terms of a Poisson-Dirichlet distribution, and we use the results obtained in the previous sections to prove Theorems 1.1 and 1.2. 2. Poisson-Dirichlet distribution In this section, we focus on the two-parameter Poisson-Dirichlet distribution denoted as PDα, θ distribution. Definition 2.1 Definition 1 in Pitman and Yor, 1997. For α, 1 and θ > α, let Y : be a family of independent r.v. such that Y has Beta1 α, θ + α distribution and write 1 V 1 Y 1, and V 1 Y i Y, if 2. i1 Let U 1 U 2 be the ranked values of V n, we say that the sequence U n is the Poisson-Dirichlet distribution with parameters α, θ. otice that for any k and n, we have P V n U k U,, V 1,..., V n 1 U k 1 {Uk {V 1,...V n 1}} 1 V 1 V 2 V n 1, for this reason we say that V n follows the size-biased pick from a PDα, θ. It is well known that there exists a strong connexion between PD distributions and PPP, we recall some of these results in the proposition below. Proposition 2.2 Proposition 1 in Pitman and Yor, 1997. Let x 1 > x 2 >... be the points of a PPPe x dx and write L + 1 eβx and U e βx /L. Then U, 1 has PDβ 1, distribution and lim n + n β U n 1/L a.s. otice from Propositions 1.3 and 2.2 that e βx n i / + 1 eβ n has the distribution of V i the size-biased pick from PDβ 1,, which makes the model solvable. Remark 2.3 Change of parameter. If V 1, V 2,... is a size-biased pick of PDα, θ, V then 2 V 1 V 1, 3 1 V 1,... has the distribution of a size-biased pick from a PDα, α+θ. Moreover, it is independent of V 1. That is, the sequence obtained from V 1, V 2,... after discarding the first sampled element V 1 and re-normalizing is a size-biased pick from a PDα, α + θ. Therefore, ordering this new sequence one obtains a PDα, α + θ sequence.

A -branching random walk with random selection 123 In what follows, we fix α, 1 and θ > α, and let c and C be positive constants, that may change from line to line and implicitly depend on α and θ. We will focus attention on the convergence and the concentration properties of Σ n : n 1 V α n 1 1 Y i α ; n. 2.1 1 Y α i1 Lemma 2.4. Set M n : n i1 1 Y i, then, there exists a positive r.v. M such that γ lim n 1 α α Mn M γ a.s. and in L 1 for all γ > θ + α, 2.2 n + with γ-moment verifying EM γ Φ θ,α γ : α γ Γθ+1Γ θ+γ. Moreover, if θ Γθ+γ+1Γ α +1 < γ < θ + α, then there exists C γ > such that P inf α Mn y C γ y γ, for all n 1 and y. 2.3 n n 1 α otice that if γ > θ, then Φ θ,α γ α γ ΓθΓ θ+γ α. Γθ+γΓ θ α α +1 Proof : Fix γ > θ + α, then Mn γ / EMn γ is a non-negative martingale with respect to its natural filtration and E Mn γ Γθ + γ + nα Γ n + θ α Γθ + nα Γ n + θ+γ Γθ + 1Γ θ+γ α + 1 Γθ + γ + 1Γ θ α α + 1 1 α γ Φ θ,α γn α, as n +. Since lim n + EMn γ/2 EMn γ 1/2 >, Kakutani s theorem yields Mn γ / EMn γ converges a.s. and in L 1 as n +, implying 2.2 with M lim n + M n n 1 α α. In particular, if < γ < θ + α we obtain from Doob s martingale inequality that P inf n 1 α α Mn y P sup M γ 1 α γ n n α y γ n n M P sup γ n EM n n γ y γ /C γ, with C γ : sup n E[Mn γ ]/n γ 1 α α <, proving 2.3. We now focus on the convergence of the series Y α α 1. Lemma 2.5. Let S n : n 1 Y αα 1 and Ψ α : α α Γ1 α 1, then, there exists a random variable S such that lim n + S n Ψ α log n S a.s. Moreover, there exists C > such that for all n and y, P S n ES n y Ce y2 α.

124 A. Cortines and B. Mallein Proof : Since Y has Beta1 α, θ + α distribution, we have EY α and VarY α 1 α α Γ1 α + O1/ Γ1 + αγ1 α 1 α 2α Γ1 α 2 + O1/, which implies that VarY α 1 < + and that ES n Ψ α log n + C S + o1 with C S R. Thanks to Y α EY α 1, 1 a.s. we deduce from Kolmogorov s three-series theorem that S n ES n and hence that S n Ψ α log n converge a.s. To bound PS n ES n y, notice that PS n ES n y e λy E [e λsn ESn] n e λy E e λα 1 Y α EY α, for all y and λ >. Taking c > such that e x 1 + x + cx 2 for x 1, 1, we obtain { E [e λα 1 Y α EY α ] e λα 1 if λ α 1 > 1; 1 + cλ 2 2α 1 VarY α if λα 1 1. Since 1 α λ α 1 < λ 1 1 α for all α, 1 and θ, there exists c cα, θ such that P S n ES n y e λy e λα 1 1 + c λ2 exp 1 α λ 1 α >λ 1 λy + λ 2 α 1 α + cλ 2, for all n and y. Let ϱ : 2 α/1 α > 2, then there exists C Cα, θ > such that P S n ES n y C exp λy + Cλ ϱ, for all n and y. Optimizing in λ > we obtain P S n ES n y C exp y ϱ/ϱ 1 C 1/1 ϱ ϱ 1/1 ϱ ϱ ϱ/1 ϱ, with C 1/1 ϱ [ ϱ 1/1 ϱ ϱ ϱ/1 ϱ] >, as ϱ > 1. The same argument, with the obvious changes, holds for PS n ES n y, therefore, there exists C > such that P S n ES n y C exp y ϱ/ϱ 1 /C, proving the second statement. With the above results, we obtain the convergence of Σ n V α tail probabilities. Lemma 2.6. With the notation of Lemmas 2.4 and 2.5, we have lim n + Σ n log n Ψ αm α a.s. and in L 1. 2 as well as its Moreover, for any < γ < α + θ there exists D γ such that for all n 1 large enough and u > we have P Σ n u log n D γ u γ α.

A -branching random walk with random selection 125 Proof : otice that Σ n n 1 S S 1 1 α M α 1 since S : and that lim n + Mn n + 1 1 α α α M α and lim n + S n log n Ψ α by Lemmas 2.4 and 2.5 respectively. Then, Stolz-Cesàro theorem yields the a.s. convergence of Σ n / log n to Ψ α M α as n. Expanding Σ n 2 we obtain E Σ 2 n n E n 1 V 2α + 2 C 1 n 1 n i1 i+1 E M 1 2α EY 2α n 1 E V i V α n 1 + 2 E Mi 1 2α E Yi 1 Y i α i1 n 1 21 α 2α + C i 21 α i α i1 n i+1 n i+1 E M α 1 M α i a.s. EY α 1 α i 1 α α Clog n 2. Therefore, sup E [ Σ n / log n 2] < + implying its L 1 convergence. To obtain bounds for PΣ n u log n, we study the two cases u 1/n and u 1/n separately. Assume first that u 1/n, then n Σ n 1 α 1 1 α Y α α M 1 α inf 1 α α M S n. For all γ < θ + α and t > such that t < E[S n ] we have α P Σ n u log n P S n t + P inf 1 α α M u log n/t C exp C 1 E[S n ] t ϱ ϱ 1 + C γ γ /α u log n t. Let < ε < 1/2 and set t u ε ES log n, since lim n n + log n Ψ α, there exists a constant c > depending only on α such that u ε log n E[S n ] for all u c. Decreasing c if necessary, we can and will assume that and hence that P Σ n u log n C exp C 1 E[S n ] u ε log n < a log n a log n ϱ/ϱ 1 + C γ u 1 εγ /α, for all u c, where a > is to be chosen conveniently small. Observe that C exp a log n ϱ/ϱ 1 < C γ u 1 εγ /α, 1/n. There- [ ] C for all u C exp α ϱ γ γ a log n ϱ 1, c and that e α γ a log n fore, taking γ 1 εγ < α + θ there exists D γ such that for all n large enough and u [ 1 n, +. P Σ n u log n D γ u γ α, ϱ ϱ 1

126 A. Cortines and B. Mallein On the other hand if u 1/n, let be such that 1 α > γ, then n PΣ n < u log n P V α u log n P V α < u log n; for all 1. 1 Observe that if u < 1/n and V α Y α < u log n for all, we have V α 1 Y u log n 11 Y 2 1 Y 1 α 1 Y α 1 1 Y 1 α. We prove by recurrence that under the above hypothesis, Y α u log n 1 log n 1 n. The case 1 holds by the assumption Y1 α u log n. Assuming that the statement holds for all i 1, that is Yi α u log n / i 1 log n 1 n then 1 log n 1 Y1 α 1 Y2 α 1 Y 1 α n 1 1 yielding Y α 1 i log n n 1 log n 1, i 1 log n i1 1 n n 1 log n n. As a consequence, for all and n suffi- < 2u log n and hence PΣ n < u log n 1 P Y α < 2u log n. ciently large Y α log n / 1 i1 1 i 1 log n n Using crude estimate for the probability distribution function of the Beta distribution, we bound the product in the display by Cu γ α ϱ u 1 α γ log n 1 α, with C an explicit constant. Since u < 1/n and 1 α γ >, the term inside the parentheses tends to zero uniformly in u. Therefore, increasing D γ > if necessary, the upper-bound P Σ n < u log n D γ u γ α holds for all n 1 and u finishing the proof. In some cases, we are able to identify the random variable Ψ α M α. Corollary 2.7. Let U n n be a PDα,, then Ψ α M α L α, where we have set 1/L lim n + n 1/α U n. Proof : By Proposition 2.2, L : lim n + n 1/α /U n exists a.s. and by Lemma 2.6 we have that Ψ α M α 1 n log n 1 V α 1 n log n 1 U α L α as n +, thus Ψ α M α L α a.s. By Lemma 2.4, the pth moments of Ψ α M α are equal to E [Ψ α M α p ] Γp + 1 Γpα + 1 Γ1 α p, for all p > 1. By Pitman and Yor 1997, Equation 3, it matches with the pth moments of L α, which implies that the two random variables have the same distribution the Mittag- Leffler α distribution, and hence that Ψ α M α L α a.s. by monotonicity. 3. Convergence of discrete exchangeable coalescent processes In this section, we study a family of coalescent processes with dynamics driven by PD-distributions and obtain a sufficient criterion for the convergence in distribution of these processes. For the sake of completeness, we include a brief introduction to coalescent theory with the main results we will use, for a detailed account we recommend the paper Berestycki 29 from where we borrow the approach.

A -branching random walk with random selection 127 Let P n be the set of partitions or equivalence classes of [n] : {1,..., n} and P the set of partitions of [ ]. A partition π P n is represented by blocks π1, π2,... listed in the increasing order of their least elements, that is, π1 is the block class containing 1, π2 the block containing the smallest element not in π1 and so on. There is a natural action of the symmetric group S n on P n setting π σ : { {σ, πi}, i [n] } for σ S n. If m < n, one can define the proection of P n onto P m by the restriction π m {π [m]}. For π, π P n, we define the coagulation of π by π to be the partition Coagπ, π { i π πi; }. With this notation, a coalescent process Πt is a discrete or continuous time Markov process in P n such that for any s, t, Πt + s CoagΠt, Π s, with Π s independent of Πt. We say that Πt is exchangeable if Π σ t and Πt have the same distribution for all permutation σ. An important class of continuous-time exchangeable coalescent processes in P are the so-called Λ-coalescents see Pitman, 1999, introduced independently by Pitman and Sagitov. They are constructed as follows: let Π n t be the restriction of Πt to [n], then Π n t; t is a Markov ump process on P n with the property that whenever there are b blocks, each k-tuple k 2 of blocks is merging to form a single block at the rate λ b,k x k 2 1 x b k Λdx, where Λ is a finite measure on [, 1]. Among such, we distinguish the Beta2 λ, λ-coalescents obtained from Λdx x 1 λ 1 x λ 1 ΓλΓ2 λ dx, where λ, 2, the case λ 1 uniform measure being the celebrated Bolthausen-Sznitman coalescent. The set P can be endowed with a topology making it a Polish space, therefore, one can study the weak convergence of processes in D [,, P, see Berestycki 29 for the definitions. Without going into details, we say that a process Π t P converges in the Skorokhod sense or in distribution to Πt, if for all n the proection Π t n converges in distribution to Πt n in D [, P n. 3.1. Coalescent processes obtained from multinomial distributions. In this section, we define a family of discrete-time coalescent processes Π t; t and prove sufficient criteria for its convergence in distribution. Let η1,... η be an - dimensional random vector satisfying 1 η 1 η 2 η and η 1. Conditionally on a realization of η, let { ξ ; } be i.i.d. random variables satisfying Pξ k η ηk and define the partition With π t ; t i.i.d. such that 1 π { { : ξ k}; k }. copies of π, let Π t be the discrete time coalescent Π {{1}, {2},..., {n}} and Π t + 1 Coag Π t, π t+1.

128 A. Cortines and B. Mallein The goal of this section is to obtain conditions under which Π t converges in distribution. First, we assume that there exist a sequence L and a function f :, 1 R + such that lim L +, + lim L P η1 > x fx and lim L E η2 + +. 3.1 Denote by c 1 E [ η 2], which corresponds to the probability that two individuals have a common ancestor one generation backward in time. Lemma 3.1. Assume that 3.1 holds and that x sup L Pη1 > x dx < +. 3.2 Then, c L 1 2xfxdx and the coalescent process Π t/c ; t R +, properly rescaled, converges in distribution to the Λ-coalescent, with Λ satisfying fx. x Λdy y 2 Proof : Denote by ν k #{ : ξ k}, then ν 1,..., ν has multinomial distribution with trials and random probabilities outcomes ηi. By Möhle and Sagitov 21, Theorem 2.1, the convergence of finite dimensional distribution of Π t is obtained from the convergence of the factorial moments of ν, that is 1 ] E [ν i1 b1... ν ia ba, with b i 2 and b b 1 +... + b a, c b i 1,...,i a1 all distinct where n a : nn 1... n a + 1. Since ν 1,...[, ν is multinomial distributed, we obtain that E [ν i1 b1... ν ia ba ] b E η b1 i 1... η ba i a ], see Cortines 216, Lemma 4.1 for a rigorous a proof. Therefore, we only have to show that for all b and a 2 [ E ηi 1 b] x b 2 Λdx and lim + c 1 lim + c 1 i 11 i 1,...,i a1 all distinct We obtain by dominated convergence that [ η 2 ] L E 1 2xL P η 1 > x dx 2xfxdx ] E [η i1 b1... η ia ba. Λdx < +, as. We also get E [ η2 2 +... + η 2 ] [ E η 2 1 η 1 ], with L E η2 tending to zero as, as ηi is ordered and sums up to 1. In particular, it implies that L c L E[η i 2 ] tends to 2xfxdx as. A similar calculation shows that for any b 2 lim L E [ ηi b] bx b 1 fxdx x b 2 Λdx λ b,b, i1

A -branching random walk with random selection 129 where λ b,b is the rate at which b blocks merge into one given that there are b blocks in total. The others λ b,k can be easily obtained using the recursion formula λ b,k λ b+1,k + λ b+1,k+1. We now consider the case a 2, cases a > 2 being treated in the same way. We have [ η b1 ] E i1 η b2 i2 i 1,i 21 distinct E [ η 1 b1η ] 2 η b2 1 i [ + E i 1 1 + E η i1 b1η 2 i 2 1 i 2 i 1 [ η b2η ] 1 2 η b1 1 i ] η b2 1 i2 3 E [ η2 ], we recall that b+. η2..+ η b η 2 +...+η 1 η 1 < 1. Since L E η2 as, the right hand side of the inequality tends to zero, concluding the proof. The next lemma gives sufficient conditions for the convergence to the Kingman s coalescent. Lemma 3.2. Assume that 3.1 holds and that xfxdx + and n 2 : x sup n L Pη1 > x dx < +. 3.3 Then, lim + c L + and the process Π n tc 1 ; t R + converges in the Skorokhod sense to the Kingman s coalescent restricted to P n. Proof : A similar argument to the one used in Lemma 3.1 shows that L c L E[η 2 1] 2xL Pη 1 > xdx. Thus, Fatou lemma and 3.3 yield lim inf + L c +, proving the first claim. By Berestycki 29, Theorem 2.5, the convergence to the Kingman s coalescent follows from lim i1 E[ν i 3 ] / 3 c. We rewrite the sum in the display as c 1 i1 E[η i 3 ] and apply Hölder inequality to obtain E [ η1 λ η1 3 λ] E [ η1 2] [ ] λ/2 E η1 23 λ 1 λ/2 2 λ for all λ < 2. Let λ, 2 be the unique solution of 23 λ / 2 λ n + 4, then we obtain from 3.3 that E [ η1 3] E [ η 1 3] [ E η E [ η1 2] 1 n+1] 1 λ/2 E [ η1 2]. + c

13 A. Cortines and B. Mallein A similar argument to the one used in Lemma 3.1 shows that c 1 i2 E[η i 3 ] tends to zero as, which proves the statement. 3.2. The Poisson-Dirichlet distribution case. In this section, we construct a coalescent using the PD distribution and obtain a criterion for its convergence in distribution. With V, 1 a size-biased pick from a PDα, θ partition, define θ : V α i1 V α i and θ 1 θ 2 θ, the order statistics of θ. In what follows, θi will stand for the η i from Section 3.1 and Π n t; t for the coalescent with transition probabilities Π n t + 1 Coag Π n t, πt n, as defined there. Theorem 3.3. With the above notation, set λ 1 + θ/α and L c α,θ log λ, where c α,θ Γ1 θ/αγ1 α θ/α Γ1 + θ 1. 1 If θ α, α, then c + 1 θ/α/l and Π t/c, t converges weakly to the Beta2 λ, λ-coalescent. 2 Otherwise, lim + c L + and Π t/c converges weakly to the Kingman s coalescent. Before proving Theorem 3.3 we obtain a couple of technical results. The next lemma studies the asymptotic behavior of θ 1. Lemma 3.4. With the notations of Theorem 3.3, we have lim L P θ1 > x λ 1 1 x 1 Beta2 λ, λdy + λγλγ2 λ x y 2. Moreover, there exists C > such that for all x, 1 Proof : Let Σ : V 2 sup L P θ1 > x Cx λ. α, 1 Y 1 then Remark 2.3 says that Σ and Y 1 are has the distribution of V 1 +... V 1 with V i a size- independent and that Σ biased pick from a PDα, α + θ distribution. By Lemma 2.6, for all ε, 1 there exists C Cε and such that Writing P θ 1 P θ 1 sup P Σ u log min Cu λ+ε, 1, for all u. 3.4 > x in terms of Σ and V 1 Y 1, we obtain 1 + 1 V 1 α Σ P P 1/y 1 > > x P V α 1 > x V α x 1 x Σ x V 1 1 V 1 > x 1 x Σ 1/α 1/α Γ1 + θ1 y α y α+θ 1 dy. Γ1 αγα + θ

A -branching random walk with random selection 131 Making the change of variables u 1 x x log 1/y 1α the display reads P θ1 > x λ x Γ1 + θ + P Σ < u log x log αγ1 αγα + θ u 2 1/α u 1/α + 1 x x log Then, we use 3.4 to bound the equation within the integral, obtaining 1/α 1+θ du. P Σ < u log 1/α 1+θ u u PΣ u log mincu ε 1, u 2, u 2+θ/α 2 1/α 1/α + 1 x x log for all large enough. In particular, there exists C > such that L P η1 > x Cx λ for all. Moreover, by dominated convergence and Lemma 2.6, we obtain lim log + λ P θ1 > x λ x Γ1 + θ + PΨ α M α < u x αγ1 αγα + θ u 1+λ λ 1 x Γ1 + θ x αλγ1 αγα + θ E Ψ α M α λ λ 1 x α α+θ 1 Γ1 α θ/α Γ1 + θ Φ θ+α,α θ + α, x λγα + θ and hence lim L P θ1 > x λ 1 x 1, proving the statement. + x λγλγ2 λ This result is used to study the asymptotic behavior of θ 1 max θ. Lemma 3.5. For all ε, 1, there exists C Cε such that P θ1 > x P θ1 > x Cx log ε 2 θ/α, for all x, 1 and large enough. Proof : otice that P θ1 > x P θ1 > x and that θ1 θ 1 if V 1 > 1/2, thanks to V i 1. Therefore, splitting the events according to V 1 > 1/2 and V 1 < 1/2 we obtain P θ1 > x P θ1 > x P θ1 > x; V 1 1 2 P θ1 > x; 1. 2 Since < V < 1 and V 1 Y 1, we have P θ1 > x, V 1 1/2 P P θ1 > x; V 1 1 2 max V α > x 1 V α; V 1 1/2 P x 1 > Y α 1 + 1 Y 1 α Σ, Y 1 1/2 P Σ < 2 α /x,

132 A. Cortines and B. Mallein where Σ V α/1 Y 1 α. By Lemma 2.6, we obtain that for all sufficiently large P θ1 > x, V 1 1/2 α+θ ε 1 Cx log α, which finishes the proof. We now study the asymptotic behavior of the second maxima in {θ 1,..., θ }, written θ 2. Lemma 3.6. For all ε, 1], there exists C > such that for any x, 1 and, P θ2 > x < Cx log ε 2 θ/α. Proof : We basically use the same method as in the previous lemma P θ 2 > x P θ 2 > x, V 1 1/2 + P θ 2 > x, V 1 > 1/2, V 2 < 1/3 + P θ 2 > x, V 1 > 1/2, V 2 > 1/3 P θ 1 > x, V 1 1/2 + P θ 2 > x, V 1 > 1/2, V 2 < 1/3 + P θ 2 > x. By Lemma 3.5, we have that P θ 1 > x, V 1 1/2 Cx log ε 2 θ/α, so the same arguments used in Lemma 3.4 yield P θ 2 > x P V α 2 1 x xv α 1 > x1 V 1 V 2 α Σ Cx log ε 2 θ/α, with Σ : 1 V 1 V 2 α 3 V α. Moreover, Σ is independent of V 1, V 2 and P θ2 > x, V 1 > 1/2, V 2 < 1/3 P max V α > x V1 α + V2 α + 1 V 1 V 2 α Σ 2 concluding the proof. P Σ C/x Cx log ε 2 θ/α, Proof of Theorem 3.3: Given ε >, Lemma 3.5 says that there exists C > such that L P θ 1 > x L P θ 1 > x Clog ε 1 x λ, for all x, 1. Therefore, by Lemma 3.4 we have that lim L P λ θ1 > x 1 1 x and + λγλγ2 λ x sup L P θ1 > x Cx λ. 3.5 We obtain from Lemma 3.6 that [ ] L E θ2 L P θ2 > x dx log ε 1 x ε 1 λ dx, 3.6 + which implies that Π t satisfies 3.1.

A -branching random walk with random selection 133 Assume now that θ α, α, so that λ, 2, then 3.5 yields x sup L P θ1 > x dx C x 1 λ dx < +. Therefore, the assumptions of Lemma 3.1 are satisfied implying that Π t/c converges in distribution to the Beta2 λ, λ-coalescent and that c L 1 θ/α as. On the other hand if θ α, we have that x 1 x λ x dx +. With k λ, we obtain from 3.5 that x k sup L P θ1 > x dx C x k λ dx < +. We apply Lemma 3.2 to conclude that Π t/c converges weakly to the Kingman s coalescent. 4. Poisson-Dirichlet representation of the, β-branching random walk We explore the relations between the, β-brw and the PDβ 1, distribution to show Theorems 1.1 and 1.2 in the case where β <. The case β is also studied, but using different methods. Proposition 4.1. Let β 1, and V n n be its size-biased pick, and set U n n be a PDβ 1,, L lim n + n β Un 1 then X 1 X eq, d 1 β log V + 1 β log L, in particular, X1 eq X eq d log 1 V 1/β + 1 β log L. Proof : By Proposition 1.3, x k, k 1 : Rank { X + p X eq, p P 1, } is the ordered points of a PPPe x dx. With L + 1 eβx and U e βx /L, we know from Proposition 2.2 that U, 1 is a PDβ 1, and that lim n + nβ U n L 1. By the definition of the, β-brw, V : e βx 1 X eq /L is the th particle sampled in the size-biased pick from U n n. Inversing the equation, we conclude that X 1 X eq 1 β log V + log L, proving the first statement. The second comes from the definition of X 1 eq. To study the case β +, we use the following representation of rightmost points of a PPPe x dx. Proposition 4.2. Let x 1 > x 2 >... > x be the rightmost points of a PPPe x dx, then x 1,..., x d Rank{Z + e 1,..., Z + e }, where e are i.i.d exponential random variables with mean 1 and Z is an independent random variable satisfying PZ dx 1! exp + 1x e x dx.

134 A. Cortines and B. Mallein Proof : It is an elementary result about PPP that + 1 δ d e x PPPdx on R + and that e x d +1 Gamma + 1, 1, moreover, conditionally on x +1, e x1,... e xn are the ranked values of i.i.d. uniform random variables on the interval [, e x +1 ]. Setting Z x +1 and U 1,..., U i.i.d. uniform random variables e x1,..., e x d Rank{e Z U 1,..., e Z U }. It is straightforward that Z and x 1,..., x satisfy the desired properties. We first use these results to compute the asymptotic behavior of the speed of the, β-brw. Proof of Theorem 1.1: Lemma 1.5 says that 1.3 holds with v,β E X 1 eq X eq. Thus, if β < Proposition 4.1 yields v,β E X1 eq X eq E log 1 V 1/β + 1 E log L. β By Lemma 2.6, log 1 1 V 1/β converges to Ψ β 1M 1/β a.s. and in L 1 as. Therefore, the logarithm of this quantity converges a.s. as well. We notice from Lemma 2.6 that for all u >, P log log 1 1 V 1/β u P 1 V 1/β e u log D 2/β e 2u. The L 1 convergence of log 1 1 V 1/β implies the existence of a constant K such that for all u >, P log log 1 In particular, log 1 V 1/β 1 V 1/β u P 1 V 1/β e u log Ke u. log log is uniformly integrable, which implies its L 1 convergence. We know from Corollary 2.7 that Ψ β 1M 1/β that lim v,β log log lim E E [ log [ log 1 V 1/β log Ψ β 1M 1/β ] + L 1/β, and hence E [log L] β ] E [log L] +. β For β we follow the ideas from Brunet et al. 27 and use Laplace methods to estimate the asymptotic mean of X1 eq. Let x 1,... x be the largest atoms of a PPPe x dx, and define for λ > λ Λλ : E exp λ log e x k E e x k. k1 k1

A -branching random walk with random selection 135 By Proposition 4.2, we have Λλ E e λz λ E k1 ee k, where e k are i.i.d exponential random variables and exp Z has Gamma+1, 1 distribution. otice that the following equalities hold: E e λz Γ+1+λ Γ+1 and λ E e e k 1 + t λ 1 E e t k1 ee k dt Γλ k1 1 Γλ + t λ 1 I t dt, 4.1 with I t Ee tee 1 the Laplace transform of e e1. The function I can be represented using the exponential integral Ei we have I x xei x+e x. Therefore, there exists K > such that I x 1 x log x Kx, for any x. In particular for x t/ log we have I t/ log 1 t/ Thus, 4.1 yields + t λ 1 I t dt 1 log λ 1 log λ + + Γλ log λ 1 K log λ. t λ 1 I t/ log dt K t λ 1 e t1 log dt Kt log. The same argument with the obvious change gives a similar lower bound, which implies Γ + 1 + λ Λλ log λ Γ + 1 1 + Olog 1, uniformly in λ [, 1], therefore log Λλ λ log log + Olog 1. consequence E log e x log Λλ k lim log log + o1, λ λ k1 which concludes the proof. As a In a similar way, we obtain the genealogy of the, β-branching random walk. Proof of Theorem 1.2: If β 1,, then Lemma 1.6 and Proposition 4.1 say that the genealogy of the, β-brw can be described by 1. in Theorem 3.3, with α 1 β and θ. Therefore, it converges to the Bolthausen-Sznitman coalescent. On the other hand if β, the genealogy of the, -BRW is again described by a coalescent process obtained from multinomial random variables. In this case, by Proposition 4.2 we can rewrite the coefficients η as η e e i1 eei, with e 1,..., e i.i.d. exponential random variables.

136 A. Cortines and B. Mallein Thanks to Pe e x x 1, Cortines 216, Theorem 1.2 c says that the genealogy of the, -BRW converges to the Bolthausen-Sznitman coalescent with c as. Acknowledgements The authors would like to thank the anonymous referee for pointing out several mistakes in a preliminary version. References E. Aïdékon and B. Jaffuel. Survival of branching random walks with absorption. Stochastic Process. Appl. 121 9, 191 1937 211. MR2819234. J. Bérard and J.-B. Gouéré. Brunet-Derrida behavior of branching-selection particle systems on the line. Comm. Math. Phys. 298 2, 323 342 21. MR2669438. J. Bérard and P. Maillard. The limiting process of -particle branching random walk with polynomial tails. Electron. J. Probab. 19, no. 22, 17 214. MR3167886. J. Berestycki,. Berestycki and J. Schweinsberg. The genealogy of branching Brownian motion with absorption. Ann. Probab. 41 2, 527 618 213. MR377519.. Berestycki. Recent progress in coalescent theory, volume 16 of Ensaios Matemáticos [Mathematical Surveys]. Sociedade Brasileira de Matemática, Rio de Janeiro 29. ISB 978-85-85818-4-1. MR2574323. E. Brunet and B. Derrida. Shift in the velocity of a front due to a cutoff. Phys. Rev. E 3 56 3, part A, 2597 264 1997. MR1473413. E. Brunet and B. Derrida. How genealogies are affected by the speed of evolution. Philosophical Magazine 92 1-3, 255 271 212. DOI: 1.18/14786435.211.6228. E. Brunet, B. Derrida, A. H. Mueller and S. Munier. Effect of selection on ancestry: an exactly soluble case and its phenomenological generalization. Phys. Rev. E 3 76 4, 4114, 2 27. MR2365627. F. Comets and A. Cortines. Finite-size corrections to the speed of a branchingselection process. ArXiv Mathematics e-prints 215. arxiv: 155.4971. A. Cortines. The genealogy of a solvable population model under selection with dynamics related to directed polymers. Bernoulli 22 4, 229 2236 216. MR349828. O. Couronné and L. Gerin. A branching-selection process related to censored Galton-Walton processes. Ann. Inst. Henri Poincaré Probab. Stat. 5 1, 84 94 214. MR3161523. R. Durrett and D. Remenik. Brunet-Derrida particle systems, free boundary problems and Wiener-Hopf equations. Ann. Probab. 39 6, 243 278 211. MR2932664. P. Maillard. Speed and fluctuations of -particle branching Brownian motion with spatial selection. Probab. Theory Related Fields 166 3-4, 161 1173 216. MR356846. B. Mallein. Branching random walk with selection at critical rate. ArXiv Mathematics e-prints 215a. arxiv: 152.739. B. Mallein. -branching random walk with α-stable spine. ArXiv Mathematics e-prints 215b. arxiv: 153.3762.

A -branching random walk with random selection 137 M. Möhle and S. Sagitov. A classification of coalescent processes for haploid exchangeable population models. Ann. Probab. 29 4, 1547 1562 21. MR188231. C. Mueller, L. Mytnik and J. Quastel. Effect of noise on front propagation in reaction-diffusion equations of KPP type. Invent. Math. 184 2, 45 453 211. MR279386. M. Pain. Velocity of the L-branching Brownian motion. Electron. J. Probab. 21, Paper o. 28, 28 216. MR3492932. J. Pitman. Coalescents with multiple collisions. Ann. Probab. 27 4, 187 192 1999. MR1742892. J. Pitman and M. Yor. The two-parameter Poisson-Dirichlet distribution derived from a stable subordinator. Ann. Probab. 25 2, 855 9 1997. MR1434129.