Applied Probability. Nathanaël Berestycki, University of Cambridge. Part II, Lent 2014

Transcription

1 Applied Probabiliy Nahanaël Beresycki, Universiy of Cambridge Par II, Len 214 c Nahanaël Beresycki 214. The copyrigh remains wih he auhor of hese noes. 1

2 Conens 1 Queueing Theory Inroducion Example: M/M/1 queue Example: M/M/ queue Burke s heorem Queues in andem Jackson Neworks Non-Markov queues: he M/G/1 queue Renewal Theory Inroducion Elemenary renewal heorem Size biased picking Equilibrium heory of renewal processes Renewal-Reward processes Example: Alernaing Renewal process Example: busy periods of M/G/1 queue G/G/1 queues and Lile s formula Populaion geneics Inroducion Moran model Fixaion The infinie sies model of muaions Kingman s n-coalescen Consisency and sie frequency specrum Kingman s infinie coalescen Inermezzo: Pólya s urn and Hoppe s urn Infinie Alleles Model Ewens sampling formula

3 1 Queueing Theory 1.1 Inroducion Suppose we have a succession of cusomers enering a queue, waiing for service. There are one or more servers delivering his service. Cusomers hen move on, eiher leaving he sysem, or joining anoher queue, ec. The quesions we have in mind are as follows: is here an equilibrium for he queue lengh? Wha is he expeced lengh of he busy period (he im during which he server is busy serving cusomers unil i empies ou)? Wha is he oal effecive rae a which cusomers are bing served? And how long do hey spend in he sysem on average? Queues form a convenien framework o address hese and relaed issues. We will be using Kendall s noaions hroughou: e.g. he ype of queue will be denoed by where: M/G/k The firs leer sands for he way cusomers arrive in he queue (M = Markovian, i.e. a Poisson process wih some rae λ). The second leer sands for he service ime of cusomers (G = general, i.e., no paricular assumpion is made on he disribuion of he service ime) The hird leer sands for he number of servers in he sysem (ypical k = 1 or k = ). 1.2 Example: M/M/1 queue Cusomers arrive a rae λ > and are served a rae µ > by a single server. Le X denoe he queue lengh (including he cusomer being served a ime ). Then X is a Markov chain on S = {, 1,...} wih q i,i+1 = λ; q i,i 1 = µ and q i,j = if j i and j i ± 1. Hence X is a birh and deah chain. Theorem 1.1. Le ρ = λ/µ. Then X is ransien if and only ρ > 1, recurren if and only ρ 1, and is posiive recurren if and only if ρ < 1. In he laer case X has an equilibrium disribuion given by π n = (1 ρ)ρ n. If W is he waiing ime in he sysem of a cusomer before being served, hen condiionally given W >, W Exp(µ λ). (1) Proof. The jump chain is given by a biased random walk on he inegers wih reflecion a : he probabiliy of jumping o he righ is p = λ/(λ + µ). Hence he chin X is ransien if and only if p > 1/2 or equivalenly λ > µ, and recurren oherwise. As concerns posiive recurrence, observe ha sup i q i < so here is a.s. no explosion by a resul from he lecures earlier. Hence we look for an invarian disribuion. Furhermore, since X is a birh and deah chain, i suffices o solve he Deailed Balance Equaions, which read: π n λ = π n+1 µ 3

4 for all n. We hus find π n+1 = (λ/µ) n+1 π inducively and deduce he desired form for π n. Noe ha π n is he disribuion of a (shifed) geomeric random variable. (Shifed because i can be equal o ). Suppose now ha a cusomer arrives in he queue a some large ime. Le N be he number of cusomers already in he queue a ha ime. We have W > if and only if N 1, and N has he disribuion of a shifed geomeric random variable. Condiionally on W >, N is hus an ordinary geomeric random variable wih parameer 1 ρ. Furhermore, if N = n, hen n W = where T i are iid exponenial random variables wih rae µ. We deduce from an exercise on Example Shee 1 ha (condiionally on W > ), W is hus an exponenial random variable wih parameer µ(1 ρ) = µ λ. Example 1.2. Wha is he expeced queue lengh a equilibrium? We have seen ha he queue lengh X a equilibrium is a shifed geomeric random variable wih success probabiliy 1 ρ. Hence E(X) = 1 1 ρ 1 = ρ 1 ρ = µ µ λ. 1.3 Example: M/M/ queue Cusomers arrive a rae λ and are served a rae µ. There are infiniely many servers, so cusomers are in fac served immediaely. Le X denoe he queue lengh a ime (which consiss only of cusomers being served a his ime). Theorem 1.3. X is a posiive recurren Markov chain for any λ, µ >. Furhermore he Poisson (ρ) disribuion is invarian where ρ = λ/µ. Proof. The raes are q i,i+1 = λ and q i,i 1 = iµ (since when here are i cusomers in he queue, he oal rae a which any one of hem is being served is iµ, by superposiion). Thus X is a birh and deah chain; hence for an invarian disribuion i suffices o solve he Deailed Balance Equaions: λπ n 1 = nµπ n or i=1 T i π n = 1 λ n µ π n 1 =... = 1 n! ( ) λ n π. µ Hence he Poisson disribuion wih parameer ρ = λ/µ is invarian. I remains o check ha X is no explosive. This is no sraighforward as he raes are unbounded. We will show by hand ha X is in fac recurren. The idea is ha for n sufficienly large, µn > λ so ha X is dominaed by a random walk biased owards o, and is hence recurren. More formally, le N be sufficienly large ha µn 2λ for all n N. If X is ransien hen we have P N+1 (X > N for all imes ) > (2) Bu so long as X N, he jump chain (Y n ) of (X, ) is such ha P(Y n+1 = Y n +1) 1/3 and P(Y n+1 = Y n 1) 2/3. I follows ha we can consruc a (2/3, 1/3) biased random walk Ỹ n such ha Y n Ỹn for all imes n. Bu Ỹ is ransien owards and hence is guaraneed 4

5 o reurn o N evenually. This conradics (2). Hence X is recurren and hus non-explosive. Since i has an invarian disribuion we deduce ha X is also posiive recurren. 1.4 Burke s heorem Burke s heorem is one of he mos inriguing (and beauiful) resuls of his course. Consider a M/M/1 queue and assume ρ = λ/µ < 1, so here is an equilibrium. Le D denoe he number of cusomers who have depared he queue up o ime. Theorem 1.4. (Burke s heorem). A equilibrium, D is a Poisson process wih rae λ, independenly of µ (so long as µ > λ). Furhermore, X is independen from (D s, s ). Remark 1.5. A firs his seems insane. For insance, he server is working a rae µ; ye he oupu is a rae λ! The explanaion is ha since here is an equilibrium, wha comes in mus be equal o wha goes ou. This makes sense from he poin of view of he sysem, bu is hard o comprehend from he poin of view of he individual worker. The independence propery also doesn look reasonable. For insance if no compleed service in he las 5 hours surely he queue is empy? I urns ou we have learn nohing abou he lengh of he queue. Proof. The proof consiss of a really nice ime-reversal argumen. Recall ha X is a birh and deah chain and has an invarian disribuion. So a equilibrium, X is reversible: hus for a given T >, if ˆX = X T we know ha ( ˆX, T ) has he same disribuion as (X, T ). Hence ˆX experiences a jump of size +1 a consan rae λ. Bu noe ha ˆX has a jump of size +1 a ime if and only a cusomer depars he queue a ime T. Since he ime reversal of a Poisson process is a Poisson process, we deduce ha (D, T ) is iself a Poisson process wih rae λ. Likewise, i is obvious ha X is independen from arrivals beween and T. Reversing he direcion of ime his shows ha X T is independen from deparures beween and T. Remark 1.6. The proof remains valid for any queue wih birh and deah queue lengh, a equilibrium, e.g. for a M/M/ queue for arbirary values of he parameers. Example 1.7. In a CD shop wih many lines he service rae of he cashiers is 2 per minue. Cusomers spend 1 on average. How many sales do hey make on average? Tha really depends on he rae a which cusomers ener he shop, while µ is basically irrelevan so long as µ is larger han he arrival rae λ. If λ = 1 per minue, hen he answer would be = Queues in andem Suppose ha here is a firs M/M/1 queue wih parameers λ and µ 1. Upon service compleion, cusomers immediaely join a second single-server queue where he rae of service is µ 2. For which values of he parameers is he chain ransien or recurren? Wha abou equilibrium? Theorem 1.8. Le X, Y denoe he queue lengh in firs (resp. second) queue. (X, Y ) is a posiive recurren Markov chain if and only if λ < µ 1 and λ < µ 2. In his case he invarian disribuion is given by π(m, n) = (1 ρ 1 )ρ m 1 (1 ρ 2 )ρ n 2 5

6 where ρ 1 = λ/µ 1 and ρ 2 = λ/µ 2. In oher words, X and Y are independen a equilibrium and are disribued according o shifed geomeric random variables wih parameers 1 ρ 1, 1 ρ 2. Proof. We firs compue he raes. From (m, n) he possible ransiions are (m + 1, n) wih rae λ (m, n) (m 1, n + 1) wih rae µ 1 if m 1 (m, n 1) wih rae µ 2 if n 1. The raes are bounded so no explosion is possible. We can check by direc compuaion ha πq = if and only π has he desired form, hence he crierion for posiive recurrence. An alernaive, more elegan or concepual proof, uses Burke s heorem. Indeed, he firs queue is an M/M/1 queue so no posiive recurrence is possible unless λ < µ 1. In his case we know ha he equilibrium disribuion is π 1 (m) = (1 ρ 1 )ρ m 1. Moreover we know by Burke s heorem ha (a equilibrium) he deparure process is a Poisson process wih rae λ. Hence he second queue is also an M/M/1 queue. Thus no equilibrium is possible unless λ < µ 2 as well. In which case he equilibrium disribuion of Y is π 2 (n) = (1 ρ 2 )ρ n 2. I remains o check independence. Inuiively his is because Y depends only on Y and he deparure process (D s, s ). Bu his is independen of X by Burke s heorem. More precisely, if X π 1 and Y π 2 are independen, hen Burke s heorem implies ha he disribuion of (X, Y ) is sill given by wo independen random variables wih disribuion π 1 and π 2. Hence (since (X, Y ) is irreducible) i follows ha his is he invarian disribuion of (X, Y ). Remark 1.9. The random variables X and Y are independen a equilibrium for a fixed ime, bu he processes (X, ) and (Y, ) canno be independen: indeed, Y has a jump of size +1 exacly when X has a jump of size 1. Remark 1.1. You may wonder abou ransience or null recurrence. I is easy o see ha if λ > µ 1, or if λ > µ 2 and λ < µ 1 hen he queue will be ransien. The equaliy cases are delicae. For insance if you assume ha λ = µ 1 = µ 2, i can be shown ha (X, Y ) is recurren. Basically his is because he jump chain is similar o a wo-dimensional simple random walk, which is recurren. However, wih hree or more queues in andem his is no longer he case: essenially because a simple random walk in Z d is ransien for d Jackson Neworks Suppose we have a nework of N single-server queues. The arrival rae ino each queue is λ i, 1 i N. The service rae ino each queue is µ i. Upon service compleion, each cusomer can eiher move o queue j wih probabiliy p ij or exi he sysem wih probabiliy p i := 1 j 1 p ij. We assume ha p i is posiive for all 1 i N, and ha p ii =. We also assume ha he sysem is irreducible in he sense ha if a cusomer arrives in queue i i is always possible for him o visi queue j a some laer ime, for arbirary 1 i, j N. Formally, he Jackson nework is a Markov chain on S = N... N (N imes), where if x = (x 1,..., x N ) hen x i denoes he number of cusomers in queue i. If e i denoes he vecor wih zeros everywhere excep 1 in he ih coordinae, hen q(n, n + e i ) = λ i q(n, n + e j e i ) = µ i p ij if n i 1 q(n, n e i = µ i p i if n i 1 6

7 Wha can be said abou equilibrium, ransience and recurrence? The problem seems very difficul o approach: he ineracion beween he queues desroys independence. Neverheless we will see ha we will ge some surprisingly explici and simple answers. The key idea is o inroduce quaniies, which we will denoe by λ i, which we will laer show o be he effecive rae a which cusomers ener queue i. We can wrie down a sysem of equaions ha hese numbers mus saisfy, called he raffic equaions, which is as follows: Definiion We say ha a vecor ( λ 1,..., λ N ) saisfies he raffic equaion if for all 1 i N: λ i = λ i + λ j p ji (3) j i The idea of (3) is ha he effecive arrival rae ino queue i consiss of arrivals from ouside he sysem (a rae λ i ) while arrivals from wihin he sysem, from queue j say, should ake place a rae λ j p ji. The reason for his guess is relaed o Burke s heorem: as he effecive oupu rae of his queue should be he same as he effecive inpu rae. Lemma There exiss a unique soluion, up o scaling, o he raffic equaions (3). Proof. Exisence: Observe ha he marix P = (p ij ) i,j N defines a sochasic marix on {,..., N}. The corresponding (discree) Markov Chain is ransien in he sense ha i is evenually absorbed a zero. Le Z n denoes a Markov chain in discree ime wih his ransiion marix, sared from he disribuion P(Z = i) = λ i /λ, for 1 i N, where λ = i λ i. Since Z is ransien, he number of visis N i o sae i by Z saisfies E(Z i ) <. Bu observe ha E(N i ) = P(Z = i) + = λ i λ + n= j=1 = λ i λ + n= j=1 P(Z n+1 = i) n= N P(Z n = j; Z n+1 = i) N P(Z n = j)p ji = λ i N λ + p ji E(N i ). j=1 Mulipliying by λ, we see ha if λ i = λe(n i ), hen which is he same hing as (3) as p ii =. Uniquenesss: see example shee. λ i = λ i + N λ j p ji We come o he main heorem of his secion. This frequenly appears in liss of he mos useful mahemaical resuls for indusry. j=1 7

8 Theorem (Jackson s heorem, 1957). Assume ha he raffic equaions have a soluion λ i such ha λ i < µ i for every 1 i N. Then he Jackson nework is posiive recurren and π(n) = N (1 ρ i ) ρ n i i=1 defines an invarian disribuion, where ρ i = λ i /µ i. A equilibrium, he processes of deparures (o he ouside) from each queue form independen Poisson processes wih raes λ i p i. Remark A equilibrium, he queue lenghs X i are hus independen for a fixed. This is exremely surprising given how much queues inerac. Proof. This heorem was proved relaively recenly, for wo reasons. One is ha i ook some ime before somebody made he bold proposal ha queues could be independen a equilibrium. The second reason is ha in fac he equilibrium is nonreversible, which always makes compuaions vasly more complicaed a priori. As we will see hese sar wih a clever rick: we will see ha here is a parial form of reversibiliy, in he sense of he Parial Balance Equaions of he following lemma. Lemma Suppose ha X is a Markov chain on some sae space S, and ha π(x) for x S. Assume ha for each x S we can find a pariion of S \ {x}, ino say S1 x,... such ha for all i 1 π(x)q(x, y) = π(y)q(y, x). (4) y S x i Then π is an invarian measure. Definiion The equaions (4) are called he Parial Balance Equaions. Proof. The assumpions say ha for each x you can group he sae ino clumps such ha he flow from x o each clump is equal o he flow from ha clump o x. I is reasonable ha his implies π is an invarian measure. The formal proof is easy: indeed, π(x) q(x, y) = π(x)q(x, y) y x i y S x i i = i y S x i y S x i π(y)q(y, x) = y x π(y)q(y, x) so πq(x) = for all x S. Here we apply his lemma as follows. Le π(n) = N in he heorem). Then define q(n, m) = π(m) q(m, n). π(n) i=1 ρn i i (a consan muliple of wha is We will check ha summing over an appropriae pariion of he sae space, m q(n, m) = q(n, m) which implies he parial balance equaions. m 8

9 Le A = {e i ; 1 i N}. Thus if n S is a sae and m A hen n + m denoes any possible sae afer arrival of a cusomer in he sysem a some queue. Le D j = {e i e j ; i j} { e j }. Thus if n S is a sae and m D j hen n + m denoes any possible sae afer deparure from a cusomer in queue j. We will show: for all n S, q(n, m) = q(n, m) (5) m D j m D j q(n, m) (6) m A q(n, m) = m A which implies ha π saisfies he parial balance equaions and is hus invarian. For he proof of (5), noe ha if m D j hen q(n, n + m) = µ j p j if m = e j, and q(n, n + m) = µ j p ji if m = e i e j. Thus he lef hand side of (5) is q(n, m) = µ j p j + µ j p ji m D j = µ j which makes sense as services occur a rae µ j. Now, Also, i j q(n, n + e i e j ) = π(n + e i e j ) q(n + e i e j, n) π(n) = ρ i ρ j µ i p ij = λ i /µ i µ i p ij ρ j = λ i p ij ρ j. q(n, n e j ) π(n e j) q(n e j, n) π(n) = λ j ρ j We deduce ha he righ hand side of (5) is given by m D j q(n, m) = λ j ρ j + i j λ i p ij ρ j = λ j ρ j = µ j, (by raffic equaions) 9

10 as desired. We now urn o (6). The lef hand side is For he righ hand side, we observe ha m A q(n, n + m) = i λ i. q(n, n + e i ) = π(n + e i) q(n + e i, n) π(n) = ρ i µ i p i = λ i µ i µ i p i = λ i p i. hence he righ hand side of (6) is given by q(n, n + m) = λ i p i m A i = λ i (1 p ij ) i j = λ i λ i p ij i j i = λ i ( λ j λ j ) i j (by raffic equaions) = j λ j, as desired. So π is an invarian disribuion. Since he raes are bounded, here can be no explosion and i follows ha, if ρ i < 1 for every i 1, we ge an invarian disribuion for he chain and hence i is posiive recurren. For he claim concerning he deparures of he queue, see Example Shee Non-Markov queues: he M/G/1 queue. Consider an M/G/1 queue: cusomers arrive in a Markovian way (as a Poisson process wih rae λ) o a single-server queue. The service ime of he nh cusomer is a random variable ξ n, and we only assume ha he (ξ n ) n 1 are i.i.d. As usual we will be ineresed in he queue lengh X, which his ime is no longer a Markov chain. Wha hope is here o sudy is long-erm behaviour wihou a Markov assumpion? Forunaely here is a hidden Markov srucure underneah in fac, we will discover wo relaed Markov processes. Le D n denoe he deparure ime of he nh cusomer. Proposiion (X(D n ), n 1) forms a (discree) Markov chain wih ransiion probabiliies given by p p 1 p 2... p p 1 p 2... p p 1 p 2... p p 1 p 2... where for all k, p k = E[exp( λξ)(λξ) k /k!]. 1

11 Remark The form of he marix is such ha he firs row is unusual. The oher rows are given by he vecor (p, p 1,...) which is pushed o he righ a each row. Proof. Assume X(D n ) >. Then he (n + 1)h cusomer begins is service immediaely a ime D n. During his service ime ξ n+1, a random number A n+1 of cusomers arrive in he queue. Then we have X(D n+1 ) = X(D n ) + A n+1 1. If however X(D n ) =, hen we have o wai unil he (n + 1)h cusomer arrives. Then during his service, a random number A n+1 of cusomers arrive, and we have X(D n+1 ) = X(D n ) + A n+1. Eiher way, by he Markov propery of he Poisson process of arrivals, he random variables A n are i.i.d. and, given ξ n, A n is Poisson (λξ n ). Hence in he saemen. The resul follows. P(A n = k) = E(P(A n = k ξ n )) = E[exp( λξ)(λξ) k /k!] = p k We wrie 1/µ = E(ξ), and call ρ = λ/µ he raffic inensiy. We deduce he following resul: Theorem If ρ 1 hen he queue is recurren: i.e., i will empy ou almos surely. If ρ > 1 hen i is ransien, meaning ha here is a posiive probabiliy ha i will never empy ou. Proof. We will give wo proofs because hey are boh insrucive. The firs one is o use he previous proposiion. Of course X is ransien/recurren in he sense of he heorem if and only if X(D n ) is ransien/recurren (in he sense of Markov chains). Bu noe ha while X(D n ) > i has he same ransiion probabiliies as a random walk on he inegers Z wih sep disribuion A n 1. Hence i is ransien if and only if E(A n 1) > or E(A n ) > 1. Bu noe ha E(A n ) = E(E(A n ξ n )) = λe(ξ) = ρ so he resul follows. The second proof consiss in uncovering a second Markov srucure, which is a branching process. Call a cusomer C 2 an offspring of cusomer C 1 if C 2 arrives during he service of C 1. This defines a family ree. By he definiion of he M/G/1 queue, he number of offsprings of each cusomer is i.i.d. given by A n. Hence he family ree is a branching process. Now, he queue empies ou if and only if he family ree is finie. As we know from branching process heory, his is equivalen o E(A n ) 1 or ρ 1, since E(A n ) = ρ. As an applicaion of he las argumen we give he following example: Example 1.2. The lengh of he busy period B of he M/G/1 queue saisfies E(B) = 1 µ λ. 11

12 To see his, adop he branching process poin of view. Le A 1 denoe he number of offsprings of he roo individual. Then we can wrie A 1 B = ξ 1 + where B i is he lengh of he busy period associaed wih he individuals forming he ih subree aached o he roo. Noe ha A 1 and ξ 1 are NOT independen. Neverheless, given A 1 and ξ 1, he B j are independen and disribued as B. Thus hence i=1 B i A 1 E(B) = E(ξ) + E(E( B i A 1, ξ 1 )) i=1 = E(ξ) + E(A 1 E(B)) = E(ξ) + ρe(b) E(B) = E(ξ) 1 ρ and he resul follows afer some simplificaions. Remark The connecion beween rees and queues is general. Since queues can be described by random walks (as we saw) his yields a general connecion beween branching processes and random walks. This is a very powerful ool o describe he geomery of large random rees. Using relaed ideas, David Aldous consruced a scaling limi of large random rees, called he Brownian coninuum random ree, in he same manner ha simple random walk on he inegers can be rescaled o an objec called Brownian moion. 12

13 2 Renewal Theory 2.1 Inroducion To explain he main problem in his secion, consider he following example. Suppose buses arrive every 1 minues on average. You go o a bus sop. How long will you have o wai? Naural answers are 5 minues, 1 minues, and π/1. This is illusraed by he following cases: if buses arrive exacly every 1 minues, we probabiliy arrive a a ime which is uniformly disribued in beween wo successive arrivals, so we expec o wai 5 minues. Bu if buses arrive afer exponenial random variables, hus forming a Poisson process, we know ha he ime for he nex bus afer any ime will be an Exponenial random variable wih mean 1 minues, by he Markov propery. We see ha he quesion is ill-posed: more informaion is needed, bu i is couner inuiive ha his quaniy appears o be so sensiive o he disribuion we choose. To see more precisely wha is happening, we inroduce he noion of a renewal process. Definiion 2.1. Le (ξ i, i 1) be i.i.d. random variables wih ξ and P(ξ > ) >. Le T n = n i=1 ξ i and se N = max{n : T n }. (N, ) is called he renewal process associaed wih ξ i. We hink of ξ i as he inerval of ime separaing wo successive renewals; T n is he ime of he nh renewal and N couns he number of renewals up o ime. Remark 2.2. Since P(ξ > ) > we have ha N < a.s. Moreover one can see ha N a.s. 2.2 Elemenary renewal heorem The firs resul, which is quie simple, ells us how many renewals have aken place by ime when large. Theorem 2.3. If 1/λ = E(ξ) < hen we have N E(N ) λ a.s.; and λ. Proof. We only prove he firs asserion here. (The second is more delicae han i looks). We noe ha we have he obvious inequaliy: T N() T N()+1. In words is greaer han he ime since he las renewal before, while i is smaller han he firs renewal afer. Dividing by N(), we ge T N() N() N() T N()+1 N(). We firs focus on he erm on he lef hand side. Since N() a.s. and since T n /n E(ξ) = 1/λ by he law of large numbers, his erm converges o 1/λ. The same reasoning applies o he erm on he righ hand side. We deduce, by comparison, ha N() 1 λ a.s. and he resul follows. 13

14 2.3 Size biased picking Suppose X 1,..., X n are i.i.d. and posiive. Le S i = X X i ; 1 i n. We use he poins S i /S n, 1 i n o ile he inerval [, 1]. This gives us a pariion of he inerval [, 1] ino n subinervals, of size Y i = X i /S n. Suppose U is an independen uniform random variable in (, 1), and le Ŷ denoe he lengh of he inerval conaining U. Wha is he disribuion of Ŷ? A firs naural guess is ha all he inervals are symmeric so we migh guess Ŷ has he same disribuion as Y = Y 1, say. However his naïve guess urns ou o be wrong. The issue is ha U ends o fall in bigger inervals han in smaller ones. This inroduces a bias, which is called a size-biasing effec. In fac, i can readily be checked ha P(Ŷ dy) = nyp(y dy). The facor y accouns for he fac ha if here is an inerval of size y hen he probabiliy U will fall in i is jus y. More generally we inroduce he following noion. Definiion 2.4. Le X be a nonnegaive random variable wih law µ, and suppose E(X) = m <. Then he size-biased disribuion ˆµ is he probabiliy disribuion given by ˆµ(dy) = y m µ(dy). A random variable ˆX wih ha disribuion is said o have he size-biased disribuion of X. Remark 2.5. Noe ha his definiion makes sense because ˆµ(dy) = (y/m)µ(dy) = m/m = 1. Example 2.6. If X is uniform on [, 1] hen ˆX has he disribuion 2xdx on (, 1). The facor x biases owards larger values of X. Example 2.7. If X is an Exponenial random variable wih rae λ hen he size-biased disribuion saisfies P( ˆX dx) = x 1/λ λe λx dx = λ 2 xe λx dx so ˆX is a Gamma (2, λ) random variable. In paricular ˆX has he same disribuion as he sum X 1 + X 2 of wo independen Exponenial random variables wih rae λ. 2.4 Equilibrium heory of renewal processes We will now sae he main heorem of his course concerning renewal processes. This deals wih he long-erm behaviour of renewal processes (N, ) wih renewal disribuion ξ, in relaion o he following se of quesions: for a large ime, how long on average unil he nex renewal? How long since he las renewal? We inroduce he following quaniies o answer hese quesions. 14

15 Definiion 2.8. Le A() = T N() be he age process, i.e, he ime ha has elapsed since he las renewal a ime. Le E() = T N()+1 be he excess a ime or residual life; i.e., he ime ha remains unil he nex renewal. Finally le L() = A() + E() = T N()+1 T N() be he lengh of he curren renewal. Wha is he disribuion of L() for large? A naïve guess migh be ha his is ξ, bu as before a size-biasing phenomenon occurs. Indeed, is more likely o fall in a big renewal inerval han a small one. We hence guess ha he disribuion of L(), for large values of, is given by ˆξ. This is he conen of he nex heorem. Theorem 2.9. Le (1/λ) = E(ξ).Then in disribuion as. Moreover, for all y, P(E() y) λ L() ˆξ (7) as and he same resul holds wih A() in place of E(). In fac, y P(ξ > x)dx. (8) (L(), E()) (ˆξ, U ˆξ) (9) in disribuion as, where U is uniform on (, 1) and is independen from ˆξ. The same resul holds wih he pair (L(), A()) insead of (L(), E()). Remark 2.1. One way o undersand ha heorem is ha L() has he size-based disribuion ˆξ and given L(), he poin falls uniformly wihin he renewal inerval of lengh L(). Tha is he meaning of he uniform random variable in he limi (9). The resricion ξ coninuous does no really need o be here. Remark Le us explain why (8) and (9) are consisen. Indeed, if U is uniform and ˆξ has he size-biased disribuion hen P(U ˆξ y) = = = = 1 P(ˆξ y/u)du 1 y/u = λ ( λxp(ξ dx))du λxp(ξ dx) 1 λxp(ξ dx)(1 y/x) (y x)p(ξ dx). 1 {u y/x} du 15

16 E() ξ 3 1 ξ 1 ξ 2 ξ 3 ξ 4 ξ 5 Figure 1: The residual life as a funcion of ime in he discree case. On he oher hand, y λ y P(ξ > z)dz = λ = λ = λ z P(ξ dx)dz P(ξ dx) P(ξ dx)(y x) 1 {z<y,z<x} dz so he random variable U ˆξ indeed has he disribuion funcion given by (8). Skech of proof. We now skech a proof of he heorem, in he case where ξ is a discree random variable aking values in {1, 2,...}. We sar by proving (8) which is slighly easier. Consider he lef-coninuous excess E() = lim s E(s) for =, 1,.... Then, as suggesed by he picure in Figure 1, (E(), =, 1,...) forms a discree Markov chain wih ransiions p i,i 1 = 1 for i 1 and p,n = P(ξ = n + 1) for all n 1. I is clearly irreducible and recurren, and an invarian measure saisfies: π n = π n+1 + π P(ξ = n + 1) hus by inducion we deduce ha π n := P(ξ = m) m n+1 is an invarian measure for his chain. This can be normalised o be a probabiliy measure if E(ξ) < in which case he invarian disribuion is π n = λp(ξ > n). 16

17 We recognise he formula (8) in he discree case where and y are resriced o be inegers. We now consider he slighly more delicae resul (9), sill in he discree case ξ {1, 2,...}. Of course, once his is proved, (7) follows. Observe ha (L(), E(); =, 1,...} also forms a discree ime Markov chain in he space N N and more precisely in he se The ransiion probabiliies are given by if k 1 and S = {(n, k) : k n 1}. p (n,k) (n,k 1) = 1 p (n,) (k,k 1) = P(ξ = k). This is an irreducible recurren chain for which an invarian measure is given by π(n, k) where: for k n 1 and π(k, k 1) = π(n, k 1) = π(n, k) π(m, )P(ξ = k). m= So aking π(n, k) = P(ξ = n) works. This can be rewrien as π(n, k) = np(ξ = n) 1 n 1 { k n 1}. Afer normalisaion, he firs facor becomes P(ˆξ = n) and he second facor ells us ha E() is uniformly disribued on {,... n 1} given L() = n in he limi. The heorem follows. Example If ξ Exp(λ) hen he renewal process is a Poisson process wih rae λ. The formula y y λ P(ξ > x)dx = λ e λx dx = 1 e λy gives us an exponenial random variable for he limi of E(). This is consisen wih he Markov propery: in fac, E() is an Exp(λ) random variable for every. Also, ˆξ = Gamma (2, λ) by Example 2.7. This can be undersood as he sum of he exponenial random variable giving us he ime unil he nex renewal and anoher independen exponenial random variable corresponding o he ime since he las renewal. This is highly consisen wih he fac ha Poisson processes are ime-reversible and he noion of bi-infinie Poisson process defined in Example Shee 2. Example If ξ is uniform on (, 1) hen for y 1 P(E y) = λ y P(ξ > u)du = λ y (1 u)du = 2(y y 2 /2). 17

18 2.5 Renewal-Reward processes We will consider a simple modificaion of renewal processes where on op of he renewal srucure here is a reward associaed o each renewal. The reward iself could be a funcion o he renewal. The formal seup is as follows. Le (ξ i, R i ) denoe i.i.d. pairs of random variables (noe ha ξ and R do no have o be independen) wih ξ and 1/λ = E(ξ) <. Le N denoe he renewal process associaed wih he (ξ i ) and le N R = denoe he oal reward colleced up o ime. We begin wih a resul elling us abou he long-erm behaviour of R which is analogous o he elemenary renewal heorem. Proposiion As, if E( R ) <, R i=1 R i λe(r); and E(R ) λe(r). Things are more ineresing if we consider he curren reward: i.e., r() = E(R N()+1 ). The size-biasing phenomenon has an impac in his seup oo. The equilibrium heory of renewal processes can be used o show he following fac: Theorem r() λe(rξ) Remark The facor ξ in he expecaion E(Rξ) comes from size-biasing: he reward R has been biased by he size ξ of he renewal in which we can find. The facor λ is 1/E(ξ). 2.6 Example: Alernaing Renewal process Suppose a machine goes on and off; on and off; ec. Each ime he machine is on, i breaks down afer a random variable X i. Once broken i akes Y i for i o be fixed by an engineer. We assume ha X i and Y i are boh i.i.d. and are independen of each oher. Le ξ i = X i + Y i which is he lengh of a full cycle. Then ξ i defines a renewal process N. Wha is he fracion of ime he machine is on in he long-run? We can associae o each renewal he reward R i which corresponds o he amoun of ime he machine was on during ha paricular cycle. Thus R i = X i. We deduce from Proposiion 2.14 ha if R is he oal amoun of ime ha he machine was on during (, ), R E(X) E(X) + E(Y ) ; and E(R ) E(X) E(X) + E(Y ). (1) In realiy here is a subley in deriving (1) from Proposiion This has o do wih he fac ha in he renewal reward process he reward is only colleced a he end of he cycle, where as in our definiion R akes ino accoun only he ime he machine was on up o ime : no up o he las renewal before ime. The discrepancy can for insance be conrolled using Theorem

19 Wha abou he probabiliy p() ha he machine is on a ime? Is here a size-biasing effec aking place here as well? I can be shown no such effec needs o be considered for his quesion, as is suggesed by (1) (since E(R ) = p(s)ds). Hence we deduce as. p() E(X) E(X) + E(Y ) 2.7 Example: busy periods of M/G/1 queue Consider a M/G/1 queue wih raffic inensiy ρ < 1. Le I n, B n denoe he lenghs of ime during which he server is successively idle and hen busy. Noe ha (B n, I n ) form an Alernaing Renewal process. (Here i is imporan o consider B n followed by I n and no he oher way around in order o ge he renewal srucure. Oherwise i is no compleely obvious ha he random variables are iid). I follows ha if p() is he probabiliy ha he server is idle a ime, hen E(I) p() E(B) + E(I). Now, by he Markov propery of arrivals, I n Exp(λ) so E(I) = 1/λ. We have also calculaed using a branching process argumen (see Example 1.2) E(B) = 1 µ λ. (11) We deduce ha p() 1 λ µ (12) which is consisen wih he case where he queue is M/M/1 in which case p() π = 1 λ/µ (leing π denoe he equilibrium disribuion of he queue lengh). 2.8 G/G/1 queues and Lile s formula Consider a G/G/1 queue. Le A n denoe he inervals beween arrival imes of cusomers and S n heir service imes. I is no hard o prove he following resul. Theorem Le 1/λ = E(A n ) and le 1/µ = E(S n ), and le ρ = λ/µ. Then if ρ < 1 he queue will empy ou almos surely, while if ρ > 1 hen wih posiive probabiliy i will never become empy. Skech of proof. Le E be he even ha he queue never empies ou. Le A be he arrival process (number of cusomers arrived up o ime ) and le D be he deparure process (number of cusomers serviced up o ime ). Then on E, we have by he elemenary renewal heorem. impossible. Hence P(E) = if ρ < 1. A λ; D µ Hence (on E), if ρ > 1, we have D A which is 19

20 We will now sae and give he skech of he proof of an ineresing resul concerning G/G/1 queues, known as Lile s formula. This relaes he long-run queue lengh, waiing imes of cusomers and arrival raes. To be precise, inroduce he following quaniies: le 1 L() = lim where X s is he queue lengh a ime s, and le 1 W = lim n n n i=1 X s ds where W i is he waiing ime including service of he ih cusomer. A priori i is no clear ha hese limis are well-defined. Theorem (Lile s formula). Assume ρ < 1. The limis defining L and W exis almos surely. Moreover, L = λw where 1/λ = E(A n ). Skech of proof. Since ρ < 1, here are infiniely many imes where he queue is idle and a new cusomer arrives. These insans, call hem T n, form a renewal process. Hence he limi for L() comes from he renewal reward heory (more specifically Proposiion 2.14), where he reward during [T n, T n+1 ] corresponds o R n = Tn+1 T n W i X s ds. Likewise he limi for exisence of W comes from he same resul and a reward R n given by he sum of waiing imes of all cusomers arriving during [T n, T n+1 ], and he elemenary renewal heorem. To prove he relaion in he here, suppose ha each cusomer pays 1 for each minue in he sysem (including when hey are geing served). Hence a ime s he operaor is collecing X s per minue, and he oal colleced up o ime is X sds. On he oher hand, if all cusomers pay upfron when hey ener he queue, hen he amoun colleced is A i=1 W i. We deduce: 1 X s ds = 1 A W i + error where error erm comes from cusomers arriving before ime and whose service ends afer ime. Bu he righ hand side can be rewrien as 1 A i=1 W i = A 1 A i=1 A i=1 W i λw as, while he lef hand side converges o L by definiion. The resul follows. 2

21 Example Waiing ime in an M/M/1 queue. Recall ha he equilibrium disribuion is π n = (1 ρ)ρ n where ρ = λ/µ < 1. Hence in ha queue L = n nπ n = 1 1 ρ 1 = λ µ λ. Hence by Lile s heorem, W = L λ = 1 µ λ. We recover (1), where in fac we had argued ha he waiing ime of a cusomer a large imes was an Exponenial random variable wih rae µ λ. 21

22 3 Populaion geneics 3.1 Inroducion Sample he DNA of n individuals from a populaion. Wha paerns of diversiy/diversiy do we expec o see? How much can be aribued o random drif vs. naural selecion? In order o answer we will assume neural muaions and deduce universal paerns of variaion. Definiion 3.1. The genome is he collecion of all geneic informaion on an individual. This informaion is sored on a number of chromosomes. Each consiss of (usually many) genes. A gene is a piece of geneic maerial coding for one specific proein. Genes hemselves are made up of acid bases: e.g. ATCTTAG... Differen versions of he same gene are called alleles. For insance, o simplify grealy, if here was a gene coding for he colour of he eye we could have he blue allele, he brown allele, ec. To simplify, we will make a convenien abuse of language and speak of an individual when we have in mind a given gene or chromosome. In paricular, for diploid populaions, every member of he populae has wo copies of he same chromosome, which means we have wo corresponding individuals. In oher words, we rea he wo chromosomes in a given member of he populaion as wo disinc individuals. So gene and individual will ofen mean he same hing. 3.2 Moran model Our basic model of a populaion dynamics will be he Moran model. This is a very crude model for he evoluion of a populaion bu neverheless capures he righ essenial feaures, and allows us o give a rigorous reamen a his level. Definiion 3.2. Le N 1. In he Moran model he populaion size is consan equal o N. A rae 1, every individual dies. Simulaneously, a uniformly random chosen individual in he populaion gives birh. Noe: he populaion size says consan hrough his mechanism. In paricular, we allow an individual o give birh jus a he ime he dies. The Moran model can be convenienly consruced in erms of Poisson processes. In he definiion of he Moran model, one can imagine ha when an individual j dies and individual i gives birh, we can hink ha he offspring of i is replacing j. By properies of Poisson processes, if he rae a which an offspring of i replaces j is chosen o be 1/N his gives us a consrucion of he Moran model: indeed he oal rae a which j dies will be N (1/N) = 1, and when his happens he individual i whose offspring is replacing j is chosen uniformly a random. Thus a consrucion of he Moran model is obained by considering independen Poisson processes le (N i,j, ge) for 1 i, j N wih raes 1/N. When N i,j has a jump his means ha individual j dies and is replaced by an offspring of individual i. Corollary 3.3. The Moran model dynamics can be exended o R, by using bi-infinie Poisson processes (N i,j, R). 22

23 3.3 Fixaion Suppose a ime =, a number of X = i of individuals carry a differen allele, called a, while all oher N i individuals carry he allele A. Le X = # individuals carrying allele a a ime, using he Moran model dynamics. Le τ = inf{ : X = or N}. We say ha a fixaes if τ < and X τ = N. We say ha here is no fixaion if τ < and X τ =. Theorem 3.4. We have ha τ < a.s. so hese are he only wo alernaives, and P(X τ = N X = i) = i/n. Moreover, E(τ X = i) = i 1 j=1 N i N N j + Remark 3.5. If p = i/n (, 1) and N hen i follows ha E(τ) is proporional o N and more precisely, E(τ) N( p log p (1 p) log(1 p)). Proof. We begin by observing ha X is a Markov chain wih j=i i j. q i,i+1 = (N i) i/n (13) (he firs facor corresponds o an individual from he A populaion dying, he second o choosing an individual from he a populaion o replace him). Likewise, q i,i 1 = i (N i)/n. (14) Hence he q i,i 1 = q i,i+1, q i = 2i(N i)/n, and X is a Birh and Deah chain whose jump is Simple Random Walk on Z, absorbed a and N. We hus ge he firs resul from he fac ha for Simple Random Walk, P i (T N < T ) = i/n. (15) Now le us wrie τ = N j=1 τ j where τ j is he oal ime spen a j. Noe ha E j (τ j ) = 1 q j E j (# visis o j) = 1 q j P j ( no reurn o j) 1 since he number of visis o j is a geomeric random variable. Now, by decoposing on he firs sep, he probabiliy o no reurn o j is given by P j ( no reurn o j) = j N j by using (15) on he inerval [, j] and [j, N] respecively. Hence Consequeny, E j (τ j ) = 1 q j 2j(N j) N = 1. E i (τ) = j = i E i (τ j ) P i (X = j for some ) E j (τ j ) = j i i j + j<i N i N j. The resul follows. 23

24 3.4 The infinie sies model of muaions Consider he case of poin muaions. These are muaions which change one base ino anoher, say A ino G. When we consider a long sequence of DNA i is exremely unlikely ha wo muaions will affec he same base or sie. We will make one simplifying assumpion ha here are infiniely many sies: i.e., no wo muaions affec he same sie. Concreely, we consider he (bi-infinie) Moran model. We assume ha independenly of he populaion dynamics, every individual is subjec o a muaion a rae u >, independenly for all individuals (neural muaions). Assuming ha no wo muaions affec he same sie, i makes sense o ask he following quesion: Sample 2 individuals from he populaion a ime =. Wha is he probabiliy hey carry he same allele? More generally we can sample n individuals from he populaion and ask how many alleles are here which are presen in only one individual of he sample? Or wo individuals? We call M j (n) = # alleles carried by exacly j individuals in he sample. The infinie sies model ells us ha if we look base by base in he DNA sequence of a sample of individuals, eiher all bases agree in he sample, or here are wo varians (bu no more). Thus, if we are given a able wih he DNA sequences of all n individuals in he sample, M j (n) will be he number of sies (bases) where exacly j individuals carry a base which differs from everyone else in he sample. Example 3.6. Suppose he DNA sequences in a a sample are as follows 1 :... A T T T C G G G T C... 2 :... A G... 3 :... C... 4 :... G C... 5 :... A... 6 : :... C... In his example n = 7. To aid visualisaion we have pu a dash if he base is idenical o ha of he firs individual in he sample. Hence we have M 2 (n) = 2 (second and fourh sies) and M 3 (n) = 1 (las bu one). Our firs resul ells us wha happens in he (unrealisic) case where n = N. Theorem 3.7. Le θ = un. Then E(M j (N)) = θ j. Proof. Muaions occur a a oal rae of u N = θ over in he ime inerval (, ]. Suppose a muaion arises a ime ( > ) on some sie. Wha is he chance ha i affecs exacly j individuals in he populaion a ime? Le X s denoe he number of individuals carrying his muaion a ime + s. Then since muaions don affec each oher, Y evolves like he Markov chain in he previous heorem, i.e., has he Q-marix given by (13) and (14). Hence he chance ha his muaion affecs exacly j individuals in he populaion a ime zero is 24

25 precisely p (1, j) where p (x, y) is he semi-group associaed wih he Q-marix. Thus as desired. E(M j (N)) = 3.5 Kingman s n-coalescen = θe 1 (τ j ) undp (1, j) = θp 1 (X = j for some )E j (τ j ) = θ (1/j) 1, Consider a Moran model defined on R. Sample n individuals a ime. Wha is he genealogical ree of his sample? Since an individual is jus a chromosome, here is jus one paren for any given individual. (This is one of he advanages of making his change of perspecive). Thus for any > here is a unique ancesor for his individual a ime. As goes furher and furher back in ime, i may happen ha he ancesor for wo individuals in he populaion become he same. We speak of a coalescence even. To pu his on a mahemaical fooing, we inroduce he noion of ancesral pariion. This is a pariion Π of he sample (idenified wih {1,..., n}) such ha i and j are in he same block of Π if and only if i and j have he same ancesor a ime. One way o hink abou Π is ha here is a block for each disinc ancesor of he populaion a ime. How does Π evolve as increases? I urns ou ha Π forms a Markov process wih values in P n = { pariions of {1,..., n} }. Theorem 3.8. (Π N/2, ) is a Markov chain in P n wih 1 if π can be obained from π by coagulaing wo of is blocks q π,π = ( ) k 2 if π = π else. This is Kingman s n-coalescen. Proof. I suffices o show ha Π is a Markov chain wih raes (2/N)q π,π. Now, recall ha each block of Π is associaed o an ancesor of he sample a ime. The rae a which his pair of blocks coalesces is 2/N, since if he ancesors are i and j a his ime hen he rae is equal o he sum of he rae for N i,j and N j,i in he Poisson consrucion of he Moran model, i.e., 2/N. All oher ransiions do no ake place, hence he resul. Properies. We lis some immediae properies of Kingman s n-coalescen. 1. Π = {1},..., {n}. 2. For sufficienly large Π = {1,..., n}. The firs such ime is he ime o he MRCA (he mos recen common ancesor) of he sample. 3. Π is a coalescing process. The only possible ransiions involve merging a pair of blocks. Each possible pair of blocks merges a rae 1 in Kingman s n-coalescen (and a rae 2/N for Π iself). 4. If K is he number of Kingman s n-coalescen hen K is a pure deah process wih raes k k 1 given by ( k 2). Moreover he jump chain is independen from K. 25

26 3.6 Consisency and sie frequency specrum A furher ineresing propery is he compaibiliy or sampling consisency. Inuiively, his means ha if we have a sample of size n, hen a subsample of size n 1 behaves as if we had direcly n 1 individuals from he populaion. Mahemaically, his can be expressed as follows. If π is a pariion of [n] = {1,..., n} hen we can speak of π [n 1], he induced pariion of [n 1] obained by resricing π o [n 1]. Proposiion 3.9. Le Π n be Kingman s n-coalescen. Then Π n [n 1] has he law of Kingman s (n 1)-coalescen. Proof. This follows direcly from he consrucion of Kingman s n-coalescen by sampling from he Moran model. Alernaively i can be shown direcly using he ransiion raes via some raher edious calculaions. The sampling consisency has some fundamenal consequences. For insance, recall he infinie sies model, where each individual is subjec o muaion a rae u >, independenly, and all muaions are disinc. Recall he quaniy M j (n), he number of disinc alleles carried by exacly j individuals. Then M j (n) depends only on he muaions which inersec wih he genealogical ree. In oher words, we have a genealogical ree wih each pair of branches coalescing a rae 2/N. Muaions fall on he ree a rae u > per branch and per uni lengh. We have shown in Theorem 3.7 ha for such a ree E(M j (N)) = θ/j where θ = un. By scaling, M j (N) would be unchanged if we consider a Kingman N-coalescen ree on which muaions fall a rae θ/2 per branch and per uni lengh. Bu his is a quesion for which he only parameer is he number N of iniial branches of he ree. We deduce he following resul: Theorem 3.1. For any 1 n N, for θ = un, E(M j (n)) = θ/j. The funcion j θ/j is called he sie frequency specrum of he infinie sies model. Example Biologiss ofen measure he so-called SNP coun S n, or Single Nucleoide Polymorphism. This is he number of sies in he sequence for which here is some variaion in he sequence. Eg in he example a few pages before, S n = 3. Then we deduce from he above heorem ha E(S n ) = θ( ) θ log n. n as n. 3.7 Kingman s infinie coalescen The sampling consisency can be used o deduce he exisence of a unique process (Π, ) aking values in pariions P of N = {1, 2,...} such ha for every n 1, Π [n] has he law of Kingman s n-coalescen. Definiion (Π, ) is called Kingman s infinie coalescen. Iniially we have Π consising of infiniely many singleons. How does i look like for posiive imes? Eg will i ever compleely coalesce? One remarkable phenomenon wih Kingman s coalescen is he following fac. 26

27 Theorem Kingman s coalescen comes down from infiniy: ha is, wih probabiliy one, he number of blocks of Π is finie a any ime >. In paricular, here is a finie ime ζ > such ha Π = {1, 2,...} for ζ. This should be viewed as some kind of big bang even, reducing he number of blocks from infiniy o finiely many in an infiniesimal amoun of ime. Proof. Wrie Π for he number of blocks. By resuls in measure heory, P( Π M) = lim n P( Πn M) = P( τ j ) j=m+1 where τ j is he ime for Π n o drop from j blocks o j 1 blocks. Hence τ j is Exponenial wih rae ( j 2). By Markov s inequaliy P( Π M) E( j=m+1 τ j) 1 1 ( j ) = 2 2 j=m+1 This ends o as M, so he resul follows. 3.8 Inermezzo: Pólya s urn and Hoppe s urn j=m+1 1 j(j 1) The proofs in his secion is no examinable bu we will use he conclusions laer on. Consider he following urn model, due o Pólya. Iniially an urn conains one whie and one black ball. A each subsequen sep, a ball is drawn from he urn. The ball is hen pu back in he urn along wih a ball of he same colour. Le X n denoe he number of black balls in he urn when here are n balls in oal in he urn. Wha is he limiing behaviour of X n? The firs ime hey see his quesion, many people believe ha X n /n will converge o 1/2 when n. Bu he resul is quie differen. Theorem We have ha as n, X n /n U almos surely, where U is a uniform random variable on (, 1). This heorem can be hough of as a rich ge richer phenomenon. Iniially here is a lo of randomness. There are a grea variey of evens ha migh happen during he firs n = 1 draws say. However, once here is a large number of balls in he urn, a law of large number kicks in. For insance if he fracion is p a ha ime, hen he probabiliy o pick a black ball will be p and he probabiliy o pick a while ball will be 1 p. Hence by he law of large numbers he fracion of black balls will end o remain close o p for a very long ime. This reinforces iself and explain why he convergence is almos sure. We will skech a differen (rigorous) proof below. Proof. We sar by making a few simple compuaion. Wha is he probabiliy o ge firs m black balls and hen n m balls (in ha order)? We see ha i is m 1 m 1 m n m n + 1 (m 1)!(n m)! =. (16) (n + 1)! 27

28 The firs facor accoun for drawing all he black balls (whose number increase from 1 o m 1 a he las draw) and he second accouns for hen drawing all whie balls, whose numbers increase from 1 o n m a he las draw. The key observaion is exchangeabiliy: if we were o compue he probabiliy of any oher sequence of draws, also resuling in m + 1 black balls and n + 1 whie balls in he urn, he probabiliy would be unchanged. This is because he boom of he fracion gives he number of balls in he urn (which can only increase by one a each draw) and he fracion gives he number of black or whie balls currenly in he urn. Bu his has o go increase from o 1 m 1 and from 1 o n respecively, albei a differen imes han in he above order. Sill he produc is unchanged. Hence ( ) n (m 1)!(n m)! P(X n+2 = m) = m (n + 1)! = n! m!(n m)! = 1 n + 1, (m 1)!(n m)! (n + 1)! so X n+2 is uniformly disribued over {1,..., n + 1}. I is hence no surprise ha he limi of X n /n, if i exiss, is uniform over [, 1]. To see why in fac he limi does exis, we asser ha X n has he same dynamics as he following (seemingly very differen) process: firs pick a number U (, 1) uniformly a random. Then inser in he urn a black ball wih probabiliy U, and a while ball wih probabiliy 1 U. Indeed, he probabiliy o ge m black balls followed by n m whie balls in ha process is given by 1 u m (1 u) n m du. This inegral can easily be evaluaed and shown o be idenical o (m 1)!(n m)!/(n + 1)!. Clearly he probabiliy is also invarian under permuaion of he sequence, so hese wo processes mus be idenical! I now follows ha by he law of large numbers, X n /n U, almos surely. I seems mad ha he wo processes considered in he proof can in fac be idenical. In he Pólya urn case, here is a complex dependency phenomenon dominaed by rich ge richer. In he second here is no such dependency i is he perhaps he mos basic process of probabiliy: i.i.d. draws, excep ha he parameer for drawing is iself random and is idenical for all draws. Hoppe s urn is a generalisaion of Pólya s urn. This is an urn wih balls of differen colours of mass 1 and a single black ball of mass θ. A each sep, we draw from he urn (wih probabiliy proporional o he mass of he ball). If i is a coloured ball, we pu back he ball in he urn along wih a ball of he same colour. If i is a black ball we pu i back in he urn along wih a ball of a new colour. I can be deduced from Pólya s heorem above ha here will be ulimaely infiniely many colours in he urn. The proporion of balls of a given colour i 1 converges a.s. o a random variable p i, and i p i = 1 a.s. 28