6 Stationary Distributions

Transcription

1 6 Stationary Distributions 6. Definition and Examles Definition 6.. Let {X n } be a Markov chain on S with transition robability matrix P. A distribution π on S is called stationary (or invariant) if π = πp, (6.) or euivalently if π j = π i i j, j S. (6.2) i S Thus, in order to find a stationary distribution of a Markov chain with a transition robability matrix P = [ i j ], we need to solve the linear system (6.) (or euivalently (6.2)) together with the conditions: π i = and π i 0 for all i S. Examle 6..2 (2-state Markov chain) Consider the transition matrix: [ ] P =. i Solving the euation πp = π, π = [π 0 π ], which is euivalent to π 0 π = 0, together with π 0 + π =, we obtain π 0 = +, π = +. (6.3) Thus we conclude: (i) If + > 0, there exists a uniue stationary distribution as in (6.3). (ii) If = = 0, a stationary distribution is not uniuely determined. In fact, any distribution π = [π 0, π ] is stationary. Moreover, we see from Examle that if 0 < + < 2, or euivalently, if r <, we have lim n Pn = [ ]. + Then lim π(n) = lim n n π(0)pn = [π 0 (0) π (0)] + [ ] [ = + It is noteworthy that the stationary distribution is obtained as a limit distribution. ]. + Examle 6..3 (3-state Markov chain) We discuss the Markov chain {X n } introduced in Examle If > 0 and b > 0, a stationary distribution is uniue and given by π = [0 0 ]. Examle 6..4 (One-dimensional RW) Consider the -dimensional random walk with right-move robability > 0 and left-move robability = > 0. Let [ π(k) ] be a distribution on Z. If it is stationary, we have π(k) = π(k ) + π(k + ), k Z. (6.4) (Case ). Then a general solution to (6.4) is given by ( ) k ( ) k π(k) = C k + C 2 = C + C 2, k Z. 22

2 This never becomes a robability distribution for any choice of C and C 2. Namely, there is no stationary distribution. (Case 2) =. In this case a general solution to (6.4) is given by π(k) = (C + C 2 k) k = C + C 2 k, k Z. This never becomes a robability distribution for any choice of C and C 2. Namely, there is no stationary distribution. Examle 6..5 (One-dimensional RW with reflection barrier) There is a uniue stationary distribution when <. In fact, ( ) k π(0) = C, π(k) = C, k, where C = 2. If, then there is no stationary distribution. 6.2 Existence and Uniueness Theorem 6.2. A Markov chain over a finite state sace S has a stationary distribution. A simle roof is based on the Brouwer s fixed-oint theorem, for details see the textbooks. Note that the stationary distribution mentioned in the above theorem is not necessarily uniue. Definition We say that a state j can be reached from a state i if there exists some n 0 such that n (i, j) > 0. By definition every state i can be reached from itself. We say that two states i and j intercommunicate if i can be reached form j and j can be reached from i, i.e., there exist m 0 and n 0 such that n (i, j) > 0 and m ( j, i) > 0. For i, j S we introduce a binary relation i j when they intercommunicate. Then becomes an euivalence relation on S : (i) i i; (ii) i j = j i; (iii) i j, j k = i k. In fact, (i) and (ii) are obvious by definition, and (iii) is verified by the Chaman-Kolmogorov euation. Thereby the state sace S is classified into a disjoint set of euivalence classes. In each euivalence class any two states intercommunicate each other. Definition A state i is called absorbing if ii =. In articular, an absorbing state is a state which constitutes an euivalence class by itself. Definition A Markov chain is called irreducible if every state can be reached from every other state, i.e., if there is only one euivalence class of intercommunicating states. Theorem An irreducible Markov chain on a finite state sace S admits a uniue stationary distribution π = [π i ]. Moreover, π i > 0 for all i S. In fact, the roof owes to the following two facts: () For an irreducible Markov chain the following assertions are euivalent: (i) it admits a stationary distribution; (ii) every state is ositive recurrent. In this case the stationary distribution π is uniue and given by π i = E(T i X 0 = i), i S. (2) Every state of an irreducible Markov chain on a finite state sace is ositive recurrent (Theorem 7.3.2). 23

3 6.3 Convergence Examle 6.3. (2-state Markov chain) We recall Examles and If + > 0, the distribution of the above Markov chain converges to the uniue stationary distribution. Consider the case of = =, i.e., the transition matrix becomes [ ] 0 P =. 0 The stationary distribution is uniue. But for a given initial distribution π(0) it is not necessarily true that π(n) converges to the stationary distribution. Roughly seaking, we need to avoid the eriodic transition in order to have the convergence to a stationary distribution. Definition For a state i S, GCD{n ; P(X n = i X 0 = i) > 0} is called the eriod of i. (When the set in the right-hand side is emty, the eriod is not defined.) A state i S is called aeriodic if its eriod is one. Theorem For an irreducible Markov chain, every state has a common eriod. Theorem Let π be a stationary distribution of an irreducible Markov chain on a finite state sace (It is uniue, see Theorem 6.2.5). If {X n } is aeriodic, for any j S we have lim P(X n = j) = π j. n Problem 0 Find all stationary distributions of the Markov chain determined by the transition diagram below. Then discuss convergence of distributions. Problem Let {X n } be the Markov chain introduced in Examle 5.2.7: b a H S D r For n =, 2,... let H n denote the robability of starting from H and terminating at D at n-ste. Similarly, for n =, 2,... let S n denote the robability of starting from S and terminating at D at n-ste. () Show that {H n } and {S n } satisfies the following linear system: H n = ah n + bs n, n 2; H = 0, S =. S n = H n + rs n, 24

4 (2) Let H and S denote the life times starting from the state H and S, resectively. Solving the linear system in (), rove the following identities for the mean life times: 6.4 Page Rank E[H] = nh n = b + + b, E[S ] = ns n = b + b. The hyerlinks among N websites give rise to a digrah (directed grah) G on N vertices. It is natural to consider a Markov chain on G, which is defined by the transition matrix P = [ i j ], where where deg i = { j ; i j} is the out-degree of i. if i j, deg i i j = 0, if i j and i j,, deg i = 0 and j = i, /2 /2 There exists a stationary state but not necessarily uniue. Taking 0 d we modify the transition matrix: Q = [ i j ], i j = d i j + ϵ, ϵ = d N. If 0 d <, the Markov chain determined by Q has necessarily a uniue stationary distribution. Choosing a suitable d <, we may understand the stationary distribution π = [π(i)] as the age rank among the websites. Problem 2 Consider the age rank introduced above. () Let π(i) be the age rank of a site i. Show that π(i) satisfies the following relation and exlain the meaning. π(i) = d N + d j: j i π( j) deg j (2) Show more examles of the age rank and discuss the role of sites which have no hyerlinks, that is, deg i = 0 (in terms of P = [ i j ] such sites corresond to absorbing states). 25

5 7 Toics in Markov Chains I: Recurrence 7. Recurrence Definition 7.. Let i S be a state. Define the first hitting time or first assage time to i by T i = inf{n ; X n = i}. If {n ; X n = i} is an emty set, we define T i =. A state i is called recurrent if P(T i < X 0 = i) =. It is called transient if P(T i = X 0 = i) > 0. Theorem 7..2 A state i S is recurrent if and only if If a state i is transient, we have n (i, i) =. n=0 n (i, i) < n=0 and n (i, i) = n=0 P(T i < X 0 = i). Proof We first ut n (i, j) = P(X n = j X 0 = i), n = 0,, 2,..., f n (i, j) = P(T j = n X 0 = i) = P(X j,..., X n j, X n = j X 0 = i), n =, 2,.... n (i, j) is nothing else but the n ste transition robability. On the other hand, f n (i, j) is the robability that the Markov chain starts from i and reach j first time after n ste. Dividing the set of samle aths from i to j in n stes according to the number of stes after which the ath reaches j for the first time, we obtain n (i, j) = n f r (i, j) n r ( j, j), i, j S, n =, 2,.... (7.) r= We next introduce the generating functions: G i j (z) = In view of (7.) we see easily that Setting i = j in (7.2), we obtain On the other hand, since G ii () = n (i, j)z n, F i j (z) = n=0 f n (i, j)z n. G i j (z) = 0 (i, j) + F i j (z)g j j (z). (7.2) G ii (z) = + F ii (z)g ii (z) G ii (z) = n (i, i), F ii () = n=0 F ii (z). f n (i, i) = P(T i < X 0 = i) we see that two conditions F ii () = and G ii () = are euivalent. The second statement is readily clear. 26

6 7.2 Random Walks on Lattices Examle 7.2. (random walk on Z) Since the random walk starting from the origin 0 returns to it only after even stes, for recurrence we only need to comute the sum of 2n (0, 0). We start with the obvious result: 2n (0, 0) = (2n)! n!n! n n, + =. Then, using the Stirling formula: n! ( n ) n 2πn (7.3) e we obtain 2n (0, 0) πn (4) n. Hence, <,, 2n (0, 0) =, = = /2. n=0 Conseuently, one-dimensional random walk is transient if, and it is recurrent if = = /2. Remark Let {a n } and {b n } be seuences of ositive numbers. We write a n b n if lim n a n /b n =. In this case, there exist two constant numbers c > 0 and c 2 > 0 such that c a n b n c 2 a n. Hence a n and b n converge or diverge at the same time. Examle (random walk on Z 2 ) Obviously, the random walk starting from the origin 0 returns to it only after even stes. Therefore, for recurrence we only need to comute the sum of 2n (0, 0). For twodimensional random walk we need to consider two directions along with x-axis and y-axis. We see easily that 2n (0, 0) = i+ j=n (2n)! i!i! j! j! ( ) 2n = (2n)! 4 n!n! Emloying the formula for the binomial coefficients: n i=0 which is a good exercise for the readers, we obtain Then we have 2n (0, 0) = ( ) 2 n = i ( 2n n ( ) 2n 4 i+ j=n ) 2 ( 4 2n (0, 0) =, which means that two-dimensional random walk is recurrent. n!n! i!i! j! j! = ( 2n n ) ( 4 ) 2n n i=0 ( ) 2 n. i ( ) 2n, (7.4) n ) 2n πn. Examle (random walk on Z 3 ) Let us consider the isotroic random walk in 3-dimension. As there are three directions, say, x, y, z-axis, we have 2n (0, 0) = i+ j+k=n (2n)! i!i! j! j!k!k! ( ) 2n = (2n)! 6 n!n! ( ) 2n 6 i+ j+k=n n!n! i!i! j! j!k!k! = ( 2n n ) ( 6 ) 2n i+ j+k=n ( ) 2 n!. i! j!k! 27

7 We note the following two facts. First, i+ j+k=n n! i! j!k! = 3n. (7.5) n! Second, the maximum value M n = max i+ j+k=n i! j!k! is attained when n 3 i, j, k n 3 + so by the Stirling formula. Then we have Therefore. 2n (0, 0) ( 2n n M n 3 3 2πn 3n ) ( 6 ) 2n 3 n M n 3 3 2π π n 3/2. 2n (0, 0) <, which imlies that the random walk is not recurrent (i.e., transient). 7.3 Positive Recurrence and Null Recurrence If a state i is recurrent, i.e., P(T i < X 0 = i) =, the mean recurrent time is defined: E(T i X 0 = i) = np(t i = n X 0 = i). The state i is called ositive recurrent if E(T i X 0 = i) <, and null recurrent otherwise. Theorem 7.3. The states in an euivalence class are all ositive recurrent, or all null recurrent, or all transient. In articular, for an irreducible Markov chain, the states are all ositive recurrent, or all null recurrent, or all transient. Theorem For an irreducible Markov chain on a finite state sace S, every state is ositive recurrent. Examle The mean recurrent time of the one-dimensional isotroic random walk is infinity, i.e., the one-dimensional isotroic random walk is null recurrent. The roof will be given in Section??. Problem 3 Let {X n } be a Markov chain described by the following transition diagram: = = = where > 0 and > 0. For a state i S let T i = inf{n ; X n = i} be the first hitting time to i. () Calculate P(T 0 = X 0 = 0), P(T 0 = 2 X 0 = 0), P(T 0 = 3 X 0 = 0), P(T 0 = 4 X 0 = 0). = (2) Find P(T 0 = n X 0 = 0) and calculate P(T 0 = n X 0 = 0), np(t 0 = n X 0 = 0). 28

8 8 Toics in Markov Chains II: Absortion 8. Absorbing States A state i is called absorbing if ii = and i j = 0 for all j i. Once a Markov chain hits an absorbing state, it stays thereat forever. Let us consider a Markov chain on a finite state sace S with some absorbing states. We set S = S a S 0, where S a denotes the set of absorbing states and S 0 the rest. According to the above artition, the transition matrix is written as [ ] I 0 P = 0 0 =. S T Then [ ] n [ ] P n I 0 I 0 = = S T T n, where S = S and S n = S n + T n S. To avoid inessential tediousness we assume the following condition (C) For any i S 0 there exist j S a and n such that (P n ) i j > 0. In other words, the Markov chain starting from i S 0 has a ositive robability of absortion. Since S is finite by assumtion, the n in (C) is chosen indeendently of i S 0. Hence (C) is euivalent to the following (C2) There exists N such that for any i S 0 there exist j S a with (P N ) i j > 0. Lemma 8.. Notations and assumtions being as above, lim n T n = 0. S n Proof We see from the obvious relation = (P N ) i j = (P N ) i j + (P N ) i j j S j S 0 j S a and condition (C2) that j S 0 (P N ) i j <, i S 0. Note that for i, j S 0 we have (P N ) i j = (T N ) i j. We choose δ < such that j S 0 (T N ) i j δ < i S 0. Now let i S 0 and n N. We see that (T n ) i j = (T n N ) ik (T N ) k j = (T n N ) ik (T N ) k j δ (T n N ) ik = δ (T n N ) i j. j S 0 j,k S 0 k S 0 j S 0 k S 0 j S 0 29

9 Reeating this rocedure, we have (T n ) i j δ k (T n kn ) i j δ k (P n kn ) i j δ k, j S 0 j S 0 j S where 0 n kn < N. Therefore, lim (T n ) i j = 0, n j S 0 from which we have lim n (T n ) i j = 0 for all i, j S 0. Remark 8..2 It is shown that every state i S 0 is transient. Theorem 8..3 Let π 0 distribution is given by = [α β] be the initial distribution (according to S = S a S 0 ). Then the limit [α + βs 0], where S = (I T) S. Proof The limit distribution is given by [ ] lim π 0P n I 0 = lim [α β] n n S n T n = lim [α + βs n βt n ]. n We see from Lemma 8.. that On the other hand, since S n = S n + T n S we have lim βt n = 0. n S n = (I + T + T T n )S and Hence which shows the result. (I T)S n = (I T n )S. lim S n = lim (I T) (I T n )S = (I T) S, n n Examle 8..4 Consider the Markov chain given by the transition diagram, which is a random walk with absorbing barriers. The transition matrix is given by [ I 0 P = = 0 0 S T 0 0 ], S = [ ] 0, T = 0 [ ]

10 Then S = (I T) S = [ ] 2 Suose that the initial distribution is given by π 0 = [α β γ δ]. Then the limit distribution is [ α + γ + ] 2 δ β + 2 γ + δ 0 0. In articular, if the Markov chain starts at the state 3, setting π 0 = [0 0 0], we obtain the limit distribution [ 2 ] 0 0, which means that the Markov chain is absorbed in the states or 2 at the ratio : 2. Problem 4 Following Examle 8..4, study the Markov chain given by the following transition diagram, where + = Gambler s Ruin We consider a random walk with absorbing barriers at A and B, where A > 0 and B > 0. This is a Markov chain on the state sace S = { A, A +,..., B, B} with the transition diagram as follows: A B We are interested in the absorbing robability, i.e., R = P(X n = A for some n =, 2,... ) = P {X n = A}, S = P(X n = B for some n =, 2,... ) = P {X n = B}. Note that the events in the right-hand sides are not the unions of disjoint events. A samle ath is shown in the following icture: B 0 A 3

11 A key idea is to introduce a similar random walk starting at k, A k B, which is denoted by X n (k). Then the original one is X n = X n (0). Let R k and S k be the robabilities that the random walk X n (k) is absorbed at A and B, resectively. We wish to find R = R 0 and S = S 0. Lemma 8.2. {R k ;, A k B} fulfills the following difference euation: R k = R k+ + R k, R A =, R B = 0. (8.) Similarly, {S k ;, A k B} fulfills the following difference euation: S k = S k+ + S k, S A = 0, S B =. (8.2) Theorem Let A and B. Let {X n } be the random walk with absorbing barriers at A and B, and with right-move robability and left-move robability ( + = ). Then the robabilities that {X n } is absorbed at the barriers are given by (/) A (/) A+B P(X n = A for some n) = (/) A+B,, B A + B, = = 2, (/) A,, P(X n = B for some n) = (/) A+B A A + B, = = 2. In articular, the random walk is absorbed at the barriers at robability. An interretation of Theorem gives the solution to the gambler s ruin roblem. Two layers A and B toss a fair coin by turns. Let A and B be their allotted oints when the game starts. They exchange oint after each trial. This game is over when one of the layers loses all the allotted oints and the other gets A + B oints. We are interested in the robability of each layer s win. For each n 0 define X n in such a way that the allotted oint of A at time n is given by A + X n. Then {X n } becomes a random walk with absorbing barrier at A and B. It then follows from Theorem that the winning robability of A and B are given by P(A) = A A + B, P(B) = B A + B, (8.3) resectively. As a result, they are roortional to the initial allotted oints. For examle, if A = and B = 00, we have P(A) = /0 and P(B) = 00/0, which sounds that almost no chance of A s win. In a fair bet the recurrence is guaranteed by Theorem 2... Even if one has much more losses than wins, continuing the game one will be back to the zero balance. However, in reality there is a barrier of limited money. (8.3) tells the effect of the barrier. It is also interesting to know the exectation of the number of coin tosses until the game is over. Theorem Let {X n } be the same as in Theorem The exected life time of this random walk until absortion is given by A A + B (/) A,, (/) A+B AB, = = 2. 32

12 Proof Let Y k be the life time of a random walk starting from the osition k ( A k B) at time n = 0 until absortion. In other words, Y k = min{ j 0 ; X (k) j We wish to comute E(Y 0 ). We see by definition that For A < k < B we have = A X (k) j = B }. E(Y A ) = E(Y B ) = 0. (8.4) E(Y k ) = In a similar manner as in the roof of Theorem we note that Inserting (8.6) into (8.5), we obtain jp(y k = j). (8.5) j= P(Y k = j) = P(Y k+ = j ) + P(Y k = j ). (8.6) E(Y k ) = jp(y k+ = j ) + jp(y k = j ) j= j= = E(Y k+ ) + E(Y k ) +. (8.7) Thus, E(Y k ) is the solution to the difference euation (8.7) with boundary condition (8.4). This difference euation is solved in a standard manner and we find Setting k = 0, we obtain the result. A + k E(Y k ) = A + B (/) A+k,, (/) A+B (A + k)(b k), = = 2. If = = /2 and A =, B = 00, the exected life time is AB = 00. The gambler A is much inferior to B in the amount of funds (as we have seen already, the robability of A s win is just /0), however, the exected life time until the game is over is 00, which sounds longer than one exects intuitively. Perhas this is because the gambler cannot uit gambling. Problem 5 (A bold gambler) In each game a gambler wins the dollars he bets with robability, and loses with robability =. The goal of the gambler is to get 5 dollars. His strategy is to bet the difference between 5 dollars and what he has. Let X n be the amount he has just after nth bet. () Analyze the Markov chain {X n } with initial condition X 0 =. (2) Comare with the steady gambler discussed in this section, who bets just dollar in each game. 33

13 9 Galton-Watson Branching Processes Consider a simlified family tree where each individual gives birth to offsring (children) and dies. The number of offsrings is random. We are interested in whether the family survives or not. A fundamental model was roosed by F. Galton in 873 and basic roerties were derived by Galton and Watson in their joint aer in the next year. The name Galton-Watson branching rocess is uite common in literatures after their aer, but it would be more fair to refer to it as BGW rocess. In fact, Irénée-Jules Bienaymé studied the same model indeendently already in Definition Let X n be the number of individuals of the nth generation. Then {X n ; n = 0,, 2,... } becomes a discretetime stochasic rocess. We assume that the number of children born from each individual obeys a common robability distribution and is indeendent of individuals and of generation. Under this assumtion {X n } becomes a Markov chain. Let us find the transition robability. Let Y be the number of children born from an individual and set P(Y = k) = k, k = 0,, 2,.... The seuence { 0,, 2,... } describes the distribution of the number of children born from an individual. In fact, what we need is the condition k 0, k =. k=0 We refer to { 0,,... } as the offsring distribution. Let Y, Y 2,... be indeendent identically distributed random variables, of which the distribution is the same as Y. Then, we define the transition robability by i (i, j) = P(X n+ = j X n = i) = P Y k = j, i, j 0, k= and 0, j, (0, j) =, j = 0. 34

14 Clearly, the state 0 is an absorbing one. The above Markov chain {X n } over the state sace {0,, 2,... } is called the Galton-Watson branching rocess with offsring distribution { k ; k = 0,, 2,... }. For simlicity we assume that X 0 =. When 0 + =, the famility tree is reduced to just a ath without branching so the situation is much simler (Problem 6). We will focus on the case where 0 + <, 2 <,..., k <,.... In the next section on we will always assume the above conditions. Problem 6 (One-child olicy) Consider the Galton-Watson branching rocess with offsring distribution satisfying 0 + =. Calculate the robabilities = P(X = 0), 2 = P(X 0, X 2 = 0),..., n = P(X 0,..., X n 0, X n = 0),... and find the extinction robability P {X n = 0} = P(X n = 0 occurs for some n ). 9.2 Generating Functions Let {X n } be the Galton-Watson branching rocess with offsring distribution { k ; k = 0,, 2,... }. Let (i, j) = P(X n+ = j X n = i) be the transition robability. We assume that X 0 =. Define the generating function of the offsring distribution by f (s) = The series in the right-hand side converges for s. We set k s k. (9.) k=0 f 0 (s) = s, f (s) = f (s), f n (s) = f ( f n (s)). Lemma 9.2. (i, j)s j = [ f (s)] i, i =, 2,.... (9.2) Proof By definition, (i, j) = P (Y + + Y i = j) = k + +k i = j k 0,...,k i 0 P(Y = k,..., Y i = k i ). Since Y,..., Y i are indeendent, we have (i, j) = P(Y = k ) P(Y i = k i ) = k + +k i = j k 0,...,k i 0 k + +k i = j k 0,...,k i 0 k ki. Hence, (i, j)s j = which roves the assertion. k + +k i = j k 0,...,k i 0 k ki s j = k s k ki s k i = [ f (s)] i, k =0 k i =0 35

15 Lemma Let n (i, j) be the n-ste transition robability of the Galton-Watson branching rocess. We have n (i, j)s j = [ f n (s)] i, i =, 2,.... (9.3) Proof We rove the assertion by induction on n. First note that (i, j) = (i, j) and f (s) = f (s) by definition. For n = we need to show that (i, j)s j = [ f (s)] i, i =, 2,..., (9.4) Which was shown in Lemma Suose that n and the claim (9.3) is valid u to n. Using the Chaman-Kolmogorov identity, we see that n+ (i, j)s j = (i, k) n (k, j)s j. k=0 Since by assumtion of induction, we obtain n (k, j)s j = [ f n (s)] k n+ (i, j)s j = (i, k)[ f n (s)] k. k=0 The right-hand side coincides with (9.4) where s is relaced by f n (s). Conseuently, we come to n+ (i, j)s j = [ f ( f n (s))] i = [ f n+ (s)] i, which roves the claim for n +. Since X 0 =, P(X n = j) = P(X n = j X 0 = ) = n (, j). In articular, P(X = j) = P(X = j X 0 = ) = (, j) = (, j) = j. Theorem Assume that the mean value of the offsring distribution is finite: m = k k <. k=0 Then we have E[X n ] = m n. Proof Differentiating (9.), we obtain f (s) = k k s k, s <. (9.5) k=0 36

16 Letting s 0, we have On the other hand, setting i = in (9.3), we have Differentiating both sides, we come to lim f (s) = m. s 0 n (, j)s j = f n (s) = f n ( f (s)). (9.6) f n(s) = j n (, j)s j = f n ( f (s)) f (s). (9.7) Letting s 0, we have lim f n(s) = s 0 j n (, j) = lim f s 0 n ( f (s)) lim f (s) = m lim f s 0 s 0 n (s). Therefore, which means that E(X n ) = lim s 0 f n(s) = m n, jp(x n = j) = j n (, j) = m n. In conclusion, the mean value of the number of individuals in the nth generation, E(X n ), decreases and converges to 0 if m < and diverges to the infinity if m >, as n. It stays at a constant if m =. We are thus suggested that extinction of the family occurs when m <. 9.3 Extinction Probability The event {X n = 0} means that the family died out until the nth generation. So = P {X n = 0} is the robability of extinction of the family. Note that the events in the right-hand side is not mutually exclusive but {X = 0} {X 2 = 0} {X n = 0}.... Therefore, it holds that = lim n P(X n = 0). (9.8) If =, this family almost surely dies out in some generation. If <, the survival robability is ositive > 0. We are interested in whether = or not. Lemma 9.3. Let f (s) be the generating function of the offsring distribution, and set f n (s) = f ( f n (s)) as before. Then we have = lim n f n (0). Therefore, satisfies the euation: = f (). (9.9) 37

17 Proof Hence, It follows from Lemma that f n (s) = n (, j)s j. f n (0) = n (, 0) = P(X n = 0 X 0 = ) = P(X n = 0), where the last identity is by the assumtion of X 0 =. The assertion is now straightforward by combining (9.8). The second assertion follows since f (s) is a continuous function on [0, ]. Lemma Assume that the offsring distribution satisfies the conditions: 0 + <, 2 <,..., k <,.... Then the generating function f (t) verifies the following roerties. () f (s) is increasing, i.e., f (s ) f (s 2 ) for 0 s s 2. (2) f (s) is strictly convex, i.e., if 0 s < s 2 and 0 < θ < we have f (θs + ( θ)s 2 ) < θ f (s ) + ( θ) f (s 2 ). Proof f (s) > 0. () is aarent since the coefficient of the ower series f (s) is non-negative. (2) follows by Lemma () If m, we have f (s) > s for 0 s <. (2) If m >, there exists a uniue s such that 0 s < and f (s) = s. Lemma f (0) f 2 (0). Theorem The extinction robability of the Galton-Watson branching rocess as above coincides with the smallest s such that s = f (s), 0 s. Moreover, if m we have =, and if m > we have <. The Galton-Watson branching rocess is called subcritical, critical and suercritical if m <, m = and m >, resectively. The survival is determined only by the mean value m of the offsring distribution. The situation changes dramatically at m = and, following the terminology of statistical hysics, we call it hase transition. Problem 7 Let b, be constant numbers such that b > 0, 0 < < and b + <. Suose that the offsring distribution given by k = b k, k =, 2,..., 0 = k. () Find the generating function f (s) of the off-sring distribution. (2) Set m = and find f n (s). k= Problem 8 Show your own model based on the Galton-Watson branching rocess with m = 0.72 (this is motivated by the total fertility rate of Jaan in 206, that is,.44). Then, by comuter simulation or by numerical comutation, estimate the extinction robability of your model. 38