1 Generating functions

Transcription

1 1 Generating functions Even quite straightforward counting problems can lead to laborious and lengthy calculations. These are greatly simplified by using generating functions. 2 Definition 1.1. Given a collection of real numbers (a ) 0, the function G(s) = G a (s) def = is called the generating function of (a ) 0. a s (1.1) Definition 1.2. If X is a discrete random variable with values in Z + def = {0, 1,... }, its (probability) generating function, G(s) G X (s) def = E ( s X) = s P(X = ), (1.2) is just the generating function of the probability mass function of X. One of the most useful applications of generating functions in probability theory is the following observation: Theorem 1.3. If X and Y are independent random variables with values in {0, 1, 2,... } and Z def = X + Y, then their generating functions satisfy 3 G Z (s) = G X+Y (s) = G X (s) G Y (s). Example 1.4. Let X 1, X 2,..., X n be independent identically distributed random variables 4 with values in {0, 1, 2,... } and let S n = X X n. Then G Sn (s) = G X1 (s)... G Xn (s) [ G X (s) n. Example 1.5. Let X 1, X 2,..., X n be i.i.d.r.v. with values in {0, 1, 2,... } and let N 0 be an integer-valued random variable independent of {X } 1. Then def S N = X X N has generating function G SN (s) = G N ( GX (s) ). (1.3) Solution. This is a straightforward application of the partition theorem for expectations. Alternatively, the result follows from the standard properties of conditional expectations: E ( z S ) [ N = E E ( z S N N ) [GX = E( (z) ) N ( = G N GX (z) ). 2 introduced by de Moivre and Euler in the early eighteenth century. 3recall: if X and Y are discrete random variables, and f, g : Z + R are arbitrary functions, then f(x) and g(y ) are independent random variables and E [ f(x)g(y ) = Ef(X) Eg(Y ); 4 from now on we shall often abbreviate this to just i.i.d.r.v. 1

2 Example 1.6. [Renewals Imagine a diligent janitor who replaces a light bulb the same day as it burns out. Suppose the first bulb is put in on day 0 and let X i be the lifetime of the ith light bulb. Assume that the individual lifetimes X i are i.i.d.r.v. s with values in {1, 2,... } having a common distribution with generating function G f (s). Consider the probabilities r n def = P ( a light bulb was replaced on day n ), f def = P ( the first light bulb was replaced on day ). We obviously have r 0 = 1, f 0 = 0, and for n 1, r n = f 1 r n 1 + f 2 r n f n r 0 = f r n. A standard computation implies that G r (s) = 1 + G f (s) G r (s) for all s 1, so that G r (s) = 1/(1 G f (s)). In general, we say a sequence (c n ) n 0 is the convolution of (a ) 0 and (b m ) m 0 (write c = a b), if c n = a b n, n 0, (1.4) The ey property of convolutions is given by the following result Theorem 1.7. [Convolution thm If c = a b, then G c (s) = G a (s) G b (s). Why do we care? If the generating function G a (s) of (a n ) n 0 is analytic near the origin, then there is a one-to-one correspondence between G a (s) and (a n ) n 0 ; namely, a can be recovered via 5 a = 1! d ds G a(s) s=0. (1.5) This result is often referred to as the uniqueness property of generating functions. Example 1.8. Let X Poi(λ) and Y Poi(µ) be independent. Then Z = X + Y is Poi(λ + µ). Solution. A straightforward computation gives G X (s) = e λ(s 1), and Theorem 1.3 implies G Z (s) = G X (s) G Y (s) = e λ(s 1) e µ(s 1) e (λ+µ)(s 1), so that the result follows by uniqueness. A similar argument implies the following result. 5 if a power series G a (s) is finite for s < r with r > 0, then it can be differentiated in the dis s < r. 2

3 Example 1.9. If X Bin(n, p) and Y Bin(m, p) are independent, then X + Y Bin(n + m, p). Another useful property of probability generating function G X (s), is that it can be used to compute moments of X: Theorem If X has generating function G(s), then 6 E [ X(X 1)... (X + 1) = G () (1). Remar The quantity E [ X(X 1)... (X + 1) is called the th factorial moment of X. Notice also that Var(X) = G X(1) + G X(1) ( G X(1) ) 2. (1.6) Proof. Fix s (0, 1) and differentiate G(s) times 7 to get G () (s) = E [ s X X(X 1)... (X + 1). Taing the limit s 1 and using the Abel theorem, 8 we obtain Remar Notice also that G () (s) E [ X(X 1)... (X + 1). lim G X(s) lim E[s X = P(X < ). s 1 s 1 This allows us to chec whether a variable is finite, if we do not now this apriori, see Example 1.14 below. Exercise Let S N be defined as in Example 1.5. Use (1.3) to compute E [ S N and Var [ S N in terms of E[X, E[N, Var[X and Var[N. Now chec your result for E [ S N and Var [ SN by directly applying the partition theorem for expectations. Example A bird lays N eggs, each being pin with probability p and blue otherwise. Assuming that N Poi(λ), find the distribution of the total number K of pin eggs. Solution. a) A standard approach in the spirit of Core A gives the value P(K = ) directly; as a by-product of this computation one deduces that K and M = N K are independent (do this!). b) One can also thin in terms of a two-stage probabilistic experiment similar to one described in Example 1.5. Here K has the same distribution as S N with N Poi(λ) and X Ber(p); consequently, G N (s) = e λ(s 1) and G X (s) = 1 + p(s 1), so that G SN (s) = e λp(s 1), ie., K Poi(λp). 6 here, G () (1) is the shorthand for G () (1 ) lim s 1 G () (s), the limiting value of the th derivative of G(s) at s = 1; 7 As G X (s) E s X 1 for all s 1, the generating function G X (s) can be differentiated many times for all s inside the dis, s < 1. 8 The Abel theorem says: If a is non-negative for all and G a (s) is finite for s < 1, then lim G a (s) = a, whether this sum is finite or equals +. By footnote 7, the Abel s 1 theorem applies to all probability generating functions. 3

4 Exercise A mature individual produces immature offspring according to a probability generating function F (s). a) Suppose we start with a population of immature individuals, each of which grows to maturity with probability p and then reproduces, independently of other individuals. Find the probability generating function of the number of immature individuals in the next generation. b) Find the probability generating function of the number of mature individuals in the next generation, given that there are mature individuals in the parent generation. c) Show that the distributions in a) and b) above have the same mean, but not necessarily the same variance. [Hint: You might prefer to first consider the case = 1, and then generalise. The next example is very important for applications. Example Let X, 1 be i.i.d.r.v. s with the common distribution P(X = 1) = p, P(X = 1) = q = 1 p. Define the simple random wal S n, n 0, via S 0 = 0 and, for n 1, S n = X 1 + X X n. Let T be the first time this random wal hits 1, ie, T def = inf { n 1 : S n = 1 }. Find the generating function of T, ie., G T (s) E [ s T. Solution. Write p = P(T = ), so that G T (s) E[s T = 0 s p. Conditioning on the outcome of the first step, and applying the partition theorem for expectations, we get G T (s) E [ s T = E [ s T X 1 = 1 p + E [ s T X 1 = 1 q = ps + qs E [ s T 2, where T 2 is the moment of first visit to state 1 starting from S 0 = 1. By partitioning on the moment T 1 of the first visit to 0, decompose P(T 2 = m) = Of course, on the event {S 0 = 1}, m 1 P ( T 1 =, T 2 = m ). P(T 2 = m S 0 = 1) P ( S 1 < 1,..., S m 1 < 1, S m = 1 S 0 = 1 ), P(T 1 = S 0 = 1) P ( S 1 < 0,..., S 1 < 0, S = 0 S 0 = 1 ), where by translation invariance the last probability is just P(T = S 0 = 0) = p. We also observe that P ( T 2 = m T 1 = ) = P ( first hit 1 from 0 after m steps ) p m. The partition theorem now implies P(T 2 = m) = m 1 P ( T 2 = m T 1 = ) P(T 1 = ) m 1 p p m = m p p m, 4

5 ie., G T2 (s) = ( G T (s) )2. We deduce that G T (s) solves the quadratic equation ϕ = ps + qsϕ 2, so that 9 G T (s) = 1 1 4pqs 2 2qs = 2ps pqs 2. We finally deduce that P(T < ) G T (1) = 1 p q 2q = { 1, p q, p/q, p < q. In particular, E(T ) = for p < q (because P(T = ) = (q p)/q > 0). For p q we obtain { E(T ) G 1 p q 1 T (1) = =, p > q, p q 2q p q, p = q, ie., E(T ) < if p > q and E(T ) = otherwise. Notice that at criticality (p = q = 1/2), the variable T is finite with probability 1, but has infinite expectation. Example In a sequence of independent Bernoulli experiments with success probability p (0, 1), let D be the first moment of two consecutive successful outcomes. Find the generating function of D. Solution. Every sequence contributing to d n = P(D = n) contains at least two successes. The probability of having exactly two successes is q n 2 p 2 (of course, n 2). Otherwise, let 1 < n 2 be the position of the first success in the sequence; it necessarily is followed by a failure, and the remaining n 1 results produce any possible sequence corresponding to the event {D = n 1}. By independence, we deduce the following renewal relation: and a standard method now gives n 3 d n = q n 2 p 2 + q 1 pq d n 1, G D (s) = p2 s 2 1 qs + pqs2 1 qs G D(s), or G D (s) = p 2 s 2 1 qs pqs 2. A straightforward computation gives G D(1) = 1+p, so that on average it taes 42 p 2 tosses of a standard symmetric dice until the first two consecutive sixes appear. Exercise Generalize the argument above to the first moment of m consecutive successes. Exercise In the setup of Example 1.15, show that d 2 = p 2, d 3 = qp 2, and, conditioning on the value of the first outcome, that d n = q d n 1 + pq d n 2, n > 3. Use these relations to again derive the generating function G D (s). 9 by recalling the fact that G T (s) 0 as s 0; 5

6 Exercise Use the method of Exercise 1.17, to derive an alternative solution to Exercise Compare the obtained expectation to that from Example Exercise A biased coin showing heads with probability p (0, 1) is flipped repeatedly. Let C w be the first moment when the word w appears in the observed sequence of results. Find the generating function of C w and the expectation E [ C w for each of the following words: HH, HT, TH and TT. Probability generating functions can also be used to chec convergence of distributions; this is justified by the following general continuity result: Theorem Let for every fixed n the sequence a 0,n, a 1,n,... be a probability distribution, ie., a,n 0 and 0 a,n = 1, and let G n (s) be the corresponding generating function, G n (s) = 0 a,ns. In order that for every fixed lim a,n = a (1.7) n it is necessary and sufficient that for every fixed s [0, 1), lim G n(s) = G(s), n where G(s) = 0 a s, the generating function of the limiting sequence (a ). Remar The convergence in (1.7) is nown as convergence in distribution! Example If X n Bin(n, p) with p = p n satisfying n p n λ as n, then for every fixed s [0, 1) G Xn (s) ( 1 + p n (s 1) ) n exp{λ(s 1)}, so that the distribution of X n converges to that of X Poi(λ). Exercise For n 1 let X (n), = 1,..., n, be independent Bernoulli random variables with Assume that as n and that P(X (n) p (n) = 1) = 1 P(X (n) δ (n) def = max 1 n p(n) 0 E = 0) = p (n). X (n) λ (0, ). Using generating functions or otherwise, show that the distribution of n X(n) converges to that of a Poisson(λ) random variable. This result is nown as the law of rare events. 6