6 PROBABILITY GENERATING FUNCTIONS

Size: px

Start display at page:

Download "6 PROBABILITY GENERATING FUNCTIONS"

Silas Powell
6 years ago
Views:

1 6 PROBABILITY GENERATING FUNCTIONS Cetain deivations pesented in this couse have been somewhat heavy on algeba. Fo example, detemining the expectation of the Binomial distibution (page 5.1 tuned out to be faily tiesome. Anothe example of had wok was detemining the set of pobabilities associated with a sum, P(X + Y = t. Many of these tasks ae geatly simplified by using pobability geneating functions... Pobability Geneating Functions Intoduction A polynomial whose coefficients ae the pobabilities associated with the diffeent outcomes of thowing a fai die was given on page 5.8: G(η = 0η η η η η η η6 The coefficients of the squae of this polynomial give ise to a highe-ode polynomial whose coefficients ae the pobabilities associated with the diffeent sums which can occu when two dice ae thown. Thee is nothing special about a fai die. The set of pobabilities associated with almost any discete distibution can be used as the coefficients of G(η whose geneal fom is: G(η = P(X = 0η 0 + P(X = 1η 1 + P(X = 2η 2 + P(X = 3η 3 + P(X = 4η 4 + (6.1 This is a powe seies which, fo any paticula distibution, is known as the associated pobability geneating function. Commonly one uses the tem geneating function, without the attibute pobability, when the context is obviously pobability. Geneating functions have inteesting popeties and can often geatly educe the amount of had wok which is involved in analysing a distibution. The cucial point to notice, in the powe seies expansion of G(η, is that the coefficient of η is the pobability P(X =. Popeties of Geneating Functions It is to be noted that G(0 = P(X = 0 and, athe moe impotantly, that: G(1 = P(X = 0 + P(X = 1 + P(X = 2 + = P(X = = 1 In the paticula case of the geneating function fo the fai die: G(1 = = 1 6.1

2 Next, conside G(η togethe with its fist and second deivatives G (η and G (η (the diffeentiation is with espect to η of couse: G(η = P(X = 0η 0 + P(X = 1η 1 + P(X = 2η 2 + P(X = 3η 3 + P(X = 4η 4 + G (η = 1 P(X = 1η P(X = 2η P(X = 3η P(X = 4η 3 + G (η = 2.1 P(X = 2η P(X = 3η P(X = 4η 2 + Now, conside G(1, G (1 and G (1: G(1 = P(X = 0 + P(X = 1 + P(X = 2 + P(X = 3 + P(X = 4 + G (1 = 1 P(X = P(X = P(X = P(X = 4 + G (1 = 2.1 P(X = P(X = P(X = 4 + At this stage, ecall the geneal fomula fo the expectation of an abitay function of a andom vaiable: E ( f(x = f(.p(x = Then, expess G(1, G (1 and G (1 in sigma notation and deive the following esults: G(1 = G (1 = G (1 = P(X = = 1.P(X = = E(X ( 1.P(X = = E ( X(X 1 (6.2 By diffeentiating the geneating function one can diectly obtain the expectation E(X. By diffeentiating again one can obtain E ( X(X 1. These two esults lead to the apid deivation of V(X. Note: G (1 + G (1 ( G (1 2 = E ( X(X 1 + E(X ( E(X 2 = E(X 2 E(X + E(X ( E(X 2 = E(X 2 ( E(X 2 = V(X (6.3 A geneating function is paticulaly helpful when the pobabilities, as coefficients, lead to a powe seies which can be expessed in a simplified fom. With many of the commonlyused distibutions, the pobabilities do indeed lead to simple geneating functions. Often it is quite easy to detemine the geneating function by simple inspection. Some examples will be consideed

3 The Binomial Distibution The set of pobabilities fo the Binomial distibution can be defined as: P(X = = ( n p q n whee = 0, 1,..., n Accodingly, fom (6.1, the geneating function is: ( n G(η = p 0 q n η = 0 (pη 0 q n p 1 q n 1 η 1 + (pη 1 q n p 2 q n 2 η (pη 2 q n 2 + p 3 q n 3 η (pη 3 q n 3 + This last expession is easily ecognised as the expansion of (q + pη n so the geneating function and its fist two deivatives ae: G(η = (q + pη n G (η = n(q + pη n 1 p G (η = n(n 1(q + pη n 2 p 2 Accodingly: G(1 = (q + p n = 1 n = 1 G (1 = n(q + p n 1 p = n.1 n 1 p = np G (1 = n(n 1(q + p n 2 p 2 = n(n 1.1 n 2 p 2 = n(n 1p 2 By (6.2, G (1 = E(X = np. This esult was deived vey much moe tediously on page 5.1. The vaiance is deived fom (6.3: V(X = G (1 + G (1 ( G (1 2 = n(n 1p 2 + np (np 2 = n 2 p 2 np 2 + np n 2 p 2 = np(1 p = npq This esult was deived on page

4 The Geometic Distibution The set of pobabilities fo the Geometic distibution can be defined as: P(X = = q p whee = 0, 1,... Remembe, this epesents successive failues (each of pobability q befoe a single success (pobability p. Note that is unbounded; thee can be an indefinite numbe of failues befoe the fist success. Fom (6.1, the geneating function is: G(η = q 0 pη 0 + q 1 pη 1 + q 2 pη 2 + q 3 pη 3 + = p ( (qη 0 + (qη 1 + (qη 2 + (qη 3 + The item in backets is easily ecognised as an infinite geometic pogession, the expansion of (1 qη 1, so the geneating function and its fist two deivatives ae: G(η = p(1 qη 1 G (η = p(1 qη 2 q G (η = 2p(1 qη 3 q 2 Accodingly: G(1 = p(1 q 1 = p.p 1 = 1 G (1 = p(1 q 2 q = p p 2 q = q p G (1 = 2p(1 q 3 q 2 = 2 p p 3 q2 = 2 q2 p 2 By (6.2, E(X = q p. The vaiance is deived fom (6.3: V(X = G (1 + G (1 ( G (1 2 = 2 q2 p 2 + q p q2 p 2 = q ( q p p + 1 ( q + p = q p p = q p 2 Both the expectation and the vaiance of the Geometic distibution ae difficult to deive without using the geneating function. 6.4

5 The Poisson Distibution The set of pobabilities fo the Poisson distibution can be defined as: P(X = = λ! e λ whee = 0, 1,... This was intoduced as the pobability of mudes in a yea when the aveage ove a long peiod is λ mudes in a yea. As with the Geometic distibution is unbounded; thee can, in pinciple, be an indefinite numbe of mudes in a yea. Fom (6.1, the geneating function is: G(η = λ0 0! e λ η 0 + λ1 1! e λ η 1 + λ2 2! e λ η 2 + λ3 3! e λ η 3 + ( (λη 0 = 0! + (λη1 1! + (λη2 2! + (λη3 3! + e λ The item in backets is easily ecognised as an exponential seies, the expansion of e (λη, so the geneating function and its fist two deivatives ae: G(η = e (λη e λ G (η = λe (λη e λ G (η = λ 2 e (λη e λ Accodingly: G(1 = e λ e λ = 1 G (1 = λe λ e λ = λ G (1 = λ 2 e λ e λ = λ 2 By (6.2, E(X = λ. The vaiance is deived fom (6.3: V(X = G (1 + G (1 ( G (1 2 = λ 2 + λ λ 2 = λ The expectation of the Poisson distibution was deived without geat difficulty on page 4.5 and it was pointed out that λ was the obvious esult since the analysis of the Poisson distibution began by taking λ as the expectation. The vaiance has not been deived befoe and it is inteesting to note that the vaiance and the expectation ae the same. 6.5

6 The Unifom Distibution In all the illustations so fa, a cucial step has been to ecognise some expansion so that the geneating function can be educed to a simple fom. In the case of the distibution fo a fai die, the geneating function cannot be simplified and little benefit accues fom using it to detemine the expectation and vaiance. Nevetheless the geneating function can be used and the following analysis is a final illustation of the use of geneating functions to deive the expectation and vaiance of a distibution. The geneating function and its fist two deivatives ae: G(η = 0η η η η η η η6 G (η = G (η = η η η η η η η η η η η4 Accodingly: G(1 = = 1 G (1 = = G (1 = = = 35 3 By (6.2, E(X = 7 2. The vaiance is deived fom (6.3: V(X = G (1 + G (1 ( G (1 2 = = 12 = The expectation and vaiance fo a fai die wee deived on pages 3.9 and 4.2 and the pesent calculations ae almost identical. This is not a fuitful use of geneating functions. Nevetheless, the geneating function fo a fai die can be vey well woth exploiting in a quite diffeent way

7 The Geneating Function as a Special Expectation The geneal fom of G(η, given in (6.1, is: G(η = P(X = 0η 0 + P(X = 1η 1 + P(X = 2η 2 + P(X = 3η 3 + P(X = 4η 4 + In sigma notation this can be ewitten: G(η = P(X = η Recall again the geneal fomula fo the expectation of an abitay function of a andom vaiable: E ( f(x = f(.p(x = Taking f(x = η X as a function of X: E ( η X = η.p(x = This is G(η and the identity: G(η = E ( η X (6.4 is the stating point fo some impotant theoy... The Sum of two Independent Random Vaiables A Theoem Suppose X and Y ae two independent andom vaiables whose values ae and s whee, s = 0, 1, 2,.... Let G X (η be the geneating function associated with X and G Y (η be the geneating function associated with Y. Fom (6.4: G X (η = E ( η X and G Y (η = E ( η Y Next, conside the geneating function associated with X + Y and call this G X+Y. Again fom (6.4: G X+Y (η = E ( η X+Y Fom the geneal fomula fo the expectation of a function of andom vaiables: G X+Y (η = E ( η X+Y = η +s P(X =, Y = s = s η η s P(X =.P(Y = s Note that P(X =, Y = s may be split only because X and Y ae independent. 6.7 s

8 The function afte the double-sigma sign can be sepaated into the poduct of two tems, the fist of which does not depend on s and the second of which does not depend on. Hence: ( ( G X+Y (η = η.p(x = η s.p(x = s = E ( η X.E ( η Y = G X (η.g Y (η In wods: the geneating function of the sum of two independent andom vaiables is the poduct of the sepaate geneating functions of the andom vaiables. This is a most useful theoem. Two Dice The theoem G X+Y (η = G X (η.g Y (η bings the analysis back to the stating point. When the geneating function fo a fai die was intoduced on page 5.8, it was shown without explanation that squaing this geneating function gives the geneating function fo the sum of two dice. The explanation is now clea. The two dice wee identical so G X = G Y = G and the squae, G 2, gave the geneating function fo X + Y. In summay: s and: G X (η = G Y (η = G(η = 0η η η η η η η6 G X+Y (η = ( G(η 2 = 0η 0 + 0η η η η η η η η η η η η12 Note that although both X and Y ae distibuted Unifom(1,6, the distibution of X + Y is emphatically not unifom. The distibution of X + Y is tiangula with a peak in the middle. If the numbe of dice is inceased, inspection of G 2, G 3, G 4 and so on shows that the shape of the distibution continues to change and tends to that of a symmetical Binomial distibution (one in which p = q = 1 2. Two Binomial Distibutions On eflection, thee has neve been any eason to suppose that combining two Unifom distibutions (in the manne of thowing two dice and adding thei scoes would poduce anothe Unifom distibution. By contast, combining two Binomial distibutions can poduce anothe Binomial distibution... Imagine that you have a pile of identical biased coins, each of which has a pobability p of landing heads when tossed. Suppose you take 3 of these coins fo youself and hand 5 othes to a fiend. 6.8

9 You then toss you coins and count the numbe of heads, clealy a numbe in the ange 0 to 3. You fiend (independently tosses the othe coins and again counts the numbe of heads, a numbe in the ange 0 to 5. You then add you head count to you fiend s head count. Let X be the andom numbe whose value is the numbe of heads you obtain and Y be the andom numbe whose value s is the numbe of heads you fiend obtains. Clealy, X is distibuted Binomial(3, p and Y is distibuted Binomial(5, p. It will be shown that X + Y is distibuted Binomial(8, p. Given that X and Y ae Binomially distibuted: P(X = = ( 3 p q 3 and P(Y = s = ( 5 p s q 5 s s whee is in the ange 0 to 3 and s is in the ange 0 to 5. Fom page 6.3, the geneating functions associated with X and Y ae: G X (η = (q + pη 3 and G Y (η = (q + pη 5 Fom the theoem: G X+Y (η = G X (η.g Y (η = (q + pη 3 (q + pη 5 = (q + pη 8 Now (q + pη 8 is the geneating function fo a andom vaiable which is distibuted Binomial(8, p. Accodingly, combining the two andom vaiables X and Y, which ae distibuted Binomial(3, p and Binomial(5, p, poduces a andom vaiable X + Y which is distibuted Binomial(8, p. On eflection, this esult is hadly supising. A thid paty watching you and you fiend could simply egad each tial as the tossing of 8 independent coins. The fact that the tossing employs two people is neithe hee no thee. The total numbe of heads is clealy distibuted Binomial(8, p. Thee is nothing special about the values 3 and 5 and the analysis eadily genealises to the case whee you have n 1 coins and you fiend has n 2 coins. The only estictions ae that all n 1 + n 2 coins must be independent and identically distibuted. Each coin individually is, of couse, distibuted Binomial(1, p and its geneating function is G(η = (q + pη 1. Fo an abitay n such coins the geneating function is: G n (η = ( G(η n = (q + pη n This is the geneating function of a andom vaiable distibuted Binomial(n, p. Glossay The following technical tem has been intoduced: Pobability Geneating Function 6.9

10 Execises VI 1. A boad game equies you to thow a 6 to stat. How many thows do you expect to have to make befoe the thow which delives you the equied 6? 2. Conside again the battling Knights of Execises II, question 7. Supposing battling Knights became a popula fom of entetainment, pomotes of such tounaments would wish to infom spectatos of the expected length of a battle. Taking a ound as the unit of measue, just what is the expected length of a battle? 3. If a andom vaiable X is distibuted Poisson(λ, show that: E ( X(X 1(X 2... (X k = λ k+1 4. Use geneating functions to find the distibution of X + Y, whee X and Y ae independent andom vaiables distibuted Binomial(n X, p X and Binomial(n Y, p Y espectively, fo the case whee n X and n Y ae abitay positive integes (they may o may not be equal but p X = p Y. In the altenative case, whee n X = n Y but p X p Y, find the mean and vaiance of X + Y. Is the distibution Binomial? 5. [Fom Pat IA of the Compute Science Tipos, 1993] X and Y ae independent andom vaiables having Poisson distibutions with paametes α and β espectively. By using pobability geneating functions, o othewise, pove that X+Y has a Poisson distibution and give its paamete. Find the conditional distibution fo X given that X + Y = n, and give its mean and vaiance. Explain you esult in wods. 6.10

Random Variables and Probability Distribution Random Variable

Random Variables and Probability Distribution Random Variable Random Vaiables and Pobability Distibution Random Vaiable Random vaiable: If S is the sample space P(S) is the powe set of the sample space, P is the pobability of the function then (S, P(S), P) is called