Discrete Distributions Chapter 6

Discrete Distributions Chapter 6 Negative Binomial Distribution section 6.3 Consider k r, r +,... independent Bernoulli trials with probability of success in one trial being p. Let the random variable X be the trial number k needed to have r-th success. Equivalently in the first k trials there are r successes (no matter when they have occurred) and the k-th trial must be a success. Since the trials are independent, the required probability is found by multiplying the two sub-probabilities: ( ) k p r ( p) k r p r ( ) k p r ( p) k r r where here we are assuming that k r, r +,.... Putting n k r, or equivalently k n + r, we can write the above equality in terms of n : ( ) n + r P (Y n) p r ( p) n n 0,, 2,... where Y X r r But we know from Combinatorics that ( ) ( n+r r n+r ) n, therefore we have ( ) n + r P (Y n) p r ( p) n n 0,, 2,... n Finally by a change of variable p ( n + r P (Y n) n, we can write: ) ( + ) r ( ) n n 0,, 2,... + Recall that therefore: ( ) n + r n (n + r )(n + r 2) (r) n! P (Y n) (n + r )(n + r 2) (r) n! ( ) r ( ) n n 0,, 2,... + +

In this new form, r can be taken to be any positive number (and not just a positive integer). So the negative binomial distribution has two positive parameters > 0 and r > 0. This distribution has an advantage over the Poisson distribution in modeling because it has one more parameter. To be able to use the table of the textbook directly, let us change n to k thoroughly: P (Y k) (k + r )(k + r 2) (r) ( ) r ( ) k k 0,, 2,... + + P (Y k) k terms {}}{ (r)(r + ) (k + r ) k ( + ) k+r k 0,, 2,... We need the values (k+r )(k+r 2) (r) to be able to calculate these probabilities. But note that Γ(k+r) (k+r )(k+r 2) (r)γ(r) (k + r )(k + r 2) (r) Γ(k + r) Γ(k + ) Γ(r) and tables of the values the Gamma function or log-gamma functions are provided in programming languages, so you can calculate the values (k+r )(k+r 2) (r) through the formula Γ(k+r) Γ(k+) Γ(r). Here is the reason for choosing the name Negative Binomial For a moment let us recall the binomial series we have learned in Calculus: ( + z) α + k which is valid for all real number α and all < z <. As an example: 4 + z ( + z) 4 + k (α)(α ) (α k + ) z k ( 4 )( 5 4 ) ( 4 k + ) z k < z < 2

Now change z to z, and substitute r for α to get: ( z) r + ( r)( r ) ( r k+) k ( z) k + (r)(r+) (r+k ) k z k dropping ( ) k + (r+k )(r+k 2) (r) k z k rearranging The series expansion k0 ( z) r ( k+r ) r z k is called the negative binomial expansion. ( z) r k0 ( k + r r ) z k Now we calculate the PGF of the negative binomial distribution. ( ) ( ) k + r r ( P N (z) E(z N ) z k P (N k) z k r + + k0 k0 ( ) r ( ) ( ) k + r k ( ) r z k + r + + k0 k0 ( ) r ( z ) r ( ) r ( + ( z) + + + + ( ) r ( + + + ( z) ) r ) r ( k + r r ( + ( z)) r ( (z )) r ) k ) ( z + ) k Expected value and Variance p(z) ( + ( z)) r p (z) r( + ( z)) r p (z) r(r + ) 2 ( + ( z)) r 2 3

p () r p () r(r + ) 2 E(N) p () r Var(N) p () + p ()( p ()) r + r 2 r( + ) Note. As we see, in the negative binomial distribution, the variance is larger than the expected value while in the Poisson distribution they are equal, therefore in modeling the data in which the sample variance seems to be larger than the sample mean, the negative binomial distribution is preferred over the Poisson distribution. If P X (z) is the PGF of X, then E(X) p () Var(X) p () + p ()( p ()) Theorem (Poisson as a limit of NBinomials). Let X n NBinomial(r n, n ) such that r n and n 0 and r n n λ > 0. Then X n d Poisson(λ) Note. Before proving this theorem, we recall from Calculus that when x 0, then the functions ln( + x) and x are equivalent in the sense that lim x 0 ln(+x) x. To see this equality apply the L Hospital rule. Once we have this, then in the quotients we can substitute x for ln( + x) whenever x 0. In fact, if g(x) is any function of x, then ln( + x) lim lim x g(x) x ln( + x) x ( x g(x) lim x ) ( ln( + x) lim x x ) x lim g(x) x x g(x) Proof of the Theorem. Set λ n r n n. Then from the assumption we have λ n λ. 4

Further: ( ) rn lim n P Xn (z) lim + n ( z) [ ] lim exp r n ln( + n ( z)) [ { }] exp lim r n ln( + n ( z)) [ { exp lim [ exp lim }] ln( n( z)) rn { ln(+ λ n( z) rn rn ) }] [ { exp lim λn( z) rn rn }] [ { }] exp lim λ n ( z) exp(λ(z )) So, we have proved that the PGF of the sequence X n s tends to the PGF of the Poisson(λ) distribution. Whence the claim. 5

Geometric Distribution The Geometric random variable with parameter 0 < q < is a variable with support {, 2,...} such that X k is the event that in a series of Bernoulli trials the first success occurs at time k. Since the first k trials should result in a failure, then P (X k) q ( q) k k, 2,... Note that P (X m) {q( q) m + q( q) m + q( q) m+ + } q( q) m { } + ( q) + ( q) 2 + q( q) m ( q)m ( q) Then for k, 2,... we have : P (X n + k X n) P (X n + k, X n) P (X n) P (X n + k) P (X n) ( q)n+k ( q) n ( q) k This property is called the memoryless property. Given that there are at least n claims, the probability distribution of the number of claims in excess of n does not depend on n : P (X n + k X n) P (X n + k X n) P (X n + k + X n) ( q) k ( q) k+ does not depend on n A random variable X has memoryless distribution if for all x s the conditional distribution (X X x) is the same for all x s. We may consider the Geometric distribution as a special case of Negative Binomial : In fact by changing k to k + and substituting Geometric() P (N k) for p we can write the probabilities in the new form: ( ) ( ) k + + k ( + ) k+ k 0,,... So, a Geometric distribution is a Negative Binomial distribution with r. Note that in this 6

new shape, we have P (N 0) + i.e. the value of the probability function at k 0 equals the probability of success. 7

(a,b,0) class section 6.5 Definition. Let p k P (X k), k 0,, 2,... be the probabilities of a discrete random variable. If there are two numbers a and b satisfying p k p k a + b k equivalently k p k p k ak + b k, 2,... then we say that X belongs to the class (a, b, 0). Note that the following four distributions are in class (a, b, 0) : Distribution a b p 0 Binomial(m, q) q q (m + )a ( q) m Negative Binomial(r, ) Geometric(q) (r )a ( + ) r 0 ( + ) Poisson(λ) 0 λ e λ As it has been shown in the literature, these four distributions are the only non-trivial distributions of the class (a, b, 0) Example. Consider a discrete random variable N of class of the (a, b, 0). Assume that P (X 0) 0., P (X ) 0.3, and P (X 2) 0.3 Calculate P (X 3). Solution. p k ( a + b ) p k k ( ) Putting k in ( ) we get: p (a + b)p 0 0.3 (a + b)(0.) a + b 3 Putting k 2 in ( ) we get: 8

p 2 (a + 2 b)p 0.3 (a + 2 b)(0.3) a + 2 b By solving the two equations thus found, we will have: a and b 4. Then the equality (8) reduces to Put k 3 in ( ) to have: p k ( + 4 ) p k k ( ) p 3 ( + 4 3 )p 2 p 3 3 p 2 3 (0.3) 0.0 Example. The Independent Insurance Company insures 25 risks, each with a 4% probability of loss. The probabilities of loss are independent. On average, how often would 4 or more risks have losses in the same year? A. Once in 3 years B. Once in 7 years C. Once in 39 years D. Once in 60 years E. Once in 72 years Solution. The distribution of loss is Binomial(m 25, q 0.04). Either find the probabilities through elementary formula p k m(m ) (m k+) q k ( q) m k or use the recursive formula: a q q 0.047 b (m + )a.08333 p 0 ( q) m (0.96) 25 0.3604 p (a + b)p 0 0.3754 p 2 (a + b 2 )p 0.877 p 3 (a + b 3 )p 2 0.0600 desired probability (p 0 + p + p 2 + p 3 ) 0.9835 0.065 4 or more risks have losses in the same year on average once in: /.065 60.6 years. Example. The distribution of accidents for 84 randomly selected policies is as follows: 9

Number of accidents Number of Policies 0 32 26 2 2 3 7 4 4 5 2 6 Total 84 Which of the following models best represents these data? (A) (B) (C) (D) (E) Negative binomial Discrete uniform Poisson Binomial Either Poisson or Binomial Solution. Number of accidents Number of Policies k p k p k 0 32 26 0.8 2 2 0.92 3 7.75 4 4 2.29 5 2 2.50 6 3.00 Total 84 0

Since the values k p k p k increase, a negative binomial distribution is best fit. Example. A discrete probability distribution has the following properties: (i) p k c( + /k)p k for k, 2,... (ii) p 0 0.5. Calculate c. Solution. This distribution is of class (a, b, 0) with a b c. Since the probabilities are non-negative we must have c 0. A value of zero for c result in p p 2 0 which together with p 0 0.5 do not result in the sum of probabilities being equal to. So we have c > 0 and then a > 0. So the distribution is of negative binomial. For the negative binomial we have a and b (r ). Then a b r r 2 p 0 0.5 0.5 ( + ) r ( + ) 2 2 0.442 c a + 0.442.442 0.293 Example. You are given the following: A portfolio consists of 0 independent risks. The distribution of the annual number of claims for each risk in the portfolio is given by a Poisson distribution with mean λ 0.. Determine the probability of the portfolio having more than claim per year. A. 5% B. 0% C. 26% D. 37% E. 63% Solution. Since the sum of independent Poisson r.v. s is distributed as Poisson, this portfolio has a Poisson distribution with parameter λ (0)(0.). So the probability of more than

one claim in one year is ( (p 0 + p ) e + e ) 0.264 2

Truncation and modification at zero section 6.6 The discrete distributions in the class (a, b, 0) all have positive probability at zero p(0) > 0. But if in a study the number of claims is under study for onlt those losses that have resulted in a claim, then the minimum observed value is and therefore p(0) 0. We can create such a counting distribution from a class (a, b, 0) by assigning the zero value to p(0) and dividing the remaining probabilities by p(0). Such a new distribution is called a zero-truncated distribution. In some insurance situations the chance of no claim is high so we may have a counting data that needs a large value of p(0) to be properly modeled. Therefore we may modify the counting model by assigning a large value to p(0) and then modify the other values of p(k) so as to have a probability distribution. This new distribution is called zero-modified. Note that in both cases of zero-truncated distribution and zero-modified distribution, the old probabilities {p(), P (2),...} are all multiplied by a constant, therefore the ratio remains the same as before for k 2, 3,..., p(k) p(k ) p(k) p(k ) a + b k k 2, 3,... A distribution with this property is said to of class (a, b, ). So by modifying a (a, b, 0) class member we get a (a, b, ) class member. Definition. A discrete random variable X {0,, 2,...} is said to belong to class (a, b, ) if its density function satisfies the equality p(k) ( a + b ) p(k ) k 2, 3,... k Note. The class (a, b, ) satisfies the same recursive form as the class (a, b, 0) does, but in the case of (a, b, ) the value p(0) is not as part of recursion. 3

Let us denote the probability function of (a, b, 0) by p(k) and that of (a, b, ) by p M (k). By assigning any number from γ [0, ) to p M (0), we then define p M by (zero-modified distribution) p M γ k 0 (k) γ p(0) p(k) k, 2,... We here verify that this is indeed a probability function: p M (0) + p M γ (k) γ + p(0) p(k) γ + γ p(0) k k γ + γ ( p(0)) γ + ( γ) p(0) The special case of γ 0 : (zero-truncated distribution) p T (k) p(k) k 0 k 0 p(0) p(k) k, 2,... Example. Let X be distributed as Geometric( 9). Calculate (i) The zero-truncated distribution. (ii) The zero-modified distribution with P M (0) 0.3 Solution. Part (i): p(0) + 0 p(0) 0.9 0 9 k 9k ( + ) k+ 0 k+ p T 0 k 0 (k) p(0) p(k) 0 9 9 k 9k 0k+ 0 k 9 k 0 k k, 2,... (zero-truncated distribution) 4

Part (ii): γ p(0) 0.7 0.9 7 9 γ p(0) p(k) 7 9 0 k+ 7(9k ) 0 k+ p M 0.3 k 0 (k) 7(9 k ) k, 2,... (zero-modified distribution) 0 k+ 9 k In the following discussion, for the sake of simplicity, we denote the base probabilities by {p 0, p,...} and the resulted probabilities due to truncation or modification by {q 0, q,...}. Theorem. Let N be in the class (a, b, 0) having the MGF M(t) and the PGF Q(z). Then the MGF and PGF of the the modified (and truncated) random variable are equal to ( ) M (t) γ p 0 p 0 + γ p 0 M(t) a weighted average ( ) Q (t) γ p 0 p 0 + γ p 0 Q(z) a weighted average 5

Proof. We only prove the first equality as the proof of the second one is similar. M (t) k0 ekt p M (k) γ + γ p 0 k ekt p k γ + γ p 0 { k0 ekt p k p 0 } { } γ + γ p 0 M(t) p 0 γ p 0 γ p 0 + γ p 0 M(t) γ p 0 p 0 + γ p 0 M(t) Note. The constant function is the MGF of the degenerate random variable which give the probability to k 0. So, the above equalities show that the modified distribution is the mixture of this degenerate distribution and the base distribution. 6

Extended Truncated Negative Binomial and Logarithmic Distributions Definition. By the extended truncated negative binomial distribution we mean the discrete distribution whose probability function is of the form: p k ( a + b ) k pk k 2, 3,... p 0 0 where a, b (r ), r 0, r >, > 0 Note. The requirements r > and > 0 are needed to ensure that this indeed defines a probability function as the following lemma shows. Lemma. ( ) k (r + )(r + 2) (r + k ) p k p + k 2, 3,... Proof. From the defining recursive formula, we have ( (r ) p k + + k( + ) Writing this for k 2, 3,... gives us: Multiplying these gives the result. ) k + (r ) (r + k ) p k p k p k k( + ) k( + ) p 2 p 3. p k r+ 2 p r+2 3 p 2 r+k k p k + r + k k p k Lemma. For any choice of 0 < p we have k p k <. Proof. Equivalently we must show that the series with positive terms ( ) k (r + )(r + 2) (r + k ) p k p + p + k k2 7

is convergent. We use the so-called ratio-test to achieve this. Set ( ) k (r + )(r + 2) (r + k ) a k + Then lim a k+ a k lim k k + r + k k + so, according to the ratio-test, the series is convergent. + < Note. Here is a closed form for p : k p ( ) k k p + p (r+)(r+2) (r+k ) k2 ( ) p + p ( ) k ( r)( r )( r 2) ( r k+) r k2 ( ) {[ p + p r + ( ) ] } k ( r)( r )( r 2) ( r k+) k r ( ) { ( ) } r p + p r r ( ) { } p + p r ( + ) r r ( ) { } p + p () r+ (+r) r { } p + p () r+ (+r) r { } p + ()r+ (+r) r { } p () r+ r Therefore p r () r+ 8

and for this one needs the assumption r 0 otherwise the denominator would be zero. Note. We recall from Calculus that d dr ( + )r+ ( + ) r+ ln( + ) Then by applying the L Hospital rule we will have: lim p lim r 0 r 0 So then for k 2 we have r ( + ) r+ lim r 0 lim r 0 p k lim r 0 p ( ( + ) r+ ln( + ) ( + ) ln( + ) ) k (r+)(r+2) (r+k ) ( ) k () (k ) () ln() ( ) k ln() k This and lim r 0 p () ln() lim p k k can be put together : ( + ) k ln( + ) k k, 2,... But the values on the right-hand side are positive numbers that sum to : ( ) k ( ) k ( ) ( ( )) k ln() k ln() k k ln() ln ( ln() ) ( ( )) ( ln ln() ) ln( + ) definition. The distribution with probability function ( ) k p(k) + ln( + ) k is called Logarithmic distribution Note. The class (a, b, ) has 4 members. The list of members is given on page 94 of the textbook and the distributional information about these distributions is given at the end of the textbook. 9