Mathematical Statistics 1 Math A 6330

Transcription

1 Mathematical Statistics 1 Math A 6330 Chapter 3 Common Families of Distributions Mohamed I. Riffi Department of Mathematics Islamic University of Gaza September 28, 2015

2 Outline 1

3 Subjects of Lecture 04 We will cover the following subjects: Discrete Uniform Distribution Hypergeometric Distribution Binomial Distribution Poisson Distribution Negative Binomial Distribution Geometric Distribution Definition A random variable X is said to have a discrete distribution if the range of X, the sample space, is countable. In most situations, the random variable has integer-valued outcomes.

4 Subjects of Lecture 04 We will cover the following subjects: Discrete Uniform Distribution Hypergeometric Distribution Binomial Distribution Poisson Distribution Negative Binomial Distribution Geometric Distribution Definition A random variable X is said to have a discrete distribution if the range of X, the sample space, is countable. In most situations, the random variable has integer-valued outcomes.

5 Discrete Uniform Distribution Definition A random variable X has a discrete uniform(1, N) distribution if P(X = x N) = 1, x = 1, 2,..., N, N where N is a specified integer. This distribution puts equal mass on each of the outcomes 1, 2,..., N.

6 Remark We use the identities k i = i=1 k(k + 1) 2 and k i 2 = i=1 k(k + 1)(2k + 1) 6 to prove that EX = N EX 2 (N + 1)(2N + 1) = 6 (N + 1)(N 1) Var X = 12

7 Hypergeometric Distribution Definition Suppose we have a large urn filled with N identical balls M of which are red and N M are green. We select K balls at random. Let X denote the number of red balls in a sample of size K, then X has a hypergeometric distribution given by ( M )( N M ) x K x P(X = x N, M, K) = ( N, x = 0, 1,..., K. K)

8 Remark Binomial coefficients of the form ( n r) have been defined only if n r, and so the range of X is additionally restricted by the pair of inequalities which can be combined as M x and N M K x, M (N K) x M. Note these identities: ( ) ( ) M M 1 x = M x x 1 and K ( ) ( ) N N 1 = N. K K 1

9 Remark Binomial coefficients of the form ( n r) have been defined only if n r, and so the range of X is additionally restricted by the pair of inequalities which can be combined as M x and N M K x, M (N K) x M. Note these identities: ( ) ( ) M M 1 x = M x x 1 and K ( ) ( ) N N 1 = N. K K 1

10 Example Mean and Variance of the Hypergeometric Distribution. EX m = K x m P(X = x) x=0 K ( )( ) /( ) M N M N = x m x K x K x=1 = KM K 1 ( )( ) /( ) M 1 N M N 1 (y + 1) m 1 N y K 1 y K 1 y=0 = KM N E(Y + 1)m 1, where Y hypergeometric(n 1, M 1, K 1).

11 Example (3.2.1) Suppose that a lot of 25 machine parts is delivered, where a part is considered acceptable only if it passes tolerance. We sample 10 parts and find that none are defective (all are within tolerance). What is the probability of this event if there are 6 defectives in the lot of 25? Solution P(X = 0) = ( 6 19 ) 0)( 10 ) = ( 25 10

12 Example (3.2.1) Suppose that a lot of 25 machine parts is delivered, where a part is considered acceptable only if it passes tolerance. We sample 10 parts and find that none are defective (all are within tolerance). What is the probability of this event if there are 6 defectives in the lot of 25? Solution P(X = 0) = ( 6 19 ) 0)( 10 ) = ( 25 10

13 Binomial Distribution Definition A r.v. X has a Bernoulli(p) distribution, 0 p 1, if { 1 with probability p, X = 0 with probability 1 p. Definition Let Y be the number of successes in n identical Bernoulli trials, each with success probability p, then Y binomial(n, p). The pmf of Y is ( ) n P(Y = y n, p) = p y (1 p) n y, y = 0, 1,..., n. y

14 Binomial Distribution Definition A r.v. X has a Bernoulli(p) distribution, 0 p 1, if { 1 with probability p, X = 0 with probability 1 p. Definition Let Y be the number of successes in n identical Bernoulli trials, each with success probability p, then Y binomial(n, p). The pmf of Y is ( ) n P(Y = y n, p) = p y (1 p) n y, y = 0, 1,..., n. y

15 Example n ( ) n EY m = y m p y (1 p) n y y y=0 n ( ) n 1 = np y m 1 p y 1 (1 p) n y y 1 y=1 n 1 ( ) n 1 = np (x + 1) m 1 p x (1 p) n 1 x x x=0 = npe(x + 1) m 1, where X binomial(n 1, p).

16 Poisson Distribution Definition A random variable X, taking values in the nonnegative integers, has a Poisson(λ) distribution if P(X = x λ) = e λ λ x, x = 0, 1,.... x! Remark P(X = x λ) = e λ x=0 x=0 λ x x! = e λ e λ = 1, where we have used the Taylor series expansion e y = i=0 y i i!.

17 Poisson Distribution Definition A random variable X, taking values in the nonnegative integers, has a Poisson(λ) distribution if P(X = x λ) = e λ λ x, x = 0, 1,.... x! Remark P(X = x λ) = e λ x=0 x=0 λ x x! = e λ e λ = 1, where we have used the Taylor series expansion e y = i=0 y i i!.

18 Exercise If X Poisson(λ), then EX = λ EX 2 = λ(λ + 1) Var X = λ M X (t) = e λ(et 1)

19 Calculating Poisson Probabilities P(X = x) = λ P(X = x 1), x = 1, 2,.... (1) x If Y binomial(n, p), then P(Y = y) = n y + 1 y Set λ = np and, if p is small, we can write p P(Y = y 1). 1 p n y + 1 y p 1 p = np p(y 1) y py λ y. Therefore, P(Y = y) = λ P(Y = y 1), which is Equation (1). y

20 Calculating Poisson Probabilities To complete the approximation, we should prove that P(X = 0) P(Y = 0). Now, for fixed λ, P(Y = 0) = (1 p) n = (1 λ n )n. lim (1 λ n n )n = e λ = P(X = 0). P(Y = 0) = (1 λ n )n e λ = P(X = 0).

21 Negative Binomial Definition In a sequence of independent Bernoulli(p) trials, let the random variable X denote the trial at which the rth success occurs. Then ( ) x 1 P(X = x r, p) = p r (1 p) x r, x = r, r + 1,..., r 1 and we say that X has a negative binomial(r, p) distribution. Remark If Y = X r, the number of failures before the rth success, then ( ) r + y 1 P(Y = y) = p y (1 p) y, y = 0, 1,.... y

22 Negative Binomial Definition In a sequence of independent Bernoulli(p) trials, let the random variable X denote the trial at which the rth success occurs. Then ( ) x 1 P(X = x r, p) = p r (1 p) x r, x = r, r + 1,..., r 1 and we say that X has a negative binomial(r, p) distribution. Remark If Y = X r, the number of failures before the rth success, then ( ) r + y 1 P(Y = y) = p y (1 p) y, y = 0, 1,.... y

23 Remark Note that ( ) r + y 1 y ( ) r = ( 1) y y y ( r)( r 1) ( r y + 1) = ( 1). (y)(y 1) (2)(1) Therefore, we may write the pmf of Y as P(Y = y) = ( 1) y ( r y ) p r (1 p) y, y = 0, 1,.... Note that x ( ) ( ) x 1 x = r. r 1 r

24 Example (Moments of Negative Binomial) Compute the expected value and the variance of a negative binomial random variable with parameters r and p. ( ) x 1 EX k = x k p r (1 p) x r r 1 x=r = r ( ) x x k 1 p r+1 (1 p) x r p r x=r = r ( ) y 1 (y 1) k 1 p r+1 (1 p) y (r+1) p r y=r+1 = r p E(Y 1)k 1, where Y negative binomial(r + 1, p).

25 Example (Moments of Negative Binomial) Compute the expected value and the variance of a negative binomial random variable with parameters r and p. ( ) x 1 EX k = x k p r (1 p) x r r 1 x=r = r ( ) x x k 1 p r+1 (1 p) x r p r x=r = r ( ) y 1 (y 1) k 1 p r+1 (1 p) y (r+1) p r y=r+1 = r p E(Y 1)k 1, where Y negative binomial(r + 1, p).

26 Remark Since Y = X r, it follows that r(1 p) EY = EX r = p r(1 p) Var Y = Var X = p 2 If we define µ = r(1 p) p, then EY = µ and Var Y = µ + 1 r µ2. If r and p 1 such that r(1 p) λ, 0 < λ <, then EY = Var Y = r(1 p) λ, p r(1 p) p 2 λ.

27 Remark Since Y = X r, it follows that r(1 p) EY = EX r = p r(1 p) Var Y = Var X = p 2 If we define µ = r(1 p) p, then EY = µ and Var Y = µ + 1 r µ2. If r and p 1 such that r(1 p) λ, 0 < λ <, then EY = Var Y = r(1 p) λ, p r(1 p) p 2 λ.

28 Example Find the expected value and the variance of the number of times one must throw a die until the outcome 1 has occurred 4 times. Solution Since the random variable of interest is a negative binomial with parameters r = 4 and p = 1 6, it follows that EX = 24 Var X = 4 ( ) 5 6 ( 1 2 = )

29 Example Find the expected value and the variance of the number of times one must throw a die until the outcome 1 has occurred 4 times. Solution Since the random variable of interest is a negative binomial with parameters r = 4 and p = 1 6, it follows that EX = 24 Var X = 4 ( ) 5 6 ( 1 2 = )

30 Inverse Binomial Sampling Example (3.2.6) suppose that in a population of fruit flies we are interested in the proportion having vestigial wings and decide to sample until we have found 100 such flies. The probability that we will have to examine at least N flies is ( ) x 1 P(X N) = p 100 (1 p) x x=n = 1 N 1 x=100 ( x 1 99 ) p 100 (1 p) x 100. For given p and N, we can evaluate this expression to determine how many fruit flies we are likely to look at.

31 Inverse Binomial Sampling Example (3.2.6) suppose that in a population of fruit flies we are interested in the proportion having vestigial wings and decide to sample until we have found 100 such flies. The probability that we will have to examine at least N flies is ( ) x 1 P(X N) = p 100 (1 p) x x=n = 1 N 1 x=100 ( x 1 99 ) p 100 (1 p) x 100. For given p and N, we can evaluate this expression to determine how many fruit flies we are likely to look at.

32 Geometric Distribution Definition The geometric distribution is the simplest of the waiting time distributions and is a special case of the negative binomial distribution, where r = 1. If X geometric(p), then its pdf is P(X = x p) = p(1 p) x 1, x = 1, 2,.... Remark The mean and variance of X geometric(p) are respectively EX = 1 p and Var X = 1 p p 2.

33 Geometric Distribution Definition The geometric distribution is the simplest of the waiting time distributions and is a special case of the negative binomial distribution, where r = 1. If X geometric(p), then its pdf is P(X = x p) = p(1 p) x 1, x = 1, 2,.... Remark The mean and variance of X geometric(p) are respectively EX = 1 p and Var X = 1 p p 2.

34 Memoryless Property of Geometric Distribution Remark X geometric(p) has the memoryless property; that is, P(X > s X > t) = P(X > s t), s > t > 0. (2) To establish (2), note for any integer n that P(X > n) = P(no success in n trials) = (1 p) n. P(X > s X > t) = P(X > s, X > t) P(X > t) = P(X > s) P(X > t) = (1 p) s t = P(X > s t).

35 Memoryless Property of Geometric Distribution Remark X geometric(p) has the memoryless property; that is, P(X > s X > t) = P(X > s t), s > t > 0. (2) To establish (2), note for any integer n that P(X > n) = P(no success in n trials) = (1 p) n. P(X > s X > t) = P(X > s, X > t) P(X > t) = P(X > s) P(X > t) = (1 p) s t = P(X > s t).

36 Example (3.2.7) The geometric distribution is sometimes used to model lifetimes or time until failure of components. For example, if the probability is.001 that a light bulb will fail on any given day, then the probability that it will last at least 30 days is P(X > 30) =.001(1.001) x 1 = (.999) 30 =.970. x=30