Chapter 8. Some Approximations to Probability Distributions: Limit Theorems

Transcription

1 Chapter 8. Some Approximations to Probability Distributions: Limit Theorems Sections : Convergence in Probability and in Distribution Jiaping Wang Department of Mathematical Science 04/22/2013, Monday

2 Outline Convergence in Probability Convergence in Distribution

3 Part 1. Convergence in Probability

4 Introduction Suppose that a coin has probability p, with 0 p 1, of coming up heads on a single flip. Suppose that we flip the coin n times, what can we say about the fraction of heads observed in the n flips? For example, if p=0.5, we draw different numbers of trials in a simulation, the result is given in the table n % % From here, we can find when n, the ratio is closer to 0.5 and thus the difference is closer to zero.

5 Definition 8.1 In mathematical notations, let X denote the number of heads observed in the n tosses. Then E(X)=np, V(X)=np(1-p). One way to measure the closeness of X/n to p is to ascertain the X probability that the distance p will be less than a preassigned small value ε so that P X n n p < ε 1. Definition 8.1: The sequence of random variables X 1,X 2,.., X n is said to convergence in probability to the constant c, if for every positive number ε, lim n P X n c < ε = 1.

6 Theorem 8.1 Weak Law of Large Numbers: Let X 1,X 2,.., X n be independent and identical distributed random variables, with E(X i )=μ and V(X i )=σ 2 < for each i=1,, n. Let X n = 1 n X n i=1 i. Then, for any positive real number ε, lim P X n n μ ε = 0 Or lim P X n n μ < ε = 1. Thus, X n converges in probability toward μ. The proof can be shown based on the Tchebysheff s theorem with X replaced by X n and σ 2 by σ 2 /n, then let k = ε n. σ

7 Theorem 8.2 Suppose that X n converges in probability toward μ 1 and Y n converges in probability toward μ 2. Then the following statements are also true. 1. X n +Y n converges in probability toward u 1 +u X n Y n converges in probability toward u 1 u X n /Y n converges in probability toward u 1 /u 2, provided u X n converges in probability toward u 1, provided P(X n 0)=1.

8 Example 8.1 Let X be a binomial random variable with probability of success p and number of trials n. Show that X/n converges in probability toward p. Answer: We have seen that we can write X as Y i with Yi=1 if the i-th trial results in Success, and Yi=0 otherwise. Then X/n=1/n Y i. Also E(Y i )=p and V(Y i )=p(1-p). Then the conditions of Theorem 8.1 are fulfilled with μ=p and σ2=p(1-p)< and thus we can conclude that, for any positive ε, lim n P( X/n-p ε)=0.

9 Example 8.2 Suppose that X 1, X 2,, X n are independent and identically distributed random Variables with E(X i ) = μ 1, E(X i2 ) = μ 2, E(X i3 ) = μ 3, E(X i4 ) = μ 4 and all assumed finite. Let S 2 denote the sample variance given by S 2 = 1 n X i X 2. Show that S 2 converges in probability to V(X i ). n Answer: Notice that S 2 = 1 X n i=1 i 2 X 2 where X = 1 X n i=1 i. The quantity 1 n X n i=1 i 2 is the average of n independent and identical distributed variables of the form X 2 i with E(X i2 )= μ 2, and V (X i2 )= μ 4 - μ 22, which is finite. Thus Theorem 8.1 tell us that 1 n X n i=1 i 2 converges to μ 2 in probability. Finally, based on n Theorem 8.2, we can have S 2 = 1 X n i=1 i 2 X 2 converges in probability to μ 2 - μ 2 1 =V(X i ). This example shows that for large samples, the sample variance has a high probability of being close to the population variance. n

10 Part 2. Convergence in Distribution

11 Definition 8.2 In the last section, we only study the convergence of certain random variables Toward constants. In this section, we study the probability distributions of certain type random variables as n tends toward infinity. Definition 8.2: Let X n be a random variable with distribution function F n (x). Let X be a random variable with distribution function F(x). If lim n F n (x)=f(x) At every point x for which F(x) is continuous, then X n is said to converge in distribution toward X. F(x) is called the limiting distribution function of X n.

12 Example 8.3 Let X 1, X 2,, X n be independent uniform random variables over the interval (θ, 0) for a negative constant θ. In addition, let Y n =min(x 1, X 2,, X n ). Find the limiting distribution of Y n. Answer: The distribution function for the uniform random variable Xi is 0, x < θ F(X i ) = P(X i x) = x θ θ, θ x 0 1, x > 0. We know G y = P Y n y = 1 P Y n > y = 1 P min X 1, X 2,, X n > y = 1 P X 1 > y P X 2 > y P X n > y = 1 1 F X y n 0, y < 0 0, y < 0 = 1 y n, θ y 0 so we can find lim G(y) = lim 1 y n, θ y 0 θ n n θ 1, y > 0. 1, y > 0. 0, y < θ = 1, y θ.

13 Theorem 8.3 Let X n and X be random variables with moment-generating functions M n (t) and M(t), respectively. If lim n M n (t)=m(t) For all real t, then X n converges in distribution toward X.

14 Example 8.4 Let X n be a binomial random variable with n trials and probability p of success on each trial. If n tends toward infinity and p tends zero with np remaining fixed. Show that X n converges in distribution toward a Poisson random variable. Answer: We know the moment-generating function for the binomial random variables X n, M n (t) is given as M n t = q + ppp n = 1 + p e t 1 n aa q = 1 p = 1 + λ n et 1 n based on np=λ. Recall that lim 1 + k n = ek. Letting k=λ(e t -1), we have n n lim M n t = exp λ e t 1 n which is the moment generating function of the Poisson random variable. As an example, when n=10 and p=0.1, we can find the true probability from the binomial Distribution is for X is less than 2 and the approximate value from the Poisson Is , they are very close. So we can approximate the probability from binomial Distribution by the Poisson distribution when n is large and p is small.

15 Example 8.5 In monitoring for a pollution, an experiment collects a small volume of water and counts the number of bacteria in the sample. Unlike earlier problems, we have only one observation. For purposes of approximating the probability distribution of counts, we can think of the volume as the quantity that is getting large. Let X denote the bacteria count per cubic centimeter of water and assume that X has a Poisson probability distribution with mean λ, which we do by showing that Y = X λ λ converges in distribution toward a standard normal random variable as λ tends toward infinity. Specifically, if the allowable pollution in a water supply is a count of 110 bacteria per cubic centimeter, approximate the probability that X will be at most 110, assuming that λ=100.

16 Solution Answer: We know the mgf for Poisson random variable X is M X (t) = exp [λ(ee 1)], thus We can have the mgf of Y as M Y t = exp t λ exp [λ(exp(t/ λ)-1)]. The term (exp(t/ λ)-1) can be written as exp (t/ λ) 1= t/ λ+ t2 + t3 + 2λ 6λ λ Thus M Y (t)=exp[ t λ+ λ(t/ λ+ t2 + t3 + + t3 + )] 2λ 6λ λ )]=exp[t2 2 6 λ When λ, M Y (t) exp(t 2 /2) which is the mgf of the standard normal distribution. So we can approximate the probability of the Poisson random variable by the standard normal distribution when λ is large enough (for example, λ 25). P X 110 = P X λ λ = P Y 1 =