arxiv: v1 [math.st] 5 Jul 2007

Transcription

1 EXPLICIT FORMULA FOR COSTRUCTIG BIOMIAL COFIDECE ITERVAL WITH GUARATEED COVERAGE PROBABILITY arxiv:77.837v [math.st] 5 Jul 27 XIJIA CHE, KEMI ZHOU AD JORGE L. ARAVEA Abstract. In this paper, we derive an explicit formula for constructing the confidence interval of binomial parameter with guaranteed coverage probability. The formula overcomes the limitation of normal approximation which is asymptotic in nature and thus inevitably introduce unknown errors in applications. Moreover, the formula is very tight in comparison with classic Clopper-Pearson s approach from the perspective of interval width. Based on the rigorous formula, we also obtain approximate formulas with excellent performance of coverage probability.. Classic Confidence Intervals The construction of confidence interval of binomial parameter is frequently encountered in communications and many other areas of science and engineering. Clopper and Pearson [3] has provided a rigorous approach for constructing confidence interval. However, the computational complexity involved with this approach is very high. The standard technique is to use normal approximation which is not accurate for rare events, especially in the context of studying the bit error rate of communication systems, blocking probability of communication networks and probability of instability of uncertain dynamic systems. Moreover, it has been recently proven by Brown, Cai and DasGupta [, 2] that the standard normal approximation approach is persistently poor. The coverage probability of the confidence interval can be significantly below the specified confidence level even for very large sample sizes. Since in many situations, it is desirable to quickly construct a confidence interval with guaranteed coverage probability, our goal is to derive a simple and rigorous formula for confidence interval construction. Let the probability space be denoted as (Ω, F, P where Ω, F, P are the sample space, the algebra of events and the probability measure respectively. Let X be a Bernoulli random variable with distribution Pr{X = } = P X, Pr{X = } = P X where P X (,. Let the sample size and confidence parameter δ (, be fixed. We refer an observation with value as a successful trial. Let K denote the number of successful trials during the i.i.d. sampling experiments. Let k = K(ω where ω is a sample point in the sample space Ω. Date: June 26. Key words and phrases. Confidence Interval, Probability, Statistics, ormal Approximation. This research was supported in part by grants from ASA (CC5-573 and LEQSF (ASA /LEQSF(2-4-.

2 2 XIJIA CHE, KEMI ZHOU AD JORGE L. ARAVEA.. Clopper-Pearson. The classic Clopper-Pearson lower confidence limit L,k,δ and upper confidence limit U,k,δ are given respectively by { { def if k = def if k = L,k,δ = and U p if k >,k,δ = p if k < where p (, is the solution of the following equation k ( (. p j ( p j = δ j 2 j= and p (, is the solution of the following equation k ( (.2 p j ( p j = δ j 2. j= The probabilistic implication of the confidence limits can be illustrated as follows: Define random variable L : Ω [, ] by L(ω = L,K(ω,δ ω Ω and random variable U : Ω [, ] by U(ω = U,K(ω,δ ω Ω. Then Pr{L P X U} > δ. The exact value of Pr{L P X U} is referred as the coverage probability. Accordingly, we refer Pr{P X < L or P X > U} as the error probability..2. ormal Approximation. It is easy to see that the equations (. and (.2 are very hard to solve and thus the confidence limits are very difficult to determine using Clopper-Pearson s approach. For large sample size, it is computationally prohibitive. To get around the difficulty, normal approximation has been widely used to develop simple approximate formulas (see, for example, [, 2, 5, 6] and the references therein. The basis of the normal approximation is the Central Limit Theorem, i.e., lim Pr P X K P X( P X < z = 2Φ(z where z > and Φ(. is the normal distribution function. Let Z δ 2 value such that Φ(Z δ = δ 2. It follows that i.e., lim Pr lim δ. { K be the critical 2 Z PX ( P X δ < P X < K } 2 + Z PX ( P X = δ, δ 2 Pr s Z 2 δ2 K + 2 Z δ 2 + Z2 δ 2 Z 2 δ K ( K < P X < s Z 2 δ2 K + 2 +Z δ 2 + Z2 δ 2 Z 2 δ K ( K = Since Z2 δ 2 for sufficiently large sample size, the lower and upper confidence limits can be estimated respectively as L k k Z ( k δ 2

3 BIOMIAL COFIDECE ITERVAL 3 and Ũ k + Z ( k δ. 2 The critical problem with the normal approximation is that it is of asymptotic nature. It is not clear how large the sample size is sufficient for the approximation error to be negligible. Such an asymptotic approach is not good enough for many practical applications involving rare events. 2. Rigorous Formula It is desirable to have a simple formula which is rigorous and very tight for the confidence interval construction. We now propose the following simple formula for constructing the confidence limits. Theorem. Define (2. L(k def = k and (2.2 U(k def = k k 2k + 4θ k( k, k =,,, + θ 2k + + 4θ k( k, k =,,, + θ with θ = 9. Then Pr {L(K < P 8ln 2 X < U(K} > δ. Moreover, δ L(k < L,k,δ < U,k,δ < U(k. Remark. L(k and U(k are tight bounds for the classic Clopper-Pearson confidence limits L,k,δ and U,k,δ (See Figures -2. A bisection search can be performed based on such bounds for computing the classic Clopper-Pearson confidence limits. To show Theorem, we need some preliminary results. The following Lemma is due to Massart [7]. Lemma. Pr { K P X + ǫ } ( ǫ exp 2 2(P X+ ǫ 3 ( PX ǫ 3 for all ǫ (, P X. Of course, the above upper bound holds trivially for ǫ P X. Thus, Lemma is actually true for any ǫ >. Lemma 2. Pr { K P X ǫ } ( ǫ exp 2 2(P X ǫ 3 ( PX+ ǫ 3 for all ǫ >. Proof. Define Y = X. Then P Y = P X. At the same time when we are conducting i.i.d. experiments for X, we are also conducting i.i.d. experiments for Y. Let the number of successful trials of the experiments for Y be denoted as K Y. Obviously, K Y = K. Applying Lemma to Y, we have Pr It follows that { K Pr { KY P Y + ǫ } ( P X + ǫ exp } ( ǫ 2 exp 2(P Y + ǫ 3 ( P Y ǫ 3. ǫ 2 2( P X + ǫ 3 [ ( P X ǫ 3 ].

4 4 XIJIA CHE, KEMI ZHOU AD JORGE L. ARAVEA The proof is thus completed by observing that Pr { K P X + ǫ } = Pr { K P X ǫ }. The following lemma can be found in [4]. Lemma 3. k ( j= j x j ( x j decreases monotonically with respect to x (, for k =,,,. ( Lemma 4. k ( j= j x j ( x j exp for k =,,,. j= (x k 2 2 ( 2 3 x+ k 3 ( 2 3 x k 3 x ( k, Proof. Consider binomial random variable X with parameter P X > k. Let K be the number of successful trials during i.i.d. sampling experiments. Then k ( P j X j ( P X j = Pr{K k}. ote that Pr{K k} = Pr { K P X ( } P X k. Applying Lemma 2 with ǫ = P X k >, we have k ( ( P j X j ( P X j (P X k exp 2 j= 2(P X PX k 3 ( P X + PX k 3 ( (P X k = exp 2 2 ( 2 3 P X + k 3 ( 2 3 P X k 3. Since the argument holds for arbitrary binomial random variable X with P X > k, the proof of the lemma is thus completed. Lemma 5. k ( j= x j ( x j (x exp k 2 x ( j (, k for k =,,. j= 2 ( 2 3 x+ k 3 ( 2 3 x k 3 Proof. Consider binomial random variable X with parameter P X < k. Let K be the number of successful trials during i.i.d. sampling experiments. Then k ( { P j K X j ( P X j = Pr{K < k} = Pr < P X + ( k } P X. Applying Lemma with ǫ = k P X >, we have that k ( ( P j X j ( P X j ( k exp P X 2 j= 2(P X + k PX 3 ( P X k PX 3 ( (P X k = exp 2 2 ( 2 3 P X + k 3 ( 2 3 P X k 3. Since the argument holds for arbitrary binomial random variable X with P X < k, the proof of the lemma is thus completed. Lemma 6. Let k. Then L,k,δ < U,k,δ.

5 BIOMIAL COFIDECE ITERVAL 5 Proof. Obviously, the lemma is true for k=,. We consider the case that k. Let S(, k, x = k j= ( j x j ( x j for x (,. otice that S(, k, p = S(, k, p + ( k p k ( p k = δ 2. Thus S(, k, p S(, k, p = δ 2 [ δ 2 ( k p k ( p k ]. otice that δ (, and that p (,, we have that ( S(, k, p S(, k, p = δ + p k ( p k >. k By Lemma 3, S(, k, x decreases monotonically with respect to x, we have p < p and complete the proof of the lemma. We are now in the position to prove Theorem. It can be easily verified that U,k,δ U(k for k =,. We need to show that U,k,δ U(k for < k <. Straightforward computation shows that U(k is the only root of equation ( (x k 2 exp 2 ( 2 3 x + k 3 ( 2 3 x k 3 with respect to x ( k,. There are two cases: U(k and U(k <. If U(k then U,k,δ U(k is trivially true. We only need to consider the case that k < U(k <. In this case, it follows from Lemma 4 that k j= ( ( [U(k] j ( U(k j exp j Recall that we have k j= k j= = δ 2 (U(k k 2 2 ( 2 3 U(k + k 3 ( 2 3 U(k k 3 ( U j,k,δ j ( U,k,δ j = δ 2, ( U j,k,δ j ( U,k,δ j k j= ( [U(k] j ( U(k j. j = δ 2. Therefore, by Lemma 3, we have that U,k,δ U(k for < k <. Thus, we have shown that U,k,δ q for all k. Similarly, by Lemma 5 and Lemma 3, we can show that L,k,δ L(k. By Lemma 6, we have L(k < L,k,δ < U,k,δ < U(k. Finally, the proof of Theorem is completed by invoking the probabilistic implication of the Clopper-Pearson confidence interval.

6 6 XIJIA CHE, KEMI ZHOU AD JORGE L. ARAVEA 3. umerical Experiments and Empirical Formulas In comparison with the Clopper-Pearson s approach, our approach is very tight from the perspective of interval width (see, for example, Figures -2. Moreover, there is no comparison on the computational complexity. Our formula is simple enough for hand calculation. Our numerical results are in agreement with the discovery made by Brown, Cai and DasGupta [, 2]. It can be seen from Figures 2-27 that the coverage probability of confidence intervals obtained by the standard normal approximation can be substantially lower than the specified confidence level δ (This is true even when the condition for applying the rule of thumb, i.e., P X ( P X > 5, is satisfied. Moreover, the situation is worse for smaller confidence parameter δ. See, for example, Figures 25-27, if one wishes to make an inference with an error frequency less than one out of, using the normal approximation can lead to a frequency of error higher than out of. In light of the excessively high error rate of inference caused by the normal approximation, the rigorous formula may be a better choice. The rigorous formula guarantees the error probability below the specify level δ. It should be noted that the rigorous formula is conservative (with actual error probability around % to 2% of the requirement. It should be noted that by tuning the parameter θ in the rigorous formula, one can obtained simple formulas which meet the specified confidence levels. For example, to construct confidence interval with confidence parameter δ =.5,.,., we can simply compute L(k and U(k defined in Theorem with θ = 2, 3, 5 respectively (The values of θ presented here are not optimal. Better coverage performance can be achieved by a fine tuning of θ. More specifically, { K Pr < P X < K { K Pr { K Pr K +2 K( K + 2 2K + 4 K 3 ( K + 3 2K + 4 K 5 ( K + 5 < P X < K < P X < K K + +2 K( K + 2 }.95; 2K + } + 4 K 3 ( K.99; + 3 2K + } + 4 K 5 ( K Confidence limits computed by these formulas for different and δ are depicted by Figures 3-2. It is interesting to note that, in most situations, the confidence limits computed by our empirical formulas almost coincide with the corresponding limits derived by Clopper-Pearson method. The numerical investigation of the coverage probability of different confidence intervals is shown in Figures It can be seen that the empirical formulas have excellent coverage performance. References [] Brown, L. D. Cai, T. DasGupta, A. (2. Interval estimation for a binomial proportion. Statistical Science 6:-33. [2] Brown, L. D. Cai, T. DasGupta, A. (22. Interval estimation for a binomial proportion and asymptotic expansions. The Annals of Statistics 3:6-2. [3] Clopper C. J. Pearson E. S. (934. The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika 26: [4] Clunies-Ross, C. W. (958. Interval estimation for the parameter of a binomial distribution. Biometrika 45:

7 BIOMIAL COFIDECE ITERVAL Lower limit by formula (3 Upper limit by formula ( Figure. Confidence Interval ( =, δ = Lower limit by formula (3 Upper limit by formula ( Figure 2. Confidence Interval ( =, δ =..

8 8 XIJIA CHE, KEMI ZHOU AD JORGE L. ARAVEA.9.8 Lower limit by formula (3 Upper limit by formula ( Figure 3. Confidence Interval ( = 5, δ = Lower limit by formula (3 Upper limit by formula ( Figure 4. Confidence Interval ( = 5, δ =..

9 BIOMIAL COFIDECE ITERVAL Lower limit by formula (3 Upper limit by formula ( Figure 5. Confidence Interval ( =, δ = Lower limit by formula (3 Upper limit by formula ( Figure 6. Confidence Interval ( =, δ =..

10 XIJIA CHE, KEMI ZHOU AD JORGE L. ARAVEA.9.8 Lower limit by formula (3 Upper limit by formula ( Figure 7. Confidence Interval ( = 5, δ = Lower limit by formula (3 Upper limit by formula ( Figure 8. Confidence Interval ( = 5, δ =..

11 BIOMIAL COFIDECE ITERVAL.9.8 Lower limit by formula (3 Upper limit by formula ( Figure 9. Confidence Interval ( =, δ =.5..9 Lower limit by formula (3 Upper limit by formula ( Figure. Confidence Interval ( =, δ =..

12 2 XIJIA CHE, KEMI ZHOU AD JORGE L. ARAVEA.9.8 Lower limit by formula (3 Upper limit by formula ( Figure. Confidence Interval ( = 5, δ = Lower limit by formula (3 Upper limit by formula ( Figure 2. Confidence Interval ( = 5, δ =..

13 BIOMIAL COFIDECE ITERVAL Lower limit by empirical formula Upper limit by empirical formula Figure 3. Confidence Interval ( = 5, δ = Lower limit by empirical formula Upper limit by empirical formula Figure 4. Confidence Interval ( = 5, δ =..

14 4 XIJIA CHE, KEMI ZHOU AD JORGE L. ARAVEA.9.8 Lower limit by empirical formula Upper limit by empirical formula Figure 5. Confidence Interval ( =, δ = Lower limit by empirical formula Upper limit by empirical formula Figure 6. Confidence Interval ( =, δ =..

15 BIOMIAL COFIDECE ITERVAL Lower limit by empirical formula Upper limit by empirical formula Figure 7. Confidence Interval ( = 5, δ = Lower limit by empirical formula Upper limit by empirical formula Figure 8. Confidence Interval ( = 5, δ =..

16 6 XIJIA CHE, KEMI ZHOU AD JORGE L. ARAVEA.9.8 Lower limit by empirical formula Upper limit by empirical formula Figure 9. Confidence Interval ( =, δ = Lower limit by empirical formula Upper limit by empirical formula Figure 2. Confidence Interval ( =, δ =..

17 BIOMIAL COFIDECE ITERVAL 7 A Coverage Probability 2 C B Sample Size Figure 2. Error Probability (P X =.5, δ =.5. A ormal, B Empirical, C Rigorous Coverage Probability 2 C A B Sample Size Figure 22. Error Probability (P X =., δ =.5. A ormal, B Empirical, C Rigorous

18 8 XIJIA CHE, KEMI ZHOU AD JORGE L. ARAVEA A Coverage Probability 2 B C Sample Size Figure 23. Error Probability (P X =.5, δ = 2. A ormal, B Empirical, C Rigorous A Coverage Probability 2 3 B C Sample Size Figure 24. Error Probability (P X =., δ = 2. A ormal, B Empirical, C Rigorous

19 BIOMIAL COFIDECE ITERVAL 9 Coverage Probability 2 3 B A 4 C Sample Size Figure 25. Error Probability (P X =.5, δ = 3. A ormal, B Empirical, C Rigorous 2 A Coverage Probability 3 4 B C Sample Size Figure 26. Error Probability (P X = 2, δ = 3. A ormal, B Empirical, C Rigorous

20 2 XIJIA CHE, KEMI ZHOU AD JORGE L. ARAVEA 2 A Coverage Probability 3 B 4 C Sample Size x 6 Figure 27. Error Probability (P X = 5, δ = 3. A ormal, B Empirical, C Rigorous [5] Hald, A. (952. Statistical Theory with Engineering Applications, pp , John Wiley and Sons. [6] John,. Kotz, L. S. Kemp, A. W. (992 Univariate Discrete Distributions, 2rd ed., pp. 24-3, Wiley. [7] Massart, P. (99. The tight constant in the Dvoretzky-Kiefer-Wolfowitz inequality. The Annals of Probability 8: Department of Electrical and Computer Engineering, Louisiana State University, Baton Rouge, LA 783 address: chan@ece.lsu.edu, kemin@ece.lsu.edu, aravena@ece.lsu.edu