Probability Inequalities for the Sum of Random Variables When Sampling Without Replacement

Transcription

1 International Journal of Statistics and Probability; Vol 2, No 4; 2013 ISSN E-ISSN Published by Canadian Center of Science and Education Probability Inequalities for the Sum of Random Variables When Sampling Without Replacement Jeremy Becnel 1, Kent Riggs 1 & Dean Young 2 1 Department of Mathematics and Statistics, Stephen F Austin State University, Nacogdoches, Texas, USA 2 Department of Statistical Systems, Baylor University, Waco, TX, USA Correspondence: Jeremy Becnel, Department of Mathematics and Statistics, Stephen F Austin State University, Nacogdoches, Texas, USA becneljj@sfasuedu Received: August 11, 2013 Accepted: September 24, 2013 Online Published: October 11, 2013 doi:105539/ijspv2n4p75 URL: Abstract Exponential-type upper bounds are formulated for the probability that the maximum of the partial sample sums of discrete random variables having finite equispaced support exceeds or differs from the population mean by a specified positive constant The new inequalities extend the work of Serfling 1974 An example of the results are given to demonstrate their efficacy Keywords: probability bounds, discrete probability 1 Introduction Serfling 1974 has obtained upper bounds for the probability that the sum of observations sampled without replacement from a finite population exceeds its expected value by a specified quantity Serfling 1974 also noted that his bound is crude due to the incorporation of the coarse variance upper bound σ 2 < b a 2 /4 and has suggested that it would be desirable to obtain a sharpening of this result involving the quantity σ 2 in place of the quantity b a 2 /4 While this problem remains unsolved, we attempt to at least partially fulfill Serfling s suggestion In order to do this we tighten his inequality bound by restricting ourselves to a particular class of discrete distributions The general problem which we address may be stated as follows Consider a finite population of size N whose members are not necessarily distinct Let the set Ω N = {x 1, x 2,,x N } be the set representation of this population Denote by X 1, X 2,,X n the values of a sample of size n drawn without replacement from Ω N Define the statistics N a = min x i, b = max x x i N i, μ = 1 i N 1 i N N, σ2 = x i μ 2 /N, 11 i=1 i=1 and let the sampling fractions be f n = n 1/N 1 and g n = n 1/N We are concerned with the behavior of the sum k S k = X i 12 i=1 for 1 k n In particular, we derive a new parameter-free upper bounds on the probabilities P n ε = PS n nμ nε 13 and R n ε = P max ε 14 n k N k where ε>0 The most familiar upper bound for 13 is the Bienayme-Chebyshev inequality, which is of the form P n ε 1 f nσ 2 15 nε 2 75

2 Serfling 1974 has derived an alternative upper bound for 13, which may be expressed as 2nε 2 P n ε exp 1 g n b a 2 16 and an alternative bound for a two-sided version of 14, which is given as R nε = P max n k N k ε ES n nμ 2r 17 nε 2r where r is a positive integer We then compare our new under bound with these under the following scenario, which is similar to an example presented by Savage 1961 Suppose one wishes to study the average height of a finite population of people Assume that all individuals in the population are between 60 and 78 inches and their heights are measured to the nearest inch The question we wish to answer is, What is the probability that the average height of a sample of 100 from a population of 4,000 individuals is within two inches of the population mean height? The main vehicle we utilize for the sharpening of inequalities 16 and 17 is the additional assumption that the random variables given by X i X 1,,X i 1 for i = 1,,n are discrete random variables with probability functions having finite, equispaced support, and whose variance is bounded above by the discrete uniform variance The remainder of the paper is as follows In Section 2 we give some mathematical preliminaries while in Section 21 we derive the main inequality results Finally, in Section 3 we present the before mentioned application of the newly-derived probability bounds 2 The Set V a,b,j From this point forward we work almost exclusively with an equispaced set of J points in the interval a, b beginning at a and ending at b We denote the set as Ω a,b,j = {a, a + c, a + 2c,,a + J 1c} where b = a + J 1c and c = b a J 1 We refer to J as the support size As before X 1, X 2,,X n denote the values of a sample of size n drawn without replacement from Ω a,b,j and S k = k i=1 X i To sharpen the inequalities 16 and 17 we work with probability distributions that have a variance bounded by the variance of a discrete uniform probability distribution This leads us to the following definition Definition 21 Let V a,b,j be the set of probability functions f with support on Ω a,b,j such that for a random variable X having probability function f the variance is bounded as per Var f X J + 1 b a 2 J 1 12 Remark 22 For f V a,b,j, the above bound on Var f X is simply the variance of a discrete uniform probability function on Ω a,b,j Remark 23 The definition for V a,b,j, although appearing somewhat restrictive, still allows for a broad and rich range of distributions In particular, it applies to a broad range of discrete unimodal distributions 21 Probability Inequalities for V a,b,j In this section we derive a new maximal probability inequality for sums of discrete unimodal random variables sampled without replacement from a set of probability functions belonging to V a,b,j We shall need the following lemmas, theorems, and corollaries to develop the new maximal probability inequality We now develop two lemmas which are used in the proof of the main theorem Lemma 24 Let X be a random variable with probability function in V a,b,j and let EX = μ Then for any λ 0 we have Ee λx μ expαe λd λd 1, where d = b a and α = J+1 1 J

3 Proof Let Z = X μ and notice that Ee λz = 1 + j=1 λ j j! EZ j = 1 + λ j j! EZ j, 21 since EZ = 0 Now let f be the probability function for X Then for any j 2 we have EZ j = b x μ j f x x=a b d j 2 x μ 2 f x x=a b = d j 2 x μ 2 f x x=a = d j 2 Var f X d 2 d j 2 J + 1 J 1 12 J + 1 d j J 1 12 since x μ d since f V a,b, f d j J 1 12 Substituting the result EZ j J+1 Ee λz 1 + into 21, we get = 1 + α λ j j! J + 1 J 1 12 = 1 + J J 1 12 λ j d j j! d j = 1 + αe λd λd 1 expαe λd λd 1 λ j d j j! Corollary 25 Let X be a random variable with probability function in V a,b,j and let EX = μ Then for any λ 0 we have Ee λx μ expβλ 2 rλ, d, where d = b a, β = J+1 b a 2 J 1 12, and rλ, d = λ j 2 d j 2 j! Proof By Lemma 24, we have 1 Ee λx μ J + 1 exp 12 J 1 eλd λd 1 Thus, 1 J + 1 exp 12 J 1 eλd λd 1 1 J + 1 λ j d j = exp 12 J 1 j! b a 2 J + 1 λ j 2 d j 2 = exp 12 J 1 λ2 j! = expβλ 2 rλ, d The following lemma uses an argument similar to Theorem 22 in Sefling s paper

4 Lemma 26 Let Ω N = {x 1, x 2,,x N } where each x k is in Ω a,b,j Also let μ = N k=1 N If the probability function of X 1 and the conditional probability functions of X k X 1, X 2,,X k 1,k= 2,,n are in V a,b,j, then, for any λ 0 and any n {1, 2,,N} we have Ee λs n nμ exp n 1 g n J J 1 eλd λd 1, where S n = n k=1 X k Proof For λ = 0, the result is obvious Given λ>0let λ k = N n λ for 1 k n N k Notice λ k is increasing in k up to λ as k goes from 1 to n Because X k μ X k 1,X 1 V a,b,j and X 1 V a,b,j letting μ k denote the conditional expectation of X k μ X k 1,,X 1 we have by Corollary 25 that Eexpλ k X k μ μ k X k 1,,X 1 expβλ k rλ k, d expβλ k rλ, d 22 because λ k λ Using the conditional independence of the random variables S k 1 and X k μ μ k given X k 1,,X 1, we can apply Corollary 36 so that Eexpλ k = Eexpλ k 1 S k 1 k 1μEexpλ k X k μ μ k X k 1,,X 1 23 Combining 23 with 22 we have that Recursively applying 24 we get where Δ n = n k=1 Replacing rλ, d with eλd λd 1 λ 2 d 2 Eexpλ k Eexpλ k 1 S k 1 k 1μ expβλ 2 krλ, d 24 Eexpλ n S n nμ expλ 2 Δ n βrλ, d, 25 N n 2 N k = 1 + N n 2 N 1 k=n n+1 1 Next, using that Δ k 2 n n1 g n from 25 we get Eexpλ n S n nμ expλ 2 n1 g n βrλ, d we have x k Eexpλ n S n nμ expλ 2 n1 g n β eλd λd 1 λ 2 d 2 = exp n 1 g n J J 1 eλ λd 1 Theorem 27 Let X k, S k, and μ be as before, then for any ε, λ > 0 we have { d1 gn J J 1ε EexpλS n nμ nε exp n 12J 1d Proof First note that by the Lemma 26 we have ln 12J 1ε 1 g n J + 1d + 1 ε d } EexpλS n nμ nε = e λnε Ee λs n nμ e λnε expαe λd λd 1 where α = n 1 g n J J 1 = expαe λd λd 1 λnε 26 In terms of λ, 26 is minimized when gλ = αe λd λd 1 λnε is minimized, which occurs at λ = 1 nε d ln αd

5 Substituting the value in 27 into 26 we get expαe λd λ d 1 λ nε { nε nε = exp α αd + 1 ln αd + 1 { = exp α + nε nε ln d αd } + nε d nε nε } d ln αd + 1 Substituting for α we get { = exp n 1 g n J J 1 + nε 12 J 1 ln d 1 g n J + 1 =exp { n d1 g n J+1+12J 1ε 12J 1d ln 12J 1ε ε d g n J+1d + 1 ε d } + nε } d 22 Main Result We now give the main result of the paper in the following theorem Theorem 28 For any ε>0and λ>0we have R n ε := P max ε n k N k exp { n d1 g n J+1+12J 1ε 12J 1d ln 12J 1ε 1 g n J+1d + 1 } ε d Proof By Proposition 34, we see that R n ε = P max ε n k N k e λε E exp λ S n nμ μ = E exp λ n S n nμ nε exp { n d1 g n J+1+12J 1ε 12J 1d ln 12J 1ε 1 g n J+1d + 1 } ε d, because Theorem 27 holds for any λ>0 here λ n As per 13 we have P n ε = PS n nμ nε Noting that P n ε R n ε and applying Theorem 28 we get the following corollary Corollary 29 For any ε>0we have P n ε exp { n d1 g n J+1+12J 1ε 12J 1d ln 12J 1ε 1 g n J+1d + 1 } ε d From 16 we have R nε = P max n k N k ε Observe that R nε = P max ε + P max n k N k ε n k N k Applying Theorem 28 to the above we get the following corollary Corollary 210 For any ε>0, R nε 2exp { n d1 g n J+1+12J 1ε 12J 1d ln 12J 1ε 1 g n J+1d + 1 ε d } 79

6 3 An Application Corollaries 29 and 210 give parameter-free maximal inequalities which are shaper than Serfling s 1974 inequalities given in 16 and 17, respectively Of course this fact is not surprising because we are utilizing the additional assumption that X 1, X 2,,X n belong to the set of probability distributions whose variance is bounded by the variance of the discrete uniform distribution as described in Definition 21 However, the extent of the improvement can be substantial as demonstrated by the following example Consider the following scenario, which is similar to an example presented by Savage 1961 Suppose one wishes to study the average height of a finite population of people Assume that all individuals in the population are between 60 and 78 inches and their heights are measured to the nearest inch The question we wish to answer is, What is the probability that the average height of a sample of 100 from a population of 4,000 individuals is within two inches of the population mean height? One possible solution is to apply the Bienayme-Chebyshev inequality given in 15, where σ 2 is replaced by the maximum possible variance of distributions with support on Ω 60,78,19, which for this problem is 81 This solution gives P X μ A second possible solution to this example is to apply the probability inequality 16 given by Serfling 1974 and assumes only finite support of a discrete random variable For our example, 16 yields P X μ If we make the additional and, in this case, reasonable assumption that the probability functions of X 1, X 2,X n are from V 60,78,19, then we may apply Chebyshev s inequality with variance bound given in Definition 21 This method yields P X μ Applying the newly-derived inequality given in Corollary 29, we get the following result P X μ Clearly, inequality 34 not only yields a better bound than inequalities 31, 32, and 33, but, moreover, considerably increases the degree of improvement As an extension of the above example, let the amount of error, ε, between X and μ be arbitrary, but retain all other values in the example Figure 1 demonstrates the sharpening of Serfling s 1974 inequality in 16 by the inequality given in Corollary 29 across values of ε from 0 to 3 Figure 1 Comparison of our bound OB to Serfling s bound SB over value of ε from 0 to 3 Acknowledgements The authors would like to thank the anynomous referee for the careful review and helpful comments pertaining to this article 80

7 References Dharmadhikari, S, & Joag-Dev, K 1989 Upper bounds for the variances of certain random variables Communications in Statistics-Theory and Methods, 18, Feller, W 1966 An Introduction to Probability Theory and Its Applications vol 2 New York: Wiley Gray, H, & Odell, P 1967 On least favorable density functions SIAM Review, 9, Jacobson, H 1969 The maximum variance of restricted unimodal distributions Annals of Statistics, 40, Klaassen, C 1985 On an inequality of chernoff Annals of Probabiliy, 13, Moors, J, & Muilwijk, J 1971 Note on a theorem of m n murthy and v k sethi Sankhyā, 33, Rayner, J 1975 Variance bounds Sankhyā, 37, Savage, I R 1961 Probability inequalities of the tchebycheff type Journal of Research of the National Bureau of Standards-B, 65, Seaman, J, Young, D, & Turner, D 1987 On the variance of certain bounded random variables The Mathematical Scientists, 12, Serfling, R J 1974 Probability inequalities for the sum in sampling without replacement The Annuals of Statistics, 2,

8 Appendix Here we present some results that are mostly standard, but are complied here for ease of reference We begin with a brief review of martingales including some results pertinent to this work 1 Martingales In the proofs of the results for this paper we need a few results about submartingales, in particular, reverse submartingales We present the necessary results here for ease of reference For a more indepth treatment of this subject see Feller 1966 For our purposes we will use the following definitions for martingales, submartingales, reverse martingales, and reverse submartingales Definition 1 Let {Z k } be a a sequence of random variables such that E Z k < We say {Z k } is a martingale if EZ k Z k 1, Z k 2, = Z k 1 for all k We say {Z k } is a submartingale if EZ k Z k 1, Z k 2, Z k 1 for all k We say {Z k } is a reverse martingale if EZ k Z k+1, Z k+2, = Z k+1 for all k We say {Z k } is a reverse submartingale if EZ k Z k+1, Z k+2, Z k+1 for all k We now present several results which exploit these properties Proposition 2 Let {Z n } N n=1 be a sequence of non-negative reverse submartingales, ie EZ n Z n+1,,z N Z n+1 Then for any c 0 we have cp max Z k c EZ n n k N Proof Let F = {max n k N Z k c} Then F can be expressed as the disjoint union of Now observe for each k = n,,n, Summing over all k we get or, equivalently, F N = {Z N c} F N 1 = {Z N < c} {Z N 1 c} F N 2 = {Z N < c} {Z N 1 < c} {Z N 2 c} F n = {Z N < c} {Z n+1 < c} {Z n c} EZ n ; F k = EEZ n Z n+1,,z N ; F k EZ k ; F k since {Z n } is a reverse submartingale cpf k since Z k c on F k N EZ n ; F k k=n N cpf k k=n EZ n ; F cpf since F = N F k Therefore, EZ n EZ n ; F cpf = cp max Z k c, n k N which yields the desired result Remark 3 If {Z k } is a reverse martingale, then applying Jensen s inequality immediately gives us that in{e λz k } is a reverse submartingale for any λ>0 Proposition 4 Let {Z k } N k=1 be a reverse martingale, ie k=n EZ k Z k+1, Z k+2, = Z k+1 82

9 For any ε>0and λ>0we have P max Z k ε EeλZ n n k N e λε Proof By Remark 3, {e λz k } is a reverse submartingale Now using c = e λε in Proposition 2 we have P max n k N eλz k e λε EeλZ n e λε A1 Combining A1 with the fact that P max n k N eλz k e λε = P max Z k ε, n k N we get the desired result 2 Finite Population Drawn Without Replacement Here, we present some results which will aid us in our quest for sharpening of inequalities 16 and 17 These results were first used without proof in Serfling s paper 1974 We give proofs here for the sake of completeness We work under the same set-up as presented in Section 1 That is Ω N = {x 1, x 2,,x N } is a finite population of size N, the members of which are not necessarily distinct Also X 1, X 2,,X n denote the values of a sample of size n drawn without replacement from Ω N and S k is the sum of the first k samples, as in 12 We also take μ as in 11 Proposition 5 Let μ k X k μ X 1, X 2,,X k 1 Then, Proof For k = 1, 2,,N 1, let T k k μ k = S k 1 k 1μ A2 N k 1 and T k N k One can easily check that T k is a reverse martingale and Tk are martingales Serfling 1974 That is, and To prove A2 note that ET k T k+1,,t N 1 = T k+1, 1 k N 2 = N k N k 1 ET k T k 1,,T 1 = T k 1, 2 k N 1 Tk = ET k+1 T k,t 1 S k+1 k + 1μ = E Tk N k 1,T 1 = N k S N k 1 E k+1 k + 1μ Tk N k,t 1 = N k N k 1 E Xk+1 μ + Tk N k,t 1 E = N k N k 1 = N k N k 1 Xk+1 μ N k T k,t 1 + E N k T k,t 1 1 N k E X k+1 μ X k,x 1 + E T k Tk,T 1 1 N k μ k+1 + Tk 83

10 Thus, Tk = μ k N k 1 + N kt k N k 1 Multiplying both sides of A3 by N k 1, we get A3 N k 1T k = μ k+1 + N kt k or Tk = μ k+1 Therefore A2 holds In the next corollary utilizes a nonobvious recursive relationship between the S k s Corollary 6 For a fixed integer any λ>0, let λ k = N n N k for 1 k n Then λ k = λ k 1 S k 1 k 1μ + λ k X k μ μ k A4 for k = 2, 3,,N Proof Using A2, we have that λ k 1 S k 1 k 1μ + λ k X k μ μ k N n = λ S k 1 k 1μ + λ N n X k μ + S k 1 k 1μ N k 1 N k N k + 1 S k 1 k 1μ = λn n + X k μ N k 1 N k + 1 S k 1 k 1μ N k N k + 1 N k + 1 S k 1 k 1μ = λn n + X k μ N k N k 1 N k = λ N n S k 1 k 1μ + X k μ N k = λ k, which yields A4 Copyrights Copyright for this article is retained by the authors, with first publication rights granted to the journal This is an open-access article distributed under the terms and conditions of the Creative Commons Attribution license 84