Estimates for probabilities of independent events and infinite series Jürgen Grahl and Shahar evo September 9, 06 arxiv:609.0894v [math.pr] 8 Sep 06 Abstract This paper deals with finite or infinite sequences of arbitrary independent events in some probability space. We find sharp lower bounds for the probability of a union of such events when the sum of their probabilities is given. The results have parallel meanings in terms of infinite series. Keywords: Probability space, independent events, Bonferroni inequalities, Borel- Cantelli lemma, infinite series Mathematics Subject Classification: 60-0, 60A05, 97K50, 40A05 Introduction This paper deals with finite or infinite sequences of arbitrary independent events in some probability space Ω, A, P. In particular, we discuss the connection between the sum of the probabilities of these events and the probability of their union. aturally, the results can be formulated both in the language of probability and the language of calculus of non-negative series. This paper is written in an expository and to some extent educational style. Part of the results in particular in sections and 3 are basically known, one of them being more or less equivalent to the Borel-Cantelli lemma. We hope that our approach, emphasizing the connections to calculus, will be of interest in itself. In Section we start with a lemma/construction that shows that for each sequence {x n } of real numbers x n [0; there is a sequence of independent events {A n } in a suitable quite simple probability space Ω, A, P such that P A n = x n for all n. In Section 3 we discuss the connections between the convergence of series of independent events and the probability of the union of these events. In particular, we give an extension of the inclusion-exclusion principle to the case of infinitely many events. In Section 4 we determine a sharp lower bound for the probability of a union of independent events when the sum of the probabilities is given and, vice versa, a sharp upper bound for the sum of the probabilities when the probability of the union is given.
The correspondence between sequences of independent events and non-negative series Throughout this paper, let Ω, A, P be a probability space, i.e. Ω is an arbitrary nonempty set, A a σ-algebra of subsets of Ω the sets considered to be measurable w.r.t. P and P : A [0; ] a probability measure. Let us first recall that infinitely many events A, A,... A are mutually independent if and only if k k P A il = P A il whenever i < i <... < i k. l= l= In the following we frequently make use of the fact that independence isn t affected if one or several events are replaced by their complements with respect to Ω. We denote the complement of an event A by A c, i.e. A c := Ω \ A. We begin with a lemma that gives a full correspondence between series with non-negative terms and less than and sequences of independent events. Lemma. If {x n } is a sequence of real numbers x n [0; ], then there exist a probability space Ω, A, P and a sequence {A n } of independent events A n A such that P A n = x n for all n. Proof. We can choose Ω := [0, ] [0, ] R, equipped with the Lebesgue measure P on the σ-algebra A of Lebesgue measurable subsets of Ω. We construct the desired sequence of sets/events A n by recursion. First, we take A to be the empty set if x = 0, and if x > 0 then we take A to be a rectangle contained in Ω, with its sides parallel to the axes and with area x. Here and in the following it doesn t matter whether we take open or closed rectangles since their boundaries form a null set anyway. Suppose we have already defined events A, A,..., A n such that P A k = x k for k =,..., n, the events A,..., A n are independent and 3 each A k k =,..., n is a finite union of rectangles with sides parallel to the axes. Then if if x n+ = 0 we define A n+ :=. If x n+ > 0, then for every k {,..., n } and for i < i <... < i k n we define k B i,i,...,i k := A il \ A j, l= j n j i,i,...,i k i.e. B i,i,...,i k consists of those points in Ω that are contained in all A il, but not in any other A j. Furthermore, for k = 0 we define B := Ω \ A A... A n.
In this way, we get a decomposition of Ω to n pairwise disjoint sets, Ω = k n i <i <...<i k n B i,i,...,i k B. For simplicity, let us re-write this as Ω = C C... C n where each C l is one of the sets B i,i,...,i k or B. The exact order is not important. Each C j is a finite union of rectangles with sides parallel to the axes. For each j we can construct a set C j C j which is a union of rectangles with sides parallel to the axes and with area P Cj = x n+ P C j. ow we define A n+ := C C... C n. From the construction it is obvious that P A n+ = x n+ and that A n+ is a finite union of rectangles with its sides parallel to the axes. It remains to show that A,..., A n+ are independent, more precisely that k k P A n+ A il = P A n+ P A il whenever i < i <... < i k n. l= ν= For this purpose we fix i,..., i n with i < i <... < i k n. Then by our construction there are j,..., j r {,..., n } such that k A il = l= r C jν. ν= Here P C jν A n+ = P Cjν = x n+ P C jν = P A n+ P C jν, and we obtain k P A n+ A il l= r = P C jν A n+ = ν= r P C jν A n+ ν= = P A n+ r P C jν ν= r = P A n+ P C jν ν= k = P A n+ P A il = P A n+ l= ν= k P A il, as desired. In such a way, we can construct the required infinite sequence {A n }. Obviously this lemma is true also for a finite number of sets. 3
3 The connection between the convergence of the series of probabilities and the probability of the union We now turn to the situation that we will deal with for the rest of this paper. We first introduce the following notation. otation. Let {x n } be a sequence of real numbers x n [0;. Then we set and generally T := x, T := x x, T 3 := x 3 x x, n T n := x n x x... x n = x n x k for all n. The quantities T n have a probabilistic meaning: In view of Lemma, we can consider the x n as probabilities of certain independent events A n in some probability space: x n = P A n. Then we have T n = P A n \ n k= A k k=, 3. i.e. T n is the probability that A n, but none of the events A,..., A n happens. In the following this correspondence will be very useful. We first collect some easy observations on the T n. Remark. For all I T n = x x... x. 3. Proof. This obviously holds for =, and if it is valid for some, then we conclude that + + T n = T n + T + = x n + x + x n = x n, so by induction our claim holds for all. Proof. 3. also follows from the probabilistic meaning of the T : For every I we have in view of 3. T n = P A n = P = x x... x, A c n where the last equality holds since A c, A c,..., A c n are also independent events. In other words, both sides of 3. denote the probability that at least one of the events A,..., A happens. 4
From 3. we immediately obtain T = x T n for all. 3.3 3 For every I the map F : R R, F x, x,..., x := T, T,..., T is injective though of course not surjective and the inverse is given by F T, T,..., T = x, x,..., x where x = T, x = T T, x 3 = T 3 T T,..., x = T T T... T. Thus, we will often say {x n } and the corresponding {T n } and vice versa. The above is also true for = in an obvious manner. 4 If σ is some permutation of {,..., } and T = T,..., T := F x σ,..., x σ, then T n = T n. This is an immediate consequence from 3.. Theorem. If the T n are defined as above, then T n < for all and Furthermore T n = if and only if x n =. T n. Proof. T n < follows immediately from 3., keeping in mind that x n < for all n. Hence u := T n. If u <, then we use that from 3.3 we have n T n = x n T k x n u for all n, 3.4 which yields k= T n u x n, hence x n u u <. 3.5 5
Suppose now that u =. We want to show that x n =. Indeed, if there exists an such that T n = T +... + T + x +n, and we obtain x n <, then x +n T T... T +n T +... + T + T T... T < since T +... + T <. This completes the proof of our Theorem. In the proof of the second statement on the case of equality we can also argue as follows: Taking the limit in 3. we obtain T n = lim T n = x n. By the theory of infinite products [, p. 9] hence to T n <. x n < is equivalent to x n > 0, Continuing with this line of ideas, we can get the following estimate for x n. Theorem 3. If x n < and u := T n <, then and this estimate is sharp. Proof. x n < log u, As in the proof of Theorem, from 3. we get u = T n = x n. Using the well-known estimate log + x < x which holds for < x we obtain x n < log x n = log x n = log u = log u. 3.6 In order to show the asymptotic sharpness of this estimate, we fix some u [0;, and we choose the x n such that finitely many of them have the same value and all others are zero. More precisely, for given I we set { u for n =,...,, x n := 0 for n >. 6
Then and T n = x n = x n = u = u x n = u log u ; the latter limit is easily calculated by considering the derivative of gx := u x at x = 0. The sharpness of the estimate can also be seen by estimating the error in the inequality log + x < x used above: From 3.6 and the Taylor expansion of the logarithm we obtain 0 < log u x n = [log x n + x n ] = x n x n 3 + x n 4 x n 5 +... If again x,..., x are all equal to x and x n = 0 for all n > where of course x depends on, in order to ensure T n = u, then x = x n < log u, hence x < log u, and we obtain 0 < log u x n x n = x < < log u. This upper bound obviously tends to 0 if which again shows the sharpness of the result. We will revisit the estimate in Theorem 3 from a slightly different point of view in the next section. We now want to give a probabilistic formulation of Theorems and 3. In order to do so we recall that if the x n are the probabilities of certain independent events A n, then T n is the probability of A n \ n k= A k. Since these sets are pairwise disjoint, we conclude that T n = P A n. So the estimate T n x n a direct consequence of T n x n is just a reformulation of the trivial inequality P A n P A n. In view of 3. it is also equivalent to the estimate x n 7 x n x n.
valid for all x n 0 which of course can also be proved by an elementary induction. The probabilistic meaning of the sum T n also carries over to the limit case. To see this, let us recall some known facts from probability theory. If {B n } n is a sequence of subsets of Ω, then we define B = lim inf n B n := k=n B k and B = lim sup B n := n k=n B k. Obviously, we always have B B. In the case of equality we write lim B n := B = B. n A sufficient condition for B = B, hence for the existence of lim n B n is that the sequence {B n } n is increasing B B B 3... or decreasing B B B 3.... When lim B n exists, then n lim P B n = P lim B n 3.7 n n see, for example [4, p. ]. We apply this to our independent events {A n }. If we set B lim B = A n, hence := A n, then P A n = P lim B = lim P B = ow we can state Theorems and 3 in terms of probability. T n. Theorem -P. Let {A n } be a sequence of independent events with P a n < for all n. Then P A n < if and only if P A n <. The direction is reminiscent of the Borel-Cantelli Lemma which can be stated as follows [, p. 96]: Let {A n } be a sequence of events. BC If P A n <, then P lim sup A n = 0. n BC If P A n = and the events A n are independent, then P lim sup A n =. n Here the zero-one law due to Borel and Kolmogorov [, p. 47] makes sure that for independent events P lim sup n A n has either the value 0 or the value. 8
In fact, BC is an immediate consequence of Theorem -P. Indeed, if P A n =, then also n= P A n = for all I, and if the events A n are independent, then Theorem -P yields P A n = for all I, so from 3.7 we deduce n= P lim sup A n = P n = n= A n = lim P n= A n =. For the sake of completeness we d also like to remind the reader of the short proof of BC: If P A n <, then for each given ε > 0 there is an I such that n=m P A n < ε for all m, hence P A n < ε for all m. Again in view of 3.7 this yields n=m P lim sup A n = lim P A n ε. n m n=m Since this holds for each ε > 0, we conclude that P lim sup n A n = 0. Theorem -P. If {A n } is a sequence of independent events with P A n < for all n and u := P A n <, then P A n < log u. ow let s consider for a moment only finitely many x n, say x,..., x, and the corresponding T,..., T. Expanding 3. we obtain T n = x n i<j x i x j + i<j<k x i x j x k +... + x x... x. The probabilistic meaning of this identity is just the inclusion-exclusion principle here for the special case of independent events: If once more we identify x n = P A n where A,..., A are independent events, then our identity takes the form P A n = P A n P A i A j 3.8 + i<j<k i<j P A i A j A k +... + P A... A. It is well-known that this identity also in the general case of non-independent events gives rise to the so-called Bonferroni inequalities see, for example [3], by truncating it either after positive or after negative terms: r r k S k P A n k S k for all admissible r, k= k= 9
where S k := P A j... A jk. j <j <...<j k Our next theorem shows that the inclusion exclusion principle holds also in the case of infinitely many independent events, i.e. that for all sums in 3.8 are convergent. Theorem 4. If {A n } is a sequence of independent events with P A n < for all n and P A n <, then P A n = k S k where S k := k= j <j <...<j k P A j... A jk. Proof. Setting x n := P A n, we can write S k as S k = x j x j... x jk. j <j <...<j k Since in any product of the form x j x j... x j, there is at least one j i with j i, and S is the sum of all possibilities of products of different x i s, we have the estimate S S n= ow, since x n converges, for every q 0; we have x n for all. 3.9 n= x n < q say for 0. Inserting this into 3.9 yields S S 0 q 0, hence lim for large enough, S = 0 for all q 0;. 3.0 q In fact, when infinitely many x n s are different from zero, then S 0 for every, and S we obtain even lim S = 0. In view of the convergence of the geometric series q n this shows that the sum k= k S k is absolutely convergent. Hence, since all terms in S have the same nonnegative sign, it follows that also the series obtained by expanding all the products in the series T n = x x... x n is absolutely convergent, and thus in any order of summation it has the same value. This proves our theorem. k= 0
4 Upper and lower bounds for probabilities Let and T n be as above. We consider the extremal problems to determine { } U s := inf T n : x n = s, 0 x,..., x for 0 s 4. and { } S u := sup x n : T n = u, 0 x,..., x for 0 u. The infimum in the definition of U s is in fact a minimum, since T +... + T = x x... x 4. { is a continuous function of x,..., x which is evaluated on the compact set x,..., x [0; ] : } x n = s. A similar reasoning shows that also the supremum in the definition of S u is a maximum. Theorem 5. U s = s and S u = U u = u. Proof. One might think of the method of Lagrange multipliers to calculate S u and U s, but as sometimes in similar situations it suffices to apply the inequality between arithmetic and geometric means. It shows that for all x,..., x [0; ] with x n = s we have x n x +... + x = s, with equality if and only if x = x =... = x = s. From this and 4. we see U s =. s On the other hand, if x,..., x [0; ] satisfy T n = u [0; ], then by 4. u = x n x +... + x, again with equality if and only if all x n are equal, in which case we have x +... + x = u. This shows the formula for S u. Obviously, S = U. Remark 6. U s and S u are strictly increasing functions of s resp. of u, while U s is a decreasing and S u an increasing function of.
Proof. That s U s and u S u are increasing is trivial. Each x,..., x [0; ] with x n = s gives rise to an x,..., x, x + [0; ] + with + x n = s by setting x + := 0, and the T,..., T corresponding to x,..., x and to x,..., x, x + are the same while T + = 0. Therefore the infimum in the definition of U + s is taken over a superset of the set appearing in the definition of U s, and we conclude that U + s U s for s. A similar resoning shows that S u is increasing. Of course, the monotonicity of U s and S u can also be verified by calculating the derivatives of the functions gx := x log x s and hy := y u /y and showing that they are non-negative. In view of, the maximum of U s over all s is attained at the first one, i.e., at = s where s denotes the smallest integer s. Hence we have Us := max U s = s s. s s So when the finite number of events is = s, the minimum of the probabilities P A n under the restriction P A n = s is the highest. Also, we have Us =. lim s In an obvious way, we can extend the definitions of S and U also to the case =. We will show that we will obtain explicit formulas for S and U by taking the limits of S and U for. First of all we note that for s > 0 the infimum in the definition of U s is not a minimum. Indeed, suppose that x n = s and x n = U s. W.l.o.g. we can assume that x > 0. Then we replace x by x, x x = x = x and x n = x n for n 3. Then x. Hence and we get a contradiction. Theorem 7., i.e. we create a new sequence {x n} where x n < x n = U s, x n = s and x x = x > U s = e s and S u = U u = log This formula for S u gives also a new proof of Theorem 3. u.
Proof. Since the infimum in the definition of U s is taken over a larger set than for any U s cf. the proof of Remark 6 and since U s e s, it is clear that U s e s. Suppose that U s < e s for some s 0. Then for some sequence {x n } with x n = s we have u 0 := T n < e s. Here infinitely many x n are positive since otherwise U s would be a lower bound for T n for sufficiently large, contradicting U s e s > u 0.. We can choose 0 so large that 0 > s and u 0 < e s 0 where s 0 := 0 x n < s. We then have 0 T n T n = u 0 < e s 0 < s 0 0 = U 0s 0. 0 This is a contradiction to the definition of U 0 s 0. Hence U s = e s. As we have mentioned already in the proof of Theorem 3, S u = u log u, so a similar reasoning as for U s shows that S u = log The functions S, U, S, U are plotted in Figure. u = U u. 6.0 5 0.8 4 0.6 3 0.4 0. 3 4 5 6 0. 0.4 0.6 0.8.0 Figure : The graphs of U s left and S u right for =,, 5, The results in Theorems 5 and 7 can be reformulated in terms of probabilities of independent events: Theorem 8. Let A,..., A be finitely many independent events. Then P A n P A n, 4.3 P A n P A n. 4.4 3
If {A n } is a sequence of independent events such that 0 < P A n <, then P A n > exp P A n, P A n < log P A n. All these estimates are best-possible. Equality in 4.3 and 4.4 occurs if P A =... = P A. The assumption in Theorem 8 that the events A n are independent is essential as the following easy counterexample demonstrates: Choose A =... = A to be one and the same event, whose probability is x = P A 0;. Then the left hand side of 4.3 is x while the right hand side is x which will be larger than x if is sufficiently large. Similarly, the left hand side of 4.4 is x while its right hand side is x, so their quotient x will become arbitrarily small for sufficiently large. x At last we take a brief look at the extremal problems opposite to those above, i.e. with supremum replaced by infimum and vice versa. Their solutions turn out to be quite simple. Theorem 9. For all I we have { } inf x n : T n = u, 0 x,..., x = u for 0 u, { sup T n : } x n = s, 0 x,..., x = min {s, } for 0 s, and this remains valid analogously also for =. Proof. By the definition of the T n we always have x n T n, hence T n x n. Therefore the infimum is at least u, and the value u is attained by taking only one event with x = T = u and x n = 0 for n. This shows the first assertion. The very same reasoning applies to the case s in the second assertion. If s >, we can choose x = and the other x n more or less arbitrary, requiring only x n = s. Then T = and T n = 0 for all n, hence T n = which is of course the maximal value. This proves also the second assertion. Acknowledgment. advice. We would like to thank Professor Ely Merzbach for his valuable References [] L.V. Ahlfors: Complex Analysis. An introduction to the theory of analytic functions of one complex variable, 3rd edition, McGraw-Hill Book Company, Düsseldorf 979 4
[] H. Bauer: Wahrscheinlichkeitstheorie, 4th edition, Walter degruyter, Berlin/ew York 99 [3] J. Galambos; I. Simonelli: Bonferroni-type inequalities with applications, Springer, ew York 996 [4] A. Gut: Probability: A Graduate Course, Springer, ew York 007. Jürgen Grahl University of Würzburg Department of Mathematics 97074 Würzburg Germany e-mail: grahl@mathematik.uni-wuerzburg.de Shahar evo Bar-Ilan University Department of Mathematics Ramat-Gan 5900 Israel e-mail: nevosh@math.biu.ac.il 5