Counts of Bernoulli Success Strings in a Multivariate Framework 1

Transcription

1 Counts of Bernoulli Success Strings in a Multivariate Framework Djilali AIT AOUDIA a, Éric MARCHANDb,2, François PRRON a a Université de Montréal, Département de mathématiques et de statistique, Montréal, QC, CANADA, H3C 3J7 b Université de Sherbrooke, Département de mathématiques, Sherbrooke, QC, CANADA, JK 2R Summary Distributional findings are obtained relative to Bernoulli arrays {X k,j, k, j,..., r + }, where the rows X k,,..., X k,r+ are independently distributed as Multinomial, p k,,..., p k,r+ for k, with p k, p k,r b+k, b r. The quantities of interest relate to the measure of the number of runs of length 2 and are S n S n,,..., S n,r, S lim n S n, where S n,j n k X k,jx k+,j. We obtain the mixed binomial moments of both the components of S n and S, as well as provide an explicit expression for the probability mass function of S. Moreover, we derive the multivariate Poisson mixture representation: S i V indep. Poissonv i with V V,..., V r Dirichlet,...,, b r +, as well as establish the distinct mixture representation: S α p α, α Beta, b with p α a multivariate mass function with Poissonα marginals given by p α s,..., s r r j0 j r e a α s +...+sr+j j s +...+s r+j!. AMS 200 subject classifications: 60C05, 6005, 625 Keywords and phrases: Arrays; Bernoulli; Binomial moments; Dirichlet; Multinomial; Poisson distribution; Poisson mixtures; Runs. Introduction In many studies of data, one is concerned with the occurrence or non-occurrence of a special event over time. In the counting of species, one wishes to know whether a new catch is from a new species; in the study of records, one wishes to assess the likelihood of a new record or not; in classification, one wishes to know whether a current observation is from a different population than past observations. And so on. Such data reduces to Bernoulli strings of 0 s and s and the counting of runs, or consecutive s has generated much interest and for a long time. November 5, Corresponding author: eric.marchand@usherbrooke.ca

2 Multivariate versions are also of interest in situations, namely for situations with r connected strings and where a motivating question is to describe the nature of the dependence. Bernoulli arrays were introduced by Ait Aoudia and Marchand 200 and consist of entries {X k,j, k, j,..., r + }, where the rows X k X k,,..., X k,r+ are independently distributed as Multinomial, p k,,..., p k,r+, k. Such arrays arise in sampling one object at a time from an urn with r + colours, with replacement and reinforcement i.e., adding balls as in a Polyà urn. Quantities of interest include the number of runs S n,j n k X k,jx k+,j of length 2 in the j th column, their limits S j lim n S n,j, and namely the random vectors S n S n,,..., S n,r, and S lim n S n.. For the univariate case, an elegant result consists of the Poisson mixture representation S U P oissonau with U Betaa, b, for p k, a, a > 0, b 0,.2 a + b + k as obtained by Mori 200, as well as Holst This class of solutions contains degenerate cases with b 0, i.e., PU, and where S is Poisson distributed with mean parameter a. Moreover, the a, b 0 Poisson solution was recognized in the mid 990 s by Persi Diaconis, as well as Hahlin 995, and earlier versions are due to Arratia, Barbour, Tavaré 992, Kolchin 97, and Goncharov 944. This elegant result has inspired much interest and lead to various findings relative to the distributions of S n,j and S j for various other configurations of the {p k,j } s, relationships and implications for Pólya urns, records, matching problems, marked Poisson processes, etc, as witnessed by the work of Chern, Hwang and Yeh 2000, Csörgó and Wu 2000, Holst 2007,2008, Huffer, Sethuraman and Sethuraman 2009, Joffe et al. 2004, 2000, Mori 200, Sethuraman and Sethuraman 2004, among others. With such Poisson distributions and Poisson mixtures arising naturally in these univariate r situations, it seems natural to investigate multivariate versions of such models. The findings of Ait Aoudia et al. 204 are contributions in this direction with analysis for the totals r j S n,j and T r j S j, and bivariate representations for the distributions of S n and S. For instance, they obtain the following multivariate extension of.2: T U P oisson au r with U Betaa, b, for p a k,l, a > 0, b 0, ra + b + k.3 which yields T Poissonλ for the degenerate case a rλ, b 0. 2

3 The main findings here are presented in Section 3, are based on the assumption p k, p k,r, with b r,.4 b + k and include: i explicit expressions Theorems 3. and 3.2 for the mixed binomial moments of S n,, S n,2,..., S n,r ; for all n, r; and of S, S 2,..., S r, ii an explicit expression for the distribution function of S Theorem 3.2, iii two mixture representations Theorem 3.3 for the distribution of S. As recalled in Section 2, mixed binomial moments yield expressions for probability mass and generating functions. The most intricate and challenging technical achievement is the binomial moment result of Theorem 3., which solves a recursive system of equations involving binomial moments related to the S n,j s and auxiliary variables W n,j, defined below in Section 3.. Solutions to such a system of equations were provided by Ait Aoudia et al. 204 in the bivariate case r 2, but our result is unified and general for r 2. The mixture representations are fascinating. On one hand, the components S i of S are distributed as independent Poissonv i conditional on V,..., V r which follows a Dirichlet distribution. This neatly summarizes the dependence structure of the S j s, as well as replicate the marginal Beta mixture representation.2. The second mixture representation is quite different as the mixing parameter is univariate Beta and the dependence is reflected otherwise through a multivariate discrete distribution with Poisson and non-independent marginals, and which connects for instance to Schur-constant distributions of Castanẽr et al Multivariate Poisson mixtures and mixed binomial moments For non-negative integer valued random variables Z, Z 2,..., Z r and constants k i {0,, 2,...}, r i,..., r, the mixed binomial moments are defined as Zi i k i, where z u is taken to be equal to 0 for u > z. Our main distributional findings involve the determination of mixed binomial moments, which imply associated expressions for the probability generating and mass functions. Indeed, the Taylor series expansion about of the probability generating function ψ Z having radius of convergence greater than can be expressed as ψ Z t [ t ] Z Z t k, t [0, ], 2. k k 0 3

4 and the probability mass function pmf of Z can be written as P Z i k k i i ki Z k. 2.2 For the multivariate case, corresponding expressions are given by r r [ t Z i Zi r i ] t j k i, t, t 2,..., t r [0, ] r, 2.3 i k 0,...,k r 0 i k i j and r P {Z j x j } r r i k i x i x k,...,x r k r j i Zi r ki k i x i i, 2.4 as long as the Taylor series expansion at t, t 2,..., t r,,..., of the probability generating function converge on an open set containing the origin. Multivariate Poisson mixtures, defined as distributions of U U,..., U r, such that U i V indep. PoissonV i and V V,..., V r is a random vector on [0, r will play a key role below. For such mixtures, it is easy to verify that mixed binomial moments assuming they exist are given by r i Ui k i r i V k i i, 2.5 k i! linking binomial moments of the U i s with mixed moments of the V i s. The specific mixing densities for V that arise below with our findings are Dirichlet a, a 2,..., a r+ ; a i > 0; densities given by Γ r+ i a i r+ i Γa i r v i a r+ i r j v a j j I D v, with D is the open r dimensional simplex defined by D {v,..., v r : v 0,..., v r 0, v + + v r }, their limiting versions as a r+ 0 which we denote Dirichleta,..., a r, 0. For multivariate Poisson mixtures with such mixing densities, we will make use of the lemma below. Hereafter, we denote γ k and γ k as the descending and ascending factorial with γ 0 γ 0, γ k k j0 γ j, and γ k k j0 γ + j for k, 2,.... Lemma 2.. Consider a multivariate Poisson mixture distribution U i V indep. PoissonV i with mixing variable V Dirichleta,..., a r+, and a i > 0, i,..., r. Then, 4

5 a U U,..., U r has mixed binomial moments r Ui Γ r+ i a i k i Γa r+ + r i a i + k i i r i a i ki k i! and, for the particular case with a,..., a r, a r+ b r +, we have r Ui r, with k k i b k i k i b For cases where a a r the distribution of U + + U r is: i a Poisson mixture with U + + U r W P oissonw, W Betar, a r+ for a r+ > 0; and ii Poisson for a r+ 0. Proof. a xpression 2.6 follows immediately from the general result, while the latter is obtained by integrating the rhs of 2.5 with respect to the Dirichlet density. b From the definition of the U i s, we have that U + + U r V PoissonW with W d V + + V r. Result i follows as one obtains readily that W Betar, a r+ for a r+ > 0, while result ii follows since W is degenerate at for a r+ 0. i, 3 Distributions of S n and S 3. Mixed binomial moments of S n We set S 0,j 0 for all j, and we define the auxiliary random vectors W n,j S n,, S n,2,..., S n,j, W n,j, S n,j+,..., S n,r, where W n,,..., W n,r are such that W n,j : S n,j + X n,j, n, j {,..., r}. 3. Our goal here is to find the mixed binomial moments of S n,, S n,2,..., S n,r and the mixed binomial moments of W n,i, i,..., r and S n,j, j,..., r, j i and derive explicitly the distribution of S n and S under assumption.4. Theorem 3.. Let r, k j r 0, j,..., r, kr r j k jr, and assume that kr > 0. Let r + r j k jr > 0, R {,..., r} and Ri R \ {i}, i R. Then, under assumption.4, we have for all r Sn,j k j r n+ kr r + +b k r n+b+ r + n kr + r +, 0 n < kr + r +, 5 3.2

6 and Wn+,i k i r for all i R. i Sn,j k j r Proof. Let N {,..., n + }. We have n+ kr r+ kir>0 +b kr n++b r + k i r>0 n kr + r + k i r > 0, 0 n < kr + r + k i r > 0, 3.3 X m,j {0, }, m, j N R, X m,j, m N. The proof will be developed in two steps called STP ON all of the k j r s are greater or equal than and STP TWO some of the k j r s are equal to 0. STP ON. Consider, for the moment, that k j r, for all j R. If S n,j k j r, for all j R then m N X m,j k j r +, for all j R. Therefore, Sn,j n < kr + r implies that 0. k j r If i R, S n,j k j r for all j Ri, and W n+,i k i r, then m N X m,j k j r + for all j Ri, and m N X m,i k i r. Therefore, Sn,j Wn+,i n < kr + r 2 implies that 0. k j r k i r i If n kr + r 2 and i R then Sn,j i k j r Wn+,i k i r is a Bernoulli random variable. The only way to obtain Wn+,i Sn,j k i r k j r i is by choosing {t,..., t r } Ri and having that if j t s, l Ri: l<s k lr + < m l Ri: l s k lr +, s Ri, X m,j if j i, kr + r k i m kr + r 0 otherwise for m, j N R. Therefore, we have for n kr + r 2, Wn+,i k i r i Sn,j k j r Bernoullir! 6 n+ m p m Bernoullir!/ + b kr+r.

7 This shows that 3.2 and 3.3 hold for n kr + r 2, k r,..., k r r > 0. Now, starting with r, knowing that 3.2 and 3.3 hold for r, n k, proceed by induction assuming that these formulas are true for n k,..., m. Condition on X m+ to obtain Sm, k Wm, Sm, p m+ + p m+ k k + [ m + b + + b k m + b + ] m k + b k m + b m + k. + b k m + b + Similarly, Wm+, Wm,i + p m+ + p m+ k k Wm,i Wm,i p m+ { + } + p m+ k k { } + + [ m + b + + b k + b k. + b k Sm, k Sm, k m + b + ] m k + b k m + b Now, let l >. Knowing that formulas 3.2 and 3.3 hold for r l, n kl + l 2, we asume that these formulas hold for r l, n kl + l 2,..., m, and seek to establish by induction that they also hold for r l, n m. We condition again on X m+ to obtain Sm,j p m+ k j l l Wm,i k i l i i Sm,j k j l + lp m+ Sm,j k j l l m kl l l + [ m + b + + b kl m + b l m + b + ] m kl l + b kl m + b l l m kl l + [ m + b l + m + b + + b kl m + b l m + b + ] m kl r + b kl m + b l m kl l [l + m kl l + ] m + b + + b kl m + b l + b kl m + kl l m + b + l. 7

8 Similarly, Wm+,i Sm,j p m+ Wm,i + Sm,j k i l k j l k i l k j l i i +p m+ Wm,j Sm,t + lp m+ k j l k t l i t Rj p m+ Wm,j Sm,t + p m+ Wm,i Sm,j k j l k t l k i l k j l t Rj i Sn,j + lp m+ k j l l m kl l m kl + l + m + b + + b kl m + b l m + b + + b kl m + b l l +[ m + b + ] m kl l, + b kl m + b l where we have made use of Pascal s binomial identity d+ c d c + d c and the induction hypothesis. When k i l in the first equality above, we proceed differently by making a link with an analogous problem where l is replaced by l also see STP TWO for more details. Simplifying, we obtain Wm+,i k i l i Sm,j k j l Sn,j k j l m kl l b + klm kl + [l + + m l kl + ] m + b + + b kl m + b l m kl l + 2 m kl l m kl + m + b l + 2 [ ] m + b + + b kl m + b l m kl l + 2 m + klm kll m + b l + 2 [ ][ + b kl m kl l + 2 m + + bm + b ] l + b kl m + kl l m + + b l, which completes the proof for k l,..., k l l > 0, l. STP TWO. Suppose that k j r > 0 for all j C, C, k j r 0 for all j R \ C, C R. 8

9 We wish to evaluate j C Sm,j k j r and Wm+,i k i l j C \{i} Sm,j, 3.4 k j r for all i C. By a symmetry argument, we can assume without loss of generality that k r,..., k r+ r > 0. Finally, linking the evaluation of 3.4 with STP ON. with r replaced by r + leads to the desired result and completes the proof. 3.2 Distribution of S and mixture representations As a consequence of Theorem 3., we obtain the following for the asymptotic case when n. Theorem 3.2. Under assumption.4, a The mixed binomial moments of S, S 2,..., S r are given by r Sj, b k with k r i k i. b The probability mass function of S S,..., S r is given by j k j P S s,..., S r s r + b t F t + r; t + b + ;, 3.6 with t r i s i and F the confluent hypergeometric function given by F γ ; γ 2 ; z j0 γ j γ 2 j z j j!. c For m N, the distribution of S r i S i t is uniform on the simplex Mt {v v, v 2,..., v r : v i 0, r i v i t}. Proof. Part a follows directly from Theorem 3. and Lebesgue s monotone convergence theorem by taking n. Part c follows from part b since P S s,..., S r s r 9

10 is a function of s + + s r. Finally, for part b, use 2.4 to obtain P S s,..., S r s r {k,...,k r: k i s i,i,...,r} mt mt m0 r i k i s i + b k m t + b m m t + b m r i ki s i r {k,...,k r: k i s i,i,...,r, r i k im} i m + r t + r m t + r m, + b t + b + t m m! which is 3.6 and completes the proof except the justification of the next to last equality. This equality can be justified by the following combinatorial argument for re-expressing the number of ways of choosing t + r objects among m + r. Number the objects through m+r and fix positive integers x,..., x r such that x + +x r t. Condition of the selection of the x th largest, x + x 2 th largest,..., x + + x r th largest numbered objects. The identity is established as the number of such possible choices is equal to r ki i x i with ki x i for i,..., r and k + + k r m. Our next result establishes two elegant mixture representations. Theorem 3.3. Under assumption.4, a The distribution of S is a multivariate Poisson mixture with S i V indep. PoissonV i for i,..., r, and V V,..., V r Dirichlet a,..., a r, a r+ b r + ; b The distribution of S admits the representation Proof. ki S α p α, α Beta, b, 3.7 with p α the multivariate probability mass function on N r given by r r e p α s,..., s r j α α s +...+s r+j j s s r + j! j0 a Part a is a direct consequence of Theorem 3.2 and Lemma s i, α 0, ]. 3.8

11 b Set t i s i. In view of expression 3.6, it suffices to show that + b t b α b g t,r α dα F t + r; t + b + ;, for all r, 2,..., b r, t 0,,..., and with g t,r α p α s,..., s r. We prove by induction on r and we make use of the integral representation 0 α γ α γ 2 γ e α dα Γγ Γγ 2 γ Γγ 2 F γ ; γ 2 ;, 3.0 as well as the recurrence relation γ 2 F γ + ; γ 2 ; γ 2 F γ ; γ 2 ; F γ + ; γ 2 + ;, 3. both valid for γ 2 > γ > 0 e.g., NIST Digital Library of Mathematical Functions. For r, we have g t, α e α α t t! and 3.9 is readily verified by using 3.0. Now, assume 3.9 holds for a given r 0 and observe that g t,r0 +α g t,r0 α g t+,r0 α by making use of Pascal s identity and by re-arranging terms. We now obtain by exploiting this along with 3. + b t b α b g t,r0 +α dα F t + r 0 ; t + b + ; which is indeed t + b + F t + r 0 + ; t + b + 2; F t + r 0 + ; t + b + ;, As a consequence of the above, we recover a Poisson mixture representation for the total number of runs of length 2, and as expressed in.3. Corollary 3.4. Under assumption.4, the total number of runs T r i S i of length 2 is Poisson distributed when b r. Otherwise for b > r, T admits the mixture representation T L PoissonL, L Betar, b r +. Proof. The result is a direct application of Theorem 3.3 and Lemma 2..

12 Remark 3.5. The above results can also be obtained using directly Theorem 3.2. instance with b r, we have using 2.3 [ [ t ] r ] r i S i t S t i k i i r i {k,...,k r: k i 0,i,...,r} i k i t k k0 {k,...,k r: k i 0,i,...r, r r k i k ik} r + k t k r r k k0 e t, For which establishes the Poisson distributional result for T when b r. The above corollary is already known and due to Ait Aoudia et al. 204, but it is presented here for sake of completeness and since our derivation differs in how the joint distribution of S, S 2,..., S r is used. The other findings above, which we now further discuss, generalize Ait Aoudia s bivariate case r 2 results. Theorem 3.2 s multivariate Poisson mixture representation of S extends in a most interesting way the known marginal distribution representation for S i, i,..., r, which is a Beta mixture of Poisson distributions. Here the components S i are clearly dependent, but the representation tells us that they are conditionally independent and the dependence is reflected through the dependence of the mixing components of the Dirichlet. In contrast to the Dirichlet mixture, the dependence in representation b of Theorem 3.2 is reflected through the conditional distributions of S, and the mixing variable α is univariate. Furthermore, it is readily verified that the conditional marginal distributions of S i α are Poissonα, which is consistent with the univariate Beta mixture result. Remark 3.6. The joint probability mass functions pmf p α in 3.8, which has Poissonα marginals, are examples of the larger class of pmf s of the form r r γx,..., x r j p x i + j, x,..., x r N, 3.2 j i j0 which were analyzed in the bivariate case r 2 and otherwise referred to by Ait Aoudia and Marchand 204. In the above form, p is a univariate pmf, γ has marginals given by p, and γ is a valid pmf as long as p is r -monotone, i.e., r j0 j pm+j 0 for all m. These conditions indeed apply for p Poissonα with α < leading to γ as in 3.8. For this and further properties of such distributions, we refer to Castañer et al r j

13 Acknowledgments Éric Marchand s and François Perron s gratefully acknowledge the support for research provided by NSRC of Canada. We are grateful to Chris Jones for useful discussions and insight on the distributions represented by 3.2. References Ait Aoudia, Djilali, Marchand, É., Perron, F. & Ben Hadj Slimene, L On runs, bivariate Poisson mixtures and distributions that arise in Bernoulli arrays. lectronic Communications in Probability, 9, -2. Ait Aoudia, D. and Marchand É On a simple construction of bivariate probability functions with fixed marginals. The American Statistician, 68, Ait Aoudia, D. and Marchand É On the number of runs for Bernoulli arrays. Journal of Applied Probability, 47, Arratia, R., Barbour, A.D., and Tavaré, S wens sampling formula. Annals of Applied Probability, 2, Poisson process approximations for the Castañer, A., Claramunt, M.M., Lefèvre, C. & Loisel, S Discrete Schur-constant models. Journal of Multivariate Analysis, 40, Chern, H.-H., Hwang, H.-K., and Yeh, Y.-N Distribution of the number of consecutive records. Random Structures Algorithms, 7, Csörgó, M. and Wu, B.W On sums of overlapping products of independent Bernoulli random variables. Ukrainian Mathematical Journal, 52, Goncharov, V On the field of combinatory analysis. Soviet Math. Izv., Ser. Math., 8, In Russian. Hahlin, L.O Double Records. Research Report # 2, Department of Mathematics, Uppsala University. Holst, L The number of two-consecutive successes in a Hoppe-Polyá urn. Journal of Applied Probability, 45, Holst, L Counts of failure strings in certain Bernoulli sequences. Journal of Applied Probability, 44,

14 Huffer, W. F., Sethuraman, J. and Sethuraman, S A study of counts of Bernoulli strings via conditional Poisson processes. Proceedings of the American Mathematical Society, 37, Joffe, A., Marchand É., Perron, F., and Popadiuk, P On sums of products of Bernoulli variables and random permutations. Journal of Theoretical Probability, 7, Joffe, A., Marchand É., Perron, F., and Popadiuk, P On sums of products of Bernoulli variables and random permutations. Research Report # 2686, Centre de Recherches Mathématiques, Montréal, Canada. Kolchin, V.F. 97. A problem of the allocation of particles in cells and cycles of random permutations. Theory of Probability and its Applications, 6, Mori, T.F On the distribution of sums of overlapping products. Acta Scientiarum Mathematica Szeged, 67, NIST Digital Library of Mathematical Functions. Release.0.9 of Online companion to [OLBC0]. Sethuraman, J. and Sethuraman, S On counts of Bernoulli strings and connections to rank orders and random permutations. A Festschrift for Herman Rubin. IMS Lecture Note Monograph Series, 45, Institute of Mathematical Statistics, pp