Exponential families of mixed Poisson distributions

Transcription

1 Exponential families of mixed Poisson distributions A. Ferrari a, G. Letac b,, J.-Y. Tourneret c a LUAN, Université de Nice Sophia-Antipolis, 68 Nice, France b LSP, Université Paul Sabatier, 362 Toulouse, France c IRIT/ENSEEIHT/TESA, 37 Toulouse, France Abstract If I =(I,..., I d ) is a random variable on [, ) d with distribution μ(dλ,..., dλ d ), the mixed Poisson distribution MP(μ) on N d is the distribution of (N (I ),..., N d (I d )) where N,..., N d are ordinary independent Poisson processes which are also independent of I. The paper proves that if F is a natural exponential family on [, ) d then MP(F) is also a natural exponential family if and only if a generating probability of F is the distribution of v + v Y + + v q Y q for some q d, for some vectors v,..., v q of [, ) d with disjoint supports and for independent standard real gamma random variables Y,..., Y q. Keywords: Natural exponential families; Multivariate gamma; Overdispersion. Introduction Consider the Poisson distribution with parameter λ > defined by P(λ)(dx) = n= λ λn e n! δ n(dx). If we randomize the parameter λ by some probability μ(dλ) on (, ) we get a new probability MP(μ) on the set N of nonnegative integers defined by MP(μ) = P(λ)μ(dλ). We have the Corresponding author. Fax: address: gerard.letac@tele2.fr (G. Letac).

2 identifiability property: P(μ) = P(μ ) if and only if μ = μ : just compute the generating function f MP(μ) (z) = z n MP(μ)({n}) n= of MP(μ) and link it to the Laplace transform L μ (θ) of μ by f MP(μ) (z) = L μ (z ). One sees MP(μ) as the distribution of N(I) where t N(t) is a standard Poisson process on N which is independent of the random variable I with distribution μ. One reason of the interest on these mixed Poisson distributions lies on the fact that they are overdispersed, in the sense that their variance is bigger than their mean. However, it should be pointed out that one easily constructs an overdispersed distribution ν concentrated on N such that no μ with ν = MP(μ) can possibly exist. An example is n= ν n z n = ( + z + z 2 )/(6 3z). There is an abundant literature on the topic for which Grandell [5] offers a good synthesis and references. Suppose now that μ belongs to a natural exponential family (NEF) F concentrated on [, ). Denote by V F (m) its variance function defined on the domain of the means M F (, ) in the sense initiated by Morris []. We consider the model MP(F) = {MP(μ); μ F}. Let us start with the following simple observation: if μ F has mean m then the variance of MP(μ) is m + V F (m). It is tantalizing to think that we have created a new natural exponential family G with variance function V G such that M G M F and such that V G (m) = m + V F (m) on M F. This is false: if p > is not an integer consider the NEF F such that M F = (, ) and V F (m) = (m ) p. Such a NEF does exist from Bar-Lev and Enis [] or Jorgensen [7]. Then one sees that V G (m) = m + (m ) p is not a variance function: from Theorems 3. and 3.2 of Letac and Mora [8] one should have M G = (, ) and thus for m > we would have the contradiction m dm m + (m ) p =. Furthermore, for some NEF F s the function m + V F (m) can be actually the variance function of some NEF G with no relation either with MP(F) or with F. A provocative example is V F (m) = m p with p > is not an integer and M F = (, ). In this case V G (m) = m+m p is the variance function of a NEF such that M G = (, ) but G is concentrated on the additive semigroup N + pn. For checking this it is enough is to compute the corresponding cumulant transform and to observe that the elements of G must be infinitely divisible with a discrete Lévy measure concentrated on N + pn. Finally Bent Jorgensen [7] offers a different construction of mixed Poisson distributions from a NEF (see the remark in Section 3 below for a description of the Jorgensen s manner.) Thus, a natural question is: if the NEF G exists do we have G MP(F)? In Section 3 Theorem says : yes, if and only if F is a gamma family, i.e. when there exists a number p > such that V F (m) = m 2 /p. In this case G is a negative binomial NEF. Section 4 extends the question to N d. We randomize (λ,..., λ d ) in the product P(λ )(dx ) P(λ d )(dx d ) by the probability μ(dλ) on [, ) d and consider the probability on N d defined by MP(μ) =... P(λ ) P(λ d )μ(dλ,..., dλ d ). We get a similar characterization (Theorem 2) which is described in the abstract above. The multivariate distributions which occur in Theorem 2 have been recently isolated and characterized 2

3 by Konstancja Bobecka and Jacek Wesołowski [2] (details are given in Section 4). The proof of Theorem 2 needs some care and the particular case d = of Theorem is a preparation to d 2. Section 2 recalls some facts about NEF. This study was motivated by statistical optics. Mixed Poisson distributions are commonly used to model data recorded from low flux objects or with short exposure times using photocounting cameras. This physical model arises from the semiclassical theory of statistical optics described in Goodman [4]. In this theory, the classical theory of propagation is used up to the camera, leading to a high flux image. Conditionally to this image, the number of photons counted on the pixels is distributed according to a Poisson distribution whose mean is the high flux intensity. A common problem for example in astrophysics is to estimate parameters of the wavefront (the mixing distribution) from photocounts recorded on a set of pixels. A description is found in Ferrari et al. [3]. A general assumption is that the vector of complex amplitudes associated to adjacent pixels of the image is a zero mean Gaussian vector, which implies that the vector of the corresponding intensities is distributed according to a multivariate gamma distribution. An important question is to derive conditions ensuring that the associated mixed Poisson distribution belongs to a NEF. This result is important since the computational complexity of most estimation or detection methods is usually reduced when applied to distributions belonging to an NEF. 2. NEF on R and R d This section describes the notations and classical facts about natural exponential families, mainly taken from Morris [] and Letac and Mora [8]. Denote by L ν (θ) = e θ,λ ν(dλ) R d the Laplace transform of a positive measure ν defined for θ R d not concentrated on any affine hyperplane. The Hölder inequality proves that the set D(ν) of θ R d such that L ν (θ) < is a convex set and that the cumulant function k ν = log L ν is a strictly convex function on this set. Denote by Θ(ν) the interior of D(ν) and assume that Θ(ν) is not empty. Then k ν is real analytic on Θ(ν). The set F(ν) of probabilities ν θ (dλ) = e θ,λ k ν(θ) ν(dλ), where θ runs Θ(ν) is called the NEF with generating measure ν. Note that F(ν) = F(ν ) does not imply ν = ν but implies only the existence of some a R d and b R such that ν(dλ) = e a,λ +b ν (dλ). Thus, a member μ of the NEF F(ν) can always be taken as a generating measure. However, some generating measures are not necessarily probabilities and can even be unbounded. We mention also that θ ν θ (dλ) is called a canonical parametrization of the NEF. Other parametrizations of the type t ν α(t) (dλ) = e α(t),λ +β(t) ν(dλ) with β(t) = k ν (α(t)), could be considered. Since θ k ν (θ) is a strictly convex function on the open set Θ(ν), the map θ m = k ν (θ) = R d λν θ(dλ) from Θ(ν) to R d is one to one. The open set M F = k ν (Θ(ν)) is called the domain of the means of F. Denote by m θ = ψ ν (m) the inverse function of k ν from M F onto Θ(ν). The Hessian 3

4 matrix k ν (θ) is the covariance matrix of the probability ν θ(dλ). Denoting V F (m) = k ν (ψ ν(m)), the map from M F to the positive definite symmetric matrices of order d defined by m V F (m) is called the variance function of F. It characterizes F since its knowledge leads by integration of a differential equation to the knowledge of the cumulant function of a generating measure of F. 3. The case of real exponential families For d =, for p and a > the gamma distribution with shape parameter p and scale parameter a is λ/a λp γ p,a (dλ) = e a p (, )(λ) dλ Γ(p). () This is a member of the NEF F generated by ν(dλ) = λp Γ(p) (λ)dλ. The domain of the means of F is M F = (, ) and its variance function is V F (m) = m 2 /p. The Laplace transform of γ p,a is L(z) = ( az) p with a suitable definition of this analytic function in {z C; Rz < /a}: we have simply to impose that it is real on the real axis. We see that the generating function of MP(γ p,a ) is (+a az) p = ( q qz )p with the notation q = a/( + a) (, ). Thus, if N MP(γ p,a ) then with the Pochhammer notation (p) = and (p) k+ = (p + k)(p) k we have Pr(N = k) = k! (p) k( q) p q k = k! (p) k, (2) ( + a) p+k a negative binomial distribution. This is a member of the NEF G of negative binomial distributions generated by k= k! (p) kδ k. The domain of the means of G is M G = (, ) and its variance function is V F (m) = m+m 2 /p. Thus, both F and G = MP(F) are NEF in this particular example. We show that this is the only case: Theorem. If the image of the NEF F(ν) on [, ) by μ MP(μ) is still an NEF, then there exists p > such that F(ν) is the family of gamma distributions with fixed shape parameter p. Proof. Denote L ν (θ) = e λθ ν(dλ) for θ Θ, where Θ is the interior of the convergence domain of L ν (θ). Note that Θ is either R or some half line (, a). Suppose that the image of F(ν) by μ MP(μ) is an NEF on N generated by some measure n= p n δ n. Consequently, there exists two functions α and β defined on Θ + such that for all n which can be rewritten λ(θ ) λn e n! ν(dλ) = p ne nα(θ)+β(θ), (3) L (n) ν (θ ) = n!p ne nα(θ)+β(θ). (4) Recall thatνis concentrated on(, ) and thus thatl ν is not a constant. Being a Laplace transform the function L ν cannot be a polynomial and L ν (n) cannot be identically. This implies p n = for 4 a k

5 all n. Eq. (??) shows that α(θ) and β(θ) are real-analytic functions on the interval Θ +. Indeed θ L ν (θ ) is analytic in the half complex plane Θ + + ir as well as its nth derivative L ν (n) (θ ). Furthermore, since L (n) ν (θ ) is positive on Θ + (because p n > ), its logarithm is real-analytic. Consequently, nα(θ) + β(θ) and (n + )α(θ) + β(θ) are real-analytic on Θ +, which implies by linear combination that α(θ) and β(θ) are real-analytic on Θ +. This proves the existence of α (θ) and β (θ). By taking the logarithms of both sides of (??) and differentiating with respect to θ we get L (n+) ν (θ ) L (n) ν (θ ) = nα (θ) + β (θ). (5) We now fix θ. Assume first that a = α (θ) = and denote p = β (θ)/α (θ). Eq. (??) can be written L (n+) ν (θ ) = a(p + n)l (n) (θ ) hence L ν (n) (θ ) = L ν (θ )(p) n a n. Since ν is concentrated on (, ) we have L ν (θ ) = a >. Since L ν (θ ) = pa2 /2 > we have p >. The Taylor formula applied to the analytic function L ν for small values of h can be written as follows: (ah) n L ν (θ + h) = L ν (θ ) (p) n n! n= = L ν (θ )( ah) p. The result L ν(θ +h) = ( ah) L ν (θ ) p is valid for any h (, /a), since the Laplace transform is an analytic function. The right-hand side of this expression is the Laplace transform of the gamma distribution γ p,a. Moreover, the Laplace transform of ν θ is e λh ν θ (dλ) = e λ(h+θ ) e k ν(θ ) ν(dλ) = L ν(θ + h) L ν (θ ) which shows that ν θ = γ p,a. In other words, the exponential family for {MP(μ θ ); θ Θ} is the family of gamma distributions with fixed shape parameter p. If α (θ) =, (??) yields L(n+) ν (θ ) = β (θ) which leads to L (n) ν (θ ) L ν (θ + h) L ν (θ ) = e β (θ)h. This is the noninteresting case where ν is a Dirac measure concentrated on the point β (θ). Our definition of NEF excludes this and the proof of Theorem is complete. Remark. For clarification it should be pointed out that Jorgensen [7, pp ] mentions a different object. Given a NEF F = F(ν) on (, ) and taking the number ψ in a suitable interval, Bent Jorgensen considers a NEF H ψ on N with a cumulant function of the form θ k ν (ψ+e θ ). In this case, introducing the reciprocal function h ψ (m) of the map z zk ν (ψ+z) he proves that ( ) m V Hψ (m) = m + V F h ψ (m) 2. h ψ (m) The Jorgensen s construction seems motivated by the particular case V F (m) = m p with p as considered by Hougaard et al. [6]. This family H ψ is obtained from F by a Poisson mixing, 5

6 process, but in a slightly complicated way. For describing it adopt the following notation: if μ is a measure on R and c > denote d c μ the image of the measure μ by the dilation x cx. Then H ψ is the set of all MP(d c ν ψ+c ) such that c is in Θ(ν). For instance if F is a gamma family generated by ν(dλ) = λp Γ(p) (, )(λ) dλ a simple calculation gives that H ψ is a negative binomial NEF with variance function V Hψ (m) = m + ψ p m2. 4. The case of multivariate exponential families A line multivariate gamma distribution governed by a nonzero vector (a,..., a d ) in [, ) d and the parameter p is the distribution of the random variable X = (a Y,..., a d Y) where Y is a real random variable with distribution γ p,. Its Laplace transform is L μ (z) = E(e θ,x ) = ( a θ a d θ d ) p. Its image by μ MP(μ) is the negative multinomial distribution on N d with generating function E(z N... z N d d ) = cp ( c(a z + + a d z d )) p where c = ( + a + + a d ). If some of the a i s are zero, (say a i > if and only if i m) then the half line image of [, ) by y (a y,..., a m y) is concentrated on [, ) m and the corresponding distribution is concentrated on N m. For an integer q consider q + subsets of {,..., d} denoted by {T,..., T q } and such that {,..., d} = q m= T m, T m =, m and T i T j =, i = j. Consider a product of line multivariate gamma distributions concentrated on [, ) T m for m =,..., q and a Dirac mass on [, ) T. More specifically, consider a distribution ν on [, ) d such that there exist nonnegative numbers a,..., a d and positive numbers p,..., p q such that the Laplace transform of ν is q k T a L ν (θ) = e k θ k m= m a k θ k p. (6) k T m Another presentation of these distributions can be helpful. If v = (v (),..., v (d) ) is in [, ) d let us call support of v the set of i {,..., d} such that v (i) >. If we now define v m (i) = a i Tm (i) then the q + vectors v,..., v q of R d have disjoint supports. Introduce the random variables Y m with distribution γ pm, such that (Y,..., Y q ) are independent. Then ν defined by (??) is the distribution of v + v Y + + v q Y q. Conversely if v,..., v q have disjoint supports the distribution of v + v Y + + v q Y q has type (??). These distributions (at least for v = ) have been characterized in Bobecka and Wesołowski [2] as follows: suppose that X and X are independent random variables of (, ) d. Write ( ) X X X d X + X = X + X,..., X d + X d Then they obtain an elegant multivariate version of a theorem due to Lukacs [9]: the random variables X + X X and X+X are independent if and only if there exists a nonzero sequence of 6

7 vectors (v,..., v q ) in [, ) d with disjoint supports and independent standard gamma variables (Y,..., Y q, Y,..., Y q ) such that X v Y + + v q Y q and X v Y + + v qy q. The real domain D(ν) of existence of this Laplace transform (??) is open and is the set Θ of θ k s such that k T m a k θ k > for all m =,..., q. For θ Θ, the element ν θ of the natural exponential family F generated by ν has the following Laplace transform: h L ν(θ + h) L ν (θ) q k T a = e k h k m= r m k T m a k h k p m, where r m = r m (θ) = ( k T m a k θ k ). Note that ν θ is also a product of line multivariate gamma distributions. The reader can verify that the familymp(f) = {MP(ν θ ); θ Θ} is indeed a natural exponential family generated by MP(ν) (warning: the parametrization θ MP(ν θ ) of MP(F) is not the canonical one). The next theorem shows that we have obtained in this way all natural exponential families F such that MP(F) is also an exponential family. It is an extension of the above Theorem. Theorem 2. If the image of the NEF F on [, ) d by μ MP(μ) is still a natural exponential family, then there exists q + disjoints subsets of {,..., d} denoted by {T, T,..., T q } and there exist nonnegative numbers a,..., a d and positive numbers p,..., p q such that F has a generating measure ν with Laplace transform (??). Proof. Similar to the proof of Theorem, the case where F is concentrated on a subspace of R d such as (,..., ) R q with q < d is discarded. This case leads to mixed Poisson distributions concentrated on (,..., ) N q. Denote by ν an arbitrary generating measure of F and L(θ) its Laplace transform defined as L(θ) = e λ,θ ν(dλ), [, ) d for θ Θ, where Θ is the interior of the domain of convergence of L(θ). Note that the fact that ν is concentrated on [, ) d implies that Θ + a Θ for any a = (a,..., a d ) such that a i. Suppose that the image of F(ν) by μ MP(μ) is an NEF on N d generated by some measure n N d p nδ n. We write R d for the vector with components equal to. We use the standard notations λ n = λ n λn d d and n! = n! n d!, for n = (n,..., n d ) in N d and λ in [, ) d. Similarly L (n) (θ) means n θ n... n d θ n L(θ). d d Thus there exist two functions α : Θ + R d and β : Θ + R such that e λ,(θ ) λ n [, ) d n! ν(dλ) = p ne n,α(θ) +β(θ), which can be rewritten L (n) (θ ) = n!p n e n,α(θ) +β(θ) n N d. (7) A discussion similar to that of Theorem shows that p n > for all n N d, since ν is not concentrated on some subspace of type (,..., ) R q. As a consequence, the real-analyticity 7

8 of α and β on Θ + can be deduced of the analyticity of L, by imitating again the proof of Theorem. Denote e i = (,...,,..., ) N d, where the unique is in position i. We also write α(θ) = (α,..., α d ), α ij = α j and β θ i i = β. By taking the logarithms of both sides of θ i (??) and applying we get θ i L (n+e i) (θ ) L (n) (θ ) = This last equality implies n, α(θ) + β(θ) = θ i θ i d n k α ik + β i. (8) k= α ij α jk = α ji α ik, α ij β j = α ji β i, (9) () for all i, j, k in {,..., d}. Indeed, by using (??) and ) can be written in two ways: L (n+e i+e j ) (θ ) = L (n) (θ ) ( d 2 θ i θ j = 2, the quantity L (n+e i+e j ) (θ θ j θ i n k α jk + β j ) ( d n k α ik + β i + α ij) k= k= ( d ) = L (n) (θ ) n k α jk + β j + α ji. k= n k α ik + β i) ( d k= Since this equality holds for all n, (??) and (??) are easily obtained. Assume first that α ij = for all i and j (separating this case is not absolutely necessary but makes the reading easier) and fix θ (as we did in the proof of Theorem ). In this case, (??) and (??) imply that there exist numbers a,..., a d and p such that a i = α ij for all j and p = β i /α ij for all i and j. Equality (??) can then be written L (n+e i) (θ ) L (n) (θ ) = a i (p + ) d n k. k= As a consequence, the following result can obtained: after noting L(θ + h) L(θ ) = ( ( d k= a k h k ) p = ) p d a k h k, () k= s= ( d ) s (p) s a k h k = s! k= (p) s s= n; n + +n d =s n!, d (a k h k ) n k. Consider now the implications of (??) and (??) in the general case where some α ij can be. For this, consider a directed graph G = (V, E) whose set of vertices V = {,..., d} is such that (i,j) is an edge if and only if α ij =. We also write i j instead of α ij = or (i,j) E and i i when the loop (i,i) exists (this loop may or may not exist). Suppose that there k= 8

9 exists k such that α ki = for all i. Eq. (??) can then be written L (n+e k) (θ ) L (n) (θ ) = β k. Thus, for any integer n k, we have L (n ke k ) (θ ) = L(θ )(β k ) n k. After multiplication by h n k k /n k! and summation (with respect to n k ) we obtain L(θ + h) = L(θ )e β k h k, for h = h k e k. More generally, denote by T the set of k such that there is no i such that k i. The above reasoning shows that k T β L(θ + h) = L(θ )e k θ k, h = h k e k. k T Some definitions about graphs need to be recalled. Consider a directed graph G = (V, E), where V is a finite set and E V V. The graph G = (V, E ) is called a subgraph of G if V V and E E (V V ). Furthermore, G is the induced graph on V if E = E (V V ). The following result can be easily obtained: Lemma. Consider the graph G defined as above on V ={,..., d} by the matrix (α ij ) satisfying (??). Then. Let i and j be distinct in V. If the induced subgraph G on V = {i, j} contains either the subgraph i j j or the subgraph i j, then the induced graph is i i j j. 2. If the induced subgraph G on V = {i, j, k} contains the subgraph i j k then G contains the subgraph k i i j j k. 3. If the induced subgraph G on V = {i, j} is either the subgraph i i j or the subgraph i j, then β j =. These results are illustrated in Fig.??. The proof of the lemma involves the three following cases:. If i j j, by setting k = j and k = i in (??), we obtain α ji = and α ii =. If i j, by setting k = j and k = i in (??), we obtain α jj = and α ii =. 2. (??) imply α ji = α jk = α jj = a j and α ij = α ii = α ik = a i. If α ij =, we obtain α ii = and α ik =. Similarly, if α jk =, we obtain α jj = and α ji =. 3. Apply (??). We come back to the proof of Theorem 2. Define the relation i j on V = {,..., d} by either i = j or the induced graph on {i, j} is i i j j. It is easy to deduce from the Lemma that is an equivalence relation. We remark that this implies that each element of T is alone in its equivalence class. Recall also that the definition of T implies that there are no arrows between two elements of T. Denote the other equivalence classes by T,..., T q. Suppose now that there exists i q m= T m and k T such that i k. Then part 3 of the Lemma implies that β k =. Eq. (??) can be used to prove that L (n+ek) = for all n N d. Thus, h L(θ + h) does not depend on h k. Since β k =, this implies that μ is concentrated on {λ R d ; λ k = }, a case which has been excluded from the beginning. There are finally no arrows between q m= T m and T. 9

10 Fig.. Illustration of the lemma. As a summary, the picture of the graph G is. a collection T of vertices without any arrow and any loop. 2. q disjoint classes T,..., T q without arrows between vertices of different classes, and with all possible arrows (including loops) inside a same class T m. Consider a fixed m in {,..., q}. For all i, j in T m, by setting k = i in (??), we obtain α ij α ji = α ji α ii. Thus, α ji = implies α ij = α ii. By using (??), for i T m and for any n N d, the following result can be obtained: L (n+e i) (θ ) L (n) (θ ) = α ii β i + n k. (2) α ii k T m Recall that the number p m = β i /α ii does not depend on i when i runs T m (indeed α ij = α ii and use (??)). Denote a k = α kk as above. The imitation of the proof of (??) and the formula (??) lead to: L(θ + h) = p m, a k h k (3) L(θ ) k T m for any h = k T m h k e k. This concludes the proof of theorem. References [] S. Bar-Lev, P. Enis, Reproducibility and natural exponential families with power variance functions, Ann. Statist. 4 (986) [2] K. Bobecka, J. Wesołowski, Multivariate Lukacs theorem, J. Multiv. Anal. 9 (24) [3] A. Ferrari, G. Letac, J.-Y. Tourneret, Multivariate mixed Poisson distributions, in: Proceedings of 2th Conference on Signal Processing (EUSIPCO 4), Vienna, Austria, September 6, 24, pp [4] J. Goodman, Statistical Optics, Wiley, New York, 985. [5] J. Grandell, Mixed Poisson Processes, Chapman & Hall, London, 997. [6] P. Hougaard, M.-T.T. Lee, G.A. Whitmore, Analysis of overdispersed count data by mixtures of Poisson variables and Poisson processes, Biometrics 53 (997) [7] B. Jorgensen, The Theory of Dispersion Models, Chapman & Hall, London, 997. [8] G. Letac, M. Mora, Natural exponential families with cubic variance functions, Ann. Statist. 8 (99) 37. [9] E. Lukacs, A characterization of the gamma distribution, Ann. Math. Statist. 26 (955) [] C.N. Morris, Natural exponential families with quadratic variance functions, Ann. Statist. (982) 65 8.