Information Geometry of Bivariate Gamma Exponential Distributions

Transcription

1 Information Geometry of Bivariate Gamma Exponential Distriutions Khadiga Ali Arwini Department of Mathematics Al-Fateh University Tripoli-Liya July 7, 9 Astract This paper is devoted to the information geometry of the family of ivariate gamma exponential distriutions, which have gamma and Pareto margins, and discuss some of its applications. We egin y considering the parameter ivariate gamma exponential manifold as a Riemannian -manifold; y following the Roa s idea to use the Fisher information matrix (FIM), and derive the α-geometry as; α-connections, α-curvature tensor, α-ricci curvature with its eigenvalues and eigenvectors, and α-scalar curvature; where the -geometry is corresponds to the geometry induced y the Levi-Civita connection, and we show that this space has a non-constant negative scalar curvature. In addition, we consider four sumanifolds as special cases, and discuss their geometrical structures, and we prove that one of these sumanifolds is an isometric isomorph of the univariate gamma manifold. Then we introduce log-ivariate gamma exponential distriutions, which have log-gamma and log-pareto margins, and we show that this family of distriutions determines a Riemannian -manifold which is isometry with the origin manifold. AMS Suject Classification (): 5B, 6G5 Key words: information geometry, statistical manifold, gamma distriution, Pareto distriution, ivariate distriutions, ivariate gamma exponential distriution Introduction Statistical manifolds are representations of families of proaility distriutions that allow differential geometric methods to e applied to prolems in stochastic processes, mathematical statistics and information theory. The origin work was due to Rao [], who considered a family of proaility distriutions as Riemannian manifold using the Fisher information matrix (FIM) as a Riemannian metric. In 975, Efron [6] defined the curvature in statistical manifolds, and gave a statistical interpretation for the curvature with application to second order efficiency. Then Amari [] introduced a one-parameter family of affine connections (α-connection), which contain the Levi-Civita connection of the Fisher metric when α =, and turn out to have importance and are part of larger systems of connections for which staility results follow [5]. He further proposed a differential-geometrical framework for constructing a higherorder asymptotic theory of statistical inference. He, defining the α-curvature of a sumanifold, pointed out important roles of the exponential and mixture curvatures and their duality in statistical inference. Several researchers studied the information geometry and its applications for some families of univariate and multivariate distriutions. Amari [] showed that the family of univariate Gaussian distriutions has a constant negative curvature. Oller [?] studied the geometry of the extreme value distriutions as, Gumel, Weiull and Frethet distriutions, and he showed that all these spaces are with constant negative curvatures, and he otained the geodesic distances in each case. Gamma manifold studied y many researcher eg [], and Dodson [] proved that every neighourhood of an exponential distriution contains a neighourhood of gamma distriutions, using an information theoretic metric topology. Adel- All, Mahmoud and Add-Ellah [] showed that the Pareto family is a space with constant positive curvature and they otained the geodesics, and they showed the relation etween the geodesic distance and the J-divergence.

2 Information Geometry of Bivariate Gamma Exponential Distriutions For ivariate and multivariate distriutions, the Fisher information metrics or the geometrical structures for most of these ivariate and multivariate distriutions has een found to e intractale; the only exceptions have een the families as, the ivariate and multivariate Gaussian distriutions, the Mckay ivariate gamma distriutions, and the Freund ivariate exponential distriutions. For a summary of ivariate and multivariate distriutions see Kotz, Balakrishnan and Johnson [7]. Sato, Sugawa and Kawaguchi [] proved that the 5-dimensional space of ivariate Gaussian distriutions has a negative constant curvature and if the covariance is zero, the space ecomes an Einstein space. After that, Skovgaard [] in 984, extended this work to the family of multivariate Gaussian model, and provided the geodesic distances. The geometry for Mckay manifold where the variales are positively correlated, otained y Arwini and Dodson [], who showed that this space can provide a metrization of departures from randomness and independence for ivariate processes, also they used the geometry of Mckay manifold in some application in hydrology, concerns tomographic images of soil samples in hydrology surveys, see [4]. After that, the geometry for Freund 4-manifold investigated y Arwini and Dodosn [], and they showed the important of this family ecause exponential distriutions represent intervals etween events for Poisson processes and Freund distriutions can model ivariate processes with positive and negative covariance. In this paper we are interested to study the Becker and Roux s and Steel and le Roux s ivariate gamma distriution, with gamma margins, which are ased on plausile physical models, and they also include the Freund distriution as a special case, see [7], the Fisher information matrix is found to e intractale. However, Steel and Roux in 989 [] studied compound distriutions of this ivariate gamma, and they otain the ivariate gamma exponential distriution with gamma and Pareto margins, hence we consider this family of distriution as a Riemannian -manifold and discuss its geometry. Bivariate gamma exponential distriutions The ivariate gamma exponential distriution has event space Ω = R + R + and proaility density function (pdf) given y f(x, y; a,, c) = a c Γ() x e (a x+c x y) for a,, c R + (.) where Γ is the gamma function which has the formula Γ(z) = t z e t dt. Figure show surface and control plots of ivariate gamma exponential distriution f(x, y; a,, c) for the range x [, 5] and y [, ], with unit parameters a = = c =.. Margins and correlation The marginal distriutions, of X and Y are gamma distriution and Pareto distriution, respectively. f X (x) = f Y (y) = a Γ() x e a x, x > (.) c a, y >. (.) + (a + c y) While the conditional marginal function f Y X given x is exponential distriution, with density function: f Y X (y x) = (c x) e (c x) y, y >. (.4)

3 Khadiga Ali Arwini y x Figure : The ivariate gamma exponential family of densities with parameters a = = c = as a surface and as a contour plot, shown for the range x [, 5] and y [, ]. The covariance and correlation coefficient of X and Y are given y: Cov(X, Y ) = c ( ), > Γ() ρ(x, Y ) =, > + Γ( + ) Γ( + ) Note that the ivariate gamma exponential distriution does not contains the independent case, with negative correlation which depends on only of the parameter. The correlation ρ decreases from to.5 where < < 4, and increases from.5 to where > 4. See figure.. Log-likelihood function and Shannon s entropy The log-likelihood function for the ivariate gamma exponential distriution (.) is l(x, y; a,, c) = log(f(x, y; a,, c)) = x (a + c y) + log(c) + (log(a) + log(x)) log(γ()). By direct calculation Shannon s information theoretic entropy for the ivariate gamma exponential distriution, which is the negative of the expectation of the log-likelihood function, is given y S f (a,, c) = l(x, y; a,, c) f(x, y; a,, c) dx dy, = + log(c) + log(γ()) ψ() (.5) Note that S f (a,, c) is independent of the parameter a. In the case when = which is the random case for the marginal function f X, the Shannon s entropy is S f (a,, c) = log(c) + γ where γ is the Euler gamma, in this case S f tends to e zero when c = e +γ. When c =, the Shannon s entropy is

4 4 Information Geometry of Bivariate Gamma Exponential Distriutions ρ(x, Y ) Figure : The correlation coefficient ρ(x, Y ) etween the variales X and Y as a function of the parameter, for the range (, 7). Note that the minimum value of ρ is.5 and occurs at = 4. S f (a,, c) = + log(γ()) ψ(). In this case S f tends to e zero when 6.5. Figure shows a plot of S f (a,, c) in the domain, c (, ), and figure 4 show plots of S f (a,, c) for the cases when = and c =.. Natural coordinate system and potential function We can rewrite the ivariate gamma exponential density funtion (.) as f(x, y; a,, c) = = a c Γ() x e (a x+c x y), = e a ( x)+c ( x y)+ (log(x)) (log(γ()) log(a) log(c)) (.6) Hence the family of ivariate gamma exponential distriutions forms an exponential family, with natural coordinate system (θ) = (θ, θ, θ ) = (a,, c), random variale (x, x, x ) = (F (x), F (x), F (x)) = ( x, x y, log x), and potential function ϕ(a,, c) = log(γ()) log(a) log(c)..4 Fisher information matrix FIM The Fisher Information (FIM) is given y the expectation of the covariance of partial derivatives of the log-likelihood function. Here the Fisher information metric components of the family of ivariate gamma exponential distriutions M with coordinate system (θ) = (θ, θ, θ ) = (a,, c) are given y g ij = = ϕ θ i θ j l(x, y, θ) θ i θ j f(x, y) dy dx, Hence, g = [g ij ] = a a a ψ () c. (.7)

5 Khadiga Ali Arwini c Figure : A surface and a contour plot for the Shannon s information entropy S f, for ivariate gamma exponential distriutions in the domain, c (, ). where ψ() = Γ () Γ() is the digamma function. and the variance covariance matrix is g = [g ij ] = a ψ () ψ () a ψ () a ψ () ψ () c. (.8) Geometry of the ivariate gamma exponential manifold. Bivariate gamma exponential manifold Let M e the family of all ivariate gamma exponential distriutions M = {f(x, y; a,, c) = a c Γ() x e (a x+c x y) a,, c R + }, x, y R + (.9) so the parameter space is R + R + R + and the random variales are (x, y) Ω = R + R +. We can consider M as a Riemannian -manifold with coordinate system (θ, θ, θ ) = (a,, c) and Fisher information metric g (.7), which is ds g = a da + ψ () d + c dc + da d. a. α-connections For each α R, the α (or (α) )-connection is the torsion-free affine connection with components ( log f log f ij,k = + α ) log f log f log f f dy dx θ i θ j θ k θ i θ j θ k

6 6 Information Geometry of Bivariate Gamma Exponential Distriutions c Figure 4: On the left, Shannon s information entropy S f for ivariate gamma exponential distriutions with unit parameter β =, in the domain c (, ). Note that S f tends to e zero when c = e +γ. On the right, Shannon s information entropy S f for ivariate gamma exponential distriutions with unit parameter c =, in the domain (, 5). Note that S f tends to e zero when 6.5. We have an affine connection (α) defined y: So y solving the equations we otain the components of (α). (α) i j, k = ij,k, where i = θ i ij,k = h= g kh Γ h(α) ij, (k =,, ). Here we give the analytic expressions for the α-connections with respect to coordinates (θ, θ, θ ) = (a,, c) : The nonzero independent components ij,k are and the components Γ i(α) jk, = (α ) a,, =, = (α ) Γ(α), = a,, = (α ) ψ (),, = α c. (.) of the (α) -connections are given y = [ ij ] = (α ) ( ψ () ) a ( ψ () ) (α ) ψ () ψ () (α ) ψ () ψ () a (α ) ψ () ψ (). (.)

7 Khadiga Ali Arwini 7 = [ ij ] = (α ) a ( ψ () ) α a ψ () a α a ψ () a (α ) ψ () ψ (). (.) = [ ij ] = α c. (.). α-curvatures By direct calculation we provide various α-curvature ojects of the ivariate gamma exponential manifold M, as: the α-curvature tensor, the α-ricci curvature, the α-scalar curvature, the α-sectional curvature, and the α-mean curvature. The α-curvature tensor components, which are defined as: ( R (α) ijkl = g hl i Γ h(α) jk j Γ h(α) ik + h= m= Γ h(α) im Γm(α) jk Γ h(α) jm Γm(α) ik ), (i, j, k, l =,, ) are given y while the other independent components are zero. ( α R (α) = R (α) = ) (ψ () + ψ ()) 4 a ( ψ. (.4) () ) Contracting R (α) ijkl with gil we otain the components R (α) jk of the α-ricci tensor R (α) = [R (α) jk ] = (α ) (ψ ()+ ψ ()) (α ) (ψ ()+ ψ ()) 4 a ( ψ () ) 4 a ( ψ () ) 4 a ( ψ () ) 4 ( ψ () ) (α ) (ψ ()+ ψ ()) (α ) ψ () (ψ ()+ ψ ()). (.5) The eigenvalues and the eigenvectors of the α-ricci tensor are given in the case when α =, y (+a ψ ()) ( ψ () ) (ψ ()+ ψ ()) ( ψ () ) (4 a + a ψ ()+a 4 ψ () ) (ψ ()+ ψ ()) 8 a ( ψ () ) (+a ψ ()) ( ψ () ) (ψ ()+ ψ ())+ ( ψ () ) (4 a + a ψ ()+a 4 ψ () ) (ψ ()+ ψ ()) 8 a ( ψ () ) (.6) a + a ψ () a + a ψ () + ( ψ () ) (4 a + a ψ ()+a 4 ψ () ) (ψ ()+ ψ ()) a( ψ () ) (ψ ()+ ψ ()) ( ψ () ) (4 a + a ψ ()+a 4 ψ () ) (ψ ()+ ψ ()) a ( ψ () ) (ψ ()+ ψ ()) (.7) By contracting the Ricci curvature components R (α) ij the scalar curvature R (α) : with the inverse metric components g ij we otain

8 8 Information Geometry of Bivariate Gamma Exponential Distriutions - 4 R (α) R () - α 4 - Figure 5: The scalar curvature R (α) for ivariate gamma exponential manifold, in the range α [, ] and (, 4) on the left, and α = and (, 4) on the right. R () is negative increasing function of the parameter from to as tends to. R (α) = ( α ) (ψ () + ψ ()) ( ψ () ). (.8) Note that the α-scalar curvature R (α) is a function of only the parameter, and in the case when α = the scalar curvature R () is negative and its increases from to. Figure?? shows the scalar curvature R (α) in the range α [, ] and (, 4), and the -scalar curvature R () in the range (, 4). The α-mean curvatures ϱ (α) (λ) (λ =,, ) are ϱ (α) () = ϱ (α) () = ( α ) (ψ () + ψ ()) ( ψ () ), ϱ (α) () =. (.9) The non-zero component of the α-sectional curvatures ϱ (α) (λ, µ) (λ, µ =,, ) is ϱ (α) (, ) = ( α ) (ψ () + ψ ()) 4 ( ψ () ). (.).4 Sumanifolds.4. Sumanifold M M: = In the case when =, the ivariate gamma exponential distriution (.) reduces to the form f(x, y; a, c) = a c x e (a x+c x y), x >, y >, a, c >. (.)

9 Khadiga Ali Arwini 9 when the x and y margins have the exponential distriution and Pareto distriution (with unit shape parameter), respectively: f X (x) = a e a x, x > (.) c a f Y (y) =, y >. (.) (a + c y) The Fisher metric with respect to the coordinate system (θ, θ ) = (a, c) is g = [ a c ]. (.4) Here we note that, the sumanifold M and the family of all independent ivariate exponential distriutions, which are the direct product of two exponential distriutions a e a x c e c y have the same Fisher metric. Hence the sumanifold M is an isometry of the manifold of independent ivariate exponential distriutions. In this manifold all the curvatures are zero, while the nonzero independent components ij,k and Γi(α) jk of the (α) -connections are.4. Sumanifold M M: c =, = α a,, = α c, = α, a = α. (.5) c In the case when c =, the ivariate gamma exponential distriution (.) reduces to the form f(x, y; a, ) = a Γ() x e (a x+x y), x >, y >, a, >. (.6) The x and y margins have the exponential and Pareto distriutions, respectively: f X (x) = f Y (y) = a Γ() x e a x, x > (.7) a, y >. (.8) + (a + y) where M and M have the same correlation coefficient (.). The Fisher metric with respect to the coordinate system (θ, θ ) = (a, ) is g = [ a a a ψ () ]. (.9)

10 Information Geometry of Bivariate Gamma Exponential Distriutions Note that the sumanifold M and the gamma manifold, which is the family of all univariate gamma proaility densities functions a Γ() x e a x, x >, a, > have the same Fisher metric. Hence, the family of ivariate gamma exponential proaility density functions with unit parameter c =, is an isometry of the gamma manifold with information theoretical metric topology. Here we give the geometrical structures of the sumanifold M. The nonzero independent components of the (α) -connections are given y:, = (α ) a,, =, = (α ) Γ(α), = a,, = (α ) ψ () (.) = (α ) ( ψ () ) a ( ψ () ) (α ) ψ () ψ () (α ) ψ () ψ () a (α ) ψ () ψ (). (.) = [ (α ) ( α) a ( ψ () ) a ψ () a ( α) a ψ () a (α ) ψ () ψ () ]. (.) The α-curvature tensor is given y while the other independent components are zero. The α-ricci curvature is R (α) = ( α R (α) = R (α) = ) (ψ () + ψ ()) 4 a ( ψ. (.) () ) (α ) (ψ ()+ ψ ()) (α ) (ψ ()+ ψ ()) 4 a ( ψ () ) 4 a ( ψ () ) (α ) (ψ ()+ ψ ()) (α ) ψ () (ψ ()+ ψ ()) 4 a ( ψ () ) 4 ( ψ () ). (.4) The α-scalar curvature is R (α) = ( α ) (ψ () + ψ ()) ( ψ () ). (.5) Note that M and M have the same scalar curvature, which is a negative non constant in the case when α = ; and this has limiting value as and as..4. Sumanifold M M: = In the case when =, the ivariate gamma exponential distriution (.) reduces to the form f(x, y; a, c) = a c x e (a x+c x y), x >, y >, a, c >. (.6) The x and y margins have the gamma distriution(with shape parameter= ) and Pareto distriution (with shape parameter=), respectively: f X (x) = a x e a x, x > (.7) f Y (y) = c a, y >. (.8) (a + c y)

11 Khadiga Ali Arwini The Fisher metric with respect to the coordinate system (θ, θ ) = (a, c) is g = [ a c The nonzero independent components of the (α) -connections are given y: ]. (.9) While all the curvatures are zero., =, = (α ) a, (α ) c, = α, a = α. (.4) c.4.4 Sumanifold M 4 M: a = c In the case when a = c, the ivariate gamma exponential distriution (.) reduces to the form f(x, y; a, ) = a+ Γ() x e a(x+x y), x >, y >, a, >. (.4) The x and y margins have the gamma and Pareto distriutions, respectively: f X (x) = f Y (y) = a Γ() x e a x, x > (.4), y >. (.4) + ( + y) and the correlation coefficient is given y ρ(x, Y ) = Γ() ( ) ( ) (Γ() Γ(+)) Γ(+) (Γ(+) Γ(+)), >. (.44) With respect to the coordinate system (θ, θ ) = (a, ) the Fisher metric is [ + ] g = a a ψ. (.45) () The nonzero independent components of the (α) -connections are given y: a, = (α ) ( + ) a,, =, = (α ) Γ(α), = a,, = (α ) ψ (). (.46) = (α ) ( (+) ψ () ) a ((+) ψ () ) (α ) ψ () (+) ψ () (α ) ψ () (+) ψ () a (α ) ψ () (+) ψ (). (.47)

12 Information Geometry of Bivariate Gamma Exponential Distriutions m n Figure 6: The log-ivariate gamma exponential family of densities with parameters a = = c = as a surface and as a contour plot, shown for the range n, m [, ]. = [ (+) (α ) a ((+) ψ () ) (α ) a ((+) ψ () ) (α ) a ((+) ψ () ) (+) (α ) ψ () (+) ψ () ]. (.48) The α-curvature tensor is given y ( α R (α) = R (α) = ) (ψ () + ( + ) ψ ()) 4 a (( + ) ψ. (.49) () ) while the other independent components are zero. α-ricci curvature is R (α) = (+) (α ) (ψ ()+(+) ψ ()) (α ) (ψ ()+(+) ψ ()) 4 a ((+) ψ () ) 4 a ((+) ψ () ) (α ) (ψ ()+(+) ψ ()) (α ) ψ () (ψ ()+(+) ψ ()) 4 a ((+) ψ () ) 4 ((+) ψ () ). (.5) The scalar curvature R () is given y: where the scalar curvature R () is negative non constant. R (α) = ( α ) (ψ () + ( + ) ψ ()) (( + ) ψ () ). (.5).5 Log-ivariate gamma exponential manifold Here we introduce a log-ivariate gamma exponential distriution, which arises from the ivariate gamma exponential distriution (.) for non-negative random variales n = e x and m = e y. So log-ivariate gamma exponential distriution, has proaility density function g(n, m) = a c n a ( log(n)) m Γ() e c log(n) log(m), < n <, < m <. (.5)

13 Khadiga Ali Arwini where a,, c >. Figure 6 show plots of the log-ivariate gamma exponential family of densities with unit parameters a, and c. Note that the margins of N and M are log-gamma distriution and log-pareto distriution, respectively. f N (n) = a na ( a log(n)) Γ() f M (m) =, < n < ( ) a c m (a c log(m)) +, < m < The covariance and correlation coefficient of the variales N and M are given y: Cov(N, M) = e a c ( a ρ(n, M) = ( ( + a) ( ) ( a e + a c E( +, ) ( ) E( +, a ) + a c c ), a +a ) a (+a) ) ( e c E( +, +a ) c ) ( ) E( +, a c ) ( ) (. c c Γ(, a c ) ( ) a Γ(, a c )) where E(n, z) is the exponential integral function, which is defined y e z t t dt. n This family of densities determines a Riemannian -manifold which is isometry with the ivariate gamma exponential manifold M. 4 Applications Still????? Acknowledgement The author would like to thank Professor Saralees Nadarajah for helpful comments concerning ivariate gamma distriutions. References [] N. H. Adel-All, M. A. W. Mahmoud and H. N. Ad-Ellah. Geomerical Properties of Pareto Distriution. Applied Mathematics and Computation 45() -9. [] Amari S. and Nagaoka H., Methods of Information Geometry, American Mathematical Society, Oxford University Press, [] Khadiga Arwini and C.T.J. Dodson. Information Geometry Near Randomness and Near Independence. Lecture Notes in Mathematics 95, Springer-Verlag, Berlin, Heidelerg, New York 8. [4] Khadiga Arwini, C.T.J. Dodson, S. Felipussi and J. Scharcanski. Information Geometric Classification of Porous Media Tomographic Images. Preprint (). [5] C.T.J. Dodson. Systems of connections for parametric models. In Proc. Workshop on Geometrization of Statistical Theory 8- Octoer 987. Ed. C.T.J. Dodson, ULDM Pulications, University of Lancaster, 987, pp 5-7.

14 4 Information Geometry of Bivariate Gamma Exponential Distriutions [6] B. Efron. Defining the curvature of a statistical prolem (with application to second order efficiency) (with discussion). Ann. Statist.,, (975) [7] Samuel Kotz, N. Balakrishnan and Norman L. Johnson. Continuous Multivariate distriutions. Volume : Models and Applications, Second Edition, John Wiley and Sons, New York,. [8] Saralees Nadarajah. The Bivariate Gamma Exponential Distriution With Application To Drought Data. Summited to Korean Society for Computational and Applied Mathematics and Korean SIGCAM. [9] Linyu Peng, Huafei Sun and Lin Jiu. The Geometric Structure of the Pareto Distriution. Boletiin de la Asociacion Matematica Venezolana, Vol. XIV, Nos. y (7) 5. [] C.R. Rao. Information and accuracy attainale in the estimation of statistical parameters. Bull. Calcutta Math. Soc. 7, (945) 8-9. [] Y.Sato, K.Sugawa and M.Kawaguchi. The geometrical structure of the parameter space of the two-dimensional normal distriution. Division of information engineering, Hokkaido University, Sapporo, Japan (977). [] L.T. Skovgaard. A Riemannian geometry of the multivariate normal model. Scand. J. Statist. (984) -. [] Steel, S. J., and le Roux, N. J.(989). A class of compound distriutions of the reparametrised ivariate gamma extension. South African Statistical Jouranl,, -4. [4] S. Yue. A Bivariate Gamma Distriution for Use in Multivariate Flood Frequency Analysis. Hydrological Processes 5(), -45. [5] S. Yue, T. B. M. J. Ouarda and B. Boee. A Review of Bivariate Gamma Distriutionks for Hydrological Applicaiton. Journal of Hydrlogy 46(), -8.