Universal connection and curvature for statistical manifold geometry

Universal connection and curvature for statistical manifold geometry Khadiga Arwini, L. Del Riego and C.T.J. Dodson Department of Mathematics University of Manchester Institute of Science and Technology Manchester M60 QD, UK k.alrawini@postgrad.umist.ac.uk dodson@umist.ac.uk and Departamento de Matemáticas Facultad de Ciencias, Universidad Autónoma de San Luis Potosí San Luis Potosí, SLP, 78900 México lilia@fc.uaslp.mx August 0, 007 Abstract Statistical manifolds are representations of smooth families of probability density functions ie subadditive measures of unit weight that allow differential geometric methods to be applied to problems in stochastic processes, mathematical statistics and information theory. It is common to have to consider a number of linear connections on a given statistical manifold and so it is important to know the corresponding universal connection and curvature; then all linear connections and their curvatures are pullbacks. An important class of statistical manifolds is that arising from the exponential families and one particular family is that of gamma distributions, which we showed recently to have important uniqueness properties in stochastic processes. Here we provide formulae for universal connections and curvatures on exponential families and give an explicit example for the manifold of gamma distributions. Key words: Universal connections and curvatures, information geometry, statistical manifold, exponential family, α-connection, gamma manifold. Introduction Information geometry is the study of geometric properties on probability density Riemannian manifolds, called statistical manifolds. Statisticians have been using for a very long time very accurate methods to choose special parameters to fit a probability density function pdf to a set of data. But it may happen that the same data can be fitted to another nearby pdf with different parameters. Information geometry began as the geometric study of statistical estimation on the set of probability distributions which constitute a statistical model as a manifold, i.e, as the manifold of pdfs to which we believe the true distribution belongs. Geometry is important to statistics because the natural setting of a Riemannian space includes studying quantities such as distance, the tangent space which provides linear approximations, geodesics and the curvature of a manifold. Fisher made this possible by defining in [] a Riemannian metric on statistical manifolds which bears his name and through which geodesics and the curvature of the Riemannian statistical manifold can be studied. Statistical inference procedures, which take into account the shape the statistical manifolds has in the entire set of pdfs can be carried out with geometric methods.

Universal connection and curvature for statistical manifold geometry More precisely, probability distributions p on a discrete set X are defined as subadditive measures: p : N {0} [0, nonnegative pk = unit weight k=0 pa B pa pb, A, B N {0} subadditivity. If X = R m, m N, then a probability density function pdf on a subset Ω satisfies: px 0 x Ω, and pxdx = Consider a family of probability distribution functions pdf on Ω R m. Suppose each element can be parameterized using real-valued variables θ Θ, so that the statistical model S is defined by S = {f θ : Ω [0, ]}; and each f θ satisfies f θ = θ Θ. The Fisher metric on S is defined by the covariance of the partial derivatives of the log-likelihood function l = log f θ : l l l g ij = E θ i θ i = E θ i j Systems of connections and universal objects The concept of system or structure of connections was introduced by Mangiarotti and Modugno [, ], they were concerned with finite-dimensional bundle representations of the space of all connections on a fibred manifold. On each system of connections there exists a unique universal connection of which every connection in the family of connections is a pullback. A similar relation holds between the corresponding universal curvature and the curvatures of the connections of the system. Definition. A system of connections on a fibred manifold p : E M is a fibred manifold p c : C M together with a first jet-valued fibred morphism ξ : C M E JE over M, such that each section Γ : M C determines a unique connection Γ = ξ Γ p, I E on E. Then C is the space of connections of the system. In the sequel we are interested in the system of linear connections on a Riemannian manifold. The system of all linear connections is the subject of studies in eg. [6, 7, 8, 9, 0]. Theorem. [, ] Let C, ξ be a system of connections on a fibred manifold p : E M. Then there is a unique connection form Λ : C M E JC M E On the fibred manifold π : C M E C with the coordinate expression Λ = dx λ λ dc a a ξ i λ dx λ i. This Λ is called the universal connection because it describes all the connections of the system. Ω Explicitly, each Γ SecC/M gives an injection Γ p, I E, of E into C E, which is a section of π and Γ coincides with the restriction of Λ to this section: Λ Γ p,ie E = Γ. A similar relation holds between its curvature Ω, called universal curvature, and the curvatures of the connections of the system.

Khadiga Arwini, L. Del Riego and C.T.J. Dodson 3 Ω = [Λ, Λ] = d ΛΛ : C M E T C E V E. So the universal curvature Ω has the coordinate expression: Ω = ξ j λ jξµ i dx λ dx µ a ξµ i dx a dx µ i. 3 Exponential family of probability density functions on R An important class of statistical manifolds is that arising from the so-called exponential family [] and one case is that of gamma distributions, which we showed recently in [4] to have important uniqueness properties for near-random stochastic processes. More details on statistical manifolds can be found in [, ]. In the present section we shall be concerned with the system of all linear connections on the manifold of an arbitrary exponential family, using the tangent bundle or the frame bundle to give the system space. We provide formulae for the universal connections and curvatures and give an explicit example for the manifold of gamma distributions. An n-dimensional set of probability density functions S = {p θ θ Θ R n } for random variable x R is said to be an exponential family [] when the density functions can be expressed in terms of functions {C, F,..., F n } on R and a function ψ on Θ as: p θ x = e {CxP i θi Fix ψθ}. 3. Then we say that θ i are its natural coordinates, and ψ is its potential function. From the normalization condition p θ x dx = we obtain: ψθ = log e {CxP i θi Fix} dx. 3. From the definition of an exponential family, and putting i = lθ, x = logp θ x to obtain and θ i, we use the log-likelihood function i lθ, x = F i x i ψθ 3.3 i j lθ, x = i j ψθ. 3.4 The Fisher information metric g [, ] on the n-dimensional space of parameters Θ R n, equivalently on the set S = {p θ θ Θ R n }, has coordinates: [g ij ] = [ i j lθ, x] dx = i j ψθ = ψ ij θ. 3.5 Then, S, g is a Riemannian n-manifold with Levi-Civita connection given by: Γ k ijθ = h= gkh i g jh j g ih h g ij = h= gkh i j h ψθ = h= ψkh θ ψ ijh θ 3.6 where [ψ hk θ] represents the inverse to [ψ hk x]. Next we obtain a family of symmetric connections which includes the Levi-Civita case and has significance in mathematical statistics. Consider for α R the function Γ α ij,k which maps each point θ Θ to the following value: Γ α ij,k θ = i j l α i l j l k lp θ = α i j k ψθ = α ψ ijk θ. 3.7 R So we have an affine connection α on the statistical manifold S, g defined by g α i j, k = Γ α ij,k, 3.8

4 Universal connection and curvature for statistical manifold geometry where g is the Fisher information metric. We call this α the α-connection and it is clearly a symmetric connection and defines an α-curvature. We have also α = α 0 α, = α α. 3.9 For a submanifold M S, the α-connection on M is simply the restriction with respect to g of the α-connection on S. Note that the 0-connection is the Riemannian or Levi-Civita connection with respect to the Fisher metric and its uniqueness implies that an α-connection is a metric connection if and only if α = 0. We have collected in the Appendix a summary of results for the α-scalar curvature for several important univariate Table and bivariate distributions Table. 3. Some geodesics for the 0-connection for the gamma manifold The geodesic equations for µ or β constant can be obtained: µ constant: the geodesic equations reduce to βt = c k t k c k for all c, c = cte β = k β β with solutions We show some explicit examples in grafica β constant: the geodesic equations reduce to µt = A exp φut B, for all µ = d log Γµ µ dµ with solutions A, B = cte We show some explicit examples in grafica 4 Universal connection and curvature for exponential families We provide explicit formulae for the universal connection and curvature on the statistical n-manifold for an arbitrary exponential family. We use the two available ways: via the tangent bundle and via the frame bundle systems of all linear connections. 4. Tangent bundle system: C T T M JT M The system of all linear connections on a manifold M has a representation on the tangent bundle with system space E = T M M C T = {α jγ T M M JT M jγ : T M T T M projects onto I T M } Here we view I T M as a section of T M T M, which is a subbundle of T M T T M, with local expression dx λ λ. The fibred morphism for this system is given by ξ T : C T M T M JT M T M T M T T M, α jγ, ν ανjγ.

Khadiga Arwini, L. Del Riego and C.T.J. Dodson 5 In coordinates x λ on M and y λ on T M ξ T = dx λ λ γ i λ i = dx λ λ y j Γ i jλ i = dx λ λ y j ψih ψ jλh i 4.0 Each section of C T M, such as Γ : M C T : x λ x λ, γ µϑ ; determines the unique linear connection Γ = ξ T Γ π T, I T M with Christoffel symbols Γ λ µϑ. On the fibred manifold π : C T M T M C T ; the universal connection is given by: h= briefly, Λ T : C T M T M JC T M T M T C T T C T M T M, x λ, v λ µν, y λ [X λ, V λ µν X λ, V λ µν, Y µ V λ µνx ν ]. Λ T = dx λ λ dv a a y µ vµν i dx ν i = dx λ λ dv a a y µ ψih ψ µνh dx ν i. 4. Explicitly, each Γ SecC T /M gives an injection Γ π T, I T M, of T M into C T T M, which is a section of π and Γ coincides with the restriction of Λ T to this section: h= Λ T Γ π T,I T M T M = Γ. and the universal curvature of the connection Λ is given by: Ω T = d ΛT Λ T : C T M T M T C T T M V T M. So here the universal curvature Ω T has the coordinate expression: Ω T = y k v j kλ jy m vmµ i dx λ dx µ a y m vmµ i dx a dx µ i = y k ψjh ψ kλh j y m ψih ψ mµh dx λ dx µ a y m ψih ψ mµh dx a dx µ i. h= h= 4. Frame bundle system: C F F M JF M A linear connection is also a principal i.e. group invariant connection on the principal bundle of frames F M with: E = F M M = F M/G consisting of linear frames ordered bases for tangent spaces with structure group the general linear group, G = Gln. Here the system space is C F = JF M/G T M T M T F M/G, consisting of G-invariant jets. The system morphism is h= ξ F : C F F M JF M T M T M T F M, [js x ], b [T x M T b F M].

6 Universal connection and curvature for statistical manifold geometry In coordinates ξ F = dx λ λ X µ µ s λ ν ν λ = dx λ λ X µ Γ λ µν ν λ = dx λ λ X µ n h= ψλh ψ µνh ν λ 4. where ν λ = is the natural base on the vertical fibre of T b λ b F M induced by coordinates b λ ν on F M. ν Each section of C F M that is projectable onto I T M, such as, ˆΓ : M C F : x λ x λ, [jγ x ] with Γ λ µν = µ s λ ν; determines the unique linear connection Γ = ξ F ˆΓ π F, I F M with Christoffel symbols Γ λ µν. On the principal G-bundle π : C F M F M C F ; the universal connection is given by: Briefly, Λ F : C F M F M JC F M F M T C F F M T C F F M F M, x λ, v λ µν, b µ ν [X λ, Y λ µν X λ, Y λ µν, b µ ν v λ µθx θ ]. Λ F = dx λ λ dv a a b µ ν vµθ λ dx θ ν λ = dx λ λ dv a a b µ ν ψλh ψ µθh dx θ ν λ. 4.3 Explicitly, each Γ SecC F /M gives an injection Γ π F, I F M, of F M into C F F M, which is a section of π and Γ coincides with the restriction of Λ F to this section: h= Λ F Γ π F,I F M F M = Γ and the universal curvature of the connection Λ is given by: Ω = d ΛF Λ F : C F M F M T C F F M V F M. So here the universal curvature form Ω F has the coordinate expression: Ω F = b k νv β kλ ν β b m ω vmµ α dx λ dx µ a b m ω vmµ α dx a dx µ ω α = b k ν ψβh ψ kλh ν β b m ω ψαh ψ mµh dx λ dx µ a b m ω ψαh ψ mµh dx a dx µ ω α. h= h= 5 Universal connection and curvature on the gamma manifold We give explicit forms for the system space and its universal connection and curvature on the statistical -manifold of gamma distributions. It is an exponential family and it includes as a special case µ = the exponential distribution itself, which complements the Poisson process on a line. We denote by G the family of gamma probability density functions {px; β, µ = β µ xµ Γµ e β x β, µ R }, x R. 5.4 So the space of parameters and hence G is topologically R R. Since the set of all gamma distributions is an exponential family with natural coordinate system β, µ and potential function ψ = log Γµ µ log β, [] the Fisher metric 3.5 is given by: [g ij ] = [ µ β β β φ µ ] h= 5.5

Khadiga Arwini, L. Del Riego and C.T.J. Dodson 7 where φµ = Γ µ Γµ is the logarithmic derivative of the gamma function. The independent nonzero Levi-Civita connection components 3.6 are given by: Γ = Γ = Γ = Γ = Γ = Γ = µ φ µ β µ φ µ, φ µ µ φ µ, β φ µ µ φ µ, µ β µ φ µ, β µ φ µ, µ φ µ µ φ µ. 5.6 5. Tangent bundle system on G The system space is C T = {α jγ T G G JT G jγ : T G T T G projects onto I T G } and the system morphism is ξ T = dx λ λ y j Γ i jλ i = dx µ φ µ φ µ β µ φ µ y µ φ µ y dx µ β µ φ µ y β µ φ µ y dx φ µ β φ µ µ φ µ y µ φ µ y dx µ φ µ β µ φ µ y µ φ µ y. The universal connection on the gamma manifold is given here by: Λ T = dx λ λ d i λj i λj y τ Γ i τνdx ν i = dx λ λ d i λj λj i µ φ µ φ µ β µ φ µ y µ φ µ y dx µ β µ φ µ y β µ φ µ y dx φ µ β φ µ µ φ µ y µ φ µ y dx µ φ µ β µ φ µ y µ φ µ y dx.

8 Universal connection and curvature for statistical manifold geometry The universal curvature on the gamma manifold is: Ω T = y k Γ j kλ jy m Γ i mµ dx λ dx µ a y m Γ i mµ dx a dx µ i = [y Γ y Γ i y Γ y Γ i y Γ y Γ i y Γ y Γ i y Γ y Γ i y Γ y Γ i y Γ y Γ i y Γ y Γ i dx dx y Γ i y Γ i dx dx ] [y Γ y Γ i y Γ y Γ i y Γ y Γ i y Γ y Γ i y Γ y Γ i y Γ y Γ i y Γ y Γ i y Γ y Γ i dx dx y Γ i y Γ i dx dx ] i [y Γ y Γ i y Γ y Γ i y Γ y Γ i y Γ y Γ i y Γ y Γ i y Γ y Γ i y Γ y Γ i y Γ y Γ i dx dx y Γ i y Γ i dx dx ] i [y Γ y Γ i y Γ y Γ i y Γ y Γ i y Γ y Γ i y Γ y Γ i y Γ y Γ i y Γ y Γ i y Γ y Γ i dx dx y Γ i y Γ i dx dx ] i [y Γ y Γ i y Γ y Γ i y Γ y Γ i y Γ y Γ i y Γ y Γ i y Γ y Γ i y Γ y Γ i y Γ y Γ i dx dx y Γ i y Γ i dx dx ] [y Γ y Γ i y Γ y Γ i y Γ y Γ i y Γ y Γ i y Γ y Γ i y Γ y Γ i y Γ y Γ i y Γ y Γ i dx dx y Γ i y Γ i dx dx ] i [y Γ y Γ i y Γ y Γ i y Γ y Γ i y Γ y Γ i y Γ y Γ i y Γ y Γ i y Γ y Γ i y Γ y Γ i dx dx y Γ i y Γ i dx dx ] i [y Γ y Γ i y Γ y Γ i y Γ y Γ i y Γ y Γ i y Γ y Γ i y Γ y Γ i y Γ y Γ i y Γ y Γ i dx dx y Γ i y Γ i dx dx ] i i =,. 5. Frame bundle system on G The system space is C F = JF G/G and the system morphism is ξ F = dx λ λ X τ Γ λ τν ν λ = dx µ φ µ φ µ β µ φ µ X µ φ µ X dx µ β µ φ µ X β µ φ µ X dx φ µ β φ µ µ φ µ X µ φ µ X dx µ φ µ β µ φ µ X µ φ µ X.

Khadiga Arwini, L. Del Riego and C.T.J. Dodson 9 The universal connection on the gamma manifold is: Λ F = dx λ λ d i λj i λj b ν τ Γ λ τθdx θ ν λ = dx λ λ d i λj λj i µ φ µ φ µ β µ φ µ b µ φ µ b dx φ µ β φ µ µ φ µ b µ φ µ b dx µ φ µ φ µ β µ φ µ b µ φ µ b dx φ µ β φ µ µ φ µ b µ φ µ b dx µ β µ φ µ b β µ φ µ b dx µ φ µ β µ φ µ b µ φ µ b dx µ β µ φ µ b β µ φ µ b dx µ φ µ β µ φ µ b µ φ µ b dx. The universal curvature on the gamma manifold is: Ω F = b k νγ β kλ ν β b m ω Γ α mµ dx λ dx µ a b m ω Γ α mµ dx a dx µ ω α. The analytic form of this is known but is rather cumbersome and omitted here. 6 Universal connection and curvature for α-connections [8] Consider an exponential family having statistical n-manifold M, g and the system C F M JF M : α, b Γ α b where C = M R is the direct product manifold of M with the standard real line. So the system space C consists of a stack of copies of M. Then every Γ SecC/M is a constant real function on M, so defining precisely one α-connection. In the case of the frame bundle system, M R M F M JF M, the universal connection on the system of α-connections is briefly, Λ : M R M F M JM R M F M T M R F M T M R F M F M, x λ, α, b µ ν [X λ, Y λ µν X λ, Y λ µν, b µ ν α X θ ]. Λ = dx λ λ dα α b µ ν α dx θ ν λ, λ, µ, ν, θ =,.., n, α R Explicitly, each Γ SecM R/M gives an injection Γ π F, I F M, of F M into M R F M, which is a section of π and Γ coincides with the restriction of Λ F to this section. The connection Λ is universal in the following sense. If Γ SecM R/M, then Γ is a constant real function on M, so the induced connection Γ = ξ Γ π F, I F M : F M JF M coincides with restriction on Λ, on the embedding by Γ π F, I F M of F M in M R M F M. So Γ is a pullback of Λ. The universal curvature on the system of α-connections is Ω : M R M F M T M R F M V F M. So Ω has the explicit form: Ω = b k ν α ν β b m ω α dx λ dx µ a b m ω α dx h dx µ ω α, ν, k, β, m, ω, λ, µ, h =,.., n, α R.

0 Universal connection and curvature for statistical manifold geometry 6. Gamma manifold α-connections We provide the α-connections for G, g the -manifold of gamma distributions. Consider the system space C = G R with the fibred morphism over G ξ : G R F G JF G : α, b Γ α b. Here the independent nonzero components Γ iα jk, of the α -connection are Γ α = α µ φ µ β µ φ, µ Γ α = α φ µ µ φ µ, Γ α = α β φ µ µ φ µ, Γ α = Γ α = α µ β µ φ µ, α β µ φ µ, Γ α = α µ φ µ µ φ µ. 6.7 These may be substituted into the above formulae for the universal connection and curvature. Acknowledgement The authors wish to thank CONACYT in Mexico for support of Del Riego during a visit to UMIST, and the Libyan Ministry of Education for a scholarship for Arwini. References [] S. Amari. Differential Geometrical Methods in Statistics, Springer Lecture Notes in Statistics 8, Springer-Verlag, Berlin 985. [] S. Amari and H. Nagaoka. Methods of Information Geometry, American Mathematical Society, Oxford University Press, 000. [3] Khadiga Arwini and C.T.J. Dodson. Neighbourhoods of randomness and information geometry of the McKay bivariate gamma 3-manifold. In Proc. International Mathematica Symposium 003 Imperial College Press, London 003 pp. 47-54. http://www.ma.umist.ac.uk/kd/preprints/ims.nb [4] Khadiga Arwini and C.T.J. Dodson. Information geometric neighbourhoods of randomness and geometry of the McKay bivariate gamma 3-manifold. Sankhya: Indian Journal of Statistics 66, 004 in press. http://www.ma.umist.ac.uk/kd/preprints/gamran.pdf [5] Khadiga Arwini and C.T.J. Dodson. Neighbourhoods of independence and associated geometry. Preprint 004. http://www.ma.umist.ac.uk/kd/preprints/nhdindep.pdf [6] L.A. Cordero, C.T.J. Dodson and M. deleon. Differential Geometry of Frame Bundles. Kluwer, Dordrecht, 989. [7] L. Del Riego and C.T.J. Dodson. Sprays, universality and stability. Math. Proc. Camb. Phil. Soc. 03988, 55-534.

Khadiga Arwini, L. Del Riego and C.T.J. Dodson [8] C.T.J. Dodson. Systems of Connections for Parametric Models. In Proc. Workshop Geometrization of Statistical Theory, Lancaster 8-3 October 987, Ed. C.T.J. Dodson, pp. 53-69, ULDM Publications, Lancaster University 987. [9] C.T.J. Dodson. Space-time edge geometry. Internat. J. Theor. Phys. 7, 6978 389-504. [0] C.T.J. Dodson and M. Modugno. Connections over connections and universal calculus. In Proc. VI Convegno Nazionale di Relativita General a Fisic Della Gravitazione Florence, 0-3 Octobar 984, Eds. R. Fabbri and M. Modugno, pp. 89-97, Pitagora Editrice, Bologna, 986. [] L.Mangiarotti and M.Modugno. Fibred spaces, jet spaces and connections for field theories. In Proc. International Meeting on Geometry and Physics, Florence, -5 October 98, ed. M.Modugno, Pitagora Editrice, Bologna, 983 pp 35-65. [] M. Modugno. Systems of vector valued forms on a fibred manifold and applications to gauge theories. In Proc. Conference Differential Geomeometric Methods in Mathematical Physics, Salamanca 985, Lecture Notes in Mathematics 5, Springer-Verlag, Berlin 987, pp. 38-64.

Universal connection and curvature for statistical manifold geometry Univariate statistical manifold Coordinates Mean Variance α-scalar curvature R α Gaussian µ, σ µ σ α Gamma β, ν β β /ν α φ νν φ ν ν φ ν Exponential β β β 0 Table : Scalar curvature of some common univariate statistical manifolds; φ = Γ Γ. Bivariate statistical manifold Coordinates Covariance α-scalar curvature Bivariate gamma Mckay M α, c, α M M: α = c, α α φ α φ α α c φ α α M M: α = α, c α c α φ α φ α α M 3 M: α α = α, c Bivariate exponential Freund F α, β, α, β α c R α M φ α α α c 0 β β α α 3 α β β α α F F : β = α, β = α α, α 0 0 F F : α = α, β = β α, β 0 F 3 F : β = β = α α α, α, β 4 α β α α α α 0 α α 4 Bivariate Gaussian N µ, µ, σ, σ, σ σ 9 α N N: σ = 0 µ, µ, σ, σ 0 α N N: σ = σ = σ, µ = µ = µ µ, σ, σ σ α N 3 N: µ = µ = 0 σ, σ, σ σ α Table : Scalar curvature of some common bivariate statistical manifolds; see equation 6.8 for R α M. Appendix For convenience of reference we summarize curvature results in Table for some common univariate exponential families, and in Table for important bivariate distributions from recent work of Arwini and Dodson [4, 5, 3]. We have used Mathematica for many calculations and we can make available the associated interactive Notebooks. The α-scalar curvature for the McKay bivariate gamma manifold M has too long an expression for Table so we give it here with: R α M = α φ α φ α φ α φ α φ α φ α φ α φ α φ α φ α φ α α α φ α φ α φ α φ α φ α α α φ α φ α φ α φ α α α, 6.8 φα i = Γ α i Γα i.