Universal connection and curvature for statistical manifold geometry

Transcription

1 Universal connection and curvature for statistical manifold geometry Khadiga Arwini, L. Del Riego and C.T.J. Dodson School of Mathematics, University of Manchester, Manchester M60 QD, UK and Departamento de Matemáticas, Facultad de Ciencias Universidad Autónoma de San Luis Potosí, San Luis Potosí, SLP, México June 7, 005 Abstract Statistical manifolds are representations of smooth families of probability density functions that allow differential geometric methods to be applied to problems in stochastic processes, mathematical statistics and information theory. It is common to have to consider a number of linear connections on a given statistical manifold and so it is important to know the corresponding universal connection and curvature; then all linear connections and their curvatures are pullbacks. An important class of statistical manifolds is that arising from the exponential families and one particular family is that of gamma distributions, which we showed recently to have important uniqueness properties in stochastic processes. Here we provide formulae for universal connections and curvatures on exponential families and give an explicit example for the manifold of gamma distributions. Introduction Information geometry is the study of Riemannian geometric properties of statistical manifolds consisting of smooth families of probability density functions. Such manifolds are endowed with the information metric of Rao [], which arose from the Fisher information matrix [3]. These parts of mathematical statistics have deep relations with general information theory; see eg Roman [3] for a modern account of information theory from a mathematical viewpoint. For our present purposes we may view a probability density function on R m as a subadditive measure function of unit weight, namely, a nonnegative map A B f : [0, f = f A f f, A, B. B Usually, a probability density function depends on a set of parameters, θ, θ,..., θ n and we say that we have an n-dimensional family of probability density functions. Let Θ be the parameter space of an n-dimensional smooth such family defined on some fixed event space {p θ θ Θ} with p θ = for all θ Θ. Then, the derivatives of the log-likelihood function, l = log p θ, yield a matrix with entries l l l g ij = p θ θ i θ j = p θ θ i θ j,

2 Universal connection and curvature for statistical manifold geometry for coordinates θ i about θ Θ R n. This gives rise to a positive definite matrix, so inducing a Riemannian metric g on Θ using for coordinates the parameters θ i ; this metric is called the information metric for the family of probability density functions the second equality here is subject to certain regularity conditions. Amari [] and Amari and Nagaoka [] provide modern accounts of the differential geometry that arises from the Fisher information metric. Systems of connections and universal objects The concept of system or structure of connections was introduced by Mangiarotti and Modugno [8, 9], they were concerned with finite-dimensional bundle representations of the space of all connections on a fibred manifold. On each system of connections there exists a unique universal connection of which every connection in the family of connections is a pullback. A similar relation holds between the corresponding universal curvature and the curvatures of the connections of the system. This is a different representation of an object similar to that introduced by Narasimhan and Ramanan [0], [] for G-bundles, also allowing a proof of Weil s theorem cf. [6, 4, 8]. Definition. A system of connections on a fibred manifold p : E M is a fibred manifold p c : C M together with a first jet-valued fibred morphism ξ : C M E JE over M, such that each section Γ : M C determines a unique connection Γ = ξ Γ p, I E on E. Then C is the space of connections of the system. In the sequel we are interested in the system of linear connections on a Riemannian manifold. The system of all linear connections is the subject of studies in eg. [6, 4, 7, 8, 9, 0, 7, ]. Theorem. [8, 9] Let C, ξ be a system of connections on a fibred manifold p : E M. Then there is a unique connection form Λ : C M E JC M E On the fibred manifold π : C M E C with the coordinate expression Λ = dx λ λ dc a a ξ i λ dx λ i. This Λ is called the universal connection because it describes all the connections of the system. Explicitly, each Γ SecC/M gives an injection Γ p, I E, of E into C E, which is a section of π and Γ coincides with the restriction of Λ to this section: Λ Γ p,ie E = Γ. A similar relation holds between its curvature, called universal curvature, and the curvatures of the connections of the system. = [Λ, Λ] = d ΛΛ : C M E T C E V E. So the universal curvature has the coordinate expression: = ξ j λ jξη i dx λ dx η a ξη i dx a dx η i. 3 Exponential family of probability density functions on R An important class of statistical manifolds is that arising from the so-called exponential family [] and one case is that of gamma distributions, which we showed recently in [4, 5] to have important uniqueness

3 Khadiga Arwini, L. Del Riego and C.T.J. Dodson 3 properties for near-random stochastic processes. Note also that Hwang and Hu [5] provided an important new characterization of gamma distributions, which helps understanding of their common application in modelling real processes. More details on statistical manifolds in general can be found in [, ] and we have provided in [6] explicit geometric neighbourhoods of independence for common bivariate processes. In the present section we shall be concerned with the system of all linear connections on the manifold of an arbitrary exponential family, using the tangent bundle or the frame bundle to give the system space. We provide formulae for the universal connections and curvatures and give an explicit example for the manifold of gamma distributions. An n-dimensional set of probability density functions S = {p θ θ Θ R n } for random variable x R is said to be an exponential family [] when the density functions can be expressed in terms of functions {C, F,..., F n } on R and a function ϕ on Θ as: p θ x = e {CxP i θi F ix ϕθ}. Then we say that θ i are its natural coordinates, and ϕ is its potential function. From the normalization condition p θx dx = we obtain: ϕθ = log e {CxP i θi F ix} dx. From the definition of an exponential family, and putting i = lθ, x = logp θ x to obtain and i lθ, x = F i x i ϕθ i j lθ, x = i j ϕθ. θ i, we use the log-likelihood function The Fisher information metric g [, ] on the n-dimensional space of parameters Θ R n, equivalently on the set S = {p θ θ Θ R n }, has coordinates: [g ij ] = [ i j lθ, x] p θ x dx = i j ϕθ = ϕ ij θ. Then, S, g is a Riemannian n-manifold with Levi-Civita connection given by: Γ k ijθ = gkh i g jh j g ih h g ij = h= h= where [ϕ hk θ] represents the inverse to [ϕ hk x]. gkh i j h ϕθ = h= ϕkh θ ϕ ijh θ Next we obtain a family of symmetric connections which includes the Levi-Civita case and has significance in mathematical statistics. Consider for α R the function Γ α ij,k which maps each point θ Θ to the following value: Γ α ij,k θ = i j l α i l j l k l p θ = α i j k ϕθ = α ϕ ijk θ. So we have an affine connection α on the statistical manifold S, g defined by g α i j, k = Γ α ij,k, where g is the Fisher information metric. We call this α the α-connection and it is clearly a symmetric connection and defines an α-curvature. We have also α = α 0 α, = α α.

4 4 Universal connection and curvature for statistical manifold geometry µ ν.5.5 z 4 0 Figure : Affine immersion using natural coordinates µ, ν in R 3 for the gamma -manifold G, g. The surface is shaded by the Gaussian curvature K G which is independent of µ and monotonically decreases from 4 to almost as ν increases to. For a submanifold M S, the α-connection on M is simply the restriction with respect to g of the α-connection on S. Note that the 0-connection is the Riemannian or Levi-Civita connection with respect to the Fisher metric and its uniqueness implies that an α-connection is a metric connection if and only if α = Example: Gamma -manifold G, g Gamma distributions form an exponential family with probability density functions: G = {px; µ, ν = µ ν xν Γν e xµ for µ, ν R }. 3. The gamma distributions are very important in, information theory, mathematical statistics and stochastic processes. This is because they contain as a special case, ν =, the negative exponential distribution that represents random states, ie Poisson processes, and because of certain uniqueness properties [5, 5]. It turns out that θ i = µ, ν is a natural coordinate system with corresponding potential function The information metric g has arc length function ϕµ, ν = log Γν ν log µ. 3. ds g = ν µ dµ µ dµ dν ψ ν dν for µ, ν R where ψν = Γ ν Γν is the digamma function. The independent nonzero Levi-Civita connection compo-

5 Khadiga Arwini, L. Del Riego and C.T.J. Dodson 5 Univariate density Coordinates Mean Variance R α Gaussian µ, σ µ σ α Gamma µ, ν µ µ /ν α ψ νν ψ ν ν ψ ν Exponential µ µ µ 0 Table : α-scalar curvature R α of the univariate Gaussian, gamma and exponential statistical manifolds; the logarithmic derivative of the gamma function is denoted by ψ = Γ /Γ. The case α = 0 corresponds to the Levi-Civita connection. nents with respect to the natural coordinates µ, ν are: Γ = Γ = Γ = Γ = Γ = Γ = ν ψ ν µ ν ψ ν, ψ ν ν ψ ν, µ ψ ν ν ψ ν, ν µ ν ψ ν, µ ν ψ ν, ν ψ ν ν ψ ν. The Riemannian -manifold G, g has been shown by Dodson and Matsuzoe [] to admit an affine immersion in R 3. This is depicted in Figure, shaded by the Gaussian curvature K G, which is independent of µ and, with increasing ν, K G monotonically decreases from 4 to : K G = ψ ν ν ψ ν 4 ν ψ ν. To compute the α-connection components it is convenient here to change to the orthogonal coordinates β = ν/µ, ν for which the metric components are given by ds = ν β dβ ψ ν dν for β, ν R. ν Proposition 3. Arwini [3] The independent nonzero components, Γ αi jk, of α are Γ α = α, β Γ α = α ν, Γ α = α ν β ν ψ ν, Γ α = α ν ψ ν ν ν ψ. ν Corollary 3. The Levi Civita connection of G, g is recovered by Γ 0i jk the curves ν = constant are geodesics. in β, ν coordinates and then

6 6 Universal connection and curvature for statistical manifold geometry Bivariate density Coordinates Covariance R α Mckay M α, c, α α /c R α M M M: α = c, α /c R M α M M: α = α, c α /c R M α M 3 : α α = α, c α /c 0 Table : α-scalar curvature R α of the McKay bivariate gamma manifold; see 3. for the formulae R α M, R M α, R M α. The case α = 0 corresponds to the Levi-Civita connection. Bivariate density Coordinates Covariance R α Freund F α, β, α, β β β α α 3 α β β α α F F : β i = α i α, α 0 0 F : α = α, β = β α, β 4 α β 0 F 3 : β i = α α α, α, β α α α α α α 4 0 Table 3: α-scalar curvature R α of the Freund bivariate exponential manifold. The case α = 0 corresponds to the Levi-Civita connection.

7 Khadiga Arwini, L. Del Riego and C.T.J. Dodson 7 Bivariate density Coordinates Covariance R α Gaussian N µ, µ, σ, σ, σ σ 9 α N N: σ = 0 µ, µ, σ, σ 0 α N : σ i = σ, µ i = µ µ, σ, σ σ α N 3 : µ = µ = 0 σ, σ, σ σ α Table 4: α-scalar curvature R α of the bivariate Gaussian manifold. The case α = 0 corresponds to the Levi-Civita connection. 3. α-scalar curvature for common distributions For convenience of reference we summarize curvature results in Table for univariate Gaussian, gamma and exponential distributions and in Tables,3,4 respectively for bivariate gamma, exponential and Gaussian distributions, from recent work of Arwini and Dodson [4, 5, 6]. We have used Mathematica for many calculations and we can make available the associated interactive Notebooks. The α-scalar curvature for the McKay bivariate gamma manifold M and its submanifolds M, M, have long expressions so we give them here: R α M = α ψ α ψ α ψ α ψ α ψ α ψ α ψ α ψ α ψ α ψ α ψ α α α ψ α ψ α ψ α ψ α ψ α α α ψ α ψ α ψ α ψ α α α, R M α = R M α = ψα i = Γ α i Γα i. α ψ α ψ α α ψ α α α ψ α ψ α α ψ α α. 4 Systems of linear connections 4. Tangent bundle system: C T T M JT M The system of all linear connections on a manifold M has a representation on the tangent bundle with system space E = T M M C T = {α jγ T M M JT M jγ : T M T T M projects onto I T M } Here we view I T M as a section of T M T M, which is a subbundle of T M T T M, with local expression dx λ λ.

8 8 Universal connection and curvature for statistical manifold geometry The fibred morphism for this system is given by In coordinates x λ on M and y λ on T M ξ T : C T M T M JT M T M T M T T M, α jγ, ν ανjγ. ξ T = dx λ λ γ i λ i = dx λ λ y j Γ i jλ i = dx λ λ y j ϕih ϕ jλh i Each section of C T M, such as Γ : M C T : x λ x λ, γ ηθ ; determines the unique linear connection Γ = ξ T Γ π T, I T M with Christoffel symbols Γ λ ηθ. On the fibred manifold π : C T M T M C T ; the universal connection is given by: h= briefly, Λ T : C T M T M JC T M T M T C T T C T M T M, x λ, vηκ, λ y λ [X λ, Vηκ λ X λ, Vηκ, λ Y η VηκX λ ]. Λ T = dx λ λ dv a a y η vηκ i dx κ i = dx λ λ dv a a y η ϕih ϕ ηκh dx κ i. Explicitly, each Γ SecC T /M gives an injection Γ π T, I T M, of T M into C T T M, which is a section of π and Γ coincides with the restriction of Λ T to this section: h= Λ T Γ π T,I T M T M = Γ. and the universal curvature of the connection Λ is given by: T = d ΛT Λ T : C T M T M T C T T M V T M. So here the universal curvature T has the coordinate expression: T = y k v j kλ jy m vmη i dx λ dx η a y m vmη i dx a dx η i = y k ϕjh ϕ kλh j y m ϕih ϕ mηh dx λ dx η i h= h= a y m ϕih ϕ mηh dx a dx η i. h= 4. Frame bundle system: C F F M JF M A linear connection is also a principal i.e. group invariant connection on the principal bundle of frames F M with: E = F M M = F M/G consisting of linear frames ordered bases for tangent spaces with structure group the general linear group, G = Gln. Here the system space is

9 Khadiga Arwini, L. Del Riego and C.T.J. Dodson 9 C F = JF M/G T M T M T F M/G, consisting of G-invariant jets. The system morphism is In coordinates ξ F : C F F M JF M T M T M T F M, [js x ], b [T x M T b F M]. ξ F = dx λ λ X η η s λ κ κ λ = dx λ λ X η Γ λ ηκ κ λ = dx λ λ X η n h= ϕλh ϕ ηκh κ λ where κ λ = is the natural base on the vertical fibre of T b λ b F M induced by coordinates b λ κ on F M. κ Each section of C F M that is projectable onto I T M, such as, ˆΓ : M C F : x λ x λ, [jγ x ] with Γ λ ηκ = η s λ κ; determines the unique linear connection Γ = ξ F ˆΓ π F, I F M with Christoffel symbols Γ λ ηκ. On the principal G-bundle π : C F M F M C F the universal connection is given by: Briefly, Λ F : C F M F M JC F M F M T C F F M T C F F M F M, x λ, vηκ, λ b η κ [X λ, Yηκ λ X λ, Yηκ, λ b η κvηθx λ θ ]. Λ F = dx λ λ dv a a b η κvηθ λ dx θ κ λ = dx λ λ dv a a b η κ ϕλh ϕ ηθh dx θ κ λ. Explicitly, each Γ SecC F /M gives an injection Γ π F, I F M, of F M into C F F M, which is a section of π and Γ coincides with the restriction of Λ F to this section: h= Λ F Γ π F,I F M F M = Γ and the universal curvature of the connection Λ is given by: = d ΛF Λ F : C F M F M T C F F M V F M. So here the universal curvature form F has the coordinate expression: F = b k κv β kλ κ β b m ω vmη α dx λ dx η a b m ω vmη α dx a dx η ω α = b k κ ϕβh ϕ kλh κ β b m ω ϕαh ϕ mηh dx λ dx η ω α h= h= a b m ω ϕαh ϕ mηh dx a dx η ω α. h= 5 Universal connection and curvature on the gamma manifold For the gamma -manifold G, g we give explicit forms for the system space and its universal connection and curvature.

10 0 Universal connection and curvature for statistical manifold geometry 5. Tangent bundle system on G, g The system space is and the system morphism is C T = {α jγ T G G JT G jγ : T G T T G projects onto I T G } ξ T = dx λ λ y j Γ i jλ i = dx ν ψ ν ψ ν µ ν ψ ν y ν ψ ν y dx ν µ ν ψ ν y µ ν ψ ν y dx ψ ν µ ψ ν ν ψ ν y ν ψ ν y dx ν ψ ν µ ν ψ ν y ν ψ ν y. The universal connection on the gamma manifold is given by: Λ T = dx λ λ d i λj i λj y τ Γ i τκdx κ i = dx λ λ d i λj λj i ν ψ ν ψ ν µ ν ψ ν y ν ψ ν y dx ν µ ν ψ ν y µ ν ψ ν y dx ψ ν µ ψ ν ν ψ ν y ν ψ ν y dx ν ψ ν µ ν ψ ν y ν ψ ν y dx. The universal curvature on the gamma manifold is: T = y k Γ j kλ jy m Γ i mκ dx λ dx κ a y m Γ i mκ dx a dx κ i i =,. The analytic form of this is known [3] but is omitted here. 5. Frame bundle system on G, g The system space is C F = JF G/G and the system morphism is ξ F = dx λ λ X τ Γ λ τκ κ λ = dx ν ψ ν ψ ν µ ν ψ ν X ν ψ ν X dx ν µ ν ψ ν X µ ν ψ ν X dx ψ ν µ ψ ν ν ψ ν X ν ψ ν X dx ν ψ ν µ ν ψ ν X ν ψ ν X.

11 Khadiga Arwini, L. Del Riego and C.T.J. Dodson The universal connection on the gamma manifold is: Λ F = dx λ λ d i λj i λj b κ τ Γ λ τθdx θ κ λ = dx λ λ d i λj λj i ν ψ ν ψ ν µ ν ψ ν b ν ψ ν b dx ψ ν µ ψ ν ν ψ ν b ν ψ ν b dx ν ψ ν ψ ν µ ν ψ ν b ν ψ ν b dx ψ ν µ ψ ν ν ψ ν b ν ψ ν b dx ν µ ν ψ ν b µ ν ψ ν b dx ν ψ ν µ ν ψ ν b ν ψ ν b dx ν µ ν ψ ν b µ ν ψ ν b dx ν ψ ν µ ν ψ ν b ν ψ ν b dx. The universal curvature on the gamma manifold is: F = b k κγ η kλ κ ηb m ω Γ α mκ dx λ dx κ a b m ω Γ α mκ dx a dx κ ω α. The analytic form of this is known [3] but is omitted here. 6 Universal connection and curvature for α-connections [0] Consider an exponential family having statistical n-manifold M, g and the system C F M JF M : α, b Γ α b where C = M R is the direct product manifold of M with the standard real line. So the system space C consists of a stack of copies of M. Then every Γ SecC/M is a constant real function on M, so defining precisely one α-connection. In the case of the frame bundle system, M R M F M JF M, the universal connection on the system of α-connections is briefly, Λ : M R M F M JM R M F M T M R F M T M R F M F M, x λ, α, b η κ [X λ, Yηκ λ X λ, Yηκ, λ b η κ α X θ ]. Λ = dx λ λ dα α b η κ α dx θ κ λ, λ, η, κ, θ =,.., n, α R Explicitly, each Γ SecM R/M gives an injection Γ π F, I F M, of F M into M R F M, which is a section of π and Γ coincides with the restriction of Λ F to this section. The connection Λ is universal in the following sense. If Γ SecM R/M, then Γ is a constant real function on M, so the induced connection Γ = ξ Γ π F, I F M : F M JF M coincides with restriction on Λ, on the embedding by Γ π F, I F M of F M in M R M F M. So Γ is a pullback of Λ. The universal curvature on the system of α-connections is : M R M F M T M R F M V F M.

12 Universal connection and curvature for statistical manifold geometry So has the explicit form: = b k κ α κ β b m ω α dx λ dx η a b m ω α dx h dx η ω α, for κ, k, β, m, ω, λ, η, h =,.., n, α R. References [] S. Amari. Differential Geometrical Methods in Statistics, Springer Lecture Notes in Statistics 8, Springer-Verlag, Berlin 985. [] S. Amari and H. Nagaoka. Methods of Information Geometry, American Mathematical Society, Oxford University Press, 000. [3] Khadiga Arwini. Differential geometry in neighbourhoods of randomness and independence PhD thesis, Department of Mathematics, UMIST 004. [4] Khadiga Arwini and C.T.J. Dodson. Neighbourhoods of randomness and information geometry of the McKay bivariate gamma 3-manifold. In Proc. International Mathematica Symposium 003 Imperial College Press, London 003 pp [5] Khadiga Arwini and C.T.J. Dodson. Information geometric neighbourhoods of randomness and geometry of the McKay bivariate gamma 3-manifold. Sankhya: Indian Journal of Statistics 66, [6] Khadiga Arwini and C.T.J. Dodson. Neighbourhoods of independence and associated geometry. Preprint [7] D.Canarutto and C.T.J.Dodson. On the bundle of principal connections and the stability of b- incompleteness of manifolds. Math. Proc. Camb. Phil. Soc. 98, [8] L.A. Cordero, C.T.J. Dodson and M. deleon. Differential Geometry of Frame Bundles. Kluwer, Dordrecht, 989. [9] L. Del Riego and C.T.J. Dodson. Sprays, universality and stability. Math. Proc. Camb. Phil. Soc , [0] C.T.J. Dodson. Systems of Connections for Parametric Models. In Proc. Workshop Geometrization of Statistical Theory, Lancaster 8-3 October 987, Ed. C.T.J. Dodson, pp , ULDM Publications, Lancaster University 987. [] C.T.J. Dodson and Hiroshi Matsuzoe. An affine embedding of the gamma manifold. Appl. Sci., [] C.T.J. Dodson and M. Modugno. Connections over connections and universal calculus. In Proc. VI Convegno Nazionale di Relativita General a Fisic Della Gravitazione Florence, 0-3 Octobar 984, Eds. R. Fabbri and M. Modugno, pp , Pitagora Editrice, Bologna, 986. [3] R.A. Fisher. Theory of statistical estimation. Proc. Camb. Phil. Soc [4] P.L. Garcia. Connections and -jet fibre bundles. Rend.Sem. Mat. Univ.Padova 47, [5] T-Y. Hwang and C-Y. Hu. On a characterization of the gamma distribution: The independence of the sample mean and the sample coefficient of variation. Annals Inst. Statist. Math. 5, [6] S. Kobayashi and K. Nomizu. Foundations of differential geometry II.Interscience New York 969. [7] S. Kumar. A remark on universal connections. Math. Ann. 60,

13 Khadiga Arwini, L. Del Riego and C.T.J. Dodson 3 [8] L. Mangiarotti and M. Modugno. Fibred spaces, jet spaces and connections for field theories. In Proc. International Meeting on Geometry and Physics, Florence, -5 October 98, ed. M.Modugno, Pitagora Editrice, Bologna, 983 pp [9] M. Modugno. Systems of vector valued forms on a fibred manifold and applications to gauge theories. In Proc. Conference Differential Geomeometric Methods in Mathematical Physics, Salamanca 985, Lecture Notes in Mathematics 5, Springer-Verlag, Berlin 987, pp [0] M.S. Narasimhan and S.Ramanan. Existence of universal connections I. Amer. J. Math. 83, [] M.S. Narasimhan and S.Ramanan. Existence of universal connections II. Amer. J. Math. 85, [] C.R. Rao. Information and accuracy attainable in the estimation of statistical parameters. Bull. Calcutta Math. Soc. 37, [3] S. Roman. Coding and Information Theory, Graduate Texts in Mathematics, 34 Springer- Verlag, New York, 99, cf. Part.