Probability on a Riemannian Manifold

Transcription

1 Probability on a Riemannian Manifold Jennifer Pajda-De La O December 2, Introduction We discuss how we can construct probability theory on a Riemannian manifold. We make comparisons to this and how probability is thought of on R n. The basic definitions for probability on R n come from Resnick [2005]. Information about probability theory on a Riemannian manifold is taken from Pennec [1999]. Fisher Information on R n and the Delta Theorem on R n are taken from Martin [2015]. We organize the paper as follows. Each section covers a different topic in probability/statistics. We summarize the importance of each concept, and then show, side by side, each concept in R n (on the left) and on a Riemannian manifold (on the right). From this paper, it is hoped that the similarities and differences between probability in R n and on a Riemannian manifold are made clear. 2 Definitions We define important terms, abbreviations, and recall important definitions here. We combine the definitions that are required for probability on R n and on a Riemannian manifold. B : a σ-field Note: A σ-field has the following properties: 1. Ω B; 2. If B B then B C B, where C denotes the complement; 3. If B i B, i 1, then i=1b i B. B(R): Borel σ-field. This is the σ-field that is generated by open subsets of R, i.e. B(R) = σ ({(a, b], a b < }) = σ ({[a, b), < a b }) = σ ({[a, b], < a b < }) = σ ({(a, b), a b }) = σ ({open sets of R}). 1

2 A : A Borel tribe / σ-field (Ω, B, P) is a probability space. In particular, P is a probability measure on the measurable space (Ω, B). Note: A probability measure has the following properties: 1. P( ) = 0; P(Ω) = 1; 2. 0 P(B) 1, B B; 3. If {B n, n 1} are disjoint events in B, then P ( n=1b n ) = n=1 P(B n). E: Expected Value E: Mean value (Fréchet Expectation) V : Variance COV : Covariance 1 A : Indicator Function of a set A M : Riemannian Manifold : Matrix Transpose iid: Independent and Identically Distributed D : Convergence in Distribution Note: We can define convergence in distribution as follows: Suppose {F, F n, n 1} are probability distributions. Then X n D X means that F n D F i.e. X n converges in distribution to X or F n converges weakly to F. Convergence in distribution only happens when the four types of convergence stated below are equivalent. Equivalence will only happen if F is proper, i.e. F ( ) = 0 and F ( ) = 1. Resnick [2005] Let {F n, n 1} be probability distribution functions and let F be a distribution function which is not necessarily proper. 1. Vague Convergence: The seqeunce {F n } converges vaguely to F, written F n v F, if for every finite interval of continuity I of F, we have F n (I) F (I). 2. Proper Convergence: The sequence {F n } converges properly to F if F n v F and F is a proper distribution function; that is F (R) = Weak Convergence: The sequence {F n } converges weakly to F, written F n w F, if F n (x) F (x) for all x C (F ). Here, C (F ) = {x R: F is continuous at x}. 4. Complete Convergence: The sequence {F n } converges complete to F, written F n c F, if F n w F and F is proper. 2

3 3 Setup In this section, we give the basic setup for each scenario. 3.1 In R n Let (Ω, B, P) be a probability space. We can define a random variable as follows: (Ω, B, P) X (R, B(R)). So X is a random variable with the property that X 1 ((, λ]) = [X λ] B, λ R. There is an induced measure on (R, B(R)). This induced measure is P X 1. We can also define a distribution function of X as 3.2 Riemannian Manifold Let M be a connected Riemannian manifold. Take a continuous collection of dot products on the tangent space T X M. The basis is i j X = Q(X). In particular, Q(X) is a positive definite matrix. The distance between two points on the manifold is defined as the minimum length among smooth curves joining the points. The curves that are created from this procedure are called geodesics. There is a unique geodesic starting at a given point X with a given tangent vector. We consider an exponential map, such that each vector is mapped to the corresponding point( on the manifold. In particular, let y = exp X XY ). The measure that we will be considering on the manifold is the volume measure. This volume measure is induced by the basis, Q(X). Specifically, dm(x) = Q(X) dx. F (A) = P X 1 (A) = P[X A]. For a point, F (x) = F ((, x]) = P[X x]. The density function is given as f(x), or f θ (x), when we need to remind ourselves what the parameter of our distribution is. On M, we can define a probability space (Ω, B, P). Let X : Ω M be a random primitive. The induced measure is P X 1. In particular, take (Ω, B, P) X (M, A ). Let p X (y) be the pdf of X. 3

4 4 Probability of a Set or Event Occurring We often want to know what is the probability of a certain event, therefore, it is important that we have a definition for this. The probability of an event occurring depends on its distribution. This can be given by its distribution function, or by its probability mass function (in the case of discrete variables), or by its probability density function (in the case of continuous variables). We denote the probability of an event A occurring by P [X A], regardless of whether the random variable X is continuous or discrete. For the discrete case, we can change the integral to a sum. 4.1 In R n P X 1 (A) = P [X A] = 1 A (X)dP Ω = 1 A (X(ω))P(dω) Ω = 1 A (x)df (x) R = df (x) A = f(x)dx. A Note that the last three equalities are only true for X as a random variable. 4.2 Riemannian Manifold P [X M] = P [X A] = M A p X (y)dm(y) = 1, p X (y)dm(y). Example: A Uniform PDF in a bounded set A: p X (y) = 1 A(y) V (A), where V (A) is the volume of the set A with respect to the measure dm. Example: A (continuous) Uniform PDF on the interval [a, b]. { 1 for x [a, b] b a f(x) = 0 otherwise = 1 [a,b](x) b a. 4

5 5 Expected Value of a Function Expected value measures a long-run average of a random variable/primitive or a function of a random variable/primitive. For example, we want to know how much we will win or lose if we are gambling at dice. Depending on the roll of the dice, we would either win a certain amount of money, lose a certain amount of money, or break-even. Assuming the dice are fair, we know the probability of each of the outcomes from rolling the dice. We can calculate how much, on average, we would gain/lose from playing the game. This is assuming that we will play the game long enough. In our calculations, we first start out with finding the expected value of a function, g(x) or ϕ(x). If we find the expected value of an identity map, i.e. g(x) = X or ϕ(x) = X, then this would be a long-run average of the random variable/primitive X. The expected value is also the first moment of a distribution. Moments are used to try and distinguish one distribution from another. 5.1 In R n Take g(x) to be a measurable function of X. Then E(g(X)) = g(x)dp Ω = g(x(ω))pd(ω) Ω = g(x)df (x) R = g(x)f(x)dx. R Note that the last two equalities are only true for X as a random variable. 5.2 Riemannian Manifold Let ϕ(x(ω)) be a Borel-real valued function defined on M. Then ϕ(x) is a real random primitive. The expected value is given as E [ϕ(x)] = ϕ(y)p X (y)dm(y). M 5

6 6 Variance Variance, or the square of the standard deviation is very important in statistics. It is also the second moment of a distribution. Variance measures the spread of the data. For example, if the variance is large, then the data-points are more spread-out from the mean, while a small variance would indicate that the data is more centered around the mean. We can also calculate variances of a function of the statistic we are interested in; this is the Delta Theorem, and will be stated in Section In R n If X 2 L 1, then X L 2. Define variance as V(X) = E (X E(X)) 2 = E ( X 2) [E (X)] 2. Theorem 1. (Delta Theorem) For random variables T n, assume that n 1/2 (T n θ) D N (0, v(θ)), where v(θ) is the asymptotic variance. Let g( ) be a function differentiable at θ, with g (θ) 0. Then n 1/2 {g(t n ) g(θ)} D N (0, v g (θ)), where v g (θ) = [g (θ)] 2 v(θ). 6.2 Riemannian Manifold Suppose dist (X, Y ) 2 = XY 2. Then σx(y) 2 = E [ dist (y, X) 2] = dist (y, z) 2 p X (z)dm(z). M 6

7 7 Covariance Covariance is important when you have multiple random variables with different distributions. It is possible for random variables to be related to one another in some way, and the covariance is one metric that displays this relationship (another is the correlation, correlation is a scaled covariance). For example, if there is a positive covariance between two random variables, this means that they will behave similarly (a positive correlation); if there is a negative covariance between two random variables, this means that they will act opposite to one another (a negative correlation). When the covariance is 0, this means that they are uncorrelated. If two random variables are independent, then their covariance will always be zero; the converse may not necessarily be true. 7.1 In R n If X, Y L 2, then COV(X, Y ) = E [(X E(X)) (Y E(Y ))] = E(XY ) E(X)E(Y ). If X and Y are random vectors, then 7.2 Riemannian Manifold Covariance calculations depend on how we view our chart. For example, if we consider our chart as a matrix, then covariance depends on the choice of basis. However, if we do not view our chart as a matrix and we view it as a bilinear form over the tangent plane, then covariance calculations do not depend on the basis. Earlier we defined E as the mean value, or Fréchet expectation. In particular, E is the set of mean primitives. We can also define it as E [X] = arg min y M E [ dist (y, X) 2]. ( ) [ ( ) ( ) ] COV X, Y = E X EX Y EY = E X Y E XE Y. Suppose X E [X]. We take X to be the unique mean value of X. Then Σ XX = COV X(X) [ = E ] XX XX. 7

8 8 Fisher Information One interpretation of Fisher Information is that variance is small if the Fisher information is big. This can be seen from the Cramér-Rao Lower Bound. We display this theorem in Section In R n Suppose θ is n dimensional and f θ (x) is the density of X with respect to P. Then the following are the FI regularity conditions. 1. f θ (x)/ θ i exists P-a.e. for each i. 2. f θ (x)dp(x) can be differentiated under the integral sign. 3. The support of f θ is the same for all θ. Fisher Information may be defined as: I X (θ) ij = COV θ log f θ (X), log f θ(x) θ i θ j }{{} score vector { } 2 = E θ log f θ (X), θ i θ j where the last equality is true provided that we can differentiate twice under the integral sign. Theorem 2. (Cramér-Rao Lower Bound) For simplicity, take θ to be a scalar, and assume that f θ satisfies the FI regularity conditions. Let X 1,..., X n f θ and let T = iid T (X 1,..., X n ) be a real-valued statistic with E θ (T ) = g(θ). Then V θ (T ) {g (θ)} 2 {ni(θ)} Riemannian Manifold Information is given by I(X). Entropy is H(X). I(X) = H(X) = E [log p X (X)]. 8

9 9 Multivariate Normal Distribution The normal distribution is highly used in statistics. In many cases, as long as the same size is large enough, we can approximate the distribution of our data as a normal distribution because of the Central Limit Theorem (CLT) for iid random variables. Note that there is also a CLT for the case where random variables are independent, but not necessarily identically distributed; this is the Lindeberg-Feller CLT. We give the CLT for the R n case below. Theorem 3. (CLT) Let {X n, n 1} be iid random variables with E(X n ) = µ and V(X n ) = σ 2. Suppose N is a random variable with N(0, 1) distribution. If S n = X X n, then S n nµ σ n D N. Moreover, the normal distribution has many nice properties so various calculations are made easier to complete. 9.1 In R n Let X = [X 1,..., X k ] be a vector such that X N (µ, Σ). Then the pdf of X is given by f(x) = (2π) k/2 Σ 1/2 { exp 1 } 2 (X µ) Σ 1 (X µ). (1) This pdf exists only when Σ is positive definite. The entropy (or I X (θ)) is given by k 2 (1 + ln (2π)) ln Σ. Suppose X N (µ, σ 2 ). Then the pdf of X is given by f(x) = ( 2πσ 2) { } 1/2 (x µ)2 exp. 2σ Riemannian Manifold The pdf [of a normal distribution on a manifold tries to minimize] the information with a fixed mean value and covariance. Suppose we have a cut locus C( X) with no continuity or differentiability constraint, a symmetric domain D( X), and a concentration matrix Γ. Suppose k is a normalization constant. Then the Normal Distribution pdf is given by Xy N ( X,Γ) [ (y) = k exp Γ Xy ] 2 [ Xy k 1 Γ Xy ] = exp dm(y) M 2 Σ = k Xy Xy [ Xy Γ Xy ] exp dm(y). 2 M Note that a high concentration matrix Γ occurs if and only if there is a small covariance matrix Σ. The equation in Section 9.2 will give the Gaussian PDF shown in Equation (1) when working in a vector space. 9

10 References Ryan Martin. Lecture notes on advanced statistical theory. Supplement to the lectures for Stat 511 at UIC given by the author, January Xavier Pennec. Probabilities and statistics on riemannian manifolds: Basic tools for geometric measurements. In Proc. of Nonlinear Signal and Image Processing (NSIP 99), pages , Sidney Resnick. A Probability Path. Birkhäuser,