Geometric Inference for Probability distributions

Transcription

1 Geometric Inference for Probability distributions F. Chazal 1 D. Cohen-Steiner 2 Q. Mérigot 2 1 Geometrica, INRIA Saclay, 2 Geometrica, INRIA Sophia-Antipolis 2009 July 8 F. Chazal, D. Cohen-Steiner, Q. Mérigot (Geometrica) Inference for Probability distributions 2009/07/08 1 / 32

2 Outline 1 Geometric inference for measures. 2 Distance to a probability measures. 3 Applications F. Chazal, D. Cohen-Steiner, Q. Mérigot (Geometrica) Inference for Probability distributions 2009/07/08 2 / 32

3 Motivation What is the (relevant) topology/geometry of a point cloud data set in R d? Motivations : Reconstruction, manifold learning and NLDR, clustering and segmentation, etc.... Chazal, D. Cohen-Steiner, Q. Mérigot (Geometrica) Inference for Probability distributions 2009/07/08 3 / 32

4 Geometric Inference Question Given an approximation C of a geometric object K, what geometric and topological quantities of K is it possible to approximate, knowing only C? The answer depends on the considered class of objects and a notion of distance between the objects (approximation) ; Some positive answers for a large class of compact sets endowed with the Hausdor distance. In this talk : - can the considered objects be probability measures on R d? - motivation : allowing approximations to have outliers or to be corrupted by non local noise. F. Chazal, D. Cohen-Steiner, Q. Mérigot (Geometrica) Inference for Probability distributions 2009/07/08 4 / 32

5 Distance functions for geometric inference Distance function and Hausfor distance Distance to a compact K R d : d K : x inf p K x p Hausdor distance between compact sets K, K R d : d H (K, K ) = inf x R d d K (x) d K (x) Replace K and C by d K and d C. Compare the topology of the osets K r = d 1 K ([0, r]) and C r = d 1 ([0, r]). C F. Chazal, D. Cohen-Steiner, Q. Mérigot (Geometrica) Inference for Probability distributions 2009/07/08 5 / 32

6 Stability properties of the osets Topological/geometric properties of the osets of K are stable with respect to Hausdor approximation : 1. Topological stability of the osets of K (CCSL'06, NSW'06). 2. Approximate normal cones (CCSL'08). 3. Boundary measures (CCSM'07), curvature measures (CCSLT'09), Voronoi covariance measures (GMO'09). F. Chazal, D. Cohen-Steiner, Q. Mérigot (Geometrica) Inference for Probability distributions 2009/07/08 6 / 32

7 The problem of outliers If K = K {x} where d K (x) > R, then d K d K > R : oset-based inference methods fail! Question : Can we generalized the previous approach by replacing the distance function by a distance-like function having a better behavior with respect to noise and outliers? F. Chazal, D. Cohen-Steiner, Q. Mérigot (Geometrica) Inference for Probability distributions 2009/07/08 7 / 32

8 The three main ingredients for stability The stability in distance-based geometric inference relies on the three following facts : 1 the 1-Lipschitz property for d K ; 2 the 1-concavity on the function d 2 K : x x 2 d 2 K (x) is convex. 3 the stability of the map K d K : d K d K = sup x R d d K (x) d K (x) = dh(k, K ) A map φ : R d R which veries (1) and (2) is called distance-like. F. Chazal, D. Cohen-Steiner, Q. Mérigot (Geometrica) Inference for Probability distributions 2009/07/08 8 / 32

9 Replacing compact sets by measures A measure µ is a mass distribution on R d. Mathematically, it is dened as a map µ that takes a (Borel) subset B R d and outputs a nonnegative number µ(b). Moreover we ask that if (B i ) are disjoint subsets, µ ( i N B ) i = i N µ(b i) µ(b) corresponds to to the mass of µ contained in B a point cloud C = {p 1,..., p n } denes a measure µ C = 1 n i δ p i the volume form on a k-dimensional submanifold M of R d denes a measure vol k M. F. Chazal, D. Cohen-Steiner, Q. Mérigot (Geometrica) Inference for Probability distributions 2009/07/08 9 / 32

10 Distance between measures The Wasserstein distance dw(µ, ν) between two probability measures µ, ν quanties the optimal cost of pushing µ onto ν, the cost of moving a small mass dx from x to y being x y 2 dx. c 1 d 1 c i π i,j d j µ ν π 1 µ and ν are discrete measures : µ = i c iδ xi, ν = j d jδ yj with j d j = i c i. 2 Transport plan : set of coecients π ij 0 with i π ij = d j and j π ij = c i. 3 Cost of a transport plan ( ) 1/2 C(π) = ij x i y j 2 π ij 4 dw(µ, ν) := inf π C(π) F. Chazal, D. Cohen-Steiner, Q. Mérigot (Geometrica) Inference for Probability distributions 2009/07/08 10 / 32

11 Wasserstein distance Examples : 1 If #C 1 = #C 2, then d 2 W (µ C 1, µ C2 ) is the cost of a minimal least-square matching between C 1 and C 2 ; 2 If C = {p 1,..., p n } and C = {p 1,..., p n k 1, o 1,..., o k } with d(o i, C) = R, then dh(c, C ) R while dw(µ C, µ C ) m(r + diam(c)) with m = k n ; 3 If µ is a probability measure, dw(µ N (0, σ), µ) σ ; 4 If X 1,..., X N are iid with law µ, then (in general), 1 N N δ i=1 X i converges to µ whp as N tends to. F. Chazal, D. Cohen-Steiner, Q. Mérigot (Geometrica) Inference for Probability distributions 2009/07/08 11 / 32

12 Outline 1 Geometric inference for measures. 2 Distance to a probability measures. 3 Applications F. Chazal, D. Cohen-Steiner, Q. Mérigot (Geometrica) Inference for Probability distributions 2009/07/08 12 / 32

13 The distance to a measure Distance function to a measure, rst attempt Let m ]0, 1[ be a positive mass, and µ a probability measure on R d : δ µ,m (x) = inf {r > 0; µ(b(x, r)) > m} x δ µ,m(x) µ(b(x, δ µ,m(x)) = m µ δ µ,m is the smallest distance needed to attain a mass of at least m ; Coincides with the distance to the k-th neighbor when m = k/n and µ = 1 n n i=1 δ p i : δ µ,k/n (µ) = x p k C (x). F. Chazal, D. Cohen-Steiner, Q. Mérigot (Geometrica) Inference for Probability distributions 2009/07/08 13 / 32

14 Unstability of µ δ µ,m Distance to a measure, rst attempt Let m ]0, 1[ be a positive mass, and µ a probability measure on R d : δ µ,m (x) = inf {r > 0; µ(b(x, r)) > m} Unstability under Wasserstein perturbations : µ ε = (1/2 ε)δ 0 + (1/2 + ε)δ 1 for ε > 0 : x < 0, δ µε,1/2(x) = x 1 for ε = 0 : x < 0, δ µ0,1/2(x) = x 0 Consequence : the map µ δ µ,m C 0 (R d ) is discontinuous whatever the (reasonable) topology on C 0 (R d ). F. Chazal, D. Cohen-Steiner, Q. Mérigot (Geometrica) Inference for Probability distributions 2009/07/08 14 / 32

15 The distance function to a measure. Denition If µ is a measure on R d and m 0 > 0, one let : ( 1 d µ,m0 : x R d m0 ) 1/2 δ 2 m µ,m(x)dm 0 0 Example. Let C = {p 1,..., p n } and µ = 1 n n δ i=1 p i. Let p k C (x) denote the kth nearest neighbor to x in C, and set m 0 = k 0 /n : ( 1 d µ,m0 (x) = k 0 k 0 k=1 x p k C (x) 2 ) 1/2 F. Chazal, D. Cohen-Steiner, Q. Mérigot (Geometrica) Inference for Probability distributions 2009/07/08 15 / 32

16 The distance function to a discrete measure. Example (continued) Let C = {p 1,..., p n } and µ = 1 n n i=1 δ p i. Let p k C (x) denote the kth nearest neighbor to x in C, and set m 0 = k 0 /n : ( 1 d µ,m0 (x) = d µ,m0 (x) = k 0 k 0 k=1 x p k C (x) 2 ) 1/2 1 k0 [ k 0 k=1 x p k C (x) ] d µ,m0 (x) F. Chazal, D. Cohen-Steiner, Q. Mérigot (Geometrica) Inference for Probability distributions 2009/07/08 16 / 32

17 Another expression for d µ,m0 Proposition The distance d µ,m0 (x) coincides with the partial Wasserstein distance between the Dirac mass m 0 δ x and µ. More precisely : m0 d µ,m0 (x) = min{dw(m 0 δ x, ν); ν µ and mass(ν) = m 0 } { ( ) } 1/2 = min y x 2 dν(y) ; ν µ, mass(ν) = m 0 R d Let µ x,m0 be a measure realizing this minimum. The measure µ x,m0 gives mass to the multiple projections of x on µ ; For the point cloud case, when m 0 = k 0 /n and x is not on a k-voronoï face, µ x,m0 = k 0 k=1 1 n δ p k C (x) F. Chazal, D. Cohen-Steiner, Q. Mérigot (Geometrica) Inference for Probability distributions 2009/07/08 17 / 32

18 1-Concavity of the squared distance function Regularity m 0 d 2 µ,m 0 (x + h) = x + h z d dµ x+h,m0 (z) R d x + h z d dµ x,m0 (z) R d m 0 d 2 µ,m 0 (x) + 2 h x z dµ x,m0 (z) + m 0 h 2 R d That is : d 2 µ,m 0 (x + h) d 2 µ,m 0 (x) + h d 2 µ,m 0 (x) + h 2 with d 2 µ,m 0 (x) := 2m 1 0 (x z)dµ x,m0 (z) R d F. Chazal, D. Cohen-Steiner, Q. Mérigot (Geometrica) Inference for Probability distributions 2009/07/08 18 / 32

19 Theorem The distance function d µ,m0 is distance-like,ie. 1 the function x d µ,m0 (x) is 1-Lipschitz ; 2 the function x x 2 d 2 µ,m 0 (x) is convex ; Theorem The map µ d µ,m0 1 m0 Lipschitz, ie from probability measures to continuous functions is dµ,m0 d µ,m 0 1 m0 dw(µ, µ ) F. Chazal, D. Cohen-Steiner, Q. Mérigot (Geometrica) Inference for Probability distributions 2009/07/08 19 / 32

20 Outline 1 Geometric inference for measures. 2 Distance to a probability measures. 3 Applications F. Chazal, D. Cohen-Steiner, Q. Mérigot (Geometrica) Inference for Probability distributions 2009/07/08 20 / 32

21 Consequences of the previous properties 1 existence of an analogous to the medial axis 2 stability of a ltered version of it (as with the µ-medial axis) under Wasserstein perturbation 3 stability of the critical function of a measure 4 the gradient d µ,m0 is L 1 -stable 5... = the distance functions d µ,m0 share many stability and regularity properties with the usual distance function. F. Chazal, D. Cohen-Steiner, Q. Mérigot (Geometrica) Inference for Probability distributions 2009/07/08 21 / 32

22 Example : square with outliers 10% outliers, k = 150 δ µ,m0, m 0 = 1/10. Chazal, D. Cohen-Steiner, Q. Mérigot (Geometrica) Inference for Probability distributions 2009/07/08 22 / 32

23 Example : square with outliers d µ,m0 d µ,m0 F. Chazal, D. Cohen-Steiner, Q. Mérigot (Geometrica) Inference for Probability distributions 2009/07/08 23 / 32

24 A 3D example Reconstruction of an oset from a noisy dataset, with 10% outliers. Chazal, D. Cohen-Steiner, Q. Mérigot (Geometrica) Inference for Probability distributions 2009/07/08 24 / 32

25 A reconstruction theorem Theorem Let µ be a probability measure of dimension at most k > 0 with compact support K R d such that r α (K) > 0 for some α (0, 1]. For any 0 < η < r α (K), there exists positive constants m 1 = m 1 (µ, α, η) > 0 and C = C(m 1 ) > 0 such that : for any m 0 < m 1 and any probability measure µ such that W 2 (µ, µ ) < C m 0, the sublevel set d 1 µ,m 0 ((, η]) is homotopy equivalent (and even isotopic) to the osets d 1 K ([0, r]) of K for 0 < r < r α (K). F. Chazal, D. Cohen-Steiner, Q. Mérigot (Geometrica) Inference for Probability distributions 2009/07/08 25 / 32

26 k-nn density estimation vs distance to a measure data Density is estimated using x m 0 ω d 1 (δ µ,m 0 (x)), m 0 = 150/1200. F. Chazal, D. Cohen-Steiner, Q. Mérigot (Geometrica) Inference for Probability distributions 2009/07/08 26 / 32

27 k-nn density estimation vs distance to a measure density log(1 + 1/d µ,m0 ) Density is estimated using x (Devroye-Wagner '77). m 0 vol d (B(x,δ µ,m 0 (x))), m 0 = 150/1200. Chazal, D. Cohen-Steiner, Q. Mérigot (Geometrica) Inference for Probability distributions 2009/07/08 27 / 32

28 k-nn density estimation vs distance to a measure 1 the gradient of the estimated density can behave wildly 2 exhibits peaks near very dense zone 1. can be xed using d µ,m0 (because of the semiconcavity) 2. shows that the distance function is a better-behaved geometric object to associate to a measure. F. Chazal, D. Cohen-Steiner, Q. Mérigot (Geometrica) Inference for Probability distributions 2009/07/08 28 / 32

29 k-nn density estimation vs distance to a measure 1 the gradient of the estimated density can behave wildly 2 exhibits peaks near very dense zone 1. can be xed using d µ,m0 (because of the semiconcavity) 2. shows that the distance function is a better-behaved geometric object to associate to a measure. F. Chazal, D. Cohen-Steiner, Q. Mérigot (Geometrica) Inference for Probability distributions 2009/07/08 29 / 32

30 Pushing data along the gradient of d µ,m0 Mean-Shift like algorithm (Comaniciu-Meer '02) Theoretical guarantees on the convergence of the algorithm and smoothness of trajectories.. Chazal, D. Cohen-Steiner, Q. Mérigot (Geometrica) Inference for Probability distributions 2009/07/08 30 / 32

31 Pushing data along the gradient of d µ,m0. Chazal, D. Cohen-Steiner, Q. Mérigot (Geometrica) Inference for Probability distributions 2009/07/08 31 / 32

32 Summary µ d µ,m0 provide a way to associate geometry to a measure in Euclidean space. d µ,m0 is robust to Wasserstein perturbations : outliers and noise are easily handled (no assumption on the nature of the noise). d µ,m0 shares regularity properties with the usual distance function to a compact. Geometric stability results in this measure-theoretic setting : topology/geometry of the sublevel sets of d µ,m0, stable notion of persistence diagram for µ,. Algorithm : for nite point clouds d µ,m0 and (d µ,m0 ) can be easily and eciently computed pointwise in any dimension. F. Chazal, D. Cohen-Steiner, Q. Mérigot (Geometrica) Inference for Probability distributions 2009/07/08 32 / 32