Statistics and Topological Data Analysis

Transcription

1 Geometry Understanding in Higher Dimensions CHAIRE D INFORMATIQUE ET SCIENCES NUMÉRIQUES Collège de France - June 2017 Statistics and Topological Data Analysis Bertrand MICHEL Ecole Centrale de Nantes - Lab. de mathématiques Jean Leray

2 Introduction : Topological Data Analysis and Statistics

3 Topological Data Analysis and Topological Inference The aim of TDA is to infer relevant qualitative and quantitative topological structures (clusters, holes...) directly from the data. data : typically point cloud X n Two popular methods in TDA : Mapper algorithm [Singh et al., 2007] and persistent homology [Edelsbrunner et al., 2002].

4 Topological Data Analysis (TDA) Why is topology interesting for data analysis? multiscale compact invariant under coordinate changes stable with respect to (small) perturbations informative topological space topological descriptors β 0 β 1 β 2 point cloud

5 Topological Data Analysis (TDA) For exploratory analysis, visualization

6 Topological Data Analysis (TDA) For exploratory analysis, visualization For feature extraction and statistical learning raw data Topological descriptors Geometric descriptors Other signatures supervised methods unsupervised methods

7 Statistics, Learning and TDA A statistical approach to TDA means that : we consider data as generated from an unknown distribution the inferred topological features by TDA methods are seen as estimators of topological quantities describing an underlying object. β 0 β 1 β 2

8 Statistics, Learning and TDA Directions of research (non-exhaustive list): Consistency / convergence of TDA methods: [Chazal15 JMLR], [Bobrowski 17 Bernouilli] Confidence regions for TDA [Fasy 14 AoS] [Chazal 15 JOCG ] Central tendency for persistent homology [Turner 14 DCG] [Fasy15 Nips] Robust methods for TDA [Chazal 17, EJS Chazal 17 JMLR] Representations of persistence in Euclidean spaces [Bubenik15 JMLR] [Adams15] Develop kernels for topological descriptors [Reininghaus 15 IEEE] [Carriere 17 ICML ] Statistical analysis of Mapper [Carriere 17]...

9 Homology and Persistent homology

10 Topological Stability and Regularity Topological inference : under regularity assumptions, topological properties of X can be recovered from (the off-sets) of a close enough object Y.

11 Topological Stability and Regularity Topological inference : under regularity assumptions, topological properties of X can be recovered from (the off-sets) of a close enough object Y. The local feature size is a local notion of regularity : For x X, lfs X (x) := d (x, M(X c )). The global version of the local feature size is the reach [Federer, 1959] : κ(x) = inf x X c lfs X(x). The reach is small if either X is not smooth or if X is close to being selfintersecting. Weak feature size and its extensions [Chazal and Lieutier, 2007] (by considering the critical values of d X ).

12 Topological Stability and Regularity Topological inference : under regularity assumptions, topological properties of X can be recovered from (the off-sets) of a close enough object Y. Example : d H (X, Y) = inf {α 0 X Y α and Y X α } Theorem [Chazal and Lieutier, 2007]: Let X and Y be two compact sets in R d and let ε > 0 be such that d H (X, Y) < ε, wfs(x) > 2ε and wfs(y) > 2ε. Then for any 0 < α < 2ε, X α and Y β are homotopy equivalent.

13 Topological Stability and Regularity Topological inference : under regularity assumptions, topological properties of X can be recovered from (the off-sets) of a close enough object Y. Example : d H (X, Y) = inf {α 0 X Y α and Y X α } Theorem [Chazal and Lieutier, 2007]: Let X and Y be two compact sets in R d and let ε > 0 be such that d H (X, Y) < ε, wfs(x) > 2ε and wfs(y) > 2ε. Then for any 0 < α < 2ε, X α and Y β are homotopy equivalent. Sampling conditions in Hausdorff metric. Statistical analysis of homotopy inference can be deduced from support estimation of a distribution under the Hausdorff metric.

14 Homology inference [Niyogi et al., 2008 and 2011] [Balakrishnan et al., 2012] : The Betti number (actually the homotopy type) of Riemannian manifolds with positive reach can be recovered with high probability from offsets of a sample on (or close to) the manifold. Homology inference Homotopy is not easy to compute in practice. Singular homology provides a algebraic description of holes in a geometric shape (connected components, loops, etc...) Betti number β k is the rank of the k-th homology group. Computational Topology : Betti numbers can be computed on simplicial complexes.

15 Persistent homology Starting from a point cloud X n, let Filt = (C α ) α A be a fitration of nested simplicial complexes. α Persistent homology: identification of persistent topological features along the filtration. multiscale information ; more stable and more robust ;

16 Barecodes and Persistence Diagrams Filtration of simplicial complexes Filt(X n ) X n Offsets Barcode

17 Barecodes and Persistence Diagrams Filtration of simplicial complexes Filt(X n ) death X n Offsets Barcode birth Dgm (Filt(X n )) Persistence diagram of the filtration Filt(X n ) built on X n.

18 Distance between persistence diagrams and stability death Add the diagonal Dgm 1 Dgm 2 Multiplicity: 2 0 birth The bottleneck distance between two diagrams Dgm 1 and Dgm 2 is d b (Dgm 1, Dgm 2 ) = inf γ Γ sup p Dgm 1 p γ(p) where Γ is the set of all the bijections between Dgm 1 and Dgm 2 and p q = max( x p x q, y p y q ).

19 Distance between persistence diagrams and stability death Add the diagonal Dgm(Filt(X)) Dgm(Filt(Y)) Multiplicity: 2 0 Theorem [Chazal et al., 2012]: For any compact metric spaces (X, ρ) and (Y, ρ ), d b (Dgm(Filt(X)), Dgm(Filt(Y))) 2 d GH (X, Y). Consequently, if X and Y are embedded in the same metric space (M, ρ) then d b (Dgm(Filt(X)), Dgm(Filt(Y))) 2 d H (X, Y). birth

20 Statistics and Persistent homology

21 Persistence diagram inference [Chazal 2015 JMLR] (M, ρ) metric space X compact set in M. X Filt(X) well defined for any compact metric space [Chazal et al., 2012] Dgm(Filt(X)) 0 0 Convergence??? X n Filt( X n ) n points sampled in X according to µ Estimator of Dgm(Filt(K)) 0 0 Dgm(Filt( X n ))

22 Persistence diagram inference For a, b > 0, µ satisfies the (a, b)-standard assumption on its support X µ if for any x X µ and any r > 0 : Theorem: For a, b > 0 : µ(b(x, r)) min(ar b, 1). P(a, b, M) : set of all the probability measures satisfying the (a, b)-standard assumption on the metric space (M, ρ). sup µ P(a,b,M) E [ ] d b (Dgm(Filt(X µ )), Dgm(Filt( X n ))) C ( ln n n ) 1/b where C only depends on a and b. Under additional technical hypotheses, for any estimator Dgm n of Dgm(Filt(X µ )): [ ] lim inf E d b (Dgm(Filt(X µ )), Dgm n ) C n 1/b n sup µ P(a,b,M) where C is an absolute constant.

23 Confidence sets for persistence diagrams [Fasy 2014 AoS] P ( Dgm(Filt(K)) ˆR ) 1 α??

24 Confidence sets for persistence diagrams [Fasy 2014 AoS] P ( Dgm(Filt(K)) ˆR ) 1 α?? Using the Hausdorff stability, we can define confidence sets for persistence diagrams: d b (Dgm (Filt(K)), Dgm (Filt(X n ))) d H (K, X n ). It is sufficient to find c n such that ( ) lim sup dh (K, X n ) > c n α. n

25 Confidence sets for persistence diagrams [Fasy 2014 AoS] Subsampling method: N subsamples X 1 b,n,..., XN b,n of size b. ) Compute T j = d H (X j b,n, X n, j = 1,..., N. Compute L b (t) = 1 N N j=1 1 T j >t, Take c b = 2L 1 b (α). If P satisfies an (a, b) standard assumption then, for n large enough : P ( W (Dgm (Filt(X µ )), Dgm (Filt(X n ))) > c b ) P ( d H (X µ, X n ) > c b ) α + O ( b n ) 1/4

26 Central tendency for persistent homology Frechet mean [Turner 2014] Use an alternative descriptor of persistence : Persistence landscapes [Bubenik, 2015]

27 Persistence landscapes [Bubnik JMLR 2015] d d+b 2 d b 2 d b 2 b X n Dgm(Filt(Xn )) persistence landscape d+b 2 Dgm = { } ( d i+b i, d i+b i ), i I 2 2 Persistence landscape λ of Dgm: λ(k, t) = kmax p D Λ p(t), t R, k N, where kmax is k-th largest value in the set. For p = ( b+d, d b ) Dgm, 2 2 t b t [b, b+d ] 2 Λ p (t) = d t t ( b+d 2, d] 0 otherwise. Stability: For any t R and any k N, λ(k, t) λ (k, t) d b (Dgm, Dgm ).

28 Subsampling methods for pers. homology [Chazal ICML 2015] joint work with F. Chazal, B. Fasy, F. Lecci, A. Rinaldo and L. Wasserman Let X = {X 1,, X m } sampled from µ. λ X : corresponding persistence landscape. Ψ m µ : the measure induced by µ m on the space of persistence landscapes. We consider the point-wise expectations of the (random) persistence landscape under this measure: E Ψ m µ [λ X (t)], t [0, T ] For S1 m,..., Sl m some independent samples of size m from µ m, the empirical counterpart of E Ψ m µ [λ X (t)] is λ m l (t) = 1 l l i=1 λ S m i (t), for all t [0, T ],

29 Subsampling methods for pers. homology [Chazal ICML 2015] Definition: The p-th Wasserstein distance between two measures µ, ν defined on (M, ρ) is W ρ,p (µ, ν) = ( inf Π M M ) 1 [ρ(x, y)] p p dπ(x, y), where the infimum is taken over all measures on M M with marginals µ and ν. Stability of the average landscape: Theorem: Let X µ m and Y ν m, where µ and ν are two probability measures on M. For any p 1 we have E Ψ m µ [λ X ] E Ψ m ν [λ Y ] 2 m 1 p Wρ,p (µ, ν).

30 Subsampling methods for pers. homology [Chazal ICML 2015] Application: Analysis of accelerometer data. Fred Fabrizio Bertrand topological features carry discriminative information no registration/calibration preprocessing step needed

31 Commercial break: Gudhi with Statistical learning Python Libraries and coming soon : Gudhi Stat with more tools for statistics and TDA.

32 Robust TDA

33 Standard TDA methods are not robust to outliers X r := x X B(x, r) = d 1 X ([0, r]) where the distance function d X to X is d X (y) = inf x X x y

34 Standard TDA methods are not robust to outliers X r := x X B(x, r) = d 1 X ([0, r]) where the distance function d X to X is d X (y) = inf x X x y

35 Some possible noise models for geometry Additive noise model P = µ Φ distribution with support the geometric shape G noise distribution Clutter noise model P = πµ + (1 π)u G distribution with support the geometric shape G Uniform distribution on the box G A few outliers P = πµ + (1 π)ψ distribution with support the geometric shape G Distribution of outliers G

36 Robust TDA with an alternative distance function? We would like to consider the sub levels of an alternative distance function related to the sampling measure, which support is X, or close to X.

37 Distance To Measure [Chazal 11 FoCM] Preliminary distance function to a measure P : Let u ]0, 1[ be a positive mass, and P a probability measure on R d : δ P,u (x) = inf {r > 0 : P (B(x, r)) u} δ P,u (x) δ P,u is the smallest distance needed to capture a mass of at least u. x δ P,u is the quantile function at u of the r.v. supp(p ) x X u where X P.

38 Distance To Measure [Chazal 11 FoCM] Preliminary distance function to a measure P : Let u ]0, 1[ be a positive mass, and P a probability measure on R d : δ P,u (x) = inf {r > 0 : P (B(x, r)) u} x δ P,u (x) Definition: Given a probability measure P on R d and m > 0, the distance function to the measure P (DTM) is defined by u supp(p ) d P,m : x R d ( 1 m m 0 ) 1/2 δp,u(x)du 2

39 Distance To Measure [Chazal 11 FoCM] Properties of the DTM : Stability under Wassertein perturbations: d P,m d Q,m 1 m W 2 (P, Q) The function x d 2 P,m (x) is semiconcave, this is ensuring strong regularity properties on the geometry of its sublevel sets. Consequently, if P is a probability distribution close to P for Wasserstein distance W 2, then the sublevel sets of d P,m provide a topologically correct approximation of the support of P.

40 Distance to The Empirical Measure (DTEM) Let X 1,..., X n sample according to P and let P n be the empirical measure. Then d 2 P n (x) = 1 k x X, k (i) 2 n k where X (1) x X (2) x X (k) x X (n) x i=1 X (i) x k = 8

41 Geometric inference with the DTM Theorem: [Chazal et al., 2011] Let µ be a measure that has dimension at most k > 0 with compact support G such that reach α (G) R > 0 for some α > 0. Let ν be another measure and ε be an upper bound on the uniform distance between d G and d ν,m0. Then, for any r [4ε/α 2, R 3ε] and any η ]0, R[, the r-sublevel sets of d µ,m0 and the η-sublevel sets of d G are homotopy equivalent as soon as: W 2 (µ, ν) R m /α 2 C(µ) 1/k m 1/k+1/2 0. In practice : X 1... X n sampled according to P. Assume W 2 (P, µ) small. P n = n i=1 δ X i : empirical measure. Than for n large enough, W 2 (P n, µ) is small and the sublevel sets of d Pn,m provide a topologically correct approximation of G. G

42 Wasserstein deconvolution and DTM denoising Additive noise model P = µ Φ distribution with support the geometric shape G noise distribution G In this case, W 2 (P, µ) can be large. Ideally we would like to denoise directly d Pn,n, but this can be hardly achieved because the DTM is not a linear functional of the measure. Alternative approach : deconvolve the observed measure [Caillerie EJS 2011] X 1,..., X n Deconvolved measure µ n d µn,m W 2 (P n, µ) W 2 ( µ n, µ) m d µn,m d µ ( 0) ( 0)

43 Wasserstein deconvolution and DTM denoising

44 DTM and persistent homology

45 DTM and persistent homology death Dgm P,m X i P d P,m 0 birth d b ( DgmP,m, Dgm Q,m ) dp,m d Q,m 1 m W 2 (P, Q) Take Q = P n... Wasserstein Stability of the DTM [Chazal et al., 2012] Stability of Persistent homology [Cohen- Steiner et al.,2005, Chazal et al., 2012]

46 Estimation of the DTM via the empirical DTM [Chazal EJS 17, Chazal JMLR 17] Quantity of interest: d 2 P n, k n (x) d 2 P, k n (x) Observe that d 2 P,m(x) = 1 m m 0 Fx 1 (u)du where F x is the cdf of x X 2 with X P. The distance to the empirical measure is the empirical counter part of the distance to P : d Pn,m(x) 2 = 1 m m 0 F 1 x,n(u)du where F x,n is the cdf of x X 2 with X P n. Finally we get that d 2 P n, k n (x) d 2 P, k n (x) = 1 m m 0 { F 1 x,n(u) Fx 1 (u) } du

47 Estimation of the DTM via the empirical DTM [Chazal EJS 17, Chazal JMLR 17] Quantity of interest: d 2 P n, k n (x) d 2 P, k n (x) Two complementary approaches of the problem: Asymptotic approach : k nn = m is fixed and n tends to infinity. Non asymptotic approach : n is fixed, and we want a tight control over the fluctuations of the empirical DTM, in function of k, which can be taken very small. We do not use Wasserstein stability for either of the two approaches. Wasserstein rates of convergence [Fournier and Guillin, 2013 ;Dereich et al., 2013] do not provide tight rates for the DTM in this context.

48 Bootstrap and significance of topological features Aim : studying the persistent homology of the sub-levels of the DTM and providing confidence regions. Two alternative boostrap methods : by bootstrapping the DTM Bottleneck Bootstrap

49 Bootstrap and significance of topological features Bootstrapping the DTM P P n

50 Bootstrap and significance of topological features Bootstrapping the DTM P P n P n P n

51 Bootstrap and significance of topological features Bootstrapping the DTM P P n P n P n

52 Bootstrap and significance of topological features Bootstrapping the DTM P P n P n P n Φ(P ) Φ(P n ) Φ(P n ) Φ(P n)

53 Bootstrap and significance of topological features Bootstrapping the DTM For m (0, 1), define c α by P ( n d 2 P,m d 2 P n,m > c α ) = α. Let X 1,..., X n be a sample from P n, and let P n be the corresponding (bootstrap) empirical measure. We consider the bootstrap quantity d P n,m(x) of d Pn,m. The bootstrap estimate ĉ α is defined by P ( n d 2 Pn,m d 2 P n,m > ĉ α X 1,..., X n ) = α where ĉ α can be approximated by Monte Carlo. Theorem: If Fx 1 is regular enough, the DTM is Hadamard differentiable at P. Consequently, the bootstrap method for the DTM is asymptotically valid.

54 Bootstrap and significance of topological features Bootstrapping the DTM Dgm : persistence diagram of the sub-levels of d P,m Dgm : persistence diagram of the sub-levels of d Pn,m. Let { } C n = E Diag : d b ( Dgm, E) ĉα n where Diag is the set of all the persistence diagrams., Then, P(Dgm C n ) = P ( d b (Dgm, Dgm) ĉα n ) P Bootstrap estimate ( ) d 2 P,m d 2 P n,m ĉα n

55 Bootstrap and significance of topological features The Bottleneck Bootstrap Dgm : persistence diagram of the sub-levels of d P,m Dgm : persistence diagram of the sub-levels of d Pn,m. Dgm : persistence diagram of the sub-levels of d P n,m. We directly bootstrap in the set of the persistence diagram by considering the random quantity d b ( Dgm, Dgm). We define ˆt α by ( ndb P ( Dgm ), Dgm) > ˆt α X 1,..., X n The quantile ˆt α can be estimated by Monte Carlo. = α.

56 Bootstrap and significance of topological features For both methods we can identify significant features by putting a band of size 2ĉ α or 2ˆt α around the diagonal: In practice, the bottleneck bootstrap can lead to more precise inferences because in many cases the stability result is not sharp enough: d b ( Dgm, Dgm) d P,m d Pn,m.

57 Concluding remarks TDA methods focus on the topological properties (homology / persistent homology) of a shape. TDA methods can be used as an exploratory method, in particuar when the point cloud is sampled on (close to) a real geometric object as a feature extraction procedure, next these extracted features can be used for learning purposes. TDA is an emerging field, at the interface maths, computer sciences, statistics. Many topics about the statistical analysis of TDA Applications in many fields of sciences ( medecine, biology, dynamic systems, astronomy, dynamical systems, physics...)