Persistence based Clustering

Transcription

1 July 9, 2009 TGDA Workshop Persistence based Clustering Primoz Skraba joint work with Frédéric Chazal, Steve Y. Oudot, Leonidas J. Guibas 1-1

2 Clustering Input samples 2-1

3 Clustering Input samples Important segments/clusters 2-2

4 Clustering Input samples Important segments/clusters ill-posed problem 2-3

5 Clustering Input samples Important segments/clusters ill-posed problem Extensive previous work - k-means - spectral clustering - mode-seeking (mean-shift) 2-4

6 Clustering Input samples Important segments/clusters ill-posed problem Extensive previous work - k-means - spectral clustering - mode-seeking (mean-shift) Our viewpoint: data points drawn at random from some unknown density distribution f 2-5

7 Definition of a Cluster Basins of attraction of significant peaks of f R X 3-1

8 Definition of a Cluster Basins of attraction of significant peaks of f R X 3-2

9 Definition of a Cluster Basins of attraction of significant peaks of f R X 3-3

10 Outline Background: scalar field analysis Algorithm Number of clusters Results (Interpretation of persistence diagrams) Spatial stability Conclusions 4-1

11 Scalar Field Analysis* Setting: X topological space, f : X R Input: A finite sampling L of X, the values of f at the sample points R f X 5-1 *[Chazal, Guibas, Oudot, Skraba 09]

12 Scalar Field Analysis* Setting: X topological space, f : X R Input: A finite sampling L of X, the values of f at the sample points R f L X 5-2 *[Chazal, Guibas, Oudot, Skraba 09]

13 Scalar Field Analysis* Setting: X topological space, f : X R Input: A finite sampling L of X, the values of f at the sample points Goal: Analyze landscape of graph(f): - prominent peaks/valleys - basins of attraction R f L X 5-3 *[Chazal, Guibas, Oudot, Skraba 09]

14 Persistence-Based Approach in a nutshell... - evolution of topology of super-level sets ˆf 1 ([α, )) as α spans R. R α 6-1 X

15 Persistence-Based Approach in a nutshell... - evolution of topology of super-level sets ˆf 1 ([α, )) as α spans R. R α 6-2 X

16 Persistence-Based Approach in a nutshell... - evolution of topology of super-level sets ˆf 1 ([α, )) as α spans R. R α 6-3 X

17 Persistence-Based Approach in a nutshell... - evolution of topology of super-level sets ˆf 1 ([α, )) as α spans R. - finite set of intervals (barcode) encode birth/death of homological features. R 6-4 X

18 Persistence-Based Approach in a nutshell... - evolution of topology of super-level sets ˆf 1 ([α, )) as α spans R. - finite set of intervals (barcode) encode birth/death of homological features. - barcode of ˆf is close to barcode of f provided that ˆf f is small. R [Cohen-Steiner, Edelsbrunner, Harer 05] 6-5 X

19 Persistence-Based Approach Assumptions: X triangulated space, f : X R Lipschitz continuous build PL approximation ˆf of f apply persistence algo. to ± ˆf [Edelsbrunner, Letscher, Zomorodian 00] ˆf 0.5 ˆf X -2.5 β 0 (6 prominent peaks) 6-6

20 Persistence-Based Approach Assumptions: X triangulated space, f : X R Lipschitz continuous build PL approximation ˆf of f apply persistence algo. to ± ˆf [Edelsbrunner, Letscher, Zomorodian 00] ˆf 0.5 ˆf 0.5 ˆf X β 0 β 1 (6 prominent peaks) (ring-shaped basin of attraction) 6-7

21 Approximation of Super-Level Sets Assumptions: X Riemannian manifold, f : X R c-lipschitz, L geodesic ε-cover of X, for some unknown ε >

22 Approximation of Super-Level Sets Assumptions: X Riemannian manifold, f : X R c-lipschitz, L geodesic ε-cover of X, for some unknown ε > 0. Access to L not X 7-2

23 Approximation of Super-Level Sets Assumptions: X Riemannian manifold, f : X R c-lipschitz, L geodesic ε-cover of X, for some unknown ε > 0. Access to L not X F α := f 1 ([α, )) L α := L F α L ε α := S p L α B X (p, ε) F α α R, L ε α+cε F α L ε α cε ε 7-3

24 Approximation of Super-Level Sets Assumptions: X Riemannian manifold, f : X R c-lipschitz, L geodesic ε-cover of X, for some unknown ε > 0. Access to L not X F α := f 1 ([α, )) L α := L F α L ε α := S p L α B X (p, ε) F α α R, L ε α+cε F α L ε α cε ε F α+cɛ 7-4

25 Approximation of Super-Level Sets Assumptions: X Riemannian manifold, f : X R c-lipschitz, L geodesic ε-cover of X, for some unknown ε > 0. Access to L not X F α := f 1 ([α, )) L α := L F α L ε α := S p L α B X (p, ε) F α cε F α α R, L ε α+cε F α L ε α cε ε F α cɛ α+cɛ 7-5

26 Approximation of Super-Level Sets Assumptions: X Riemannian manifold, f : X R c-lipschitz, L geodesic ε-cover of X, for some unknown ε > 0. Access to L not X F α := f 1 ([α, )) F α cε L α := L F α L ε α := S p L α B X (p, ε) F α α R, L ε α+cε F α L ε α cε ε the filtrations {F α } α R and {L ε α} α R are cε-interleaved their barcodes are cε-close. F α cɛ α+cɛ 7-6 [Chazal, Cohen-Steiner, Glisse, Guibas, Oudot 09]

27 Approximation of Super-Level Sets Assumptions: X Riemannian manifold, f : X R c-lipschitz, L geodesic ε-cover of X, for some unknown ε > 0. Guarantee: here, interleaved is in some generalized sense, because arrows of the diagrams may not represent inclusions - see paper δ ε, {F α } α R and {R δ (L α ) R 2δ (L α )} α R are 2cδ-interleaved [Chazal, Cohen-Steiner, Glisse, Guibas, Oudot 09] the bound of this theorem has been improved since the writing of the final version of the SODA paper. their barcodes are 2cδ-close. 8-1

28 Approximation of Super-Level Sets Assumptions: X Riemannian manifold, f : X R c-lipschitz, L geodesic ε-cover of X, for some unknown ε > 0. f f R L β 0 β 1 (6 prominent peaks) (ring-shaped basin of attraction) X 8-2

29 Approximation of Super-Level Sets Assumptions: X Riemannian manifold, f : X R c-lipschitz, L geodesic ε-cover of X, for some unknown ε > 0. f f ˆf ˆf β 0 β β 0 β 1 8-3

30 Approximation of Super-Level Sets Assumptions: X Riemannian manifold, f : X R c-lipschitz, L geodesic ε-cover of X, for some unknown ε > 0. 5,000 ˆf τ β 0 8-4

31 Homological Features and Clusters Samples drawn from f Estimate ˆf from samples X 9-1

32 Homological Features and Clusters Samples drawn from f Estimate ˆf from samples R 9-2 X Clusters: Prominent peaks correspond to persistent connected components of the super-level set filtration of f

33 Computing Clusters How do we compute clusters from a barcode? 10-1

34 Computing Clusters How do we compute clusters from a barcode? Input: Samples with estimated density ˆf 10-2

35 Computing Clusters How do we compute clusters from a barcode? Input: Samples with estimated density ˆf Two steps: 1. Mode-seeking step [ Koontz et. al. 76] 10-3

36 Computing Clusters How do we compute clusters from a barcode? Input: Samples with estimated density ˆf Two steps: 1. Mode-seeking step [ Koontz et. al. 76] 10-4

37 Computing Clusters How do we compute clusters from a barcode? Input: Samples with estimated density ˆf Two steps: 1. Mode-seeking step [ Koontz et. al. 76] 2. Merge clusters according to persistence 10-5

38 Algorithm Input: f(x), R δ, α 11-1

39 Algorithm Input: f(x), R δ, α 1. Sort x according to f 2. For x L 2a. For neighbors of x in R δ If no higher neighbors new cluster else assign x to f 2b. For adjacent clusters y to x if f(y) f(x) α merge into oldest adjacent cluster 11-2

40 Putting it together Estimate density Run algorithm with α = - Standard persistence algorithm Use persistence diagram to choose threshold Re-run algorithm 5, τ 5000

41 Putting it together Estimate density Run algorithm with α = - Standard persistence algorithm Use persistence diagram to choose threshold Re-run algorithm 5, τ 5000

42 Theoretical Guarantees Applying the result from scalar field work 13-1

43 Theoretical Guarantees Applying the result from scalar field work Approximation depends on cδ 13-2

44 Theoretical Guarantees Applying the result from scalar field work Approximation depends on cδ Whole space is not uniformly sampled 13-3

45 Theoretical Guarantees Applying the result from scalar field work Approximation depends on cδ Whole space is not uniformly sampled smaller δ larger δ Approximation result holds in well-sampled regions w.h.p. 13-4

46 Theoretical Guarantees Applying the result from scalar field work Approximation depends on cδ Whole space is not uniformly sampled smaller δ larger δ Approximation result holds in well-sampled regions w.h.p. More points more of the space 13-5

47 Number of Clusters Define a signal-to-noise ratio Definition:Given two values d 2 > d 1 0, the persistence diagram D 0 f is called (d 1, d 2 )-separated if every point of D 0 f lies either in the region D 1 above the diagonal line y = x d 1 or in the region D 2 below the diagonal y = x d 2 and to the right of the vertical line x = d 2. d 1 d 2 D 1 D d 2

48 Approximation Assume enough points that up to cδ is well-sampled w.h.p. d 1 d 2 III I V - 0cδ 0 II d 2d2 +cδ IV 15-1

49 Approximation Assume enough points that up to cδ is well-sampled w.h.p. d 1 τ d 2 I V III cδ - 0 II 0 d 1 τ IV 15-2

50 Approximation Assume enough points that up to cδ is well-sampled w.h.p. 2cδ III cδ 0 - d 2 - cδ d 2 V d 2 +cδ IV 15-3

51 Feedback and Interpreting Diagrams If peaks are prominent enough, we will get the right number of clusters Practically, - Gives a sense of stability of the number of clusters - Choice of threshold transparent w.r.t. number of clusters Rips parameter δ = spatial scale - Trade-off Small δ = good approximation Large δ = holds over a larger part of the space 16-1

52 Experiments Synthetic dataset Image segmentation Alanine-dipeptide conformations 17-1

53 4 Rings Interlocking rings in R3 600k (100k + 500k) points total 18-1

54 4 Rings

55 4 Rings

56 Image Segmentation Each pixel is assigned color coordinates in LUV space 21-1

57 Mandrill x

58 Landscape x10

59 Street x

60 Koala x

61 Incorporating Spatial Information Neighborhood graph: proximity in LUV space and image

62 Alanine-dipeptide Conformations Clustering in 22-dim space 192k points 27-1

63 Alanine-dipeptide Conformations x10 3 Metastability Rank Prominence Number of clusters

64 Spatial stability Number of clusters are correct Can we say anything about the clusters themselves? 1. Each prominent cluster has a stable part 2. Unstable part can be very large 29-1

65 Spatial stability Number of clusters are correct Can we say anything about the clusters themselves? 1. Each prominent cluster has a stable part 2. Unstable part can be very large z m 1 m 2 z z m s 2 s2 s 1 m s 2 s

66 Spatial stability Number of clusters are correct Can we say anything about the clusters themselves? 1. Each prominent cluster has a stable part 2. Unstable part can be very large 29-3

67 Stable Part Idea: Prominent clusters have a minimum size under c-lipschitz assumption Under small pertubations, prominent peak part of the same cluster Soft clustering 1. Run the algorithm multiple times, with small pertubations 2. Find one-to-one correspondance between clusters 3. Find stable and unstable parts 30-1

68 Conclusions Practical clustering algorithm (efficient in space and time) General framework - Use your favorite density estimator - Choice of neighborhood graph Easily-interpreted feedback - No black box effect Theoretical guarantees - Number of clusters - Spatial stability Soft-clustering Higher-dimensional features 31-1