SPECTRAL CLUSTERING OF LARGE NETWORKS

Transcription

1 SPECTRAL CLUSTERING OF LARGE NETWORKS A. Fender, N. Emad, S. Petiton, M. Naumov May 8th, 207

2 Introduction Agenda Laplacian Modularity Conclusions 2

3 THE CLUSTERING PROBLEM Example : detect relevant groups based on frequent co-purchasing on Amazon.com Data: V. Krebs Visualization: M. Bastian, S. Heymann, and M. Jacomy. Gephi: An Open Source Software for exploring and manipulating networks

4 THE CLUSTERING PROBLEM Pink Liberal Yellow Neutral Green Conservative Data: V. Krebs Visualization: M. Bastian, S. Heymann, and M. Jacomy. Gephi: An Open Source Software for exploring and manipulating networks

5 CLUSTERING ALGORITHMS Spectral A x x Build a matrix, solve an eigenvalue problem, use eigenvectors for clustering Hierarchical / Agglomerative fine coarse Build a hierarchy (fine to coarse), partition coarse, propagate results back to fine level Local refinements Switch one node at a time 5

6 Laplacian 6

7 RATIO CUT COST FUNCTION Objective function: Cost = p i= E i V i where E i : # of edges cut and V i : # of nodes in i-th partition A compromise between small edge-cut and balanced partitions Cost = = 5 3 M. Naumov, T. Moon. Parallel spectral graph partitioning. Nvidia Technical Report,

8 GRAPH LAPLACIAN L = D A D: degree matrix A: adjacency matrix = 3 2 L D A M. Naumov, T. Moon. Parallel spectral graph partitioning. Nvidia Technical Report,

9 GRAPH LAPLACIAN For a vector x with elements that are 0 or : Number of edges cut x T L x E = = 2 V E Number of elements V = x T x = 2 M. Naumov, T. Moon. Parallel spectral graph partitioning. Nvidia Technical Report,

10 MINIMIZATION PROBLEM min V i p i= E i V i min x i p i= x T i Lx i x i T x i where x i 0, n and x i x j V E 5 Next step Relax requirements on x, and let x i take real values M. Naumov, T. Moon. Parallel spectral graph partitioning. Nvidia Technical Report,

11 K-MEANS POINTS CLUSTERING Points Centroids p p 2 c c k p l Lloyd s Algorithm: Select centroids Compute distance of points to centroids Assign points to the closest centroid Recompute centroid position M. Naumov, T. Moon. Parallel spectral graph partitioning. Nvidia Technical Report, 206.

12 EDGE CUT MINIMIZATION PIPELINE Graph Preprocessing Laplacian Eigensolver Points Clustering Clustering x x 2 x x 2 2

13 SPECTRAL EDGE CUT MINIMIZATION 80% hit rate Balanced cut minimization Ground truth 3

14 Modularity 4

15 MODULARITY FUNCTION Measures the difference between how well vertices are assigned into clusters for the current graph G = (V,E) versus a random graph R = (V,F). G = (V,E) R = (V,F) Q = 2ω i j (w ij v iv j 2ω ) δ c i c(j) for some assignment c(.) into clusters. A. Fender, N. Emad, S. Petiton, M. Naumov. Parallel Modularity Clustering. ICCS, 207 5

16 MODULARITY MATRIX Let matrix B = A 2ω vvt then modularity Q = 2ω Tr(XT BX) where Tr(.) is the trace (sum of diagonal elements) and X = [x,, x p ] is such that x ik = if c i = k. A v A. Fender, N. Emad, S. Petiton, M. Naumov. Parallel Modularity Clustering. ICCS, 207 6

17 MODULARITY MAXIMIZATION PIPELINE Graph Preprocessing Modularity Eigensolver Points Clustering Clustering ω vvt x 4 x 5 x x 2 7

18 SPECTRAL MODULARITY MAXIMIZATION 84% hit rate Spectral Modularity maximization Ground truth 8

19 PROFILING The eigensolver takes 90% of the time The sparse matrix vector multiplication takes 90% of the time in the eigensolver 9

20 MODULARITY VS. LAPLACIAN CLUSTERING Modularity higher and steadier modularity score 3x speedup over Laplacian Laplacian homogeneous cluster sizes Nvidia Titan X (Pascal), Intel Core GHz 20

21 SPEEDUP AND QUALITY VS. AGGLOMERATIVE * 0.8s on network with 00 million edges on a single Titan X GPU Speedup 3x over agglomerative* scheme Tradeoff Speed vs. quality *: D. LaSalle and G. Karypis. Multithreaded Modularity Based Graph Clustering Using the Multilevel Paradigm Parallel Distrib. Comput., Vol. 76, pp , 205. Nvidia Titan X (Pascal), Intel Core GHz 2

22 Spectral Clustering in CUDA Toolkit 9.0 release of nvgraph 22

23 CUDA TOOLKIT 9.0 nvgraph API nvgraphstatus_t nvgraphspectralclustering ( nvgraphhandle_t handle, const nvgraphgraphdescr_t graph_descr, const size_t weight_index, const struct SpectralClusteringParameter *params, int *clustering, void *eig_vals, void *eig_vects); struct SpectralClusteringParameter { int n_clusters; int n_eig_vects; nvgraphspectralclusteringtype_t alg float evs_tolerance int evs_max_iter; float kmean_tolerance; }; 23

24 Conclusions 24

25 Software Framework similar for both Laplacian - Minimum balanced cut [] SPECTRAL CLUSTERING Probably the most common metric, with balancing involved in the cost function Requires careful choice of eigensolver Modularity maximization [2] Widely used in analysis of social networks Faster to compute [] M. Naumov, T. Moon. Parallel spectral graph partitioning. Nvidia Technical Report, 206. [2] A. Fender, N. Emad, S. Petiton, M. Naumov. Parallel Modularity Clustering. ICCS,

26 Thank you H729 - Accelerated Libraries Monday and 4:00, Pod B 26

27