Graph Partitioning Using Random Walks
|
|
- Everett Fowler
- 5 years ago
- Views:
Transcription
1 Graph Partitioning Using Random Walks A Convex Optimization Perspective Lorenzo Orecchia Computer Science
2 Why Spectral Algorithms for Graph Problems in practice? Simple to implement Can exploit very efficient linear algebra routines Perform well in practice for many problems in theory? Connections between spectral and combinatorial objects Connections to Markov Chains and Probability Theory Intuitive geometric viewpoint RECENT ADVANCES: Fast algorithms for fundamental combinatorial problems rely onspectraland optimizationideas
3 Spectral Algorithms for Graph Partitioning Spectral algorithms are widely used in many graph-partitioning applications: clustering, image segmentation, community-detection, etc. CLASSICAL VIEW: - Based on Cheeger s Inequality - Eigenvectors sweep-cuts reveal sparse cuts in the graph
4 Spectral Algorithms for Graph Partitioning Spectral algorithms are widely used in many graph-partitioning applications: clustering, image segmentation, community-detection, etc. CLASSICAL VIEW: NEW TREND: - Based on Cheeger s Inequality - Eigenvectors sweep-cuts reveal sparse cuts in the graph - Random walk vectors replace eigenvectors: Fast Algorithms for Graph Partitioning Local Graph Partitioning Empirical Network Analysis - Different random walks: PageRank, Heat-Kernel, etc.
5 Graphs As Linear Operators Adjacency Matrix A A ji = A ij = ( w ij if i 6= j 0 if i = j Weights are similarities
6 Adjacency Matrix A Graphs As Linear Operators Diagonal Degree Matrix D d i = X j w ij, D = diag(d)
7 Adjacency Matrix A Graphs As Linear Operators Diagonal Degree Matrix D Random Walk Matrix W W = AD 1 Probability transition matrix
8 Adjacency Matrix A Graphs As Linear Operators Diagonal Degree Matrix D Random Walk Matrix W Laplacian Matrix L W = AD 1 L = D A =(I W )D
9 Adjacency Matrix A Graphs As Linear Operators Diagonal Degree Matrix D Random Walk Matrix W Laplacian Matrix L W = AD 1 L = D A =(I W )D L is positive semidefinite, i.e., L º 0
10 Adjacency Matrix A Graphs As Linear Operators Diagonal Degree Matrix D Random Walk Matrix W Laplacian Matrix L W = AD 1 L = D A =(I W )D L is positive semidefinite, i.e., L º 0 Form x T L x measures how quickly random walk mixes from vector x All-one vector 1 is 0-eigenvalue, because D1 is stationary distribution
11 Example: Spectral Clustering Small eigenvalues of Laplacian control clustering structure of graph min = 1 =0 by construction 2 =0 if G is disconnected Robust to noise when large eigengap Components are recovered by computing 2 nd smallest eigenvector aka Fiedler vector
12 Why Random Walks? A Practitioner s View Advantages of Random Walks: 1) Quick approximation to eigenvector in massive graphs Fiedler vector can be approximating by applying power method to W: For random y 0 s.t. y0 T D 1 1 = 0, compute D 1 W t y 0 Heuristic: For massive graphs, pick t as large as computationally affordable.
13 Why Random Walks? A Practitioner s View Advantages of Random Walks: 1) Quick approximation to eigenvector in massive graphs 2) Statistical robustness Real-world graphs are noisy GROUND TRUTH GRAPH
14 Why Random Walks? A Practitioner s View Advantages of Random Walks: 1) Quick approximation to eigenvector in massive graphs 2) Statistical robustness Real-world graphs are noisy NOISY MEASUREMENT GROUND-TRUTH GRAPH INPUT GRAPH GOAL: estimate eigenvector of ground-truth graph.
15 Why Random Walks? A Practitioner s View Advantages of Random Walks: 1) Quick approximation to eigenvector in massive graphs 2) Statistical robustness NOISY MEASUREMENT GROUND-TRUTH GRAPH INPUT GRAPH GOAL: estimate eigenvector of ground-truthgraph. OBSERVATION: eigenvector of input graph can have very large variance, as it can be very sensitive to noise RANDOM-WALK VECTORS provide better, morestableestimates.
16 What s Wrong With Eigenvectors? 1) Computationally, eigenvector computation may be slow 2) Statistically, eigenvectors are not robust GROUND-TRUTH GRAPH
17 What s Wrong With Eigenvectors? 1) Computationally, eigenvector computation may be slow 2) Statistically, eigenvectors are not robust + GROUND-TRUTH GRAPH _ Eigenvector
18 What s Wrong With Eigenvectors? 1) Computationally, eigenvector computation may be slow 2) Statistically, eigenvectors are not robust + GROUND-TRUTH GRAPH Eigenvector _ NOISY MEASUREMENT
19 What s Wrong With Eigenvectors? 1) Computationally, eigenvector computation may be slow 2) Statistically, eigenvectors are not robust GROUND-TRUTH GRAPH + Eigenvector NOISY MEASUREMENT Eigenvector Changes Completely _ OBSERVATION: eigenvector of input graph can be very sensitive to noise
20 Talk Outline Stable Analogues of the Laplacian Eigenvectors: Formulate eigenvector as convex optimization problem Regularization yields stable optimum Natural connection to random walks Applications to Graph Partitioning: Spectral Clustering Balanced Graph Partitioning Detecting Well-Connected Subgraphs
21 Stable Analogues of Laplacian Eigenvectors
22 The Laplacian Eigenvalue Problem SOCP Formulation for Regular G min x T Lx s.t. kxk 2 =1 x T 1 = 0 x 2 R n. Eigenvector is one-dimensional embedding of the graph: 0
23 The Laplacian Eigenvalue Problem SOCP Formulation for Regular G min x T Lx s.t. kxk 2 =1 x T 1 = 0 x 2 R n. Eigenvector is one-dimensional embedding of the graph: ISSUES: 1. Non-convexity 2. Stability requires multidimensional embeddings 0
24 The Laplacian Eigenvalue Problem SOCP Formulation for Regular G min x T Lx s.t. kxk 2 =1 x T 1 = 0 x 2 R n. One-dimensional embedding: Multidimensional SOCP min Tr(Y T LY ) s.t. Tr(Y T Y ) = 1 Tr(Y T 11 T )= 0 Y 2 R n k. Multidimensional embedding: 0 Programs have same one-dimensional optimum.
25 Laplacian Eigenvalue as a Semidefinite Program SOCP Formulation for Regular G min x T Lx s.t. kxk 2 =1 x T 1 = 0 x 2 R n. One-dimensional embedding: min SDP Formulation L X s.t. I X =1 11 T X =0 Multidimensional embedding: X 0 0 Programs have same optimum. Take optimal solution: X = x (x ) T
26 Laplacian Eigenvalue as a Semidefinite Program SDP Formulation min L X s.t. I X =1 11 T X =0 X 0 Spectrahedron NB: Linear SDP is a convex representation of the original eigenvector problem Rank-1 optimal solution, i.e., 1-dimensional vector embedding
27 Regularization 101 Regularization is a fundamental technique in optimization OPTIMIZATION PROBLEM WELL-BEHAVED OPTIMIZATION PROBLEM Stable optimum Unique optimal solution Smoothness conditions
28 What is Regularization? Regularization is a fundamental technique in optimization OPTIMIZATION PROBLEM WELL-BEHAVED OPTIMIZATION PROBLEM min x2h L(x)+ F (x) Parameter > 0 Regularizer F Benefits of Regularization in Learning and Statistics: Decreases sensitivity to noise Prevents overfitting
29 Regularized Spectral Optimization SDP Formulation min L X F (X) s.t. I X =1 11 T X =0 X 0 ASSUMPTION: The regularizer F is 1-strongly convex over wrt some norm. Regularizer F 0 Parameter 0 RESULTS: Let X? be the optimal solution. 1. The embedding corresponding to will be multidimensional. X? X? 2. The embedding of can be approximated well in time O ( E polylog( E ))
30 Regularized Spectral Optimization F 0, 0 min L X F (X) s.t. I X =1 11 T X =0 X 0 ASSUMPTION: The regularizer F is 1-strongly convex over wrt some norm. RESULTS: Let X? be the optimal solution: 1. Multidimensional embedding, 2. Efficiently Computable, 3. Approximation guarantee: Objective value of SDP is near 2 L X? 2 apple max X2 F (X), 3. Stability aka smoothness under arbitrary perturbation H: kx? (L + H) X? (L)k apple khk.
31 Examples of Regularizers Applicable regularizers are SDP-versions of common regularizers von Neumann Entropy F H (X) =Tr(X log X) p-norm, p > 1 F p (X) = 1 p X p p = 1 p Tr(Xp ) 1/2-Quasi-Norm, F 1 2 (X) = 2 Tr(X 1 2 ) For each regularizer, we have a different trade-off between regularization and stability.
32 Regularized Eigenvectors and Random Walks Regularized SDP: min L X F (X) s.t. I X =1 11 T X =0 X 0 REGULARIZER F = F H Entropy OPTIMAL SOLUTION OF REGULARIZED PROGRAM HEAT-KERNEL DIFFUSION X? / e tl where t depends on F = F p p-norm F = F 1 2 ½-Quasi-Norm LAZY RANDOM WALK X? / (ai +(1 a)w ) 1 p 1 PERSONALIZED PAGERANK X? / (I aw ) 1 This interpretation has computational and analytical advantages where a depends on
33 Applications
34 Robust Spectral Clustering PROBLEM: Consider spectral clustering in the wild: - no eigengap assumption - input graph may be noisy APPROACH: Perform k-means of regularized spectral embedding. NOISY MEASUREMENT REGULARIZED EMBEDDING RESULT: guarantees on stability of clusters under different levels of noise With Zhenyu Liao (BU). In progress.
35 Balanced Graph Partitioning GOAL: Find large sparse cut S µ V : SPARSITY GUARANTEE on Conductance: (S) = E(S, S apple min{vol(s), vol( S)} SIZE GUARANTEE on Volume: 1 2 > vol(s ) vol(v ) > b S With Nisheeth Vishnoi (EPFL) and Sushant Sachdeva (Yale)
36 Eigenvector Approach: The Worst Case Recursive Eigenvector Algorithm: Compute smallest Laplacian eigenvector; Use it to find a cut; If cut is unbalanced, remove it and reiterate on residual graph. - (n) nearly-disconnected components
37 Eigenvector Approach: The Worst Case Recursive Eigenvector Algorithm: Compute smallest Laplacian eigenvector; Use it to find a cut; If cut is unbalanced, remove it and reiterate on residual graph. S S 3 2 S 1 - (n) components - (n) eigenvector computations
38 Regularized Eigenvector Approach EXPANDER The regularized eigenvector embedding reveals sparse large cut in one shot. RESULT: Balanced Graph Partitioning in Nearly-Linear Time O ( E polylog( E ))
39 Detecting Well-Connected Subsets GOAL: Detect subset S that is well-connected. Suppose second-smallest eigenvector looks like this: S 0 Question: what can we say about the connectivity of S? Using our stable analogue of the eigenvector, we obtain multidimensional embedding: S We can guarantee strong lower bounds on connectivity of the subgraph induced by S. Joint work with Nisheeth Vishnoi (EPFL) and Sushant Sachdeva (Yale)
40 and more Different Graph Partitioning Formulations: Local graph partitioning Different partitioning objectives Sparsification: Fastest construction of graph sparsifiers with constant average degree Uses PageRank regularizer Dynamically Evolving Networks: Analysis of local edge switching protocols that guarantee global connectivity properties THE END More Details at orecchia.net
Certifying the Global Optimality of Graph Cuts via Semidefinite Programming: A Theoretic Guarantee for Spectral Clustering
Certifying the Global Optimality of Graph Cuts via Semidefinite Programming: A Theoretic Guarantee for Spectral Clustering Shuyang Ling Courant Institute of Mathematical Sciences, NYU Aug 13, 2018 Joint
More informationLocally-biased analytics
Locally-biased analytics You have BIG data and want to analyze a small part of it: Solution 1: Cut out small part and use traditional methods Challenge: cutting out may be difficult a priori Solution 2:
More informationLecture 13: Spectral Graph Theory
CSE 521: Design and Analysis of Algorithms I Winter 2017 Lecture 13: Spectral Graph Theory Lecturer: Shayan Oveis Gharan 11/14/18 Disclaimer: These notes have not been subjected to the usual scrutiny reserved
More informationGraph Partitioning Algorithms and Laplacian Eigenvalues
Graph Partitioning Algorithms and Laplacian Eigenvalues Luca Trevisan Stanford Based on work with Tsz Chiu Kwok, Lap Chi Lau, James Lee, Yin Tat Lee, and Shayan Oveis Gharan spectral graph theory Use linear
More informationLecture: Local Spectral Methods (1 of 4)
Stat260/CS294: Spectral Graph Methods Lecture 18-03/31/2015 Lecture: Local Spectral Methods (1 of 4) Lecturer: Michael Mahoney Scribe: Michael Mahoney Warning: these notes are still very rough. They provide
More informationFour graph partitioning algorithms. Fan Chung University of California, San Diego
Four graph partitioning algorithms Fan Chung University of California, San Diego History of graph partitioning NP-hard approximation algorithms Spectral method, Fiedler 73, Folklore Multicommunity flow,
More informationLecture: Local Spectral Methods (3 of 4) 20 An optimization perspective on local spectral methods
Stat260/CS294: Spectral Graph Methods Lecture 20-04/07/205 Lecture: Local Spectral Methods (3 of 4) Lecturer: Michael Mahoney Scribe: Michael Mahoney Warning: these notes are still very rough. They provide
More informationLecture 14: Random Walks, Local Graph Clustering, Linear Programming
CSE 521: Design and Analysis of Algorithms I Winter 2017 Lecture 14: Random Walks, Local Graph Clustering, Linear Programming Lecturer: Shayan Oveis Gharan 3/01/17 Scribe: Laura Vonessen Disclaimer: These
More informationSpectral and Electrical Graph Theory. Daniel A. Spielman Dept. of Computer Science Program in Applied Mathematics Yale Unviersity
Spectral and Electrical Graph Theory Daniel A. Spielman Dept. of Computer Science Program in Applied Mathematics Yale Unviersity Outline Spectral Graph Theory: Understand graphs through eigenvectors and
More informationLecture: Local Spectral Methods (2 of 4) 19 Computing spectral ranking with the push procedure
Stat260/CS294: Spectral Graph Methods Lecture 19-04/02/2015 Lecture: Local Spectral Methods (2 of 4) Lecturer: Michael Mahoney Scribe: Michael Mahoney Warning: these notes are still very rough. They provide
More informationData Analysis and Manifold Learning Lecture 7: Spectral Clustering
Data Analysis and Manifold Learning Lecture 7: Spectral Clustering Radu Horaud INRIA Grenoble Rhone-Alpes, France Radu.Horaud@inrialpes.fr http://perception.inrialpes.fr/ Outline of Lecture 7 What is spectral
More informationConvex Optimization of Graph Laplacian Eigenvalues
Convex Optimization of Graph Laplacian Eigenvalues Stephen Boyd Abstract. We consider the problem of choosing the edge weights of an undirected graph so as to maximize or minimize some function of the
More informationDiffusion and random walks on graphs
Diffusion and random walks on graphs Leonid E. Zhukov School of Data Analysis and Artificial Intelligence Department of Computer Science National Research University Higher School of Economics Structural
More informationLECTURE NOTE #11 PROF. ALAN YUILLE
LECTURE NOTE #11 PROF. ALAN YUILLE 1. NonLinear Dimension Reduction Spectral Methods. The basic idea is to assume that the data lies on a manifold/surface in D-dimensional space, see figure (1) Perform
More informationLecture 12: Introduction to Spectral Graph Theory, Cheeger s inequality
CSE 521: Design and Analysis of Algorithms I Spring 2016 Lecture 12: Introduction to Spectral Graph Theory, Cheeger s inequality Lecturer: Shayan Oveis Gharan May 4th Scribe: Gabriel Cadamuro Disclaimer:
More information1 Matrix notation and preliminaries from spectral graph theory
Graph clustering (or community detection or graph partitioning) is one of the most studied problems in network analysis. One reason for this is that there are a variety of ways to define a cluster or community.
More informationSpectral Graph Theory and its Applications. Daniel A. Spielman Dept. of Computer Science Program in Applied Mathematics Yale Unviersity
Spectral Graph Theory and its Applications Daniel A. Spielman Dept. of Computer Science Program in Applied Mathematics Yale Unviersity Outline Adjacency matrix and Laplacian Intuition, spectral graph drawing
More informationLecture 7. 1 Normalized Adjacency and Laplacian Matrices
ORIE 6334 Spectral Graph Theory September 3, 206 Lecturer: David P. Williamson Lecture 7 Scribe: Sam Gutekunst In this lecture, we introduce normalized adjacency and Laplacian matrices. We state and begin
More informationSpectral Clustering on Handwritten Digits Database
University of Maryland-College Park Advance Scientific Computing I,II Spectral Clustering on Handwritten Digits Database Author: Danielle Middlebrooks Dmiddle1@math.umd.edu Second year AMSC Student Advisor:
More informationRandom Sampling of Bandlimited Signals on Graphs
Random Sampling of Bandlimited Signals on Graphs Pierre Vandergheynst École Polytechnique Fédérale de Lausanne (EPFL) School of Engineering & School of Computer and Communication Sciences Joint work with
More informationLecture 12 : Graph Laplacians and Cheeger s Inequality
CPS290: Algorithmic Foundations of Data Science March 7, 2017 Lecture 12 : Graph Laplacians and Cheeger s Inequality Lecturer: Kamesh Munagala Scribe: Kamesh Munagala Graph Laplacian Maybe the most beautiful
More informationMLCC Clustering. Lorenzo Rosasco UNIGE-MIT-IIT
MLCC 2018 - Clustering Lorenzo Rosasco UNIGE-MIT-IIT About this class We will consider an unsupervised setting, and in particular the problem of clustering unlabeled data into coherent groups. MLCC 2018
More informationGraphs, Vectors, and Matrices Daniel A. Spielman Yale University. AMS Josiah Willard Gibbs Lecture January 6, 2016
Graphs, Vectors, and Matrices Daniel A. Spielman Yale University AMS Josiah Willard Gibbs Lecture January 6, 2016 From Applied to Pure Mathematics Algebraic and Spectral Graph Theory Sparsification: approximating
More informationSpectral Clustering. Spectral Clustering? Two Moons Data. Spectral Clustering Algorithm: Bipartioning. Spectral methods
Spectral Clustering Seungjin Choi Department of Computer Science POSTECH, Korea seungjin@postech.ac.kr 1 Spectral methods Spectral Clustering? Methods using eigenvectors of some matrices Involve eigen-decomposition
More informationEIGENVECTOR NORMS MATTER IN SPECTRAL GRAPH THEORY
EIGENVECTOR NORMS MATTER IN SPECTRAL GRAPH THEORY Franklin H. J. Kenter Rice University ( United States Naval Academy 206) orange@rice.edu Connections in Discrete Mathematics, A Celebration of Ron Graham
More informationSummary: A Random Walks View of Spectral Segmentation, by Marina Meila (University of Washington) and Jianbo Shi (Carnegie Mellon University)
Summary: A Random Walks View of Spectral Segmentation, by Marina Meila (University of Washington) and Jianbo Shi (Carnegie Mellon University) The authors explain how the NCut algorithm for graph bisection
More informationNetworks and Their Spectra
Networks and Their Spectra Victor Amelkin University of California, Santa Barbara Department of Computer Science victor@cs.ucsb.edu December 4, 2017 1 / 18 Introduction Networks (= graphs) are everywhere.
More informationApproximating the Exponential, the Lanczos Method and an Õ(m)-Time Spectral Algorithm for Balanced Separator
Approximating the Exponential, the Lanczos Method and an Õm-Time Spectral Algorithm for Balanced Separator Lorenzo Orecchia MIT Cambridge, MA, USA. orecchia@mit.edu Sushant Sachdeva Princeton University
More informationData Analysis and Manifold Learning Lecture 3: Graphs, Graph Matrices, and Graph Embeddings
Data Analysis and Manifold Learning Lecture 3: Graphs, Graph Matrices, and Graph Embeddings Radu Horaud INRIA Grenoble Rhone-Alpes, France Radu.Horaud@inrialpes.fr http://perception.inrialpes.fr/ Outline
More informationIntroduction to Spectral Graph Theory and Graph Clustering
Introduction to Spectral Graph Theory and Graph Clustering Chengming Jiang ECS 231 Spring 2016 University of California, Davis 1 / 40 Motivation Image partitioning in computer vision 2 / 40 Motivation
More informationCommunities, Spectral Clustering, and Random Walks
Communities, Spectral Clustering, and Random Walks David Bindel Department of Computer Science Cornell University 26 Sep 2011 20 21 19 16 22 28 17 18 29 26 27 30 23 1 25 5 8 24 2 4 14 3 9 13 15 11 10 12
More informationFitting a Graph to Vector Data. Samuel I. Daitch (Yale) Jonathan A. Kelner (MIT) Daniel A. Spielman (Yale)
Fitting a Graph to Vector Data Samuel I. Daitch (Yale) Jonathan A. Kelner (MIT) Daniel A. Spielman (Yale) Machine Learning Summer School, June 2009 Given a collection of vectors x 1,..., x n IR d 1,, n
More informationLaplacian Matrices of Graphs: Spectral and Electrical Theory
Laplacian Matrices of Graphs: Spectral and Electrical Theory Daniel A. Spielman Dept. of Computer Science Program in Applied Mathematics Yale University Toronto, Sep. 28, 2 Outline Introduction to graphs
More informationLOCAL CHEEGER INEQUALITIES AND SPARSE CUTS
LOCAL CHEEGER INEQUALITIES AND SPARSE CUTS CAELAN GARRETT 18.434 FINAL PROJECT CAELAN@MIT.EDU Abstract. When computing on massive graphs, it is not tractable to iterate over the full graph. Local graph
More informationRegularized Laplacian Estimation and Fast Eigenvector Approximation
Regularized Laplacian Estimation and Fast Eigenvector Approximation Patrick O. Perry Information, Operations, and Management Sciences NYU Stern School of Business New York, NY 1001 pperry@stern.nyu.edu
More informationFast Linear Iterations for Distributed Averaging 1
Fast Linear Iterations for Distributed Averaging 1 Lin Xiao Stephen Boyd Information Systems Laboratory, Stanford University Stanford, CA 943-91 lxiao@stanford.edu, boyd@stanford.edu Abstract We consider
More informationDissertation Defense
Clustering Algorithms for Random and Pseudo-random Structures Dissertation Defense Pradipta Mitra 1 1 Department of Computer Science Yale University April 23, 2008 Mitra (Yale University) Dissertation
More informationSpectral Clustering. Guokun Lai 2016/10
Spectral Clustering Guokun Lai 2016/10 1 / 37 Organization Graph Cut Fundamental Limitations of Spectral Clustering Ng 2002 paper (if we have time) 2 / 37 Notation We define a undirected weighted graph
More informationSpectral clustering. Two ideal clusters, with two points each. Spectral clustering algorithms
A simple example Two ideal clusters, with two points each Spectral clustering Lecture 2 Spectral clustering algorithms 4 2 3 A = Ideally permuted Ideal affinities 2 Indicator vectors Each cluster has an
More informationData Mining and Analysis: Fundamental Concepts and Algorithms
: Fundamental Concepts and Algorithms dataminingbook.info Mohammed J. Zaki 1 Wagner Meira Jr. 2 1 Department of Computer Science Rensselaer Polytechnic Institute, Troy, NY, USA 2 Department of Computer
More informationGraph Sparsification I : Effective Resistance Sampling
Graph Sparsification I : Effective Resistance Sampling Nikhil Srivastava Microsoft Research India Simons Institute, August 26 2014 Graphs G G=(V,E,w) undirected V = n w: E R + Sparsification Approximate
More informationLecture: Some Practical Considerations (3 of 4)
Stat260/CS294: Spectral Graph Methods Lecture 14-03/10/2015 Lecture: Some Practical Considerations (3 of 4) Lecturer: Michael Mahoney Scribe: Michael Mahoney Warning: these notes are still very rough.
More informationAlgorithmic Primitives for Network Analysis: Through the Lens of the Laplacian Paradigm
Algorithmic Primitives for Network Analysis: Through the Lens of the Laplacian Paradigm Shang-Hua Teng Computer Science, Viterbi School of Engineering USC Massive Data and Massive Graphs 500 billions web
More informationA Local Spectral Method for Graphs: With Applications to Improving Graph Partitions and Exploring Data Graphs Locally
Journal of Machine Learning Research 13 (2012) 2339-2365 Submitted 11/12; Published 9/12 A Local Spectral Method for Graphs: With Applications to Improving Graph Partitions and Exploring Data Graphs Locally
More informationFinding normalized and modularity cuts by spectral clustering. Ljubjana 2010, October
Finding normalized and modularity cuts by spectral clustering Marianna Bolla Institute of Mathematics Budapest University of Technology and Economics marib@math.bme.hu Ljubjana 2010, October Outline Find
More informationLecture 10: October 27, 2016
Mathematical Toolkit Autumn 206 Lecturer: Madhur Tulsiani Lecture 0: October 27, 206 The conjugate gradient method In the last lecture we saw the steepest descent or gradient descent method for finding
More informationCS224W: Social and Information Network Analysis Jure Leskovec, Stanford University
CS224W: Social and Information Network Analysis Jure Leskovec Stanford University Jure Leskovec, Stanford University http://cs224w.stanford.edu Task: Find coalitions in signed networks Incentives: European
More informationManifold Regularization
9.520: Statistical Learning Theory and Applications arch 3rd, 200 anifold Regularization Lecturer: Lorenzo Rosasco Scribe: Hooyoung Chung Introduction In this lecture we introduce a class of learning algorithms,
More informationUsing Local Spectral Methods in Theory and in Practice
Using Local Spectral Methods in Theory and in Practice Michael W. Mahoney ICSI and Dept of Statistics, UC Berkeley ( For more info, see: http: // www. stat. berkeley. edu/ ~ mmahoney or Google on Michael
More informationConvex Optimization M2
Convex Optimization M2 Lecture 8 A. d Aspremont. Convex Optimization M2. 1/57 Applications A. d Aspremont. Convex Optimization M2. 2/57 Outline Geometrical problems Approximation problems Combinatorial
More informationSpectral Graph Theory Lecture 2. The Laplacian. Daniel A. Spielman September 4, x T M x. ψ i = arg min
Spectral Graph Theory Lecture 2 The Laplacian Daniel A. Spielman September 4, 2015 Disclaimer These notes are not necessarily an accurate representation of what happened in class. The notes written before
More informationOn sums of eigenvalues, and what they reveal about graphs and manifolds
On sums of eigenvalues, and what they reveal about graphs and manifolds Evans Harrell Georgia Tech Université F. Rabelais www.math.gatech.edu/~harrell Tours le 30 janvier, 2014 Copyright 2014 by Evans
More informationLecture 24: Element-wise Sampling of Graphs and Linear Equation Solving. 22 Element-wise Sampling of Graphs and Linear Equation Solving
Stat260/CS294: Randomized Algorithms for Matrices and Data Lecture 24-12/02/2013 Lecture 24: Element-wise Sampling of Graphs and Linear Equation Solving Lecturer: Michael Mahoney Scribe: Michael Mahoney
More informationConvex Optimization of Graph Laplacian Eigenvalues
Convex Optimization of Graph Laplacian Eigenvalues Stephen Boyd Stanford University (Joint work with Persi Diaconis, Arpita Ghosh, Seung-Jean Kim, Sanjay Lall, Pablo Parrilo, Amin Saberi, Jun Sun, Lin
More information8.1 Concentration inequality for Gaussian random matrix (cont d)
MGMT 69: Topics in High-dimensional Data Analysis Falll 26 Lecture 8: Spectral clustering and Laplacian matrices Lecturer: Jiaming Xu Scribe: Hyun-Ju Oh and Taotao He, October 4, 26 Outline Concentration
More informationComputer Vision Group Prof. Daniel Cremers. 10a. Markov Chain Monte Carlo
Group Prof. Daniel Cremers 10a. Markov Chain Monte Carlo Markov Chain Monte Carlo In high-dimensional spaces, rejection sampling and importance sampling are very inefficient An alternative is Markov Chain
More informationMarkov Chains and Spectral Clustering
Markov Chains and Spectral Clustering Ning Liu 1,2 and William J. Stewart 1,3 1 Department of Computer Science North Carolina State University, Raleigh, NC 27695-8206, USA. 2 nliu@ncsu.edu, 3 billy@ncsu.edu
More informationA Statistical Look at Spectral Graph Analysis. Deep Mukhopadhyay
A Statistical Look at Spectral Graph Analysis Deep Mukhopadhyay Department of Statistics, Temple University Office: Speakman 335 deep@temple.edu http://sites.temple.edu/deepstat/ Graph Signal Processing
More informationMachine Learning for Data Science (CS4786) Lecture 11
Machine Learning for Data Science (CS4786) Lecture 11 Spectral clustering Course Webpage : http://www.cs.cornell.edu/courses/cs4786/2016sp/ ANNOUNCEMENT 1 Assignment P1 the Diagnostic assignment 1 will
More informationSum-of-Squares Method, Tensor Decomposition, Dictionary Learning
Sum-of-Squares Method, Tensor Decomposition, Dictionary Learning David Steurer Cornell Approximation Algorithms and Hardness, Banff, August 2014 for many problems (e.g., all UG-hard ones): better guarantees
More informationCOMPSCI 514: Algorithms for Data Science
COMPSCI 514: Algorithms for Data Science Arya Mazumdar University of Massachusetts at Amherst Fall 2018 Lecture 8 Spectral Clustering Spectral clustering Curse of dimensionality Dimensionality Reduction
More informationLaplacian Eigenmaps for Dimensionality Reduction and Data Representation
Laplacian Eigenmaps for Dimensionality Reduction and Data Representation Neural Computation, June 2003; 15 (6):1373-1396 Presentation for CSE291 sp07 M. Belkin 1 P. Niyogi 2 1 University of Chicago, Department
More information18.S096: Spectral Clustering and Cheeger s Inequality
18.S096: Spectral Clustering and Cheeger s Inequality Topics in Mathematics of Data Science (Fall 015) Afonso S. Bandeira bandeira@mit.edu http://math.mit.edu/~bandeira October 6, 015 These are lecture
More informationData Mining Techniques
Data Mining Techniques CS 622 - Section 2 - Spring 27 Pre-final Review Jan-Willem van de Meent Feedback Feedback https://goo.gl/er7eo8 (also posted on Piazza) Also, please fill out your TRACE evaluations!
More informationNetwork Localization via Schatten Quasi-Norm Minimization
Network Localization via Schatten Quasi-Norm Minimization Anthony Man-Cho So Department of Systems Engineering & Engineering Management The Chinese University of Hong Kong (Joint Work with Senshan Ji Kam-Fung
More informationDimension reduction for semidefinite programming
1 / 22 Dimension reduction for semidefinite programming Pablo A. Parrilo Laboratory for Information and Decision Systems Electrical Engineering and Computer Science Massachusetts Institute of Technology
More informationUnsupervised dimensionality reduction
Unsupervised dimensionality reduction Guillaume Obozinski Ecole des Ponts - ParisTech SOCN course 2014 Guillaume Obozinski Unsupervised dimensionality reduction 1/30 Outline 1 PCA 2 Kernel PCA 3 Multidimensional
More informationEigenvalue Problems Computation and Applications
Eigenvalue ProblemsComputation and Applications p. 1/36 Eigenvalue Problems Computation and Applications Che-Rung Lee cherung@gmail.com National Tsing Hua University Eigenvalue ProblemsComputation and
More informationSpectral Clustering. Zitao Liu
Spectral Clustering Zitao Liu Agenda Brief Clustering Review Similarity Graph Graph Laplacian Spectral Clustering Algorithm Graph Cut Point of View Random Walk Point of View Perturbation Theory Point of
More informationContribution from: Springer Verlag Berlin Heidelberg 2005 ISBN
Contribution from: Mathematical Physics Studies Vol. 7 Perspectives in Analysis Essays in Honor of Lennart Carleson s 75th Birthday Michael Benedicks, Peter W. Jones, Stanislav Smirnov (Eds.) Springer
More informationOn a Cut-Matching Game for the Sparsest Cut Problem
On a Cut-Matching Game for the Sparsest Cut Problem Rohit Khandekar Subhash A. Khot Lorenzo Orecchia Nisheeth K. Vishnoi Electrical Engineering and Computer Sciences University of California at Berkeley
More informationPartitioning Algorithms that Combine Spectral and Flow Methods
CS369M: Algorithms for Modern Massive Data Set Analysis Lecture 15-11/11/2009 Partitioning Algorithms that Combine Spectral and Flow Methods Lecturer: Michael Mahoney Scribes: Kshipra Bhawalkar and Deyan
More informationLaplacian and Random Walks on Graphs
Laplacian and Random Walks on Graphs Linyuan Lu University of South Carolina Selected Topics on Spectral Graph Theory (II) Nankai University, Tianjin, May 22, 2014 Five talks Selected Topics on Spectral
More informationFacebook Friends! and Matrix Functions
Facebook Friends! and Matrix Functions! Graduate Research Day Joint with David F. Gleich, (Purdue), supported by" NSF CAREER 1149756-CCF Kyle Kloster! Purdue University! Network Analysis Use linear algebra
More informationThe Second Eigenvalue of the Google Matrix
The Second Eigenvalue of the Google Matrix Taher H. Haveliwala and Sepandar D. Kamvar Stanford University {taherh,sdkamvar}@cs.stanford.edu Abstract. We determine analytically the modulus of the second
More informationSemi Supervised Distance Metric Learning
Semi Supervised Distance Metric Learning wliu@ee.columbia.edu Outline Background Related Work Learning Framework Collaborative Image Retrieval Future Research Background Euclidean distance d( x, x ) =
More informationAn indicator for the number of clusters using a linear map to simplex structure
An indicator for the number of clusters using a linear map to simplex structure Marcus Weber, Wasinee Rungsarityotin, and Alexander Schliep Zuse Institute Berlin ZIB Takustraße 7, D-495 Berlin, Germany
More informationSTA141C: Big Data & High Performance Statistical Computing
STA141C: Big Data & High Performance Statistical Computing Lecture 12: Graph Clustering Cho-Jui Hsieh UC Davis May 29, 2018 Graph Clustering Given a graph G = (V, E, W ) V : nodes {v 1,, v n } E: edges
More informationCommunities, Spectral Clustering, and Random Walks
Communities, Spectral Clustering, and Random Walks David Bindel Department of Computer Science Cornell University 3 Jul 202 Spectral clustering recipe Ingredients:. A subspace basis with useful information
More informationFast Approximation Algorithms for Graph Partitioning using Spectral and Semidefinite-Programming Techniques
Fast Approximation Algorithms for Graph Partitioning using Spectral and Semidefinite-Programming Techniques Lorenzo Orecchia Electrical Engineering and Computer Sciences University of California at Berkeley
More informationLearning Eigenfunctions: Links with Spectral Clustering and Kernel PCA
Learning Eigenfunctions: Links with Spectral Clustering and Kernel PCA Yoshua Bengio Pascal Vincent Jean-François Paiement University of Montreal April 2, Snowbird Learning 2003 Learning Modal Structures
More informationLecture 1. 1 if (i, j) E a i,j = 0 otherwise, l i,j = d i if i = j, and
Specral Graph Theory and its Applications September 2, 24 Lecturer: Daniel A. Spielman Lecture. A quick introduction First of all, please call me Dan. If such informality makes you uncomfortable, you can
More informationNonlinear Dimensionality Reduction
Outline Hong Chang Institute of Computing Technology, Chinese Academy of Sciences Machine Learning Methods (Fall 2012) Outline Outline I 1 Kernel PCA 2 Isomap 3 Locally Linear Embedding 4 Laplacian Eigenmap
More informationSpectral Clustering on Handwritten Digits Database Mid-Year Pr
Spectral Clustering on Handwritten Digits Database Mid-Year Presentation Danielle dmiddle1@math.umd.edu Advisor: Kasso Okoudjou kasso@umd.edu Department of Mathematics University of Maryland- College Park
More informationNetwork Topology Inference from Non-stationary Graph Signals
Network Topology Inference from Non-stationary Graph Signals Rasoul Shafipour Dept. of Electrical and Computer Engineering University of Rochester rshafipo@ece.rochester.edu http://www.ece.rochester.edu/~rshafipo/
More informationSpectral Graph Theory
Spectral raph Theory to appear in Handbook of Linear Algebra, second edition, CCR Press Steve Butler Fan Chung There are many different ways to associate a matrix with a graph (an introduction of which
More informationSparse Subspace Clustering
Sparse Subspace Clustering Based on Sparse Subspace Clustering: Algorithm, Theory, and Applications by Elhamifar and Vidal (2013) Alex Gutierrez CSCI 8314 March 2, 2017 Outline 1 Motivation and Background
More informationData-dependent representations: Laplacian Eigenmaps
Data-dependent representations: Laplacian Eigenmaps November 4, 2015 Data Organization and Manifold Learning There are many techniques for Data Organization and Manifold Learning, e.g., Principal Component
More informationLearning on Graphs and Manifolds. CMPSCI 689 Sridhar Mahadevan U.Mass Amherst
Learning on Graphs and Manifolds CMPSCI 689 Sridhar Mahadevan U.Mass Amherst Outline Manifold learning is a relatively new area of machine learning (2000-now). Main idea Model the underlying geometry of
More informationSpectral Graph Theory
Spectral Graph Theory Aaron Mishtal April 27, 2016 1 / 36 Outline Overview Linear Algebra Primer History Theory Applications Open Problems Homework Problems References 2 / 36 Outline Overview Linear Algebra
More informationAlgorithms, Graph Theory, and the Solu7on of Laplacian Linear Equa7ons. Daniel A. Spielman Yale University
Algorithms, Graph Theory, and the Solu7on of Laplacian Linear Equa7ons Daniel A. Spielman Yale University Rutgers, Dec 6, 2011 Outline Linear Systems in Laplacian Matrices What? Why? Classic ways to solve
More informationEugene Wigner [4]: in the natural sciences, Communications in Pure and Applied Mathematics, XIII, (1960), 1 14.
Introduction Eugene Wigner [4]: The miracle of the appropriateness of the language of mathematics for the formulation of the laws of physics is a wonderful gift which we neither understand nor deserve.
More informationData Mining and Matrices
Data Mining and Matrices 08 Boolean Matrix Factorization Rainer Gemulla, Pauli Miettinen June 13, 2013 Outline 1 Warm-Up 2 What is BMF 3 BMF vs. other three-letter abbreviations 4 Binary matrices, tiles,
More informationSpectra of Adjacency and Laplacian Matrices
Spectra of Adjacency and Laplacian Matrices Definition: University of Alicante (Spain) Matrix Computing (subject 3168 Degree in Maths) 30 hours (theory)) + 15 hours (practical assignment) Contents 1. Spectra
More informationInformation Recovery from Pairwise Measurements
Information Recovery from Pairwise Measurements A Shannon-Theoretic Approach Yuxin Chen, Changho Suh, Andrea Goldsmith Stanford University KAIST Page 1 Recovering data from correlation measurements A large
More informationThe Nyström Extension and Spectral Methods in Learning
Introduction Main Results Simulation Studies Summary The Nyström Extension and Spectral Methods in Learning New bounds and algorithms for high-dimensional data sets Patrick J. Wolfe (joint work with Mohamed-Ali
More informationSVD, Power method, and Planted Graph problems (+ eigenvalues of random matrices)
Chapter 14 SVD, Power method, and Planted Graph problems (+ eigenvalues of random matrices) Today we continue the topic of low-dimensional approximation to datasets and matrices. Last time we saw the singular
More informationCS598 Machine Learning in Computational Biology (Lecture 5: Matrix - part 2) Professor Jian Peng Teaching Assistant: Rongda Zhu
CS598 Machine Learning in Computational Biology (Lecture 5: Matrix - part 2) Professor Jian Peng Teaching Assistant: Rongda Zhu Feature engineering is hard 1. Extract informative features from domain knowledge
More informationPredicting Graph Labels using Perceptron. Shuang Song
Predicting Graph Labels using Perceptron Shuang Song shs037@eng.ucsd.edu Online learning over graphs M. Herbster, M. Pontil, and L. Wainer, Proc. 22nd Int. Conf. Machine Learning (ICML'05), 2005 Prediction
More informationAcyclic Semidefinite Approximations of Quadratically Constrained Quadratic Programs
Acyclic Semidefinite Approximations of Quadratically Constrained Quadratic Programs Raphael Louca & Eilyan Bitar School of Electrical and Computer Engineering American Control Conference (ACC) Chicago,
More information