Introduction to Spectral Graph Theory and Graph Clustering
|
|
- Jemimah Black
- 6 years ago
- Views:
Transcription
1 Introduction to Spectral Graph Theory and Graph Clustering Chengming Jiang ECS 231 Spring 2016 University of California, Davis 1 / 40
2 Motivation Image partitioning in computer vision 2 / 40
3 Motivation Community detection in network analysis 3 / 40
4 Outline I. Graph and graph Laplacian Graph Weighted graph Graph Laplacian II. Graph clustering Graph clustering Normalized cut Spectral clustering 4 / 40
5 I.1 Graph An (undirected) graph is a pair G = (V, E), where {v i } is a set of vertices; E is a subset of V V. Remarks: An edge is a pair {u, v} with u v (no self-loop); There is at most one edge from u to v (simple graph). 5 / 40
6 I.1 Graph For every vertex v V, the degree d(v) of v is the number of edges adjacent to v: d(v) = {u V {u, v} E}. Let d i = d(v i ), the degree matrix D = D(G) = diag(d 1,..., d n ). D = / 40
7 I.1 Graph Given a graph G = (V, E), with V = n and E = m, the incidence matrix D(G) of G is an n m matrix with d ij = { 1, if k s.t. ej = {v i, v k } 0, otherwise D(G) = e 1 e 2 e 3 e 4 e 5 v v v v / 40
8 I.1 Graph Given a graph G = (V, E), with V = n and E = m, the adjacency matrix A(G) of G is a symmetric n n matrix with a ij = { 1, if {vi, v j } E 0, otherwise. A(G) = / 40
9 I.2 Weighted graph A weighted graph is a pair G = (V, W ) where V = {v i } is a set of vertices and V = n; W R n n is called weight matrix with { wji 0 if i j w ij = 0 if i = j The underlying graph of G is Ĝ = (V, E) with E = {{v i, v j } w ij > 0}. If w ij {0, 1}, W = A, the adjacency matrix of Ĝ. Since w ii = 0, there is no self-loops in Ĝ. 9 / 40
10 I.2 Weighted graph For every vertex v i V, the degree d(v i ) of v i is the sum of the weights of the edges adjacent to v i : d(v i ) = n w ij. j=1 Let d i = d(v i ), the degree matrix D = D(G) = diag(d 1,..., d n ). Remark: Let d = diag(d) and denote 1 = (1,..., 1) T, then d = W / 40
11 I.2 Weighted graph Given a subset of vertices A V, we define the volume vol(a) by vol(a) = d(v i ) = n w ij. v i A v i A j=1 Remarks: If vol(a) = 0, all the vertices in A are isolated. Example: If A = {v 1, v 3 }, then vol(a) =d(v 1 ) + d(v 3 ) =(w 12 + w 13 )+ (w 31 + w 32 + w 34 ) 11 / 40
12 I.2 Weighted graph Given two subsets of vertices A, B V, we define the links links(a, B) by Remarks: links(a, B) = A and B are not necessarily distinct; v i A,v j B w ij. Since W is symmetric, links(a, B) = links(b, A); vol(a) = links(a, V ). 12 / 40
13 I.2 Weighted graph The quantity cut(a) is defined by cut(a) = links(a, V A). The quantity assoc(a) is defined by assoc(a) = links(a, A). Remarks: cut(a) measures how many links escape from A; assoc(a) measures how many links stay within A; cut(a) + assoc(a) = vol(a). 13 / 40
14 I.3 Graph Laplacian Given a weighted graph G = (V, W ), the (graph) Laplacian L of G is defined by L = D W. where D is the degree matrix of G. Remark D = diag(w 1). 14 / 40
15 I.3 Graph Laplacian Properties of Laplacian 1. x T Lx = 1 2 n i,j=1 w ij(x i x j ) 2 for x R n. 2. L 0 if w ij 0 for all i, j; 3. L 1 = 0; 4. If the underlying graph of G is connected, then 0 = λ 1 < λ 2 λ 3... λ n ; 5. If the underlying graph of G is connected, then the dimension of the nullspace of L is / 40
16 I.3 Graph Laplacian Proofs Property 1. Since L = D W, we have x T Lx = x T Dx x T W x n n = d i x 2 i w ij x i x j i=1 i,j=1 = 1 n 2 ( d i x 2 i 2 i n w ij x i x j + i,j=1 = 1 n 2 ( w ij x 2 i 2 = 1 2 i,j=1 n d j x 2 j) j=1 n w ij x i x j + i,j=1 n w ij (x i x j ) 2. i,j=1 n w ij x 2 j) i,j=1 16 / 40
17 I.3 Graph Laplacian Property 2. Since L T = D W T = D W = L, L is symmetric. Since x T Lx = 1 n 2 i,j=1 w ij(x i x j ) 2 and w ij 0 for all i, j, we have x T Lx 0. Property 3. L 1 = (D W )1 = D1 W 1 = d d = 0. Properties 4 and 5: skip for now, see 2.2 of [Gallier 14]. 17 / 40
18 Outline I. Graph and graph Laplacian Graph Weighted graph Graph Laplacian II. Graph clustering Graph clustering Normalized cut Spectral clustering 18 / 40
19 II.1 Graph clustering k-way partitioning: given a weighted graph G = (V, W ), find a partition A 1, A 2,..., A k of V, such that A 1 A 2... A k = V ; A 1 A 2... A k = ; for any i and j, the edges between (A i, A j ) have low weight and the edges within A i have high weight. If k = 2, it is a two-way partitioning. 19 / 40
20 II.1 Graph clustering Recall: (two-way) cut: cut(a) = links(a, Ā) = where Ā = V A. v i A,v j Ā w ij 20 / 40
21 II.1 Graph clustering problems The mincut is defined by min cut(a) = min w ij. v i A,v j Ā In practice, the mincut easily yields unbalanced partitions. min cut(a) = = 3; 21 / 40
22 II.2 Normalized cut The normalized cut 1 is defined by Ncut(A) = cut(a) vol(a) + cut(ā) vol(ā). 1 Jianbo Shi and Jitendra Malik, / 40
23 II.2 Normalized cut Minimal Ncut: min Ncut(A). min Ncut(A) = = / 40
24 II.2 Normalized cut 1. Let x = (x 1,..., x n ) be the indicator vector, such that { 1 if vi A x i = 1 if v i Ā. 2. Then (1 + x) T D(1 + x) = 4 v i A d i = 4 vol(a); (1 + x) T W (1 + x) = 4 v i A,v j A w ij = 4 assoc(a). (1 + x) T L(1 + x) = 4 (vol(a) assoc(a)) = 4 cut(a); and (1 x) T D(1 x) = 4 v i Ā d i = 4 vol(ā); (1 x) T W (1 x) = 4 v i Ā,v j Ā w ij = 4 assoc(ā). (1 x) T L(1 x) = 4 (vol(ā) assoc(ā)) = 4 cut(ā). (Please verify it after class.) 24 / 40
25 II.2 Normalized cut 3. Ncut(A) can now be written as Ncut(A) = 1 4 ( (1 + x) T L(1 + x) k1 T D1 + (1 ) x)t L(1 x) (1 k)1 T D1 = 1 4 ((1 + x) b(1 x))t L((1 + x) b(1 x)) b1 T. D1 where k = vol(a)/vol(v ), b = k/(1 k) and vol(v ) = 1 T D1. 4. Let y = (1 + x) b(1 x), we have Ncut(A) = 1 4 y T Ly b1 T D1 where y i = { 2 if vi A 2b if v i Ā. 25 / 40
26 II.2 Normalized cut 5. Since b = k/(1 k) = vol(a)/vol(ā), we have In addition, 1 4 yt Dy = d i + b 2 d i = vol(a) + b 2 vol(ā) v i A v i Ā = b(vol(ā) + vol(a)) = b 1T D1. y T D1 = y T d = 2 d i 2b v i A v i Ā = 2 vol(a) 2b vol(ā) = 0 d i 26 / 40
27 II.2 Normalized cut 6. The minimal normalized cut is to solve the following binary optimization: y = arg min y s.t. y T Ly y T Dy y(i) {2, 2b} y T D1 = 0 (1) 7. Relaxation y = arg min y s.t. y T Ly y T Dy y R n y T D1 = 0 (2) 27 / 40
28 II.2 Normalized cut Variational principle Let A, B R n n, A T = A, B T = B > 0 and λ 1 λ 2... λ n be the eigenvalues of Au = λbu with corresponding eigenvectors u 1, u 2,..., u n, then and min x x T Ax x T Bx = λ 1, arg min x x T Ax x T Bx = u 1 x T Ax min x T Bu 1 =0 x T Bx = λ x T Ax 2, arg min x T Bu 1 =0 x T Bx = u 2. More general form exists. 28 / 40
29 II.2 Normalized cut For the matrix pair (L, D), it is known that (λ 1, y 1 ) = (0, 1). By the variational principle, the relaxed minimal Ncut (2) is equivalent to finding the second smallest eigenpair (λ 2, y 2 ) of Ly = λdy (3) Remarks: L is extremely sparse and D is diagonal; Precision requirement for eigenvectors is low, say O(10 3 ). 29 / 40
30 II.2 Normalized cut Image segmentation: original graph 30 / 40
31 II.2 Normalized cut Image segmentation: heatmap of eigenvectors 31 / 40
32 II.2 Normalized cut Image segmentation: result of min Ncut 32 / 40
33 II.3 Spectral clustering Ncut remaining issues Once the indicator vector is computed, how to search the splitting point that the resulting partition has the minimal Ncut(A) value? How to use the extreme eigenvectors to do the k-way partitioning? The above two problems are addressed in spectral clustering algorithm. 33 / 40
34 II.3 Spectral clustering Spectral clustering algorithm [Ng et al, 2002] Given a weighted graph G = (V, W ), 1. compute the normalized Laplacian L n = D 1 2 (D W )D 1 2 ; 2. find k eigenvectors X = [x 1,..., x k ] corresponding to the k smallest eigenvalues of L n ; 3. form Y R n k by normalizing each row of X such that Y (i, :) = X(i, ; )/ X(i, :) ; 4. treat each Y (i, :) as a point, cluster them into k clusters via K-means with label c i = {1,..., k}. The label c i indicates the cluster that v i belongs to. 34 / 40
35 II.3 Spectral clustering Synthetic example: original data 35 / 40
36 II.3 Spectral clustering Synthetic example: computed eigenvectors 36 / 40
37 II.3 Spectral clustering Synthetic example: 2-way clustering 37 / 40
38 II.3 Spectral clustering Synthetic example: 3-way clustering 38 / 40
39 II.3 Spectral clustering Synthetic example: 4-way clustering 39 / 40
40 References 1. Jean Gallier, Notes on elementary spectral graph theory applications to graph clustering using normalized cuts, Jianbo Shi and Jitendra Malik, Normalized cuts and image segmentation, Andrew Y Ng, Michael I. Jordan and Yair Weiss, On spectral clustering: Analysis and an algorithm, / 40
Laplacian Eigenmaps for Dimensionality Reduction and Data Representation
Introduction and Data Representation Mikhail Belkin & Partha Niyogi Department of Electrical Engieering University of Minnesota Mar 21, 2017 1/22 Outline Introduction 1 Introduction 2 3 4 Connections to
More informationSpectral Theory of Unsigned and Signed Graphs Applications to Graph Clustering: a Survey
Spectral Theory of Unsigned and Signed Graphs Applications to Graph Clustering: a Survey Jean Gallier Department of Computer and Information Science University of Pennsylvania Philadelphia, PA 19104, USA
More informationSummary: A Random Walks View of Spectral Segmentation, by Marina Meila (University of Washington) and Jianbo Shi (Carnegie Mellon University)
Summary: A Random Walks View of Spectral Segmentation, by Marina Meila (University of Washington) and Jianbo Shi (Carnegie Mellon University) The authors explain how the NCut algorithm for graph bisection
More informationNotes on Elementary Spectral Graph Theory Applications to Graph Clustering Using Normalized Cuts
University of Pennsylvania ScholarlyCommons Technical Reports (CIS) Department of Computer & Information Science 1-1-2013 Notes on Elementary Spectral Graph Theory Applications to Graph Clustering Using
More informationMATH 567: Mathematical Techniques in Data Science Clustering II
This lecture is based on U. von Luxburg, A Tutorial on Spectral Clustering, Statistics and Computing, 17 (4), 2007. MATH 567: Mathematical Techniques in Data Science Clustering II Dominique Guillot Departments
More informationSpectral Theory of Unsigned and Signed Graphs Applications to Graph Clustering. Some Slides
Spectral Theory of Unsigned and Signed Graphs Applications to Graph Clustering Some Slides Jean Gallier Department of Computer and Information Science University of Pennsylvania Philadelphia, PA 19104,
More information17.1 Directed Graphs, Undirected Graphs, Incidence Matrices, Adjacency Matrices, Weighted Graphs
Chapter 17 Graphs and Graph Laplacians 17.1 Directed Graphs, Undirected Graphs, Incidence Matrices, Adjacency Matrices, Weighted Graphs Definition 17.1. A directed graph isa pairg =(V,E), where V = {v
More informationSpectral Clustering on Handwritten Digits Database
University of Maryland-College Park Advance Scientific Computing I,II Spectral Clustering on Handwritten Digits Database Author: Danielle Middlebrooks Dmiddle1@math.umd.edu Second year AMSC Student Advisor:
More informationNotes on Elementary Spectral Graph Theory Applications to Graph Clustering
Notes on Elementary Spectral Graph Theory Applications to Graph Clustering Jean Gallier Department of Computer and Information Science University of Pennsylvania Philadelphia, PA 19104, USA e-mail: jean@cis.upenn.edu
More informationSpectral Clustering. Zitao Liu
Spectral Clustering Zitao Liu Agenda Brief Clustering Review Similarity Graph Graph Laplacian Spectral Clustering Algorithm Graph Cut Point of View Random Walk Point of View Perturbation Theory Point of
More informationSpectral Clustering. Guokun Lai 2016/10
Spectral Clustering Guokun Lai 2016/10 1 / 37 Organization Graph Cut Fundamental Limitations of Spectral Clustering Ng 2002 paper (if we have time) 2 / 37 Notation We define a undirected weighted graph
More informationMachine Learning for Data Science (CS4786) Lecture 11
Machine Learning for Data Science (CS4786) Lecture 11 Spectral clustering Course Webpage : http://www.cs.cornell.edu/courses/cs4786/2016sp/ ANNOUNCEMENT 1 Assignment P1 the Diagnostic assignment 1 will
More informationMATH 567: Mathematical Techniques in Data Science Clustering II
Spectral clustering: overview MATH 567: Mathematical Techniques in Data Science Clustering II Dominique uillot Departments of Mathematical Sciences University of Delaware Overview of spectral clustering:
More informationSpectral Clustering. Spectral Clustering? Two Moons Data. Spectral Clustering Algorithm: Bipartioning. Spectral methods
Spectral Clustering Seungjin Choi Department of Computer Science POSTECH, Korea seungjin@postech.ac.kr 1 Spectral methods Spectral Clustering? Methods using eigenvectors of some matrices Involve eigen-decomposition
More informationCS 556: Computer Vision. Lecture 13
CS 556: Computer Vision Lecture 1 Prof. Sinisa Todorovic sinisa@eecs.oregonstate.edu 1 Outline Perceptual grouping Low-level segmentation Ncuts Perceptual Grouping What do you see? 4 What do you see? Rorschach
More informationLecture 12: Introduction to Spectral Graph Theory, Cheeger s inequality
CSE 521: Design and Analysis of Algorithms I Spring 2016 Lecture 12: Introduction to Spectral Graph Theory, Cheeger s inequality Lecturer: Shayan Oveis Gharan May 4th Scribe: Gabriel Cadamuro Disclaimer:
More informationSTA141C: Big Data & High Performance Statistical Computing
STA141C: Big Data & High Performance Statistical Computing Lecture 12: Graph Clustering Cho-Jui Hsieh UC Davis May 29, 2018 Graph Clustering Given a graph G = (V, E, W ) V : nodes {v 1,, v n } E: edges
More informationGraphs in Machine Learning
Graphs in Machine Learning Michal Valko INRIA Lille - Nord Europe, France Partially based on material by: Ulrike von Luxburg, Gary Miller, Doyle & Schnell, Daniel Spielman January 27, 2015 MVA 2014/2015
More informationData Analysis and Manifold Learning Lecture 7: Spectral Clustering
Data Analysis and Manifold Learning Lecture 7: Spectral Clustering Radu Horaud INRIA Grenoble Rhone-Alpes, France Radu.Horaud@inrialpes.fr http://perception.inrialpes.fr/ Outline of Lecture 7 What is spectral
More informationA spectral clustering algorithm based on Gram operators
A spectral clustering algorithm based on Gram operators Ilaria Giulini De partement de Mathe matiques et Applications ENS, Paris Joint work with Olivier Catoni 1 july 2015 Clustering task of grouping
More informationChapter 4. Signed Graphs. Intuitively, in a weighted graph, an edge with a positive weight denotes similarity or proximity of its endpoints.
Chapter 4 Signed Graphs 4.1 Signed Graphs and Signed Laplacians Intuitively, in a weighted graph, an edge with a positive weight denotes similarity or proximity of its endpoints. For many reasons, it is
More informationCS 664 Segmentation (2) Daniel Huttenlocher
CS 664 Segmentation (2) Daniel Huttenlocher Recap Last time covered perceptual organization more broadly, focused in on pixel-wise segmentation Covered local graph-based methods such as MST and Felzenszwalb-Huttenlocher
More informationCOMPSCI 514: Algorithms for Data Science
COMPSCI 514: Algorithms for Data Science Arya Mazumdar University of Massachusetts at Amherst Fall 2018 Lecture 8 Spectral Clustering Spectral clustering Curse of dimensionality Dimensionality Reduction
More informationSpectral clustering. Two ideal clusters, with two points each. Spectral clustering algorithms
A simple example Two ideal clusters, with two points each Spectral clustering Lecture 2 Spectral clustering algorithms 4 2 3 A = Ideally permuted Ideal affinities 2 Indicator vectors Each cluster has an
More informationLearning Spectral Graph Segmentation
Learning Spectral Graph Segmentation AISTATS 2005 Timothée Cour Jianbo Shi Nicolas Gogin Computer and Information Science Department University of Pennsylvania Computer Science Ecole Polytechnique Graph-based
More informationData Analysis and Manifold Learning Lecture 3: Graphs, Graph Matrices, and Graph Embeddings
Data Analysis and Manifold Learning Lecture 3: Graphs, Graph Matrices, and Graph Embeddings Radu Horaud INRIA Grenoble Rhone-Alpes, France Radu.Horaud@inrialpes.fr http://perception.inrialpes.fr/ Outline
More informationGraphs in Machine Learning
Graphs in Machine Learning Michal Valko Inria Lille - Nord Europe, France TA: Pierre Perrault Partially based on material by: Ulrike von Luxburg, Gary Miller, Mikhail Belkin October 16th, 2017 MVA 2017/2018
More informationMLCC Clustering. Lorenzo Rosasco UNIGE-MIT-IIT
MLCC 2018 - Clustering Lorenzo Rosasco UNIGE-MIT-IIT About this class We will consider an unsupervised setting, and in particular the problem of clustering unlabeled data into coherent groups. MLCC 2018
More informationSolving Constrained Rayleigh Quotient Optimization Problem by Projected QEP Method
Solving Constrained Rayleigh Quotient Optimization Problem by Projected QEP Method Yunshen Zhou Advisor: Prof. Zhaojun Bai University of California, Davis yszhou@math.ucdavis.edu June 15, 2017 Yunshen
More information1 Matrix notation and preliminaries from spectral graph theory
Graph clustering (or community detection or graph partitioning) is one of the most studied problems in network analysis. One reason for this is that there are a variety of ways to define a cluster or community.
More informationSpectral Techniques for Clustering
Nicola Rebagliati 1/54 Spectral Techniques for Clustering Nicola Rebagliati 29 April, 2010 Nicola Rebagliati 2/54 Thesis Outline 1 2 Data Representation for Clustering Setting Data Representation and Methods
More informationCertifying the Global Optimality of Graph Cuts via Semidefinite Programming: A Theoretic Guarantee for Spectral Clustering
Certifying the Global Optimality of Graph Cuts via Semidefinite Programming: A Theoretic Guarantee for Spectral Clustering Shuyang Ling Courant Institute of Mathematical Sciences, NYU Aug 13, 2018 Joint
More informationLinear Spectral Hashing
Linear Spectral Hashing Zalán Bodó and Lehel Csató Babeş Bolyai University - Faculty of Mathematics and Computer Science Kogălniceanu 1., 484 Cluj-Napoca - Romania Abstract. assigns binary hash keys to
More informationNetworks and Their Spectra
Networks and Their Spectra Victor Amelkin University of California, Santa Barbara Department of Computer Science victor@cs.ucsb.edu December 4, 2017 1 / 18 Introduction Networks (= graphs) are everywhere.
More informationLecture 13: Spectral Graph Theory
CSE 521: Design and Analysis of Algorithms I Winter 2017 Lecture 13: Spectral Graph Theory Lecturer: Shayan Oveis Gharan 11/14/18 Disclaimer: These notes have not been subjected to the usual scrutiny reserved
More informationSpectral Graph Theory and its Applications. Daniel A. Spielman Dept. of Computer Science Program in Applied Mathematics Yale Unviersity
Spectral Graph Theory and its Applications Daniel A. Spielman Dept. of Computer Science Program in Applied Mathematics Yale Unviersity Outline Adjacency matrix and Laplacian Intuition, spectral graph drawing
More informationFinding normalized and modularity cuts by spectral clustering. Ljubjana 2010, October
Finding normalized and modularity cuts by spectral clustering Marianna Bolla Institute of Mathematics Budapest University of Technology and Economics marib@math.bme.hu Ljubjana 2010, October Outline Find
More informationGraph Partitioning Using Random Walks
Graph Partitioning Using Random Walks A Convex Optimization Perspective Lorenzo Orecchia Computer Science Why Spectral Algorithms for Graph Problems in practice? Simple to implement Can exploit very efficient
More informationDiffuse interface methods on graphs: Data clustering and Gamma-limits
Diffuse interface methods on graphs: Data clustering and Gamma-limits Yves van Gennip joint work with Andrea Bertozzi, Jeff Brantingham, Blake Hunter Department of Mathematics, UCLA Research made possible
More informationSome notes on SVD, dimensionality reduction, and clustering. Guangliang Chen
Some notes on SVD, dimensionality reduction, and clustering Guangliang Chen Contents 1. Introduction 4 2. Review of necessary linear algebra 4 3. Matrix SVD and its applications 8 Practice problems set
More informationMarkov Chains and Spectral Clustering
Markov Chains and Spectral Clustering Ning Liu 1,2 and William J. Stewart 1,3 1 Department of Computer Science North Carolina State University, Raleigh, NC 27695-8206, USA. 2 nliu@ncsu.edu, 3 billy@ncsu.edu
More informationCS224W: Social and Information Network Analysis Jure Leskovec, Stanford University
CS224W: Social and Information Network Analysis Jure Leskovec Stanford University Jure Leskovec, Stanford University http://cs224w.stanford.edu Task: Find coalitions in signed networks Incentives: European
More informationAn indicator for the number of clusters using a linear map to simplex structure
An indicator for the number of clusters using a linear map to simplex structure Marcus Weber, Wasinee Rungsarityotin, and Alexander Schliep Zuse Institute Berlin ZIB Takustraße 7, D-495 Berlin, Germany
More informationLecture 1: Graphs, Adjacency Matrices, Graph Laplacian
Lecture 1: Graphs, Adjacency Matrices, Graph Laplacian Radu Balan January 31, 2017 G = (V, E) An undirected graph G is given by two pieces of information: a set of vertices V and a set of edges E, G =
More informationAn Introduction to Spectral Graph Theory
An Introduction to Spectral Graph Theory Mackenzie Wheeler Supervisor: Dr. Gary MacGillivray University of Victoria WheelerM@uvic.ca Outline Outline 1. How many walks are there from vertices v i to v j
More informationSPECTRAL CLUSTERING AND KERNEL PRINCIPAL COMPONENT ANALYSIS ARE PURSUING GOOD PROJECTIONS
SPECTRAL CLUSTERING AND KERNEL PRINCIPAL COMPONENT ANALYSIS ARE PURSUING GOOD PROJECTIONS VIKAS CHANDRAKANT RAYKAR DECEMBER 5, 24 Abstract. We interpret spectral clustering algorithms in the light of unsupervised
More informationSpectral Generative Models for Graphs
Spectral Generative Models for Graphs David White and Richard C. Wilson Department of Computer Science University of York Heslington, York, UK wilson@cs.york.ac.uk Abstract Generative models are well known
More informationLecture Introduction. 2 Brief Recap of Lecture 10. CS-621 Theory Gems October 24, 2012
CS-62 Theory Gems October 24, 202 Lecture Lecturer: Aleksander Mądry Scribes: Carsten Moldenhauer and Robin Scheibler Introduction In Lecture 0, we introduced a fundamental object of spectral graph theory:
More informationStatistical and Computational Analysis of Locality Preserving Projection
Statistical and Computational Analysis of Locality Preserving Projection Xiaofei He xiaofei@cs.uchicago.edu Department of Computer Science, University of Chicago, 00 East 58th Street, Chicago, IL 60637
More informationMATH 829: Introduction to Data Mining and Analysis Clustering II
his lecture is based on U. von Luxburg, A Tutorial on Spectral Clustering, Statistics and Computing, 17 (4), 2007. MATH 829: Introduction to Data Mining and Analysis Clustering II Dominique Guillot Departments
More informationOn the Equivalence of Nonnegative Matrix Factorization and Spectral Clustering
On the Equivalence of Nonnegative Matrix Factorization and Spectral Clustering Chris Ding, Xiaofeng He, Horst D. Simon Published on SDM 05 Hongchang Gao Outline NMF NMF Kmeans NMF Spectral Clustering NMF
More informationSpectral Graph Theory Lecture 2. The Laplacian. Daniel A. Spielman September 4, x T M x. ψ i = arg min
Spectral Graph Theory Lecture 2 The Laplacian Daniel A. Spielman September 4, 2015 Disclaimer These notes are not necessarily an accurate representation of what happened in class. The notes written before
More informationK-Means, Expectation Maximization and Segmentation. D.A. Forsyth, CS543
K-Means, Expectation Maximization and Segmentation D.A. Forsyth, CS543 K-Means Choose a fixed number of clusters Choose cluster centers and point-cluster allocations to minimize error can t do this by
More informationLinear algebra and applications to graphs Part 1
Linear algebra and applications to graphs Part 1 Written up by Mikhail Belkin and Moon Duchin Instructor: Laszlo Babai June 17, 2001 1 Basic Linear Algebra Exercise 1.1 Let V and W be linear subspaces
More informationMa/CS 6b Class 23: Eigenvalues in Regular Graphs
Ma/CS 6b Class 3: Eigenvalues in Regular Graphs By Adam Sheffer Recall: The Spectrum of a Graph Consider a graph G = V, E and let A be the adjacency matrix of G. The eigenvalues of G are the eigenvalues
More informationSpectral Clustering on Handwritten Digits Database Mid-Year Pr
Spectral Clustering on Handwritten Digits Database Mid-Year Presentation Danielle dmiddle1@math.umd.edu Advisor: Kasso Okoudjou kasso@umd.edu Department of Mathematics University of Maryland- College Park
More informationCSE 291. Assignment Spectral clustering versus k-means. Out: Wed May 23 Due: Wed Jun 13
CSE 291. Assignment 3 Out: Wed May 23 Due: Wed Jun 13 3.1 Spectral clustering versus k-means Download the rings data set for this problem from the course web site. The data is stored in MATLAB format as
More informationLecture 12 : Graph Laplacians and Cheeger s Inequality
CPS290: Algorithmic Foundations of Data Science March 7, 2017 Lecture 12 : Graph Laplacians and Cheeger s Inequality Lecturer: Kamesh Munagala Scribe: Kamesh Munagala Graph Laplacian Maybe the most beautiful
More informationLearning from Sensor Data: Set II. Behnaam Aazhang J.S. Abercombie Professor Electrical and Computer Engineering Rice University
Learning from Sensor Data: Set II Behnaam Aazhang J.S. Abercombie Professor Electrical and Computer Engineering Rice University 1 6. Data Representation The approach for learning from data Probabilistic
More informationGraph fundamentals. Matrices associated with a graph
Graph fundamentals Matrices associated with a graph Drawing a picture of a graph is one way to represent it. Another type of representation is via a matrix. Let G be a graph with V (G) ={v 1,v,...,v n
More informationData Mining and Analysis: Fundamental Concepts and Algorithms
: Fundamental Concepts and Algorithms dataminingbook.info Mohammed J. Zaki 1 Wagner Meira Jr. 2 1 Department of Computer Science Rensselaer Polytechnic Institute, Troy, NY, USA 2 Department of Computer
More informationSpace-Variant Computer Vision: A Graph Theoretic Approach
p.1/65 Space-Variant Computer Vision: A Graph Theoretic Approach Leo Grady Cognitive and Neural Systems Boston University p.2/65 Outline of talk Space-variant vision - Why and how of graph theory Anisotropic
More information1 Matrix notation and preliminaries from spectral graph theory
Graph clustering (or community detection or graph partitioning) is one of the most studied problems in network analysis. One reason for this is that there are a variety of ways to define a cluster or community.
More informationAn Algorithmist s Toolkit September 10, Lecture 1
18.409 An Algorithmist s Toolkit September 10, 2009 Lecture 1 Lecturer: Jonathan Kelner Scribe: Jesse Geneson (2009) 1 Overview The class s goals, requirements, and policies were introduced, and topics
More informationPower Grid Partitioning: Static and Dynamic Approaches
Power Grid Partitioning: Static and Dynamic Approaches Miao Zhang, Zhixin Miao, Lingling Fan Department of Electrical Engineering University of South Florida Tampa FL 3320 miaozhang@mail.usf.edu zmiao,
More informationEmbedding and Spectrum of Graphs
Embedding and Spectrum of Graphs Ali Shokoufandeh, Department of Computer Science, Drexel University Overview Approximation Algorithms Geometry of Graphs and Graphs Encoding the Geometry Spectral Graph
More informationSpectral and Electrical Graph Theory. Daniel A. Spielman Dept. of Computer Science Program in Applied Mathematics Yale Unviersity
Spectral and Electrical Graph Theory Daniel A. Spielman Dept. of Computer Science Program in Applied Mathematics Yale Unviersity Outline Spectral Graph Theory: Understand graphs through eigenvectors and
More information1 T 1 = where 1 is the all-ones vector. For the upper bound, let v 1 be the eigenvector corresponding. u:(u,v) E v 1(u)
CME 305: Discrete Mathematics and Algorithms Instructor: Reza Zadeh (rezab@stanford.edu) Final Review Session 03/20/17 1. Let G = (V, E) be an unweighted, undirected graph. Let λ 1 be the maximum eigenvalue
More informationSpectral Graph Theory and You: Matrix Tree Theorem and Centrality Metrics
Spectral Graph Theory and You: and Centrality Metrics Jonathan Gootenberg March 11, 2013 1 / 19 Outline of Topics 1 Motivation Basics of Spectral Graph Theory Understanding the characteristic polynomial
More informationEigenvalue Problems Computation and Applications
Eigenvalue ProblemsComputation and Applications p. 1/36 Eigenvalue Problems Computation and Applications Che-Rung Lee cherung@gmail.com National Tsing Hua University Eigenvalue ProblemsComputation and
More informationLecture 7. 1 Normalized Adjacency and Laplacian Matrices
ORIE 6334 Spectral Graph Theory September 3, 206 Lecturer: David P. Williamson Lecture 7 Scribe: Sam Gutekunst In this lecture, we introduce normalized adjacency and Laplacian matrices. We state and begin
More informationGRAPH PARTITIONING WITH MATRIX COEFFICIENTS FOR SYMMETRIC POSITIVE DEFINITE LINEAR SYSTEMS
GRAPH PARTITIONING WITH MATRIX COEFFICIENTS FOR SYMMETRIC POSITIVE DEFINITE LINEAR SYSTEMS EUGENE VECHARYNSKI, YOUSEF SAAD, AND MASHA SOSONKINA Abstract. Prior to the parallel solution of a large linear
More informationACO Comprehensive Exam March 17 and 18, Computability, Complexity and Algorithms
1. Computability, Complexity and Algorithms (a) Let G(V, E) be an undirected unweighted graph. Let C V be a vertex cover of G. Argue that V \ C is an independent set of G. (b) Minimum cardinality vertex
More informationGraph Partitioning Algorithms and Laplacian Eigenvalues
Graph Partitioning Algorithms and Laplacian Eigenvalues Luca Trevisan Stanford Based on work with Tsz Chiu Kwok, Lap Chi Lau, James Lee, Yin Tat Lee, and Shayan Oveis Gharan spectral graph theory Use linear
More informationLaplacian Eigenmaps for Dimensionality Reduction and Data Representation
Laplacian Eigenmaps for Dimensionality Reduction and Data Representation Neural Computation, June 2003; 15 (6):1373-1396 Presentation for CSE291 sp07 M. Belkin 1 P. Niyogi 2 1 University of Chicago, Department
More informationUnsupervised Clustering of Human Pose Using Spectral Embedding
Unsupervised Clustering of Human Pose Using Spectral Embedding Muhammad Haseeb and Edwin R Hancock Department of Computer Science, The University of York, UK Abstract In this paper we use the spectra of
More informationFitting a Graph to Vector Data. Samuel I. Daitch (Yale) Jonathan A. Kelner (MIT) Daniel A. Spielman (Yale)
Fitting a Graph to Vector Data Samuel I. Daitch (Yale) Jonathan A. Kelner (MIT) Daniel A. Spielman (Yale) Machine Learning Summer School, June 2009 Given a collection of vectors x 1,..., x n IR d 1,, n
More informationFisher s Linear Discriminant Analysis
Fisher s Linear Discriminant Analysis Seungjin Choi Department of Computer Science and Engineering Pohang University of Science and Technology 77 Cheongam-ro, Nam-gu, Pohang 37673, Korea seungjin@postech.ac.kr
More informationGrouping with Bias. Stella X. Yu 1,2 Jianbo Shi 1. Robotics Institute 1 Carnegie Mellon University Center for the Neural Basis of Cognition 2
Grouping with Bias Stella X. Yu, Jianbo Shi Robotics Institute Carnegie Mellon University Center for the Neural Basis of Cognition What Is It About? Incorporating prior knowledge into grouping Unitary
More informationMATH 285 HW3. Xiaoyan Chong
MATH 285 HW3 Xiaoyan Chong November 20, 2015 1. (a) Draw the graph Figure 1: Graph (b) Find the degrees of all vertices d 1 = 0 +.15 +.15 +.3 + 0 =.6 d 2 =.15 + 0 +.85 + 0 + 0 = 1 d 3 =.15 +.85 + 0 + 0
More informationMining of Massive Datasets Jure Leskovec, AnandRajaraman, Jeff Ullman Stanford University
Note to other teachers and users of these slides: We would be delighted if you found this our material useful in giving your own lectures. Feel free to use these slides verbatim, or to modify them to fit
More information1 Review: symmetric matrices, their eigenvalues and eigenvectors
Cornell University, Fall 2012 Lecture notes on spectral methods in algorithm design CS 6820: Algorithms Studying the eigenvalues and eigenvectors of matrices has powerful consequences for at least three
More informationCOMMUNITY DETECTION IN SPARSE NETWORKS VIA GROTHENDIECK S INEQUALITY
COMMUNITY DETECTION IN SPARSE NETWORKS VIA GROTHENDIECK S INEQUALITY OLIVIER GUÉDON AND ROMAN VERSHYNIN Abstract. We present a simple and flexible method to prove consistency of semidefinite optimization
More informationPart I: Preliminary Results. Pak K. Chan, Martine Schlag and Jason Zien. Computer Engineering Board of Studies. University of California, Santa Cruz
Spectral K-Way Ratio-Cut Partitioning Part I: Preliminary Results Pak K. Chan, Martine Schlag and Jason Zien Computer Engineering Board of Studies University of California, Santa Cruz May, 99 Abstract
More information5 Quiver Representations
5 Quiver Representations 5. Problems Problem 5.. Field embeddings. Recall that k(y,..., y m ) denotes the field of rational functions of y,..., y m over a field k. Let f : k[x,..., x n ] k(y,..., y m )
More information8.1 Concentration inequality for Gaussian random matrix (cont d)
MGMT 69: Topics in High-dimensional Data Analysis Falll 26 Lecture 8: Spectral clustering and Laplacian matrices Lecturer: Jiaming Xu Scribe: Hyun-Ju Oh and Taotao He, October 4, 26 Outline Concentration
More informationGrouping with Directed Relationships
Grouping with Directed Relationships Stella. Yu Jianbo Shi Robotics Institute Carnegie Mellon University Center for the Neural Basis of Cognition Grouping: finding global structures coherence salience
More information6.207/14.15: Networks Lectures 4, 5 & 6: Linear Dynamics, Markov Chains, Centralities
6.207/14.15: Networks Lectures 4, 5 & 6: Linear Dynamics, Markov Chains, Centralities 1 Outline Outline Dynamical systems. Linear and Non-linear. Convergence. Linear algebra and Lyapunov functions. Markov
More informationSparse Subspace Clustering
Sparse Subspace Clustering Based on Sparse Subspace Clustering: Algorithm, Theory, and Applications by Elhamifar and Vidal (2013) Alex Gutierrez CSCI 8314 March 2, 2017 Outline 1 Motivation and Background
More informationOn the inverse matrix of the Laplacian and all ones matrix
On the inverse matrix of the Laplacian and all ones matrix Sho Suda (Joint work with Michio Seto and Tetsuji Taniguchi) International Christian University JSPS Research Fellow PD November 21, 2012 Sho
More informationRandom walks and anisotropic interpolation on graphs. Filip Malmberg
Random walks and anisotropic interpolation on graphs. Filip Malmberg Interpolation of missing data Assume that we have a graph where we have defined some (real) values for a subset of the nodes, and that
More informationA lower bound for the Laplacian eigenvalues of a graph proof of a conjecture by Guo
A lower bound for the Laplacian eigenvalues of a graph proof of a conjecture by Guo A. E. Brouwer & W. H. Haemers 2008-02-28 Abstract We show that if µ j is the j-th largest Laplacian eigenvalue, and d
More informationSpectral Clustering using Multilinear SVD
Spectral Clustering using Multilinear SVD Analysis, Approximations and Applications Debarghya Ghoshdastidar, Ambedkar Dukkipati Dept. of Computer Science & Automation Indian Institute of Science Clustering:
More informationLinear Systems of ODE: Nullclines, Eigenvector lines and trajectories
Linear Systems of ODE: Nullclines, Eigenvector lines and trajectories James K. Peterson Department of Biological Sciences and Department of Mathematical Sciences Clemson University October 6, 2013 Outline
More informationLecture 1: From Data to Graphs, Weighted Graphs and Graph Laplacian
Lecture 1: From Data to Graphs, Weighted Graphs and Graph Laplacian Radu Balan February 5, 2018 Datasets diversity: Social Networks: Set of individuals ( agents, actors ) interacting with each other (e.g.,
More informationFunctional Analysis Review
Outline 9.520: Statistical Learning Theory and Applications February 8, 2010 Outline 1 2 3 4 Vector Space Outline A vector space is a set V with binary operations +: V V V and : R V V such that for all
More informationECS 289 / MAE 298, Network Theory Spring 2014 Problem Set # 1, Solutions. Problem 1: Power Law Degree Distributions (Continuum approach)
ECS 289 / MAE 298, Network Theory Spring 204 Problem Set #, Solutions Problem : Power Law Degree Distributions (Continuum approach) Consider the power law distribution p(k) = Ak γ, with support (i.e.,
More informationRelations Between Adjacency And Modularity Graph Partitioning: Principal Component Analysis vs. Modularity Component Analysis
Relations Between Adjacency And Modularity Graph Partitioning: Principal Component Analysis vs. Modularity Component Analysis Hansi Jiang Carl Meyer North Carolina State University October 27, 2015 1 /
More informationLast lecture: Recurrence relations and differential equations. The solution to the differential equation dx
Last lecture: Recurrence relations and differential equations The solution to the differential equation dx = ax is x(t) = ce ax, where c = x() is determined by the initial conditions x(t) Let X(t) = and
More informationCommunities, Spectral Clustering, and Random Walks
Communities, Spectral Clustering, and Random Walks David Bindel Department of Computer Science Cornell University 26 Sep 2011 20 21 19 16 22 28 17 18 29 26 27 30 23 1 25 5 8 24 2 4 14 3 9 13 15 11 10 12
More information