Laplacian Matrices of Graphs: Spectral and Electrical Theory

Transcription

1 Laplacian Matrices of Graphs: Spectral and Electrical Theory Daniel A. Spielman Dept. of Computer Science Program in Applied Mathematics Yale University Toronto, Sep. 28, 2

2 Outline Introduction to graphs Physical metaphors Laplacian matrices Spectral graph theory A very fast survey Trailer for lectures 2 and 3

3 Graphs and Networks V: a set of vertices (nodes) E: a set of edges an edge is a pair of vertices Dan Nikhil Donna Allan Shang- Hua Maria Gary Difficult to draw when big

4 Examples of Graphs

5 Examples of Graphs

6 Examples of Graphs

7 How to understand a graph Use physical metaphors Edges as rubber bands Edges as resistors Examine processes Diffusion of gas Spilling paint Identify structures Communities

8 How to understand a graph Use physical metaphors Edges as rubber bands Edges as resistors Examine processes Diffusion of gas Spilling paint Identify structures Communities

9 Graphs as Spring Networks View edges as rubber bands or ideal linear springs spring constant (for now) Nail down some ver8ces, let rest se<le

10 Graphs as Spring Networks View edges as rubber bands or ideal linear springs spring constant (for now) Nail down some ver8ces, let rest se<le When stretched to length poten8al energy is 2 /2

11 Graphs as Spring Networks Nail down some ver8ces, let rest se<le. x(a) a Physics: posi8on minimizes total poten8al energy (x(a) x(b)) 2 2 (a,b) E subject to boundary constraints (nails)

12 Graphs as Spring Networks Nail down some ver8ces, let rest se<le x(a) a Energy minimized when free ver8ces are averages of neighbors x(a) = d a (a,b) E x(b) d a is degree of a, number of attached edges

13 Tutte s Theorem 63 If nail down a face of a planar 3- connected graph, get a planar embedding!

14 Tutte s Theorem connected: cannot break graph by curng 2 edges

15 Tutte s Theorem connected: cannot break graph by curng 2 edges

16 Tutte s Theorem connected: cannot break graph by curng 2 edges

17 Tutte s Theorem connected: cannot break graph by curng 2 edges

18 Graphs as Resistor Networks View edges as resistors connecting vertices Apply voltages at some vertices. Measure induced voltages and current flow. V V

19 Graphs as Resistor Networks View edges as resistors connecting vertices Apply voltages at some vertices. Measure induced voltages and current flow. Current flow measures strength of connection between endpoints. More short disjoint paths lead to higher flow.

20 Graphs as Resistor Networks View edges as resistors connecting vertices Apply voltages at some vertices. Measure induced voltages and current flow..5v.5v V V.375V.625V

21 Graphs as Resistor Networks View edges as resistors connecting vertices Apply voltages at some vertices. Measure induced voltages and current flow. Induced voltages minimize (v(a) v(b)) 2 (a,b) E Subject to fixed voltages (by battery)

22 Learning on Graphs [Zhu- Ghahramani- Lafferty 3] Infer values of a function at all vertices from known values at a few vertices. Minimize (a,b) E Subject to known values (x(a) x(b)) 2

23 Learning on Graphs [Zhu- Ghahramani- Lafferty 3] Infer values of a function at all vertices from known values at a few vertices. Minimize (a,b) E Subject to known values (x(a) x(b))

24 The Laplacian quadratic form (x(a) x(b)) 2 (a,b) E

25 The Laplacian matrix of a graph x T Lx = (a,b) E (x(a) x(b)) 2

26 The Laplacian matrix of a graph x T Lx = (a,b) E (x(a) x(b)) 2 To minimize subject to boundary constraints, set derivative to zero. Solve equation of form Lx = b

27 Weighted Graphs (a, b) Edge assigned a non-negative real weight w a,b R measuring strength of connection spring constant /resistance x T Lx = (a,b) E w a,b (x(a) x(b)) 2

28 Weighted Graphs (a, b) Edge assigned a non-negative real weight w a,b R measuring strength of connection spring constant /resistance x T Lx = (a,b) E w a,b (x(a) x(b)) 2 I ll show the matrix entries tomorrow

29 Measuring boundaries of sets Boundary: edges leaving a set S

30 Measuring boundaries of sets Boundary: edges leaving a set S

31 Measuring boundaries of sets Boundary: edges leaving a set Characteristic Vector of S: x(a) = a in S a not in S S S

32 Measuring boundaries of sets Boundary: edges leaving a set Characteristic Vector of S: x(a) = a in S a not in S S S x T Lx = (x(a) x(b)) 2 = boundary(s) (a,b) E

33 Cluster Quality Number of edges leaving S Size of S = boundary(s) S def = Φ(S) (sparsity) S S

34 Cluster Quality Number of edges leaving S Size of S = boundary(s) S def = Φ(S) (sparsity) S S = xt Lx x T x = (a,b) E (x(a) x(b))2 a x(a)2 The Rayleigh Quotient of x with respect to L

35 Spectral Graph Theory A n-by-n symmetric matrix has n real eigenvalues λ λ 2 λ n and eigenvectors v,...,v n such that Lv i = λ i v i

36 Spectral Graph Theory A n-by-n symmetric matrix has n real eigenvalues λ λ 2 λ n and eigenvectors v,...,v n such that These eigenvalues and eigenvectors tell us a lot about a graph! Theorems Algorithms Heuristics Lv i = λ i v i

37 The Rayleigh Quotient and Eigenvalues A n-by-n symmetric matrix has n real eigenvalues λ λ 2 λ n and eigenvectors v,...,v n such that Lv i = λ i v i Courant-Fischer Theorem: λ =min x= x T Lx x T x v = arg min x= x T Lx x T x

38 The Courant Fischer Theorem

39 The Courant Fischer Theorem λ k = min x v,...,v k x T Lx x T x

40 The first eigenvalue λ =min x= =min x= x T Lx x T x (a,b) E (x(a) x(b))2 x 2 Setting x(a) = for all a We find λ = and v =

41 The second eigenvalue λ 2 > if and only if G is connected Proof: if G is not connected, are two functions with Rayleigh quotient zero

42 The second eigenvalue λ 2 > if and only if G is connected Proof: if G is not connected, are two functions with Rayleigh quotient zero

43 The second eigenvalue λ 2 > if and only if G is connected Proof: if G is connected, x means a x(a) = So must be an edge (a,b) for which x(a) <x(b) and so (x(a) x(b)) 2 > + -

44 The second eigenvalue λ 2 > if and only if G is connected Proof: if G is connected, x means a x(a) = So must be an edge (a,b) for which x(a) <x(b) and so (x(a) x(b)) 2 >

45 The second eigenvalue λ 2 > if and only if G is connected Fiedler ( 73) called the algebraic connectivity of a graph The further from, the more connected.

46 Cheeger s Inequality [Cheeger 7] [Alon-Milman 85, Jerrum-Sinclair 89, Diaconis-Stroock 9]. is big if and only if G does not have good clusters. 2. If is small, can use v 2 to find a good cluster.

47 Cheeger s Inequality [Cheeger 7] [Alon-Milman 85, Jerrum-Sinclair 89, Diaconis-Stroock 9]. is big if and only if G does not have good clusters. When every vertex has d edges, λ 2 /2 min S n/2 Φ(S) 2dλ 2 Φ(S) = boundary(s) S

48 Cheeger s Inequality [Cheeger 7] [Alon-Milman 85, Jerrum-Sinclair 89, Diaconis-Stroock 9]. is big if and only if G does not have good clusters. 2. If is small, can use v 2 to find a good cluster. In a moment

49 Spectral Graph Drawing [Hall 7] Arbitrary Drawing

50 Spectral Graph Drawing [Hall 7] Plot vertex a at (v 2 (a),v 3 (a)) draw edges as straight lines Arbitrary Drawing 9 7 Spectral Drawing 8

51 A Graph

52 Drawing of the graph using v 2, v 3 Plot vertex a at (v 2 (a),v 3 (a))

53

54 The Airfoil Graph, original coordinates

55 The Airfoil Graph, spectral coordinates

56 The Airfoil Graph, spectral coordinates

57 Spectral drawing of Streets in Rome

58 Spectral drawing of Erdos graph: edge between co-authors of papers

59 Dodecahedron Best embedded by first three eigenvectors

60 Spectral graph drawing: Tutte justification Condition for eigenvector Gives x(a) = d a λ (a,b) E x(b) for all a λ small says x(a) near average of neighbors

61 Spectral graph drawing: Tutte justification Condition for eigenvector Gives x(a) = d a λ (a,b) E x(b) for all a λ small says x(a) near average of neighbors For planar graphs: λ 2 8d/n [S-Teng 96] λ 3 O(d/n) [Kelner-Lee-Price-Teng 9]

62 Small eigenvalues are not enough Plot vertex a at (v 3 (a),v 4 (a))

63 Spectral Graph Partitioning [Donath-Hoffman 72, Barnes 82, Hagen-Kahng 92] S = {a : v 2 (a) t} for some t

64 Spectral Graph Partitioning [Donath-Hoffman 72, Barnes 82, Hagen-Kahng 92] S = {a : v 2 (a) t} for some Cheeger s Inequality says there is a t t so that Φ(S) 2dλ 2

65 Major topics in spectral graph theory Graph Isomorphism: determining if two graphs are the same Independent sets: large sets of vertices containing no edges Graph Coloring: so that edges connect different colors

66 Major topics in spectral graph theory Graph Isomorphism Independent sets Graph Coloring Behavior under graph transformations Random Walks and Diffusion PageRank and Hits Colin de Verdière invariant Special Graphs from groups from meshes Machine learning Image processing

67 Solving linear equations in Laplacians For energy minimization and computation of eigenvectors and eigenvalues Can do it in time nearly-linear in the number of edges in the graph! A powerful computational primitive.

68 Maximum flow problem Send as much stuff as possible from s to t. At most one unit can go through each edge. s t

69 Maximum flow problem Send as much stuff as possible from s to t. At most one unit can go through each edge. s t

70 Maximum flow problem Send as much stuff as possible from s to t. At most one unit can go through each edge. s t Standard approach: incrementally add flow paths

71 Maximum flow problem Send as much stuff as possible from s to t. At most one unit can go through each edge. s t Standard approach: incrementally add flow paths

72 Maximum flow problem Send as much stuff as possible from s to t. At most one unit can go through each edge. s t Standard approach: incrementally add flow paths

73 Maximum flow problem Send as much stuff as possible from s to t. At most one unit can go through each edge. s t Standard approach: incrementally add flow paths Issue: sometimes requires backtracking

74 Maximum flow problem, electrical approach [Christiano-Kelner-Madry-S-Teng ]. Try the electrical flow. 2.8 V.9.9 s V t

75 Maximum flow problem, electrical approach [Christiano-Kelner-Madry-S-Teng ]. Try the electrical flow. 2. Increase resistance when too much flow 2.8 V.9.9 s V t

76 Maximum flow problem, electrical approach [Christiano-Kelner-Madry-S-Teng ]. Try the electrical flow. 2. Increase resistance when too much flow 2. V.5.5 s V t

77 Solving linear equations in Laplacians For energy minimization and computation of eigenvectors and eigenvalues Can do it in time nearly-linear in the number of edges in the graph! Key ideas: how to approximate a graph by a tree or by a very sparse graph random matrix theory numerical linear algebra

78 Approximating Graphs A graph H is an -approximation of G if for all x + xt L H x x T L G x +

79 Approximating Graphs A graph H is an -approximation of G if for all x + xt L H x x T L G x + To solve linear equations quickly, approximate G by a simpler graph H

80 Approximating Graphs A graph H is an -approximation of G if for all x + xt L H x x T L G x + A very strong notion of approximation Preserves all electrical and spectral properties

81 Approximating Graphs A graph H is an -approximation of G if for all x + xt L H x x T L G x + Theorem [Batson-S-Srivastava 9] Every graph G has an -approximation H with V (2 + ) 2 / 2 edges

82 Approximating Graphs A graph H is an -approximation of G if for all x + xt L H x x T L G x + Theorem [Batson-S-Srivastava 9] Every graph G has an -approximation H with V (2 + ) 2 / 2 edges A powerful technique in linear algebra many applications

83 To learn more Lectures 2 and 3: More precision More notation Similar sophistication

84 To learn more See my lecture notes from Spectral Graph Theory and Graphs and Networks