An Algorithmist s Toolkit September 15, Lecture 2

Size: px

Start display at page:

Download "An Algorithmist s Toolkit September 15, Lecture 2"

Barbara Parrish
6 years ago
Views:

1 An Algorithmist s Toolkit September 15, 007 Lecture Lecturer: Jonathan Kelner Scribe: Mergen Nachin Administrative Details Signup online for scribing. Review of Lecture 1 All of the following are covered in detail in the notes for Lecture 1: The definition of L G, specifically that L G = D G A G, where D G is a diagonal matrix of degrees and A G is the adjacency matrix of graph G. The action of L G on a vector x, namely that [L G x] i = deg(i) (x i average of x on neighbors of i) The eigenvalues of L G are λ 1 λ n with corresponding eigenvectors v 1,..., v n. The first and the most trivial eigenvector is v 1 = 1 with an eigenvalue λ 1 = 0. We are mostly interested in v and λ. 3 Properties of the Laplacian 3.1 Some simple properties Lemma 1 (Edge Union) If G and H are two graphs on the same vertex set with disjoint edge sets, L G H = L G + L H (additivity) (1) Lemma (Isolated Vertices) If a vertex i G is isolated, then the corresponding row and column of the Laplacian are zero, i.e. [L G ] i,j = [L G ] j,i = 0 for all j. Lemma 3 (Disjoint Union) These together imply that the Laplacian of the disjoint union of G and H is direct sum of L G and L H, i.e.: ( ) LG 0 L G H = LG L H = () 0 L H Proof Consider the graph G v(h) = (V G V H, E G ), namely the graph consisting of G along with the vertex set of H as disjoint vertices. Define v(g) H similarly. By the second remark, ( LG 0 ) ( ) 0 0 L G v(h) = and Lv(G) H = LH By definition, G H = (G v(h)) (v(g) H), and so by the first remark: ( ) LG 0 L G H = L G L H =. 0 L H This implies the Laplacian is the direct sum of the Laplacians of the connected components. Thus, -1

2 Theorem 4 (Disjoint Union Spectrum) If L G has eigenvectors v 1,..., v n with eigenvalues λ 1,..., λ n, and L H has eigenvectors w 1,..., w n with eigenvalues µ 1,..., µ n, then L G H has eigenvectors: with corresponding eigenvalues: Proof v 1 0,..., v n 0, 0 w 1,..., 0 w n λ 1,..., λ n, µ 1,..., µ n. By the previous lemma, ( ) ( ) ( ) L L G H (v 1 0) = G 0 v 1 λ = 1 v 1 0 L H 0 0 Thus v 1 0 is an eigenvector of L G H with eigenvalue λ 1. The rest follow by symmetry. 3. The Laplacian of an edge Definition 5 Let L e be the Laplacian of the graph on n vertices consisting of just the edge e. Example 1 If e is the edge (v 1, v ), then L e = By additivity, this lets us write: L G = L e (3) e E This will allow us to prove a number of facts about the Laplacian by proving them for one edge and adding them up. The more general technique, which we ll use more later, is to bound Laplacians by adding matrices of substructures. So for an edge e, ( ) 1 1 L e = [zeros]. 1 1 Note that: ( ) ( ) ( ) ( ) 1 ( = 1 1 = ), and so 1 ( ) T 1 is an eigenvector with eigenvalue. This decomposition implies: ( ) ( ) x T L e x = ( ) 1 ( ) x x 1 x = (x 1 x 1 x ). (4) Remark The Laplacian is a quadratic form, specifically: x T L G x = x T ( L e )x = x T L e x = (x i x j ) (5) e E e E (i,j) E This implies that L is positive semidefinite. -

3 3.3 Review of Positive Semidefiniteness Definition 6 A symmetric matrix M is positive semidefinite (PSD) if x R n, x T Mx 0. M is positive definite (PD) if the inequality is strict x = 0. Lemma 7 M is PSD iff all eigenvalues λ i 0. Similarly M is PD iff all eigenvalues λ i > 0. Proof Let s consider the matrix M in its eigenbasis, that is M = Q T ΛQ. Clearly, y T Λy = i λ iy i 0 for all y R n iff λ i 0 for all i. Similar for PD matrix. Lemma 8 (PSD Matrix Decomposition) M is PSD iff there exists a matrix A such that M = A T A. (6) Note that A can be (n k) for any k, and that it need not be square. Importantly, note that A is not unique. Proof ( ) If M is positive semidefinite, recall that M can be diagonalized as M = Q T ΛQ, thus ( ) T ( ) M = Q T Λ 1/ Λ 1/ Q = Λ 1/ Q Λ 1/ Q, where Λ 1/ has λ i on the diagonal. ( ) If M = A T A, then Letting y = (Ax) R k, we see that: x T Mx = x T A T Ax = (Ax) T (Ax) x T Mx = y T y = y Factoring the Laplacian We know from the previous section that we can factor L as A T A using eigenvectors, but there also exists a much nicer factorization which we will show here. Definition 9 Let m be the number of edges and n be the number of vertices. Then the incidence matrix = G is the m n matrix given by: 1 if e = (v, w) and v < w e,v = 1 if e = (v, w) and v > w (7) 0 otherwise. -3

4 Example The Laplacian and the Incidence matrix of the graph G= 3 4 b 1 c a is L G = 1 = G Lemma 10 L G = T. [ ] ( ) ( ) Proof Observe that T ij = (ith column of ) (jth column of ) = e [ ] e,vi [ ] e,vj This gives three cases: When i = j, When i = j and no edge exists between v i and v j, [ ] ( ) T = [ ] = 1 = deg(i). ij e When i = j and exists an edge e between v i and v j, [ ] ( ) ( ) ( ) ( ) T ij = [ ] e,vi [ ] e,vj = [ ] e,v i [ ] e,v j = 1. e,v i e incident to v i [ ] ( ) ( ) T ij = [ ] e,vi [ ] e,vj = 0 as every edge is non-incident to at least one of v i, v j. Corollary 11 Note that This gives another proof that L is PSD. 3.5 Dimension of the Null Space e e x T L G x = x = (x i x j ), (i,j) E Theorem 1 If G is connected, the null space is 1-dimensional and spanned by the vector 1. Proof Let x null(l), i.e. L G x = 0. This implies x T L G x = (x i x j ) = 0. (i,j) E Thus, x i = x j for every (i, j) E. As G is connected, this means that all x i are equal. Thus every member of the null space is a multiple of 1. Corollary 13 If G is connected, λ > 0. Corollary 14 The dimension of the null space of L G is exactly the number of connected components of G. -4

5 4 Spectra of Some Common Graphs We compute the spectra of some graphs: Lemma 15 (Complete graph) The Laplacian for the complete graph K n on n vertices has eigenvalue 0 with multiplicity 1 and eigenvalue n with multiplicity n 1 and associated eigenspace {x x 1 = 0}. Proof By corollary 13, we conclude that eigenvalue 0 has multiplicity 1. Now, take any vector v which is orthogonal to 1 and consider [L Kn v] i. Note that this value is equal to (n 1)v i v j = nv i v j = nv i Hence any vector v which is orthogonal to 1 is an eigenvector with an eigenvalue n. j=i Lemma 16 (Ring graph) The Laplacian for the ring graph R n on n vertices has eigenvectors x k (u) = sin(πku/n), and y k (u) = cos(πku/n) for 0 k n/. Both x k and y k have eigenvalue cos(πk/n). Note that, x 0 = 0 should be ignored and y 0 is 1, and when n is even x n/ = 0 should be ignored and we only have y n/. Proof. The best way to see is to plot the graph on the circle using these vectors as coordinates. Below is the plot for a k = 3. j Just consider vertex 1. Keep in mind that sin(x) = sin(x) cos(x). Then, [Lx k ] 1 = x k (1) x k (0) x k () = sin(πk/n) 0 sin(πk/n) = sin(πk/n) sin(πk/n) cos(πk/n) = ( cos(πk/n)) sin(πk/n) = ( cos(πk/n))x k (1) Note that this shows that x(u) = R(e πi(ku+c)/n ) is an eigenvector for any k Z, c C. Lemma 17 (Path graph) The Laplacian for the path graph P n on n vertices has the same eigenvalues as R n and eigenvectors v k (u) = sin(πku/n + π/n) for 0 k < n -5

6 Proof R P We will realize P n as quotient of R n. Suppose z was an eigenvector of L Rn in which z i = z n 1 i for 0 i < n. Take the first n components of z and call this vector v. Note that for 0 < i < n: [L Pn v] i = (v i neighbors of i in P n ) = (z i neighbors of i in R n ) = (z i neighbors of i in R n ) + (z n i 1 neighbors of (n i 1) in R n ) 1 = ([L R n z] i + [L Rn z] n i 1 ) 1 = (λz i + λz n i 1 ) = λz i = λv i Now consider the case when i = 0. [L Pn v] 0 = v 0 v 1 = v 0 v 1 + v 0 = z 0 z 1 + z 0 = z 0 z 1 + z n 1 = λz 0 = λv 0 Hence, v is an eigenvector of L Pn. Now we show that such v exists, that is, there exists eigenvector z of L Rn in which z i = z n 1 i for 0 i < n. Take z, z k (u) = sin(πku/n + π/n) = sin(πku/n) cos(π/n) + cos(πku/n) sin(π/n) = x k (u) cos(π/n) + y k sin(π/n) We see that z k is in the span of x k and y k. Hence, z k is an eigenvector of L Rn cos(πk/n) by lemma 16. Check that z k satisfies z k (i) = z k (n 1 i) with an eigenvalue 4.1 Graph Products The next natural example is the grid graph, which will follow from general theory about product graphs. -6

7 Definition 18 Let G = (V, E), and H = (W, F ). The product graph G H has vertex set V W and edge set: ((v 1, w), (v, w)), (v 1, v ) E, w W and ((v, w 1 ), (v, w )), (w 1, w ) F, v V. Example 3 P n P m = G n,m. We see that the vertices of P n P m are: (v 1, w 1 ) (v 1, w ) (v 1, w m ) (v, w 1 ) (v, w ) (v, w m ) v(g n,m ) = (v n, w 1 ) (v n, w ) (v n, w m ) The vertices are written in the above layout because it makes the edges intuitive. The edges are: For a fixed w, i.e. a column in the above layout, a copy of P n. For a fixed v, i.e. a row in the above layout, a copy of P m. Thus P n P m = G n,m, which is to say the product of two path graphs is the grid graph. Theorem 19 (Graph Products) If L G has eigenvectors v 1,..., v n with eigenvalues λ 1,..., λ n, and L H has eigenvectors w 1,..., w k with eigenvalues µ 1,..., µ k, then L G H has, for all 1 i n, 1 j k, an eigenvector: z ij (v, w) = x i (v)y j (w) of eigenvalue λ i + µ j. Note importantly that eigenvalues add here, they do not multiply. Proof Let A m be the graph with m isolated vertices. We can then decompose the product as: G H = (G A k ) (A n H), i.e. the edge union of k disjoint copies of G and n disjoint copies of H, exactly as in the definition. By Lemmas 1 and 3 we have L G H = L G Ak + L An H = L G I k + I n L H Consider z ij = x i y j as above for a fixed i and j, we see that: L G H z ij = (L G I k ) (x i y j ) + (I n L H ) (x i y j ) = (λ i x i y j ) + (x i µ j y j ) = (λ i + µ j ) (x i y j ) = (λ i + µ j )z ij. Corollary 0 G n,m has eigenvectors and eigenvalues completely determined by those of P n and P m as above. 5 Why is this called the Laplacian? It turns out that the graph Laplacian is very naturally related to the continuous Laplacian. d In 1 dimension, the continuous Laplacian is dx. d f + d f In dimensions, the continuous Laplacian is f = dx dy. -7

8 5.1 Discretizing Derivatives, 1d case Consider a 1d function f : R R, which we wish to discretize at the points (..., k, k 1, k, k+1, k+,...): We approximate the first derivative at the line midpoints, df, up to scaling, by taking the differences dx between the values at the adjacent points: The discrete first derivative of f is a function on edges, and is, up to scaling Pn f, the incidence matrix d f of P n defined earlier. In order to compute the second derivative at the original points, dx, again up to scaling, we take the differences of the adjacent midpoints at the vertices: The discrete second derivative of f is thus, up to scaling, L Pn f. 5. Discretizing Derivatives, d case Here we discretize f : R R on a grid: f(k-1,k+1) f(k,k+1) f(k+1,k+1) f(k-1,k) f(k,k) f(k+1,k) f(k-1,k-1) f(k,k-1) f(k+1,k-1) To compute the discrete derivative in the x and y directions, we ll just look at a little piece. On the df df horizontal edges, we approximate dx up to scaling, and do likewise on the vertical edges with dy : -8

9 f(k,k+1)-f(k,k) f(k,k)-f(k-1,k) f(k+1,k)-f(k,k) f(k,k)-f(k,k-1) Again, the discrete derivative of f is a function on edges. When we consider the concatenation of the two discretization of the directional derivatives, we see that the discretization of the gradient, up to scaling, is Gn,m f. Finally we use this to compute the discretized Laplacian, up to scaling, and get: f(k,k+1)+f(k,k-1)+f(k+1,k)+f(k-1,k) -4 f(k,k) Thus the discretized Laplacian in two dimensions of f is L Gn,m f. 5.3 A Note on Fourier Coefficients We observed that paths and rings had eigenvectors that looked like Fourier coefficients. In the continuous case: d sin(kx + c) dx d cos(kx + c) dx = k sin(kx + c) = k cos(kx + c) Thus sin(kx + c) and cos(kx + c) are eigenfunctions of the d eigenvalue k. 6 Bounding Laplacian Eigenvalues dx operator, i.e. the 1d Laplacian, both with Lemma 1 (Sum of the eigenvalues) Given an n-vertex graph G with degrees d i, where d max = max i d i, and Laplacian L G with eigenvalues λ i, λ i = d i d max n (8) Proof i The first two expressions are both the trace, the upper bound is trivial. i Lemma (Bounds on λ and λ n ) Given λ i and d i as above, i d i λ n 1 (9) -9

10 λ n i d i n 1 (10) n Proof By the previous slide and the fact that λ 1 = 0, we get i= λ i = i d i. As λ λ n, the bounds follow immediately. 7 Bounding λ and λ max Theorem 3 (Courant-Fischer Formula) For any n n symmetric matrix A, x T Ax λ 1 = min x T Ax = min x =1 x=0 x T x x T Ax λ = min x T Ax = min (11) x =1 x=0 x T x x v 1 x v 1 x T Ax λ max = max x T Ax = max x =1 x=0 x T x Proof We consider the diagonalization A = Q T ΛQ. As seen earlier, x T Ax = (Qx) T Λ(Qx). As Q is orthogonal, we also have Qx = x. Thus it suffices to consider diagonal matrices. Moreover, all of the equalities on the right follow immediately from linearity. Thus we need to consider, for x = 1: λ 1 x 1 T Λx = (x 1 λix x x n ).... i = λ i x i = x. We compute the gradient and find: λ n [ ( )] λ i x i λ i x 3 i 3 x x T Λx = i x = λ i(x i x i ( x i ) i ) thus all extremal values occur when one x i = 1 and the rest are 0. The identities follow immediately. Corollary 4 (Rayleigh Quotient) Letting G = (V, E) be a graph with Laplacian L G, x n λ 1 = 0 v 1 = 1 x T L G x (i,j) E (x i x j ) λ = min x T = P x min x v 1 x=0 i V x i (1) x=0 x=0 x T L G x (i,j) E (x i x j ) λ max = max = max x=0 x T x x=0 i V x i The Rayleigh Quotient is a useful tool for bounding graph spectra. Whereas before we had to consider all possible vectors x, now in order to get an upper bound on λ we need only produce a vector with small Rayleigh quotient. Likewise to get a lower bound on λ max we need only to find a vector with large Rayleigh quotient. Examples next lecture! i -10

11 MIT OpenCourseWare Topics in Theoretical Computer Science: An Algorithmist's Toolkit Fall 009 For information about citing these materials or our Terms of Use, visit:

Specral Graph Theory and its Applications September 7, Lecture 2 L G1,2 = 1 1.

Specral Graph Theory and its Applications September 7, 2004 Lecturer: Daniel A. Spielman Lecture 2 2.1 The Laplacian, again We ll begin by redefining the Laplacian. First, let G 1,2 be the graph on two