ORIE 6334 Spectral Graph Theory November 8, Lecture 22

Transcription

1 ORIE 6334 Spectral Graph Theory November 8, 206 Lecture 22 Lecturer: David P. Williamson Scribe: Shijin Rajakrishnan Recap In the previous lectures, we explored the problem of finding spectral sparsifiers saw both deterministic romized algorithms that given a graph G with n vertices finds a spectral sparsifier with O ( ) n log n ɛ edges. In this lecture, we improve 2 upon this result to find linear-sized spectral sparsifiers, or more concretely, spectral sparsifiers with O ( ) n ɛ edges. 2 This is an interesting result since we noted in the past that a spectral sparsifier is a generalization of the cut sparsifier of a graph, the best results known previous to this one for cut sparsifiers were that a cut sparsifier with n log O() n/ɛ 2 edges can be found in nearly linear time []. Recall that given a graph G = (V, E), a weighted graph H = (V, E ) with weights w(i, j) is an ɛ-spectral sparsifier of G if ( ɛ)l G L H ( + ɛ)l G. In the last lecture, we showed that ( ɛ)l G w(i, j)(e i e j )(e i e j ) T ( + ɛ)l G (i,j) E if only if ( ɛ)i (i,j) E w(i, j)x ij x ij T ( + ɛ)i, where the vectors x ij = L 2 G (e i e j ), that (i,j) E x ijx ij T = I (recall that I is our continuing fudge of an identity matrix, which is actually L G L G ; for any vector v such that v T e = 0, I v = v). Our goal is, given vectors v,..., v m such that m v iv T i = I, to show that there exists some weights w i 0 a subset S [m] s.t S n ɛ 2 ( ɛ) 2 I i S w i v i v i T ( + ɛ) 2 I. 0 This lecture is derived from Spielman 205, Lecture 8, spielman/56/lect8-5.pdf. 22-

2 Definition The vectors v i are said to be in isotropic position if v i v T i = I. Note that for vectors v i in isotropic position, for any A, v T i Av i = (v i v T i ) A = ( v i v T i ) A = I A = tr(a). In this lecture, we prove a weaker version of the theorem first then mention how to extend it to the general case. The version we prove is as follows. Theorem Given vectors v,..., v m that are in isotropic postion, we can find a subset S [m], weights w i 0 such that S 6n 2 The Algorithm I w i v i v T i 3I. 3 3n i S The basic idea behind the algorithm is that we greedily pick vectors weights such that we control how the maximum minimum eigenvalues change. Algorithm : Linear sized spectral sparsifier l n; u n l /3; u 2 w i 0 i A 0 for k to 6n do Pick v i, c s.t. λ min (A + cv i v T i ) l + l λ max (A + cv i v T i ) u + u w i w i + c; T A A + cv i v i l l + l u u + u end return A Note that initially λ min (A) = λ max (A) = 0, which satisfies l = n, u = n as bounds. At the end of the algorithm, λ min (A) n + (6n) 3 = n, λ max(a) n + (6n)(2) = 3n. So we divide the matrix by 3n to get the bound as given in the theorem. 22-2

3 3 Barrier Functions The genius of the result is that rather than working with λ min λ max directly, we use barrier functions, L(l, A) U(u, A), which we now define. We let L(l, A) = U(u, A) = λ i l = tr((a li) ), u λ i = tr((ui A) ), for l λ... λ n u where the λ i are the eigenvalues of A. Initially, l 0 λ min (A) λ max (A) u 0. This implies that U(u, A) L(l, A) are both positive bounded. During the course of the algorithm, we pick vectors increase l u so that the barrier functions don t increase in value, so we can ensure that the eigenvalues lie in the range [l, u]. Initially U(u, 0) = n = L( n, 0) = n n =. n What we want in each iteration is to find a vector v a weight c such that U(u + u, A + cvv T ) U(u, A) L(l + l, A + cvv T ) L(l, A), so that the barrier functions do not increase as we update A, u, l through the run of the algorithm. 4 The Analysis We now turn towards proving that such a selection of v c can be made in each iteration of the loop. We will show that there exists matrices U A L A such that the following two lemmas hold. Lemma 2 The barrier functions do not increase in an iteration; that is, U(u + u, A + cvv T ) U(u, A) holds for c, v, if L(l + l, A + cvv T ) L(l, A) v T U A v c vt L A v. In the original paper due to Batson, Spielman, Srivastava, as well as other notes on their result, the notation Φ u (A) is used for U(a, A) Φ l (A) is used for L(l, A). 22-3

4 Lemma 3 v T i U A v i u + U(u, A) m l /L(l, A) l v T i L A v i. From the choice of our parameters, note that we get u + U(u, A) 2 + = 3 2, l /L(l, A) l 3 /3 = 3 2. From Lemma 3, we see that this implies vi T U A v i 3 m 2 vi T L A v i, therefore, there exists some c, v i such that v i T U A v i c v i T L A v i. Then Lemma 2 ensures that there exists a vector v i weight c so that we can add cv i vi T to A without increasing the barrier functions. The only remaining part is to prove the two lemmata. To prove Lemma 2, we first analyze what the addition of the vector cvv T does to the matrix A, by using the following formula. Theorem 4 (Sherman-Morrison formula) For a nonsingular symmetric matrix X a vector v (X vv T ) = X + X vv T X v T X v. The formula expresses a rank- update to the inverse of a matrix. Proof of Lemma 2: On adding the matrix cvv T to A, the barrier function changes to U(u, A + cvv T ) = tr((ui (A + cvv T )) ) = tr((ui A cvv T ) ) = tr((ui A) ) + c tr((ui A) vv T (ui A) ) cv T (ui A) v = U(u, A) + c tr((ui A) vv T (ui A) ) cv T (ui A) v = U(u, A) + c tr(vt (ui A) 2 v) cv T (ui A) v v T (ui A) 2 v = U(u, A) + c cv T (ui A) v 22-4

5 The third inequality follows from the Sherman-Morrison formula, the fourth by the definition of U(u, A), the fifth by the cyclic property of the trace. So as we add cvv T, the barrier function increases. To counteract this, we increase the value of u so to keep the barrier function constant. Let û u + u. Then we want to see under what values of c v the barrier function doesn t increase. So then U(û, A + cvv T v T (ûi A) 2 v ) = U(û, A) + c cv T (ûi A) v v T (ûi A) 2 v = U(u, A) (U(u, A) U(û, A)) + c cv T (ûi A) v We want U(û, A + cvv T ) U(u, A), which will be true if v T (ûi A) 2 v U(u, A) U(û, A) c cv T (ûi A) v, which holds if c vt (ûi A) 2 v U(u, A) U(û, A) + vt (ûi A ) v = v T U A v, where we define U A (ûi A) 2 U(u, A) U(û, A) + (ûi A). This proves one half of the Lemma. The other half can be proved similarly, but with ˆl = l + l, 2 (A ˆlI) L A = L(ˆl, A) L(l, A) (A ˆlI). We now turn to the proof of the second Lemma. Proof of Lemma 3, upper bound: Notice that if u > λ max (A), δ > 0, then the barrier function is decreasing in u, ie, U(u + δ, A) < U(u, A). Now, since the vectors v i are in isotropic position, v T i U A v i = tr(u A ) = tr((ûi A) 2 ) U(u, A) U(û, A) + tr((ûi A) ) tr((ûi A) 2 ) + U(u, A). U(u, A) U(û, A) 22-5

6 To bound the first term, we note that d d U(u, A) = du du u λ i = d 2 du U(u, A) = 2 2 (u λ i ) > 0, 3 thus U(u, A) is decreasing convex in u. In particular, convexity implies that which yields leading to (u λ i ) 2 = tr((ui A) 2 ), U(u, A) U(û, A) ( u) d du U(û, A) = u tr((ûi A) 2 ), tr((ui A) 2 ) U(u, A) U(û, A) u, tr((ûi A) 2 ) U(u, A) U(û, A) u. We can prove the lower bound for Lemma 3 in a similar way. Proof of Lemma 3, lower bound: In this case, we have Now, v T i L A v i = tr(l A ) d dl L(l, A) = d dl = tr((a ˆlI) 2 ) L(ˆl, A) L(l, A) tr((a ˆlI) ) = tr((a ˆlI) 2 ) L(ˆl, A) L(l, A) L(ˆl, A). λ i l = (λ i l) 2 = tr((a li) 2 ), d 2 dl L(l, A) = 2 2 (λ i l) > 0, 3 thus L(l, A) is increasing convex in l. Then by convexity, L(l, A) L(ˆl, A) l d dl L(ˆl, A) l tr((a ˆlI) 2 ). 22-6

7 Rearranging terms, we get tr((a ˆlI) 2 ) L(ˆl, A) L(l, A) l. To bound the second term, we claim that L(ˆl, A) L(l, A) l L(l, A)L(ˆl, A). If the claim is true, then by rearranging terms we get L(ˆl, A) /L(l, A) l, as desired. To prove the claim we observe that L(ˆl, A) L(l, A) = [ l l λ i ˆl ] λ i l [ = λ i l (λ i ˆl) ] l (λ i l)(λ i ˆl) [ ] = l l (λ i l)(λ i ˆl) [ ] = (λ i l)(λ i ˆl) [ ] [ ] λ i l λ i ˆl = L(l, A)L(ˆl, A), as long as l λ min, as we have been guaranteeing. To get a stronger (more general) result, we change the parameters u, l, then prove instead of tr(l A ) L(l, A) l tr(l A ) l /L(l, A) l. The proof of this is messier involves more algebra, but is not really bad. Having seen the correctness of the algorithm, we turn towards the question of its runtime. The general usage of a spectral sparsifier is to reduce the dependence of the runtime of algorithms that depend on the number of edges by creating a sparse approximation. The algorithm described runs far too slowly to realize gains using a sparsifier for cut algorithms (for instance). The faster algorithm due to Lee Sun 205, constructs a sparsifier with O( qn ɛ 2 ) edges in Õ(qmn 2 ) time. 22-7

8 References [] Wai Shing Fung, Ramesh Hariharan, Nicholas JA Harvey, Debmalya Panigrahi, A General Framework for Graph Sparsification. Proceedings of the Forty- Third Annual ACM Symposium on the Theory of Computing, pp 7-80,