# U.C. Berkeley CS294: Spectral Methods and Expanders Handout 11 Luca Trevisan February 29, 2016

Size: px
Start display at page:

Download "U.C. Berkeley CS294: Spectral Methods and Expanders Handout 11 Luca Trevisan February 29, 2016"

## Transcription

1 U.C. Berkeley CS294: Spectral Methods and Expanders Handout Luca Trevisan February 29, 206 Lecture : ARV In which we introduce semi-definite programming and a semi-definite programming relaxation of sparsest cut, and we reduce its analysis to a key lemma that we will prove in the next lecture(s) The Goemans-Linial Relaxation Recall that, for two undirected graphs G, H, the sparsest cut problem is to optimize sc(g, H) := min S V {} E G S (u) S (v) {} E H S (u) S (v) and the Leighton-Rao relaxation is obtained by noting that if we define := S (u) S (v) then d(, ) is a semimetric over V, meaning that the following quantity is a relaxation of (G, H): LR(G, H) = min d : V V R d semimetric {} E G {} E H If G is r-regular, H is a clique, and 0 = λ λ 2 λ n are the eigenvalues of the normalized Laplacian of G, then r n λ 2 = min f:v R {} E G f(u) f(v) 2 {} E Kn f(u) f(v) 2 () which is a relaxation of sc(g, K n ), because, for every S, every u and every v, S (u) S (v) = S (u) S (v) 2. We note that if we further relax () by allowing V to be mapped into a higher dimension space R m instead of R, and we replace by 2, the optimum remains the same.

2 Fact λ 2 = inf m,f :V R m {} E F (u) F (v) 2 G {} E Kn F (u) F (v) 2 Proof: For every F : V R m, if we write F (v) = (f (v),..., f n (v)), we have {} E G F (u) F (v) 2 {} E F (u) F (v) 2 Kn i {} E (f i (u) f i (v)) 2 = G i {} E Kn (f i (u) f i (v)) 2 {} E (f i (u) f i (v)) 2 min G i=,...,m {} E (f i (u) f i (v)) 2 Kn λ 2 The above observations give the following comparison between the Leighton-Rao relaxation and the spectral relaxation: both are obtained by replacing S (u) S (v) with a distance function ; in the Leighton-Rao relaxation, is constrained to satisfy the triangle inequality; in the spectral relaxation, is constrained to be the square of the Euclidean distance between F (u) and F (v) for some mapping F : V R m. The Arora-Rao-Vazirani relaxation is obtained by enforcing both conditions, that is, by considering distance functions that satisfy the triangle inequality and can be realized of F (u) F (v) 2 for some mapping F : V R m. Definition 2 A semimetric d : V V R is called of negative type if there is a dimension m and a mapping F : V R m such that = F (u) F (v) 2 for every u, v V. With the above definition, we can formulate the Goemans-Linial relaxation as ARV (G, H) := min d : V V R d semimetric of negative type {} E G {} E H (2) Remark 3 The relaxation (2) was first proposed by Goemans and Linial. Arora, Rao and Vazirani were the first to prove that it achieves an approximation guarantee which is better than the approximation guarantee of the Leighton-Rao relaxation. 2

3 We have, by definition, sc(g, H) ARV (G, H) LR(G, H) and, when H is a clique and G is r-regular, sc(g, K n ) ARV (G, K n ) r n λ 2(G) and so the approximation results that we have proved for λ 2 and LR apply to ARV. For every graphs G and H: and for every r-regular graph G ARV (G, H) O(log V ) sc(g, H) n r ARV (G, K n) 8 n r sc(g, K n) Interestingly, known examples of graphs for which LR and λ 2 give poor approximation are complementary. When H is a clique, if G is a cycle, then r n λ 2 is a poor approximation of sc(g, K n ), but LR(G, K n ) is a good approximation of sc(g, K n ); if G is a constant-degree expander then LR(G, K n ) is a poor approximation of sc(g, K n ), but r n λ 2 is a good approximation. When Goemans and Linial (separately) proposed to study the relaxation (2), they conjectured that it would always provide a constant-factor approximation of sc(g, H). Unfortunately, the conjecture turned out to be false, but Arora, Rao and Vazirani were able to prove that (2) does provide a strictly better approximation than the Leighton-Rao relaxation. In the next lectures, we will present parts of the proof of the following results. Theorem 4 There is a constant c such that, for every graph G = (V, E), sc(g, K n ) c log V ARV (G, K n ) Theorem 5 There is a constant c such that, for every graphs G = (V, E G ), H = (V, E H ), sc(g, H) c log V log log V ARV (G, H) Theorem 6 There is a constant c and an infinite family of graphs G n = (V n, E n ) such that sc(g n, K n ) c log log V n ARV (G n, K n ) 3

4 Theorem 7 There are families of graphs G n = (V n, EG n ) and H n (V n, EH n ) such that, for every ɛ > 0 and every sufficiently larger n, ARV (G n, K n ) (log V ) 2 ɛ sc(g n, K n ) 2 Polynomial Time Solvability In this section we show that the Ellipsoid algorithm can compute ARV (G, H) in polynomial time. Definition 8 If C R m is a set, then a separation oracle for C is a procedure that, on input x R m, If x C, outputs yes If x C, outputs coefficients a,..., a m, b such that x i a i < b but, for every z C, i z i a i b i Note that a set can have a separation oracle only if it is convex. Under certain additional mild conditions, if C has a polynomial time computable separation oracle, then the optimization problem minimize subject to i ct x Ax b x C is solvable in polynomial time using the Ellipsoid Algorithm. It remains to see how to put the Arora-Rao-Vazirani relaxation into the above form. Recall that a matrix X R n n is positive semidefinite if all its eigenvalues are nonnegative. We will use the set of all n n positive semidefinite matrices as our set C (thinking of them as n 2 -dimensional vectors). If we think of two matrices M, M R n n as n 2 -dimensional vectors, then their inner product is M M := i,j M i,j M i,j 4

5 Lemma 9 The set of n n positive semidefinite matrices has a separation oracle computable in time polynomial in n. Proof: Given a symmetric matrix X, its smallest eigenvalue is min z R n, z = zt Xz the vector achieving the minimum is a corresponding eigenvector, and both the smallest eigenvalue and the corresponding eigenvector can be computed in polynomial time. If we find that the smallest eigenvalue of X is non-negative, then we answer yes. Otherwise, if z is an eigenvector of the smallest eigenvalue we output the matrix A = z T z. We see that we have A X = z T Xz < 0 but that, for every positive semidefinite matrix M, we have A M = z T Mz 0 This implies that any optimization problem of the following form can be solved in polynomial time minimize subject to C X A X b A m X b m X 0 (3) where C, A,..., A m are square matrices of coefficients, b,..., b m are scalars, and X is a square matrix of variables. An optimization problem like the one above is called a semidefinite program. It remains to see how to cast the Arora-Rao-Vazirani relaxation as a semidefinite program. Lemma 0 For a symmetric matrix M R n n, the following properties are equivalent:. M is positive semidefinite; 5

6 2. there are vectors x,..., x n R d such that, for all i, j, M i,j = x i, x j ; 3. for every vector z R n, z T Mz 0 Proof: That () and (3) are equivalent follows from the characterization of the smallest eigenvalue of M as the minimum of z T Mz over all unit vectors z. To see that (2) (3), suppose that vectors x,..., x n exist as asserted in (2), and let X be the matrix whose columns are the vectors x,..., x n, so that X T X = M. Take any vector z, and see that z T Mz = z T X T Xz = Xz 2 0 Finally, to see that () (2), let λ,..., λ n be the eigenvalues of M with multiplicities, and let v,..., v n be a corresponding orthonormal set of eigenvectors. Then M = i λ k v k v T k that is, M i,j = k λ k v k (i)v k (j) = x i, x j if we define x,..., x n as the vectors such that x i (k) := λ k v k (i). This means that the generic semidefinite program (4) can be rewritten as an optimization problem in which the variables are the vectors x,..., x n as in part (2) of the above lemma. minimize subject to i,j C i,j x i, x j i,j A i,j x i, x j b i,j Am i,j x i, x j b m x i R m i {,..., n} where the dimension m is itself a variable (although one could fix it, without loss of generality, to be equal to n). In this view, a semidefinite program is an optimization problem in which we wish to select n vectors such that their pairwise inner products satisfy certain linear inequalities, while optimizing a cost function that is linear in their pairwise inner product. The square of the Euclidean distance between two vectors is a linear function of inner products 6 (4)

7 x y 2 = x y, x y = x, x 2 x, y + y, y and so, in a semidefinite program, we can include expressions that are linear in the pairwise squared distances (or squared norms) of the vectors. The ARV relaxation can be written as follows minimize {} E G x u x v 2 subject to x {} EH u x v 2 = x u x v 2 x u x w 2 + x w x v 2 x u R d u, v, w V u V and so it is a semidefinite program, and it can be solved in polynomial time. Remark Our discussion of polynomial time solvability glossed over important issues about numerical precision. To run the Ellipsoid Algorithm one needs, besides the separation oracle, to be given a ball that is entirely contained in the set of feasible solutions and a ball that entirely contains the set of feasible solutions, and the running time of the algorithm is polynomial in the size of the input, polylogarithmic in the ratio of the volumes of the two balls, and polylogarithmic in the desired amount of precision. At the end, one doesn t get an optimal solution, which might not have a finite-precision exact representation, but an approximation within the desired precision. The algorithm is able to tolerate a bounded amount of imprecision in the separation oracle, which is an important feature because we do not have exact algorithms to compute eigenvalues and eigenvectors (the entries in the eigenvector might not have a finite-precision representation). The Ellipsoid algorithm is typically not a practical algorithm. Algorithms based on the interior point method have been adapted to semidefinite programming, and run both in worst-case polynomial time and in reasonable time in practice. Arora and Kale have developed an Õ(( V + E )2 /ɛ O() ) time algorithm to solve the ARV relaxation within a multiplicative error ( + ɛ). The dependence on the error is worse than that of generic algorithms, which achieve polylogarithmic dependency, but this is not a problem in this application, because we are going to lose an O( log V ) factor in the rounding, so an extra constant factor coming from an approximate solution of the relaxation is a low-order consideration. 3 Rounding when H is a clique Given the equivalence between the sparsest cut problem and the L relaxation of sparsest cut, it will be enough to prove the following result. 7

8 Theorem 2 (Rounding of ARV) Let G = (V, E) be a graph, and {x v } v V feasible solution of the relaxation ARV (G, K n ). Then there is a mapping f : V R such that {} E f(u) f(v) {} f(u) f(v) O( {} E log V ) x u x v 2 {} x u x v 2 be a As in the rounding of the Leighton-Rao relaxation via Bourgain s theorem, we will identify a set S V, and define f S (v) := min s S x s x v 2 (5) Recall that, as we saw in the proof of Bourgain s embedding theorem, no matter how we choose the set S we have f S (u) f S (v) x u x v 2 (6) where we are not using any facts about 2 other than the fact that, for solutions of the ARV relaxation, it is a distance function that obeys the triangle inequality. This means that, in order to prove the theorem, we just have to find a set S V such that f S (u) f S (v) O( log V ) x u x v 2 (7) and this is a considerable simplification because the above expression is completely independent of the graph! The remaining problem is purely one about geometry. Recall that if we have a set of vectors {x v } v V such that the distance function := x u x v 2 satisfies the triangle inequality, then we say that d(, ) is a (semi-)metric of negative type. After these preliminaries observations, our goal is to prove the following theorem. Theorem 3 (Rounding of ARV Revisited) If d(, ) is a semimetric of negative type over a set V, then there is a set S such that if we define f S (v) := min{d(s, v)} s S we have f S (u) f S (v) O( log V ) 8

9 Furthermore, the set S can be found in randomized polynomial time with high probability given a set of vector {x v } v V such that = x u x v 2. Since the statement is scale-invariant, we can restrict ourselves, with no loss of generality, to the case = V 2. Remark 4 Let us discuss some intuition before continuing with the proof. As our experience proving Bourgain s embedding theorem shows us, it is rather difficult to pick sets such that f S (u) f S (v) is not much smaller than. Here we have a somewhat simpler case to solve because we are not trying to preserve all distances, but only the average pairwise distance. A simple observation is that if we find a set S which contains Ω( V ) elements and such that Ω( V ) elements of V are at distance Ω(δ) from S, then we immediately get f S(u) f S (v) Ω(δ V 2 ), because there will be Ω( V 2 ) pairs u, v such that f S (u) = 0 and f S (v) δ. In particular, if we could find such a set with δ = /O( log V ) then we would be done. Unfortunately this is too much to ask for in general, because we always have f S (u) f S (v), which means that if we want f S(u) f S (v) to have Ω(V 2 ) noticeably large terms we must also have that is noticeably large for Ω( V 2 ) pairs of points, which is not always true. There is, however, the following argument, which goes back to Leighton and Rao: either there are Ω( V ) points concentrated in a ball whose radius is a quarter (say) of the average pairwise distance, and then we can use that ball to get an l mapping with only constant error; or there are Ω( V ) points in a ball of radius twice the average pairwise distance, such that the pairwise distances of the points in the ball account for a constant fraction of all pairwise distances. In particular, the sum of pairwise distances includes Ω( V 2 ) terms which are Ω(). After we do this reduction and some scaling, we are left with the task of proving the following theorem: suppose we are given an n-point negative type metric in which the points are contained in a ball of radius and are such that the sum of pairwise distances is Ω(n 2 ); then there is a subset S of size Ω(n) such that there are Ω(n) points whose distance from the set is /O( log n). This theorem is the main result of the Arora-Rao- Vazirani paper. (Strictly speaking, this form of the theorem was proved later by Lee Arora, Rao and Vazirani had a slightly weaker formulation.) We begin by considering the case in which a constant fraction of the points are concentrated in a small ball. Definition 5 (Ball) For a point z V and a radius r > 0, the ball of radius r and center z is the set B(z, r) := {v : d(z, v) r} 9

10 Lemma 6 For every vertex z, if we define S := B(z, /4), then f S (u) f S (v) S 2 V Proof: Our first calculation is to show that the typical value of f S (u) is rather large. We note that for every two vertices u and v, if we call a a closest vertex in S to u, and b a closest vertex to v in S, we have d(u, a) + d(a, z) + d(z, b) + d(b, v) and so f S (u) + f S (v) + 2 V 2 = 2 V f S (v) + v V 2 2 that is, v f S (v) V 2 Now we can get a lower bound to the sum of l distances given by the embedding f S ( ). f S (u) f S (v) u S,v V f S (v) = S v f S (v) S V 2 This means that if there is a vertex z such that B(z, /4) = Ω( V ), or even B(z, /4) = Ω( V / log V ), then we are done. Otherwise, we will find a set of Ω( V ) vertices such that their average pairwise distances are within a constant factor of their maximum pairwise distances, and then we 0

11 will work on finding an embedding for such a set of points. (The condition that the average distance is a constant fraction of the maximal distance will be very helpful in subsequent calculations.) Lemma 7 Suppose that for every vertex z we have B(z, /4) V /4. Then there is a vertex w such that, if we set S = B(w, 2), we have S 2 V S 8 S 2 Proof: Let w be a vertex that maximizes B(w, 2) ; then B(w, 2) V /2, because if we had B(u, 2) < V /2 for every vertex u, then we would have > u 2 ( V B(u, 2) ) > V 2 Regarding the sum of pairwise distances of elements of S, we have S > u S ( S B(u, /4) ) S 4 4 S 2 The proof of the main theorem now reduces to proving the following geometric fact. Lemma 8 (ARV Main Lemma) Let d be a negative-type metric over a set V such that the points are contained in a unit ball and have constant average distance, that is, there is a vertex z such that d(v, z) for every v V V c V 2 Then there are sets S, T V such that S, T Ω( V ); for every u S and every v S, /O( log V ) where the multiplicative factors hidden in the O( ) and Ω( ) notations depend only on c.

12 Indeed, applying the ARV Main Lemma to tells us that there are subsets 2 S, T of B(z, 2), both of size Ω( B(z, 2) ) = Ω(n) such that /O( log n) for every u S and v T. If we consider the Frechet embedding f S, we have f S (u) f S (v) {} = S T u S,v T O( log n) n 2 O( log n) O( log n) It remains the prove the ARV Main Lemma. {} f S (u) f S (v) 2

### U.C. Berkeley CS294: Beyond Worst-Case Analysis Handout 8 Luca Trevisan September 19, 2017

U.C. Berkeley CS294: Beyond Worst-Case Analysis Handout 8 Luca Trevisan September 19, 2017 Scribed by Luowen Qian Lecture 8 In which we use spectral techniques to find certificates of unsatisfiability

### Topic: Balanced Cut, Sparsest Cut, and Metric Embeddings Date: 3/21/2007

CS880: Approximations Algorithms Scribe: Tom Watson Lecturer: Shuchi Chawla Topic: Balanced Cut, Sparsest Cut, and Metric Embeddings Date: 3/21/2007 In the last lecture, we described an O(log k log D)-approximation

### U.C. Berkeley Better-than-Worst-Case Analysis Handout 3 Luca Trevisan May 24, 2018

U.C. Berkeley Better-than-Worst-Case Analysis Handout 3 Luca Trevisan May 24, 2018 Lecture 3 In which we show how to find a planted clique in a random graph. 1 Finding a Planted Clique We will analyze

### SDP Relaxations for MAXCUT

SDP Relaxations for MAXCUT from Random Hyperplanes to Sum-of-Squares Certificates CATS @ UMD March 3, 2017 Ahmed Abdelkader MAXCUT SDP SOS March 3, 2017 1 / 27 Overview 1 MAXCUT, Hardness and UGC 2 LP

### Positive Semi-definite programing and applications for approximation

Combinatorial Optimization 1 Positive Semi-definite programing and applications for approximation Guy Kortsarz Combinatorial Optimization 2 Positive Sem-Definite (PSD) matrices, a definition Note that

### Lecture: Expanders, in theory and in practice (2 of 2)

Stat260/CS294: Spectral Graph Methods Lecture 9-02/19/2015 Lecture: Expanders, in theory and in practice (2 of 2) Lecturer: Michael Mahoney Scribe: Michael Mahoney Warning: these notes are still very rough.

### Lecture 17: Cheeger s Inequality and the Sparsest Cut Problem

Recent Advances in Approximation Algorithms Spring 2015 Lecture 17: Cheeger s Inequality and the Sparsest Cut Problem Lecturer: Shayan Oveis Gharan June 1st Disclaimer: These notes have not been subjected

### ORIE 6334 Spectral Graph Theory November 22, Lecture 25

ORIE 64 Spectral Graph Theory November 22, 206 Lecture 25 Lecturer: David P. Williamson Scribe: Pu Yang In the remaining three lectures, we will cover a prominent result by Arora, Rao, and Vazirani for

### Graph Partitioning Algorithms and Laplacian Eigenvalues

Graph Partitioning Algorithms and Laplacian Eigenvalues Luca Trevisan Stanford Based on work with Tsz Chiu Kwok, Lap Chi Lau, James Lee, Yin Tat Lee, and Shayan Oveis Gharan spectral graph theory Use linear

### CSC Linear Programming and Combinatorial Optimization Lecture 10: Semidefinite Programming

CSC2411 - Linear Programming and Combinatorial Optimization Lecture 10: Semidefinite Programming Notes taken by Mike Jamieson March 28, 2005 Summary: In this lecture, we introduce semidefinite programming

### U.C. Berkeley CS294: Beyond Worst-Case Analysis Handout 2 Luca Trevisan August 29, 2017

U.C. Berkeley CS94: Beyond Worst-Case Analysis Handout Luca Trevisan August 9, 07 Scribe: Mahshid Montazer Lecture In this lecture, we study the Max Cut problem in random graphs. We compute the probable

### A Linear Round Lower Bound for Lovasz-Schrijver SDP Relaxations of Vertex Cover

A Linear Round Lower Bound for Lovasz-Schrijver SDP Relaxations of Vertex Cover Grant Schoenebeck Luca Trevisan Madhur Tulsiani Abstract We study semidefinite programming relaxations of Vertex Cover arising

### In English, this means that if we travel on a straight line between any two points in C, then we never leave C.

Convex sets In this section, we will be introduced to some of the mathematical fundamentals of convex sets. In order to motivate some of the definitions, we will look at the closest point problem from

### Partitioning Algorithms that Combine Spectral and Flow Methods

CS369M: Algorithms for Modern Massive Data Set Analysis Lecture 15-11/11/2009 Partitioning Algorithms that Combine Spectral and Flow Methods Lecturer: Michael Mahoney Scribes: Kshipra Bhawalkar and Deyan

### DS-GA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra.

DS-GA 1002 Lecture notes 0 Fall 2016 Linear Algebra These notes provide a review of basic concepts in linear algebra. 1 Vector spaces You are no doubt familiar with vectors in R 2 or R 3, i.e. [ ] 1.1

### Lecture 14: Random Walks, Local Graph Clustering, Linear Programming

CSE 521: Design and Analysis of Algorithms I Winter 2017 Lecture 14: Random Walks, Local Graph Clustering, Linear Programming Lecturer: Shayan Oveis Gharan 3/01/17 Scribe: Laura Vonessen Disclaimer: These

### CS 6820 Fall 2014 Lectures, October 3-20, 2014

Analysis of Algorithms Linear Programming Notes CS 6820 Fall 2014 Lectures, October 3-20, 2014 1 Linear programming The linear programming (LP) problem is the following optimization problem. We are given

### Introduction to Semidefinite Programming I: Basic properties a

Introduction to Semidefinite Programming I: Basic properties and variations on the Goemans-Williamson approximation algorithm for max-cut MFO seminar on Semidefinite Programming May 30, 2010 Semidefinite

### 8 Approximation Algorithms and Max-Cut

8 Approximation Algorithms and Max-Cut 8. The Max-Cut problem Unless the widely believed P N P conjecture is false, there is no polynomial algorithm that can solve all instances of an NP-hard problem.

### Notes for Lecture 14

U.C. Berkeley CS278: Computational Complexity Handout N4 Professor Luca Trevisan 3/3/2008 Notes for Lecture 4 In the last lecture, we saw how a series of alternated squaring and zig-zag products can turn

### ORIE 6334 Spectral Graph Theory December 1, Lecture 27 Remix

ORIE 6334 Spectral Graph Theory December, 06 Lecturer: David P. Williamson Lecture 7 Remix Scribe: Qinru Shi Note: This is an altered version of the lecture I actually gave, which followed the structure

### 1 Review: symmetric matrices, their eigenvalues and eigenvectors

Cornell University, Fall 2012 Lecture notes on spectral methods in algorithm design CS 6820: Algorithms Studying the eigenvalues and eigenvectors of matrices has powerful consequences for at least three

### Lecture 5. Max-cut, Expansion and Grothendieck s Inequality

CS369H: Hierarchies of Integer Programming Relaxations Spring 2016-2017 Lecture 5. Max-cut, Expansion and Grothendieck s Inequality Professor Moses Charikar Scribes: Kiran Shiragur Overview Here we derive

### A better approximation ratio for the Vertex Cover problem

A better approximation ratio for the Vertex Cover problem George Karakostas Dept. of Computing and Software McMaster University October 5, 004 Abstract We reduce the approximation factor for Vertex Cover

### arxiv:cs/ v1 [cs.cc] 7 Sep 2006

Approximation Algorithms for the Bipartite Multi-cut problem arxiv:cs/0609031v1 [cs.cc] 7 Sep 2006 Sreyash Kenkre Sundar Vishwanathan Department Of Computer Science & Engineering, IIT Bombay, Powai-00076,

### Grothendieck s Inequality

Grothendieck s Inequality Leqi Zhu 1 Introduction Let A = (A ij ) R m n be an m n matrix. Then A defines a linear operator between normed spaces (R m, p ) and (R n, q ), for 1 p, q. The (p q)-norm of A

### Lecture 3: Semidefinite Programming

Lecture 3: Semidefinite Programming Lecture Outline Part I: Semidefinite programming, examples, canonical form, and duality Part II: Strong Duality Failure Examples Part III: Conditions for strong duality

### U.C. Berkeley CS294: Beyond Worst-Case Analysis Handout 12 Luca Trevisan October 3, 2017

U.C. Berkeley CS94: Beyond Worst-Case Analysis Handout 1 Luca Trevisan October 3, 017 Scribed by Maxim Rabinovich Lecture 1 In which we begin to prove that the SDP relaxation exactly recovers communities

### 1 Matrix notation and preliminaries from spectral graph theory

Graph clustering (or community detection or graph partitioning) is one of the most studied problems in network analysis. One reason for this is that there are a variety of ways to define a cluster or community.

### Cuts & Metrics: Multiway Cut

Cuts & Metrics: Multiway Cut Barna Saha April 21, 2015 Cuts and Metrics Multiway Cut Probabilistic Approximation of Metrics by Tree Metric Cuts & Metrics Metric Space: A metric (V, d) on a set of vertices

### Lecture 8 : Eigenvalues and Eigenvectors

CPS290: Algorithmic Foundations of Data Science February 24, 2017 Lecture 8 : Eigenvalues and Eigenvectors Lecturer: Kamesh Munagala Scribe: Kamesh Munagala Hermitian Matrices It is simpler to begin with

### A Unified Theorem on SDP Rank Reduction. yyye

SDP Rank Reduction Yinyu Ye, EURO XXII 1 A Unified Theorem on SDP Rank Reduction Yinyu Ye Department of Management Science and Engineering and Institute of Computational and Mathematical Engineering Stanford

Chapter 11 Approximation Algorithms Slides by Kevin Wayne. Copyright @ 2005 Pearson-Addison Wesley. All rights reserved. 1 Approximation Algorithms Q. Suppose I need to solve an NP-hard problem. What should

### Doubling metric spaces and embeddings. Assaf Naor

Doubling metric spaces and embeddings Assaf Naor Our approach to general metric spaces bears the undeniable imprint of early exposure to Euclidean geometry. We just love spaces sharing a common feature

### UC Berkeley CS 170: Efficient Algorithms and Intractable Problems Handout 22 Lecturer: David Wagner April 24, Notes 22 for CS 170

UC Berkeley CS 170: Efficient Algorithms and Intractable Problems Handout 22 Lecturer: David Wagner April 24, 2003 Notes 22 for CS 170 1 NP-completeness of Circuit-SAT We will prove that the circuit satisfiability

### Notes taken by Costis Georgiou revised by Hamed Hatami

CSC414 - Metric Embeddings Lecture 6: Reductions that preserve volumes and distance to affine spaces & Lower bound techniques for distortion when embedding into l Notes taken by Costis Georgiou revised

### Lecture 18: March 15

CS71 Randomness & Computation Spring 018 Instructor: Alistair Sinclair Lecture 18: March 15 Disclaimer: These notes have not been subjected to the usual scrutiny accorded to formal publications. They may

### Lecture 8: Spectral Graph Theory III

A Theorist s Toolkit (CMU 18-859T, Fall 13) Lecture 8: Spectral Graph Theory III October, 13 Lecturer: Ryan O Donnell Scribe: Jeremy Karp 1 Recap Last time we showed that every function f : V R is uniquely

### 1 Primals and Duals: Zero Sum Games

CS 124 Section #11 Zero Sum Games; NP Completeness 4/15/17 1 Primals and Duals: Zero Sum Games We can represent various situations of conflict in life in terms of matrix games. For example, the game shown

### Unique Games and Small Set Expansion

Proof, beliefs, and algorithms through the lens of sum-of-squares 1 Unique Games and Small Set Expansion The Unique Games Conjecture (UGC) (Khot [2002]) states that for every ɛ > 0 there is some finite

### Notes for Lecture 2. Statement of the PCP Theorem and Constraint Satisfaction

U.C. Berkeley Handout N2 CS294: PCP and Hardness of Approximation January 23, 2006 Professor Luca Trevisan Scribe: Luca Trevisan Notes for Lecture 2 These notes are based on my survey paper [5]. L.T. Statement

### Geometric Constraints II

Geometric Constraints II Realizability, Rigidity and Related theorems. Embeddability of Metric Spaces Section 1 Given the matrix D d i,j 1 i,j n corresponding to a metric space, give conditions under which

### Finite Metric Spaces & Their Embeddings: Introduction and Basic Tools

Finite Metric Spaces & Their Embeddings: Introduction and Basic Tools Manor Mendel, CMI, Caltech 1 Finite Metric Spaces Definition of (semi) metric. (M, ρ): M a (finite) set of points. ρ a distance function

### On the Power of Robust Solutions in Two-Stage Stochastic and Adaptive Optimization Problems

MATHEMATICS OF OPERATIONS RESEARCH Vol. xx, No. x, Xxxxxxx 00x, pp. xxx xxx ISSN 0364-765X EISSN 156-5471 0x xx0x 0xxx informs DOI 10.187/moor.xxxx.xxxx c 00x INFORMS On the Power of Robust Solutions in

### 1 Matrix notation and preliminaries from spectral graph theory

Graph clustering (or community detection or graph partitioning) is one of the most studied problems in network analysis. One reason for this is that there are a variety of ways to define a cluster or community.

### Functional Analysis Review

Outline 9.520: Statistical Learning Theory and Applications February 8, 2010 Outline 1 2 3 4 Vector Space Outline A vector space is a set V with binary operations +: V V V and : R V V such that for all

### Lecture 7: Positive Semidefinite Matrices

Lecture 7: Positive Semidefinite Matrices Rajat Mittal IIT Kanpur The main aim of this lecture note is to prepare your background for semidefinite programming. We have already seen some linear algebra.

### Assignment 1: From the Definition of Convexity to Helley Theorem

Assignment 1: From the Definition of Convexity to Helley Theorem Exercise 1 Mark in the following list the sets which are convex: 1. {x R 2 : x 1 + i 2 x 2 1, i = 1,..., 10} 2. {x R 2 : x 2 1 + 2ix 1x

### On a Cut-Matching Game for the Sparsest Cut Problem

On a Cut-Matching Game for the Sparsest Cut Problem Rohit Khandekar Subhash A. Khot Lorenzo Orecchia Nisheeth K. Vishnoi Electrical Engineering and Computer Sciences University of California at Berkeley

### 25.1 Markov Chain Monte Carlo (MCMC)

CS880: Approximations Algorithms Scribe: Dave Andrzejewski Lecturer: Shuchi Chawla Topic: Approx counting/sampling, MCMC methods Date: 4/4/07 The previous lecture showed that, for self-reducible problems,

### Stanford University CS366: Graph Partitioning and Expanders Handout 13 Luca Trevisan March 4, 2013

Stanford University CS366: Graph Partitioning and Expanders Handout 13 Luca Trevisan March 4, 2013 Lecture 13 In which we construct a family of expander graphs. The problem of constructing expander graphs

### Lectures 6, 7 and part of 8

Lectures 6, 7 and part of 8 Uriel Feige April 26, May 3, May 10, 2015 1 Linear programming duality 1.1 The diet problem revisited Recall the diet problem from Lecture 1. There are n foods, m nutrients,

### 1 T 1 = where 1 is the all-ones vector. For the upper bound, let v 1 be the eigenvector corresponding. u:(u,v) E v 1(u)

CME 305: Discrete Mathematics and Algorithms Instructor: Reza Zadeh (rezab@stanford.edu) Final Review Session 03/20/17 1. Let G = (V, E) be an unweighted, undirected graph. Let λ 1 be the maximum eigenvalue

### 16.1 Min-Cut as an LP

600.469 / 600.669 Approximation Algorithms Lecturer: Michael Dinitz Topic: LPs as Metrics: Min Cut and Multiway Cut Date: 4//5 Scribe: Gabriel Kaptchuk 6. Min-Cut as an LP We recall the basic definition

### Lecture 10: October 27, 2016

Mathematical Toolkit Autumn 206 Lecturer: Madhur Tulsiani Lecture 0: October 27, 206 The conjugate gradient method In the last lecture we saw the steepest descent or gradient descent method for finding

### Duality of LPs and Applications

Lecture 6 Duality of LPs and Applications Last lecture we introduced duality of linear programs. We saw how to form duals, and proved both the weak and strong duality theorems. In this lecture we will

### Distances, volumes, and integration

Distances, volumes, and integration Scribe: Aric Bartle 1 Local Shape of a Surface A question that we may ask ourselves is what significance does the second fundamental form play in the geometric characteristics

### U.C. Berkeley CS294: Beyond Worst-Case Analysis Handout 3 Luca Trevisan August 31, 2017

U.C. Berkeley CS294: Beyond Worst-Case Analysis Handout 3 Luca Trevisan August 3, 207 Scribed by Keyhan Vakil Lecture 3 In which we complete the study of Independent Set and Max Cut in G n,p random graphs.

### The Hilbert Space of Random Variables

The Hilbert Space of Random Variables Electrical Engineering 126 (UC Berkeley) Spring 2018 1 Outline Fix a probability space and consider the set H := {X : X is a real-valued random variable with E[X 2

### Vertical Versus Horizontal Poincare Inequalities. Assaf Naor Courant Institute

Vertical Versus Horizontal Poincare Inequalities Assaf Naor Courant Institute Bi-Lipschitz distortion a metric space and (M; d M ) (X; k k X ) a Banach space. c X (M) = the infimum over those D 2 (1; 1]

### 16.1 L.P. Duality Applied to the Minimax Theorem

CS787: Advanced Algorithms Scribe: David Malec and Xiaoyong Chai Lecturer: Shuchi Chawla Topic: Minimax Theorem and Semi-Definite Programming Date: October 22 2007 In this lecture, we first conclude our

### Some notes on streaming algorithms continued

U.C. Berkeley CS170: Algorithms Handout LN-11-9 Christos Papadimitriou & Luca Trevisan November 9, 016 Some notes on streaming algorithms continued Today we complete our quick review of streaming algorithms.

### Math 350 Fall 2011 Notes about inner product spaces. In this notes we state and prove some important properties of inner product spaces.

Math 350 Fall 2011 Notes about inner product spaces In this notes we state and prove some important properties of inner product spaces. First, recall the dot product on R n : if x, y R n, say x = (x 1,...,

### Convex and Semidefinite Programming for Approximation

Convex and Semidefinite Programming for Approximation We have seen linear programming based methods to solve NP-hard problems. One perspective on this is that linear programming is a meta-method since

### Fréchet embeddings of negative type metrics

Fréchet embeddings of negative type metrics Sanjeev Arora James R Lee Assaf Naor Abstract We show that every n-point metric of negative type (in particular, every n-point subset of L 1 ) admits a Fréchet

### Massachusetts Institute of Technology 6.854J/18.415J: Advanced Algorithms Friday, March 18, 2016 Ankur Moitra. Problem Set 6

Massachusetts Institute of Technology 6.854J/18.415J: Advanced Algorithms Friday, March 18, 2016 Ankur Moitra Problem Set 6 Due: Wednesday, April 6, 2016 7 pm Dropbox Outside Stata G5 Collaboration policy:

### Lecture 12 : Graph Laplacians and Cheeger s Inequality

CPS290: Algorithmic Foundations of Data Science March 7, 2017 Lecture 12 : Graph Laplacians and Cheeger s Inequality Lecturer: Kamesh Munagala Scribe: Kamesh Munagala Graph Laplacian Maybe the most beautiful

### Throughout these notes we assume V, W are finite dimensional inner product spaces over C.

Math 342 - Linear Algebra II Notes Throughout these notes we assume V, W are finite dimensional inner product spaces over C 1 Upper Triangular Representation Proposition: Let T L(V ) There exists an orthonormal

### CS675: Convex and Combinatorial Optimization Fall 2016 Convex Optimization Problems. Instructor: Shaddin Dughmi

CS675: Convex and Combinatorial Optimization Fall 2016 Convex Optimization Problems Instructor: Shaddin Dughmi Outline 1 Convex Optimization Basics 2 Common Classes 3 Interlude: Positive Semi-Definite

### 6.842 Randomness and Computation March 3, Lecture 8

6.84 Randomness and Computation March 3, 04 Lecture 8 Lecturer: Ronitt Rubinfeld Scribe: Daniel Grier Useful Linear Algebra Let v = (v, v,..., v n ) be a non-zero n-dimensional row vector and P an n n

### Lagrange Multipliers

Optimization with Constraints As long as algebra and geometry have been separated, their progress have been slow and their uses limited; but when these two sciences have been united, they have lent each

### Spectral Processing. Misha Kazhdan

Spectral Processing Misha Kazhdan [Taubin, 1995] A Signal Processing Approach to Fair Surface Design [Desbrun, et al., 1999] Implicit Fairing of Arbitrary Meshes [Vallet and Levy, 2008] Spectral Geometry

### Hardness of Embedding Metric Spaces of Equal Size

Hardness of Embedding Metric Spaces of Equal Size Subhash Khot and Rishi Saket Georgia Institute of Technology {khot,saket}@cc.gatech.edu Abstract. We study the problem embedding an n-point metric space

### 1 Adjacency matrix and eigenvalues

CSC 5170: Theory of Computational Complexity Lecture 7 The Chinese University of Hong Kong 1 March 2010 Our objective of study today is the random walk algorithm for deciding if two vertices in an undirected

### Approximation Algorithms

Approximation Algorithms Chapter 26 Semidefinite Programming Zacharias Pitouras 1 Introduction LP place a good lower bound on OPT for NP-hard problems Are there other ways of doing this? Vector programs

### Fiedler s Theorems on Nodal Domains

Spectral Graph Theory Lecture 7 Fiedler s Theorems on Nodal Domains Daniel A. Spielman September 19, 2018 7.1 Overview In today s lecture we will justify some of the behavior we observed when using eigenvectors

### Delsarte s linear programming bound

15-859 Coding Theory, Fall 14 December 5, 2014 Introduction For all n, q, and d, Delsarte s linear program establishes a series of linear constraints that every code in F n q with distance d must satisfy.

### Topic: Primal-Dual Algorithms Date: We finished our discussion of randomized rounding and began talking about LP Duality.

CS787: Advanced Algorithms Scribe: Amanda Burton, Leah Kluegel Lecturer: Shuchi Chawla Topic: Primal-Dual Algorithms Date: 10-17-07 14.1 Last Time We finished our discussion of randomized rounding and

### Linear Algebra, Summer 2011, pt. 2

Linear Algebra, Summer 2, pt. 2 June 8, 2 Contents Inverses. 2 Vector Spaces. 3 2. Examples of vector spaces..................... 3 2.2 The column space......................... 6 2.3 The null space...........................

### Embeddings of finite metric spaces in Euclidean space: a probabilistic view

Embeddings of finite metric spaces in Euclidean space: a probabilistic view Yuval Peres May 11, 2006 Talk based on work joint with: Assaf Naor, Oded Schramm and Scott Sheffield Definition: An invertible

### Lecture 1: Entropy, convexity, and matrix scaling CSE 599S: Entropy optimality, Winter 2016 Instructor: James R. Lee Last updated: January 24, 2016

Lecture 1: Entropy, convexity, and matrix scaling CSE 599S: Entropy optimality, Winter 2016 Instructor: James R. Lee Last updated: January 24, 2016 1 Entropy Since this course is about entropy maximization,

### Fiedler s Theorems on Nodal Domains

Spectral Graph Theory Lecture 7 Fiedler s Theorems on Nodal Domains Daniel A Spielman September 9, 202 7 About these notes These notes are not necessarily an accurate representation of what happened in

### Integer Linear Programs

Lecture 2: Review, Linear Programming Relaxations Today we will talk about expressing combinatorial problems as mathematical programs, specifically Integer Linear Programs (ILPs). We then see what happens

### Lecture 12: Introduction to Spectral Graph Theory, Cheeger s inequality

CSE 521: Design and Analysis of Algorithms I Spring 2016 Lecture 12: Introduction to Spectral Graph Theory, Cheeger s inequality Lecturer: Shayan Oveis Gharan May 4th Scribe: Gabriel Cadamuro Disclaimer:

### 14 Singular Value Decomposition

14 Singular Value Decomposition For any high-dimensional data analysis, one s first thought should often be: can I use an SVD? The singular value decomposition is an invaluable analysis tool for dealing

### Key words. sparsification, approximation algorithm, graph partitioning, routing, metric space

VERTEX SPARSIFICATION AND OBLIVIOUS REDUCTIONS ANKUR MOITRA Abstract. Given an undirected, capacitated graph G = (V, E) and a set K V of terminals of size k, we construct an undirected, capacitated graph

### Lecture 5. Ch. 5, Norms for vectors and matrices. Norms for vectors and matrices Why?

KTH ROYAL INSTITUTE OF TECHNOLOGY Norms for vectors and matrices Why? Lecture 5 Ch. 5, Norms for vectors and matrices Emil Björnson/Magnus Jansson/Mats Bengtsson April 27, 2016 Problem: Measure size of

### Lecture Notes 1: Vector spaces

Optimization-based data analysis Fall 2017 Lecture Notes 1: Vector spaces In this chapter we review certain basic concepts of linear algebra, highlighting their application to signal processing. 1 Vector

### BBM402-Lecture 20: LP Duality

BBM402-Lecture 20: LP Duality Lecturer: Lale Özkahya Resources for the presentation: https://courses.engr.illinois.edu/cs473/fa2016/lectures.html An easy LP? which is compact form for max cx subject to

### Lecture 22: Hyperplane Rounding for Max-Cut SDP

CSCI-B609: A Theorist s Toolkit, Fall 016 Nov 17 Lecture : Hyperplane Rounding for Max-Cut SDP Lecturer: Yuan Zhou Scribe: Adithya Vadapalli 1 Introduction In the previous lectures we had seen the max-cut

### Iterative solvers for linear equations

Spectral Graph Theory Lecture 15 Iterative solvers for linear equations Daniel A. Spielman October 1, 009 15.1 Overview In this and the next lecture, I will discuss iterative algorithms for solving linear

### princeton univ. F 17 cos 521: Advanced Algorithm Design Lecture 6: Provable Approximation via Linear Programming

princeton univ. F 17 cos 521: Advanced Algorithm Design Lecture 6: Provable Approximation via Linear Programming Lecturer: Matt Weinberg Scribe: Sanjeev Arora One of the running themes in this course is

### 8 Eigenvectors and the Anisotropic Multivariate Gaussian Distribution

Eigenvectors and the Anisotropic Multivariate Gaussian Distribution Eigenvectors and the Anisotropic Multivariate Gaussian Distribution EIGENVECTORS [I don t know if you were properly taught about eigenvectors

### U.C. Berkeley CS270: Algorithms Lecture 21 Professor Vazirani and Professor Rao Last revised. Lecture 21

U.C. Berkeley CS270: Algorithms Lecture 21 Professor Vazirani and Professor Rao Scribe: Anupam Last revised Lecture 21 1 Laplacian systems in nearly linear time Building upon the ideas introduced in the

### Lecture 5 : Projections

Lecture 5 : Projections EE227C. Lecturer: Professor Martin Wainwright. Scribe: Alvin Wan Up until now, we have seen convergence rates of unconstrained gradient descent. Now, we consider a constrained minimization

### CSC Linear Programming and Combinatorial Optimization Lecture 12: The Lift and Project Method

CSC2411 - Linear Programming and Combinatorial Optimization Lecture 12: The Lift and Project Method Notes taken by Stefan Mathe April 28, 2007 Summary: Throughout the course, we have seen the importance

### MIT Algebraic techniques and semidefinite optimization February 14, Lecture 3

MI 6.97 Algebraic techniques and semidefinite optimization February 4, 6 Lecture 3 Lecturer: Pablo A. Parrilo Scribe: Pablo A. Parrilo In this lecture, we will discuss one of the most important applications