Introduction to Semidefinite Programming I: Basic properties a
|
|
- Regina Hall
- 5 years ago
- Views:
Transcription
1 Introduction to Semidefinite Programming I: Basic properties and variations on the Goemans-Williamson approximation algorithm for max-cut MFO seminar on Semidefinite Programming May 30, 2010
2 Semidefinite programming duality Complexity Positive semidefinite matrices Definition: For a symmetric n n matrix X, the following conditions are equivalent: 1. X is positive semidefinite (written X 0) if all eigenvalues of X are nonnegative. 2. u T Xu 0 for all u R n. 3. X = UU T for some matrix U R n p. 4. For some vectors v 1,..., v n R p, X ij = vi T v j ( i, j [n]). Say that X is the Gram matrix of the v i s. 5. All principal minors of X are nonnegative. Definition: X is positive definite (written X 0) if all eigenvalues of X are positive.
3 Semidefinite programming duality Complexity Notation S n : the space of n n symmetric matrices. S n + : the cone of positive semidefinite matrices. S n ++ : the cone of positive definite matrices. S n ++ is the interior of the cone S n +. Trace inner product on S n : A B = Tr(A T B) = n A ij B ij i,j=1 The PSD cone is self-dual: For X S n, A S + n A B 0 B S + n
4 Semidefinite programming duality Complexity Primal/dual semidefinite programs Given matrices C, A 1,..., A m S n and a vector b R m Primal SDP: p := max X C X such that A j X = b j (j = 1,..., m), X 0 Dual SDP: d := min y b y such that m j=1 y ja j C 0 Weak duality: p d Pf: If X is primal feasible and y is dual feasible, then ( m ) 0 y j A j C X = j=1 j y j (A j X ) C X = b y C X
5 Semidefinite programming duality Complexity Analogy between LP and SDP Given vectors c, a 1,..., a m R n and b R m Primal/dual LP: max x c x such that a j x = b j ( j m), x R n + min y b y such that m y j a j c 0 j=1 Primal SDP: max X C X such that A j X = b j (j = 1,..., m), X S + n SDP is the analogue of LP, replacing R n + by S + n. Get LP when C, A j are diagonal matrices.
6 Semidefinite programming duality Complexity Strong duality: p = d? Strong duality holds for LP, but we need some regularity condition (e.g., Slater condition) to have strong duality for SDP! Primal (P) / Dual (D) SDP s: p d (P) p = sup C X s.t. A j X = b j (j = 1,..., m), X 0 (D) d = inf b y s.t. m j=1 y ja j C 0 Strong duality Theorem: 1. If (P) is strictly feasible ( X 0 feasible for (P)) and bounded (p < ), then p = d and (D) attains its infimum. 2. If (D) is strictly feasible ( y with j y ja j C 0) and bounded (d > ), then p = d and (P) attains its supremum.
7 Semidefinite programming duality Complexity Proof of 2. Assume d R and j ỹja j C 0 ỹ p = max X 0? C X d = inf y b y A j X = b j j y ja j C 0 Goal: There exists X feasible for (P) with C X d. WMA b 0 (else, b = 0 implies d = 0 and choose X = 0). Set { M := y j A j C y R m, b y d }. Fact: M S ++ n =. j Pf: Otherwise, let y for which b y d and j y ja j C 0. Then one can find y with b y < b y d and j y j A j C 0. This contradicts the minimality of d.
8 Semidefinite programming duality Complexity Sketch of proof for 2. (continued) As M S ++ n =, there is a hyperplane separating M and S ++ n. That is, there exists Z 0 non-zero with Z Y 0 Y M, i.e., b y d = Z ( j y j A j C) 0 By Farkas lemma, there exists µ R + for which (Z A j ) j = µb and µd Z C If µ = 0, then 0 }{{} Z ( ỹ j A j C) > 0, a contradiction. 0 j }{{} 0 Hence µ > 0 and Z/µ is feasible for (P) with C (Z/µ) d. QED.
9 Semidefinite programming duality Complexity An example with duality gap 0 x 12 0 p = min x 12 s.t. x 12 x x 12 = min 1 2 E 12 X s.t. E 11 X = 0 a E 13 X = 0 b E 23 X = 0 c (E E 12) X = 1 y X 0 d = max y s.t. 1 2 E 12 ae 11 be 13 ce 23 y(e E 12) 0 y+1 a 2 b = max y s.t. y c 0 b c y Thus, p = 0, d = 1 non-zero duality gap
10 Semidefinite programming duality Complexity Complexity Recall: An LP with rational data has a rational optimum solution whose bit size is polynomially bounded in terms of the bit length of the input data. Not true for SDP: 2 = max x s.t. Any solution to ( ) x , 0 1 satisfies x 1 2 2n 1. ( ) 1 x 0 x 2 ( ) ( ) x2 x 1 xn x 0,..., n 1 0 x 1 1 x n 1 1
11 Semidefinite programming duality Complexity Complexity (continued) Theorem: SDP can be solved in polynomial time to an arbitrary prescribed precision. [Assuming certain technical conditions hold.] Theoretically: Use the ellipsoid method [since checking whether X 0 is in P, e.g. with Gaussian elimination] Practically: Use e.g. interior-point algorithms. More precisely: Let K denote the feasible region of the SDP. Assume we know R N s.t. X K with X R if K. Given ɛ > 0, the ellipsoid based algorithm, either finds X at distance at most ɛ from K such that C X C X ɛ X K at distance at least ɛ from the border, or claims: there is no such X. The rumnning time is polynomial in n, m, the bit size of A j, C, b, log R, and log(1/ɛ).
12 Semidefinite programming duality Complexity Feasibility of SDP Feasibility SDP problem (F): Given integer A 0, A j S n, decide whether there exists x R m s.t. A 0 + m j=1 x ja j 0? (F) NP (F) co-np. [Ramana 97] (F) P for fixed n or m. [Porkolab-Khachiyan 97] Testing existence of a rational solution is in P for fixed dimension m. [Porkolab-Khachiyan 97] More on complexity and algorithms for SDP in other lectures.
13 Semidefinite programming duality Complexity Use SDP to express convex quadratic constraints Consider the convex quadratic constraint: x T Ax b T x + c where A 0. Write A = B T B for some B R p n. ( ) Then: x T Ax b T Ip Bx x + c x T B T b T 0 x + c Use Schur complement: Given C 0, ( ) C B B T 0 A B T C 1 B 0 A
14 Semidefinite programming duality Complexity The S-lemma [Yakubovich 1971] Consider the quadratic polynomials: ( ) ( ) f (x) = x T Ax + 2a T x + α = (1 x T α a T 1 ) a A x ( ) ( ) g(x) = x T Bx + 2b T x + β = (1 x T β b T 1 ) b B x Question: Characterize when (*) f (x) 0 = g(x) 0 Answer: Assume f (x) > 0 for some x. Then, (*) holds IFF ( ) ( ) β b T α a T λ 0 for some λ 0 b B a A
15 Semidefinite programming duality Complexity Testing sums of squares of polynomials Question: How to check whether a polynomial p(x) = α N n α 2d p αx α can be written as a sum of squares: p(x)? = m j=1 (u j(x)) 2 for some polynomials u j? Answer: Use semidefinite programming: x α = x α 1 1 x n αn Write u j (x) = (a j ) T [x] d [x] d = (x α ) α d j (u j(x)) 2 = ([x] d ) T ( a j aj T ) [x] d j }{{} A 0 Test feasibility of SDP: A β,γ = p α ( α 2d), A 0 β,γ: β, γ d β+γ=α
16 Two milestone applications of SDP to combinatorial optimization Approximate maximum stable sets and minimum vertex coloring with the theta number. Work of Lovász [1979], Grötschel-Lovász-Schrijver [1981] (First non-trivial) approximation algorithm for max-cut of Goemans-Williamson [1995]
17 G = (V, E) graph. S V stable set if S contains no edge. α(g):= maximum size of a stable set stability number χ(g):= minimum number of colors needed to color the vertices so that adjacent vertices receive distinct colors. Computing α(g), χ(g) is an NP-hard problem. of Lovász [1979]: ϑ(g) := max J X s.t. Tr(X ) = 1, X ij = 0 (ij E), X 0 Lovász sandwich theorem: α(g) ϑ(g) χ(ḡ). Can compute α(g), χ(ḡ) via SDP for graphs with α(g) = χ(ḡ).
18 Maximum cuts in graphs G = (V, E), n = V, w = (w e ) e E edge weights. S V cut δ(s) := all edges cut by the partition (S, V \ S). Max-Cut problem: Find a cut of maximum weight. Max-Cut is NP-hard. No (16/17 + ɛ)-approximation algorithm unless P=NP. mc(g) Max-Cut is in P for graphs with no K 5 minor, since it can be computed with the LP: [Barahona-Mahjoub 86] max w T x s.t. x ij x ik x jk 0, x ij +x ik +x jk 2 i, j, k V An easy 1/2-approximation algorithm for w 0: Consider the random partition (S, V \ S), where i S with prob. 1/2 : E(w(S)) = w(e)/2 mc(g)/2
19 Goemans-Williamson approximation algorithm for Max-Cut Encode a partition (S, V \ S) by a vector x {±1} n. Encode the cut δ(s) by the matrix X = xx T. Reformulate Max-Cut: max 1 2 ij E w ij(1 x i x j ) s.t. x {±1} n Solve the SDP relaxation: max 1 2 ij E w ij(1 X ij ) s.t. X 0, diag(x ) = e v 1,..., v n unit vectors s.t. X = (vi T v j ) is opt. for SDP. Randomized rounding: Pick a random hyperplane H with normal r. partition (S, V \ S) depending on the sign of v T i r.
20 Performance analysis Theorem: For w 0, mc(g) E(w(δ(S))) sdp(g) mc(g). }{{} Basic lemma: Prob(ij δ(s)) = arccos(v T i v j ) π. E(w(δ(S))) = ij E w ij Prob(ij δ(s)) = ij E w ij arccos(vi T v j ) π = ij E w ij (1 v T i v j ) 2 sdp(g) α GW 2 arccos(vi T v j ) π } 1 vi T v j {{ } α GW 0.878
21 Extension to ±1 quadratic programming Given A S n Integer problem: ip(a) := max x T Ax s.t. x {±1} n SDP relaxation: sdp(a) := max A X s.t. X 0, diag(x ) = e. A = 1 4 L w, L w : Laplacian matrix of (G, w) where L w (i, i) = w(δ(i)), L w (i, j) = w ij. Max-Cut When A 0, Ae = e, A ij 0 (i j) approx. alg. When A 0 2 π ( 0.636)-approx. alg. [Nesterov 97] When diag(a) = 0 Grothendieck constant
22 Nesterov 2 π-approximation algorithm Solve SDP: Let v 1,..., v n unit vectors s.t. X = (v T i v j ) maximizes A X. Random hyperplane rounding: Pick a random unit vector r. random ±1 vector: x = (sgn(r T v i )) n i=1 Lemma 1 [identity of Grothendieck] E(xx T ) = 2 π arcsin X. Proof: E(sgn(r T v i ) sgn(r T v j )) = 1 2 Prob(sgn(r T v i ) sgn(r T v j )) = 1 2 arccos(v i T v j ) π = 2 π ( π 2 arccos(v i T v j )) = 2 π arcsin(v T i v j ).
23 Global performance analysis Lemma 2: arcsin X X 0 Proof: arcsin x x = k a kx 2k+1 where a k 0. Global analysis: E(x T Ax) = A E(xx T ) Therefore: For A 0, = A (E(xx T ) 2 π X ) + 2 π A X = }{{} A ( 2 π arcsin X 2 π X ) + 2 π A X 0 }{{} 0 }{{} 0 2 π A X. ip(a) 2 π sdp(a).
24 Grothendieck inequality Assume diag(a) = 0. The support graph G A has as edges the pairs ij with A ij 0. Definition: The Grothendieck constant K(G) of a graph G is the smallest constant K for which sdp(a) K ip(a) for all A S n with G A G. Theorem: ([Gr. 53] [Krivine 77] [Alon-Makarychev(x2)-Naor 05]) For G complete bipartite, π 2 K(G) π 1 2 ln( ) Ω(log(ω(G))) K(G) O(log(ϑ(Ḡ))).
25 Sketch of proof for Krivine s upper bound: Show: K(K n,m ) π 2 π 1 2 ln(1+ 2) 1 ln(1+ 2) =: π 2c? c := ln(1 + 2) 1. Let A R n m. Let u i (i n) and v j (j m) be unit vectors in H maximizing sdp(a) = i n, j m a iju i v j. 2. Construct new unit vectors S(u i ), T (v j ) Ĥ satisfying arcsin(s(u i ) T (v j )) = c u i v j 3. Pick a random unit vector r Ĥ. Define the ±1 vectors x, y x i = sgn(r T S(u i )), y j = sgn(r T T (v j )) 4. Analysis: E(x T Ay) = i,j a ij E(x i y j ) = i,j a ij 2 π arcsin(s(u i) T (v j )) = i,j a ij 2 π c u i v j = 2 π c sdp(a).
26 Proof (continued) Step 2. Given unit vectors u, v H, construct S(u), T (v) Ĥ satisfying arcsin(s(u) T (v)) = c u v. sin x = k 0 ( 1)k x2k+1 (2k+1)!, sinh x = k 0 x2k+1 (2k+1)! Set c := sinh 1 (1) = ln(1 + 2). ( c Set S(u) = 2k+1 (2k+1)! )k u (2k+1) Ĥ := k 0 H (2k+1), T (v) = (( 1) k c 2k+1 (2k+1)! )k v (2k+1) Ĥ. Then, S(u) T (v) = k ( 1)k c2k+1 (2k+1)! (u v)2k+1 = sin(c u v). Thus: arcsin(s(u) T (v)) = c u v.
27 A reformulation of the theta number Theorem [Alon-Makarychev(x2)-Naor 05] The smallest constant C for which sdp( A) C sdp(a) for all A S n with G A G is C = ϑ(ḡ) 1. Geometrically: E n := {X S n X 0, diag(x ) = e} the elliptope E(G) R E : the projection of E n onto the edge set of G. Theorem [AMMN]: E(G) (ϑ(ḡ) 1) E(G).
28 Link with matrix completion : Given a partial n n matrix, whose entries are specified on the diagonal (say equal to 1) and on a subset E of the positions (given by x R E ), decide whether it can be completed to a PSD matrix. Equivalently, decide whether x E(G)? A necessary condition: Each fully specified principal submatrix is PSD. [Clique condition] The clique condition is sufficient IFF G is a chordal graph (i.e. no induced circuit of length 4). 1 1 a? b? a? b? 1 1 is not completable to PSD
29 Another necessary condition Fact: a + b + c 2π 1 cos a cos b cos a 1 cos c a b c 0 0 a + b c 0 cos b cos c 1 a b + c 0 Write x = cos a for some a [0, π] E. If x E(G) then [Metric condition] a(f ) a(c \ F ) π( F 1) C circuit, F C odd. The metric condition is sufficient IFF G has no K 4 minor. 1 1/2 1/2 1/2 1/2 1 1/2 1/2 1/2 1/2 1 1/2 0 1/2 1/2 1/2 1 while 2π 3 (1, 1, 1, 1) satisfies the triangle inequalities.
30 Geometrically Introduction CUT ±1 (G) E(G), with equality IFF G has no K 3 minor. E(G) cos(π MET 01 (G)), with equality IFF G has no K 4 minor. E(G) cos(π CUT 01 (G)), with equality IFF G has no K 4 minor. The Goemans-Williamson randomized rounding argument shows: If v 1,..., v n are unit vectors and a ij := arccos(vi T v j ) are their pairwise angles, then c ij a ij π c 0 1 i<j n if c z c 0 is any inequality valid for the cuts of K n.
31 Extension to max k-cut [Frieze Jerrum 95] Max k-cut: Given G = (V, E), w R E +, k 2, find a partition P = (S 1,..., S k ) maximizing w(p) = e E e is cut by P w e. Pick unit vectors a 1,..., a k R k with a T i a j = 1 k 1 Model max k-cut: SDP relax.: mc k (G) = ij E w ij(1 xi T x j ) s.t. x 1,..., x n {a 1,..., a k }. max k 1 k sdp k (G) = max k 1 k ij E w ij(1 vi T v j ) s.t. v i unit vectors, vi T v j 1 k 1. for i j. Randomized rounding: Pick k independent random unit vectors r 1,..., r k partition P = (S 1,..., S k ) where S h = {i vi T r h vi T r h h }.
32 Analysis Introduction The probability that edge ij is not cut, i.e., v i, v j are both closer to the same r h, is equal to k times a function f (v T i v j ). E(w(P)) = ij E w ij(1 kf (vi T v j )) = ij E w 1 kf (vi T v j ) k ij 1 vi T v j k 1 }{{} α k sdp k (G). 1 kf (t) k α k :=min 1 k 1 t 1 1 t k 1 k 1 k (1 v i T v j ) α 2 = α GW 0.878: GW approximation ratio for max-cut. α 3 = π 2 arccos 2 ( 1/4) > [de Klerk et al.] α 100 > 0.99.
SDP Relaxations for MAXCUT
SDP Relaxations for MAXCUT from Random Hyperplanes to Sum-of-Squares Certificates CATS @ UMD March 3, 2017 Ahmed Abdelkader MAXCUT SDP SOS March 3, 2017 1 / 27 Overview 1 MAXCUT, Hardness and UGC 2 LP
More informationLNMB PhD Course. Networks and Semidefinite Programming 2012/2013
LNMB PhD Course Networks and Semidefinite Programming 2012/2013 Monique Laurent CWI, Amsterdam, and Tilburg University These notes are based on material developed by M. Laurent and F. Vallentin for the
More informationSemidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 5
Semidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 5 Instructor: Farid Alizadeh Scribe: Anton Riabov 10/08/2001 1 Overview We continue studying the maximum eigenvalue SDP, and generalize
More informationCSC Linear Programming and Combinatorial Optimization Lecture 10: Semidefinite Programming
CSC2411 - Linear Programming and Combinatorial Optimization Lecture 10: Semidefinite Programming Notes taken by Mike Jamieson March 28, 2005 Summary: In this lecture, we introduce semidefinite programming
More information8 Approximation Algorithms and Max-Cut
8 Approximation Algorithms and Max-Cut 8. The Max-Cut problem Unless the widely believed P N P conjecture is false, there is no polynomial algorithm that can solve all instances of an NP-hard problem.
More informationCopositive Programming and Combinatorial Optimization
Copositive Programming and Combinatorial Optimization Franz Rendl http://www.math.uni-klu.ac.at Alpen-Adria-Universität Klagenfurt Austria joint work with M. Bomze (Wien) and F. Jarre (Düsseldorf) and
More informationMIT Algebraic techniques and semidefinite optimization February 14, Lecture 3
MI 6.97 Algebraic techniques and semidefinite optimization February 4, 6 Lecture 3 Lecturer: Pablo A. Parrilo Scribe: Pablo A. Parrilo In this lecture, we will discuss one of the most important applications
More informationThe maximal stable set problem : Copositive programming and Semidefinite Relaxations
The maximal stable set problem : Copositive programming and Semidefinite Relaxations Kartik Krishnan Department of Mathematical Sciences Rensselaer Polytechnic Institute Troy, NY 12180 USA kartis@rpi.edu
More informationSemidefinite Programming
Semidefinite Programming Basics and SOS Fernando Mário de Oliveira Filho Campos do Jordão, 2 November 23 Available at: www.ime.usp.br/~fmario under talks Conic programming V is a real vector space h, i
More informationLift-and-Project Techniques and SDP Hierarchies
MFO seminar on Semidefinite Programming May 30, 2010 Typical combinatorial optimization problem: max c T x s.t. Ax b, x {0, 1} n P := {x R n Ax b} P I := conv(k {0, 1} n ) LP relaxation Integral polytope
More informationModeling with semidefinite and copositive matrices
Modeling with semidefinite and copositive matrices Franz Rendl http://www.math.uni-klu.ac.at Alpen-Adria-Universität Klagenfurt Austria F. Rendl, Singapore workshop 2006 p.1/24 Overview Node and Edge relaxations
More informationOn the Sandwich Theorem and a approximation algorithm for MAX CUT
On the Sandwich Theorem and a 0.878-approximation algorithm for MAX CUT Kees Roos Technische Universiteit Delft Faculteit Electrotechniek. Wiskunde en Informatica e-mail: C.Roos@its.tudelft.nl URL: http://ssor.twi.tudelft.nl/
More informationLecture: Examples of LP, SOCP and SDP
1/34 Lecture: Examples of LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2018.html wenzw@pku.edu.cn Acknowledgement:
More informationLecture 5. Max-cut, Expansion and Grothendieck s Inequality
CS369H: Hierarchies of Integer Programming Relaxations Spring 2016-2017 Lecture 5. Max-cut, Expansion and Grothendieck s Inequality Professor Moses Charikar Scribes: Kiran Shiragur Overview Here we derive
More informationFour new upper bounds for the stability number of a graph
Four new upper bounds for the stability number of a graph Miklós Ujvári Abstract. In 1979, L. Lovász defined the theta number, a spectral/semidefinite upper bound on the stability number of a graph, which
More informationSummer School: Semidefinite Optimization
Summer School: Semidefinite Optimization Christine Bachoc Université Bordeaux I, IMB Research Training Group Experimental and Constructive Algebra Haus Karrenberg, Sept. 3 - Sept. 7, 2012 Duality Theory
More informationComplexity of the positive semidefinite matrix completion problem with a rank constraint
Complexity of the positive semidefinite matrix completion problem with a rank constraint M. E.-Nagy 1, M. Laurent 1,2, and A. Varvitsiotis 1 1 Centrum Wiskunde & Informatica (CWI), Amsterdam, The Netherlands.
More informationConvex and Semidefinite Programming for Approximation
Convex and Semidefinite Programming for Approximation We have seen linear programming based methods to solve NP-hard problems. One perspective on this is that linear programming is a meta-method since
More informationGrothendieck s Inequality
Grothendieck s Inequality Leqi Zhu 1 Introduction Let A = (A ij ) R m n be an m n matrix. Then A defines a linear operator between normed spaces (R m, p ) and (R n, q ), for 1 p, q. The (p q)-norm of A
More informationLecture Semidefinite Programming and Graph Partitioning
Approximation Algorithms and Hardness of Approximation April 16, 013 Lecture 14 Lecturer: Alantha Newman Scribes: Marwa El Halabi 1 Semidefinite Programming and Graph Partitioning In previous lectures,
More informationSemidefinite Programming Basics and Applications
Semidefinite Programming Basics and Applications Ray Pörn, principal lecturer Åbo Akademi University Novia University of Applied Sciences Content What is semidefinite programming (SDP)? How to represent
More informationCopositive Programming and Combinatorial Optimization
Copositive Programming and Combinatorial Optimization Franz Rendl http://www.math.uni-klu.ac.at Alpen-Adria-Universität Klagenfurt Austria joint work with I.M. Bomze (Wien) and F. Jarre (Düsseldorf) IMA
More informationApproximation Algorithms
Approximation Algorithms Chapter 26 Semidefinite Programming Zacharias Pitouras 1 Introduction LP place a good lower bound on OPT for NP-hard problems Are there other ways of doing this? Vector programs
More informationLecture 10. Semidefinite Programs and the Max-Cut Problem Max Cut
Lecture 10 Semidefinite Programs and the Max-Cut Problem In this class we will finally introduce the content from the second half of the course title, Semidefinite Programs We will first motivate the discussion
More informationSemidefinite programs and combinatorial optimization
Semidefinite programs and combinatorial optimization Lecture notes by L. Lovász Microsoft Research Redmond, WA 98052 lovasz@microsoft.com http://www.research.microsoft.com/ lovasz Contents 1 Introduction
More informationApplications of the Inverse Theta Number in Stable Set Problems
Acta Cybernetica 21 (2014) 481 494. Applications of the Inverse Theta Number in Stable Set Problems Miklós Ujvári Abstract In the paper we introduce a semidefinite upper bound on the square of the stability
More information6.854J / J Advanced Algorithms Fall 2008
MIT OpenCourseWare http://ocw.mit.edu 6.85J / 8.5J Advanced Algorithms Fall 008 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms. 8.5/6.85 Advanced Algorithms
More information- Well-characterized problems, min-max relations, approximate certificates. - LP problems in the standard form, primal and dual linear programs
LP-Duality ( Approximation Algorithms by V. Vazirani, Chapter 12) - Well-characterized problems, min-max relations, approximate certificates - LP problems in the standard form, primal and dual linear programs
More informationAgenda. Applications of semidefinite programming. 1 Control and system theory. 2 Combinatorial and nonconvex optimization
Agenda Applications of semidefinite programming 1 Control and system theory 2 Combinatorial and nonconvex optimization 3 Spectral estimation & super-resolution Control and system theory SDP in wide use
More informationMustapha Ç. Pinar 1. Communicated by Jean Abadie
RAIRO Operations Research RAIRO Oper. Res. 37 (2003) 17-27 DOI: 10.1051/ro:2003012 A DERIVATION OF LOVÁSZ THETA VIA AUGMENTED LAGRANGE DUALITY Mustapha Ç. Pinar 1 Communicated by Jean Abadie Abstract.
More informationLecture 3: Semidefinite Programming
Lecture 3: Semidefinite Programming Lecture Outline Part I: Semidefinite programming, examples, canonical form, and duality Part II: Strong Duality Failure Examples Part III: Conditions for strong duality
More informationChapter 3. Some Applications. 3.1 The Cone of Positive Semidefinite Matrices
Chapter 3 Some Applications Having developed the basic theory of cone programming, it is time to apply it to our actual subject, namely that of semidefinite programming. Indeed, any semidefinite program
More information4. Algebra and Duality
4-1 Algebra and Duality P. Parrilo and S. Lall, CDC 2003 2003.12.07.01 4. Algebra and Duality Example: non-convex polynomial optimization Weak duality and duality gap The dual is not intrinsic The cone
More informationLecture Note 5: Semidefinite Programming for Stability Analysis
ECE7850: Hybrid Systems:Theory and Applications Lecture Note 5: Semidefinite Programming for Stability Analysis Wei Zhang Assistant Professor Department of Electrical and Computer Engineering Ohio State
More informationCSC Linear Programming and Combinatorial Optimization Lecture 12: The Lift and Project Method
CSC2411 - Linear Programming and Combinatorial Optimization Lecture 12: The Lift and Project Method Notes taken by Stefan Mathe April 28, 2007 Summary: Throughout the course, we have seen the importance
More informationI.3. LMI DUALITY. Didier HENRION EECI Graduate School on Control Supélec - Spring 2010
I.3. LMI DUALITY Didier HENRION henrion@laas.fr EECI Graduate School on Control Supélec - Spring 2010 Primal and dual For primal problem p = inf x g 0 (x) s.t. g i (x) 0 define Lagrangian L(x, z) = g 0
More informationConic approach to quantum graph parameters using linear optimization over the completely positive semidefinite cone
Conic approach to quantum graph parameters using linear optimization over the completely positive semidefinite cone Monique Laurent 1,2 and Teresa Piovesan 1 1 Centrum Wiskunde & Informatica (CWI), Amsterdam,
More informationLMI MODELLING 4. CONVEX LMI MODELLING. Didier HENRION. LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ. Universidad de Valladolid, SP March 2009
LMI MODELLING 4. CONVEX LMI MODELLING Didier HENRION LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ Universidad de Valladolid, SP March 2009 Minors A minor of a matrix F is the determinant of a submatrix
More informationU.C. Berkeley CS294: Spectral Methods and Expanders Handout 11 Luca Trevisan February 29, 2016
U.C. Berkeley CS294: Spectral Methods and Expanders Handout Luca Trevisan February 29, 206 Lecture : ARV In which we introduce semi-definite programming and a semi-definite programming relaxation of sparsest
More informationSEMIDEFINITE PROGRAM BASICS. Contents
SEMIDEFINITE PROGRAM BASICS BRIAN AXELROD Abstract. A introduction to the basics of Semidefinite programs. Contents 1. Definitions and Preliminaries 1 1.1. Linear Algebra 1 1.2. Convex Analysis (on R n
More informationA new graph parameter related to bounded rank positive semidefinite matrix completions
Mathematical Programming manuscript No. (will be inserted by the editor) Monique Laurent Antonios Varvitsiotis A new graph parameter related to bounded rank positive semidefinite matrix completions the
More informationSolving large Semidefinite Programs - Part 1 and 2
Solving large Semidefinite Programs - Part 1 and 2 Franz Rendl http://www.math.uni-klu.ac.at Alpen-Adria-Universität Klagenfurt Austria F. Rendl, Singapore workshop 2006 p.1/34 Overview Limits of Interior
More informationc 2000 Society for Industrial and Applied Mathematics
SIAM J. OPIM. Vol. 10, No. 3, pp. 750 778 c 2000 Society for Industrial and Applied Mathematics CONES OF MARICES AND SUCCESSIVE CONVEX RELAXAIONS OF NONCONVEX SES MASAKAZU KOJIMA AND LEVEN UNÇEL Abstract.
More informationLecture 17 (Nov 3, 2011 ): Approximation via rounding SDP: Max-Cut
CMPUT 675: Approximation Algorithms Fall 011 Lecture 17 (Nov 3, 011 ): Approximation via rounding SDP: Max-Cut Lecturer: Mohammad R. Salavatipour Scribe: based on older notes 17.1 Approximation Algorithm
More informationA notion of Total Dual Integrality for Convex, Semidefinite and Extended Formulations
A notion of for Convex, Semidefinite and Extended Formulations Marcel de Carli Silva Levent Tunçel April 26, 2018 A vector in R n is integral if each of its components is an integer, A vector in R n is
More informationLinear and non-linear programming
Linear and non-linear programming Benjamin Recht March 11, 2005 The Gameplan Constrained Optimization Convexity Duality Applications/Taxonomy 1 Constrained Optimization minimize f(x) subject to g j (x)
More informationLecture 8: The Goemans-Williamson MAXCUT algorithm
IU Summer School Lecture 8: The Goemans-Williamson MAXCUT algorithm Lecturer: Igor Gorodezky The Goemans-Williamson algorithm is an approximation algorithm for MAX-CUT based on semidefinite programming.
More information6-1 The Positivstellensatz P. Parrilo and S. Lall, ECC
6-1 The Positivstellensatz P. Parrilo and S. Lall, ECC 2003 2003.09.02.10 6. The Positivstellensatz Basic semialgebraic sets Semialgebraic sets Tarski-Seidenberg and quantifier elimination Feasibility
More informationSemidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 4
Semidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 4 Instructor: Farid Alizadeh Scribe: Haengju Lee 10/1/2001 1 Overview We examine the dual of the Fermat-Weber Problem. Next we will
More informationCOT 6936: Topics in Algorithms! Giri Narasimhan. ECS 254A / EC 2443; Phone: x3748
COT 6936: Topics in Algorithms! Giri Narasimhan ECS 254A / EC 2443; Phone: x3748 giri@cs.fiu.edu https://moodle.cis.fiu.edu/v2.1/course/view.php?id=612 Gaussian Elimination! Solving a system of simultaneous
More informationJOHN THICKSTUN. p x. n sup Ipp y n x np x nq. By the memoryless and stationary conditions respectively, this reduces to just 1 yi x i.
ESTIMATING THE SHANNON CAPACITY OF A GRAPH JOHN THICKSTUN. channels and graphs Consider a stationary, memoryless channel that maps elements of discrete alphabets X to Y according to a distribution p y
More informationLINEAR PROGRAMMING III
LINEAR PROGRAMMING III ellipsoid algorithm combinatorial optimization matrix games open problems Lecture slides by Kevin Wayne Last updated on 7/25/17 11:09 AM LINEAR PROGRAMMING III ellipsoid algorithm
More informationSemidefinite Programming and Integer Programming
Semidefinite Programming and Integer Programming Monique Laurent and Franz Rendl CWI, Kruislaan 413, 1098 SJ Amsterdam, The Netherlands monique@cwi.nl Universität Klagenfurt, Institut für Mathematik Universitätstrasse
More information1 The independent set problem
ORF 523 Lecture 11 Spring 2016, Princeton University Instructor: A.A. Ahmadi Scribe: G. Hall Tuesday, March 29, 2016 When in doubt on the accuracy of these notes, please cross chec with the instructor
More informationLecture 4: January 26
10-725/36-725: Conve Optimization Spring 2015 Lecturer: Javier Pena Lecture 4: January 26 Scribes: Vipul Singh, Shinjini Kundu, Chia-Yin Tsai Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer:
More informationDistance Geometry-Matrix Completion
Christos Konaxis Algs in Struct BioInfo 2010 Outline Outline Tertiary structure Incomplete data Tertiary structure Measure diffrence of matched sets Def. Root Mean Square Deviation RMSD = 1 n x i y i 2,
More informationConic Linear Optimization and its Dual. yyye
Conic Linear Optimization and Appl. MS&E314 Lecture Note #04 1 Conic Linear Optimization and its Dual Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A.
More informationExample: feasibility. Interpretation as formal proof. Example: linear inequalities and Farkas lemma
4-1 Algebra and Duality P. Parrilo and S. Lall 2006.06.07.01 4. Algebra and Duality Example: non-convex polynomial optimization Weak duality and duality gap The dual is not intrinsic The cone of valid
More informationNotes on the decomposition result of Karlin et al. [2] for the hierarchy of Lasserre by M. Laurent, December 13, 2012
Notes on the decomposition result of Karlin et al. [2] for the hierarchy of Lasserre by M. Laurent, December 13, 2012 We present the decomposition result of Karlin et al. [2] for the hierarchy of Lasserre
More informationE5295/5B5749 Convex optimization with engineering applications. Lecture 5. Convex programming and semidefinite programming
E5295/5B5749 Convex optimization with engineering applications Lecture 5 Convex programming and semidefinite programming A. Forsgren, KTH 1 Lecture 5 Convex optimization 2006/2007 Convex quadratic program
More informationELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications
ELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications Professor M. Chiang Electrical Engineering Department, Princeton University March
More informationSemidefinite Programming (SDP)
Chapter 3 Semidefinite Programming (SDP) You should all be familiar with linear programs, the theory of duality, and the use of linear programs to solve a variety of problems in polynomial time; flow or
More informationNetwork Optimization
Network Optimization Fall 2014 - Math 514 Lecturer: Thomas Rothvoss University of Washington, Seattle Last changes: December 4, 2014 2 Contents 1 The MaxCut problem 5 1.1 Semidefinite programs.........................................
More informationConvex Optimization M2
Convex Optimization M2 Lecture 8 A. d Aspremont. Convex Optimization M2. 1/57 Applications A. d Aspremont. Convex Optimization M2. 2/57 Outline Geometrical problems Approximation problems Combinatorial
More informationCO 250 Final Exam Guide
Spring 2017 CO 250 Final Exam Guide TABLE OF CONTENTS richardwu.ca CO 250 Final Exam Guide Introduction to Optimization Kanstantsin Pashkovich Spring 2017 University of Waterloo Last Revision: March 4,
More informationLecture 8. Strong Duality Results. September 22, 2008
Strong Duality Results September 22, 2008 Outline Lecture 8 Slater Condition and its Variations Convex Objective with Linear Inequality Constraints Quadratic Objective over Quadratic Constraints Representation
More informationGeometric Ramifications of the Lovász Theta Function and Their Interplay with Duality
Geometric Ramifications of the Lovász Theta Function and Their Interplay with Duality by Marcel Kenji de Carli Silva A thesis presented to the University of Waterloo in fulfillment of the thesis requirement
More informationRelaxations of combinatorial problems via association schemes
1 Relaxations of combinatorial problems via association schemes Etienne de Klerk, Fernando M. de Oliveira Filho, and Dmitrii V. Pasechnik Tilburg University, The Netherlands; Nanyang Technological University,
More informationCOURSE ON LMI PART I.2 GEOMETRY OF LMI SETS. Didier HENRION henrion
COURSE ON LMI PART I.2 GEOMETRY OF LMI SETS Didier HENRION www.laas.fr/ henrion October 2006 Geometry of LMI sets Given symmetric matrices F i we want to characterize the shape in R n of the LMI set F
More informationRelaxations and Randomized Methods for Nonconvex QCQPs
Relaxations and Randomized Methods for Nonconvex QCQPs Alexandre d Aspremont, Stephen Boyd EE392o, Stanford University Autumn, 2003 Introduction While some special classes of nonconvex problems can be
More informationU.C. Berkeley CS294: Beyond Worst-Case Analysis Handout 12 Luca Trevisan October 3, 2017
U.C. Berkeley CS94: Beyond Worst-Case Analysis Handout 1 Luca Trevisan October 3, 017 Scribed by Maxim Rabinovich Lecture 1 In which we begin to prove that the SDP relaxation exactly recovers communities
More informationSemidefinite Programming
Chapter 2 Semidefinite Programming 2.0.1 Semi-definite programming (SDP) Given C M n, A i M n, i = 1, 2,..., m, and b R m, the semi-definite programming problem is to find a matrix X M n for the optimization
More informationLecture: Introduction to LP, SDP and SOCP
Lecture: Introduction to LP, SDP and SOCP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2015.html wenzw@pku.edu.cn Acknowledgement:
More informationIII. Applications in convex optimization
III. Applications in convex optimization nonsymmetric interior-point methods partial separability and decomposition partial separability first order methods interior-point methods Conic linear optimization
More informationBBM402-Lecture 20: LP Duality
BBM402-Lecture 20: LP Duality Lecturer: Lale Özkahya Resources for the presentation: https://courses.engr.illinois.edu/cs473/fa2016/lectures.html An easy LP? which is compact form for max cx subject to
More informationLECTURE 10 LECTURE OUTLINE
LECTURE 10 LECTURE OUTLINE Min Common/Max Crossing Th. III Nonlinear Farkas Lemma/Linear Constraints Linear Programming Duality Convex Programming Duality Optimality Conditions Reading: Sections 4.5, 5.1,5.2,
More informationEE 227A: Convex Optimization and Applications October 14, 2008
EE 227A: Convex Optimization and Applications October 14, 2008 Lecture 13: SDP Duality Lecturer: Laurent El Ghaoui Reading assignment: Chapter 5 of BV. 13.1 Direct approach 13.1.1 Primal problem Consider
More informationLecture #21. c T x Ax b. maximize subject to
COMPSCI 330: Design and Analysis of Algorithms 11/11/2014 Lecture #21 Lecturer: Debmalya Panigrahi Scribe: Samuel Haney 1 Overview In this lecture, we discuss linear programming. We first show that the
More informationLecture 7: Positive Semidefinite Matrices
Lecture 7: Positive Semidefinite Matrices Rajat Mittal IIT Kanpur The main aim of this lecture note is to prepare your background for semidefinite programming. We have already seen some linear algebra.
More informationLecture 8 : Eigenvalues and Eigenvectors
CPS290: Algorithmic Foundations of Data Science February 24, 2017 Lecture 8 : Eigenvalues and Eigenvectors Lecturer: Kamesh Munagala Scribe: Kamesh Munagala Hermitian Matrices It is simpler to begin with
More information4TE3/6TE3. Algorithms for. Continuous Optimization
4TE3/6TE3 Algorithms for Continuous Optimization (Duality in Nonlinear Optimization ) Tamás TERLAKY Computing and Software McMaster University Hamilton, January 2004 terlaky@mcmaster.ca Tel: 27780 Optimality
More informationApproximation Algorithms for Maximum. Coverage and Max Cut with Given Sizes of. Parts? A. A. Ageev and M. I. Sviridenko
Approximation Algorithms for Maximum Coverage and Max Cut with Given Sizes of Parts? A. A. Ageev and M. I. Sviridenko Sobolev Institute of Mathematics pr. Koptyuga 4, 630090, Novosibirsk, Russia fageev,svirg@math.nsc.ru
More informationHierarchies. 1. Lovasz-Schrijver (LS), LS+ 2. Sherali Adams 3. Lasserre 4. Mixed Hierarchy (recently used) Idea: P = conv(subset S of 0,1 n )
Hierarchies Today 1. Some more familiarity with Hierarchies 2. Examples of some basic upper and lower bounds 3. Survey of recent results (possible material for future talks) Hierarchies 1. Lovasz-Schrijver
More informationWHEN DOES THE POSITIVE SEMIDEFINITENESS CONSTRAINT HELP IN LIFTING PROCEDURES?
MATHEMATICS OF OPERATIONS RESEARCH Vol. 6, No. 4, November 00, pp. 796 85 Printed in U.S.A. WHEN DOES THE POSITIVE SEMIDEFINITENESS CONSTRAINT HELP IN LIFTING PROCEDURES? MICHEL X. GOEMANS and LEVENT TUNÇEL
More informationLecture 6: Conic Optimization September 8
IE 598: Big Data Optimization Fall 2016 Lecture 6: Conic Optimization September 8 Lecturer: Niao He Scriber: Juan Xu Overview In this lecture, we finish up our previous discussion on optimality conditions
More informationInteger Linear Programs
Lecture 2: Review, Linear Programming Relaxations Today we will talk about expressing combinatorial problems as mathematical programs, specifically Integer Linear Programs (ILPs). We then see what happens
More informationLinear algebra for computational statistics
University of Seoul May 3, 2018 Vector and Matrix Notation Denote 2-dimensional data array (n p matrix) by X. Denote the element in the ith row and the jth column of X by x ij or (X) ij. Denote by X j
More informationLecture 1. 1 Conic programming. MA 796S: Convex Optimization and Interior Point Methods October 8, Consider the conic program. min.
MA 796S: Convex Optimization and Interior Point Methods October 8, 2007 Lecture 1 Lecturer: Kartik Sivaramakrishnan Scribe: Kartik Sivaramakrishnan 1 Conic programming Consider the conic program min s.t.
More informationRelations between Semidefinite, Copositive, Semi-infinite and Integer Programming
Relations between Semidefinite, Copositive, Semi-infinite and Integer Programming Author: Faizan Ahmed Supervisor: Dr. Georg Still Master Thesis University of Twente the Netherlands May 2010 Relations
More informationLectures 6, 7 and part of 8
Lectures 6, 7 and part of 8 Uriel Feige April 26, May 3, May 10, 2015 1 Linear programming duality 1.1 The diet problem revisited Recall the diet problem from Lecture 1. There are n foods, m nutrients,
More informationThere are several approaches to solve UBQP, we will now briefly discuss some of them:
3 Related Work There are several approaches to solve UBQP, we will now briefly discuss some of them: Since some of them are actually algorithms for the Max Cut problem (MC), it is necessary to show their
More informationThe moment-lp and moment-sos approaches
The moment-lp and moment-sos approaches LAAS-CNRS and Institute of Mathematics, Toulouse, France CIRM, November 2013 Semidefinite Programming Why polynomial optimization? LP- and SDP- CERTIFICATES of POSITIVITY
More information1 Positive definiteness and semidefiniteness
Positive definiteness and semidefiniteness Zdeněk Dvořák May 9, 205 For integers a, b, and c, let D(a, b, c) be the diagonal matrix with + for i =,..., a, D i,i = for i = a +,..., a + b,. 0 for i = a +
More informationOperations Research. Report Applications of the inverse theta number in stable set problems. February 2011
Operations Research Report 2011-01 Applications of the inverse theta number in stable set problems Miklós Ujvári February 2011 Eötvös Loránd University of Sciences Department of Operations Research Copyright
More informationConvexification of Mixed-Integer Quadratically Constrained Quadratic Programs
Convexification of Mixed-Integer Quadratically Constrained Quadratic Programs Laura Galli 1 Adam N. Letchford 2 Lancaster, April 2011 1 DEIS, University of Bologna, Italy 2 Department of Management Science,
More informationApplications of semidefinite programming, symmetry and algebra to graph partitioning problems
Applications of semidefinite programming, symmetry and algebra to graph partitioning problems Edwin van Dam and Renata Sotirov Tilburg University, The Netherlands Summer 2018 Edwin van Dam (Tilburg) SDP,
More informationIE 521 Convex Optimization
Lecture 14: and Applications 11th March 2019 Outline LP SOCP SDP LP SOCP SDP 1 / 21 Conic LP SOCP SDP Primal Conic Program: min c T x s.t. Ax K b (CP) : b T y s.t. A T y = c (CD) y K 0 Theorem. (Strong
More informationLagrangian-Conic Relaxations, Part I: A Unified Framework and Its Applications to Quadratic Optimization Problems
Lagrangian-Conic Relaxations, Part I: A Unified Framework and Its Applications to Quadratic Optimization Problems Naohiko Arima, Sunyoung Kim, Masakazu Kojima, and Kim-Chuan Toh Abstract. In Part I of
More informationSemidefinite Programming, Combinatorial Optimization and Real Algebraic Geometry
Semidefinite Programming, Combinatorial Optimization and Real Algebraic Geometry assoc. prof., Ph.D. 1 1 UNM - Faculty of information studies Edinburgh, 16. September 2014 Outline Introduction Definition
More information1 Introduction Semidenite programming (SDP) has been an active research area following the seminal work of Nesterov and Nemirovski [9] see also Alizad
Quadratic Maximization and Semidenite Relaxation Shuzhong Zhang Econometric Institute Erasmus University P.O. Box 1738 3000 DR Rotterdam The Netherlands email: zhang@few.eur.nl fax: +31-10-408916 August,
More information