Lecture: Examples of LP, SOCP and SDP
|
|
- Dinah Johnston
- 5 years ago
- Views:
Transcription
1 1/34 Lecture: Examples of LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University Acknowledgement: this slides is based on Prof. Farid Alizadeh and Prof. Lieven Vandenberghe s lecture notes
2 Problems with absolute values 2/34 c i x i, assume c 0 i Ax b Reformulation 1: c i z i i Ax b x i z i c i z i i Ax b z i x i z i Reformulation 2: x i = x i + xi, x i +, xi 0. Then x i = x i + + xi c i (x i + + xi ) i Ax + Ax b, x +, x 0
3 3/34 Problems with absolute values data fitting: Compressive sensing x Ax b x Ax b 1 x 1, Ax = b (LP) µ x Ax + b 2 (QP, SOCP) Ax b, x 1 1
4 4/34 Quadratic Programg (QP) q(x) = x Qx + a x + β assume Q 0, Q = Q Ax = b x 0 q(x) = ū 2 + β 1 4 a Q 1 a, where ū = Q 1/2 x Q 1/2 a. equivalent SOCP u 0 ū = Q 1/2 x Q 1/2 a Ax = b x 0, (u 0, ū) Q 0
5 Quadratic constraints 5/34 q(x) = x B Bx + a x + β 0 is equivalent to where ū = ( Bx ) a x+β+1 2 (u 0, ū) Q 0, and u 0 = 1 a x β 2
6 Norm imization problems 6/34 Let v i = A i x + b i R n i. x i v i is equivalent to i v i0 v i = A i x + b i (v i0, v i ) Q 0 x max 1 i r v i is equivalent to t v i = A i x + b i (t, v i ) Q 0
7 Norm imization problems 7/34 Let v i = A i x + b i R n i. v [1],..., v [r] are the norms v 1,..., v r sorted in nonincreasing order x i v [i] is equivalent to u i + kt i v i = A i x + b i v i u i + t u i 0
8 Rotated Quadratic Cone 8/34 rotated cone w w xy, where x, y 0, is equivalent to ( ) 2w x + y x y Minimize the harmonic mean of positive affine functions 1/(a i x + β i ), a i x + β i > 0 i is equivalent to i u i v i = a i x + β i 1 u i v i u i 0
9 9/34 Logarithmic Tchebychev approximation x max 1 i r ln(a i x) ln b i Since ln(a i x) ln b i = ln max(a i x/b i, b i /a i x), the problem is equivalent to t 1 (a i x/b i )t a i x/b i t t 0 Inequalities involving geometric means ( n 1/n (a i x + b i )) t i=1
10 10/34 n=4 max 4 (a i x b i ) i=1 max w 3 a i x b i 0 (a 1 x b 1 )(a 2 x b 2 ) w 2 1 (a 3 x b 3 )(a 4 x b 4 ) w 2 2 w 1 w 2 w 2 3 w i 0 This can be extended to products of rational powers of affine functions
11 Robust linear programg 11/34 the parameters in LP are often uncertain There can be uncertainty in c, a i, b. c x a i x b i two common approaches to handling uncertainty (in a i, for simplicity) deteristic model: constraints must hold for all a i E i c x a i x b i, for all a i E i stochastic model: a i is random variable; constraints must hold with probability η c x prob(a i x b i ) η
12 deteristic approach via SOCP 12/34 Choose an ellipsoid as E i : E i = {ā i + P i u u 2 1}, ā i R n, P i R n n Robust LP c x a i x b i, for all a i E i is equivalent to the SOCP c x ā i x + P i x 2 b i since sup (ā i + P i u) x = ā i x + P i x 2 u 2 1
13 stochastic approach via SOCP a i is Gaussian with mean ā i, covariance Σ i ( a i N (ā i, Σ i ) a i x is Gaussian r.v. with mean ā i x, variance x Σ i x; hence ( prob(a bi ā ) i x i x b i ) = Φ Σ 1/2 x 2 where Φ(x) = (1/ 2π) x e t2 /2 dt is CDF of N (0, 1) robust LP c x prob(a i x b i ) η is equivalent to the SOCP c x ā i x + Φ 1 (η) Σ 1/2 x 2 b i 13/34
14 14/34 SDP Standard Form S n = {X R n n X = X}, S n + = {X S n X 0}, S n ++ = {X S n X 0} Define linear operator A : S n R m : A(X) = ( A 1, X,..., A m, X ), X S n. Since A(X) y = m i=1 y i A i, X = m i=1 y ia i, X, the adjoint of A: A (y) = m y i A i i=1 The SDP standard form: (P) C, X A(X) = b X 0 (D) max b y A (y) + S = C S 0
15 Facts on matrix calcuation If A, B R m n, then Tr(AB ) = Tr(B A) If U, V S n and Q is orthogonal, then U, V = Q UQ, Q UQ If X S n, then U = Q ΛQ, where Q Q = I and Λ is diagonal. Matrix norms: X F = λ(x) 2, X 2 = λ(x), λ(x) = diag(λ) X 0 v Xv for all v R n λ(x) 0 X = B B The dual cone of S+ n is S+ n If X 0, then X ii 0. If X ii = 0, then X ik = X ki = 0 for all k. If X 0, then PXP 0 for any P of approriate dimensions ( ) X11 X If X = 12 X12 0, then X X X 0 iff every principal submatrix is positive semidefinite (psd). 15/34
16 Facts on matrix calcuation ( ) A B Let U = B with A and C symmetric and A 0. Then C U 0 ( or 0) C B A 1 B 0 ( or 0). The matrix C B A 1 B is the Schur complement of A in U: ( ) ( ) ( ) ( A B I 0 A 0 I A B = 1 ) B C B A 1 I 0 C B A 1 B 0 I If A S n, then x Ax = A, xx If A 0, then A, B > 0 for every nonzero B 0 and {B 0 A, B β} is bounded for β > 0 If A, B 0, then A, B = 0 iff AB = 0 A, B S n, then A and B are commute iff AB is symmetric, iff A and B can be simultaneously diagonalized 16/34
17 17/34 Eigenvalue optimization imizing the largest eigenvalue λ max (A 0 + i x ia i ): λ max (A 0 + i x i A i ) can be expressed as an SDP and its dual is z max A 0, Y zi i x i A i A 0 A i, Y = k I, Y = 1 Y 0 follows from λ max (A) t A ti
18 Eigenvalue optimization 18/34 Let A i R m n. Minimizing the 2-norm of A(x) = A 0 + i x ia i : can be expressed as an SDP x A(x) 2 x,t t ( ) ti A(x) A(x) 0 ti Constraint follows from A 2 t A A t 2 I, t 0 ( ) ti A(x) A(x) 0 ti
19 Eigenvalue optimization 19/34 Let Λ k (A) indicate sum of the k largest eigenvalues of A. Then imizing Λ k (A 0 + i x ia i ): Λ k (A 0 + i x i A i ) can be expressed as an SDP kz + Tr(X) zi + X i x i A i A 0 since X 0 Λ k (A) t t kz Tr(X) 0, zi + X A, X 0
20 20/34 The following problems can be expressed as SDP maximizing sum of the k smallest eigenvalues of A 0 + i x ia i imizing sum of the k absolute-value-wise largest eigenvalues imizing sum of the k largest singular values of A 0 + i x ia i
21 Quadratically Constrained Quadratic Programg 21/34 Consider QCQP x A 0 x + 2b 0 x + c 0 assume A i S n x A i x + 2b i x + c i 0, i = 1,..., m If A 0 0 and A i = B i B i, i = 1,..., m, then it is a SOCP If A i S n but may be indefinite x A i x + 2b i x + c i = A i, xx + 2b i x + c i The original problem is equivalent to TrA 0 X + 2b 0 x + c 0 TrA i X + 2b i x + c i 0, i = 1,..., m X = xx
22 QCQP 22/34 If A i S n but may be indefinite ( x A i x + 2b Ai b i x + c i = i b i c i ), ( ) X x x := Ā 1 i, X X 0 is equivalent to X xx The SDP relaxation is TrA 0 X + 2b 0 x + c 0 TrA i X + 2b i x + c i 0, i = 1,..., m X xx Maxcut: max x Wx, xi 2 = 1 Phase retrieval: a i x = b i, the value of a i x is complex
23 Max cut 23/34 For graph (V, E) and weights w ij = w ji 0, the maxcut problem is (Q) max x 1 2 w ij (1 x i x j ), x i { 1, 1} i<j Relaxation: (P) 1 max v i R n 2 w ij (1 v i v j ), v i 2 = 1 i<j Equivalent SDP of (P): (SDP) 1 max X S n 2 w ij (1 X ij ), X ii = 1, X 0 i<j
24 24/34 Max cut: rounding procedure Goemans and Williamson s randomized approach Solve (SDP) to obtain an optimal solution X. Compute the decomposition X = V V, where V = [v 1, v 2,..., v n ] Generate a vector r uniformly distributed on the unit sphere, i.e., r 2 = 1 Set x i = { 1 v i r 0 1 otherwise
25 Max cut: theoretical results Let W be the objective function value of x and E(W) be the expected value. Then E(W) = 1 w ij arccos(v i v j ) π Goemans and Williamson showed: E(W) α 1 w ij (1 v i v j ) 2 i<j i<j where 2 θ α = 0 θπ π 1 cos θ > Let Z(SDP) and Z (Q) be the optimal values of (SDP) and (Q) E(W) αz (SDP) αz (Q) 25/34
26 SDP-Representablity 26/34 What kind of problems can be expressed by SDP and SOCP? Definition: A set X R n is SDP-representable (or SDP-Rep for short) if it can be expressed linearly as the feasible region of an SDP X = { x there exist u R k such that for some A i, B j, C R m m : x i A i + u j B j + C 0 i j
27 SDP-Representablity 27/34 Definition: A function f (x) is SDP-Rep if its epigraph is SDP-representable epi(f ) = {(x 0, x) f (x) x 0 } If X is SDP-Rep, then x X c x is an SDP If f (x) is SDP-Rep, then x f (x) is an SDP
28 28/34 A calculus of SDP-Rep sets and functions SDP-Rep sets and functions remain so under finitely many applications of most convex-preserving operations. If X and Y are SDP-Rep then so are Minkowski sum X + Y intersection X Y Affine pre-image A 1 (X) if A is affine Affine map A(X) if A is affine Cartesian Product: X Y = {(x, y) x X, y Y}
29 29/34 SDP-Rep Functions If functions f i, i = 1,..., m and g are SDP-Rep. Then the following are SDP-Rep nonnegative sum i α if i for α i 0 maximum max i f i composition: g(f 1 (x),..., f m (x)) if f i : R n R and g : R m R Legendre transform f (y) = max x y x f (x)
30 Positive Polynomials 30/34 The set of nonnegative polynomials of a given degree forms a proper cone P n = {(p 0,..., p n ) p 0 + p 1 t p n t t > 0 for all t I} where I is any of [a, b], [a, ) or (, ) Important fact: The cone of positive polynomials is SDP-Rep To see this we need to introduce another problem
31 The Moment cone 31/34 The Moment space and Moment cone Let (c 0, c 1,..., c n ) be such that there is a probability measure F where c i = t i df, for i = 0,..., n. The set of such vectors is the Moment space I The Moment cone M n = {ac There is a distribution F : c i = t i df and a 0} I
32 The Moment cone 32/34 The moment cone is also SDP-Rep: The discrete Hamburger moment problem: I = R, c M 2n+1 c 0 c 1... c n c 1 c 2... c n c n c n+1... c 2n This is the Hankel matrix
33 The Moment cone 33/34 The discrete Stieltjes moment problem I = [0, ), c M m c 0 c 1... c n c 1 c 2... c n 1 c 1 c 2... c n+1 c 2 c 3... c n , and c n c n+1... c 2n c n 1 c n... c 2n 1 where m = n 2 The Hausdorff moment problem where I = [0, 1] is similarly SDP-Rep
34 Moment and positive polynomial cones 34/34 P n = M n, i.e., i.e. moment cones and nonnegative polynomials are dual of each other If {u 0 (x),..., u n (x)}, x I are linearly independent functions (possibly of several variables) The cone of polynomials that can be expressed as sum of squares is SDP-Rep. if I is a one dimensional set then positive polynomials and sum of square polynomials coincide In general except for I one-dimensional positive polynomials properly include sum of square polynomials
Lecture: Convex Optimization Problems
1/36 Lecture: Convex Optimization Problems http://bicmr.pku.edu.cn/~wenzw/opt-2015-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghe s lecture notes Introduction 2/36 optimization
More information4. Convex optimization problems
Convex Optimization Boyd & Vandenberghe 4. Convex optimization problems optimization problem in standard form convex optimization problems quasiconvex optimization linear optimization quadratic optimization
More informationLecture: Introduction to LP, SDP and SOCP
Lecture: Introduction to LP, SDP and SOCP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2015.html wenzw@pku.edu.cn Acknowledgement:
More informationSemidefinite Programming Basics and Applications
Semidefinite Programming Basics and Applications Ray Pörn, principal lecturer Åbo Akademi University Novia University of Applied Sciences Content What is semidefinite programming (SDP)? How to represent
More information4. Convex optimization problems
Convex Optimization Boyd & Vandenberghe 4. Convex optimization problems optimization problem in standard form convex optimization problems quasiconvex optimization linear optimization quadratic optimization
More informationConvex optimization problems. Optimization problem in standard form
Convex optimization problems optimization problem in standard form convex optimization problems linear optimization quadratic optimization geometric programming quasiconvex optimization generalized inequality
More informationConvex Optimization. (EE227A: UC Berkeley) Lecture 6. Suvrit Sra. (Conic optimization) 07 Feb, 2013
Convex Optimization (EE227A: UC Berkeley) Lecture 6 (Conic optimization) 07 Feb, 2013 Suvrit Sra Organizational Info Quiz coming up on 19th Feb. Project teams by 19th Feb Good if you can mix your research
More informationInterior Point Methods: Second-Order Cone Programming and Semidefinite Programming
School of Mathematics T H E U N I V E R S I T Y O H F E D I N B U R G Interior Point Methods: Second-Order Cone Programming and Semidefinite Programming Jacek Gondzio Email: J.Gondzio@ed.ac.uk URL: http://www.maths.ed.ac.uk/~gondzio
More informationIE 521 Convex Optimization
Lecture 14: and Applications 11th March 2019 Outline LP SOCP SDP LP SOCP SDP 1 / 21 Conic LP SOCP SDP Primal Conic Program: min c T x s.t. Ax K b (CP) : b T y s.t. A T y = c (CD) y K 0 Theorem. (Strong
More informationConvex Optimization and Modeling
Convex Optimization and Modeling Convex Optimization Fourth lecture, 05.05.2010 Jun.-Prof. Matthias Hein Reminder from last time Convex functions: first-order condition: f(y) f(x) + f x,y x, second-order
More informationLecture: Duality of LP, SOCP and SDP
1/33 Lecture: Duality of LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2017.html wenzw@pku.edu.cn Acknowledgement:
More informationIntroduction to Semidefinite Programming I: Basic properties a
Introduction to Semidefinite Programming I: Basic properties and variations on the Goemans-Williamson approximation algorithm for max-cut MFO seminar on Semidefinite Programming May 30, 2010 Semidefinite
More informationELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications
ELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications Professor M. Chiang Electrical Engineering Department, Princeton University March
More informationMIT Algebraic techniques and semidefinite optimization February 14, Lecture 3
MI 6.97 Algebraic techniques and semidefinite optimization February 4, 6 Lecture 3 Lecturer: Pablo A. Parrilo Scribe: Pablo A. Parrilo In this lecture, we will discuss one of the most important applications
More informationLMI MODELLING 4. CONVEX LMI MODELLING. Didier HENRION. LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ. Universidad de Valladolid, SP March 2009
LMI MODELLING 4. CONVEX LMI MODELLING Didier HENRION LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ Universidad de Valladolid, SP March 2009 Minors A minor of a matrix F is the determinant of a submatrix
More informationLecture Note 5: Semidefinite Programming for Stability Analysis
ECE7850: Hybrid Systems:Theory and Applications Lecture Note 5: Semidefinite Programming for Stability Analysis Wei Zhang Assistant Professor Department of Electrical and Computer Engineering Ohio State
More informationLecture 20: November 1st
10-725: Optimization Fall 2012 Lecture 20: November 1st Lecturer: Geoff Gordon Scribes: Xiaolong Shen, Alex Beutel Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer: These notes have not
More informationLecture: Algorithms for LP, SOCP and SDP
1/53 Lecture: Algorithms for LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2018.html wenzw@pku.edu.cn Acknowledgement:
More informationLecture 6: Conic Optimization September 8
IE 598: Big Data Optimization Fall 2016 Lecture 6: Conic Optimization September 8 Lecturer: Niao He Scriber: Juan Xu Overview In this lecture, we finish up our previous discussion on optimality conditions
More informationAdvances in Convex Optimization: Theory, Algorithms, and Applications
Advances in Convex Optimization: Theory, Algorithms, and Applications Stephen Boyd Electrical Engineering Department Stanford University (joint work with Lieven Vandenberghe, UCLA) ISIT 02 ISIT 02 Lausanne
More informationWhat can be expressed via Conic Quadratic and Semidefinite Programming?
What can be expressed via Conic Quadratic and Semidefinite Programming? A. Nemirovski Faculty of Industrial Engineering and Management Technion Israel Institute of Technology Abstract Tremendous recent
More informationCOURSE ON LMI PART I.2 GEOMETRY OF LMI SETS. Didier HENRION henrion
COURSE ON LMI PART I.2 GEOMETRY OF LMI SETS Didier HENRION www.laas.fr/ henrion October 2006 Geometry of LMI sets Given symmetric matrices F i we want to characterize the shape in R n of the LMI set F
More informationCSCI 1951-G Optimization Methods in Finance Part 10: Conic Optimization
CSCI 1951-G Optimization Methods in Finance Part 10: Conic Optimization April 6, 2018 1 / 34 This material is covered in the textbook, Chapters 9 and 10. Some of the materials are taken from it. Some of
More informationEE Applications of Convex Optimization in Signal Processing and Communications Dr. Andre Tkacenko, JPL Third Term
EE 150 - Applications of Convex Optimization in Signal Processing and Communications Dr. Andre Tkacenko JPL Third Term 2011-2012 Due on Thursday May 3 in class. Homework Set #4 1. (10 points) (Adapted
More informationLecture: Duality.
Lecture: Duality http://bicmr.pku.edu.cn/~wenzw/opt-2016-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghe s lecture notes Introduction 2/35 Lagrange dual problem weak and strong
More informationHandout 6: Some Applications of Conic Linear Programming
ENGG 550: Foundations of Optimization 08 9 First Term Handout 6: Some Applications of Conic Linear Programming Instructor: Anthony Man Cho So November, 08 Introduction Conic linear programming CLP, and
More informationLecture 1. 1 Conic programming. MA 796S: Convex Optimization and Interior Point Methods October 8, Consider the conic program. min.
MA 796S: Convex Optimization and Interior Point Methods October 8, 2007 Lecture 1 Lecturer: Kartik Sivaramakrishnan Scribe: Kartik Sivaramakrishnan 1 Conic programming Consider the conic program min s.t.
More informationLecture Semidefinite Programming and Graph Partitioning
Approximation Algorithms and Hardness of Approximation April 16, 013 Lecture 14 Lecturer: Alantha Newman Scribes: Marwa El Halabi 1 Semidefinite Programming and Graph Partitioning In previous lectures,
More informationCOM Optimization for Communications 8. Semidefinite Programming
COM524500 Optimization for Communications 8. Semidefinite Programming Institute Comm. Eng. & Dept. Elect. Eng., National Tsing Hua University 1 Semidefinite Programming () Inequality form: min c T x s.t.
More informationApproximation Algorithms
Approximation Algorithms Chapter 26 Semidefinite Programming Zacharias Pitouras 1 Introduction LP place a good lower bound on OPT for NP-hard problems Are there other ways of doing this? Vector programs
More informationConvex Optimization in Classification Problems
New Trends in Optimization and Computational Algorithms December 9 13, 2001 Convex Optimization in Classification Problems Laurent El Ghaoui Department of EECS, UC Berkeley elghaoui@eecs.berkeley.edu 1
More informationl p -Norm Constrained Quadratic Programming: Conic Approximation Methods
OUTLINE l p -Norm Constrained Quadratic Programming: Conic Approximation Methods Wenxun Xing Department of Mathematical Sciences Tsinghua University, Beijing Email: wxing@math.tsinghua.edu.cn OUTLINE OUTLINE
More informationRelaxations and Randomized Methods for Nonconvex QCQPs
Relaxations and Randomized Methods for Nonconvex QCQPs Alexandre d Aspremont, Stephen Boyd EE392o, Stanford University Autumn, 2003 Introduction While some special classes of nonconvex problems can be
More informationContinuous Optimisation, Chpt 9: Semidefinite Optimisation
Continuous Optimisation, Chpt 9: Semidefinite Optimisation Peter J.C. Dickinson DMMP, University of Twente p.j.c.dickinson@utwente.nl http://dickinson.website/teaching/2017co.html version: 28/11/17 Monday
More informationLinear and non-linear programming
Linear and non-linear programming Benjamin Recht March 11, 2005 The Gameplan Constrained Optimization Convexity Duality Applications/Taxonomy 1 Constrained Optimization minimize f(x) subject to g j (x)
More information12. Interior-point methods
12. Interior-point methods Convex Optimization Boyd & Vandenberghe inequality constrained minimization logarithmic barrier function and central path barrier method feasibility and phase I methods complexity
More informationContinuous Optimisation, Chpt 9: Semidefinite Problems
Continuous Optimisation, Chpt 9: Semidefinite Problems Peter J.C. Dickinson DMMP, University of Twente p.j.c.dickinson@utwente.nl http://dickinson.website/teaching/2016co.html version: 21/11/16 Monday
More informationCSC Linear Programming and Combinatorial Optimization Lecture 10: Semidefinite Programming
CSC2411 - Linear Programming and Combinatorial Optimization Lecture 10: Semidefinite Programming Notes taken by Mike Jamieson March 28, 2005 Summary: In this lecture, we introduce semidefinite programming
More informationORF 523 Lecture 9 Spring 2016, Princeton University Instructor: A.A. Ahmadi Scribe: G. Hall Thursday, March 10, 2016
ORF 523 Lecture 9 Spring 2016, Princeton University Instructor: A.A. Ahmadi Scribe: G. Hall Thursday, March 10, 2016 When in doubt on the accuracy of these notes, please cross check with the instructor
More informationN. L. P. NONLINEAR PROGRAMMING (NLP) deals with optimization models with at least one nonlinear function. NLP. Optimization. Models of following form:
0.1 N. L. P. Katta G. Murty, IOE 611 Lecture slides Introductory Lecture NONLINEAR PROGRAMMING (NLP) deals with optimization models with at least one nonlinear function. NLP does not include everything
More informationConvex Optimization. 4. Convex Optimization Problems. Prof. Ying Cui. Department of Electrical Engineering Shanghai Jiao Tong University
Conve Optimization 4. Conve Optimization Problems Prof. Ying Cui Department of Electrical Engineering Shanghai Jiao Tong University 2017 Autumn Semester SJTU Ying Cui 1 / 58 Outline Optimization problems
More information15. Conic optimization
L. Vandenberghe EE236C (Spring 216) 15. Conic optimization conic linear program examples modeling duality 15-1 Generalized (conic) inequalities Conic inequality: a constraint x K where K is a convex cone
More information6.854J / J Advanced Algorithms Fall 2008
MIT OpenCourseWare http://ocw.mit.edu 6.85J / 8.5J Advanced Algorithms Fall 008 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms. 8.5/6.85 Advanced Algorithms
More informationCS675: Convex and Combinatorial Optimization Fall 2016 Convex Optimization Problems. Instructor: Shaddin Dughmi
CS675: Convex and Combinatorial Optimization Fall 2016 Convex Optimization Problems Instructor: Shaddin Dughmi Outline 1 Convex Optimization Basics 2 Common Classes 3 Interlude: Positive Semi-Definite
More informationSDP Relaxations for MAXCUT
SDP Relaxations for MAXCUT from Random Hyperplanes to Sum-of-Squares Certificates CATS @ UMD March 3, 2017 Ahmed Abdelkader MAXCUT SDP SOS March 3, 2017 1 / 27 Overview 1 MAXCUT, Hardness and UGC 2 LP
More informationSemidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 2
Semidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 2 Instructor: Farid Alizadeh Scribe: Xuan Li 9/17/2001 1 Overview We survey the basic notions of cones and cone-lp and give several
More informationSecond-Order Cone Programming
Second-Order Cone Programming F. Alizadeh D. Goldfarb 1 Introduction Second-order cone programming (SOCP) problems are convex optimization problems in which a linear function is minimized over the intersection
More informationIII. Applications in convex optimization
III. Applications in convex optimization nonsymmetric interior-point methods partial separability and decomposition partial separability first order methods interior-point methods Conic linear optimization
More informationAgenda. Applications of semidefinite programming. 1 Control and system theory. 2 Combinatorial and nonconvex optimization
Agenda Applications of semidefinite programming 1 Control and system theory 2 Combinatorial and nonconvex optimization 3 Spectral estimation & super-resolution Control and system theory SDP in wide use
More informationCanonical Problem Forms. Ryan Tibshirani Convex Optimization
Canonical Problem Forms Ryan Tibshirani Convex Optimization 10-725 Last time: optimization basics Optimization terology (e.g., criterion, constraints, feasible points, solutions) Properties and first-order
More informationThere are several approaches to solve UBQP, we will now briefly discuss some of them:
3 Related Work There are several approaches to solve UBQP, we will now briefly discuss some of them: Since some of them are actually algorithms for the Max Cut problem (MC), it is necessary to show their
More informationConvex relaxation. In example below, we have N = 6, and the cut we are considering
Convex relaxation The art and science of convex relaxation revolves around taking a non-convex problem that you want to solve, and replacing it with a convex problem which you can actually solve the solution
More informationHW1 solutions. 1. α Ef(x) β, where Ef(x) is the expected value of f(x), i.e., Ef(x) = n. i=1 p if(a i ). (The function f : R R is given.
HW1 solutions Exercise 1 (Some sets of probability distributions.) Let x be a real-valued random variable with Prob(x = a i ) = p i, i = 1,..., n, where a 1 < a 2 < < a n. Of course p R n lies in the standard
More informationSemidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 5
Semidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 5 Instructor: Farid Alizadeh Scribe: Anton Riabov 10/08/2001 1 Overview We continue studying the maximum eigenvalue SDP, and generalize
More informationQuadratic reformulation techniques for 0-1 quadratic programs
OSE SEMINAR 2014 Quadratic reformulation techniques for 0-1 quadratic programs Ray Pörn CENTER OF EXCELLENCE IN OPTIMIZATION AND SYSTEMS ENGINEERING ÅBO AKADEMI UNIVERSITY ÅBO NOVEMBER 14th 2014 2 Structure
More informationHandout 6: Some Applications of Conic Linear Programming
G00E04: Convex Optimization and Its Applications in Signal Processing Handout 6: Some Applications of Conic Linear Programming Instructor: Anthony Man Cho So June 3, 05 Introduction Conic linear programming
More informationPreliminaries Overview OPF and Extensions. Convex Optimization. Lecture 8 - Applications in Smart Grids. Instructor: Yuanzhang Xiao
Convex Optimization Lecture 8 - Applications in Smart Grids Instructor: Yuanzhang Xiao University of Hawaii at Manoa Fall 2017 1 / 32 Today s Lecture 1 Generalized Inequalities and Semidefinite Programming
More informationSemidefinite Programming
Semidefinite Programming Basics and SOS Fernando Mário de Oliveira Filho Campos do Jordão, 2 November 23 Available at: www.ime.usp.br/~fmario under talks Conic programming V is a real vector space h, i
More informationLecture 7: Positive Semidefinite Matrices
Lecture 7: Positive Semidefinite Matrices Rajat Mittal IIT Kanpur The main aim of this lecture note is to prepare your background for semidefinite programming. We have already seen some linear algebra.
More information3. Convex functions. basic properties and examples. operations that preserve convexity. the conjugate function. quasiconvex functions
3. Convex functions Convex Optimization Boyd & Vandenberghe basic properties and examples operations that preserve convexity the conjugate function quasiconvex functions log-concave and log-convex functions
More informationConvex Optimization M2
Convex Optimization M2 Lecture 8 A. d Aspremont. Convex Optimization M2. 1/57 Applications A. d Aspremont. Convex Optimization M2. 2/57 Outline Geometrical problems Approximation problems Combinatorial
More informationConvex relaxation. In example below, we have N = 6, and the cut we are considering
Convex relaxation The art and science of convex relaxation revolves around taking a non-convex problem that you want to solve, and replacing it with a convex problem which you can actually solve the solution
More informationEE5138R: Problem Set 5 Assigned: 16/02/15 Due: 06/03/15
EE538R: Problem Set 5 Assigned: 6/0/5 Due: 06/03/5. BV Problem 4. The feasible set is the convex hull of (0, ), (0, ), (/5, /5), (, 0) and (, 0). (a) x = (/5, /5) (b) Unbounded below (c) X opt = {(0, x
More informationSemidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 4
Semidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 4 Instructor: Farid Alizadeh Scribe: Haengju Lee 10/1/2001 1 Overview We examine the dual of the Fermat-Weber Problem. Next we will
More informationApplications of Robust Optimization in Signal Processing: Beamforming and Power Control Fall 2012
Applications of Robust Optimization in Signal Processing: Beamforg and Power Control Fall 2012 Instructor: Farid Alizadeh Scribe: Shunqiao Sun 12/09/2012 1 Overview In this presentation, we study the applications
More informationConvex Optimization M2
Convex Optimization M2 Lecture 3 A. d Aspremont. Convex Optimization M2. 1/49 Duality A. d Aspremont. Convex Optimization M2. 2/49 DMs DM par email: dm.daspremont@gmail.com A. d Aspremont. Convex Optimization
More informationSecond-order cone programming
Math. Program., Ser. B 95: 3 51 (2003) Digital Object Identifier (DOI) 10.1007/s10107-002-0339-5 F. Alizadeh D. Goldfarb Second-order cone programming Received: August 18, 2001 / Accepted: February 27,
More informationELE539A: Optimization of Communication Systems Lecture 6: Quadratic Programming, Geometric Programming, and Applications
ELE539A: Optimization of Communication Systems Lecture 6: Quadratic Programming, Geometric Programming, and Applications Professor M. Chiang Electrical Engineering Department, Princeton University February
More informationSemidefinite and Second Order Cone Programming Seminar Fall 2012 Project: Robust Optimization and its Application of Robust Portfolio Optimization
Semidefinite and Second Order Cone Programming Seminar Fall 2012 Project: Robust Optimization and its Application of Robust Portfolio Optimization Instructor: Farid Alizadeh Author: Ai Kagawa 12/12/2012
More information3. Convex functions. basic properties and examples. operations that preserve convexity. the conjugate function. quasiconvex functions
3. Convex functions Convex Optimization Boyd & Vandenberghe basic properties and examples operations that preserve convexity the conjugate function quasiconvex functions log-concave and log-convex functions
More informationCommon-Knowledge / Cheat Sheet
CSE 521: Design and Analysis of Algorithms I Fall 2018 Common-Knowledge / Cheat Sheet 1 Randomized Algorithm Expectation: For a random variable X with domain, the discrete set S, E [X] = s S P [X = s]
More information12. Interior-point methods
12. Interior-point methods Convex Optimization Boyd & Vandenberghe inequality constrained minimization logarithmic barrier function and central path barrier method feasibility and phase I methods complexity
More informationLecture: Cone programming. Approximating the Lorentz cone.
Strong relaxations for discrete optimization problems 10/05/16 Lecture: Cone programming. Approximating the Lorentz cone. Lecturer: Yuri Faenza Scribes: Igor Malinović 1 Introduction Cone programming is
More informationNonlinear Programming Models
Nonlinear Programming Models Fabio Schoen 2008 http://gol.dsi.unifi.it/users/schoen Nonlinear Programming Models p. Introduction Nonlinear Programming Models p. NLP problems minf(x) x S R n Standard form:
More informationConvexification of Mixed-Integer Quadratically Constrained Quadratic Programs
Convexification of Mixed-Integer Quadratically Constrained Quadratic Programs Laura Galli 1 Adam N. Letchford 2 Lancaster, April 2011 1 DEIS, University of Bologna, Italy 2 Department of Management Science,
More informationLecture 4: September 12
10-725/36-725: Conve Optimization Fall 2016 Lecture 4: September 12 Lecturer: Ryan Tibshirani Scribes: Jay Hennig, Yifeng Tao, Sriram Vasudevan Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer:
More informationAnalysis and Linear Algebra. Lectures 1-3 on the mathematical tools that will be used in C103
Analysis and Linear Algebra Lectures 1-3 on the mathematical tools that will be used in C103 Set Notation A, B sets AcB union A1B intersection A\B the set of objects in A that are not in B N. Empty set
More informationSymmetric Matrices and Eigendecomposition
Symmetric Matrices and Eigendecomposition Robert M. Freund January, 2014 c 2014 Massachusetts Institute of Technology. All rights reserved. 1 2 1 Symmetric Matrices and Convexity of Quadratic Functions
More informationLinear Algebra & Analysis Review UW EE/AA/ME 578 Convex Optimization
Linear Algebra & Analysis Review UW EE/AA/ME 578 Convex Optimization January 9, 2015 1 Notation 1. Book pages without explicit citation refer to [1]. 2. R n denotes the set of real n-column vectors. 3.
More informationminimize x x2 2 x 1x 2 x 1 subject to x 1 +2x 2 u 1 x 1 4x 2 u 2, 5x 1 +76x 2 1,
4 Duality 4.1 Numerical perturbation analysis example. Consider the quadratic program with variables x 1, x 2, and parameters u 1, u 2. minimize x 2 1 +2x2 2 x 1x 2 x 1 subject to x 1 +2x 2 u 1 x 1 4x
More informationL. Vandenberghe EE236C (Spring 2016) 18. Symmetric cones. definition. spectral decomposition. quadratic representation. log-det barrier 18-1
L. Vandenberghe EE236C (Spring 2016) 18. Symmetric cones definition spectral decomposition quadratic representation log-det barrier 18-1 Introduction This lecture: theoretical properties of the following
More informationSEMIDEFINITE PROGRAM BASICS. Contents
SEMIDEFINITE PROGRAM BASICS BRIAN AXELROD Abstract. A introduction to the basics of Semidefinite programs. Contents 1. Definitions and Preliminaries 1 1.1. Linear Algebra 1 1.2. Convex Analysis (on R n
More information5. Duality. Lagrangian
5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationAgenda. 1 Cone programming. 2 Convex cones. 3 Generalized inequalities. 4 Linear programming (LP) 5 Second-order cone programming (SOCP)
Agenda 1 Cone programming 2 Convex cones 3 Generalized inequalities 4 Linear programming (LP) 5 Second-order cone programming (SOCP) 6 Semidefinite programming (SDP) 7 Examples Optimization problem in
More informationConvex Functions. Daniel P. Palomar. Hong Kong University of Science and Technology (HKUST)
Convex Functions Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470 - Convex Optimization Fall 2017-18, HKUST, Hong Kong Outline of Lecture Definition convex function Examples
More informationCS295: Convex Optimization. Xiaohui Xie Department of Computer Science University of California, Irvine
CS295: Convex Optimization Xiaohui Xie Department of Computer Science University of California, Irvine Course information Prerequisites: multivariate calculus and linear algebra Textbook: Convex Optimization
More informationConvex Optimization and l 1 -minimization
Convex Optimization and l 1 -minimization Sangwoon Yun Computational Sciences Korea Institute for Advanced Study December 11, 2009 2009 NIMS Thematic Winter School Outline I. Convex Optimization II. l
More informationConstrained optimization
Constrained optimization DS-GA 1013 / MATH-GA 2824 Optimization-based Data Analysis http://www.cims.nyu.edu/~cfgranda/pages/obda_fall17/index.html Carlos Fernandez-Granda Compressed sensing Convex constrained
More informationElementary linear algebra
Chapter 1 Elementary linear algebra 1.1 Vector spaces Vector spaces owe their importance to the fact that so many models arising in the solutions of specific problems turn out to be vector spaces. The
More informationA Quick Tour of Linear Algebra and Optimization for Machine Learning
A Quick Tour of Linear Algebra and Optimization for Machine Learning Masoud Farivar January 8, 2015 1 / 28 Outline of Part I: Review of Basic Linear Algebra Matrices and Vectors Matrix Multiplication Operators
More informationComparing Convex Relaxations for Quadratically Constrained Quadratic Programming
Comparing Convex Relaxations for Quadratically Constrained Quadratic Programming Kurt M. Anstreicher Dept. of Management Sciences University of Iowa European Workshop on MINLP, Marseille, April 2010 The
More informationOptimisation convexe: performance, complexité et applications.
Optimisation convexe: performance, complexité et applications. Introduction, convexité, dualité. A. d Aspremont. M2 OJME: Optimisation convexe. 1/128 Today Convex optimization: introduction Course organization
More informationEE 227A: Convex Optimization and Applications October 14, 2008
EE 227A: Convex Optimization and Applications October 14, 2008 Lecture 13: SDP Duality Lecturer: Laurent El Ghaoui Reading assignment: Chapter 5 of BV. 13.1 Direct approach 13.1.1 Primal problem Consider
More informationNonconvex Quadratic Programming: Return of the Boolean Quadric Polytope
Nonconvex Quadratic Programming: Return of the Boolean Quadric Polytope Kurt M. Anstreicher Dept. of Management Sciences University of Iowa Seminar, Chinese University of Hong Kong, October 2009 We consider
More informationLargest dual ellipsoids inscribed in dual cones
Largest dual ellipsoids inscribed in dual cones M. J. Todd June 23, 2005 Abstract Suppose x and s lie in the interiors of a cone K and its dual K respectively. We seek dual ellipsoidal norms such that
More informationMath Camp Lecture 4: Linear Algebra. Xiao Yu Wang. Aug 2010 MIT. Xiao Yu Wang (MIT) Math Camp /10 1 / 88
Math Camp 2010 Lecture 4: Linear Algebra Xiao Yu Wang MIT Aug 2010 Xiao Yu Wang (MIT) Math Camp 2010 08/10 1 / 88 Linear Algebra Game Plan Vector Spaces Linear Transformations and Matrices Determinant
More information13. Convex programming
CS/ISyE/ECE 524 Introduction to Optimization Spring 2015 16 13. Convex programming Convex sets and functions Convex programs Hierarchy of complexity Example: geometric programming Laurent Lessard (www.laurentlessard.com)
More informationConvex Optimization Boyd & Vandenberghe. 5. Duality
5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationThe proximal mapping
The proximal mapping http://bicmr.pku.edu.cn/~wenzw/opt-2016-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghes lecture notes Outline 2/37 1 closed function 2 Conjugate function
More informationLecture 7: Weak Duality
EE 227A: Conve Optimization and Applications February 7, 2012 Lecture 7: Weak Duality Lecturer: Laurent El Ghaoui 7.1 Lagrange Dual problem 7.1.1 Primal problem In this section, we consider a possibly
More information