# Lecture 5. The Dual Cone and Dual Problem

Size: px
Start display at page:

## Transcription

1 IE Lecture 5. The Dual Cone and Dual Problem

2 IE For a convex cone K, its dual cone is defined as K = {y x, y 0, x K}. The inner-product can be replaced by x T y if the coordinates of the vectors are given. Clearly, K is always a closed set, regardless if K is closed or not.

3 IE In light of the dual cone, the problem of finding a lower bound on the optimal value for the standard conic optimization problem can be done as follows. Introduce the Lagrangian multiplier y for the equality constraint b Ax = 0, and s for the conic constraint x K. The objective becomes c T x + y T (b Ax) s T x = y T b + (c A T y s) T x. This will be a lower bound on the optimal value whenever c A T y s = 0 and s K. The quest to find the best lower bound leads to the dual problem for the standard conic optimization problem: maximize subject to b T y A T y + s = c s K. (1)

4 IE As an example, for the standard SDP problem (primal), minimize subject to C X A i X = b i, i = 1,..., m X 0, its dual problem is in the form of maximize subject to m i=1 b iy i m i=1 y ia i + S = C S 0.

5 IE The so-called weak duality relationship is immediate, by the construction of the dual problem. This is shown in the next theorem. Theorem 1 Let x be feasible for the primal problem, and (y, s) be feasible to its dual. Then, b T y c T x. Moreover, if b T y = c T x then x T s = 0, and they are both optimal to their own respective problems. If a pair of primal-dual feasible solutions x and (y, s) both exist and they satisfy x T s = 0 then we call them a pair of complementary optimal solutions.

6 IE A very useful theorem is the following. Theorem 2 Let x be feasible for the primal problem, and (y, s) be feasible to its dual, and x int K and s int K. Then, the sets of optimal solutions for both problems are nonempty and compact, and any pair of primal-dual optimal solutions will be complementary to each other.

7 IE Let us take some moments to study the structures of the dual cones. Let us start with the topological structures. If rel (K) is denoted as the relative interior of K, then (rel (K)) = K. Let the summation of two sets A and B be A + B = {z z = x + y with x A, y B}. It is clear that the sum of two convex sets remains convex. However, the sum of two closed convex sets may not be closed any more.

8 IE Example 1 The set R SOC(3) is convex but not closed. For instance, the point (0, 1, 0) T is not in the set, but it is in the closure of the set, since for any ϵ > 0, we may write 1 ϵ 1 = 1 1 ϵ ϵ ϵ 1, 0 1 where the second term is in SOC(3). 1 ϵ

9 IE However, if A and B are both polyhedra, then A + B remains a polyhedron, hence closed. It is interesting to note that the set-summation and the set-intersection operations are a duality pair: Proposition 1 Suppose that K 1 and K 2 are convex cones. It holds that (K 1 + K 2 ) = K 1 K 2. Proof. For y (K 1 + K 2 ), it follows that 0 (x 1 + x 2 ) T y for all x 1 K 1 and x 2 K 2. Since 0 K 2, it follows that 0 (x 1 ) T y for all x 1 K 1, thus y K 1, and similarly y K 2. Hence, y K 1 K 2. Now, take any y K 1 K 2, and any x 1 K 1 and x 2 K 2. We have (x 1 + x 2 ) T y = (x 1 ) T y + (x 2 ) T y 0, and so y (K 1 + K 2 ). This proves (K 1 + K 2 ) = K 1 K 2 as desired.

10 IE The following bipolar theorem can be considered as a cornerstone for convex analysis. Theorem 3 Suppose that K is a convex cone. Then (K ) = cl K. Proof. For any x K, by the definition of the dual cone, x T y 0 for all y K, and so K (K ). Therefore cl K (K ). The other containing relationship (K ) cl K can be shown by contradiction. Suppose such is not true then there exists z (K ) \ cl K. By the separation theorem for convex cones, there is vector c such that c T z < 0, and c T x 0 x cl K. The last relation suggests that c K, which contradicts with the first relation because, as z (K ), it should lead to c T z 0, but not the opposite. This contradiction completes the proof.

11 IE This proof suggests that the bipolar theorem for convex cone is equivalent to the separation theorem itself. Suppose that K is a closed convex cone, and v K. Then, the bipolar theorem asserts that v (K ), and this implies that there is y K such that v T y < 0, and x T y 0 for any x K because y K, hence a separation exists.

12 IE In case K is closed, the bipolar theorem is what in folklore understood as the dual of the dual is the primal itself again. Combining Theorem 3 and Proposition 1, it follows that cl (K 1 + K 2 ) = (K 1 K 2 ). (2) It is also useful to compute the effect of linear transformation on a convex cone in the context of dual operation. Let A be an invertible matrix. Then, we have (A T K) = A 1 K, (3) by observing that y (A T K) (A T x) T y 0 x K Ay K y A 1 K.

13 IE A convex cone is called solid if it is full dimensional. Mathematically, K R n is solid span K = R n, or K + ( K) = R n. A convex cone is called pointed if it does not contain any nontrivial subspace; i.e. K ( K) = {0}. The following result proves to be useful: Proposition 2 If K is solid then K is pointed, and if K is pointed then K is solid.

14 IE Proof. Suppose that K is solid, i.e. there is a ball B(x 0 ; δ) K with radius δ > 0 and centered at x 0, and at the same suppose that there is a line Rc K, then 0 s(x 0 + t) T c, t δ and s R, which is clearly impossible. This contradiction shows that if K is solid then K must be pointed. By the bipolar theorem, the second part follows by symmetry.

15 IE It is easy to compute the following dual cones (R n +) = R n +. SOC(n) = SOC(n). (S n n + ) = S n n +, and (H n n + ) = H n n +. The above standard cones are so regular that their corresponding dual cones coincide with the original cones themselves (the primal cones). In general, they are not the same. As an example, consider the following L p cone, where p 1, L p (n + 1) = t x t R, x Rn, t x p.

16 IE One can show that (L p (n + 1)) = L q (n + 1) where 1/p + 1/q = 1. Now, the self-duality of SOC(n + 1) is due to the fact that SOC(n + 1) = L 2 (n + 1). In convex analysis, the dual object for a convex function f(x) is known as the conjugate of f(x), defined as f (s) = sup{( s) T x f(x) x dom f}, where dom f stands for the domain of the function f. The conjugate function is also known as the Legendre-Fenchel transformation.

17 IE Examples of some popular conjugate functions are: 1. For f(x) = 1 2 xt Qx + b T x where Q 0, the conjugate function is f (s) = 1 2 (s + b)t Q 1 (s + b). 2. For f(x) = n i=1 ex i, the conjugate function is f (s) = n i=1 s i log( s i ) + n i=1 s i. 3. For f(x) = n i=1 x i log x i, the conjugate function is f (s) = n i=1 e1+s i. 4. For f(x) = c T x, the conjugate function is f (s) = 0 for s = c and f (s) = + when s c. 5. For f(x) = n i=1 log x i, the conjugate function is f (s) = n n i=1 log s i. 6. For f(x) = max 1 i n x i, the conjugate function is f (s) = 0 if n i=1 s i 1, and f (s) = + otherwise.

18 IE If f(x) is strictly convex and differentiable, then f (s) is also strictly convex and differentiable. The famous bi-conjugate theorem states that f = cl f. There is certainly a relationship between the two dual objects: the dual cone and the conjugate function. Let us now set out to find the exact relationship. First of all, suppose that f is a convex function, mapping from its domain in R n to R. Its epigraphy is a set in R n+1 defined as epi (f) := t t f(x), t R, x dom (f) x.

19 IE It is well known that f is a convex function if and only if epi (f) is a convex set. It turns out that H (epi (f )) is the dual of H (epi (f)), after taking the closure and reorienting itself. To be precise, we have: Theorem 4 Suppose that f is a convex function, and p K := cl q p > 0, q pf(x/p) 0. x Then, K = cl u v s v > 0, u vf (s/v) 0.

20 IE Let L 2 := 0, 1 1, 0 and L := L 2 I n = [L 2, 0 2,n ; 0 n,2, I n ]. The effect of applying L to x is to swap the position of x 1 and x 2. Equivalently, Theorem 4 can be stated as (H (epi (f))) = cl L (H (epi (f ))). (4)

21 IE In linear programming, it is known that whenever the primal and the dual problems are both feasible then complementary solutions must exist. This important property, however, is lost, for a general conic optimization model, as the following example shows. Consider minimize x 11 subject to x 12 + x 21 = 1 x 11, x 12 x 21, x This problem is clearly feasible. However, there is no attainable optimal solution.

22 IE Its dual problem is maximize subject to y y 0, 1 1, 0 + s 11, s 12 s 21, s 22 = 1, 0 0, 0 s 11, s 12 s 21, s 22 0, or, more explicitly, maximize subject to y 1, y y, 0 0. The dual problem is feasible and has an optimal solution; however, a pair of primal-dual complementary optimal solutions fail to exist, although both primal and dual problems are feasible.

23 IE One may guess that the unfortunate situation was caused by the non-attainable optimal solution. Next example shows that even if both primal and dual problems are nicely feasible with attainable optimal solutions, there might still be a positive duality gap, meaning that the complementarity condition does not hold. minimize x 12 x x 33 subject to x 12 + x 21 + x 33 = 1 x 22 = 0 x 11, x 12, x 13 x 21, x 22, x 23 x 31, x 32, x 33 0.

24 IE The dual of this problem is maximize y 1 subject to 0, 1 y 1, 0 1 y 1, y 2, 0 0, 0, 2 y 1 0. The primal problem has an optimal solution, e.g. 1, 0, 0 X = 0, 0, 0, with the optimal value equal to 2. The dual 0, 0, 1 problem also is feasible, and has an optimal solution e.g. (y1, y2) = ( 1, 1), with optimal value equal to 1. Both problems are feasible and have attainable optimal solutions; however, their optimal solutions are not complementary to each other, and there is a positive duality gap: c T x b T y = (x ) T s > 0.

25 IE Key References: Yu. Nesterov and A. Nemirovsky, Interior Point Polynomial Methods in Convex Programming, Studies in Applied Mathematics 13, SIAM, J.F. Sturm, Theory and Algorithms for Semidefinite Programming. In High Performance Optimization, eds. H. Frenk, K. Roos, T. Terlaky, and S. Zhang, Kluwer Academic Publishers, S. Zhang, A New Self-Dual Embedding Method for Convex Programming, Journal of Global Optimization, 29, , 2004.

### Lecture 5. Theorems of Alternatives and Self-Dual Embedding

IE 8534 1 Lecture 5. Theorems of Alternatives and Self-Dual Embedding IE 8534 2 A system of linear equations may not have a solution. It is well known that either Ax = c has a solution, or A T y = 0, c

### 4. Algebra and Duality

4-1 Algebra and Duality P. Parrilo and S. Lall, CDC 2003 2003.12.07.01 4. Algebra and Duality Example: non-convex polynomial optimization Weak duality and duality gap The dual is not intrinsic The cone

### Lecture 6: Conic Optimization September 8

IE 598: Big Data Optimization Fall 2016 Lecture 6: Conic Optimization September 8 Lecturer: Niao He Scriber: Juan Xu Overview In this lecture, we finish up our previous discussion on optimality conditions

### Convex Optimization M2

Convex Optimization M2 Lecture 3 A. d Aspremont. Convex Optimization M2. 1/49 Duality A. d Aspremont. Convex Optimization M2. 2/49 DMs DM par email: dm.daspremont@gmail.com A. d Aspremont. Convex Optimization

### Example: feasibility. Interpretation as formal proof. Example: linear inequalities and Farkas lemma

4-1 Algebra and Duality P. Parrilo and S. Lall 2006.06.07.01 4. Algebra and Duality Example: non-convex polynomial optimization Weak duality and duality gap The dual is not intrinsic The cone of valid

### A New Self-Dual Embedding Method for Convex Programming

A New Self-Dual Embedding Method for Convex Programming Shuzhong Zhang October 2001; revised October 2002 Abstract In this paper we introduce a conic optimization formulation to solve constrained convex

### 15. Conic optimization

L. Vandenberghe EE236C (Spring 216) 15. Conic optimization conic linear program examples modeling duality 15-1 Generalized (conic) inequalities Conic inequality: a constraint x K where K is a convex cone

### Supplement: Universal Self-Concordant Barrier Functions

IE 8534 1 Supplement: Universal Self-Concordant Barrier Functions IE 8534 2 Recall that a self-concordant barrier function for K is a barrier function satisfying 3 F (x)[h, h, h] 2( 2 F (x)[h, h]) 3/2,

### Lecture 1. 1 Conic programming. MA 796S: Convex Optimization and Interior Point Methods October 8, Consider the conic program. min.

MA 796S: Convex Optimization and Interior Point Methods October 8, 2007 Lecture 1 Lecturer: Kartik Sivaramakrishnan Scribe: Kartik Sivaramakrishnan 1 Conic programming Consider the conic program min s.t.

### Conic Linear Optimization and its Dual. yyye

Conic Linear Optimization and Appl. MS&E314 Lecture Note #04 1 Conic Linear Optimization and its Dual Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A.

### Extreme Abridgment of Boyd and Vandenberghe s Convex Optimization

Extreme Abridgment of Boyd and Vandenberghe s Convex Optimization Compiled by David Rosenberg Abstract Boyd and Vandenberghe s Convex Optimization book is very well-written and a pleasure to read. The

### Convex Optimization & Lagrange Duality

Convex Optimization & Lagrange Duality Chee Wei Tan CS 8292 : Advanced Topics in Convex Optimization and its Applications Fall 2010 Outline Convex optimization Optimality condition Lagrange duality KKT

### On duality gap in linear conic problems

On duality gap in linear conic problems C. Zălinescu Abstract In their paper Duality of linear conic problems A. Shapiro and A. Nemirovski considered two possible properties (A) and (B) for dual linear

### Course Notes for EE227C (Spring 2018): Convex Optimization and Approximation

Course Notes for EE227C (Spring 2018): Convex Optimization and Approximation Instructor: Moritz Hardt Email: hardt+ee227c@berkeley.edu Graduate Instructor: Max Simchowitz Email: msimchow+ee227c@berkeley.edu

### Optimality Conditions for Constrained Optimization

72 CHAPTER 7 Optimality Conditions for Constrained Optimization 1. First Order Conditions In this section we consider first order optimality conditions for the constrained problem P : minimize f 0 (x)

### Lecture: Duality of LP, SOCP and SDP

1/33 Lecture: Duality of LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2017.html wenzw@pku.edu.cn Acknowledgement:

### On duality theory of conic linear problems

On duality theory of conic linear problems Alexander Shapiro School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, Georgia 3332-25, USA e-mail: ashapiro@isye.gatech.edu

### Optimization for Machine Learning

Optimization for Machine Learning (Problems; Algorithms - A) SUVRIT SRA Massachusetts Institute of Technology PKU Summer School on Data Science (July 2017) Course materials http://suvrit.de/teaching.html

### Global Optimization of Polynomials

Semidefinite Programming Lecture 9 OR 637 Spring 2008 April 9, 2008 Scribe: Dennis Leventhal Global Optimization of Polynomials Recall we were considering the problem min z R n p(z) where p(z) is a degree

### Exact SDP Relaxations for Classes of Nonlinear Semidefinite Programming Problems

Exact SDP Relaxations for Classes of Nonlinear Semidefinite Programming Problems V. Jeyakumar and G. Li Revised Version:August 31, 2012 Abstract An exact semidefinite linear programming (SDP) relaxation

### IE 521 Convex Optimization

Lecture 1: 16th January 2019 Outline 1 / 20 Which set is different from others? Figure: Four sets 2 / 20 Which set is different from others? Figure: Four sets 3 / 20 Interior, Closure, Boundary Definition.

### Lagrangian Duality Theory

Lagrangian Duality Theory Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye Chapter 14.1-4 1 Recall Primal and Dual

### Lecture: Introduction to LP, SDP and SOCP

Lecture: Introduction to LP, SDP and SOCP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2015.html wenzw@pku.edu.cn Acknowledgement:

### LECTURE 13 LECTURE OUTLINE

LECTURE 13 LECTURE OUTLINE Problem Structures Separable problems Integer/discrete problems Branch-and-bound Large sum problems Problems with many constraints Conic Programming Second Order Cone Programming

### LECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE

LECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE CONVEX ANALYSIS AND DUALITY Basic concepts of convex analysis Basic concepts of convex optimization Geometric duality framework - MC/MC Constrained optimization

### POLYNOMIAL OPTIMIZATION WITH SUMS-OF-SQUARES INTERPOLANTS

POLYNOMIAL OPTIMIZATION WITH SUMS-OF-SQUARES INTERPOLANTS Sercan Yıldız syildiz@samsi.info in collaboration with Dávid Papp (NCSU) OPT Transition Workshop May 02, 2017 OUTLINE Polynomial optimization and

### 5. Duality. Lagrangian

5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized

### Strong duality in Lasserre s hierarchy for polynomial optimization

Strong duality in Lasserre s hierarchy for polynomial optimization arxiv:1405.7334v1 [math.oc] 28 May 2014 Cédric Josz 1,2, Didier Henrion 3,4,5 Draft of January 24, 2018 Abstract A polynomial optimization

### Optimality Conditions for Nonsmooth Convex Optimization

Optimality Conditions for Nonsmooth Convex Optimization Sangkyun Lee Oct 22, 2014 Let us consider a convex function f : R n R, where R is the extended real field, R := R {, + }, which is proper (f never

### A Simple Derivation of a Facial Reduction Algorithm and Extended Dual Systems

A Simple Derivation of a Facial Reduction Algorithm and Extended Dual Systems Gábor Pataki gabor@unc.edu Dept. of Statistics and OR University of North Carolina at Chapel Hill Abstract The Facial Reduction

### Robust Farkas Lemma for Uncertain Linear Systems with Applications

Robust Farkas Lemma for Uncertain Linear Systems with Applications V. Jeyakumar and G. Li Revised Version: July 8, 2010 Abstract We present a robust Farkas lemma, which provides a new generalization of

### Optimization and Optimal Control in Banach Spaces

Optimization and Optimal Control in Banach Spaces Bernhard Schmitzer October 19, 2017 1 Convex non-smooth optimization with proximal operators Remark 1.1 (Motivation). Convex optimization: easier to solve,

### New stopping criteria for detecting infeasibility in conic optimization

Optimization Letters manuscript No. (will be inserted by the editor) New stopping criteria for detecting infeasibility in conic optimization Imre Pólik Tamás Terlaky Received: March 21, 2008/ Accepted:

### Lecture 8. Strong Duality Results. September 22, 2008

Strong Duality Results September 22, 2008 Outline Lecture 8 Slater Condition and its Variations Convex Objective with Linear Inequality Constraints Quadratic Objective over Quadratic Constraints Representation

### Lecture 3: Semidefinite Programming

Lecture 3: Semidefinite Programming Lecture Outline Part I: Semidefinite programming, examples, canonical form, and duality Part II: Strong Duality Failure Examples Part III: Conditions for strong duality

### MAT-INF4110/MAT-INF9110 Mathematical optimization

MAT-INF4110/MAT-INF9110 Mathematical optimization Geir Dahl August 20, 2013 Convexity Part IV Chapter 4 Representation of convex sets different representations of convex sets, boundary polyhedra and polytopes:

### Inequality Constraints

Chapter 2 Inequality Constraints 2.1 Optimality Conditions Early in multivariate calculus we learn the significance of differentiability in finding minimizers. In this section we begin our study of the

### Convex Optimization Theory. Chapter 5 Exercises and Solutions: Extended Version

Convex Optimization Theory Chapter 5 Exercises and Solutions: Extended Version Dimitri P. Bertsekas Massachusetts Institute of Technology Athena Scientific, Belmont, Massachusetts http://www.athenasc.com

### 1 Review of last lecture and introduction

Semidefinite Programming Lecture 10 OR 637 Spring 2008 April 16, 2008 (Wednesday) Instructor: Michael Jeremy Todd Scribe: Yogeshwer (Yogi) Sharma 1 Review of last lecture and introduction Let us first

### I.3. LMI DUALITY. Didier HENRION EECI Graduate School on Control Supélec - Spring 2010

I.3. LMI DUALITY Didier HENRION henrion@laas.fr EECI Graduate School on Control Supélec - Spring 2010 Primal and dual For primal problem p = inf x g 0 (x) s.t. g i (x) 0 define Lagrangian L(x, z) = g 0

### Lecture 8 Plus properties, merit functions and gap functions. September 28, 2008

Lecture 8 Plus properties, merit functions and gap functions September 28, 2008 Outline Plus-properties and F-uniqueness Equation reformulations of VI/CPs Merit functions Gap merit functions FP-I book:

### CSCI 1951-G Optimization Methods in Finance Part 01: Linear Programming

CSCI 1951-G Optimization Methods in Finance Part 01: Linear Programming January 26, 2018 1 / 38 Liability/asset cash-flow matching problem Recall the formulation of the problem: max w c 1 + p 1 e 1 = 150

### Lecture 9 Monotone VIs/CPs Properties of cones and some existence results. October 6, 2008

Lecture 9 Monotone VIs/CPs Properties of cones and some existence results October 6, 2008 Outline Properties of cones Existence results for monotone CPs/VIs Polyhedrality of solution sets Game theory:

### Facial Reduction and Geometry on Conic Programming

Facial Reduction and Geometry on Conic Programming Masakazu Muramatsu UEC Bruno F. Lourenço, and Takashi Tsuchiya Seikei University GRIPS Based on the papers of LMT: 1. A structural geometrical analysis

### LECTURE 10 LECTURE OUTLINE

LECTURE 10 LECTURE OUTLINE Min Common/Max Crossing Th. III Nonlinear Farkas Lemma/Linear Constraints Linear Programming Duality Convex Programming Duality Optimality Conditions Reading: Sections 4.5, 5.1,5.2,

### Largest dual ellipsoids inscribed in dual cones

Largest dual ellipsoids inscribed in dual cones M. J. Todd June 23, 2005 Abstract Suppose x and s lie in the interiors of a cone K and its dual K respectively. We seek dual ellipsoidal norms such that

### 12. Interior-point methods

12. Interior-point methods Convex Optimization Boyd & Vandenberghe inequality constrained minimization logarithmic barrier function and central path barrier method feasibility and phase I methods complexity

### Summer School: Semidefinite Optimization

Summer School: Semidefinite Optimization Christine Bachoc Université Bordeaux I, IMB Research Training Group Experimental and Constructive Algebra Haus Karrenberg, Sept. 3 - Sept. 7, 2012 Duality Theory

### Some Properties of the Augmented Lagrangian in Cone Constrained Optimization

MATHEMATICS OF OPERATIONS RESEARCH Vol. 29, No. 3, August 2004, pp. 479 491 issn 0364-765X eissn 1526-5471 04 2903 0479 informs doi 10.1287/moor.1040.0103 2004 INFORMS Some Properties of the Augmented

### SEMIDEFINITE PROGRAM BASICS. Contents

SEMIDEFINITE PROGRAM BASICS BRIAN AXELROD Abstract. A introduction to the basics of Semidefinite programs. Contents 1. Definitions and Preliminaries 1 1.1. Linear Algebra 1 1.2. Convex Analysis (on R n

### EE 227A: Convex Optimization and Applications October 14, 2008

EE 227A: Convex Optimization and Applications October 14, 2008 Lecture 13: SDP Duality Lecturer: Laurent El Ghaoui Reading assignment: Chapter 5 of BV. 13.1 Direct approach 13.1.1 Primal problem Consider

### Supplement: Hoffman s Error Bounds

IE 8534 1 Supplement: Hoffman s Error Bounds IE 8534 2 In Lecture 1 we learned that linear program and its dual problem (P ) min c T x s.t. (D) max b T y s.t. Ax = b x 0, A T y + s = c s 0 under the Slater

### Lecture Note 5: Semidefinite Programming for Stability Analysis

ECE7850: Hybrid Systems:Theory and Applications Lecture Note 5: Semidefinite Programming for Stability Analysis Wei Zhang Assistant Professor Department of Electrical and Computer Engineering Ohio State

### Semidefinite Programming

Semidefinite Programming Basics and SOS Fernando Mário de Oliveira Filho Campos do Jordão, 2 November 23 Available at: www.ime.usp.br/~fmario under talks Conic programming V is a real vector space h, i

### Optimality, Duality, Complementarity for Constrained Optimization

Optimality, Duality, Complementarity for Constrained Optimization Stephen Wright University of Wisconsin-Madison May 2014 Wright (UW-Madison) Optimality, Duality, Complementarity May 2014 1 / 41 Linear

### Convex Optimization Boyd & Vandenberghe. 5. Duality

5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized

### Victoria Martín-Márquez

A NEW APPROACH FOR THE CONVEX FEASIBILITY PROBLEM VIA MONOTROPIC PROGRAMMING Victoria Martín-Márquez Dep. of Mathematical Analysis University of Seville Spain XIII Encuentro Red de Análisis Funcional y

### Semidefinite Programming

Semidefinite Programming Notes by Bernd Sturmfels for the lecture on June 26, 208, in the IMPRS Ringvorlesung Introduction to Nonlinear Algebra The transition from linear algebra to nonlinear algebra has

### LMI MODELLING 4. CONVEX LMI MODELLING. Didier HENRION. LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ. Universidad de Valladolid, SP March 2009

LMI MODELLING 4. CONVEX LMI MODELLING Didier HENRION LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ Universidad de Valladolid, SP March 2009 Minors A minor of a matrix F is the determinant of a submatrix

### Continuous Optimisation, Chpt 9: Semidefinite Optimisation

Continuous Optimisation, Chpt 9: Semidefinite Optimisation Peter J.C. Dickinson DMMP, University of Twente p.j.c.dickinson@utwente.nl http://dickinson.website/teaching/2017co.html version: 28/11/17 Monday

### Lecture 1: Entropy, convexity, and matrix scaling CSE 599S: Entropy optimality, Winter 2016 Instructor: James R. Lee Last updated: January 24, 2016

Lecture 1: Entropy, convexity, and matrix scaling CSE 599S: Entropy optimality, Winter 2016 Instructor: James R. Lee Last updated: January 24, 2016 1 Entropy Since this course is about entropy maximization,

Iranian Journal of Operations Research Vol. 2, No. 2, 20, pp. 29-34 A semidefinite relaxation scheme for quadratically constrained quadratic problems with an additional linear constraint M. Salahi Semidefinite

### Lagrange Duality. Daniel P. Palomar. Hong Kong University of Science and Technology (HKUST)

Lagrange Duality Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470 - Convex Optimization Fall 2017-18, HKUST, Hong Kong Outline of Lecture Lagrangian Dual function Dual

### GEORGIA INSTITUTE OF TECHNOLOGY H. MILTON STEWART SCHOOL OF INDUSTRIAL AND SYSTEMS ENGINEERING LECTURE NOTES OPTIMIZATION III

GEORGIA INSTITUTE OF TECHNOLOGY H. MILTON STEWART SCHOOL OF INDUSTRIAL AND SYSTEMS ENGINEERING LECTURE NOTES OPTIMIZATION III CONVEX ANALYSIS NONLINEAR PROGRAMMING THEORY NONLINEAR PROGRAMMING ALGORITHMS

### Strong Duality and Minimal Representations for Cone Optimization

Strong Duality and Minimal Representations for Cone Optimization Levent Tunçel Henry Wolkowicz August 2008, revised: December 2010 University of Waterloo Department of Combinatorics & Optimization Waterloo,

### Semidefinite Programming

Chapter 2 Semidefinite Programming 2.0.1 Semi-definite programming (SDP) Given C M n, A i M n, i = 1, 2,..., m, and b R m, the semi-definite programming problem is to find a matrix X M n for the optimization

### Lagrange duality. The Lagrangian. We consider an optimization program of the form

Lagrange duality Another way to arrive at the KKT conditions, and one which gives us some insight on solving constrained optimization problems, is through the Lagrange dual. The dual is a maximization

### Lecture: Duality.

Lecture: Duality http://bicmr.pku.edu.cn/~wenzw/opt-2016-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghe s lecture notes Introduction 2/35 Lagrange dual problem weak and strong

### Lecture 1: Convex Sets January 23

IE 521: Convex Optimization Instructor: Niao He Lecture 1: Convex Sets January 23 Spring 2017, UIUC Scribe: Niao He Courtesy warning: These notes do not necessarily cover everything discussed in the class.

### COURSE ON LMI PART I.2 GEOMETRY OF LMI SETS. Didier HENRION henrion

COURSE ON LMI PART I.2 GEOMETRY OF LMI SETS Didier HENRION www.laas.fr/ henrion October 2006 Geometry of LMI sets Given symmetric matrices F i we want to characterize the shape in R n of the LMI set F

### Research overview. Seminar September 4, Lehigh University Department of Industrial & Systems Engineering. Research overview.

Research overview Lehigh University Department of Industrial & Systems Engineering COR@L Seminar September 4, 2008 1 Duality without regularity condition Duality in non-exact arithmetic 2 interior point

### Semidefinite Programming Basics and Applications

Semidefinite Programming Basics and Applications Ray Pörn, principal lecturer Åbo Akademi University Novia University of Applied Sciences Content What is semidefinite programming (SDP)? How to represent

### Introduction to Mathematical Programming IE406. Lecture 10. Dr. Ted Ralphs

Introduction to Mathematical Programming IE406 Lecture 10 Dr. Ted Ralphs IE406 Lecture 10 1 Reading for This Lecture Bertsimas 4.1-4.3 IE406 Lecture 10 2 Duality Theory: Motivation Consider the following

### More First-Order Optimization Algorithms

More First-Order Optimization Algorithms Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye Chapters 3, 8, 3 The SDM

### Lecture 17: Primal-dual interior-point methods part II

10-725/36-725: Convex Optimization Spring 2015 Lecture 17: Primal-dual interior-point methods part II Lecturer: Javier Pena Scribes: Pinchao Zhang, Wei Ma Note: LaTeX template courtesy of UC Berkeley EECS

### Functional Analysis I

Functional Analysis I Course Notes by Stefan Richter Transcribed and Annotated by Gregory Zitelli Polar Decomposition Definition. An operator W B(H) is called a partial isometry if W x = X for all x (ker

### l p -Norm Constrained Quadratic Programming: Conic Approximation Methods

OUTLINE l p -Norm Constrained Quadratic Programming: Conic Approximation Methods Wenxun Xing Department of Mathematical Sciences Tsinghua University, Beijing Email: wxing@math.tsinghua.edu.cn OUTLINE OUTLINE

### Continuous Optimisation, Chpt 9: Semidefinite Problems

Continuous Optimisation, Chpt 9: Semidefinite Problems Peter J.C. Dickinson DMMP, University of Twente p.j.c.dickinson@utwente.nl http://dickinson.website/teaching/2016co.html version: 21/11/16 Monday

### On self-concordant barriers for generalized power cones

On self-concordant barriers for generalized power cones Scott Roy Lin Xiao January 30, 2018 Abstract In the study of interior-point methods for nonsymmetric conic optimization and their applications, Nesterov

### Research Note. A New Infeasible Interior-Point Algorithm with Full Nesterov-Todd Step for Semi-Definite Optimization

Iranian Journal of Operations Research Vol. 4, No. 1, 2013, pp. 88-107 Research Note A New Infeasible Interior-Point Algorithm with Full Nesterov-Todd Step for Semi-Definite Optimization B. Kheirfam We

### Constrained Optimization and Lagrangian Duality

CIS 520: Machine Learning Oct 02, 2017 Constrained Optimization and Lagrangian Duality Lecturer: Shivani Agarwal Disclaimer: These notes are designed to be a supplement to the lecture. They may or may

### E5295/5B5749 Convex optimization with engineering applications. Lecture 5. Convex programming and semidefinite programming

E5295/5B5749 Convex optimization with engineering applications Lecture 5 Convex programming and semidefinite programming A. Forsgren, KTH 1 Lecture 5 Convex optimization 2006/2007 Convex quadratic program

### Lecture 14: Optimality Conditions for Conic Problems

EE 227A: Conve Optimization and Applications March 6, 2012 Lecture 14: Optimality Conditions for Conic Problems Lecturer: Laurent El Ghaoui Reading assignment: 5.5 of BV. 14.1 Optimality for Conic Problems

### Chapter 2 Convex Analysis

Chapter 2 Convex Analysis The theory of nonsmooth analysis is based on convex analysis. Thus, we start this chapter by giving basic concepts and results of convexity (for further readings see also [202,

### Convex Optimization and SVM

Convex Optimization and SVM Problem 0. Cf lecture notes pages 12 to 18. Problem 1. (i) A slab is an intersection of two half spaces, hence convex. (ii) A wedge is an intersection of two half spaces, hence

### Chapter 1. Preliminaries. The purpose of this chapter is to provide some basic background information. Linear Space. Hilbert Space.

Chapter 1 Preliminaries The purpose of this chapter is to provide some basic background information. Linear Space Hilbert Space Basic Principles 1 2 Preliminaries Linear Space The notion of linear space

### Lecture: Algorithms for LP, SOCP and SDP

1/53 Lecture: Algorithms for LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2018.html wenzw@pku.edu.cn Acknowledgement:

### EE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 17

EE/ACM 150 - Applications of Convex Optimization in Signal Processing and Communications Lecture 17 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory May 29, 2012 Andre Tkacenko

### Lecture 1: Background on Convex Analysis

Lecture 1: Background on Convex Analysis John Duchi PCMI 2016 Outline I Convex sets 1.1 Definitions and examples 2.2 Basic properties 3.3 Projections onto convex sets 4.4 Separating and supporting hyperplanes

### Lecture 8: Semidefinite programs for fidelity and optimal measurements

CS 766/QIC 80 Theory of Quantum Information (Fall 0) Lecture 8: Semidefinite programs for fidelity and optimal measurements This lecture is devoted to two examples of semidefinite programs: one is for

### Lecture: Cone programming. Approximating the Lorentz cone.

Strong relaxations for discrete optimization problems 10/05/16 Lecture: Cone programming. Approximating the Lorentz cone. Lecturer: Yuri Faenza Scribes: Igor Malinović 1 Introduction Cone programming is

### Dedicated to Michel Théra in honor of his 70th birthday

VARIATIONAL GEOMETRIC APPROACH TO GENERALIZED DIFFERENTIAL AND CONJUGATE CALCULI IN CONVEX ANALYSIS B. S. MORDUKHOVICH 1, N. M. NAM 2, R. B. RECTOR 3 and T. TRAN 4. Dedicated to Michel Théra in honor of

### On smoothness properties of optimal value functions at the boundary of their domain under complete convexity

On smoothness properties of optimal value functions at the boundary of their domain under complete convexity Oliver Stein # Nathan Sudermann-Merx June 14, 2013 Abstract This article studies continuity

### Real Symmetric Matrices and Semidefinite Programming

Real Symmetric Matrices and Semidefinite Programming Tatsiana Maskalevich Abstract Symmetric real matrices attain an important property stating that all their eigenvalues are real. This gives rise to many

### Week 3 Linear programming duality

Week 3 Linear programming duality This week we cover the fascinating topic of linear programming duality. We will learn that every minimization program has associated a maximization program that has the

### Selected Examples of CONIC DUALITY AT WORK Robust Linear Optimization Synthesis of Linear Controllers Matrix Cube Theorem A.

. Selected Examples of CONIC DUALITY AT WORK Robust Linear Optimization Synthesis of Linear Controllers Matrix Cube Theorem A. Nemirovski Arkadi.Nemirovski@isye.gatech.edu Linear Optimization Problem,

### CHAPTER 2: CONVEX SETS AND CONCAVE FUNCTIONS. W. Erwin Diewert January 31, 2008.

1 ECONOMICS 594: LECTURE NOTES CHAPTER 2: CONVEX SETS AND CONCAVE FUNCTIONS W. Erwin Diewert January 31, 2008. 1. Introduction Many economic problems have the following structure: (i) a linear function

Additional Homework Problems Robert M. Freund April, 2004 2004 Massachusetts Institute of Technology. 1 2 1 Exercises 1. Let IR n + denote the nonnegative orthant, namely IR + n = {x IR n x j ( ) 0,j =1,...,n}.

### Extended Monotropic Programming and Duality 1

March 2006 (Revised February 2010) Report LIDS - 2692 Extended Monotropic Programming and Duality 1 by Dimitri P. Bertsekas 2 Abstract We consider the problem minimize f i (x i ) subject to x S, where

### 6-1 The Positivstellensatz P. Parrilo and S. Lall, ECC

6-1 The Positivstellensatz P. Parrilo and S. Lall, ECC 2003 2003.09.02.10 6. The Positivstellensatz Basic semialgebraic sets Semialgebraic sets Tarski-Seidenberg and quantifier elimination Feasibility