Lecture 5. The Dual Cone and Dual Problem


 Kerry Mosley
 1 years ago
 Views:
Transcription
1 IE Lecture 5. The Dual Cone and Dual Problem
2 IE For a convex cone K, its dual cone is defined as K = {y x, y 0, x K}. The innerproduct can be replaced by x T y if the coordinates of the vectors are given. Clearly, K is always a closed set, regardless if K is closed or not.
3 IE In light of the dual cone, the problem of finding a lower bound on the optimal value for the standard conic optimization problem can be done as follows. Introduce the Lagrangian multiplier y for the equality constraint b Ax = 0, and s for the conic constraint x K. The objective becomes c T x + y T (b Ax) s T x = y T b + (c A T y s) T x. This will be a lower bound on the optimal value whenever c A T y s = 0 and s K. The quest to find the best lower bound leads to the dual problem for the standard conic optimization problem: maximize subject to b T y A T y + s = c s K. (1)
4 IE As an example, for the standard SDP problem (primal), minimize subject to C X A i X = b i, i = 1,..., m X 0, its dual problem is in the form of maximize subject to m i=1 b iy i m i=1 y ia i + S = C S 0.
5 IE The socalled weak duality relationship is immediate, by the construction of the dual problem. This is shown in the next theorem. Theorem 1 Let x be feasible for the primal problem, and (y, s) be feasible to its dual. Then, b T y c T x. Moreover, if b T y = c T x then x T s = 0, and they are both optimal to their own respective problems. If a pair of primaldual feasible solutions x and (y, s) both exist and they satisfy x T s = 0 then we call them a pair of complementary optimal solutions.
6 IE A very useful theorem is the following. Theorem 2 Let x be feasible for the primal problem, and (y, s) be feasible to its dual, and x int K and s int K. Then, the sets of optimal solutions for both problems are nonempty and compact, and any pair of primaldual optimal solutions will be complementary to each other.
7 IE Let us take some moments to study the structures of the dual cones. Let us start with the topological structures. If rel (K) is denoted as the relative interior of K, then (rel (K)) = K. Let the summation of two sets A and B be A + B = {z z = x + y with x A, y B}. It is clear that the sum of two convex sets remains convex. However, the sum of two closed convex sets may not be closed any more.
8 IE Example 1 The set R SOC(3) is convex but not closed. For instance, the point (0, 1, 0) T is not in the set, but it is in the closure of the set, since for any ϵ > 0, we may write 1 ϵ 1 = 1 1 ϵ ϵ ϵ 1, 0 1 where the second term is in SOC(3). 1 ϵ
9 IE However, if A and B are both polyhedra, then A + B remains a polyhedron, hence closed. It is interesting to note that the setsummation and the setintersection operations are a duality pair: Proposition 1 Suppose that K 1 and K 2 are convex cones. It holds that (K 1 + K 2 ) = K 1 K 2. Proof. For y (K 1 + K 2 ), it follows that 0 (x 1 + x 2 ) T y for all x 1 K 1 and x 2 K 2. Since 0 K 2, it follows that 0 (x 1 ) T y for all x 1 K 1, thus y K 1, and similarly y K 2. Hence, y K 1 K 2. Now, take any y K 1 K 2, and any x 1 K 1 and x 2 K 2. We have (x 1 + x 2 ) T y = (x 1 ) T y + (x 2 ) T y 0, and so y (K 1 + K 2 ). This proves (K 1 + K 2 ) = K 1 K 2 as desired.
10 IE The following bipolar theorem can be considered as a cornerstone for convex analysis. Theorem 3 Suppose that K is a convex cone. Then (K ) = cl K. Proof. For any x K, by the definition of the dual cone, x T y 0 for all y K, and so K (K ). Therefore cl K (K ). The other containing relationship (K ) cl K can be shown by contradiction. Suppose such is not true then there exists z (K ) \ cl K. By the separation theorem for convex cones, there is vector c such that c T z < 0, and c T x 0 x cl K. The last relation suggests that c K, which contradicts with the first relation because, as z (K ), it should lead to c T z 0, but not the opposite. This contradiction completes the proof.
11 IE This proof suggests that the bipolar theorem for convex cone is equivalent to the separation theorem itself. Suppose that K is a closed convex cone, and v K. Then, the bipolar theorem asserts that v (K ), and this implies that there is y K such that v T y < 0, and x T y 0 for any x K because y K, hence a separation exists.
12 IE In case K is closed, the bipolar theorem is what in folklore understood as the dual of the dual is the primal itself again. Combining Theorem 3 and Proposition 1, it follows that cl (K 1 + K 2 ) = (K 1 K 2 ). (2) It is also useful to compute the effect of linear transformation on a convex cone in the context of dual operation. Let A be an invertible matrix. Then, we have (A T K) = A 1 K, (3) by observing that y (A T K) (A T x) T y 0 x K Ay K y A 1 K.
13 IE A convex cone is called solid if it is full dimensional. Mathematically, K R n is solid span K = R n, or K + ( K) = R n. A convex cone is called pointed if it does not contain any nontrivial subspace; i.e. K ( K) = {0}. The following result proves to be useful: Proposition 2 If K is solid then K is pointed, and if K is pointed then K is solid.
14 IE Proof. Suppose that K is solid, i.e. there is a ball B(x 0 ; δ) K with radius δ > 0 and centered at x 0, and at the same suppose that there is a line Rc K, then 0 s(x 0 + t) T c, t δ and s R, which is clearly impossible. This contradiction shows that if K is solid then K must be pointed. By the bipolar theorem, the second part follows by symmetry.
15 IE It is easy to compute the following dual cones (R n +) = R n +. SOC(n) = SOC(n). (S n n + ) = S n n +, and (H n n + ) = H n n +. The above standard cones are so regular that their corresponding dual cones coincide with the original cones themselves (the primal cones). In general, they are not the same. As an example, consider the following L p cone, where p 1, L p (n + 1) = t x t R, x Rn, t x p.
16 IE One can show that (L p (n + 1)) = L q (n + 1) where 1/p + 1/q = 1. Now, the selfduality of SOC(n + 1) is due to the fact that SOC(n + 1) = L 2 (n + 1). In convex analysis, the dual object for a convex function f(x) is known as the conjugate of f(x), defined as f (s) = sup{( s) T x f(x) x dom f}, where dom f stands for the domain of the function f. The conjugate function is also known as the LegendreFenchel transformation.
17 IE Examples of some popular conjugate functions are: 1. For f(x) = 1 2 xt Qx + b T x where Q 0, the conjugate function is f (s) = 1 2 (s + b)t Q 1 (s + b). 2. For f(x) = n i=1 ex i, the conjugate function is f (s) = n i=1 s i log( s i ) + n i=1 s i. 3. For f(x) = n i=1 x i log x i, the conjugate function is f (s) = n i=1 e1+s i. 4. For f(x) = c T x, the conjugate function is f (s) = 0 for s = c and f (s) = + when s c. 5. For f(x) = n i=1 log x i, the conjugate function is f (s) = n n i=1 log s i. 6. For f(x) = max 1 i n x i, the conjugate function is f (s) = 0 if n i=1 s i 1, and f (s) = + otherwise.
18 IE If f(x) is strictly convex and differentiable, then f (s) is also strictly convex and differentiable. The famous biconjugate theorem states that f = cl f. There is certainly a relationship between the two dual objects: the dual cone and the conjugate function. Let us now set out to find the exact relationship. First of all, suppose that f is a convex function, mapping from its domain in R n to R. Its epigraphy is a set in R n+1 defined as epi (f) := t t f(x), t R, x dom (f) x.
19 IE It is well known that f is a convex function if and only if epi (f) is a convex set. It turns out that H (epi (f )) is the dual of H (epi (f)), after taking the closure and reorienting itself. To be precise, we have: Theorem 4 Suppose that f is a convex function, and p K := cl q p > 0, q pf(x/p) 0. x Then, K = cl u v s v > 0, u vf (s/v) 0.
20 IE Let L 2 := 0, 1 1, 0 and L := L 2 I n = [L 2, 0 2,n ; 0 n,2, I n ]. The effect of applying L to x is to swap the position of x 1 and x 2. Equivalently, Theorem 4 can be stated as (H (epi (f))) = cl L (H (epi (f ))). (4)
21 IE In linear programming, it is known that whenever the primal and the dual problems are both feasible then complementary solutions must exist. This important property, however, is lost, for a general conic optimization model, as the following example shows. Consider minimize x 11 subject to x 12 + x 21 = 1 x 11, x 12 x 21, x This problem is clearly feasible. However, there is no attainable optimal solution.
22 IE Its dual problem is maximize subject to y y 0, 1 1, 0 + s 11, s 12 s 21, s 22 = 1, 0 0, 0 s 11, s 12 s 21, s 22 0, or, more explicitly, maximize subject to y 1, y y, 0 0. The dual problem is feasible and has an optimal solution; however, a pair of primaldual complementary optimal solutions fail to exist, although both primal and dual problems are feasible.
23 IE One may guess that the unfortunate situation was caused by the nonattainable optimal solution. Next example shows that even if both primal and dual problems are nicely feasible with attainable optimal solutions, there might still be a positive duality gap, meaning that the complementarity condition does not hold. minimize x 12 x x 33 subject to x 12 + x 21 + x 33 = 1 x 22 = 0 x 11, x 12, x 13 x 21, x 22, x 23 x 31, x 32, x 33 0.
24 IE The dual of this problem is maximize y 1 subject to 0, 1 y 1, 0 1 y 1, y 2, 0 0, 0, 2 y 1 0. The primal problem has an optimal solution, e.g. 1, 0, 0 X = 0, 0, 0, with the optimal value equal to 2. The dual 0, 0, 1 problem also is feasible, and has an optimal solution e.g. (y1, y2) = ( 1, 1), with optimal value equal to 1. Both problems are feasible and have attainable optimal solutions; however, their optimal solutions are not complementary to each other, and there is a positive duality gap: c T x b T y = (x ) T s > 0.
25 IE Key References: Yu. Nesterov and A. Nemirovsky, Interior Point Polynomial Methods in Convex Programming, Studies in Applied Mathematics 13, SIAM, J.F. Sturm, Theory and Algorithms for Semidefinite Programming. In High Performance Optimization, eds. H. Frenk, K. Roos, T. Terlaky, and S. Zhang, Kluwer Academic Publishers, S. Zhang, A New SelfDual Embedding Method for Convex Programming, Journal of Global Optimization, 29, , 2004.
Lecture 5. Theorems of Alternatives and SelfDual Embedding
IE 8534 1 Lecture 5. Theorems of Alternatives and SelfDual Embedding IE 8534 2 A system of linear equations may not have a solution. It is well known that either Ax = c has a solution, or A T y = 0, c
More information4. Algebra and Duality
41 Algebra and Duality P. Parrilo and S. Lall, CDC 2003 2003.12.07.01 4. Algebra and Duality Example: nonconvex polynomial optimization Weak duality and duality gap The dual is not intrinsic The cone
More informationLecture 6: Conic Optimization September 8
IE 598: Big Data Optimization Fall 2016 Lecture 6: Conic Optimization September 8 Lecturer: Niao He Scriber: Juan Xu Overview In this lecture, we finish up our previous discussion on optimality conditions
More informationConvex Optimization M2
Convex Optimization M2 Lecture 3 A. d Aspremont. Convex Optimization M2. 1/49 Duality A. d Aspremont. Convex Optimization M2. 2/49 DMs DM par email: dm.daspremont@gmail.com A. d Aspremont. Convex Optimization
More informationExample: feasibility. Interpretation as formal proof. Example: linear inequalities and Farkas lemma
41 Algebra and Duality P. Parrilo and S. Lall 2006.06.07.01 4. Algebra and Duality Example: nonconvex polynomial optimization Weak duality and duality gap The dual is not intrinsic The cone of valid
More informationA New SelfDual Embedding Method for Convex Programming
A New SelfDual Embedding Method for Convex Programming Shuzhong Zhang October 2001; revised October 2002 Abstract In this paper we introduce a conic optimization formulation to solve constrained convex
More information15. Conic optimization
L. Vandenberghe EE236C (Spring 216) 15. Conic optimization conic linear program examples modeling duality 151 Generalized (conic) inequalities Conic inequality: a constraint x K where K is a convex cone
More informationSupplement: Universal SelfConcordant Barrier Functions
IE 8534 1 Supplement: Universal SelfConcordant Barrier Functions IE 8534 2 Recall that a selfconcordant barrier function for K is a barrier function satisfying 3 F (x)[h, h, h] 2( 2 F (x)[h, h]) 3/2,
More informationConic Linear Optimization and its Dual. yyye
Conic Linear Optimization and Appl. MS&E314 Lecture Note #04 1 Conic Linear Optimization and its Dual Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A.
More informationLecture 1. 1 Conic programming. MA 796S: Convex Optimization and Interior Point Methods October 8, Consider the conic program. min.
MA 796S: Convex Optimization and Interior Point Methods October 8, 2007 Lecture 1 Lecturer: Kartik Sivaramakrishnan Scribe: Kartik Sivaramakrishnan 1 Conic programming Consider the conic program min s.t.
More informationExtreme Abridgment of Boyd and Vandenberghe s Convex Optimization
Extreme Abridgment of Boyd and Vandenberghe s Convex Optimization Compiled by David Rosenberg Abstract Boyd and Vandenberghe s Convex Optimization book is very wellwritten and a pleasure to read. The
More informationConvex Optimization & Lagrange Duality
Convex Optimization & Lagrange Duality Chee Wei Tan CS 8292 : Advanced Topics in Convex Optimization and its Applications Fall 2010 Outline Convex optimization Optimality condition Lagrange duality KKT
More informationOn duality gap in linear conic problems
On duality gap in linear conic problems C. Zălinescu Abstract In their paper Duality of linear conic problems A. Shapiro and A. Nemirovski considered two possible properties (A) and (B) for dual linear
More informationCourse Notes for EE227C (Spring 2018): Convex Optimization and Approximation
Course Notes for EE227C (Spring 2018): Convex Optimization and Approximation Instructor: Moritz Hardt Email: hardt+ee227c@berkeley.edu Graduate Instructor: Max Simchowitz Email: msimchow+ee227c@berkeley.edu
More informationOptimality Conditions for Constrained Optimization
72 CHAPTER 7 Optimality Conditions for Constrained Optimization 1. First Order Conditions In this section we consider first order optimality conditions for the constrained problem P : minimize f 0 (x)
More informationLecture: Duality of LP, SOCP and SDP
1/33 Lecture: Duality of LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2017.html wenzw@pku.edu.cn Acknowledgement:
More informationOn duality theory of conic linear problems
On duality theory of conic linear problems Alexander Shapiro School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, Georgia 333225, USA email: ashapiro@isye.gatech.edu
More informationOptimization for Machine Learning
Optimization for Machine Learning (Problems; Algorithms  A) SUVRIT SRA Massachusetts Institute of Technology PKU Summer School on Data Science (July 2017) Course materials http://suvrit.de/teaching.html
More informationExact SDP Relaxations for Classes of Nonlinear Semidefinite Programming Problems
Exact SDP Relaxations for Classes of Nonlinear Semidefinite Programming Problems V. Jeyakumar and G. Li Revised Version:August 31, 2012 Abstract An exact semidefinite linear programming (SDP) relaxation
More informationGlobal Optimization of Polynomials
Semidefinite Programming Lecture 9 OR 637 Spring 2008 April 9, 2008 Scribe: Dennis Leventhal Global Optimization of Polynomials Recall we were considering the problem min z R n p(z) where p(z) is a degree
More informationIE 521 Convex Optimization
Lecture 1: 16th January 2019 Outline 1 / 20 Which set is different from others? Figure: Four sets 2 / 20 Which set is different from others? Figure: Four sets 3 / 20 Interior, Closure, Boundary Definition.
More informationLagrangian Duality Theory
Lagrangian Duality Theory Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye Chapter 14.14 1 Recall Primal and Dual
More informationLecture: Introduction to LP, SDP and SOCP
Lecture: Introduction to LP, SDP and SOCP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2015.html wenzw@pku.edu.cn Acknowledgement:
More informationLECTURE 13 LECTURE OUTLINE
LECTURE 13 LECTURE OUTLINE Problem Structures Separable problems Integer/discrete problems Branchandbound Large sum problems Problems with many constraints Conic Programming Second Order Cone Programming
More informationLECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE
LECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE CONVEX ANALYSIS AND DUALITY Basic concepts of convex analysis Basic concepts of convex optimization Geometric duality framework  MC/MC Constrained optimization
More informationPOLYNOMIAL OPTIMIZATION WITH SUMSOFSQUARES INTERPOLANTS
POLYNOMIAL OPTIMIZATION WITH SUMSOFSQUARES INTERPOLANTS Sercan Yıldız syildiz@samsi.info in collaboration with Dávid Papp (NCSU) OPT Transition Workshop May 02, 2017 OUTLINE Polynomial optimization and
More informationStrong duality in Lasserre s hierarchy for polynomial optimization
Strong duality in Lasserre s hierarchy for polynomial optimization arxiv:1405.7334v1 [math.oc] 28 May 2014 Cédric Josz 1,2, Didier Henrion 3,4,5 Draft of January 24, 2018 Abstract A polynomial optimization
More information5. Duality. Lagrangian
5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationOptimality Conditions for Nonsmooth Convex Optimization
Optimality Conditions for Nonsmooth Convex Optimization Sangkyun Lee Oct 22, 2014 Let us consider a convex function f : R n R, where R is the extended real field, R := R {, + }, which is proper (f never
More informationA Simple Derivation of a Facial Reduction Algorithm and Extended Dual Systems
A Simple Derivation of a Facial Reduction Algorithm and Extended Dual Systems Gábor Pataki gabor@unc.edu Dept. of Statistics and OR University of North Carolina at Chapel Hill Abstract The Facial Reduction
More informationOptimization and Optimal Control in Banach Spaces
Optimization and Optimal Control in Banach Spaces Bernhard Schmitzer October 19, 2017 1 Convex nonsmooth optimization with proximal operators Remark 1.1 (Motivation). Convex optimization: easier to solve,
More informationRobust Farkas Lemma for Uncertain Linear Systems with Applications
Robust Farkas Lemma for Uncertain Linear Systems with Applications V. Jeyakumar and G. Li Revised Version: July 8, 2010 Abstract We present a robust Farkas lemma, which provides a new generalization of
More informationNew stopping criteria for detecting infeasibility in conic optimization
Optimization Letters manuscript No. (will be inserted by the editor) New stopping criteria for detecting infeasibility in conic optimization Imre Pólik Tamás Terlaky Received: March 21, 2008/ Accepted:
More informationLecture 8. Strong Duality Results. September 22, 2008
Strong Duality Results September 22, 2008 Outline Lecture 8 Slater Condition and its Variations Convex Objective with Linear Inequality Constraints Quadratic Objective over Quadratic Constraints Representation
More informationLecture 3: Semidefinite Programming
Lecture 3: Semidefinite Programming Lecture Outline Part I: Semidefinite programming, examples, canonical form, and duality Part II: Strong Duality Failure Examples Part III: Conditions for strong duality
More informationMATINF4110/MATINF9110 Mathematical optimization
MATINF4110/MATINF9110 Mathematical optimization Geir Dahl August 20, 2013 Convexity Part IV Chapter 4 Representation of convex sets different representations of convex sets, boundary polyhedra and polytopes:
More informationInequality Constraints
Chapter 2 Inequality Constraints 2.1 Optimality Conditions Early in multivariate calculus we learn the significance of differentiability in finding minimizers. In this section we begin our study of the
More informationConvex Optimization Theory. Chapter 5 Exercises and Solutions: Extended Version
Convex Optimization Theory Chapter 5 Exercises and Solutions: Extended Version Dimitri P. Bertsekas Massachusetts Institute of Technology Athena Scientific, Belmont, Massachusetts http://www.athenasc.com
More informationI.3. LMI DUALITY. Didier HENRION EECI Graduate School on Control Supélec  Spring 2010
I.3. LMI DUALITY Didier HENRION henrion@laas.fr EECI Graduate School on Control Supélec  Spring 2010 Primal and dual For primal problem p = inf x g 0 (x) s.t. g i (x) 0 define Lagrangian L(x, z) = g 0
More information1 Review of last lecture and introduction
Semidefinite Programming Lecture 10 OR 637 Spring 2008 April 16, 2008 (Wednesday) Instructor: Michael Jeremy Todd Scribe: Yogeshwer (Yogi) Sharma 1 Review of last lecture and introduction Let us first
More informationLecture 8 Plus properties, merit functions and gap functions. September 28, 2008
Lecture 8 Plus properties, merit functions and gap functions September 28, 2008 Outline Plusproperties and Funiqueness Equation reformulations of VI/CPs Merit functions Gap merit functions FPI book:
More informationLecture 9 Monotone VIs/CPs Properties of cones and some existence results. October 6, 2008
Lecture 9 Monotone VIs/CPs Properties of cones and some existence results October 6, 2008 Outline Properties of cones Existence results for monotone CPs/VIs Polyhedrality of solution sets Game theory:
More informationCSCI 1951G Optimization Methods in Finance Part 01: Linear Programming
CSCI 1951G Optimization Methods in Finance Part 01: Linear Programming January 26, 2018 1 / 38 Liability/asset cashflow matching problem Recall the formulation of the problem: max w c 1 + p 1 e 1 = 150
More informationFacial Reduction and Geometry on Conic Programming
Facial Reduction and Geometry on Conic Programming Masakazu Muramatsu UEC Bruno F. Lourenço, and Takashi Tsuchiya Seikei University GRIPS Based on the papers of LMT: 1. A structural geometrical analysis
More informationLECTURE 10 LECTURE OUTLINE
LECTURE 10 LECTURE OUTLINE Min Common/Max Crossing Th. III Nonlinear Farkas Lemma/Linear Constraints Linear Programming Duality Convex Programming Duality Optimality Conditions Reading: Sections 4.5, 5.1,5.2,
More informationLargest dual ellipsoids inscribed in dual cones
Largest dual ellipsoids inscribed in dual cones M. J. Todd June 23, 2005 Abstract Suppose x and s lie in the interiors of a cone K and its dual K respectively. We seek dual ellipsoidal norms such that
More information12. Interiorpoint methods
12. Interiorpoint methods Convex Optimization Boyd & Vandenberghe inequality constrained minimization logarithmic barrier function and central path barrier method feasibility and phase I methods complexity
More informationSummer School: Semidefinite Optimization
Summer School: Semidefinite Optimization Christine Bachoc Université Bordeaux I, IMB Research Training Group Experimental and Constructive Algebra Haus Karrenberg, Sept. 3  Sept. 7, 2012 Duality Theory
More informationSome Properties of the Augmented Lagrangian in Cone Constrained Optimization
MATHEMATICS OF OPERATIONS RESEARCH Vol. 29, No. 3, August 2004, pp. 479 491 issn 0364765X eissn 15265471 04 2903 0479 informs doi 10.1287/moor.1040.0103 2004 INFORMS Some Properties of the Augmented
More informationSemidefinite Programming
Semidefinite Programming Basics and SOS Fernando Mário de Oliveira Filho Campos do Jordão, 2 November 23 Available at: www.ime.usp.br/~fmario under talks Conic programming V is a real vector space h, i
More informationSEMIDEFINITE PROGRAM BASICS. Contents
SEMIDEFINITE PROGRAM BASICS BRIAN AXELROD Abstract. A introduction to the basics of Semidefinite programs. Contents 1. Definitions and Preliminaries 1 1.1. Linear Algebra 1 1.2. Convex Analysis (on R n
More informationEE 227A: Convex Optimization and Applications October 14, 2008
EE 227A: Convex Optimization and Applications October 14, 2008 Lecture 13: SDP Duality Lecturer: Laurent El Ghaoui Reading assignment: Chapter 5 of BV. 13.1 Direct approach 13.1.1 Primal problem Consider
More informationOptimality, Duality, Complementarity for Constrained Optimization
Optimality, Duality, Complementarity for Constrained Optimization Stephen Wright University of WisconsinMadison May 2014 Wright (UWMadison) Optimality, Duality, Complementarity May 2014 1 / 41 Linear
More informationSupplement: Hoffman s Error Bounds
IE 8534 1 Supplement: Hoffman s Error Bounds IE 8534 2 In Lecture 1 we learned that linear program and its dual problem (P ) min c T x s.t. (D) max b T y s.t. Ax = b x 0, A T y + s = c s 0 under the Slater
More informationLecture Note 5: Semidefinite Programming for Stability Analysis
ECE7850: Hybrid Systems:Theory and Applications Lecture Note 5: Semidefinite Programming for Stability Analysis Wei Zhang Assistant Professor Department of Electrical and Computer Engineering Ohio State
More informationLMI MODELLING 4. CONVEX LMI MODELLING. Didier HENRION. LAASCNRS Toulouse, FR Czech Tech Univ Prague, CZ. Universidad de Valladolid, SP March 2009
LMI MODELLING 4. CONVEX LMI MODELLING Didier HENRION LAASCNRS Toulouse, FR Czech Tech Univ Prague, CZ Universidad de Valladolid, SP March 2009 Minors A minor of a matrix F is the determinant of a submatrix
More informationConvex Optimization Boyd & Vandenberghe. 5. Duality
5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationSemidefinite Programming
Semidefinite Programming Notes by Bernd Sturmfels for the lecture on June 26, 208, in the IMPRS Ringvorlesung Introduction to Nonlinear Algebra The transition from linear algebra to nonlinear algebra has
More informationVictoria MartínMárquez
A NEW APPROACH FOR THE CONVEX FEASIBILITY PROBLEM VIA MONOTROPIC PROGRAMMING Victoria MartínMárquez Dep. of Mathematical Analysis University of Seville Spain XIII Encuentro Red de Análisis Funcional y
More informationLecture 1: Entropy, convexity, and matrix scaling CSE 599S: Entropy optimality, Winter 2016 Instructor: James R. Lee Last updated: January 24, 2016
Lecture 1: Entropy, convexity, and matrix scaling CSE 599S: Entropy optimality, Winter 2016 Instructor: James R. Lee Last updated: January 24, 2016 1 Entropy Since this course is about entropy maximization,
More informationA semidefinite relaxation scheme for quadratically constrained quadratic problems with an additional linear constraint
Iranian Journal of Operations Research Vol. 2, No. 2, 20, pp. 2934 A semidefinite relaxation scheme for quadratically constrained quadratic problems with an additional linear constraint M. Salahi Semidefinite
More informationContinuous Optimisation, Chpt 9: Semidefinite Optimisation
Continuous Optimisation, Chpt 9: Semidefinite Optimisation Peter J.C. Dickinson DMMP, University of Twente p.j.c.dickinson@utwente.nl http://dickinson.website/teaching/2017co.html version: 28/11/17 Monday
More informationLagrange Duality. Daniel P. Palomar. Hong Kong University of Science and Technology (HKUST)
Lagrange Duality Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470  Convex Optimization Fall 201718, HKUST, Hong Kong Outline of Lecture Lagrangian Dual function Dual
More informationStrong Duality and Minimal Representations for Cone Optimization
Strong Duality and Minimal Representations for Cone Optimization Levent Tunçel Henry Wolkowicz August 2008, revised: December 2010 University of Waterloo Department of Combinatorics & Optimization Waterloo,
More informationGEORGIA INSTITUTE OF TECHNOLOGY H. MILTON STEWART SCHOOL OF INDUSTRIAL AND SYSTEMS ENGINEERING LECTURE NOTES OPTIMIZATION III
GEORGIA INSTITUTE OF TECHNOLOGY H. MILTON STEWART SCHOOL OF INDUSTRIAL AND SYSTEMS ENGINEERING LECTURE NOTES OPTIMIZATION III CONVEX ANALYSIS NONLINEAR PROGRAMMING THEORY NONLINEAR PROGRAMMING ALGORITHMS
More informationSemidefinite Programming
Chapter 2 Semidefinite Programming 2.0.1 Semidefinite programming (SDP) Given C M n, A i M n, i = 1, 2,..., m, and b R m, the semidefinite programming problem is to find a matrix X M n for the optimization
More informationLagrange duality. The Lagrangian. We consider an optimization program of the form
Lagrange duality Another way to arrive at the KKT conditions, and one which gives us some insight on solving constrained optimization problems, is through the Lagrange dual. The dual is a maximization
More informationLecture: Duality.
Lecture: Duality http://bicmr.pku.edu.cn/~wenzw/opt2016fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghe s lecture notes Introduction 2/35 Lagrange dual problem weak and strong
More informationLecture 1: Convex Sets January 23
IE 521: Convex Optimization Instructor: Niao He Lecture 1: Convex Sets January 23 Spring 2017, UIUC Scribe: Niao He Courtesy warning: These notes do not necessarily cover everything discussed in the class.
More informationCOURSE ON LMI PART I.2 GEOMETRY OF LMI SETS. Didier HENRION henrion
COURSE ON LMI PART I.2 GEOMETRY OF LMI SETS Didier HENRION www.laas.fr/ henrion October 2006 Geometry of LMI sets Given symmetric matrices F i we want to characterize the shape in R n of the LMI set F
More informationSemidefinite Programming Basics and Applications
Semidefinite Programming Basics and Applications Ray Pörn, principal lecturer Åbo Akademi University Novia University of Applied Sciences Content What is semidefinite programming (SDP)? How to represent
More informationResearch overview. Seminar September 4, Lehigh University Department of Industrial & Systems Engineering. Research overview.
Research overview Lehigh University Department of Industrial & Systems Engineering COR@L Seminar September 4, 2008 1 Duality without regularity condition Duality in nonexact arithmetic 2 interior point
More informationIntroduction to Mathematical Programming IE406. Lecture 10. Dr. Ted Ralphs
Introduction to Mathematical Programming IE406 Lecture 10 Dr. Ted Ralphs IE406 Lecture 10 1 Reading for This Lecture Bertsimas 4.14.3 IE406 Lecture 10 2 Duality Theory: Motivation Consider the following
More informationMore FirstOrder Optimization Algorithms
More FirstOrder Optimization Algorithms Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye Chapters 3, 8, 3 The SDM
More informationLecture 17: Primaldual interiorpoint methods part II
10725/36725: Convex Optimization Spring 2015 Lecture 17: Primaldual interiorpoint methods part II Lecturer: Javier Pena Scribes: Pinchao Zhang, Wei Ma Note: LaTeX template courtesy of UC Berkeley EECS
More informationFunctional Analysis I
Functional Analysis I Course Notes by Stefan Richter Transcribed and Annotated by Gregory Zitelli Polar Decomposition Definition. An operator W B(H) is called a partial isometry if W x = X for all x (ker
More informationl p Norm Constrained Quadratic Programming: Conic Approximation Methods
OUTLINE l p Norm Constrained Quadratic Programming: Conic Approximation Methods Wenxun Xing Department of Mathematical Sciences Tsinghua University, Beijing Email: wxing@math.tsinghua.edu.cn OUTLINE OUTLINE
More informationContinuous Optimisation, Chpt 9: Semidefinite Problems
Continuous Optimisation, Chpt 9: Semidefinite Problems Peter J.C. Dickinson DMMP, University of Twente p.j.c.dickinson@utwente.nl http://dickinson.website/teaching/2016co.html version: 21/11/16 Monday
More informationConstrained Optimization and Lagrangian Duality
CIS 520: Machine Learning Oct 02, 2017 Constrained Optimization and Lagrangian Duality Lecturer: Shivani Agarwal Disclaimer: These notes are designed to be a supplement to the lecture. They may or may
More informationOn selfconcordant barriers for generalized power cones
On selfconcordant barriers for generalized power cones Scott Roy Lin Xiao January 30, 2018 Abstract In the study of interiorpoint methods for nonsymmetric conic optimization and their applications, Nesterov
More informationLecture 14: Optimality Conditions for Conic Problems
EE 227A: Conve Optimization and Applications March 6, 2012 Lecture 14: Optimality Conditions for Conic Problems Lecturer: Laurent El Ghaoui Reading assignment: 5.5 of BV. 14.1 Optimality for Conic Problems
More informationResearch Note. A New Infeasible InteriorPoint Algorithm with Full NesterovTodd Step for SemiDefinite Optimization
Iranian Journal of Operations Research Vol. 4, No. 1, 2013, pp. 88107 Research Note A New Infeasible InteriorPoint Algorithm with Full NesterovTodd Step for SemiDefinite Optimization B. Kheirfam We
More informationE5295/5B5749 Convex optimization with engineering applications. Lecture 5. Convex programming and semidefinite programming
E5295/5B5749 Convex optimization with engineering applications Lecture 5 Convex programming and semidefinite programming A. Forsgren, KTH 1 Lecture 5 Convex optimization 2006/2007 Convex quadratic program
More informationLecture 8: Semidefinite programs for fidelity and optimal measurements
CS 766/QIC 80 Theory of Quantum Information (Fall 0) Lecture 8: Semidefinite programs for fidelity and optimal measurements This lecture is devoted to two examples of semidefinite programs: one is for
More informationChapter 2 Convex Analysis
Chapter 2 Convex Analysis The theory of nonsmooth analysis is based on convex analysis. Thus, we start this chapter by giving basic concepts and results of convexity (for further readings see also [202,
More informationConvex Optimization and SVM
Convex Optimization and SVM Problem 0. Cf lecture notes pages 12 to 18. Problem 1. (i) A slab is an intersection of two half spaces, hence convex. (ii) A wedge is an intersection of two half spaces, hence
More informationChapter 1. Preliminaries. The purpose of this chapter is to provide some basic background information. Linear Space. Hilbert Space.
Chapter 1 Preliminaries The purpose of this chapter is to provide some basic background information. Linear Space Hilbert Space Basic Principles 1 2 Preliminaries Linear Space The notion of linear space
More informationLecture: Algorithms for LP, SOCP and SDP
1/53 Lecture: Algorithms for LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2018.html wenzw@pku.edu.cn Acknowledgement:
More informationEE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 17
EE/ACM 150  Applications of Convex Optimization in Signal Processing and Communications Lecture 17 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory May 29, 2012 Andre Tkacenko
More informationLecture: Cone programming. Approximating the Lorentz cone.
Strong relaxations for discrete optimization problems 10/05/16 Lecture: Cone programming. Approximating the Lorentz cone. Lecturer: Yuri Faenza Scribes: Igor Malinović 1 Introduction Cone programming is
More informationLecture 1: Background on Convex Analysis
Lecture 1: Background on Convex Analysis John Duchi PCMI 2016 Outline I Convex sets 1.1 Definitions and examples 2.2 Basic properties 3.3 Projections onto convex sets 4.4 Separating and supporting hyperplanes
More informationOn smoothness properties of optimal value functions at the boundary of their domain under complete convexity
On smoothness properties of optimal value functions at the boundary of their domain under complete convexity Oliver Stein # Nathan SudermannMerx June 14, 2013 Abstract This article studies continuity
More informationWeek 3 Linear programming duality
Week 3 Linear programming duality This week we cover the fascinating topic of linear programming duality. We will learn that every minimization program has associated a maximization program that has the
More informationCHAPTER 2: CONVEX SETS AND CONCAVE FUNCTIONS. W. Erwin Diewert January 31, 2008.
1 ECONOMICS 594: LECTURE NOTES CHAPTER 2: CONVEX SETS AND CONCAVE FUNCTIONS W. Erwin Diewert January 31, 2008. 1. Introduction Many economic problems have the following structure: (i) a linear function
More informationDedicated to Michel Théra in honor of his 70th birthday
VARIATIONAL GEOMETRIC APPROACH TO GENERALIZED DIFFERENTIAL AND CONJUGATE CALCULI IN CONVEX ANALYSIS B. S. MORDUKHOVICH 1, N. M. NAM 2, R. B. RECTOR 3 and T. TRAN 4. Dedicated to Michel Théra in honor of
More informationReal Symmetric Matrices and Semidefinite Programming
Real Symmetric Matrices and Semidefinite Programming Tatsiana Maskalevich Abstract Symmetric real matrices attain an important property stating that all their eigenvalues are real. This gives rise to many
More informationSelected Examples of CONIC DUALITY AT WORK Robust Linear Optimization Synthesis of Linear Controllers Matrix Cube Theorem A.
. Selected Examples of CONIC DUALITY AT WORK Robust Linear Optimization Synthesis of Linear Controllers Matrix Cube Theorem A. Nemirovski Arkadi.Nemirovski@isye.gatech.edu Linear Optimization Problem,
More information