Lecture: Introduction to LP, SDP and SOCP


 Steven Daniel
 2 years ago
 Views:
Transcription
1 Lecture: Introduction to LP, SDP and SOCP Zaiwen Wen Beijing International Center For Mathematical Research Peking University Acknowledgement: this slides is based on Prof. Farid Alizadeh lecture notes
2 Linear Programming (LP) Primal Dual min c 1 x c n x n max b 1 y b m y m s.t. a 11 x a 1n x n = b 1... a m1 x a mn x n = b m x i 0 s.t. a 11 y a m1 y m c 1... a 1n y a mn y m c n 2/29
3 3/29 Linear Programming (LP) more succinctly Primal (P) Dual (D) min c x max b y s.t. Ax = b x 0 s.t. A y + s = c s 0
4 4/29 Weak duality Suppose x is feasible to (P) (y, s) is feasible to (D) Then 0 x s because x i s i 0 = x (c A y) = c x (Ax) y = c x b y = duality gap
5 5/29 Key Properties of LP Strong duality: If both Primal and Dual are feasible then at the optimum c x = b y x s = 0 complementary slackness: This implies x s = x 1 s x n s n = 0 and therefore x i s i = 0
6 6/29 complementarity Putting together primal feasibility, dual feasibility and complementarity together we get a square system of equations Ax = b A y + s = c x i s i = 0 for i = 1,..., n At least in principle this system determines the primal and dual optimal values
7 Algebraic characterization 7/29 We can define x s = (x 1 s 1,..., x n s n ) and L x : y (x 1 y 1,..., x n y n ) i.e. L x = Diag(x) We can write complementary slackness conditions as x s = L x s = L x L s 1 = 0 1, the vector of all ones, is the identity element: x 1 = x
8 Semidefinite Programming (SDP) 8/29 X Y means that the the symmetric matrix X Y is positive semidefinite X is positive semidefinite a Xa 0 for all vector a X = B B all eigenvalues of X is nonnegative
9 Semidefinite Programming (SDP) X, Y = ij X ijy ij = Tr(XY) Primal (P) min C 1, X C n, X n max Dual (D) b 1 y b m y m s.t. A 11, X A 1n, X n = b 1... A m1, X A mn, X n = b m X i 0 s.t. A 11 y A m1 y m + S 1 = c 1... A 1n y A mn y m + S n = c n S i 0 9/29
10 Simplified SDP 10/29 For simplicity we deal with single variable SDP: Primal (P) Dual (D) min C, X max s.t. A 1, X = b 1... A m, X = b m X 0 s.t. A single variable LP is trivial b y y i A i + S = C i S 0 But a single matrix SDP is as general as a multiple matrix
11 Weak duality in SDP 11/29 Just as in LP X, S = C, X b y Also if both X 0 and S 0 then because S 1/2 XS 1/2 0 X, S = Tr(XS 1/2 S 1/2 ) = Tr(S 1/2 XS 1/2 ) 0 Thus X, S = C, X b y 0
12 12/29 Complementarity Slackness Theorem X 0 and S 0 and X, S = 0 implies XS = 0 Proof: X, S = Tr(XS 1/2 S 1/2 ) = Tr(S 1/2 XS 1/2 ) Thus Tr(S 1/2 XS 1/2 ) = 0. Since S 1/2 XS 1/2 0, then S 1/2 XS 1/2 = 0 = S 1/2 X 1/2 X 1/2 S 1/2 = 0 X 1/2 S 1/2 = 0 = XS = 0
13 Algebraic properties of SDP 13/29 For reasons to become clear later it is better to write complementary slackness conditions as XS + SX 2 = 0 It can be shown that if X 0 and S 0, then XS = 0 iff XS + SX = 0
14 Algebraic properties of SDP 14/29 Definition: X S = XS+SX 2 The binary operation is commutative X S = S X is not associative: X (Y Z) (X Y) Z in general But X (X X) = (X X) X. Thus X p = X p is well defined In general X (X 2 Y) = X 2 (X Y) The identity matrix I is identity w.r.t Define the operator L X : Y X Y, thus X S = L X (S) = L X (L S (I))
15 Constraint Qualifications 15/29 Unlike LP we need some conditions for the optimal values of Primal and Dual SDP to coincide Here are two: If there is primalfeasible X 0 (i.e. X is positive definite) If there is dualfeasible S 0 When strong duality holds X, S = 0
16 KKT Condition 16/29 Thus just like LP The system of equations A i, X = b i, for i = 1,..., m y i A i + S = C i X S = 0 Gives us a square system
17 Second Order Cone Programming (SOCP) 17/29 For simplicity we deal with single variable SOCP: Primal (P) Dual (D) min c x max b y s.t. Ax = b x Q 0 s.t. A y + s = c s Q 0 the vectors x, s, c are indexed from zero If z = (z 0, z 1,..., z n ) and z = (z 1,..., z n ) z Q 0 z 0 z
18 Illustration of SOC 18/29 Q = {z z 0 z }
19 19/29 Weak Duality in SOCP The single block SOCP is not as trivial as LP but it still can be solved analytically weak duality: Again as in LP and SDP If x, s Q 0, then x s = c x b y = duality gap x s = x 0 s 0 + x s x s + x s since x, s Q 0 x s + x s CauchySchwartz inequality 0
20 Complementary Slackness for SOCP 20/29 Given x Q 0, s Q 0 and x s = 0. Assume x 0 > 0 and s 0 > 0 We have ( ) ( ) x 2 0 s 2 0 n i=1 x 2 i n s 2 i x0 2 i=1 n i=1 ( ) x s = 0 x 0 s 0 = i s 2 i x2 0 s 2 0 x i s i 2x 2 0 = Adding (*), (**), (***), we get 0 ( ) 2 n i=1 x i + s ix 0 s 0 n i=1 2x i s i x 0 s 0 This implies x i s 0 + x 0 s i = 0, for i = 1,..., n
21 Illustration of SOC 21/29 When x Q 0, s Q 0 are orthogonal both must be on the boundary in such a way that their projection on the x 1,..., x n plane is collinear
22 Strong Duality 22/29 at the optimum c x = b y x s = 0 Like SDP constraint qualifications are required If there is primalfeasible x Q 0 If there is dualfeasible s Q 0
23 23/29 Complementary Slackness for SOCP Thus again we have a square system Ax = b, A y + s = c x s = 0 x 0 s i + s 0 x i = 0
24 Algebraic properties of SOCP 24/29 Let us define a binary operation for vectors x and s both indexed from zero x 0 s 0 x s x 1. s 1. = x 0 s 1 + s 0 x 1. x n s n x 0 s n + s 0 x n
25 Algebraic properties of SOCP 25/29 The binary operation is commutative x s = s x is not associative: x (y z) (x y) z in general But x (x x) = (x x) x. Thus x p = x p is well defined In general x (x 2 y) = x 2 (x y) The identity matrix I is identity w.r.t e = (1, 0,..., 0) is the identity: x e = x
26 Algebraic properties of SOCP 26/29 Define the operator L x : y x y ( x0 x L x = Arw(x) = ) x x 0 I x s = Arw(x)s = Arw(x)Arw(s)e
27 Summary 27/29 Properties LP SDP SOCP ( ) x binary operator x s = (x i s i ) X S = XS+SX 2 x s = s x 0 s + s 0 x identity 1 I e = (1, 0,..., 0) associative yes no no L X y Diag(x)y Y XY+YX 2 y Arw(x)y Primal feasibility Ax = b A i, X = b i Ax = b dual feasility A y + s = c i y ia i + S = C A y + s = c complementarity L x L s 1 = 0 L X (L S (I)) = 0 L x L s e = 0
28 28/29 Conic LP A set K R n is a proper cone if It is a cone: If x K = ax K for all α 0 It is convex: x, y K = αx + (1 α)y K for α [0, 1] It is pointed: K ( K) = {0} It is closed It has nonempty interior in R n dual cone: K = {x for all z K, x, z 0}
29 Conic LP 29/29 ConicLP is defined as the following optimization problem: Primal (P) Dual (D) min c x max s.t. Ax = b x K s.t. For LP K is the nonnegative orthant b y A y + s = c s K For SDP K is the cone of positive semidefinite matrices For SOCP K is the circular or Lorentz cone In all three cases the cones are selfdual K = K
Lecture: Algorithms for LP, SOCP and SDP
1/53 Lecture: Algorithms for LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2018.html wenzw@pku.edu.cn Acknowledgement:
More informationSemidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 4
Semidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 4 Instructor: Farid Alizadeh Scribe: Haengju Lee 10/1/2001 1 Overview We examine the dual of the FermatWeber Problem. Next we will
More informationLecture: Examples of LP, SOCP and SDP
1/34 Lecture: Examples of LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2018.html wenzw@pku.edu.cn Acknowledgement:
More informationLecture: Duality of LP, SOCP and SDP
1/33 Lecture: Duality of LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2017.html wenzw@pku.edu.cn Acknowledgement:
More informationLecture 14: Optimality Conditions for Conic Problems
EE 227A: Conve Optimization and Applications March 6, 2012 Lecture 14: Optimality Conditions for Conic Problems Lecturer: Laurent El Ghaoui Reading assignment: 5.5 of BV. 14.1 Optimality for Conic Problems
More informationConic Linear Optimization and its Dual. yyye
Conic Linear Optimization and Appl. MS&E314 Lecture Note #04 1 Conic Linear Optimization and its Dual Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A.
More informationI.3. LMI DUALITY. Didier HENRION EECI Graduate School on Control Supélec  Spring 2010
I.3. LMI DUALITY Didier HENRION henrion@laas.fr EECI Graduate School on Control Supélec  Spring 2010 Primal and dual For primal problem p = inf x g 0 (x) s.t. g i (x) 0 define Lagrangian L(x, z) = g 0
More informationConvex Optimization M2
Convex Optimization M2 Lecture 3 A. d Aspremont. Convex Optimization M2. 1/49 Duality A. d Aspremont. Convex Optimization M2. 2/49 DMs DM par email: dm.daspremont@gmail.com A. d Aspremont. Convex Optimization
More information12. Interiorpoint methods
12. Interiorpoint methods Convex Optimization Boyd & Vandenberghe inequality constrained minimization logarithmic barrier function and central path barrier method feasibility and phase I methods complexity
More informationLecture 1. 1 Conic programming. MA 796S: Convex Optimization and Interior Point Methods October 8, Consider the conic program. min.
MA 796S: Convex Optimization and Interior Point Methods October 8, 2007 Lecture 1 Lecturer: Kartik Sivaramakrishnan Scribe: Kartik Sivaramakrishnan 1 Conic programming Consider the conic program min s.t.
More information15. Conic optimization
L. Vandenberghe EE236C (Spring 216) 15. Conic optimization conic linear program examples modeling duality 151 Generalized (conic) inequalities Conic inequality: a constraint x K where K is a convex cone
More informationELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications
ELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications Professor M. Chiang Electrical Engineering Department, Princeton University March
More informationSEMIDEFINITE PROGRAM BASICS. Contents
SEMIDEFINITE PROGRAM BASICS BRIAN AXELROD Abstract. A introduction to the basics of Semidefinite programs. Contents 1. Definitions and Preliminaries 1 1.1. Linear Algebra 1 1.2. Convex Analysis (on R n
More informationSummer School: Semidefinite Optimization
Summer School: Semidefinite Optimization Christine Bachoc Université Bordeaux I, IMB Research Training Group Experimental and Constructive Algebra Haus Karrenberg, Sept. 3  Sept. 7, 2012 Duality Theory
More informationLecture: Cone programming. Approximating the Lorentz cone.
Strong relaxations for discrete optimization problems 10/05/16 Lecture: Cone programming. Approximating the Lorentz cone. Lecturer: Yuri Faenza Scribes: Igor Malinović 1 Introduction Cone programming is
More informationLecture: Duality.
Lecture: Duality http://bicmr.pku.edu.cn/~wenzw/opt2016fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghe s lecture notes Introduction 2/35 Lagrange dual problem weak and strong
More informationCSC Linear Programming and Combinatorial Optimization Lecture 10: Semidefinite Programming
CSC2411  Linear Programming and Combinatorial Optimization Lecture 10: Semidefinite Programming Notes taken by Mike Jamieson March 28, 2005 Summary: In this lecture, we introduce semidefinite programming
More informationConvex Optimization. (EE227A: UC Berkeley) Lecture 6. Suvrit Sra. (Conic optimization) 07 Feb, 2013
Convex Optimization (EE227A: UC Berkeley) Lecture 6 (Conic optimization) 07 Feb, 2013 Suvrit Sra Organizational Info Quiz coming up on 19th Feb. Project teams by 19th Feb Good if you can mix your research
More informationSemidefinite Programming
Semidefinite Programming Basics and SOS Fernando Mário de Oliveira Filho Campos do Jordão, 2 November 23 Available at: www.ime.usp.br/~fmario under talks Conic programming V is a real vector space h, i
More informationMidterm Review. Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A.
Midterm Review Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye (LY, Chapter 14, Appendices) 1 Separating hyperplane
More informationDuality. Lagrange dual problem weak and strong duality optimality conditions perturbation and sensitivity analysis generalized inequalities
Duality Lagrange dual problem weak and strong duality optimality conditions perturbation and sensitivity analysis generalized inequalities Lagrangian Consider the optimization problem in standard form
More informationA Review of Linear Programming
A Review of Linear Programming Instructor: Farid Alizadeh IEOR 4600y Spring 2001 February 14, 2001 1 Overview In this note we review the basic properties of linear programming including the primal simplex
More informationLecture 6: Conic Optimization September 8
IE 598: Big Data Optimization Fall 2016 Lecture 6: Conic Optimization September 8 Lecturer: Niao He Scriber: Juan Xu Overview In this lecture, we finish up our previous discussion on optimality conditions
More information12. Interiorpoint methods
12. Interiorpoint methods Convex Optimization Boyd & Vandenberghe inequality constrained minimization logarithmic barrier function and central path barrier method feasibility and phase I methods complexity
More informationAgenda. 1 Duality for LP. 2 Theorem of alternatives. 3 Conic Duality. 4 Dual cones. 5 Geometric view of cone programs. 6 Conic duality theorem
Agenda 1 Duality for LP 2 Theorem of alternatives 3 Conic Duality 4 Dual cones 5 Geometric view of cone programs 6 Conic duality theorem 7 Examples Lower bounds on LPs By eliminating variables (if needed)
More informationIntroduction to Mathematical Programming IE406. Lecture 10. Dr. Ted Ralphs
Introduction to Mathematical Programming IE406 Lecture 10 Dr. Ted Ralphs IE406 Lecture 10 1 Reading for This Lecture Bertsimas 4.14.3 IE406 Lecture 10 2 Duality Theory: Motivation Consider the following
More informationLagrangian Duality Theory
Lagrangian Duality Theory Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye Chapter 14.14 1 Recall Primal and Dual
More informationInterior Point Methods: SecondOrder Cone Programming and Semidefinite Programming
School of Mathematics T H E U N I V E R S I T Y O H F E D I N B U R G Interior Point Methods: SecondOrder Cone Programming and Semidefinite Programming Jacek Gondzio Email: J.Gondzio@ed.ac.uk URL: http://www.maths.ed.ac.uk/~gondzio
More informationInterior Point Algorithms for Constrained Convex Optimization
Interior Point Algorithms for Constrained Convex Optimization Chee Wei Tan CS 8292 : Advanced Topics in Convex Optimization and its Applications Fall 2010 Outline Inequality constrained minimization problems
More information5. Duality. Lagrangian
5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationThe Q Method for SecondOrder Cone Programming
The Q Method for SecondOrder Cone Programming Yu Xia Farid Alizadeh July 5, 005 Key words. Secondorder cone programming, infeasible interior point method, the Q method Abstract We develop the Q method
More informationThe Q Method for Symmetric Cone Programmin
The Q Method for Symmetric Cone Programming The Q Method for Symmetric Cone Programmin Farid Alizadeh and Yu Xia alizadeh@rutcor.rutgers.edu, xiay@optlab.mcma Large Scale Nonlinear and Semidefinite Progra
More informationCS711008Z Algorithm Design and Analysis
CS711008Z Algorithm Design and Analysis Lecture 8 Linear programming: interior point method Dongbo Bu Institute of Computing Technology Chinese Academy of Sciences, Beijing, China 1 / 31 Outline Brief
More informationSemidefinite Programming Basics and Applications
Semidefinite Programming Basics and Applications Ray Pörn, principal lecturer Åbo Akademi University Novia University of Applied Sciences Content What is semidefinite programming (SDP)? How to represent
More informationProblem 1 (Exercise 2.2, Monograph)
MS&E 314/CME 336 Assignment 2 Conic Linear Programming January 3, 215 Prof. Yinyu Ye 6 Pages ASSIGNMENT 2 SOLUTIONS Problem 1 (Exercise 2.2, Monograph) We prove the part ii) of Theorem 2.1 (Farkas Lemma
More informationSemidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 5
Semidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 5 Instructor: Farid Alizadeh Scribe: Anton Riabov 10/08/2001 1 Overview We continue studying the maximum eigenvalue SDP, and generalize
More informationContinuous Optimisation, Chpt 9: Semidefinite Problems
Continuous Optimisation, Chpt 9: Semidefinite Problems Peter J.C. Dickinson DMMP, University of Twente p.j.c.dickinson@utwente.nl http://dickinson.website/teaching/2016co.html version: 21/11/16 Monday
More information14. Duality. ˆ Upper and lower bounds. ˆ General duality. ˆ Constraint qualifications. ˆ Counterexample. ˆ Complementary slackness.
CS/ECE/ISyE 524 Introduction to Optimization Spring 2016 17 14. Duality ˆ Upper and lower bounds ˆ General duality ˆ Constraint qualifications ˆ Counterexample ˆ Complementary slackness ˆ Examples ˆ Sensitivity
More informationLecture 5. The Dual Cone and Dual Problem
IE 8534 1 Lecture 5. The Dual Cone and Dual Problem IE 8534 2 For a convex cone K, its dual cone is defined as K = {y x, y 0, x K}. The innerproduct can be replaced by x T y if the coordinates of the
More informationLinear and Combinatorial Optimization
Linear and Combinatorial Optimization The dual of an LPproblem. Connections between primal and dual. Duality theorems and complementary slack. Philipp Birken (Ctr. for the Math. Sc.) Lecture 3: Duality
More informationSemidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 2
Semidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 2 Instructor: Farid Alizadeh Scribe: Xuan Li 9/17/2001 1 Overview We survey the basic notions of cones and conelp and give several
More informationIII. Applications in convex optimization
III. Applications in convex optimization nonsymmetric interiorpoint methods partial separability and decomposition partial separability first order methods interiorpoint methods Conic linear optimization
More informationOptimality, Duality, Complementarity for Constrained Optimization
Optimality, Duality, Complementarity for Constrained Optimization Stephen Wright University of WisconsinMadison May 2014 Wright (UWMadison) Optimality, Duality, Complementarity May 2014 1 / 41 Linear
More informationLecture 10: Linear programming duality and sensitivity 00
Lecture 10: Linear programming duality and sensitivity 00 The canonical primal dual pair 1 A R m n, b R m, and c R n maximize z = c T x (1) subject to Ax b, x 0 n and minimize w = b T y (2) subject to
More informationEE 227A: Convex Optimization and Applications October 14, 2008
EE 227A: Convex Optimization and Applications October 14, 2008 Lecture 13: SDP Duality Lecturer: Laurent El Ghaoui Reading assignment: Chapter 5 of BV. 13.1 Direct approach 13.1.1 Primal problem Consider
More information4. Algebra and Duality
41 Algebra and Duality P. Parrilo and S. Lall, CDC 2003 2003.12.07.01 4. Algebra and Duality Example: nonconvex polynomial optimization Weak duality and duality gap The dual is not intrinsic The cone
More informationE5295/5B5749 Convex optimization with engineering applications. Lecture 5. Convex programming and semidefinite programming
E5295/5B5749 Convex optimization with engineering applications Lecture 5 Convex programming and semidefinite programming A. Forsgren, KTH 1 Lecture 5 Convex optimization 2006/2007 Convex quadratic program
More informationLecture Note 5: Semidefinite Programming for Stability Analysis
ECE7850: Hybrid Systems:Theory and Applications Lecture Note 5: Semidefinite Programming for Stability Analysis Wei Zhang Assistant Professor Department of Electrical and Computer Engineering Ohio State
More informationLECTURE 13 LECTURE OUTLINE
LECTURE 13 LECTURE OUTLINE Problem Structures Separable problems Integer/discrete problems Branchandbound Large sum problems Problems with many constraints Conic Programming Second Order Cone Programming
More informationLagrange Duality. Daniel P. Palomar. Hong Kong University of Science and Technology (HKUST)
Lagrange Duality Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470  Convex Optimization Fall 201718, HKUST, Hong Kong Outline of Lecture Lagrangian Dual function Dual
More informationLecture: Convex Optimization Problems
1/36 Lecture: Convex Optimization Problems http://bicmr.pku.edu.cn/~wenzw/opt2015fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghe s lecture notes Introduction 2/36 optimization
More information61 The Positivstellensatz P. Parrilo and S. Lall, ECC
61 The Positivstellensatz P. Parrilo and S. Lall, ECC 2003 2003.09.02.10 6. The Positivstellensatz Basic semialgebraic sets Semialgebraic sets TarskiSeidenberg and quantifier elimination Feasibility
More informationSemidefinite Programming, Combinatorial Optimization and Real Algebraic Geometry
Semidefinite Programming, Combinatorial Optimization and Real Algebraic Geometry assoc. prof., Ph.D. 1 1 UNM  Faculty of information studies Edinburgh, 16. September 2014 Outline Introduction Definition
More informationConvex Optimization & Lagrange Duality
Convex Optimization & Lagrange Duality Chee Wei Tan CS 8292 : Advanced Topics in Convex Optimization and its Applications Fall 2010 Outline Convex optimization Optimality condition Lagrange duality KKT
More informationLecture 7: Convex Optimizations
Lecture 7: Convex Optimizations Radu Balan, David Levermore March 29, 2018 Convex Sets. Convex Functions A set S R n is called a convex set if for any points x, y S the line segment [x, y] := {tx + (1
More informationIntroduction to Machine Learning Lecture 7. Mehryar Mohri Courant Institute and Google Research
Introduction to Machine Learning Lecture 7 Mehryar Mohri Courant Institute and Google Research mohri@cims.nyu.edu Convex Optimization Differentiation Definition: let f : X R N R be a differentiable function,
More informationConvex Optimization and Modeling
Convex Optimization and Modeling Convex Optimization Fourth lecture, 05.05.2010 Jun.Prof. Matthias Hein Reminder from last time Convex functions: firstorder condition: f(y) f(x) + f x,y x, secondorder
More informationMore FirstOrder Optimization Algorithms
More FirstOrder Optimization Algorithms Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye Chapters 3, 8, 3 The SDM
More informationLecture 4: January 26
10725/36725: Conve Optimization Spring 2015 Lecturer: Javier Pena Lecture 4: January 26 Scribes: Vipul Singh, Shinjini Kundu, ChiaYin Tsai Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer:
More informationConvex Optimization Boyd & Vandenberghe. 5. Duality
5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationUsing Schur Complement Theorem to prove convexity of some SOCfunctions
Journal of Nonlinear and Convex Analysis, vol. 13, no. 3, pp. 41431, 01 Using Schur Complement Theorem to prove convexity of some SOCfunctions JeinShan Chen 1 Department of Mathematics National Taiwan
More informationLecture 17: Primaldual interiorpoint methods part II
10725/36725: Convex Optimization Spring 2015 Lecture 17: Primaldual interiorpoint methods part II Lecturer: Javier Pena Scribes: Pinchao Zhang, Wei Ma Note: LaTeX template courtesy of UC Berkeley EECS
More informationConvex Optimization and Modeling
Convex Optimization and Modeling Duality Theory and Optimality Conditions 5th lecture, 12.05.2010 Jun.Prof. Matthias Hein Program of today/next lecture Lagrangian and duality: the Lagrangian the dual
More informationLinear and nonlinear programming
Linear and nonlinear programming Benjamin Recht March 11, 2005 The Gameplan Constrained Optimization Convexity Duality Applications/Taxonomy 1 Constrained Optimization minimize f(x) subject to g j (x)
More informationCuts for mixed 01 conic programs
Cuts for mixed 01 conic programs G. Iyengar 1 M. T. Cezik 2 1 IEOR Department Columbia University, New York. 2 GERAD Université de Montréal, Montréal TUChemnitz Workshop on Integer Programming and Continuous
More informationLecture 3: Lagrangian duality and algorithms for the Lagrangian dual problem
Lecture 3: Lagrangian duality and algorithms for the Lagrangian dual problem Michael Patriksson 00 The Relaxation Theorem 1 Problem: find f := infimum f(x), x subject to x S, (1a) (1b) where f : R n R
More informationLecture 5. Theorems of Alternatives and SelfDual Embedding
IE 8534 1 Lecture 5. Theorems of Alternatives and SelfDual Embedding IE 8534 2 A system of linear equations may not have a solution. It is well known that either Ax = c has a solution, or A T y = 0, c
More informationConvexification of MixedInteger Quadratically Constrained Quadratic Programs
Convexification of MixedInteger Quadratically Constrained Quadratic Programs Laura Galli 1 Adam N. Letchford 2 Lancaster, April 2011 1 DEIS, University of Bologna, Italy 2 Department of Management Science,
More informationMotivation. Lecture 2 Topics from Optimization and Duality. network utility maximization (NUM) problem:
CDS270 Maryam Fazel Lecture 2 Topics from Optimization and Duality Motivation network utility maximization (NUM) problem: consider a network with S sources (users), each sending one flow at rate x s, through
More informationExample Problem. Linear Program (standard form) CSCI5654 (Linear Programming, Fall 2013) Lecture7. Duality
CSCI5654 (Linear Programming, Fall 013) Lecture7 Duality Lecture 7 Slide# 1 Lecture 7 Slide# Linear Program (standard form) Example Problem maximize c 1 x 1 + + c n x n s.t. a j1 x 1 + + a jn x n b j
More informationIntroduction to Semidefinite Programming I: Basic properties a
Introduction to Semidefinite Programming I: Basic properties and variations on the GoemansWilliamson approximation algorithm for maxcut MFO seminar on Semidefinite Programming May 30, 2010 Semidefinite
More informationBBM402Lecture 20: LP Duality
BBM402Lecture 20: LP Duality Lecturer: Lale Özkahya Resources for the presentation: https://courses.engr.illinois.edu/cs473/fa2016/lectures.html An easy LP? which is compact form for max cx subject to
More informationLecture #21. c T x Ax b. maximize subject to
COMPSCI 330: Design and Analysis of Algorithms 11/11/2014 Lecture #21 Lecturer: Debmalya Panigrahi Scribe: Samuel Haney 1 Overview In this lecture, we discuss linear programming. We first show that the
More informationL. Vandenberghe EE236C (Spring 2016) 18. Symmetric cones. definition. spectral decomposition. quadratic representation. logdet barrier 181
L. Vandenberghe EE236C (Spring 2016) 18. Symmetric cones definition spectral decomposition quadratic representation logdet barrier 181 Introduction This lecture: theoretical properties of the following
More informationGlobal Optimization of Polynomials
Semidefinite Programming Lecture 9 OR 637 Spring 2008 April 9, 2008 Scribe: Dennis Leventhal Global Optimization of Polynomials Recall we were considering the problem min z R n p(z) where p(z) is a degree
More informationConvex Optimization and SVM
Convex Optimization and SVM Problem 0. Cf lecture notes pages 12 to 18. Problem 1. (i) A slab is an intersection of two half spaces, hence convex. (ii) A wedge is an intersection of two half spaces, hence
More informationSemidefinite and Second Order Cone Programming Seminar Fall 2012 Project: Robust Optimization and its Application of Robust Portfolio Optimization
Semidefinite and Second Order Cone Programming Seminar Fall 2012 Project: Robust Optimization and its Application of Robust Portfolio Optimization Instructor: Farid Alizadeh Author: Ai Kagawa 12/12/2012
More informationChapter 1. Preliminaries
Introduction This dissertation is a reading of chapter 4 in part I of the book : Integer and Combinatorial Optimization by George L. Nemhauser & Laurence A. Wolsey. The chapter elaborates links between
More informationConic Linear Programming. Yinyu Ye
Conic Linear Programming Yinyu Ye December 2004, revised October 2017 i ii Preface This monograph is developed for MS&E 314, Conic Linear Programming, which I am teaching at Stanford. Information, lecture
More informationPrimalDual Geometry of Level Sets and their Explanatory Value of the Practical Performance of InteriorPoint Methods for Conic Optimization
PrimalDual Geometry of Level Sets and their Explanatory Value of the Practical Performance of InteriorPoint Methods for Conic Optimization Robert M. Freund M.I.T. June, 2010 from papers in SIOPT, Mathematics
More informationInner products and Norms. Inner product of 2 vectors. Inner product of 2 vectors x and y in R n : x 1 y 1 + x 2 y x n y n in R n
Inner products and Norms Inner product of 2 vectors Inner product of 2 vectors x and y in R n : x 1 y 1 + x 2 y 2 + + x n y n in R n Notation: (x, y) or y T x For complex vectors (x, y) = x 1 ȳ 1 + x 2
More informationAgenda. 1 Cone programming. 2 Convex cones. 3 Generalized inequalities. 4 Linear programming (LP) 5 Secondorder cone programming (SOCP)
Agenda 1 Cone programming 2 Convex cones 3 Generalized inequalities 4 Linear programming (LP) 5 Secondorder cone programming (SOCP) 6 Semidefinite programming (SDP) 7 Examples Optimization problem in
More informationEE/AA 578, Univ of Washington, Fall Duality
7. Duality EE/AA 578, Univ of Washington, Fall 2016 Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationSecondOrder Cone Programming
SecondOrder Cone Programming F. Alizadeh D. Goldfarb 1 Introduction Secondorder cone programming (SOCP) problems are convex optimization problems in which a linear function is minimized over the intersection
More information4.6 Linear Programming duality
4.6 Linear Programming duality To any minimization (maximization) LP we can associate a closely related maximization (minimization) LP Different spaces and objective functions but in general same optimal
More informationCSCI5654 (Linear Programming, Fall 2013) Lecture8. Lecture 8 Slide# 1
CSCI5654 (Linear Programming, Fall 2013) Lecture8 Lecture 8 Slide# 1 Today s Lecture 1. Recap of dual variables and strong duality. 2. Complementary Slackness Theorem. 3. Interpretation of dual variables.
More information4TE3/6TE3. Algorithms for. Continuous Optimization
4TE3/6TE3 Algorithms for Continuous Optimization (Duality in Nonlinear Optimization ) Tamás TERLAKY Computing and Software McMaster University Hamilton, January 2004 terlaky@mcmaster.ca Tel: 27780 Optimality
More informationSDP Relaxations for MAXCUT
SDP Relaxations for MAXCUT from Random Hyperplanes to SumofSquares Certificates CATS @ UMD March 3, 2017 Ahmed Abdelkader MAXCUT SDP SOS March 3, 2017 1 / 27 Overview 1 MAXCUT, Hardness and UGC 2 LP
More informationSemidefinite Programming
Chapter 2 Semidefinite Programming 2.0.1 Semidefinite programming (SDP) Given C M n, A i M n, i = 1, 2,..., m, and b R m, the semidefinite programming problem is to find a matrix X M n for the optimization
More information4. Convex optimization problems
Convex Optimization Boyd & Vandenberghe 4. Convex optimization problems optimization problem in standard form convex optimization problems quasiconvex optimization linear optimization quadratic optimization
More informationContinuous Optimisation, Chpt 9: Semidefinite Optimisation
Continuous Optimisation, Chpt 9: Semidefinite Optimisation Peter J.C. Dickinson DMMP, University of Twente p.j.c.dickinson@utwente.nl http://dickinson.website/teaching/2017co.html version: 28/11/17 Monday
More informationThe fundamental theorem of linear programming
The fundamental theorem of linear programming Michael Tehranchi June 8, 2017 This note supplements the lecture notes of Optimisation The statement of the fundamental theorem of linear programming and the
More informationm i=1 c ix i i=1 F ix i F 0, X O.
What is SDP? for a beginner of SDP Copyright C 2005 SDPA Project 1 Introduction This note is a short course for SemiDefinite Programming s SDP beginners. SDP has various applications in, for example, control
More informationLecture 7 Duality II
L. Vandenberghe EE236A (Fall 201314) Lecture 7 Duality II sensitivity analysis twoperson zerosum games circuit interpretation 7 1 Sensitivity analysis purpose: extract from the solution of an LP information
More informationSecondorder cone programming
Outline Secondorder cone programming, PhD Lehigh University Department of Industrial and Systems Engineering February 10, 2009 Outline 1 Basic properties Spectral decomposition The cone of squares The
More information1 Outline Part I: Linear Programming (LP) InteriorPoint Approach 1. Simplex Approach Comparison Part II: Semidenite Programming (SDP) Concludin
Sensitivity Analysis in LP and SDP Using InteriorPoint Methods E. Alper Yldrm School of Operations Research and Industrial Engineering Cornell University Ithaca, NY joint with Michael J. Todd INFORMS
More informationConic Linear Programming. Yinyu Ye
Conic Linear Programming Yinyu Ye December 2004, revised January 2015 i ii Preface This monograph is developed for MS&E 314, Conic Linear Programming, which I am teaching at Stanford. Information, lecture
More informationExample: feasibility. Interpretation as formal proof. Example: linear inequalities and Farkas lemma
41 Algebra and Duality P. Parrilo and S. Lall 2006.06.07.01 4. Algebra and Duality Example: nonconvex polynomial optimization Weak duality and duality gap The dual is not intrinsic The cone of valid
More informationLecture 8 Plus properties, merit functions and gap functions. September 28, 2008
Lecture 8 Plus properties, merit functions and gap functions September 28, 2008 Outline Plusproperties and Funiqueness Equation reformulations of VI/CPs Merit functions Gap merit functions FPI book:
More informationYinyu Ye, MS&E, Stanford MS&E310 Lecture Note #06. The Simplex Method
The Simplex Method Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye (LY, Chapters 2.32.5, 3.13.4) 1 Geometry of Linear
More information