Agenda. 1 Duality for LP. 2 Theorem of alternatives. 3 Conic Duality. 4 Dual cones. 5 Geometric view of cone programs. 6 Conic duality theorem
|
|
- Marianna Houston
- 5 years ago
- Views:
Transcription
1 Agenda 1 Duality for LP 2 Theorem of alternatives 3 Conic Duality 4 Dual cones 5 Geometric view of cone programs 6 Conic duality theorem 7 Examples
2 Lower bounds on LPs By eliminating variables (if needed) any LP can be put in inequality form minimize c T x subject to Ax b We use this form for simplicity Purpose of Lagrange duality: get a lower bound on optimal value p?
3 Lagrange duality Lagrangian: L : R n R m! R L(x, )=c T x T (Ax b) : Lagrange multipliers (Lagrange) dual function g( )=inf x2r n L(x, )= Suppose 0 & x is feasible ( b T A T = c 1 otherwise L(x, )=c T x T (Ax b) apple c T x This implies that g( ) apple c T x for all feasible x ) g( ) apple inf x:ax b ct x = p?! Dual function always gives a lower bound on optimal value p?
4 Dual problem Dual problem: best lower bound on p? maximize g( ) subject to 0 () maximize subject to A T = c 0 b T (Lagrange) dual problem This is an LP Optimal value d? obeys d? apple p? :weakduality
5 LP duality Theorem For LP, strong duality always holds (remarkable and very useful fact) d? = p? p? = 1 (primal infeasible) =) d? = p? p? = 1 =) d? = 1 (dual infeasible)
6 Theorem of alternatives and strong LP duality min {c T x : Ax b} How do we decide if LP is feasible? How can we certify feasible set is empty? One possible approach: combine inequalities and get a contradiction 0 and Ax b =) T Ax T b Get such a contradiction if A T =0and b T >0 (only way to be sure) Remarkably, this is su cient but also necessary Theorem (Theorem of alternatives) Exactly one of (1) and (2) is true (1) 9x s.t. Ax b (2) 9 0 s.t. A T =0and b T >0
7 Lower bounds for LPs min {c T x : Ax b} Theorem of alternatives: a apple p? i a apple p?, {x : Ax b and c T x<a} = ; 9( 0, ) 0 s.t. 0c + A T =0 0 + b T 0 If LP is feasible, can take 0 > 0, hencea apple p? i system below has a solution (D) A T b T 0 = c a Proposition (D) has a sol (, a) =) a apple p? LP is feasible and a apple p? =) existence of a sol to (D)
8 Strong duality Proposition (D) has a sol (, a) =) a apple p? LP is feasible and a apple p? =) existence of a sol to (D) (, a) sol to (D) =) (, b T ) sol to (D) Since b T a, best lower bound max {b T : A T = c, 0} Conclusion This proves strong duality feasible for dual problem =) b T apple p? (d? apple p? ) If LP feasible, then for every a apple p?, 9 dual feasible with b T a ) d? = p?
9 Proof of theorem of alternatives
10 Similar notion for cone programs? Conic program minimize subject to c T x Ax K b Interested in something similar to LP s 0 and Ax b 0 =) h, Ax bi 0 For cones, we would like a set C such that 2 C and a K 0 =) h, ai 0 If 2 C and x is feasible, we get a lower bound hc, xi h, Ax bi applehc, xi
11 Or similarly For LP s: 0 gives min {c T x : Ax K 0} Ax b =) T Ax T b If A T = c, we get a lower bound For cones, we would like a set C such that 2 C and a K 0 =) h, ai 0 Best lower bound max {b T : 2 C and A T = c}
12 Dual cones Definition Dual cone K K = { 2 R n : T a 0 8a 2 K}
13 Properties of dual cone K 6= ; (nonempty set) 1 K is a closed convex cone 2 int(k) 6= ; =) K is pointed 3 K closed, convex and pointed =) int(k ) 6= ; 4 K closed, convex and pointed =) K closed and convex (K ) = K Corollary: K proper () K proper
14 Dual of a conic problem Primal (P) minimize subject to c T x Ax K b Lagrangian L(x, )=c T x T (Ax b) 2 K Dual (D) maximize subject to A T = c 2 K b T Weak duality: d apple p x feasible, feasible =) b T apple c T x
15 Dual of conic problem, continued (P) minimize c T x subject to A i x Ki b i, i =1,...,m (D) maximize subject to P b T P i i A T i i = c i K i 0, i =1,...,m
16 Geometrical view of primal/dual pair (P) minimize subject to c T x Ax K b (D) maximize subject to A T = c K 0 b T Primal: minimize linear function over an a ne slice of cone K Dual: maximize linear function over an a ne slice of cone K Same problem: only di erence due to how we represent the problem
17 Assume 9d : c = A T d () c 2 row(a) otherwise dual is infeasible: d? = 1 and p? 2 {±1} Assume for simplicity A has full column rank (so that c = A T d as a solution) Primal feasible set: v = Ax b (i) v 2 V b, V = range(a) (ii) v K 0 (iii) Gives c T x = d T Ax = d T b + d T v Conic program can be formulated as min {d T b + d T v : v 2 V b, v K 0} Dual feasible set: A T = c, 2 K () 2 V? + d, 2 K So that dual problem is max {b T : 2 V? + d, K 0}
18 Geometric view of primal/dual pair, continued dual feasible K * primal feasible R - b K
19 Symmetry of duality (P) minimize subject to c T x Ax K b (D) If K is proper, then (P) is dual to (D) because K = K maximize subject to A T = c K 0 b T dual of (D) minimize T c subject to v K 0 v = A b () minimize ct x subject to Ax b K 0
20 Examples of dual cones The following are all self-dual cones: (a) K = R n + K = { : X ia i 0 8a 0} = R n + (b) K = L R n+1, L = {(x, t) :kxk applet} (y, s) 2 K () (y, x)+st 0 8(x, t) 2 L t x
21 L = {(x, t) :kxk applet} (y, s) 2 L () hy, xi + st 0 8(x, t) 2 L Claim: L = L (i) L L Take (y, s) 2 L, hy, xi + st kykkxk + st 0 (ii) L L Take (y, s) 2 L and set (x, t) =( y, kyk) 2 L hy, yi + skyk 0 =) kyk apples () (y, s) 2 L
22 (c) K = S n + Claim: K = K K = { 2 S n : Tr( X) 0 8X 0} (i) K S n + Take 2 K. Then for all x 2 R n and so 0 Tr( xx T )=Tr(x T x) =x T x 0 (ii) S n + K Take 0 and decompose X 0 as X = P kx k x T k,with k 0 Tr(X ) = X k ktr(x k x T k ) = X k kx T k x k 0
23 Implication of conic self duality Primal LP SOCP SDP Dual LP SOCP SDP
24 Example: minimum total-variation denoising Constraints minimize kxk TV subject to kx bk apple kx bk apple () (D ij x, t ij ) 2 L ij () () apple x b 2 L0 () apple D 0 0 I P tij minimize subject to kx bk apple kd ij xkapplet ij apple x t apple I L 0 apple x t L0 apple b L = Y L ij
25 Dual maximize T 0 apple b subject to 0 L0 0 ij Lij 0 Dual variables apple I apple u0 apple D T 0 s 0 0 I apple u s + apple 0 1 =0 0 = apple apple u0 uij : ku s 0 kapples 0 ij = 0 s ij : ku ij kapples ij Dual problem (SOCP) max hu 0,bi s 0 s. t. ku 0 kapples 0, ku ij kapples ij u 0 + D T u =0 s =1 () max hb, DT ui kd T uk s. t. ku ij kapple1
26 Example: SDP maximize hc, xi subject to x 1 F x m F m F 0 (F i 2 S p ) Dual cone problem maximize hf 0, i subject to 0 hf i, i = c i, i =1,...,m
27 SDP formulation minimize ka 0 (x 1 A x m A m )k minimize subject to t apple ti A(x) A T (x) ti with A(x) =A 0 (x 1 A x m A m ) apple I 0 X apple 0 LMI () t x 0 I i A T i 0 i Ai 0 apple 0 A0 + A T 0 0 0
28 Dual problem With = apple T 12 22, this gives apple 0 A0 maximize h A T, i 0 0 subject to apple 0 0 Ai h A T, i =0 apple i 0 I 0 h, i =1 0 I maximize 2Tr(A T 0 12 ) subject to Tr(A T i 12) =0 Tr( ) = 1 0 () maximize 2Tr(A T 0 12 ) subject to Tr(A T i 12) =0 Tr( 11 )+Tr( 22 )=1 apple T
29 Conic duality theorem Can get results paralleling LP duality provided primal is strictly feasible Strict feasibility means that V b \ int(k) 6= ; Theorem (1) Assume (P) is strictly feasible (9x : Ax K b)andp? 6= 1. Then(D)is solvable and d? = p?. (2) Assume (D) is strictly feasible (9 : K 0,AT = c) andd? 6= 1. Then (P) is solvable and d? = p?. (3) Assume one of the problems is bounded and strictly feasible. Then (x, ) primal/dual optimal i b T = c T x () T (Ax b) =0
30 Proof (1) Claim: If (P) is strictly feasible and 9x : Ax K b then (D) is solvable and d? = p? Enough to show that 9 dual feasible s.t. b T p? If c =0,thenp? =0and? =0does the job. So we assume c 6= 0. M = {Ax b : x 2 R n,c T x apple p? } R - b K M 6= ; (image of halfspace) M \ int(k) =; Why? Assume v 2 M \ int(k) () v = Ax 0 b with Ax 0 b K 0 and c T x 0 apple p? and hence x 0 is primal optimal 9 0 s.t. Ax b 0 8x : kx x 0kapple c 6= 0 =) 9x s.t. kx x 0kapple and c T x<c T x 0 apple p? =)(= M d
31 M is convex, nonempty, and M \ int(k) =; Separation theorem for convex sets: 9 6= 0 s.t sup h, vi apple inf h, vi v2m v2int(k) hence T v bounded below on int(k) =) T y 0 8y 2 int(k) =) T y 0 8y 2 K () 2 K In summary sup h, vi apple0 & 2 K () 2 K v2m T (Ax b) apple 0 8x : c T x apple p?! Linear form bounded above on a half-plane. This is possible only if A T = µc µ 0
32 We claim µ>0 Why? µ =0 =) A T =0, T b 0 Take x 0 s.t. Ax 0 b K 0 This is a contradiction K 0 and 6= 0 =) T (Ax 0 b) > 0 =) T b>0? = µ 1 obeys? 2 K?, A T? = c, hc, xi appleh?,bi 8x : c T x apple p? Taking an x such that c T x = p? =) value of dual objective is at least p?
33 End of proof (2) Same by symmetry (3) x primal feasible, dual feasible =) c T x b T =(Ax b) T! Complementary slackness i zero duality gap Enough to show (x, ) optimal () DG =0 DG is zero gives c T x b T = c T x p? + d? b T + p? d? =0 since c T x p? =0and d? b T =0must hold In conclusion, x primal optimal dual optimal nota bene: no strict feasibility required Conversely, p? = d? =) c T x = b T
34 Example Primal / dual strict feasibility (P) minimize c T x subject to x 1 F x n F n F 0 (D) maximize hf 0, i subject to 0 hf i, i = c i (x, ) optimal () c T x hf 0, i = h X x i F i F 0, i =0
35 Example for which p? >d? Feasible set: apple 0 x1 x 1 x 1 + x 2 0 x minimize x2 1 0 x subject to 4x 1 x 1 + x x 1 +1 () x 1 + x 2 0 x x 1 1 () x 1 =0 x 2 0 ) p? =0
36 Dual x x which gives maximize 2 33 subject to dual feasible =) 33 = ) d? = SDP violates conditions of conic duality theorem Both (P) and (D) are feasible, but not strictly 6= LP where strong duality always holds
37 Nuclear norm Theorem Operator norm and nuclear norm are dual Proof: want to show kxk = max ky kapple1 hy,xi (a) Take SVD of X = U V T and set Y = UV T.Then (b) In the other direction (D) maximize subject to Tr(Y T X)=Tr( ) = kxk =) max ky kapple1 hy,xi hy,xi apple I Y Y T I 0 duality $ (P) kxk 1 minimize apple2 Tr( 1)+ 1 2 Tr( 2) 1 X subject to X T 0 2 (D) has variable Y and (P) has variables 1, 2 1 = U U T 2 = V V T =) apple 1 X X T 2 = apple U V apple 0 0 apple U V T val(d) apple val(p) apple Tr( ) = kxk
38 Consequence minimize subject to kxk X 2C If C is SDP-representable, this is an SDP; e.g. A(X) =b ka(x) bk 2 apple ka (A(X) b)k apple... CVX knows about this
I.3. LMI DUALITY. Didier HENRION EECI Graduate School on Control Supélec - Spring 2010
I.3. LMI DUALITY Didier HENRION henrion@laas.fr EECI Graduate School on Control Supélec - Spring 2010 Primal and dual For primal problem p = inf x g 0 (x) s.t. g i (x) 0 define Lagrangian L(x, z) = g 0
More informationConvex Optimization M2
Convex Optimization M2 Lecture 3 A. d Aspremont. Convex Optimization M2. 1/49 Duality A. d Aspremont. Convex Optimization M2. 2/49 DMs DM par email: dm.daspremont@gmail.com A. d Aspremont. Convex Optimization
More information4. Algebra and Duality
4-1 Algebra and Duality P. Parrilo and S. Lall, CDC 2003 2003.12.07.01 4. Algebra and Duality Example: non-convex polynomial optimization Weak duality and duality gap The dual is not intrinsic The cone
More informationIntroduction to Mathematical Programming IE406. Lecture 10. Dr. Ted Ralphs
Introduction to Mathematical Programming IE406 Lecture 10 Dr. Ted Ralphs IE406 Lecture 10 1 Reading for This Lecture Bertsimas 4.1-4.3 IE406 Lecture 10 2 Duality Theory: Motivation Consider the following
More informationLP Duality: outline. Duality theory for Linear Programming. alternatives. optimization I Idea: polyhedra
LP Duality: outline I Motivation and definition of a dual LP I Weak duality I Separating hyperplane theorem and theorems of the alternatives I Strong duality and complementary slackness I Using duality
More informationConic Linear Optimization and its Dual. yyye
Conic Linear Optimization and Appl. MS&E314 Lecture Note #04 1 Conic Linear Optimization and its Dual Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A.
More informationLecture 14: Optimality Conditions for Conic Problems
EE 227A: Conve Optimization and Applications March 6, 2012 Lecture 14: Optimality Conditions for Conic Problems Lecturer: Laurent El Ghaoui Reading assignment: 5.5 of BV. 14.1 Optimality for Conic Problems
More informationLecture: Introduction to LP, SDP and SOCP
Lecture: Introduction to LP, SDP and SOCP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2015.html wenzw@pku.edu.cn Acknowledgement:
More informationOptimization for Communications and Networks. Poompat Saengudomlert. Session 4 Duality and Lagrange Multipliers
Optimization for Communications and Networks Poompat Saengudomlert Session 4 Duality and Lagrange Multipliers P Saengudomlert (2015) Optimization Session 4 1 / 14 24 Dual Problems Consider a primal convex
More informationLagrangian Duality Theory
Lagrangian Duality Theory Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye Chapter 14.1-4 1 Recall Primal and Dual
More informationExample: feasibility. Interpretation as formal proof. Example: linear inequalities and Farkas lemma
4-1 Algebra and Duality P. Parrilo and S. Lall 2006.06.07.01 4. Algebra and Duality Example: non-convex polynomial optimization Weak duality and duality gap The dual is not intrinsic The cone of valid
More informationLagrange duality. The Lagrangian. We consider an optimization program of the form
Lagrange duality Another way to arrive at the KKT conditions, and one which gives us some insight on solving constrained optimization problems, is through the Lagrange dual. The dual is a maximization
More informationSemidefinite Programming
Semidefinite Programming Basics and SOS Fernando Mário de Oliveira Filho Campos do Jordão, 2 November 23 Available at: www.ime.usp.br/~fmario under talks Conic programming V is a real vector space h, i
More information5. Duality. Lagrangian
5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationWHY DUALITY? Gradient descent Newton s method Quasi-newton Conjugate gradients. No constraints. Non-differentiable ???? Constrained problems? ????
DUALITY WHY DUALITY? No constraints f(x) Non-differentiable f(x) Gradient descent Newton s method Quasi-newton Conjugate gradients etc???? Constrained problems? f(x) subject to g(x) apple 0???? h(x) =0
More informationEE364a Review Session 5
EE364a Review Session 5 EE364a Review announcements: homeworks 1 and 2 graded homework 4 solutions (check solution to additional problem 1) scpd phone-in office hours: tuesdays 6-7pm (650-723-1156) 1 Complementary
More informationEE 227A: Convex Optimization and Applications October 14, 2008
EE 227A: Convex Optimization and Applications October 14, 2008 Lecture 13: SDP Duality Lecturer: Laurent El Ghaoui Reading assignment: Chapter 5 of BV. 13.1 Direct approach 13.1.1 Primal problem Consider
More informationLecture: Duality.
Lecture: Duality http://bicmr.pku.edu.cn/~wenzw/opt-2016-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghe s lecture notes Introduction 2/35 Lagrange dual problem weak and strong
More information12. Interior-point methods
12. Interior-point methods Convex Optimization Boyd & Vandenberghe inequality constrained minimization logarithmic barrier function and central path barrier method feasibility and phase I methods complexity
More informationAgenda. 1 Cone programming. 2 Convex cones. 3 Generalized inequalities. 4 Linear programming (LP) 5 Second-order cone programming (SOCP)
Agenda 1 Cone programming 2 Convex cones 3 Generalized inequalities 4 Linear programming (LP) 5 Second-order cone programming (SOCP) 6 Semidefinite programming (SDP) 7 Examples Optimization problem in
More informationLecture: Duality of LP, SOCP and SDP
1/33 Lecture: Duality of LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2017.html wenzw@pku.edu.cn Acknowledgement:
More informationConvex Optimization Boyd & Vandenberghe. 5. Duality
5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationLagrange Duality. Daniel P. Palomar. Hong Kong University of Science and Technology (HKUST)
Lagrange Duality Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470 - Convex Optimization Fall 2017-18, HKUST, Hong Kong Outline of Lecture Lagrangian Dual function Dual
More informationDuality. Lagrange dual problem weak and strong duality optimality conditions perturbation and sensitivity analysis generalized inequalities
Duality Lagrange dual problem weak and strong duality optimality conditions perturbation and sensitivity analysis generalized inequalities Lagrangian Consider the optimization problem in standard form
More informationSummer School: Semidefinite Optimization
Summer School: Semidefinite Optimization Christine Bachoc Université Bordeaux I, IMB Research Training Group Experimental and Constructive Algebra Haus Karrenberg, Sept. 3 - Sept. 7, 2012 Duality Theory
More informationLecture 10: Linear programming. duality. and. The dual of the LP in standard form. maximize w = b T y (D) subject to A T y c, minimize z = c T x (P)
Lecture 10: Linear programming duality Michael Patriksson 19 February 2004 0-0 The dual of the LP in standard form minimize z = c T x (P) subject to Ax = b, x 0 n, and maximize w = b T y (D) subject to
More informationEE/AA 578, Univ of Washington, Fall Duality
7. Duality EE/AA 578, Univ of Washington, Fall 2016 Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationSolutions Chapter 5. The problem of finding the minimum distance from the origin to a line is written as. min 1 2 kxk2. subject to Ax = b.
Solutions Chapter 5 SECTION 5.1 5.1.4 www Throughout this exercise we will use the fact that strong duality holds for convex quadratic problems with linear constraints (cf. Section 3.4). The problem of
More informationELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications
ELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications Professor M. Chiang Electrical Engineering Department, Princeton University March
More informationConvex Optimization and Modeling
Convex Optimization and Modeling Duality Theory and Optimality Conditions 5th lecture, 12.05.2010 Jun.-Prof. Matthias Hein Program of today/next lecture Lagrangian and duality: the Lagrangian the dual
More information14. Duality. ˆ Upper and lower bounds. ˆ General duality. ˆ Constraint qualifications. ˆ Counterexample. ˆ Complementary slackness.
CS/ECE/ISyE 524 Introduction to Optimization Spring 2016 17 14. Duality ˆ Upper and lower bounds ˆ General duality ˆ Constraint qualifications ˆ Counterexample ˆ Complementary slackness ˆ Examples ˆ Sensitivity
More informationSparse Optimization Lecture: Dual Certificate in l 1 Minimization
Sparse Optimization Lecture: Dual Certificate in l 1 Minimization Instructor: Wotao Yin July 2013 Note scriber: Zheng Sun Those who complete this lecture will know what is a dual certificate for l 1 minimization
More information12. Interior-point methods
12. Interior-point methods Convex Optimization Boyd & Vandenberghe inequality constrained minimization logarithmic barrier function and central path barrier method feasibility and phase I methods complexity
More informationLinear and Combinatorial Optimization
Linear and Combinatorial Optimization The dual of an LP-problem. Connections between primal and dual. Duality theorems and complementary slack. Philipp Birken (Ctr. for the Math. Sc.) Lecture 3: Duality
More informationAgenda. Interior Point Methods. 1 Barrier functions. 2 Analytic center. 3 Central path. 4 Barrier method. 5 Primal-dual path following algorithms
Agenda Interior Point Methods 1 Barrier functions 2 Analytic center 3 Central path 4 Barrier method 5 Primal-dual path following algorithms 6 Nesterov Todd scaling 7 Complexity analysis Interior point
More informationExtreme Abridgment of Boyd and Vandenberghe s Convex Optimization
Extreme Abridgment of Boyd and Vandenberghe s Convex Optimization Compiled by David Rosenberg Abstract Boyd and Vandenberghe s Convex Optimization book is very well-written and a pleasure to read. The
More informationMath 413/513 Chapter 6 (from Friedberg, Insel, & Spence)
Math 413/513 Chapter 6 (from Friedberg, Insel, & Spence) David Glickenstein December 7, 2015 1 Inner product spaces In this chapter, we will only consider the elds R and C. De nition 1 Let V be a vector
More information4TE3/6TE3. Algorithms for. Continuous Optimization
4TE3/6TE3 Algorithms for Continuous Optimization (Duality in Nonlinear Optimization ) Tamás TERLAKY Computing and Software McMaster University Hamilton, January 2004 terlaky@mcmaster.ca Tel: 27780 Optimality
More informationsubject to (x 2)(x 4) u,
Exercises Basic definitions 5.1 A simple example. Consider the optimization problem with variable x R. minimize x 2 + 1 subject to (x 2)(x 4) 0, (a) Analysis of primal problem. Give the feasible set, the
More information15. Conic optimization
L. Vandenberghe EE236C (Spring 216) 15. Conic optimization conic linear program examples modeling duality 15-1 Generalized (conic) inequalities Conic inequality: a constraint x K where K is a convex cone
More informationLecture 5. Theorems of Alternatives and Self-Dual Embedding
IE 8534 1 Lecture 5. Theorems of Alternatives and Self-Dual Embedding IE 8534 2 A system of linear equations may not have a solution. It is well known that either Ax = c has a solution, or A T y = 0, c
More informationHW1 solutions. 1. α Ef(x) β, where Ef(x) is the expected value of f(x), i.e., Ef(x) = n. i=1 p if(a i ). (The function f : R R is given.
HW1 solutions Exercise 1 (Some sets of probability distributions.) Let x be a real-valued random variable with Prob(x = a i ) = p i, i = 1,..., n, where a 1 < a 2 < < a n. Of course p R n lies in the standard
More informationAdditional Homework Problems
Additional Homework Problems Robert M. Freund April, 2004 2004 Massachusetts Institute of Technology. 1 2 1 Exercises 1. Let IR n + denote the nonnegative orthant, namely IR + n = {x IR n x j ( ) 0,j =1,...,n}.
More informationLecture 1. 1 Conic programming. MA 796S: Convex Optimization and Interior Point Methods October 8, Consider the conic program. min.
MA 796S: Convex Optimization and Interior Point Methods October 8, 2007 Lecture 1 Lecturer: Kartik Sivaramakrishnan Scribe: Kartik Sivaramakrishnan 1 Conic programming Consider the conic program min s.t.
More informationDeterminant maximization with linear. S. Boyd, L. Vandenberghe, S.-P. Wu. Information Systems Laboratory. Stanford University
Determinant maximization with linear matrix inequality constraints S. Boyd, L. Vandenberghe, S.-P. Wu Information Systems Laboratory Stanford University SCCM Seminar 5 February 1996 1 MAXDET problem denition
More information1 Review of last lecture and introduction
Semidefinite Programming Lecture 10 OR 637 Spring 2008 April 16, 2008 (Wednesday) Instructor: Michael Jeremy Todd Scribe: Yogeshwer (Yogi) Sharma 1 Review of last lecture and introduction Let us first
More informationConvex Optimization and Support Vector Machine
Convex Optimization and Support Vector Machine Problem 0. Consider a two-class classification problem. The training data is L n = {(x 1, t 1 ),..., (x n, t n )}, where each t i { 1, 1} and x i R p. We
More informationSEMIDEFINITE PROGRAM BASICS. Contents
SEMIDEFINITE PROGRAM BASICS BRIAN AXELROD Abstract. A introduction to the basics of Semidefinite programs. Contents 1. Definitions and Preliminaries 1 1.1. Linear Algebra 1 1.2. Convex Analysis (on R n
More informationSemidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 4
Semidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 4 Instructor: Farid Alizadeh Scribe: Haengju Lee 10/1/2001 1 Overview We examine the dual of the Fermat-Weber Problem. Next we will
More information4. Convex optimization problems
Convex Optimization Boyd & Vandenberghe 4. Convex optimization problems optimization problem in standard form convex optimization problems quasiconvex optimization linear optimization quadratic optimization
More informationCSC Linear Programming and Combinatorial Optimization Lecture 10: Semidefinite Programming
CSC2411 - Linear Programming and Combinatorial Optimization Lecture 10: Semidefinite Programming Notes taken by Mike Jamieson March 28, 2005 Summary: In this lecture, we introduce semidefinite programming
More informationMidterm Review. Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A.
Midterm Review Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye (LY, Chapter 1-4, Appendices) 1 Separating hyperplane
More informationConvex Optimization & Lagrange Duality
Convex Optimization & Lagrange Duality Chee Wei Tan CS 8292 : Advanced Topics in Convex Optimization and its Applications Fall 2010 Outline Convex optimization Optimality condition Lagrange duality KKT
More informationConvex Optimization. (EE227A: UC Berkeley) Lecture 6. Suvrit Sra. (Conic optimization) 07 Feb, 2013
Convex Optimization (EE227A: UC Berkeley) Lecture 6 (Conic optimization) 07 Feb, 2013 Suvrit Sra Organizational Info Quiz coming up on 19th Feb. Project teams by 19th Feb Good if you can mix your research
More informationConvex Optimization. (EE227A: UC Berkeley) Lecture 28. Suvrit Sra. (Algebra + Optimization) 02 May, 2013
Convex Optimization (EE227A: UC Berkeley) Lecture 28 (Algebra + Optimization) 02 May, 2013 Suvrit Sra Admin Poster presentation on 10th May mandatory HW, Midterm, Quiz to be reweighted Project final report
More informationOptimisation and Operations Research
Optimisation and Operations Research Lecture 9: Duality and Complementary Slackness Matthew Roughan http://www.maths.adelaide.edu.au/matthew.roughan/ Lecture_notes/OORII/
More informationLecture 5. The Dual Cone and Dual Problem
IE 8534 1 Lecture 5. The Dual Cone and Dual Problem IE 8534 2 For a convex cone K, its dual cone is defined as K = {y x, y 0, x K}. The inner-product can be replaced by x T y if the coordinates of the
More informationLecture: Convex Optimization Problems
1/36 Lecture: Convex Optimization Problems http://bicmr.pku.edu.cn/~wenzw/opt-2015-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghe s lecture notes Introduction 2/36 optimization
More informationChapter 1. Preliminaries
Introduction This dissertation is a reading of chapter 4 in part I of the book : Integer and Combinatorial Optimization by George L. Nemhauser & Laurence A. Wolsey. The chapter elaborates links between
More informationLecture 3: Lagrangian duality and algorithms for the Lagrangian dual problem
Lecture 3: Lagrangian duality and algorithms for the Lagrangian dual problem Michael Patriksson 0-0 The Relaxation Theorem 1 Problem: find f := infimum f(x), x subject to x S, (1a) (1b) where f : R n R
More informationLECTURE 10 LECTURE OUTLINE
LECTURE 10 LECTURE OUTLINE Min Common/Max Crossing Th. III Nonlinear Farkas Lemma/Linear Constraints Linear Programming Duality Convex Programming Duality Optimality Conditions Reading: Sections 4.5, 5.1,5.2,
More informationLecture 8: Semidefinite programs for fidelity and optimal measurements
CS 766/QIC 80 Theory of Quantum Information (Fall 0) Lecture 8: Semidefinite programs for fidelity and optimal measurements This lecture is devoted to two examples of semidefinite programs: one is for
More informationExample Problem. Linear Program (standard form) CSCI5654 (Linear Programming, Fall 2013) Lecture-7. Duality
CSCI5654 (Linear Programming, Fall 013) Lecture-7 Duality Lecture 7 Slide# 1 Lecture 7 Slide# Linear Program (standard form) Example Problem maximize c 1 x 1 + + c n x n s.t. a j1 x 1 + + a jn x n b j
More informationConstrained Optimization and Lagrangian Duality
CIS 520: Machine Learning Oct 02, 2017 Constrained Optimization and Lagrangian Duality Lecturer: Shivani Agarwal Disclaimer: These notes are designed to be a supplement to the lecture. They may or may
More informationNote 3: LP Duality. If the primal problem (P) in the canonical form is min Z = n (1) then the dual problem (D) in the canonical form is max W = m (2)
Note 3: LP Duality If the primal problem (P) in the canonical form is min Z = n j=1 c j x j s.t. nj=1 a ij x j b i i = 1, 2,..., m (1) x j 0 j = 1, 2,..., n, then the dual problem (D) in the canonical
More informationConstrained optimization
Constrained optimization DS-GA 1013 / MATH-GA 2824 Optimization-based Data Analysis http://www.cims.nyu.edu/~cfgranda/pages/obda_fall17/index.html Carlos Fernandez-Granda Compressed sensing Convex constrained
More informationminimize x x2 2 x 1x 2 x 1 subject to x 1 +2x 2 u 1 x 1 4x 2 u 2, 5x 1 +76x 2 1,
4 Duality 4.1 Numerical perturbation analysis example. Consider the quadratic program with variables x 1, x 2, and parameters u 1, u 2. minimize x 2 1 +2x2 2 x 1x 2 x 1 subject to x 1 +2x 2 u 1 x 1 4x
More informationLecture Note 18: Duality
MATH 5330: Computational Methods of Linear Algebra 1 The Dual Problems Lecture Note 18: Duality Xianyi Zeng Department of Mathematical Sciences, UTEP The concept duality, just like accuracy and stability,
More informationSemidefinite Programming Basics and Applications
Semidefinite Programming Basics and Applications Ray Pörn, principal lecturer Åbo Akademi University Novia University of Applied Sciences Content What is semidefinite programming (SDP)? How to represent
More information3.10 Lagrangian relaxation
3.10 Lagrangian relaxation Consider a generic ILP problem min {c t x : Ax b, Dx d, x Z n } with integer coefficients. Suppose Dx d are the complicating constraints. Often the linear relaxation and the
More informationTrust Region Problems with Linear Inequality Constraints: Exact SDP Relaxation, Global Optimality and Robust Optimization
Trust Region Problems with Linear Inequality Constraints: Exact SDP Relaxation, Global Optimality and Robust Optimization V. Jeyakumar and G. Y. Li Revised Version: September 11, 2013 Abstract The trust-region
More informationWeek 3 Linear programming duality
Week 3 Linear programming duality This week we cover the fascinating topic of linear programming duality. We will learn that every minimization program has associated a maximization program that has the
More information3. Duality: What is duality? Why does it matter? Sensitivity through duality.
1 Overview of lecture (10/5/10) 1. Review Simplex Method 2. Sensitivity Analysis: How does solution change as parameters change? How much is the optimal solution effected by changing A, b, or c? How much
More informationLecture 7: Semidefinite programming
CS 766/QIC 820 Theory of Quantum Information (Fall 2011) Lecture 7: Semidefinite programming This lecture is on semidefinite programming, which is a powerful technique from both an analytic and computational
More informationMAT-INF4110/MAT-INF9110 Mathematical optimization
MAT-INF4110/MAT-INF9110 Mathematical optimization Geir Dahl August 20, 2013 Convexity Part IV Chapter 4 Representation of convex sets different representations of convex sets, boundary polyhedra and polytopes:
More informationLinear and non-linear programming
Linear and non-linear programming Benjamin Recht March 11, 2005 The Gameplan Constrained Optimization Convexity Duality Applications/Taxonomy 1 Constrained Optimization minimize f(x) subject to g j (x)
More informationDuality Theory of Constrained Optimization
Duality Theory of Constrained Optimization Robert M. Freund April, 2014 c 2014 Massachusetts Institute of Technology. All rights reserved. 1 2 1 The Practical Importance of Duality Duality is pervasive
More informationLecture 6: Conic Optimization September 8
IE 598: Big Data Optimization Fall 2016 Lecture 6: Conic Optimization September 8 Lecturer: Niao He Scriber: Juan Xu Overview In this lecture, we finish up our previous discussion on optimality conditions
More informationLECTURE 13 LECTURE OUTLINE
LECTURE 13 LECTURE OUTLINE Problem Structures Separable problems Integer/discrete problems Branch-and-bound Large sum problems Problems with many constraints Conic Programming Second Order Cone Programming
More informationLecture: Examples of LP, SOCP and SDP
1/34 Lecture: Examples of LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2018.html wenzw@pku.edu.cn Acknowledgement:
More informationA Review of Linear Programming
A Review of Linear Programming Instructor: Farid Alizadeh IEOR 4600y Spring 2001 February 14, 2001 1 Overview In this note we review the basic properties of linear programming including the primal simplex
More informationPart IB Optimisation
Part IB Optimisation Theorems Based on lectures by F. A. Fischer Notes taken by Dexter Chua Easter 2015 These notes are not endorsed by the lecturers, and I have modified them (often significantly) after
More informationConvex Optimization and SVM
Convex Optimization and SVM Problem 0. Cf lecture notes pages 12 to 18. Problem 1. (i) A slab is an intersection of two half spaces, hence convex. (ii) A wedge is an intersection of two half spaces, hence
More informationOptimality, Duality, Complementarity for Constrained Optimization
Optimality, Duality, Complementarity for Constrained Optimization Stephen Wright University of Wisconsin-Madison May 2014 Wright (UW-Madison) Optimality, Duality, Complementarity May 2014 1 / 41 Linear
More informationThe Kuhn-Tucker Problem
Natalia Lazzati Mathematics for Economics (Part I) Note 8: Nonlinear Programming - The Kuhn-Tucker Problem Note 8 is based on de la Fuente (2000, Ch. 7) and Simon and Blume (1994, Ch. 18 and 19). The Kuhn-Tucker
More informationStrong Duality and Minimal Representations for Cone Optimization
Strong Duality and Minimal Representations for Cone Optimization Levent Tunçel Henry Wolkowicz August 2008, revised: December 2010 University of Waterloo Department of Combinatorics & Optimization Waterloo,
More informationCOM S 578X: Optimization for Machine Learning
COM S 578X: Optimization for Machine Learning Lecture Note 4: Duality Jia (Kevin) Liu Assistant Professor Department of Computer Science Iowa State University, Ames, Iowa, USA Fall 2018 JKL (CS@ISU) COM
More informationLecture 8. Strong Duality Results. September 22, 2008
Strong Duality Results September 22, 2008 Outline Lecture 8 Slater Condition and its Variations Convex Objective with Linear Inequality Constraints Quadratic Objective over Quadratic Constraints Representation
More informationA notion of Total Dual Integrality for Convex, Semidefinite and Extended Formulations
A notion of for Convex, Semidefinite and Extended Formulations Marcel de Carli Silva Levent Tunçel April 26, 2018 A vector in R n is integral if each of its components is an integer, A vector in R n is
More informationLecture: Algorithms for LP, SOCP and SDP
1/53 Lecture: Algorithms for LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2018.html wenzw@pku.edu.cn Acknowledgement:
More informationLECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE
LECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE CONVEX ANALYSIS AND DUALITY Basic concepts of convex analysis Basic concepts of convex optimization Geometric duality framework - MC/MC Constrained optimization
More informationAn E cient A ne-scaling Algorithm for Hyperbolic Programming
An E cient A ne-scaling Algorithm for Hyperbolic Programming Jim Renegar joint work with Mutiara Sondjaja 1 Euclidean space A homogeneous polynomial p : E!R is hyperbolic if there is a vector e 2E such
More informationLecture 10: Linear programming duality and sensitivity 0-0
Lecture 10: Linear programming duality and sensitivity 0-0 The canonical primal dual pair 1 A R m n, b R m, and c R n maximize z = c T x (1) subject to Ax b, x 0 n and minimize w = b T y (2) subject to
More informationOptimization for Machine Learning
Optimization for Machine Learning (Problems; Algorithms - A) SUVRIT SRA Massachusetts Institute of Technology PKU Summer School on Data Science (July 2017) Course materials http://suvrit.de/teaching.html
More informationDiscrete Optimization
Prof. Friedrich Eisenbrand Martin Niemeier Due Date: April 15, 2010 Discussions: March 25, April 01 Discrete Optimization Spring 2010 s 3 You can hand in written solutions for up to two of the exercises
More informationLecture #21. c T x Ax b. maximize subject to
COMPSCI 330: Design and Analysis of Algorithms 11/11/2014 Lecture #21 Lecturer: Debmalya Panigrahi Scribe: Samuel Haney 1 Overview In this lecture, we discuss linear programming. We first show that the
More informationTutorial on Convex Optimization: Part II
Tutorial on Convex Optimization: Part II Dr. Khaled Ardah Communications Research Laboratory TU Ilmenau Dec. 18, 2018 Outline Convex Optimization Review Lagrangian Duality Applications Optimal Power Allocation
More informationSDP Relaxations for MAXCUT
SDP Relaxations for MAXCUT from Random Hyperplanes to Sum-of-Squares Certificates CATS @ UMD March 3, 2017 Ahmed Abdelkader MAXCUT SDP SOS March 3, 2017 1 / 27 Overview 1 MAXCUT, Hardness and UGC 2 LP
More informationand to estimate the quality of feasible solutions I A new way to derive dual bounds:
Lagrangian Relaxations and Duality I Recall: I Relaxations provide dual bounds for the problem I So do feasible solutions of dual problems I Having tight dual bounds is important in algorithms (B&B), and
More informationReview Solutions, Exam 2, Operations Research
Review Solutions, Exam 2, Operations Research 1. Prove the weak duality theorem: For any x feasible for the primal and y feasible for the dual, then... HINT: Consider the quantity y T Ax. SOLUTION: To
More information