LP Duality: outline. Duality theory for Linear Programming. alternatives. optimization I Idea: polyhedra
|
|
- Amice Wheeler
- 6 years ago
- Views:
Transcription
1 LP Duality: outline I Motivation and definition of a dual LP I Weak duality I Separating hyperplane theorem and theorems of the alternatives I Strong duality and complementary slackness I Using duality theory to I Characterize unbounded LPs I Resolution theorem and its converse: two ways to look at polyhedra IOE 610: LP II, Fall 2013 Duality Page 63 Duality theory for Linear Programming I Special case of Lagrangian Duality Theory for (general) optimization I Idea: I Find a relaxation of your optimization problem, to obtain a dual (i.e., optimistic) bound on its optimal bound I Try to find a tight dual bound through the above methodology IOE 610: LP II, Fall 2013 Duality Page 64
2 For linear programming (in standard form): (P) minimize s.t. c T x Ax = b x 0, where A 2< m n. Assume (P) has an optimal solution x?. I L(x, p) =c T x + p T (b Ax), where p 2< m I g(p) =min x 0 c T x + p T (b Ax) I Note: for any p, g(p) apple c T x? I Thus, to obtain a tight lower bound on c T x?,wemaywantto maximize p g(p), which is the dual problem I Exploring g(p) further, we conclude that the dual problem can be formulated as (D) maximize s.t. p T b p T A apple c T IOE 610: LP II, Fall 2013 Duality Page 65 For linear programming (in inequality form) (P) minimize c T x s.t. Ax b where A 2< m n. Assume (P) has an optimal solution x?. I L(x, p) =c T x + p T (b Ax), where p 2< m I g(p) =min x c T x + p T (b Ax) I For any p 0, g(p) apple c T x? I Thus, to obtain a tight lower bound on c T x?,wemaywantto maximize p 0 g(p), which is the dual problem I Exploring g(p) further, we conclude that the dual problem can be formulated as (D) maximize s.t. p T b p T A = c T p 0 IOE 610: LP II, Fall 2013 Duality Page 66
3 The dual problem for LP in general form I Introduce a variable p i in the dual problem for every non-sign constraint in the primal I We will have a constraint in the dual for every variable, and vice versa in the primal (P) minimize c T x (D) maximize p T b s.t. a T i x b i, i 2 M 1, s.t. p i 0, i 2 M 1 a T i x apple b i, i 2 M 2, p i apple 0, i 2 M 2 a T i x = b i, i 2 M 3, p i free, i 2 M 3 x j 0, j 2 N 1, p T A j apple c j, j 2 N 1 x j apple 0, j 2 N 2, p T A j c j, j 2 N 2 x j free, j 2 N 3, p T A j = c j, j 2 N 3 IOE 610: LP II, Fall 2013 Duality Page 67 The relations between primal and dual problems: PRIMAL minimize maximize DUAL b i 0 constraints apple b i apple 0 variables = b i free 0 apple c j variables apple 0 c j constraints free = c j Theorem 4.1 ( Dual of the dual is the primal ) If we transform the dual into an equivalent minimization problem and then form its dual, we obtain a problem equivalent to the original problem. IOE 610: LP II, Fall 2013 Duality Page 68
4 Properties of primal-dual pairs Theorem 4.3 (Weak duality) If x is a feasible solution to the primal problem and p is a feasible solution to the dual problem, then Corollary 4.1 p T b apple c T x. If the cost of the primal (dual) is 1 (+1), then the dual (primal) problem must be infeasible. Corollary 4.2 Let x and p be feasible solutions to the primal and the dual, respectively, and suppose that p T b = c T x.thenx and p are optimal solutions to the primal and the dual, respectively. IOE 610: LP II, Fall 2013 Duality Page 69 Strong duality and Complementary slackness Theorem 4.4 Strong duality Theorem 4.5 Complementary slackness We ll come back to it. IOE 610: LP II, Fall 2013 Duality Page 70
5 Duals of equivalent LPs are equivalent Atechnicalitybeforeweproceed: Theorem 4.2 Suppose that we have transformed a linear programming problem 1 into another linear programming problem 2, by a sequence of transformations of the following types: I Replace a free variable with the di erence of two nonnegative variables. I Replace an inequality constraint by an equality constraint involving a nonnegative slack variable. I If some row of the matrix A in a feasible standard form problem is a linear combination of the other rows, eliminate the corresponding equality constraint. Then the duals of 1 and 2 are equivalent, i.e., they are either both infeasible, or they have the same optimal cost. IOE 610: LP II, Fall 2013 Duality Page 71 Separating hyperplane theorem Theorem 4.11 (Separating hyperplane theorem) Let S be a nonempty closed convex subset of < n and let x? 62 S. Then 9c 2< n such that c T x? < c T x 8x 2 S. I Definition: AsetS < n is called closed if, for every sequence x 1, x 2,... of elements of S that converges to some x 2< n, x 2 S. I Intersection of closed sets is closed. I Every polyhedron is a closed set Theorem 4.10 (Weierstrass theorem) If f : < n!<is a continuous function, and if S < n is a nonempty, closed, and bounded set, then I There exists some x? 2 S such that f (x? ) apple f (x) 8x 2 S I There exists some y? 2 S such that f (y? ) f (x) 8x 2 S. Note: all three assumptions are important! IOE 610: LP II, Fall 2013 Separating hyperplanes and duality theory Page 72
6 A theorem of alternative Farkas lemma (also, Theorem 4.6) Let A 2< m n and let b 2< m. Then, exactly one of the following two alternatives holds: (a) 9x 0 such that Ax = b (b) 9p such that p T A 0 T and p T b < 0. Proof I It is easy to show that both alternatives cannot hold. I Suppose the first system does not have a solution. Let S = {Ax : x 0} = {y 2< m : 9x 0 s.t. y = Ax}. I S is nonempty, convex and closed (?), and b 62 S. I Thus, b and S can be separated by a hyperplane... IOE 610: LP II, Fall 2013 Separating hyperplanes and duality theory Page 73 Strong duality theorem Theorem 4.4 (Strong Duality Theorem) If a linear programming problem has an optimal solution, so does its dual, and the respective optimal costs are equal. Proof We will prove the theorem for the primal-dual pair of the form (P) minimize c T x (D) maximize b T p s.t. Ax b s.t. A T p = c, p 0 Proof outline: I Let x? be an optimal solution of (P), and let I be the set of indices of constraints active at x?. I There does not exist a vector d 2< n such that c T d < 0and a T i d 0 8i 2 I I By Farkas lemma, P i2i p ia i = c, p i solution. Let p i =0, i 62 I. I p is feasible for (D), and c T x? = b T p. 0, i 2 I has a IOE 610: LP II, Fall 2013 Separating hyperplanes and duality theory Page 74
7 Summary of relationship between (P) and (D) There are 4 possible outcomes for a primal-dual pair of LPs (P) and (D): I (P) and (D) are both infeasible I (P) is infeasible and (D) is unbounded I (P) is unbounded and (D) is infeasible I (P) and (D) both have optimal solutions of equal value IOE 610: LP II, Fall 2013 Separating hyperplanes and duality theory Page 75 Complementary slackness Theorem 4.5 (Complementary Slackness) Let x and p be feasible solutions to the primal and the dual problem, respectively. The vectors x and p are optimal solutions for the two respective problems i p i (a T i x b i )=0, 8i (c j p T A j )x j =0, 8j. IOE 610: LP II, Fall 2013 Separating hyperplanes and duality theory Page 76
8 Geometric interpretation of duality and complementary slackness (P) minimize c T x (D) maximize b T p s.t. a T P m i x b i, i =1,...,m s.t. i=1 a ip i = c p 0 Let x 2< n and let I = i : a T i x = b i I For x to be an optimal solution of (P), it is necessary and su cient that there exists p 2< m such that together they satisfy: (a) a T i x b i for all i (Primal feasiblity) (b) p i =0foralli 62 I (Complementary slackness) (c) P m i=1 a ip i = c (Dual feasiblity) (d) p 0 (Dual feasibility) I What if x is feasible, but not optimal? I What if x is an optimal BFS? A degenerate optimal BFS? IOE 610: LP II, Fall 2013 Interpretation and uses of duality theory Page 77 Strong Duality)Farkas lemma Farkas lemma (also, Theorem 4.6) Let A 2< m n and let b 2< m. Then, exactly one of the following two alternatives holds: (a) 9x 0 such that Ax = b (b) 9p such that A T p 0andb T p < 0 Proof Consider (P) minimize b T p (D) maximize 0 T x s.t. A T p 0 s.t. Ax = b x 0 I If (a) holds, (D) is feasible and has optimal value 0. By strong duality, (P) has optimal value 0, and (b) does not hold I If (a) does not hold, (D) is infeasible, and (P) is either infeasible or unbounded I (P) is feasible: take p = 0 I Hence, (P) is unbounded, i.e., (b) holds IOE 610: LP II, Fall 2013 Interpretation and uses of duality theory Page 78
9 Characterization of unbounded LPs I If we know an LP has a finite optimal value, search for an optimal solution can be restricted to a (finite set) of extreme points of the feasible region (If necessary, of an equivalent problem in standard form) I If an LP is unbounded, how can we find that out? what constitutes a proof of its unboundedness? I Can the search for a proof of unboundedness be restricted to some finite set? IOE 610: LP II, Fall 2013 Interpretation and uses of duality theory Page 79 Cones Definition 4.1 AsetC < n is a cone if x 2 C for all 0 and all x 2 C. Note: I 0 2 C if C 6= ;. I The only possible extreme point of a cone is 0 if it is the case, we say that the cone is pointed Theorem 4.12 (Special case of Thm. 2.6) Let C be a polyhedral cone defined by the constraints a T i x 0, i =1,...,m. Then the following are equivalent: (a) The 0 vector is an extreme point of C (b) C does not contain a line (c) There exist n vectors out of the family a 1,...,a m,whichare linearly independent. IOE 610: LP II, Fall 2013 Interpretation and uses of duality theory Page 80
10 Rays and recession cones I A recession cone of a set P 6= ; at point y 2 P is the set I If P is a polyhedron: {d : y + d 2 P 8 0} I P = {x : Ax b} 6= ;: the recession cone at y 2 P is: {d : A(y + d) b, 8 0} = {d : Ad 0} independent of the point y. I P = {x 0 : Ax = b}: the recession cone is {d 0 : Ad = 0}. Definition 4.2: Extreme rays (a) A nonzero element x of a polyhedral cone C < n is called an extreme ray if there are n 1 linearly independent constraints that are active at x. (b) An extreme ray of the recession cone associated with a nonempty polyhedron P is also called an extreme ray of P. IOE 610: LP II, Fall 2013 Interpretation and uses of duality theory Page 81 Observations about extreme rays I If x is an extreme ray, x with > 0 is also an extreme ray. We call two extreme rays equivalent if one is a positive multiple of the other. I If two extreme rays are equivalent, they correspond to the same n 1 active constraints. Any n 1 linearly independent constraints define a line and can lead to at most two non-equivalent extreme rays. I Since there is a finite number of ways we can choose n 1 active constraints, the number of extreme rays of a polyhedron is finite (we do not distinguish between equivalent ones). I A finite collection of extreme rays is a complete set of extreme rays is it contains exactly one representative from each equivalence class. IOE 610: LP II, Fall 2013 Interpretation and uses of duality theory Page 82
11 Characterization of unbounded LPs Theorem 4.13 Consider the problem of minimizing c T x over a pointed polyhedral cone C = {x : a T i x 0, i =1,...,m}. The optimal cost is equal to 1 i some extreme ray d of C satisfies c T d < 0. Theorem 4.14 Consider the problem of minimizing c T x s.t. Ax b, andassume that the feasible set has at least one extreme point. The optimal cost is equal to 1 if an only if some extreme ray d of the feasible set satisfies c T d < 0. IOE 610: LP II, Fall 2013 Interpretation and uses of duality theory Page 83 A solution approach for LPs (P) minimize c T x s.t. x 2 P 1. Check if P = ;. If so, (P) is infeasible; if not, continue. 2. For every extreme ray d of P, checkifc T d < 0. If so, (P) is unbounded; if not continue. 3. For every extreme point x if P, evaluatec T x. Pickanextreme point with the smallest value: it is an optimal solution of (P). IOE 610: LP II, Fall 2013 Interpretation and uses of duality theory Page 84
12 Representation of polyhedra Theorem 4.15 (Resolution theorem) Let P = {x : Ax b} be a nonempty polyhedron with at least one extreme point. Let x 1,...,x k be the extreme points, and let w 1,...,w r be a complete set of extreme rays of P. Let ( kx ) rx kx Q = ix i + j w j : i 0, j 0, i =1. Then Q = P. Corollary 4.4 i=1 j=1 i=1 Anonemptyboundedpolyhedronistheconvexhullofitsextremepoints. Corollary 4.5 If the cone C = {x : Ax 0} is pointed, then every element of C can be represented as a nonnegative linear combination of the extreme rays of C. IOE 610: LP II, Fall 2013 Interpretation and uses of duality theory Page 85 Key to proof of Resolution Theorem Showing Q P is easy. Suppose z 2 P, butz 62 Q. Consider and (LP z ) maximize 0 T + 0 T kx rx s.t. ix i + j w j = z i=1 kx i=1 i =1 j=1 0, 0 (LD z ) minimize z T p + q s.t. (x i ) T p + q 0 8i (w j ) T p 0 8j (LP z )isinfeasible,hence(ld z ) is unbounded. IOE 610: LP II, Fall 2013 Interpretation and uses of duality theory Page 86
13 Converse to the resolution theorem I AsetQ is finitely generated if it is specified in the form 8 9 < kx rx kx = Q = : ix i + j w j : i 0, j 0, i =1 ;, i=1 j=1 where x 1,...,x k and w 1,...,w r are some given elements of < n. I The resolution theorem claims that every polyhedron with at least one extreme point is finitely generated. Theorem 4.16 A finitely generated set is a polyhedron. In particular, a convex hull of finitely many vectors is a (bounded) polyhedron. i=1 IOE 610: LP II, Fall 2013 Interpretation and uses of duality theory Page 87 Impact of Resolution Theorem and it converse Every polyhedron with at least one extreme point can be given/described/represented two ways: I {x : Ax b} for some A 2< m n and b 2< m n I x = P k i=1 ix i + P r j=1 jw j : i 0, j 0, P k for some x 1,...,x k, w 1,...,w r 2< n. i=1 i =1 o IOE 610: LP II, Fall 2013 Interpretation and uses of duality theory Page 88
Introduction to Mathematical Programming IE406. Lecture 10. Dr. Ted Ralphs
Introduction to Mathematical Programming IE406 Lecture 10 Dr. Ted Ralphs IE406 Lecture 10 1 Reading for This Lecture Bertsimas 4.1-4.3 IE406 Lecture 10 2 Duality Theory: Motivation Consider the following
More informationChapter 1. Preliminaries
Introduction This dissertation is a reading of chapter 4 in part I of the book : Integer and Combinatorial Optimization by George L. Nemhauser & Laurence A. Wolsey. The chapter elaborates links between
More informationMidterm Review. Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A.
Midterm Review Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye (LY, Chapter 1-4, Appendices) 1 Separating hyperplane
More information4. Algebra and Duality
4-1 Algebra and Duality P. Parrilo and S. Lall, CDC 2003 2003.12.07.01 4. Algebra and Duality Example: non-convex polynomial optimization Weak duality and duality gap The dual is not intrinsic The cone
More informationCO 250 Final Exam Guide
Spring 2017 CO 250 Final Exam Guide TABLE OF CONTENTS richardwu.ca CO 250 Final Exam Guide Introduction to Optimization Kanstantsin Pashkovich Spring 2017 University of Waterloo Last Revision: March 4,
More informationLinear Programming Inverse Projection Theory Chapter 3
1 Linear Programming Inverse Projection Theory Chapter 3 University of Chicago Booth School of Business Kipp Martin September 26, 2017 2 Where We Are Headed We want to solve problems with special structure!
More information3. Linear Programming and Polyhedral Combinatorics
Massachusetts Institute of Technology 18.433: Combinatorial Optimization Michel X. Goemans February 28th, 2013 3. Linear Programming and Polyhedral Combinatorics Summary of what was seen in the introductory
More informationI.3. LMI DUALITY. Didier HENRION EECI Graduate School on Control Supélec - Spring 2010
I.3. LMI DUALITY Didier HENRION henrion@laas.fr EECI Graduate School on Control Supélec - Spring 2010 Primal and dual For primal problem p = inf x g 0 (x) s.t. g i (x) 0 define Lagrangian L(x, z) = g 0
More informationLP Relaxations of Mixed Integer Programs
LP Relaxations of Mixed Integer Programs John E. Mitchell Department of Mathematical Sciences RPI, Troy, NY 12180 USA February 2015 Mitchell LP Relaxations 1 / 29 LP Relaxations LP relaxations We want
More informationNote 3: LP Duality. If the primal problem (P) in the canonical form is min Z = n (1) then the dual problem (D) in the canonical form is max W = m (2)
Note 3: LP Duality If the primal problem (P) in the canonical form is min Z = n j=1 c j x j s.t. nj=1 a ij x j b i i = 1, 2,..., m (1) x j 0 j = 1, 2,..., n, then the dual problem (D) in the canonical
More informationEE364a Review Session 5
EE364a Review Session 5 EE364a Review announcements: homeworks 1 and 2 graded homework 4 solutions (check solution to additional problem 1) scpd phone-in office hours: tuesdays 6-7pm (650-723-1156) 1 Complementary
More information3. Linear Programming and Polyhedral Combinatorics
Massachusetts Institute of Technology 18.453: Combinatorial Optimization Michel X. Goemans April 5, 2017 3. Linear Programming and Polyhedral Combinatorics Summary of what was seen in the introductory
More informationLECTURE 10 LECTURE OUTLINE
LECTURE 10 LECTURE OUTLINE Min Common/Max Crossing Th. III Nonlinear Farkas Lemma/Linear Constraints Linear Programming Duality Convex Programming Duality Optimality Conditions Reading: Sections 4.5, 5.1,5.2,
More informationSpring 2017 CO 250 Course Notes TABLE OF CONTENTS. richardwu.ca. CO 250 Course Notes. Introduction to Optimization
Spring 2017 CO 250 Course Notes TABLE OF CONTENTS richardwu.ca CO 250 Course Notes Introduction to Optimization Kanstantsin Pashkovich Spring 2017 University of Waterloo Last Revision: March 4, 2018 Table
More informationMAT-INF4110/MAT-INF9110 Mathematical optimization
MAT-INF4110/MAT-INF9110 Mathematical optimization Geir Dahl August 20, 2013 Convexity Part IV Chapter 4 Representation of convex sets different representations of convex sets, boundary polyhedra and polytopes:
More information1 Review Session. 1.1 Lecture 2
1 Review Session Note: The following lists give an overview of the material that was covered in the lectures and sections. Your TF will go through these lists. If anything is unclear or you have questions
More informationExample: feasibility. Interpretation as formal proof. Example: linear inequalities and Farkas lemma
4-1 Algebra and Duality P. Parrilo and S. Lall 2006.06.07.01 4. Algebra and Duality Example: non-convex polynomial optimization Weak duality and duality gap The dual is not intrinsic The cone of valid
More informationLecture 10: Linear programming duality and sensitivity 0-0
Lecture 10: Linear programming duality and sensitivity 0-0 The canonical primal dual pair 1 A R m n, b R m, and c R n maximize z = c T x (1) subject to Ax b, x 0 n and minimize w = b T y (2) subject to
More informationDuality Theory, Optimality Conditions
5.1 Duality Theory, Optimality Conditions Katta G. Murty, IOE 510, LP, U. Of Michigan, Ann Arbor We only consider single objective LPs here. Concept of duality not defined for multiobjective LPs. Every
More informationThe Simplex Algorithm
8.433 Combinatorial Optimization The Simplex Algorithm October 6, 8 Lecturer: Santosh Vempala We proved the following: Lemma (Farkas). Let A R m n, b R m. Exactly one of the following conditions is true:.
More information1 Review of last lecture and introduction
Semidefinite Programming Lecture 10 OR 637 Spring 2008 April 16, 2008 (Wednesday) Instructor: Michael Jeremy Todd Scribe: Yogeshwer (Yogi) Sharma 1 Review of last lecture and introduction Let us first
More informationLectures 6, 7 and part of 8
Lectures 6, 7 and part of 8 Uriel Feige April 26, May 3, May 10, 2015 1 Linear programming duality 1.1 The diet problem revisited Recall the diet problem from Lecture 1. There are n foods, m nutrients,
More informationLECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE
LECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE CONVEX ANALYSIS AND DUALITY Basic concepts of convex analysis Basic concepts of convex optimization Geometric duality framework - MC/MC Constrained optimization
More informationOptimization WS 13/14:, by Y. Goldstein/K. Reinert, 9. Dezember 2013, 16: Linear programming. Optimization Problems
Optimization WS 13/14:, by Y. Goldstein/K. Reinert, 9. Dezember 2013, 16:38 2001 Linear programming Optimization Problems General optimization problem max{z(x) f j (x) 0,x D} or min{z(x) f j (x) 0,x D}
More informationChapter 1 Linear Programming. Paragraph 5 Duality
Chapter 1 Linear Programming Paragraph 5 Duality What we did so far We developed the 2-Phase Simplex Algorithm: Hop (reasonably) from basic solution (bs) to bs until you find a basic feasible solution
More informationWeek 3 Linear programming duality
Week 3 Linear programming duality This week we cover the fascinating topic of linear programming duality. We will learn that every minimization program has associated a maximization program that has the
More informationYinyu Ye, MS&E, Stanford MS&E310 Lecture Note #06. The Simplex Method
The Simplex Method Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye (LY, Chapters 2.3-2.5, 3.1-3.4) 1 Geometry of Linear
More informationReview Solutions, Exam 2, Operations Research
Review Solutions, Exam 2, Operations Research 1. Prove the weak duality theorem: For any x feasible for the primal and y feasible for the dual, then... HINT: Consider the quantity y T Ax. SOLUTION: To
More informationLinear Programming and the Simplex method
Linear Programming and the Simplex method Harald Enzinger, Michael Rath Signal Processing and Speech Communication Laboratory Jan 9, 2012 Harald Enzinger, Michael Rath Jan 9, 2012 page 1/37 Outline Introduction
More informationLinear Programming. Chapter Introduction
Chapter 3 Linear Programming Linear programs (LP) play an important role in the theory and practice of optimization problems. Many COPs can directly be formulated as LPs. Furthermore, LPs are invaluable
More informationConic Linear Optimization and its Dual. yyye
Conic Linear Optimization and Appl. MS&E314 Lecture Note #04 1 Conic Linear Optimization and its Dual Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A.
More informationInteger Programming, Part 1
Integer Programming, Part 1 Rudi Pendavingh Technische Universiteit Eindhoven May 18, 2016 Rudi Pendavingh (TU/e) Integer Programming, Part 1 May 18, 2016 1 / 37 Linear Inequalities and Polyhedra Farkas
More informationChapter 3, Operations Research (OR)
Chapter 3, Operations Research (OR) Kent Andersen February 7, 2007 1 Linear Programs (continued) In the last chapter, we introduced the general form of a linear program, which we denote (P) Minimize Z
More informationAgenda. 1 Duality for LP. 2 Theorem of alternatives. 3 Conic Duality. 4 Dual cones. 5 Geometric view of cone programs. 6 Conic duality theorem
Agenda 1 Duality for LP 2 Theorem of alternatives 3 Conic Duality 4 Dual cones 5 Geometric view of cone programs 6 Conic duality theorem 7 Examples Lower bounds on LPs By eliminating variables (if needed)
More informationKey Things We Learned Last Time. IE418 Integer Programming. Proving Facets Way #2 Indirect. A More Abstract Example
Key Things We Learned Last Time IE48: Integer Programming Department of Industrial and Systems Engineering Lehigh University 8th March 5 A face F is said to be a facet of P if dim(f ) = dim(p ). All facets
More informationIE 5531: Engineering Optimization I
IE 5531: Engineering Optimization I Lecture 7: Duality and applications Prof. John Gunnar Carlsson September 29, 2010 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 29, 2010 1
More informationLinear Programming. Larry Blume Cornell University, IHS Vienna and SFI. Summer 2016
Linear Programming Larry Blume Cornell University, IHS Vienna and SFI Summer 2016 These notes derive basic results in finite-dimensional linear programming using tools of convex analysis. Most sources
More informationConvex Optimization M2
Convex Optimization M2 Lecture 3 A. d Aspremont. Convex Optimization M2. 1/49 Duality A. d Aspremont. Convex Optimization M2. 2/49 DMs DM par email: dm.daspremont@gmail.com A. d Aspremont. Convex Optimization
More informationApproximate Farkas Lemmas in Convex Optimization
Approximate Farkas Lemmas in Convex Optimization Imre McMaster University Advanced Optimization Lab AdvOL Graduate Student Seminar October 25, 2004 1 Exact Farkas Lemma Motivation 2 3 Future plans The
More informationExtreme Abridgment of Boyd and Vandenberghe s Convex Optimization
Extreme Abridgment of Boyd and Vandenberghe s Convex Optimization Compiled by David Rosenberg Abstract Boyd and Vandenberghe s Convex Optimization book is very well-written and a pleasure to read. The
More informationCS261: A Second Course in Algorithms Lecture #9: Linear Programming Duality (Part 2)
CS261: A Second Course in Algorithms Lecture #9: Linear Programming Duality (Part 2) Tim Roughgarden February 2, 2016 1 Recap This is our third lecture on linear programming, and the second on linear programming
More informationLinear and Integer Optimization (V3C1/F4C1)
Linear and Integer Optimization (V3C1/F4C1) Lecture notes Ulrich Brenner Research Institute for Discrete Mathematics, University of Bonn Winter term 2016/2017 March 8, 2017 12:02 1 Preface Continuous updates
More informationFarkas Lemma, Dual Simplex and Sensitivity Analysis
Summer 2011 Optimization I Lecture 10 Farkas Lemma, Dual Simplex and Sensitivity Analysis 1 Farkas Lemma Theorem 1. Let A R m n, b R m. Then exactly one of the following two alternatives is true: (i) x
More informationBBM402-Lecture 20: LP Duality
BBM402-Lecture 20: LP Duality Lecturer: Lale Özkahya Resources for the presentation: https://courses.engr.illinois.edu/cs473/fa2016/lectures.html An easy LP? which is compact form for max cx subject to
More informationA Review of Linear Programming
A Review of Linear Programming Instructor: Farid Alizadeh IEOR 4600y Spring 2001 February 14, 2001 1 Overview In this note we review the basic properties of linear programming including the primal simplex
More informationLinear Algebra Review: Linear Independence. IE418 Integer Programming. Linear Algebra Review: Subspaces. Linear Algebra Review: Affine Independence
Linear Algebra Review: Linear Independence IE418: Integer Programming Department of Industrial and Systems Engineering Lehigh University 21st March 2005 A finite collection of vectors x 1,..., x k R n
More informationDiscrete Optimization
Prof. Friedrich Eisenbrand Martin Niemeier Due Date: April 15, 2010 Discussions: March 25, April 01 Discrete Optimization Spring 2010 s 3 You can hand in written solutions for up to two of the exercises
More informationAppendix PRELIMINARIES 1. THEOREMS OF ALTERNATIVES FOR SYSTEMS OF LINEAR CONSTRAINTS
Appendix PRELIMINARIES 1. THEOREMS OF ALTERNATIVES FOR SYSTEMS OF LINEAR CONSTRAINTS Here we consider systems of linear constraints, consisting of equations or inequalities or both. A feasible solution
More informationIntroduction to linear programming using LEGO.
Introduction to linear programming using LEGO. 1 The manufacturing problem. A manufacturer produces two pieces of furniture, tables and chairs. The production of the furniture requires the use of two different
More informationLecture #21. c T x Ax b. maximize subject to
COMPSCI 330: Design and Analysis of Algorithms 11/11/2014 Lecture #21 Lecturer: Debmalya Panigrahi Scribe: Samuel Haney 1 Overview In this lecture, we discuss linear programming. We first show that the
More informationLecture: Algorithms for LP, SOCP and SDP
1/53 Lecture: Algorithms for LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2018.html wenzw@pku.edu.cn Acknowledgement:
More informationLecture 7 Duality II
L. Vandenberghe EE236A (Fall 2013-14) Lecture 7 Duality II sensitivity analysis two-person zero-sum games circuit interpretation 7 1 Sensitivity analysis purpose: extract from the solution of an LP information
More informationAlgorithmic Game Theory and Applications. Lecture 7: The LP Duality Theorem
Algorithmic Game Theory and Applications Lecture 7: The LP Duality Theorem Kousha Etessami recall LP s in Primal Form 1 Maximize c 1 x 1 + c 2 x 2 +... + c n x n a 1,1 x 1 + a 1,2 x 2 +... + a 1,n x n
More informationSimplex method(s) for solving LPs in standard form
Simplex method: outline I The Simplex Method is a family of algorithms for solving LPs in standard form (and their duals) I Goal: identify an optimal basis, as in Definition 3.3 I Versions we will consider:
More informationIntroduction to Mathematical Programming
Introduction to Mathematical Programming Ming Zhong Lecture 22 October 22, 2018 Ming Zhong (JHU) AMS Fall 2018 1 / 16 Table of Contents 1 The Simplex Method, Part II Ming Zhong (JHU) AMS Fall 2018 2 /
More information4.6 Linear Programming duality
4.6 Linear Programming duality To any minimization (maximization) LP we can associate a closely related maximization (minimization) LP Different spaces and objective functions but in general same optimal
More informationToday: Linear Programming (con t.)
Today: Linear Programming (con t.) COSC 581, Algorithms April 10, 2014 Many of these slides are adapted from several online sources Reading Assignments Today s class: Chapter 29.4 Reading assignment for
More information{ move v ars to left, consts to right { replace = by t wo and constraints Ax b often nicer for theory Ax = b good for implementations. { A invertible
Finish remarks on min-cost ow. Strongly polynomial algorithms exist. { Tardos 1985 { minimum mean-cost cycle { reducing -optimality { \xing" arcs of very high reduced cost { best running running time roughly
More informationLecture 10: Linear programming. duality. and. The dual of the LP in standard form. maximize w = b T y (D) subject to A T y c, minimize z = c T x (P)
Lecture 10: Linear programming duality Michael Patriksson 19 February 2004 0-0 The dual of the LP in standard form minimize z = c T x (P) subject to Ax = b, x 0 n, and maximize w = b T y (D) subject to
More informationConvex Optimization and Support Vector Machine
Convex Optimization and Support Vector Machine Problem 0. Consider a two-class classification problem. The training data is L n = {(x 1, t 1 ),..., (x n, t n )}, where each t i { 1, 1} and x i R p. We
More informationThursday, May 24, Linear Programming
Linear Programming Linear optimization problems max f(x) g i (x) b i x j R i =1,...,m j =1,...,n Optimization problem g i (x) f(x) When and are linear functions Linear Programming Problem 1 n max c x n
More informationLinear Programming. Operations Research. Anthony Papavasiliou 1 / 21
1 / 21 Linear Programming Operations Research Anthony Papavasiliou Contents 2 / 21 1 Primal Linear Program 2 Dual Linear Program Table of Contents 3 / 21 1 Primal Linear Program 2 Dual Linear Program Linear
More informationLinear and Combinatorial Optimization
Linear and Combinatorial Optimization The dual of an LP-problem. Connections between primal and dual. Duality theorems and complementary slack. Philipp Birken (Ctr. for the Math. Sc.) Lecture 3: Duality
More informationCSCI5654 (Linear Programming, Fall 2013) Lecture-8. Lecture 8 Slide# 1
CSCI5654 (Linear Programming, Fall 2013) Lecture-8 Lecture 8 Slide# 1 Today s Lecture 1. Recap of dual variables and strong duality. 2. Complementary Slackness Theorem. 3. Interpretation of dual variables.
More informationInteger Programming ISE 418. Lecture 13. Dr. Ted Ralphs
Integer Programming ISE 418 Lecture 13 Dr. Ted Ralphs ISE 418 Lecture 13 1 Reading for This Lecture Nemhauser and Wolsey Sections II.1.1-II.1.3, II.1.6 Wolsey Chapter 8 CCZ Chapters 5 and 6 Valid Inequalities
More information4TE3/6TE3. Algorithms for. Continuous Optimization
4TE3/6TE3 Algorithms for Continuous Optimization (Duality in Nonlinear Optimization ) Tamás TERLAKY Computing and Software McMaster University Hamilton, January 2004 terlaky@mcmaster.ca Tel: 27780 Optimality
More informationChap6 Duality Theory and Sensitivity Analysis
Chap6 Duality Theory and Sensitivity Analysis The rationale of duality theory Max 4x 1 + x 2 + 5x 3 + 3x 4 S.T. x 1 x 2 x 3 + 3x 4 1 5x 1 + x 2 + 3x 3 + 8x 4 55 x 1 + 2x 2 + 3x 3 5x 4 3 x 1 ~x 4 0 If we
More information14. Duality. ˆ Upper and lower bounds. ˆ General duality. ˆ Constraint qualifications. ˆ Counterexample. ˆ Complementary slackness.
CS/ECE/ISyE 524 Introduction to Optimization Spring 2016 17 14. Duality ˆ Upper and lower bounds ˆ General duality ˆ Constraint qualifications ˆ Counterexample ˆ Complementary slackness ˆ Examples ˆ Sensitivity
More informationConvex Optimization & Lagrange Duality
Convex Optimization & Lagrange Duality Chee Wei Tan CS 8292 : Advanced Topics in Convex Optimization and its Applications Fall 2010 Outline Convex optimization Optimality condition Lagrange duality KKT
More information"SYMMETRIC" PRIMAL-DUAL PAIR
"SYMMETRIC" PRIMAL-DUAL PAIR PRIMAL Minimize cx DUAL Maximize y T b st Ax b st A T y c T x y Here c 1 n, x n 1, b m 1, A m n, y m 1, WITH THE PRIMAL IN STANDARD FORM... Minimize cx Maximize y T b st Ax
More informationLinear Programming: Simplex
Linear Programming: Simplex Stephen J. Wright 1 2 Computer Sciences Department, University of Wisconsin-Madison. IMA, August 2016 Stephen Wright (UW-Madison) Linear Programming: Simplex IMA, August 2016
More informationChapter 2: Linear Programming Basics. (Bertsimas & Tsitsiklis, Chapter 1)
Chapter 2: Linear Programming Basics (Bertsimas & Tsitsiklis, Chapter 1) 33 Example of a Linear Program Remarks. minimize 2x 1 x 2 + 4x 3 subject to x 1 + x 2 + x 4 2 3x 2 x 3 = 5 x 3 + x 4 3 x 1 0 x 3
More informationOptimization methods NOPT048
Optimization methods NOPT048 Jirka Fink https://ktiml.mff.cuni.cz/ fink/ Department of Theoretical Computer Science and Mathematical Logic Faculty of Mathematics and Physics Charles University in Prague
More information5. Duality. Lagrangian
5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationPart 1. The Review of Linear Programming Introduction
In the name of God Part 1. The Review of Linear Programming 1.1. Spring 2010 Instructor: Dr. Masoud Yaghini Outline The Linear Programming Problem Geometric Solution References The Linear Programming Problem
More informationLINEAR PROGRAMMING II
LINEAR PROGRAMMING II LP duality strong duality theorem bonus proof of LP duality applications Lecture slides by Kevin Wayne Last updated on 7/25/17 11:09 AM LINEAR PROGRAMMING II LP duality Strong duality
More informationLinear Programming. Murti V. Salapaka Electrical Engineering Department University Of Minnesota, Twin Cities
Linear Programming Murti V Salapaka Electrical Engineering Department University Of Minnesota, Twin Cities murtis@umnedu September 4, 2012 Linear Programming 1 The standard Linear Programming (SLP) problem:
More informationCSCI 1951-G Optimization Methods in Finance Part 01: Linear Programming
CSCI 1951-G Optimization Methods in Finance Part 01: Linear Programming January 26, 2018 1 / 38 Liability/asset cash-flow matching problem Recall the formulation of the problem: max w c 1 + p 1 e 1 = 150
More information2.098/6.255/ Optimization Methods Practice True/False Questions
2.098/6.255/15.093 Optimization Methods Practice True/False Questions December 11, 2009 Part I For each one of the statements below, state whether it is true or false. Include a 1-3 line supporting sentence
More informationDuality in Linear Programming
Duality in Linear Programming Gary D. Knott Civilized Software Inc. 1219 Heritage Park Circle Silver Spring MD 296 phone:31-962-3711 email:knott@civilized.com URL:www.civilized.com May 1, 213.1 Duality
More information4. Duality and Sensitivity
4. Duality and Sensitivity For every instance of an LP, there is an associated LP known as the dual problem. The original problem is known as the primal problem. There are two de nitions of the dual pair
More informationDuality of LPs and Applications
Lecture 6 Duality of LPs and Applications Last lecture we introduced duality of linear programs. We saw how to form duals, and proved both the weak and strong duality theorems. In this lecture we will
More informationConvex Optimization. (EE227A: UC Berkeley) Lecture 28. Suvrit Sra. (Algebra + Optimization) 02 May, 2013
Convex Optimization (EE227A: UC Berkeley) Lecture 28 (Algebra + Optimization) 02 May, 2013 Suvrit Sra Admin Poster presentation on 10th May mandatory HW, Midterm, Quiz to be reweighted Project final report
More informationLagrangian Duality Theory
Lagrangian Duality Theory Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye Chapter 14.1-4 1 Recall Primal and Dual
More informationExample Problem. Linear Program (standard form) CSCI5654 (Linear Programming, Fall 2013) Lecture-7. Duality
CSCI5654 (Linear Programming, Fall 013) Lecture-7 Duality Lecture 7 Slide# 1 Lecture 7 Slide# Linear Program (standard form) Example Problem maximize c 1 x 1 + + c n x n s.t. a j1 x 1 + + a jn x n b j
More information8. Geometric problems
8. Geometric problems Convex Optimization Boyd & Vandenberghe extremal volume ellipsoids centering classification placement and facility location 8 Minimum volume ellipsoid around a set Löwner-John ellipsoid
More informationOptimality, Duality, Complementarity for Constrained Optimization
Optimality, Duality, Complementarity for Constrained Optimization Stephen Wright University of Wisconsin-Madison May 2014 Wright (UW-Madison) Optimality, Duality, Complementarity May 2014 1 / 41 Linear
More informationConstrained Optimization and Lagrangian Duality
CIS 520: Machine Learning Oct 02, 2017 Constrained Optimization and Lagrangian Duality Lecturer: Shivani Agarwal Disclaimer: These notes are designed to be a supplement to the lecture. They may or may
More informationMotivating examples Introduction to algorithms Simplex algorithm. On a particular example General algorithm. Duality An application to game theory
Instructor: Shengyu Zhang 1 LP Motivating examples Introduction to algorithms Simplex algorithm On a particular example General algorithm Duality An application to game theory 2 Example 1: profit maximization
More informationOPTIMISATION 3: NOTES ON THE SIMPLEX ALGORITHM
OPTIMISATION 3: NOTES ON THE SIMPLEX ALGORITHM Abstract These notes give a summary of the essential ideas and results It is not a complete account; see Winston Chapters 4, 5 and 6 The conventions and notation
More information12. Interior-point methods
12. Interior-point methods Convex Optimization Boyd & Vandenberghe inequality constrained minimization logarithmic barrier function and central path barrier method feasibility and phase I methods complexity
More informationOptimization for Communications and Networks. Poompat Saengudomlert. Session 4 Duality and Lagrange Multipliers
Optimization for Communications and Networks Poompat Saengudomlert Session 4 Duality and Lagrange Multipliers P Saengudomlert (2015) Optimization Session 4 1 / 14 24 Dual Problems Consider a primal convex
More information1 Maximal Lattice-free Convex Sets
47-831: Advanced Integer Programming Lecturer: Amitabh Basu Lecture 3 Date: 03/23/2010 In this lecture, we explore the connections between lattices of R n and convex sets in R n. The structures will prove
More informationDuality and Projections
Duality and Projections What s the use? thst@man.dtu.dk DTU-Management Technical University of Denmark 1 Outline Projections revisited... Farka s lemma Proposition 2.22 and 2.23 Duality theory (2.6) Complementary
More informationICS-E4030 Kernel Methods in Machine Learning
ICS-E4030 Kernel Methods in Machine Learning Lecture 3: Convex optimization and duality Juho Rousu 28. September, 2016 Juho Rousu 28. September, 2016 1 / 38 Convex optimization Convex optimisation This
More informationMath 5593 Linear Programming Week 1
University of Colorado Denver, Fall 2013, Prof. Engau 1 Problem-Solving in Operations Research 2 Brief History of Linear Programming 3 Review of Basic Linear Algebra Linear Programming - The Story About
More informationPart 1. The Review of Linear Programming
In the name of God Part 1. The Review of Linear Programming 1.5. Spring 2010 Instructor: Dr. Masoud Yaghini Outline Introduction Formulation of the Dual Problem Primal-Dual Relationship Economic Interpretation
More informationmin 4x 1 5x 2 + 3x 3 s.t. x 1 + 2x 2 + x 3 = 10 x 1 x 2 6 x 1 + 3x 2 + x 3 14
The exam is three hours long and consists of 4 exercises. The exam is graded on a scale 0-25 points, and the points assigned to each question are indicated in parenthesis within the text. If necessary,
More information3 The Simplex Method. 3.1 Basic Solutions
3 The Simplex Method 3.1 Basic Solutions In the LP of Example 2.3, the optimal solution happened to lie at an extreme point of the feasible set. This was not a coincidence. Consider an LP in general form,
More informationSparse Optimization Lecture: Dual Certificate in l 1 Minimization
Sparse Optimization Lecture: Dual Certificate in l 1 Minimization Instructor: Wotao Yin July 2013 Note scriber: Zheng Sun Those who complete this lecture will know what is a dual certificate for l 1 minimization
More information