Week 3 Linear programming duality
|
|
- James Brown
- 5 years ago
- Views:
Transcription
1 Week 3 Linear programming duality This week we cover the fascinating topic of linear programming duality. We will learn that every minimization program has associated a maximization program that has the same value. This surprising connection will allow us to efficiently certify the optimality or the infeasibility of a given linear program. 3.1 Bounding the value of a linear program To develop some intuition, we will explore the idea of bounding the value of the following linear program: 2x 1 + 3x 2 6x 1 + 4x 2 10 x 1 + 5x 2 4 x 1, x 2 0 We can get a first easy upperbound by noting that for any feasible solution (x 1, x 2 ) we have 2x 1 + 3x 2 6x 1 + 4x 2 10, where the first inequality follows from the fact that x 1, x 2 0 and the second from the first constraint of the program. We can get another upperbound by noting that for any feasible solution (x 1, x 2 ) we have 2x 1 + 3x 2 2x x 2 = 2(x 1 + 5x 2 ) 2 4 = 8, where the second inequality follows from the second constraint of the program. We can get an even better upperbound if we combine the two constraints: 2x 1 + 3x (6x 1 + 4x 2 ) (x 1 + 5x 2 ) = Taking a more systematic approach, we could ask to find multipliers y 1, y 2 0 for each constraint to obtain the best possible upperbound on the value of our program. Copyright 2015 Julián Mestre. This work is licensed under a Creative Commons Attribution-NonCommercial- ShareAlike 4.0 International License. For the most recent version of these notes, please visit the class webpage.
2 week 3 linear programming duality discrete optimization 2 This leads to the following linear program 1 10y 1 + 4y 2 6y 1 + y 2 2 4y 1 + 5y 2 3 y 1, y In order to have the inequality 2x 1 + 3x 2 y 1 (6x 1 + 4x 2 ) + y 2 (x 1 + 5x 2 ), we ask that 2 6y 1 + y 2, and 3 4y 1 + 5y 2, so that we get the upperbound The best possible upperbound is attained by y 1 = 26 7 and y 2 = 10 which yields a bound of How good an upperbound is this? As 26, 10y 1 + 4y 2, which we should try to. good as it gets! Indeed, there is a solution to the original program with the same value 2. The auxiliary linear program is the dual 2 The solution x 1 = and x 2 = 13 7 has value Amazing! program of the original program, which is called primal. In the rest of this lecture, we will define formally how to derive the dual of a given program and to prove the remarkable property that the value of the primal and dual programs is the same, provided they are both feasible. 3.2 How to derive the dual program Let A R m n be a matrix with columns A 1,..., A n and rows a 1,..., a m. With each primal linear program we associate the dual program j c jx j a T i x b i i M 1 a T i x b i i M 2 a T i x = b i i M 3 x j 0 j N 1 x j 0 j N 2 x j free j N 3 i b iy i y T A j c j j N 1 y T A j c j j N 2 y T A j = c j j N 3 y i 0 i M 1 y i 0 i M 2 y i free i M 3 (P) (D) Of course, we must have and m = M 1 + M 2 + M 3 n = N 1 + N 2 + N 3. A variable x j or y i is free if it can take positive and negative values. We can define the dual of a maximizing program by reversing the direction of the transformation. In this way, taking the dual of the dual brings us back to the linear program we started from! 3.3 Weak duality The Weak Duality Theorem states that if (P) and (D) are feasible then the value of (P) is greater than or equal to the value of (D). Theorem 3.1. Let x and y be feasible solutions of (P) and (D) respectively. Then.
3 week 3 linear programming duality discrete optimization 3 Proof. One could say that the dual has been defined with the express purpose to make this proof go through: c j x j y T A j x j = y T Ax = y i a T i x y i b i. (3.1) j i i j The first inequality holds because, if j N 1 then x j 0 and c j y T A j, so c j x j y T A j x j ; if j N 2 then x j 0 and c j y T A j, so c j x j y T A j x j ; if j N 3 then x j free and y T A j = c j, so c j x j = y T A j x j. The second inequality can be justified in a similar fashion. Corollary 3.1. If (P) has unbounded objective then (D) is infeasible. Similarly, if (D) has unbounded objective then (P) is infeasible. 3.4 Strong duality The collorary follows from Theorem 3.1: Any feasible solution of (D) sets a lowerbound on the value of (P). Similarly, any solution of (P) sets an upperbound on the value of (D). The Strong Duality Theorem states that the value of the primal and the dual programs must be the same, provided the programs have bounded objective. Theorem 3.2. If a linear program is feasible and has bounded objective, so does its dual. Furthermore, the value of the two programs is the same. Proof. We will prove the theorem only for linear programs in standard form 3 : 3 In the Exercises we will show that it indeed holds for any linear program Ax = b The dual program is: y T A c T y free Suppose the primal program has bounded objective. Then if we run Simplex on the primal program, it will return an optimal basic feasible solution x. Let B be the basis associated with x. Recall that the termination criterion for Simplex was that reduced costs at B were non-negative; that is, c T c B T A 1 B A 0. Therefore, the solution y T = c T B A 1 B is dual feasible. The cost of this solution is = y T b = c B T A 1 B b = c B T x B =. From Theorem 3.1, it follows that y is an optimal dual solution. Furthermore, x and y have the same objective value.
4 week 3 linear programming duality discrete optimization Complementary slackness Strong duality constrains the structure of the optimal solutions of primal and dual programs. We now show that a pair of optimal primal and dual solutions must obey the so-called complementary slackness property. Theorem 3.3. Let x and y be feasible solutions to (P) and (D) respectively. Then x and y are optimal if and only if i) x j = 0 or y T A j = c j for all j N 1 N 2 N 3, and ii) y i = 0 or a i T x = b i for all i M 1 M 2 M 3. Notice that this Theorem only provides interesting information for indices associated with inequality constraints (j N 1 N 2 and i M 1 M 2 ). For equality constraints (j N 3 and i M 3 ) the property holds trivially. Proof. Consider the proof of Theorem 3.1 applied to the optimal solution x and y. By Theorem 3.2 we know that if the solution are optimal then =, therefore, each of the inequalities in (3.1) must in fact, be met with equality. Conversely, if we have equality throughout the cost of the solutions must be the same. It is easy to check that the first inequality in (3.1) is met with equality if and only if i) holds. Similarly, the second inequality in (3.1) is met with equality if and only if ii) holds. 3.6 Certifying infeasibility If the program (P) and its dual (D) are feasible, Theorem 3.2 tells us that they must have the same value. Therefore, if one of them is feasible and it is has unbounded objective, then the other one must necessarily be infeasible. This connection between (P) and (D) can be exploited to design a simple certificate of infeasibility. Theorem 3.4 (Farkas lemma). Let A R m n and b R m. Then exactly one of the following alternatives holds, i) there exists x R n such that Ax = b and, or ii) there exists y R m such that A T y 0 and < 0. Proof. Let us first argue that both alternatives cannot hold simultaneously. Indeed, if Ax = b then y T Ax = y T b. Notice that and y T A 0, so y T A. On the other hand, we know that y T b < 0, os we get a contradiction. Now assume that there is no x R n such that and Ax = b. Then the following linear program is infeasible 0 x While the statement and the proof of Theorem 3.4 may seem abstract and unintuitive at first, there is a neat geometric interpretation of the Theorem statement. We can think of Ax as a positive linear combination of the columns of A. Therefore, the system is Ax = b, is infeasible if and only if b lies outside conical hull of the columns of A. Intuitively b lies outside this cone if and only if there is a hyperplane separating the two. The vector y is the normal direction of this hyperplane. {Ax : } A 1 A 2 A 3 Ax = b A 4 b y
5 week 3 linear programming duality discrete optimization 5 By Theorem 3.2, this means its dual must be either infeasible or have unbounded objective. The dual is the following program A T y 0 y free The program is feasible since y = 0 is a valid solution. Therefore, there must be a solution y such that A T y 0 and < Physical interpretation of duality The exposition in this section is not meant to be a rigorous mathematical argument, but rather a helpful image for building intuition. Feel free to skip this part if you find it confusing. Consider the linear program and its dual Ax b x free y T A = c T y 0 Let A have rows a T 1,..., at m. Picture a physical representation of the polyhedron P = {x : Ax b} where the hyperplanes a i x = b i are solid walls. Imagine placing a ball inside P and letting it fall under the influence of a gravitation force that pull the ball in the opposite direction of c. If the ball is small enough, after some time it will come to a rest near some corner x of P. The ball is stationary because the gravitational pull is perfectly cancelled by the forces exerted by the walls it rests on. In other words, there are coefficients y 1,..., y n 0 such that a 1 a 3 c y 3 a 3 y 1 a 1 c = i y i a i. Notice that y is a dual feasible solution. Furthermore, y i > 0 only for those hyperplanes that meet at x. Therefore, if a i x > b i then y i = 0. Therefore, In other words, for all i y i = 0 or a i x = b i. = i y i b i = i y i a i T x = y T Ax = c T x. This is precisely the complementary slackness condition ii) of Theorem 3.3 In other words, we get strong duality. Exercises 1. Each linear program can be classified into infeasible, feasible with unbounded objective, or feasible with bounded objective.
6 week 3 linear programming duality discrete optimization 6 Therefore, a pair of primal and dual programs can potentially be classified into nine possible types. Not all nine combination can be achieved. For each combination either argue why it cannot be achieved, or show example (a pair of primal and dual programs). Hint: it is possible to have an unfeasible-unfeasible primal-dual pair of programs. 2. Our proof of Theorem 3.2 only works for linear programs in standard form. Show that it also holds for programs of the form: Ax b First turn the above program into an equivalent 4 program in 4 We already did this in Week 1. standard form by introducing additional slack variables. Then show that the duals of the original program and the one in standard form are equivalent. 3. Our proof of Theorem 3.2 only works for linear programs in standard form. Show that it also holds for programs of the form: A T y c y 0 Notice that the proof of Theorem 3.2 does not work here because the program is not in standard form, but rather it is dual of a program in standard form. 4. Let A R m n and b R m. Prove that exactly one of the following alternatives holds, i) there exists x R n such that Ax b, or ii) there exists y R m such that A T y = 0 and y 0 and > 0. Prove that the system Ax = 0 and x > 0 is infeasible if and only if the system A T y 0 and A T y = 0 is feasible. Let a and a 1,..., a m be vectors in R n. Prove that the following statements are equivalent: (a) for all x R n such that we have a x max i a i x, or (b) there exists λ R m such that λ 0 and i λ i = 1 and a i λ ia i. Here x > 0 means every coordinate of x is strictly positive.
7 week 3 linear programming duality discrete optimization 7 Solutions of selected exercises 2. Our proof of Theorem 3.2 only works for linear programs in standard form. Show that it also holds for programs of the form: Ax b Solution. Call the above program (P). Its dual, which we will denote by (D), is A T y c y 0 We introduce an equivalent program (P ) in standard form by adding dummy variables that eat up the slack. Ax Iz = b z 0 We call the dual of this new program (D ): A T y c Iy 0 y free Notice that (D ) is in fact equivalent to (D), since Iy 0 is the same constraint as y 0. To finish the argument, we note that if (P) is feasible and has a bounded objective then (P ) is feasible and has bounded objective, because they are, in essence, the same program. Using the proof of Theorem 3.2 from the notes, we get that (D ) is feasible and has the same value of (P ). But since (D ) and (D) are equivalent, it follows that (D) is also feasible and has the same value as (P).
Introduction to Mathematical Programming IE406. Lecture 10. Dr. Ted Ralphs
Introduction to Mathematical Programming IE406 Lecture 10 Dr. Ted Ralphs IE406 Lecture 10 1 Reading for This Lecture Bertsimas 4.1-4.3 IE406 Lecture 10 2 Duality Theory: Motivation Consider the following
More informationDiscrete Optimization
Prof. Friedrich Eisenbrand Martin Niemeier Due Date: April 15, 2010 Discussions: March 25, April 01 Discrete Optimization Spring 2010 s 3 You can hand in written solutions for up to two of the exercises
More informationThe Simplex Algorithm
8.433 Combinatorial Optimization The Simplex Algorithm October 6, 8 Lecturer: Santosh Vempala We proved the following: Lemma (Farkas). Let A R m n, b R m. Exactly one of the following conditions is true:.
More informationCO 250 Final Exam Guide
Spring 2017 CO 250 Final Exam Guide TABLE OF CONTENTS richardwu.ca CO 250 Final Exam Guide Introduction to Optimization Kanstantsin Pashkovich Spring 2017 University of Waterloo Last Revision: March 4,
More information4. Algebra and Duality
4-1 Algebra and Duality P. Parrilo and S. Lall, CDC 2003 2003.12.07.01 4. Algebra and Duality Example: non-convex polynomial optimization Weak duality and duality gap The dual is not intrinsic The cone
More informationChapter 1 Linear Programming. Paragraph 5 Duality
Chapter 1 Linear Programming Paragraph 5 Duality What we did so far We developed the 2-Phase Simplex Algorithm: Hop (reasonably) from basic solution (bs) to bs until you find a basic feasible solution
More informationSpring 2017 CO 250 Course Notes TABLE OF CONTENTS. richardwu.ca. CO 250 Course Notes. Introduction to Optimization
Spring 2017 CO 250 Course Notes TABLE OF CONTENTS richardwu.ca CO 250 Course Notes Introduction to Optimization Kanstantsin Pashkovich Spring 2017 University of Waterloo Last Revision: March 4, 2018 Table
More informationLP Duality: outline. Duality theory for Linear Programming. alternatives. optimization I Idea: polyhedra
LP Duality: outline I Motivation and definition of a dual LP I Weak duality I Separating hyperplane theorem and theorems of the alternatives I Strong duality and complementary slackness I Using duality
More informationLinear Programming Duality
Summer 2011 Optimization I Lecture 8 1 Duality recap Linear Programming Duality We motivated the dual of a linear program by thinking about the best possible lower bound on the optimal value we can achieve
More informationI.3. LMI DUALITY. Didier HENRION EECI Graduate School on Control Supélec - Spring 2010
I.3. LMI DUALITY Didier HENRION henrion@laas.fr EECI Graduate School on Control Supélec - Spring 2010 Primal and dual For primal problem p = inf x g 0 (x) s.t. g i (x) 0 define Lagrangian L(x, z) = g 0
More information1 Seidel s LP algorithm
15-451/651: Design & Analysis of Algorithms October 21, 2015 Lecture #14 last changed: November 7, 2015 In this lecture we describe a very nice algorithm due to Seidel for Linear Programming in lowdimensional
More informationLinear Programming Inverse Projection Theory Chapter 3
1 Linear Programming Inverse Projection Theory Chapter 3 University of Chicago Booth School of Business Kipp Martin September 26, 2017 2 Where We Are Headed We want to solve problems with special structure!
More informationExample Problem. Linear Program (standard form) CSCI5654 (Linear Programming, Fall 2013) Lecture-7. Duality
CSCI5654 (Linear Programming, Fall 013) Lecture-7 Duality Lecture 7 Slide# 1 Lecture 7 Slide# Linear Program (standard form) Example Problem maximize c 1 x 1 + + c n x n s.t. a j1 x 1 + + a jn x n b j
More informationFarkas Lemma, Dual Simplex and Sensitivity Analysis
Summer 2011 Optimization I Lecture 10 Farkas Lemma, Dual Simplex and Sensitivity Analysis 1 Farkas Lemma Theorem 1. Let A R m n, b R m. Then exactly one of the following two alternatives is true: (i) x
More information1 Review Session. 1.1 Lecture 2
1 Review Session Note: The following lists give an overview of the material that was covered in the lectures and sections. Your TF will go through these lists. If anything is unclear or you have questions
More information"SYMMETRIC" PRIMAL-DUAL PAIR
"SYMMETRIC" PRIMAL-DUAL PAIR PRIMAL Minimize cx DUAL Maximize y T b st Ax b st A T y c T x y Here c 1 n, x n 1, b m 1, A m n, y m 1, WITH THE PRIMAL IN STANDARD FORM... Minimize cx Maximize y T b st Ax
More informationReview Solutions, Exam 2, Operations Research
Review Solutions, Exam 2, Operations Research 1. Prove the weak duality theorem: For any x feasible for the primal and y feasible for the dual, then... HINT: Consider the quantity y T Ax. SOLUTION: To
More informationLecture 10: Linear programming. duality. and. The dual of the LP in standard form. maximize w = b T y (D) subject to A T y c, minimize z = c T x (P)
Lecture 10: Linear programming duality Michael Patriksson 19 February 2004 0-0 The dual of the LP in standard form minimize z = c T x (P) subject to Ax = b, x 0 n, and maximize w = b T y (D) subject to
More informationConic Linear Optimization and its Dual. yyye
Conic Linear Optimization and Appl. MS&E314 Lecture Note #04 1 Conic Linear Optimization and its Dual Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A.
More informationAlgorithmic Game Theory and Applications. Lecture 7: The LP Duality Theorem
Algorithmic Game Theory and Applications Lecture 7: The LP Duality Theorem Kousha Etessami recall LP s in Primal Form 1 Maximize c 1 x 1 + c 2 x 2 +... + c n x n a 1,1 x 1 + a 1,2 x 2 +... + a 1,n x n
More informationCSC373: Algorithm Design, Analysis and Complexity Fall 2017 DENIS PANKRATOV NOVEMBER 1, 2017
CSC373: Algorithm Design, Analysis and Complexity Fall 2017 DENIS PANKRATOV NOVEMBER 1, 2017 Linear Function f: R n R is linear if it can be written as f x = a T x for some a R n Example: f x 1, x 2 =
More informationDuality of LPs and Applications
Lecture 6 Duality of LPs and Applications Last lecture we introduced duality of linear programs. We saw how to form duals, and proved both the weak and strong duality theorems. In this lecture we will
More informationLecture 5. x 1,x 2,x 3 0 (1)
Computational Intractability Revised 2011/6/6 Lecture 5 Professor: David Avis Scribe:Ma Jiangbo, Atsuki Nagao 1 Duality The purpose of this lecture is to introduce duality, which is an important concept
More informationChapter 3, Operations Research (OR)
Chapter 3, Operations Research (OR) Kent Andersen February 7, 2007 1 Linear Programs (continued) In the last chapter, we introduced the general form of a linear program, which we denote (P) Minimize Z
More informationLinear Programming. Larry Blume Cornell University, IHS Vienna and SFI. Summer 2016
Linear Programming Larry Blume Cornell University, IHS Vienna and SFI Summer 2016 These notes derive basic results in finite-dimensional linear programming using tools of convex analysis. Most sources
More informationDuality in Linear Programming
Duality in Linear Programming Gary D. Knott Civilized Software Inc. 1219 Heritage Park Circle Silver Spring MD 296 phone:31-962-3711 email:knott@civilized.com URL:www.civilized.com May 1, 213.1 Duality
More informationLinear programming. Saad Mneimneh. maximize x 1 + x 2 subject to 4x 1 x 2 8 2x 1 + x x 1 2x 2 2
Linear programming Saad Mneimneh 1 Introduction Consider the following problem: x 1 + x x 1 x 8 x 1 + x 10 5x 1 x x 1, x 0 The feasible solution is a point (x 1, x ) that lies within the region defined
More informationLinear Programming. Chapter Introduction
Chapter 3 Linear Programming Linear programs (LP) play an important role in the theory and practice of optimization problems. Many COPs can directly be formulated as LPs. Furthermore, LPs are invaluable
More informationLecture 10: Linear programming duality and sensitivity 0-0
Lecture 10: Linear programming duality and sensitivity 0-0 The canonical primal dual pair 1 A R m n, b R m, and c R n maximize z = c T x (1) subject to Ax b, x 0 n and minimize w = b T y (2) subject to
More informationExample: feasibility. Interpretation as formal proof. Example: linear inequalities and Farkas lemma
4-1 Algebra and Duality P. Parrilo and S. Lall 2006.06.07.01 4. Algebra and Duality Example: non-convex polynomial optimization Weak duality and duality gap The dual is not intrinsic The cone of valid
More informationCS261: A Second Course in Algorithms Lecture #9: Linear Programming Duality (Part 2)
CS261: A Second Course in Algorithms Lecture #9: Linear Programming Duality (Part 2) Tim Roughgarden February 2, 2016 1 Recap This is our third lecture on linear programming, and the second on linear programming
More informationPart 1. The Review of Linear Programming
In the name of God Part 1. The Review of Linear Programming 1.5. Spring 2010 Instructor: Dr. Masoud Yaghini Outline Introduction Formulation of the Dual Problem Primal-Dual Relationship Economic Interpretation
More informationA Review of Linear Programming
A Review of Linear Programming Instructor: Farid Alizadeh IEOR 4600y Spring 2001 February 14, 2001 1 Overview In this note we review the basic properties of linear programming including the primal simplex
More informationCS 6820 Fall 2014 Lectures, October 3-20, 2014
Analysis of Algorithms Linear Programming Notes CS 6820 Fall 2014 Lectures, October 3-20, 2014 1 Linear programming The linear programming (LP) problem is the following optimization problem. We are given
More informationLecture 5. Theorems of Alternatives and Self-Dual Embedding
IE 8534 1 Lecture 5. Theorems of Alternatives and Self-Dual Embedding IE 8534 2 A system of linear equations may not have a solution. It is well known that either Ax = c has a solution, or A T y = 0, c
More informationLectures 6, 7 and part of 8
Lectures 6, 7 and part of 8 Uriel Feige April 26, May 3, May 10, 2015 1 Linear programming duality 1.1 The diet problem revisited Recall the diet problem from Lecture 1. There are n foods, m nutrients,
More informationUnderstanding the Simplex algorithm. Standard Optimization Problems.
Understanding the Simplex algorithm. Ma 162 Spring 2011 Ma 162 Spring 2011 February 28, 2011 Standard Optimization Problems. A standard maximization problem can be conveniently described in matrix form
More informationBBM402-Lecture 20: LP Duality
BBM402-Lecture 20: LP Duality Lecturer: Lale Özkahya Resources for the presentation: https://courses.engr.illinois.edu/cs473/fa2016/lectures.html An easy LP? which is compact form for max cx subject to
More information1. Algebraic and geometric treatments Consider an LP problem in the standard form. x 0. Solutions to the system of linear equations
The Simplex Method Most textbooks in mathematical optimization, especially linear programming, deal with the simplex method. In this note we study the simplex method. It requires basically elementary linear
More informationFinite Dimensional Optimization Part III: Convex Optimization 1
John Nachbar Washington University March 21, 2017 Finite Dimensional Optimization Part III: Convex Optimization 1 1 Saddle points and KKT. These notes cover another important approach to optimization,
More informationII. Analysis of Linear Programming Solutions
Optimization Methods Draft of August 26, 2005 II. Analysis of Linear Programming Solutions Robert Fourer Department of Industrial Engineering and Management Sciences Northwestern University Evanston, Illinois
More informationExtreme Abridgment of Boyd and Vandenberghe s Convex Optimization
Extreme Abridgment of Boyd and Vandenberghe s Convex Optimization Compiled by David Rosenberg Abstract Boyd and Vandenberghe s Convex Optimization book is very well-written and a pleasure to read. The
More information1 Overview. 2 Extreme Points. AM 221: Advanced Optimization Spring 2016
AM 22: Advanced Optimization Spring 206 Prof. Yaron Singer Lecture 7 February 7th Overview In the previous lectures we saw applications of duality to game theory and later to learning theory. In this lecture
More informationNote 3: LP Duality. If the primal problem (P) in the canonical form is min Z = n (1) then the dual problem (D) in the canonical form is max W = m (2)
Note 3: LP Duality If the primal problem (P) in the canonical form is min Z = n j=1 c j x j s.t. nj=1 a ij x j b i i = 1, 2,..., m (1) x j 0 j = 1, 2,..., n, then the dual problem (D) in the canonical
More informationEE364a Review Session 5
EE364a Review Session 5 EE364a Review announcements: homeworks 1 and 2 graded homework 4 solutions (check solution to additional problem 1) scpd phone-in office hours: tuesdays 6-7pm (650-723-1156) 1 Complementary
More informationLINEAR PROGRAMMING II
LINEAR PROGRAMMING II LP duality strong duality theorem bonus proof of LP duality applications Lecture slides by Kevin Wayne Last updated on 7/25/17 11:09 AM LINEAR PROGRAMMING II LP duality Strong duality
More informationIntroduction to Operations Research Prof. G. Srinivasan Department of Management Studies Indian Institute of Technology, Madras
Introduction to Operations Research Prof. G. Srinivasan Department of Management Studies Indian Institute of Technology, Madras Module - 03 Simplex Algorithm Lecture 15 Infeasibility In this class, we
More informationMidterm Review. Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A.
Midterm Review Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye (LY, Chapter 1-4, Appendices) 1 Separating hyperplane
More informationCSCI 1951-G Optimization Methods in Finance Part 01: Linear Programming
CSCI 1951-G Optimization Methods in Finance Part 01: Linear Programming January 26, 2018 1 / 38 Liability/asset cash-flow matching problem Recall the formulation of the problem: max w c 1 + p 1 e 1 = 150
More informationMATH 373 Section A1. Final Exam. Dr. J. Bowman 17 December :00 17:00
MATH 373 Section A1 Final Exam Dr. J. Bowman 17 December 2018 14:00 17:00 Name (Last, First): Student ID: Email: @ualberta.ca Scrap paper is supplied. No notes or books are permitted. All electronic equipment,
More informationIE 5531: Engineering Optimization I
IE 5531: Engineering Optimization I Lecture 7: Duality and applications Prof. John Gunnar Carlsson September 29, 2010 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 29, 2010 1
More informationIE 521 Convex Optimization Homework #1 Solution
IE 521 Convex Optimization Homework #1 Solution your NAME here your NetID here February 13, 2019 Instructions. Homework is due Wednesday, February 6, at 1:00pm; no late homework accepted. Please use the
More information(P ) Minimize 4x 1 + 6x 2 + 5x 3 s.t. 2x 1 3x 3 3 3x 2 2x 3 6
The exam is three hours long and consists of 4 exercises. The exam is graded on a scale 0-25 points, and the points assigned to each question are indicated in parenthesis within the text. Problem 1 Consider
More information3. Linear Programming and Polyhedral Combinatorics
Massachusetts Institute of Technology 18.433: Combinatorial Optimization Michel X. Goemans February 28th, 2013 3. Linear Programming and Polyhedral Combinatorics Summary of what was seen in the introductory
More informationSensitivity Analysis and Duality
Sensitivity Analysis and Duality Part II Duality Based on Chapter 6 Introduction to Mathematical Programming: Operations Research, Volume 1 4th edition, by Wayne L. Winston and Munirpallam Venkataramanan
More informationLecture 1: Entropy, convexity, and matrix scaling CSE 599S: Entropy optimality, Winter 2016 Instructor: James R. Lee Last updated: January 24, 2016
Lecture 1: Entropy, convexity, and matrix scaling CSE 599S: Entropy optimality, Winter 2016 Instructor: James R. Lee Last updated: January 24, 2016 1 Entropy Since this course is about entropy maximization,
More informationLECTURE 10 LECTURE OUTLINE
LECTURE 10 LECTURE OUTLINE Min Common/Max Crossing Th. III Nonlinear Farkas Lemma/Linear Constraints Linear Programming Duality Convex Programming Duality Optimality Conditions Reading: Sections 4.5, 5.1,5.2,
More informationToday: Linear Programming (con t.)
Today: Linear Programming (con t.) COSC 581, Algorithms April 10, 2014 Many of these slides are adapted from several online sources Reading Assignments Today s class: Chapter 29.4 Reading assignment for
More informationConvex Optimization M2
Convex Optimization M2 Lecture 3 A. d Aspremont. Convex Optimization M2. 1/49 Duality A. d Aspremont. Convex Optimization M2. 2/49 DMs DM par email: dm.daspremont@gmail.com A. d Aspremont. Convex Optimization
More informationSection Notes 9. IP: Cutting Planes. Applied Math 121. Week of April 12, 2010
Section Notes 9 IP: Cutting Planes Applied Math 121 Week of April 12, 2010 Goals for the week understand what a strong formulations is. be familiar with the cutting planes algorithm and the types of cuts
More informationTopic: Primal-Dual Algorithms Date: We finished our discussion of randomized rounding and began talking about LP Duality.
CS787: Advanced Algorithms Scribe: Amanda Burton, Leah Kluegel Lecturer: Shuchi Chawla Topic: Primal-Dual Algorithms Date: 10-17-07 14.1 Last Time We finished our discussion of randomized rounding and
More informationSupplementary lecture notes on linear programming. We will present an algorithm to solve linear programs of the form. maximize.
Cornell University, Fall 2016 Supplementary lecture notes on linear programming CS 6820: Algorithms 26 Sep 28 Sep 1 The Simplex Method We will present an algorithm to solve linear programs of the form
More informationmin 4x 1 5x 2 + 3x 3 s.t. x 1 + 2x 2 + x 3 = 10 x 1 x 2 6 x 1 + 3x 2 + x 3 14
The exam is three hours long and consists of 4 exercises. The exam is graded on a scale 0-25 points, and the points assigned to each question are indicated in parenthesis within the text. If necessary,
More informationSection Notes 9. Midterm 2 Review. Applied Math / Engineering Sciences 121. Week of December 3, 2018
Section Notes 9 Midterm 2 Review Applied Math / Engineering Sciences 121 Week of December 3, 2018 The following list of topics is an overview of the material that was covered in the lectures and sections
More informationIntroduction to linear programming using LEGO.
Introduction to linear programming using LEGO. 1 The manufacturing problem. A manufacturer produces two pieces of furniture, tables and chairs. The production of the furniture requires the use of two different
More informationLecture Note 18: Duality
MATH 5330: Computational Methods of Linear Algebra 1 The Dual Problems Lecture Note 18: Duality Xianyi Zeng Department of Mathematical Sciences, UTEP The concept duality, just like accuracy and stability,
More informationOPTIMISATION 2007/8 EXAM PREPARATION GUIDELINES
General: OPTIMISATION 2007/8 EXAM PREPARATION GUIDELINES This points out some important directions for your revision. The exam is fully based on what was taught in class: lecture notes, handouts and homework.
More informationSensitivity Analysis and Duality in LP
Sensitivity Analysis and Duality in LP Xiaoxi Li EMS & IAS, Wuhan University Oct. 13th, 2016 (week vi) Operations Research (Li, X.) Sensitivity Analysis and Duality in LP Oct. 13th, 2016 (week vi) 1 /
More informationDuality Theory, Optimality Conditions
5.1 Duality Theory, Optimality Conditions Katta G. Murty, IOE 510, LP, U. Of Michigan, Ann Arbor We only consider single objective LPs here. Concept of duality not defined for multiobjective LPs. Every
More informationLinear and Combinatorial Optimization
Linear and Combinatorial Optimization The dual of an LP-problem. Connections between primal and dual. Duality theorems and complementary slack. Philipp Birken (Ctr. for the Math. Sc.) Lecture 3: Duality
More informationSummer School: Semidefinite Optimization
Summer School: Semidefinite Optimization Christine Bachoc Université Bordeaux I, IMB Research Training Group Experimental and Constructive Algebra Haus Karrenberg, Sept. 3 - Sept. 7, 2012 Duality Theory
More information3. Linear Programming and Polyhedral Combinatorics
Massachusetts Institute of Technology 18.453: Combinatorial Optimization Michel X. Goemans April 5, 2017 3. Linear Programming and Polyhedral Combinatorics Summary of what was seen in the introductory
More informationSummary of the simplex method
MVE165/MMG630, The simplex method; degeneracy; unbounded solutions; infeasibility; starting solutions; duality; interpretation Ann-Brith Strömberg 2012 03 16 Summary of the simplex method Optimality condition:
More informationOPTIMISATION /09 EXAM PREPARATION GUIDELINES
General: OPTIMISATION 2 2008/09 EXAM PREPARATION GUIDELINES This points out some important directions for your revision. The exam is fully based on what was taught in class: lecture notes, handouts and
More informationOptimisation and Operations Research
Optimisation and Operations Research Lecture 9: Duality and Complementary Slackness Matthew Roughan http://www.maths.adelaide.edu.au/matthew.roughan/ Lecture_notes/OORII/
More informationAppendix PRELIMINARIES 1. THEOREMS OF ALTERNATIVES FOR SYSTEMS OF LINEAR CONSTRAINTS
Appendix PRELIMINARIES 1. THEOREMS OF ALTERNATIVES FOR SYSTEMS OF LINEAR CONSTRAINTS Here we consider systems of linear constraints, consisting of equations or inequalities or both. A feasible solution
More informationOPTIMISATION 3: NOTES ON THE SIMPLEX ALGORITHM
OPTIMISATION 3: NOTES ON THE SIMPLEX ALGORITHM Abstract These notes give a summary of the essential ideas and results It is not a complete account; see Winston Chapters 4, 5 and 6 The conventions and notation
More informationLecture #21. c T x Ax b. maximize subject to
COMPSCI 330: Design and Analysis of Algorithms 11/11/2014 Lecture #21 Lecturer: Debmalya Panigrahi Scribe: Samuel Haney 1 Overview In this lecture, we discuss linear programming. We first show that the
More informationMAT-INF4110/MAT-INF9110 Mathematical optimization
MAT-INF4110/MAT-INF9110 Mathematical optimization Geir Dahl August 20, 2013 Convexity Part IV Chapter 4 Representation of convex sets different representations of convex sets, boundary polyhedra and polytopes:
More informationOptimization WS 13/14:, by Y. Goldstein/K. Reinert, 9. Dezember 2013, 16: Linear programming. Optimization Problems
Optimization WS 13/14:, by Y. Goldstein/K. Reinert, 9. Dezember 2013, 16:38 2001 Linear programming Optimization Problems General optimization problem max{z(x) f j (x) 0,x D} or min{z(x) f j (x) 0,x D}
More informationLinear Programming: Chapter 5 Duality
Linear Programming: Chapter 5 Duality Robert J. Vanderbei September 30, 2010 Slides last edited on October 5, 2010 Operations Research and Financial Engineering Princeton University Princeton, NJ 08544
More informationLecture 7 Duality II
L. Vandenberghe EE236A (Fall 2013-14) Lecture 7 Duality II sensitivity analysis two-person zero-sum games circuit interpretation 7 1 Sensitivity analysis purpose: extract from the solution of an LP information
More informationCS261: A Second Course in Algorithms Lecture #8: Linear Programming Duality (Part 1)
CS261: A Second Course in Algorithms Lecture #8: Linear Programming Duality (Part 1) Tim Roughgarden January 28, 2016 1 Warm-Up This lecture begins our discussion of linear programming duality, which is
More informationIntroduction to Mathematical Programming
Introduction to Mathematical Programming Ming Zhong Lecture 22 October 22, 2018 Ming Zhong (JHU) AMS Fall 2018 1 / 16 Table of Contents 1 The Simplex Method, Part II Ming Zhong (JHU) AMS Fall 2018 2 /
More informationLinear Programming Duality P&S Chapter 3 Last Revised Nov 1, 2004
Linear Programming Duality P&S Chapter 3 Last Revised Nov 1, 2004 1 In this section we lean about duality, which is another way to approach linear programming. In particular, we will see: How to define
More informationConstrained Optimization and Lagrangian Duality
CIS 520: Machine Learning Oct 02, 2017 Constrained Optimization and Lagrangian Duality Lecturer: Shivani Agarwal Disclaimer: These notes are designed to be a supplement to the lecture. They may or may
More informationTheory and Internet Protocols
Game Lecture 2: Linear Programming and Zero Sum Nash Equilibrium Xiaotie Deng AIMS Lab Department of Computer Science Shanghai Jiaotong University September 26, 2016 1 2 3 4 Standard Form (P) Outline
More informationICS-E4030 Kernel Methods in Machine Learning
ICS-E4030 Kernel Methods in Machine Learning Lecture 3: Convex optimization and duality Juho Rousu 28. September, 2016 Juho Rousu 28. September, 2016 1 / 38 Convex optimization Convex optimisation This
More informationLinear and Integer Optimization (V3C1/F4C1)
Linear and Integer Optimization (V3C1/F4C1) Lecture notes Ulrich Brenner Research Institute for Discrete Mathematics, University of Bonn Winter term 2016/2017 March 8, 2017 12:02 1 Preface Continuous updates
More informationLinear Programming Redux
Linear Programming Redux Jim Bremer May 12, 2008 The purpose of these notes is to review the basics of linear programming and the simplex method in a clear, concise, and comprehensive way. The book contains
More information15-780: LinearProgramming
15-780: LinearProgramming J. Zico Kolter February 1-3, 2016 1 Outline Introduction Some linear algebra review Linear programming Simplex algorithm Duality and dual simplex 2 Outline Introduction Some linear
More informationOptimization methods NOPT048
Optimization methods NOPT048 Jirka Fink https://ktiml.mff.cuni.cz/ fink/ Department of Theoretical Computer Science and Mathematical Logic Faculty of Mathematics and Physics Charles University in Prague
More informationWeek 2. The Simplex method was developed by Dantzig in the late 40-ties.
1 The Simplex method Week 2 The Simplex method was developed by Dantzig in the late 40-ties. 1.1 The standard form The simplex method is a general description algorithm that solves any LPproblem instance.
More information4. Duality and Sensitivity
4. Duality and Sensitivity For every instance of an LP, there is an associated LP known as the dual problem. The original problem is known as the primal problem. There are two de nitions of the dual pair
More informationChapter 1. Preliminaries
Introduction This dissertation is a reading of chapter 4 in part I of the book : Integer and Combinatorial Optimization by George L. Nemhauser & Laurence A. Wolsey. The chapter elaborates links between
More informationHow to Take the Dual of a Linear Program
How to Take the Dual of a Linear Program Sébastien Lahaie January 12, 2015 This is a revised version of notes I wrote several years ago on taking the dual of a linear program (LP), with some bug and typo
More informationDuality and Projections
Duality and Projections What s the use? thst@man.dtu.dk DTU-Management Technical University of Denmark 1 Outline Projections revisited... Farka s lemma Proposition 2.22 and 2.23 Duality theory (2.6) Complementary
More informationApproximate Farkas Lemmas in Convex Optimization
Approximate Farkas Lemmas in Convex Optimization Imre McMaster University Advanced Optimization Lab AdvOL Graduate Student Seminar October 25, 2004 1 Exact Farkas Lemma Motivation 2 3 Future plans The
More informationReview Questions, Final Exam
Review Questions, Final Exam A few general questions 1. What does the Representation Theorem say (in linear programming)? 2. What is the Fundamental Theorem of Linear Programming? 3. What is the main idea
More informationOptimization 4. GAME THEORY
Optimization GAME THEORY DPK Easter Term Saddle points of two-person zero-sum games We consider a game with two players Player I can choose one of m strategies, indexed by i =,, m and Player II can choose
More information