Linear programming: Theory
|
|
- Mark Oliver
- 5 years ago
- Views:
Transcription
1 Division of the Humanities and Social Sciences Ec 181 KC Border Convex Analsis and Economic Theor Winter 2018 Topic 28: Linear programming: Theor 28.1 The saddlepoint theorem for linear programming The material for this chapter is based largel on the beautifull written book b David Gale [2]. A maximum linear program in standard inequalit form 1 is a constrained maximization problem of the form maximize x p x xa q (1) x 0 (2) where x and p belong to R n, q belongs to R m, and A is n m. The program is feasible if there is some x satisfing the constraints (1) and (2). Ever maximum linear program in standard inequalit form has a dual program, which is the minimization problem: q (3) A p (4) 0. (5) The original maximum linear program ma be called the primal program to distinguish it from the dual. Let us start b examining the Lagrangean for the primal program. Write (1) as q j (xa) j 0, j = 1,..., m, and let j denote the Lagrange multiplier for this constraint. Incorporate (2) b setting the domain X = R n +. The Lagrangean is then L(x, ) = p x + q xa. (6) 1 Gale [2] refers to this as the standard form of a linear program. However Dantzig [1] uses the term standard form in a different fashion. KC Border: for Ec 181, Winter 2018 src: LPTheor v ::15.57
2 KC Border Linear programming: Theor 28 2 Treating the dual as the problem of maximizing q A p 0, and using x to denote the vector of Lagrange multipliers, the Lagrangean for the dual is: q xa + x p, which is just (6) again. Consequentl, b the eas half of the Saddlepoint Theorem, if ( x, ȳ) is a saddlepoint of L(x, ) = p x + q xa over R n R m, then x is optimal for the primal program and ȳ is optimal (minimal) for the dual program. In particular, if there is a saddlepoint, then both programs are feasible. If we knew that both programs satisfied Slater s Condition, then the Saddlepoint Theorem would assert that an pair of optimal solutions would be a saddlepoint of the Lagrangean. Remarkabl, for the linear programming case, we do not need Slater s Condition Saddlepoint Theorem for Linear Programming The following are equivalent. 1. The Lagrangean has a saddlepoint over R n + R m The primal has an optimal solution. 3. The dual has an optimal solution. L(x, ) = p x + q xa 4. Both the primal and dual are feasible. Also, the following are equivalent. a. ( x, ȳ) is a saddlepoint of the Lagrangean. b. x is optimal for the primal and ȳ is optimal for the dual. c. x is feasible for the primal program and ȳ is feasible for the dual, and p x = q ȳ. The proof is broken down into a series of lemmas. The first of which is both eas and useful Lemma (Optimalit Criterion for LP) If x is feasible for the primal program and is feasible for its dual, then p x q. If in addition p x = q, then x is optimal and is optimal for the dual program, and p x = q = xa. Proof : Suppose x satisfies (1) and 0. Then xa q. Likewise if satisfies (4) and x 0, then xa x p. Combining these pieces proves the lemma. v ::15.57 src: LPTheor KC Border: for Ec 181, Winter 2018
3 KC Border Linear programming: Theor 28 3 If b some means we have found an optimum for the primal and the dual, then the Optimalit Criterion allows a simple wa to prove that it is an optimum. Most other optimization techniques do not include a proof that the outcome is optimal. The remaining gap is to show that if x is optimal, then the dual has an optimal solution ȳ and that p x = q ȳ (instead of p x < q ȳ). This brings us to the following Fundamental Dualit Theorem of LP If both a maximum linear program in standard inequalit form and its dual are feasible, then both have optimal solutions, and the values of the two programs are the same. If one of the programs is infeasible, neither has an optimum. Proof : (Gale [2]) Start b assuming both programs are feasible. We alread know that if x and are feasible for the primal and dual respectivel, then p x q. Thus it suffices to find a solution (x, ) 0 to the inequalities or, in matrix form xa q A p q x p 0, [x, ] A 0 p [q, p, 0]. (7) 0 A q Either these inequalities have a solution, or else b Farkas s Lemma , there is a nonnegative vector u 0, where u R m +, v R n +, and α R +, satisfing α A 0 p u A q α 0 (8) and [q, p, 0] u < 0. (9) α We shall show that this latter set of inequalities does not have a solution: Suppose b wa of contradiction that (8) and (9) have a nonnegative solution. Rewriting (8), we have Au αp (10) and va αq, (11) KC Border: for Ec 181, Winter 2018 src: LPTheor v ::15.57
4 KC Border Linear programming: Theor 28 4 while (9) becomes q u < p v. (12) Let x 0 be feasible for the primal, that is, xa q. Then xau q u (13) since u 0. Similarl let ȳ 0 be feasible for the dual, that is, Aȳ p. Then vaȳ v p (14) since v 0. We next show that α 0. For suppose α = 0. Then (10) becomes Au 0, which implies xau 0, since x 0. Also (11) implies vaȳ 0, since ȳ 0. Combining this with (13) and (14) ields q u xau 0 vaȳ v p, which contradicts (12). This shows that α > 0, so we ma without loss of generalit assume α = 1. In this case, (10) becomes Au p and (11) becomes va q, which impl that v is feasible for the primal program and u is feasible for the dual. Therefore, b Lemma , q u p v, which again contradicts (12). This contradiction shows that if both programs are feasible, then both have optimal solutions and both programs have the same value. If either program is infeasible, then certainl it cannot have an optimal solution. So suppose that the primal program is infeasible, but the dual is feasible. That is, xa q has no nonnegative solution, so b the theorem of the alternative again, there is a nonnegative satisfing A 0 and q < 0. Let z be an feasible nonnegative solution to the dual. Then z + α is feasible for an α 0, and q (z + α) = q z + αq as α. Therefore no optimal solution exists for the dual. A similar argument works if the dual is infeasible, but the primal is feasible. Proof of Theorem : B the eas half of the Saddlepoint Theorem, if ( x, ȳ) is a saddlepoint of the Lagrangean, then x is optimal for the primal and ȳ is optimal for the dual. Thus (1) = (2) & (3) and (a) = (b). B the Optimalit Criterion if x is optimal for the primal and ȳ is optimal for the dual, then ( x, ȳ) is a saddlepoint of the Lagrangean. It follows that (b) = (c) = (a) and (2) & (3) = (1). Clearl (2) & (3) = (4). The Fundamental Dualit Theorem implies (2) (3) and that (4) = (2) & (3). This finishes the proof of the Saddlepoint Theorem for Linear Programing. v ::15.57 src: LPTheor KC Border: for Ec 181, Winter 2018
5 KC Border Linear programming: Theor 28 5 Gale refers to the following result as the Equilibrium Theorem. It is also known as the Complementar Slackness Theorem. It is a simple consequence of the Optimalit Criterion (Lemma ) Complementar Slackness Theorem Suppose x and are feasible for the primal and dual respectivel. The are optimal if and onl if both (xa) j < q j = j = 0 (15) and (A) i > p i = x i = 0. (16) Proof : Suppose x and are feasible for the primal and dual respectivel. From xa q and 0, we have xa q with equalit if and onl if (15) holds. Similarl, (16) holds if and onl if xa = p x. The conclusion now follows from the Optimalit Criterion , which sas that x and are optimal if and onl if p x = q = xa. You ma suspect that it is possible to combine linear constraints with more general concave constraints that satisf Slater s Condition. This is indeed the case as Uzawa [4] has shown. (See also Moore [3].) 28.2 Other formulations Not ever linear program comes to us alread in standard inequalit form, nor is the inequalit form alwas the easiest to work with. There are other forms, some of which have names, and all of which can be translated into one another. In fact, we just translated a standard minimum inequalit form into a standard maximum inequalit form above. Each of these forms also has a dual, and the program and its dual satisf the Fundamental Dualit Theorem of LP That is, if both a linear program (in an form) and its dual are feasible, then both have optimal solutions, and the values of the two programs are the same. If one of the programs is infeasible, neither has a solution. Table summarizes these forms and their dual programs The primal is the dual of the dual Remember that the dual of a maximum linear program in standard inequalit form is a linear program of the form KC Border: for Ec 181, Winter 2018 src: LPTheor v ::15.57
6 KC Border Linear programming: Theor 28 6 q A p 0 where x and p belong to R n, q belongs to R m, and A is n m. Let us call this a minimum linear program in standard inequalit form. Now the dual program itself can be rewritten as the following maximum LP in standard inequalit form: maximize q ( A ) p 0, where A is the transpose of A. The dual of this program is: x p x A x p x 0, or maximize x p x xa p x 0, which is the primal. Thus the dual of a minimum LP in standard inequalit form is a maximum LP in standard inequalit form, and vice-versa. Moreover the dual of the dual is the primal The general form Let us start with a linear program in general maximum form, which allows for both linear inequalities and equations, and optional sign constraints on the components of x. v ::15.57 src: LPTheor KC Border: for Ec 181, Winter 2018
7 KC Border Linear programming: Theor 28 7 maximize x n p x = p j x j j=1 x A j q j j J L x A j = q j j J E x A j q j j J G x i 0 i I N x i free i I F x i 0 i I P where J L J E J G = {1,..., n}, and I N I F I P = {1,..., m}. We can translate this into standard inequalit maximum form as follows. Start b rewriting all the constraints as inequalities, x A j q j j J L x A j q j j J E x ( A j ) q j j J E x ( A j ) q j j J G. Next replace x b u v, where both 0 and v 0. This places no sign restrictions on the components of u v. To capture the requirement that x i 0 for i I N, we require u e i 0, where e i is the i th unit coordinate vector in R n. (Do ou see wh this works?) Similarl x i 0 corresponds to v e i 0. Thus our rewritten problem is maximize p (u v) u,v (u v) A j q j j J L (u v) A j q j j J E (u v) ( A j ) q j j J E (u v) ( A j ) q j j J G u e i 0 i I N v e i 0 i I P u 0 v 0. Or in matrix form maximize u,v (p, p) (u, v) KC Border: for Ec 181, Winter 2018 src: LPTheor v ::15.57
8 KC Border Linear programming: Theor 28 8 (u, v) AL A E A E A G I N 0 P (q L, q E, q E, q G, 0 N, 0 P ), A L A E A E A G 0 N I P where A L, A E, and A G are m-rowed matrices whose columns are A j for j J M, j J E, and j G respectivel, the columns of I N and I P are unit coordinate vectors e i for i N and i P respectivel, and q L, q E, and q P have components q j for j L, j E, and j G respectivel. The zeros are of the dimension the need to be. The dual of this maximum problem in standard inequalit form is thus the following: z,w z 0, w 0, and (q L, q E, q E, q G, 0 N, 0 P ) (z L, z E, w E, z G, z N, z P ) AL A E A E A G I N 0 P A L A E A E A G 0 N I P z L z E w E p, z G p z N z P where z L, z E, w E, z G, z N, z P are of the appropriate dimensions. Define R m b z j j J L j = z j w j j J E so that z j j J G and rewrite the dual as j 0 j J L j unsigned j J E j 0 j J G z 0, w 0, and A + IN 0 A 0 I P q z N z P p, p v ::15.57 src: LPTheor KC Border: for Ec 181, Winter 2018
9 KC Border Linear programming: Theor 28 9 where A is the m n matrix of columns A 1,..., A n. What this sas is A + z i e i p i N A z i e i p i P Since z 0, the dual can be written: q j 0 j J L j free j J E j 0 j J G A i p i i I N A i = p i i I F A i p i i I P Recall that the variables in the dual are the Lagrange multipliers for the primal. Thus we see that, the Lagrange multipliers associated with the equalit constraints (j J E ) are not a priori restricted in sign, while the multipliers for the inequalit constraints (i J L ) are nonnegative, and the multipliers for the inequalit constraints (i J G ) are nonpositive. Since the primal variables are the Lagrange multipliers for the dual program, the nonnegativit constraints (i I P ) on the primal correspond to inequalit constraints in the dual, the nonpositivit constraints (i I N ) on the primal correspond to inequalit constraints in the dual, and the unrestricted primal variable are associated with equalit constraints in the dual Canonical (equalit) form There is one more useful form for linear programs, the canonical or equalit form. 2 In it, all the constraints are equations, and all the variables are nonnegative. An LP is in canonical maximum form if it is written as: 2 The equalit form is what Gale [2] calls the canonical form. Dantzig uses the term canonical in a different fashion. KC Border: for Ec 181, Winter 2018 src: LPTheor v ::15.57
10 KC Border Linear programming: Theor maximize x p x xa = q x 0 To transform an inequalit form into the equalit form, introduce slack variables z R m and observe that xa q xa + z = q, z 0. It follows from the characterization of the dual to the general maximum problem that the dual program can be written as the decidedl non-equalit minimum problem q A p Note the lack of sign restrictions on. In the canonical maximization problem, we are looking to write q as linear combination of the rows of A. In discussing algorithms for solving linear programs it is useful to be able to restrict attention to solutions where q is a linear combination of a linearl independent set of rows. That is, we want to make sure that if an optimum exists, then there is an optimum x where {A i : x i > 0} is a linearl independent set, where A i denotes the i th row of A. We shall call such a x a basic solution to the linear program. To prove the existence of a basic solution we must first derive and prove the proper version of the Complementar Slackness Theorem for canonical programs Theorem (Complementar Slackness for Canonical From) Let x and be feasible for the canonical maximum problem and its dual. That is, x 0, xa = q, and A p. Then x and are optimal if and onl for i = 1,..., m, A i > p i = x i = 0. Similarl, let x and be feasible for the canonical minimum problem and its dual. Then x and are optimal if and onl for i = 1,..., m, A i < p i = x i = 0. Proof : I ll prove the result for the maximization problem. So let x 0, xa = q, and A p. v ::15.57 src: LPTheor KC Border: for Ec 181, Winter 2018
11 KC Border Linear programming: Theor ( = ) Assume that x and are optimal and that A i > p i. Since x and are optimal, we know b the Fundamental Dualit Theorem that p x = q. Since x 0 and A p, we have that On the other hand xa = q, so Combining these we get x A p x = q. m q = xa = x A = (A i )x i. i=1 xa = p x or m (A i p i )x i = 0. i=1 Since each x i 0 and each A i p i 0, the onl wa the sum can be zero is that each product term (A i p i )x i is zero. So if A i p i > 0, then x i = 0. which means if that can happens is ( = ) Assume that x i = 0 whenever A i > p i. Then since A p and x 0, we must have xa = p x. But xa = q so q = xa = p, which implies thatx and are both optimal Theorem If a canonical (equalit constraint) linear programming problem has an optimal solution, then it has a basic optimal solution. Proof : Let x and be an optimal solution to the primal and the dual. Put B = {i : x i > 0}. Then b the contrapositive of Complementar Slackness , A i = p i for all i B. B Lemma on nonnegative basic solutions there exists x satisfing xa = q such that B = {i : x i > 0} B and {A i : x i > 0} is linearl independent. Now for all i B B we also have A i = p i. That is, A i > p i implies x i = 0, so b Complementar Slackness again, x is also optimal Linear equations as LPs It is possible to recast the problem of solving linear equations and inequalities as LP problems. Consider the problem of finding a nonnegative solution to a sstem of equations. That is, find x such that xa = q x 0. KC Border: for Ec 181, Winter 2018 src: LPTheor v ::15.57
12 KC Border Linear programming: Theor Primal program Dual program General maximum form General minimum form maximize x p x q xa j q j j J L xa j = q j j J E xa j q j j J G x i 0 i I N x i free i I F x i 0 i I P j 0 j J L j free j J E j 0 j J G A i p i i I N A i = p i i I F A i p i i I P Standard maximum form Standard minimum form maximize x p x q xa q x 0 0 A p Canonical maximum form maximize x p x q xa = q x 0 A p free Canonical minimum form x p x maximize q Ax = q x 0 A p free Table v :: Selected forms src: LPTheor of linear programs KCand Border: theirfor duals. Ec 181, Winter 2018 The characterization of the general form implies the other results. The primal constraints ma also be expressed in terms of Ax rather than xa, in which case
13 KC Border Linear programming: Theor Consider the linear program in equalit minimum form: x,z 1 z xa + z = q [x, z] 0 Here 1 is the vector whose components are all 1. Without loss of generalit we ma assume q 0, for if q j < 0 we ma multipl A j and q j b 1 without affecting the solution set. Then note that this program is feasible, since x = 0, z = q is a nonnegative feasible solution. Since we require z 0, we have 1 z 0 and 1 z = 0 if and onl if z = 0, in which case xa = q. Thus, if this linear program has value 0 if and onl xa = q, x 0 has a solution, and an optimal (x, z) provides a nonnegative solution to the equation. At this point ou might be inclined to sa so what? In the next lecture, I describe the simplex algorithm, which is a special version of Gauss Jordan elimination, that is a reasonabl efficient and easil programmable method for solving linear programs. In other words, it also finds nonnegative solutions to linear equations when the exist. References [1] G. B. Dantzig Linear programming and extensions. Princeton: Princeton Universit Press. [2] D. Gale Theor of linear economic models. Chicago: Universit of Chicago Press. Reprint of the 1960 edition published b McGraw-Hill. [3] J. C. Moore Some extensions of the Kuhn Tucker results in concave programming. In J. P. Quirk and A. M. Zarle, eds., Papers in Quantitative Economics, 1, pages Lawrence, Kansas: Universit of Kansas Press. [4] H. Uzawa The Kuhn Tucker conditions in concave programming. In K. J. Arrow, L. Hurwicz, and H. Uzawa, eds., Studies in Linear and Nonlinear Programming, number 2 in Stanford Mathematical Studies in the Social Sciences, chapter 3, pages Stanford, California: Stanford Universit Press. KC Border: for Ec 181, Winter 2018 src: LPTheor v ::15.57
14
ECE 307- Techniques for Engineering Decisions
ECE 307- Techniques for Engineering Decisions Lecture 4. Dualit Concepts in Linear Programming George Gross Department of Electrical and Computer Engineering Universit of Illinois at Urbana-Champaign DUALITY
More informationCPS 616 ITERATIVE IMPROVEMENTS 10-1
CPS 66 ITERATIVE IMPROVEMENTS 0 - APPROACH Algorithm design technique for solving optimization problems Start with a feasible solution Repeat the following step until no improvement can be found: change
More informationConstrained maxima and Lagrangean saddlepoints
Division of the Humanities and Social Sciences Ec 181 KC Border Convex Analysis and Economic Theory Winter 2018 Topic 10: Constrained maxima and Lagrangean saddlepoints 10.1 An alternative As an application
More informationLinear and Combinatorial Optimization
Linear and Combinatorial Optimization The dual of an LP-problem. Connections between primal and dual. Duality theorems and complementary slack. Philipp Birken (Ctr. for the Math. Sc.) Lecture 3: Duality
More informationLINEAR PROGRAMMING II
LINEAR PROGRAMMING II LP duality strong duality theorem bonus proof of LP duality applications Lecture slides by Kevin Wayne Last updated on 7/25/17 11:09 AM LINEAR PROGRAMMING II LP duality Strong duality
More informationA. Derivation of regularized ERM duality
A. Derivation of regularized ERM dualit For completeness, in this section we derive the dual 5 to the problem of computing proximal operator for the ERM objective 3. We can rewrite the primal problem as
More informationIntroduction to Mathematical Programming IE406. Lecture 10. Dr. Ted Ralphs
Introduction to Mathematical Programming IE406 Lecture 10 Dr. Ted Ralphs IE406 Lecture 10 1 Reading for This Lecture Bertsimas 4.1-4.3 IE406 Lecture 10 2 Duality Theory: Motivation Consider the following
More informationOptimality, Duality, Complementarity for Constrained Optimization
Optimality, Duality, Complementarity for Constrained Optimization Stephen Wright University of Wisconsin-Madison May 2014 Wright (UW-Madison) Optimality, Duality, Complementarity May 2014 1 / 41 Linear
More informationLecture 10: Linear programming duality and sensitivity 0-0
Lecture 10: Linear programming duality and sensitivity 0-0 The canonical primal dual pair 1 A R m n, b R m, and c R n maximize z = c T x (1) subject to Ax b, x 0 n and minimize w = b T y (2) subject to
More informationLecture 10: Linear programming. duality. and. The dual of the LP in standard form. maximize w = b T y (D) subject to A T y c, minimize z = c T x (P)
Lecture 10: Linear programming duality Michael Patriksson 19 February 2004 0-0 The dual of the LP in standard form minimize z = c T x (P) subject to Ax = b, x 0 n, and maximize w = b T y (D) subject to
More informationConvex Analysis and Economic Theory Winter 2018
Division of the Humanities and Social Sciences Ec 181 KC Border Convex Analysis and Economic Theory Winter 2018 Topic 0: Vector spaces 0.1 Basic notation Here are some of the fundamental sets and spaces
More informationAn introductory example
CS1 Lecture 9 An introductory example Suppose that a company that produces three products wishes to decide the level of production of each so as to maximize profits. Let x 1 be the amount of Product 1
More informationI.3. LMI DUALITY. Didier HENRION EECI Graduate School on Control Supélec - Spring 2010
I.3. LMI DUALITY Didier HENRION henrion@laas.fr EECI Graduate School on Control Supélec - Spring 2010 Primal and dual For primal problem p = inf x g 0 (x) s.t. g i (x) 0 define Lagrangian L(x, z) = g 0
More information4TE3/6TE3. Algorithms for. Continuous Optimization
4TE3/6TE3 Algorithms for Continuous Optimization (Duality in Nonlinear Optimization ) Tamás TERLAKY Computing and Software McMaster University Hamilton, January 2004 terlaky@mcmaster.ca Tel: 27780 Optimality
More informationIn applications, we encounter many constrained optimization problems. Examples Basis pursuit: exact sparse recovery problem
1 Conve Analsis Main references: Vandenberghe UCLA): EECS236C - Optimiation methods for large scale sstems, http://www.seas.ucla.edu/ vandenbe/ee236c.html Parikh and Bod, Proimal algorithms, slides and
More informationSummary Notes on Maximization
Division of the Humanities and Social Sciences Summary Notes on Maximization KC Border Fall 2005 1 Classical Lagrange Multiplier Theorem 1 Definition A point x is a constrained local maximizer of f subject
More informationWeek 3 September 5-7.
MA322 Weekl topics and quiz preparations Week 3 September 5-7. Topics These are alread partl covered in lectures. We collect the details for convenience.. Solutions of homogeneous equations AX =. 2. Using
More informationConvex Optimization & Lagrange Duality
Convex Optimization & Lagrange Duality Chee Wei Tan CS 8292 : Advanced Topics in Convex Optimization and its Applications Fall 2010 Outline Convex optimization Optimality condition Lagrange duality KKT
More informationHow to Take the Dual of a Linear Program
How to Take the Dual of a Linear Program Sébastien Lahaie January 12, 2015 This is a revised version of notes I wrote several years ago on taking the dual of a linear program (LP), with some bug and typo
More informationCO 250 Final Exam Guide
Spring 2017 CO 250 Final Exam Guide TABLE OF CONTENTS richardwu.ca CO 250 Final Exam Guide Introduction to Optimization Kanstantsin Pashkovich Spring 2017 University of Waterloo Last Revision: March 4,
More informationDuality Theory, Optimality Conditions
5.1 Duality Theory, Optimality Conditions Katta G. Murty, IOE 510, LP, U. Of Michigan, Ann Arbor We only consider single objective LPs here. Concept of duality not defined for multiobjective LPs. Every
More informationIntroduction to linear programming using LEGO.
Introduction to linear programming using LEGO. 1 The manufacturing problem. A manufacturer produces two pieces of furniture, tables and chairs. The production of the furniture requires the use of two different
More informationNonlinear Optimization
Nonlinear Optimization Etienne de Klerk (UvT)/Kees Roos e-mail: C.Roos@ewi.tudelft.nl URL: http://www.isa.ewi.tudelft.nl/ roos Course WI3031 (Week 4) February-March, A.D. 2005 Optimization Group 1 Outline
More information"SYMMETRIC" PRIMAL-DUAL PAIR
"SYMMETRIC" PRIMAL-DUAL PAIR PRIMAL Minimize cx DUAL Maximize y T b st Ax b st A T y c T x y Here c 1 n, x n 1, b m 1, A m n, y m 1, WITH THE PRIMAL IN STANDARD FORM... Minimize cx Maximize y T b st Ax
More informationAnother max flow application: baseball
CS124 Lecture 16 Spring 2018 Another max flow application: baseball Suppose there are n baseball teams, and team 1 is our favorite. It is the middle of baseball season, and some games have been played
More informationConic Linear Optimization and its Dual. yyye
Conic Linear Optimization and Appl. MS&E314 Lecture Note #04 1 Conic Linear Optimization and its Dual Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A.
More informationCh 3 Alg 2 Note Sheet.doc 3.1 Graphing Systems of Equations
Ch 3 Alg Note Sheet.doc 3.1 Graphing Sstems of Equations Sstems of Linear Equations A sstem of equations is a set of two or more equations that use the same variables. If the graph of each equation =.4
More informationIntroductory notes on stochastic rationality. 1 Stochastic choice and stochastic rationality
Division of the Humanities and Social Sciences Introductory notes on stochastic rationality KC Border Fall 2007 1 Stochastic choice and stochastic rationality In the standard theory of rational choice
More informationLecture 5. Theorems of Alternatives and Self-Dual Embedding
IE 8534 1 Lecture 5. Theorems of Alternatives and Self-Dual Embedding IE 8534 2 A system of linear equations may not have a solution. It is well known that either Ax = c has a solution, or A T y = 0, c
More informationLinear Programming. Larry Blume Cornell University, IHS Vienna and SFI. Summer 2016
Linear Programming Larry Blume Cornell University, IHS Vienna and SFI Summer 2016 These notes derive basic results in finite-dimensional linear programming using tools of convex analysis. Most sources
More informationChapter 11 Optimization with Equality Constraints
Ch. - Optimization with Equalit Constraints Chapter Optimization with Equalit Constraints Albert William Tucker 95-995 arold William Kuhn 95 oseph-ouis Giuseppe odovico comte de arane 76-. General roblem
More informationUniversity of California, Davis Department of Agricultural and Resource Economics ARE 252 Lecture Notes 2 Quirino Paris
University of California, Davis Department of Agricultural and Resource Economics ARE 5 Lecture Notes Quirino Paris Karush-Kuhn-Tucker conditions................................................. page Specification
More informationIE 5531: Engineering Optimization I
IE 5531: Engineering Optimization I Lecture 7: Duality and applications Prof. John Gunnar Carlsson September 29, 2010 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 29, 2010 1
More informationMidterm Review. Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A.
Midterm Review Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye (LY, Chapter 1-4, Appendices) 1 Separating hyperplane
More informationORIE 6300 Mathematical Programming I August 25, Lecture 2
ORIE 6300 Mathematical Programming I August 25, 2016 Lecturer: Damek Davis Lecture 2 Scribe: Johan Bjorck Last time, we considered the dual of linear programs in our basic form: max(c T x : Ax b). We also
More informationThe Simplex Algorithm
8.433 Combinatorial Optimization The Simplex Algorithm October 6, 8 Lecturer: Santosh Vempala We proved the following: Lemma (Farkas). Let A R m n, b R m. Exactly one of the following conditions is true:.
More informationOPTIMISATION 2007/8 EXAM PREPARATION GUIDELINES
General: OPTIMISATION 2007/8 EXAM PREPARATION GUIDELINES This points out some important directions for your revision. The exam is fully based on what was taught in class: lecture notes, handouts and homework.
More informationMath 5593 Linear Programming Week 1
University of Colorado Denver, Fall 2013, Prof. Engau 1 Problem-Solving in Operations Research 2 Brief History of Linear Programming 3 Review of Basic Linear Algebra Linear Programming - The Story About
More informationSpring 2017 CO 250 Course Notes TABLE OF CONTENTS. richardwu.ca. CO 250 Course Notes. Introduction to Optimization
Spring 2017 CO 250 Course Notes TABLE OF CONTENTS richardwu.ca CO 250 Course Notes Introduction to Optimization Kanstantsin Pashkovich Spring 2017 University of Waterloo Last Revision: March 4, 2018 Table
More informationEE364a Review Session 5
EE364a Review Session 5 EE364a Review announcements: homeworks 1 and 2 graded homework 4 solutions (check solution to additional problem 1) scpd phone-in office hours: tuesdays 6-7pm (650-723-1156) 1 Complementary
More informationChap 2. Optimality conditions
Chap 2. Optimality conditions Version: 29-09-2012 2.1 Optimality conditions in unconstrained optimization Recall the definitions of global, local minimizer. Geometry of minimization Consider for f C 1
More informationLP Duality: outline. Duality theory for Linear Programming. alternatives. optimization I Idea: polyhedra
LP Duality: outline I Motivation and definition of a dual LP I Weak duality I Separating hyperplane theorem and theorems of the alternatives I Strong duality and complementary slackness I Using duality
More informationNotes on Convexity. Roy Radner Stern School, NYU. September 11, 2006
Notes on Convexit Ro Radner Stern School, NYU September, 2006 Abstract These notes are intended to complement the material in an intermediate microeconomic theor course. In particular, the provide a rigorous
More informationLecture 1. 1 Conic programming. MA 796S: Convex Optimization and Interior Point Methods October 8, Consider the conic program. min.
MA 796S: Convex Optimization and Interior Point Methods October 8, 2007 Lecture 1 Lecturer: Kartik Sivaramakrishnan Scribe: Kartik Sivaramakrishnan 1 Conic programming Consider the conic program min s.t.
More informationUnderstanding the Simplex algorithm. Standard Optimization Problems.
Understanding the Simplex algorithm. Ma 162 Spring 2011 Ma 162 Spring 2011 February 28, 2011 Standard Optimization Problems. A standard maximization problem can be conveniently described in matrix form
More informationIntroduction to Mathematical Programming
Introduction to Mathematical Programming Ming Zhong Lecture 22 October 22, 2018 Ming Zhong (JHU) AMS Fall 2018 1 / 16 Table of Contents 1 The Simplex Method, Part II Ming Zhong (JHU) AMS Fall 2018 2 /
More informationInequalities and alternatives
Division of the Humanities and Social Sciences Ec 181 KC Border Convex Analysis and Economic Theory Winter 2018 Topic 25: Inequalities and alternatives This chapter discusses both the solution of linear
More informationx 1 + x 2 2 x 1 x 2 1 x 2 2 min 3x 1 + 2x 2
Lecture 1 LPs: Algebraic View 1.1 Introduction to Linear Programming Linear programs began to get a lot of attention in 1940 s, when people were interested in minimizing costs of various systems while
More information4 Inverse function theorem
Tel Aviv Universit, 2013/14 Analsis-III,IV 53 4 Inverse function theorem 4a What is the problem................ 53 4b Simple observations before the theorem..... 54 4c The theorem.....................
More informationFarkas Lemma, Dual Simplex and Sensitivity Analysis
Summer 2011 Optimization I Lecture 10 Farkas Lemma, Dual Simplex and Sensitivity Analysis 1 Farkas Lemma Theorem 1. Let A R m n, b R m. Then exactly one of the following two alternatives is true: (i) x
More informationLecture 5. x 1,x 2,x 3 0 (1)
Computational Intractability Revised 2011/6/6 Lecture 5 Professor: David Avis Scribe:Ma Jiangbo, Atsuki Nagao 1 Duality The purpose of this lecture is to introduce duality, which is an important concept
More informationDuality Theory of Constrained Optimization
Duality Theory of Constrained Optimization Robert M. Freund April, 2014 c 2014 Massachusetts Institute of Technology. All rights reserved. 1 2 1 The Practical Importance of Duality Duality is pervasive
More information8.7 Systems of Non-Linear Equations and Inequalities
8.7 Sstems of Non-Linear Equations and Inequalities 67 8.7 Sstems of Non-Linear Equations and Inequalities In this section, we stud sstems of non-linear equations and inequalities. Unlike the sstems of
More information4. Algebra and Duality
4-1 Algebra and Duality P. Parrilo and S. Lall, CDC 2003 2003.12.07.01 4. Algebra and Duality Example: non-convex polynomial optimization Weak duality and duality gap The dual is not intrinsic The cone
More informationOPTIMISATION /09 EXAM PREPARATION GUIDELINES
General: OPTIMISATION 2 2008/09 EXAM PREPARATION GUIDELINES This points out some important directions for your revision. The exam is fully based on what was taught in class: lecture notes, handouts and
More informationWeek 3 Linear programming duality
Week 3 Linear programming duality This week we cover the fascinating topic of linear programming duality. We will learn that every minimization program has associated a maximization program that has the
More informationChap6 Duality Theory and Sensitivity Analysis
Chap6 Duality Theory and Sensitivity Analysis The rationale of duality theory Max 4x 1 + x 2 + 5x 3 + 3x 4 S.T. x 1 x 2 x 3 + 3x 4 1 5x 1 + x 2 + 3x 3 + 8x 4 55 x 1 + 2x 2 + 3x 3 5x 4 3 x 1 ~x 4 0 If we
More informationGeneralization to inequality constrained problem. Maximize
Lecture 11. 26 September 2006 Review of Lecture #10: Second order optimality conditions necessary condition, sufficient condition. If the necessary condition is violated the point cannot be a local minimum
More informationCourse Notes for EE227C (Spring 2018): Convex Optimization and Approximation
Course Notes for EE227C (Spring 2018): Convex Optimization and Approximation Instructor: Moritz Hardt Email: hardt+ee227c@berkeley.edu Graduate Instructor: Max Simchowitz Email: msimchow+ee227c@berkeley.edu
More informationTRINITY COLLEGE DUBLIN THE UNIVERSITY OF DUBLIN. School of Mathematics
JS and SS Mathematics JS and SS TSM Mathematics TRINITY COLLEGE DUBLIN THE UNIVERSITY OF DUBLIN School of Mathematics MA3484 Methods of Mathematical Economics Trinity Term 2015 Saturday GOLDHALL 09.30
More informationNote 3: LP Duality. If the primal problem (P) in the canonical form is min Z = n (1) then the dual problem (D) in the canonical form is max W = m (2)
Note 3: LP Duality If the primal problem (P) in the canonical form is min Z = n j=1 c j x j s.t. nj=1 a ij x j b i i = 1, 2,..., m (1) x j 0 j = 1, 2,..., n, then the dual problem (D) in the canonical
More informationCS261: A Second Course in Algorithms Lecture #9: Linear Programming Duality (Part 2)
CS261: A Second Course in Algorithms Lecture #9: Linear Programming Duality (Part 2) Tim Roughgarden February 2, 2016 1 Recap This is our third lecture on linear programming, and the second on linear programming
More informationChapter 2: Linear Programming Basics. (Bertsimas & Tsitsiklis, Chapter 1)
Chapter 2: Linear Programming Basics (Bertsimas & Tsitsiklis, Chapter 1) 33 Example of a Linear Program Remarks. minimize 2x 1 x 2 + 4x 3 subject to x 1 + x 2 + x 4 2 3x 2 x 3 = 5 x 3 + x 4 3 x 1 0 x 3
More informationLecture 18: Optimization Programming
Fall, 2016 Outline Unconstrained Optimization 1 Unconstrained Optimization 2 Equality-constrained Optimization Inequality-constrained Optimization Mixture-constrained Optimization 3 Quadratic Programming
More informationConvex Optimization M2
Convex Optimization M2 Lecture 3 A. d Aspremont. Convex Optimization M2. 1/49 Duality A. d Aspremont. Convex Optimization M2. 2/49 DMs DM par email: dm.daspremont@gmail.com A. d Aspremont. Convex Optimization
More informationExtreme Abridgment of Boyd and Vandenberghe s Convex Optimization
Extreme Abridgment of Boyd and Vandenberghe s Convex Optimization Compiled by David Rosenberg Abstract Boyd and Vandenberghe s Convex Optimization book is very well-written and a pleasure to read. The
More informationAlgorithms and Theory of Computation. Lecture 13: Linear Programming (2)
Algorithms and Theory of Computation Lecture 13: Linear Programming (2) Xiaohui Bei MAS 714 September 25, 2018 Nanyang Technological University MAS 714 September 25, 2018 1 / 15 LP Duality Primal problem
More informationWhen are Sums Closed?
Division of the Humanities and Social Sciences Ec 181 KC Border Convex Analysis and Economic Theory Fall 2018 Winter 2019 Topic 20: When are Sums Closed? 20.1 Is a sum of closed sets closed? Example 0.2.2
More informationDiscrete Optimization
Prof. Friedrich Eisenbrand Martin Niemeier Due Date: April 15, 2010 Discussions: March 25, April 01 Discrete Optimization Spring 2010 s 3 You can hand in written solutions for up to two of the exercises
More informationLectures 9 and 10: Constrained optimization problems and their optimality conditions
Lectures 9 and 10: Constrained optimization problems and their optimality conditions Coralia Cartis, Mathematical Institute, University of Oxford C6.2/B2: Continuous Optimization Lectures 9 and 10: Constrained
More informationDEPARTMENT OF STATISTICS AND OPERATIONS RESEARCH OPERATIONS RESEARCH DETERMINISTIC QUALIFYING EXAMINATION. Part I: Short Questions
DEPARTMENT OF STATISTICS AND OPERATIONS RESEARCH OPERATIONS RESEARCH DETERMINISTIC QUALIFYING EXAMINATION Part I: Short Questions August 12, 2008 9:00 am - 12 pm General Instructions This examination is
More informationPart 1. The Review of Linear Programming
In the name of God Part 1. The Review of Linear Programming 1.5. Spring 2010 Instructor: Dr. Masoud Yaghini Outline Introduction Formulation of the Dual Problem Primal-Dual Relationship Economic Interpretation
More informationSensitivity Analysis and Duality
Sensitivity Analysis and Duality Part II Duality Based on Chapter 6 Introduction to Mathematical Programming: Operations Research, Volume 1 4th edition, by Wayne L. Winston and Munirpallam Venkataramanan
More informationReview Solutions, Exam 2, Operations Research
Review Solutions, Exam 2, Operations Research 1. Prove the weak duality theorem: For any x feasible for the primal and y feasible for the dual, then... HINT: Consider the quantity y T Ax. SOLUTION: To
More information2.098/6.255/ Optimization Methods Practice True/False Questions
2.098/6.255/15.093 Optimization Methods Practice True/False Questions December 11, 2009 Part I For each one of the statements below, state whether it is true or false. Include a 1-3 line supporting sentence
More informationSharpening the Karush-John optimality conditions
Sharpening the Karush-John optimality conditions Arnold Neumaier and Hermann Schichl Institut für Mathematik, Universität Wien Strudlhofgasse 4, A-1090 Wien, Austria email: Arnold.Neumaier@univie.ac.at,
More informationLectures 6, 7 and part of 8
Lectures 6, 7 and part of 8 Uriel Feige April 26, May 3, May 10, 2015 1 Linear programming duality 1.1 The diet problem revisited Recall the diet problem from Lecture 1. There are n foods, m nutrients,
More informationLinear Programming, Lecture 4
Linear Programming, Lecture 4 Corbett Redden October 3, 2016 Simplex Form Conventions Examples Simplex Method To run the simplex method, we start from a Linear Program (LP) in the following standard simplex
More informationWritten Examination
Division of Scientific Computing Department of Information Technology Uppsala University Optimization Written Examination 202-2-20 Time: 4:00-9:00 Allowed Tools: Pocket Calculator, one A4 paper with notes
More informationLecture 1.1: The Minimax Theorem
Algorithmic Game Theor, Compleit and Learning Jul 17, 2017 Lecture 1.1: The Mini Theorem Lecturer: Constantinos Daskalakis Scribe: Constantinos Daskalakis In these notes, we show that a Nash equilibrium
More informationCHAPTER 2: CONVEX SETS AND CONCAVE FUNCTIONS. W. Erwin Diewert January 31, 2008.
1 ECONOMICS 594: LECTURE NOTES CHAPTER 2: CONVEX SETS AND CONCAVE FUNCTIONS W. Erwin Diewert January 31, 2008. 1. Introduction Many economic problems have the following structure: (i) a linear function
More informationChapter 1. Preliminaries
Introduction This dissertation is a reading of chapter 4 in part I of the book : Integer and Combinatorial Optimization by George L. Nemhauser & Laurence A. Wolsey. The chapter elaborates links between
More informationA Polyhedral Cone Counterexample
Division of the Humanities and Social Sciences A Polyhedral Cone Counterexample KCB 9/20/2003 Revised 12/12/2003 Abstract This is an example of a pointed generating convex cone in R 4 with 5 extreme rays,
More informationAlgorithmic Game Theory and Applications. Lecture 7: The LP Duality Theorem
Algorithmic Game Theory and Applications Lecture 7: The LP Duality Theorem Kousha Etessami recall LP s in Primal Form 1 Maximize c 1 x 1 + c 2 x 2 +... + c n x n a 1,1 x 1 + a 1,2 x 2 +... + a 1,n x n
More informationAffine transformations
Reading Optional reading: Affine transformations Brian Curless CSE 557 Autumn 207 Angel and Shreiner: 3., 3.7-3. Marschner and Shirle: 2.3, 2.4.-2.4.4, 6..-6..4, 6.2., 6.3 Further reading: Angel, the rest
More informationMAT016: Optimization
MAT016: Optimization M.El Ghami e-mail: melghami@ii.uib.no URL: http://www.ii.uib.no/ melghami/ March 29, 2011 Outline for today The Simplex method in matrix notation Managing a production facility The
More informationMotivation. Lecture 2 Topics from Optimization and Duality. network utility maximization (NUM) problem:
CDS270 Maryam Fazel Lecture 2 Topics from Optimization and Duality Motivation network utility maximization (NUM) problem: consider a network with S sources (users), each sending one flow at rate x s, through
More information3.7 InveRSe FUnCTIOnS
CHAPTER functions learning ObjeCTIveS In this section, ou will: Verif inverse functions. Determine the domain and range of an inverse function, and restrict the domain of a function to make it one-to-one.
More information4. Duality and Sensitivity
4. Duality and Sensitivity For every instance of an LP, there is an associated LP known as the dual problem. The original problem is known as the primal problem. There are two de nitions of the dual pair
More information14. Duality. ˆ Upper and lower bounds. ˆ General duality. ˆ Constraint qualifications. ˆ Counterexample. ˆ Complementary slackness.
CS/ECE/ISyE 524 Introduction to Optimization Spring 2016 17 14. Duality ˆ Upper and lower bounds ˆ General duality ˆ Constraint qualifications ˆ Counterexample ˆ Complementary slackness ˆ Examples ˆ Sensitivity
More informationON THE PARAMETRIC LCP: A HISTORICAL PERSPECTIVE
ON THE PARAMETRIC LCP: A HISTORICAL PERSPECTIVE Richard W. Cottle Department of Management Science and Engineering Stanford University ICCP 2014 Humboldt University Berlin August, 2014 1 / 55 The year
More informationThe Karush-Kuhn-Tucker (KKT) conditions
The Karush-Kuhn-Tucker (KKT) conditions In this section, we will give a set of sufficient (and at most times necessary) conditions for a x to be the solution of a given convex optimization problem. These
More informationTheory and Internet Protocols
Game Lecture 2: Linear Programming and Zero Sum Nash Equilibrium Xiaotie Deng AIMS Lab Department of Computer Science Shanghai Jiaotong University September 26, 2016 1 2 3 4 Standard Form (P) Outline
More informationLinear Programming: Simplex
Linear Programming: Simplex Stephen J. Wright 1 2 Computer Sciences Department, University of Wisconsin-Madison. IMA, August 2016 Stephen Wright (UW-Madison) Linear Programming: Simplex IMA, August 2016
More information4.6 Linear Programming duality
4.6 Linear Programming duality To any minimization (maximization) LP we can associate a closely related maximization (minimization) LP Different spaces and objective functions but in general same optimal
More informationApplications of Linear Programming
Applications of Linear Programming lecturer: András London University of Szeged Institute of Informatics Department of Computational Optimization Lecture 9 Non-linear programming In case of LP, the goal
More informationBBM402-Lecture 20: LP Duality
BBM402-Lecture 20: LP Duality Lecturer: Lale Özkahya Resources for the presentation: https://courses.engr.illinois.edu/cs473/fa2016/lectures.html An easy LP? which is compact form for max cx subject to
More information4. Convex optimization problems (part 1: general)
EE/AA 578, Univ of Washington, Fall 2016 4. Convex optimization problems (part 1: general) optimization problem in standard form convex optimization problems quasiconvex optimization 4 1 Optimization problem
More information5. Duality. Lagrangian
5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationSymmetric and Asymmetric Duality
journal of mathematical analysis and applications 220, 125 131 (1998) article no. AY975824 Symmetric and Asymmetric Duality Massimo Pappalardo Department of Mathematics, Via Buonarroti 2, 56127, Pisa,
More information