IE 5531: Engineering Optimization I
|
|
- Dorcas McKenzie
- 5 years ago
- Views:
Transcription
1 IE 5531: Engineering Optimization I Lecture 3: Linear Programming, Continued Prof. John Gunnar Carlsson September 15, 2010 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
2 Pop quiz Write the region above in the form Ax b. For points A, B, and C, give a vector c such that c T x is minimized at that point (or explain why none exists) Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
3 Administrivia Lecture slides 1, 2, 3 posted PS 1 posted this evening Xi Chen's oce hours: Tuesdays 10:00-12:00, ME 1124, Table B Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
4 Today: Linear Programming, continued Linearization Mathematical preliminaries Simplex method Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
5 Recap A linear program (LP) is a mathematical optimization problem in which the objective function and all constraint functions are linear: minimize 2x 1 x 2 +4x 3 s.t. x 1 + x 2 + x 4 2 3x 2 x 3 = 5 x 3 + x 4 3 x 1 0 x 3 0 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
6 2-dimensional LPs If x R 2, it is easy to solve a linear program Consider the problem minimize x 1 x 2 s.t. x 1 + 2x 2 3 2x 1 + x 2 3 x 1, x 2 0 How do we solve this? Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
7 The graphical method Draw half-spaces corresponding to the constraints: Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
8 The graphical method Draw half-spaces corresponding to the constraints: Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
9 The graphical method Draw half-spaces corresponding to the constraints: Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
10 The graphical method Draw half-spaces corresponding to the constraints: Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
11 The graphical method Draw level sets of the objective function (they're lines orthogonal to c) Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
12 Active constraints We say a constraint a T x b is active at a point x if we have a T x = b In the previous example we had two active constraints: x 1 + 2x 2 = 3 and 2x 1 + x 2 = 3, while x 1, x 2 > 0 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
13 Facts about LP All LP problems fall into one of three classes: Problem is infeasible: the feasible region is empty Problem is unbounded: the feasible region is unbounded in the objective function direction Problem is feasible and bounded: There exists an optimal solution x There may be a unique optimal solution or multiple optimal solutions All optimal solutions are on a face of the feasible region There is always at least one corner optimizer if the face has a corner If a corner point is not worse than its neighboring corners, then it is optimal Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
14 Linearizing a problem LP can also be used to model certain non-linear problems A convex function is a function f ( ) : R n R satisfying f (λx + (1 λ) y) λf (x) + (1 λ) f (y) for all x, y R n and λ [0, 1] (bowl-shaped) A concave function is a function f ( ) : R n R satisfying f (λx + (1 λ) y) λf (x) + (1 λ) f (y) for all x, y R n and λ [0, 1] (hill-shaped) We claim that any piecewise linear convex function can be minimized by solving an LP Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
15 Linearizing a problem Consider the function f ( ) dened by { f (x) = max i=1,...,m c T i x + d i } It is easy to prove that this function is convex We can solve the problem minimize f (x) s.t. Ax b by solving an LP Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
16 Linearizing a problem The LP is minimize z s.t. z c T i Ax b x + d i i {1,..., m} Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
17 Absolute values Problems involving absolute values can be handled as well; consider minimize n i=1 c i x i s.t. Ax b Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
18 Absolute values The LP is minimize n i=1 c i z i s.t. z i x i i {1,..., n} z i x i i {1,..., n} Ax b Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
19 Data tting Consider the unconstrained problem of minimizing the largest residual minimize max b i a T i x i i where a i and b i are given, for i {1,..., m} Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
20 Data tting Consider the unconstrained problem of minimizing the largest residual minimize max b i a T i x i i where a i and b i are given, for i {1,..., m} The LP is minimize z s.t. z b i a T i x i ) z (b i a Ti x i Note that we can impose additional linear constraints on x, say C x d We could even impose something like n i=1 x i q! Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
21 Polyhedra Denition A polyhedron is a set that can be described in the form {x R n : Ax b} where A is an m n matrix and b R n. By the equivalence of linear programs, we know that a set of the form is also a polyhedron {x R n : Ax = b, x 0} Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
22 Boundedness Denition A set S R n is bounded if there exists a constant K such that S is contained in a ball of radius K. Note: a linear program can be bounded, but have an unbounded feasible set! However, if a linear program has a bounded feasible set, it must be bounded Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
23 Hyperplanes and half spaces Let a R n be nonzero and let b be a scalar. Denition The set { x : a T x = b} is called a hyperplane. Denition The set { x : a T x b} is called a half-space. Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
24 Convex sets Denition A set S R n is convex if, for any x, y S and any λ [0, 1], we have λx + (1 λ) y S. Intuitively, this means that the line segment between two points in the set must also lie in the set Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
25 Facts about convex sets The intersection of convex sets is convex Every polyhedron is convex The sub-level set of a convex function is convex (converse?) Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
26 Linear independence We say a set of vectors x 1,... x k is linearly dependent if there exist real numbers a 1,..., a k, not all of which are zero, such that a 1 x a k x k = 0 If no such real numbers exist, we say that x 1,..., x k is linearly independent If x 1,..., xn R n are linearly independent, then the matrix (x 1,..., xn) is invertible Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
27 Real functions Weierstrass theorem: a continuous function f (x) dened on a compact (closed and bounded) region S R n has a minimizer in S. Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
28 Gradient The gradient of a function f (x) : R n R, is the vector f dened by f / x 1 f =. f / x n The gradient vector always points in the direction that the function is increasing the fastest The gradient of a linear function c T x is c Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
29 Extreme points Denition Let P be a polyhedron. A point x P is said to be an extreme point of P if we cannot nd two vectors y, z P, both dierent from x, and a scalar λ [0, 1], such that x = λy + (1 λ) z In other words, x does not lie on the line segment between two other points in P Denition Let P be a polyhedron. A point x P is said to be a vertex of P if there exists some c such that c T x < c T y for all y P not equal to x. The vector c is said to dene a supporting hyperplane to P at x. Vertices and extreme points are the same thing! Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
30 Algorithmic interpretation of extreme points We gave two geometric interpretations of vertices/extreme points However, this does not suggest how a computer might nd an extreme point How can a computer recognize a vertex? How can we make a computer tell that two corners are neighboring? How can we make a computer terminate and declare optimality? How can we recognize vertices/extreme points directly from the polyhedron {x : Ax = b, x 0}? Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
31 Basic feasible solution Consider a polyhedron dened by {x : Ax = b, x 0} where A is an m n matrix and b R n. What describes the extreme points? Select m linearly independent columns, denoted by the indices B, from A, and solve A B x B = b Then, set all other variables x N to 0 If all entries x B 0, then x is called a basic feasible solution (BFS) A basic feasible solution is the same thing as a corner or extreme point this is an algebraic description, rather than a geometric description Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
32 Example Consider the polyhedron {x : Ax = b, x 0}, where ( ) ( ) A = ; b = If we take B = {1, 2}, then we solve ( ) x B = ( and nd that x B = (1; 2). Thus the BFS is x = (1; 2; 0; 0) However, if we take B = {2, 3}, then we solve ( ) ( ) x B = 16 and nd that x B = (3.42, 0.5), which is not a BFS ) Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
33 Example We can enumerate all of the vertices of the polyhedron {x : Ax = b, x 0}, where A = by choosing all subsets B ( ) ( 17 ; b = 16 ) B {1, 2} {1, 3} {1, 4} {2, 3} {2, 4} {3, 4} xb (1; 2) (2.41; 0.70) (2.8; 0.6) (3.42; 0.5) (3.11; 0.33) (5.09; 3.73) BFS? Y Y Y N N N Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
34 The simplex method One way to solve a linear program is clearly to write out all of the BFS's, although that would clearly be slow A better strategy is to start at a BFS, and move to a better neighboring BFS if one is available If no neighboring BFS exists, we're done! How to identify a neighboring BFS? Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
35 Neighboring basic solutions Two basic solutions are neighboring or adjacent if they dier by exactly one basic (or nonbasic) variable A basic feasible solution is optimal if no better neighboring feasible solution exists How to check if this is true? Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
36 Optimality test Consider the BFS (0, 0, 1, 1, 1.5); is this optimal for the problem minimize x 1 2x 2 s.t. x 1 +x 3 = 1 x 2 + x 4 = 1 x 1 + x 2 +x 5 = 1.5 x 1,x 2, x 3,x 4, x 5 0 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
37 Optimality test Consider the BFS (0, 0, 1, 1, 1.5); is this optimal for the problem minimize x 1 2x 2 s.t. x 1 +x 3 = 1 x 2 + x 4 = 1 x 1 + x 2 +x 5 = 1.5 x 1,x 2, x 3,x 4, x 5 0 No, it isn't; the basic set is {3, 4, 5}; if we increase x 1 while decreasing x 3 and x 5, the objective function decreases Thus, a better basic set has 1 in it, and we should remove 3 or 5 (don't know which one yet) Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
38 Optimality test Consider the BFS (0, 0, 1, 1, 1.5); is this optimal for the problem minimize x 1 + 2x 2 s.t. x 1 +x 3 = 1 x 2 + x 4 = 1 x 1 + x 2 +x 5 = 1.5 x 1,x 2, x 3,x 4, x 5 0 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
39 Optimality test Consider the BFS (0, 0, 1, 1, 1.5); is this optimal for the problem minimize x 1 + 2x 2 s.t. x 1 +x 3 = 1 x 2 + x 4 = 1 x 1 + x 2 +x 5 = 1.5 x 1,x 2, x 3,x 4, x 5 0 Yes, it is; our basic set is {3, 4, 5} and the objective function is 0. If we exchange any indices, the objective function becomes positive. Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
40 LP canonical form A standard-form LP is said to be in canonical form at a basic feasible solution if the objective coecients to all the basic variables are zero the constraint matrix for the basic variables form an identity matrix (with some permutation if necessary) If the LP is in canonical form, then it's easy to tell if the current BFS is optimal Can we always transform an LP problem to an equivalent LP in canonical form? Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
41 Transforming to canonical form Consider the constraint Ax = b and suppose that ( A = (A 1, A 2 ) and x = y z ) It is therefore the case that Ax = A 1 y + A 2 z = b Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
42 Transforming to canonical form If we let y = x B and z = x N, then we nd that A B x B + A N x N = b A B x B = b A N x N x B = A 1 B (ignore the fact that x N = 0 for now) The objective function is b A 1 B A N x N c T (x B; x N) = c T B x B + c T N x N = c T B (A 1 B = c T B A 1 B = c T B A 1 b A 1 B b ct B A 1 B B b + ( A N x N ) + c T N x N A N x N + c T N x N c T N ct B A 1 B A N ) x N Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
43 An equivalent LP We can ignore the constant term c T B A 1 b which doesn't contribute to B the optimization The alternative LP is minimize r T x s.t. Āx = b x 0 where r B = 0, r N = c N A T N ) (A 1 T B c B, Ā = A 1 A, b = A 1 B B b Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
44 Optimality testing Note that c Ā T c B = c ( ) A T A 1 T B c B = c (A B, A N ) ( T A 1 ( ) A T B ( A 1 B = c ( = c = = ( ( c B c N A T N ) A T B (A 1 T B A T N (A 1 B ) c N A T N ( 0 A T N (A 1 B ) T B c B ) T c B ) T ) c B c B ) T c B (A 1 B ) T c B ) = r ) Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
45 Optimality test The vector r = c Ā T c B = c ( ) A T A 1 T B c B is called a reduced cost coecient vector We often write y = ( ) A 1 T B c B so that r = c A T y Note that if r N 0 (equivalently r 0) at a BFS with basic variable set B, then the BFS is an optimal basic solution and A B is an optimal basis Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
46 Example Consider the example minimize x 1 + 2x 2 +3x 3 x 4 s.t. x 1 +x 3 = 1 x 2 + x 4 = 1 x 1 + x 2 +x 5 = 1.5 x 1,x 2, x 3,x 4, x 5 0 We set B = {1, 2, 3} so that x = (0.5, 1, 0.5, 0, 0) as A B = ; A 1 = B and therefore r N = c N A T N optimal ( ) (A 1 T 6 B c B = 3 ) ; this is not Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
47 Simplex tableau While performing the simplex algorithm, we maintain a simplex tableau that organizes the intermediate canonical form data: B r T c T B b basis indices Ā b What does the upper-right corner represent? Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
48 Simplex tableau While performing the simplex algorithm, we maintain a simplex tableau that organizes the intermediate canonical form data: B r T c T B b basis indices Ā b What does the upper-right corner represent? Since b = A 1 b, B ct B b = c T B A 1 B b = ct B x B, the negative objective function value Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
49 Simplex tableau example The problem minimize x 1 2x 2 s.t. x 1 +x 3 = 1 x 2 + x 4 = 1 x 1 + x 2 +x 5 = 1.5 x 1,x 2, x 3,x 4, x 5 0 has the following tableau for B = {3, 4, 5}: B Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
50 Finding a better neighbor point If one of the indices of r is negative, our basic set is not optimal We make an eort to nd a better neighboring basic solution (that diers by the current basic solution by exactly one basic variable), as long as the reduced cost coecient of the entering variable is negative Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
51 Changing basis With B = {3, 4, 5}, B Try inserting variable x 1 into the basic set; the constraint says x1 + 1 x2 + 0 x3 + 1 x4 + 0 x5 = }{{} i.e. 0 x 3 x 4 x 5 = x Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
52 Minimum ratio test The question is: how much can we increase x 1 while the current basic variable remain feasible (non-negative)? This is easy to gure out with the minimum ratio test (MRT): 1 Select the entering variable x e with reduced cost r e < 0 2 If Ā e 0 then the problem is unbounded 3 The MRT: What does θ represent? θ = min { bi Ā ie : Ā ie > 0 } Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
53 Minimum ratio test θ represents the largest amount that x e can be increased before one (or more) of the current basic variables x i becomes zero (and leaves the feasible set) Suppose that the minimum ratio is attained by one unique basic variable index o. Then x e is the entering basic variable and x o is the out-going basic variable: x o = b o ā oe θ = 0 x i = b i ā ie θ > 0 i 0 Thus the new basic set contains x e and drops x o Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
54 Tie breaking If the MRT does not give a single index, but instead a set of one or more, we choose one of these arbitrarily We say that the new basic feasible solution is degenerate because some of the basic variables x B just happen to be 0 We'll deal with this later; for now, we can just pretend that the degeneracies are actually ɛ > 0 and continue Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
55 The simplex algorithm Initialize the simplex algorithm with a feasible basic set B, so that x B 0. Let N be the remaining indices. Write the simplex tableau. 1 Test for termination. Find r e = min {r j } j N If r e 0, the solution is optimal. Otherwise, determine whether the column of Ā e contains a positive entry. If not, the objective function is unbounded below. Otherwise, let x e be the entering basic variable 2 Determine the outgoing variable. Use the MRT to determine the outgoing variable x o. 3 Update the basic set. Update B and A B and transform the problem to canonical form. Return to step 1. Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
56 Expanded simplex tableau B basis indices ) T c B c T B A 1 B b A 1 B A A 1 B b c A T ( A 1 B Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, / 49
IE 5531: Engineering Optimization I
IE 5531: Engineering Optimization I Lecture 5: The Simplex method, continued Prof. John Gunnar Carlsson September 22, 2010 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 22, 2010
More informationYinyu Ye, MS&E, Stanford MS&E310 Lecture Note #06. The Simplex Method
The Simplex Method Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye (LY, Chapters 2.3-2.5, 3.1-3.4) 1 Geometry of Linear
More information1 Review Session. 1.1 Lecture 2
1 Review Session Note: The following lists give an overview of the material that was covered in the lectures and sections. Your TF will go through these lists. If anything is unclear or you have questions
More informationORF 522. Linear Programming and Convex Analysis
ORF 522 Linear Programming and Convex Analysis The Simplex Method Marco Cuturi Princeton ORF-522 1 Reminder: Basic Feasible Solutions, Extreme points, Optima Some important theorems last time for standard
More informationIE 5531: Engineering Optimization I
IE 5531: Engineering Optimization I Lecture 7: Duality and applications Prof. John Gunnar Carlsson September 29, 2010 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 29, 2010 1
More informationIE 5531: Engineering Optimization I
IE 5531: Engineering Optimization I Lecture 15: Nonlinear optimization Prof. John Gunnar Carlsson November 1, 2010 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I November 1, 2010 1 / 24
More informationLecture 2: The Simplex method
Lecture 2 1 Linear and Combinatorial Optimization Lecture 2: The Simplex method Basic solution. The Simplex method (standardform, b>0). 1. Repetition of basic solution. 2. One step in the Simplex algorithm.
More informationMath 273a: Optimization The Simplex method
Math 273a: Optimization The Simplex method Instructor: Wotao Yin Department of Mathematics, UCLA Fall 2015 material taken from the textbook Chong-Zak, 4th Ed. Overview: idea and approach If a standard-form
More information9.1 Linear Programs in canonical form
9.1 Linear Programs in canonical form LP in standard form: max (LP) s.t. where b i R, i = 1,..., m z = j c jx j j a ijx j b i i = 1,..., m x j 0 j = 1,..., n But the Simplex method works only on systems
More informationSpring 2017 CO 250 Course Notes TABLE OF CONTENTS. richardwu.ca. CO 250 Course Notes. Introduction to Optimization
Spring 2017 CO 250 Course Notes TABLE OF CONTENTS richardwu.ca CO 250 Course Notes Introduction to Optimization Kanstantsin Pashkovich Spring 2017 University of Waterloo Last Revision: March 4, 2018 Table
More informationMath Models of OR: Some Definitions
Math Models of OR: Some Definitions John E. Mitchell Department of Mathematical Sciences RPI, Troy, NY 12180 USA September 2018 Mitchell Some Definitions 1 / 20 Active constraints Outline 1 Active constraints
More informationIE 5531: Engineering Optimization I
IE 5531: Engineering Optimization I Lecture 14: Unconstrained optimization Prof. John Gunnar Carlsson October 27, 2010 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I October 27, 2010 1
More informationStandard Form An LP is in standard form when: All variables are non-negativenegative All constraints are equalities Putting an LP formulation into sta
Chapter 4 Linear Programming: The Simplex Method An Overview of the Simplex Method Standard Form Tableau Form Setting Up the Initial Simplex Tableau Improving the Solution Calculating the Next Tableau
More information1 Overview. 2 Extreme Points. AM 221: Advanced Optimization Spring 2016
AM 22: Advanced Optimization Spring 206 Prof. Yaron Singer Lecture 7 February 7th Overview In the previous lectures we saw applications of duality to game theory and later to learning theory. In this lecture
More informationSimplex Algorithm Using Canonical Tableaus
41 Simplex Algorithm Using Canonical Tableaus Consider LP in standard form: Min z = cx + α subject to Ax = b where A m n has rank m and α is a constant In tableau form we record it as below Original Tableau
More informationCSCI 1951-G Optimization Methods in Finance Part 01: Linear Programming
CSCI 1951-G Optimization Methods in Finance Part 01: Linear Programming January 26, 2018 1 / 38 Liability/asset cash-flow matching problem Recall the formulation of the problem: max w c 1 + p 1 e 1 = 150
More information3 The Simplex Method. 3.1 Basic Solutions
3 The Simplex Method 3.1 Basic Solutions In the LP of Example 2.3, the optimal solution happened to lie at an extreme point of the feasible set. This was not a coincidence. Consider an LP in general form,
More information1. Algebraic and geometric treatments Consider an LP problem in the standard form. x 0. Solutions to the system of linear equations
The Simplex Method Most textbooks in mathematical optimization, especially linear programming, deal with the simplex method. In this note we study the simplex method. It requires basically elementary linear
More informationDr. Maddah ENMG 500 Engineering Management I 10/21/07
Dr. Maddah ENMG 500 Engineering Management I 10/21/07 Computational Procedure of the Simplex Method The optimal solution of a general LP problem is obtained in the following steps: Step 1. Express the
More informationLinear Programming. Murti V. Salapaka Electrical Engineering Department University Of Minnesota, Twin Cities
Linear Programming Murti V Salapaka Electrical Engineering Department University Of Minnesota, Twin Cities murtis@umnedu September 4, 2012 Linear Programming 1 The standard Linear Programming (SLP) problem:
More informationLinear Programming and the Simplex method
Linear Programming and the Simplex method Harald Enzinger, Michael Rath Signal Processing and Speech Communication Laboratory Jan 9, 2012 Harald Enzinger, Michael Rath Jan 9, 2012 page 1/37 Outline Introduction
More informationCO350 Linear Programming Chapter 8: Degeneracy and Finite Termination
CO350 Linear Programming Chapter 8: Degeneracy and Finite Termination 27th June 2005 Chapter 8: Finite Termination 1 The perturbation method Recap max c T x (P ) s.t. Ax = b x 0 Assumption: B is a feasible
More informationAM 121: Intro to Optimization Models and Methods Fall 2018
AM 121: Intro to Optimization Models and Methods Fall 2018 Lecture 5: The Simplex Method Yiling Chen Harvard SEAS Lesson Plan This lecture: Moving towards an algorithm for solving LPs Tableau. Adjacent
More informationIntroduce the idea of a nondegenerate tableau and its analogy with nondenegerate vertices.
2 JORDAN EXCHANGE REVIEW 1 Lecture Outline The following lecture covers Section 3.5 of the textbook [?] Review a labeled Jordan exchange with pivoting. Introduce the idea of a nondegenerate tableau and
More informationSimplex Method for LP (II)
Simplex Method for LP (II) Xiaoxi Li Wuhan University Sept. 27, 2017 (week 4) Operations Research (Li, X.) Simplex Method for LP (II) Sept. 27, 2017 (week 4) 1 / 31 Organization of this lecture Contents:
More informationCO 602/CM 740: Fundamentals of Optimization Problem Set 4
CO 602/CM 740: Fundamentals of Optimization Problem Set 4 H. Wolkowicz Fall 2014. Handed out: Wednesday 2014-Oct-15. Due: Wednesday 2014-Oct-22 in class before lecture starts. Contents 1 Unique Optimum
More information3 Development of the Simplex Method Constructing Basic Solution Optimality Conditions The Simplex Method...
Contents Introduction to Linear Programming Problem. 2. General Linear Programming problems.............. 2.2 Formulation of LP problems.................... 8.3 Compact form and Standard form of a general
More informationΩ R n is called the constraint set or feasible set. x 1
1 Chapter 5 Linear Programming (LP) General constrained optimization problem: minimize subject to f(x) x Ω Ω R n is called the constraint set or feasible set. any point x Ω is called a feasible point We
More informationDistributed Real-Time Control Systems. Lecture Distributed Control Linear Programming
Distributed Real-Time Control Systems Lecture 13-14 Distributed Control Linear Programming 1 Linear Programs Optimize a linear function subject to a set of linear (affine) constraints. Many problems can
More informationSimplex method(s) for solving LPs in standard form
Simplex method: outline I The Simplex Method is a family of algorithms for solving LPs in standard form (and their duals) I Goal: identify an optimal basis, as in Definition 3.3 I Versions we will consider:
More informationAM 121: Intro to Optimization
AM 121: Intro to Optimization Models and Methods Lecture 6: Phase I, degeneracy, smallest subscript rule. Yiling Chen SEAS Lesson Plan Phase 1 (initialization) Degeneracy and cycling Smallest subscript
More informationMath Models of OR: Extreme Points and Basic Feasible Solutions
Math Models of OR: Extreme Points and Basic Feasible Solutions John E. Mitchell Department of Mathematical Sciences RPI, Troy, NY 1180 USA September 018 Mitchell Extreme Points and Basic Feasible Solutions
More informationExample Bases and Basic Feasible Solutions 63 Let q = >: ; > and M = >: ;2 > and consider the LCP (q M). The class of ; ;2 complementary cones
Chapter 2 THE COMPLEMENTARY PIVOT ALGORITHM AND ITS EXTENSION TO FIXED POINT COMPUTING LCPs of order 2 can be solved by drawing all the complementary cones in the q q 2 - plane as discussed in Chapter.
More informationTIM 206 Lecture 3: The Simplex Method
TIM 206 Lecture 3: The Simplex Method Kevin Ross. Scribe: Shane Brennan (2006) September 29, 2011 1 Basic Feasible Solutions Have equation Ax = b contain more columns (variables) than rows (constraints),
More informationThe dual simplex method with bounds
The dual simplex method with bounds Linear programming basis. Let a linear programming problem be given by min s.t. c T x Ax = b x R n, (P) where we assume A R m n to be full row rank (we will see in the
More informationIE 5531: Engineering Optimization I
IE 5531: Engineering Optimization I Lecture 19: Midterm 2 Review Prof. John Gunnar Carlsson November 22, 2010 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I November 22, 2010 1 / 34 Administrivia
More informationLecture 2: The Simplex method. 1. Repetition of the geometrical simplex method. 2. Linear programming problems on standard form.
Lecture 2: The Simplex method. Repetition of the geometrical simplex method. 2. Linear programming problems on standard form. 3. The Simplex algorithm. 4. How to find an initial basic solution. Lecture
More informationLinear programs Optimization Geoff Gordon Ryan Tibshirani
Linear programs 10-725 Optimization Geoff Gordon Ryan Tibshirani Review: LPs LPs: m constraints, n vars A: R m n b: R m c: R n x: R n ineq form [min or max] c T x s.t. Ax b m n std form [min or max] c
More informationLesson 27 Linear Programming; The Simplex Method
Lesson Linear Programming; The Simplex Method Math 0 April 9, 006 Setup A standard linear programming problem is to maximize the quantity c x + c x +... c n x n = c T x subject to constraints a x + a x
More informationChapter 5 Linear Programming (LP)
Chapter 5 Linear Programming (LP) General constrained optimization problem: minimize f(x) subject to x R n is called the constraint set or feasible set. any point x is called a feasible point We consider
More informationLecture 4: Algebra, Geometry, and Complexity of the Simplex Method. Reading: Sections 2.6.4, 3.5,
Lecture 4: Algebra, Geometry, and Complexity of the Simplex Method Reading: Sections 2.6.4, 3.5, 10.2 10.5 1 Summary of the Phase I/Phase II Simplex Method We write a typical simplex tableau as z x 1 x
More informationOperations Research Lecture 2: Linear Programming Simplex Method
Operations Research Lecture 2: Linear Programming Simplex Method Notes taken by Kaiquan Xu@Business School, Nanjing University Mar 10th 2016 1 Geometry of LP 1.1 Graphical Representation and Solution Example
More informationThe Simplex Method for Solving a Linear Program Prof. Stephen Graves
The Simplex Method for Solving a Linear Program Prof. Stephen Graves Observations from Geometry feasible region is a convex polyhedron an optimum occurs at a corner point possible algorithm - search over
More informationA Review of Linear Programming
A Review of Linear Programming Instructor: Farid Alizadeh IEOR 4600y Spring 2001 February 14, 2001 1 Overview In this note we review the basic properties of linear programming including the primal simplex
More informationmin 4x 1 5x 2 + 3x 3 s.t. x 1 + 2x 2 + x 3 = 10 x 1 x 2 6 x 1 + 3x 2 + x 3 14
The exam is three hours long and consists of 4 exercises. The exam is graded on a scale 0-25 points, and the points assigned to each question are indicated in parenthesis within the text. If necessary,
More informationLinear programs, convex polyhedra, extreme points
MVE165/MMG631 Extreme points of convex polyhedra; reformulations; basic feasible solutions; the simplex method Ann-Brith Strömberg 2015 03 27 Linear programs, convex polyhedra, extreme points A linear
More informationMATH 4211/6211 Optimization Linear Programming
MATH 4211/6211 Optimization Linear Programming Xiaojing Ye Department of Mathematics & Statistics Georgia State University Xiaojing Ye, Math & Stat, Georgia State University 0 The standard form of a Linear
More informationNotes on Simplex Algorithm
Notes on Simplex Algorithm CS 9 Staff October 8, 7 Until now, we have represented the problems geometrically, and solved by finding a corner and moving around Now we learn an algorithm to solve this without
More informationOPERATIONS RESEARCH. Linear Programming Problem
OPERATIONS RESEARCH Chapter 1 Linear Programming Problem Prof. Bibhas C. Giri Department of Mathematics Jadavpur University Kolkata, India Email: bcgiri.jumath@gmail.com MODULE - 2: Simplex Method for
More informationChapter 4 The Simplex Algorithm Part I
Chapter 4 The Simplex Algorithm Part I Based on Introduction to Mathematical Programming: Operations Research, Volume 1 4th edition, by Wayne L. Winston and Munirpallam Venkataramanan Lewis Ntaimo 1 Modeling
More informationIn Chapters 3 and 4 we introduced linear programming
SUPPLEMENT The Simplex Method CD3 In Chapters 3 and 4 we introduced linear programming and showed how models with two variables can be solved graphically. We relied on computer programs (WINQSB, Excel,
More informationPart 1. The Review of Linear Programming
In the name of God Part 1. The Review of Linear Programming 1.2. Spring 2010 Instructor: Dr. Masoud Yaghini Outline Introduction Basic Feasible Solutions Key to the Algebra of the The Simplex Algorithm
More informationIntroduction to Mathematical Programming IE406. Lecture 3. Dr. Ted Ralphs
Introduction to Mathematical Programming IE406 Lecture 3 Dr. Ted Ralphs IE406 Lecture 3 1 Reading for This Lecture Bertsimas 2.1-2.2 IE406 Lecture 3 2 From Last Time Recall the Two Crude Petroleum example.
More informationLinear Algebra Review: Linear Independence. IE418 Integer Programming. Linear Algebra Review: Subspaces. Linear Algebra Review: Affine Independence
Linear Algebra Review: Linear Independence IE418: Integer Programming Department of Industrial and Systems Engineering Lehigh University 21st March 2005 A finite collection of vectors x 1,..., x k R n
More informationIE 400 Principles of Engineering Management. The Simplex Algorithm-I: Set 3
IE 4 Principles of Engineering Management The Simple Algorithm-I: Set 3 So far, we have studied how to solve two-variable LP problems graphically. However, most real life problems have more than two variables!
More informationDeveloping an Algorithm for LP Preamble to Section 3 (Simplex Method)
Moving from BFS to BFS Developing an Algorithm for LP Preamble to Section (Simplex Method) We consider LP given in standard form and let x 0 be a BFS. Let B ; B ; :::; B m be the columns of A corresponding
More informationFebruary 17, Simplex Method Continued
15.053 February 17, 2005 Simplex Method Continued 1 Today s Lecture Review of the simplex algorithm. Formalizing the approach Alternative Optimal Solutions Obtaining an initial bfs Is the simplex algorithm
More informationLecture: Algorithms for LP, SOCP and SDP
1/53 Lecture: Algorithms for LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2018.html wenzw@pku.edu.cn Acknowledgement:
More information3. THE SIMPLEX ALGORITHM
Optimization. THE SIMPLEX ALGORITHM DPK Easter Term. Introduction We know that, if a linear programming problem has a finite optimal solution, it has an optimal solution at a basic feasible solution (b.f.s.).
More informationThe Primal-Dual Algorithm P&S Chapter 5 Last Revised October 30, 2006
The Primal-Dual Algorithm P&S Chapter 5 Last Revised October 30, 2006 1 Simplex solves LP by starting at a Basic Feasible Solution (BFS) and moving from BFS to BFS, always improving the objective function,
More informationTermination, Cycling, and Degeneracy
Chapter 4 Termination, Cycling, and Degeneracy We now deal first with the question, whether the simplex method terminates. The quick answer is no, if it is implemented in a careless way. Notice that we
More informationMotivating examples Introduction to algorithms Simplex algorithm. On a particular example General algorithm. Duality An application to game theory
Instructor: Shengyu Zhang 1 LP Motivating examples Introduction to algorithms Simplex algorithm On a particular example General algorithm Duality An application to game theory 2 Example 1: profit maximization
More informationORF 522. Linear Programming and Convex Analysis
ORF 5 Linear Programming and Convex Analysis Initial solution and particular cases Marco Cuturi Princeton ORF-5 Reminder: Tableaux At each iteration, a tableau for an LP in standard form keeps track of....................
More informationMS-E2140. Lecture 1. (course book chapters )
Linear Programming MS-E2140 Motivations and background Lecture 1 (course book chapters 1.1-1.4) Linear programming problems and examples Problem manipulations and standard form problems Graphical representation
More informationA TOUR OF LINEAR ALGEBRA FOR JDEP 384H
A TOUR OF LINEAR ALGEBRA FOR JDEP 384H Contents Solving Systems 1 Matrix Arithmetic 3 The Basic Rules of Matrix Arithmetic 4 Norms and Dot Products 5 Norms 5 Dot Products 6 Linear Programming 7 Eigenvectors
More informationLP. Lecture 3. Chapter 3: degeneracy. degeneracy example cycling the lexicographic method other pivot rules the fundamental theorem of LP
LP. Lecture 3. Chapter 3: degeneracy. degeneracy example cycling the lexicographic method other pivot rules the fundamental theorem of LP 1 / 23 Repetition the simplex algorithm: sequence of pivots starting
More informationIE 5531: Engineering Optimization I
IE 5531: Engineering Optimization I Lecture 12: Nonlinear optimization, continued Prof. John Gunnar Carlsson October 20, 2010 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I October 20,
More informationCO350 Linear Programming Chapter 8: Degeneracy and Finite Termination
CO350 Linear Programming Chapter 8: Degeneracy and Finite Termination 22th June 2005 Chapter 8: Finite Termination Recap On Monday, we established In the absence of degeneracy, the simplex method will
More informationLecture 11: Post-Optimal Analysis. September 23, 2009
Lecture : Post-Optimal Analysis September 23, 2009 Today Lecture Dual-Simplex Algorithm Post-Optimal Analysis Chapters 4.4 and 4.5. IE 30/GE 330 Lecture Dual Simplex Method The dual simplex method will
More informationNumerical Optimization
Linear Programming Computer Science and Automation Indian Institute of Science Bangalore 560 012, India. NPTEL Course on min x s.t. Transportation Problem ij c ijx ij 3 j=1 x ij a i, i = 1, 2 2 i=1 x ij
More informationMATH2070 Optimisation
MATH2070 Optimisation Linear Programming Semester 2, 2012 Lecturer: I.W. Guo Lecture slides courtesy of J.R. Wishart Review The standard Linear Programming (LP) Problem Graphical method of solving LP problem
More information4.5 Simplex method. min z = c T x s.v. Ax = b. LP in standard form
4.5 Simplex method min z = c T x s.v. Ax = b x 0 LP in standard form Examine a sequence of basic feasible solutions with non increasing objective function value until an optimal solution is reached or
More informationLecture slides by Kevin Wayne
LINEAR PROGRAMMING I a refreshing example standard form fundamental questions geometry linear algebra simplex algorithm Lecture slides by Kevin Wayne Last updated on 7/25/17 11:09 AM Linear programming
More informationNotes taken by Graham Taylor. January 22, 2005
CSC4 - Linear Programming and Combinatorial Optimization Lecture : Different forms of LP. The algebraic objects behind LP. Basic Feasible Solutions Notes taken by Graham Taylor January, 5 Summary: We first
More informationThe Simplex Algorithm and Goal Programming
The Simplex Algorithm and Goal Programming In Chapter 3, we saw how to solve two-variable linear programming problems graphically. Unfortunately, most real-life LPs have many variables, so a method is
More informationContents. 4.5 The(Primal)SimplexMethod NumericalExamplesoftheSimplexMethod
Contents 4 The Simplex Method for Solving LPs 149 4.1 Transformations to be Carried Out On an LP Model Before Applying the Simplex Method On It... 151 4.2 Definitions of Various Types of Basic Vectors
More information2.098/6.255/ Optimization Methods Practice True/False Questions
2.098/6.255/15.093 Optimization Methods Practice True/False Questions December 11, 2009 Part I For each one of the statements below, state whether it is true or false. Include a 1-3 line supporting sentence
More informationECE 307 Techniques for Engineering Decisions
ECE 7 Techniques for Engineering Decisions Introduction to the Simple Algorithm George Gross Department of Electrical and Computer Engineering University of Illinois at Urbana-Champaign ECE 7 5 9 George
More informationTRANSPORTATION PROBLEMS
Chapter 6 TRANSPORTATION PROBLEMS 61 Transportation Model Transportation models deal with the determination of a minimum-cost plan for transporting a commodity from a number of sources to a number of destinations
More informationIE 400: Principles of Engineering Management. Simplex Method Continued
IE 400: Principles of Engineering Management Simplex Method Continued 1 Agenda Simplex for min problems Alternative optimal solutions Unboundedness Degeneracy Big M method Two phase method 2 Simplex for
More informationDual Basic Solutions. Observation 5.7. Consider LP in standard form with A 2 R m n,rank(a) =m, and dual LP:
Dual Basic Solutions Consider LP in standard form with A 2 R m n,rank(a) =m, and dual LP: Observation 5.7. AbasisB yields min c T x max p T b s.t. A x = b s.t. p T A apple c T x 0 aprimalbasicsolutiongivenbyx
More informationLINEAR PROGRAMMING I. a refreshing example standard form fundamental questions geometry linear algebra simplex algorithm
Linear programming Linear programming. Optimize a linear function subject to linear inequalities. (P) max c j x j n j= n s. t. a ij x j = b i i m j= x j 0 j n (P) max c T x s. t. Ax = b Lecture slides
More information15-780: LinearProgramming
15-780: LinearProgramming J. Zico Kolter February 1-3, 2016 1 Outline Introduction Some linear algebra review Linear programming Simplex algorithm Duality and dual simplex 2 Outline Introduction Some linear
More informationCO 250 Final Exam Guide
Spring 2017 CO 250 Final Exam Guide TABLE OF CONTENTS richardwu.ca CO 250 Final Exam Guide Introduction to Optimization Kanstantsin Pashkovich Spring 2017 University of Waterloo Last Revision: March 4,
More informationLinear Programming and its Extensions Prof. Prabha Shrama Department of Mathematics and Statistics Indian Institute of Technology, Kanpur
Linear Programming and its Extensions Prof. Prabha Shrama Department of Mathematics and Statistics Indian Institute of Technology, Kanpur Lecture No. # 03 Moving from one basic feasible solution to another,
More informationLecture 6 Simplex method for linear programming
Lecture 6 Simplex method for linear programming Weinan E 1,2 and Tiejun Li 2 1 Department of Mathematics, Princeton University, weinan@princeton.edu 2 School of Mathematical Sciences, Peking University,
More informationDiscrete Optimization
Prof. Friedrich Eisenbrand Martin Niemeier Due Date: April 15, 2010 Discussions: March 25, April 01 Discrete Optimization Spring 2010 s 3 You can hand in written solutions for up to two of the exercises
More informationLP Duality: outline. Duality theory for Linear Programming. alternatives. optimization I Idea: polyhedra
LP Duality: outline I Motivation and definition of a dual LP I Weak duality I Separating hyperplane theorem and theorems of the alternatives I Strong duality and complementary slackness I Using duality
More informationThe Simplex Method. Lecture 5 Standard and Canonical Forms and Setting up the Tableau. Lecture 5 Slide 1. FOMGT 353 Introduction to Management Science
The Simplex Method Lecture 5 Standard and Canonical Forms and Setting up the Tableau Lecture 5 Slide 1 The Simplex Method Formulate Constrained Maximization or Minimization Problem Convert to Standard
More informationLinear Programming. Operations Research. Anthony Papavasiliou 1 / 21
1 / 21 Linear Programming Operations Research Anthony Papavasiliou Contents 2 / 21 1 Primal Linear Program 2 Dual Linear Program Table of Contents 3 / 21 1 Primal Linear Program 2 Dual Linear Program Linear
More informationChapter 1. Preliminaries
Introduction This dissertation is a reading of chapter 4 in part I of the book : Integer and Combinatorial Optimization by George L. Nemhauser & Laurence A. Wolsey. The chapter elaborates links between
More information{ move v ars to left, consts to right { replace = by t wo and constraints Ax b often nicer for theory Ax = b good for implementations. { A invertible
Finish remarks on min-cost ow. Strongly polynomial algorithms exist. { Tardos 1985 { minimum mean-cost cycle { reducing -optimality { \xing" arcs of very high reduced cost { best running running time roughly
More information"SYMMETRIC" PRIMAL-DUAL PAIR
"SYMMETRIC" PRIMAL-DUAL PAIR PRIMAL Minimize cx DUAL Maximize y T b st Ax b st A T y c T x y Here c 1 n, x n 1, b m 1, A m n, y m 1, WITH THE PRIMAL IN STANDARD FORM... Minimize cx Maximize y T b st Ax
More information4.5 Simplex method. LP in standard form: min z = c T x s.t. Ax = b
4.5 Simplex method LP in standard form: min z = c T x s.t. Ax = b x 0 George Dantzig (1914-2005) Examine a sequence of basic feasible solutions with non increasing objective function values until an optimal
More informationIntroduction to Operations Research Prof. G. Srinivasan Department of Management Studies Indian Institute of Technology, Madras
Introduction to Operations Research Prof. G. Srinivasan Department of Management Studies Indian Institute of Technology, Madras Module - 03 Simplex Algorithm Lecture 15 Infeasibility In this class, we
More informationChapter 7 Network Flow Problems, I
Chapter 7 Network Flow Problems, I Network flow problems are the most frequently solved linear programming problems. They include as special cases, the assignment, transportation, maximum flow, and shortest
More information1 Review of last lecture and introduction
Semidefinite Programming Lecture 10 OR 637 Spring 2008 April 16, 2008 (Wednesday) Instructor: Michael Jeremy Todd Scribe: Yogeshwer (Yogi) Sharma 1 Review of last lecture and introduction Let us first
More information3. Vector spaces 3.1 Linear dependence and independence 3.2 Basis and dimension. 5. Extreme points and basic feasible solutions
A. LINEAR ALGEBRA. CONVEX SETS 1. Matrices and vectors 1.1 Matrix operations 1.2 The rank of a matrix 2. Systems of linear equations 2.1 Basic solutions 3. Vector spaces 3.1 Linear dependence and independence
More informationConstrained and Unconstrained Optimization Prof. Adrijit Goswami Department of Mathematics Indian Institute of Technology, Kharagpur
Constrained and Unconstrained Optimization Prof. Adrijit Goswami Department of Mathematics Indian Institute of Technology, Kharagpur Lecture - 01 Introduction to Optimization Today, we will start the constrained
More informationAnn-Brith Strömberg. Lecture 4 Linear and Integer Optimization with Applications 1/10
MVE165/MMG631 Linear and Integer Optimization with Applications Lecture 4 Linear programming: degeneracy; unbounded solution; infeasibility; starting solutions Ann-Brith Strömberg 2017 03 28 Lecture 4
More information