# Yinyu Ye, MS&E, Stanford MS&E310 Lecture Note #06. The Simplex Method

Save this PDF as:

Size: px
Start display at page:

Download "Yinyu Ye, MS&E, Stanford MS&E310 Lecture Note #06. The Simplex Method"

## Transcription

1 The Simplex Method Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. yyye (LY, Chapters , ) 1

2 Geometry of Linear Programming (LP) Consider the following LP problem: maximize x 1 +2x 2 subject to x 1 1 x 2 1 x 1 +x x 1, x

3 LP Geometry depicted in two variable space a1 a2 a2 a3 c If the direction of c is contained by the norm cone of a corner point, then the point is optimal. a3 a4 Each corner point has a norm direction cone a4 Objective contour a5 Figure 1: Feasible region with objective contours 3

4 Solution (decision, point): any specification of values for all decision variables, regardless of whether it is a desirable or even allowable choice. Feasible Solution: a solution for which all the constraints are satisfied. Feasible Region (constraint set, feasible set): the collection of all feasible solution. Interior, Boundary, and Face Extreme Point or Corner Point or Vertex Objective Function Contour: iso-profit (or iso-cost) line. Optimal Solution: a feasible solution that has the most favorable objective value. Optimal Objective Value: the value of the objective function evaluated at an optimal solution. Active Constraint: binding constraint. 4

5 Definition of Face and Extreme Points Let P be a polyhedron in R n, then F is a face of P if and only if there is a vector b for which F is the set of points attaining max{b T y : y P } provided this maximum is finite. A polyhedron has only finite many faces; each face is a nonempty polyhedron. A vector y P is an extreme point or a vertex of P if y is not a convex combination of two distinct feasible points. 5

6 Theory of LP All LP problems fall into one of the following three classes: Problem is infeasible: feasible region is empty. Problem is unbounded: feasible region is unbounded towards the optimizing direction. Problem is feasible and bounded: There exists an optimal solution or optimizer. There may be a unique optimizer or multiple optimizers. All optimizers are on a face of the feasible region. There is always at least one corner (extreme) optimizer if the face has a corner. If a corner point is not worse than all its adjacent or neighboring corners, then it is optimal. 6

7 History of the Simplex Method George B. Dantzig s Simplex Method for LP stands as one of the most significant algorithmic achievements of the 20th century. It is now over 50 years old and still going strong. The basic idea of the simplex method to confine the search to corner points of the feasible region (of which there are only finitely many) in a most intelligent way. In contrast, interior-point methods will move in the interior of the feasible region, hoping to by-pass many corner points on the boundary of the region. The key for the simplex method is to make computers see corner points; and the key for interior-point methods is to stay in the interior of the feasible region. 7

8 From Geometry to Algebra How to make computer recognize a corner point? How to make computer tell that two corners are neighboring? How to make computer terminate and declare optimality? How to make computer identify a better neighboring corner? 8

9 LP Standard Form minimize c 1 x 1 + c 2 x c n x n subject to a 11 x 1 + a 12 x a 1n x n = b 1, a 21 x 1 + a 22 x a 2n x n = b 2,. a m1 x 1 + a m2 x a mn x n = b m, x j 0, j = 1, 2,..., n. Equivalently, (LP ) minimize c T x subject to Ax = b, x 0. (LD) maximize b T y subject to A T y + s = c, s 0. 9

10 Reduction to the Standard Form Eliminating free variables: substitute with the difference of two nonnegative variables Eliminating inequalities: add slack variables x = x + x where x +, x 0. a T x b a T x + s = b, s 0, a T x b a T x s = b, s 0. Eliminating upper bounds: move them to constraints x 3 x + s = 3, s 0. Eliminating nonzezro lower bounds: shift the decision variables x 3 x := x 3. Change max c T x to min c T x. 10

11 An LP Example in Standard Form minimize x 1 2x 2 subject to x 1 +x 3 = 1 x 2 +x 4 = 1 x 1 +x 2 +x 5 = 1.5 x 1, x 2, x 3, x 4, x

12 Basic and Basic Feasible Solution (BFS) In the LP standard form, let s assume that we selected m linearly independent columns, denoted by the index set B from A and solve A B x B = b for the m-vector x B. By setting the variables x N of x corresponding to the remaining columns of A equal to zero, we obtain a solution x of Ax = b. Then x is said to be a basic solution to (LP) with respect to basis A B. The components of x B are called basic variables and those of x N are called nonbasic variables. Two basic solutions are adjacent if they differ by exactly one basic (or nonbasic) variable. If a basic solution satisfies x B 0, then x is called a basic feasible solution (BFS), and it is an extreme point of the feasible region. If one or more components in x B has value zero, x is said to be degenerate. 12

13 Geometry vs Algebra Theorem 1 Consider the polyhedron in the standard LP form. Then a basic feasible solution and a corner point are equivalent; the former is algebraic and the latter is geometric. In the LP example: Basis 3,4,5 1,4,5 3,4,1 3,2,5 3,4,2 1,2,3 1,2,4 1,2,5 Feasible? x 1, x 2 0, 0 1, 0 1.5, 0 0, 1 0, 1.5.5, 1 1,.5 1, 1 Neighboring Basic Solutions: Two basic solutions are neighboring or adjacent if they differ by exactly one basic (or nonbasic) variable. A basic feasible solution is optimal if no better neighboring BFS exists, or the direction to each neighbor BFS is non-improving. 13

14 Theorem 2 (The Fundamental Theorem of LP in Algebraic form) a Given (LP) and (LD) where A has full row rank m, i) if there is a feasible solution, there is a basic feasible solution (Carathéodory s theorem); ii) if there is an optimal solution, there is an optimal basic solution. The simplex method is to proceed from one BFS (a corner point of the feasible region) to an adjacent or neighboring one, in such a way as to continuously improve the value of the objective function. We now prove ii) in the next slide. a Text p.21 14

15 Rounding to an Optimal Basic Feasible Solution for LP 1. Start with any optimal solution x 0 = x after removing all variables with the zero value, thus x 0 > 0. Let A 0 be the remaining constraint matrix corresponding to x 0 such that A 0 x 0 = b. If the columns of A 0 is linearly independent, then x 0 is already a BFS (if x 0 < m then fill any rest of independent columns to make it a basis so that the BFS is degenerate). Otherwise, let k = 0 and go to the next step. 2. Find any vector d such that A k d = 0, d 0, and let x k+1 := x k + αd where α is chosen so that x k+1 0 and at least one entry of x k+1 equals Remove all zero entries from x k+1 and let A k+1 be the remaining constraint matrix corresponding to x k+1 such that A k+1 x k+1 = b. 4. If the columns of A k+1 is linearly independent, then stop with the BFS x k+1 ; otherwise set k := k + 1 and return to Step 2. The final output would be a BFS, and it should be optimal because it remains complementary to any optimal dual vector that is complementary to the initial x. 15

16 Optimality test Consider a BFS (x 1, x 2, x 3, x 4, x 5 ) = (0, 0, 1, 1, 1.5) with basic variables {3, 4, 5}. It s NOT optimal to the problem minimize x 1 2x 2 subject to x 1 +x 3 = 1 x 2 +x 4 = 1 x 1 +x 2 +x 5 = 1.5 x 1, x 2, x 3, x 4, x 5 0; 16

17 but it would be optimal to the below problem minimize x 1 +2x 2 subject to x 1 +x 3 = 1 x 2 +x 4 = 1 x 1 +x 2 +x 5 = 1.5 x 1, x 2, x 3, x 4, x 5 0. At a BFS, when the objective coefficients to all the basic variables are zero and those to all the non-basic variables are non-negative, then the BFS is optimal if the objective is to be minimized. 17

18 LP Canonical Form An LP in the standard form is said to be in canonical form at a BFS if The objective coefficients to all the basic variables are zero, that is, the objective function contains only non-basic variables and The constraint matrix for the basic variables form an identity matrix (with some permutation if necessary). It s easy to see whether or not the current BFS is optimal or not, if the LP is in canonical form. Is there a way to transform the original LP problem to an equivalent LP in the canonical form? That is, is there an algorithm that the feasible set and optimal solution set remain the same when compared to the original LP? 18

19 Transforming to canonical form Suppose the basis of a BFS is A B and the rest is A N. One can transform the equality constraint to A 1 B Ax = A 1 B b, (so that x B = A 1 B b A 1 B A N x N ) That is, we express x B in terms of x N, the degree of freedom variables. Then the objective function becomes c T x = c T Bx B + c T Nx N = c T BA 1 B b ct BA 1 B A N x N + c T Nx N = c T BA 1 B b + (ct N c T BA 1 B A N )x N. The transformed problem is then in canonical form with respect to the BFS A B ; and it s equivalent to the original LP. That is, its feasible set and optimal solution set don t change. 19

20 Equivalent LP By ignoring the constant term in the objective function, we have an equivalent intermediate LP problem in canonical form: minimize subject to r T x Āx = b, x 0, where r B := 0, r N := c N A T N (A 1 B )T c B, Ā := A 1 B A, b := A 1 B b. Vector r is the so-called reduced cost coefficient vector or reduced gradient vector. 20

21 Optimality Test Revisited The reduced gradient vector r R n can be written by r = c ĀT c B = c A T (A 1 B )T c B = c A T y, where y = (A 1 B )T c B is called the shadow price or dual vector. Theorem 3 If r N 0 ( equivalently r 0) at a BFS with basic variable set B, then the BFS x is a primal optimal basic solution and y is a dual optimal basic solution, both with A B being an optimal basis,. The proof is simply from the primal feasibility, dual feasibility and complementarity with dual slack vector s = r. 21

22 In the LP Example, let the basic variable set B = {1, 2, 3} so that A B = and A 1 B = y T = (0, 1, 1) and r T = (0, 0, 0, 1, 1) The corresponding optimal corner solution is (x 1, x 2 ) = (0.5, 1) in the original problem. 22

23 Canonical Tableau The intermediate canonical form data can be organized in a tableau: B r T c T B b Basis Indices Ā b The up-right corner quantity represents negative of the objective value at the current basic solution. With the initial basic variable set B = {3, 4, 5} for the example, the canonical tableau is B

24 Moving to a Better Neighboring Corner As we saw in the previous slides, the BFS given by the canonical form is optimal if the reduced-cost vector r 0. What to do if it is not optimal? An effort can be made to find a better neighboring basic feasible solution. That is, a new BFS that differs from the current one by exactly one new basic variable, as long as the reduced cost coefficient of the entering variable is negative. 24

25 Changing Basis for the LP Example With the initial basic variable set B = {3, 4, 5}: B Select the entering basic variable x 1 with r 1 = 1 < 0 and consider x x 4 = 1 0 x 1. x 5 How much can we increase x 1 while the current basic variables remain feasible (non-negative)? This connects to the definition of minimum ratio test (MRT)

26 Minimum Ratio Test (MRT) Select the entering variable x e with its reduced cost r e < 0; If column Ā.e 0, then Unbounded Minimum Ratio Test (MRT): θ := min { } bi : Ā Āie > 0. ie 26

27 How Much Can the New Basic Variable be Increased? What is θ? It is the largest amount that x e can be increased before one (or more) of the current basic variables x i decreases to zero. For the moment, assume that the minimum ratio in the MRT is attained by exactly one basic variable index, o. Then x o is called the out-going basic variable: That is, in the basic variable set. x o = b o ā oe θ = 0 x i = b i ā ie θ > 0 i o. x o x e 27

28 Tie Breaking If the MRT does not result in a single index, but rather in a set of two or more indices for which the minimum ratio is attained, we select one of these indices as out-going arbitrarily. Again, when x e reaches θ we have generated a corner point of the feasible region that is adjacent to the one associated with the index set B. The new basic variable set associated with this new corner point is o e. But the new BFS is degenerate, one of its basic variable has value 0. Pretending that ϵ > 0 but arbitrarily small and continue the transformation process to the new canonical form. 28

29 The Simplex Algorithm 0. Initialize with a minimization problem in feasible canonical form with respect to a basic index set B. Let N denote the complementary index set. 1. Test for termination: first find r e = min j N {r j }. If r e 0, stop. The solution is optimal. Otherwise determine whether the column of Ā.e contains a positive entry. If not, the objective function is unbounded below. Terminate. Let x e be the entering basic variable. 2. Determine the outgoing: execute the MRT to determine the outgoing variable x o. 3. Update basis: update B and A B and transform the problem to canonical form and return to Step 1. 29

30 Transform to a New Canonical Form: Pivoting The passage from one canonical form to the next can be carried out by an algebraic process called pivoting. In a nutshell, a pivot on the nonzero (in our case, positive) pivot element ā oe amounts to (a) expressing x e in terms of all the non-basic variables; (b) replacing x e in all other formula by using the expression. In terms of Ā, this has the effect of making the column of x e look like the column of an identity matrix with 1 in the o th row and zero elsewhere. 30

31 Pivoting: Gauss-Jordan Elimination Process Here is what happens to the current table: 1. All the entries in row o are divided by the pivot element ā oe (This produces a 1 (one) in the column of x s ). 2. For all i o, all other entries are modified according to the rule (When j = e, the new entry is just 0 (zero).) ā ij ā ij āoj ā oe ā ie. The right-hand side and the objective function row are modified in the same way. 31

32 The Initial Canonical Form of the Example Choose e = 2 and MRT would decide o = 4 with θ = 1: B MRT If necessary, deviding the pivoting row by the pivot value to make the pivot element equal to

33 Transform to a New Canonical Form by Pivoting Objective row := Objective Row + 2 Pivot Row : Bottom row := Bottom Row - Pivot Row : This is a new canonical form! B B

34 Let s Keep Going Choose e = 1 and MRT would decide o = 5 with θ = 1/2: B MRT

35 Transform to a New Canonical Form by Pivoting Objective Row := Objective Row + Pivot Row : Second Row := Second Row - Pivot Row : B Stop! It s optimal:) B

36 Detailed Canonical Tableau for Production If the original LP is the production problem: maximize p T x The initial canonical tableau for minimization would be The intermediate canonical tableau would be subject to Rx b(> 0), x 0. B p T 0 0 Basis Indices R I b B r T y T c T B b Basis Indices Ā A 1 B b 36

### IE 5531: Engineering Optimization I

IE 5531: Engineering Optimization I Lecture 5: The Simplex method, continued Prof. John Gunnar Carlsson September 22, 2010 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 22, 2010

### IE 5531: Engineering Optimization I

IE 5531: Engineering Optimization I Lecture 3: Linear Programming, Continued Prof. John Gunnar Carlsson September 15, 2010 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 15, 2010

### Midterm Review. Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A.

Midterm Review Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye (LY, Chapter 1-4, Appendices) 1 Separating hyperplane

### 1. Algebraic and geometric treatments Consider an LP problem in the standard form. x 0. Solutions to the system of linear equations

The Simplex Method Most textbooks in mathematical optimization, especially linear programming, deal with the simplex method. In this note we study the simplex method. It requires basically elementary linear

### CO 602/CM 740: Fundamentals of Optimization Problem Set 4

CO 602/CM 740: Fundamentals of Optimization Problem Set 4 H. Wolkowicz Fall 2014. Handed out: Wednesday 2014-Oct-15. Due: Wednesday 2014-Oct-22 in class before lecture starts. Contents 1 Unique Optimum

### Math Models of OR: Some Definitions

Math Models of OR: Some Definitions John E. Mitchell Department of Mathematical Sciences RPI, Troy, NY 12180 USA September 2018 Mitchell Some Definitions 1 / 20 Active constraints Outline 1 Active constraints

### 9.1 Linear Programs in canonical form

9.1 Linear Programs in canonical form LP in standard form: max (LP) s.t. where b i R, i = 1,..., m z = j c jx j j a ijx j b i i = 1,..., m x j 0 j = 1,..., n But the Simplex method works only on systems

### Simplex Algorithm Using Canonical Tableaus

41 Simplex Algorithm Using Canonical Tableaus Consider LP in standard form: Min z = cx + α subject to Ax = b where A m n has rank m and α is a constant In tableau form we record it as below Original Tableau

### Ω R n is called the constraint set or feasible set. x 1

1 Chapter 5 Linear Programming (LP) General constrained optimization problem: minimize subject to f(x) x Ω Ω R n is called the constraint set or feasible set. any point x Ω is called a feasible point We

### Math 273a: Optimization The Simplex method

Math 273a: Optimization The Simplex method Instructor: Wotao Yin Department of Mathematics, UCLA Fall 2015 material taken from the textbook Chong-Zak, 4th Ed. Overview: idea and approach If a standard-form

### CSCI 1951-G Optimization Methods in Finance Part 01: Linear Programming

CSCI 1951-G Optimization Methods in Finance Part 01: Linear Programming January 26, 2018 1 / 38 Liability/asset cash-flow matching problem Recall the formulation of the problem: max w c 1 + p 1 e 1 = 150

### Chapter 5 Linear Programming (LP)

Chapter 5 Linear Programming (LP) General constrained optimization problem: minimize f(x) subject to x R n is called the constraint set or feasible set. any point x is called a feasible point We consider

### Lecture 4: Algebra, Geometry, and Complexity of the Simplex Method. Reading: Sections 2.6.4, 3.5,

Lecture 4: Algebra, Geometry, and Complexity of the Simplex Method Reading: Sections 2.6.4, 3.5, 10.2 10.5 1 Summary of the Phase I/Phase II Simplex Method We write a typical simplex tableau as z x 1 x

### Review Solutions, Exam 2, Operations Research

Review Solutions, Exam 2, Operations Research 1. Prove the weak duality theorem: For any x feasible for the primal and y feasible for the dual, then... HINT: Consider the quantity y T Ax. SOLUTION: To

### Note 3: LP Duality. If the primal problem (P) in the canonical form is min Z = n (1) then the dual problem (D) in the canonical form is max W = m (2)

Note 3: LP Duality If the primal problem (P) in the canonical form is min Z = n j=1 c j x j s.t. nj=1 a ij x j b i i = 1, 2,..., m (1) x j 0 j = 1, 2,..., n, then the dual problem (D) in the canonical

### Part 1. The Review of Linear Programming

In the name of God Part 1. The Review of Linear Programming 1.2. Spring 2010 Instructor: Dr. Masoud Yaghini Outline Introduction Basic Feasible Solutions Key to the Algebra of the The Simplex Algorithm

### More First-Order Optimization Algorithms

More First-Order Optimization Algorithms Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye Chapters 3, 8, 3 The SDM

### Standard Form An LP is in standard form when: All variables are non-negativenegative All constraints are equalities Putting an LP formulation into sta

Chapter 4 Linear Programming: The Simplex Method An Overview of the Simplex Method Standard Form Tableau Form Setting Up the Initial Simplex Tableau Improving the Solution Calculating the Next Tableau

### Dr. Maddah ENMG 500 Engineering Management I 10/21/07

Dr. Maddah ENMG 500 Engineering Management I 10/21/07 Computational Procedure of the Simplex Method The optimal solution of a general LP problem is obtained in the following steps: Step 1. Express the

### min 4x 1 5x 2 + 3x 3 s.t. x 1 + 2x 2 + x 3 = 10 x 1 x 2 6 x 1 + 3x 2 + x 3 14

The exam is three hours long and consists of 4 exercises. The exam is graded on a scale 0-25 points, and the points assigned to each question are indicated in parenthesis within the text. If necessary,

### OPERATIONS RESEARCH. Linear Programming Problem

OPERATIONS RESEARCH Chapter 1 Linear Programming Problem Prof. Bibhas C. Giri Department of Mathematics Jadavpur University Kolkata, India Email: bcgiri.jumath@gmail.com MODULE - 2: Simplex Method for

### + 5x 2. = x x. + x 2. Transform the original system into a system x 2 = x x 1. = x 1

University of California, Davis Department of Agricultural and Resource Economics ARE 5 Optimization with Economic Applications Lecture Notes Quirino Paris The Pivot Method for Solving Systems of Equations...................................

### A Review of Linear Programming

A Review of Linear Programming Instructor: Farid Alizadeh IEOR 4600y Spring 2001 February 14, 2001 1 Overview In this note we review the basic properties of linear programming including the primal simplex

### Part 1. The Review of Linear Programming

In the name of God Part 1. The Review of Linear Programming 1.5. Spring 2010 Instructor: Dr. Masoud Yaghini Outline Introduction Formulation of the Dual Problem Primal-Dual Relationship Economic Interpretation

### Contents. 4.5 The(Primal)SimplexMethod NumericalExamplesoftheSimplexMethod

Contents 4 The Simplex Method for Solving LPs 149 4.1 Transformations to be Carried Out On an LP Model Before Applying the Simplex Method On It... 151 4.2 Definitions of Various Types of Basic Vectors

### Lecture 2: The Simplex method

Lecture 2 1 Linear and Combinatorial Optimization Lecture 2: The Simplex method Basic solution. The Simplex method (standardform, b>0). 1. Repetition of basic solution. 2. One step in the Simplex algorithm.

### Lecture slides by Kevin Wayne

LINEAR PROGRAMMING I a refreshing example standard form fundamental questions geometry linear algebra simplex algorithm Lecture slides by Kevin Wayne Last updated on 7/25/17 11:09 AM Linear programming

### The Simplex Algorithm

8.433 Combinatorial Optimization The Simplex Algorithm October 6, 8 Lecturer: Santosh Vempala We proved the following: Lemma (Farkas). Let A R m n, b R m. Exactly one of the following conditions is true:.

### 1 Review Session. 1.1 Lecture 2

1 Review Session Note: The following lists give an overview of the material that was covered in the lectures and sections. Your TF will go through these lists. If anything is unclear or you have questions

### MATH2070 Optimisation

MATH2070 Optimisation Linear Programming Semester 2, 2012 Lecturer: I.W. Guo Lecture slides courtesy of J.R. Wishart Review The standard Linear Programming (LP) Problem Graphical method of solving LP problem

### 2.098/6.255/ Optimization Methods Practice True/False Questions

2.098/6.255/15.093 Optimization Methods Practice True/False Questions December 11, 2009 Part I For each one of the statements below, state whether it is true or false. Include a 1-3 line supporting sentence

### Chap6 Duality Theory and Sensitivity Analysis

Chap6 Duality Theory and Sensitivity Analysis The rationale of duality theory Max 4x 1 + x 2 + 5x 3 + 3x 4 S.T. x 1 x 2 x 3 + 3x 4 1 5x 1 + x 2 + 3x 3 + 8x 4 55 x 1 + 2x 2 + 3x 3 5x 4 3 x 1 ~x 4 0 If we

### 3 The Simplex Method. 3.1 Basic Solutions

3 The Simplex Method 3.1 Basic Solutions In the LP of Example 2.3, the optimal solution happened to lie at an extreme point of the feasible set. This was not a coincidence. Consider an LP in general form,

### Lesson 27 Linear Programming; The Simplex Method

Lesson Linear Programming; The Simplex Method Math 0 April 9, 006 Setup A standard linear programming problem is to maximize the quantity c x + c x +... c n x n = c T x subject to constraints a x + a x

### MAT016: Optimization

MAT016: Optimization M.El Ghami e-mail: melghami@ii.uib.no URL: http://www.ii.uib.no/ melghami/ March 29, 2011 Outline for today The Simplex method in matrix notation Managing a production facility The

### TIM 206 Lecture 3: The Simplex Method

TIM 206 Lecture 3: The Simplex Method Kevin Ross. Scribe: Shane Brennan (2006) September 29, 2011 1 Basic Feasible Solutions Have equation Ax = b contain more columns (variables) than rows (constraints),

### In Chapters 3 and 4 we introduced linear programming

SUPPLEMENT The Simplex Method CD3 In Chapters 3 and 4 we introduced linear programming and showed how models with two variables can be solved graphically. We relied on computer programs (WINQSB, Excel,

### LINEAR PROGRAMMING I. a refreshing example standard form fundamental questions geometry linear algebra simplex algorithm

Linear programming Linear programming. Optimize a linear function subject to linear inequalities. (P) max c j x j n j= n s. t. a ij x j = b i i m j= x j 0 j n (P) max c T x s. t. Ax = b Lecture slides

### The dual simplex method with bounds

The dual simplex method with bounds Linear programming basis. Let a linear programming problem be given by min s.t. c T x Ax = b x R n, (P) where we assume A R m n to be full row rank (we will see in the

### Linear Programming. Linear Programming I. Lecture 1. Linear Programming. Linear Programming

Linear Programming Linear Programming Lecture Linear programming. Optimize a linear function subject to linear inequalities. (P) max " c j x j n j= n s. t. " a ij x j = b i # i # m j= x j 0 # j # n (P)

### The Simplex Method for Solving a Linear Program Prof. Stephen Graves

The Simplex Method for Solving a Linear Program Prof. Stephen Graves Observations from Geometry feasible region is a convex polyhedron an optimum occurs at a corner point possible algorithm - search over

### Farkas Lemma, Dual Simplex and Sensitivity Analysis

Summer 2011 Optimization I Lecture 10 Farkas Lemma, Dual Simplex and Sensitivity Analysis 1 Farkas Lemma Theorem 1. Let A R m n, b R m. Then exactly one of the following two alternatives is true: (i) x

### 3. Vector spaces 3.1 Linear dependence and independence 3.2 Basis and dimension. 5. Extreme points and basic feasible solutions

A. LINEAR ALGEBRA. CONVEX SETS 1. Matrices and vectors 1.1 Matrix operations 1.2 The rank of a matrix 2. Systems of linear equations 2.1 Basic solutions 3. Vector spaces 3.1 Linear dependence and independence

### Distributed Real-Time Control Systems. Lecture Distributed Control Linear Programming

Distributed Real-Time Control Systems Lecture 13-14 Distributed Control Linear Programming 1 Linear Programs Optimize a linear function subject to a set of linear (affine) constraints. Many problems can

### CSC Design and Analysis of Algorithms. LP Shader Electronics Example

CSC 80- Design and Analysis of Algorithms Lecture (LP) LP Shader Electronics Example The Shader Electronics Company produces two products:.eclipse, a portable touchscreen digital player; it takes hours

### CSC Design and Analysis of Algorithms. Lecture 12. Linear Programming (LP) LP Shader Electronics Example

CSC 801- Design and Analysis of Algorithms Lecture 12 Linear Programming (LP) LP Shader Electronics Example The Shader Electronics Company produces two products: 1. Eclipse, a portable touchscreen digital

### ECE 307 Techniques for Engineering Decisions

ECE 7 Techniques for Engineering Decisions Introduction to the Simple Algorithm George Gross Department of Electrical and Computer Engineering University of Illinois at Urbana-Champaign ECE 7 5 9 George

### Dual Basic Solutions. Observation 5.7. Consider LP in standard form with A 2 R m n,rank(a) =m, and dual LP:

Dual Basic Solutions Consider LP in standard form with A 2 R m n,rank(a) =m, and dual LP: Observation 5.7. AbasisB yields min c T x max p T b s.t. A x = b s.t. p T A apple c T x 0 aprimalbasicsolutiongivenbyx

### Simplex Method for LP (II)

Simplex Method for LP (II) Xiaoxi Li Wuhan University Sept. 27, 2017 (week 4) Operations Research (Li, X.) Simplex Method for LP (II) Sept. 27, 2017 (week 4) 1 / 31 Organization of this lecture Contents:

### Developing an Algorithm for LP Preamble to Section 3 (Simplex Method)

Moving from BFS to BFS Developing an Algorithm for LP Preamble to Section (Simplex Method) We consider LP given in standard form and let x 0 be a BFS. Let B ; B ; :::; B m be the columns of A corresponding

### Chapter 4 The Simplex Algorithm Part I

Chapter 4 The Simplex Algorithm Part I Based on Introduction to Mathematical Programming: Operations Research, Volume 1 4th edition, by Wayne L. Winston and Munirpallam Venkataramanan Lewis Ntaimo 1 Modeling

### Motivating examples Introduction to algorithms Simplex algorithm. On a particular example General algorithm. Duality An application to game theory

Instructor: Shengyu Zhang 1 LP Motivating examples Introduction to algorithms Simplex algorithm On a particular example General algorithm Duality An application to game theory 2 Example 1: profit maximization

### (P ) Minimize 4x 1 + 6x 2 + 5x 3 s.t. 2x 1 3x 3 3 3x 2 2x 3 6

The exam is three hours long and consists of 4 exercises. The exam is graded on a scale 0-25 points, and the points assigned to each question are indicated in parenthesis within the text. Problem 1 Consider

### 4.5 Simplex method. LP in standard form: min z = c T x s.t. Ax = b

4.5 Simplex method LP in standard form: min z = c T x s.t. Ax = b x 0 George Dantzig (1914-2005) Examine a sequence of basic feasible solutions with non increasing objective function values until an optimal

### Linear Programming: Simplex

Linear Programming: Simplex Stephen J. Wright 1 2 Computer Sciences Department, University of Wisconsin-Madison. IMA, August 2016 Stephen Wright (UW-Madison) Linear Programming: Simplex IMA, August 2016

### Slack Variable. Max Z= 3x 1 + 4x 2 + 5X 3. Subject to: X 1 + X 2 + X x 1 + 4x 2 + X X 1 + X 2 + 4X 3 10 X 1 0, X 2 0, X 3 0

Simplex Method Slack Variable Max Z= 3x 1 + 4x 2 + 5X 3 Subject to: X 1 + X 2 + X 3 20 3x 1 + 4x 2 + X 3 15 2X 1 + X 2 + 4X 3 10 X 1 0, X 2 0, X 3 0 Standard Form Max Z= 3x 1 +4x 2 +5X 3 + 0S 1 + 0S 2

### The Primal-Dual Algorithm P&S Chapter 5 Last Revised October 30, 2006

The Primal-Dual Algorithm P&S Chapter 5 Last Revised October 30, 2006 1 Simplex solves LP by starting at a Basic Feasible Solution (BFS) and moving from BFS to BFS, always improving the objective function,

### MATH 4211/6211 Optimization Linear Programming

MATH 4211/6211 Optimization Linear Programming Xiaojing Ye Department of Mathematics & Statistics Georgia State University Xiaojing Ye, Math & Stat, Georgia State University 0 The standard form of a Linear

### Chapter 1 Linear Programming. Paragraph 5 Duality

Chapter 1 Linear Programming Paragraph 5 Duality What we did so far We developed the 2-Phase Simplex Algorithm: Hop (reasonably) from basic solution (bs) to bs until you find a basic feasible solution

### AM 121: Intro to Optimization Models and Methods Fall 2018

AM 121: Intro to Optimization Models and Methods Fall 2018 Lecture 5: The Simplex Method Yiling Chen Harvard SEAS Lesson Plan This lecture: Moving towards an algorithm for solving LPs Tableau. Adjacent

### Linear Programming Duality

Summer 2011 Optimization I Lecture 8 1 Duality recap Linear Programming Duality We motivated the dual of a linear program by thinking about the best possible lower bound on the optimal value we can achieve

### March 13, Duality 3

15.53 March 13, 27 Duality 3 There are concepts much more difficult to grasp than duality in linear programming. -- Jim Orlin The concept [of nonduality], often described in English as "nondualism," is

### 3. THE SIMPLEX ALGORITHM

Optimization. THE SIMPLEX ALGORITHM DPK Easter Term. Introduction We know that, if a linear programming problem has a finite optimal solution, it has an optimal solution at a basic feasible solution (b.f.s.).

### Lecture 10: Linear programming duality and sensitivity 0-0

Lecture 10: Linear programming duality and sensitivity 0-0 The canonical primal dual pair 1 A R m n, b R m, and c R n maximize z = c T x (1) subject to Ax b, x 0 n and minimize w = b T y (2) subject to

### Introduction to optimization

Introduction to optimization Geir Dahl CMA, Dept. of Mathematics and Dept. of Informatics University of Oslo 1 / 24 The plan 1. The basic concepts 2. Some useful tools (linear programming = linear optimization)

### CO350 Linear Programming Chapter 6: The Simplex Method

CO350 Linear Programming Chapter 6: The Simplex Method 8th June 2005 Chapter 6: The Simplex Method 1 Minimization Problem ( 6.5) We can solve minimization problems by transforming it into a maximization

### F 1 F 2 Daily Requirement Cost N N N

Chapter 5 DUALITY 5. The Dual Problems Every linear programming problem has associated with it another linear programming problem and that the two problems have such a close relationship that whenever

### Lecture 11: Post-Optimal Analysis. September 23, 2009

Lecture : Post-Optimal Analysis September 23, 2009 Today Lecture Dual-Simplex Algorithm Post-Optimal Analysis Chapters 4.4 and 4.5. IE 30/GE 330 Lecture Dual Simplex Method The dual simplex method will

### OPTIMISATION 3: NOTES ON THE SIMPLEX ALGORITHM

OPTIMISATION 3: NOTES ON THE SIMPLEX ALGORITHM Abstract These notes give a summary of the essential ideas and results It is not a complete account; see Winston Chapters 4, 5 and 6 The conventions and notation

### Lecture 5 Simplex Method. September 2, 2009

Simplex Method September 2, 2009 Outline: Lecture 5 Re-cap blind search Simplex method in steps Simplex tableau Operations Research Methods 1 Determining an optimal solution by exhaustive search Lecture

### Simplex method(s) for solving LPs in standard form

Simplex method: outline I The Simplex Method is a family of algorithms for solving LPs in standard form (and their duals) I Goal: identify an optimal basis, as in Definition 3.3 I Versions we will consider:

### The augmented form of this LP is the following linear system of equations:

1 Consider the following LP given in standard form: max z = 5 x_1 + 2 x_2 Subject to 3 x_1 + 2 x_2 2400 x_2 800 2 x_1 1200 x_1, x_2 >= 0 The augmented form of this LP is the following linear system of

### 4.5 Simplex method. min z = c T x s.v. Ax = b. LP in standard form

4.5 Simplex method min z = c T x s.v. Ax = b x 0 LP in standard form Examine a sequence of basic feasible solutions with non increasing objective function value until an optimal solution is reached or

### CO350 Linear Programming Chapter 6: The Simplex Method

CO50 Linear Programming Chapter 6: The Simplex Method rd June 2005 Chapter 6: The Simplex Method 1 Recap Suppose A is an m-by-n matrix with rank m. max. c T x (P ) s.t. Ax = b x 0 On Wednesday, we learned

### Linear Programming Duality P&S Chapter 3 Last Revised Nov 1, 2004

Linear Programming Duality P&S Chapter 3 Last Revised Nov 1, 2004 1 In this section we lean about duality, which is another way to approach linear programming. In particular, we will see: How to define

### The Graphical Method & Algebraic Technique for Solving LP s. Métodos Cuantitativos M. En C. Eduardo Bustos Farías 1

The Graphical Method & Algebraic Technique for Solving LP s Métodos Cuantitativos M. En C. Eduardo Bustos Farías The Graphical Method for Solving LP s If LP models have only two variables, they can be

### Chapter 1: Linear Programming

Chapter 1: Linear Programming Math 368 c Copyright 2013 R Clark Robinson May 22, 2013 Chapter 1: Linear Programming 1 Max and Min For f : D R n R, f (D) = {f (x) : x D } is set of attainable values of

### CS Algorithms and Complexity

CS 50 - Algorithms and Complexity Linear Programming, the Simplex Method, and Hard Problems Sean Anderson 2/15/18 Portland State University Table of contents 1. The Simplex Method 2. The Graph Problem

### Linear Programming and the Simplex method

Linear Programming and the Simplex method Harald Enzinger, Michael Rath Signal Processing and Speech Communication Laboratory Jan 9, 2012 Harald Enzinger, Michael Rath Jan 9, 2012 page 1/37 Outline Introduction

### Discrete Optimization

Prof. Friedrich Eisenbrand Martin Niemeier Due Date: April 15, 2010 Discussions: March 25, April 01 Discrete Optimization Spring 2010 s 3 You can hand in written solutions for up to two of the exercises

### Math Models of OR: Sensitivity Analysis

Math Models of OR: Sensitivity Analysis John E. Mitchell Department of Mathematical Sciences RPI, Troy, NY 8 USA October 8 Mitchell Sensitivity Analysis / 9 Optimal tableau and pivot matrix Outline Optimal

### LINEAR PROGRAMMING 2. In many business and policy making situations the following type of problem is encountered:

LINEAR PROGRAMMING 2 In many business and policy making situations the following type of problem is encountered: Maximise an objective subject to (in)equality constraints. Mathematical programming provides

### END3033 Operations Research I Sensitivity Analysis & Duality. to accompany Operations Research: Applications and Algorithms Fatih Cavdur

END3033 Operations Research I Sensitivity Analysis & Duality to accompany Operations Research: Applications and Algorithms Fatih Cavdur Introduction Consider the following problem where x 1 and x 2 corresponds

### IE 400: Principles of Engineering Management. Simplex Method Continued

IE 400: Principles of Engineering Management Simplex Method Continued 1 Agenda Simplex for min problems Alternative optimal solutions Unboundedness Degeneracy Big M method Two phase method 2 Simplex for

### Lecture: Algorithms for LP, SOCP and SDP

1/53 Lecture: Algorithms for LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2018.html wenzw@pku.edu.cn Acknowledgement:

### LP Duality: outline. Duality theory for Linear Programming. alternatives. optimization I Idea: polyhedra

LP Duality: outline I Motivation and definition of a dual LP I Weak duality I Separating hyperplane theorem and theorems of the alternatives I Strong duality and complementary slackness I Using duality

### Lecture 2: The Simplex method. 1. Repetition of the geometrical simplex method. 2. Linear programming problems on standard form.

Lecture 2: The Simplex method. Repetition of the geometrical simplex method. 2. Linear programming problems on standard form. 3. The Simplex algorithm. 4. How to find an initial basic solution. Lecture

### The use of shadow price is an example of sensitivity analysis. Duality theory can be applied to do other kind of sensitivity analysis:

Sensitivity analysis The use of shadow price is an example of sensitivity analysis. Duality theory can be applied to do other kind of sensitivity analysis: Changing the coefficient of a nonbasic variable

### THE UNIVERSITY OF HONG KONG DEPARTMENT OF MATHEMATICS. Operations Research I

LN/MATH2901/CKC/MS/2008-09 THE UNIVERSITY OF HONG KONG DEPARTMENT OF MATHEMATICS Operations Research I Definition (Linear Programming) A linear programming (LP) problem is characterized by linear functions

### 21. Solve the LP given in Exercise 19 using the big-m method discussed in Exercise 20.

Extra Problems for Chapter 3. Linear Programming Methods 20. (Big-M Method) An alternative to the two-phase method of finding an initial basic feasible solution by minimizing the sum of the artificial

### The Simplex Algorithm and Goal Programming

The Simplex Algorithm and Goal Programming In Chapter 3, we saw how to solve two-variable linear programming problems graphically. Unfortunately, most real-life LPs have many variables, so a method is

### Special cases of linear programming

Special cases of linear programming Infeasible solution Multiple solution (infinitely many solution) Unbounded solution Degenerated solution Notes on the Simplex tableau 1. The intersection of any basic

### Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

Dr. Said Bourazza Department of Mathematics Jazan University 1 P a g e Contents: Chapter 0: Modelization 3 Chapter1: Graphical Methods 7 Chapter2: Simplex method 13 Chapter3: Duality 36 Chapter4: Transportation

### ORF 522. Linear Programming and Convex Analysis

ORF 522 Linear Programming and Convex Analysis The Simplex Method Marco Cuturi Princeton ORF-522 1 Reminder: Basic Feasible Solutions, Extreme points, Optima Some important theorems last time for standard

### Operations Research Lecture 2: Linear Programming Simplex Method

Operations Research Lecture 2: Linear Programming Simplex Method Notes taken by Kaiquan Xu@Business School, Nanjing University Mar 10th 2016 1 Geometry of LP 1.1 Graphical Representation and Solution Example

### Lectures 6, 7 and part of 8

Lectures 6, 7 and part of 8 Uriel Feige April 26, May 3, May 10, 2015 1 Linear programming duality 1.1 The diet problem revisited Recall the diet problem from Lecture 1. There are n foods, m nutrients,

### Example Bases and Basic Feasible Solutions 63 Let q = >: ; > and M = >: ;2 > and consider the LCP (q M). The class of ; ;2 complementary cones

Chapter 2 THE COMPLEMENTARY PIVOT ALGORITHM AND ITS EXTENSION TO FIXED POINT COMPUTING LCPs of order 2 can be solved by drawing all the complementary cones in the q q 2 - plane as discussed in Chapter.

### Introduction. Very efficient solution procedure: simplex method.

LINEAR PROGRAMMING Introduction Development of linear programming was among the most important scientific advances of mid 20th cent. Most common type of applications: allocate limited resources to competing