# 1. Algebraic and geometric treatments Consider an LP problem in the standard form. x 0. Solutions to the system of linear equations

Size: px
Start display at page:

Download "1. Algebraic and geometric treatments Consider an LP problem in the standard form. x 0. Solutions to the system of linear equations"

## Transcription

1 The Simplex Method Most textbooks in mathematical optimization, especially linear programming, deal with the simplex method. In this note we study the simplex method. It requires basically elementary linear algebra, but many calculations are in block matrix form. We do not treat the degenerate case as we said before, but we will make some remarks in this case.. Algebraic and geometric treatments Consider an LP problem in the standard form min s.t. Solutions to the system of linear equations c t x Ax = b x 0 () Ax = b correspond to an hypersurface in R n and the positivity requirement x 0. If the matrix A is an n m-matrix of full rank, then the hyperplane {x Ax = b} has dimension n m. We shall assume the full rank in the sequel. If you are in a feasible point, you perhaps would like to go in the negative gradient direction, that is, c, for minimizing the objective function. However, this direction normally lies outside the hypersurface {x Ax = b}. Naturally, there are two ways to deal with the problem. (Think why it is natural.) To project the direction c onto the hyperplane {x Ax = b} and go along this projected direction To solve the m variables by () and express them in terms of the rest of n m variables. The simplex method uses the second alternative. Without loss of generality, we assume that the first m columns of the matrix A are linearly independent. Then () can be written as () A B x B + A N x N = b where x B is a vector containing the first m components of x, and x N the rest of n m. The matrix A B is formed by the first m columns of A and A N the rest of columns. The sub-indexes B, N stand for basic and nonbasic respectively. They can also be interpreted as multiindex sets. Then B consists of those indexes where the variables are included in x B in the order they appear in the problem. For instance, B = (, 4, 3), then x B = (x, x 4, x 3 ), and A B = (a, a 4, a 3 ). Notice that A B is an m m matrix. By our full rank assumption, it has full rank, and hence, inevitable. Consequently x B can be solved uniquely. More precisely, by pre-multiplying A B we obtain x B + A B A N x N = A B b, and then x B = A B b A B A N x N. We know by elementary linear algebra that x N can be chosen freely. If we set x N = 0, we have a solution to (). This is by definition a basic solution. The matrix A B is called the corresponding base matrix. The components in x B are basic variables

2 and x N are nonbasic variables. Clearly, the basic solution x B = A B b, x N = 0 is feasible if x 0, that is, x B = A B b 0. As we showed in the previous section, a basic feasible solution lies at a corner of the feasible set, i.e. an extreme point of the convex feasible set. Now we can compute the objective function at this basic solution. We partition the vector c as c t = (c t B, ct N ) z =c t Bx B + c t Nx N = c t BA B b ct BA B A Nx N + c t N x N =(c t N c t BA B A N)x N + c t BA B b =: ct Nx N + z At the basic solution x N = 0 and z = z. The quantity c t = c t N ct B A B A N is called reduced cost. It tells us how much the objective function varies as x N varies. Observe that c N has two parts: One is c t N which represents a direct contribution to the objective function from x N, and the other, c t B A B A N, an indirect contribution from c N through x B, that varies as x N varies. Now denote ĀN = A B A N and b = A B b we have the following transformed problem (3) min s.t. z = c t x N + z = z + c j x j j N x B = b ĀNx N x B, x N 0. The corresponding basic solution is (x t B, xt N ) = ( b t, 0). Assume that this basic solution is feasible, that is b 0. If some components in c N, e.g. c j, is negative, we see that it is meaningful to increase x j from 0. If only x j increases, then x B varies according to (4) x B = b ā j x j where ā j is the column corresponding to x j in the matrix ĀN. This solution is feasible as long as x B 0. Equation (4) shows that Note the following facts: (x B ) i = b i ā ij x j j N. If ā ij 0, the corresponding variable (x B ) i increases or stays constant as x j increase. The condition (x B ) i 0 has no effect in this case, because (x B ) i will never become 0. If, on the other hand, ā ij > 0, the corresponding variable (x B ) i decreases and it becomes 0 when x j = b i ā ij. This shows that the solution is feasible only if x j min i { b i ā ij ā ij > 0}. Note that, if x j = min i { b i ā ij ā ij > 0} (we call this the smallest replacement quantity), then some of basic variable become 0, that is, that/those for which the minimum with respect to i is attained and we reach another extreme point. Now it is the time to change coordinate system in the hypersurface {x Ax = b}. The variable x j now is greater than 0 and it becomes basic variable. The variable which became 0 is a nonbasic variable. This is the essence of the simplex method. In short, the simplex method is based on the fact that a given feasible basic solution

3 can be improved if some component in c N is negative. It is natural to ask: What happens if c N 0? Is the solution optimal then? The answers are affirmative. Theorem.. Assume that b 0 and c N 0 in the transformed LP problem (3). Then the corresponding basic solution is optimal. Proof. We show this by relaxation. Since b 0, the basic solution (x t B, xt N ) = ( b t, 0) is feasible. If we relax the condition x B = b ĀN x N from the transformed LP problem (3), we get a relaxed problem min z = c t Nx N + z = j N c j x j + z s.t. x B, x N 0 Because c j 0 and we can minimize for each variable x j separately, it is obvious that (x t B, xt N ) = ( b t, 0) is optimal to the relaxed problem above, and hence to the transformed problem (3), since the solution is feasible. It gives automatically same value of objective function, for we have the same objective function. We summarize the basic structure of the simplex to close this section: (i) START at a feasible basic solution. (ii) Transform the problem to the form of (3) by determining the basic variables. (iii) If all c j 0, STOP. We are at optimum. (iv) If some c j < 0, increase the corresponding x j until (x B ) i = 0. We are now at a new basic solution. GO TO Step.. Tableau form For hand calculation of the simplex method, we usually put all data in a tableau. Now we do an example of the simplex method step by step through the example given in the previous section. We repeat it here. max 80x + 60x s.t. x + x 00 x + x 50 5x + 0x 800 x, x 0 The feasible region is depicted in the previous section. Number the corners from the origin anticlockwise we have corner : (0, 0), corner : (75, 0), corner 3: (50, 50), corner 4: (40, 60), corner 5: (0, 80). First transform the problem to minimizing 80x 60x. Then, introduce slack variables x 3, x 4, x 5 0. We put the equations and the objective function in the following matrix form: x x x 3 x 4 x 5 b

4 [ ] A b The tableau is of the form c t. This is called the simplex tableau. 0 This tableau is already in a feasible basic solution form. The feasible basic solution is x = 0, x = 0, x 3 = 00, x 4 = 50, x 5 = 800. It corresponds to corner. Now we have to decide which variable enters the basic variable, and which variable departs from the basic variable. In our notation, N = {, } and B = {3, 4, 5}. So c t N = ( 80, 60). The last row in the tableau shows that there are negative costs c and c. It seems natural to choose the first component x because it corresponds the most negative one in the reduced cost. To decide the departing variable we compute 00/, 50/, 800/5. Then the second quantity is the minimum of these three. Hence the variable x 4 departs B. Now B = {3,, 5}, and N = {4, }. The new tableau is x x x 3 x 4 x 5 b The new basic solution is x = 75, x = 0, x 3 = 5, x 4 = 0, x 5 = 45. It corresponds to corner. There is one negative component in c N. That is c. So the variable x enters the index set B. From min{5/, 75/ 5, 45/ } = 50, the variable x 3 departs from the basic variables. So the new index sets are B = {,, 5} and N = {4, 3}. The next tableau is x x x 3 x 4 x 5 b The new basic solution is x = x = x 5 = 50, x 3 = x 4 = 0, that is, corner 3. Remember that we obtained the same optimal solution geometrically. In this problem the solution is unique and bounded. In general, boundedness and uniqueness may be violated. Moreover if there are two optimal feasible solutions, then there are infinitely many optimal solutions. (Prove it as an exercise.) This happens if all basic variables are positive and in the optimal tableau a zero occurs in the objective row in a column corresponding to a nonbasic variable and including a positive entry. Example.. Example of infinitely many solutions. min z = 40x 00x s.t. 0x + 5x 500 4x + 0x 000 x + 3x 900 x, x 0 Now we turn to discussing when a solution is unbounded. Remember that we stopped increasing a nonbasic variable x j with negative reduced cost c j when some basic variable became zero. If all ā ij 0 then no basic variable will decrease as x j increases. No matter how much x j increases, we have feasible solutions. Since 4

5 the objective function decreases as x j increases, the objective function will decrease unboundedly. We say that the problem has an unbounded solution. Example.. Example of unbounded solution. min z = 3x x s.t. x x 3x x 6 x, x 0 Compute these two examples as exercise! In conclusion, we have the following algorithm: the simplex method. Step. Obtain the initial basic solution by setting each slack variable equal to the corresponding resource and setting the other variables equal to zero. Step. Choose the entering variable from among those having negative reduced cost. Step 3. Choose the departing variable to be the basic variable in row producing the smallest replacement quantity. Step 4. There is no lower bound on the objective function when one column has entirely nonpositive coefficients above a negative component of the reduced cost. Step 5. The solution is optimal when all entries in the objective row are nonnegative. Step 6. If all basic variables are positive, an optimal solution is not unique if in the optimal tableau a zero occurs in the objective row in a column corresponding to a nonbasic variable and including a positive entry. Finally we make an important remark. We have seen that the simplex method stops in two cases: (i) c 0. In this case, we have an optimal solution. (ii) For some c j < 0 all ā ij 0. In this case we have unbounded solution. The question is whether or not we must have these two cases. We take a close look now. In every iteration of the simplex method, the objective function decreases or stays constant: If x j is the entering variable then x j increases from 0 to x j = min{ b i ā ij a ij > 0} 0. The change in the objective function is therefore z = c j x j = 0 only if x j = min{ b i ā ij a ij > 0} = 0, which means that a basic variable is zero in a basic solution. In an earlier iteration there are two basic variables at the same time as the entering variable increases. However, only one of these variables has departed from the basis. This situation is called degenerate case and means that more than n m hyperplanes of type (x B ) i = 0 meet at the same point in the hyersurface {x Ax = b}. One can argue that it is a rare situation. Nevertheless it does happen to all LP problems. If the degeneration never happens the objective function strictly decreases in each iteration. Then we can never come back to an extreme point we have already visited, because it will give the same value of the objective function. Moreover, we know that it has only finitely many basic solutions. So if degeneration never occurs, the simplex method must stop after finitely many steps, in either case (i) or case (ii). At degeneration the simplex method will go on as a circle. It is called cyclic. In practice, the cyclic situation seldom will be a case. We will do not discuss this in details. See for example our textbook for further discussion. 5

6 Next, even though the simplex method is finite, it is of no interest in practice, because a practically small problem such as 00 constraints and 00 variables can have up to 0 30 feasible basic solutions. If the simplex method were to pass all these basic solutions in such a problem and uses only 0 9 seconds per basic solution, then it would take 0 seconds which is about years, that is, much more than the length of the earth s life. However, the simplex method in practice takes much less time. Nowadays the problem of the abovementioned size takes just several seconds on a PC. One usually counts as follows: The number of iterations increases approximately linearly by the number of constraints, while work on each iteration increases as square of the number of constraints. This can be used as a practical complexity to the simplex method. For implementation the simplex method the following flow chart can be used. START Insert slack variables Generate initial solution (rule ) Optimal? (rule 5) YES Unique? (rule 6) YES NO NO STOP Obtain next solution by pivoting Choose the entering variable (rule ) Deter,ine other solutions Determine replacement quantities Determine the departing variable (rule 3) YES replacement quantity found? NO Solutions unbounded (rule 4) 3. The two phase simplex method In the preceding section we assumed implicitly that the initial simplex tableau is ready for further computing, i.e. we already have a basic feasible solution. It is clear that one needs to find it. This is the task of this section. It turns out that the problem of finding such a feasible basic solution can be solved by the simplex method! 6

7 Now consider the system of equations (5) Ax = b, x 0. We assume that b 0. (Note that this is always possible.) Introduce now the so-called artificial variables y 0. In other words, we can think of this as measuring the error between the left and right hand sides in the equations. Then we have another system of linear equations with all variables nonnegative: (6) Ax + y = b, x 0, y 0 In this second equation, it is easy to find a feasible basic solution, that is y = b, x = 0. If we want to have a feasible solution to the first equation we have to drive y to zero. Then we can solve the following Phase I problem. m min (7) s.t. i= y i Ax + y = b x, y 0 It is still an LP problem, and therefore it can be solved by the simplex method. Theorem 3.. The LP problem (7) always have a finite positive optimal value. This optimal value is given by the simplex method. If the optimal value is greater than 0, then the system (5) has no feasible solution. If the optimal value is zero then the system (5) has a feasible solution, and the simplex method gives the desired result. Proof. Assume that x is a feasible solution to (5). Then, ( x t, 0) is a feasible basic solution to (6). Thus the optimal value of (7) is equal to 0. If the optimal value of (7) is greater than 0, then (5) cannot have a feasible solution. Conversely, if the optimal value of (7) is 0, then the simplex method will give the optimal basic solution with value 0. Since y i = 0 and y i 0, it must hold that all y i = 0 and we obtain a feasible solution of (5). Example 3.. min 3x + x s.t. x + x x + x 3 x, x 0 From the figure we see that the optimum is attained at point 6. First we introduce surplus variables x 3 and x 4 to obtain equality constraints. So we get the following tableau. x x x 3 x 4 b From this we do not see any natural initial basis. Thus we introduce artificial variables y and y. In Phase I we shall minimize y + y. This gives the next tableau. 7

8 x x x x x 3 x 4 y y b The last row in this tableau is the objective function of Phase I while the third row is the actual objective row. As said before, it is suitable to choose y, y as basic variables. Eliminate the s in the corresponding columns yields the initial simplex tableau. x x x 3 x 4 y y b Thus y =, y = 3, x = x = x 3 = x 4 = 0 form a basic solution, corner. In this tableau c and c are negative. We let x be entering variable. Next min{/, 3/} =, which meets at first row so y is a departing variable. Now we pivot component (, ) and get x x x 3 x 4 y y b The corresponding basic solution is x =, y = the rest rest variables are 0, which is corner. Now c, c 3 are negative. Choose x to enter the basis. According to min{/, /} = which meets at row. So y departs. Pivoting (, )-component gives a new tableau 8

9 x x x 3 x 4 y y b The basic solution now is x =, x =, and the rest is 0. So corner 4 is reached. Now y = y = 0 and the artificial objective function is zero. Thus, Phase I is finished and we have found a basic feasible solution to solve the original LP by simplex. We can drop the last row to continue Phase I. In the third row, we see that c 4 < 0 and x 4 enters the basis. Now there is only one choice of the departing variable, because (, 4)-component is negative. Pivoting (, 4)-component gives x x x 3 x 4 y y b The corresponding basic solution is x =, x 4 = and the rest is 0, that is corner 6. Now all reduced costs are nonnegative, so we are at optimum. The whole structure of the simplex method is where Step A. From the artificial objective function as the sum of the artificial variables, obtain the initial feasible basic solution by setting each slack variable and each artificial variable equal to the corresponding constant on the right hand side of the constraint. Step B. There is no feasible solution if the sum of the artificial variables is greater than zero. 4. Duality theory Consider the LP problem in standard form. The dual problem is max b t u s.t A t u c. Note that there is no sign restriction on the variables u. We know that the variables u is Lagrange multipliers. So it is natural to see if the simplex multipliers have something to do with the dual problem. Assume the primal problem is solved until the optimum is attained. Then x B is basic variable. The optimality condition is c N 0, that is c t N πt N 0 or N [ t π ] c N. However, the constraints on the dual B variable u is c A t u = [B, N] t t u = N t u, c B B t u, c N N t u. Therefore, the simplex multipliers satisfy half of the dual constraints, if the optimality condition is satisfied. The other half is c B B t )u is satisfied automatically with equality, since π = (c t B B ) t = (B t ) c B and B t π = c B π is a feasible dual solution. The above argument means that π is optimal if it gives the same objective function value as the primal solution does (according to weak duality). The primal objective value is z = c t B A B b while π yields the dual objective value bt π = b t (A t B ) c B = c t B A B b = z. Therefore, we have proved: Theorem. Ass. x B = A B b, x N = 0 is simplex-optimal (i.e. c 0) basic solution to an LP in standard form. Then the corresponding simplex multiplier, c t B A B, is optimal dual solution. Moreover, the primal and dual optimal values are equal. How primal and dual relation looks like in other cases. If an LP problem is solved 9

10 START Insert slack, surplus & artificial variables (rule 0) Generating initial solution (rule A) Apply simplex method using the artificial objective function NO All artificial variables zero? YES No feasible solution (rule B) Drop nonbasic artificial variables and objective function Apply simplex method to actual objective function STOP by the simplex method, according to above discussion, the following cases occur: Phase I gives optimal value greater than 0, that is there is no feasible solution. ( ) Phase I gives optimal value 0 and Phase II gives a bounded optimum. (B) Phase I gives optimal value 0 and Phase II gives unbounded solution. ( ) If P occurs then the dual cannot have a feasible solution according to weak duality. So the dual has no solution, D. Similarly, D implies P. By the theorem P B implies D B and vise verse (due to the symmetry). The only case we have not considered is the relation between P and D. Ex. min x, s.t. x x x + x 0, x, x 0. Its dual: min u s.t. u u 0, u + u u, u 0. None of these problems has solutions. We conclude by the following tableau. (The cross stands for impossible situation.) 0

11 D D B D P X P B X X P X X Complementary slackness: Let ( x, ȳ) be a pair of feasible solutions to the primal P resp. D. Then x and ȳ are resp. optimal to P and D iff (c A t ȳ) j x j = 0 for j =,..., n. Exerceises. () Solve the following LP problem: min 7x x + 3x 3 x x 3 3x + x x 3 s.t. x + x + x 3 x, x, x 3 0 () Consider the following LP problem: max x + x + x 3 x + x + x 3 5 x + x x 3 4 s.t. x x x, x, x 3 0 Find the optimal value and an optimal point. points? Give all optimal points if yes. Are there other optimal Yishao Zhou//December, 04

### Yinyu Ye, MS&E, Stanford MS&E310 Lecture Note #06. The Simplex Method

The Simplex Method Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye (LY, Chapters 2.3-2.5, 3.1-3.4) 1 Geometry of Linear

### Lecture 2: The Simplex method

Lecture 2 1 Linear and Combinatorial Optimization Lecture 2: The Simplex method Basic solution. The Simplex method (standardform, b>0). 1. Repetition of basic solution. 2. One step in the Simplex algorithm.

### Standard Form An LP is in standard form when: All variables are non-negativenegative All constraints are equalities Putting an LP formulation into sta

Chapter 4 Linear Programming: The Simplex Method An Overview of the Simplex Method Standard Form Tableau Form Setting Up the Initial Simplex Tableau Improving the Solution Calculating the Next Tableau

### Chap6 Duality Theory and Sensitivity Analysis

Chap6 Duality Theory and Sensitivity Analysis The rationale of duality theory Max 4x 1 + x 2 + 5x 3 + 3x 4 S.T. x 1 x 2 x 3 + 3x 4 1 5x 1 + x 2 + 3x 3 + 8x 4 55 x 1 + 2x 2 + 3x 3 5x 4 3 x 1 ~x 4 0 If we

### Summary of the simplex method

MVE165/MMG630, The simplex method; degeneracy; unbounded solutions; infeasibility; starting solutions; duality; interpretation Ann-Brith Strömberg 2012 03 16 Summary of the simplex method Optimality condition:

### Review Solutions, Exam 2, Operations Research

Review Solutions, Exam 2, Operations Research 1. Prove the weak duality theorem: For any x feasible for the primal and y feasible for the dual, then... HINT: Consider the quantity y T Ax. SOLUTION: To

### 3 The Simplex Method. 3.1 Basic Solutions

3 The Simplex Method 3.1 Basic Solutions In the LP of Example 2.3, the optimal solution happened to lie at an extreme point of the feasible set. This was not a coincidence. Consider an LP in general form,

### Ω R n is called the constraint set or feasible set. x 1

1 Chapter 5 Linear Programming (LP) General constrained optimization problem: minimize subject to f(x) x Ω Ω R n is called the constraint set or feasible set. any point x Ω is called a feasible point We

### The dual simplex method with bounds

The dual simplex method with bounds Linear programming basis. Let a linear programming problem be given by min s.t. c T x Ax = b x R n, (P) where we assume A R m n to be full row rank (we will see in the

### A Review of Linear Programming

A Review of Linear Programming Instructor: Farid Alizadeh IEOR 4600y Spring 2001 February 14, 2001 1 Overview In this note we review the basic properties of linear programming including the primal simplex

### Slack Variable. Max Z= 3x 1 + 4x 2 + 5X 3. Subject to: X 1 + X 2 + X x 1 + 4x 2 + X X 1 + X 2 + 4X 3 10 X 1 0, X 2 0, X 3 0

Simplex Method Slack Variable Max Z= 3x 1 + 4x 2 + 5X 3 Subject to: X 1 + X 2 + X 3 20 3x 1 + 4x 2 + X 3 15 2X 1 + X 2 + 4X 3 10 X 1 0, X 2 0, X 3 0 Standard Form Max Z= 3x 1 +4x 2 +5X 3 + 0S 1 + 0S 2

### Dr. Maddah ENMG 500 Engineering Management I 10/21/07

Dr. Maddah ENMG 500 Engineering Management I 10/21/07 Computational Procedure of the Simplex Method The optimal solution of a general LP problem is obtained in the following steps: Step 1. Express the

### Farkas Lemma, Dual Simplex and Sensitivity Analysis

Summer 2011 Optimization I Lecture 10 Farkas Lemma, Dual Simplex and Sensitivity Analysis 1 Farkas Lemma Theorem 1. Let A R m n, b R m. Then exactly one of the following two alternatives is true: (i) x

### Linear Programming Redux

Linear Programming Redux Jim Bremer May 12, 2008 The purpose of these notes is to review the basics of linear programming and the simplex method in a clear, concise, and comprehensive way. The book contains

### Part 1. The Review of Linear Programming

In the name of God Part 1. The Review of Linear Programming 1.5. Spring 2010 Instructor: Dr. Masoud Yaghini Outline Introduction Formulation of the Dual Problem Primal-Dual Relationship Economic Interpretation

### The Simplex Algorithm

8.433 Combinatorial Optimization The Simplex Algorithm October 6, 8 Lecturer: Santosh Vempala We proved the following: Lemma (Farkas). Let A R m n, b R m. Exactly one of the following conditions is true:.

### 3. THE SIMPLEX ALGORITHM

Optimization. THE SIMPLEX ALGORITHM DPK Easter Term. Introduction We know that, if a linear programming problem has a finite optimal solution, it has an optimal solution at a basic feasible solution (b.f.s.).

### Introduction to Mathematical Programming IE406. Lecture 10. Dr. Ted Ralphs

Introduction to Mathematical Programming IE406 Lecture 10 Dr. Ted Ralphs IE406 Lecture 10 1 Reading for This Lecture Bertsimas 4.1-4.3 IE406 Lecture 10 2 Duality Theory: Motivation Consider the following

### CSCI 1951-G Optimization Methods in Finance Part 01: Linear Programming

CSCI 1951-G Optimization Methods in Finance Part 01: Linear Programming January 26, 2018 1 / 38 Liability/asset cash-flow matching problem Recall the formulation of the problem: max w c 1 + p 1 e 1 = 150

### The use of shadow price is an example of sensitivity analysis. Duality theory can be applied to do other kind of sensitivity analysis:

Sensitivity analysis The use of shadow price is an example of sensitivity analysis. Duality theory can be applied to do other kind of sensitivity analysis: Changing the coefficient of a nonbasic variable

### Simplex Method for LP (II)

Simplex Method for LP (II) Xiaoxi Li Wuhan University Sept. 27, 2017 (week 4) Operations Research (Li, X.) Simplex Method for LP (II) Sept. 27, 2017 (week 4) 1 / 31 Organization of this lecture Contents:

### Note 3: LP Duality. If the primal problem (P) in the canonical form is min Z = n (1) then the dual problem (D) in the canonical form is max W = m (2)

Note 3: LP Duality If the primal problem (P) in the canonical form is min Z = n j=1 c j x j s.t. nj=1 a ij x j b i i = 1, 2,..., m (1) x j 0 j = 1, 2,..., n, then the dual problem (D) in the canonical

### 9.1 Linear Programs in canonical form

9.1 Linear Programs in canonical form LP in standard form: max (LP) s.t. where b i R, i = 1,..., m z = j c jx j j a ijx j b i i = 1,..., m x j 0 j = 1,..., n But the Simplex method works only on systems

### 21. Solve the LP given in Exercise 19 using the big-m method discussed in Exercise 20.

Extra Problems for Chapter 3. Linear Programming Methods 20. (Big-M Method) An alternative to the two-phase method of finding an initial basic feasible solution by minimizing the sum of the artificial

### OPTIMISATION 3: NOTES ON THE SIMPLEX ALGORITHM

OPTIMISATION 3: NOTES ON THE SIMPLEX ALGORITHM Abstract These notes give a summary of the essential ideas and results It is not a complete account; see Winston Chapters 4, 5 and 6 The conventions and notation

### 3. Vector spaces 3.1 Linear dependence and independence 3.2 Basis and dimension. 5. Extreme points and basic feasible solutions

A. LINEAR ALGEBRA. CONVEX SETS 1. Matrices and vectors 1.1 Matrix operations 1.2 The rank of a matrix 2. Systems of linear equations 2.1 Basic solutions 3. Vector spaces 3.1 Linear dependence and independence

### MAT016: Optimization

MAT016: Optimization M.El Ghami e-mail: melghami@ii.uib.no URL: http://www.ii.uib.no/ melghami/ March 29, 2011 Outline for today The Simplex method in matrix notation Managing a production facility The

### MATH 4211/6211 Optimization Linear Programming

MATH 4211/6211 Optimization Linear Programming Xiaojing Ye Department of Mathematics & Statistics Georgia State University Xiaojing Ye, Math & Stat, Georgia State University 0 The standard form of a Linear

### Math Models of OR: Some Definitions

Math Models of OR: Some Definitions John E. Mitchell Department of Mathematical Sciences RPI, Troy, NY 12180 USA September 2018 Mitchell Some Definitions 1 / 20 Active constraints Outline 1 Active constraints

### Chapter 5 Linear Programming (LP)

Chapter 5 Linear Programming (LP) General constrained optimization problem: minimize f(x) subject to x R n is called the constraint set or feasible set. any point x is called a feasible point We consider

### Part 1. The Review of Linear Programming

In the name of God Part 1. The Review of Linear Programming 1.2. Spring 2010 Instructor: Dr. Masoud Yaghini Outline Introduction Basic Feasible Solutions Key to the Algebra of the The Simplex Algorithm

### OPRE 6201 : 3. Special Cases

OPRE 6201 : 3. Special Cases 1 Initialization: The Big-M Formulation Consider the linear program: Minimize 4x 1 +x 2 3x 1 +x 2 = 3 (1) 4x 1 +3x 2 6 (2) x 1 +2x 2 3 (3) x 1, x 2 0. Notice that there are

### Linear programming. Saad Mneimneh. maximize x 1 + x 2 subject to 4x 1 x 2 8 2x 1 + x x 1 2x 2 2

Linear programming Saad Mneimneh 1 Introduction Consider the following problem: x 1 + x x 1 x 8 x 1 + x 10 5x 1 x x 1, x 0 The feasible solution is a point (x 1, x ) that lies within the region defined

### CO350 Linear Programming Chapter 6: The Simplex Method

CO350 Linear Programming Chapter 6: The Simplex Method 8th June 2005 Chapter 6: The Simplex Method 1 Minimization Problem ( 6.5) We can solve minimization problems by transforming it into a maximization

### F 1 F 2 Daily Requirement Cost N N N

Chapter 5 DUALITY 5. The Dual Problems Every linear programming problem has associated with it another linear programming problem and that the two problems have such a close relationship that whenever

### TIM 206 Lecture 3: The Simplex Method

TIM 206 Lecture 3: The Simplex Method Kevin Ross. Scribe: Shane Brennan (2006) September 29, 2011 1 Basic Feasible Solutions Have equation Ax = b contain more columns (variables) than rows (constraints),

### IE 5531: Engineering Optimization I

IE 5531: Engineering Optimization I Lecture 5: The Simplex method, continued Prof. John Gunnar Carlsson September 22, 2010 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 22, 2010

### "SYMMETRIC" PRIMAL-DUAL PAIR

"SYMMETRIC" PRIMAL-DUAL PAIR PRIMAL Minimize cx DUAL Maximize y T b st Ax b st A T y c T x y Here c 1 n, x n 1, b m 1, A m n, y m 1, WITH THE PRIMAL IN STANDARD FORM... Minimize cx Maximize y T b st Ax

### Simplex Algorithm Using Canonical Tableaus

41 Simplex Algorithm Using Canonical Tableaus Consider LP in standard form: Min z = cx + α subject to Ax = b where A m n has rank m and α is a constant In tableau form we record it as below Original Tableau

### In Chapters 3 and 4 we introduced linear programming

SUPPLEMENT The Simplex Method CD3 In Chapters 3 and 4 we introduced linear programming and showed how models with two variables can be solved graphically. We relied on computer programs (WINQSB, Excel,

### Lesson 27 Linear Programming; The Simplex Method

Lesson Linear Programming; The Simplex Method Math 0 April 9, 006 Setup A standard linear programming problem is to maximize the quantity c x + c x +... c n x n = c T x subject to constraints a x + a x

### Example. 1 Rows 1,..., m of the simplex tableau remain lexicographically positive

3.4 Anticycling Lexicographic order In this section we discuss two pivoting rules that are guaranteed to avoid cycling. These are the lexicographic rule and Bland s rule. Definition A vector u R n is lexicographically

### 1 Review Session. 1.1 Lecture 2

1 Review Session Note: The following lists give an overview of the material that was covered in the lectures and sections. Your TF will go through these lists. If anything is unclear or you have questions

### Dual Basic Solutions. Observation 5.7. Consider LP in standard form with A 2 R m n,rank(a) =m, and dual LP:

Dual Basic Solutions Consider LP in standard form with A 2 R m n,rank(a) =m, and dual LP: Observation 5.7. AbasisB yields min c T x max p T b s.t. A x = b s.t. p T A apple c T x 0 aprimalbasicsolutiongivenbyx

### Special cases of linear programming

Special cases of linear programming Infeasible solution Multiple solution (infinitely many solution) Unbounded solution Degenerated solution Notes on the Simplex tableau 1. The intersection of any basic

### Foundations of Operations Research

Solved exercises for the course of Foundations of Operations Research Roberto Cordone The dual simplex method Given the following LP problem: maxz = 5x 1 +8x 2 x 1 +x 2 6 5x 1 +9x 2 45 x 1,x 2 0 1. solve

### Chapter 1: Linear Programming

Chapter 1: Linear Programming Math 368 c Copyright 2013 R Clark Robinson May 22, 2013 Chapter 1: Linear Programming 1 Max and Min For f : D R n R, f (D) = {f (x) : x D } is set of attainable values of

### Lecture 11: Post-Optimal Analysis. September 23, 2009

Lecture : Post-Optimal Analysis September 23, 2009 Today Lecture Dual-Simplex Algorithm Post-Optimal Analysis Chapters 4.4 and 4.5. IE 30/GE 330 Lecture Dual Simplex Method The dual simplex method will

### MVE165/MMG631 Linear and integer optimization with applications Lecture 5 Linear programming duality and sensitivity analysis

MVE165/MMG631 Linear and integer optimization with applications Lecture 5 Linear programming duality and sensitivity analysis Ann-Brith Strömberg 2017 03 29 Lecture 4 Linear and integer optimization with

### The Simplex Method for Solving a Linear Program Prof. Stephen Graves

The Simplex Method for Solving a Linear Program Prof. Stephen Graves Observations from Geometry feasible region is a convex polyhedron an optimum occurs at a corner point possible algorithm - search over

### Linear and Combinatorial Optimization

Linear and Combinatorial Optimization The dual of an LP-problem. Connections between primal and dual. Duality theorems and complementary slack. Philipp Birken (Ctr. for the Math. Sc.) Lecture 3: Duality

### ORF 522. Linear Programming and Convex Analysis

ORF 5 Linear Programming and Convex Analysis Initial solution and particular cases Marco Cuturi Princeton ORF-5 Reminder: Tableaux At each iteration, a tableau for an LP in standard form keeps track of....................

### CO350 Linear Programming Chapter 6: The Simplex Method

CO50 Linear Programming Chapter 6: The Simplex Method rd June 2005 Chapter 6: The Simplex Method 1 Recap Suppose A is an m-by-n matrix with rank m. max. c T x (P ) s.t. Ax = b x 0 On Wednesday, we learned

### Linear Programming Duality P&S Chapter 3 Last Revised Nov 1, 2004

Linear Programming Duality P&S Chapter 3 Last Revised Nov 1, 2004 1 In this section we lean about duality, which is another way to approach linear programming. In particular, we will see: How to define

### Math 273a: Optimization The Simplex method

Math 273a: Optimization The Simplex method Instructor: Wotao Yin Department of Mathematics, UCLA Fall 2015 material taken from the textbook Chong-Zak, 4th Ed. Overview: idea and approach If a standard-form

### II. Analysis of Linear Programming Solutions

Optimization Methods Draft of August 26, 2005 II. Analysis of Linear Programming Solutions Robert Fourer Department of Industrial Engineering and Management Sciences Northwestern University Evanston, Illinois

### Algebraic Simplex Active Learning Module 4

Algebraic Simplex Active Learning Module 4 J. René Villalobos and Gary L. Hogg Arizona State University Paul M. Griffin Georgia Institute of Technology Time required for the module: 50 Min. Reading Most

### min 4x 1 5x 2 + 3x 3 s.t. x 1 + 2x 2 + x 3 = 10 x 1 x 2 6 x 1 + 3x 2 + x 3 14

The exam is three hours long and consists of 4 exercises. The exam is graded on a scale 0-25 points, and the points assigned to each question are indicated in parenthesis within the text. If necessary,

### Chapter 3, Operations Research (OR)

Chapter 3, Operations Research (OR) Kent Andersen February 7, 2007 1 Linear Programs (continued) In the last chapter, we introduced the general form of a linear program, which we denote (P) Minimize Z

### Contents. 4.5 The(Primal)SimplexMethod NumericalExamplesoftheSimplexMethod

Contents 4 The Simplex Method for Solving LPs 149 4.1 Transformations to be Carried Out On an LP Model Before Applying the Simplex Method On It... 151 4.2 Definitions of Various Types of Basic Vectors

### The augmented form of this LP is the following linear system of equations:

1 Consider the following LP given in standard form: max z = 5 x_1 + 2 x_2 Subject to 3 x_1 + 2 x_2 2400 x_2 800 2 x_1 1200 x_1, x_2 >= 0 The augmented form of this LP is the following linear system of

### Introduction to Operations Research Prof. G. Srinivasan Department of Management Studies Indian Institute of Technology, Madras

Introduction to Operations Research Prof. G. Srinivasan Department of Management Studies Indian Institute of Technology, Madras Module - 03 Simplex Algorithm Lecture 15 Infeasibility In this class, we

### Introduction to optimization

Introduction to optimization Geir Dahl CMA, Dept. of Mathematics and Dept. of Informatics University of Oslo 1 / 24 The plan 1. The basic concepts 2. Some useful tools (linear programming = linear optimization)

### Developing an Algorithm for LP Preamble to Section 3 (Simplex Method)

Moving from BFS to BFS Developing an Algorithm for LP Preamble to Section (Simplex Method) We consider LP given in standard form and let x 0 be a BFS. Let B ; B ; :::; B m be the columns of A corresponding

### MATH 445/545 Homework 2: Due March 3rd, 2016

MATH 445/545 Homework 2: Due March 3rd, 216 Answer the following questions. Please include the question with the solution (write or type them out doing this will help you digest the problem). I do not

### Spring 2017 CO 250 Course Notes TABLE OF CONTENTS. richardwu.ca. CO 250 Course Notes. Introduction to Optimization

Spring 2017 CO 250 Course Notes TABLE OF CONTENTS richardwu.ca CO 250 Course Notes Introduction to Optimization Kanstantsin Pashkovich Spring 2017 University of Waterloo Last Revision: March 4, 2018 Table

### Week 3: Simplex Method I

Week 3: Simplex Method I 1 1. Introduction The simplex method computations are particularly tedious and repetitive. It attempts to move from one corner point of the solution space to a better corner point

### Chapter 1 Linear Programming. Paragraph 5 Duality

Chapter 1 Linear Programming Paragraph 5 Duality What we did so far We developed the 2-Phase Simplex Algorithm: Hop (reasonably) from basic solution (bs) to bs until you find a basic feasible solution

### Gauss-Jordan Elimination for Solving Linear Equations Example: 1. Solve the following equations: (3)

The Simple Method Gauss-Jordan Elimination for Solving Linear Equations Eample: Gauss-Jordan Elimination Solve the following equations: + + + + = 4 = = () () () - In the first step of the procedure, we

### The Dual Simplex Algorithm

p. 1 The Dual Simplex Algorithm Primal optimal (dual feasible) and primal feasible (dual optimal) bases The dual simplex tableau, dual optimality and the dual pivot rules Classical applications of linear

### Linear Programming and the Simplex method

Linear Programming and the Simplex method Harald Enzinger, Michael Rath Signal Processing and Speech Communication Laboratory Jan 9, 2012 Harald Enzinger, Michael Rath Jan 9, 2012 page 1/37 Outline Introduction

### M340(921) Solutions Problem Set 6 (c) 2013, Philip D Loewen. g = 35y y y 3.

M340(92) Solutions Problem Set 6 (c) 203, Philip D Loewen. (a) If each pig is fed y kilograms of corn, y 2 kilos of tankage, and y 3 kilos of alfalfa, the cost per pig is g = 35y + 30y 2 + 25y 3. The nutritional

### 4.5 Simplex method. min z = c T x s.v. Ax = b. LP in standard form

4.5 Simplex method min z = c T x s.v. Ax = b x 0 LP in standard form Examine a sequence of basic feasible solutions with non increasing objective function value until an optimal solution is reached or

### December 2014 MATH 340 Name Page 2 of 10 pages

December 2014 MATH 340 Name Page 2 of 10 pages Marks [8] 1. Find the value of Alice announces a pure strategy and Betty announces a pure strategy for the matrix game [ ] 1 4 A =. 5 2 Find the value of

### 56:171 Operations Research Fall 1998

56:171 Operations Research Fall 1998 Quiz Solutions D.L.Bricker Dept of Mechanical & Industrial Engineering University of Iowa 56:171 Operations Research Quiz

### Linear programming. Starch Proteins Vitamins Cost (\$/kg) G G Nutrient content and cost per kg of food.

18.310 lecture notes September 2, 2013 Linear programming Lecturer: Michel Goemans 1 Basics Linear Programming deals with the problem of optimizing a linear objective function subject to linear equality

### Linear Programming: Simplex

Linear Programming: Simplex Stephen J. Wright 1 2 Computer Sciences Department, University of Wisconsin-Madison. IMA, August 2016 Stephen Wright (UW-Madison) Linear Programming: Simplex IMA, August 2016

### Introduction. Very efficient solution procedure: simplex method.

LINEAR PROGRAMMING Introduction Development of linear programming was among the most important scientific advances of mid 20th cent. Most common type of applications: allocate limited resources to competing

### Systems Analysis in Construction

Systems Analysis in Construction CB312 Construction & Building Engineering Department- AASTMT by A h m e d E l h a k e e m & M o h a m e d S a i e d 3. Linear Programming Optimization Simplex Method 135

### 4. Duality Duality 4.1 Duality of LPs and the duality theorem. min c T x x R n, c R n. s.t. ai Tx = b i i M a i R n

2 4. Duality of LPs and the duality theorem... 22 4.2 Complementary slackness... 23 4.3 The shortest path problem and its dual... 24 4.4 Farkas' Lemma... 25 4.5 Dual information in the tableau... 26 4.6

### IE 5531: Engineering Optimization I

IE 5531: Engineering Optimization I Lecture 7: Duality and applications Prof. John Gunnar Carlsson September 29, 2010 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I September 29, 2010 1

### Worked Examples for Chapter 5

Worked Examples for Chapter 5 Example for Section 5.2 Construct the primal-dual table and the dual problem for the following linear programming model fitting our standard form. Maximize Z = 5 x 1 + 4 x

### CHAPTER 2. The Simplex Method

CHAPTER 2 The Simplex Method In this chapter we present the simplex method as it applies to linear programming problems in standard form. 1. An Example We first illustrate how the simplex method works

### LP. Lecture 3. Chapter 3: degeneracy. degeneracy example cycling the lexicographic method other pivot rules the fundamental theorem of LP

LP. Lecture 3. Chapter 3: degeneracy. degeneracy example cycling the lexicographic method other pivot rules the fundamental theorem of LP 1 / 23 Repetition the simplex algorithm: sequence of pivots starting

### OPERATIONS RESEARCH. Linear Programming Problem

OPERATIONS RESEARCH Chapter 1 Linear Programming Problem Prof. Bibhas C. Giri Department of Mathematics Jadavpur University Kolkata, India Email: bcgiri.jumath@gmail.com MODULE - 2: Simplex Method for

### 2.098/6.255/ Optimization Methods Practice True/False Questions

2.098/6.255/15.093 Optimization Methods Practice True/False Questions December 11, 2009 Part I For each one of the statements below, state whether it is true or false. Include a 1-3 line supporting sentence

### Optimisation and Operations Research

Optimisation and Operations Research Lecture 9: Duality and Complementary Slackness Matthew Roughan http://www.maths.adelaide.edu.au/matthew.roughan/ Lecture_notes/OORII/

### Distributed Real-Time Control Systems. Lecture Distributed Control Linear Programming

Distributed Real-Time Control Systems Lecture 13-14 Distributed Control Linear Programming 1 Linear Programs Optimize a linear function subject to a set of linear (affine) constraints. Many problems can

### Linear programs, convex polyhedra, extreme points

MVE165/MMG631 Extreme points of convex polyhedra; reformulations; basic feasible solutions; the simplex method Ann-Brith Strömberg 2015 03 27 Linear programs, convex polyhedra, extreme points A linear

### Lecture 4: Algebra, Geometry, and Complexity of the Simplex Method. Reading: Sections 2.6.4, 3.5,

Lecture 4: Algebra, Geometry, and Complexity of the Simplex Method Reading: Sections 2.6.4, 3.5, 10.2 10.5 1 Summary of the Phase I/Phase II Simplex Method We write a typical simplex tableau as z x 1 x

### The simplex algorithm

The simplex algorithm The simplex algorithm is the classical method for solving linear programs. Its running time is not polynomial in the worst case. It does yield insight into linear programs, however,

### Discrete Optimization

Prof. Friedrich Eisenbrand Martin Niemeier Due Date: April 15, 2010 Discussions: March 25, April 01 Discrete Optimization Spring 2010 s 3 You can hand in written solutions for up to two of the exercises

### (includes both Phases I & II)

(includes both Phases I & II) Dennis ricker Dept of Mechanical & Industrial Engineering The University of Iowa Revised Simplex Method 09/23/04 page 1 of 22 Minimize z=3x + 5x + 4x + 7x + 5x + 4x subject

### Duality Theory, Optimality Conditions

5.1 Duality Theory, Optimality Conditions Katta G. Murty, IOE 510, LP, U. Of Michigan, Ann Arbor We only consider single objective LPs here. Concept of duality not defined for multiobjective LPs. Every

### Summary of the simplex method

MVE165/MMG631,Linear and integer optimization with applications The simplex method: degeneracy; unbounded solutions; starting solutions; infeasibility; alternative optimal solutions Ann-Brith Strömberg

### Week_4: simplex method II

Week_4: simplex method II 1 1.introduction LPs in which all the constraints are ( ) with nonnegative right-hand sides offer a convenient all-slack starting basic feasible solution. Models involving (=)

### (P ) Minimize 4x 1 + 6x 2 + 5x 3 s.t. 2x 1 3x 3 3 3x 2 2x 3 6

The exam is three hours long and consists of 4 exercises. The exam is graded on a scale 0-25 points, and the points assigned to each question are indicated in parenthesis within the text. Problem 1 Consider

### Understanding the Simplex algorithm. Standard Optimization Problems.

Understanding the Simplex algorithm. Ma 162 Spring 2011 Ma 162 Spring 2011 February 28, 2011 Standard Optimization Problems. A standard maximization problem can be conveniently described in matrix form