Introduction to Operations Research (Week 5: Linear Programming: More on Simplex) José Rui Figueira Instituto Superior Técnico Universidade de Lisboa (figueira@tecnico.ulisboa.pt) March 14-15, 2016
This slides are currently available for personal use of IST LEGI FIO students in an unpublished draft form only. The slides cannot be copied, reproduced, or distributed in any form.
Part Matrix-Vector Form of Simplex Method
Contents 1. Introduction 2. Another view of the graphical method 3. The matrix-form of Simplex method 4. Illustration with the tabular form 5. The Gauss-Jordan and the Simplex method 6. An example 7. Conclusions
1. Introduction Brief Introduction Some variants of the simplex method. Algebraic form: longhand notation. Tabular form: simple row operations (as in Gauss-Jordan method). Algebraic form: shorthand notation (matrix-vector notation). Tabular form: matrix-vector notation. A little bit more on matrix-vector notation. References: [Bazaraa et al., 1990, Goldfarb and Todd, 1989, Hillier and Lieberman, 2005, Taha, 2010]. J.R. Figueira (IST) FIO March 14-15, 2016 5 / 30
2. Another view of the graphical method 3D Representation f (x 1, x 2 ) f (x c ) f (x b ) f (x d ) (5, 3, 13) f (x e ) f (x a ) (4, 6, 14) Z (2, 7, 11) (3, 1, 7) f (x f ) (1, 6, 8) (1, 3, 5) ( f x 1, f ) = f = f = (2, 1) x 2 xf x a x b X x c x e x d x 2 x 1 J.R. Figueira (IST) FIO March 14-15, 2016 6 / 30
2. Another view of the graphical method Another View of the Objective Function Shape 15 14 13 12 11 10 f (x1, x2) 9 8 7 5 6 4.5 5 4 4 1.4 1.6 1.8 2 2.2 2.4 2.6 2.8 3 3.2 3.4 3.6 x1 3.8 4 4.2 4.4 4.6 4.8 5 1.5 2 2.5 3 x2 3.5 J.R. Figueira (IST) FIO March 14-15, 2016 7 / 30
2. Another view of the graphical method 2D Representation f (x 1, x 2 ) z = 5 x a x f x e z = 11 x b X x c = x = (4, 6) xd x 2 z = 14 x 1 J.R. Figueira (IST) FIO March 14-15, 2016 8 / 30
3. The matrix-form of Simplex method A Shorthand Form (1) Consider the matrix form of an LP. maximize z(x) = c x subject to Ax = b with x 0. The problem can be reformulated as follows. max subject to: ẑ Ax = b ẑ c x = 0 x 0 Given a basis B we can make the following partitions. A = (B N) c = (c B c N ) x = (x B x N ) J.R. Figueira (IST) FIO March 14-15, 2016 9 / 30
3. The matrix-form of Simplex method A Shorthand Form (2) The problem can be rewritten as follows. max ẑ subject to: Bx B + Nx N = b ẑ cb x B cn x N = 0 x B 0 x N 0 Again, we can form the system: Bx B + Nx N = b ẑ c B x B c N x N = 0 J.R. Figueira (IST) FIO March 14-15, 2016 10 / 30
3. The matrix-form of Simplex method A Shorthand Form (3) We can analyze separately the two equations (Consider the first one). Bx B + Nx N = b Bx B = b Nx N (first step to make x B as a function of x N ) B 1 Bx B = B 1( ) b Nx N (multiply both sides by B 1, the inverse of B) Ix B = B 1 b B 1 Nx N (I is the identity matrix) x B = B 1 b B 1 Nx N (since Ix B = x B ) x B = B 1 b (since x N = 0) Now consider the second equation: ẑ cb x B cn x N = 0 ẑ cb ( B 1 b B 1 ) Nx N c N x N = 0 ẑ = cb B 1 b ( cb B 1 N cn) xn ẑ = cb B 1 b (= cb x B) J.R. Figueira (IST) FIO March 14-15, 2016 11 / 30
3. The matrix-form of Simplex method A Shorthand Form (4) And, we have also our dictionary: x B = B 1 b B 1 Nx N x B = B 1 b ẑ = c B B 1 b ( c B B 1 N c N) xn ẑ = c B B 1 b Reduced costs (Consider z N = c B B 1 N) ẑ + ( c B B 1 N c N) xn = ẑ + ( z N c N) xn = ẑ + c N x N = c B B 1 b c N = ( z N c N) are thus the reduced costs. The reduced cost for the basic variables are equal to zero ( c B = 0). J.R. Figueira (IST) FIO March 14-15, 2016 12 / 30
4. Illustration with the tabular form The tabular form: Starting Tableau Consider the following data concerning the initial Tableau: A = (Ā, I) where N = Ā and B = I at the beginning. x I is the vector of the initial basic variables. x R is the vector of the initial nonbasic variables or remaining variables. c c R c I c B x B x R x I c I x I Ā I b z ẑ c ( c I ci Ā Ā cr ) c I 0 c I b J.R. Figueira (IST) FIO March 14-15, 2016 13 / 30
4. Illustration with the tabular form Tableau at some iteration t Consider a given basis B. x I is till the initial basic variables. x R is still the initial nonbasic variables. B 1 is the basis inverse, a key element for simplex computations. c c R c I c B x B x R x I c B x B B 1 Ā B 1 B 1 b z cb B 1 Ā cb B 1 ẑ c ( cb B 1 Ā cr ) ( c B B 1 ci ) c B B 1 b J.R. Figueira (IST) FIO March 14-15, 2016 14 / 30
5. The Gauss-Jordan and the Simplex method Some important Remarks The steps of the simplex method are equivalent to those of the Gauss-Jordan method. In both cases we make elementary row operations. For this reason, we can obtain the optimal Tableau (or other) by pre-multiplying the initial Tableau by a suitable matrix. Let us consider again the system Ax = b and ẑ = c x. And the partition in basic and nonbasic variables. From Linear Algebra we know that B 1( B I b ) = ( B 1 B B 1 I B 1 b ) = ( I B 1 B 1 b ). J.R. Figueira (IST) FIO March 14-15, 2016 15 / 30
5. The Gauss-Jordan and the Simplex method Another view of the matrix form (1) Consider our system again. { Bx B + Nx N = b ẑ cb x B cn x N = 0 We can consider the system in an extended matrix form. ( 0 B N ) b 1 cb cn 0 The basis for this extended matrix is the following: ( ) 0 B 1 c B And its inverse is as follows. ( ) 1 0 B 1 cb = ( 0 B 1 1 c B B 1 ) J.R. Figueira (IST) FIO March 14-15, 2016 16 / 30
5. The Gauss-Jordan and the Simplex method Another view of the matrix form (2) Let us pre-multiply the extended matrix by the inverse. ( 0 B 1 1 c B B 1 ) ( 0 B N ) b 1 cb cn 0 = ( 0 I B 1 N B 1 b ) 1 0 cn + c B B 1 N cb B 1 b The Simplex Tableau is thus obtained again. The basic variables form canonical columns in the Tableau (see the Identity Matrix). The reduced cost are equal to zero for the basic variables. When B is the optimal basis we get the optimal Tableau. J.R. Figueira (IST) FIO March 14-15, 2016 17 / 30
6. An example Back to our example Consider x B = (x 6 x 1 x 5 x 2 ). The basis B is as follows (see initial columns, 6, 1, 5,and 2 of A). 0 2 0 1 B = 0 1 0 2 0 1 1 0 1 0 0 1 We a method (for instance the Gauss-Jordan) to compute the basis inverse, B 1. 1 0 2 0 1 1/3 2/3 0 1 B 1 = 0 1 0 2 0 1 1 0 = 2/3 1/3 0 0 2/3 1/3 1 0 1 0 0 1 1/3 2/3 0 0 J.R. Figueira (IST) FIO March 14-15, 2016 18 / 30
6. An example Optimal Tableau c (3 4) (0 0 0 0) c B x B (x 1 x 2 ) (x 3 x 4 x 5 x 6 ) ˆb 0 x 6 0 0 1/3 2/3 0 1 2 3 x 1 1 0 2/3 1/3 0 0 7 0 x 5 0 0 2/3 1/3 1 0 1 4 x 2 0 1 1/3 2/3 0 0 4 z (3 4) (2/3 5/3 0 0) ẑ c (0 0) (2/3 5/3 0 0) 37 J.R. Figueira (IST) FIO March 14-15, 2016 19 / 30
7. Conclusions A Shorthand Form All variants of simplex method are important for a better understating of the method itself. All are equivalent. Some are more appropriated than other according to the circumstances. Matrix-vector form leads to another version: The revised simplex method. This one is more important for those who want to implement simplex. J.R. Figueira (IST) FIO March 14-15, 2016 20 / 30
Part How to Deal With Particular LPs in Simplex Method
Contents 1. Introduction 2. Multiple Optima 3. Degeneracy 4. Unboundedness 5. Infeasibility 6. Conclusions
1. Introduction A Brief Introduction Solve an LP does not imply always to get a single optima. There are many cases to be considered. Degeneracy is one of the most common. A basic solution, x, is called degenerate if more than n constraints are biding at x. We will be back to this definitions in one of the next lectures. J.R. Figueira (IST) FIO March 14-15, 2016 23 / 30
2. Multiple Optima The objective function coincides with a constraint max z(x) = 2x 1 + 4x 2 subject to: x 1 + 2x 2 5 x 1 + x 2 4 x 1, x 2 0 max z(x) = 2x 1 + 4x 2 subject to: x 1 + 2x 2 + x 3 = 5 x 1 + x 2 + x 4 = 4 x 1, x 2, x 3, x 4 0 x 2 5 4 3 2 1 0 1 0 1 2 3 4 5 1 x 1 c j 2 4 0 0 Basis x j x 1 x 2 x 3 x 4 ˆbi 0 x 3 1 2 1 0 5 0 x 4 1 1 0 1 4 z j 0 0 0 0 c j 2 4 0 0 0 0 x 4 1/2 0 1/2 1 3/2 4 x 2 1/2 1 1/2 0 5/2 z j 2 4 2 0 c j 0 0 2 0 10 4 x 2 0 1 1 1 1 2 x 1 1 0 1 2 3 z j 2 4 2 0 c j 0 0 2 0 10 J.R. Figueira (IST) FIO March 14-15, 2016 24 / 30
3. Degeneracy At least one basic variable has a zero value max z(x) = 3x 1 + 9x 2 subject to: x 1 + 4x 2 8 x 1 + 2x 2 4 x 1, x 2 0 max z(x) = 3x 1 + 9x 2 subject to: x 1 + 4x 2 + x 3 = 8 x 1 + 2x 2 + x 4 = 4 x 1, x 2, x 3, x 4 0 5 4 3 2 1 x 2 0 0 1 2 3 4 5 6 7 8 9 10 x 1 c j 3 9 0 0 Basis x j x 1 x 2 x 3 x 4 ˆbi 0 x 3 1 4 1 0 8 0 x 4 1 2 0 1 4 z j 0 0 0 0 c j 3 9 0 0 0 0 x 4 1/2 0 1/2 1 0 9 x 2 1/4 1 1/4 0 2 z j 9/4 9 9/4 0 c j 3/4 0 9/4 0 18 9 x 2 0 1 1/2 1/2 2 3 x 1 1 0 1 2 0 z j 3 9 3/2 3/2 c j 0 0 3/2 3/2 18 J.R. Figueira (IST) FIO March 14-15, 2016 25 / 30
4. Unboundedness The objective has not a finite value max z(x) = 2x 1 + x 2 subject to: x 1 x 2 2 2x 1 8 x 1, x 2 0 max z(x) = 2x 1 + x 2 subject to: x 1 x 2 + x 3 = 2 2x 1 + x 4 = 8 x 1, x 2, x 3, x 4 0 x 2 5 4 3 2 1 0 1 0 1 2 3 4 5 1 2 x 1 c j 2 1 0 0 0 x 3 1 1 1 0 2 0 x 4 2 0 0 1 8 Basis x j x 1 x 2 x 3 x 4 ˆbi z j 0 0 0 0 c j 2 1 0 0 0 2 x 1 1 1 1 0 2 0 x 4 0 2 2 1 4 z j 2 2 2 0 c j 0 3 2 0 4 2 x 1 1 0 0 1/2 4 1 x 2 0 1 1 1/2 2 z j 2 1 1 3/2 c j 0 0 1 3/2 10 J.R. Figueira (IST) FIO March 14-15, 2016 26 / 30
5. Infeasibility The polyhedron is empty max z(x) = 2x 1 + 2x 2 subject to: x 1 + x 2 3 2x 2 7 x 1, x 2 0 max z(x) = 2x 1 + 2x 2 subject to: x 1 + x 2 + x 3 = 3 2x 2 x 4 = 7 x 1, x 2, x 3, x 4 0 x 2 Solve this problem! 5 4 3 2 1 0 0 1 2 3 4 5 x 1 J.R. Figueira (IST) FIO March 14-15, 2016 27 / 30
6. Conclusions Some conclusions These situations are not so rare in practice. Unboundedness and infeasibility are in general related to a bad formulation of the problem. Degeneracy can lead cycling, which implies to use different pricing rules for preventing cycling. Multiple optima is not in general bad since we can dispose of several programs with the same value, and give more options for the choice of one. But, in some cases multiple optima can pose many problems. J.R. Figueira (IST) FIO March 14-15, 2016 28 / 30
Part Bibliography
References Bazaraa, M., Jarvis, J., and Sherali, H. (1990). Linear Programming and Network Flows. John Wiley & Sons, New York, USA, second edition. Goldfarb, D. and Todd, M. J. (1989). Linear programming. In Nemhauser, G. L., Rinnoy Kan, A. H. G., and Todd, M. J., editors, Optimization, volume 1 of Handbooks in Operations Research and Management Science, pages 141 170. North Holland, Amsterdam, The Netherlands. Hillier, F. and Lieberman, G. (2005). Introduction to Operations Research. The McGraw-Hill Companies, Inc., New York, USA, eighth edition. Taha, H. (2010). Operations Research: An Introduction. Prentice-Hall, Upper Saddle River, New Jersey, USA, 9th edition.