Introduction to Operations Research

Size: px

Start display at page:

Download "Introduction to Operations Research"

Elinor Moore
5 years ago
Views:

1 Introduction to Operations Research (Week 4: Linear Programming: More on Simplex and Post-Optimality) José Rui Figueira Instituto Superior Técnico Universidade de Lisboa March 7-8, 2016

2 This slides are currently available for personal use of IST LEGI FIO students in an unpublished draft form only. The slides cannot be copied, reproduced, or distributed in any form.

3 Part The Termination of the Simplex Method and Other Aspects

4 Contents 1. Introduction 2. Non-degenerate LPs: Theoretical results 3. Degeneracy and cycling 4. Avoiding degeneracy: The perturbation method 5. Avoiding cycling 6. The computational behavior of the simplex method 7. Numerical stability of the simplex method 8. Conclusions

5 1. Introduction Brief Introduction The simplex algorithm tries to increase the value of a non-basic variable x k. When there is no degeneracy, then x k will be positive after pivoting, which leads to the improvement of the objective value function and the algorithm terminates after a finite number of iterations. When in presence of degeneracy, cycling may occurs. In such a situation, there is a sequence of pivots that starts from some basic feasible solution and ends exactly at the same basic feasible solution. Simplex algorithm cycles forever. How to prevent degeneracy? How to prevent cycling? What is the computational behavior of the simplex algorithm in the worst-case? What about the stability of the simplex algorithm? J.R. Figueira (IST) FIO March 7-8, / 50

6 2. Non-degenerate LPs: Theoretical results Comments and Remarks If we are sure that all the basic feasible solutions are not degenerate the following result holds. Theorem 1 (Termination of the simplex algorithm). If all basic feasible solutions are non-degenerate, then the simplex algorithm terminates (after performing a finite number of iterations), regardless of the choice of the candidate entering variables (and the corresponding leaving variables). The issue is that in practice we do not know wether an LP is free of degenerate basic feasible solutions. Degeneracy should be avoided since it may lead to cycling. Let us illustrate these two concepts. J.R. Figueira (IST) FIO March 7-8, / 50

7 3. Degeneracy and cycling An Example by Klee-Minty max z(x) = 3/4x 1 20x 2 + 1/2x 3 + 6x subject to: 1/4x 1 8x 2 x 3 + 9x 4 0 1/2x 1 12x 2 1/2x 3 + 3x 4 0 x 3 1 x 1, x 2, x 3, x 4 0 After introducing the slack variables, x 5, x 6, and x 7, the following sequence of bases occurs: (Please verify by making the necessary pivoting operations in the simplex Tabeaus.) 1. x B := (x 5, x 6, x 7 ) := (0, 0, 1) and z = x B := (x 1, x 6, x 7 ) := (0, 0, 1) and z = x B := (x 1, x 2, x 7 ) := (0, 0, 1) and z = x B := (x 3, x 2, x 7 ) := (0, 0, 1) and z = x B := (x 3, x 4, x 7 ) := (0, 0, 1) and z = x B := (x 5, x 4, x 7 ) := (0, 0, 1) and z = x B := (x 5, x 6, x 7 ) := (0, 0, 1) and z = 3. J.R. Figueira (IST) FIO March 7-8, / 50

8 4. Avoiding degeneracy: The perturbation method Comments and Remarks Consider the following two problems, named P and P ɛ, respectively (ɛ is a small enough positive value). { max c x : Ax = b, x 0 } (P) x R n { max c x : Ax = b ɛ, x 0 } (P ɛ ) x R n where the components of the perturbed RHS vector b ɛ are the following The following result holds. b 1 + ɛ, b 2 + ɛ 2,..., b i + ɛ i,..., b m + ɛ m. Theorem 2 (Termination of the simplex algorithm regained). The simplex algorithm terminates for P ɛ. Moreover, if P ɛ is unbounded, then P is also unbounded; and, if P ɛ has an optimal solution, then the corresponding basis is also optimal for P. J.R. Figueira (IST) FIO March 7-8, / 50

9 5. Avoiding cycling Comments and Remarks Good news: We do not really to have to deal with a concrete ɛ. Two important rules can be used to prevent cycling: Lexicographic rule. Applied to the leaving variable (more details are needed). Bland s rule. Consider the strictly negative c j. The entering variable is the one corresponding to the smallest index j. When in presence of ties for the selection of the leaving variable, chose the one with the smallest index i. The following result holds. Theorem 3 (Alternative termination result). The simplex method terminates, if the Bland s (lexicographic) rule is applied when solving P. J.R. Figueira (IST) FIO March 7-8, / 50

10 6. The computational behavior of the simplex method Comments and Remarks Each solution provided by the simplex algorithm is a basic feasible solution with m basic variables (the number of functional constraints). The number of ways of selecting m basic variables out of n is equal to n!/(n m)!m!, which is an upper bound on the number of basic feasible solutions. How many vertices can be visited by the simplex algorithm in the worst case? Klee and Minty (1972) created an LP with n variables and n constraints and 2 n vertices. The standard simplex algorithm passes through each one of the extreme points before reaching the optimal solution (the exponential running time behavior of simplex algorithm for general LPs). The example was built by distorting a cube (hypercube). J.R. Figueira (IST) FIO March 7-8, / 50

11 6. The computational behavior of the simplex method The Klee-Minty Example max z(x) = 2 n 1 x n 2 x n 3 x x n 1 + x n subject to: x 1 5 4x 1 + x x 1 + 4x 2 + x n x n 1 x n 2 x x n 1 + x n 5 n x 1, x 2, x 3,... x n 1, x n 0 J.R. Figueira (IST) FIO March 7-8, / 50

12 7. Numerical stability of the simplex method Comments and Remarks For implementation purposes, the revised simplex method is preferable (to the full tableau method) due to its smaller memory requirements and smaller average running time. The revised simplex method is a variant of the simplex method in the matrix-vector form. We know the importance of the B 1 in this method. Every time this matrix is updated there is a risk of introducing roundoff or truncation errors, which accumulate and might eventually lead to highly inaccurate results. There are several ways of dealing with this problem, but they are out of the contents of this course. J.R. Figueira (IST) FIO March 7-8, / 50

13 8. Conclusions The future of simplex algorithms Simplex for large size LPs. Improved revised versions of the simplex. Use the dual revised simplex. Exploiting parallelism. J.R. Figueira (IST) FIO March 7-8, / 50

14 Part Finding an Initial Basic Feasible Solution

15 Contents 1. Introduction 2. A small example 3. The problem in the equality form 4. Introducing artificial variables 5. Solving the Big M model by the simplex method 6. The two-phases application of the simplex method 7. Conclusions

16 1. Introduction Brief Introduction For a large amount of LPs, we do not dispose of a good starting basic feasible solution. It occurs in particular whenever the LP contains a positive RHS and constraints of the types and =. There are two well-known techniques to find a feasible basic solutions when the polyhedron is not empty. But, if it is the case these techniques are able to provide such an information. These two techniques are: the successive application of the simplex method in two-phases and solving the big M model. IOR Tutorial: References: [Bazaraa et al., 1990, Goldfarb and Todd, 1989, Hillier and Lieberman, 2005, Taha, 2010]. J.R. Figueira (IST) FIO March 7-8, / 50

17 2. A small example An Example: Equality-Inequality Form min z(x 1, x 2 ) = x 1 + 1/4x 2 subject to: 3x 1 + x 2 = 3 4x 1 + 3x 2 6 x 1 + 2x 2 4 x 1, x 2 0 It is easy to see that we cannot start with the standard solution x 1 = x 2 = 0. This solution is not feasible. The first and the second constraints are not standard (as in the previous Lectures). How to deal with the problem? J.R. Figueira (IST) FIO March 7-8, / 50

18 2. A small example Graphical Method x D 1 C 0 A B x 1 J.R. Figueira (IST) FIO March 7-8, / 50

19 3. The problem in the equality form An Example: Equality Form Consider the problem in an equality form. The variable considered to transform an inequality of the type into an equation is called a surplus variable. Slack and surplus variables have a substantive meaning. min z(x 1, x 2, x 3, x 4 ) = x 1 + 1/4x 2 0x 3 + 0x 4 subject to: 3x 1 + x 2 = 3 4x 1 + 3x 2 x 3 = 6 x 1 + 2x 2 + x 4 = 4 x 1, x 2, x 3, x 4 0 Let us try to construct an initial basis:? = 3 3x 1 x 2? = 6 4x 1 3x 2 x 4 = 4 x 1 2x 2 + x 3 It is not possible! J.R. Figueira (IST) FIO March 7-8, / 50

20 4. Introducing artificial variables Artificial Variables Consider only the constraints and add two artificial variables: 3x 1 + x 2 + a 1 = 3 4x 1 + 3x 2 x 3 + a 2 = 6 x 1 + 2x 2 + x 4 = 4 x 1, x 2, a 1, x 3, a 2, x 4 0 Now, construct an initial basis: a 1 = 3 3x 1 x 2 a 2 = 6 4x 1 3x 2 x 4 = 4 x 1 2x 2 + x 3 Artificial variables have no meaning. They should be removed from the basis as soon as possible. If they still in the basis at the optimum, then the problem has no feasible solutions. J.R. Figueira (IST) FIO March 7-8, / 50

21 5. Solving the Big M model by the simplex method Comments and Remarks The main objective is to remove artificial variables from the basis. When it is possible, we get a first basic feasible solution. If the artificial variables cannot be removed, the problem has no feasible solutions. In Big M method we should form the following objective function (M is a large enough positive value). min z(x) = x 1 + 1/4x 2 + Ma 1 0x 3 + Ma 2 + 0x 4. The next step is just to apply simplex method as we know it. If the problem is a maximization problem, the coefficients in the objective function are as follows Ma 1,... When an artificial variable leaves the basis we do not need to update its column, unless we need to perform some kind of post-optimality analysis. Next slide contains the resolution of the example. J.R. Figueira (IST) FIO March 7-8, / 50

22 5. Solving the Big M model by the simplex method c j 1 1/4 M 0 M 0 M a (3/3 = 1) M a (6/4 = 1.5) Basis x j x 1 x 2 a 1 x 3 a 2 x 4 ˆbi Ratio 0 x (4/1 = 4) t = 0 z j 7M 4M M M M 0 A c j (7M 1) (4M 1/4) 0 M 0 0 9M 1 x 1 1 1/ (1 : 1/3 = 3) M a 2 0 5/ (2 : 5/3 = 6/5) 0 x 4 0 5/ (3 : 5/3 = 9/5) t = 1 z j 1 (1 + 5M)/3 M M 0 B c j 0 (5M/3 + 1/12) M M 1 x /5 0 3/5 (3/5 : 1/5 = 3) 1/4 x /5 0 6/5 0 x (1/1 = 1) t = 2 z j 1 1/4 1/20 0 C c j 0 0 1/20 0 9/10 1 x /5 2/5 1/4 x /5 9/5 0 x t = 3 z j 1 1/4 0 1/20 D c j /20 17/20 J.R. Figueira (IST) FIO March 7-8, / 50

23 6. The two-phases application of the simplex method Comments and Remarks The objective is the same: find a starting feasible basis, if possible. This should be done in two phases. The first phase consist of minimizing the following objective function (it is always a minimization). min a 1 + a 2. If the optimal value of this objective function is zero, it means we found a starting basic feasible solutions and we can go to the second phase. Otherwise, the problem has no feasible solutions. In Phase II, optimize the original objective function from the optimal solution of Phase I. min z(x) = x 1 + 1/4x 2. When the original problem is a maximization problem, the second phase is also a maximization problem. Let us consider the same example. J.R. Figueira (IST) FIO March 7-8, / 50

24 6. The two-phases application of the simplex method Phase I: (Consider the first objective function) c j a (3/3 = 1) 1 a (6/4 = 1.5) Basis x j x 1 x 2 a 1 x 3 a 2 x 4 ˆbi Ratio 0 x (4/1 = 4) t = 0 z j A c j x 1 1 1/ (1 : 1/3 = 3) 1 a 2 0 5/ (2 : 5/3 = 6/5) 0 x 4 0 5/ (3 : 5/3 = 9/5) t = 1 z j 1 5/ B c j 0 5/ x /5 0 3/5 0 x /5 0 6/5 0 x t = 2 z j C c j At the end the objective function has value zero. Go to Phase II. J.R. Figueira (IST) FIO March 7-8, / 50

25 6. The two-phases application of the simplex method Phase II: (Consider the last tableau of Phase I) c j???? Basis x j x 1 x 2 x 3 x 4 ˆbi Ratio? x /5 0 3/5? x /5 0 6/5? x z j???? c j????? c j 1 1/4 0 0 Basis x j x 1 x 2 x 3 x 4 ˆbi Ratio 1 x /5 0 3/5 1/4 x /5 0 6/5 0 x t = 0 z j 1 1/4 1/20 0 C c j 0 0 1/20 0 9/10 1 x /5 2/5 1/4 x /5 9/5 0 x t = 1 z j 1 1/4 0 1/20 D c j /20 17/20 J.R. Figueira (IST) FIO March 7-8, / 50

26 7. Conclusions Some conclusions These two techniques are used when there is no an obvious starting solution. You should apply the two techniques to a problem with no feasible solutions (see PB Classes). Unless post-optimality analysis will be required, you can discard the columns of artificial variables once they leave the basis. Apply the two techniques to minimization as well as to maximization problems. J.R. Figueira (IST) FIO March 7-8, / 50

27 Part Post-Optimality and Economic Interpretation

28 Contents 1. Introduction 2. A general product-mix allocation problem 3. Characterizing the resources 4. Sensitivity analysis: Graphical approach 5. Sensitivity analysis: Tabular computations 6. Sensitivity analysis: Matrix-vector computations 7. Conclusions

29 1. Introduction Brief Introduction The economic interpretation of the data and the results is extremely important. Some post-optimality analysis can help in doing it. Post-optimality includes: adding/removing variables or/and constraints, discrete changes in data, parametric techniques, robustness analysis, stability analysis, sensitivity analysis,... We shall deal with sensitivity analysis for each c j and b i, considered separately. In such a kind of analysis we are interested in identifying the range that allows to maintain the structure of the optimal solution (i.e, basic and non-basic variables will not change). J.R. Figueira (IST) FIO March 7-8, / 50

30 2. A general product-mix allocation problem A Classical Maximization Problem Example 2 (A Classical Product Mix Resource Allocation Problem). The CorkMaxPro Company is the largest Portuguese producer and exporter of cork made products. Over the last five years, the company has gradually expanded the geography and the volume of its production activities, which consist of producing a product mix (i.e., designing and making the product lines or assortments that the company offers to its clients), here generically denoted by P 1,..., P j,..., and P n. The company s plants possess several resources, R 1,..., R i,..., and R m, to make the different assortments. The quantity available of each resource is, b 1,..., b i,..., and b m, respectively. The allocation of resources to the production is as follows: allocate a ij units of the resource R i to make one unit of the assortment P j. The market absorbs all the production and the revenue per unit of product sold is c j Euros. The Company s CEO needs to know the next year production plan (activity levels) in such a way that the resource constraints should be taken into account and the overall revenue to be earned from the selling of the assortments in the market should be maximized. J.R. Figueira (IST) FIO March 7-8, / 50

31 2. A general product-mix allocation problem Relevant Information from the Problem Description Sets: P = {P 1,..., P j,..., P n } is the set of products. R = {R 1,..., R i,..., R m } is the set of resources. Indices: j {1,..., n} is the set of the product indices. i {1,..., m} is the set of the resources indices. Parameters (Data): b i is the available amount of resource R i, i = 1,..., m. a ij is the amount of resource R i, i = 1,..., m, allocated to the production of one unit of product P j, j = 1,..., n. c j is the per unit revenue earned when P j is sold, j = 1,..., n. Decision Variables:... x j is the amount of product P j, j = 1,..., n, to be produced. J.R. Figueira (IST) FIO March 7-8, / 50

32 2. A general product-mix allocation problem The Linear Programming (LP) Model maximize z(x) = c 1 x c j x j c n x n subject to: a 11 x a 1j x j a 1n x n b 1... a i1 x a ij x j a in x n b i... a m1 x a mj x j a mn x n b m x 1,... x j,... x n 0 J.R. Figueira (IST) FIO March 7-8, / 50

33 3. Characterizing the resources Some definitions (1) Definition 1 (Slack Variables). From a technical point of view, in linear programming, a (non-negative) slack variable, say x n+i (or s i instead), is a variable that is added to a (lower than or equal) inequality type constraint, n j=1 a ijx j b i, to transform it in an equality type constraint: a i1 x a ij x j a in x n + s i = b i, s i 0 From an economic point of view, s i represents the unused amount of resource availability, b i. J.R. Figueira (IST) FIO March 7-8, / 50

34 3. Characterizing the resources Some definitions (2) Definition 2 (Biding Constraints). A constraint, i, is said biding when a i1 x a ij x j a in x n = b i. In other words, when the whole amount of resource R i, b i, was allocated to the production of P j, for j = 1,..., n. In this situation s i = 0. When s i > 0 the constraint is called non-biding. In such a case: a i1 x a ij x j a in x n < b i, and some amount of resource remains unused. J.R. Figueira (IST) FIO March 7-8, / 50

35 3. Characterizing the resources Some definitions (3) Definition 3 (Scarce and Abundant Resources). A resource R i is called scarce if it is associated with a biding constraint, i.e., when s i = 0. Otherwise, it is called abundant (s i > 0). This classification of resources is very important for an economical interpretation of the problem. J.R. Figueira (IST) FIO March 7-8, / 50

36 3. Characterizing the resources Simplex method: Graphical Illustration x B C D A x J.R. Figueira (IST) FIO March 7-8, / 50

37 3. Characterizing the resources Comments and Remarks Constraint 1 is biding: resource R 1 is scarce (Silicone). Constraint 2 is biding: resource R 2 is scarce (Hight-Tech machine). Constraint 3 is non-biding: resource R 3 is abundant (Robot 1). Constraint 4 is non-biding: resource R 4 is abundant (Robot 2). In what follow, these information are very important. J.R. Figueira (IST) FIO March 7-8, / 50

38 4. Sensitivity analysis: Graphical approach Relevant Questions: Sensitivity Analysis to the RHS When performing the sensitivity analysis of the availability of Resource 1, we do not care about the other data (ceteris paribus condition): b min 1 b 1 b max 1. As for a scarce resource, the most relevant question is to know how much we can increase its availability (this leads always to improve the value of the objective function). As for an abundant resource, the most relevant question is to know how much we can decrease its availability (this does not change the value of objective function). For instance, during crises periods, it is also important to how much we can decrease the availability of a scarce resource (this leads always to a degradation of the value of the objective function). Increase the availability of an abundant resource is not relevant (it can be increased till infinite). J.R. Figueira (IST) FIO March 7-8, / 50

39 4. Sensitivity analysis: Graphical approach Graphical Illustration: Upper Bound of b 1 x D H x J.R. Figueira (IST) FIO March 7-8, / 50

40 4. Sensitivity analysis: Graphical approach Graphical Illustration: Lower bound of b 1 x C D x J.R. Figueira (IST) FIO March 7-8, / 50

41 4. Sensitivity analysis: Graphical approach Comments and Remarks Compute b max 1. Observe the graphical representation. - When increasing the resource availability we can reach point H = (8, 3.5). - At this point the availability is Thus, b max 1 = At point H the value of the objective function is When increasing the availability of R 1 from 18 to 19.5, the objective function increases from 37 to The unitary impact is thus (38 37)/( ) = 2/3. Compute b min 1. - The availability of R 1 can be decreased till point C = (3, 6). - Thus, b min 1 = 12. Conclusion: 12 b J.R. Figueira (IST) FIO March 7-8, / 50

42 4. Sensitivity analysis: Graphical approach Upper bound of c 1 Lower bound of c 2 x D x J.R. Figueira (IST) FIO March 7-8, / 50

43 4. Sensitivity analysis: Graphical approach Objective function coefficients In the sensitivity analysis of c j, we should stay at point D. If we rotate clockwise the objective function we can reach the c max 1. It reaches its maximum value when the slope of the objective function is the same as the slope of the Constraint 1. It suffices to perform such a slope analysis. From the objective function: c max 1 x 1 + 4x 2 = K x 2 = K/4 (c max 1 /4)x 1 From the Equation 1: 2x 1 + x 2 = 18 x 2 = 18 2x 1 Conclusion: c max 1 /4 = 2 c max 1 = 8. J.R. Figueira (IST) FIO March 7-8, / 50

44 5. Sensitivity analysis: Tabular computations c j Basis x j x 1 x 2 x 3 x 4 x 5 x 6 ˆbi Ratio 0 x x x x t = 0 z j A z j c j x x x x t = 1 z j B z j c j x x x x t = 2 z j C z j c j J.R. Figueira (IST) FIO March 7-8, / 50

45 5. Sensitivity analysis: Tabular computations c j Basis x j x 1 x 2 x 3 x 4 x 5 x 6 ˆbi Ratio 0 x /3 2/ /3 1 3 x /3 1/ /3 1 0 x /3 1/ /3 1 4 x /3 2/ /3 1 t = 3 z j 3 4 2/3 5/3 0 0 D z j c j 0 0 2/3 5/ /3 1 Construct the system: 2 + 1/3 1 0 ( 1 6) 7 + 2/3 1 0 ( 1 21/2) 1 2/3 1 0 ( 1 3/2) 4 1/3 1 0 ( 1 12) Thus, Consequently, 12 b J.R. Figueira (IST) FIO March 7-8, / 50

46 5. Sensitivity analysis: Tabular computations c j (3 + δ 1 ) Basis x j x 1 x 2 x 3 x 4 x 5 x 6 ˆbi Ratio 0 x /3 2/ (3 + δ 1 ) x /3 1/ x /3 1/ x /3 2/ t = 3 z j (3 + δ 1 ) 4 (2/3 + 2/3δ 1 ) (5/3 1/3δ 1 ) 0 0 D z j c j 0 0 (2/3 + 2/3δ 1 ) (5/3 1/3δ 1 ) δ 1 Construct the system: { 2/3 + 2/3δ1 0 (δ 1 1) 5/3 1/3δ 1 0 (δ 1 5) Thus, 1 δ 1 5 Consequently, 2 c 1 8 J.R. Figueira (IST) FIO March 7-8, / 50

47 6. Sensitivity analysis: Matrix-vector computations Matrix computations As for b 1 we have to guarantee that B 1 b /3 2/ / /3 1/ /3 1/ / / /3 2/ /3 1 0 As for the coefficients of the objective function there are two cases: Our x 1 is basic: (c δ 1 B ) B 1 N (c δ 1 B ) 0. When x j is non-basic we have: z j (c j + δ j ) 0 J.R. Figueira (IST) FIO March 7-8, / 50

48 7. Conclusions Results Resource R 1 : 12 b Resource R 2 : 12 b Resource R 3 : 7 b 3 +. Resource R 4 : 4 b 4 +. Coefficient of x 1 : 2 c 1 8. Coefficient of x 2 : 1.5 c 2 6. J.R. Figueira (IST) FIO March 7-8, / 50

49 Part Bibliography

50 References Bazaraa, M., Jarvis, J., and Sherali, H. (1990). Linear Programming and Network Flows. John Wiley & Sons, New York, USA, second edition. Goldfarb, D. and Todd, M. J. (1989). Linear programming. In Nemhauser, G. L., Rinnoy Kan, A. H. G., and Todd, M. J., editors, Optimization, volume 1 of Handbooks in Operations Research and Management Science, pages North Holland, Amsterdam, The Netherlands. Hillier, F. and Lieberman, G. (2005). Introduction to Operations Research. The McGraw-Hill Companies, Inc., New York, USA, eighth edition. Taha, H. (2010). Operations Research: An Introduction. Prentice-Hall, Upper Saddle River, New Jersey, USA, 9th edition.

Introduction to Operations Research

Introduction to Operations Research (Week 5: Linear Programming: More on Simplex) José Rui Figueira Instituto Superior Técnico Universidade de Lisboa (figueira@tecnico.ulisboa.pt) March 14-15, 2016 This