Simplex Method for LP (II) Xiaoxi Li Wuhan University Sept. 27, 2017 (week 4) Operations Research (Li, X.) Simplex Method for LP (II) Sept. 27, 2017 (week 4) 1 / 31
Organization of this lecture Contents: Tie for entering/leaving variable Unbounded objective z Multiple optimal solutions "=" or " " constraint: the artificial-variable technique and the Big-M method Negative variables allowed The Two-Phase method for the artificial-variable technique Operations Research (Li, X.) Simplex Method for LP (II) Sept. 27, 2017 (week 4) 2 / 31
Tie for the entering or leaving variable When there is a tie for determining the entering or the leaving variable, just choose one arbitrarily. For an example: Eq. Basic variables z x 1 x 2 s 1 s 2 s 3 RHS (0) z 1-3 -3 0 0 0 0 (1) s 1 0 1 0 1 0 0 4 (2) s 2 0 0 2 0 1 0 12 (3) s 3 0 3 2 0 0 1 18 It is possible to choose either x 1 or x 2 as the entering variable. They will lead to different optimal solution paths. Operations Research (Li, X.) Simplex Method for LP (II) Sept. 27, 2017 (week 4) 3 / 31
No leaving variable: unbounded objective z For an example: Eq. Basic variables z x 1 x 2 s 1 s 2 RHS (0) z 1-3 -5 0 0 0 (1) s 1 0 1 0 1 0 0 (2) s 2 0 3-2 0 1 6 Every coefficient in the pivot column is either zero or negative. Then the entering variable x 2 can be increased infinitely large (while x 1 = 0): { x 1 + 0x 2 + s 1 = 4 = s 1 = 4 0 3x 1 2x 2 + s 2 = 6 = 2x 2 = s 2 6 6. As x 2 increases while keeping x 1 = 0, the objective z tends to infinity. Operations Research (Li, X.) Simplex Method for LP (II) Sept. 27, 2017 (week 4) 4 / 31
Multiple optimal solutions. It is possible for us to use the simplex method to identify all the CPF solutions if there are multiple. Then, by taking convex combinations of the CPF solutions, we obtain all the optimal solutions. When there are multiple optimal solutions, at least one of the nonbasic variables s i has a zero coefficient in the final Eq. (0). Increasing such variable s i will not increase the objective z, thus it is possible to leave the supporting hyperplane "s i = 0" to arrive at an adjacent CPF optimal solution (having the same z value). Operations Research (Li, X.) Simplex Method for LP (II) Sept. 27, 2017 (week 4) 5 / 31
For an example: Max z = 3x 1 + 2x 2, Initialization: Eq. Basic variables z x 1 x 2 s 1 s 2 s 3 RHS Optimal? (0) z 1-3 -2 0 0 0 0 NO (1) s 1 0 1 0 1 0 0 4 (2) s 2 0 0 2 0 1 0 12 (3) s 3 0 3 2 0 0 1 18 The pivot number: 1. Operations Research (Li, X.) Simplex Method for LP (II) Sept. 27, 2017 (week 4) 6 / 31
Iteration 1: Eq. Basic variables z x 1 x 2 s 1 s 2 s 3 RHS Optimal? (0) z 1 0-2 3 0 0 12 NO (1) x 1 0 1 0 1 0 0 4 (2) s 2 0 0 2 0 1 0 12 (3) s 3 0 0 2-3 0 1 6 Iteration 2: Eq. Basic variables z x 1 x 2 s 1 s 2 s 3 RHS Optimal? (0) z 1 0 0 0 0 1 18 Yes (1) x 1 0 1 0 1 0 0 4 (2) s 2 0 0 0 3 1-1 6 (3) x 2 0 0 1-3/2 0 1/2 3 Optimality Test: after Iteration 2, s 1,s 3 are nonbasic variables, and their coefficients in Eq. (0) are respectively 0 and 1. = (4,3,0,6,0) is an optimal solution. Operations Research (Li, X.) Simplex Method for LP (II) Sept. 27, 2017 (week 4) 7 / 31
The coefficient for the nonbasic variable s 1 in Eq. (0) is 0, thus it is possible to find another BF optimal solution following an Extra Iteration: Eq. Basic variables z x 1 x 2 s 1 s 2 s 3 RHS Optimal? (0) z 1 0 0 0 0 1 18 Yes (1) x 1 0 1 0 0-1/3 1/3 2 (2) s 1 0 0 0 1 1/3-1/3 2 (3) x 2 0 0 1 0 1/2 0 6 We obtain another optimal BF solution (2,6,2,0,0). The set of optimal solutions is {α(4,3,0,6,0) + (1 α)(2,6,2,0,0) for some α [0,1]}. Check. Now s 2 is a nonbasic variable with 0 its coefficient in Eq. (0). Re-do an extra iteration by increasing s 2 leads to the previous simplex table. Operations Research (Li, X.) Simplex Method for LP (II) Sept. 27, 2017 (week 4) 8 / 31
The artificial-variable technique for LP in non-standard forms So far, we only studied the simplex method for LP in standard form. In this section we aim at solving the LP in non-standard forms, which are sorted into one of the following situations: z is minimized. some RHS constant "b i " is negative. some functional constraints with "=" or " ". some nonnegative constraint "x j 0" is withdrew. Idea: transform the non-standard LP problem to some equivalent LP in standard form, mainly by introducing the artificial variable. Operations Research (Li, X.) Simplex Method for LP (II) Sept. 27, 2017 (week 4) 9 / 31
z = c 1 x 1 + + c n x n is minimized. Let z = z be the new objective function, then subject to the same constraints, Max z = c 1 x 1 c n x n is equivalent to Min z = c 1 x 1 + + c n x n Some RHS constant "b i " is negative. Multiplying both sides 1 to obtain a positive constant on the RHS. For example: 5x 1 + 6x 2 5 5x 1 6x 2 5, which is in standard form; 5x 1 + 6x 2 3 5x 1 6x 2 3, with " ", to be solved. 5x 1 + 6x 2 = 1 5x 1 6x 2 = 1, with "=", to be solved. Operations Research (Li, X.) Simplex Method for LP (II) Sept. 27, 2017 (week 4) 10 / 31
Some functional constraints with "=". For an example: s.t. Maximize z = 3x 1 + 5x 2 x 1 4 1 2x 2 12 2 3x 1 + 2x 2 =18 3 x 1,x 2 0 + (0.1) The problem is as before except that we have replaced in Eq. + 3x 1 + 2x 2 18 by 3x 1 + 2x 2 = 18. Operations Research (Li, X.) Simplex Method for LP (II) Sept. 27, 2017 (week 4) 11 / 31
The feasible region for the LP problem with an equality constraint: Operations Research (Li, X.) Simplex Method for LP (II) Sept. 27, 2017 (week 4) 12 / 31
As before, we introduce slack variables to constraints 1, 2 to obtain: s.t. Maximize z = 3x 1 + 5x 2 x 1 + s 1 = 4 1 2x 2 + s 2 = 12 2 3x 1 + 2x 2 = 18 3 x 1, x 2, s 1, s 2 0 + (0.2) The previous LP (0.1) is not in standard form, thus this LP (0.2) is not in canonical form: there is no slack variable in constraint 3. The crucial point is that the Initialization step of the simplex method can not be carried out directly since there is not an obvious BF solution ( as the origin (0,0,4,12,18) in standard LP (0.1) ). Operations Research (Li, X.) Simplex Method for LP (II) Sept. 27, 2017 (week 4) 13 / 31
To solve the problem, introduce an artificial variable s 3 0 in 3, i.e. 3x 1 + 2x 2 + s 3 = 18 3, to obtain an initial BF solution (x 1,x 2,s 1,s 2, s 3 ) = (0,0,4,12,18), where {s 1,s 2, s 3 } are the basic variables. This is as to extend the feasible region to include the origin (0,0). However, in order to obtain an equivalent artificial problem, one puts a heavy penalty in the objective function for s 3 0 being strictly positive, i.e. Max z = 3x 1 + 5x 2 M s 3, where M > 0 is sufficiently large. The technique of introducing an artificial variable into the a LP constraint is called the artificial-variable technique; the method of relaxing the constraint and introducing a huge penalty on infeasible solutions (outside the pre-relaxed region) is called the Big-M method. Operations Research (Li, X.) Simplex Method for LP (II) Sept. 27, 2017 (week 4) 14 / 31
Define the artificial problem as: Define s 3 = 18 3x 1 2x 2 s.t. Maximize z = 3x 1 + 5x 2 M s 3 x 1 4 1 2x 2 12 2 3x 1 + 2x 2 18 3 ( so 3x1 + 2x 2 + s 3 = 18 ) x 1, x 2, s 3 0 + (0.3) Operations Research (Li, X.) Simplex Method for LP (II) Sept. 27, 2017 (week 4) 15 / 31
Graphical analysis for the artificial problem: Operations Research (Li, X.) Simplex Method for LP (II) Sept. 27, 2017 (week 4) 16 / 31
Claim Suppose that LP (0.1) has a nonempty feasible region. Then for M > 0 sufficiently large a, any optimal solution (x1,x 2, s 3 ) to LP problem (0.3) satisfies: (i). (x1,x 2 ) is optimal to LP problem (0.1); (ii). s 3 = 0. a Compared with other coefficients/constants in LP (0.1). To apply the simplex method to artificial problem (0.3) (in standard form!), we first write its associated LP problem in canonical form. Step I. Introduce slack variables "s 1,s 2 " to " " constraints 1 and 2, and put the objective equation into the equation system: Eq.(0) z 3x 1 5x 2 + M s 3 = 0 Eq.(1) x 1 + s 1 = 4 Eq.(2) 2x 2 + s 2 = 12 Eq.(3) 3x 1 + 2x 2 + s 3 = 18 Operations Research (Li, X.) Simplex Method for LP (II) Sept. 27, 2017 (week 4) 17 / 31
Step II. Use elementary algebraic operations to write the objective function in terms of the nonbasic variables {x 1,x 2 } only. {s 1,s 2, s 3 } are basic variables, and {x 1,x 2 } are the nonbasic variables, so we make the operation Eq.(0 ) = Eq.(0) M Eq.(3) to obtain the new equation system (ready to apply the simplex method): Eq.(0) z (3M + 3)x 1 (2M + 5)x 2 = 18M Eq.(1) x 1 + s 1 = 4 Eq.(2) 2x 2 + s 2 = 12 Eq.(3) 3x 1 + 2x 2 + s 3 = 18 Operations Research (Li, X.) Simplex Method for LP (II) Sept. 27, 2017 (week 4) 18 / 31
We now put the coefficients of the above system into the simplex table to start solving the artificial problem with simplex method. Initialization Eq. Basic variables z x 1 x 2 s 1 s 2 s 3 RHS (0) z 1-3M-3-2M-5 0 0 0-18M (1) s 1 0 1 0 1 0 0 4 (2) s 2 0 0 2 0 1 0 12 (3) s 3 0 3 2 0 0 1 18 Keep in mind: M > 0 is larger than any ordinary number you encounter during the solution process. In view of this, we see that the initial BF solution (0,0,4,12,18) is not optimal and we choose x 1 as the entering variable since 3M + 3 > 2M + 5. The minimum ratio test identifies s 1 as the leaving variable so "1" is the pivot number. Operations Research (Li, X.) Simplex Method for LP (II) Sept. 27, 2017 (week 4) 19 / 31
Iteration 1 Iteration 2 Eq. Basic variables z x 1 x 2 s 1 s 2 s 3 RHS (0) z 1 0-2M-5 3M+3 0 0-6M+12 (1) x 1 0 1 0 1 0 0 4 (2) s 2 0 0 2 0 1 0 12 (3) s 3 0 0 2-3 0 1 6 Eq. Basic variables z x 1 x 2 s 1 s 2 s 3 RHS (0) z 1 0 0-9/2 0 M+5/2 27 (1) x 1 0 1 0 1 0 0 4 (2) s 2 0 0 0 3 1-1 6 (3) x 2 0 0 1-3/2 0 1/2 3 Iteration 3 (after which it is optimal) Eq. Basic variables z x 1 x 2 s 1 s 2 s 3 RHS (0) z 1 0 0 0 3/2 M+1 36 (1) x 1 0 1 0 0-1/3 1/3 2 (2) s 1 0 0 0 1 1/3-1/3 2 (3) x 2 0 0 1 0 1/2 0 6 Operations Research (Li, X.) Simplex Method for LP (II) Sept. 27, 2017 (week 4) 20 / 31
The optimal solution path is (0,0) (4,0) (4,3) (2,6); After Iteration 2, the artificial variable s 3 leaves the basis, i.e. we reach at the supporting hyperplane " s 3 = 0" ("3x 1 + 2x 2 = 18"), thus the feasible region of the original LP (0.1) Another iteration leads us to the optimal solution (2, 6). Operations Research (Li, X.) Simplex Method for LP (II) Sept. 27, 2017 (week 4) 21 / 31
Some functional constraints with " ". Adding one surplus variable first and then one artificial variable. For an example: s.t. Maximize z = 0.4x 1 + 0.5x 2 0.3x 1 + 0.1x 2 2.7 1 0.6x 1 + 0.4x 2 6 2 x 1, x 2 0 + (0.4) Let "x 5 0" be a surplus variable. The system (0.4) is equivalent to: 0.3x 1 + 0.1x 2 2.7 1 0.6x 1 + 0.4x 2 x 5 = 6 2 x 1, x 2, x 5 0 + = It is then a problem of LP with an "=" constraint, which can be solved by introducing an artificial variable (same as before). (0.5) Operations Research (Li, X.) Simplex Method for LP (II) Sept. 27, 2017 (week 4) 22 / 31
Remark. It is possible to have several constraints of type " ", "=" or a mix of them. In this case, several artificial variables are needed (one for each constraint), and we put a big M for each of them in the objective function. At an optimal solution, all artificial variables should be zero. See pages. 120-125 in H-L for an example. It is possible to obtain at an optimal solution to an artificial LP problem that the artificial variable is nonzero. In this case, we deduce that there is no feasibility for the original LP problem. The coefficient for nonbasic variable in Eq. (0) is in the form of am + b. In identifying the entering variable, when there is a tie on the multiplication term "a", we look at the addition term "b". Operations Research (Li, X.) Simplex Method for LP (II) Sept. 27, 2017 (week 4) 23 / 31
Negative variables allowed In some applications, the constraint x j 0 for some j may not be required. A negative x j may be interpreted as a decrease in stock. Two possibilities: x j b j for some b j R. In this case, we define x j = x j b j 0 to be a decision variable, and substitute "x j = x j + b j" in other equations. x j R with no constraint. Introduce x j +,xj variables with x j = x j + xj. 0 two decision Operations Research (Li, X.) Simplex Method for LP (II) Sept. 27, 2017 (week 4) 24 / 31
For an example. s.t. Maximize z = 3x 1 + 5x 2 x 1 4 2x 2 12 3x 1 + 2x 2 18 x 2 0 is equivalent to the following (letting x j = x + j x j ) s.t. Maximize z = 3x 1 + 3x 1 + 5x 2 x 1 + x 1 4 2x 2 12 3x 1 + 3x 1 + 2x 2 18 x 1 +, x 1, x 2 0 Operations Research (Li, X.) Simplex Method for LP (II) Sept. 27, 2017 (week 4) 25 / 31
The Two-Phase method An easier way to treat the artificial problem involving no "Big M". Idea: first, start from the origin and arrive at the feasible region, by minimizing only the penalties; next, optimize the objective function within the feasible region. Apply the simplex method in two phases: Phase I. Minimize a new objective function as the sum of all the artificial variables. Conclude that the problem is infeasible if the resulting optimal solution contains a strictly positive artificial variable; Proceed to Phase II when all artificial variables leave the basic variables, thus equal zero, at an optimal solution. Phase II. Take the optimal solution in Phase I as an initial BF solution, and apply the simplex method to the original LP problem. Operations Research (Li, X.) Simplex Method for LP (II) Sept. 27, 2017 (week 4) 26 / 31
For an example. s.t. Maximize z = 3x 1 + 5x 2 x 1 4 1 2x 2 12 2 3x 1 + 2x 2 = 18 3 x 1,x 2 0 + Prepare as in the Big-M method: introducing slack variables (or even surplus variable if needed) and artificial variables to obtain an equivalent artificial problem in canonical form: s.t. Maximize z = 3x 1 + 5x 2 + M s 3 x 1 + s 1 = 4 1 2x 2 + s 2 = 12 2 3x 1 + 2x 2 + s 3 = 18 3 x 1, x 2, s 1, s 2, s 3 0 + (0.6) Operations Research (Li, X.) Simplex Method for LP (II) Sept. 27, 2017 (week 4) 27 / 31
To solve the above artificial LP (0.6), we use the two-phase method instead of the Big-M method. Phase I. s.t. Minimize y = s 3 x 1 + s 1 = 4 1 2x 2 + s 2 = 12 2 3x 1 + 2x 2 + s 3 = 18 3 x 1, x 2, s 1, s 2, s 3 0 + {s 1,s 2, s 3 } are the initial basic variables, so we need first to replace s 3 by x 1 and x 2 using equation " s 3 = 18 3x 1 2x 2 ". Further, this is a minimization problem, defining y = y, one obtains the Eq.(0) : y 3x 1 2x 2 = 18. Operations Research (Li, X.) Simplex Method for LP (II) Sept. 27, 2017 (week 4) 28 / 31
Initialization Iteration 1 Iteration 2 Eq. Basic variables -y x 1 x 2 s 1 s 2 s 3 RHS (0) -y 1-3 -2 0 0 0-18 (1) s 1 0 1 0 1 0 0 4 (2) s 2 0 0 2 0 1 0 12 (3) s 3 0 3 2 0 0 1 18 Eq. Basic variables -y x 1 x 2 s 1 s 2 s 3 RHS (0) -y 1 0-2 3 0 0-6 (1) x 1 0 1 0 1 0 0 4 (2) s 2 0 0 2 0 1 0 12 (3) s 3 0 0 2-3 0 1 6 Eq. Basic variables -y x 1 x 2 s 1 s 2 s 3 RHS (0) -y 1 0 0 0 0 1 0 (1) x 1 0 1 0 1 0 0 4 (2) s 2 0 0 0 3 1-1 6 (3) x 2 0 0 1-3/2 0 1/2 3 Operations Research (Li, X.) Simplex Method for LP (II) Sept. 27, 2017 (week 4) 29 / 31
After Iteration 2, it is optimal for the Phase I problem, where the artificial variable s 3 leaves the basis. The optimal BF solution is such that (x 1,s 2,x 2 ) are basic variables equal to (4,6,3). Phase II. Maximize z = 3x 1 + 5x 2 x 1 + s 1 = 4 1 3s 1 + s 2 s 3 = 6 2 s.t. x 2 3 2 s 1 + 1 2 s 3 = 3 3 x 1, x 2, s 1, s 2, s 3 0 + It s ready to apply the simplex method to the above LP in canonical form except that the objective function needs to be revised to contain only nonbasic variables {s 1, s 3 }: 1 : x 1 = 4 s 1 ; 3 : x 2 = 3 + 3 2 s 1 1 2 s 3, we obtain the revised objective function: z = 3(4 s 1 ) + 5(3 + 3 2 s 1 1 2 s 3) = 27 + 9 2 s 1 5 2 s 3. Operations Research (Li, X.) Simplex Method for LP (II) Sept. 27, 2017 (week 4) 30 / 31
Initialization (of Phase II, compare with Iteration in Big-M method) Eq. Basic variables z x 1 x 2 s 1 s 2 s 3 RHS (0) z 1 0 0-9/2 0 x 27 (1) x 1 0 1 0 1 0 x 4 (2) s 2 0 0 0 3 1 x 6 (3) x 2 0 0 1-3/2 0 x 3 Iteration 1 (of Phase II) Eq. Basic variables z x 1 x 2 s 1 s 2 s 3 RHS (0) z 1 0 0 0 3/2 x 36 (1) x 1 0 1 0 0-1/3 x 2 (2) s 1 0 0 0 1 1/3 x 2 (3) x 2 0 0 1 0 1/2 x 6 It is optimal the BF solution: (x 1,s 1,x 2 ) = (2,2,6) basic variables and (s 2, s 3 ) = (0,0) the nonbasic variables. The optimal BF solution to the original LP problem is then (x 1,x 2 ) = (1,6) and the optimal value is 36. Operations Research (Li, X.) Simplex Method for LP (II) Sept. 27, 2017 (week 4) 31 / 31