The simplex algorithm The simplex algorithm is the classical method for solving linear programs. Its running time is not polynomial in the worst case. It does yield insight into linear programs, however, and is often remarkably fast in practice. The simplex algorithm bears some similarity to Gaussian elimination (iteration)
Gaussian elimination begins with a system of linear equalities whose solution is unknown. In each iteration, we rewrite this system in a equivalent form that has some additional structure. After some number of iterations, we have rewritten the system so that the solution is simple to obtain.
To find the solution of the following system of linear equations: 2x + y - z = 8 ----- (1) -3x - y + 2z = -11 ----- (2) -2x + y + 2z = -3 ----- (3) Eliminate x in both (2) and (3): (2) + 3/2(1) (2) 0x + 1/2 y + 1/2 z = 1 ----(2) (3) + (1) (3) 0x + 2 y + z = 5 ----(3)
Eliminate y in (3): (3) - 4 (2) (3) 0x + 0y z = 1 -----(3) We have that (upper triangle) 2x + y z = 8 0x + 1/2 y + 1/2 z = 1 ----(1) ----(2) 0x + 0y z = 1 ----(3) Then, we have z = -1 from (3), y = 3 from (2) and z=-1, and x = 2 from (1) and y=3,z=-1
An example of the simplex algorithm Eq-1 Subject to We first convert the linear program from standard form into slack form.
Convert standard form to slack form Non-negative constraints are the only inequality constraints. All other constraints are equality constraints 1. Introduce slack variables x 4, x 5, x 6 for inequality X 4 = 30 x 1 x 2 3x 3 X 5 = 24 2x 1 2x 2 5x 3 X 6 = 36 4x 1 x 2 2x 3
Clearly, slack variables x 4, x 5, x 6 must be nonnegative, and they are called basic varibles, and the original variables x 1,x 2,x 3 are called non-basic variables. Therefore, we have x 1, x 2, x 3, x 4, x 5, x 6 > 0 _ 2. Let the object function be in slack form. z = 0 + 2x 1 3x 2 + 3x 3
The LP in Slack form
A solution is feasible if all of x 1, x 2,..., x 6 are Nonnegative there can be an infinite number of feasible solutions since there are 6 variables and only 3 equations Eq-2
Any setting of variables x 1, x 2, x 3 will define values for x 4, x 5, x 6. A solution is feasible if all x 1, x 2, x 3, x 4, x 5, x 6 are non-negative Basic solution is obtained by setting all nonbasic variables to zero and then calculating the values for basic variables as well as the value of object function z. (x 1, x 2, x 3, x 4, x 5, x 6, z) = (0,0,0,30,24,36,0).
the basic solution: set all the (non-basic) variables on the right-hand side to 0 and then compute the values of the (basic) variables on the left-hand side. is the basic solution it has value z = (3 0) + (1 0) + (2 0) = 0. If a basic solution is also feasible, we call it a basic feasible solution. If a basic solution is not feasible, we shall call Initialize-simplex procedure to find a basic solution if it exists. A basic solution corresponds to a vertex of simplex.
Our goal, in each iteration, is to reformulate the linear program so that the basic solution has a greater objective value. METHOD: 1.We choose a non-basic variable with positive coefficient in objective function. 2. We increase the value of this variable to a limit that will violate any of these constraints. 3. Then the non-basic variable as entering variable and the basic variable as leaving variable in this constraint will exchange places. This is called pivoting.
In our example, choose x 1 in objective function. increasing the value of x 1. As we increase x 1, the values of x 4, x 5, and x 6 all decrease. we cannot allow any of them to become negative in order to be a feasible solution. The third constraint is the tightest constraint, and it limits how much we can increase x 1 (=9). Obtain a new constraint from 3 rd constraint
To rewrite the other constraints with x 6 on the right-hand side, we substitute x 6 for x 1 we obtain X 4 = 21-3/4 x 2-5/2 x 3 + 1/4 x 6 X 5 = 6 3/2 X 2-4 X 3 + 1/2 X 6
Similarly, to rewrite our linear program in the following form: Eq3. this operation is called a pivot. a pivot chooses a non-basic variable x e (x 1 ),called the entering variable, and a basic variable x l (x 6 ) called the leaving variable, and exchanges their roles.
The linear program described in eq-3 is equivalent to the linear program described in eq-2. However the values of object function z are different. If we set zero to non-basic variables and calculate the values for basic variables and finally find the value for z, we have (9, 0, 0, 21, 6, 0) and z = (3*9) + (1*0) + (2*0) = 27.
we wish to find a new variable whose value might be increased. We do not increase x 6 since its coefficient is negative so this will decrease z We can try x 2 or x 3, say x 3. The third constraint is again the tightest one, and we will therefore rewrite the third constraint so that x 3 is on the left-hand side and x 5 is on the right-hand side. We then substitute this new equation into eq-3 and obtain the new, but equivalent, system
Eq-4 We obtain (33/4, 0, 3/2, 69/4, 0, 0) and z= 111/4. (27.75) The only way to increase the value of z is to increase x 2. since it is positive term.
We increase x 2 to 4, and it becomes non-basic. we solve eq-4 for x 2 and substitute in the other equations to obtain Eq-5.
At this point, all coefficients in the objective function are negative. As we shall see later in this chapter, this situation occurs only when we have rewritten the linear program so that the basic solution is an optimal solution. for this problem, the solution (8, 4, 0, 18, 0, 0), with objective value 28, is optimal.
Check the slack variables in the original form with the final solution for x 1, x 2, x 3 = 8, 4, 0, we have that x 4, x 5, x 6 = 18, 0, 0. That is, only slack variable x 4 has big slack. The values of coefficients in the original form of the example are integers, however in many real problems, it is real numbers. The coefficients intermediate form and the solution may also be real numbers.
Pivoting We now formalize the procedure for pivoting. The procedure PIVOT takes as input a slack form, given by the tuple (N, B, A, b, c, v), the index l of the leaving variable x l, and the index e of the entering variable x e. It returns the tuple describing the new slack form.
e e e
PIVOT works as follows. Lines 2 5 compute the coefficients in the new equation for x e by rewriting the equation that has x l on the left-hand side to instead have x e on the left-hand side. Lines 7 11 update the remaining equations by substituting the right-hand side of this new equation for each occurrence of x e.
Lines 13 16 do the same substitution for the objective function, and lines 18 and 19 update the sets of non-basic and basic variables. Line 20 returns the new slack form. As given, if a le = 0, PIVOT would cause an error by dividing by 0, PIVOT is called only when a le 0.
Lemma 29.1: Consider a call to PIVOT(N, B, A, b, c, v, l, e) in which a le 0. Let the values returned from the call be and let x denote the basic solution after the call. Then
The formal simplex algorithm we could have had several other issues to address: How do we determine if a linear program is feasible? What do we do if the linear program is feasible, but the initial basic solution is not feasible? How do we determine if a linear program is unbounded? How do we choose the entering and leaving variables?
We therefore assume that we have a procedure INITIALIZE-SIMPLEX(A, b, c) that takes as input a linear program in standard form, that is, an m n matrix A = (a ij ), an m-dimensional vector b = (b i ), and an n-dimensional vector c = (c j ). If the problem is infeasible, it returns a message that the program is infeasible and then terminates. Otherwise, it returns a slack form for which the initial basic solution is feasible.
For L has a feasible solution: Let L be a linear program in standard form,. maximize Σ j=1 to n c j x j subject to Σ j=1 to n a ij x j < b j for i=1,2,...,m x j > 0 for j = 1,2,...,n.
Let L max be the following linear program with n+1 variables. maximize x 0 subject to Σ j=1 to n a ij x j - x 0 < _ b i for i=1,, m x j _ > 0 for j=0,,n. Then, L is feasible iff the optimal objective solution for L max is 0.
The procedure SIMPLEX takes as input a linear program in standard form, as just described. It returns an n-vector that is an optimal solution to the linear program.
maximize 2x 1 - x 2 _ subject to 2x 1 - x 2 < 2 x 1-5x 2 < -4 x 1, x 2 > 0 If we set x 1 and x 2 to zero as the basic solution, then in the slack form: maximize z= 2x 1 - x 2 subject to x 3 = 2-2x 1 + x 2 x 4 = -4 - x 1 + 5x 2 x 1, x 2, x 3, x 4 _ > 0 x 4 would be negative, which is not a feasible. (x 1,x 2,x 3,x 4,z)=(0,0,2,-4,0)
We set an auxiliary linear program: maximize - x 0 subject to 2x 1 - x 2 - x _ 0 < 2 x 1-5x 2 - x _ 0 < -4 x 1, x 2, x 0 _ > 0 and the slack form: z = - x 0 x 3 = 2-2x 1 + x 2 + x 0 x 4 = -4 - x 1 + 5x 2 + x 0 x 0 = 4 + x 1-5x 2 + x 4
Call PIVOT: x 0 as entering varible and x 4 as leaving variable. We have a new slack form: z= - 4 - x 1 + 5x 2 - x 4 x 0 = 4 + x 1-5x 2 + x 4 x 3 = 6 - x 1-4x 2 + x 4 with (x 0,x 1,x 2,x 3,x 4 ) = (4,0,0,6,0), which is feasible and z=-4. However, x 0 is not zero. We continue to call PIVOT: x 2 as entering varible and x 0 as leaving variable. We have a new slack form: z = - x 0 x 2 = 4/5-1/5x 0 + 1/5x 1 + 1/5x 4 x 3 = 14/5 + 4/5x 0-9/5x 1 + 1/5x 4 x 3 = 6 - x 1-4 ( 4/5-1/5x 0 + 1/5x 1 + 1/5x 4 )+ x 4 x 0 = 5/4 ( -14/5 + 9/5x 1 + x 3-5x 4 ) = -7/2 +9/4x 1 + 5/4x 3-1/4x 4
the basic solution: (x 0,x 1,x 2,x 3,x 4,z) = (0,0,4/5,14/5,0,0). Hence the initial linear program is feasible by the lemma. To obtain the maximum of the object function, we rewrite the initial slack form: maximize z= 2x 1 - x 2 = 2x 1 - (4/5-1/5x 0 + 1/5x 1 + 1/5x 4 ) = 4/5 + 9/5x 1-1/5x 4 subject to x 3 = 14/5-9/5x 1 + 1/5x 4 x 2 = 4/5 + 1/5x 1 x 1, x 2, x 3, x _ 4 > 0 (note that since x 0 =0 and it is removed) (x 0,x 1,x 2,x 3,x 4,z) = (0,0,4/5,14/5,0,4/5)
If x 0 not equal to zero, then the initial linear program has no feasible solution. To see this, note that variable x 0 is nonnegative by constraint. Hence in an optimal solution, the value of objective function -x 0 must be negative. Moreover, the value must be finite as variable x i = 0 for i = 1,2,...,n and variable x 0 = min i=1 to n {b i }. Therefore, the value of objective function is - min i=1 to n {b i }.
Convert a general form to standard form If the objective function to be minimize, then negative the coefficients of the objective function to obtain the maximize form. If a variable x i has a negative constraint replace occurrences of x i with x i - x i, and add constraints: x i > _ 0 and x i _ > 0. If a constraint is in equality form, then replace it with two inequality forms: > and < If a constraint is a _ > form, then multiply the constraint by -1 to obtain a < _ form.
The SIMPLEX procedure works as follows. In line 1, it calls the procedure INITIALIZESIMPLEX(A, b, c), described above, which either determines that the linear program is infeasible or returns a slack form for which the basic solution is feasible. The main part of the algorithm is given in the while loop in lines 2 11. If all the coefficients in the objective function are negative, then the while loop terminates. Otherwise, in line 3, we select a variable x e whose coefficient in the objective function is positive to be the entering variable.
While we have the freedom to choose any such variable as the entering variable, we assume that we use some prespecified deterministic rule.
Next, in lines 4 8, we check each constraint, and we pick the one that most severely limits the amount by which we can increase x e without violating any of the non-negativity constraints. the basic variable associated with this constraint is x l. Again, we may have the freedom to choose one of several variables as the leaving variable, but we assume that we use some pre-specified deterministic rule.
If none of the constraints limits the amount by which the entering variable can increase, the algorithm returns "unbounded" in line 10. Otherwise, line 11 exchanges the roles of the entering and leaving variables by calling the subroutine PIVOT(N, B, A, b, c, v, l, e), as described above.
Lines 12 15 compute a solution for the original _ linear-programming variables x 1, x 2,..., x n by setting all the nonbasic variables to 0 and each basic variable to b i. We shall see that this solution can be proven to be an optimal solution to the linear program. Finally, line 16 returns the computed values of these original linear-programming variables.
To show that SIMPLEX is correct, we first show that if SIMPLEX has an initial feasible solution and eventually terminates, then it either returns a feasible solution or determines that the linear program is unbounded. Then, we show that SIMPLEX terminates. Finally, we can show that the solution returned is optimal.
Lemma 29.2: Given a linear program (A, b, c), suppose that the call to INITIALIZE-SIMPLEX in line 1 of SIMPLEX returns a slack form for which the basic solution is feasible. Then if SIMPLEX returns a solution in line 16, that solution is a feasible solution to the linear program. If SIMPLEX returns "unbounded" in line 10, the linear program is unbounded.
At each iteration, SIMPLEX maintains A, b, c, and v in addition to the sets N and B. Although explicitly maintaining A, b, c, and v is essential for the efficient implementation of the simplex algorithm, it is not strictly necessary. In other words, the slack form is uniquely determined by the sets of basic and nonbasic variables. Before proving this fact, we prove a useful algebraic lemma.
Lemma 29.3: Let I be a set of indices. For each i Є I, let α i and β i be real numbers, and let x i be a real-valued variable. Let γ be any real number. Suppose that for any settings of the x i, we have Then α i = β i for each i Є I, and γ = 0.
Lemma 29.4: Let (A, b, c) be a linear program in standard form. Given a set B of basic variables, the associated slack form is uniquely determined. it is possible that an iteration leaves the objective value unchanged. This phenomenon is called degeneracy
The objective value is changed by the assignment in line 13 of PIVOT. Since SIMPLEX calls PIVOT only when c e > 0, the only way for the objective value to remain unchanged (i.e., )is for to be 0. This value is assigned as in line 2 of PIVOT. Since we always call PIVOT with a le 0, we see that for to equal 0, and hence the objective value to be unchanged, we must have b l = 0.
Indeed, this situation can occur. Consider the linear program Suppose that we choose x 1 as the entering variable and x 4 as the leaving variable. After pivoting, we obtain
At this point, our only choice is to pivot with x 3 entering and x 5 leaving. Since x 5 = 0, the objective value of 8 remains unchanged after pivoting: The objective value has not changed, but our representation has.
We say that SIMPLEX cycles if the slack forms at two different iterations are identical, in which case, since SIMPLEX is a deterministic algorithm, it will cycle through the same series of slack forms forever. Lemma 29.5: If SIMPLEX fails to terminate in at most iterations, then it cycles.
Cycling is theoretically possible, but extremely rare. It is avoidable by choosing the entering and leaving variables somewhat more carefully. One option is to perturb the input slightly so that it is impossible to have two solutions with the same objective value. A second is to break ties lexicographically, and a third is to break ties by always choosing the variable with the smallest index. The last strategy is known as Bland's rule.
Lemma 29.6: If in lines 3 and 8 of SIMPLEX, ties are always broken by choosing the variable with the smallest index, then SIMPLEX must terminate. Lemma 29.7: Assuming that INITIALIZE-SIMPLEX returns a slack form for which the basic solution is feasible, SIMPLEX either reports that a linear program is unbounded, or it terminates with a feasible solution in at most iterations.