4.5 Simplex method. LP in standard form: min z = c T x s.t. Ax = b

Transcription

1 4.5 Simplex method LP in standard form: min z = c T x s.t. Ax = b x 0 George Dantzig ( ) Examine a sequence of basic feasible solutions with non increasing objective function values until an optimal solution is reached or the LP is found to be unbounded (G. Dantzig 1947). At each iteration, we move from a basic feasible solution to a neighboring basic feasible solution. E. Amaldi -- Foundations of Operations Research -- Politecnico di Milano 1

2 Geometrically: c Generate a path along the edges of the polyhedron of the feasible solutions until an optimal vertex is reached. adjacent vertices E. Amaldi -- Foundations of Operations Research -- Politecnico di Milano 2

3 Given the correspondence between the basic feasible solutions and the vertices, we need to describe how to : Findan initial vertex or establish that the LP is infeasible. By applying the same method to another LP, see end of chapter. Move from a current vertex to a better adjacent vertex (in terms of objective function value) or establish that the LP is unbounded. Determine whether the current vertex is optimal. E. Amaldi -- Foundations of Operations Research -- Politecnico di Milano 3

4 Example: Move to an adjacent vertex x 1 + x 2 + s 1 = 6 (I) 2x 1 + x 2 + s 2 = 8 (II) x 1, x 2, s 1, s 2 0 Move from vertex 1) to vertex 5): In 1) x 1 = 0, x 2 = 0 s 1 = 6, s 2 = 8 with x B =[s 1,s 2 ], x N =[x 1,x 2 ] In 5) x 2 = 0, s 2 = 0 x 1 = 4, s 1 = 2 with x B =[x 1,s 1 ], x N =[x 2,s 2 ] Thus x 1 enters the basis B and s 2 exits the basis B. Observation: When moving from the current vertex to an adjacent vertex, we substitute one column of B (that of s 2 ) with one column of N (that of x 1 ). E. Amaldi -- Foundations of Operations Research -- Politecnico di Milano 4 P 2) 1) x 2 3) 6) 5) 4) (II) (I) x 1

5 By expressing the basic variables in terms of the non basic variables, we obtain s 1 =6 -x 1 -x 2 s 2 =8-2x 1 -x 2 Now we increase x 1 while keeping x 2 = 0. 2) x 2 3) 6) 1) 5) 4) (II) (I) Since s 1 = 6 - x 1 0 implies x 1 6 and s 2 =8-2x 1 0 implies x 1 8/2=4, the upper limit on the increase of x 1 is: x 1 min{6, 4}=4. P x 1 We move from vertex 1) to vertex 5) by letting x 1 enter the basis and s 2 exit the basis (s 1 = 2 and s 2 = 0). Note: When x 1 = 6, we obtain the infeasible basic solution 4). E. Amaldi -- Foundations of Operations Research -- Politecnico di Milano 5

6 General case: Given a basis B, the system Bx B + Nx N = b can be expressed in canonical form n Ax b a x =b for i= 1,...,m j= 1 ij j i x B Nx B b x Nx b 1 1 B N B N which emphasizes the basic feasible solution (x B,x N )=(B -1 b,0). This amounts to pre-multiply the system by B -1 : B -1 B x B + B -1 N x N = B -1 b I N b E. Amaldi -- Foundations of Operations Research -- Politecnico di Milano 6

7 In the canonical form n m x a x b for i 1,..., m B i ij N j i j 1 I x B + N x N = b the basic variables are expressed in terms of the non basic variables: x B = b - N x N. If we increase the value of a non basic x s (from value 0) while keeping all the other non basic variables to 0, the system becomes x a x b x b a x for i 1,..., m B is s i B i is s i i E. Amaldi -- Foundations of Operations Research -- Politecnico di Milano 7

8 x To guarantee for each i, we need to satisfy B i 0 bi bi - aisxs 0 xs for ais 0 a is The value of x s can be increased up to * b i min for ais 0 ais i 1,..., m The value of the basic variable x r of index If a is 0 for every i, there is no limit to the increase of x s r b i arg min for ais 0 i 1,..., m ais decreases to 0 and exits from the basis. E. Amaldi -- Foundations of Operations Research -- Politecnico di Milano 8

9 4.5.2 Reduced costs and optimality test Given a LP min{ c T x : Ax = b, x 0} and a feasible basis B of A, Ax = b can be rewritten as with B -1 b 0. B x B + N x N = b x B = B -1 b-b -1 N x N Basic feasible solution: x B = B -1 b, x N = 0 E. Amaldi -- Foundations of Operations Research -- Politecnico di Milano 9

10 By substitution we express the objective function in terms of only the non basic variables (for the current basis B): x B c T x = (c T B c T N) x = (c T B c T N N) B -1 b B -1 Nx N x N c T x=c T B B -1 b c T B B -1 Nx N + c T N x N = c T B B -1 b+ ( c T N c T B B -1 N ) x N only in terms of the non basic variables z 0 = cost of the basic feasible solution x B = B -1 b, x N = 0 c T N c T N c T B B -1 N reduced costs of the non basic variables x N E. Amaldi -- Foundations of Operations Research -- Politecnico di Milano 10

11 Definition: 0 T c T N c T c T c T B B -1 A = [c T B c T B B -1 B, c T N c T B B -1 N] c is the vector of reduced costs with respect to the basis B. I Defined for non basic as well as basic variables c j = change in the objective function value if the non basic variable x j is increased by 1 unit while the other non basic variables are kept equal to 0. The solution value changes by z = c j E. Amaldi -- Foundations of Operations Research -- Politecnico di Milano 11

12 Optimality test Given any LP min{ c T x : Ax = b, x 0 }( max{ } ) and a feasible basis B. If all reduced costs of the non basic variables c N are non negative (non positive) the basic feasible solution (x T B, x T N), where x B = B -1 b 0 and x N = 0, of cost c T B B -1 b is optimal. Proof c T 0 T implies that c T x= c T B B -1 b+c T N x N c T B B -1 b x 0, A x = b. Observation: This optimality condition is sufficient but in general not necessary. E. Amaldi -- Foundations of Operations Research -- Politecnico di Milano 12

13 x 2 Example: (II) c A P min x 1 x 2 3) T c x 1 x 2 + s 1 = 1 (I) x 1 + x 2 + s 2 = 3 (II) x 1, x 2, s 1, s 2 0 1) 2) In (4): x B = [x 1,s 2 ] = [1,2], z = -1 x N = [x 2,s 1 ] (I) 4) x 1 B Increasing x 2 from 0 to 1 keeping s 1 =0, we proceed along the edge to (x 1,x 2 ) = (2,1), z = -3 N T T c B 1 0 c 1 0 N B T T T 1 c N c N c BB N 2 0 E. Amaldi -- Foundations of Operations Research -- Politecnico di Milano 13 Since c 2 =-2 < 0, increasing x 2 to 1 (keeping the other non basic variables to 0) we improve the solution by -2

14 4.5.3 Changing basis (for minimization LP) Consider a feasible basis B and a non basic x s (in x N ) with reduced cost c s < 0. Increase x s as much as possible (x s enters the basis ) while keeping the other non basic variables equal to 0. The basic variable x r (in x B ) such that x r 0 imposes the tightest upper bound on the increase of x s (x r leaves the basis). If > 0, the new basic feasible solution has a better objective function value. The new basis differs w.r.t. the previous one by a single column (adjacent vertices). E. Amaldi -- Foundations of Operations Research -- Politecnico di Milano 14

15 To go from the canonical form of the current basic feasible solution B -1 B x B + B -1 N x N = B -1 b to that of an adjacent basic feasible solution, it is not necessary to compute B -1 from scratch. B -1 of the new basis B can be obtained incrementally by applying to the inverse of the previous basis (which differs w.r.t a single column) a unique pivoting operation. E. Amaldi -- Foundations of Operations Research -- Politecnico di Milano 15

16 Pivoting operation Given a matrix A 1. Select a coefficient a rs 0 (the pivot ) 2. Divide the r-th row by a rs 3. For each row with i r and a is 0, substract the r-th row multiplied by a is. Observation: The same operations (which do not affect the set of feasible solutions) are used in the Gaussian elimination method to solve systems of linear equations. E. Amaldi -- Foundations of Operations Research -- Politecnico di Milano 16

17 Example: min z = x 1 + x 2 + x 3 x 1 + 2x 2 + x 3 x 4 = 3 4x 2 x 3 + x 5 = 2 2x 1 + 3x 3 + x 4 = 5 x i 0 i =1,,5 To put the system in canonical form w.r.t. the basis where x 1, x 2, x 5 are basic pivot r A b ½ 1 ½ -½ /2-4 5 s columns of basic variables 0 1 -¼ -¾ /2 ½ 0 ¼ 1 5/2 E. Amaldi -- Foundations of Operations Research -- Politecnico di Milano 17

18 Moving to an adjacent vertex (basic feasible solution) Goals : i) improve the objective function value ii) preserve feasibility 1) Which non basic variable enters the basis? Choice of the pivot column s Any one with reduced cost c j < 0. One that yields the maximum z w.r.t. z = c BT B -1 b (the actual decrement z also depends on ). Bland s rule : s = min{ j : c j < 0}. For maximization problems : c j > 0 E. Amaldi -- Foundations of Operations Research -- Politecnico di Milano 18

19 Choice of the pivot row r otherwise no limit! 2) Which basic variable leaves the basis? bi Min ratio test: index i with smallest = * among those with a is >0. ais Thightest upper bound on increase of x s bi Bland s rule : r = min{ i : = *, a is > 0} ais randomly Unboundedness: If c j < 0 with a ij 0 i, no element of the j-th column can play the role of a pivot. The minimization problem is unbounded! E. Amaldi -- Foundations of Operations Research -- Politecnico di Milano 19

20 4.5.4 Tableau representation System z = c T x Ax = b with (implicit) nonnegativity constraints Initial tableau: - right hand side of the objective function x 1 x n 0 c T objetive function m rows b A right hand side vector E. Amaldi -- Foundations of Operations Research -- Politecnico di Milano 20

21 Consider a basis B and a partition A = [B N] x 1 x m x m+1 x n 0 c T B c T N b B N z = c T x z by pivoting operations (or pre-multiplying by B -1 ) we put the tableau in canonical form with respect to B: x 1 x m x m+1 x n -z x B[1] x B[m] -z c T N b I N z = c T B B -1 b+ c T N x N z 0 b= B -1 b basic variables E. Amaldi -- Foundations of Operations Research -- Politecnico di Milano 21

22 Example: min z = - x 1 -x 2 6x 1 + 4x 2 +x 3 = 24 3x 1 2x 2 + x 4 = 6 x i 0 i =1,, 4 Tableau w.r.t. the basis with columns 3, 4: r s x 1 x 2 x 3 x 4 -z x x pivot basis I 2x2 x 1 enters in the basis and x 4 exits the basis Pivot w.r.t. amounts to deriving an expression for x 1 from the pivot row and substituting it in all other rows. E. Amaldi -- Foundations of Operations Research -- Politecnico di Milano 22

23 Tableau w.r.t. the new basis: x 1 x 2 x 3 x 4 -z 2 0-5/3 0 1/3 x x /3 0 1/3 basis corresponding basic feasible solution: x T = (2, 0, 12, 0) with z = -2. reduced costs x 1 x E. Amaldi -- Foundations of Operations Research -- Politecnico di Milano 23

24 x 2 only non basic variable to enter the bais (c 2 = -5/3 < 0) x 3 only basic variable that exits the basis (a rs = 8 > 0) r s x 1 x 2 x 3 x 4 -z 2 0-5/3 0 1/3 x x /3 0 1/3 x 1 x 2 x 3 x 4 -z 6 ½ 0 ¼ 0 x 2 6 3/2 1 ¼ 0 x ½ 1 x 1 x 2 x 3 x 4 -z 9/ /24-1/12 x 2 3/ /8-1/4 x /12 1/6 s All reduced costs 0 r optimal basic (feasible) solution: x* T = (0, 6, 0, 18) with z * = - 6 E. Amaldi -- Foundations of Operations Research -- Politecnico di Milano 24

25 Simplex algorithm (LP with min) BEGIN Let B[1],,B[m] be the column indices of the inital feasible basis B; Construct the initial tableau A = {a[i,j]: 0 i m, 0 j n} in canonical form w.r.t. B; unbounded := false; optimal := false; WHILE (optimal = false) AND (unbounded = false) THEN IF a[0,j] 0 j=1,,m THEN optimal := true; /* for LP with min */ ELSE Select a non basic variable x s with a[0,s] < 0; IF a[i,s] 0 i=1,,m THEN unbounded := true; ELSE reduced costs a[ i,0] Determine index r that minimizes a[ i, s] with 1 i m and a[i,s] > 0; pivot(r,s) /* update tableau */ B[r] := s; END-IF END-IF Procedure pivot(r,s)? END E. Amaldi -- Foundations of Operations Research -- Politecnico di Milano 25

26 4.5.5 Degenerate basic feasible solutions and convergence Definition: A basic feasible solution x is degenarate if it contains at least one basic variable =0. x with more than n-m zeroes correspond to several distinct bases! Same vertex: x More than n constraints ( the m of Ax = b and more than n-m among the n of x 0 ) are satisfied with equality ( active ). E. Amaldi -- Foundations of Operations Research -- Politecnico di Milano 26

27 In the presence of degenerate basic (feasible) solutions, the basis change may not decrease the objective function value: If the current basic feasible solution is degenerate, * can be = 0 and hence the new basic feasible solution is indentical (same vertex). Also if * > 0, several basic variables may go to 0 when x s is increased to *. Thus yielding a degenerate basic feasible solution. One can cycle through a sequence of degenerate bases associated to the same vertex. E. Amaldi -- Foundations of Operations Research -- Politecnico di Milano 27

28 Several anticycling rules have been proposed for the choice of the variables that enter or exit the bases (indices r and s). Bland s rule: Among all candidate variables to enter/exit the basis (x s and x r ) always select the one with smallest index. Proposition: The Simplex algorithm with Bland s rule terminates after n m iterations. Robert. Bland finite number of pivots E. Amaldi -- Foundations of Operations Research -- Politecnico di Milano 28

29 In some pathological cases (see e.g. Klee Minty 72), the number of iterations grows exponentially w.r.t. n and/or m. However the Simplex algorithm is overall very efficient. Extensive experimental campaigns: The number of iterations grows linearly w.r.t. m ( m. 3m) and very slowly ( logarithmically) w.r.t. n. E. Amaldi -- Foundations of Operations Research -- Politecnico di Milano 29

30 4.5.6 Two-phase simplex method Phase 1: Determine an intial basic feasible solution. Example: a submatrix I 2x2 of A! min z = x 1 + x 3 x 1 +2x 2 5 x 2 +2x 3 = 6 x 1, x 2, x 3 0 x 1 + 2x 2 + x 4 = 5 x 4 0 Given (P) min z = c T x Ax = b x 0 Assumption: b 0 E. Amaldi -- Foundations of Operations Research -- Politecnico di Milano 30

31 Auxilliary problem with artificial variables y i, 1 i m, (P A ) min v m i 1 y i A x+ I y= b x 0, y 0 an obvious initial basic feasible solution y = b 0 1) If v * > 0, then (P) is infeasible. 2) If v * = 0, clearly y * = 0 and x * is a basic feasible solution of (P). E. Amaldi -- Foundations of Operations Research -- Politecnico di Milano 31

32 For 2) there are two cases: If y i is non basic i, with 1 i m, delete the corresponding columns and obtain a tableau in canonical form w.r.t. a basis; the row of z must be determined by substitution. If a basic y i (the basic feasible solution is degenerate), we perform a pivot operation w.r.t. a coefficient 0 of the row of y i so as to exchange y i with a non basic variable x j. cf. example E. Amaldi -- Foundations of Operations Research -- Politecnico di Milano 32

33 Example: min v = y 1 + y 2 (P A ) x 2 + 4x 3 + y 1 = 2-2x 1 + x 2-6x 3 + y 2 = 2 x 1, x 2, x 3, y 1, y 2 0 min z = x 1 + x x 3 x 2 + 4x 3 = 2 (P) -2x 1 + x 2-6x 3 = 2 x 1, x 2, x 3 0 Put v = y 1 +y 2 in canonical form by substituting the expression of y 1 and y 2 in terms of x 1, x 2 and x 3. x 1 x 2 x 3 y 1 y 2 -v y y E. Amaldi -- Foundations of Operations Research -- Politecnico di Milano 33

34 x 1 x 2 x 3 y 1 y 2 -v y y x 1 x 2 x 3 y 1 y 2 -v x optimal value v * = 0 y E. Amaldi -- Foundations of Operations Research -- Politecnico di Milano 34

35 By selecting as pivot the coefficient 2 of the row of y 2, we obtain: Equivalent optimal basis x 1 x 2 x 3 y 1 y 2 -v x x ½ -½ 0 The column 1 of I has been transferred in the area the original variables. optimal basic feasible solution of (P A ) x= (0, 2, 0) is a basic feasible solution of (P). E. Amaldi -- Foundations of Operations Research -- Politecnico di Milano 35

36 z = x 1 +x x 3 By substituting: canonical form non basic variable x 2 = 2-4x 3 x 1 = - 5x 3 z = 2 + x 3 x 1 x 2 x 3 -z x x Tableau corresponding to the initial basic feasible solution of (P). Since the basic feasible solution found is (already) optimal, here no need for the second phase! E. Amaldi -- Foundations of Operations Research -- Politecnico di Milano 36

37 4.5.7 Polynomial-time algorithms for LP Ellipsoid method (L. Khachiyan 1979) Theoretically important. Interior point methods (N. Karmarkar 1984, ) Narendra Karmarkar (1957-) Very efficient variants (e.g. barrier methods) for some types of instances (e.g. sparse and large-scale). E. Amaldi -- Foundations of Operations Research -- Politecnico di Milano 37