The Dual Simplex Algorithm

Size: px

Start display at page:

Download "The Dual Simplex Algorithm"

Alaina Watson
5 years ago
Views:

1 p. 1 The Dual Simplex Algorithm Primal optimal (dual feasible) and primal feasible (dual optimal) bases The dual simplex tableau, dual optimality and the dual pivot rules Classical applications of linear programming: the use of the primal and the dual simplex methods, examples

2 p. 2 Recall: Linear Programming Duality Consider the (primal) linear program: z = max c T x s.t. Ax = b x 0 where A is an m n matrix, b is a column m-vector, x is a column n-vector, and c T is a row n-vector By the Karush-Kuhn-Tucker Conditions, x is optimal if and only if there is (v T,w T ) so that Ax = b, x 0 (P) c T w T A+v T = 0, v T 0 (D) v T x = 0 (CS)

3 p. 3 Recall: Linear Programming Duality Let x be a basic feasible solution and let B denote the corresponding basis matrix z x B x N RHS z 1 0 c B T B 1 N c N T c B T B 1 b 0. sor x B 0 I m B 1 N B 1 b 1..m sorok Choose the dual variables as follows: w T = c B T B 1, v T = [ 0 }{{} basic c T B B 1 T N c N ] } {{ } nonbasic (P) holds since x is feasible (CS) holds identically since v T x = 0x B +(c B T B 1 N c N T )0 0

4 p. 4 Recall: Linear Programming Duality One of the constraints of (D), namely c T w T A+v T = 0 also holds identically Separating to basic and nonbasic components: c T w T A+v T = (c B T,c N T ) Component-wise: w T (B,N)+(0,c B T B 1 N c N T ) c B T w T B +0 = c B T c B T B 1 B 0 (basic) c N T w T N +(c B T B 1 N c N T ) = c B T B 1 N +c B T B 1 N 0 (nonbasic) The other part of (D), v T 0, only holds if B is optimal

5 p. 5 The Dual Simplex Method Correspondingly, the primal simplex method develops a basis that satisfies the (P), (D), and (CS) conditions simultaneously In each iteration it satisfies the primal conditions (P), the complementary slackness conditions (CS), and the dual conditions (D) partially We have optimality when (D) is fully satisfied The dual simplex method is the dual of the primal simplex: it converges through a series of dual feasible bases into a dual optimal (primal feasible) basis in every iteration it fulfills (D), (CS) and (P) partially optimality when (P) is fully satisfied Useful when it is easy to find a dual feasible (primal optimal) initial basis

6 p. 6 The Dual Simplex Method Consider the standard form linear program: max c T x s.t. Ax = b x 0 where A is an m n matrix, b is a column m-vector, x is a column n-vector, and c T is a row n-vector Let B be a basis that satisfies the primal optimality conditions (i.e., dual feasible) c B T B 1 N c N T 0 but is not primal feasible (i.e., not dual optimal) B 1 b 0

7 p. 7 The Dual Simplex Method The simplex tableau for basis B (dual) feasible if j N : z j 0 (dual) optimal, if i {1,...,m} : b i 0 The goal is to obtain a simplex tableau that is dual optimal, maintaining dual feasibility along the way In terms of the tableau, this means that in row 0 we always have nonnegative elements (dual feasibility) but the RHS column may contain negative elements (not dual optimal) Eventually, the RHS column will also become nonnegative This is attained through a sequence of (dual) pivots For brevity, we merely state the method without proofs

8 p. 8 The Dual Simplex Method Choose the leaving variable x r first as the basic variable with the smallest value in the current basis: r = argmin bi i {1,...,m} Lemma: after the pivot we obtain a primal optimal basic feasible solution (row 0 is nonnegative), if the entering variable x k is chosen according to: k = argmin j N { z j y rj : y rj < 0 Lemma: if j N : y rj 0, then the dual is unbounded and the primal is infeasible Pivot on row r and column k }

9 The Dual Simplex Method: Example Consider the linear program min 2x 1 + 3x 2 + 4x 3 s.t. x 1 + 2x 2 + x 3 3 2x 1 x 2 + 3x 3 4 x 1, x 2, x 3 0 Bringing to standard form and converting to maximization (note the eventual inversion!): max 2x 1 3x 2 4x 3 s.t. x 1 + 2x 2 + x 3 x 4 = 3 2x 1 x 2 + 3x 3 x 5 = 4 x 1, x 2, x 3, x 4, x 5 0 Cannot use the primal simplex since the initial basis formed by the slack variables is not (primal) feasible p. 9

10 p. 10 The Dual Simplex Method: Example Let us use the dual simplex (after inverting the constraints): max 2x 1 3x 2 4x 3 s.t. x 1 2x 2 x 3 + x 4 = 3 2x 1 + x 2 3x 3 + x 5 = 4 x 1, x 2, x 3, x 4, x 5 0 We can do this since the slack variables for a primal optimal (dual feasible) initial basis c T B B 1 N c T N = 0 [ 2 3 4] 0 [ ] [ ] 3 3 Not dual optimal: B 1 b = I 2 = 0 4 4

11 p. 11 The Dual Simplex Method: Example The initial simplex tableau: z x 1 x 2 x 3 x 4 x 5 RHS z x x The most negative basic variable leaves the basis: x 5 The entering variable is x 1 as z 1 y 51 = min{ z j y 5j : y 5j < 0} Divide the j-the element of row 0 with the j-th element of the r-th row if that is negative and invert, and take the minimum If we choose the leaving and entering variable this way, we get a dual feasible basis after the pivot

12 The Dual Simplex Method: Example z x 1 x 2 x 3 x 4 x 5 RHS z x x z x 1 x 2 x 3 x 4 x 5 RHS z x x z x 1 x 2 x 3 x 4 x 5 RHS z x x p. 12

13 p. 13 The Dual Simplex Method After the pivot the RHS element of the pivot row is always nonnegative, since first we divided the row of x r by y rk < 0 and so we invert all elements, this way b r < 0 as well If the basis is not dual degenerate (z k > 0), then after the pivot the objective function value decreases In fact, the current basis satisfies the (primal) optimality conditions but it lies outside the feasible region of the primal Making it feasible is possible only at the price of decreasing the primal objective It is not the primal maximization problem that we are solving now but rather the dual minimization problem We do not need to rewrite the problem into the dual to apply the dual simplex method, it can run directly on the (primal) simplex tableau

14 p. 14 The Dual Simplex Method: Example The new basis is dual feasible (primal optimal) but still not dual optimal, as x 4 = 1 < 0 x 4 leaves the basis and x 2 = argmin{ z j : y y 4j < 0} 4j enters x 1 x 2 x 3 x 4 x 5 RHS 9 z x x The resultant basis is both dual optimal and dual feasible The optimum for the maximization problem is z = 28 5, attained at the point x = [ ] T

15 p. 15 The Dual Simplex Method: Example Observe that the objective function value for the maximization problem has decreased in each iteration max 2x 1 3x 2 4x 3 : Of course, this is because we have in fact solved the dual minimization problem min{w T b : w T A c T } Choosing w T = c B B 1 the dual objective function w T b = c B B 1 b can be read from the simplex tableau in each step (row zero, RHS column) minw T b : Originally we had a minimization problem (invert!), whose optimum is thus z = 28 5

16 p. 16 The Dual Simplex Method: Example Solve the below linear program: min 2x 1 + 3x 2 + 5x 3 + 6x 4 s.t. x 1 + 2x 2 + 3x 3 + x 4 2 2x 1 + x 2 x 3 + 3x 4 3 x 1, x 2, x 3, x 4 0 Standard form, as a maximization (note: invert!) max 2x 1 3x 2 5x 3 6x 4 s.t. x 1 + 2x 2 + 3x 3 + x 4 x 5 = 2 2x 1 + x 2 x 3 + 3x 4 + x 6 = 3 x 1, x 2, x 3, x 4, x 5, x 6 0 Multiplying the first constraint by (-1) we obtain a primal optimal initial basis

17 p. 17 The Dual Simplex Method: Example In general, slack variables constitute a primal feasible basis if b 0, and a dual feasible basis if c T 0 We can use the dual simplex now z x 1 x 2 x 3 x 4 x 5 x 6 RHS z x x z x 1 x 2 x 3 x 4 x 5 x 6 RHS z x x

18 The Dual Simplex Method: Example z x 1 x 2 x 3 x 4 x 5 x 6 RHS 8 1 z x x The minimum is 19 5 the point x = [ 8 5, attained by the minimization problem at ] T The dual of the original minimization problem: 8 5 max 2w 1 3w 2 s.t. w 1 2w 2 2 2w 1 + w 2 3 3w 1 w 2 5 w 1 + 3w 2 6 w 1 0 w 2, 0 p. 18

19 p. 19 The Dual Simplex Method: Example w 2 is odd since w 2 0 (the simplex requires nonnegativity) Let w 2 = w 2 0 max 2w 1 + 3w 2 s.t. w 1 + 2w 2 2 2w 1 w 2 3 3w 1 + w 2 5 w 1 3w 2 6 w 1, w 2 0 The slack variables supply a primal feasible initial basis in the dual, since the RHS is nonnegatove and all constraints are of the type Solving by the primal simplex: the optimum is 19 5 and w 1 = 8 5, w 2 = w 2 = 1 5

20 p. 20 Optimal Employee Work Schedule At a railway station, the distribution of work is such that the number of staff needed is 3 persons between 0 and 4 o clock, 8 persons between 4 and 8 o clock, 10 persons between 8 and 12 o clock, 8 persons between 12 and 16 o clock, 14 persons between 16 and 20 o clock, 5 persons between 20 and 24 o clock Shifts start every day at 0, 4, 8, 12, 16, and 20 o clock and keep 8 hours Task: obtain an optimal schedule that requires the smallest staff (fewest persons during the day working in total)

21 p. 21 Optimal Employee Work Schedule Indicate the number of workers starting in each shift by x 1, x 2, x 3, x 4, x 5, and x 6 Then, the task is to minimize the objective function x 1 +x 2 +x 3 +x 4 +x 5 +x 6 From 0 until 4 o clock, the 20-o clock and 0-o-clock shifts are in work, at least 3 persons x 1 +x 6 3 From 4 until 8 o clock at least 8 persons are needed x 1 +x 2 8 Similarly for the rest of the shifts Of course, all variables are nonnegative

22 For now, we merely hope that what we obtain eventually will be integer-valued p. 22 Optimal Employee Work Schedule The linear program min x 1 + x 2 + x 3 + x 4 + x 5 + x 6 s.t. x 1 + x 2 8 x 2 + x 3 10 x 3 + x 4 8 x 4 + x 5 14 x 5 + x 6 5 x 1 + x 6 3 x 1, x 2, x 3, x 4, x 5, x 6 0 Variables are continuous, even though we cannot schedule employees partially Should pose this as an integer linear program, by requiring variables to be integer-valued: NP-hard problem

23 p. 23 Optimal Employee Work Schedule Introduce slack variables s 1, s 2,... to convert the constraints x i +x j b to the form x i +x j s = b,z 0 Writing as a maximization problem (note to ourselves: invert result at the end) and ignoring the column of z for now x 1 x 2 x 3 x 4 x 5 x 6 s 1 s 2 s 3 s 4 s 5 s 6 RHS z s s s s s s

24 p. 24 Optimal Employee Work Schedule Slack variables form a trivial initial basis (it is worth inverting all rows) Primal optimal basis but not primal feasible: use the dual simplex! s 4 leaves the basis and x 4 enters x 1 x 2 x 3 x 4 x 5 x 6 s 1 s 2 s 3 s 4 s 5 s 6 RHS z s s s s s s

25 p. 25 Optimal Employee Work Schedule x 1 x 2 x 3 x 4 x 5 x 6 s 1 s 2 s 3 s 4 s 5 s 6 RHS z s s s x s s s 2 leaves the basis and x 2 enters

26 p. 26 Optimal Employee Work Schedule x 1 x 2 x 3 x 4 x 5 x 6 s 1 s 2 s 3 s 4 s 5 s 6 RHS z s x s x s s s 5 leaves, x 5 enters Dual degenerate pivot

27 p. 27 Optimal Employee Work Schedule x 1 x 2 x 3 x 4 x 5 x 6 s 1 s 2 s 3 s 4 s 5 s 6 RHS z s x s x x s s 6 leaves, x 1 enters

28 Optimal Employee Work Schedule x 1 x 2 x 3 x 4 x 5 x 6 s 1 s 2 s 3 s 4 s 5 s 6 RHS z s x s x x x Optimal tableau: primal optimal and now also primal feasible 27 persons employed in total: 3 in the 0 o clock shift, 10 in the 4 o clock shift, 9 in the 12 o clock shift, and 5 in the 16 o clock shift, and no one works in the rest of the shifts Observe that there is surplus staff from 4 until 8 (s 1 = 5) and from 12 until 16 (s 3 = 1)! p. 28

29 p. 29 Optimal Production Planning The four-month prognosis for an item in a store is as follows: 1st month 2nd month 3rd month 4th month Business Plan [t] Purchase cost [musd/t] Storage capacity [t] Storage costs [musd/t] At the beginning and end of the period the stock in the storage is zero The stock changes uniformly during a month and the storage cost per month is based on the average quantity in stock Task: fulfill the business plan in each month, taking into account the storage capacities, with the lowest purchase and storage costs

30 p. 30 Optimal Production Planning Denote the quantity of the items purchased in each month by x 1, x 2, x 3, and x 4 [tonnes] Denote the stock at the end of each month by r 1, r 2, and r 3 [tonnes] (no stock at the end of the period) From the quantity purchased in the first month, 5 tonnes must be sold according to the business plan and the rest goes into stock x 1 = 5+r 1 In the rest of the months, the stock at the beginning plus the purchased quantity covers the monthly business plan and the stock at the end of the month r 1 +x 2 = 6+r 2 r 2 +x 3 = 8+r 3 r 3 +x 4 = 6

31 p. 31 Optimal Production Planning As the stock changes uniformly during the month, the average stock in each month is r 1 2, r 1 +r 2, 2 r 2 +r 3 2, r 3 2 The storage cost for the entire period [million USD]: 1.5 ( r1 2 + r 1 +r r 2 +r r 3 2 ) = 1.5r r r 3 The purchase cost: 4x 1 +3x 2 +2x 3 +5x 4 [million USD] Finally, the stock cannot exceed the storage capacity: r 1,r 2,r 3 10 Evidently, all variables are nonnegative

32 p. 32 Optimal Production Planning The linear program: min 4x 1 +3x 2 +2x 3 +5x r r r 3 s.t. x 1 r 1 = 5 x 2 +r 1 r 2 = 6 x 3 +r 2 r 3 = 8 x 4 +r 3 = 6 r 1 10 r 2 10 r 3 10 x 1, x 2, x 3, x 4, r 1, r 2, r 3 0 The objective function in maximization form: max 4x 1 3x 2 2x 3 5x 4 1.5r 1 1.5r 2 1.5r 3 Must be inverted when written into the simplex tableau!

33 p. 33 Optimal Production Planning We still need to find an initial basis the slacks for the storage constraints (s 1, s 2, s 3 ) are OK x 1, x 2, x 3, and x 4 would also work, but we first need to zero out the corresponding objective function coefficients to obtain a valid simplex tableau z x 1 x 2 x 3 x 4 r 1 r 2 r 3 s 1 s 2 s 3 RHS z x x x x s s s

34 p. 34 Optimal Production Planning Subtract four times the row of x 1 from row 0, this way eliminating the reduced cost for x 1 : z x 1 x 2 x 3 x 4 r 1 r 2 r 3 s 1 s 2 s 3 RHS z x x x x s s s Note how the objective function value has changed!

35 p. 35 Optimal Production Planning Similarly, cancel the reduced cost for x 2, x 3, and x 4 by elementary row operations z x 1 x 2 x 3 x 4 r 1 r 2 r 3 s 1 s 2 s 3 RHS z x x x x s s s Primal feasible tableau, solve with the primal simplex Note the nonzero objective function in the initial tableau!

36 p. 36 Optimal Production Planning z x 1 x 2 x 3 x 4 r 1 r 2 r 3 s 1 s 2 s 3 RHS z x x x r s s s The quantity of items to be purchased in each month is 5, 6, and 14 tonnes, no purchase in the last month Stock is created only in the 3rd month, 6 tonnes The total cost is 75 million USD

37 Network Routing A subscriber wishes to transfer 2-2 units of traffic between points A-B and C-D in a telecommunications network The network service provider establishes a path P 1 between A-B and paths P 2 and P 3 between C-D The A-B link capacity is 3 units, and 2 units for C-D Pricing is progressive: the first 1 unit of traffic through a link costs 1 unit, every additional unit costs 3 units Task: find the minimal cost assignment of traffic demands to paths p. 37

38 p. 38 Network Routing Denote the quantity of traffic routed to paths P 1, P 2, and P 3 by f 1, f 2, and f 3 The demand is 2 units of flow between A-B and 2 units between C-D f 1 2, f 2 +f 3 2 Denote the total load at each link by l 1 and l 2, these must satisfy the capacity constraints l 1 = f 1 +f 2 3, l 2 = f 3 2 Let the price of traffic routed to each link be c 1 and c 2 { li if l c i = i 1 i {1,2} 1+3(l i 1) if l i > 1 Task is to minimize c 1 +c 2 : nonlinear objective function!

39 p. 39 Network Routing Trick: linearize the objective function Piecewise linear objective function Approximate piecewise minc i c i l i c i 3l i 2 The smallest possible cost by minimization The piecewise objective is convex: we can use linear programming

40 p. 40 Network Routing The linear program: min c 1 +c 2 s.t. f 1 +f 2 3 f 3 2 f 1 +f 2 c 1 0 3f 1 +3f 2 c 1 2 f 3 c 2 0 3f 3 c 2 2 f 1 2 f 2 +f 3 2 f 1,f 2,f 3,c 1,c 2, 0

41 p. 41 Network Routing Standard form: slack variables constitute an initial basis Converting to maximization and inverting the last two constraints we get a dual feasible initial basis Use the dual simplex! z f 1 f 2 f 3 c 1 c 2 s 1 s 2 s 3 s 4 s 5 s 6 s 7 s 8 RHS z s s s s s s s s

42 p. 42 Network Routing The optimal tableau: z f 1 f 2 f 3 c 1 c 2 s 1 s 2 s 3 s 4 s 5 s 6 s 7 s 8 RHS z s s s c c f f f Route 2 unit to path P 1 and route 1 unit to each of the paths P 2 and P 3 The total cost is 8 units

Part 1. The Review of Linear Programming

In the name of God Part 1. The Review of Linear Programming 1.5. Spring 2010 Instructor: Dr. Masoud Yaghini Outline Introduction Formulation of the Dual Problem Primal-Dual Relationship Economic Interpretation