4. The Dual Simplex Method

Transcription

1 4. The Dual Simplex Method Javier Larrosa Albert Oliveras Enric Rodríguez-Carbonell Problem Solving and Constraint Programming (RPAR) Session 4 p.1/34

2 Basic Idea (1) Algorithm as explained so far known as primal simplex: starting with feasible basis, look for optimal basis while keeping feasibility Alternative algorithm known as dual simplex: starting with optimal basis, look for feasible basis while keeping optimality Session 4 p.2/34

3 Basic Idea (2) min x y 2x+y 3 2x+y 6 x+2y 6 y 0 = min x y 2x+y s 1 = 3 2x+y +s 2 = 6 x+2y +s 3 = 6 x,y,s 1,s 2,s 3 0 min 6+y +s 3 x = 6 2y s 3 s 1 = 9 3y 2s 3 s 2 = 6+3y +2s 3 Basis (x,s 1,s 2 ) is optimal but not feasible! Session 4 p.3/34

4 Basic Idea (3) y 2x+y 6 min x y 2x+y 3 y 0 x+2y 6 (6,0) x Session 4 p.4/34

5 Basic Idea (4) Let us make the violating variables non-negative... Increase s 2 by making it non-basic... while preserving optimality If y replaces s 2 in the basis, then y = 1 3 (s s 3 ), x y = (s 2 +s 3 ) If s 3 replaces s 2 in the basis, then s 3 = 1 2 (s y), x y = (s 2 y) To preserve optimality, y must replace s 2 Session 4 p.5/34

6 Basic Idea (5) min 6+y +s 3 min s s 3 x = 6 2y s 3 = x = s s 3 s 1 = 9 3y 2s 3 y = s s 3 s 2 = 6+3y +2s 3 s 1 = 3 s 2 Current basis is feasible and optimal! Session 4 p.6/34

7 Basic Idea (6) y 2x+y 6 (2,2) min x y 2x+y 3 y 0 x+2y 6 (6,0) x Session 4 p.7/34

8 Outline of the Dual Simplex Algorithm 1. Initialization: Pick an optimal basis. 2. Dual Pricing: If all basic values are 0, then return OPTIMAL. Else pick a basic variable with value < Dual Ratio test: Compute best value preserving optimality, i.e. sign constraints on reduced costs. If best value does not exist, then return INFEASIBLE. Else select non-basic variable to be exchanged with violating basic variable. 4. Update: Update the tableau and go to 2. Session 4 p.8/34

9 Duality (1) The way the dual simplex works is best understood using the theory of duality We can get lower bounds on LP optimum value by combining constraints with convenient multipliers min x y 2x+y 3 2x+y 6 x+2y 6 y 0 min x y 2x+y 3 2x y 6 x 2y 6 y 0 1 ( x 2y 6 ) 1 ( y 0 ) x 2y 6 y 0 x y 6 Session 4 p.9/34

10 Duality (2) min x y 2x+y 3 2x y 6 x 2y 6 y 0 1 ( 2x+y 3 ) 2 ( 2x y 6 ) 1 ( ) 2x+y 3 4x 2y 12 x y 9 Session 4 p.10/34

11 Duality (3) min x y 2x+y 3 2x y 6 x 2y 6 y 0 µ 1 ( 2x+y 3 ) µ 2 ( 2x y 6 ) µ 3 ( x 2y 6 ) 2µ 1 x+µ 1 y 3µ 1 2µ 2 x µ 2 y 6µ 2 µ 3 x 2µ 3 y 6µ 3 (2µ 1 2µ 2 µ 3 )x + (µ 1 µ 2 2µ 3 )y 3µ 1 6µ 2 6µ 3 If µ 1 0, µ 2 0, µ 3 0, 2µ 1 2µ 2 µ 3 1 and µ 1 µ 2 2µ 3 1 then 3µ 1 6µ 2 6µ 3 is a lower bound Session 4 p.11/34

12 Duality (4) Best possible lower bound can be found by solving max 3µ 1 6µ 2 6µ 3 2µ 1 2µ 2 µ 3 1 µ 1 µ 2 2µ 3 1 µ 1,µ 2,µ 3 0 Best solution is given by (µ 1,µ 2,µ 3 ) = (0, 1 3, 1 3 ) 0 ( 2x+y 3 ) ( 2x y 6 ) ( x 2y 6 ) x y 4 Matches with optimum! Session 4 p.12/34

13 Dual Problem (1) Given a LP (called primal) its dual is the LP min c T x Ax b max b T y A T y c y 0 Primal variables associated with columns of A Dual variables (multipliers) associated with rows of A Objective and right-hand side vectors swap their roles Session 4 p.13/34

14 Dual Problem (2) Prop. The dual of the dual is the primal. Proof: max b T y min ( b) T y A T y c = A T y c y 0 y 0 max c T x ( A T ) T x b = min c T x Ax b One says the primal and the dual form primal-dual pair Session 4 p.14/34

15 Dual Problem (3) Prop. min c T x Ax = b and max bt y A T y c form a primal-dual pair Proof: min c T x Ax = b = min c T x Ax b Ax b max b T y 1 b T y 2 A T y 1 A T y 2 c y 1,y 2 0 y:=y 1 y 2 = max b T y A T y c Session 4 p.15/34

16 Duality Theorems (1) Th. (Weak Duality) Let (P,D) be a primal-dual pair (P) min c T x Ax = b and (D) max bt y A T y c If x is feasible solution top andy is feasible solution to D then y T b c T x Proof: c A T y 0 and imply (c A T y) T. Hence y T b = y T Ax = (A T y) T x c T x Session 4 p.16/34

17 Duality Theorems (2) Feasible solutions to D give lower bounds on P Feasible solutions to P give upper bounds on D Can the two bounds ever be equal? Th. (Strong Duality) Let (P,D) be a primal-dual pair (P) min c T x Ax = b and (D) max bt y A T y c If any of P or D has a feasible solution and a finite optimum then the same holds for the other problem and the two optimum values are equal. Session 4 p.17/34

18 Duality Theorems (3) Proof (Th. of Strong Duality): By symmetry it is sufficient to prove only one direction. Wlog. let us assume P is feasible with finite optimum. After executing the Simplex algorithm to P we find B optimal feasible basis. Then: c T B B 1 a j = c j for all j B c T B B 1 a j c j for all j R (optimality conds hold) So π T := c T B B 1 is dual feasible: π T A c T, i.e. A T π c. Moreover, c T B β = ct B B 1 b = π T b = b T π By the theorem of weak duality, π is optimum for D If B optimal feasible basis for P, then simplex multipliers π T := c T B B 1 are optimal feasible solution for D. Session 4 p.18/34

19 Duality Theorems (4) Prop. Let (P,D) be a primal-dual pair (P) min c T x Ax = b and (D) max bt y A T y c If P (resp., D) has a feasible solution but the objective value is not bounded, then D (resp., P) is infeasible Proof: By contradiction. If y were a feasible solution to D, by weak duality theorem objective of P would be bounded from below! Session 4 p.19/34

20 Duality Theorems (5) Prop. Let (P,D) be a primal-dual pair (P) min c T x Ax = b and (D) max bt y A T y c If P (resp., D) has a feasible solution but the objective value is not bounded, then D (resp., P) is infeasible And the converse? Session 4 p.20/34

21 Duality Theorems (5) Prop. Let (P,D) be a primal-dual pair (P) min c T x Ax = b and (D) max bt y A T y c If P (resp., D) has a feasible solution but the objective value is not bounded, then D (resp., P) is infeasible And the converse? min 3x 1 +5x 2 x 1 +2x 2 = 3 2x 1 +4x 2 = 1 x 1,x 2 free max 3y 1 +y 2 y 1 +2y 2 = 3 2y 1 +4y 2 = 5 x 1,x 2 free Session 4 p.20/34

22 Duality Theorems (6) Primal unbounded Dual unbounded Primal infeasible Dual infeasible = Dual infeasible = Primal { infeasible = Dual infeasible unbounded { = Primal infeasible unbounded Session 4 p.21/34

23 Karush Kuhn Tucker Optimality Conds (1) Consider a primal-dual pair of the form min c T x Ax = b and max b T y A T y +w = c w 0 Karush-Kuhn-Tucker (KKT) optimality conditions are Ax = b x,w 0 A T y +w = c x T w = 0 (complementary slackness) They are necessary and sufficient conditions for optimality of the pair of primal-dual solutions (x,(y, w)) Used, e.g., as a test for quality in LP solvers Session 4 p.22/34

24 Karush Kuhn Tucker Optimality Conds (2) (KKT) min c T x max b T y Ax = b (P) Ax = b (D) A T y +w = c A T y +w = c w 0 x,w 0 x T w = 0 Th. (x,(y,w)) is solution to KKT iff x optimal solution to P and (y,w) optimal solution to D Proof: By 0 = x T w = x T (c A T y) = c T x b T y, Weak Duality x is feasible solution to P, (y,w) is feasible solution to D. By Strong Duality x T w = x T (c A T y) = c T x b T y = 0 as both solutions are optimal Session 4 p.23/34

25 Towards Dual Simplex: Relating Bases (1) (P) min z = c T x Ax = b (D) max Z = bt y A T y c max Z = b T y A T y +w = c w 0 Let B be basis of P. Reorder rows in D so that B-basic variables are first m. Reorder columns in D so that the matrix is ( B T I 0 R T 0 I ) y w B w R Submatrix of vars y and vars w R : ( ˆB = B T 0 R T I ) Session 4 p.24/34

26 Towards Dual Simplex: Relating Bases (2) ˆB = (y,w R ) is a basis of D: ( ˆB 1 = ˆB = ( B T 0 R T I ) B T 0 R T B T I ) Each var w j in D is associated to var a x j in P. w j is ˆB-basic iff x j is not B-basic Session 4 p.25/34

27 Dual Feasibility is Primal Optimality Let s apply simplex algorithm to dual problem Let s see correspondence of dual feasibility in primal LP ˆB 1 c = ( B T 0 R T B T I )( c B c R ) = ( B T c B R T B T c B +c R ) There is no restriction on the sign of y 1,...,y m Variables w j have to be non-negative. But R T B T c B +c R 0 iff c T R ct B B 1 R 0 iff d T R 0 ˆB is dual feasible iff d j 0 for all j R Dual feasibility is primal optimality! Session 4 p.26/34

28 Dual Optimality is Primal Feasibility ˆB-basic dual vars: (y w R ) with costs (b T 0) Non ˆB-basic dual vars: w B with costs (0) Optimality condition: reduced costs 0 (maximization!) 0 ( 0 ( ) 0 ) ( ( b T 0 b T B T 0 ) B T 0 R T B T ) I ( = 0 I I = 0 ) β T iff β 0 For all 1 p m, w kp is not dual improving iff β p 0 Dual optimality is primal feasibility! Session 4 p.27/34

29 Improving a Non-Optimal Solution (1) Let p (1 p m) be such that β p < 0 b T B T e p < 0 Let ρ p = B T e p, so b T ρ p = β p. If w kp takes value t 0: ( ) ( ( y(t) w R (t) B T c B d R = ˆB 1 c ˆB 1 te p = ) ( B T c B tρ p d R +tr T ρ p ) B T 0 R T B T I )( te p 0 Dual objective value improvement is Z = b T y(t) b T y(0) = tb T ρ p = tβ p ) = Session 4 p.28/34

30 Improving a Non-Optimal Solution (2) Only w variables need to be 0: for j R w j (t) = d j +ta T j ρ p = d j +tρ T pa j = d j +te T pb 1 a j = d j +te T pα j = d j +tα p j w j (t) 0 d j +tα p j 0 If α p j 0 the constraint is satisfied for all t 0 If α p j < 0 we need d j α p j t Best improvement achieved with Θ D := min{ d j α p j α p j < 0} Variable w q is blocking when Θ D = d q α p q Session 4 p.29/34

31 Improving a Non-Optimal Solution (3) 1. If Θ D = + (there is no j such that j R and α p j < 0): Value of dual objective can be increased infinitely. Dual LP is unbounded. Primal LP is infeasible. 2. If Θ D < + and w q is blocking: When setting w kp = Θ D sign of dual slack basic vars (primal reduced costs of non-basic vars) is respected In particular, w q (Θ D ) = d q +Θ D αq p = d q +( d q α p)αp q = 0 q We can make a basis change: In dual: w kp enters ˆB and w q leaves In primal: x kp leaves B and x q enters Session 4 p.30/34

32 Update We forget about dual LP and work only with primal LP New basic indices: B = B {kp } {q} New objective value: Z = Z Θ D β p New dual basic sol: ȳ = y Θ D ρ p d j = d j +Θ D α p j if j R, d kp = Θ D New primal basic sol: β p = Θ P, where Θ P = β p α p q βi = β i Θ P α i q if i p New basis inverse: B 1 = EB 1 where E = (e 1,...,e p 1,η,e p+1,...,e m and (( ) ( ) ( ) η T α 1 = q α p 1,..., α p q, 1 α p+1 q α p q α p q q α p q,...,( α m q α p q )) T Session 4 p.31/34

33 Algorithmic Description (1) 1. Initialization: Find an initial dual feasible basis B Compute B 1, β = B 1 b, y T = c T B B 1, d T R = ct R yt R, Z = b T y 2. Dual Pricing: If for all i B,β i 0 then return OPTIMAL Else let p be such that β p < 0. Compute ρ T p = e T pb 1 and α p j = ρt pa j for j R 3. Dual Ratio test: Compute J = {j j R,α p j < 0}. If J = then return INFEASIBLE Else compute Θ D = min j J ( d j ) and q st. Θ α p D = d q j α p q Session 4 p.32/34

34 Algorithmic Description (2) 4. Update: B = B {k p } {q} Z = Z Θ D β p Dual solution ȳ = y Θ D ρ p d j = d j +Θ D α p j if j R, d kp = Θ D Primal solution Compute α q = B 1 a q and Θ P = β p β p = Θ P, B 1 = EB 1 βi = β i Θ P α i q if i p α p q Go to 2. Session 4 p.33/34

35 Primal vs. Dual Simplex PRIMAL Ratio test: O(m) divs Can handle bounds efficiently Many years of research and implementation There are classes of LP s for which it is the best Not suitable for solving LP s with integer variables DUAL Ratio test: O(n m) divs Can handle bounds efficiently (not explained here) Developments in the 90 s made it an alternative Nowadays on average it gives better performance Suitable for solving LP s with integer variables Session 4 p.34/34