IP Duality. Menal Guzelsoy. Seminar Series, /21-07/28-08/04-08/11. Department of Industrial and Systems Engineering Lehigh University

Size: px

Start display at page:

Download "IP Duality. Menal Guzelsoy. Seminar Series, /21-07/28-08/04-08/11. Department of Industrial and Systems Engineering Lehigh University"

Abel Clarke
6 years ago
Views:

1 IP Duality Department of Industrial and Systems Engineering Lehigh University Seminar Series, /21-07/28-08/04-08/11

2 Outline Duality Theorem 1 Duality Theorem Introduction Optimality Conditions 2 Separability & Lagrangean Relaxation Cutting Planes Branch and Bound Example 3 Characterization Solution Procedure 4 With an OSF With Branch and Bound Inference Duality 5 Methods to Solve PIPs

3 References: Duality Theorem Diego Klabjan A New Subadditive Approach to Integer Programming: Theory and Algorithms In Proceedings of the Ninth Conference on Integer Programming and Combinatorial Optimization, Cambridge, MA, pages L.A. Wolsey Integer Programming Duality: Price Functions and Sensitivity Analaysis Math Programming, 20 L. Schrage and L.A. Wolsey Sensitivity analysis for branch and bound integer programming. Operations Research, 33(5): , M. Dawande and J. N. Hooker Inference-based sensitivity analysis for mixed integer/linear programming, revised 1997, Operations Research 48 (2000)

4 References: Duality Theorem Jenkins, L. (1990). Parametric Methods in Integer Linear Programming. Annals of Operation Research. 27, Jenkins,L. Parametric-objective integer programming using knapsack facets and Gomory cutting planes. Eur. J. Oper. Res. 31 (1987) Wang, HF & Jyh-Shing Horng, 1996, Structural Approach to Parametric Analysis of an IP, European J. of OR, 92, Marsten RE and Morin TL, Parametric integer programming: the right-hand side case, Annals of. Discr. Math. 1 (1977) Geoffrion A.M. and Nauss R. Parametric and postoptimality analysis in integer.linear programming. Management Science, 23(5): , Rountree S., and B. Gillett, Parametric integer linear programming: A synthesis of branch and bound with cutting planes, European Journal of Operational Research, 10, , 1982

5 Duality Theorem Introduction Optimality Conditions Z = max s.t nx c j x j j=1 nx a j x j b a j, b Z m (P) j=1 x j 0 integer, j = 1,..., n W = min F(b) s.t F(a j ) c j j = 1,..., n (D) F Γ m where Γ m = {F : R m R F is superadditive, nondecreasing and F(0) = 0}.

6 Z=W Duality Theorem Introduction Optimality Conditions Weak Duality: cx F(b) for all primal feasible vectors x and dual feasible functions F Z W (P) is infeasible if W (D) is infeasible if Z Strong Duality: If either (P) or (D) has a finite optimal value, then there exists x and F s.t Z = cx = F (b) = W If (P) is infeasible, either (D) is infeasible or W If (D) is infeasible, either (P) is infeasible or Z

7 Complementary Slackness Duality Theorem Introduction Optimality Conditions Let x and F be optimal solutions to (P) and (D). Then, c j F (a j ) 0, j = 1,..., n If x j > 0, then c j F (a j ) = 0 F is nondecreasing and superadditive F (b) = F ( P n j=1 a jx j ) and F (b P n j=1 a jx j ) = 0

8 Separability & Lagrangean Relaxation Cutting Planes Branch and Bound Example What type of dual optimal functions can be generated? What else can we know about the structure of F? Theorem of Alternative: P n j=1 π jx j π 0 is a valid inequality for Q = {x Ax b, x 0, and integer} iff there exists F Γ m s.t F (a j ) π j, j = 1,..., n and F (b) π 0. conv[q] {x nx F(a j )x j F(b), F Γ m, x 0} j=1

9 Separability & Lagrangean Relaxation Cutting Planes Branch and Bound Example Consider the problems P 1, D 1, D 2 : Z 1 = max cx nx s.t F 1 (h j )x j F 1 (h) F 1 G (P 1 ) j=1 x conv[x : Ex e, x 0, integer] W 1 = inf F 1 G nx max{ [(c j F 1 (h j ))x j ] + F 1 (h) : Ex e, x 0, integer} (D 1 ) j=1 W 2 = inf {F(b) : F(a j ) c j, F G Γ m r } (D 2 )

10 Propositions Duality Theorem Separability & Lagrangean Relaxation Cutting Planes Branch and Bound Example Z Z 1 W 1 W 2 If D 1 or D 2 has a finite optimum value, then W 1 = W 2

11 Propositions Duality Theorem Separability & Lagrangean Relaxation Cutting Planes Branch and Bound Example If G = L r or Γ r, problem D 2 is a strong dual of problem P 1. If P has a finite optimum value, there is an optimal dual function of the form: F(d) = P r i=1 u id i + F 2 (d r+1,..., d m), u i 0, i = 1,..., r, F 2 Γ m r if and only if Z = W nx 1 = min max{ (c j uh j )x j + uh : Ex e, x 0, integer} u 0 j=1 if and only if max{cx : Hx h, x conv[ex e, x 0, integral]} has an optimal solution in integers. If P has a finite optimum value, it has an optimal dual price function that is separable, F Γ r Γ m r if and only if max{cx : x conv[hx h, x 0, integral] x conv[ex e, x 0, integral]} has an optimal solution in integers.

12 Gomory fractional cutting plane algorithm Initialization: Set r = 1. Iteration r: Solve the problem P r 1 : Separability & Lagrangean Relaxation Cutting Planes Branch and Bound Example Z r 1 = max s.t nx c j x j j=1 nx a ij x j b i i = 1,..., m + r 1 j=1 x 0 If P r 1 is infeasible or the solution is integer, stop. Add the cut P n j=1 a (m+r)jx j b m+r which has the form: nx mx f r (a j )x j f r (b) where f r (d) = j=1 i=1 Xr 1 i d i + λ r 1 m+i f i (d) and λ r 1 i=1 λ r+i = (λ r 1 1,..., λ r 1 m+r 1 ) 0

13 Separability & Lagrangean Relaxation Cutting Planes Branch and Bound Example If the Gomory cutting plane algorithm terminates finitely and P has a finite optimum value, there exists an optimal value function of the form F(d) = G r (d) = mx rx ui r d i + um+if r i (d) i=1 i=1 for some finite value r and if u r is dual optimal, Z r = G r (b)

14 Branch and Bound Algorithm Separability & Lagrangean Relaxation Cutting Planes Branch and Bound Example Replace P by a family of problems: x is optimal if: (P t) Z t = max{cx : Ax, x X t}, t = 1,..., r where x is optimal for some problem P t r t=1x t = {x : x 0, integer} Z t = cx Z t t t where Z t is an upper bound on the optimal value of P t In this case it is not true that the algorithms generate an optimal superadditive value function. However, an optimal dual function with some other structure can be constructed.

15 Separability & Lagrangean Relaxation Cutting Planes Branch and Bound Example It is shown that the problem min F(b) s.t F(Ax) cx x X t F {F : R m R {± }} and nondecreasing is a strong dual of problem P t. Hence, the problem min F(b) s.t F(Ax) cx x 0, integer ( D) F {F : R m R} and nondecreasing is a strong dual of problem P. Let X t = {x : 0 g t j x j h t j, j = 1,..., n, x 0, integer}. Then if a standard branch and bound algorithm terminates with a finite optimal value, then D has an optimal solution of the form: for some finite value of r. F (d) = max t=1,...,r [αt + π t d], π t 0

16 Alternative formulation Duality Theorem Separability & Lagrangean Relaxation Cutting Planes Branch and Bound Example Let P be the problem: Z = max{cx : Ax b, x g, x h, x 0, integer} with the same X t and r t=1x t = {x : 0 g x h, x 0, integer}. P has a dual problem D: D = min{f(b, g, h) : F(a j, e j, e j ) c j, j = 1,..., n, F Γ m+2n } If a standard branch and bound algorithm terminates with a finite optimal value, then D has a feasible solution of the form: F(d) = min t=1,...,r ut d and nx j=1 F(a j, e j, e j )x j max t=1,...,r F(b, gt, h t ) (1) is a valid cut for P.

17 Separability & Lagrangean Relaxation Cutting Planes Branch and Bound Example

18 Duality Theorem Characterization Solution Procedure For the problem min cx s.t Ax = b, A Z m + n, b Z m +, c Z n x Z n + given a vector α R m, we define a generator subadditive function F α : R m + R as P F α(d) = αd max i E (αa i c i )x i A E x d x Z E + where E = {i N : αa i > c i } and A E is the submatrix of A consisting of the columns with indices in E. Also we define a divergent generator subadditive function F β : R m + R as F β (d) = βd max P i E (βa i)x i A E x d x Z E +

19 Characterization Solution Procedure b = ˆ c = ˆ A =

20 Properties Duality Theorem Characterization Solution Procedure Weak duality: F α is feasible subadditive function and hence F α(b) cx, α and feasible primal feasible vectors. Strong duality: If the IP is feasible, then there exists an α s.t F α is a generator optimal subadditive function(ofs),i.e., F α(b) = z IP. If the IP is infeasible, then there exists a divergent generator subadditive function F β s.t Fβ > 0 Example Complementary slackness: Let x be an optimal IP solution. If x i > 0 then αa i c i in any generator OSF.

21 Properties Duality Theorem Characterization Solution Procedure Reduced cost fixing: Let F be a feasible subadditive dual function and let z IP be an upper bound on z IP. If c k F(a k ) > 0 and v = zip F(b) c k F(a k ) > 0 for a column k N, then there is an optimal IP solution x with x k v 1 Valid Inequalities: For any generator feasible subadditive function X c i x i + X (αa i )x i F α(b) (2) i E i H is a valid inequality. Example

22 Basic generator subadditive functions Characterization Solution Procedure For an E N we can find a generator subadditive function F α with the best objective value and such that E(α ) E by the problem P E max{η : (η, α) Q b (E)} s.t Q b (E) = {η + α(a E x b) c E x x Z E +, A E x b (η, α) R R m } αa i c i i H = N \ E A generator subadditive function F α is called a basic generator subadditive function(bg) if (F α(b), α) is an extreme point of the polyhedron Q b (E(α)). (Similar argument for divergent generator subadditive functions). BG functions yield facet-defining valid inequalities and hence suffice to solve the IP as an LP.

23 Main frame Duality Theorem Characterization Solution Procedure Given E solve P E to get an α. Next set E = E i and update α. Iterate until F α(b) = cx. However P E is hard to solve. Therefore, use row generation. Let Ê Zn + s.t. Ê {x Zn + : A E x b} where E = x Ê {i N : x i > 0} Note that if A E x b, then x is not necessarily in Ê. Hence, subadditivity may not be satisfied.

24 Maintaining subadditivity Duality Theorem Characterization Solution Procedure Let S(x) = {y Z n + : y x}. Given Ê Zn + and a vector α, define π(x) = αax max {(αa c)y} y Ê,y S(x) π is dual feasible if π(e i ) c i, i N. If π is dual feasible and subadditive and x is feasible to the IP, then π(x) X i N π(e i )x i cx Hence, π provides weak duality. In addition consider Ê = {x Z E +, A E x b} where E is defined based on the α value of an OSF. Then, strong duality follows.

25 D(Ê, Ĥ) Duality Theorem Characterization Solution Procedure We have the new problem: max π max π(x) x Z n +,Ax=b π(e i ) c i i N π subadditive Alternatively, α that gives the largest dual objective value can be obtained from D(Ê, Ĥ): max π 0 π 0 + α(ay b) cy y Ê αax cx x Ĥ where Ĥ = {x Zn + : x / Ê, S(x) \ {x} Ê}.

26 The Algorithm Duality Theorem Characterization Solution Procedure Ê =, Ĥ = {e i : i N}, α = optimal dual vector of the LP relaxation, w D = LOOP Choose a vector x Ĥ, Ê = Ê { x}, Ĥ = Ĥ { x + e i : i N, y Ê, y x + e i, y Z n +} Update α by solving D(Ê, Ĥ). Let π 0 be the optimal value. w D = max{w D, π0 } If w D = min{cx : x Ĥ, Ax = b}, then we have solved the IP and exit. END LOOP

27 Computing a generator OSF Characterization Solution Procedure It has 3 stages: Find π based on the previous algorithm. Improve π and get it subadditive in {x Z S + : A S x = b, c s x z IP } for an S N. Obtain a generator OSF. Start with E = x Ê {i N : x i > 0} where Ê is obtained in previous stage. Keep expanding E and solve P E till getting the optimal OSF.

28 With an optimal solution pair (x, F ) With an OSF With Branch and Bound Inference Duality Changes in the rhs: b b F remains feasible. Z F (b ) x {y F ( P n j=1 a j y j ) = P n j=1 c j y j } if F is still optimal. Changes in the objective: c c x remains primal feasible. Z c x If c j F (a j ) j, then F remains dual feasible. Z F (b) If c j F (a j ) when xj = 0 and c j = c j when xj > 0, then x remains optimal.

29 With an OSF With Branch and Bound Inference Duality Adding a new column: ( c, ã) (x, 0) remains primal feasible. Z Z (x, 0) remains optimal if F (ã) c Adding a new constraint: P n j=1 h jx j h 0 If x is still feasible, x is optimal. The price function F : R m+1 R defined by F(d, d m+1 ) = F (d) is dual feasible for the new problem. Z Z = F(b, h 0 )

30 After a mixed-binary program is solved With an OSF With Branch and Bound Inference Duality Branch and bound algorithm exits with a tree of LP problems of the form (for node t): Q t (b) : Z t (b) = max cx nx s.t. a j x j b (π t ) where j=1 x j 0 for j N (θ t j ) L t j x j U t j for j I (µ t j ) L t j = Uj t = 0 for j F0 t L t j = Uj t = 1 for j F1 t L t j = 0, U t j = 1 for j I F t 1 F t 0 and π t, µ t are the dual feasible prices, I = {j N : x j {0, 1}} and F t 0, F t 1 are the sets of variables in I fixed to 0 and 1 respectively.

31 With an OSF With Branch and Bound Inference Duality Let C t (d) and B t (d) be the upper bounding functions on Q t (d) and the corresponding mixed-ip problem. We want to get B 1 (d). For each node t: C t (d) = π t d + X j I F t 0 F t 1 max{0, µ t j } + X j F t 1 For each terminal node with a feasible solution to Q t (b): B t (d) = C t (d) For a terminal node with no feasible solution: B t (d) = { if Ct (d) < 0 + if C t (d) 0 For any noterminal node having two offsprings L(t) and R(t): B t (d) = min {C t (d), max {B L(t) (d), B R(t) (d)}} µ t j

32 With an OSF With Branch and Bound Inference Duality Z B 1 (d) when problem is solved with a rhs of d. If P has been solved to optimality and a new column a j is added to the problem, then there exists an optimal solution to the new problem with x j = 0 if c j B 1 (b) B 1 (b a j )

33 With an OSF With Branch and Bound Inference Duality Next time...

34 Methods to Solve PIPs Parametric objective MIP z(φ) = min (c + φc )x s.t Ax b (Pφ) x 0 x j integer, j J N Parametric rhs MIP z(θ) = min cx s.t Ax (b + θb ) (Pθ) x 0 x j integer, j J N Parametric A-matrix MIP z(λ) = min cx s.t (A + λa )x b (Pλ) x 0 x j integer, j J N where φ, θ, α [0, 1] and c, b, A are the change vectors and the matrix.

35 Methods to Solve PIPs By solving Pφ for instance, we mean to determine an optimal solution and its value, x (φ) and z (φ) for all 0 φ 1. z(φ) and z(θ) - Graphs here The solution procedure usually follows to solve a number of problems Pφ k, k = 1,..K with φ k determined through an algorithm.

36 Objective case - IP Duality Theorem Methods to Solve PIPs Concavity may be used to define upper and lower bounds on z(φ) : z LB (φ), z UB (φ) Graph Some φ is removed from the set {φ i } and P φ is solved. If z( φ) = z UB ( φ) then z(φ) = z UB (φ) for all φ ˆ φl, φ R. If z( φ) z UB ( φ), then a new x( φ) has been found. Then the intersection point of (c + φc )x( φ) with the parts UB( φ) define new values φ i and φ i to the left and the right of φ respectively. Jenkins showed that this procedure requires solving 2p-1 point-values IPs where p is the number of different optimal solutions found for φ [0, 1]. He uses cutting planes algorithm together with some post optimality analysis to determine ranges for φ.

37 RHS Case - IP Duality Theorem Methods to Solve PIPs The hyperplane defined by constraint i cannot pass any integral point unless the integer portion of b i + θb i changes. Let d = 1 lcm{b 1,..., b n} Then, any b i + θb i, for i = 1,..., m will never be integer when (w 1)d < θ < wd with w = 1,..., 1/d. Thus, all of the m hyperplanes will not pass any integral point during these intervals of θ. Now, let wd with w = 0, 1,...1/d be the candidates of θ.

38 Methods to Solve PIPs When θ varies, the current optimal solution will not change until at least one constraint passes some integral point not every candidate of θ will make b i + θb i integer such that the i th constraint is binding at some integral point. Let these θ values be the principal candidates. If ζ r and ζ r+1 are two adjacent principal candidates of θ, then when θ varies within (ζ r, ζ r+1 ), there exists at most one optimal solution to Pθ If ζ r and ζ r+1 are two adjacent principal candidates and x(ζ r ) = x(ζ r+1 ), then x(ζ r ) is optimal for all ζ r θ ζ r+1 Using some elimination techniques(tolerance intervals for the optimal solutions), solve a sequence of candidate problems to get Pθ

39 Methods to Solve PIPs A branch and cut algorithm for general monotone (b 0) parametric IP It solves R -the number of principal candidates- problems simulatenously, i.e., min cx s.t Ax (b + t r) (P tr ) x 0 integer where t r = θ r b.

40 Methods to Solve PIPs Use dual simplex method on the LP-relaxations of all R problems till the relaxation of P b is at optimality. Save results as LBs on members. Add cuts to the first primal feasible and unfathomed member and resolve Check for integer solutions and update UBs. Note that UB(P t) is valid for another member with a smaller t and the members are fathomed from a node if UB(P t) LB(P t). If a member is fathomed, than move to the next member in decreasing order. If it is not primal feasible, pivot until optimality. Repeat this step. Branch. Select a node and pivot on the first unfathomed member. Repeat the previous step until all nodes are fathomed. Heuristics can also be used to improve the UBs.

41 Methods to Solve PIPs A branch and bound algorithm for binary IP A standart branch and bound algorithm with upper and lower bounds depending on θ. UB(θ): For the known feasible solutions x k, k K define: θ k 1 = min{θ θ k 2 = max{θ nx j=1 nx j=1 A j x k j b + θb } A j x k j b + θb } where θ k 1 = θ k 2 = if the indicated set is empty. Then Hence, UB k (θ) = { nx j=1 c j x k j if θ k 1 θ θ k 2, otherwise} UB(θ) = min {UB k (θ) k K }

42 Methods to Solve PIPs LB q (θ) of a node q is basically the LP relaxation of P q θ. A lower bound function to LB q (θ) can be derived from an optimal dual solution. If the primal problem is feasible: LB q (θ) (u q b )θ + u q b + v q e where u q and v q are the optimal dual values. If the primal problem is infeasible, then there exists an extreme ray (y q, z q ) 0 such that y q b + z q e > 0. If y q b 0, then LB q (θ) = + for all 0 θ 1. Otherwise, LB q (θ) = for all 0 θ θ. and LB q (θ) (u q b )θ + u q b + v q e for all θ θ 1 where (u q, v q ) is an extreme point and Graph of LB q (θ) θ = y q (b) z q e y q b

43 Methods to Solve PIPs Begin with P 0. Update UB(θ) when an integer solution is obtained. Besides, use heuristics for several values of theta to get integer solutions. Fathoming: if LB q (θ) UB(θ) for all 0 θ 1, then the corresponding node will not yield a better descendent. (Do not fathom even the optimal primal solution is all integer). When the algorithm terminates, P(θ) = UB(θ) Example

44 Methods to Solve PIPs

45 Next Time Duality Theorem Methods to Solve PIPs A subset of More of IP Duality MIP Duality Sensitivity Analysis, Inference based SA Parametric Integer Programming Warm Starting

A New Subadditive Approach to Integer Programming: Theory and Algorithms

A New Subadditive Approach to Integer Programming: Theory and Algorithms Diego Klabjan Department of Mechanical and Industrial Engineering University of Illinois at Urbana-Champaign Urbana, IL email: klabjan@uiuc.edu