and to estimate the quality of feasible solutions I A new way to derive dual bounds:

Transcription

1 Lagrangian Relaxations and Duality I Recall: I Relaxations provide dual bounds for the problem I So do feasible solutions of dual problems I Having tight dual bounds is important in algorithms (B&B), and to estimate the quality of feasible solutions I A new way to derive dual bounds: I I I Lagrangian dual Strength of Lagrangian dual Solving Lagrangian dual problem IOE 518: Introduction to IP, Winter 2012 Lagrangian relaxations and duality Page 95 c Marina A. Epelman Lagrangian relaxation Basic idea: consider the problem (IP) z =max{c T x : Ax apple b, Dx apple d, x 2 Z n +} =max{c T x : Dx apple d, x 2 X }. (5) I Suppose, the set X = {x 2 Z n +, Ax apple b} is easy to optimize over, and it is the m constraints Dx apple d that make the problem di cult. I A possible relaxation: max{c T x : x 2 X } probably weak, since some constraints are ignored I A way to take constraints into account: dual variables or Lagrange multipliers: choose u 2 R m + and define (IP(u)) z(u) =max{c T x + u T (d Dx) :x 2 X }. Proposition 10.1 Problem (IP(u)) is a relaxation of (IP) for all u 0. IOE 518: Introduction to IP, Winter 2012 Lagrangian relaxations Page 96 c Marina A. Epelman

2 Example: Lagrangian relaxation of UFL Let c ij be the unit profit resulting from supplying client i 2 M from location j 2 N, andf j unit cost of opening location j 2 N (IP) z =max P i2m Pj2N c P P ijx ij j2n f jy j j2n x ij =18i 2 M x ij y j apple 0 8i 2 M, 8j 2 N x 2 R M N +, y 2 B N Dualizing the demand constraints: z(u) =max P i2m Pj2N (c P ij u i )x ij j2n f jy j + P i2m u i (IP(u)) x ij y j apple 0 8i 2 M, 8j 2 N x 2 R M N +, y 2 B N IOE 518: Introduction to IP, Winter 2012 Lagrangian relaxations Page 97 c Marina A. Epelman Solving Lagrangian relaxations for UFL (IP(u)) z(u) =max P i2m Pj2N (c P ij u i )x ij j2n f jy j + P i2m u i x ij y j apple 0 8i 2 M, 8j 2 N x 2 R M N +, y 2 B N z(u) = P j2n z j(u)+ P i2m u i,where (IP j (u)) z j (u) =max P i2m (c ij u i )x ij f j y j x ij y j apple 0 8i 2 M 0 8i 2 M, y j 2 B x ij (IP j (u)) can be solved by inspection! ( z j (u) =max 0, X max{c ij i2m u i, 0} ) f j IOE 518: Introduction to IP, Winter 2012 Lagrangian relaxations Page 98 c Marina A. Epelman

3 Example: Lagrangian relaxation of STSP Symmetric TSP problem on undirected graph G =(V, E) canbe formulated as follows: min P e2e c ex e P e2 (i) x e =2, i 2 V ( P e2 (1) x e = 2) ( P e2e x e = n) Resulting subgraph is connected Let s dualize the degree constraints for nodes other than 1: min P e2e c ex e +2 P i2v u P P i i2v u i e2 (i) x e P P e2 (1) x e =2 e2e x e = n Resulting subgraph is connected IOE 518: Introduction to IP, Winter 2012 Lagrangian relaxations Page 99 c Marina A. Epelman Solving Lagrangian relaxations for STSP The above Lagrangian relaxation can be rewritten as 2 P i2v u i + min P e2e (c e u i u j )x e P P e2 (1) x e =2 e2e x e = n Resulting subgraph is connected, where i and j are endpoints of e. The feasible region of the relaxation consists precisely of 1-trees. The minimum cost 1-tree can be found by a greedy algorithm. IOE 518: Introduction to IP, Winter 2012 Lagrangian relaxations Page 100 c Marina A. Epelman

4 Lagrangian duality What is the best choice of u? Since (IP(u)) is a relaxation of (IP) 8u 0, z(u) z. To obtain the best upper bound: (LD) w LD = min{z(u) :u 0}. (LD) is the Lagrangian Dual Problem When is (LD) a strong dual? Proposition 10.2 Let x(u) solve (IP(u)) for some u 0. Suppose Dx(u) apple d and [Dx(u)] i = d i for all i such that u i > 0(complementarity). Then x(u) is optimal for (IP). Note: If constraints of the form Dx = d are dualized, the corresponding Lagrange multipliers are unrestricted in sign: u 2 R m. In the above proposition, complementarity condition is automatically satisfied. IOE 518: Introduction to IP, Winter 2012 Lagrangian Duality Page 101 c Marina A. Epelman Strength of the Lagrangian dual Theorem 10.3 w LD =max{c T x : Dx apple d, x 2 conv(x )}. Proof We will prove for finite X = {x 1,...,x T },butresultapplies for unbounded X as well. In this case, w LD =min u 0 max t=1,...,t {c T x t + u T (d Dx t )}, whichcanbe written as w LD =min u, c T x t + u T (d Dx t ), t =1,...,T u 0, unrestricted Taking the LP dual (dual variables: µ 2 R T ): w LD =max µ P T t=1 (ct x t )µ t = max c T x P T t=1 (Dx t d)µ t apple 0 Dx apple d P T t=1 µ t =1 x 2 conv(x ) µ 0 IOE 518: Introduction to IP, Winter 2012 Lagrangian Duality Page 102 c Marina A. Epelman

5 Lagrangian Dual vs. LP relaxation z apple w LD apple z LP To see the second inequality, note that conv(x ) = conv({x 2 Z n + : Ax apple b}) {x 2 R n + : Ax apple b} Each, and all, of the inequalities above can be made strict. Corollary (a) z = w LD for all c if and only if conv(x \ {x : Dx apple d}) = conv(x ) \ {x : Dx apple d}. (b) z LP = w LD for all c if and only if conv(x )={x 0:Ax apple b}. IOE 518: Introduction to IP, Winter 2012 Lagrangian Duality Page 103 c Marina A. Epelman Solving the Lagrangian dual Definition 10.1 Let f : R m! R be a convex function. A vector subgradient of f at u if for all v 2 R m, f (v) f (u)+ (u) T (v u). (u) 2 R m is a Note: For a continuously di erentiable convex function f, (u) = rf (u) is a subgradient. w LD =minz(u), where z(u) = max u 0 t=1,...,t ct x t + u T (d Dx t ). Here, z(u) is a piecewise linear convex function. For z(u), d Dx(u) is a subgradient, where x(u) 2 X solves (IP(u)). Subgradient algorithm for (LD) Initialization: u = u 0 Iteration k: Let u = u k. Solve (IP(u k )) to find x(u k ), and set u k+1 =max{u k µ k (d Dx(u k )), 0} for µ k > 0; k k +1 IOE 518: Introduction to IP, Winter 2012 Lagrangian Duality Page 104 c Marina A. Epelman

6 Choice of the step size u k+1 =max{u k µ k (d Dx(u k )), 0} how to choose µ k > 0? I µ k! 0and P k µ k!1as k!1(e.g., µ k =1/k). Guaranteed to converge, but too slow. I µ k = µ 0 k for some 0 < < 1. Fast progress; converges if µ 0 and are su ciently large but cannot be determined in advance. z(u I µ k = k ) w k,where0< kd Dx k (u)k 2 k < 2and w w LD.Either converges to w, orfindsu k with w z(u k ) w LD.Fast progress, popular choice. IOE 518: Introduction to IP, Winter 2012 Lagrangian Duality Page 105 c Marina A. Epelman Which Lagrangian dual? Each IP has a multitude of Lagrangian duals associated with it, depending on which constraints are dualized. To decide which dual to work with, consider: I The strength of the resulting dual bound w LD (see Theorem 10.3) I Ease of solution of the Lagrangian subproblems (IP(u)) (problem-specific) I Ease of solution of the Lagrangian dual (hard to estimate a priori) IOE 518: Introduction to IP, Winter 2012 Lagrangian Duality Page 106 c Marina A. Epelman

7 Example: Lagrangian duals for GAP Generalized assignment problem (GAP): z =max P n P m i=1 c ijx ij P n x ij apple 1, i =1,...,m P m i=1 a ijx ij apple b j, j =1,...,n x 2 B mn Possible Lagrangian relaxations/dual problems: 1. Dualize both sets of constraints 2. Dualize first set of assignments constraints 3. Dualize the second set of generalized assignment constraints IOE 518: Introduction to IP, Winter 2012 Lagrangian Duality Page 107 c Marina A. Epelman Comparing Lagrangian dual I First dual: w 1 LD =min u 0,v 0 z 1 (u, v), where z 1 (u, v) = max x2b mn nx mx (c ij u i a ij v j )x ij + i=1 mx u i + i=1 nx v j b j I The strength of the bound: w 1 LD = z LP, since conv(b mn )=[0, 1] mn I Ease of solution of the Lagrangian subproblems (IP(u)): can be solved by inspection (decide the value of each x ij separately). I Ease of solution of the Lagrangian dual: m + n dual variables IOE 518: Introduction to IP, Winter 2012 Lagrangian Duality Page 108 c Marina A. Epelman

8 Comparing Lagrangian duals II Second dual: w 2 LD =min u 0 z 2 (u), where z 2 (u) =max P n P m i=1 (c ij u i )x ij + P m i=1 u i P m i=1 a ijx ij apple b j, j =1,...,n x 2 B mn I The strength of the bound: potentially stronger that the LP relaxation: w 2 LD apple z LP. I Ease of solution of the Lagrangian subproblems (IP(u)): decomposes into n knapsack problems (decide the values of each group x 1j,...,x mj separately). I Ease of solution of the Lagrangian dual: m dual variables. IOE 518: Introduction to IP, Winter 2012 Lagrangian Duality Page 109 c Marina A. Epelman Comparing Lagrangian duals III Third dual: wld 3 =min v 0 z 3 (v), where z =max P n P m i=1 (c ij a ij v j )x ij + P n P v jb j n x ij apple 1, i =1,...,m x 2 B mn I The strength of the bound: w 3 LD = z LP, since conv({x 2 B mn : nx x ij apple 1, i =1,...,m}) = {x 2 [0, 1] mn : nx x ij apple 1, i =1,...,m} I Ease of solution of the Lagrangian subproblems (IP(u)): can be solved by inspection (decide the values of each group x i1,...,x in separately). I Ease of solution of the Lagrangian dual: n dual variables. IOE 518: Introduction to IP, Winter 2012 Lagrangian Duality Page 110 c Marina A. Epelman