3.10 Lagrangian relaxation

Transcription

1 3.10 Lagrangian relaxation Consider a generic ILP problem min {c t x : Ax b, Dx d, x Z n } with integer coefficients. Suppose Dx d are the complicating constraints. Often the linear relaxation and the relaxation by elimination of Dx d yield weak bounds (e.g., TSP/UFL deleting cut-set/demand constraints) More general setting: min {c t x : Dx d, x X R n } (1) Idea: Delete the complicating constraints Dx d and, for each one of them, add to the objective function a term with a multiplier u i, which penalizes its violation and that is 0 for all feasible solutions of problem (1). Edoardo Amaldi (PoliMI) Optimization Academic year / 40

2 Definition: Given a problem z = min {c t x : Dx d, x X R n } (2) For each vector of Lagrange multipliers u 0, the Lagrangian subproblem is w(u) = min {c t x + u t (d Dx) : x X R n } (3) where L(x, u) = c t x + u t (d Dx) is the Lagrangian function of the primal problem (2), w(u) = min {L(x, u) : x X R n } is the dual function. Proposition: For any u 0, the Lagrangian subproblem (3) is a relaxation of problem (2). Proof: Clearly {x X : Dx d} X. Moreover, for each u 0 and each feasible x for (2), we have w(u) c t x. Indeed w(u) c t x + u t (d Dx) c t x since u t (d Dx) 0 and w(u) is the minimum value of c t x + u t (d Dx) for x X. Corollary: If z = min {c t x : Dx d, x X } is finite, then w(u) z u 0. Edoardo Amaldi (PoliMI) Optimization Academic year / 40

3 To determine the tightest lower bound Definition: The Lagrangian dual of the primal problem (2) is Only nonnegativity constraints! w = max u 0 w(u). (4) Note: By relaxing (dualizing) linear constraints, the objective function remains linear. The other constraints can be of any type, provided subproblem (3) is sufficiently easy. For LPs the Lagrangian dual coincides with the LP dual (exercise 5.2). Corollary: (Weak Duality) For each pair of feasible solutions x {x X : Dx d} of the primal (2) and u 0 of the Lagrangian dual (4), we have w(u) c t x. (5) Edoardo Amaldi (PoliMI) Optimization Academic year / 40

4 Consequences: i) If x is feasible for the primal (2), ũ is feasible for the Lagrangian dual (4) and c t x = w(ũ), then x and ũ are optimal for respectively (2) and (4). ii) In particular w = max u 0 w(u) z = min {c t x : Dx d, x X }. If one problem is unbounded, the other one is infeasible. Recall: For any primal-dual pair of bounded LPs we have strong duality (w = z ). Observation: Unlike for LPs, discrete optimization problems can have a duality gap, that is w < z. Lagrangian relaxation of equality constraints: Only difference: the Lagrange multipliers of equalities are unrestriced in sign (u i R). If all the m relaxed/dualized constraints are equality constraints, the Lagrangian dual is: max w(u) u R m Edoardo Amaldi (PoliMI) Optimization Academic year / 40

5 Example 1: Binary knapsack Consider max z = 10x 1 + 4x x 3 s.t. 3x 1 + x 2 + 4x 3 4 x 1, x 2, x 3 {0, 1} Relaxing the capacity constraint, we obtain the Lagrangian function: L(x, u) = 10x 1 + 4x x 3 + u(4 3x 1 x 2 4x 3) Dual function: w(u) = max x {0,1} 3 10x 1 + 4x x 3 + u(4 3x 1 x 2 4x 3) u 0 The Lagrangian subproblem: w(u) = max (10 3u)x1 + (4 u)x2 + (14 4u)x3 + 4u x {0,1} 3 optimal solution: x i = 1 (x i = 0) if nonnegative (nonpositive) coefficient and to an arbitrary value if coefficient 0. Lagrangian dual: min w(u) = min ( max (10 3u)x1 + (4 u)x2 + (14 4u)x3 + 4u ) u 0 u 0 x {0,1} 3 Edoardo Amaldi (PoliMI) Optimization Academic year / 40

6 Dual function: w(u) = max x {0,1} 3 (10 3u)x 1 + (4 u)x 2 + (14 4u)x 3 + 4u Values of u for which the coefficients of x 1, x 2, x 3 are nonpositive: u 10 3 for x1, u 14 4 for x 3 and u 4 for x 2. Optimal solution of the Lagrangian subproblem as a function of u: x = (1, 1, 1) for u [0, 10 3 ], x = (0, 1, 1) for u [ 10 3, 14 4 ], x = (0, 1, 0) for u [ 14 4, 4], x = (0, 0, 0) for u [4, ). Thus w(u) = 28 4u for u [0, 10 3 ] 18 u for u [ 10 3, 14 4 ] 4 + 3u for u [ 14 4, 4] 4u for u [4, ) Edoardo Amaldi (PoliMI) Optimization Academic year / 40

7 Dual function: w(u) u Lagrangian dual: min w(u) = min ( max (10 3u)x1 + (4 u)x2 + (14 4u)x3 + 4u ) u 0 u 0 x {0,1} 3 optimal solution u = 14 4 with w = w(u ) = Edoardo Amaldi (PoliMI) Optimization Academic year / 40

8 Example 2: Uncapacitated Facility Location (UFL) Variant with profits p ij, fixed costs f j for opening the depots in the candidate sites, and total profit to be maximized. MILP formulation: z = max s.t. i M j N p ijx ij j N f jy j j N x ij = 1 i M (6) x ij y j i M, j N y j {0, 1} j N 0 x ij 1 i M, j N Relaxing the demand constraints (6), we obtain the Lagrangian subproblem: w(u) = max i M j N (p ij u i )x ij j N f jy j + i M u i s.t. x ij y j i M, j N (7) y j {0, 1} j N (8) 0 x ij 1 i M, j N (9) which decomposes into N independent subproblems, one for each candidate site j. Edoardo Amaldi (PoliMI) Optimization Academic year / 40

9 Indeed w(u) = j N w j(u) + i M u i where w j (u) = max i M (p ij u i )x ij f j y j (10) s.v. x ij y j i M y j {0, 1} 0 x ij 1 i M For each j N, the subproblem (10) can be solved by inspection: If y j = 0, then x ij = 0 for each i and the objective function value is 0. If y j = 1, it is convenient to serve all clients with positive profit, namely x ij = 1 for all i such that p ij u i > 0, with an objective function value of max{p ij u i, 0} f j. i M Thus w j (u) = max{0, i M max{p ij u i, 0} f j }. Example: see Chapter 10 of L. Wolsey, Integer Programming, p Edoardo Amaldi (PoliMI) Optimization Academic year / 40

10 Sometimes solving the Lagrangian subproblem (3) yields an optimal solution of the primal (2). Proposition: If u 0 and i) x(u) is an optimal solution of the Lagrangian subproblem (3) ii) Dx(u) d iii) (Dx(u)) i = d i for each u i > 0 (complementary slackness conditions), then x(u) is an optimal solution of the primal poblem (2). Proof: Due to (i) we have w w(u) = c t x(u) + u t (d Dx(u)) and to (iii) we have c t x(u) + u t (d Dx(u)) = c t x(u). According to (ii), x(u) is a feasible solution of the primal (2) and hence c t x(u) z. Thus w c t x(u) + u t (d Dx(u)) = c t x(u) z and, since w z, everything holds with equality and x(u) is an optimal solution of the primal (2). Observation: If only equalities, conditions (iii) are automatically satisfied, and an optimal solution of Lagrangian subproblem is optimal for primal (2) if it is feasible. Edoardo Amaldi (PoliMI) Optimization Academic year / 40

11 Proposition: The dual function w(u) is concave. Proof: Consider any pair u 1 0 and u 2 0. For any α such that 0 α 1, let x be an optimal solution of the Lagrangian subproblem (3) for ũ = αu 1 + (1 α)u 2, namely w(ũ) = c t x + ũ t (d D x). By definition of w(u), we have w(u 1 ) c t x + u t 1 (d D x) and w(u 2 ) c t x + u t 2 (d D x). Multipying the first inequality by α and the second one by 1 α, we obtain αw(u 1 ) + (1 α)w(u 2 ) c t x + (αu 1 + (1 α)u 2 ) t (d D x) = w(αu 1 + (1 α)u 2 ). Edoardo Amaldi (PoliMI) Optimization Academic year / 40

12 LP characterization of the Lagrangian dual How strong is the lower bound on z we obtain by solving the Lagrangian dual? Theorem: Consider a generic ILP problem min {c t x : Ax b, Dx d, x Z n } with integer coefficients. Let w(u) = min {c t x + u t (d Dx) : Ax b, x Z n } be the dual function, w = max u 0 w(u) be the optimal value of the Lagrangian dual and X = {x Z n : Ax b}, then w = min {c t x : Dx d, x conv(x )}. (11) The problem is convexified, the Lagrangian dual is characterized in terms of a Linear Program. Corollary 1: Since conv(x ) {x R n : Ax b}, z LP = min {c t x : Ax b, Dx d, x R n } w z. Depending on the objective function, these inequalities can be strict, i.e, z LP < w < z. Edoardo Amaldi (PoliMI) Optimization Academic year / 40

13 Illustration: from D. Bertsimas, R. Weismantel, Optimization over integers, Dynamic Ideas, 2005, p Consider the ILP problem min 3x1 x 2 s.t. x 1 x 2 1 x 1 + 2x 2 5 3x 1 + 2x 2 3 6x 1 + x 2 15 x 1, x 2 0 integer - Represent graphically the feasible region and the optimal solutions of the ILP and of its linear relaxation: x ILP = (1, 2) with z ILP = 1 and x LP = (1/5, 6/5) with z LP = 3/5. - Apply Lagrangian relaxation to the first constraint. For every u 0, the Lagrangian subproblem is w(u) = min 3x 1 x 2 + u( 1 x 1 + x 2) (x 1,x 2 ) X where X is the set of all integer solutions that satisfy all the other constraints. Edoardo Amaldi (PoliMI) Optimization Academic year / 40

14 - Use the theorem to find the optimal value of the Lagrangian dual w = max u 0 w(u) and the corresponding optimal solution x D = x(u ), where u is an optimal solution of the Lagrangian dual. Represent conv(x ) and the polyhedron conv(x ) {(x 1, x 2) R 2 : x 1 x 2 1}. We obtain x D = (1/3, 4/3) with w = 1/3. Thus, we have: z LP = 3/5 < w = 1/3 < z ILP = 1 Drawing the dual function w(u) it is possibile to verify that the optimal solution of the Lagrangian dual is u = 5/3 with w = 1/3. Edoardo Amaldi (PoliMI) Optimization Academic year / 40

15 Proof: Consider the case where X = {x 1,..., x k } with k finite even though huge. The Lagrangian dual is equivalent to maximize a nondifferentiable piecewise linear concave function: w = max w(u) = max {min u 0 u 0 x X [ct x + u t (d Dx)]} = max { min u 0 1 l k [ct x l + u t (d Dx l )]} and it can be expressed as the following Linear Program: w = max y which contains a huge number of constraints. s.t. c t x l + u t (d Dx l ) y l u 0, y R m Taking its dual and applying strong duality, we obtain: k w = min (c t x l )µ l s.t. l=1 k (Dx l d)µ l 0 l=1 k µ l = 1 l=1 µ l 0 l. Edoardo Amaldi (PoliMI) Optimization Academic year / 40

16 w = min s.t. k (c t x l )µ l l=1 k (Dx l d)µ l 0 l=1 k µ l = 1 l=1 µ l 0 l. Setting x = k l=1 µ lx l with k l=1 µ l = 1 and µ l 0 for each l, we obtain: w = min c t x s.t. Dx d x conv(x ). The result can be extended to the case where X is the feasible region of any ILP. Example of a nondifferentiable piecewise linear concave dual function w(u). Edoardo Amaldi (PoliMI) Optimization Academic year / 40

17 When is the Lagrangian relaxation bound not stronger than the linear relaxation one? Corollary 2: If X = {x Z n : Ax b} and conv(x ) = {x R n : Ax b}, then w = max u 0 w(u) = z LP = min {c t x : Ax b, Dx d, x R n }. Example: Given a generic binary knapsack problem and its linear relaxation max s.t. z = n i=1 p ix i n i=1 a ix i b x i {0, 1} n z LP KP = max x [0,1] n{ p i x i : i=1 i n a i x i b}. The Lagrangian relaxation is as weak as the linear relaxation. Indeed, X = {x {0, 1} n } and obviously conv(x ) = {x [0, 1] n }, whose constraints 0 x i 1 are already contained in the linear relaxation. According to Corollary 2, we have w = z LP KP. i=1 Edoardo Amaldi (PoliMI) Optimization Academic year / 40

18 Solution of the Lagrangian duals Generalization of the gradient method for functions of class C 1 to functions piecewise C 1 (not everywhere differentiable) Definition: Let C R n be a convex set and f : C R a convex function on C a vector γ R n is a subgradient of f at x C if f (x) f (x) + γ t (x x) x C the subdifferential, denoted by f (x), is the set of all subgradients of f at x. Examples: - For f (x) = x 2, at x = 3 the only subgradient is γ = 6. Indeed 0 (x 3) 2 = x 2 6x + 9 implies that for each x: f (x) = x 2 6x 9 = 9 + 6(x 3) = f (x) + 6(x x) - For f (x) = x it is clear that: γ = 1 if x > 0, γ = 1 if x < 0, and f (x) = [ 1, 1] if x = 0 Edoardo Amaldi (PoliMI) Optimization Academic year / 40

19 Properties: 1) A convex function f : C R has at least one subgradient at each interior point x of C. N.B.: The existence of (at least) one subgradient at any point of int(c), with C convex, is a necessary and sufficient condition for f to be convex on int(c). 2) If f is convex and x C, f (x) is a nonempty, convex, closed and bounded set. 3) x is a global minimum of f if and only if 0 f (x ). Edoardo Amaldi (PoliMI) Optimization Academic year / 40

20 Subgradient method Consider min x R n f (x) with f (x) convex. Start from an abitrary x 0. At the k-th iteration: consider a γ k f (x k ) and set with α k > 0 x k+1 := x k α k γ k Observation: We do not perform a 1-D search because for nondifferentiable functions a subgradient γ f (x) is not necessarily a descent direction! Example: next page For sufficiently small stepsizes, one gets closer to an optimal solution: Lemma: If x k is a non-optimal solution and x any optimal solution, then implies that x k+1 x < x k x 2. 0 < α k < 2 f (x k ) f (x ) γ k 2 Edoardo Amaldi (PoliMI) Optimization Academic year / 40

21 Example: min 1 x1,x 2 1 f (x 1, x 2) with f (x 1, x 2) = max{ x 1, x 1 + x 2, x 1 2x 2} not everywhere differentiable Level curves in brown, points of nondifferentiability in green (of type: (t, 0), ( t, 2t) and ( t, t) for t 0), global minimum x = (0, 0). If x k = (1, 0) and we consider γ k = (1, 1) f (x k ), f (x) increases along {x R 2 : x = x k α k γ k, α k 0} but if α k is sufficiently small then x k+1 = x k α k γ k is closer to x. From Chapter 8, Bazaraa et al., Nonlinear Programming, Wiley, 2006, p Edoardo Amaldi (PoliMI) Optimization Academic year / 40

22 The method can be easily extended to the case with bound constraints by including a Edoardo Amaldi (PoliMI) Optimization Academic year / 40 Theorem: If f is convex, lim x f (x) = +, lim k α k = 0 and k=0 α k =, the subgradient method terminates after a finite number of iterations with an optimal solution x or generates an infinite sequence {x k } admitting a subsequence converging to x. Choice of the stepsize: In practice {α k } such that lim k α k = 0, k=0 α k = (e.g., α k = 1/k) are too slow. An alternative: α k = α 0ρ k, for a given ρ < 1. A more sophisticated and popular rule: α k = ε k f (x k ) ˆf γ k 2, where 0 < ε k < 2 and ˆf is either the optimal value (minimum) f (x ) or an estimate. Stopping criterion: prescribed maximum number of iterations because, even if 0 f (x k ) that subgradient may non be considered at x k. N.B.: Since it is not a monotone method, one needs to store the best solution x k found so far.

23 Subgradient method for solving the Lagrangian dual Lagrangian dual: max u 0 w(u) where w(u) = min {c t x + u t (d Dx) : x X R n } is concave and piecewise linear. Simple characterization of the subgradients of w(u): Proposition: Consider ũ 0 and X (ũ) = {x X : w(ũ) = c t x + ũ t (d Dx)} the set of optimal solutions of the Lagrangian subproblem (3). Then For each x(ũ) X (ũ), the vector (d Dx(ũ)) w(ũ). Each subgradient of w(u) at ũ can be expressed as a convex combination of subgradients (d Dx(ũ)) with x(ũ) X (ũ). Edoardo Amaldi (PoliMI) Optimization Academic year / 40

24 Proof: (first point) By definition of w(u): w(u) c t x + u t (d Dx) x X, u 0. In particular, for any x(ũ) X (ũ) we have w(u) c t x(ũ) + u t (d Dx(ũ)) u 0 (12) and clearly w(ũ) = c t x(ũ) + ũ t (d Dx(ũ)). (13) Substracting (13) from (12) we obtain w(u) w(ũ) (u t ũ t )(d Dx(ũ)) u 0 which is equivalent to w(u) w(ũ) + (u t ũ t )(d Dx(ũ)) u 0. Thus (d Dx(ũ)) w(ũ). Edoardo Amaldi (PoliMI) Optimization Academic year / 40

25 Procedure: 1) Select an initial u 0 and set k := 0. 2) Solve the Lagrangian subproblem w(u k ) = min {c t x + u t k (d Dx) : x X }. Let x(u k ) be the optimal solution found, then (d Dx(u k )) is a subgradient of w(u) at u k. 3) Update the Lagrange multipliers: u k+1 = max{0, u k + α k (d Dx(u k ))} with, for instance, α k = ε k ŵ w(u k ) d Dx(u k ) 2, where ŵ is an estimate of the optimal value w. 4) Set k := k + 1 Edoardo Amaldi (PoliMI) Optimization Academic year / 40

26 Example: Lagrangian relaxation for the binary knapsack problem Consider max z = 10x 1 + 4x x 3 s.t. 3x 1 + x 2 + 4x 3 4 x 1, x 2, x 3 {0, 1} Lagrangian dual: min w(u) = min ( max (10 3u)x1 + (4 u)x2 + (14 4u)x3 + 4u ) u 0 u 0 x {0,1} 3 with the dual function w(u) = 28 4u for u [0, 10 3 ] 18 u for u [ 10 3, 14 4 ] 4 + 3u for u [ 14 4, 4] 4u for u [4, ) Edoardo Amaldi (PoliMI) Optimization Academic year / 40

27 w(u) u Optimal solution u = 14 4 with w = w(u ) = Edoardo Amaldi (PoliMI) Optimization Academic year / 40

28 Lagrangian dual: min w(u) = min ( max (10 3u)x1 + (4 u)x2 + (14 4u)x3 + 4u ) u 0 u 0 x {0,1} 3 Subgradient γ k = (4 3x k 1 x k 2 4x k 3 ), where x k = x(u k ) is any optimal solution of the Lagrangian subproblem at the k-th iteration. Lagrange multiplier update: u k+1 = max{0, u k α k γ k } with α k = 1 2 k Subgradient method: k u k α k w(u k ) x k γ k max (10, 4, 14) t x + 4u = 28 (1, 1, 1) max ( 2, 0, 2) t x + 4u = (0, 0, 0)* max (4, 2, 6) t x + 4u = (1, 1, 1) max (1, 1, 2) t x + 4u = (1, 1, 1) max ( 0.5, 0.5, 0) t x + 4u = (0, 1, 0)* 3 Symbol : there are several optimal solutions x k and we choose the lexicographically smallest one (set to 0 each variable xi k with zero coefficient) N.B.: The optimal value of the multiplier u = 14 4 considered subgradient is nonzero. is reached at iteration k = 4 but the Edoardo Amaldi (PoliMI) Optimization Academic year / 40

29 Lagrangian relaxation for the STSP (Held & Karp) Symmetric TSP: Given an undirected graph G = (V, E) with a cost c e Z + for each edge e E, determine a Hamiltonian cycle of minimum total cost. ILP formulation: min s.t. e E cexe e δ(i) e E(S) xe = 2 i V (14) xe S 1 S V, 2 S n 1 (15) x e {0, 1} e E where E(S) = {{i, j} E : i S, j S} Observation: i) Due to the presence of constraints (14), half of the subtour-elimination ones (15) are redundant: e E(S) xe S 1 iff e E(S) xe S 1, where S = V \ S. Thus all the constraints (15) with 1 S can be deleted. ii) Summing over all the degree constraints (14) and dividing by 2, we obtain xe = n that can be added to the formulation. e E Edoardo Amaldi (PoliMI) Optimization Academic year / 40

30 Recall that a Hamiltonian cycle is a 1-tree (i.e., a spanning tree on nodes {2,..., n} plus two edges incident to node 1) in which all nodes have exactly two incident edges. Since e E cexe + i V u i(2 e δ(i) xe) = e={i,j} E (ce u i u j )x e + 2 i V u i, relaxing/dualizing the degree constraints (14) for all nodes except for node 1, we obtain the Lagrangian subproblem: w(u) = min s.t. e E (ce u i u j )x e + 2 i V u i e δ(1) xe = 2 e E(S) xe S 1 e E xe = n x e {0, 1} S V, 2 S n 1, 1 S e E where u 1 = 0 and E(S) = {{i, j} E : i S, j S}. N.B.: The set of feasible solutions of this problem coincides with the set of all 1-trees. To find a minimum cost 1-tree: determine a minimum cost spanning tree on nodes {2,..., n} (Kruskal or Prim) and select two smallest cost edges incident to node 1. Edoardo Amaldi (PoliMI) Optimization Academic year / 40

31 Observation: Since constraints e δ(1) xe = 2 e E(S) xe S 1 e E xe = n S V, 2 S n 1, 1 S with x 0 describe the convex hull of the (binary) incidence vectors of 1-trees, Corollary 2 implies that w = z LP. The linear relaxation min s.t. e E cexe e δ(i) xe = 2 e E(S) i V xe S 1 S V, 2 S n 1 x e 0 e E with an exponential number of constraints can thus be solved without considering them explicitly. Since the dualized constraints are equations, the Lagrangian dual is: max w(u) u R V : u 1 =0 Edoardo Amaldi (PoliMI) Optimization Academic year / 40

32 Example: taken from L. Wolsey, Integer Programming, p Consider the undirected graph G = (V, E) with 5 nodes and the cost matrix: Dual function: w(u k ) = min e ui e={i,j} E(c k uj k )xe k + 2 i V u k i : x k incidence vector of a 1-tree Notation: cij k = c e ui k uj k for e = {i, j} E Subgradient γ k with γi k = (2 e δ(i) x e k ), where x k = x(u k ) is an optimal solution of the Lagrangian subproblem at the k-th iteration. Since e δ(1) xe = 2 is not relaxed, γk 1 = 0 for all k. Starting from u1 k = 0 for k = 0, this implies that u1 k = 0 for each k 1. Edoardo Amaldi (PoliMI) Optimization Academic year / 40

33 A feasible solution of cost 148 found with a primal heuristic: x 12 = x 23 = x 34 = x 45 = x 51 = 1 and x ij = 0 for all other {i, j} E Solution of the Lagrangian dual starting from u 0 = 0 with ε = 1: Solving the Lagrangian subproblem with costs: C 0 = C = (c 0 e = c e for each e E since u 0 = 0), we find x(u 0 ) that corresponds to the 1-tree of cost 130: x 12 = x 13 = x 23 = x 34 = x 35 = 1 and x ij = 0 for all other {i, j} E Edoardo Amaldi (PoliMI) Optimization Academic year / 40

34 Knowing x(u 0 ), we can compute the value of the dual function: w(u 0 ) = , equivalent to 1-tree cost + 2 i V u0 i. Subgradient Update the Lagrange multipliers: γ 0 = u 1 = u 0 + (ŵ w(u0 )) γ 0 ( ) = 0 + γ = Thus C 1 = Edoardo Amaldi (PoliMI) Optimization Academic year / 40

35 As optimal solution x(u 1 ) of the Lagrangian subproblem with cost matrix C 1 we find the 1-tree of cost 143: x 12 = x 13 = x 23 = x 34 = x 45 = 1 and x ij = 0 for all other {i, j} E and w(u 1 ) = i V u1 i = 143. Since we have u 2 = u 1 + (ŵ w(u1 )) γ 1 = γ 2 1 γ 1 = , + ( ) = Edoardo Amaldi (PoliMI) Optimization Academic year / 40

36 Therefore C 2 = and we obtain x(u 2 ) that corresponds to the 1-tree of cost 147.5: x 12 = x 15 = x 23 = x 35 = x 45 = 1 and x ij = 0 for all other {i, j} E and w(u 2 ) = Since all costs c e are integer, the feasible solution of cost 148 found by the heuristic is optimal. Edoardo Amaldi (PoliMI) Optimization Academic year / 40

37 Choice of the Lagrangian dual For problems with different groups of constraints, we need to decide which ones to relax. Choice criteria: i) Strength of the bound w obtained by solving the Lagrangian dual, ii) Difficulty of solving the Lagrangian subproblems w(u) = min {c t x + u t (d Dx) : x X R n }, iii) Difficulty of solving the Lagrangian dual: w = max u 0 w(u). For (i) we have the characterization of the Lagrangian dual bound w in terms of LP. The difficulty of the Lagrangian subproblems depends on the specific problem. The difficulty of the Lagrangian dual depends, among others, on the number of dual variables. We look for a reasonable trade-off. Edoardo Amaldi (PoliMI) Optimization Academic year / 40

38 Example: Generalized assignment problem Given a set I of processes and a set J of machines with c ij cost for executing process i I on machine j J, w ij the amount of resource required to execute process i I on machine j J, b j the total amount of resource available on machine j J. assign the processes to the machines so as to minimize the total cost, while respecting the resource constraints. Assumption: Once started processes cannot be interrupted. ILP formulation min z = s.t. i I j J c ijx ij j J x ij = 1 i I w ijx ij b j x ij {0, 1} i I j J i I, j J where x ij = 1 if process i I is assigned to machine j J, and x ij = 0 otherwise. Edoardo Amaldi (PoliMI) Optimization Academic year / 40

39 Three possible Lagrangian relaxations: 1) Relaxing the capacity constraints: w 1(u) = min i I j J c ijx ij j J u j(b j i I w ijx ij ) s.t. j J x ij = 1 x ij {0, 1} i I i I, j J Trivial solution: assign each process i I to machine j J with min c ij + u j w ij. N.B.: If the integrality conditions were relaxed, the solution would not change because for each i conv({x i {0, 1} J : j J x ij = 1}) = {x R J : j J x ij = 1, 0 x i 1}. 2) Relaxing the assignment constraints: w 2(v) = min i I j J c ijx ij i I v i( j J x ij 1) s.t. i I w ijx ij b j x ij {0, 1} j J i I, j J Since the variables x ij corresponding to different machines j are not linked, the subproblem decomposes into J independent binary knapsack subproblems, with profit c ij + v i (max version) and weight w ij for item i I. Edoardo Amaldi (PoliMI) Optimization Academic year / 40

40 3) Relaxing all the constraints, we obtain the Lagrangian subproblem: w 3(u, v) = min i I j J c ijx ij i I v i( j J x ij 1) j J u j(b j i I w ijx ij ) x ij {0, 1} i I, j J Trivial solution: set x ij = 1 if c ij v i + u j w ij < 0, x ij = 0 if c ij v i + u j w ij > 0 and arbitrarily x ij = 0 or x ij = 1 if c ij v i + u j w ij = 0. Observations: i) According to Corollary 2, the first and third Lagrangian relaxation bounds are as weak as (not stronger than) the linear relaxation one. ii) Since the ideal formulation for a binary knapsack problem contains many other inequalities (e.g., cover inequalities), namely for each j J conv({x j {0, 1} I : i I w ij x ij b j }) {x j R I : i I w ij x ij b j, 0 x j 1}, the second Lagrangian relaxation provides a potentially stronger bound. Edoardo Amaldi (PoliMI) Optimization Academic year / 40