Extended Formulations, Lagrangian Relaxation, & Column Generation: tackling large scale applications

Transcription

1 Extended Formulations, Lagrangian Relaxation, & Column Generation: tackling large scale applications François Vanderbeck University of Bordeaux INRIA Bordeaux-Sud-Ouest part : Defining Extended Formulations François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. / 72

2 Outline Extented Formulations Formulation Extended Formulation Reformulation Decomposition & Reformulation 2 Examples Steiner Tree Problem Traveling Salesman Problem Capacitated Network Design 3 How to build them Variable Splitting DP based reformulation LP separation Union of Polyhedra Reduced coefficient & basis 4 Interests of Reformulations François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 2 / 72

3 Extented Formulations Outline Extented Formulations Formulation Extended Formulation Reformulation Decomposition & Reformulation 2 Examples Steiner Tree Problem Traveling Salesman Problem Capacitated Network Design 3 How to build them Variable Splitting DP based reformulation LP separation Union of Polyhedra Reduced coefficient & basis 4 Interests of Reformulations François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 3 / 72

4 Extented Formulations Outline Extented Formulations Formulation Extended Formulation Reformulation Decomposition & Reformulation 2 Examples Steiner Tree Problem Traveling Salesman Problem Capacitated Network Design 3 How to build them Variable Splitting DP based reformulation LP separation Union of Polyhedra Reduced coefficient & basis 4 Interests of Reformulations François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 4 / 72

5 Extented Formulations Combinatorial Optimization Problem (CO) min{c(s) : s S} where S is the discrete set of feasible solutions. (7) (6) (5) S (4) François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 5 / 72

6 Extented Formulations Combinatorial Optimization Problem (CO) min{c(s) : s S} where S is the discrete set of feasible solutions. Formulation A polyhedron P = {x n : Ax a} is a formulation for (CO) iff min{c(s) : s S} min{cx : x P I = P n }. x2 x2 (7) 5 5 (6) (5) 3 (7) 3 (7) (6) (5) (6) (5) S (4) 2 (4) 2 (4) x x François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 5 / 72

7 Extented Formulations MIP - formulation & Generic Solver Integer Program (IP) min{cx : x X} where X = P n with P = {x n + : Ax a}. x2 5 3 (7) (6) (5) 2 (4) x François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 6 / 72

8 Extented Formulations MIP - formulation & Generic Solver Integer Program (IP) min{cx : x X} where X = P n with P = {x n + : Ax a}. Mixed Integer Program (MIP) min{cx + hy : (x, y) X} where X = P ( n p ) with P = {(x, y) n + p + : Gx + Hy b}. François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 6 / 72

9 Extented Formulations MIP - formulation & Generic Solver Integer Program (IP) min{cx : x X} where X = P n with P = {x n + : Ax a}. x2 x2 P0 5 5 x < 3 x > 4 3 (7) 3 (7) P P2 (6) (5) (6) (5) 2 2 (4) (4) x x François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 6 / 72

10 Extented Formulations MIP - formulation & Generic Solver Integer Program (IP) min{cx : x X} where X = P n with P = {x n + : Ax a}. x2 x2 P0 5 5 x < 3 x > 4 3 (7) 3 (7) P P2 (6) (5) (6) (5) 2 2 (4) (4) x x MIP solvers are efficient but fail beyond a certain size. They barely exploit problem structure. The quality of the formulation is key for the solver. François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 6 / 72

11 Extented Formulations Alternative formulations A formulation is typically not unique P and P can be alternative formulations for (CO) if (CO) min{cx : x P n } min{c x : x P n } x2 5 3 (7) (6) (5) 2 (4) x warning: can expressed in different variable-spaces. François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 7 / 72

12 Extented Formulations Quality of Formulations Stronger formulation (in the same space) Formulation P n is a stronger than P n if P P. Then, min{cx : x P } min{cx : x P} x2 5 3 (7) (6) (5) 2 (4) x Dual Bound quality + considerations of Size + Symmetry issues François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 8 / 72

13 Extented Formulations Ideal Formulation The Convex hull of an IP set, P I conv(p I ) is the smallest closed convex set containing P I. x2 5 3 (7) (6) (5) 2 (4) x conv(p I ) is an ideal polyhedron / formulation If P I is defined by rational data, conv(p I ) is a polyhedron. François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 9 / 72

14 Extented Formulations Outline Extented Formulations Formulation Extended Formulation Reformulation Decomposition & Reformulation 2 Examples Steiner Tree Problem Traveling Salesman Problem Capacitated Network Design 3 How to build them Variable Splitting DP based reformulation LP separation Union of Polyhedra Reduced coefficient & basis 4 Interests of Reformulations François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 0 / 72

15 Extented Formulations Extended Formulation Given an initial compact formulation: x2 5 3 (7) (6) (5) 2 (4) x François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. / 72

16 Extented Formulations Extended Formulation w x2 (5) (4) x François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 2 / 72

17 Extented Formulations Extended Formulation w x2 (5) (4) x François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 3 / 72

18 Extented Formulations Extended Formulation w x2 (5) (4) x François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 4 / 72

19 Extented Formulations Projection The Projection of Q = {(x, w) n+e : Gx + Hw d} on the x-space is: proj x (Q) := {x n : w e such that (x, w) Q}. w x2 (5) (4) x François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 5 / 72

20 Extented Formulations Projection The Projection of Q = {(x, w) n+e : Gx + Hw d} on the x-space is: proj x (Q) := {x n : w e such that (x, w) Q}. Farka s Lemma Given x, {w n + : Hw (d G x )} if and only if v m + : vh 0, v(d G x ) 0. Hence, a polyhedral description of the projection in the x-space is: x > 2 3 x x2 > 0 proj x (Q) = {x n : v j (d Gx) 0 j J} x + 2 x2 > 0 {v j } j J, exteme rays. of {v m + : vh 0}. x + x2 > 6 François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 5 / 72

21 Extented Formulations Extended Formulations An extended formulation for an IP set P I n is a polyhedron Q = {(x, w) n+e : Gx + Hw d} such that P I = proj x (Q) n. w x2 (5) (4) x François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 6 / 72

22 Extented Formulations Extended Formulations An extended formulation for an IP set P I n is a polyhedron Q = {(x, w) n+e : Gx + Hw d} such that P I = proj x (Q) n. w x2 (5) (4) x François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 6 / 72

23 Extented Formulations Tight Extended Formulations A tight extended formulation for an IP set P I n is a polyhedron Q = {(x, w) n+e : Gx + Hw d} such that conv(p I ) = proj x (Q). w x2 (5) (4) x François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 7 / 72

24 Extented Formulations Compact Extended Formulations A formulation (resp. extended f.) is Compact if the length of the description of P (resp. Q) is polynomial in the input length of the description of CO. w 5 3 x2 5 (5) (4) x François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 8 / 72

25 Extented Formulations Compact Extended Formulations A formulation (resp. extended f.) is Compact if the length of the description of P (resp. Q) is polynomial in the input length of the description of CO. w 5 3 x2 5 (5) (4) x Compactness of an Ideal Formulation An ideal formulation cannot be compact unless CO is in. François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 8 / 72

26 Extented Formulations IP Extended Formulations An extended IP-formulation for an IP set P I n is an IP-set Q I = {(x, w) n e : Gx + Hw b} s.t. P I = proj x Q I. w x2 (5) (4) x François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 9 / 72

27 Extented Formulations Outline Extented Formulations Formulation Extended Formulation Reformulation Decomposition & Reformulation 2 Examples Steiner Tree Problem Traveling Salesman Problem Capacitated Network Design 3 How to build them Variable Splitting DP based reformulation LP separation Union of Polyhedra Reduced coefficient & basis 4 Interests of Reformulations François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 20 / 72

28 Extented Formulations Reformulation Change of variables: x=t w x2 5 3 (7) 2 (6) (5) W3 (4) W x W François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 2 / 72

29 Extented Formulations Reformulation: a special case of extended formulation An extended formulation based on a change of variables: x = Tw. Then, Q = {(x, w) n+e : Tw = x Ew e}. proj x (Q) = T(W) := {x = Tw n : Ew e, w }{{} e }. w W A reformulation for an IP-set P I n is a polyhedron W along a linear transformation, x = Tw, s.t. P I =T(W) n A IP-reformulation for an IP-set P I n is an IP-set W I = W e along a linear transformation, x = Tw, s.t., P I = T(W I ) François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 22 / 72

30 Extented Formulations Minkowski s representation: a special case of reformulation Polyhedron conv(p I ) can be defined by its extreme points and rays: Q = {(x, λ, µ) n G + R + : x = x g λ g + v r µ r, λ g = } g G r R g G change of variables: x = X λ + V µ. François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 23 / 72

31 Extented Formulations Outline Extented Formulations Formulation Extended Formulation Reformulation Decomposition & Reformulation 2 Examples Steiner Tree Problem Traveling Salesman Problem Capacitated Network Design 3 How to build them Variable Splitting DP based reformulation LP separation Union of Polyhedra Reduced coefficient & basis 4 Interests of Reformulations François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 24 / 72

32 Extented Formulations Extended formulation based on a subset of constraints Original formulation [F] min c x A x a B x b x n Subproblem P B x b x n + P I = P n Decomposition + SP Reformulation François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 25 / 72

33 Extented Formulations Extended formulation based on a subset of constraints Original formulation [F] min c x A x a B x b x n Subproblem P B x b x n + P I x = Tw : Ew e, w e P I = P n Extended reformulation [R] min c T w A T w a E w e w p François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 25 / 72

34 Extented Formulations Extended formulation based on a subset of constraints Original formulation [F] min c x A x a B x b x n Subproblem P B x b x n + P I = P n Special case: Dantzig-Wolfe Reformulation [M] min c x g λ g g G A x g λ g a g G λ g = g G λ {0, } G Applying Minkowski x = g G xg λ g x x 2 x 3 x 4 François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 26 / 72

35 Extented Formulations Price Decomposition (Dantzig) (IP) z = min{cx : Ax a, Bx b, x n + } where Ax a represent complicating constraints while the set Bx b is more tractable Relaxing Ax a while penalizing (pricing) their violation in the objective Lagrangian relaxation Reformulate the problem as selection of solutions to set Bx b that satisfy Ax a Dantzig-Wolfe Reformulation Column Generation François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 27 / 72

36 Extented Formulations Dantzig-Wolfe Decomposition: The block diagonal case min c x + c 2 x c K x K A x + A 2 x A K x K a B x b B 2 x 2 b B K x K x n +, x2 n 2 +,... xk n K +. b K Relaxing the constraints Ax a decomposes the problem into K smaller size optimization problems: min{c k x k : B k x k b K } The complicating constraints only depend on the aggregate variables: y = K k= xk Y = {y n : Ay a}. + François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 28 / 72

37 Extented Formulations Extended formulation based on a subset of variables Original formulation [F] min c x + h y Gx + Hy d x n, y p Subproblem P Hy d Gx P I y = Tw : Ew e(x), w e y q P I = P q Extended reformulation [R] min cx + h T w G x + H T w d E w e(x) x n, w e François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 29 / 72

38 Extented Formulations Resource Decomposition (Benders) min cx + hy Gx + Hy d x n, y p + The integer variables x are seen as the important decisions: ex. network design Fix x and compute the associated optimal y (solve SP). A feedback loop allowing one to adjust the x solution after obtaining the associated y: Bender s cuts. François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 30 / 72

39 Extented Formulations Benders Decomposition min{cx + hy : Gx + Hy d, x Z n, y R p + } min{cx + φ(x) : x proj x (Q) n } a MIP where Q = {(x, y) R n R p + : Gx + Hy d} φ(x) = min{hy : Hy d Gx, y R p + } = max{u(d Gx) : uh h, u R m + } = max t=,...,t {ut (d Gx)} for x proj x (Q) u t are extreme points of U = {u m + : uh h}, vr are extreme rays; Bender s Master min cx + σ σ u t (d Gx) t =,, T v r (d Gx) 0 r =,, R x n François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 3 / 72

40 Extented Formulations Benders Decomposition: The block diagonal case min cx + h y + h 2 y h K y K G x + H y d G 2 x + H 2 y 2 d G K x + H K y K d K x n, y k q k =,..., K Fixing x leads to a decomposition per block in y k variables If moreover, blocks are identical, i.e. (H k, h k ) = (H, h) k, Benders cut generators obtained for one SP are valid forall k François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 32 / 72

41 Extented Formulations Resource Splitting (Dantzig) min cx +h y +h 2 y 2 +h 3 y 3 x x x 2 x 3 = 0 G x + H y d G 2 x 2 + H 2 y 2 d 2 G 3 x 3 + H 3 y 3 d 3 x n, (x k, y k ) n+q k =,..., 3 split x using x = k xk (or x = x k k) Lagrangian dualization of constraints x = k xk (or x = x k k) François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 33 / 72

42 Examples Outline Extented Formulations Formulation Extended Formulation Reformulation Decomposition & Reformulation 2 Examples Steiner Tree Problem Traveling Salesman Problem Capacitated Network Design 3 How to build them Variable Splitting DP based reformulation LP separation Union of Polyhedra Reduced coefficient & basis 4 Interests of Reformulations François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 34 / 72

43 Examples Outline Extented Formulations Formulation Extended Formulation Reformulation Decomposition & Reformulation 2 Examples Steiner Tree Problem Traveling Salesman Problem Capacitated Network Design 3 How to build them Variable Splitting DP based reformulation LP separation Union of Polyhedra Reduced coefficient & basis 4 Interests of Reformulations François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 35 / 72

44 Examples Example: Steiner Tree B 6 7 A C 2 3 D François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 36 / 72

45 Examples Example: Steiner Tree B 6 7 A C 2 3 D François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 37 / 72

46 Examples Example: Steiner Tree B 6 7 A C 2 3 D Special cases (that are easy ): T = {i} : shortest path from r to i T = V \ {r} : minimum cost spanning tree François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 37 / 72

47 Examples Steiner Tree: Arc flow formulation Variables j V (i) j V (i) x ij {0, } arc (i, j) is used or not y ij number of connections going through (i, j) min y ji y ji (i,j) A j V + (r) j V + (i) j V + (i) c ij x ij y rj = T y ij = i T y ij = 0 i V \ (T {r}) y ij T x ij (i, j) A y A +, x {0, } A A C 2 3 François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 38 / 72 B 6 7 D

48 Examples Example: Steiner Tree B 6 7 A C 2 3 D François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 39 / 72

49 Examples Steiner Tree: Multi commodity flow formulation Variable splitting j V (i) j V (i) w t ij {0, } arc (i, j) is used to connect terminal t y ij = k wt ij defines a linear transformation min (i,j) A j V + (r) w t ji j V + (i) w t ji j V + (i) c ij x ij w t rj = t T w t ij = i = t T w t ij = 0 i V \ {r, k}, t T w t ij x ij (i, j) A, t T w K A +, x {0, } A A C 2 3 François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 40 / 72 B 6 7 D

50 Examples Steiner Tree: Path flow formulation Decomposition λ t p {0, } path p is used to connect terminal t min c ij x ij (i,j) A p P(k) p P(k) λ t p = t T δ p ij λt p x ij (i, j) A, t T λ t p {0, } P(k) t T x ij {0, } François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 4 / 72

51 Examples Example: Steiner Tree S B 6 7 A C 2 3 D François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 42 / 72

52 Examples Steiner Tree: Network design formulation min c ij x ij (i,j) A (i,j) δ + (S) projection in the x-space x ij S r, T \ S x {0, } A, S B 6 7 A C 2 3 D François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 43 / 72

53 Examples Steiner Tree: Network design formulation min c ij x ij (i,j) A (i,j) δ + (S) projection in the x-space x ij S r, T \ S x {0, } A, S B 6 7 A C 2 3 D Note: This projection onto the x space has the same LP value than the multi-commodity flow formulation is better than the initial compact aggregate flow formulation. François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 43 / 72

54 Examples Outline Extented Formulations Formulation Extended Formulation Reformulation Decomposition & Reformulation 2 Examples Steiner Tree Problem Traveling Salesman Problem Capacitated Network Design 3 How to build them Variable Splitting DP based reformulation LP separation Union of Polyhedra Reduced coefficient & basis 4 Interests of Reformulations François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 44 / 72

55 Examples Multi-commodity flow: Three-Index Flow for the ATSP min c ij x ij x ij = x ij = i V j j i S j V\S min c ij x ij w t ij w t ji = j j x ij S with φ S V x {0, } A i = r t = i 0 otherwise t V \ {r} w t ij x ij (i, j) A, t V \ {r} x {0, } A, w [0, ] (i, j) A, t V \ {r} François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 45 / 72

56 Examples Outline Extented Formulations Formulation Extended Formulation Reformulation Decomposition & Reformulation 2 Examples Steiner Tree Problem Traveling Salesman Problem Capacitated Network Design 3 How to build them Variable Splitting DP based reformulation LP separation Union of Polyhedra Reduced coefficient & basis 4 Interests of Reformulations François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 46 / 72

57 Examples Multi-Commodity Capacitated Network Design [F] min{ c k ij xk ij + f ij y ij ijk ij x k ji x k = d k i, k ij i j j x k u ij ij y ij i, j k x k ij 0 i, j, k y ij i, j} [SP ij ] min{ c k x k + f y : k x k u y k x k min{d k, u y} k} François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 47 / 72

58 Examples Network Design: Extended form. for the SPs Let y s ij = and xks ij = x k ij if y ij = s. [SP ij ] min{ c k ij xks + f ij ij s y s ij : ks (s ) u ij y s ij s y s ij k s x ks ij s u ij y s ij s x ks ij min{d k, s u ij } y s ij k, s} Extended formulation for the arc design subproblem (Union of Polyhedra) [Croxton, Gendron and Magnanti OR07] François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 48 / 72

59 Examples Network Design: extended formulation [R] min{ c k ij xks + f ij ij s y s ij ijks js ijs x ks ji js (s ) u ij y s ij k x ks ij = d k i i, k x ks ij s u ij y s ij i, j, s 0 x ks d k y s i, j, k, s ij ij y s = i, j ij s y s ij {0, } i, j, s} [Frangioni & Gendron, DAM09] François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 49 / 72

60 Examples Network Design: Union of Polyhedra François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 50 / 72

61 Examples Network Design: column genenration formulation [M] min{ (c k ij xg + f ks ij s y g s ) λij g i,j,s,g G ij x g ks λij g x g ks λij g = d k i, k i js g G ij js g G ij λ ij g i, j g G ij λ ij g {0, } i, j, g G ij } [Frangioni & Gendron WP0] François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 5 / 72

62 How to build them Outline Extented Formulations Formulation Extended Formulation Reformulation Decomposition & Reformulation 2 Examples Steiner Tree Problem Traveling Salesman Problem Capacitated Network Design 3 How to build them Variable Splitting DP based reformulation LP separation Union of Polyhedra Reduced coefficient & basis 4 Interests of Reformulations François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 52 / 72

63 How to build them Outline Extented Formulations Formulation Extended Formulation Reformulation Decomposition & Reformulation 2 Examples Steiner Tree Problem Traveling Salesman Problem Capacitated Network Design 3 How to build them Variable Splitting DP based reformulation LP separation Union of Polyhedra Reduced coefficient & basis 4 Interests of Reformulations François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 53 / 72

64 How to build them Ways to obtain extended formulations Variable Splitting Multi-Commodity Flow: x ij = k xk ij Unary expansion: x = u q=0 q w q, u q=0 w q =, w {0, } u+ Binary expansion: x = log u p=0 w p,, w {0, } log u Dynamic Programming Solver Network Flow LP [Martin et al] Separation is easy Separation LP [Martin et al] Reduced coefficent / basis reformulations [Aardal et al] Union of Polyhedra [Balas]... François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 54 / 72

65 How to build them Unary expansion: Time-Indexed Formulation Single machine scheduling problem (with integer data): S 3 S 2 S t S j S i + p i or S i S j + p j i, j requires big M formulation: S j S i + p i M( x ij ). François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 55 / 72

66 How to build them Unary expansion: Time-Indexed Formulation Single machine scheduling problem (with integer data): S 3 3 S 2 2 S t S j S i + p i or S i S j + p j i, j Change of variables: S j = t w jt with w jt = iff job j starts at the beginning of [t, t + ]. w j0 = j J w jt w j,t pj = 0 t j J j J t François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 55 / 72

67 How to build them Ways to obtain extended formulations Variable Splitting Multi-Commodity Flow: x ij = k xk ij Unary expansion: x = u q=0 q w q, u q=0 w q =, w {0, } u+ Binary expansion: x = log u p=0 w p,, w {0, } log u Dynamic Programming Solver Network Flow LP [Martin et al] Separation is easy Separation LP [Martin et al] Reduced coefficent / basis reformulations [Aardal et al] Union of Polyhedra [Balas]... François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 56 / 72

68 How to build them Outline Extented Formulations Formulation Extended Formulation Reformulation Decomposition & Reformulation 2 Examples Steiner Tree Problem Traveling Salesman Problem Capacitated Network Design 3 How to build them Variable Splitting DP based reformulation LP separation Union of Polyhedra Reduced coefficient & basis 4 Interests of Reformulations François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 57 / 72

69 How to build them DP based reformulation: the knapsack example max{ p i x i : a i x i b, x i } i i DP Recursion: V(c) = max i=,...,n:c ai {V(c a i ) + p i } in LP form: min V(b) V(c) V(c a i ) p i i =,..., n, c = a i,, b V(0) = 0 its Dual: longest path problem François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 58 / 72

70 How to build them DP based reformulation: the knapsack example max{ p i x i : a i x i b, x i } i DP Recursion: V(c) = max i=,...,n:c ai {V(c a i ) + p i } in LP form: min V(b) i V(c) V(c a i ) p i i =,..., n, c = a i,, b V(0) = 0 its Dual: longest path problem max n b ai j= r=0 c iw ic i w ic = c = 0 i w ic i w i,c a i = 0 c =,, b i w i,c a i = c = b w ic 0 i =,, n; c = 0,, b a i François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 58 / 72

71 How to build them DP based reformulation: Multi-Echelon Lot-Sizing Variables x e,t production of intermediate product of echelon e in period t s e,t stock of echelon e product at the end of period t x e,t + s e,t = x e+,t + s e,t for e =,..., E x e,t + s e,t = d t + s e,t for e = E e = e = 2 e = 3 t François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 59 / 72

72 How to build them DP based reformulation: Multi-Echelon Lot-Sizing Dominance property opt solution where x e,t s e,t = 0 e, t, production plan is a tree: e = e = 2 e = 3 t François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 60 / 72

73 How to build them DP based reformulation: Multi-Echelon Lot-Sizing Dominance property opt solution where x e,t s e,t = 0 e, t, production plan is a tree: e = e = 2 e = 3 Dynamic programming State (e, t, a, b) corresponds to accumulating at echelon e in period t a production covering exactly the demand of periods a,..., b. V(e, t, a, b) = min{v(e, t +, a, b), min {V(e +, t, a, l) + l=a,...,b ck et Dk al + f k et + V(e, t +, l +, b)}} François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 60 / 72 t

74 How to build them DP based reformulation: Multi-Echelon Lot-Sizing DP Recursion: V(e, t, a, b) = min{v(e, t +, a, b), min {V(e +, t, a, l) + l=a,...,b ck et Dk al + f k et + V(e, t +, l +, b)}} in LP form: max V(,,, T) V(e, t, a, b) V(e, t +, a, b) e, t, a, b V(e, t, a, b) V(e +, t, a, l) + c k et Dk al + f k et + V(e, t +, l +, b) e, t, a, b, l V(E +, t, a, b) = 0 t, a, b its Dual: flow on hyper-arcs w e,t,a,l,b = if at echelon e in period t production covers demands from period a to period l, while the rest of demand up to b, shall be covered in the future. François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 6 / 72

75 How to build them DP based reformulations [Martin et al OR90] When a problem can be solved by dynamic programming, V(l) = min { V(j) + c(j, l)}, (J,l) j J an extended formulation consist in modeling a decision tree in an hyper-graph François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 62 / 72

76 How to build them Ways to obtain extended formulations Variable Splitting Multi-Commodity Flow: x ij = k xk ij Unary expansion: x = u q=0 q w q, u q=0 w q =, w {0, } u+ Binary expansion: x = log u p=0 w p,, w {0, } log u Dynamic Programming Solver Network Flow LP [Martin et al] Separation is easy Separation LP [Martin et al] Reduced coefficent / basis reformulations [Aardal et al] Union of Polyhedra [Balas]... François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 63 / 72

77 How to build them Outline Extented Formulations Formulation Extended Formulation Reformulation Decomposition & Reformulation 2 Examples Steiner Tree Problem Traveling Salesman Problem Capacitated Network Design 3 How to build them Variable Splitting DP based reformulation LP separation Union of Polyhedra Reduced coefficient & basis 4 Interests of Reformulations François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 64 / 72

78 How to build them Reformulation of the Uncapacitated Lot-Sizing: LP sep. & reform. min n t= p tx t + n t= h ts t + n t= q ty t s t + x t = d t + s t t x t My t t s, x R n +, y {0, }n François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 65 / 72

79 How to build them Reformulation of the Uncapacitated Lot-Sizing: LP sep. & reform. min n t= p tx t + n t= h ts t + n t= q ty t s t + x t = d t + s t t x t My t t s, x R n +, y {0, }n Facet-defining inequalities: L = {,..., l}, S L x j + d jl y j d l j S j L\S Let µ jl = min{x j, d jl y j } for j l n a tight and compact extended formulation is obtained from the OF by adding: l j= µ jl d l l n µ jl x j j l n µ jl d jl y j j l n. François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 65 / 72

80 How to build them LP separation & reformulation: example 2 Robust Optimization: min{ c x A ξ x a ξ Ξ with A ξ = A + k Ak ξ k x n }, and Ξ = {ξ K : B ξ b}. The separation problem: a ij x + min{ a k ij ξ kx j : Bξ b, ξ K } a i0? i j k j a ij x + max{u b : u B a k ij x j k, u m } a i0? j j The extended formulation: min{ c x a ij x + u b a i0 i j u B a k x ij j k u m x n }. j François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 66 / 72 i

81 How to build them Outline Extented Formulations Formulation Extended Formulation Reformulation Decomposition & Reformulation 2 Examples Steiner Tree Problem Traveling Salesman Problem Capacitated Network Design 3 How to build them Variable Splitting DP based reformulation LP separation Union of Polyhedra Reduced coefficient & basis 4 Interests of Reformulations François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 67 / 72

82 How to build them The k Configuration example n Y = {(x 0, x) {0, } n+ : kx 0 + x j n}. n n Y 0 = {x 0 = 0, x j n} Y = {x 0 =, x j n k} j= j= j= Tight extended formulation: x j = x 0 j + x j j =,..., n x 0 j x 0 j =,..., n x j x 0 j =,..., n n x j (n k)x 0 j= x [0, ] 3n 2 François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 68 / 72

83 How to build them Outline Extented Formulations Formulation Extended Formulation Reformulation Decomposition & Reformulation 2 Examples Steiner Tree Problem Traveling Salesman Problem Capacitated Network Design 3 How to build them Variable Splitting DP based reformulation LP separation Union of Polyhedra Reduced coefficient & basis 4 Interests of Reformulations François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 69 / 72

84 How to build them The knapsack problem example X = P n = P 2 n where P = {x [0, ] 5 : 97x + 65x x x x 5 36} P 2 = {x [0, ] 5 : 5x + 3x 2 + 3x 3 + 2x 4 + x 5 6} P 2 P François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 70 / 72

85 Interests of Reformulations Outline Extented Formulations Formulation Extended Formulation Reformulation Decomposition & Reformulation 2 Examples Steiner Tree Problem Traveling Salesman Problem Capacitated Network Design 3 How to build them Variable Splitting DP based reformulation LP separation Union of Polyhedra Reduced coefficient & basis 4 Interests of Reformulations François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 7 / 72

86 Interests of Reformulations Extended formulation: Interests Improved formulation (better LP bound & rounding heuristic) extra variables tighter relations, linearisation w x2 (5) (4) x François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 72 / 72

87 Interests of Reformulations Extended formulation: Interests Improved formulation (better LP bound & rounding heuristic) 2 Simpler formulation (captures the combinatorial structure) extra variables fewer constaints structure built into var. definitions François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 72 / 72

88 Interests of Reformulations Extended formulation: Interests Improved formulation (better LP bound & rounding heuristic) 2 Simpler formulation (captures the combinatorial structure) 3 Direct use of a MIP-Solver (solved by standard tools) François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 72 / 72

89 Interests of Reformulations Extended formulation: Interests Improved formulation (better LP bound & rounding heuristic) 2 Simpler formulation (captures the combinatorial structure) 3 Direct use of a MIP-Solver (solved by standard tools) 4 Rich variable space (to express cuts or branching) Vehicle routing: x a = l=0,...,c wa l w a q = if vehicle on arc a with load l, lw a l lw a l = d i l a δ (i) l a δ + (i) l w_l d_i i l w_l knapsack cover cuts. [Uchoa] François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 72 / 72

90 Interests of Reformulations Extended formulation: Interests Improved formulation (better LP bound & rounding heuristic) 2 Simpler formulation (captures the combinatorial structure) 3 Direct use of a MIP-Solver (solved by standard tools) 4 Rich variable space (to express cuts or branching) Vehicle routing: x a = l=0,...,c wa l w a q = if vehicle on arc a with load l, lw a l lw a l = d i l a δ (i) l a δ + (i) knapsack cover cuts. l w_l [Uchoa] 5 Reformulation can help to eliminate Symmetries d_i i l w_l François Vanderbeck Extended Formulations, Lagrangian Relaxation, Column Gener. 72 / 72