Integer Programming Reformulations: Dantzig-Wolfe & Benders Decomposition the Coluna Software Platform

Similar documents
Stabilization in Column Generation: numerical study

Reformulation and Decomposition of Integer Programs

Column Generation for Extended Formulations

Extended Formulations, Lagrangian Relaxation, & Column Generation: tackling large scale applications

Pedro Munari - COA 2017, February 10th, University of Edinburgh, Scotland, UK 2

Lecture 9: Dantzig-Wolfe Decomposition

Feasibility Pump Heuristics for Column Generation Approaches

Outline. Relaxation. Outline DMP204 SCHEDULING, TIMETABLING AND ROUTING. 1. Lagrangian Relaxation. Lecture 12 Single Machine Models, Column Generation

Acceleration and Stabilization Techniques for Column Generation

Lagrangian Relaxation in MIP

Lecture 8: Column Generation

Branching in Branch-and-Price: a Generic Scheme

Column generation for extended formulations

A generic view of Dantzig Wolfe decomposition in mixed integer programming

Lecture 8: Column Generation

Column Generation. MTech Seminar Report. Soumitra Pal Roll No: under the guidance of

A Column Generation Based Heuristic for the Dial-A-Ride Problem

Column Generation. ORLAB - Operations Research Laboratory. Stefano Gualandi. June 14, Politecnico di Milano, Italy

An Integer Cutting-Plane Procedure for the Dantzig-Wolfe Decomposition: Theory

On the exact solution of a large class of parallel machine scheduling problems

Integer program reformulation for robust branch-and-cut-and-price

Decomposition-based Methods for Large-scale Discrete Optimization p.1

Decomposition and Reformulation in Integer Programming

Interior-Point versus Simplex methods for Integer Programming Branch-and-Bound

The Fixed Charge Transportation Problem: A Strong Formulation Based On Lagrangian Decomposition and Column Generation

Decomposition methods in optimization

Decomposition Methods for Integer Programming

Large-scale optimization and decomposition methods: outline. Column Generation and Cutting Plane methods: a unified view

1 Solution of a Large-Scale Traveling-Salesman Problem... 7 George B. Dantzig, Delbert R. Fulkerson, and Selmer M. Johnson

Benders Decomposition

Strengthened Benders Cuts for Stochastic Integer Programs with Continuous Recourse

Notes on Dantzig-Wolfe decomposition and column generation

3.10 Lagrangian relaxation

Generation and Representation of Piecewise Polyhedral Value Functions

3.10 Column generation method

A Benders Algorithm for Two-Stage Stochastic Optimization Problems With Mixed Integer Recourse

Classification of Dantzig-Wolfe Reformulations for MIP s

Lagrange Relaxation: Introduction and Applications

Convex Optimization. Dani Yogatama. School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, USA. February 12, 2014

Integer Programming ISE 418. Lecture 16. Dr. Ted Ralphs

A Hub Location Problem with Fully Interconnected Backbone and Access Networks

Introduction to Bin Packing Problems

Vehicle Routing and MIP

Network Flows. 6. Lagrangian Relaxation. Programming. Fall 2010 Instructor: Dr. Masoud Yaghini

1. Gradient method. gradient method, first-order methods. quadratic bounds on convex functions. analysis of gradient method

Applications. Stephen J. Stoyan, Maged M. Dessouky*, and Xiaoqing Wang

3.10 Column generation method

Linear Programming Inverse Projection Theory Chapter 3

Solving Dual Problems

Lecture 7: Lagrangian Relaxation and Duality Theory

Lecture 10: Linear programming duality and sensitivity 0-0

LECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE

An Integrated Column Generation and Lagrangian Relaxation for Flowshop Scheduling Problems

Bilevel Integer Programming

A branch-and-price algorithm for parallel machine scheduling using ZDDs and generic branching

An Adaptive Partition-based Approach for Solving Two-stage Stochastic Programs with Fixed Recourse

Column Generation I. Teo Chung-Piaw (NUS)

8. Geometric problems

Lecture 3: Lagrangian duality and algorithms for the Lagrangian dual problem

3.7 Cutting plane methods

Integer Programming ISE 418. Lecture 8. Dr. Ted Ralphs

Bilevel Integer Linear Programming

A PARALLEL INTERIOR POINT DECOMPOSITION ALGORITHM FOR BLOCK-ANGULAR SEMIDEFINITE PROGRAMS IN POLYNOMIAL OPTIMIZATION

KNAPSACK PROBLEMS WITH SETUPS

Uses of duality. Geoff Gordon & Ryan Tibshirani Optimization /

DM545 Linear and Integer Programming. Lecture 13 Branch and Bound. Marco Chiarandini

Separation, Inverse Optimization, and Decomposition. Some Observations. Ted Ralphs 1 Joint work with: Aykut Bulut 1

A Capacity Scaling Procedure for the Multi-Commodity Capacitated Network Design Problem. Ryutsu Keizai University Naoto KATAYAMA

Separation, Inverse Optimization, and Decomposition. Some Observations. Ted Ralphs 1 Joint work with: Aykut Bulut 1

Introduction to Integer Linear Programming

Multicommodity Flows and Column Generation

Lagrangean relaxation

Linear Programming. Leo Liberti. LIX, École Polytechnique. Operations research courses / LP theory p.

Big Data Analytics: Optimization and Randomization

Linear and Integer Programming - ideas

Section Notes 8. Integer Programming II. Applied Math 121. Week of April 5, expand your knowledge of big M s and logical constraints.

Interior Point Cutting Plane Methods in Integer Programming

Stochastic Integer Programming An Algorithmic Perspective

Projection in Logic, CP, and Optimization

Cutting Plane Separators in SCIP

0-1 Reformulations of the Network Loading Problem

8. Geometric problems

Stochastic Decomposition

Accelerating Benders Decomposition by Local Branching

ELE539A: Optimization of Communication Systems Lecture 16: Pareto Optimization and Nonconvex Optimization

Convex Optimization Lecture 16

Solution Methods for Stochastic Programs

Advanced Linear Programming: The Exercises

15-780: LinearProgramming

Local Cuts and Two-Period Convex Hull Closures for Big-Bucket Lot-Sizing Problems

Partial Path Column Generation for the Elementary Shortest Path Problem with Resource Constraints

Logic-based Benders Decomposition

minimize x subject to (x 2)(x 4) u,

EE364a Review Session 5

Bilevel Integer Optimization: Theory and Algorithms

Benders Decomposition Methods for Structured Optimization, including Stochastic Optimization

Coordinate Descent and Ascent Methods

Dual methods and ADMM. Barnabas Poczos & Ryan Tibshirani Convex Optimization /36-725

The two-machine flowshop total completion time problem: A branch-and-bound based on network-flow formulation

Section Notes 9. Midterm 2 Review. Applied Math / Engineering Sciences 121. Week of December 3, 2018

Transcription:

Integer Programming Reformulations: Dantzig-Wolfe & Benders Decomposition the Coluna Software Platform François Vanderbeck B. Detienne, F. Clautiaux, R. Griset, T. Leite, G. Marques, V. Nesello, A. Pessoa, E. Uchoa, R. Sadykov, H. Sen, I. Tahiri, L. Wolsey ROADEF, February 2019 François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 1 / 54

Formulation Combinatorial Optimization Problem (CO) min{c(s) : s S} where S is the discrete set of feasible solutions. (7) (6) (5) S (4) François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 2 / 54

Formulation Combinatorial Optimization Problem (CO) min{c(s) : s S} where S is the discrete set of feasible solutions. Integer Programming Formulation A polyhedron P = {x n : Ax a} is a formulation for (CO) iff min{c(s) : s S} min{cx : x P I = P n }. x2 x2 (7) 5 5 (6) (5) 3 (7) 3 (7) (6) (5) (6) (5) S (4) 2 (4) 2 (4) 1 1 1 2 3 4 x1 1 2 3 4 x1 François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 2 / 54

Alternative formulations A formulation is typically not unique P and P can be alternative formulations for (CO) if (CO) min{cx : x P n } min{c x : x P n } x2 5 3 (7) (6) (5) 2 1 (4) 1 2 3 4 x1 More generally: can expressed in different variable-spaces. François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 3 / 54

Quality of Formulations Stronger formulation (in the same space) Formulation P n is a stronger than P n if P P. Then, min{cx : x P } min{cx : x P} x2 5 3 (7) (6) (5) 2 1 (4) 1 2 3 4 x1 François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 4 / 54

Extended Formulation Given an initial compact formulation: x2 5 3 (7) (6) (5) 2 1 (4) 1 2 3 4 x1 François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 5 / 54

Extended Formulation w 5 3 5 x2 (5) 2 3 1 2 (4) 1 1 2 3 4 x1 François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 6 / 54

Extended Formulation w 5 3 5 x2 (5) 2 3 1 2 (4) 1 1 2 3 4 x1 François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 7 / 54

Extended Formulation w 5 3 5 x2 (5) 2 3 1 2 (4) 1 1 2 3 4 x1 François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 8 / 54

Projection The Projection of Q = {(x, w) n+e : Gx + Hw d} on the x-space is: proj x (Q) := {x n : w e such that(x, w) Q}. w 5 3 5 x2 (5) 2 3 1 2 (4) 1 1 2 3 4 x1 François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 9 / 54

Take away messages Integer Programming Reformulations A powerful way to exploit the combinatorial structure. Benders & Dantzig Wolfe decomposition are generic schemes to derive & handle reformulations Size can be coped with using dynamic generation: a small % of variables and constraints are needed; hence it scales up to real-life applications. With efficiency enhancement features, these approaches can be highly competitive Implementation can be generic, with tools such as Coluna.jl François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 10 / 54

Outline 1 Sub-gradient linearization : Dantzig & Benders 2 Solution Methods 3 Efficient Implementation 4 Integer Solutions 5 Coluna.jl François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 11 / 54

Subgradient linearization of a convex function f (x) max f(x) g(x) 0 x 0 where f (.) and/or g(.) are defined by a bundle of linear pieces that are generated dynamically during the optimization process. x François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 12 / 54

Subgradient linearization of convex functions η max η η f(x) g(x) 0 x 0 where f (.) and/or g(.) are defined by a bundle of linear pieces that are generated dynamically during the optimization process. x François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 13 / 54

Subgradient linearization of convex functions η max η η f(x) g(x) 0 x 0 where f (.) and/or g(.) are defined by a bundle of linear pieces that are generated dynamically during the optimization process. x François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 13 / 54

Subgradient linearization of convex functions η max η η f(x) g(x) 0 x 0 where f (.) and/or g(.) are defined by a bundle of linear pieces that are generated dynamically during the optimization process. x François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 13 / 54

Subgradient linearization of convex functions η max η η f(x) g(x) 0 x 0 where f (.) and/or g(.) are defined by a bundle of linear pieces that are generated dynamically during the optimization process. x François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 13 / 54

Subgradient linearization of convex functions η max η η f(x) g(x) 0 x 0 where f (.) and/or g(.) are defined by a bundle of linear pieces that are generated dynamically during the optimization process. x François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 13 / 54

Subgradient linearization of convex functions η max η η f(x) g(x) 0 x 0 where f (.) and/or g(.) are defined by a bundle of linear pieces that are generated dynamically during the optimization process. Separation: Evaluate the true fonction f at the current point x yields a proof of optimality or a violated constraint. x François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 13 / 54

Outline 1 Sub-gradient linearization : Dantzig & Benders 2 Solution Methods 3 Efficient Implementation 4 Integer Solutions 5 Coluna.jl François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 14 / 54

Dantzig-Wolfe Decomposition Assume a bounded integer integer problem with structure: [F] min c x : A x a x X = { B x b x n } François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 15 / 54

Dantzig-Wolfe Decomposition Assume a bounded integer integer problem with structure: [F] min c x : A x a x X = { B x b where B is typically block diagonal x n } [F B ] min c 1 x 1 + c 2 x 2 +... + c K x K A 1 x 1 + A 2 x 2 +... + A K x K a B 1 x 1 b 1 B 2 x 2 b 2...... B K x K x 1 p 1 x 2 p 2..., x K p K b K François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 15 / 54

Dantzig-Wolfe Decomposition Assume a bounded integer integer problem with structure: [F] min c x : A x a x X = { B x b x n } Assume that subproblem [SP] min{c x : x X} (1) is relatively easy to solve compared to problem [F]. Then, X = {x q } q Q François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 15 / 54

Dantzig-Wolfe Decomposition Assume a bounded integer integer problem with structure: [F] min c x : A x a x X = { B x b x n } Assume that subproblem [SP] min{c x : x X} (1) is relatively easy to solve compared to problem [F]. Then, X = {x q } q Q conv(x) = {x n + : x = x q λ q, λ q = 1, λ q 0 q Q} q Q q Q François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 15 / 54

Dantzig-Wolfe Decomposition Assume a bounded integer integer problem with structure: [F] min c x : A x a x X = { B x b x n } Assume that subproblem [SP] min{c x : x X} (1) is relatively easy to solve compared to problem [F]. Then, X = {x q } q Q conv(x) = {x n + : x = x q λ q, λ q = 1, λ q 0 q Q} q Q q Q [F] min{ cx q λ q : Ax q λ q a, λ q = 1, x q λ q n } q Q q Q q Q q François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 15 / 54

The Cutting Stock Problem (CSP) 7 x 656 688 waste : 0 min 432 432 480 2 x waste : 0 422 352 568 2 x waste : 2 420 572 352 2 x waste : 0 8 x 672 672 waste : 0 4 x 688 608 waste : 48 5 x 722 568 waste : 54 4 x 560 560 waste : 224 8 x 722 572 waste : 50 6 x 692 572 waste : 80 560 432 352 4 x waste : 0 s.t. K k=1 y k K x ik d i i k=1 w i x ik W y k k i x ik i, k y k {0, 1} k 8 x 576 608 waste : 160 432 400 488 3 x waste : 24 488 488 352 4 x waste : 16 576 400 352 5 x waste : 16 14 x 692 608 waste : 44 2 x 576 400 waste : 368 Number of distinct cutting patterns : 17 Number of rolls : 88 Total waste : 5090 min q λ q x q i λ q d i i = 1,, n q λ q 0 q François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 16 / 54

The Vehicle Routing Problem (VRP) min s.t. c e x r e e,r e δ(i),r x r e 2 i x r X r r 6, 4, 3 min e,r,q Q r c e x q e λ q e δ(i),r,q Q r x q e λ q 2 i q Q r λ q U r r λ q 0 r, q Q r François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 17 / 54

Lagrangian Duality [F] = min {c x : A x a; B x b; x n } L(π) = min q Q {c xq + π (a Ax q )} [LD] = max L(π) π m + η η L(π) π o π π François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 18 / 54

The Lagrangian Dual as an LP η [LD] max min {π a + (c π m + q Q πa)xq }; max{η, η cx q + π(a Ax q ) q Q, π m +, η 1 }; η L(π) π L(π) π François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 19 / 54

The Lagrangian Dual as an LP η [LD] max min {π a + (c π m + q Q πa)xq }; max{η, η cx q + π(a Ax q ) q Q, π m +, η 1 }; min{ (cx q )λ q, q Q (Ax q )λ q a, q Q λ q = 1, q Q λ q 0 q Q}; η L(π) L(π) François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 19 / 54 π π

Cut generation in the dual L(u) x2 x1 L(u) x2 x1 x3 x3 x4 x4 x5 x5 u u L(u) x2 x1 L(u) x2 x1 x3 x3 x4 x4 x5 x5 u u François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 20 / 54

Column generation in the primal Xlp Xlp Xlp Xlp François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 21 / 54

Properties Restricted Master: DUAL DM t max L t () where L t (π) = min q Q t{πa + (c πa)x q } PRIMAL PM t min{cx : Ax a, x conv({x q } q Q t)}. Valid upper bound: UB η L(π) (π t, η t ) π François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 22 / 54

Properties Restricted Master: DUAL DM t max L t () where L t (π) = min q Q t{πa + (c πa)x q } PRIMAL PM t min{cx : Ax a, x conv({x q } q Q t)}. Valid upper bound: UB η L(π) (π t, η t ) SubProb: Computing L(π t ) yields a 1 most violated constr. for DM t 2 most neg. red. cost col. for PM t 3 a correction on L t (.) at point π t 4 a sub-gradient g t = (a A x t ) of L(.) π Valid lower bound: LB François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 22 / 54

The benefits of the approach Better Dual Bound: Lagrangian duality Mastering the size of the formulation: projection w 5 3 5 x2 (5) 2 3 1 2 (4) 1 1 2 3 4 x1 François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 23 / 54

The benefits of the approach Better Dual Bound: Lagrangian duality Mastering the size of the formulation: projection Divide and conquer & Allow to Exploit problem structure complexity size + +... François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 23 / 54

The benefits of the approach Better Dual Bound: Lagrangian duality Mastering the size of the formulation: projection Divide and conquer & Allow to Exploit problem structure Avoid symmetry between identical blocks & Allow a compact representation of a solution min cx 1 + c x 2 +... + c x K A x 1 + A x 2 +... + A x K a B x 1 b 1 B x 2 b 2...... B x K x 1 p 1 x 2 p 2..., x K p K If identical subproblems, solution x q is taken λ q times b K François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 23 / 54

Dual Decompositions On a subset of constraints (Dantzig-Wolfe): multi-resources min c 1 x 1 + c 2 x 2 +... + c K x K A x 1 + A x 2 +... + A x K a B 1 x 1 b 1 B 2 x 2 b 2...... B K x K b K On a subset of variables (Benders): multi-decision-levels min a x + b 1 y 1 + b 2 y 2 +... + b K y K A x + B 1 y 1 c 1 A x + B 2 y 2 c 2 A x +...... A x + B K y K c K François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 24 / 54

Capacitated Facility Location P fx + Pi i i s.t. y Pj ij i yij yij xi min François Vanderbeck (U. de Bordeaux & Atoptima) P i,j cij yij si xi i dj j 0 i, j {0, 1}, i Integer Programming Reformulations 25 / 54

Multi Commodity Network Design min i,j f ijx ij + i,j,k c ijky ijk s.t. k y ijk u ij x ij i, j j y ijk j y jik = b ik i, k y ijk 0 i, j, k x ij {0, 1} i, j François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 26 / 54

Benders Decomposition max a x + b 1 y 1 + b 2 y 2 +... + b K y K A x + B 1 y 1 c 1 A x + B 2 y 2 c 2 A x +...... A x + B K y K c K max { a x + b y : A x + B y c, x n, y p + }, François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 27 / 54

Benders Decomposition max { a x + b y : A x + B y c, φ(x) := max {b y : B y c A x, x n, y p + }, y p + } First stage: max x,η {η : η a x + φ(x), x n, η }, François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 28 / 54

Benders Decomposition max { a x + b y : A x + B y c, φ(x) := max {b y : B y c A x, x n, y p + }, y p + } First stage: max x,η {η : η a x + φ(x), x n, η }, Second stage: min ν 0? b y + ν (η a x) B y + 1 ν (A x c) (y, ν) p+1 + François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 28 / 54

Benders Decomposition max { a x + b y : A x + B y c, φ(x) := max {b y : B y c A x, x n, y p + }, y p + } First stage: max x,η {η : η a x + φ(x), x n, η }, u q 0 (η a x) + uq (A x c) 0 q Second stage: min ν 0? b y + ν (η a x) B y + 1 ν (A x c) (y, ν) p+1 + max u 0 (η a x) + u (A x c) 0? u 0 b u B 0 u 0 + 1x 1 (u 0, u) 1+m 2 + {u 1,..., u T } are its extreme dual solutions. François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 28 / 54

Benders Decomposition max { a x + b y : A x + B y c, φ(x) := max {b y : B y c A x, x n, y p + }, y p + } First stage: Relaxed Master UB max x,η {η : η a x + φ(x), x n, η }, Second stage: min ν 0? b y + ν (η a x) B y + 1 ν (A x c) (y, ν) p+1 + u q 0 (η a x) + uq (A x c) 0 q Sub-Problem Lagrangian LB max u 0 (η a x) + u (A x c) 0? u 0 b u B 0 u 0 + 1x 1 (u 0, u) 1+m 2 + {u 1,..., u T } are its extreme dual solutions. François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 28 / 54

Benders Relaxed Master PRIMAL: [M t ] UB max x,η {η : x q 0 η + (xq A x q 0 a) x cxq q Q t x n +, η }, (x t, η t ) DUAL: [DM t ] min{ (c x q ) λ q : q Q t (x q A x q 0 a) λ q 0, q Q t x q 0 λ q = 1, q λ q 0 q Q t }. (π t, σ t ) = q (uq, u q 0 )λ q SP: L(x, η) = η + min q {c u q (u q A u q 0 a) x uq 0 η} u, LB François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 29 / 54

Outline 1 Sub-gradient linearization : Dantzig & Benders 2 Solution Methods 3 Efficient Implementation 4 Integer Solutions 5 Coluna.jl François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 30 / 54

Method s Classification 1 Ascent: max π {L(π)} Subgradient Volume Conjugate SubGradient Subgradient: π t+1 = (π t + s t η L(π) g t g t )+ + simple unconstraint optimization + low memory use - myoptic : use local info. - slow convergence tail π François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 31 / 54

Method s Classification 1 Ascent: Subgradient Volume Conjugate SubGradient f (x) 2 Polyhedral - Active Set: Kelley Bundle x + use global info + restricted formulation - combinatorics of active set - oscillation bt extreme sol. François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 31 / 54

Method s Classification 1 Ascent: Subgradient Volume Conjugate SubGradient 2 Polyhedral - Active Set: Kelley Bundle 3 Polyhedral - Interior Point: Analytic Center Cutting Plane Primal Dual Col Gen + nb iterations less sensitive to size - full size solution François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 31 / 54

Comparing Methods Best Methods? Ascent methods are computationally light but only allow a rough optimization (good for: warm start, huge problems [Nesterov-14]) Active set methods are best when combinatoric is low Extreme solutions are good to yield integer solutions Otherwise, Interior Point approaches (ACCPM, PDCGM) take over Quality of the implementation François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 32 / 54

Outline 1 Sub-gradient linearization: Dantzig & Benders 2 Solution Methods 3 Efficient Implementation 4 Integer Solutions 5 Coluna.jl François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 33 / 54

Convergence Issue {π t } t π Π Oscillations: π t (bang-bang), {L(π t )} (yo-yo) and π t π > π t 1 π Tailing-off effect: cuts co-linear with objective (inherent to cutting plane proc.) step size very small. Multiple optima 844 760 675 591 506 422 338 253 169 François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 34 / 54

Stabilization - Acceleration Techniques 1 Proximality of a stability center (to avoid oscillations): Volume: π t = ( ˆπ + s t dt d t )+. Bundle: π t = argmax π {πa + σ 1 2θ π ˆπ 2 } ( ˆπ, ˆL) L(π) François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 35 / 54

Stabilization - Acceleration Techniques 1 Proximality of a stability center (to avoid oscillations): 2 Central Point Cutting (to avoid tailing and degeneracy): Optimal face center Chebyshev center Analytic center (of trust polyhedra) Interior Point LP-Solver η = η t ( ˆπ, ˆL) (π t, η t ) η ˆL η ˆL ( ˆπ, ˆL) François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 35 / 54

Stabilization - Acceleration Techniques 1 Proximality of a stability center (to avoid oscillations): 2 Central Point Cutting (to avoid tailing and degeneracy): 3 Smoothing (to avoid oscillations): Primal Smoothing Volume: d t = α d t 1 + (1 α) g t Dual Smoothing (also contributes to centralize) [Neame-99] π t = α π t 1 + (1 α) π t [Wentges-97] π t = α ˆπ + (1 α) π t G x y0 G x(a)=x a G OUT SEP d y2 y1 IN f(x)=c1 f(x)=c3<c2 f(x)=c2<c1 François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 35 / 54

Outline 1 Sub-gradient linearization: Dantzig & Benders 2 Solution Methods 3 Efficient Implementation 4 Integer Solutions 5 Coluna.jl François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 36 / 54

Integrality enforcement in Branch-and-Price w 5 3 5 x2 (5) 2 3 1 2 (4) 1 1 2 3 4 x1 François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 37 / 54

Integrality enforcement in Branch-and-Price w 5 3 5 x2 (5) 2 3 1 2 (4) 1 1 2 3 4 x1 François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 38 / 54

Integrality enforcement in Branch-and-Price Assume a bounded integer integer problem with structure: Assume that subproblem [F] min c x : A x a x Z = { B x b x n } [SP] min{c x : x Z} (2) with conv(z) = {x n + : x = x q λ q, λ q = 1, λ q 0 q Q} q Q q Q The reformulation of [F] is : min{ cx q λ q : Ax q λ q a, λ q = 1, q Q q Q q Q q xq λ q n } François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 39 / 54

Integrality enforcement in Branch-and-Price Assume a bounded integer integer problem with structure: Assume that subproblem [F] min c x : A x a x Z = { B x b x n } [SP] min{c x : x Z} (2) with conv(z) = {x n + : x = x q λ q, λ q = 1, λ q 0 q Q} q Q q Q The reformulation of [F] is : min{ cx q λ q : Ax q λ q a, λ q = 1, q Q But in the case of identical sub-problems... q Q q Q q xq λ q n } François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 39 / 54

Integrality enforcement in B&P: identical SP A Mapping that partitions the aggregate SP solutions: λ 1 = 1 2 λ 2 = 1 2 λ 3 = 1 2 λ 4 = 1 2 4 1 1 1 4 1 2 2 2 2 1 1 1 1 x 1 = x 2 = 4 1 1 2 2 1 1 François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 40 / 54

Integrality enforcement in B&P: identical SP A Mapping that partitions the aggregate SP solutions: 1 2 1 1 2 1 2 λ q 1 2 1 1 x 1 1 1 1 1 0 0 0 x 2 1 1 0 0 1 1 0 x 3 0 0 1 0 1 0 1 x 4 1 0 1 0 1 1 0 X 1 = {x X : x 1 1} and X 12 = {x X : x 1 1, x 2 1} Branching: [V, MP 2009] q X 12 λ q 2 or q (X 1 \X 12 ) λ q 2 or q (X\X 1 ) λ q 3 dynamically increases the # of SP, up to K non-empty nested SP François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 41 / 54

Integer Benders Algorithm (text-book) Oracle Multi-cuts: separation for each integer solution of the pool François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 42 / 54

Benders using Lazy Constraints Oracle Oracle Oracle Oracle Oracle François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 43 / 54

Benders using Cut-and-Branch Oracle Oracle Oracle Oracle Oracle Oracle François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 44 / 54

Benders using Branch-and-Cut Oracle Oracle Oracle Oracle Oracle Oracle Oracle Oracle Oracle François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 45 / 54

Outline 1 Sub-gradient linearization: Dantzig & Benders 2 Solution Methods 3 Efficient Implementation 4 Integer Solutions 5 Coluna.jl François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 46 / 54

Software What: Uses: Coluna.jl a Column-and-Row generation code Open-Source: MPLicence in Julia 1.0 Extending JuMP / JuliaOpt and BlockDecomposition.jl Dantzig-Wolfe & Benders decomposition Robust & Stochastic Optimization Machine Learning First release: Jan. 19 Support: Math Optimization Society (MOS) & JuliaOpt URL https://github.com/atoptima/coluna.jl François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 47 / 54

Coluna Environment François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 48 / 54

Cutting Stock Problem: identical subproblems François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 49 / 54

Vehicle Routing Problem: Q-route subproblem oracle François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 50 / 54

Vehicle Routing Problem: Cut & Pricing oracles François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 51 / 54

Production Problem: combining Dantzig & Benders François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 52 / 54

Coluna s Future Present Modeling JuMP Extension Automated Reformul. Algorithms Column Generation Pricing Call-backs Branch-and-Price Preprocessing Primal Heuristics Applications RCSP Oracle VRP Application (MIT L) François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 53 / 54

Coluna s Future Present Coming Soon Modeling JuMP Extension Automated Reformul. Algorithms Column Generation Pricing Call-backs Branch-and-Price Preprocessing Primal Heuristics Applications RCSP Oracle VRP Application (MIT L) Efficiency Stabilization Strong Branching Further Primal Heuristics Pricing Strategies Parallelisation Cutomization Cutting Callback Branching Callback Functionality Ad-hoc Branching Benders Decomposition Nested Decomposition François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 53 / 54

Coluna & the community Welcome β-users Integration Algorithms: Subgradient, Interior Point, Primal Heuristics Oracles: Pricing, Separation Code review Co-development François Vanderbeck (U. de Bordeaux & Atoptima) Integer Programming Reformulations 54 / 54