Advanced Mixed Integer Programming (MIP) Formulation Techniques

Transcription

1 Advanced Mixed Integer Programming (MIP) Formulation Techniques Juan Pablo Vielma Massachusetts Institute of Technology Center for Nonlinear Studies, Los Alamos National Laboratory. Los Alamos, New Mexico, August, 2016.

2 Traveling Salesman Problem (TSP): Visit Cities Fast Advanced MIP Formulations 1 / 37

3 MIP = Avoid Enumeration Number of tours for 49 cities Fastest supercomputer Assuming one floating point operation per tour: > years times the age of the universe! How long does it take on an iphone? Less than a second! 4 iterations of cutting plane method! = 48!/ flops Dantzig, Fulkerson and Johnson 1954 did it by hand! For more info see tutorial in ConcordeTSP app Cutting planes are the key for effectively solving (even NPhard) MIP problems in practice. Advanced MIP Formulations 2 / 37

4 50+ Years of MIP = Significant Solver Speedups Algorithmic Improvements (Machine Independent): CPLEX v1.2 (1991) v11 (2007): 29,000x speedup Gurobi v1 (2009) v6.5 (2015): 48.7x speedup Commercial, but free for academic use (Reasonably) effective free / open source solvers: GLPK, CBC and SCIP (free only for non-commercial) Easy to use, fast and versatile modeling languages Julia based JuMP modelling language Linear MIP solvers very mature and effective: Convex nonlinear MIP getting there (quadratic nearly there) Advanced MIP Formulations 3 / 37

5 What do YOU need to do to use MIP? 1. Use JuMP 2. Construct a MIP formulation of your problem This talk: From non-convex constraints to linear MIP formulations One illustrative example Beyond linear MIP: Convex nonlinear MIP See Miles talk on Thursday Advanced MIP Formulations 4 / 37

6 Example: Experimental Design in Marketing Think Simulation-Based Optimization

7 (Custom) Product Recommendations via CBCA Feature SX530 RX100 Zoom 50x 3.6x Prize $ $ Weight ounces 7.5 ounces Prefer Feature TG-4 G9 Waterproof Yes No Prize $ $ Weight 7.36 lb 7.5 lb Prefer We recommend: Feature TG-4 Galaxy 2 Waterproof Yes No Prize $ $ Viewfinder Electronic Optical Prefer Advanced MIP Formulations 6 / 37

8 Towards Optimal Product Recommendation Find enough information about preferences to recommend Feature SX530 RX100 Zoom 50x 3.6x Prize $ $ Weight ounces 7.5 ounces Prefer Feature TG-4 G9 Waterproof Yes No Prize $ $ Weight 7.36 lb 7.5 lb Prefer We recommend: Feature TG-4 Galaxy 2 Waterproof Yes No Prize $ $ Viewfinder Electronic Optical Prefer How do I pick the next (1 st ) question to obtain the largest reduction of uncertainty or variance on preferences Advanced MIP Formulations 7 / 37

9 Choice-based Conjoint Analysis Feature Chewbacca BB-8 Wookiee Yes No Droid No Yes Blaster Yes No 0 1 1A = x 2 0 I would buy toy Product Profile x 1 x 2 Advanced MIP Formulations 8 / 37

10 MNL Preference Model Utilities for 2 products, n features (e.g. n = 12) U 1 = x = X n i=1 i x 1 i + 1 U 2 = x = X n i=1 i x 2 i + 2 part-worths product profile noise (gumbel) Utility maximizing customer: x 1 x 2, U 1 U 2 Noise can result in response error: L x 1 x 2 = P x 1 x 2 = e x1 e x1 + e x2 Advanced MIP Formulations 9 / 37

11 Next Question To Reduce Variance : Bayesian Prior Distribution of Feature SX530 RX100 Zoom 50x 3.6x Prize $ $ Weight ounces 7.5 ounces Prefer Bayesian Update Posterior Distribution Feature TG-4 Galaxy 2 Waterproof Yes No Prize $ $ MCMC Bayesian Update Posterior Distribution Viewfinder Electronic Optical 0.10 Prefer Black-box objective: Question Selection = Enumeration Question selection by Mixed Integer Programming (MIP) Advanced MIP Formulations 10 / 37

12 Bayesian Update and Geometric Updates Prior distribution Answer likelihood Posterior distribution N(µ, ) x 1 x 2 f x 1 x 2 ( ; µ, ) f x 1 x 2 = R L x 1 x 2 Multidimension al Integration? ( ; µ, ) L x 1 x 2 R ( ; µ, ) L ( x1 x 2 ) d non-convex on x 1,x 2 2 {0, 1} n Advanced MIP Formulations 11 / 37

13 D-Efficiency and Posterior Covariance Matrix x 1 x 2 N(µ, ) x 1 x 2 cov( )= 1 Variance = D-Efficiency: Non-convex function f x 1,x 2 := E,x 1 / x 2 det( i ) 1/p Even evaluating expected D-Efficiency for a question requires multidimensional integration cov( )= 2 Advanced MIP Formulations 12 / 37

14 Standard Question Selection Criteria ( µ) 0 1 ( µ) apple r Choice balance: Minimize distance to center µ x 1 x 2 µ Postchoice symmetry: Maximize variance of question x 1 x 2 0 x 1 x 2 Advanced MIP Formulations 13 / 37

15 D-efficiency: Balance Question Trade-off D-efficiency = Non-convex function distance: d := µ x 1 x 2 f(d, v) variance: v := x 1 x 2 0 x 1 x 2 of Can evaluate f(d, v) with 1-dim integral Advanced MIP Formulations 14 / 37

16 Optimization Model min f(d, v) s.t. µ x 1 x 2 = d x 1 x 2 0 x 1 x 2 = v A 1 x 1 + A 2 x 2 apple b x 1 6= x 2 x 1,x 2 2 {0, 1} n Advanced MIP Formulations 15 / 37

17 Technique 1: Binary Quadratic x 1,x 2 2 {0, 1} n x 1 x 2 0 x 1 x 2 = v X l i,j apple x l i, X l i,j apple x l j, X l i,j x l i + x l j 1, X l i,j 0 apple X l i,j = x l i x l j (l 2 {1, 2}, i,j 2 {1,...,n}) : W i,j = x 1 i x 2 j : W i,j apple x 1 i, W i,j apple x 2 j, W i,j x 1 i + x 2 j 1, W i,j 0 nx i,j=1 X 1 i,j + X 2 i,j W i,j W j,i i,j = v Advanced MIP Formulations 16 / 37

18 Technique 1: Binary Quadratic x 1,x 2 2 {0, 1} n x 1 6= x 2, kx 1 x 2 k Xi,j l = x l i x l j (l 2 {1, 2}, i,j 2 {1,...,n}) : Xi,j l apple x l i, Xi,j l apple x l j, Xi,j l x l i + x l j 1, Xi,j l 0 W i,j = x 1 i x 2 j : W i,j apple x 1 i, W i,j apple x 2 j, W i,j x 1 i + x 2 j 1, W i,j 0 nx i,j=1 X 1 i,j + X 2 i,j W i,j W j,i 1 Advanced MIP Formulations 17 / 37

19 Technique 2: Piecewise Linear Functions D-efficiency = Non-convex function distance: d := µ x 1 x 2 f(d, v) variance: v := x 1 x 2 0 x 1 x 2 of Can evaluate f(d, v) with 1-dim integral Piecewise Linear Interpolation MIP formulation Advanced MIP Formulations 18 / 37

20 f(d 3 ) f(d 2 ) f(d 5 ) f(d 1 ) f(d 4 ) Simple Formulation for Univariate Functions z = f(x) d 1 d 2 d 3 d 4 d 5 Size = O (# of segments) Non-Ideal: Fractional Extreme Points x z = X 5 1= X 5 j=1 j=1 dj f(d j ) j j, j 0 Advanced MIP Formulations 19 / 37

21 f(d 3 ) f(d 2 ) f(d 5 ) f(d 1 ) f(d 4 ) Advanced Formulation for Univariate Functions z = f(x) d 1 d 2 d 3 d 4 d 5 Size = O (log 2 # of segments) Ideal: Integral Extreme Points x z = X 5 1= X 5 j=1 j=1 y 2 {0, 1} 2 dj f(d j ) j j, j 0 Advanced MIP Formulations 20 / 37

22 Computational Performance Advanced formulations provide an computational advantage Advantage is significantly more important for free solvers State of the art commercial solvers can be significantly better that free solvers Still, free is free! Time [s] Time [s] Simple CPLEX GLPK Advanced Simple Advanced Advanced MIP Formulations 21 / 37

23 Formulation Improvements can be Significant GUROBI Simple Advanced MIP Formulations 22 / 37

24 Constructing Advanced Formulations

25 Abstracting Univariate Functions P z = f(x) = X i := 2 5 : j =0 8j 5/2 T i z T j=1 i := {i, i +1} i 2 {1,...,4} f(d 3 ) f(d 2 ) 5 = f(d 5 ) f(d 1 ) f(d 4 ) d 1 d 2 d 3 d 4 d 5 T x 1= X 5 j=1 dj f(d j ) j j, j 0 [ 4 2 P i 5 i=1 Advanced MIP Formulations 24 / 37

26 Abstraction Works for Multivariate Functions P i := { 2 m : j =0 8v j /2 T i } f(x,y) v m y x 2 [ n i=1 P i m Advanced MIP Formulations 25 / 37

27 Complete Abstraction V := n 2 R V + : X v2v o v =1, P i = 2 V : v =0 8v /2 T i T i = cliques of a graph Advanced MIP Formulations 26 / 37

28 From Cliques to (Complement) Conflict Graph SOS Advanced MIP Formulations 27 / 37

29 From Conflict Graph to Bi-clique Cover SOS Advanced MIP Formulations 28 / 37

30 From Bi-clique Cover to Formulation SOS2 0 apple apple 1 y 1 0 apple 3 apple y 1 0 apple apple 1 y 2 0 apple apple y Advanced MIP Formulations 29 / 37

31 Ideal Formulation from Bi-clique Cover Conflict Graph G =(V,E) E = {(u, v) :u, v 2 V, u 6= s.t. u, v 2 T i } Bi-clique cover A j, B j t j=1, Aj, B j V 8{u, v} 2 E 9j s.t. u 2 A j ^ v 2 B j Formulation X v2a j v apple 1 y j 8j 2 [t] X v2b j v apple y j 8j 2 [t] y 2 {0, 1} t Advanced MIP Formulations 30 / 37

32 Recursive Construction of Cover for SOS2, Step 1 Base case n=2 1 : Step 1 recursion : Reflect Graph / Cover Stick Graph / Cover Repeat for all bi-cliques from 2 k-1 to cover all edges within first and last half of conflict graph Advanced MIP Formulations 31 / 37

33 Recursive Construction of Cover for SOS2, Step 2 Only edges missing are those between first and last half of conflict graph Step 2 : Add one more bi-clique 1 2 n/2 n/2 n/2 + n n Cover has log 2 n bi-cliques. For non-power of two just delete extra nodes. Advanced MIP Formulations 32 / 37

34 Grid Triangulations: Step 1 = SOS2 for Inter-Box Covers all arcs between boxes Advanced MIP Formulations 33 / 37

35 Grid Triangulations: Step 2 = Ad-hoc Intra-Box Covers all arcs within boxes Sometimes 1 additional cover Advanced MIP Formulations 34 / 37

36 Grid Triangulations: Step 2 = Ad-hoc Intra-Box Sometimes 2 additional covers Sometimes more, but always less than 9 Simple rules to get (near) optimal in Fall 16 Advanced MIP Formulations 35 / 37

37 Summary and Main Messages Always choose Chewbacca! MIP can solve very challenging problems in practice Commercial solvers best, but free solvers reasonable Both easily accessible and integrated into complex systems through the JuMP Advanced formulations yield important speed-ups and are (relatively) easy to learn Advanced MIP Formulations 36 / 37

38 More Information JuMP: Ask Miles and MIP Formulations: Mixed integer linear programming formulation techniques. V. SIAM Review 57, pp Advanced Formulation: Small independent branching formulations for unions of V- polyhedra. Joey Huchette and V arxiv: Marketing Application: Ellipsoidal methods for adaptive choice-based conjoint analysis. Denis Saure and V Advanced MIP Formulations 37 / 37