Mixed Integer Programming (MIP) for Causal Inference and Beyond

Transcription

1 Mixed Integer Programming (MIP) for Causal Inference and Beyond Juan Pablo Vielma Massachusetts Institute of Technology Columbia Business School New York, NY, October, 2016.

2 Traveling Salesman Problem (TSP): Visit Cities Fast

3 Traveling Salesman Problem (TSP): Visit Cities Fast Paradoxes, Contradictions, and the Limits of Science Many research results define boundaries of what cannot be known, predicted, or described. Classifying these limitations shows us the structure of science and reason. Noson S. Yanofsky S A computer would have to check all these possible routes to find the shortest one.

4 MIP = Avoid Enumeration Number of tours for 49 cities Fastest supercomputer flops Assuming one floating point operation per tour: > years times the age of the universe! How long does it take on an iphone? Less than a second! 4 iterations of cutting plane method! = 48!/ Dantzig, Fulkerson and Johnson 1954 did it by hand! For more info see tutorial in ConcordeTSP app Cutting planes are the key for effectively solving (even NPhard) MIP problems in practice.

5 Using IP to visit Germany 45 cities(1832) 120 cities(1977) 15,112 cities(2004)

6 50+ Years of MIP = Significant Solver Speedups Algorithmic Improvements (Machine Independent): CPLE v1.2 (1991) v11 (2007): 29,000x speedup Gurobi v1 (2009) v6.5 (2015): 48.7x speedup Commercial, but free for academic use (Reasonably) effective free / open source solvers: GLPK, CBC and SCIP (free only for non-commercial) Easy to use, fast and versatile modeling languages Julia based JuMP modelling language Linear MIP solvers very mature and effective: Convex nonlinear MIP getting there (quadratic nearly there)

7 Matching Treated Units: T = {t 1,...,t T } Control Units: C = {c 1,...,c C } Observed Covariates: P = {p 1,...,p P } Covariate Values: x t = x t p p2p, x c = x c p p2p, t 2 T c 2 C

8 Maximum Cardinality Exact Matching max s.t. t2t c2c m t,c m t,c apple 1, 8c 2 C t2t m t,c apple 1, c2c 8t 2 T m t,c =0 8t, c x t 6= x c 0 apple m t,c apple 1 m t,c 2 {0, 1} 8t 2 T, c2 C. Solve time for truncated Lalonde CPS 429 = s Why? Can solve relaxation.

9 Maximum Cardinality Marginal Means t2t max c2c m t,c s.t. m t,c apple 1, t2t m t,c apple 1, c2c t2t 8c 2 C 8t 2 T x c pm t,c 8p 2 P x t pm t,c = c2c c2c t2t m t,c 2 {0, 1} 8t 2 T, c2 C. Solve time for truncated Lalonde CPS 429 = 444 s Why? One reason = relaxation has fractions.

10 Maximum Cardinality Fine Balance, Take 1 max s.t. t2t c2c m t,c t2t mp t,c apple 1, 8c 2 C, p 2 P c2c mp t,c apple 1, 8t 2 T, p 2 P m p t,c =0 8t, c x t p 6= x c p, p 2 P t2t mp t,c = t2t mq t,c c2c mp t,c = c2c mq t,c 8c 2 C, p, q 2 P 8t 2 T, p, q 2 P m p t,c 2 {0, 1} 8t 2 T,c2 C, p 2 P. Solve time for truncated Lalonde CPS 429 = 0.81 s Why? One reason = relaxation has fewer fractions.

11 Basic Branch-and-Bound x (1) FRAC z = 1.71 x 1 apple 0 x (2) INT z = 0 x 2 2 (3) FRAC z = 1.5 x 1 apple x 1 3x 1 +2x 2 apple 6 2x 1 + x 2 apple 0 x 1,x 2 0 (4) INT z = 1 Linear Programming (LP) Relaxation (5) INF x 1,x 2 2 Z

12 Stronger Formulations = Faster Solves? x x (1) FRAC z = 1.71 x 1 apple 0 x (2) INT z = 0 (3) INT z = x x 1 3x 1 +2x 2 apple 6 2x 1 + x 2 apple 0 x 1,x 2 0 x 1,x 2 2 Z x 1 +x 2 apple 2 2x 1 +x 2 apple 0 x 1,x 2 0 x 1,x 2 2 Z

13 Cutting Plane Example: Chátal-Gomory Cuts ( ) P := x 2 R 2 x 1 + x 2 apple 3, : 5x 1 3x 2 apple 3 3 H := x 2 R 2 :4x 1 +3x 2 apple 10.5 {z } 2 Z if x 2 Z Valid for H \ Z

14 Branch-and-Bound and Cuts (Branch-and-Cut) x 2 3 (1) FRAC z = (2) FRAC z = 1 x 1 apple 0 x x 1 3x 1 +2x 2 apple 6 (3) INT z = 0 (4) INT z = 1 2x 1 + x 2 apple 0 x 1,x 2 0 x 1,x 2 2 Z

15 Partial Solves and GAP x 2 3 (1) FRAC z = 1.71 x 1 apple 0 x (2) INT z = 0 (3) FRAC z = x 1 GAP =100 bestnode bestinteger bestinteger BB GAP = = BB + GAP = = 50%

16 Maximum Cardinality Marginal Means t2t max c2c m t,c s.t. m t,c apple 1, t2t m t,c apple 1, c2c t2t 8c 2 C 8t 2 T x c pm t,c 8p 2 P x t pm t,c = c2c c2c t2t m t,c 2 {0, 1} 8t 2 T, c2 C. Solve time for truncated Lalonde CPS 429 = 444 s Optimal = 17, LP Relaxation = 19.35

17 Maximum Cardinality Marginal Means Root relaxation: objective e+01, 1947 iterations, 0.16 seconds Nodes Current Node Objective Bounds Work Expl Unexpl Obj Depth IntInf Incumbent BestBd Gap It/Node Time Black Magic ( s s s s s H % s... * % s Cutting planes: Gomory: 1 Flow cover: 1 Explored nodes ( simplex iterations) in seconds Optimal solution found (tolerance 1.00e-04) Best objective e+01, best bound e+01, gap 0.0% Better Feasible Solution Optimal Solution Bound

18 Maximum Cardinality Fine Balance, Take 1 max s.t. t2t c2c m t,c t2t mp t,c apple 1, 8c 2 C, p 2 P c2c mp t,c apple 1, 8t 2 T, p 2 P m p t,c =0 8t, c x t p 6= x c p, p 2 P t2t mp t,c = t2t mq t,c c2c mp t,c = c2c mq t,c 8c 2 C, p, q 2 P 8t 2 T, p, q 2 P m p t,c 2 {0, 1} 8t 2 T,c2 C, p 2 P. Solve time for truncated Lalonde CPS 429 = 0.81 s Optimal = 10, LP Relaxation = 11

19 Maximum Cardinality Fine Balance, Take 1 Root relaxation: objective e+01, 1490 iterations, 0.03 seconds Nodes Current Node Objective Bounds Work Expl Unexpl Obj Depth IntInf Incumbent BestBd Gap It/Node Time s s s H % - 0s % - 0s Cutting planes: Zero half: 2 Explored 8 nodes (9933 simplex iterations) in 0.78 seconds Optimal solution found (tolerance 1.00e-04) Best objective e+01, best bound e+01, gap 0.0% Better Feasible Solution Optimal Solution Bound

20 Maximum Cardinality Fine Balance, Take 1 max s.t. t2t c2c m t,c t2t mp t,c apple 1, 8c 2 C, p 2 P c2c mp t,c apple 1, 8t 2 T, p 2 P m p t,c =0 8t, c x t p 6= x c p, p 2 P t2t mp t,c = t2t mq t,c c2c mp t,c = c2c mq t,c 8c 2 C, p, q 2 P 8t 2 T, p, q 2 P m p t,c 2 {0, 1} 8t 2 T,c2 C, p 2 P. Solve time for truncated Lalonde CPS 429 = 0.81 s Scalability? Size = T C P

21 Maximum Cardinality Fine Balance, Take 2 max s.t. t2t p,k t2t c2c m t,c m t,c apple 1, t2t m t,c apple 1, c2c c/2c p,k m t,c = t/2t p,k Solve time for truncated Lalonde CPS 429 = 0.61 s Scalability? Size = c2c p,k m t,c 8c 2 C 8t 2 T 8p 2 P,k 2 K(p) m t,c 2 {0, 1} 8t 2 T, c 2 C. T C + P K(p) ={x c p} c2p [ {x t p} t2t C p,k = {c 2 C : x c p = k} T p,k = {t 2 T : x t p = k}

22 Take 1 Revisited max s.t. t2t c2c m t,c t2t mp t,c apple 1, 8c 2 C, p 2 P c2c mp t,c apple 1, 8t 2 T, p 2 P m p t,c =0 8t, c x t p 6= x c p, p 2 P t2t mp t,c = t2t mq t,c c2c mp t,c = c2c mq t,c 8c 2 C, p, q 2 P 8t 2 T, p, q 2 P m p t,c 2 {0, 1} 8t 2 T,c2 C, p 2 P. Solve time for truncated Lalonde CPS 429 = 0.81 s Scalability? Size = T C P

23 Take 1 Revisited max s.t. t2t x t m p t,c = y c, 8p 2 P, k 2 K(p), c 2 C p,k t2t p,k m p t,c = x t, 8S 2 F, k 2 K(p), t 2 T p,k c2c p,k m p t,c 2 {0, 1} 8p 2 P, k 2 K(p), t 2 T p,k, c 2 C p,k x t 2 {0, 1} 8t 2 T y c 2 {0, 1} 8c 2 C.

24 Maximum Cardinality Fine Balance, Take 3 max s.t. t2t t2t x t t2t p,k x t = x t = c2c y c, Solve time for truncated Lalonde CPS 429 = s Scalability? Size = c2c p,k y c, 8p 2 P, k 2 K(p) x t 2 {0, 1} 8t 2 T y c 2 {0, 1} 8c 2 C. P ( T + C ) K(p) ={x c p} c2p [ {x t p} t2t C p,k = {c 2 C : x c p = k} T p,k = {t 2 T : x t p = k}

25 Simple Formulation for Univariate Functions f(d 3 ) f(d 2 ) f(d 5 ) f(d 1 ) f(d 4 ) z = f(x) d 1 d 2 d 3 d 4 d 5 Size = O (# of segments) Non-Ideal: Fractional Extreme Points x z = 5 1= 5 j=1 j=1 dj f(d j ) j j, j 0

26 Advanced Formulation for Univariate Functions f(d 3 ) f(d 2 ) f(d 5 ) f(d 1 ) f(d 4 ) z = f(x) x z d 1 d 2 d 3 d 4 d 5 Size = O (log 2 # of segments) Ideal: Integral Extreme Points = 5 1= 5 j=1 j=1 y 2 {0, 1} 2 dj f(d j ) j j, j 0

27 Extended Formulation for PWL Functions S = gr (f) = f(d 3 ) f(d 2 ) f(d 1 ) f(d 4 ) k[ i=1 ( (x, z) 2 R 2 : 0 d 1 d 2 d 3 d 4 d i apple x apple d i+1 m i x + c i = z ) d i y i apple x i apple d i+1 y i m i x i + c i y i = z i k i=1 xi = x k i=1 zi = z k i=1 y i =1 MC Formulation: y 2 {0, 1} k 8i 2 [k] 8i 2 [k]

28 Abstracting Univariate Functions P z = f(x) = i := 2 5 : j =0 8j 5/2 T i T j=1 i := {i, i +1} i 2 {1,...,4} f(d 3 ) 5 = f(d 2 ) f(d 5 ) f(d 1 ) f(d 4 ) x z 1= 5 j=1 dj f(d j ) j j, j 0 d 1 d 2 d 3 d 4 d 5 T [ 4 i=1 P i 5

29 Abstraction Works for Multivariate Functions P i := { 2 m : j =0 8v j /2 T i } f(x,y) v m y x 2 [ n i=1 P i m

30 Complete Abstraction V := n 2 R V + : v2v o v =1, P i = 2 V : v =0 8v /2 T i T i = cliques of a graph

31 From Cliques to (Complement) Conflict Graph SOS

32 From Conflict Graph to Bi-clique Cover SOS

33 From Bi-clique Cover to Formulation SOS2 0 apple apple 1 y 1 0 apple 3 apple y 1 0 apple apple 1 y 2 0 apple apple y

34 Ideal Formulation from Bi-clique Cover Conflict Graph G =(V,E) E = {(u, v) :u, v 2 V, u 6= s.t. u, v 2 T i } Bi-clique cover A j, B j t j=1, Aj, B j V 8{u, v} 2 E 9j s.t. u 2 A j ^ v 2 B j Formulation v2a j v apple 1 y j 8j 2 [t] v2b j v apple y j 8j 2 [t] y 2 {0, 1} t

35 Recursive Construction of Cover for SOS2, Step 1 Base case n=2 1 : Step 1 recursion : Reflect Graph / Cover Stick Graph / Cover Repeat for all bi-cliques from 2 k-1 to cover all edges within first and last half of conflict graph

36 Recursive Construction of Cover for SOS2, Step 2 Only edges missing are those between first and last half of conflict graph Step 2 : Add one more bi-clique 1 2 n/2 n/2 n/2 + n n Cover has log 2 n bi-cliques. For non-power of two just delete extra nodes.

37 Grid Triangulations: Step 1 = SOS2 for Inter-Box Covers all arcs between boxes

38 Grid Triangulations: Step 2 = Ad-hoc Intra-Box Covers all arcs within boxes Sometimes 1 additional cover

39 Grid Triangulations: Step 2 = Ad-hoc Intra-Box Sometimes 2 additional covers Sometimes more, but always less than 9 Simple rules to get (near) optimal in Fall 16