Using Sparsity to Design Primal Heuristics for MILPs: Two Stories

Size: px

Start display at page:

Download "Using Sparsity to Design Primal Heuristics for MILPs: Two Stories"

Darren Goodman
5 years ago
Views:

1 for MILPs: Two Stories Santanu S. Dey Joint work with: Andres Iroume, Marco Molinaro, Domenico Salvagnin, Qianyi Wang MIP Workshop, 2017

2 Sparsity in real" Integer Programs (IPs) Real" IPs are sparse: The average number (median) of non-zero entries in the constraint matrix of MIPLIB 2010 instances is 1.63% (0.17%). Many have "arrow shape" [Bergner, Caprara, Furini, Lübbecke, SPARSITY IN IPs Malaguti, Traversi 11] or "almost decomposable structure" of the constraint Many IPs matrix. in practice have sparse constraints Other Many example, have almost two-stage decomposable Stochasticstructure: IPs: 2-stage stochastic programming x As proxy, we keep in mind decomposable x

3 3 Sparsity in real" Integer Programs (IPs) Real" IPs are sparse: The average number (median) of non-zero entries in the constraint matrix of MIPLIB 2010 instances is 1.63% (0.17%). Many have "arrow shape" [Bergner, Caprara, Furini, Lübbecke, SPARSITY IN IPs Malaguti, Traversi 11] or "almost decomposable structure" of the constraint Many IPs matrix. in practice have sparse constraints Other Many example, have almost two-stage decomposable Stochasticstructure: IPs: 2-stage stochastic programming x As proxy, we keep in mind decomposable Goal: Exploit sparsity of IPs while designing primal heuristics? x

4 1 Story 1: Pump Joint work with: Andres Iroume 1, Marco Molinaro 2, and Domenico Salvagnin 3 1 Revenue Analytics, USA 2 PUC-Rio, Brazil 3 IBM Italy and DEI, University of Padova, Italy

5 5 : Feasibility FP + [Fischetti, Glover, Lodi 05] Vanilla Feasibility Pump Input: Mixed-binary LP (with binary variables x and continuous variables y) Solve the linear programming relaxation, and let ( x, ȳ) be an optimal solution While x is not integral do: Round: Round x to closest 0/1 values, call the obtained vector x. Project: Let ( x, ȳ) be the point in the LP relaxation that minimizes i x i x i (we say, x = l 1 -proj( x)).

6 6 : Feasibility FP + [Fischetti, Glover, Lodi 05] Vanilla Feasibility Pump Input: Mixed-binary LP (with binary variables x and continuous variables y) Solve the linear programming relaxation, and let ( x, ȳ) be an optimal solution While x is not integral do: Round: Round x to closest 0/1 values, call the obtained vector x. Project: Let ( x, ȳ) be the point in the LP relaxation that minimizes i x i x i (we say, x = l 1 -proj( x)). Problem: The above algorithm may cycle: Revisit the same x {0, 1} n is different iterations (stalling). Solution: Randomly perturb x.

7 7 : Feasibility FP + [Fischetti, Glover, Lodi 05] Vanilla Feasibility Pump Input: Mixed-binary LP (with binary variables x and continuous variables y) Solve the linear programming relaxation, and let ( x, ȳ) be an optimal solution while x is not integral do: Round: Round x to closest 0/1 values, call the obtained vector x. If stalling detected: Randomly perturb x to a different 0/1 vector. Project (l 1 -proj): Let ( x, ȳ) be the point in the LP relaxation that minimizes i x i x i.

8 8 Feasibility FP + FP is very successful in practice (For example, the original FP finds feasible solutions for 96.3% of the instances in MIPLIB 2003 instances). Many improvements and generalizations: [Achterberg, Berthold 07], [Bertacco, Fischetti, Lodi 07], [Bonami, Cornuéjols, Lodi, Margot 09], [Fischetti, Salvagnin 09], [Boland, Eberhard, Engineer, Tsoukalas 12], [D Ambrosio, Frangioni, Liberti, Lodi 12], [De Santis, Lucidi, Rinaldi 13], [Boland, Eberhard, Engineer, Fischetti, Savelsbergh, Tsoukalas 14], [Geißler, Morsi, Schewe, Schmidt 17],... Some directions of research: Take objective function into account Mixed-integer programs with general integer variables. Mixed-integer Non-linear programs (MINLP) Alternative projection and rounding steps

9 9 Feasibility FP + FP is very successful in practice (For example, the original FP finds feasible solutions for 96.3% of the instances in MIPLIB 2003 instances). Many improvements and generalizations: [Achterberg, Berthold 07], [Bertacco, Fischetti, Lodi 07], [Bonami, Cornuéjols, Lodi, Margot 09], [Fischetti, Salvagnin 09], [Boland, Eberhard, Engineer, Tsoukalas 12], [D Ambrosio, Frangioni, Liberti, Lodi 12], [De Santis, Lucidi, Rinaldi 13], [Boland, Eberhard, Engineer, Fischetti, Savelsbergh, Tsoukalas 14], [Geißler, Morsi, Schewe, Schmidt 17],... Some directions of research: Take objective function into account Mixed-integer programs with general integer variables. Mixed-integer Non-linear programs (MINLP) Alternative projection and rounding steps Randomization step plays significant role but has not been explicitly studied. We focus on changing the randomization step by "thinking about sparsity".

10 10 Sparse IPs Decomposable IPs FP + As discussed earlier real integer programs are sparse. A common example of sparse integer programs is those that are almost decomposable.

11 11 Sparse IPs Decomposable IPs SPARSITY IN IPs FP + Many IPs in practice have sparse constraints Many have almost decomposable structure: As 2-stage discussed stochastic earlier real integer programs are sparse. programming x A common example of sparse integer programs is those that are almost decomposable. As proxy, As proxy, we we keep keep in in mind decomposable. x

12 12 Agenda FP + Propose a modification of for the mixed-binary case. Show that this modified algorithm "works well" on-mixed-binary instances that are decomposable.

13 13 Agenda FP + Propose a modification of for the mixed-binary case. Show that this modified algorithm "works well" on-mixed-binary instances that are decomposable. Analyze randomization based on + Feasibility Pump. Show that this version of FP "works well" on single-row decomposable instances.

14 Agenda FP + Propose a modification of for the mixed-binary case. Show that this modified algorithm "works well" on-mixed-binary instances that are decomposable. Analyze randomization based on + Feasibility Pump. Show that this version of FP "works well" on single-row decomposable instances. Implementation of FP with new randomization step that combines ideas from the previous randomization and new randomization. The new method shows small but consistent improvement over FP.

15 1.1

16 16 FP + is effective primal heuristic used in SAT community [Schoning 99] : WALKSAT for pure-binary IPs - Start with a uniformly random point x {0,1} n. If feasible, done - Else, pick any violated constraint and randomly pick a variable x i in its support is effective - Flip value primal of x i heuristic used in SAT community [Schöning 99] - Repeat from Step 2 x = = for pure binary IPs Start with a uniformly random point x {0, 1} n. If feasible, done While x is infeasible do Pick any violated constraint and randomly pick a variable x i in its support Flip value of x i

17 17 Performance of FP + [Schöning 99] returns a feasible solution in 2 n iterations, in expectation. Key Ideas:

18 18 Performance of FP + [Schöning 99] returns a feasible solution in 2 n iterations, in expectation. Key Ideas: Consider a fixed integer feasible solution x. Track the number of coordinates that different from x.

19 19 Performance of FP + [Schöning 99] returns a feasible solution in 2 n iterations, in expectation. Key Ideas: Consider a fixed integer feasible solution x. Track the number of coordinates that different from x. In each step, with probability at least 1 s }{{} # non-zeros in violated constraint we choose to flip coordinate where they differ

20 20 Performance of FP + [Schöning 99] returns a feasible solution in 2 n iterations, in expectation. Key Ideas: Consider a fixed integer feasible solution x. Track the number of coordinates that different from x. In each step, with probability at least 1 s }{{} # non-zeros in violated constraint we choose to flip coordinate where they differ a positive constant With probability atleast 1 s, reduce by 1 the number of coordinates they differ.

21 21 2-stage stochastic programming good for decomposable instances As proxy, we keep in mind decomposable x FP + x Observation Each iteration depends only on one part. Overall execution can be split into independent executions over each part Put together bound from previous page over all parts

22 2-stage stochastic programming good for decomposable instances As proxy, we keep in mind decomposable x FP + x Observation Each iteration depends only on one part. Overall execution can be split into independent executions over each part Put together bound from previous page over all parts Consequences Find feasible solution in k2 n/k iterations, in expectation. Compare this to total enumeration 2 n iterations.

23 1.2 Mixed-binary version of

24 Mixed-binary version of FP + (l) for Mixed-binary IPs Input: Mixed-binary LP-relaxation {(x, y) Ax + By b} (with binary variables x and continuous variables y); parameter: l Start with a uniformly random point x {0, 1} n. If ȳ such that ( x, ȳ) is feasible, done While x Proj x (P) is infeasible do Generate minimal projected certificate of infeasibility: λ Ax λ b 1. a valid inequality for Proj x (P) i.e.,λ 0, λ B = violating x: (λ A) x > λ b 3. minimal with respect to support of λ. (Can be obtained by solving a LP) Randomly pick l variables (with replacement) in the support of minimal projected certificate. Flip value of these variables

25 25 FP + Key Observation: If a set is decomposable, then minimal certificate has a support contained in exactly one disjoint set of variables.

26 26 FP + Key Observation: If a set is decomposable, then minimal certificate has a support contained in exactly one disjoint set of variables. Theorem Consider a feasible decomposable mixed-binary set P I = P I 1... P I k, where for all i [k] we have P I i = P i ({0, 1} n i R d i ), P i = {(x i, y i ) [0, 1] n i R d i : A i x i + B i y i c i }. (1) Let s i be such that each constraint in P i has at most s i binary variables, and define γ i := min{s i (d i + 1), n i }. Then with probability at least 1 δ, (1) returns a feasible solution within ln(k/δ) n i 2 n i log γ i i iterations.

27 1.3 Feasibility Pump +

28 28 FP + (FPW) FP + Vanilla Feasibility Pump Input: Mixed-binary LP (with binary variables x and continuous variables y) Solve the linear programming relaxation, and let ( x, ȳ) be an optimal solution while x is not integral do: Round: Round x to closest 0/1 values, call the obtained vector x. If Stalling detected: "Randomly" perturb x to a different 0/1 vector. Project: Let ( x, ȳ) be the point in the LP relaxation that minimizes i x i x i.

29 29 FP + (FPW) FP + Feasibility Pump + Input: Mixed-binary LP (with binary variables x and continuous variables y) Solve the linear programming relaxation, and let ( x, ȳ) be an optimal solution while x is not integral do: Round: Round x to closest 0/1 values, call the obtained vector x. If Stalling detected: "Randomly" perturb x to a different 0/1 vector. < Use mixed-binary (l) for random update Project: Let ( x, ȳ) be the point in the LP relaxation that minimizes i x i x i.

30 30 Analysis of Feasibility Pump + (FPW) FP + 1. We are not able to analyze this algorithm WFP for a general mixed-binary IP Issue: From previous proof, with probability 1/s j randomization makes progress, but projection+rounding in next iteration could ruin everything

31 31 Analysis of Feasibility Pump + (FPW) FP + 1. We are not able to analyze this algorithm WFP for a general mixed-binary IP Issue: From previous proof, with probability 1/s j randomization makes progress, but projection+rounding in next iteration could ruin everything 2. Can analyze running-time for decomposable 1-row instances, i.e. instances of the following kind: a i x i + b i y i } = c i x i {0, 1} n i, y i R d i [k] i +.

32 32 Main result FP + Theorem Consider a feasible decomposable 1-row instances set as shown in the previous slide. Then with probability at least 1 δ, Feasibility Pump + (2) returns a feasible solution within T = ln(k/δ) i [k] n i (n i + 1) 2 2n i log n i ln(k/δ) k( n + 1) 2 2 n log n 2 iterations, where n = max i n i.

33 33 Main result FP + Theorem Consider a feasible decomposable 1-row instances set as shown in the previous slide. Then with probability at least 1 δ, Feasibility Pump + (2) returns a feasible solution within T = ln(k/δ) i [k] n i (n i + 1) 2 2n i log n i ln(k/δ) k( n + 1) 2 2 n log n 2 iterations, where n = max i n i. Note: Naive Feasibility Pump with original randomization may fail to converge for these instances.

34 34 FP + Proof sketch - I (Like before) We can split the execution into independent executions over each constraint.

35 35 FP + Proof sketch - I (Like before) We can split the execution into independent executions over each constraint. Notation: For x {0, 1} n : AltProj( x) := round (l 1 -proj( x)).

36 36 FP + Proof sketch - I (Like before) We can split the execution into independent executions over each constraint. Notation: For x {0, 1} n : AltProj( x) := round (l 1 -proj( x)). Proposition (Length of cycle) All cycles are due to short cycles, i.e. randomization is invoked only when AltProj( x) = x.

37 37 FP + Proof sketch - I (Like before) We can split the execution into independent executions over each constraint. Notation: For x {0, 1} n : AltProj( x) := round (l 1 -proj( x)). Proposition (Length of cycle) All cycles are due to short cycles, i.e. randomization is invoked only when AltProj( x) = x. Notation (Stabilization): AltProj ( x) = x, where AltProj k ( x) = AltProj k+1 ( x) = x for some k Z ++.

38 38 FP + Proof sketch - I (Like before) We can split the execution into independent executions over each constraint. Notation: For x {0, 1} n : AltProj( x) := round (l 1 -proj( x)). Proposition (Length of cycle) All cycles are due to short cycles, i.e. randomization is invoked only when AltProj( x) = x. Notation (Stabilization): AltProj ( x) = x, where AltProj k ( x) = AltProj k+1 ( x) = x for some k Z ++. # of iterations FPW [# iterations of AltProj ] }{{} Number of stallings max x {0,1} n min{k : altproj k ( x) = altproj ( x)}. }{{} Worst-case stabilization time

39 39 FP + Proof sketch - I (Like before) We can split the execution into independent executions over each constraint. Notation: For x {0, 1} n : AltProj( x) := round (l 1 -proj( x)). Proposition (Length of cycle) All cycles are due to short cycles, i.e. randomization is invoked only when AltProj( x) = x. Notation (Stabilization): AltProj ( x) = x, where AltProj k ( x) = AltProj k+1 ( x) = x for some k Z ++. # of iterations FPW [# iterations of AltProj ] }{{} Number of stallings max x {0,1} n min{k : altproj k ( x) = altproj ( x)}. }{{} Worst-case stabilization time Proposition (Worst-case stabilization time) For any x {0, 1} n, AltProj n+1 ( x) = AltProj( x).

40 40 FP + Proof sketch - II [x 2 := AltProj ( x 1 )] [ x 2 := WALKSAT(x 2 )] [x 3 := AltProj ( x 2 )] [ x 3 := WALKSAT(x 3 )] [x 4 := AltProj ( x 3 )] [ x 4 := WALKSAT(x 4 )]... Like last time, we target a point x Proj x (P) {0, 1} n.

41 FP + Proof sketch - II [x 2 := AltProj ( x 1 )] [ x 2 := WALKSAT(x 2 )] [x 3 := AltProj ( x 2 )] [ x 3 := WALKSAT(x 3 )] [x 4 := AltProj ( x 3 )] [ x 4 := WALKSAT(x 4 )]... Like last time, we target a point x Proj x (P) {0, 1} n. Key Question: What if we get closer to x in the step, but then go far away in the AltProj step.

42 FP + Proof sketch - II [x 2 := AltProj ( x 1 )] [ x 2 := WALKSAT(x 2 )] [x 3 := AltProj ( x 2 )] [ x 3 := WALKSAT(x 3 )] [x 4 := AltProj ( x 3 )] [ x 4 := WALKSAT(x 4 )]... Like last time, we target a point x Proj x (P) {0, 1} n. Key Question: What if we get closer to x in the step, but then go far away in the AltProj step. Lemma Consider a point x Proj x (P) {0, 1} n, and a point x {0, 1} n not in Proj x (P) {0, 1} n. Suppose altproj( x) = x.

43 43 FP + Proof sketch - II [x 2 := AltProj ( x 1 )] [ x 2 := WALKSAT(x 2 )] [x 3 := AltProj ( x 2 )] [ x 3 := WALKSAT(x 3 )] [x 4 := AltProj ( x 3 )] [ x 4 := WALKSAT(x 4 )]... Like last time, we target a point x Proj x (P) {0, 1} n. Key Question: What if we get closer to x in the step, but then go far away in the AltProj step. Lemma Consider a point x Proj x (P) {0, 1} n, and a point x {0, 1} n not in Proj x (P) {0, 1} n. Suppose altproj( x) = x. Then there is a point x {0, 1} n satisfying the following: 1. (close to x) x x (closer to x ) x x 0 x x 0 1

44 FP + Proof sketch - II [x 2 := AltProj ( x 1 )] [ x 2 := WALKSAT(x 2 )] [x 3 := AltProj ( x 2 )] [ x 3 := WALKSAT(x 3 )] [x 4 := AltProj ( x 3 )] [ x 4 := WALKSAT(x 4 )]... Like last time, we target a point x Proj x (P) {0, 1} n. Key Question: What if we get closer to x in the step, but then go far away in the AltProj step. Lemma Consider a point x Proj x (P) {0, 1} n, and a point x {0, 1} n not in Proj x (P) {0, 1} n. Suppose altproj( x) = x. Then there is a point x {0, 1} n satisfying the following: 1. (close to x) x x (closer to x ) x x 0 x x (projection control) l 1 -proj( x ) x Moreover, if we have the equality l 1 -proj( x ) x 1 = 1 in Item 3, then 2 x x 0 x x 0 2.

45 FP + Proof sketch - II [x 2 := AltProj ( x 1 )] [ x 2 := WALKSAT(x 2 )] [x 3 := AltProj ( x 2 )] [ x 3 := WALKSAT(x 3 )] [x 4 := AltProj ( x 3 )] [ x 4 := WALKSAT(x 4 )]... Like last time, we target a point x Proj x (P) {0, 1} n. Key Question: What if we get closer to x in the step, but then go far away in the AltProj step. Lemma Consider a point x Proj x (P) {0, 1} n, and a point x {0, 1} n not in Proj x (P) {0, 1} n. Suppose altproj( x) = x. Then there is a point x {0, 1} n satisfying the following: 1. (close to x) x x (closer to x ) x x 0 x x (projection control) l 1 -proj( x ) x Moreover, if we have the equality l 1 -proj( x ) x 1 = 1 in Item 3, then 2 x x 0 x x 0 2. Corollary Let x be a coordinate-wise maximal solution in {0, 1} n Proj x (P). Consider any point x {0, 1} n \ Proj x (P) satisfying altproj ( x) = x, and let x {0, 1} n be a point constructed in Lemma above with respect to x and x. Then altproj ( x ) x 0 x x

46 1.4

47 47 FP + Proposed randomization All features (such as constraint propagation) which are part of the Feasibility Pump 2.0 ([Fischetti, Salvagnin 09]) code have been left unchanged. The only change is in the randomization step.

48 48 FP + Proposed randomization All features (such as constraint propagation) which are part of the Feasibility Pump 2.0 ([Fischetti, Salvagnin 09]) code have been left unchanged. The only change is in the randomization step. Old randomization Define fractionality of i th variable: x i x i. Let F be the number of variables with positive fractionality. Randomly generate an integer TT (uniformly from {10,..., 30}). Flip the min{f, TT } variables with highest fractionality.

49 49 FP + Proposed randomization All features (such as constraint propagation) which are part of the Feasibility Pump 2.0 ([Fischetti, Salvagnin 09]) code have been left unchanged. The only change is in the randomization step. Old randomization Define fractionality of i th variable: x i x i. Let F be the number of variables with positive fractionality. Randomly generate an integer TT (uniformly from {10,..., 30}). Flip the min{f, TT } variables with highest fractionality. Flip the min{f, TT } variables with highest fractionality. If F < TT, then: let S be the union of the supports of the constraints that are not satisfied by the current point ( x, ȳ). Select uniformly at random min{ S, TT F } indices from S, and flip the values in x for all the selected indices.

50 50 Computational experiments FP + Two classes of : 1. Two-stage stochastic models (randomly generated) Ax + D i y i b i, i {1,..., k} x {0, 1} p y i {0, 1} q i {1,..., k} 2. MIPLIB 2010 Two algorithms: 1. FP: Feasibility pump FPWM: Feasibility pump the modified randomization above

51 51 Results for stochastic instances FP + # found itr. time (s) % gap % modified seed FP FPWM FP FPWM FP FPWM FP FPWM FPWM % 39% 22% % 36% 26% % 40% 25% % 35% 23% % 35% 25% % 37% 27% % 39% 27% % 37% 25% % 37% 26% % 38% 24% Avg % 37% 25% Table: Aggregated results by seed on two-stage stochastic models. 150 instances k {10, 20, 30, 40, 50} p = q {10, 20}

52 52 Results for MIPLIB 2010 instances FP + # found itr. time (s) % gap % modified seed FP FPWM FP FPWM FP FPWM FP FPWM FPWM % 48% 29% % 50% 22% % 47% 33% % 48% 25% % 51% 27% % 50% 32% % 49% 26% % 48% 31% % 49% 27% % 49% 31% Avg % 49% 28% Table: Aggregated results by seed on MIPLIB2010.

53 53 Conclusions FP + First ever analysis of running time of Feasibility Pump (even if it is for a special class of instances) Suggested changes are trivial to implement and appears to dominate feasibility pump almost consistently.

54 54 Conclusions FP + First ever analysis of running time of Feasibility Pump (even if it is for a special class of instances) Suggested changes are trivial to implement and appears to dominate feasibility pump almost consistently. "Designing for sparse instances" helps!

55 2 Story 2: Joint work with: Qianyi Wang 4 and Marco Molinaro 5 4 Uber Inc. 5 PUC-Rio, Brazil

56 56 Packing instances Key ideas Some definitions Algorithm and main result max s.t. where A and b is non-negative. c x Ax b x Z n +,

57 57 Packing instances Key ideas Some definitions Algorithm and main result max s.t. where A and b is non-negative. High-level sketch of algorithm The algorithm runs in two phases. c x Ax b x Z n +,

58 58 Packing instances Key ideas Some definitions Algorithm and main result max s.t. where A and b is non-negative. High-level sketch of algorithm The algorithm runs in two phases. c x Ax b x Z n +, 1. In the first phase, the sparse packing problem is partitioned into smaller parts and then these small integer programs are solved.

59 59 Packing instances Key ideas Some definitions Algorithm and main result max s.t. where A and b is non-negative. High-level sketch of algorithm The algorithm runs in two phases. c x Ax b x Z n +, 1. In the first phase, the sparse packing problem is partitioned into smaller parts and then these small integer programs are solved. 2. In the second phase, the optimal solutions of the smaller are patched together into a feasible solution for the original problem by exploiting the sparsity structure of the constraint matrix.

60 2.1 Key ideas

61 61 Key idea - I Key ideas Some definitions Algorithm and main result Suppose the matrix A looks like this: 5 blocks of variables. Unshaded boxes correspond to zeros in A A 1 x 1 + A 2 x 2 + A 3 x 3 + Ax 4 + A 5 x 5 b, x i Z n i +

62 62 Key idea - I Key ideas Some definitions Algorithm and main result Suppose we fix variables in blocks {2, 3, 4, 5} to zero and find a non-zero feasible solution in variables of block 1. A 1 x A 2 x 2 + A 3 x 3 + Ax 4 + A 5 x 5 b, x i Z n i + Let this solutions be: ( x 1, 0, 0, 0, 0)

63 63 Key idea - I Key ideas Some definitions Algorithm and main result Similarly we can fix variables in blocks {1, 3, 4, 5} to zero and find a non-zero feasible solution in variables of block 2. 0 A 1 x 1 + A 2 x A 3 x 3 + Ax 4 + A 5 x 5 b, x i Z n i + Let this solutions be: (0, x 2, 0, 0, 0)

64 64 Key idea - I Key ideas Some definitions Algorithm and main result Is ( x 1, 0, 0, 0, 0) + (0, x 2, 0, 0, 0) guaranteed to be a feasible solution?

65 65 Key idea - I Key ideas Some definitions Algorithm and main result Is ( x 1, 0, 0, 0, 0) + (0, x 2, 0, 0, 0) guaranteed to be a feasible solution? No!

66 66 Key idea - I Key ideas Some definitions Algorithm and main result Is ( x 1, 0, 0, 0, 0) + (0, 0, x 3, 0, 0) guaranteed to be a feasible solution?

67 67 Key idea - I Key ideas Some definitions Algorithm and main result Is ( x 1, 0, 0, 0, 0) + (0, 0, x 3, 0, 0) guaranteed to be a feasible solution? Yes!

68 68 Key idea - I A graph theoretic perspective Key ideas Some definitions Algorithm and main result

69 Key idea - I A graph theoretic perspective 1 2 3 4 5 1 2 3 4 5 Key ideas Some

69 69 Key idea - I A graph theoretic perspective Key ideas Some definitions Algorithm and main result {1, 2} is not a stable set in the graph!

70 70 Key idea - I A graph theoretic perspective Key ideas Some definitions Algorithm and main result {1, 3} is a stable set in the graph!

71 71 Key idea - II Generalizing the notion of stable sets Key ideas Some definitions Algorithm and main result

72 72 Key idea - II Generalizing the notion of stable sets Key ideas Some definitions Algorithm and main result

definitions Algorithm and main result 3 4 2 1 5

73 73 Key idea - II Generalizing the notion of stable sets Key ideas Some definitions Algorithm and main result {{2, 3}, {5}} is a general-stable set in the graph!

74 2.2 Some definitions

75 75 Describing sparsity of A J1 J2 J3 J4 J5 J6 Key ideas Some definitions Algorithm and main result The matrix A with: 1. Column partition J := {J 1,..., J 6 }. 2. Unshaded boxes correspond to zeros in A. 3. Shaded boxes may have non-zero entries.

76 76 Describing sparsity of A J1 J2 J3 J4 J5 J6 v2 v3 v1 v4 Key ideas Some definitions Algorithm and main result v6 v5 The matrix A with: 1. Column partition J := {J 1,..., J 6 }. 2. Unshaded boxes correspond to zeros in A. 3. Shaded boxes may have non-zero entries. The corresponding graph G A,J : 1. One node for every block of variables. 2. (v i, v j ) E if and only if there is a row in A with non-zero entries in both parts J i and J j.

77 Some graph-theoretic definition I: General-stable set Key ideas Some definitions Algorithm and main result Definition (General stable set) Let G = (V, E) be a simple graph. Let V be a collection of subsets of the vertices V. We call a collection of subsets of vertices M 2 V a general stable set subordinate to V if the following hold: 1. Every set in M is contained in a set in V. 2. The sets in M are pairwise disjoint. 3. There are no edges of G with endpoints in distinct sets in M.

78 78 Some graph-theoretic definition I: General-stable set Key ideas Some definitions Algorithm and main result Definition (General stable set) Let G = (V, E) be a simple graph. Let V be a collection of subsets of the vertices V. We call a collection of subsets of vertices M 2 V a general stable set subordinate to V if the following hold: 1. Every set in M is contained in a set in V. 2. The sets in M are pairwise disjoint. 3. There are no edges of G with endpoints in distinct sets in M. Example: 1. V = {{v 1, v 2 }, {v 2, v 3 }, {v 3, v 4 }, {v 4, v 5}, {v 1, v 5}} 2. M = {{v 3 }, {v 1, v 5}} Original Graph: v1 v2 General Stable Set: v1 v2 v5 v3 v5 v3 v4 v4

79 79 Key ideas Some definitions Some graph-theoretic definition II: General-chromatic number Consider a simple graph G = (V, E) and a collection V of subset of vertices. General-chromatic number with respect to V (Denoted as η (G)): V It is the smallest number of general-stable sets M 1,..., M k subordinate to V that cover all vertices of the graph (that is, every vertex v V belongs to a set in one of the M i s). Algorithm and main result

80 80 Key ideas Some definitions Algorithm and main result Some graph-theoretic definition II: General-chromatic number Consider a simple graph G = (V, E) and a collection V of subset of vertices. General-chromatic number with respect to V (Denoted as η (G)): V It is the smallest number of general-stable sets M 1,..., M k subordinate to V that cover all vertices of the graph (that is, every vertex v V belongs to a set in one of the M i s). Fractional general-chromatic number with respect to V (Denoted as η(g)): V Given a general stable set M subordinate to V, let χ M {0, 1} V denote its incidence vector (that is, for each vertex v V, χ M (v) = 1 if v belongs to a set in M, and χ M (v) = 0 otherwise.) Then we define the fractional general-chromatic number η V (G) = min M y M s.t. M y M χ M 1 (2) y M 0 M, where the summations range over all general stable sets subordinate to V.

81 2.3 Algorithm

82 82 Algorithm Input: Interaction graph G A,J = (V, E); A collection V of subset of vertices. Let M be the collection of all general stable sets wrt V. Key ideas Some definitions Algorithm and main result

83 83 Key ideas Some definitions Algorithm and main result Algorithm Input: Interaction graph G A,J = (V, E); A collection V of subset of vertices. Let M be the collection of all general stable sets wrt V. Solve Sub: For W U V. w W := max s.t. c x Ax b x j = 0, variables not in W. x j Z ++ Optimal solution: x W

84 84 Key ideas Some definitions Algorithm and main result Algorithm Input: Interaction graph G A,J = (V, E); A collection V of subset of vertices. Let M be the collection of all general stable sets wrt V. Solve Sub: For W U V. w W := max s.t. c x Ax b x j = 0, variables not in W. x j Z ++ Optimal solution: x W Patching Integer program: z A := max Û w W y W s.t. y W1 + y W2 1 if W 1 W 2 y {0, 1} W y W1 + y W2 1 if v i W i, (v 1, v 2 ) E Optimal solution: y

85 85 Key ideas Some definitions Algorithm and main result Algorithm Input: Interaction graph G A,J = (V, E); A collection V of subset of vertices. Let M be the collection of all general stable sets wrt V. Solve Sub: For W U V. w W := max s.t. c x Ax b x j = 0, variables not in W. x j Z ++ Optimal solution: x W Patching Integer program: z A := max Û w W y W s.t. y W1 + y W2 1 if W 1 W 2 y {0, 1} W Optimal solution: y Output: x A := M W y W x W y W1 + y W2 1 if v i W i, (v 1, v 2 ) E

86 86 Performance guarantee Key ideas Some definitions Algorithm and main result Theorem Let x be the optimal solution of packing IP, and x A be the solution produced by Algorithm, with column partition J (resulting in packing interaction graph G A,J ) and list V. Then ( ) c x A 1 c x. η V (G A,J )

87 2.4

88 88 Two-stage instances Key ideas Some definitions Algorithm and main result Table: Gaps for two-stage instances Instance Features #Variables Algo. CPLEX GRASP nv1-5s100/z (2) NA(5) nv1-5s100/z nv1-5s100/z nv1-5s100/z nv1-5s150/z nv1-5s150/z nv1-5s150/z nv1-5s150/z nv1-5s200/z5 1, nv1-5s200/z10 1, nv1-5s200/z15 1, nv1-5s200/z20 1,

89 89 Three stage instances Key ideas Some definitions Algorithm and main result Table: Gaps for three stage instances Instance Features #Variables Algo. CPLEX GRASP nv1-5-25s100/z5 3, nv1-5-25s100/z10 3, nv1-5-25s100/z15 3, nv1-5-25s100/z20 3, nv1-5-25s150/z5 4, nv1-5-25s150/z10 4, nv1-5-25s150/z15 4, nv1-5-25s150/z20 4, nv1-5-25s200/z5 6, nv1-5-25s200/z10 6, nv1-5-25s200/z15 6, nv1-5-25s200/z20 6,

90 90 Key ideas Some definitions Algorithm and main result Random Graph Instances Table: Relative gap of various algorithms (random graph instances) Instance Features #Variables Algo. CPLEX GRASP nv50pb3s400/z5 20, nv50pb3s400/z10 20, nv50pb3s400/z15 20, (1) (1) nv50pb3s400/z20 20,000 NA(5) NA(5) nv50pb5s400/z5 20, nv50pb5s400/z10 20, nv50pb5s400/z15 20, nv50pb5s400/z20 20,000 NA(5) (1) nv50pb8s400/z5 20, nv50pb8s400/z10 20, nv50pb8s400/z15 20, nv50pb8s400/z20 20, nv50pb10s400/z5 20, nv50pb10s400/z10 20, nv50pb10s400/z15 20, nv50pb10s400/z20 20,

91 91 Key ideas Some definitions Algorithm and main result Random Graph Instances - II Table: Relative gap of various algorithms (random graph instances) - II Instance Features #Variables Gap A Gap IP Gap GS GS nv100pb3s200/z5 20, nv100pb3s200/z10 20, nv100pb3s200/z15 20, nv100pb3s200/z20 20, nv100pb5s200/z5 20, nv100pb5s200/z10 20, nv100pb5s200/z15 20, nv100pb5s200/z20 20, nv100pb8s200/z5 20, nv100pb8s200/z10 20, nv100pb8s200/z15 20, nv100pb8s200/z20 20, nv100pb10s200/z5 20, nv100pb10s200/z10 20, nv100pb10s200/z15 20, nv100pb10s200/z20 20,

92 92 Conclusion Key ideas Some definitions Algorithm and main result Some comments: Our approximation algorithm almost always obtains a significantly better solution than CPLEX and GRASP.

93 93 Conclusion Key ideas Some definitions Algorithm and main result Some comments: Our approximation algorithm almost always obtains a significantly better solution than CPLEX and GRASP. Our algorithm is very easily parallelizable. We may solve each of the sub-ips separately. Dual bounds: Also parallelizable: z UB = max s.t. c T x j variables in (U) 0 x 1 c j x j }{{} w U, U V Opt.obj.ofsubproblem [Huchette, D., Vielma 16]

Improving the Randomization Step in Feasibility Pump using WalkSAT

Improving the Randomization Step in Feasibility Pump using Santanu S. Dey Joint work with: Andres Iroume, Marco Molinaro, Domenico Salvagnin Discrepancy & IP workshop, 2018 Sparsity in real" Integer Programs