Improving the Randomization Step in Feasibility Pump using WalkSAT

Transcription

1 Improving the Randomization Step in Feasibility Pump using Santanu S. Dey Joint work with: Andres Iroume, Marco Molinaro, Domenico Salvagnin Discrepancy & IP workshop, 2018

2 Sparsity in real" Integer Programs (IPs) Mixed-binary FP + Real" IPs are sparse: The average number (median) of non-zero entries in the constraint matrix of MIPLIB 2010 instances is 1.63% (0.17%). Many have "arrow shape" [Bergner, Caprara, Furini, Lübbecke, SPARSITY IN IPs Malaguti, Traversi 11] or "almost decomposable structure" of the constraint Many IPs matrix. in practice have sparse constraints Other Many example, have almost two-stage decomposable Stochasticstructure: IPs: 2-stage stochastic programming x As proxy, we keep in mind decomposable problems x

3 3 Sparsity in real" Integer Programs (IPs) Mixed-binary FP + Real" IPs are sparse: The average number (median) of non-zero entries in the constraint matrix of MIPLIB 2010 instances is 1.63% (0.17%). Many have "arrow shape" [Bergner, Caprara, Furini, Lübbecke, SPARSITY IN IPs Malaguti, Traversi 11] or "almost decomposable structure" of the constraint Many IPs matrix. in practice have sparse constraints Other Many example, have almost two-stage decomposable Stochasticstructure: IPs: 2-stage stochastic programming x As proxy, we keep in mind decomposable problems Goal: Exploit sparsity of IPs while designing primal heuristics, cutting-plane, branching rules... x

4 1 Feasibility Pump

5 5 : Feasibility Pump (FP) Mixed-binary FP + [Fischetti, Glover, Lodi 05] Vanilla Feasibility Pump Input: Mixed-binary LP (with binary variables x and continuous variables y) Solve the linear programming relaxation, and let ( x, ȳ) be an optimal solution While x is not integral do: Round: Round x to closest 0/1 values, call the obtained vector x. Project: Let ( x, ȳ) be the point in the LP relaxation that minimizes i x i x i (we say, x = l 1 -proj( x)).

6 6 : Feasibility Pump (FP) Mixed-binary FP + [Fischetti, Glover, Lodi 05] Vanilla Feasibility Pump Input: Mixed-binary LP (with binary variables x and continuous variables y) Solve the linear programming relaxation, and let ( x, ȳ) be an optimal solution While x is not integral do: Round: Round x to closest 0/1 values, call the obtained vector x. Project: Let ( x, ȳ) be the point in the LP relaxation that minimizes i x i x i (we say, x = l 1 -proj( x)). Problem: The above algorithm may cycle: Revisit the same x {0, 1} n is different iterations (stalling). Solution: Randomly perturb x.

7 7 : Feasibility Pump (FP) Mixed-binary FP + [Fischetti, Glover, Lodi 05] Vanilla Feasibility Pump Input: Mixed-binary LP (with binary variables x and continuous variables y) Solve the linear programming relaxation, and let ( x, ȳ) be an optimal solution while x is not integral do: Round: Round x to closest 0/1 values, call the obtained vector x. If stalling detected: Randomly perturb x to a different 0/1 vector. Project (l 1 -proj): Let ( x, ȳ) be the point in the LP relaxation that minimizes i x i x i.

8 8 Feasibility Pump (FP) Mixed-binary FP + FP is very successful in practice (For example, the original FP finds feasible solutions for 96.3% of the instances in MIPLIB 2003 instances). Many improvements and generalizations: [Achterberg, Berthold 07], [Bertacco, Fischetti, Lodi 07], [Bonami, Cornuéjols, Lodi, Margot 09], [Fischetti, Salvagnin 09], [Boland, Eberhard, Engineer, Tsoukalas 12], [D Ambrosio, Frangioni, Liberti, Lodi 12], [De Santis, Lucidi, Rinaldi 13], [Boland, Eberhard, Engineer, Fischetti, Savelsbergh, Tsoukalas 14], [Geißler, Morsi, Schewe, Schmidt 17],... Some directions of research: Take objective function into account Mixed-integer programs with general integer variables. Mixed-integer Non-linear programs (MINLP) Alternative projection and rounding steps

9 9 Feasibility Pump (FP) Mixed-binary FP + FP is very successful in practice (For example, the original FP finds feasible solutions for 96.3% of the instances in MIPLIB 2003 instances). Many improvements and generalizations: [Achterberg, Berthold 07], [Bertacco, Fischetti, Lodi 07], [Bonami, Cornuéjols, Lodi, Margot 09], [Fischetti, Salvagnin 09], [Boland, Eberhard, Engineer, Tsoukalas 12], [D Ambrosio, Frangioni, Liberti, Lodi 12], [De Santis, Lucidi, Rinaldi 13], [Boland, Eberhard, Engineer, Fischetti, Savelsbergh, Tsoukalas 14], [Geißler, Morsi, Schewe, Schmidt 17],... Some directions of research: Take objective function into account Mixed-integer programs with general integer variables. Mixed-integer Non-linear programs (MINLP) Alternative projection and rounding steps Randomization step plays significant role but has not been explicitly studied. We focus on changing the randomization step by "thinking about sparsity".

10 10 Sparse IPs Decomposable IPs Mixed-binary FP + As discussed earlier real integer programs are sparse. A common example of sparse integer programs is those that are almost decomposable.

11 11 Sparse IPs Decomposable IPs SPARSITY IN IPs Mixed-binary FP + Many IPs in practice have sparse constraints Many have almost decomposable structure: As 2-stage discussed stochastic earlier real integer programs are sparse. programming x A common example of sparse integer programs is those that are almost decomposable. As proxy, As proxy, we we keep keep in in mind decomposable problems. x

12 12 Agenda Mixed-binary FP + Propose a modification of for the mixed-binary case. Show that this modified algorithm "works well" on-mixed-binary instances that are decomposable.

13 13 Agenda Mixed-binary FP + Propose a modification of for the mixed-binary case. Show that this modified algorithm "works well" on-mixed-binary instances that are decomposable. Analyze randomization based on + Feasibility Pump. Show that this version of FP "works well" on single-row decomposable instances.

14 Agenda Mixed-binary FP + Propose a modification of for the mixed-binary case. Show that this modified algorithm "works well" on-mixed-binary instances that are decomposable. Analyze randomization based on + Feasibility Pump. Show that this version of FP "works well" on single-row decomposable instances. Implementation of FP with new randomization step that combines ideas from the previous randomization and new randomization. The new method shows small but consistent improvement over FP.

15 2

16 16 Mixed-binary FP + is effective primal heuristic used in SAT community [Schoning 99] : WALKSAT for pure-binary IPs - Start with a uniformly random point x {0,1} n. If feasible, done - Else, pick any violated constraint and randomly pick a variable x i in its support is effective - Flip value primal of x i heuristic used in SAT community [Schöning 99] - Repeat from Step 2 = x = for pure binary IPs Start with a uniformly random point x {0, 1} n. If feasible, done While x is infeasible do Pick any violated constraint and randomly pick a variable x i in its support Flip value of x i

17 17 Performance of Mixed-binary FP + [Schöning 99] With probability 1 δ, WALKSAT returns a feasible solution in log( 1 δ )2n iterations. Key Ideas:

18 18 Performance of Mixed-binary FP + [Schöning 99] With probability 1 δ, WALKSAT returns a feasible solution in log( 1 δ )2n iterations. Key Ideas: Consider a fixed integer feasible solution x. Track the number of coordinates that are different from x.

19 19 Performance of Mixed-binary FP + [Schöning 99] With probability 1 δ, WALKSAT returns a feasible solution in log( 1 δ )2n iterations. Key Ideas: Consider a fixed integer feasible solution x. Track the number of coordinates that are different from x. In each step, with probability at least 1 s }{{} # non-zeros in violated constraint we choose to flip coordinate where they differ.

20 20 Performance of Mixed-binary FP + [Schöning 99] With probability 1 δ, WALKSAT returns a feasible solution in log( 1 δ )2n iterations. Key Ideas: Consider a fixed integer feasible solution x. Track the number of coordinates that are different from x. In each step, with probability at least 1 s }{{} # non-zeros in violated constraint we choose to flip coordinate where they differ. a positive constant With probability atleast 1 s, reduce by 1 the number of coordinates they differ.

21 21 2-stage stochastic programming good for decomposable instances x As proxy, we keep in mind decomposable problems Mixed-binary FP + x Observation Each iteration depends only on one part. Overall execution can be split into independent executions over each part Put together bound from previous page over all parts

22 2-stage stochastic programming good for decomposable instances x As proxy, we keep in mind decomposable problems Mixed-binary FP + x Observation Each iteration depends only on one part. Overall execution can be split into independent executions over each part Put together bound from previous page over all parts Consequences Find feasible solution in k2 n/k iterations. Compare this to total enumeration 2 n iterations.

23 3 Mixed-binary version of

24 24 Mixed-binary version of Mixed-binary FP + (l) for Mixed-binary IPs Input: Mixed-binary LP-relaxation {(x, y) Ax + By b} (with binary variables x and continuous variables y); parameter: l Start with a uniformly random point x {0, 1} n. If ȳ such that ( x, ȳ) is feasible, done While x Proj x (P) do Generate minimal (wrt support of λ) projected certificate of infeasibility: λ Ax λ b 1. a valid inequality for Proj x (P) i.e.,λ 0, λ B = violating x: (λ A) x > λ b (Can be obtained by solving a LP) Randomly pick l variables (with replacement) in the support of minimal projected certificate. Flip value of these variables

25 25 Mixed-binary Key Observation: If a set is decomposable, then minimal certificate has a support contained in exactly one disjoint set of variables. Mixed-binary FP +

26 26 Mixed-binary Mixed-binary FP + Key Observation: If a set is decomposable, then minimal certificate has a support contained in exactly one disjoint set of variables. Theorem Consider a feasible decomposable mixed-binary set P I = P I 1... P I k, where for all i [k] we have P I i = P i ({0, 1} n i R d i ), P i = {(x i, y i ) [0, 1] n i R d i : A i x i + B i y i c i }. (1) Let s i be such that each constraint in P i has at most s i binary variables, and define γ i := min{s i (d i + 1), n i }. Then with probability at least 1 δ, Mixed-binary (1) returns a feasible solution within ln(k/δ) n i 2 n i log γ i i iterations.

27 4 Feasibility Pump +

28 28 FP + (FPW) Mixed-binary FP + Vanilla Feasibility Pump Input: Mixed-binary LP (with binary variables x and continuous variables y) Solve the linear programming relaxation, and let ( x, ȳ) be an optimal solution while x is not integral do: Round: Round x to closest 0/1 values, call the obtained vector x. If Stalling detected: "Randomly" perturb x to a different 0/1 vector. Project: Let ( x, ȳ) be the point in the LP relaxation that minimizes i x i x i.

29 29 FP + (FPW) Mixed-binary FP + Feasibility Pump + Input: Mixed-binary LP (with binary variables x and continuous variables y) Solve the linear programming relaxation, and let ( x, ȳ) be an optimal solution while x is not integral do: Round: Round x to closest 0/1 values, call the obtained vector x. If Stalling detected: "Randomly" perturb x to a different 0/1 vector. < Use mixed-binary (l) for random update Project: Let ( x, ȳ) be the point in the LP relaxation that minimizes i x i x i.

30 30 Analysis of Feasibility Pump + (FPW) Mixed-binary FP + 1. We are not able to analyze this algorithm WFP for a general mixed-binary IP Issue: From previous proof, with probability 1/s j randomization makes progress, but projection+rounding in next iteration could ruin everything

31 31 Analysis of Feasibility Pump + (FPW) Mixed-binary FP + 1. We are not able to analyze this algorithm WFP for a general mixed-binary IP Issue: From previous proof, with probability 1/s j randomization makes progress, but projection+rounding in next iteration could ruin everything 2. Can analyze running-time for decomposable 1-row instances, i.e. instances of the following kind: a i x i + b i y i } = c i x i {0, 1} n i, y i R d i [k] i +.

32 32 Main result Mixed-binary FP + Theorem Consider a feasible decomposable 1-row instances set as shown in the previous slide. Then with probability at least 1 δ, Feasibility Pump + (2) returns a feasible solution within T = ln(k/δ) i [k] n i (n i + 1) 2 2n i log n i ln(k/δ) k( n + 1) 2 2 n log n 2 iterations, where n = max i n i.

33 33 Main result Mixed-binary FP + Theorem Consider a feasible decomposable 1-row instances set as shown in the previous slide. Then with probability at least 1 δ, Feasibility Pump + (2) returns a feasible solution within T = ln(k/δ) i [k] n i (n i + 1) 2 2n i log n i ln(k/δ) k( n + 1) 2 2 n log n 2 iterations, where n = max i n i. Note: Naive Feasibility Pump with original randomization (and no re-start) may fail to converge for these instances.

34 34 Proof sketch - I (Like before) We can split the execution into independent executions over each constraint. Mixed-binary FP +

35 35 Proof sketch - I (Like before) We can split the execution into independent executions over each constraint. Notation: For x {0, 1} n : AltProj( x) := round (l 1 -proj( x)). Mixed-binary FP +

36 36 Mixed-binary FP + Proof sketch - I (Like before) We can split the execution into independent executions over each constraint. Notation: For x {0, 1} n : AltProj( x) := round (l 1 -proj( x)). Proposition (Length of cycle) All cycles are due to short cycles, i.e. randomization is invoked only when AltProj( x) = x.

37 37 Mixed-binary FP + Proof sketch - I (Like before) We can split the execution into independent executions over each constraint. Notation: For x {0, 1} n : AltProj( x) := round (l 1 -proj( x)). Proposition (Length of cycle) All cycles are due to short cycles, i.e. randomization is invoked only when AltProj( x) = x. Notation (Stabilization): AltProj ( x) = x, where AltProj k ( x) = AltProj k+1 ( x) = x for some k Z ++.

38 38 Mixed-binary FP + Proof sketch - I (Like before) We can split the execution into independent executions over each constraint. Notation: For x {0, 1} n : AltProj( x) := round (l 1 -proj( x)). Proposition (Length of cycle) All cycles are due to short cycles, i.e. randomization is invoked only when AltProj( x) = x. Notation (Stabilization): AltProj ( x) = x, where AltProj k ( x) = AltProj k+1 ( x) = x for some k Z ++. # of iterations FPW [# iterations of AltProj ] }{{} Number of stallings max x {0,1} n min{k : altproj k ( x) = altproj ( x)}. }{{} Worst-case stabilization time

39 39 Mixed-binary FP + Proof sketch - I (Like before) We can split the execution into independent executions over each constraint. Notation: For x {0, 1} n : AltProj( x) := round (l 1 -proj( x)). Proposition (Length of cycle) All cycles are due to short cycles, i.e. randomization is invoked only when AltProj( x) = x. Notation (Stabilization): AltProj ( x) = x, where AltProj k ( x) = AltProj k+1 ( x) = x for some k Z ++. # of iterations FPW [# iterations of AltProj ] }{{} Number of stallings max x {0,1} n min{k : altproj k ( x) = altproj ( x)}. }{{} Worst-case stabilization time Proposition (Worst-case stabilization time) For any x {0, 1} n, AltProj n+1 ( x) = AltProj( x).

40 40 Proof sketch - II Mixed-binary FP + [x 2 := AltProj ( x 1 )] [ x 2 := WALKSAT(x 2 )] [x 3 := AltProj ( x 2 )] [ x 3 := WALKSAT(x 3 )] [x 4 := AltProj ( x 3 )] [ x 4 := WALKSAT(x 4 )]... Like last time, we target a point x Proj x (P) {0, 1} n.

41 Proof sketch - II Mixed-binary FP + [x 2 := AltProj ( x 1 )] [ x 2 := WALKSAT(x 2 )] [x 3 := AltProj ( x 2 )] [ x 3 := WALKSAT(x 3 )] [x 4 := AltProj ( x 3 )] [ x 4 := WALKSAT(x 4 )]... Like last time, we target a point x Proj x (P) {0, 1} n. Key Question: What if we get closer to x in the step, but then go far away in the AltProj step.

42 Proof sketch - II Mixed-binary FP + [x 2 := AltProj ( x 1 )] [ x 2 := WALKSAT(x 2 )] [x 3 := AltProj ( x 2 )] [ x 3 := WALKSAT(x 3 )] [x 4 := AltProj ( x 3 )] [ x 4 := WALKSAT(x 4 )]... Like last time, we target a point x Proj x (P) {0, 1} n. Key Question: What if we get closer to x in the step, but then go far away in the AltProj step. Lemma Consider a point x Proj x (P) {0, 1} n, and a point x {0, 1} n not in Proj x (P) {0, 1} n. Suppose altproj( x) = x.

43 43 Proof sketch - II Mixed-binary FP + [x 2 := AltProj ( x 1 )] [ x 2 := WALKSAT(x 2 )] [x 3 := AltProj ( x 2 )] [ x 3 := WALKSAT(x 3 )] [x 4 := AltProj ( x 3 )] [ x 4 := WALKSAT(x 4 )]... Like last time, we target a point x Proj x (P) {0, 1} n. Key Question: What if we get closer to x in the step, but then go far away in the AltProj step. Lemma Consider a point x Proj x (P) {0, 1} n, and a point x {0, 1} n not in Proj x (P) {0, 1} n. Suppose altproj( x) = x. Then there is a point x {0, 1} n satisfying the following: 1. (close to x) x x (closer to x ) x x 0 x x 0 1

44 Mixed-binary FP + Proof sketch - II [x 2 := AltProj ( x 1 )] [ x 2 := WALKSAT(x 2 )] [x 3 := AltProj ( x 2 )] [ x 3 := WALKSAT(x 3 )] [x 4 := AltProj ( x 3 )] [ x 4 := WALKSAT(x 4 )]... Like last time, we target a point x Proj x (P) {0, 1} n. Key Question: What if we get closer to x in the step, but then go far away in the AltProj step. Lemma Consider a point x Proj x (P) {0, 1} n, and a point x {0, 1} n not in Proj x (P) {0, 1} n. Suppose altproj( x) = x. Then there is a point x {0, 1} n satisfying the following: 1. (close to x) x x (closer to x ) x x 0 x x (projection control) l 1 -proj( x ) x Moreover, if we have the equality l 1 -proj( x ) x 1 = 1 in Item 3, then 2 x x 0 x x 0 2.

45 Mixed-binary FP + Proof sketch - II [x 2 := AltProj ( x 1 )] [ x 2 := WALKSAT(x 2 )] [x 3 := AltProj ( x 2 )] [ x 3 := WALKSAT(x 3 )] [x 4 := AltProj ( x 3 )] [ x 4 := WALKSAT(x 4 )]... Like last time, we target a point x Proj x (P) {0, 1} n. Key Question: What if we get closer to x in the step, but then go far away in the AltProj step. Lemma Consider a point x Proj x (P) {0, 1} n, and a point x {0, 1} n not in Proj x (P) {0, 1} n. Suppose altproj( x) = x. Then there is a point x {0, 1} n satisfying the following: 1. (close to x) x x (closer to x ) x x 0 x x (projection control) l 1 -proj( x ) x Moreover, if we have the equality l 1 -proj( x ) x 1 = 1 in Item 3, then 2 x x 0 x x 0 2. Corollary Let x be a coordinate-wise maximal solution in {0, 1} n Proj x (P). Consider any point x {0, 1} n \ Proj x (P) satisfying altproj ( x) = x, and let x {0, 1} n be a point constructed in Lemma above with respect to x and x. Then altproj ( x ) x 0 x x

46 5

47 47 Mixed-binary FP + Proposed randomization All features (such as constraint propagation) which are part of the Feasibility Pump 2.0 ([Fischetti, Salvagnin 09]) code have been left unchanged. The only change is in the randomization step.

48 48 Mixed-binary FP + Proposed randomization All features (such as constraint propagation) which are part of the Feasibility Pump 2.0 ([Fischetti, Salvagnin 09]) code have been left unchanged. The only change is in the randomization step. Old randomization Define fractionality of i th variable: x i x i. Let F be the number of variables with positive fractionality. Randomly generate an integer TT (uniformly from {10,..., 30}). Flip the min{f, TT } variables with highest fractionality.

49 49 Mixed-binary FP + Proposed randomization All features (such as constraint propagation) which are part of the Feasibility Pump 2.0 ([Fischetti, Salvagnin 09]) code have been left unchanged. The only change is in the randomization step. Old randomization Define fractionality of i th variable: x i x i. Let F be the number of variables with positive fractionality. Randomly generate an integer TT (uniformly from {10,..., 30}). Flip the min{f, TT } variables with highest fractionality. New randomization Flip the min{f, TT } variables with highest fractionality. If F < TT, then: let S be the union of the supports of the constraints that are not satisfied by the current point ( x, ȳ). Select uniformly at random min{ S, TT F } indices from S, and flip the values in x for all the selected indices.

50 50 Computational experiments Mixed-binary FP + Two classes of problems: 1. Two-stage stochastic models (randomly generated) Ax + D i y i b i, i {1,..., k} x {0, 1} p y i {0, 1} q i {1,..., k} 2. MIPLIB 2010 Two algorithms: 1. FP: Feasibility pump FPWM: Feasibility pump the modified randomization above

51 51 Results for stochastic instances Mixed-binary FP + # found itr. time (s) % gap % modified seed FP FPWM FP FPWM FP FPWM FP FPWM FPWM % 39% 22% % 36% 26% % 40% 25% % 35% 23% % 35% 25% % 37% 27% % 39% 27% % 37% 25% % 37% 26% % 38% 24% Avg % 37% 25% Table: Aggregated results by seed on two-stage stochastic models. 150 instances k {10, 20, 30, 40, 50} p = q {10, 20}

52 52 Results for MIPLIB 2010 instances Mixed-binary FP + # found itr. time (s) % gap % modified seed FP FPWM FP FPWM FP FPWM FP FPWM FPWM % 48% 29% % 50% 22% % 47% 33% % 48% 25% % 51% 27% % 50% 32% % 49% 26% % 48% 31% % 49% 27% % 49% 31% Avg % 49% 28% Table: Aggregated results by seed on MIPLIB2010.

53 53 Conclusions Mixed-binary FP + First ever analysis of running time of Feasibility Pump (even if it is for a special class of instances) Suggested changes are trivial to implement and appears to dominate feasibility pump almost consistently.

54 54 Conclusions Mixed-binary FP + First ever analysis of running time of Feasibility Pump (even if it is for a special class of instances) Suggested changes are trivial to implement and appears to dominate feasibility pump almost consistently. "Designing for sparse instances" helps!