Tightening linearizations of non-linear binary optimization problems

Transcription

1 Tightening linearizations of non-linear binary optimization problems Elisabeth Rodríguez Heck and Yves Crama QuantOM, HEC Management School, University of Liège Partially supported by Belspo - IAP Project COMEX January 2016, Louvain-la-Neuve, ORBEL 30

2 Denitions Denition: Pseudo-Boolean functions A pseudo-boolean function is a mapping f : {0, 1} n R. Multilinear representation Every pseudo-boolean function f can be represented uniquely by a multilinear polynomial (Hammer, Rosenberg, Rudeanu [5]). Example: f (x 1, x 2, x 3 ) = 9x 1 x 2 x 3 + 8x 1 x 2 6x 2 x 3 + x 1 2x 2 + x 3 1 / 19

3 Pseudo-Boolean Optimization Many problems formulated as optimization of a pseudo-boolean function Pseudo-Boolean Optimization min f (x) x {0,1} n Optimization is N P-hard, even if f is quadratic (MAX-2-SAT, MAX-CUT modelled by quadratic f ). Approaches: Linearization: extensive literature in integer programming. Quadratization: exact algorithms, heuristics, polyhedral results... Direct resolution methods 2 / 19

4 Pseudo-Boolean Optimization Many problems formulated as optimization of a pseudo-boolean function Pseudo-Boolean Optimization min f (x) x {0,1} n Optimization is N P-hard, even if f is quadratic (MAX-2-SAT, MAX-CUT modelled by quadratic f ). Approaches: Linearization: extensive literature in integer programming. Quadratization: exact algorithms, heuristics, polyhedral results... Direct resolution methods 2 / 19

5 Standard linearization (SL) min a S x k + {0,1} n S S k S i=1 n a i x i, S = {S {1,..., n} a S 0 and S 2} (non-constant and non-linear monomials) 1. Substitute monomials min a S y S + S S i=1 n a i x i s.t. y S = k S x k, S S y S {0, 1}, x k {0, 1}, S S k = 1,..., n 3 / 19

6 Standard linearization (SL) min a S x k + {0,1} n S S k S i=1 n a i x i, S = {S {1,..., n} a S 0 and S 2} (non-constant and non-linear monomials) 1. Substitute monomials min a S y S + S S i=1 n a i x i s.t. y S = k S x k, S S 2. Linearize constraints min a S y S + S S i=1 n a i x i s.t. y S x k, k S, S S y S x k ( S 1), S S k S y S {0, 1}, x k {0, 1}, S S k = 1,..., n y S {0, 1}, x k {0, 1}, S S k = 1,..., n 3 / 19

7 Standard linearization (SL) min a S x k + {0,1} n S S k S i=1 n a i x i, S = {S {1,..., n} a S 0 and S 2} (non-constant and non-linear monomials) 1. Substitute monomials min a S y S + S S i=1 n a i x i s.t. y S = k S x k, S S 2. Linearize constraints min a S y S + S S i=1 n a i x i s.t. y S x k, k S, S S y S x k ( S 1), S S k S y S {0, 1}, x k {0, 1}, S S k = 1,..., n y S {0, 1}, x k {0, 1}, S S k = 1,..., n 3 / 19

8 Standard linearization (SL) min a S x k + {0,1} n S S k S i=1 n a i x i, S = {S {1,..., n} a S 0 and S 2} (non-constant and non-linear monomials) 1. Substitute monomials min a S y S + S S i=1 n a i x i s.t. y S = k S x k, S S 3. Linear relaxation min a S y S + S S i=1 n a i x i s.t. y S x k, k S, S S y S x k ( S 1), S S k S y S {0, 1}, x k {0, 1}, S S k = 1,..., n 0 y S 1, S S 0 x k 1, k = 1,..., n 3 / 19

9 Linearized problem Linearized problem min (x,y) PSL S S a S y S + n a i x i, where i=1 P SL = conv({(x, y) {0, 1} n+ S y S x k k S, y S k S x k ( S 1)}) Relaxing integrality constraints we obtain the standard linearization polytope P SL = conv({(x, y) [0, 1] n+ S y S x k k S, y S k S x k ( S 1)}) 4 / 19

10 Linearized problem Linearized problem min (x,y) PSL S S a S y S + n a i x i, where i=1 P SL = conv({(x, y) {0, 1} n+ S y S x k k S, y S k S x k ( S 1)}) Relaxing integrality constraints we obtain the standard linearization polytope P SL = conv({(x, y) [0, 1] n+ S y S x k k S, y S k S x k ( S 1)}) For a function containing a single non-linear monomial, P SL = P SL 4 / 19

11 Linearized problem Linearized problem min (x,y) PSL S S a S y S + n a i x i, where i=1 P SL = conv({(x, y) {0, 1} n+ S y S x k k S, y S k S x k ( S 1)}) Relaxing integrality constraints we obtain the standard linearization polytope P SL = conv({(x, y) [0, 1] n+ S y S x k k S, y S k S x k ( S 1)}) For a function containing a single non-linear monomial, P SL = P SL 4 / 19

12 Linearized problem Linearized problem min (x,y) PSL S S a S y S + n a i x i, where i=1 P SL = conv({(x, y) {0, 1} n+ S y S x k k S, y S k S x k ( S 1)}) Relaxing integrality constraints we obtain the standard linearization polytope P SL = conv({(x, y) [0, 1] n+ S y S x k k S, y S k S x k ( S 1)}) Not for two non-linear terms, in general P SL is a very weak relaxation!!! 4 / 19

13 Denition of the 2-links Denition: 2-links Consider two monomials S, T S such that S T 2, and their corresponding variables y S, y T. The 2-link between S and T is y S y T x i + T \S i T \S For S T < 2, the 2-links are implied by the standard linearization. 5 / 19

14 Denition of the 2-links Denition: 2-links Consider two monomials S, T S such that S T 2, and their corresponding variables y S, y T. The 2-link between S and T is y S y T x i + T \S i T \S For S T < 2, the 2-links are implied by the standard linearization. Interpretation: if y S = i S x i = 1 and y T = i T x i = 0, a variable in T \S must be zero. 5 / 19

15 Denition of the 2-links Denition: 2-links Consider two monomials S, T S such that S T 2, and their corresponding variables y S, y T. The 2-link between S and T is y S y T x i + T \S i T \S For S T < 2, the 2-links are implied by the standard linearization. Interpretation: if y S = i S x i = 1 and y T = i T x i = 0, a variable in T \S must be zero. 5 / 19

16 Validity and strength Proposition: Validity For any S, T S, the corresponding 2-link is valid for P SL. Proposition: Facet-dening for nested monomials Consider a pseudo-boolean function dened on l monomials such that S (1) S (2) S (l) and S (1) 2. Then, the 2-links corresponding to consecutive monomials in the nest y S (k) y S (k+1) x i + S (k+1) \S (k) i S (k+1) \S (k) y S (k+1) y S (k), for k = 1,..., l 1, are facet-dening for P,nest SL. 6 / 19

17 Validity and strength Proposition: Validity For any S, T S, the corresponding 2-link is valid for P SL. Proposition: Facet-dening for nested monomials Consider a pseudo-boolean function dened on l monomials such that S (1) S (2) S (l) and S (1) 2. Then, the 2-links corresponding to consecutive monomials in the nest y S (k) y S (k+1) x i + S (k+1) \S (k) i S (k+1) \S (k) y S (k+1) y S (k), for k = 1,..., l 1, are facet-dening for P,nest SL. 6 / 19

18 The case of two non-linear monomials Consider f containing exactly two non-linear terms S, T ( S T 2), and the corresponding 2-links y S y T x i + T \S (1) i T \S y T y S x i + S\T (2) i S\T Proposition: Facet-dening (1) and (2) are facet-dening for P SL. 7 / 19

19 The case of two non-linear monomials Consider f containing exactly two non-linear terms S, T ( S T 2), and the corresponding 2-links y S y T x i + T \S (1) i T \S y T y S x i + S\T (2) i S\T Proposition: Facet-dening (1) and (2) are facet-dening for P SL. Notation P 2links SL = P SL {(x, y S, y T ) such that (1), (2) are satised} 7 / 19

20 The case of two non-linear monomials Consider f containing exactly two non-linear terms S, T ( S T 2), and the corresponding 2-links y S y T x i + T \S (1) i T \S y T y S x i + S\T (2) i S\T Proposition: Facet-dening (1) and (2) are facet-dening for P SL. Notation P 2links SL = P SL {(x, y S, y T ) such that (1), (2) are satised} 7 / 19

21 The case of two non-linear monomials Theorem: Perfect formulation for two intersecting non-linear monomials P 2links SL = P SL Idea of the proof: 8 / 19

22 The case of two non-linear monomials Theorem: Perfect formulation for two intersecting non-linear monomials P 2links SL = P SL Idea of the proof: Consider the four polytopes obtained by xing y S, y T to 0 or to 1 in P 2links SL : P 00 : y S = 0, y T = 0 P 11 : y S = 1, y T = 1 P 10 : y S = 1, y T = 0 P 01 : y S = 0, y T = 1 8 / 19

23 The case of two non-linear monomials Theorem: Perfect formulation for two intersecting non-linear monomials P 2links SL = P SL Idea of the proof: Consider the four polytopes obtained by xing y S, y T to 0 or to 1 in P 2links SL : P 00 : y S = 0, y T = 0 P 11 : y S = 1, y T = 1 P 10 : y S = 1, y T = 0 P 01 : y S = 0, y T = 1 Observe that they all have integer vertices. 8 / 19

24 The case of two non-linear monomials Theorem: Perfect formulation for two intersecting non-linear monomials P 2links SL = P SL Idea of the proof: Consider the four polytopes obtained by xing y S, y T to 0 or to 1 in P 2links SL : P 00 : y S = 0, y T = 0 P 11 : y S = 1, y T = 1 P 10 : y S = 1, y T = 0 P 01 : y S = 0, y T = 1 Observe that they all have integer vertices. See that P 2links SL = conv(p 00 P 11 P 10 P 01 ). 8 / 19

25 The case of two non-linear monomials Theorem: Perfect formulation for two intersecting non-linear monomials P 2links SL = P SL Idea of the proof: Consider the four polytopes obtained by xing y S, y T to 0 or to 1 in P 2links SL : P 00 : y S = 0, y T = 0 P 11 : y S = 1, y T = 1 P 10 : y S = 1, y T = 0 P 01 : y S = 0, y T = 1 Observe that they all have integer vertices. See that P 2links SL = conv(p 00 P 11 P 10 P 01 ). 8 / 19

26 : Motivation Example of function containing 3 non-linear monomials for which optimizing over PSL 2links leads to a fractional solution: f (x) = 5x 1 x 2 x 4 3x 1 x 3 x 4 3x 1 x 2 x 3 + 2x 3 It is hopeless (unless P = N P) to nd a concise perfect formulation for the general case. 9 / 19

27 : Motivation Example of function containing 3 non-linear monomials for which optimizing over PSL 2links leads to a fractional solution: f (x) = 5x 1 x 2 x 4 3x 1 x 3 x 4 3x 1 x 2 x 3 + 2x 3 It is hopeless (unless P = N P) to nd a concise perfect formulation for the general case. Are the 2-links still useful for the general case? 9 / 19

28 : Motivation Example of function containing 3 non-linear monomials for which optimizing over PSL 2links leads to a fractional solution: f (x) = 5x 1 x 2 x 4 3x 1 x 3 x 4 3x 1 x 2 x 3 + 2x 3 It is hopeless (unless P = N P) to nd a concise perfect formulation for the general case. Are the 2-links still useful for the general case? 9 / 19

29 : Objectives Two objectives of the computational experiments: Quality of the bounds: of P SL and P 2links SL. Computational performance: of exact resolution methods with dierent types of cuts Method name CPLEX cuts 2-links (User cuts) No cuts (P SL ) User cuts (PSL 2links ) CPLEX cuts CPLEX & user cuts 10 / 19

30 : Objectives Two objectives of the computational experiments: Quality of the bounds: of P SL and P 2links SL. Computational performance: of exact resolution methods with dierent types of cuts Method name CPLEX cuts 2-links (User cuts) No cuts (P SL ) User cuts (PSL 2links ) CPLEX cuts CPLEX & user cuts 10 / 19

31 Random instances: Generation Same generation procedure as Buchheim, Rinaldi ([1]). Input: n (variables), m (monomials). 11 / 19

32 Random instances: Generation Same generation procedure as Buchheim, Rinaldi ([1]). Input: n (variables), m (monomials). Monomials are always generated by randomly choosing the variables in it and the coecient. 11 / 19

33 Random instances: Generation Same generation procedure as Buchheim, Rinaldi ([1]). Input: n (variables), m (monomials). Monomials are always generated by randomly choosing the variables in it and the coecient. 1 Same-degree. All monomials have the same degree d = 3 or / 19

34 Random instances: Generation Same generation procedure as Buchheim, Rinaldi ([1]). Input: n (variables), m (monomials). Monomials are always generated by randomly choosing the variables in it and the coecient. 1 Same-degree. All monomials have the same degree d = 3 or 4. 2 Random-degree. The degree d of each monomial is chosen with probability 2 d 1 (less high-degree monomials, more low-degree monomials). 11 / 19

35 Random instances: Generation Same generation procedure as Buchheim, Rinaldi ([1]). Input: n (variables), m (monomials). Monomials are always generated by randomly choosing the variables in it and the coecient. 1 Same-degree. All monomials have the same degree d = 3 or 4. 2 Random-degree. The degree d of each monomial is chosen with probability 2 d 1 (less high-degree monomials, more low-degree monomials). 11 / 19

36 Random instances: Results Instance LP gap (%) IP execution times (secs) d n m P SL PSL 2links no cuts user cplex c & u >3600 > Table: Results for random (same-degree) instances 12 / 19

37 Random instances: Results Instance LP gap (%) IP execution times (secs) d n m P SL PSL 2links no cuts user cplex c & u Table: Results for random (random-degree) instances 13 / 19

38 Vision instances: Idea Figure: Image from "Corel database" with additive Gaussian noise [6]. Figure: Restoration of an old digitalized image with scratches [6]. 14 / 19

39 Vision instances: Input generation Base image (a) top left rect (b) centre rect (c) cross Figure: Base images: size Perturbation: None, Low (change any pixel with probability 5%), High (change zero's with probability 50%). 15 / 19

40 Vision instances: Input generation Base image (a) top left rect (b) centre rect (c) cross Figure: Base images: size Perturbation: None, Low (change any pixel with probability 5%), High (change zero's with probability 50%). 15 / 19

41 Vision instances: Image restoration model Input: p ij {0, 1} value of pixel (i, j) in the input image. Variables: x ij {0, 1} value assigned to pixel (i, j) in the output. Objective function: min f (x) = L(x) + P(x) 16 / 19

42 Vision instances: Image restoration model Input: p ij {0, 1} value of pixel (i, j) in the input image. Variables: x ij {0, 1} value assigned to pixel (i, j) in the output. Objective function: min f (x) = L(x) + P(x) 1 Similarity between input and output L(x) = 25(p ij x ij ) 2 (linear). 16 / 19

43 Vision instances: Image restoration model Input: p ij {0, 1} value of pixel (i, j) in the input image. Variables: x ij {0, 1} value assigned to pixel (i, j) in the output. Objective function: min f (x) = L(x) + P(x) 1 Similarity between input and output L(x) = 25(p ij x ij ) 2 (linear). 2 Smoothness of the image (polynomial: 2 2 windows - degree 4). 16 / 19

44 Vision instances: Image restoration model Input: p ij {0, 1} value of pixel (i, j) in the input image. Variables: x ij {0, 1} value assigned to pixel (i, j) in the output. Objective function: min f (x) = L(x) + P(x) 1 Similarity between input and output L(x) = 25(p ij x ij ) 2 (linear). 2 Smoothness of the image (polynomial: 2 2 windows - degree 4). Window assignments Penalty / 19

45 Vision instances: Image restoration model Input: p ij {0, 1} value of pixel (i, j) in the input image. Variables: x ij {0, 1} value assigned to pixel (i, j) in the output. Objective function: min f (x) = L(x) + P(x) 1 Similarity between input and output L(x) = 25(p ij x ij ) 2 (linear). 2 Smoothness of the image (polynomial: 2 2 windows - degree 4). Window assignments Penalty / 19

46 Vision instances: Results Instance (10 15) LP gap (%) IP execution times (secs) Base image Perturbation P SL PSL 2links no cuts user cplex c & u top left rect none > 3600 > top left rect low > 3600 > top left rect high > 3600 > centre rect none > 3600 > centre rect low > 3600 > centre rect high > 3600 > cross none > 3600 > cross low > 3600 > cross high > 3600 > Table: Results for vision instances of size 10 15, n = 150 m = / 19

47 Vision instances: Results Instance (15 15) LP gap (%) IP execution times (secs) Base image Perturbation P SL PSL 2links no cuts user cplex c & u top left rect none > 3600 > top left rect low > 3600 > top left rect high > 3600 > centre rect none > 3600 > centre rect low > 3600 > centre rect high > 3600 > cross none > 3600 > cross low > 3600 > cross high > 3600 > Table: Results for vision instances of size 15 15, n = 225 m = / 19

48 Main contributions: Denition of the 2-links to strengthen P SL linearization. For f containing two intersecting non-linear monomials, PSL 2links complete description. is a 19 / 19

49 Main contributions: Denition of the 2-links to strengthen P SL linearization. For f containing two intersecting non-linear monomials, PSL 2links complete description. for the general case: is a 19 / 19

50 Main contributions: Denition of the 2-links to strengthen P SL linearization. For f containing two intersecting non-linear monomials, PSL 2links is a complete description. for the general case: Random instances: results depend on ratio m n, adding 2-links helps. 19 / 19

51 Main contributions: Denition of the 2-links to strengthen P SL linearization. For f containing two intersecting non-linear monomials, PSL 2links is a complete description. for the general case: Random instances: results depend on ratio m n, adding 2-links helps. Vision instances: CPLEX cuts make the dierence from infeasible to feasible. Adding 2-links always improves even more. 19 / 19

52 Main contributions: Denition of the 2-links to strengthen P SL linearization. For f containing two intersecting non-linear monomials, PSL 2links is a complete description. for the general case: Open questions: Random instances: results depend on ratio m n, adding 2-links helps. Vision instances: CPLEX cuts make the dierence from infeasible to feasible. Adding 2-links always improves even more. Other problem structures for which 2-links can help computationally? Provide a complete description? When is SL enough? 19 / 19

53 Main contributions: Denition of the 2-links to strengthen P SL linearization. For f containing two intersecting non-linear monomials, PSL 2links is a complete description. for the general case: Open questions: Random instances: results depend on ratio m n, adding 2-links helps. Vision instances: CPLEX cuts make the dierence from infeasible to feasible. Adding 2-links always improves even more. Other problem structures for which 2-links can help computationally? Provide a complete description? When is SL enough? Similar inequalities linking 3 terms? 19 / 19

54 Main contributions: Denition of the 2-links to strengthen P SL linearization. For f containing two intersecting non-linear monomials, PSL 2links is a complete description. for the general case: Open questions: Random instances: results depend on ratio m n, adding 2-links helps. Vision instances: CPLEX cuts make the dierence from infeasible to feasible. Adding 2-links always improves even more. Other problem structures for which 2-links can help computationally? Provide a complete description? When is SL enough? Similar inequalities linking 3 terms? 19 / 19

55 Some references I C. Buchheim and G. Rinaldi. Ecient reduction of polynomial zero-one optimization to the quadratic case. SIAM Journal on Optimization, 18(4): , Y. Crama and P. L. Hammer. Boolean functions: Theory, algorithms, and applications. Cambridge University Press, New York, N. Y., R. Fortet. L'algèbre de boole et ses applications en recherche opérationelle. Cahiers du Centre d'études de recherche opérationelle, 4:536, F. Glover and E. Woolsey. Further reduction of zero-one polynomial programming problems to zero-one linear programming problems. Operations Research, 21(1):156161, P. L. Hammer, I. Rosenberg, and S. Rudeanu. On the determination of the minima of pseudo-boolean functions. Studii si Cercetari Matematice, 14:359364, in Romanian.

56 Some references II S. Roth. High-Order Markov Random Fields for Low-Level Vision. PhD thesis, Brown University, Providence, Rhode Island, USA, 2007.