Feasibility Pump for Mixed Integer Nonlinear Programs 1

Transcription

1 Feasibility Pump for Mixed Integer Nonlinear Programs 1 Presenter: 1 by Pierre Bonami, Gerard Cornuejols, Andrea Lodi and Francois Margot

2 Mixed Integer Linear or Nonlinear Programs (MILP/MINLP) Optimize objective function: min f (x, y) f :R n 1 R n 2 R

3 Mixed Integer Linear or Nonlinear Programs (MILP/MINLP) Optimize objective function: Subject to constraints: min f (x, y) f :R n 1 R n 2 R g(x, y) b, g : R n 1 R n 2 R m x Z n 1 y R n 2

4 The relationship between MINLP, NLP and MILP 2 If f and g are linear functions MILP. If all variables are continuous NLP. 2 A Tutorial on Mixed-Integer Non-Linear Programming by A. Letchfold, 2010

5 Mixed Integer Nonlinear Programs Applications 3 best-subset multiple linear regression various portfolio optimization problems robust versions of MILPs machine scheduling problems with min-variance objective inventory-routing problems design of networks for electricity transmission design of chemical processes scheduling gas- or coal-fired power stations 3 A Tutorial on Mixed-Integer Non-Linear Programming by A. Letchfold, 2010

6 Methods for Solving MINLP Branch and bound method (BB) (Gupta and Ravindran, 1985; Nabar and Schrage, 1991; Borchers and Mitchell, 1994; Stubbs and Mehrotra, 1996; Leyffer, 1998). Generalized Benders Decomposition (GBD) (Geoffrion, 1972). Outer-Approximation (OA) (Duran and Grossmann, 1986; Yuan et al., 1988; Fletcher and Leyffer, 1994). LP/NLP based branch and bound (Quesada and Grossmann, 1992). Extended Cutting Plane Method (ECP) (Westerlund and Pettersson, 1995).

7 A Feasibility Pump for Mixed Integer Linear Program (MILP) Generate a sequence of integer infeasible points ( x 0, ȳ 0 ),..., ( x k, ȳ k ) that satisfy the continuous relaxation.

8 A Feasibility Pump for Mixed Integer Linear Program (MILP) Generate a sequence of integer infeasible points ( x 0, ȳ 0 ),..., ( x k, ȳ k ) that satisfy the continuous relaxation. Sequence of integer feasible points (ˆx 1, ŷ 1 ),..., (ˆx k+1, ŷ k+1 ) that do not necessarily satisfy all constraints (ˆx i+1 is a componentwise rounding of x i ; ŷ i+1 = ȳ i ).

9 A Feasibility Pump for Mixed Integer Linear Program (MILP) Generate a sequence of integer infeasible points ( x 0, ȳ 0 ),..., ( x k, ȳ k ) that satisfy the continuous relaxation. Sequence of integer feasible points (ˆx 1, ŷ 1 ),..., (ˆx k+1, ŷ k+1 ) that do not necessarily satisfy all constraints (ˆx i+1 is a componentwise rounding of x i ; ŷ i+1 = ȳ i ). Sequence ( x i, ȳ i ) is generated by solving LP whose objective is to minimize the distance between x to ˆx i according to L 1 norm.

10 A Feasibility Pump for Mixed Integer Linear Program (MILP) Generate a sequence of integer infeasible points ( x 0, ȳ 0 ),..., ( x k, ȳ k ) that satisfy the continuous relaxation. Sequence of integer feasible points (ˆx 1, ŷ 1 ),..., (ˆx k+1, ŷ k+1 ) that do not necessarily satisfy all constraints (ˆx i+1 is a componentwise rounding of x i ; ŷ i+1 = ȳ i ). Sequence ( x i, ȳ i ) is generated by solving LP whose objective is to minimize the distance between x to ˆx i according to L 1 norm. Two sequences has the property: at each iteration the distance between x i to ˆx i is non-increasing (the procedure may cycle so need to use random restarts).

11 A Feasibility Pump for Mixed Integer Nonlinear Program (MINLP) Construct two sequences: s 1 = ( x 0, ȳ 0 ),..., ( x k, ȳ k ) s 2 = (ˆx 1, ŷ 1 ),..., (ˆx k+1, ŷ k+1 )

12 A Feasibility Pump for Mixed Integer Nonlinear Program (MINLP) Construct two sequences: s 1 = ( x 0, ȳ 0 ),..., ( x k, ȳ k ) s 2 = (ˆx 1, ŷ 1 ),..., (ˆx k+1, ŷ k+1 ) that satisfy the properties: - points ( x i, ȳ i ) satisfy g( x i, ȳ i ) b, but x i is "not necessarily" in Z n1 - points (ˆx i, ŷ i ) do not satisfy g(ˆx i, ŷ i ) b, but ˆx i Z n1

13 A Feasibility Pump for Mixed Integer Nonlinear Program (MINLP) Construct two sequences: s 1 = ( x 0, ȳ 0 ),..., ( x k, ȳ k ) s 2 = (ˆx 1, ŷ 1 ),..., (ˆx k+1, ŷ k+1 ) that satisfy the properties: - points ( x i, ȳ i ) satisfy g( x i, ȳ i ) b, but x i is "not necessarily" in Z n1 - points (ˆx i, ŷ i ) do not satisfy g(ˆx i, ŷ i ) b, but ˆx i Z n1 ( x i, ȳ i ) is generated by solving NLPs

14 A Feasibility Pump for Mixed Integer Nonlinear Program (MINLP) Construct two sequences: s 1 = ( x 0, ȳ 0 ),..., ( x k, ȳ k ) s 2 = (ˆx 1, ŷ 1 ),..., (ˆx k+1, ŷ k+1 ) that satisfy the properties: - points ( x i, ȳ i ) satisfy g( x i, ȳ i ) b, but x i is "not necessarily" in Z n1 - points (ˆx i, ŷ i ) do not satisfy g(ˆx i, ŷ i ) b, but ˆx i Z n1 ( x i, ȳ i ) is generated by solving NLPs (ˆx i, ŷ i ) is generated by solving MILPs

15 Feasibility Pump When the Functions g j are Convex Construct the sequence (ˆx 1, ŷ 1 ),..., (ˆx k+1, ŷ k+1 ) by Outer Approximation of the region g(x, y) b. It linearizes the constraints of the continuous relaxation of MINLP to build a mixed integer linear relaxation of MINLP.

16 Continuous Relaxation of MINLP ( x, ȳ) is a feasible solution and g j is convex, so the constraints are valid: (( ) )) g j ( x, ȳ) + g j ( x, ȳ) T x ( x b y ȳ j, j = 1,..., m

17 Continuous Relaxation of MINLP ( x, ȳ) is a feasible solution and g j is convex, so the constraints are valid: (( ) )) g j ( x, ȳ) + g j ( x, ȳ) T x ( x b y ȳ j, j = 1,..., m Therefore, given any set of points ( x 0, ȳ 0 ),..., ( x i 1, ȳ i 1 ) can build a relaxation of the feasible set of MINLP: (( ) )) x ( x g( x k, ȳ k ) + J g ( x k, ȳ k ) T k y ȳ k b, k = 0,..., i 1 x Z n 1 y R n 2

18 Continuous Relaxation of MINLP ( x, ȳ) is a feasible solution and g j is convex, so the constraints are valid: (( ) )) g j ( x, ȳ) + g j ( x, ȳ) T x ( x b y ȳ j, j = 1,..., m Therefore, given any set of points ( x 0, ȳ 0 ),..., ( x i 1, ȳ i 1 ) can build a relaxation of the feasible set of MINLP: (( ) )) x ( x g( x k, ȳ k ) + J g ( x k, ȳ k ) T k y ȳ k b, k = 0,..., i 1 x Z n 1 y R n 2 Solution: (ˆx i, ŷ i ).

19 The Basic Feasibility Pump: Step 1 Step 1: Feasibility Outer Approximation (FOA) i Choose ( x 0, ȳ 0 ) to be an optimal solution of the continuous relaxation of MINLP.

20 The Basic Feasibility Pump: Step 1 Step 1: Feasibility Outer Approximation (FOA) i Choose ( x 0, ȳ 0 ) to be an optimal solution of the continuous relaxation of MINLP. For i 1 find a point (ˆx i, ŷ i ) that solves: min (x x i 1 ) 1 s.t. (( ) )) x ( x g( x k, ȳ k ) + J g ( x k, ȳ k ) T k y ȳ k b, k = 0,..., i 1 x Z n 1 y R n 2

21 The Basic Feasibility Pump: Step 2 Step 2: Feasibility Pump - NonLinear Programs (FP NLP) i Compute ( x i, ȳ i ) by solving NLP: min (x ˆx i ) 2 s.t. g(x, y) b x R n 1 y R n 2

22 The Basic Feasibility Pump: Step 2 Step 2: Feasibility Pump - NonLinear Programs (FP NLP) i Compute ( x i, ȳ i ) by solving NLP: min (x ˆx i ) 2 s.t. g(x, y) b x R n 1 y R n 2 The basic FP iterates between solving (FOA) i and (FP NLP) i until either a feasible solution of MINLP is found or (FOA) i becomes infeasible.

23 The basic FP iterates between solving FOA and FP NLP until either a feasible solution of MINLP is found or FOA i becomes infeasible. See Figure 1 Illustration for an illustration of The of the Feasibility Pump. Pump y x 1 y 1 ˆx 1 ŷ 1 g x y b x 0 y 0 x 2 y 2 ˆx 2 ŷ x

24 Enhanced Feasibility Pump (SFOA) i At iteration k > 0 we have a point (ˆx k, ŷ k ) outside the convex region g(x, y) b and a point ( x k, ȳ k ) on its boundary that minimizes (x x k ) 2. Then ( x k ˆx k ) T (x x k ) 0 is valid for MINLP.

25 Enhanced Feasibility Pump (SFOA) i At iteration k > 0 we have a point (ˆx k, ŷ k ) outside the convex region g(x, y) b and a point ( x k, ȳ k ) on its boundary that minimizes (x x k ) 2. Then ( x k ˆx k ) T (x x k ) 0 is valid for MINLP. Add additional constraint: min (x x i 1 ) 1 s.t. (( ) )) x ( x g( x k, ȳ k ) + J g ( x k, ȳ k ) T k y ȳ k b, k = 0,..., i 1 ( x k ˆx k ) T (x x k ) 0, k = 1,..., i 1 x Z n 1 y R n 2

26 Enhanced Feasibility Pump (SFOA) i In general, FP is a heuristic, but when the region S := {(x, y) R n 1 R n 2 : g(x, y) b} is convex, the enhanced FP is an exact algorithm.

27 Enhanced Feasibility Pump (SFOA) i In general, FP is a heuristic, but when the region S := {(x, y) R n 1 R n 2 : g(x, y) b} is convex, the enhanced FP is an exact algorithm. When the region g(x, y) b is convex and the integer variables x are bounded, the enhanced FP either finds a feasible solution or proves that none exists.

28 Feasibility Pump Algorithm 1: i = 0; 2: initialize ( x 0, ȳ 0 ) and (ˆx 0, ŷ 0 )); 3: while ( ( x i, ȳ i ) (ˆx i, ŷ i ) and CPU time < limit) do 4: increase i; 5: solve (FOA i )/(SFOA i ) (minimize distance to ( x i, ȳ i ) s.t. x Z n 1, y R n 2 ) to yield (ˆx i, ŷ i ); 6: solve (FP NLP) i (minimize distance to (ˆx i 1, ŷ i 1 ) s.t. x R n 1, y R n 2 ) to yield ( x i, ȳ i ); 7: end while.

29 Feasibility Pump When the Region g(x, y) b is convex If g(x, y) b is convex but some of the functions are nonconvex (e.g., g j ), then constraint may cut off the part of the feasible region unless ( x, ȳ) satisfies the constraint g j (x, y) b j with equality g j (x, y) = b j.

30 Feasibility Pump When the Region g(x, y) b is convex If g(x, y) b is convex but some of the functions are nonconvex (e.g., g j ), then constraint may cut off the part of the feasible region unless ( x, ȳ) satisfies the constraint g j (x, y) b j with equality g j (x, y) = b j. The constraint is valid for any point ( x, ȳ) when g j is convex: (( ) )) g j ( x, ȳ) + g j ( x, ȳ) T x ( x b y ȳ j, j I( x, ȳ), where I( x, ȳ) = {j : either g j is convex or g j ( x, ȳ) = b j } {1,..., m}.

31 Feasibility Pump (FP OA) i When the Region g(x, y) b is Convex min (x x i 1 ) 1 s.t. (( ) )) x ( x g j ( x k, ȳ k ) + g j ( x k, ȳ k ) T k y ȳ k b j, x Z n 1 y R n 2 j I( x, ȳ), k = 0,..., i 1 ( x k ˆx k ) T (x x k ) 0, k = 1,..., i 1 Constraints (FP OA) i give a valid outer approximation of MINLP given the region g(x, y) b is convex.

32 Enhanced Feasibility Pump (FP OA) i When the Region g(x, y) b is Convex Starts with feasible solution ( x 0, ȳ 0 ) of the continuous relaxation of MILNP.

33 Enhanced Feasibility Pump (FP OA) i When the Region g(x, y) b is Convex Starts with feasible solution ( x 0, ȳ 0 ) of the continuous relaxation of MILNP. Iterates solving (FP OA) i and (FP NLP) i for i 1 until either a feasible solution of MINLP is found or (FP OA) i becomes infeasible.

34 Basic Feasibility Pump Convergence If linearly independent constraint qualification holds at each point ( x i, ȳ i ), then the basic FP cannot cycle. Constraint qualification holds if the vectors g j ( x i, ȳ i ) are linearly independent for all i J, where J {1,, m} is a set of indices for for which g j (x, y) b j is satisfied with equality by ( x, ȳ).

35 Basic Feasibility Pump Convergence If linearly independent constraint qualification holds at each point ( x i, ȳ i ), then the basic FP cannot cycle. Constraint qualification holds if the vectors g j ( x i, ȳ i ) are linearly independent for all i J, where J {1,, m} is a set of indices for for which g j (x, y) b j is satisfied with equality by ( x, ȳ). Theorem In the basic FP, let (ˆx i, ŷ i ) be an optimal solution of (FOA) i and ( x i, ȳ i ) an optimal solution of (FP NLP) i. If the constraint qualification for (FP NLP i ) holds at ( x i, ȳ i ), then x i x k for all k = 0,..., m.

36 Enhanced Feasibility Pump Convergence Theorem The enhanced Feasibility Pump cannot cycle. If the integer variables x are bounded, the enhanced FP terminates in a finite number of iterations. If, in addition, the region g(x, y) b is convex, the enhanced FP is an exact algorithm: either it finds a feasible solution of MINLP if one exists, or it proves that none exists.

37 Conclusions and Opened Research Questions Fact MINLPs are not only NP-hard, they are worse than NP-hard! MINLP has many applications MINLP are much more difficult to solve than NLP and MILP Several methods are available for convex case Nonconvex problems Need more representative real-world test problems