RC386 (W060-09) February, 006 Mathematics IBM Research Report A Feasibility Pump for Mied Integer Nonlinear Programs Pierre Bonami, Gérard Cornuéjols, Andrea Lodi*, François Margot Tepper School of Business Carnegie Mellon University Pittsburgh, PA *DEIS University of Bologna viale Risorgimento 4036 Bologna, Italy (Part of this research was carried out when Andrea Lodi was Herman Goldstine Fellow of the IBM T. J. Watson Research Center) Research Division Almaden - Austin - Beijing - Haifa - India - T. J. Watson - Tokyo - Zurich LIMITED DISTRIBUTION NOTICE: This report has been submitted for publication outside of IBM and will probably be copyrighted if accepted for publication. Ithas been issued as a Research Report for early dissemination of its contents. In view of the transfer of copyright to the outside publisher, its distribution outside of IBM prior to publication should be limited to peer communications and specific requests. After outside publication, requests should be filled only by reprints or legally obtained copies of the article (e.g., payment of royalties). Copies may be requested from IBM T. J. Watson Research Center, P. O. Bo 8, Yorktown Heights, NY 0598 USA (email: reports@us.ibm.com). Some reports are available on the internet at http://domino.watson.ibm.com/library/cyberdig.nsf/home.
Pierre Bonami Gérard Cornuéjols Andrea Lodi François Margot A Feasibility Pump for Mied Integer Nonlinear Programs Abstract We present an algorithm for finding a feasible solution to a conve mied integer nonlinear program. This algorithm, called Feasibility Pump, alternates between solving nonlinear programs and mied integer linear programs. We also discuss how the algorithm can be iterated so as to improve the first solution it finds, as well as its integration within an outer approimation scheme. We report computational results. Introduction Finding a good feasible solution to a Mied Integer Linear Program (MILP) can be difficult, and sometimes just finding a feasible solution is an issue. Fischetti, Glover and Lodi [6] developed a heuristic for the latter which they called Feasibility Pump. Here we propose a heuristic for finding a feasible solution for Mied Integer NonLinear Programs P. Bonami Tepper School of Business, Carnegie Mellon University, Pittsburgh PA 53, USA. Supported in part by a grant from IBM. E-mail: pbonami@andrew.cmu.edu G. Cornuéjols Tepper School of Business, Carnegie Mellon University, Pittsburgh PA 53, USA; and LIF, Faculté des Sciences de Luminy, 388 Marseille, France. Supported in part by NSF grant DMI- 035885 and ONR grant N0004-03--088. E-mail: gc0v@andrew.cmu.edu A. Lodi DEIS, University of Bologna, viale Risorgimento, 4036 Bologna, Italy. Part of this research was carried out when Andrea Lodi was Herman Goldstine Fellow of the IBM T.J. Watson Research Center whose support is gratefully acknowledged. E-mail: alodi@deis.unibo.it F. Margot Tepper School of Business, Carnegie Mellon University, Pittsburgh PA 53, USA. Supported in part by a grant from IBM and by ONR grant N0004-03--088. E-mail: fmargot@andrew.cmu.edu
Pierre Bonami et al. MINLP min f y s t : g y b n n y n where f is a function from n n to and g is a function from n to m. If a variable is nonnegative, the corresponding inequality is part of the constraints g y b. In this paper, all functions are assumed to be differentiable everywhere. For MILP (when both f and g are linear functions), the basic principle of the Feasibility Pump consists in generating a sequence of points 0 y 0 k y k that satisfy the continuous relaation. Associated with the sequence 0 y 0 k y k of integer infeasible points is a sequence ˆ ŷ ˆ k ŷ k of points which are integer feasible but do not necessarily satisfy the other constraints of the problem. Specifically, each ˆ i is the componentwise rounding of i and ŷ i y i. The sequence i y i is generated by solving a linear program whose objective function is to minimize the distance of to ˆ i according to the L norm. The two sequences have the property that at each iteration the distance between i and ˆ i is nonincreasing. This basic procedure may cycle and Fischetti, Glover and Lodi use randomization to restart the procedure. For MINLP, we construct two sequences 0 y 0 k y k and ˆ ŷ ˆ k ŷ k with the following properties. The points i y i in the first sequence satisfy g i y i b but i n. The points ˆ i ŷ i in the second sequence do not satisfy g ˆ i ŷ i b but they satisfy ˆ i n. The sequence i y i is generated by solving nonlinear programs (NLP) and the sequence ˆ i ŷ i is generated by solving MILPs. We call this procedure Feasibility Pump for MINLP and we present two versions, a basic version and an enhanced version, which we denote basic FP and enhanced FP respectively. Unlike the procedure of Fischetti, Glover and Lodi, the enhanced FP cannot cycle and it is finite when all the integer variables are bounded. The Feasibility Pump for MINLP is a heuristic in general, but when the region S : n y n : g y b! is conve, the enhanced version is an eact algorithm: either it finds a feasible solution or it proves that none eists. The paper is organized as follows. In Section we outline two versions of the Feasibility Pump for MINLP assuming that the functions g j are conve. We present the basic version of our algorithm as well as an enhanced version. In Section 3, we present the enhanced FP in the more general case where the region g y" b is conve. In Section 4, we study the convergence of these algorithms. When constraint qualification holds, we show that the basic Feasibility Pump cannot cycle. When constraint qualification does not hold, we give an eample showing that the basic FP can cycle. On the other hand, we prove that the enhanced version never cycles. It follows that, when the region g y# b is conve and the integer variables are bounded, the enhanced FP either finds a feasible solution or proves that none eists. In Section 5, we present computational results showing the effectiveness of the method. In Section 6, we discuss how the algorithm can be iterated so as to improve the first solution it finds and we report computational eperiments for such an iterated Feasibility Pump algorithm. Finally, in Section 7,
A Feasibility Pump for Mied Integer Nonlinear Programs 3 we discuss the integration of the Feasibility Pump within the Outer Approimation [4] approach and we report computational results. Feasibility Pump When the Functions g j Are Conve In this section, we consider the case where each of the functions g j is conve for j m. To construct the sequence ˆ ŷ ˆ k ŷ k, we use an Outer Approimation of the region g y b. This technique was first proposed by Duran and Grossmann [4]. It linearizes the constraints of the continuous relaation of MINLP to build a mied integer linear relaation of MINLP. Consider any feasible solution y of the continuous relaation of MINLP. By conveity of the functions g j, the constraints g j y g j y T y y are valid for MINLP. Therefore, given any set of points we can build a relaation of the feasible set of MINLP g k y k J g k y k y n n y k y k b j j m () 0 y 0 i y i!, b k 0 i where J g denotes the Jacobian matri of function g. Our basic algorithm generates ˆ i ŷ i using this relaation. The basic Feasibility Pump. Initially, we choose 0 y 0 to be an optimal solution of the continuous relaation of MINLP. More generally, when the objective f y is not important, we could start from any feasible solution of this continuous relaation. Then, for i, we start by finding a point ˆ i ŷ i in the current outer approimation of the constraints that solves FOA i min i s t : g k y k J g k y k y n n y k y k b k 0 i We then compute i y i by solving the NLP
4 Pierre Bonami et al. FP NLP i min ˆ i s t : g y b n n y The basic FP iterates between solving FOA i and FP NLP i until either a feasible solution of MINLP is found or FOA i becomes infeasible. See Figure for an illustration of the Feasibility Pump. y y ˆ ŷ g y b 0 y 0 y ˆ ŷ 0 Fig. Illustration of the Feasibility Pump (both basic and enhanced). The tangent lines to the feasible region represent outer approimation constraints () added in iterations 0 and. The enhanced Feasibility Pump (case where the functions g i are conve). In addition to the inequalities (), other valid linear inequalities for MINLP may improve the outer approimation. For eample, at iteration k 0, we have a point ˆ k ŷ k outside the conve region g y b and a point k y k on its boundary that minimizes ˆ k. Then the inequality k ˆ k T k 0 () is valid for MINLP. This is because the hyperplane that goes through k and is orthogonal to the vector k ˆ k is tangent at k to the projection of the conve region
A Feasibility Pump for Mied Integer Nonlinear Programs 5 g y b onto the -space. Furthermore this hyperplane separates ˆ k ŷ k from the conve region g y b. Therefore, we can add constraint () to FOA i for any i k. We denote by SFOA i the resulting strengthening of the outer approimation FOA i : SFOA i min i s t : g k y k J g k y k k y y k y k b k 0 i ˆ k T k 0 k i n n Let ˆ i ŷ i denote the solution found by solving SFOA i. The enhanced Feasibility Pump for MINLP when the functions g i are conve iterates between solving SFOA i and FP NLP i until either a feasible solution of MINLP is found or SFOA i becomes infeasible. 3 Feasibility Pump When the Region g y" b is Conve Let us now consider the case where the region g y b is conve but some of the functions defining it are nonconve. Assume g j is nonconve. Then, constraint () may cut off part of the feasible region in general, unless y satisfies the constraint g j y b j with equality, namely g j y b j, as proved in the net lemma. Lemma Assume the region S z n : g z# b! is conve and let z " S such that g j z b j. If g j is differentiable at z, then is valid for all z S. T g j z z z 0 Proof: Take any z S z : g j z b j! with z z. By conveity of S, we have that, for any λ 0, z λ z z S. As S z : g j z b j! and g j z b j, we get It follows that g j z λ z z g j z 0 lim λ 0 g j z λ z z λ g j z T g j z z z 0
6 Pierre Bonami et al. Lemma shows that constraint () is valid for any point y when g j is conve, and for those points y such that g j y b j when g j is nonconve. Therefore for our most general version of the Feasibility Pump, we define the program FP OA i obtained from SFOA i by using only the following subset of constraints of type () T g j y g j y b y y j j I y (3) where I y j : either g j is conve or g j y b j! m!. Thus the program FP OA i reads as follows: min i s t : g j k y k g j k y k T k y y k b j i FP OA j I k y k k 0 i k y ˆ k T k 0 k i n n OA i gives a valid outer appro- b is conve. This way of defining the constraints of FP imation of MINLP provided the region g y" The enhanced Feasibility Pump (case where the region g y b is conve). The algorithm starts with a feasible solution 0 ȳ 0 of the continuous relaation of MINLP and then iterates between solving FP OA i i and FP NLP for i until either a feasible solution of MINLP is found or FP OA i becomes infeasible. Note that in the case where the region g y b is nonconve, the method can still be applied, but the outer approimation constraints (3) are not always valid. This may result in the problem FP OA i being infeasible and the method failing while there eists some integer feasible solution to MINLP. 4 Convergence Consider a point y such that g y b. Let m! be the set of indices for which g j y b j is satisfied with equality by y. The linear independence constraint qualification (constraint qualification for short) is said to hold at y if the vectors g j y for i are linearly independent [5]. The net theorem shows that if the constraint qualification holds at each point i y i, then the basic FP cannot cycle. Theorem In the basic FP, let ˆ i ŷ i i be an optimal solution of FOA and i y i an optimal solution of FP NLP i. If the constraint qualification for FP NLP i holds at i y i, then i k for all k 0 i.
A Feasibility Pump for Mied Integer Nonlinear Programs 7 Proof: Suppose that k i for some k i (i.e. k y k is an optimal solution of FP NLP i ). Let h ˆ i. By property of the norm, ˆ i satisfies h k T ˆ i as the equality is derived from h k h k 0 (4) h. ˆ i T ˆ i, implying h ˆ i Now, as a minimizer of h over g y b satisfying the constraint qualification, the point ( k y k satisfies the KKT conditions h k T 0 λ g k y k λj g k y k (5) b 0 (6) for some λ 0. As a feasible solution of FOA i, ˆ i ŷ i satisfies the outerapproimation the constraints in k y k and therefore, since λ 0 : λg k y k λj g k y k ˆ i k ŷ i y k λb Using (5) and (6), this implies that h k T ˆ i k 0 which contradicts (4) and proves Theorem. The proof of Theorem shows that, when the constraint qualification holds, constraint () is implied by the outer approimations constraints at k y k. Note that Theorem still holds if we replace the L -norm by any L p -norm in the objective of FP NLP i. Note also that if we replace the outer approimation i constraints in FOA by constraints (3), Theorem still holds when the region g y" b is conve. Net we give an eample showing that when the constraint qualification does not hold the basic algorithm may cycle. Eample Consider the following constraint set for a 3-variable MINLP: y y 4 y 0 y 0 0! The first and third constraints imply that y 0. Figure illustrates the feasible region of the continuous relaation, namely the line segment joining the points 0 0 and 0. y (7)
8 Pierre Bonami et al. y 0 y Fig. Illustration of Eample. 0 0 Starting from the point ˆ ŷ ŷ 0, and solving FP NLP, we get the point y y 0. The Jacobian of g and g y y are given by J g y y 0 y y 0 0 0 0 0 0 0 g y y Therefore the outer approimation constraints () for the point y y are given by 4 0 0 0 0 4 0 0 0 y 0 0 0 0 y 0 0 0 Among these constraints, the last four are linear constraints already present in (7), and after simplification the first one yields y 0 Since all these constraints are satisfied by ˆ ŷ ŷ 0, this point is a feasible solution of the FOA and it is easy to verify that it is indeed optimum. Therefore we encounter a cycle. This happens since the constraint qualification does not hold at the point y y 0. (Indeed, the first and third rows of A J g y y are linearly dependent.) 4 0 0 In the net theorem, we consider the convergence of the enhanced Feasibility Pump for MINLP. In particular we prove that it cannot cycle. This is a difference with the Feasibility Pump of Fischetti, Glover and Lodi for MILP, where cycling can occur.
A Feasibility Pump for Mied Integer Nonlinear Programs 9 Theorem The enhanced Feasibility Pump cannot cycle. If the integer variables are bounded, the enhanced FP terminates in a finite number of iterations. If, in addition, the region g y" b is conve, the enhanced FP is an eact algorithm: either it finds a feasible solution of MINLP if one eists, or it proves that none eists. Proof: If for some k 0, k ȳ k is integer feasible, the Feasibility Pump terminates. So we may assume that k ȳ k is not integer feasible. Since ˆ k ŷ k k y k, the point ˆ k ŷ k does not satisfy constraint () and therefore ˆ k cannot be repeated when solving FP OA i. Thus the enhanced Feasibility Pump cannot cycle. If the integer variables are bounded, the enhanced Feasibility Pump is a finite algorithm, since there is only a finite number of possible different values for ˆ k. The last part of the theorem follows from the fact that FP OA i is a valid relaation of MINLP when the region g y" b is conve. Eample We run the enhanced Feasibility Pump starting from the point where we were stuck with the basic algorithm in Eample, namely y y 0. The corresponding inequality (), namely, is added to FOA. Solving the resulting ILP FP OA yields ˆ ŷ ŷ 0 0 0. Solving FP NLP, we get the point y y 0 0, which is feasible for (7) and we stop. Note that, although ˆ i cannot be repeated in the enhanced Feasibility Pump, the point i y i could be repeated as shown by considering a slightly modified version of Eample. Eample 3 Change the second constraint in Eample to y 0. Starting from the point ˆ ŷ ŷ 0, and solving FP NLP, we get the point y y 0. As in Eample, the constraint qualification does not hold at this point. Here inequality () is. We add this inequality to FOA to get FP OA. Solving this integer program yields ˆ ŷ ŷ 0 0 0. Solving FP NLP, we get the point y y 0, which is the same as y y. Although a point y is repeated, the enhanced Feasibility Pump does not cycle. Indeed, now inequality () is. Adding it to FOA 3 yields FP OA 3 which is infeasible. This proves that the starting MINLP is infeasible. 5 Computational Results The Feasibility Pump for MINLP has been implemented in the COIN infrastructure [] using a new framework for MINLP []. Our implementation uses to solve the nonlinear programs and to solve the mied integer linear programs. All the tests were performed on an IBM IntellistationZ Pro with an Intel Xeon 3.GHz CPU, gigabytes of RAM and running Linu Fedora Core 3. We tested the Feasibility Pump on a set of 66 conve MINLP instances gathered from different sources, and featuring applications from operations research
0 Pierre Bonami et al. and chemical engineering. Those instances are discussed in [, 8]. In these instances, the objective function and all the functions g j are conve. The basic Feasibility Pump never cycles on the instances in our test set. This means that using the enhanced FP is not necessary for these instances. Therefore all the results reported in this paper are obtained with the basic Feasibility Pump. FP OA Name previous best value time # iter value time # iter BatchS0006M 769440* 786499 0 78384 BatchS08M 45* 36499 0 43886 3 BatchS508M 54347* 69878 0 545 3 BatchS00M 95349* 40369 3640 0 CLay0304M 406.40* 5969.0 0 406.40 5 4 CLay0304H 406.40* 6509.0 0 0 406.40 7 3 CLay0305M 809.50* 9006.76 0 3 878.46 35 CLay0305H 809.50* 8646.44 0 878.46 4 3 FLay04M 54.4* 54.4 0 54.4 0 FLay04H 54.4* 54.4 0 54.4 0 FLay05M 64.50* 64.50 0 64.50 FLay05H 64.50* 64.50 0 64.50 0 FLay06M 66.93* 66.93 66.93 4 FLay06H 66.93* 66.93 0 66.93 9 fo7 7.75 7.75 7.75 9 fo7 0.73 9.94 0 0.73 3 fo8.38 38.0 0 3.9 3 fo9 3.46 49.80 3 4 4.00 96 o7 6.94 59.38 0 8.85 5650 4 o7 3.64 7.5 3 5 700 RSyn0830H -50.07* -509.49-50.07 RSyn0830M -50.07* -49.53 0-497.87 0 RSyn0830M0H -730.5* -77. 0-78.3 0 RSyn0830M0M -730.5* -663.55 0-7.45 365 RSyn0830M03H -543.06* -538.0-535.46 RSyn0830M03M -543.06* -98.98 0-53.09 44 RSyn0830M04H -59.07* -59.03-5.04 RSyn0830M04M -59.07* -436.44-50.39 584 RSyn0840H -35.55* -37.50 0-35.55 RSyn0840M -35.55* -3.4 0-35.55 0 RSyn0840M0H -734.98* -73.3 0-73.3 0 RSyn0840M0M -734.98* -599.57 0-7.98 09 RSyn0840M03H -74.65* -79.53-73.53 RSyn0840M03M -74.65* -55.9 3 3-70.8 695 RSyn0840M04H -564.50* -538.83-544.04 RSyn0840M04M -563.50* -478.67 5 3-488.87 700 Table FP vs. first solution found by OA (on the first 36 instances of the test set). Column labeled previous best gives the best known solution obtained using and and * indicates that the value is known to be optimal; columns labeled value report the objective value of the solution found, where indicates that no solution is found; columns labeled time show the CPU time in seconds rounded to the closest integer (with a maimum of hours of CPU time allowed); columns labeled # iter give the number of iterations. In order to guarantee convergence to an optimal solution in Theorems and it is important to find an optimum solution i ȳ i of FP NLP i. On the other hand, it is not necessary to obtain an optimum solution of FP OA i. In our
A Feasibility Pump for Mied Integer Nonlinear Programs FP OA Name previous best value time # iter value time # iter SLay07H 64749* 663 0 69509 0 SLay07M 64749* 6554 0 6587 0 SLay08H 84960* 9345 0 504 0 SLay08M 84960* 9849 0 9849 0 SLay09H 07806* 0858 0 5989 0 SLay09M 07806* 588 0 750 0 SLay0H 9580* 5688 0 56490 SLay0M 9580* 3640 0 6337 0 Syn30H -38.6* -.86 0 -.86 0 Syn30M -38.6* -5.9 0-5.9 0 Syn30M0H -399.68* -387.37 0-387.37 0 Syn30M0M -399.68* -386.5 0-386.5 0 Syn30M03H -654.5* -64.84 0-64.84 0 Syn30M03M -654.5* -646.05 0-646.05 Syn30M04H -865.7* -88. 0-88. 0 Syn30M04M -865.7* -85.75 0-856.05 Syn40H -67.7* -6.9 0-6.9 0 Syn40M -67.7* -55.7 0-55.7 0 Syn40M0H -388.77* -387.04 0-388.77 0 Syn40M0M -388.77* -37.48 0-376.48 Syn40M03H -395.5* -38.64 4-38.64 0 Syn40M03M -395.5* -33.69 0-354.69 4 Syn40M04H -90.75* -87.7 0-837.7 0 Syn40M04M -90.75* -765.0 0-805.70 7 trimloss 5.3* 5.3 0 3 5.3 0 6 trimloss4 8.3*.7 8.3 893 7 trimloss5 3.0 3 700 trimloss6 6.7 4 4 700 trimloss7 3. 553 700 trimloss.7 453 43 700 Table FP vs. first solution found by OA (on the remaining 30 instances). Table is the continuation of Table. Symbol * indicates that the value is known to be optimal, indicates that no solution is found. implementation of the Feasibility Pump, we do not insist on solving the MILP i FP OA to optimality. Once a feasible solution has been found and has not been improved for 5000 nodes of the branch-and-cut algorithm, we use this solution as our point ˆ i ŷ i. This reduces the amount of time spent solving MILPs and it improves the overall computing time of the Feasibility Pump. In a first eperiment, we compare the solution obtained with the Feasibility Pump to the first solution obtained by the Outer Approimation algorithm (OA for short) as implemented in [] and using and as subsolvers. Tables and summarize this comparison. The following comments can be made about the results of tables and. The Feasibility Pump finds a feasible solution in less than a second in most cases. Overall, FP is much faster than OA. Although on the instances both FP and OA require several iterations to find a feasible solution, FP is roughly ten times faster. The, and instances are particularly challenging for OA, which fails to find a feasible solution within the -hour time limit for 5 of these instances. By contrast, FP finds a feasible solution to each of them. The cases of,, and are noteworthy
Pierre Bonami et al. since no feasible solution was known prior to this work. The column previous best contains the best known solution from [3] and [9]. is an MINLP solver based on the outer approimation technique whereas is a solver based on branch-and-bound. 6 Iterating the Feasibility Pump for MINLP In the net two sections, we assume that we have a conve MINLP, that is we assume that both the region g y b and the objective function f are conve. This section investigates the heuristic obtained by iterating the FP, i.e. calling several time in a row FP, each time trying to find a solution strictly better than the last solution found. More precisely, to take into account the cost function f y of MINLP, we add to FOA i a new variable α and the constraint f y# α. Initially, the variable α is unbounded. Each time a new feasible solution with value z U to MINLP is found, the upper bound on α is decreased to z U δ for some small δ 0. As a result, the current best known feasible solution becomes infeasible and it is possible to restart FP from the optimal solution of the relaation of MINLP. Note that () is used to generate outer approimations of the conve constraint f y α. If eecuted long enough, this algorithm will ultimately find the optimal solution of MINLP and prove its optimality by application of Theorem under the assumption that the integer variables are bounded and δ is small enough. Here, we do not use it as an eact algorithm but instead we just run it for a limited time. We call this heuristic Iterated Feasibility Pump for MINLP (or IFP for short). Table 3 compares the best solutions found by iterated FP and by OA with a time limit of minute of CPU time. In our eperiments, we use δ 0 4. The following comments can be made about the results of Table 3. IFP produces good feasible solutions for all but of the instances within the -minute time limit, whereas OA fails to find a feasible solution for 5 of the instances. In terms of the quality of solutions found, OA finds a strictly better solution than IFP in 9 cases while IFP is the winner in 0 cases. OA can prove optimality of its solution in 36 instances and IFP in 30 instances. 7 Application to Outer Approimation Decomposition We now present a new variation of the Outer Approimation Decomposition algorithm of Duran and Grossmann [4] which integrates the Feasibility Pump algorithm. Duran and Grossmann assume that all the functions are conve and that the constraint qualification holds at all optimal points. Our variation of outer approimation does not need the assumption that all functions are conve, provided that the region g y" b is a conve set and that f y is conve. In this algorithm we alternate between solving four different problems. The first one is a linear outer approimation of MINLP with the original conve ob-
A Feasibility Pump for Mied Integer Nonlinear Programs 3 IFP OA IFP OA Name value value Name value value BatchS0006M 769440 * 769440 * SLay07H 64749 * 64749 * BatchS08M 45 * 45 * SLay07M 64749 * 64749 * BatchS508M 54347 * 54347 * SLay08H 84960 * 84960 * BatchS00M 978 95349 SLay08M 84960 * 84960 * CLay0304H 4079.00 406.40 * SLay09H 07983 0890 CLay0304M 4359.60 406.40 * SLay09M 07806 085 CLay0305H 809.50 * 809.50 * SLay0H 356 3640 CLay0305M 809.50 * 878.46 SLay0M 984 30447 FLay04H 54.4 * 54.4 * Syn30H -38.6 * -38.6 * FLay04M 54.4 * 54.4 * Syn30M -38.6 * -38.6 * FLay05H 64.50 64.50 Syn30M0H -399.68 * -399.68 * FLay05M 64.50 64.50 Syn30M0M -399.68 * -399.68 * FLay06H 66.93 66.93 Syn30M03H -654.5 * -654.6 * FLay06M 66.93 66.93 Syn30M03M -654.5 * -654.5 * fo7 7.75 * 7.75 Syn30M04H -865.7 * -865.7 * fo7 0.73 0.73 Syn30M04M -865.7 * -865.7 fo8.38 Syn40H -67.7 * -67.7 * fo9 8.95 Syn40M -67.7 * -67.7 * o7 4.83 Syn40M0H -388.77 * -388.77 * o7 39.87 Syn40M0M -388.77 * -388.77 * RSyn0830H -50.07 * -50.07 * Syn40M03H -395.5 * -395.5 * RSyn0830M -50.07 * -50.07 * Syn40M03M -395.06-395.5 RSyn0830M0H -730.5-730.5 * Syn40M04H -87.7-90.75 * RSyn0830M0M -730.5 Syn40M04M -864.76-90.75 RSyn0830M03H -54.84-543.06 * trimloss 5.3 * 5.3 * RSyn0830M03M -543.06 trimloss4 8.3 RSyn0830M04H -57.86-59.07 * trimloss5.8 RSyn0830M04M -50.88 trimloss6 6.7 RSyn0840H -35.55 * -35.55 * trimloss7 RSyn0840M -35.55 * -35.55 * trimloss RSyn0840M0H -734.98-734.98 * RSyn0840M0M -734.98 RSyn0840M03H -74.65-74.65 * RSyn0840M03M -74.65 RSyn0840M04H -56.37-564.50 * RSyn0840M04M -499.9 Table 3 IFP vs. OA (at most minute of CPU time). Columns labeled value report the objective value of the solution found; symbol * denotes proven optimality and indicates that no solution is found. jective function f y being linearized as well: minα s t : f k y k T k y y k α f k y k i OA g j k y k T k y y k b j g j k y k k k 0 i j I k y k k 0 i ˆ k T k 0 k K n n y α
4 Pierre Bonami et al. where I y j : either g j is conve or g j y b j! as defined in Section 3 and K i! is the subset of iterations where k y k is obtained by solving FP NLP k and k ˆ k. Note that OA i is an MILP. The second problem is MINLP with fied: min f ˆ i y s t : i NLP g ˆ i y b n y Note that NLP i is a nonlinear program. The third one is FP NLP i as defined in Section. Recall that this is a nonlinear program. The fourth one is the following MILP which looks for a better solution than the best found so far: min i s t : f k y k T k y y k f k y k z U δ k 0 i i FP OA g j k y k T k y y k b j g j k y k k j I k y k k 0 i ˆ k T k 0 k K n n y where z U is the current upper bound on the value of MINLP and δ 0 is a small value indicating the desired improvement in objective function value. As in OA k, K denotes the subset of iterations where k y k is obtained by solving FP NLP k and k ˆ k. The overall algorithm is given in Figure 3. A cursory description of its steps is as follows: Initially, we solve the continuous relaation of MINLP to optimality to obtain a starting point 0 y 0. Then, we compute the optimum ˆ ŷ of OA where K /0. Similarly, at subsequent iterations, we obtain ˆ i ŷ i and a lower bound ˆα i on the value of MINLP by solving OA i. We then solve NLP i with fied at ˆ i. If NLP i is feasible, then i y i given by i ˆ i and y i is the optimal solution of NLP i. Moreover, f i y i is an upper bound on the optimal solution of MINLP. Otherwise, NLP i is infeasible and we perform at least one iteration of the Feasibility Pump. More precisely, FP starts by solving FP NLP i, obtaining the point i y i. Then ˆ i ŷ i is obtained by solving FP OA i. Additional iterations of the Feasibility Pump are possibly performed, solving alternatively FP NLP and FP OA until either a better feasible solution of MINLP is found, or a proof is obtained that no such solution eists, or some iteration or time limit is reached (five iterations and at most two minutes CPU time are used in the eperiments). If no better feasible solution of MINLP eists, the algorithm
A Feasibility Pump for Mied Integer Nonlinear Programs 5 terminates. In the other two cases, a sequence of points k y k is generated for k i l. In the case where a better feasible solution l y l is found by FP NLP l, i.e. FP NLP l has objective value zero, we solve NLP l to check whether there eists a better solution than l y l with respect to the original objective function f. If we find such an improved solution we replace y l by this solution. In both cases, the algorithm reverts to solving OA l with all the outer approimation constraints generated for the points i y i l y l!. The algorithm continues iterating between the four problems as described above. The algorithm terminates when the lower bound given by OA and the best upper bound found are equal within a specified tolerance ε. z U : ; z L : ; 0 y 0 : optimal solution of the continuous relaation of MINLP; K : /0; i : ; Choose convergence parameters ε and δ while z U z L ε do Let ˆα i ˆ i ŷ i be the optimal solution of OA i ; z L : ˆα i ; if NLP i is feasible; then Let i : ˆ and y i be the optimal solution to NLP i ; z U : min z U f i y i ; i : i ; else Let i y i be the optimal solution of FP NLP i ; l : i while l ˆ l l i 5 od fi od i : l time in this FP minutes do if FP OA l is feasible fi then Let ˆ l ŷ l be the optimal solution of FP OA l ; Let l y l be the optimal solution of FP NLP l ; if l ˆ l then replace y l by the optimal solution of NLP l ; z U : min z U f l y l ; else K : K l ; fi l : l ; else z U is optimal, eit; Fig. 3 Enhanced Outer Approimation algorithm.
6 Pierre Bonami et al. The integration of Feasibility Pump into the Outer Approimation algorithm enhances the behavior of OA for instances with conve feasible sets defined by nonconve constraints. This is shown by the following eample (see also Figure 4). Eample 4 min s t : y y sin y sin 0! y 5π 3 5π 3 Although the constraints of the above problem are nonconve, the feasible region 0 0 Fig. 4 Illustration of Eample 4. of the continuous relaation is conve. Solving the continuous relaation gives the point 0 y 0 0 3. The first outer approimation problem, namely OA ma y : y 0!!, has solution ˆ ŷ. At this point, the classical OA tries to minimize the violation of the constraints over the line selecting the point 0. Then it adds the following two constraints 5π 6 y 5π 6 3 and 5π 6 y 5π 6 which make the corresponding MILP infeasible. This occurs because the point 0 is taken outside of the feasible region of the continuous relaation and, since the constraints are not defined by conve functions, generating the supporting hyperplanes at this point induces invalid constraints. On the other hand, the OA enhanced by FP NLP warm starting from ˆ ŷ solves FP NLP finding the closest NLP feasible point y 0 6 0 to ˆ in L -norm. Starting from such a point FP OA converges to ˆ ŷ 0 0 which is integer and NLP feasible (as proved by the net FP NLP iteration). 3
A Feasibility Pump for Mied Integer Nonlinear Programs 7 The behavior outlined by the previous eample is generalized by the following theorem. Theorem 3 Consider an MINLP with conve objective function and conve region g y b. Assume that the integer variables are bounded. If the constraint qualification holds at every optimal solution of NLP i, the modified version of OA converges to an optimal solution or proves that none eists even when the conve region g y" b is defined by nonconve constraints. Proof: If ˆ i ˆ k for all i k with i k then the algorithm terminates in a finite number of iterations since the integer variables are bounded. Furthermore the algorithm finds an optimum solution of MINLP or proves that none eists since the constraints of OA i and FP OA i are valid outer approimations of MINLP. Now suppose that ˆ i ŷ i satisfies ˆ i ˆ k for some k i. We consider two cases. First suppose that ˆ k was obtained by solving OA k. Then we claim that NLP k is feasible, for suppose not. Then the algorithm calls the Feasibility Pump. Since NLP k is infeasible, the first iteration of FP finds a point k ˆ k. Therefore, k K and constraint k ˆ k T ˆ k 0 has been added. This implies that ˆ k can never be repeated, a contradiction. This proves the claim. Second, suppose that ˆ k was obtained by solving FP OA k. Then k is obtained by solving FP NLP k. If k ˆ k then, by the argument used above, ˆ k cannot be repeated, a contradiction. Therefore k ˆ k which implies that NLP k is feasible. Thus in both cases NLP k is feasible, k ˆ k and y k is the optimum solution of NLP k. Since y k is optimal for NLP k and the constraint qualification holds, y k satisfies the KKT conditions: y f k y k λ j g j k y k λ 0 m j b j λ j y g j k y k 0 j m where y g j denotes the gradient of g j with respect to the y variables. Since i k and ˆ i ŷ i i i is feasible for OA or FP OA the following outer approimation constraints are satisfied by ˆ i ŷ i : Since ˆ i g j k y k k we have T ˆ i ŷ i k y k b j g j k y k j I k y k y g j k y k T ŷ i y k b j g j k y k j I k y k Using the above KKT conditions, this implies that y f k y k T ŷ i y k 0 (8) There are now two possible cases. First, if ˆ i ŷ i is a solution of OA i, it satisfies the inequality f k y k T 0 ŷ i y k ˆα i f k y k
8 Pierre Bonami et al. Then equation (8) implies that α i f k y k showing that k y k is an optimum solution, and therefore the algorithm terminates with the correct answer. In the second case, ˆ i ŷ i is a solution of FP OA i. Then it satisfies f k y k T 0 ŷ i y k f k y k z U and equation (8) implies that z U δ f k y k. This is a contradiction to the facts that k y k is feasible to MINLP and that z U is the best known upper bound. δ Computational eperiments. In Table 4 we report computational results comparing the OA with the enhanced OA coupled with FP (OA+FP). The latter is implemented as a modification of the OA algorithm implemented in [], which is used as the OA code. Our procedure is set as follows. We start by performing one minute of iterated Feasibility Pump in order to find a good solution. We then start the enhanced OA algorithm. Since the primary goal of the FP in the enhanced OA is to quickly find improved feasible solutions, we put a limit of two minutes and five iterations for each call to the FP inside the enhanced OA. OA+FP OA Name ub tub lb tlb ub tub lb tlb CLay0304M 406.4 79 * 8 406.4 * 4 CLay0305H 809.5 4 * 3 809.5 4 * 4 CLay0305M 809.5 4 * 4 809.5 75 * 75 fo7 7.75 4 * 03 7.75 0 * 8 fo7 0.73 60 * 60 0.73 4 * 97 fo8.38 573 * 835.38 77 * 906 fo9 3.46 60 * 63 3.46 535 * 604 o7 6.94 89 * 3 8.86 565 4.08 700 o7 3.64 5 * 6055.79 700 RSyn0830M0M -730.5 * 78-730.5 3837 * 57 RSyn0830M03M -543.06 5 * 08-538.9 5933-548.46 700 RSyn0830M04M -50.88 46-3067.54 700-50.39 5697-36.9 700 RSyn0840M0M -734.98 383 * 383-734.98 846 * 846 RSyn0840M03M -74.65 348 * 348-734.53 700-789.93 700 RSyn0840M04M -556.60 4-638.63 700-488.87 700-3599.77 700 SLay0M 9580 778 * 34 9580 336 853 700 trimloss4 8.3 0 * 43 8.3 785 * 785 trimloss5 0.7 485 3.3 700 5.9 700 trimloss6 6.5 040 3.5 700 6.5 700 trimloss7 7.5 387.6 700 3.3 700 trimloss 5.4 700 9.6 700 Table 4 Comparison between OA and its enhanced version on a subset of difficult instances. Columns labeled ub and lb report the upper and lower bound values; columns labeled tub and tlb give the CPU time in seconds for obtaining those upper and lower bounds; symbol * denotes proven optimality and indicates that no solution is found. The results of Table 4 show that OA+FP can solve 5 instances whereas the classical OA algorithm solves only 0 within the -hour time limit. Furthermore, OA+FP finds a feasible solution in all but one instance, whereas the classical OA
A Feasibility Pump for Mied Integer Nonlinear Programs 9 algorithm fails to find a feasible solution in 5 cases. In addition to being more robust, OA+FP is also competitive in terms of computing time, on most instances. Acknowledgements We would like to thank all the members of the IBM-CMU working group on MINLP (Larry Biegler, Andy Conn, Ignacio Grossmann, Carl Laird, Jon Lee, Nick Sawaya and Andreas Waechter) for numerous discussions on this topic. Special thanks to Andy Conn for his helpful comments on an earlier draft of the paper. References. P. Bonami, A. Wächter, L.T. Biegler, A.R. Conn, G. Cornuéjols, I.E. Grossmann, C.D. Laird, J. Lee, A. Lodi, F. Margot and N.W. Sawaya. An algorithmic framework for conve mied integer nonlinear programs. Technical Report RC377, IBM T.J. Watson Research Center, 005.. COIN-OR. 3..! 4. M. Duran and I.E. Grossmann. An outer-approimation algorithm for a class of miedinteger nonlinear programs. Mathematical Programming, 36:307 339, 986. 5. A.V. Fiacco, G.P. McCormick, Nonlinear Programming: Sequential Unconstrained Minimization Techniques, Wiley & Sons, 968; republished 990, SIAM Philadelphia. 6. M. Fischetti, F. Glover and A. Lodi. The Feasibility Pump. Mathematical Programming, 04:9 04, 004. 7. R. Fletcher and S. Leyffer. Solving mied integer nonlinear programs by outer approimation. Mathematical Programming, 66:37 349, 994. 8. N.W. Sawaya, C.D. Laird and P. Bonami. A novel library of nonlinear mied-integer and generalized disjunctive programming problems. In preparation, 006. 9.. " ## $ %!