Using Interior-Point Methods within Mixed-Integer Nonlinear Programming

Size: px

Start display at page:

Download "Using Interior-Point Methods within Mixed-Integer Nonlinear Programming"

Jonas Arnold
5 years ago
Views:

1 Using Interior-Point Methods within Mixed-Integer Nonlinear Programming. Hande Y. Benson Drexel University IMA - MINLP - p. 1/34

2 Motivation: Discrete Variables Motivation: Discrete Variables Interior-Point Methods Handling discrete variables generally requires a bilevel approach: Upper level: Branch-and-bound, branch-and-cut, outer approximation Lower level: Active-set methods, interior-point methods Active-set methods are considered superior at the lower level because They can be warmstarted They can identify infeasible problems They can handle fixed variables naturally A textbook interior-point method cannot do any of these. But current interior-point implementations outperform active-set implementations on many large problems As sizes of mixed-integer nonlinear programming problems grow, there will be a definite need for interior-point methods at the lower level. IMA - MINLP - p. 2/34

3 Interior-Point Methods Each NLP relaxation has the form: Motivation: Discrete Variables Interior-Point Methods Add slack variables: min x,y f(x, y) s.t. h(x, y) 0 l y u min x,y,g,t f(x, y) s.t. h(x, y) w = 0 y g = l y + t = u w, g, t 0. IMA - MINLP - p. 3/34

4 Interior-Point Methods - 2 First-order conditions for the log barrier problem are Motivation: Discrete Variables Interior-Point Methods h(x, y) w = 0 y g = l y + t = u x f(x, y) A T x λ = 0 y f(x, y) A T y λ z + s = 0 W Λe = µe GZe = µe T Se = µe Use Newton s Method to solve this system. At each iteration, solve the reduced KKT system: 2 Hxx Hxy A T x x 6 4 Hxy (Hyy + D) A T 7 B y yc A = Ax Ay E λ 0 xf(x, y) A T x λ y f(x, y) A T y λ z + s D g (l y) D t (u y) µg 1 e + µt 1 e µλ 1 e h(x, y) 1 C A where E = W Λ 1, D = Dg + D t, Dg = G 1 Z, D t = T 1 S. IMA - MINLP - p. 4/34

5 Interior-Point Methods - 3 Motivation: Discrete Variables Interior-Point Methods At each iteration: choose steplengths to ensure that slacks remain strictly positive and sufficient progress toward optimality and feasibility is attained. value of the barrier parameter may also be updated as a function of (W (k+1) Λ (k+1) e, G (k+1) Z (k+1) e, T (k+1) S (k+1) e). Stopping criteria: primal infeasibility < ɛ dual infeasibility < ɛ average complementarity < ɛ IMA - MINLP - p. 5/34

6 IMA - MINLP - p. 6/34

7 Warmstarting: Branch-and-Bound Optimal solution at the parent: (x, y, g, t, λ, z, s ). Warmstarting: Branch-and-Bound Warmstarting: Outer Approximation Infeasibility Identification Fixed Variables Current node: Branch on some variable y j The following must hold: l j < y j < u j, gj > 0, zj = 0 t j > 0, s j = 0. WLOG, assume that l j < y j < y j The only term affected is D tj (u j y j ) = s j (u j y j )/t j. However, at the first iteration s j (u j y j )/t j = 0. The algorithm will get stuck at this nonoptimal and, in fact, infeasible solution. IMA - MINLP - p. 7/34

8 Warmstarting: Outer Approximation Warmstarting: Branch-and-Bound Warmstarting: Outer Approximation Infeasibility Identification Fixed Variables min x,y (x 0.25) 2 + y s.t. 60x 3 y y {0, 1} At each iteration of OA, an NLP subproblem is solved for a fixed value of y. The reduced KKT system: [ ] ( ) ( ) 2 360xλ 180x 2 x 2x x 2 λ 180x 2 w = µ λ λ λ (y 60x3 ) Let y = 1 for the first subproblem. Then, x = 0.25, w = 0.062, and λ = 0. Let y = 0 for the next subproblem. In the first iteration, x = 0 and y > 0, but very close to 0. Then, w = µ λ w w λ λ = 1. The steplength is shortened to less than The algorithm becomes stuck at the old solution. IMA - MINLP - p. 8/34

9 Infeasibility Identification Warmstarting: Branch-and-Bound Warmstarting: Outer Approximation Infeasibility Identification Fixed Variables An infeasible interior-point method does not have to start and/or stay feasible. Cannot get a certificate of infeasibility - only heuristics available. IMA - MINLP - p. 9/34

10 Fixed Variables Consider the following problem: Warmstarting: Branch-and-Bound Warmstarting: Outer Approximation Infeasibility Identification Fixed Variables min y y 2 s.t. 1 y 1. The optimality conditions of this problem are: y g = 1 y + t = 1 2y z + s = 0 gz = 0 ts = 0. When y = 1, we have both g and t equal 0 and at the optimal solution, the dual variables z and s are free to take on any nonnegative values as long as they satisfy the equality z s = 2. IMA - MINLP - p. 10/34

11 IMA - MINLP - p. 11/34

12 Previous Work on Warmstarting IPMs Previous Work Primal-Dual Penalty Model Solving the Penalty Problem Exactness of the Penalty Model Initialization Updates Benefits of the Primal-Dual Penalty Model One last improvement... Approach: Find a suitable starting point Identify an iterate close to the central path of the original problem Modify the iterate so it is well-centered for the new problem Solve the new problem from this point Works well in theory and practice: Gondzio (1998), Gondzio and Grothey (2003, 2006), Gondzio and Vial (1999), Yildirim and Wright (2002), John and Yildirim (2006) Mostly for LPs and QPs and only certain types of data perturbations IMA - MINLP - p. 12/34

13 Previous Work on Warmstarting IPMs Previous Work Primal-Dual Penalty Model Solving the Penalty Problem Exactness of the Penalty Model Initialization Updates Benefits of the Primal-Dual Penalty Model One last improvement... We propose a different approach: Change the problem, not the starting point Also investigated by Waltz & Ordonez, and Engau, Anjos, & Vanelli (2008) Our approach (Benson & Shanno (2005) and Benson (2007)) Corrects the numerical issues in the KKT system at the optimum of the original problem Allows the nonnegative variables to become negative to encourage longer steps Solves the new problem from the optimum of the original problem without modification IMA - MINLP - p. 13/34

14 Primal-Dual Penalty Model The primal problem: Previous Work Primal-Dual Penalty Model Solving the Penalty Problem Exactness of the Penalty Model Initialization Updates Benefits of the Primal-Dual Penalty Model One last improvement... min x,y The primal penalty problem: f(x, y) s.t. h(x, y) 0 l y u. min f(x, y) + c T wξ w + c T g ξ g + c T t ξ t x,y,w,g,t,ξ w,ξ g,ξ t s.t. h(x, y) w = 0 y g = l y + t = u ξ w w b λ ξ g g b z ξ t t b s ξ w, ξ g, ξ t 0, IMA - MINLP - p. 14/34

15 Primal-Dual Penalty Model The dual problem: Previous Work Primal-Dual Penalty Model Solving the Penalty Problem Exactness of the Penalty Model Initialization Updates Benefits of the Primal-Dual Penalty Model One last improvement... max λ,z,s dual_obj(λ, z, s; x, y) s.t. x f(x, y) A T x λ = 0 y f(x, y) A T y λ z + s = 0 λ, z, s 0. The dual penalty problem: max λ,z,s dual_obj(λ, z, s; x, y) b T λ ψ λ b T z ψ z b T s ψ s s.t. x f(x, y) A T x λ = 0 y f(x, y) A T y λ z + s = 0 ψ λ λ c w ψ λ ψ z z c g ψ z ψ s s c t ψ s ψ λ, ψ z, ψ s 0. IMA - MINLP - p. 15/34

16 Solving the Penalty Problem First-order conditions: Previous Work Primal-Dual Penalty Model Solving the Penalty Problem Exactness of the Penalty Model Initialization Updates Benefits of the Primal-Dual Penalty Model One last improvement... h(x, y) w = 0 y g = l y + t = u (W + Ξ w )(Λ + Ψ λ )e = µe (G + Ξ g )(Z + Ψ z )e = µe (T + Ξ t )(S + Ψ s )e = µe The reduced KKT system has E = Dg = D t = Ψ λ (B λ W )e = µe Ψ z (B z G)e = µe Ψ s (B s T )e = µe. x f(x, y) A T x λ = 0 y f(x, y) A T y λ z + s = 0 Ξ w (C w Λ Ψ λ )e = µe Ξ g (C g Z Ψ z )e = µe Ξ t (C t S Ψ s )e = µe (Λ + Ψ λ ) 1 (W + Ξw) + Ξw(Cw Λ Ψ λ ) Ψλ (B λ W ) 1«1 (Z + Ψz ) 1 (G + Ξg ) + Ξg (Cg Z Ψz ) Ψ z (Bz G) 1 (S + Ψs) 1 (T + Ξ t ) + Ξ t (C t S Ψs) Ψ s(bs T ) 1 and an appropriately modified rhs. IMA - MINLP - p. 16/34

17 Exactness of the Penalty Model Set the penalty parameters so that Previous Work Primal-Dual Penalty Model Solving the Penalty Problem Exactness of the Penalty Model Initialization Updates Benefits of the Primal-Dual Penalty Model One last improvement... b λ > w, c w > λ, b z > ḡ, c g > z, b s > t, c t > s. Then, the optimality conditions of the penalty problem reduce to the optimality conditions of the original problem. IMA - MINLP - p. 17/34

18 Computational Issues: Initialization Previous Work Primal-Dual Penalty Model Solving the Penalty Problem Exactness of the Penalty Model Initialization Updates Benefits of the Primal-Dual Penalty Model One last improvement... How do we initialize the relaxation variables and the penalty parameters in order to reach the new optimum quickly after a warmstart? Relaxation variables: ξ w = max(h(x, y) w, 0) + β ψ λ = β ξ g = max(x g l, 0) + β ψ z = β ξ t = max(x + t u, 0) + β ψ s = β where β is a small parameter, currently set to 10 5 M, where M is the greater of 1 and the largest primal or dual slack value. For discrete variables, β = 1. Penalty parameters: b λ = 2(w + κ) c w = 2(λ + ψ λ + κ) b z = 2(g + κ) c g = 2(z + ψ z + κ) b s = 2(t + κ) c t = 2(s + ψ s + κ), where κ is a constant with a default value of 1. IMA - MINLP - p. 18/34

19 Computational Issues: Updates Previous Work Primal-Dual Penalty Model Solving the Penalty Problem Exactness of the Penalty Model Initialization Updates Benefits of the Primal-Dual Penalty Model One last improvement... If necessary, how do we update the penalty parameters? Static updates Dynamic updates If If If If If If w (k+1) i g (k+1) j t (k+1) j λ (k+1) i z (k+1) j s (k+1) j > 0.9b (k) λi, then b(k+1) λi > 0.9b (k) zj > 0.9b (k) sj, then b(k+1) zj, then b(k+1) sj + ψ (k) λi > 0.9c wi w (k), then c (k+1) wi + ψ (k) zj + ψ (k) sj > 0.9c(k) gj > 0.9c (k) tj, then c(k+1) gj, then c(k+1) tj = 10b (k) λi, i = 1,..., m. = 10b (k) zj, j = 1,..., p. = 10b (k) sj = 10c (k) wi = 10c (k) gj = 10c (k) tj, j = 1,..., p., i = 1,..., m., j = 1,..., p., j = 1,..., p. IMA - MINLP - p. 19/34

20 Benefits of the Primal-Dual Penalty Model Warmstarting Previous Work Primal-Dual Penalty Model Solving the Penalty Problem Exactness of the Penalty Model Initialization Updates Benefits of the Primal-Dual Penalty Model One last improvement... Primal and dual infeasibility/unboundedness detection Handling of fixed variables Bounded sets of optimal primal and dual solutions Handling of complementarity conditions Detection of nonkkt optima Relieving of the jamming phenomenon IMA - MINLP - p. 20/34

21 One last improvement... A binary variable y can also be expressed as Previous Work Primal-Dual Penalty Model Solving the Penalty Problem Exactness of the Penalty Model Initialization Updates Benefits of the Primal-Dual Penalty Model One last improvement... 0 y 1 y 0. Not guaranteed to give the optimal solution, but it can provide integer feasible solutions. Computational effort is not significantly more than solving one subproblem: D = D g + D t 2 Λ + (2Y I) Λ W 1 (2Y I) with an appropriately modified rhs. IMA - MINLP - p. 21/34

22 IMA - MINLP - p. 22/34

23 Binary variables Mixed-integer nonlinear programming problems from MINLPLib: Binary variables Warmstarting NAME #MPEC #node f MPEC f MPEC opt? alan E E+000 Y batch E E+005 N batchdes E E+005 N ex E E+000 Y ex1223a E E+000 Y ex1223b E E+000 Y ex E E+001 N ex E E+001 N ex E E+000 N fac E E+007 N fuel E E+003 Y gbd E E+000 Y gkocis E E+000 N johnall E E+002 Y meanvarx E E+001 N nous E E+000 Y IMA - MINLP - p. 23/34

24 Binary variables - 2 Mixed-integer nonlinear programming problems from MINLPLib: Binary variables Warmstarting NAME #MPEC #node f MPEC f MPEC opt? oaer E E+000 Y procsel E E+000 Y ravem E E+005 N st_e E E+000 Y st_miqp E E+002 N st_miqp E E+003 Y st_test E E-012 N st_test E E+002 Y st_test E E+002 N synthes E E+000 N synthes E E+001 Y synthes E E+001 N IMA - MINLP - p. 24/34

25 Warmstarting Mixed-integer nonlinear programming problems from MINLPLib: Binary variables Warmstarting Problem WarmIters ColdIters %Impr #Nodes #Inf f(x ) alan E+00 batch* E+05 batchdes E+05 du-opt E+00 du-opt E+00 eg_all_s E+00 eg_disc_s E+00 eg_disc2_s E+00 eg_int_s E+00 ex E+00 ex1223a E+00 ex1223b E+00 ex E+01 ex E+01 ex E+00 fac E+07 IMA - MINLP - p. 25/34

26 Warmstarting - 2 Binary variables Warmstarting Mixed-integer nonlinear programming problems from MINLPLib: Problem WarmIters ColdIters %Impr #Nodes #Inf f(x ) fuel* E+03 gbd E+00 gear E-05 gkocis E+00 johnall E+02 meanvarx E+01 nous E+00 nvs E+01 nvs E-01 nvs E+00 nvs E+01 nvs E+02 nvs E+02 nvs E+02 nvs E+02 nvs E+04 IMA - MINLP - p. 26/34

27 Warmstarting - 3 Binary variables Warmstarting Mixed-integer nonlinear programming problems from MINLPLib: Problem WarmIters ColdIters %Impr #Nodes #Inf f(x ) nvs E+00 nvs E+03 nvs E+02 nvs E+03 nvs E+02 nvs E+03 nvs E+03 oaer E+00 prob E+03 prob E+01 procsel E+00 ravem E+05 st_e E+00 st_miqp E+02 st_miqp E+00 st_miqp E+03 IMA - MINLP - p. 27/34

28 Warmstarting - 4 Mixed-integer nonlinear programming problems from MINLPLib: Binary variables Warmstarting Problem WarmIters ColdIters %Impr #Nodes #Inf f(x ) st_test E-12 st_test E+00 st_test E+00 st_test E+02 st_test E+02 st_test E+04 st_testgr E+01 st_testgr E+01 st_testph E+01 synthes E+00 synthes E+01 synthes E+01 tloss E+01 OVERALL IMA - MINLP - p. 28/34

29 MILPs Binary variables Warmstarting Mixed-integer linear programming problems solved using branch-and-bound: Problem WarmIters ColdIters Diet Diet Diet Diet Diet Diet Diet Diet Diet Diet HL HL HL415-4 (inf) (inf) (inf) HL HL HL Problem WarmIters ColdIters Synthes Synthes Synthes Synthes Synthes Synthes Synthes Synthes Synthes Synthes Synthes Synthes Synthes Synthes IMA - MINLP - p. 29/34

30 Cutting Stock Binary variables Warmstarting Master problems of the cutting stock model: Problem WarmIters ColdIters Master Master Master IMA - MINLP - p. 30/34

31 IMA - MINLP - p. 31/34

32 Mixed-Integer SOCP MISOCP Interesting application areas, e.g. portfolio optimization with cardinality constraints, facility location problems with fixed costs. Interior-point methods have good convergence properties and computational performance. Homogeneous self-dual approach allows for a similar type of warmstart capability. A naive implementation of SeDuMi + Outer Approximation Looking to implement a re-centering scheme The primal-dual penalty approach also extends naturally to SOCPs. IMA - MINLP - p. 32/34

33 IMA - MINLP - p. 33/34

34 Interior-point methods can be warmstarted when regularization is used. Primal-dual regularization allows for warmstarts after any change to the problem. Moral of the story: We re working on it! IMA - MINLP - p. 34/34

RESEARCH ARTICLE. Mixed Integer Nonlinear Programming Using Interior-Point Methods

RESEARCH ARTICLE. Mixed Integer Nonlinear Programming Using Interior-Point Methods Optimization Methods and Software Vol. 00, No. 00, Month 200x, 1 20 RESEARCH ARTICLE Mixed Integer Nonlinear Programming Using Interior-Point Methods Hande Y. Benson Department of Decision Sciences Bennett