Infeasibility Detection in Nonlinear Optimization

Size: px

Start display at page:

Download "Infeasibility Detection in Nonlinear Optimization"

Hester Perry
5 years ago
Views:

1 Infeasibility Detection in Nonlinear Optimization Frank E. Curtis, Lehigh University Hao Wang, Lehigh University SIAM Conference on Optimization 2011 May 16, 2011 (See also Infeasibility Detection in Nonlinear Programming, Figen Oztoprak, MS71) Infeasibility Detection in Nonlinear Optimization 1 of 35

2 Outline Motivation Penalty-SQO Method Penalty-Interior-Point Method Unified Framework(?) Infeasibility Detection in Nonlinear Optimization 2 of 35

3 Outline Motivation Penalty-SQO Method Penalty-Interior-Point Method Unified Framework(?) Infeasibility Detection in Nonlinear Optimization 3 of 35

4 Constrained minimization Is this how we should formulate optimization problems? min x s.t. f (x) ( c E (x) = 0 c I (x) 0 (OP) Infeasibility Detection in Nonlinear Optimization 4 of 35

5 Feasibility violation minimization Suppose that if (OP) is infeasible, then we want to solve» min v(x) := c E (x) x max{c I (FP) (x), 0} Many algorithms/codes do this already, by either switching from solving one problem to the other; transitioning from solving one problem to the other. But are they doing it efficiently? Infeasibility Detection in Nonlinear Optimization 5 of 35

6 Numerical experiments: Infeasible optimization problems Iterations and f evaluations for 8 infeasible optimization problems (2-3 variables): Prob. Ipopt Knitro-Direct Knitro-Active Filter Iter. Eval. Iter. Eval. Iter. Eval. Iter. Eval *10000 * *1060 * *76 * *10000 *43652 *10000 * Problems also run with Knitro-CG and LOQO, but they failed every time. (We want your infeasible test problems!) Infeasibility Detection in Nonlinear Optimization 6 of 35

7 Numerical experiments: Feasibility problems (solved directly) Iterations and f evaluations for 8 feasibility problems (2-3 variables): Problem Ipopt Knitro-Direct Knitro-Active Filter Iter. Eval. Iter. Eval. Iter. Eval. Iter. Eval = If we can switch/transition efficiently, then our current tools work well. Infeasibility Detection in Nonlinear Optimization 7 of 35

8 Rapid detection of infeasibility Problem type Global convergence Fast local convergence Feasible Infeasible? Infeasibility Detection in Nonlinear Optimization 8 of 35

9 Two-phase strategy Infeasibility Detection in Nonlinear Optimization 9 of 35

10 Two-phase strategy Exploratory step determines possible progress toward feasibility. Rapid infeasibility detection requires rapid transition/switch to feasibility steps. (Important to have consistency between ways the two steps measure feasibility.) Infeasibility Detection in Nonlinear Optimization 10 of 35

11 Outline Motivation Penalty-SQO Method Penalty-Interior-Point Method Unified Framework(?) Infeasibility Detection in Nonlinear Optimization 11 of 35

12 Penalty methods Penalization provides one mechanism for transitioning to infeasibility detection: min ρf (x) + x,r,s,t et r + e T s + e T t 8 >< c E (x) = r s s.t. c >: I (x) t (r, s, t) 0 (PP) How should ρ be updated? Infeasibility Detection in Nonlinear Optimization 12 of 35

13 Literature (Rich history of SQP methods) Fletcher, Leyffer (2002) Byrd, Gould, Nocedal (2005) Byrd, Nocedal, Waltz (2008) Byrd, Curtis, Nocedal (2010) Byrd, López-Calva, Nocedal (2010) Gould, Robinson (2010) Morales, Nocedal, Wu (2010) Infeasibility Detection in Nonlinear Optimization 13 of 35

14 Steering and FILTER strategies Steering methods solve a sequence of constrained subproblems: FILTER:... we make use of a property of the phase I algorithm in our QP solver. If an infeasible QP is detected, the solver exists with a solution of [an LP minimizing the l 1 norm of violated constraints]. Infeasibility Detection in Nonlinear Optimization 14 of 35

15 SQuID strategy Compute v k as the optimal value for min d,r,s,t et r + e T s + e T t 8 c E (x k ) + c E (x k ) T d = r s >< c I (x s.t. k ) + c I (x k ) T d t d >: (r, s, t) 0 Update ρ for rapid infeasibility detection (Byrd, Curtis, Nocedal, 2010) Compute d k as the search direction from min ρ(f (x k) + f (x k ) T d) + e T r + e T s + e T t + 1 d,r,s,t 2 dt H(x k, λ k ; ρ)d 8 c E (x k ) + c E (x k ) T d = r s >< c I (x k ) + c I (x k ) T d t s.t. e >: T r + e T s + e T t v k (r, s, t) 0 Update ρ for descent Infeasibility Detection in Nonlinear Optimization 15 of 35

16 Numerical experiments: Infeasible optimization problems Iterations and f evaluations for 8 infeasible optimization problems (2-3 variables): Prob. Filter SQuID Iter. Eval. Iter. Eval Infeasibility Detection in Nonlinear Optimization 16 of 35

17 Outline Motivation Penalty-SQO Method Penalty-Interior-Point Method Unified Framework(?) Infeasibility Detection in Nonlinear Optimization 17 of 35

18 Penalty and interior-point methods Constrained subproblems in penalty methods can be expensive: min ρf (x) + x,r,s,t et r + e T s + e T t 8 >< c E (x) = r s s.t. c >: I (x) t (r, s, t) 0 (PP) Interior-point methods are more efficient for large-scale problems: min f (x) µ X ln u i x,u 8 >< c E (x) = 0 s.t. c >: I (x) = u u 0 (IP) Infeasibility Detection in Nonlinear Optimization 18 of 35

19 Penalty-interior-point method Applying an interior-point reformulation to (PP): X min ρf (x) µ (ln r i + ln s i ) + X (ln t i + ln u i ) + e T r + e T s + e T t x,r,s,t,u ( c E (x) = r s s.t. c I (x) = t u (PIP) The optimization problem (OP) and feasibility problem (FP) can be solved via (PIP): µ 0 and ρ ρ > 0 to solve (OP). µ 0 and ρ 0 to solve (FP). Infeasibility Detection in Nonlinear Optimization 19 of 35

20 Literature Previous work with similar motivations: Jittorntrum and Osborne (1980) Polyak (1982, 1992, 2008) Breitfeld and Shanno (1994, 1996) Goldfarb, Polyak, Scheinberg, and Yuzefovich (1999) Gould, Orban, and Toint (2003) Chen and Goldfarb (2006, 2006) Benson, Sen, and Shanno (2008) Infeasibility Detection in Nonlinear Optimization 20 of 35

21 Literature Previous work with similar motivations: Jittorntrum and Osborne (1980) Polyak (1982, 1992, 2008) Breitfeld and Shanno (1994, 1996) Goldfarb, Polyak, Scheinberg, and Yuzefovich (1999) Gould, Orban, and Toint (2003) Chen and Goldfarb (2006, 2006) Benson, Sen, and Shanno (2008) Parameter updates are essential to have a practical algorithm. Conservative strategy: 1. Fix ρ > 0 and (approximately) solve (PIP) for µ If infeasible, then reduce ρ and go to step 1. (ρ reduced too slowly to ensure fast convergence for infeasible problems.) Infeasibility Detection in Nonlinear Optimization 21 of 35

22 PIPAL strategy Newton system has the form A ρ,µd = ρq 1 + µq 2 + q 3. Steering strategy now efficient! (Only one factorization required.) Solution for (ρ, µ) = (0, µ) indicates progress possible toward primal feasibility. Adjust ρ to aim for primal feasibility. Adjust µ to aim for dual feasibility and complementarity. Infeasibility Detection in Nonlinear Optimization 22 of 35

23 Conservative strategy ρ μ Each dot may require at least a few iterations. Each row may require the computational effort of an entire interior-point solve. For an infeasible problem, we need both ρ and µ to reduce to zero. Infeasibility Detection in Nonlinear Optimization 23 of 35

24 PIPAL strategy ρ μ Blue line separates acceptable and unacceptable ρ values (for primal feasibility). µ then set to ensure progress toward dual feasibility and complementarity. For an infeasible problem, ρ and µ can reduce rapidly. Infeasibility Detection in Nonlinear Optimization 24 of 35

25 Numerical results: Feasible problems (sample size = 422) Infeasibility Detection in Nonlinear Optimization 25 of 35

26 Numerical results: Feasible problems w/ ρ decrease (sample size = 136) Infeasibility Detection in Nonlinear Optimization 26 of 35

27 Numerical results: Degenerate problems (sample size = 114) Added constraints: c i (x) 2 0 Infeasibility Detection in Nonlinear Optimization 27 of 35

28 Numerical results: Infeasible problems (sample size = 90) Added constraints: c i (x) 2 1 Infeasibility Detection in Nonlinear Optimization 28 of 35

29 Outline Motivation Penalty-SQO Method Penalty-Interior-Point Method Unified Framework(?) Infeasibility Detection in Nonlinear Optimization 29 of 35

30 Constrained minimization Is this how we should formulate optimization problems? min x s.t. f (x) ( c E (x) = 0 c I (x) 0 (OP) That is, should the strategy be the following: Attempt to solve (OP), but... if it is infeasible, then solve» min v(x) := c E (x) x max{c I (FP) (x), 0} Infeasibility Detection in Nonlinear Optimization 30 of 35

31 Minimization over set of feasibility violation minimizers Let X be the solution set of feasibility violation minimization:» min v(x) := c E (x) x max{c I (FP) (x), 0} We should solve min x f (x) s.t. x X. (Even better would be a single problem or set of conditions for solving (UP).) (UP) Infeasibility Detection in Nonlinear Optimization 31 of 35

32 1 0 Consider the following model (AMPL presolver turned off): var x; minimize f: (x-1)^2; s.t. c: 1 <= 0; Solver Inf. Declared? Iterations Final (x, y) CONOPT No FILTER Yes IPOPT Yes KNITRO Yes LANCELOT Yes LOQO No MINOS Yes MOSEK Yes PENNON No SNOPT Yes Infeasibility Detection in Nonlinear Optimization 32 of 35

33 y 2 1 Consider the following model (AMPL presolver turned off): var x; var y; minimize f: (x-1)^2; s.t. c: y^2 + 1 <= 0; Solver Inf. Declared? Iterations Final (x, y) CONOPT Yes 3 (0.0000,0.0000) FILTER Yes 2 (0.0000,0.0000) IPOPT Yes 6 (0.0099,0.0000) KNITRO Yes 4 (0.3765,0.0000) LANCELOT Yes 1 (1.0000,0.0000) LOQO No 500 (0.4569,0.0000) MINOS Yes 2 (1.0000,0.0000) MOSEK Yes 1 (0.0000,0.0000) PENNON No 0 (1.0000,0.0000) SNOPT Yes 3 (1.0000,0.0000) Infeasibility Detection in Nonlinear Optimization 33 of 35

34 Penalty and switching methods Penalization provides one mechanism for transitioning to infeasibility detection: min ρf (x) + x,r,s,t et r + e T s + e T t 8 >< c E (x) = r s s.t. c >: I (x) t (r, s, t) 0 (PP) However, once x X is found, f (x) has been thrown out (since ρ 0). Switching methods throw out f (x) even earlier. Infeasibility Detection in Nonlinear Optimization 34 of 35

35 What is needed? General-purpose algorithms that: require no constraint qualifications to do something useful minimize the objective over the set of feasibility violation minimizers are as efficient and robust as contemporary methods when problems are nice Infeasibility Detection in Nonlinear Optimization 35 of 35

Infeasibility Detection and an Inexact Active-Set Method for Large-Scale Nonlinear Optimization

Infeasibility Detection and an Inexact Active-Set Method for Large-Scale Nonlinear Optimization Frank E. Curtis, Lehigh University involving joint work with James V. Burke, University of Washington Daniel