arxiv: v1 [math.oc] 20 Aug 2014

Transcription

1 Adaptive Augmented Lagrangian Methods: Algorithms and Practical Numerical Experience Detailed Version Frank E. Curtis, Nicholas I. M. Gould, Hao Jiang, and Daniel P. Robinson arxiv: v1 [math.oc] 20 Aug 2014 In this paper, we consider augmented Lagrangian (AL) algorithms for solving large-scale nonlinear optimization problems that execute adaptive strategies for updating the penalty parameter. Our work is motivated by the recently proposed adaptive AL trust region method by Curtis, Jiang, and Robinson [Math. Prog., DOI: /s y, 2013]. The first focal point of this paper is a new variant of the approach that employs a line search rather than a trust region strategy, where a critical algorithmic feature for the line search strategy is the use of convexified piecewise quadratic models of the AL function for computing the search directions. We prove global convergence guarantees for our line search algorithm that are on par with those for the previously proposed trust region method. A second focal point of this paper is the practical performance of the line search and trust region algorithm variants in Matlab software, as well as that of an adaptive penalty parameter updating strategy incorporated into the Lancelot software. We test these methods on problems from the CUTEst and COPS collections, as well as on challenging test problems related to optimal power flow. Our numerical experience suggests that the adaptive algorithms outperform traditional AL methods in terms of efficiency and reliability. As with traditional AL algorithms, the adaptive methods are matrix-free and thus represent a viable option for solving extreme-scale problems. Keywords: nonlinear optimization, nonconvex optimization, large-scale optimization, augmented Lagrangians, matrix-free methods, steering methods AMS Subject Classification: 49M05, 49M15, 49M29, 49M37, 65K05, 65K10, 90C06, 90C30, 93B40 1. Introduction Augmented Lagrangian (AL) methods [25, 33] have recently regained popularity due to growing interests in solving extreme-scale nonlinear optimization problems. These methods are attractive in such settings as they can be implemented matrix-free [2, 3, 11, 28] and have global and local convergence guarantees under relatively weak assumptions [18, 26]. Furthermore, certain variants of AL methods [20, 21] have proved to be very efficient for solving certain structured problems [6, 34, 36]. A new AL trust region method was recently proposed and analyzed in [15]. The novel feature of that algorithm is an adaptive strategy for updating the penalty parameter inspired by techniques for performing such updates in the context of exact penalty methods [7, 8, 29]. This feature is designed to overcome a potentially serious drawback of traditional AL methods, which is that they may be ineffective during some (early) iterations due to poor choices of the penalty parameter and/or Lagrange multiplier estimates. In such situations, the poor choices of these quantities may lead to little or no improvement in the primal space and, in fact, the iterates may diverge from even a well-chosen initial Department of Industrial and Systems Engineering, Lehigh University, Bethlehem, PA, USA. frank.e.curtis@gmail.com. Supported by Department of Energy grant DE SC STFC-Rutherford Appleton Laboratory, Numerical Analysis Group, R18, Chilton, OX11 0QX, UK. nick.gould@stfc.ac.uk. Supported by Engineering and Physical Sciences Research Council grant EP/I013067/1. Department of Applied Mathematics and Statistics, Johns Hopkins University, Baltimore, MD, USA. hjiang13@jhu.edu, daniel.p.robinson@gmail.com. Supported by National Science Foundation grant DMS

2 iterate. The key idea for avoiding this behavior in the algorithm proposed in [15] is to adaptively update the penalty parameter during the step computation in order to ensure that the trial step yields a sufficiently large reduction in linearized constraint violation, thus steering the optimization process steadily toward constraint satisfaction. The contributions of this paper are two-fold. First, we present an AL line search method based on the same framework employed for the trust region method in [15]. The main difference between our new approach and that in [15], besides the differences inherent in using line searches instead of a trust region strategy, is that we utilize a convexified piecewise quadratic model of the AL function to compute the search direction in each iteration. With this modification, we prove that our line search method achieves global convergence guarantees on par with those proved for the trust region method in [15]. The second contribution of this paper is that we perform extensive numerical experiments with a Matlab implementation of the adaptive algorithms (i.e., both line search and trust region variants) and an implementation of an adaptive penalty parameter updating strategy in the Lancelot software [12]. We test these implementations on problems from the CUTEst [22] and COPS [5] collections, as well as on test problems related to optimal power flow [37]. Our results indicate that our adaptive algorithms outperform traditional AL methods in terms of efficiency and reliability. The remainder of the paper is organized as follows. In 2, we present our adaptive AL line search method and state convergence results. Details and proofs of these results, which draw from those in [15], can be found in Appendices A and B. We then provide numerical results in 3 to illustrate the effectiveness of our implementations of our adaptive AL algorithms. We give conclusions in 4. Notation. We often drop function arguments once a function is defined. We also use a subscript on a function name to denote its value corresponding to algorithmic quantities using the same subscript. For example, for a function f : R n R, if x k is the value for the variable x during iteration k of an algorithm, then f k := f(x k ). We also often use subscripts for constants to indicate the algorithmic quantity to which they correspond. For example, γ µ denotes a parameter corresponding to the algorithmic quantity µ. 2. An Adaptive Augmented Lagrangian Line Search Algorithm 2.1 Preliminaries We assume that all problems under our consideration are formulated as minimize f(x) subject to c(x) = 0, l x u. (1) x R n Here, we assume that the objective function f : R n R and constraint function c : R n R m are twice continuously differentiable, and that the variable lower bound vector l R n and upper bound vector u R n satisfy l u. Our goal is to design an algorithm that will compute a first-order primal-dual stationary point for problem (1). However, in order for the algorithm to be suitable as a general-purpose approach, it should have mechanisms for terminating and providing useful information when an instance of (1) is (locally) infeasible. In such cases, we have designed our algorithm so that it transitions to finding an infeasible first-order stationary point for the nonlinear feasibility problem where v : R n R is defined as v(x) = 1 2 c(x) 2 2. minimize v(x) subject to l x u, (2) x R n 2

3 As implied by the previous paragraph, our algorithm requires first-order stationarity conditions for problems (1) and (2), which can be stated in the following manner. First, introducing a Lagrange multiplier vector y R m, we define the Lagrangian for problem (1), call it l : R n R m R, by l(x, y) = f(x) c(x) T y. Then, defining the gradient of the objective function g : R n R n by g(x) = f(x), the Jacobian of the constraint functions J : R n R m n by J(x) = c(x), and the projection operator onto the bounds P : R n R n, component-wise for i {1,..., n}, by [P (x)] i = l i u i x i if x i l i if x i u i otherwise we may introduce the primal-dual stationarity measure F L : R n R m R n given by F L (x, y) = P (x x l(x, y)) x = P (x (g(x) J(x) T y)) x. First-order primal-dual stationary points for (1) can then be characterized as zeros of the primal-dual stationarity measure F OPT : R n R m R n+m defined by stacking the stationarity measure F L and the constraint function c, i.e., a first-order primal-dual stationary point for (1) is any pair (x, y) with l x u satisfying 0 = F OPT (x, y) = ( ) FL (x, y) = c(x) ( ) P (x x l(x, y)) x. (3) y l(x, y) Similarly, a first-order primal stationary point for (2) is any x with l x u satisfying where F FEAS : R n R n is defined by 0 = F FEAS (x), (4) F FEAS (x) = P (x x v(x)) x = P (x J(x) T c(x)) x. In particular, if l x u, v(x) > 0, and (4) holds, then x is an infeasible stationary point for problem (1). Over the past decades, a variety of effective numerical methods have been proposed for solving large-scale bound-constrained optimization problems. Hence, the critical issue in solving problem (1) is how to handle the presence of the equality constraints. As in the wide variety of penalty methods that have been proposed, the strategy adopted by AL methods is to remove these constraints, but push the algorithm to satisfy them through the addition of influential terms in the objective. In this manner, problem (1) (or at least (2)) can be solved via a sequence of bound-constrained subproblems thus allowing AL methods to exploit the methods that are available for subproblems of this type. Specifically, AL methods consider a sequence of subproblems in which the objective is a weighted sum of the Lagrangian l and the constraint violation measure v. By scaling l by a penalty parameter µ 0, each subproblem involves the minimization of a function L : R n R m R R, called the augmented Lagrangian (AL), defined by L(x, y, µ) = µl(x, y) + v(x) = µ(f(x) c(x) T y) c(x)

4 Observe that the gradient of the AL with respect to x, evaluated at (x, y, µ), is given by x L(x, y, µ) = µ ( g(x) J(x) T π(x, y, µ) ), where we define the function π : R n R m R R m by π(x, y, µ) = y 1 µ c(x). (5) Hence, each subproblem to be solved in an AL method has the form minimize L(x, y, µ) subject to l x u. (6) x R n Given a pair (y, µ), a first-order stationary point for problem (6) is any zero of the primaldual stationarity measure F AL : R n R m R R n, defined similarly to F L but with the Lagrangian replaced by the augmented Lagrangian; i.e., given (y, µ), a first-order stationary point for (6) is any x satisfying 0 = F AL (x, y, µ) = P (x x L(x, y, µ)) x. (7) Given a pair (y, µ) with µ > 0, a traditional AL method proceeds by (approximately) solving (6), which is to say that it finds a point, call it x(y, µ), that (approximately) satisfies (7). If the resulting pair (x(y, µ), y) is not a first-order primal-dual stationary point for (1), then the method would modify the Lagrange multiplier y or penalty parameter µ so that, hopefully, the solution of the subsequent subproblem (of the form (6)) yields a better primal-dual solution estimate for (1). The function π plays a critical role in this procedure. In particular, observe that if c(x(y, µ)) = 0, then π(x(y, µ), y, µ) = y and (7) would imply F OPT (x(y, µ), y) = 0, i.e., that (x(y, µ), y) is a first-order primal-dual stationary point for (1). Hence, if the constraint violation at x(y, µ) is sufficiently small, then a traditional AL method would set the new value of y as π(x, y, µ). Otherwise, if the constraint violation is not sufficiently small, then the penalty parameter is decreased to place a higher priority on reducing it during subsequent iterations. 2.2 Algorithm Description Our AL line search algorithm is similar to the AL trust region method proposed in [15], except for two key differences: it executes line searches rather than using a trust region framework, and it employs a convexified piecewise quadratic model of the AL function for computing the search direction in each iteration. The main motivation for utilizing a convexified model is to ensure that each computed search direction is a direction of strict descent for the AL function from the current iterate, which is necessary to ensure the wellposedness of the line search. However, it should be noted that, practically speaking, the convexification of the model does not necessarily add any computational difficulties when computing each direction; see Similar to the trust region method proposed in [15], a critical component of our algorithm is the adaptive strategy for updating the penalty parameter µ during the search direction computation. This is used to ensure steady progress i.e., steer the algorithm toward solving (1) (or at least (2)) by monitoring predicted improvements in linearized feasibility. The central component of each iteration of our algorithm is the search direction computation. In our approach, this computation is performed based on local models of the constraint violation measure v and the AL function L at the current iterate, which at iteration k is given by (x k, y k, µ k ). The local models that we employ for these functions 4

5 are, respectively, q v : R n R and q : R n R, defined as follows: q v (s; x) = 1 2 c(x) + J(x)s 2 2 q(s; x, y, µ) = L(x, y) + x L(x, y) T s + max{ 1 2 st (µ 2 xxl(x, y) + J(x) T J(x))s, 0}. We note that q v is a typical Gauss-Newton model of the constraint violation measure v, and q is a convexification of a second-order approximation of the augmented Lagrangian. (We use the notation q rather than simply q to distinguish between the model above and the second-order model without the max that appears extensively in [15].) Our algorithm computes two types of steps during each iteration. The purpose of the first step, which we refer to as the steering step, is to gauge the progress towards linearized feasibility that may be achieved (locally) from the current iterate. This is done by (approximately) minimizing our model q v of the constraint violation measure v within the bound constraints and a trust region. Then, a step of the second type is computed by (approximately) minimizing our model q of the AL function L within the bound constraints and a trust region. If the reduction in the model q v yielded by the latter step is sufficiently large say, compared to that yielded by the steering step then the algorithm proceeds using this step as the search direction. Otherwise, the penalty parameter may be reduced, in which case a step of the latter type is recomputed. This process repeats iteratively until a search direction is computed that yields a sufficiently large (or at least not too negative) reduction in q v. As such, the iterate sequence is intended to make steady progress toward (or at least approximately maintain) constraint satisfaction throughout the optimization process, regardless of the initial penalty parameter value. We now describe this process in more detail. During iteration k, the steering step r k is computed via the optimization subproblem given by minimize q v (r; x k ) subject to l x k + r u, r 2 θ k, (8) r R n where, for some constant δ > 0, the trust region radius is defined to be θ k := δ F FEAS (x k ) 2 0. (9) A consequence of this choice of trust region radius is that it forces the steering step to be smaller in norm as the iterates of the algorithm approach any stationary point of the constraint violation measure [35]. This prevents the steering step from being too large relative to the progress that can be made toward minimizing v. While (8) is a convex optimization problem for which there are efficient methods, in order to reduce computational expense our algorithm only requires r k to be an approximate solution of (8). In particular, we merely require that r k yields a reduction in q v that is proportional to that yielded by the associated Cauchy step (see (16a) later on), which is defined to be r k := r(x k, θ k ) := P (x k β k J T k c k) x k (10) for β k := β(x k, θ k ) such that, for some ε r (0, 1), the step r k satisfies q v (r k ; x k ) := q v (0; x k ) q v (r k ; x k ) ε r r T k J T k c k and r k 2 θ k. (11) Appropriate values for β k and r k along with auxiliary nonnegative scalar quantities ε k and Γ k to be used in subsequent calculations in our method are computed by Algorithm 1. The quantity q v (r k ; x k ) representing the predicted reduction in constraint violation yielded by r k is guaranteed to be positive at any x k that is not a first-order 5

6 stationary point for v subject to the bound constraints; see part (i) of Lemma A.4. We define a similar reduction q v (r k ; x k ) for the steering step r k. Algorithm 1 Cauchy step computation for the feasibility subproblem (8) 1: procedure Cauchy feasibility(x k, θ k ) 2: restrictions : θ k 0. 3: available constants : {ε r, γ} (0, 1). 4: Compute the smallest integer l k 0 satisfying P (x k γ lk Jk Tc k) x k 2 θ k. 5: if l k > 0 then 6: Set Γ k min{2, 1 2 (1 + P (x k γ lk 1 Jk Tc k) x k 2 /θ k )}. 7: else 8: Set Γ k 2. 9: end if 10: Set β k γ lk, r k P (x k β k Jk Tc k) x k, and ε k 0. 11: while r k does not satisfy (11) do 12: Set ε k max(ε k, q v (r k ; x k )/r T k J k Tc k). 13: Set β k γβ k and r k P (x k β k Jk Tc k) x k. 14: end while 15: return : (β k, r k, ε k, Γ k ) 16: end procedure After computing a steering step r k, we proceed to compute a trial step s k via minimize q(s; x k, y k, µ k ) subject to l x k + s u, s 2 Θ k, (12) s R n where, given Γ k > 1 from the output of Algorithm 1, we define the trust region radius Θ k := Θ(x k, y k, µ k, Γ k ) = Γ k δ F AL (x k, y k, µ k ) 2 0. (13) As for the steering step, we allow inexactness in the solution of (12) by only requiring the step s k to satisfy a Cauchy decrease condition (see (16b) later on), where the Cauchy step for problem (12) is s k := s(x k, y k, µ k, Θ k, ε k ) := P (x k α k x L(x k, y k, µ k )) x k (14) for α k = α(x k, y k, µ k, Θ k, ε k ) such that, for ε k 0 returned from Algorithm 1, s k yields q(s k ; x k, y k, µ k ) := q(0; x k, y k, µ k ) q(s k ; x k, y k, µ k ) (ε k + ε r ) s T k 2 xl(x k, y k, µ k ) and s k 2 Θ k. (15) Algorithm 2 describes our procedure for computing α k and s k. (The importance of incorporating Γ k in (13) and ε k in (15) is revealed in the proofs of Lemmas A.2 and A.3.) The quantity q(s k ; x k, y k, µ k ) representing the predicted reduction in L(, y k, µ k ) yielded by s k is guaranteed to be positive at any x k that is not a first-order stationary point for L(, y k, µ k ) subject to the bound constraints; see part (ii) of Lemma A.4. A similar quantity q(s k ; x k, y k, µ k ) is also used for the search direction s k. Our complete algorithm is given as Algorithm 3 on page 8. In particular, the kth iteration proceeds as follows. Given the kth iterate tuple (x k, y k, µ k ), the algorithm first determines whether the first-order primal-dual stationarity conditions for (1) or the firstorder stationarity condition for (2) are satisfied. If either is the case, then the algorithm 6

7 Algorithm 2 Cauchy step computation for the Augmented Lagrangian subproblem (12). 1: procedure Cauchy AL(x k, y k, µ k, Θ k, ε k ) 2: restrictions : µ k > 0, Θ k > 0, and ε k 0. 3: available constant : γ (0, 1). 4: Set α k 1 and s k P (x k α k x L(x k, y k, µ k )) x k. 5: while (15) is not satisfied do 6: Set α k γα k and s k P (x k α k x L(x k, y k, µ k )) x k. 7: end while 8: return : (α k, s k ) 9: end procedure terminates, but otherwise the method enters the while loop in line 11 to check for stationarity with respect to the AL function. This loop is guaranteed to terminate finitely; see Lemma A.1. Next, after computing appropriate trust region radii and Cauchy steps, the method enters a block for computing the steering step r k and trial step s k. Through the while loop on line 19, the overall goal of this block is to compute (approximate) solutions of subproblems (8) and (12) satisfying q(s k ; x k, y k, µ k ) κ 1 q(s k ; x k, y k, µ k ) > 0, l x k + s k u, s k 2 Θ k, (16a) q v (r k ; x k ) κ 2 q v (r k ; x k ) 0, l x k + r k u, r k 2 θ k, (16b) and q v (s k ; x k ) min{κ 3 q v (r k ; x k ), v k 1 2 (κ tt j ) 2 }. (16c) In these conditions, the method employs user-provided constants {κ 1, κ 2, κ 3, κ t } (0, 1) and the algorithmic quantity t j > 0 representing the jth constraint violation target. It should be noted that, for sufficiently small µ > 0, many approximate solutions to (8) and (12) satisfy (16), but for our purposes (see Theorem 2.2) it is sufficient that, for sufficiently small µ > 0, they are at least satisfied by r k = r k and s k = s k. A complete description of the motivations underlying conditions (16) can be found in [15, Section 3]. In short, (16a) and (16b) are Cauchy decrease conditions while (16c) ensures that the trial step predicts progress toward constraint satisfaction, or at least predicts that any increase in constraint violation is limited (when the right-hand side is negative). With the search direction s k in hand, the method proceeds to perform a backtracking line search along the strict descent direction s k for L(, y k, µ k ) at x k. Specifically, for a given γ α (0, 1), the method computes the smallest integer l 0 such that L(x k + γ l αs k, y k, µ k ) L(x k, y k, µ k ) η s γ l α q(s k ; x k, y k, µ k ), (17) and then sets α k γ l α and x k+1 x k + α k s k. The remainder of the iteration is then composed of potential modifications of the Lagrange multiplier vector and target values for the accuracies in minimizing the constraint violation measure and AL function subject to the bound constraints. First, the method checks whether the constraint violation at the next primal iterate x k+1 is sufficiently small compared to the target t j > 0. If this requirement is met, then a multiplier vector ŷ k+1 that satisfies F L (x k+1, ŷ k+1 ) 2 min { F L (x k+1, y k ) 2, F L (x k+1, π(x k+1, y k, µ k )) 2 } (18) is computed. Two obvious potential choices for ŷ k+1 are y k and π(x k+1, y k, µ k ), but another viable candidate would be an approximate least-squares multiplier estimate (which may be computed via a linearly constrained optimization subproblem). The method then checks if either F L (x k+1, ŷ k+1 ) 2 or F AL (x k+1, y k, µ k ) 2 is sufficiently small with respect 7

8 Algorithm 3 Adaptive Augmented Lagrangian Line Search Algorithm 1: Choose {γ, γ µ, γ α, γ t, γ T, κ 1, κ 2, κ 3, ε r, κ t, η s, η vs } (0, 1) and {δ, ɛ, Y } (0, ) such that η vs η s. 2: Choose initial primal-dual pair (x 0, y 0 ) and initialize {µ 0, δ 0, t 0, t 1, T 1, Y 1 } (0, ) such that Y 1 Y and y 0 2 Y 1. 3: Set k 0, k 0 0, and j 1. 4: loop 5: if F OPT (x k, y k ) = 0, then 6: return the first-order stationary solution (x k, y k ). 7: end if 8: if c k 2 > 0 and F FEAS (x k ) = 0, then 9: return the infeasible stationary point x k. 10: end if 11: while F AL (x k, y k, µ k ) = 0, do 12: Set µ k γ µ µ k. 13: end while 14: Define θ k by (9). 15: Use Algorithm 1 to compute (β k, r k, ε k, Γ k ) = Cauchy feasibility(x k, θ k ). 16: Define Θ k by (13). 17: Use Algorithm 2 to compute (α k, s k ) = Cauchy AL(x k, y k, µ k, Θ k, ε k ). 18: Compute approximate solutions r k to (8) and s k to (12) that satisfy (16a) (16b). 19: while (16c) is not satisfied or F AL (x k, y k, µ k ) = 0, do 20: Set µ k γ µ µ k and define Θ k by (13). 21: Use Algorithm 2 to compute (α k, s k ) = Cauchy AL(x k, y k, µ k, Θ k, ε k ). 22: Compute an approximate solution s k to (12) satisfying (16a). 23: end while 24: Set α k γα l where l 0 is the smallest integer satisfying (17). 25: Set x k+1 x k + α k s k. 26: if c k+1 2 t j, then 27: Compute any ŷ k+1 satisfying (18). 28: if min{ F L (x k+1, ŷ k+1 ) 2, F AL (x k+1, y k, µ k ) 2 } T j, then 29: Set k j k + 1 and Y j+1 max{y, t ɛ j 1 }. 30: Set t j+1 min{γ t t j, t 1+ɛ j } and T j+1 γ T T j. 31: Set y k+1 from (19) where α y satisfies (20). 32: Set j j : else 34: Set y k+1 y k. 35: end if 36: else 37: Set y k+1 y k. 38: end if 39: Set µ k+1 µ k. 40: Set k k : end loop to the target value T j > 0. If so, then new target values t j+1 < t j and T j+1 < T j are set, Y j+1 Y j is chosen, and a new Lagrange multiplier vector is set as y k+1 (1 α y )y k + α y ŷ k+1, (19) 8

9 where α y is the largest value in [0, 1] such that (1 α y )y k + α y ŷ k+1 2 Y j+1. (20) This updating procedure is well-defined since the choice α y 0 results in y k+1 y k, for which (20) is satisfied since y k 2 Y j Y j+1. If either line 26 or line 28 in Algorithm 3 tests false, then the method simply sets y k+1 y k. We note that unlike more traditional augmented Lagrangian approaches [2, 11], the penalty parameter is not adjusted on the basis of a test like that on line 26, but instead relies on our steering procedure. Moreover, in our approach we decrease the target values at a linear rate for simplicity, but more sophisticated approaches may be used [11]. 2.3 Well-posedness and global convergence In this section, we state two vital results, namely that Algorithm 3 is well posed, and that limit points of the iterate sequence have desirable properties. Proofs of these results, which are similar to those in [15], are given in Appendices A and B. In order to show well-posedness of the algorithm, we make the following formal assumption. Assumption 2.1 At each given x k, the objective function f and constraint function c are both twice-continuously differentiable. Under this assumption, we have the following theorem. Theorem 2.2 Suppose that Assumption 2.1 holds. Then the kth iteration of Algorithm 3 is well posed. That is, either the algorithm will terminate in line 6 or 9, or it will compute µ k > 0 such that F AL (x k, y k, µ k ) 0 and for the steps s k = s k and r k = r k the conditions in (16) will be satisfied, in which case (x k+1, y k+1, µ k+1 ) will be computed. According to Theorem 2.2, we have that Algorithm 3 will either terminate finitely or produce an infinite sequence of iterates. If it terminates finitely which can only occur if line 6 or 9 is executed then the algorithm has computed a first-order stationary solution or an infeasible stationary point and there is nothing else to prove about the algorithm s performance in such cases. Therefore, it remains to focus on the global convergence properties of Algorithm 3 under the assumption that the sequence {(x k, y k, µ k )} is infinite. For such cases, we make the following additional assumption. Assumption 2.3 The primal sequences {x k } and {x k + s k } are contained in a convex compact set over which the objective function f and constraint function c are both twicecontinuously differentiable. Our main global convergence result for Algorithm 3 is as follows. Theorem 2.4 Suppose that Assumptions 2.2 and 2.3 hold. Then one of the following must hold: (i) every limit point x of {x k } is an infeasible stationary point; (ii) µ k 0 and there exists an infinite ordered set K N such that every limit point of {(x k, ŷ k )} k K is first-order stationary for (1); or (iii) µ k 0, every limit point of {x k } is feasible, and if there exists a positive integer p such that µ kj 1 γ p µµ kj 1 1 for all sufficiently large j, then there exists an infinite ordered set J N such that any limit point of either {(x kj, ŷ kj )} j J or {(x kj, y kj 1)} j J is 9

10 first-order stationary for (1). Observe that the conclusions are exactly the same as in [15, Theorem 3.14]. We also call the readers attention to the comments following [15, Theorem 3.14], which discuss the consequences of these results. In particular, these comments suggest how Algorithm 3 may be modified to guarantee convergence to first-order stationary points, even in case (iii) of the theorem. However, as mentioned in [15], we do not consider these modifications to the algorithm to have practical benefits. This perspective is supported by the numerical tests presented in the following section. 3. Numerical Experiments In this section, we provide evidence that steering can have a positive effect on the performance of AL algorithms. To best illustrate the influence of steering, we implemented and tested algorithms in two pieces of software. First, in Matlab, we implemented our adaptive AL line search algorithm, i.e., Algorithm 3, and the adaptive AL trust region method given as [15, Algorithm 4]. Since these methods were implemented from scratch, we had control over every aspect of the code, which allowed us to implement all features described in this paper and in [15]. Second, we implemented a simple modification of the AL trust region algorithm in the Lancelot software package [12]. Our only modification to Lancelot was to incorporate a basic form of steering; i.e., we did not change other aspects of Lancelot, such as the mechanisms for triggering a multiplier update. In this manner, we were also able to isolate the effect that steering had on numerical performance, though it should be noted that there were differences between Algorithm 3 and our implemented algorithm in Lancelot in terms of, e.g., the multiplier updates. While we provide an extensive amount of information about the results of our experiments in this section, further information can be found in Appendix C. 3.1 Matlab implementation Implementation details Our Matlab software was comprised of six algorithm variants. The algorithms were implemented as part of the same package so that most of the algorithmic components were exactly the same; the primary differences related to the step acceptance mechanisms and the manner in which the Lagrange multiplier estimates and penalty parameter were updated. First, for comparison against algorithms that utilized our steering mechanism, we implemented line search and trust region variants of a basic augmented Lagrangian method, given as [15, Algorithm 1]. We refer to these algorithms as BAL-LS (basic augmented Lagrangian, line search) and BAL-TR (trust region), respectively. These algorithms clearly differed in that one used a line search and the other used a trust region strategy for step acceptance, but the other difference was that, like Algorithm 3 in this paper, BAL-LS employed a convexified model of the AL function. (We discuss more details about the use of this convexified model below.) The other algorithms implemented in our software included two variants of Algorithm 3 and two variants of [15, Algorithm 4]. The first variants of each, which we refer to as AAL-LS and AAL-TR (adaptive, as opposed to basic), were straightforward implementations of these algorithms, whereas the latter variants, which we refer to as AAL-LS-safe and AAL-TR-safe, included an implementation of a safeguarding procedure for the steering mechanism. The safeguarding procedure will be described in detail shortly. The main per-iteration computational expense for each algorithm variant can be at- 10

11 tributed to the search direction computations. For computing a search direction via an approximate solve of (12) or [15, Prob. (3.8)], all algorithms essentially used the same procedure. For simplicity, all algorithms considered variants of these subproblems in which the l 2 -norm trust region was replaced by an l -norm trust region so that the subproblems were bound-constrained. (The same modification was used in the Cauchy step calculations.) Then, starting with the Cauchy step as the initial solution estimate and defining the initial working set by the bounds identified as active by the Cauchy step, a projected conjugate gradient (PCG) method was used to compute an improved solution. During the PCG routine, if a new trial solution violated a bound constraint that was not already part of the working set, then this bound was added to the working set and the PCG routine was reinitialized. By contrast, if the reduced subproblem (corresponding to the current working set) was solved sufficiently accurately, then a check for termination was performed. In particular, multiplier estimates were computed for the working set elements. If these multiplier estimates were all nonnegative (or at least larger than a small negative number), then the subproblem was deemed to be solved and the routine terminated. Otherwise, an element corresponding to the most negative multiplier estimate was removed from the working set and the PCG routine was reinitialized. We do not claim that the precise manner in which we implemented this approach guaranteed convergence to an exact solution of the subproblem. However, the approach just described was based on well-established methods for solving bound-constrained quadratic optimization problems (QPs), and we found that it worked very well in our experiments. It should be noted that if, at any time, negative curvature was encountered in the PCG routine, then the solver terminated with the current PCG iterate. In this manner, the solutions were generally less accurate when negative curvature was encountered, but we claim that this did not have too adverse an effect on the performance of any of the algorithms. A few additional comments are necessary to describe our search direction computation procedures. First, it should be noted that for the line search algorithms, the Cauchy step calculation in Algorithm 2 was performed with (15) as stated (i.e., with q), but the above PCG routine to compute the search direction was applied to (12) without the convexification for the quadratic term. However, we claim that this choice remains consistent with the stated algorithms since, for all algorithm variants, we performed a sanity check after the computation of the search direction. In particular, the reduction in the model of the AL function yielded by the search direction was compared against that yielded by the corresponding Cauchy step. If the Cauchy step actually provided a better reduction in the model, then the computed search direction was replaced by the Cauchy step. In this sanity check for the line search algorithms, we computed the model reductions with the convexification of the quadratic term (i.e., with q), which implies that, overall, our implemented algorithm guaranteed Cauchy decrease in the appropriate model for all algorithms. Second, we remark that for the algorithms that employed a steering mechanism, we did not employ the same procedure to approximately solve (8) or [15, Prob. (3.4)]. Instead, we simply used the Cauchy steps as approximate solutions of these subproblems. Finally, we note that in the steering mechanism, we checked condition (16c) with the Cauchy steps for each subproblem, despite the fact that the search direction was computed as a more accurate solution of (12) or [15, Prob. (3.8)]. This had the effect that the algorithms were able to modify the penalty parameter via the steering mechanism prior to computing the search direction; only Cauchy steps for the subproblems were needed for steering. Most of the other algorithmic components were implemented similarly to the algorithm in [15]. As an example, for the computation of the estimates {ŷ k+1 } (which are required to satisfy (18)), we checked whether F L (x k+1, π(x k+1, y k, µ k )) 2 F L (x k+1, y k ) 2 ; if so, then we set ŷ k+1 π(x k+1, y k, µ k ), and otherwise we set ŷ k+1 y k. Furthermore, 11

12 for prescribed tolerances {κ opt, κ feas, µ min } (0, ), we terminated an algorithm with a declaration that a stationary point was found if F L (x k, y k ) κ opt and c k κ feas, (21) and terminated with a declaration that an infeasible stationary point was found if F FEAS (x k ) κ opt, c k > κ feas, and µ k µ min. (22) As in [15], this latter set of conditions shows that we did not declare that an infeasible stationary point was found unless the penalty parameter had already been reduced below a prescribed tolerance. This helps in avoiding premature termination when the algorithm could otherwise continue and potentially find a point satisfying (21), which was always the preferred outcome. Each algorithm terminated with a message of failure if neither (21) nor (22) was satisfied within k max iterations. It should also be noted that the problems were pre-scaled so that the l -norms of the gradients of the problem functions at the initial point would be less than or equal to a prescribed constant G > 0. The values for all of these parameters, as well as other input parameter required in the code, are summarized in Table 1. (Values for parameters related to updating the trust region radii required by [15, Algorithm 4] were set as in [15].) Table 1. Input parameter values used in our Matlab software. Parameter Value Parameter Value Parameter Value Parameter Value γ 0.5 κ 2 1 η vs 0.9 µ min 10 8 γ µ 0.1 κ ɛ 0.5 k max 10 4 γ t 0.1 ε r 10 4 µ 0 1 G 10 2 γ T 0.1 κ t 0.9 κ opt 10 5 κ 1 1 η s 10 4 κ feas 10 5 We close this subsection with a discussion of some additional differences between the algorithms as stated in this paper and in [15] and those implemented in our software. We claim that none of these differences represents a significant departure from the stated algorithms; we merely made some adjustments to simplify the implementation and to incorporate features that we found to work well in our experiments. First, while all algorithms use the input parameter γ µ given in Table 1 for decreasing the penalty parameter, we decrease the penalty parameter less significantly in the steering mechanism. In particular, in line 20 of Algorithm 3 and line 20 of [15, Algorithm 4], we replace γ µ with 0.7. Second, in the line search algorithms, rather than set the trust region radii as in (9) and (13) where δ appears as a constant value, we defined a dynamic sequence, call it {δ k }, that depended on the step-size sequence {α k }. In this manner, δ k replaced δ in (9) and (13) for all k. We initialized δ 0 1. Then, for all k, if α k = 1, then we set δ k δ k, and if α k < 1, then we set δ k δ k. Third, to simplify our implementation, we effectively ignored the imposed bounds on the multiplier estimates by setting Y and Y 1. This choice implies that we always chose α y 1 in (19). Fourth, we initialized the target values as t 1 max{10 2, min{10 4, c k }} (23) and T 1 max{10 0, min{10 2, F L (x k, y k ) }}. (24) 12

13 Finally, in AAL-LS-safe and AAL-TR-safe, we safeguard the steering procedure by shutting it off whenever the penalty parameter was smaller than a prescribed tolerance. Specifically, we considered the while condition in line 19 of Algorithm 3 and line 19 of [15, Algorithm 4] to be satisfied whenever µ k Results on CUTEst test problems We tested our Matlab algorithms on the subset of problems from the CUTEst [24] collection that have at least one general constraint and at most 1000 variables and 1000 constraints. This set contains 383 test problems. However, the results that we present in this section are only for those problems for which at least one of our six solvers obtained a successful result, i.e., where (21) or (22) was satisfied. This led to a set of 323 problems that are represented in the numerical results in this section. To illustrate the performance of our Matlab software, we use performance profiles as introduced by Dolan and Moré [17] to provide a visual comparison of different measures of performance. Consider a performance profile that measures performance in terms of required iterations until termination. For such a profile, if the graph associated with an algorithm passes through the point (α, 0.β), then, on β% of the problems, the number of iterations required by the algorithm was less than 2 α times the number of iterations required by the algorithm that required the fewest number of iterations. At the extremes of the graph, an algorithm with a higher value on the vertical axis may be considered a more efficient algorithm, whereas an algorithm on top at the far right of the graph may be considered more reliable. Since, for most problems, comparing values in the performance profiles for large values of α is not enlightening, we truncated the horizontal axis at 16 and simply remark on the numbers of failures for each algorithm. Figures 1 and 2 show the results for the three line search variants, namely BAL-LS, AAL-LS, and AAL-LS-safe. The numbers of failures for these algorithms were 25, 3, and 16, respectively. The same conclusion may be drawn from both profiles: the steering variants (with and without safeguarding) were both more efficient and more reliable than the basic algorithm, where efficiency is measured by either the number of iterations (Figure 1) or the number of function evaluations (Figure 2) required. We display the profile for the number of function evaluations required since, for a line search algorithm, this value is always at least as large as the number of iterations, and will be strictly greater whenever backtracking is required to satisfy (17) (yielding α k < 1). From these profiles, one may observe that unrestricted steering (in AAL-LS) yielded superior performance to restricted steering (in AAL-LS-safe) in terms of both efficiency and reliability; this suggests that safeguarding the steering mechanism may diminish its potential benefits BAL-LS AAL-LS AAL-LS-safe Figure 1. Performance profile for iterations: line search algorithms on the CUTEst set. 0.2 BAL-LS AAL-LS AAL-LS-safe Figure 2. Performance profile for function evaluations: line search algorithms on the CUTEst set. Figures 3 and 4 show the results for the three trust region variants, namely BAL-TR, 13

14 AAL-TR, and AAL-TR-safe, the numbers of failures for which were 30, 12, and 20, respectively. Again, as for the line search algorithms, the same conclusion may be drawn from both profiles: the steering variants (with and without safeguarding) are both more efficient and more reliable than the basic algorithm, where now we measure efficiency by either the number of iterations (Figure 3) or the number of gradient evaluations (Figure 4) required before termination. We observe the number of gradient evaluations here (as opposed to the number of function evaluations) since, for a trust region algorithm, this value is never larger than the number of iterations, and will be strictly smaller whenever a step is rejected and the trust-region radius is decreased because of insufficient decrease in the AL function. These profiles also support the other observation that was made by the results for our line search algorithms, i.e., that unrestricted steering may be superior to restricted steering in terms of efficiency and reliability BAL-TR AAL-TR AAL-TR-safe Figure 3. Performance profile for iterations: trust region algorithms on the CUTEst set. 0.2 BAL-TR AAL-TR AAL-TR-safe Figure 4. Performance profile for gradient evaluations: trust region algorithms on the CUTEst set. The performance profiles in Figures 1 4 suggest that steering has practical benefits, and that safeguarding the procedure may limit its potential benefits. However, to be more confident in these claims, one should observe the final penalty parameter values typically produced by the algorithms. These observations are important since one may be concerned whether the algorithms that employ steering yield final penalty parameter values that are often significantly smaller than those yielded by basic AL algorithms. To investigate this possibility in our experiments, we collected the final penalty parameter values produced by all six algorithms; the results are in Table 2. The column titled µ final gives a range for the final value of the penalty parameter. (For example, the value 27 in the BAL-LS column indicates that the final penalty parameter value computed by our basic line search AL algorithm fell in the range [10 2, 10 1 ) for 27 of the problems.) Table 2. Numbers of CUTEst problems for which the final penalty parameter values were in the given ranges. µ final BAL-LS AAL-LS AAL-LS-safe BAL-TR AAL-TR AAL-TR-safe [10 1, 1) [10 2, 10 1 ) [10 3, 10 2 ) [10 4, 10 3 ) [10 5, 10 4 ) [10 6, 10 5 ) (0, 10 6 ) We remark on two observations about the data in Table 2. First, as may be expected, the algorithms that employ steering typically reduce the penalty parameter below its ini- 14

15 tial value on some problems on which the other algorithms do not reduce it at all. This, in itself, is not a major concern, since a reasonable reduction in the penalty parameter may cause an algorithm to locate a stationary point more quickly. Second, we remark that the number of problems for which the final penalty parameter was very small (say, less than 10 4 ) was similar for all algorithms, even those that employed steering. This suggests that while steering was able to aid in guiding the algorithms toward constraint satisfaction, the algorithms did not reduce the value to such a small value that feasibility became the only priority. Overall, our conclusion from Table 2 is that steering typically decreases the penalty parameter more than does a traditonal updating scheme, but one should not expect that the final penalty parameter value will be reduced unnecessarily small due to steering; rather, steering can have the intended benefit of improving efficiency and reliability by guiding a method toward constraint satisfaction more quickly Results on COPS test problems We also tested our Matlab software on the large-scale constrained problems available in the COPS [5] collection. This test set was designed to provide difficult test cases for nonlinear optimization software; the problems include examples from fluid dynamics, population dynamics, optimal design, mesh smoothing, and optimal control. For our purposes, we solved the smallest versions of the AMPL models [1, 19] provided in the collection. All of our solvers failed to solve the problems named chain, dirichlet, henon, lane emden, and robot1, so these problems were excluded. The remaining set consisted of the following 17 problems: bearing, camshape, catmix, channel, elec, gasoil, glider, marine, methanol, minsurf, pinene, polygon, rocket, steering, tetra, torsion, and triangle. Since the size of this test set is relatively small, we have decided to display pair-wise comparisons of algorithms in the manner suggested in [30]. That is, for a performance measure of interest (e.g., number of iterations required until termination), we compare solvers, call them A and B, on problem j with the logarithmic outperforming factor { r j AB := log 2(m j A /mj B ), where m j A is the measure for A on problem j m j B is the measure for B on problem j. (25) Therefore, if the measure of interest is iterations required, then r j AB = p would indicate that solver A required 2 p the iterations required by solver B. For all plots, we focus our attention on the range p [ 2, 2]. The results of our experiments are given in Figures 5 8. For the same reasons as discussed in 3.1.2, we display results for iterations and function evaluations for the line search algorithms, and display results for iterations and gradient evaluations for the trust region algorithms. In addition, here we ignore the results for AAL-LS-safe and AAL-TR-safe since, as in the results in 3.1.2, we did not see benefits in safeguarding the steering mechanism. In each figure, a positive (negative) bar indicates that the algorithm whose name appears above (below) the horizontal axis yielded a better value for the measure on a particular problem. The results are displayed according to the order of the problems listed in the previous paragraph. In Figures 5 and 6 for the line search algorithms, the red bars for problems catmix and polygon indicate that AAL-LS failed on the former and BAL-LS failed on the latter; similarly, in Figures 7 and 8 for the trust region algorithms, the red bar for catmix indicates that AAL-TR failed on it. The results in Figures 5 and 6 indicate that AAL-LS more often outperforms BAL-LS in terms of iterations and functions evaluations, though the advantage is not overwhelming. On the other hand, it is clear from Figures 7 and 8 that, despite the one failure, AAL-TR is generally superior to BAL-TR. We conclude from these results that steering was beneficial 15

16 on this test set, especially in terms of the trust region methods BAL LS 1 BAL LS AAL LS 1 AAL LS Figure 5. Outperforming factors for iterations: line search algorithms on the COPS set. Figure 6. Outperforming factors for function evaluations: line search algorithms on the COPS set BAL TR 1 BAL TR AAL TR 1 AAL TR Figure 7. Outperforming factors for iterations: trust region algorithms on the COPS set. Figure 8. Outperforming factors for gradient evaluations: trust region algorithms on the COPS set Results on optimal power flow (OPF) test problems As a third and final set of experiments for our Matlab software, we tested our algorithms on a collection of optimal power flow (OPF) problems modeled in AMPL using data sets obtained from MATPOWER [37]. OPF problems represent a challenging set of nonconvex problems. The active and reactive power flow and the network balance equations give rise to equality constraints involving nonconvex functions while the inequality constraints are linear and result from placing operating limits on quantities such as flows, voltages, and various control variables. The control variables include the voltages at generator buses and the active-power output of the generating units. The state variables consist of the voltage magnitudes and angles at each node as well as reactive and active flows in each link. Our test set was comprised of 28 problems modeled on systems having 14 to 662 nodes from the IEEE test set. In particular, there are seven IEEE systems, each modeled in four different ways: (i) in Cartesian coordinates; (ii) in polar coordinates; (iii) with basic approximations to the sin and cos functions in the problem functions; and (iv) with linearized constraints based on DC power flow equations (in place of AC power flow). It should be noted that while linearizing the constraints in 16