NUMERICAL OPTIMIZATION J. Ch. Gilbert
Numerical optimization (past) The discipline deals with the classical smooth (nonconvex) problem min {f(x) : c E (x) = 0, c I (x) 0}. Applications: variable added lens design (Essilor), seismic tomography (IFP), tire industry (Michelin). Contributions to interior point (IP) methods Technique: solving min {f(x) µ P m i=1 log s i : c E (x) = 0, c I (x) + s = 0 for µ 0. Theoretical convergence studies of the quasi-newton version of the approach (with P. Armand, S. Jégou). Introduction and theoretical convergence study of one of the 1st nonlinear IP approach (with R. Byrd, J. Nocedal). Ill-behaved central paths for convex problems (with E. Karas, C. Gonzaga). Development of the prorotype general purpose solver OPINL (with A. Fuduli). Contributions to the SQP (= Newton) approach Technique: solving a sequence of (convex) QP s min {g d+ 1 2 d Hd : c E (x)+c E (x)d = 0, c I(x)+c I (x)d 0}. Truncated-Newton version with line-search (with L. Chauvier, A. Fuduli). Global linear convergence of the AL approach for solving a convex QP (with F. Delbos). Development of the general SQP solver SQPAL using the AL for solving QP s (in / with IFP). PhD thesis: E. Karas (2002), X. Jonsson (2002), F. Delbos (2004). A book: Numerical Optimization: Theoretical and Practical Aspects, J.F. Bonnans, J.Ch. Gilbert, C. Lemaréchal, and C.A. Sagastizábal, Springer, 2003. J.Ch. Gilbert INRIA-Rocquencourt October 14, 2004 2
Numerical optimization: back to SQP The discipline deals with the classical smooth (nonconvex) problem (P EI ) < : min x f(x) c E (x) = 0 c I (x) 0. Applications: variable added lens design (Essilor), seismic tomography (IFP), tire industry (Michelin), potentially many more... The sequential quadratic programming (SQP) approach (mid 70) It is a Newton-like method. It solves a sequence of QP s. < : min d g d + 1 2 d Hd c E (x) + c E (x)d = 0 c I (x) + c I (x)d 0. Properties: few iterations, QP expensive to solve. Interior point (IP) approach (late 90) It is a penalty method. It solves a sequence of nonlinear OP s without inequalities. < : min x f(x) µ P m i=1 log s i c E (x) = 0 c I (x) + s = 0. Properties: more iterations, LS less expensive to solve, ill-conditioned LS. For very large problems: SQP looks better that IP! J.Ch. Gilbert INRIA-Rocquencourt October 14, 2004 3
Solving a quadratic optimization problem (QP) The problem to solve Let Q S n and C be m n. (QP) j minx 1 2 x Qx + q x l Cx u. Difficulties: inequality constraints Q 0 = NP-hardness Standard approach within SQP (e.g., SNOPT) active set method Solves for a sequence of updated working set W {1,..., m} and implicit bounds l Cx u: j minx 1 2 x Qx + q x C i x = l i or u i, for i W. Properties: finite termination, W changes slowly. Forgotten approach augmented Lagrangian (AL) method (requires Q 0) Solves for a sequence of multipliers updated by λ + = λ + r(cx + y + ): (ALP) j 1 minx 2 x Qx + q x + λ (Cx y) + r 2 Cx y 2 l y u. Properties: (ALP) solved by GP-AS-CG (for example): efficient since only bound constraints and FIAC property, r can change at each iteration, infinite termination, but fast convergence with a global convergence rate: for some L > 0, at each iteration Cx + y + Cx y min 1, L «. r J.Ch. Gilbert INRIA-Rocquencourt October 14, 2004 4
Solving large scale seismic tomography problems with SQPAL QPAL and SQPAL QPAL: solver of (QP) with the AL approach (Fortran-90 version at IFP, Matlab version at Inria). SQPAL: SQP algorithm to solve (P EI ) using QPAL as QP solver. A seismic tomography problem (with F. Delbos, R. Glowinski, D. Sinoquet) Can solve problems with up to n 1.5 10 4 variables and up to m 10 4 constraints. J.Ch. Gilbert INRIA-Rocquencourt October 14, 2004 5
Numerical experiments with QPAL ncg 10 5 10 4 Experiments for n = 1,..., 10 3, m = max(1, n/2 ), and random data. 10 3 10 2 10 1 Total number of CG iterations n CG in terms of n. the behavior looks linear : n CG 6.67n 1.16 (n CG O(n 1.5 ) for IP). 10 0 10 0 10 1 10 2 10 3 n 12 ncgrel 10 6 4 The figure gives n CG w.r.t. n CG for the unconstrained problem. The ratio looks bounded (O(n 0.5 ) for IP). 2 0 10 0 10 1 10 2 10 3 n J.Ch. Gilbert INRIA-Rocquencourt October 14, 2004 6
The future of QPAL Open questions Behavior when the QP is infeasible (extension of a work by R. Glowinski). Solving inexactly the bound constrained augmented Lagrangian (see Dostál, Friedlander, and Santos). Make more precise the finite identification property of the AL approach. Use of QPAL in SQP with trust regions. Polynomial complexity of the algorithm? (suggested by the previous numerical experiments) Comparison with IP. Can we say something when the QP is nonconvex? (... close to P = NP!) Possible application to shape optimization (aeronautic and tire industry) and many other domains. J.Ch. Gilbert INRIA-Rocquencourt October 14, 2004 7
Optimization problems with complementarity constraints (MPEC) MPEC (mathematical programming with equilibrium constraints): an intensive international activity on the problem < : min x f(x) c I (x) = 0 and c I (x) 0 0 p(x) q(x) 0 complementarity constraints. Difficulty: complementarity constraints are not qualified! Numerical methods IP approach: already widely explored. Renewal of the SQP approach (for some classes of problems). Does a sequential QP+(linear complementarity constraint) can be used? Can an augmented Lagrangian approach be useful? Possible applications Robotics, contact problems (tire industry, Michelin),... Optimization of many systems with threshold effects: stratigraphic identification (IFP),... Special cases of bilevel optimization problems: yield management (in networks, France Télécom),... J.Ch. Gilbert INRIA-Rocquencourt October 14, 2004