Lecture 15: SQP methods for equality constrained optimization
|
|
- Bernard White
- 5 years ago
- Views:
Transcription
1 Lecture 15: SQP methods for equality constrained optimization Coralia Cartis, Mathematical Institute, University of Oxford C6.2/B2: Continuous Optimization Lecture 15: SQP methods for equality constrained optimization p. 1/22
2 Nonlinear equality-constrained problems again! min f(x) subject to c(x) = 0, (ecp) x R n where f : R n R, c = (c 1,...,c m ) : R n R m are C 1 or C 2 (when needed), with m n. easily generalized to inequality constraints... but may be better to use interior-point methods for these. (ecp): attempt to find local solutions or at least KKT points: L(x,y) = f(x) J(x) T y = 0 and c(x) = 0 nonlinear and square system in x and y (linear in y) = use Newton s method to find a correction (s,w) to (x,y) 2 L(x,y) J(x) T J(x) 0 s w = L(x,y) c(x) Lecture 15: SQP methods for equality constrained optimization p. 2/22
3 Alternative formulations Recall: the Lagrangian function is L(x,y) = f(x) y T c(x). unsymmetric: 2 L(x,y) J(x) T J(x) 0 s w = L(x,y) c(x) or symmetric: 2 L(x,y) J(x) T J(x) 0 s w = L(x,y) c(x) or (with y + = y + w) symmetric: 2 L(x,y) J(x) T J(x) 0 s y + = [unsymmetric also possible] f(x) c(x) Lecture 15: SQP methods for equality constrained optimization p. 3/22
4 Some more details often approximate with symmetric B 2 L(x,y) = e.g. B J(x)T J(x) 0 solve system using s y + unsymmetric (LU) factorization of = symmetric (indefinite) factorization of f(x) c(x) B J(x)T J(x) 0 B J(x)T J(x) 0 symmetric factorizations of B and the Schur complement J(x)B 1 J(x) T iterative method (GMRES(k), MINRES, CG within N(J), etc.) Lecture 15: SQP methods for equality constrained optimization p. 4/22
5 An alternative interpretation QP : minimize s IR n f(x) T s st Bs subject to J(x)s = c(x) QP = quadratic program first-order model of constraints c(x + s) second-order model of objective f(x + s)... but B includes curvature of constraints Solution to (QP) satisfies B J(x)T J(x) 0 s y + = f(x) c(x) Lecture 15: SQP methods for equality constrained optimization p. 5/22
6 Sequential quadratic programming - SQP or successive quadratic programming or recursive quadratic programming (RQP) A basic SQP method Given (x 0,y 0 ), set k = 0 Until convergence iterate: Compute a suitable symmetric B k using (x k,y k ) Find the solution of (QP k ) s k = arg min s IR n f(xk ) T s + 1 2s T B k s subject to J(x k )s = c(x k ) along with associated Lagrange multiplier estimates y k+1. Set x k+1 = x k + s k and let k := k + 1. Lecture 15: SQP methods for equality constrained optimization p. 6/22
7 Sequential quadratic programming - SQP... Advantanges of SQP: simple fast quadratically convergent with B k = 2 L(x k,y k ) superlinearly convergent with good B k 2 L(x k,y k ) don t actually need B k 2 L(x k,y k ) Issues with pure SQP how to choose B k? [similar to Newton s method for unconstrained opt and systems] what if QP k is unbounded from below? and when? how do we globalize the SQP iteration? Lecture 15: SQP methods for equality constrained optimization p. 7/22
8 Sequential quadratic programming - SQP... The QP k subproblem: minimize s IR n f(x k ) T s + 1 2sB k s subject to J(x k )s = c(x k ) need constraints to be consistent OK if J(x k ) is full rank need B k to be positive (semi-) definite when J(x k )s = 0 N T k Bk N k positive (semi-) definite where the columns of N k form a basis for the null space of J(x k ) Bk J(x k ) T J(x k ) 0 (is non-singular and) has m ve eigenvalues Lecture 15: SQP methods for equality constrained optimization p. 8/22
9 Linesearch SQP methods s k = arg min s IR n f(xk ) T s + 1 2s T B k s subject to J(x k )s = c(x k ) Linesearch SQP: Set x k+1 = x k + α k s k, where α k is chosen so that Φ(x k + α k s k,σ k ) < Φ(x k,σ k ) where Φ(x,σ) is a suitable merit function and σ k are parameters. Recall unconstrained GLM: crucial that s k is a descent direction for Φ(x,σ k ) at x k ; normally require that B k is positive definite. Lecture 15: SQP methods for equality constrained optimization p. 9/22
10 Suitable merit functions for SQP Recall the quadratic penalty function: Φ(x,σ) = f(x) + 1 2σ c(x) 2 Theorem 30: Suppose that B k is positive definite, and that (s k,y k+1 ) are the SQP search direction and its associated Lagrange multiplier estimates for the problem the (ecp) problem at x k. Then if x k is not a KKT point of (ecp), then s k is a descent direction for the quadratic penalty function Φ(x,σ k ) at x k whenever σ k c(xk ) y k+1. Lecture 15: SQP methods for equality constrained optimization p. 10/22
11 Suitable merit functions for SQP... Proof of Theorem 30: SQP direction s k and associated multiplier estimates y k+1 satisfy B k s k J(x k ) T y k+1 = f(x k ) ( ) J(x k )s k = c(x k ) ( ) (*) + (**) = (s k ) T f(x k ) = (s k ) T B k s k + (s k ) T J(x k ) T y k+1 (**) = c(x k ) T J(x k )s k = c(x k ) 2. = (s k ) T B k s k c(x k ) T y k+1 These, the positive definiteness of B k, the Cauchy-Schwarz inequality, the required bound on σ k, and s k 0 if x k is not critical = Lecture 15: SQP methods for equality constrained optimization p. 11/22
12 Suitable merit functions for SQP... Proof of Theorem 30: (continued) (s k ) T x Φ(x k,σ k ) = (s k ) ( f(x T k ) + 1 ) ) T c(x k ) σ kj(xk = (s k ) T B k s k c(x k ) T y k+1 c(xk ) 2 ( ) σ k c(x k < c(x k ) ) y k+1 0, and so s k is descent for Φ(x k,σ k ). Other suitable merit functions: The non-differentiable exact penalty function: [widely used] Ψ(x,ρ) = f(x) + ρ c(x), where can be any norm and ρ > 0. recall that minimizers of (ecp) correspond to those of Ψ(x, ρ) for ρ sufficiently large (and finite! ρ > y D ); equivalent of Th 30 holds (ρ k y k+1 D ). σ k Lecture 15: SQP methods for equality constrained optimization p. 12/22
13 The Maratos effect merit function may prevent acceptance of the full SQP step (so α k 1) arbitrarily close to x = slow convergence x * 0.1 x k +s k x k f(x) = 2(x x2 2 1) x 1 and c(x) = x x2 2 1; solution: x = (1,0), y = 2. 3 Here: l 1 non-differentiable exact penalty function (ρ = 1) but other merit fcts. have similar behaviour. Lecture 15: SQP methods for equality constrained optimization p. 13/22
14 Avoiding the Maratos effect The Maratos effect occurs because the curvature of the constraints is not adequately represented by linearization in the SQP model: c(x k + s k ) = O( s k 2 ) = need to correct for this curvature Use a second-order correction from x k + s k : c(x k + s k + s k C ) = o( sk 2 ). Also, do not want to destroy potential for fast convergence = s k C = o(sk ). Lecture 15: SQP methods for equality constrained optimization p. 14/22
15 Popular second-order corrections minimum norm solution to c(x k + s k ) + J(x k + s k )s k C = 0 I J(x k + s k ) T J(x k + s k ) 0 sk C y k+1 C = 0 c(x k + s k ) minimum norm solution to c(x k + s k ) + J(x k )s k C = 0 I J(xk ) T J(x k ) 0 sk C y k+1 C = 0 c(x k + s k ) another SQP step from x k + s k...and so on... 2 L(x k + s k,y+ k ) J(xk + s k ) T J(x k + s k ) 0 sk C y k+1 C = L(xk + s k ) c(x k + s k ) Lecture 15: SQP methods for equality constrained optimization p. 15/22
16 Second-order corrections in action x * x k +s k +s c k 0.1 x k +s k x k f(x) = 2(x x 2 2 1) x 1 and c(x) = x x 2 2 1; solution: x = (1,0), y = 3 2. Here: l 1 non-differentiable exact penalty function (ρ = 1) but other merit fcts. have similar behaviour. fast convergence x k + s k + s k C reduces Φ = global convergence Lecture 15: SQP methods for equality constrained optimization p. 16/22
17 Trust-region SQP methods Obvious trust-region approach: s k = arg min s IR n f(xk ) T s st B k s subject to J(x k )s = c(x k ) and s k do not require that B k be positive definite = can use B k = 2 L(x k,y k ) if k < CRIT where CRIT def = min s subject to J(x k )s = c(x k ) = no solution to trust-region subproblem = simple trust-region approach to SQP is flawed if c(x k ) 0 = need to consider alternatives Lecture 15: SQP methods for equality constrained optimization p. 17/22
18 Ã È ÈÈÈÈ ÈÕ Æ Infeasibility of the SQP step Ì Ð Ò Ö Þ ÓÒ ØÖ ÒØ ØÖÙ Ø Ö ÓÒ ¹ Ì + Ì ÒÓÒÐ Ò Ö ÓÒ ØÖ ÒØ Lecture 15: SQP methods for equality constrained optimization p. 18/22
19 An alternative: Composite-step methods Aim: find composite step s k = n k + t k where the normal step n k moves towards feasibility of the linearized constraints (within the trust region) J(x k )n k + c(x k ) < c(x k ) (model objective may get worse) the tangential step t k reduces the model objective function (within the trust-region) without sacrificing feasibility obtained from n k J(x k )(n k + t k ) = J(x k )n k = J(x k )t k = 0 Lecture 15: SQP methods for equality constrained optimization p. 19/22
20 Ò È ÈÈÈÈ ÈÕ Ò Normal and tangential steps Points on dotted line are all potential tangential steps Ì Ð Ò Ö Þ ÓÒ ØÖ ÒØ Æ Ö Ø ÔÓ ÒØ ÓÒ Ð Ò Ö Þ ÓÒ ØÖ ÒØ ÐÓ ØÓ Ò Ö Ø ÔÓ ÒØ ØÖÙ Ø Ö ÓÒ ¹ Ì + + Lecture 15: SQP methods for equality constrained optimization p. 20/22
21 Constraint reduction [Byrd-Omojokun] normal step: replace J(x k )s = c(x k ) and s k by approximately minimize J(x k )n + c(x k ) subject to n k tangential step: (approximate) arg min t IR n subject to ( f(x k ) + B k n k ) T t + 1 2t T B k t J(x k )t = 0 and n k + t k use conjugate gradients to solve both subproblems = Cauchy conditions satisfied in both cases globally convergent basis of successful KNITRO package Lecture 15: SQP methods for equality constrained optimization p. 21/22
22 Other alternatives composite step SQP methods constraint relaxation (Vardi) - not addressed here constraint reduction (Byrd Omojokun) -covered constraint lumping (Celis Dennis Tapia) - not addressed the Sl p QP method of Fletcher - not addressed here the filter-sqp approach of Fletcher and Leyffer - not addressed An important class of methods not addressed: active-set methods for linearly-constrained nonlinear problems (ie, generalization of simplex methods from the LP to the nonlinear case). Lecture 15: SQP methods for equality constrained optimization p. 22/22
Lecture 11 and 12: Penalty methods and augmented Lagrangian methods for nonlinear programming
Lecture 11 and 12: Penalty methods and augmented Lagrangian methods for nonlinear programming Coralia Cartis, Mathematical Institute, University of Oxford C6.2/B2: Continuous Optimization Lecture 11 and
More informationNonlinear Optimization: What s important?
Nonlinear Optimization: What s important? Julian Hall 10th May 2012 Convexity: convex problems A local minimizer is a global minimizer A solution of f (x) = 0 (stationary point) is a minimizer A global
More informationMethods for Unconstrained Optimization Numerical Optimization Lectures 1-2
Methods for Unconstrained Optimization Numerical Optimization Lectures 1-2 Coralia Cartis, University of Oxford INFOMM CDT: Modelling, Analysis and Computation of Continuous Real-World Problems Methods
More informationAlgorithms for Constrained Optimization
1 / 42 Algorithms for Constrained Optimization ME598/494 Lecture Max Yi Ren Department of Mechanical Engineering, Arizona State University April 19, 2015 2 / 42 Outline 1. Convergence 2. Sequential quadratic
More information1 Computing with constraints
Notes for 2017-04-26 1 Computing with constraints Recall that our basic problem is minimize φ(x) s.t. x Ω where the feasible set Ω is defined by equality and inequality conditions Ω = {x R n : c i (x)
More informationAlgorithms for constrained local optimization
Algorithms for constrained local optimization Fabio Schoen 2008 http://gol.dsi.unifi.it/users/schoen Algorithms for constrained local optimization p. Feasible direction methods Algorithms for constrained
More informationA Trust Funnel Algorithm for Nonconvex Equality Constrained Optimization with O(ɛ 3/2 ) Complexity
A Trust Funnel Algorithm for Nonconvex Equality Constrained Optimization with O(ɛ 3/2 ) Complexity Mohammadreza Samadi, Lehigh University joint work with Frank E. Curtis (stand-in presenter), Lehigh University
More informationPart 4: Active-set methods for linearly constrained optimization. Nick Gould (RAL)
Part 4: Active-set methods for linearly constrained optimization Nick Gould RAL fx subject to Ax b Part C course on continuoue optimization LINEARLY CONSTRAINED MINIMIZATION fx subject to Ax { } b where
More informationSF2822 Applied Nonlinear Optimization. Preparatory question. Lecture 9: Sequential quadratic programming. Anders Forsgren
SF2822 Applied Nonlinear Optimization Lecture 9: Sequential quadratic programming Anders Forsgren SF2822 Applied Nonlinear Optimization, KTH / 24 Lecture 9, 207/208 Preparatory question. Try to solve theory
More informationPart 3: Trust-region methods for unconstrained optimization. Nick Gould (RAL)
Part 3: Trust-region methods for unconstrained optimization Nick Gould (RAL) minimize x IR n f(x) MSc course on nonlinear optimization UNCONSTRAINED MINIMIZATION minimize x IR n f(x) where the objective
More informationLecture 13: Constrained optimization
2010-12-03 Basic ideas A nonlinearly constrained problem must somehow be converted relaxed into a problem which we can solve (a linear/quadratic or unconstrained problem) We solve a sequence of such problems
More informationScientific Computing: An Introductory Survey
Scientific Computing: An Introductory Survey Chapter 6 Optimization Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002. Reproduction permitted
More informationScientific Computing: An Introductory Survey
Scientific Computing: An Introductory Survey Chapter 6 Optimization Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002. Reproduction permitted
More information4TE3/6TE3. Algorithms for. Continuous Optimization
4TE3/6TE3 Algorithms for Continuous Optimization (Algorithms for Constrained Nonlinear Optimization Problems) Tamás TERLAKY Computing and Software McMaster University Hamilton, November 2005 terlaky@mcmaster.ca
More informationPart 5: Penalty and augmented Lagrangian methods for equality constrained optimization. Nick Gould (RAL)
Part 5: Penalty and augmented Lagrangian methods for equality constrained optimization Nick Gould (RAL) x IR n f(x) subject to c(x) = Part C course on continuoue optimization CONSTRAINED MINIMIZATION x
More informationE5295/5B5749 Convex optimization with engineering applications. Lecture 8. Smooth convex unconstrained and equality-constrained minimization
E5295/5B5749 Convex optimization with engineering applications Lecture 8 Smooth convex unconstrained and equality-constrained minimization A. Forsgren, KTH 1 Lecture 8 Convex optimization 2006/2007 Unconstrained
More information6.252 NONLINEAR PROGRAMMING LECTURE 10 ALTERNATIVES TO GRADIENT PROJECTION LECTURE OUTLINE. Three Alternatives/Remedies for Gradient Projection
6.252 NONLINEAR PROGRAMMING LECTURE 10 ALTERNATIVES TO GRADIENT PROJECTION LECTURE OUTLINE Three Alternatives/Remedies for Gradient Projection Two-Metric Projection Methods Manifold Suboptimization Methods
More informationMS&E 318 (CME 338) Large-Scale Numerical Optimization
Stanford University, Management Science & Engineering (and ICME) MS&E 318 (CME 338) Large-Scale Numerical Optimization 1 Origins Instructor: Michael Saunders Spring 2015 Notes 9: Augmented Lagrangian Methods
More informationAM 205: lecture 19. Last time: Conditions for optimality, Newton s method for optimization Today: survey of optimization methods
AM 205: lecture 19 Last time: Conditions for optimality, Newton s method for optimization Today: survey of optimization methods Quasi-Newton Methods General form of quasi-newton methods: x k+1 = x k α
More informationLectures 9 and 10: Constrained optimization problems and their optimality conditions
Lectures 9 and 10: Constrained optimization problems and their optimality conditions Coralia Cartis, Mathematical Institute, University of Oxford C6.2/B2: Continuous Optimization Lectures 9 and 10: Constrained
More informationMultidisciplinary System Design Optimization (MSDO)
Multidisciplinary System Design Optimization (MSDO) Numerical Optimization II Lecture 8 Karen Willcox 1 Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Today s Topics Sequential
More informationConstrained Nonlinear Optimization Algorithms
Department of Industrial Engineering and Management Sciences Northwestern University waechter@iems.northwestern.edu Institute for Mathematics and its Applications University of Minnesota August 4, 2016
More informationDetermination of Feasible Directions by Successive Quadratic Programming and Zoutendijk Algorithms: A Comparative Study
International Journal of Mathematics And Its Applications Vol.2 No.4 (2014), pp.47-56. ISSN: 2347-1557(online) Determination of Feasible Directions by Successive Quadratic Programming and Zoutendijk Algorithms:
More information5 Handling Constraints
5 Handling Constraints Engineering design optimization problems are very rarely unconstrained. Moreover, the constraints that appear in these problems are typically nonlinear. This motivates our interest
More informationLecture 3: Linesearch methods (continued). Steepest descent methods
Lecture 3: Linesearch methods (continued). Steepest descent methods Coralia Cartis, Mathematical Institute, University of Oxford C6.2/B2: Continuous Optimization Lecture 3: Linesearch methods (continued).
More informationAM 205: lecture 19. Last time: Conditions for optimality Today: Newton s method for optimization, survey of optimization methods
AM 205: lecture 19 Last time: Conditions for optimality Today: Newton s method for optimization, survey of optimization methods Optimality Conditions: Equality Constrained Case As another example of equality
More informationISM206 Lecture Optimization of Nonlinear Objective with Linear Constraints
ISM206 Lecture Optimization of Nonlinear Objective with Linear Constraints Instructor: Prof. Kevin Ross Scribe: Nitish John October 18, 2011 1 The Basic Goal The main idea is to transform a given constrained
More informationOutline. Scientific Computing: An Introductory Survey. Optimization. Optimization Problems. Examples: Optimization Problems
Outline Scientific Computing: An Introductory Survey Chapter 6 Optimization 1 Prof. Michael. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002. Reproduction
More informationA New Penalty-SQP Method
Background and Motivation Illustration of Numerical Results Final Remarks Frank E. Curtis Informs Annual Meeting, October 2008 Background and Motivation Illustration of Numerical Results Final Remarks
More informationSurvey of NLP Algorithms. L. T. Biegler Chemical Engineering Department Carnegie Mellon University Pittsburgh, PA
Survey of NLP Algorithms L. T. Biegler Chemical Engineering Department Carnegie Mellon University Pittsburgh, PA NLP Algorithms - Outline Problem and Goals KKT Conditions and Variable Classification Handling
More informationInfeasibility Detection and an Inexact Active-Set Method for Large-Scale Nonlinear Optimization
Infeasibility Detection and an Inexact Active-Set Method for Large-Scale Nonlinear Optimization Frank E. Curtis, Lehigh University involving joint work with James V. Burke, University of Washington Daniel
More informationNonmonotone Trust Region Methods for Nonlinear Equality Constrained Optimization without a Penalty Function
Nonmonotone Trust Region Methods for Nonlinear Equality Constrained Optimization without a Penalty Function Michael Ulbrich and Stefan Ulbrich Zentrum Mathematik Technische Universität München München,
More informationOn Lagrange multipliers of trust-region subproblems
On Lagrange multipliers of trust-region subproblems Ladislav Lukšan, Ctirad Matonoha, Jan Vlček Institute of Computer Science AS CR, Prague Programy a algoritmy numerické matematiky 14 1.- 6. června 2008
More informationminimize x subject to (x 2)(x 4) u,
Math 6366/6367: Optimization and Variational Methods Sample Preliminary Exam Questions 1. Suppose that f : [, L] R is a C 2 -function with f () on (, L) and that you have explicit formulae for
More informationAn Inexact Newton Method for Nonlinear Constrained Optimization
An Inexact Newton Method for Nonlinear Constrained Optimization Frank E. Curtis Numerical Analysis Seminar, January 23, 2009 Outline Motivation and background Algorithm development and theoretical results
More informationAn Inexact Sequential Quadratic Optimization Method for Nonlinear Optimization
An Inexact Sequential Quadratic Optimization Method for Nonlinear Optimization Frank E. Curtis, Lehigh University involving joint work with Travis Johnson, Northwestern University Daniel P. Robinson, Johns
More informationA Trust-Funnel Algorithm for Nonlinear Programming
for Nonlinear Programming Daniel P. Robinson Johns Hopins University Department of Applied Mathematics and Statistics Collaborators: Fran E. Curtis (Lehigh University) Nic I. M. Gould (Rutherford Appleton
More informationAn Inexact Newton Method for Optimization
New York University Brown Applied Mathematics Seminar, February 10, 2009 Brief biography New York State College of William and Mary (B.S.) Northwestern University (M.S. & Ph.D.) Courant Institute (Postdoc)
More informationNonlinear Programming
Nonlinear Programming Kees Roos e-mail: C.Roos@ewi.tudelft.nl URL: http://www.isa.ewi.tudelft.nl/ roos LNMB Course De Uithof, Utrecht February 6 - May 8, A.D. 2006 Optimization Group 1 Outline for week
More informationWhat s New in Active-Set Methods for Nonlinear Optimization?
What s New in Active-Set Methods for Nonlinear Optimization? Philip E. Gill Advances in Numerical Computation, Manchester University, July 5, 2011 A Workshop in Honor of Sven Hammarling UCSD Center for
More informationConstrained Optimization
1 / 22 Constrained Optimization ME598/494 Lecture Max Yi Ren Department of Mechanical Engineering, Arizona State University March 30, 2015 2 / 22 1. Equality constraints only 1.1 Reduced gradient 1.2 Lagrange
More informationPOWER SYSTEMS in general are currently operating
TO APPEAR IN IEEE TRANSACTIONS ON POWER SYSTEMS 1 Robust Optimal Power Flow Solution Using Trust Region and Interior-Point Methods Andréa A. Sousa, Geraldo L. Torres, Member IEEE, Claudio A. Cañizares,
More informationTrust-Region SQP Methods with Inexact Linear System Solves for Large-Scale Optimization
Trust-Region SQP Methods with Inexact Linear System Solves for Large-Scale Optimization Denis Ridzal Department of Computational and Applied Mathematics Rice University, Houston, Texas dridzal@caam.rice.edu
More informationOptimization. Escuela de Ingeniería Informática de Oviedo. (Dpto. de Matemáticas-UniOvi) Numerical Computation Optimization 1 / 30
Optimization Escuela de Ingeniería Informática de Oviedo (Dpto. de Matemáticas-UniOvi) Numerical Computation Optimization 1 / 30 Unconstrained optimization Outline 1 Unconstrained optimization 2 Constrained
More informationSeminal papers in nonlinear optimization
Seminal papers in nonlinear optimization Nick Gould, CSED, RAL, Chilton, OX11 0QX, England (n.gould@rl.ac.uk) December 7, 2006 The following papers are classics in the field. Although many of them cover
More informationThe Squared Slacks Transformation in Nonlinear Programming
Technical Report No. n + P. Armand D. Orban The Squared Slacks Transformation in Nonlinear Programming August 29, 2007 Abstract. We recall the use of squared slacks used to transform inequality constraints
More informationQuiz Discussion. IE417: Nonlinear Programming: Lecture 12. Motivation. Why do we care? Jeff Linderoth. 16th March 2006
Quiz Discussion IE417: Nonlinear Programming: Lecture 12 Jeff Linderoth Department of Industrial and Systems Engineering Lehigh University 16th March 2006 Motivation Why do we care? We are interested in
More informationNonlinear Optimization Solvers
ICE05 Argonne, July 19, 2005 Nonlinear Optimization Solvers Sven Leyffer leyffer@mcs.anl.gov Mathematics & Computer Science Division, Argonne National Laboratory Nonlinear Optimization Methods Optimization
More informationSteering Exact Penalty Methods for Nonlinear Programming
Steering Exact Penalty Methods for Nonlinear Programming Richard H. Byrd Jorge Nocedal Richard A. Waltz April 10, 2007 Technical Report Optimization Technology Center Northwestern University Evanston,
More informationA PRIMAL-DUAL TRUST REGION ALGORITHM FOR NONLINEAR OPTIMIZATION
Optimization Technical Report 02-09, October 2002, UW-Madison Computer Sciences Department. E. Michael Gertz 1 Philip E. Gill 2 A PRIMAL-DUAL TRUST REGION ALGORITHM FOR NONLINEAR OPTIMIZATION 7 October
More informationSequential Quadratic Programming Methods
Sequential Quadratic Programming Methods Klaus Schittkowski Ya-xiang Yuan June 30, 2010 Abstract We present a brief review on one of the most powerful methods for solving smooth constrained nonlinear optimization
More informationConstrained optimization: direct methods (cont.)
Constrained optimization: direct methods (cont.) Jussi Hakanen Post-doctoral researcher jussi.hakanen@jyu.fi Direct methods Also known as methods of feasible directions Idea in a point x h, generate a
More informationPDE-Constrained and Nonsmooth Optimization
Frank E. Curtis October 1, 2009 Outline PDE-Constrained Optimization Introduction Newton s method Inexactness Results Summary and future work Nonsmooth Optimization Sequential quadratic programming (SQP)
More informationOptimization Methods
Optimization Methods Decision making Examples: determining which ingredients and in what quantities to add to a mixture being made so that it will meet specifications on its composition allocating available
More informationScientific Computing: An Introductory Survey
Scientific Computing: An Introductory Survey Chapter 6 Optimization Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002. Reproduction permitted
More informationPenalty and Barrier Methods General classical constrained minimization problem minimize f(x) subject to g(x) 0 h(x) =0 Penalty methods are motivated by the desire to use unconstrained optimization techniques
More informationStructural and Multidisciplinary Optimization. P. Duysinx and P. Tossings
Structural and Multidisciplinary Optimization P. Duysinx and P. Tossings 2018-2019 CONTACTS Pierre Duysinx Institut de Mécanique et du Génie Civil (B52/3) Phone number: 04/366.91.94 Email: P.Duysinx@uliege.be
More informationOptimization Problems with Constraints - introduction to theory, numerical Methods and applications
Optimization Problems with Constraints - introduction to theory, numerical Methods and applications Dr. Abebe Geletu Ilmenau University of Technology Department of Simulation and Optimal Processes (SOP)
More informationCONSTRAINED NONLINEAR PROGRAMMING
149 CONSTRAINED NONLINEAR PROGRAMMING We now turn to methods for general constrained nonlinear programming. These may be broadly classified into two categories: 1. TRANSFORMATION METHODS: In this approach
More informationRecent Adaptive Methods for Nonlinear Optimization
Recent Adaptive Methods for Nonlinear Optimization Frank E. Curtis, Lehigh University involving joint work with James V. Burke (U. of Washington), Richard H. Byrd (U. of Colorado), Nicholas I. M. Gould
More informationOn nonlinear optimization since M.J.D. Powell
On nonlinear optimization since 1959 1 M.J.D. Powell Abstract: This view of the development of algorithms for nonlinear optimization is based on the research that has been of particular interest to the
More information10 Numerical methods for constrained problems
10 Numerical methods for constrained problems min s.t. f(x) h(x) = 0 (l), g(x) 0 (m), x X The algorithms can be roughly divided the following way: ˆ primal methods: find descent direction keeping inside
More informationInexact-Restoration Method with Lagrangian Tangent Decrease and New Merit Function for Nonlinear Programming 1 2
Inexact-Restoration Method with Lagrangian Tangent Decrease and New Merit Function for Nonlinear Programming 1 2 J. M. Martínez 3 December 7, 1999. Revised June 13, 2007. 1 This research was supported
More informationCONVEXIFICATION SCHEMES FOR SQP METHODS
CONVEXIFICAION SCHEMES FOR SQP MEHODS Philip E. Gill Elizabeth Wong UCSD Center for Computational Mathematics echnical Report CCoM-14-06 July 18, 2014 Abstract Sequential quadratic programming (SQP) methods
More informationPenalty and Barrier Methods. So we again build on our unconstrained algorithms, but in a different way.
AMSC 607 / CMSC 878o Advanced Numerical Optimization Fall 2008 UNIT 3: Constrained Optimization PART 3: Penalty and Barrier Methods Dianne P. O Leary c 2008 Reference: N&S Chapter 16 Penalty and Barrier
More informationFeasible Interior Methods Using Slacks for Nonlinear Optimization
Feasible Interior Methods Using Slacks for Nonlinear Optimization Richard H. Byrd Jorge Nocedal Richard A. Waltz February 28, 2005 Abstract A slack-based feasible interior point method is described which
More informationLecture 18: Optimization Programming
Fall, 2016 Outline Unconstrained Optimization 1 Unconstrained Optimization 2 Equality-constrained Optimization Inequality-constrained Optimization Mixture-constrained Optimization 3 Quadratic Programming
More informationWritten Examination
Division of Scientific Computing Department of Information Technology Uppsala University Optimization Written Examination 202-2-20 Time: 4:00-9:00 Allowed Tools: Pocket Calculator, one A4 paper with notes
More informationUnconstrained optimization
Chapter 4 Unconstrained optimization An unconstrained optimization problem takes the form min x Rnf(x) (4.1) for a target functional (also called objective function) f : R n R. In this chapter and throughout
More informationx 2. x 1
Feasibility Control in Nonlinear Optimization y M. Marazzi z Jorge Nocedal x March 9, 2000 Report OTC 2000/04 Optimization Technology Center Abstract We analyze the properties that optimization algorithms
More information1. Introduction. We consider the general smooth constrained optimization problem:
OPTIMIZATION TECHNICAL REPORT 02-05, AUGUST 2002, COMPUTER SCIENCES DEPT, UNIV. OF WISCONSIN TEXAS-WISCONSIN MODELING AND CONTROL CONSORTIUM REPORT TWMCC-2002-01 REVISED SEPTEMBER 2003. A FEASIBLE TRUST-REGION
More information2.098/6.255/ Optimization Methods Practice True/False Questions
2.098/6.255/15.093 Optimization Methods Practice True/False Questions December 11, 2009 Part I For each one of the statements below, state whether it is true or false. Include a 1-3 line supporting sentence
More informationComputational Finance
Department of Mathematics at University of California, San Diego Computational Finance Optimization Techniques [Lecture 2] Michael Holst January 9, 2017 Contents 1 Optimization Techniques 3 1.1 Examples
More informationInexact Newton Methods and Nonlinear Constrained Optimization
Inexact Newton Methods and Nonlinear Constrained Optimization Frank E. Curtis EPSRC Symposium Capstone Conference Warwick Mathematics Institute July 2, 2009 Outline PDE-Constrained Optimization Newton
More informationLecture 15 Newton Method and Self-Concordance. October 23, 2008
Newton Method and Self-Concordance October 23, 2008 Outline Lecture 15 Self-concordance Notion Self-concordant Functions Operations Preserving Self-concordance Properties of Self-concordant Functions Implications
More informationAn Active Set Strategy for Solving Optimization Problems with up to 200,000,000 Nonlinear Constraints
An Active Set Strategy for Solving Optimization Problems with up to 200,000,000 Nonlinear Constraints Klaus Schittkowski Department of Computer Science, University of Bayreuth 95440 Bayreuth, Germany e-mail:
More informationOptimization and Root Finding. Kurt Hornik
Optimization and Root Finding Kurt Hornik Basics Root finding and unconstrained smooth optimization are closely related: Solving ƒ () = 0 can be accomplished via minimizing ƒ () 2 Slide 2 Basics Root finding
More informationHYBRID FILTER METHODS FOR NONLINEAR OPTIMIZATION. Yueling Loh
HYBRID FILTER METHODS FOR NONLINEAR OPTIMIZATION by Yueling Loh A dissertation submitted to The Johns Hopkins University in conformity with the requirements for the degree of Doctor of Philosophy. Baltimore,
More informationLecture 3. Optimization Problems and Iterative Algorithms
Lecture 3 Optimization Problems and Iterative Algorithms January 13, 2016 This material was jointly developed with Angelia Nedić at UIUC for IE 598ns Outline Special Functions: Linear, Quadratic, Convex
More informationComputational Optimization. Augmented Lagrangian NW 17.3
Computational Optimization Augmented Lagrangian NW 17.3 Upcoming Schedule No class April 18 Friday, April 25, in class presentations. Projects due unless you present April 25 (free extension until Monday
More informationA SECOND DERIVATIVE SQP METHOD : THEORETICAL ISSUES
A SECOND DERIVATIVE SQP METHOD : THEORETICAL ISSUES Nicholas I. M. Gould Daniel P. Robinson Oxford University Computing Laboratory Numerical Analysis Group Technical Report 08/18 November 2008 Abstract
More informationAN AUGMENTED LAGRANGIAN AFFINE SCALING METHOD FOR NONLINEAR PROGRAMMING
AN AUGMENTED LAGRANGIAN AFFINE SCALING METHOD FOR NONLINEAR PROGRAMMING XIAO WANG AND HONGCHAO ZHANG Abstract. In this paper, we propose an Augmented Lagrangian Affine Scaling (ALAS) algorithm for general
More informationSome Theoretical Properties of an Augmented Lagrangian Merit Function
Some Theoretical Properties of an Augmented Lagrangian Merit Function Philip E. GILL Walter MURRAY Michael A. SAUNDERS Margaret H. WRIGHT Technical Report SOL 86-6R September 1986 Abstract Sequential quadratic
More informationA SECOND DERIVATIVE SQP METHOD : LOCAL CONVERGENCE
A SECOND DERIVATIVE SQP METHOD : LOCAL CONVERGENCE Nic I. M. Gould Daniel P. Robinson Oxford University Computing Laboratory Numerical Analysis Group Technical Report 08-21 December 2008 Abstract In [19],
More informationAnalysis of Inexact Trust-Region Interior-Point SQP Algorithms. Matthias Heinkenschloss Luis N. Vicente. TR95-18 June 1995 (revised April 1996)
Analysis of Inexact rust-region Interior-Point SQP Algorithms Matthias Heinkenschloss Luis N. Vicente R95-18 June 1995 (revised April 1996) Department of Computational and Applied Mathematics MS 134 Rice
More informationApplications of Linear Programming
Applications of Linear Programming lecturer: András London University of Szeged Institute of Informatics Department of Computational Optimization Lecture 9 Non-linear programming In case of LP, the goal
More informationOn the Local Quadratic Convergence of the Primal-Dual Augmented Lagrangian Method
Optimization Methods and Software Vol. 00, No. 00, Month 200x, 1 11 On the Local Quadratic Convergence of the Primal-Dual Augmented Lagrangian Method ROMAN A. POLYAK Department of SEOR and Mathematical
More informationTrust Region Methods. Lecturer: Pradeep Ravikumar Co-instructor: Aarti Singh. Convex Optimization /36-725
Trust Region Methods Lecturer: Pradeep Ravikumar Co-instructor: Aarti Singh Convex Optimization 10-725/36-725 Trust Region Methods min p m k (p) f(x k + p) s.t. p 2 R k Iteratively solve approximations
More informationNumerisches Rechnen. (für Informatiker) M. Grepl P. Esser & G. Welper & L. Zhang. Institut für Geometrie und Praktische Mathematik RWTH Aachen
Numerisches Rechnen (für Informatiker) M. Grepl P. Esser & G. Welper & L. Zhang Institut für Geometrie und Praktische Mathematik RWTH Aachen Wintersemester 2011/12 IGPM, RWTH Aachen Numerisches Rechnen
More informationKey words. minimization, nonlinear optimization, large-scale optimization, constrained optimization, trust region methods, quasi-newton methods
SIAM J. OPTIM. c 1998 Society for Industrial and Applied Mathematics Vol. 8, No. 3, pp. 682 706, August 1998 004 ON THE IMPLEMENTATION OF AN ALGORITHM FOR LARGE-SCALE EQUALITY CONSTRAINED OPTIMIZATION
More informationOptimal Newton-type methods for nonconvex smooth optimization problems
Optimal Newton-type methods for nonconvex smooth optimization problems Coralia Cartis, Nicholas I. M. Gould and Philippe L. Toint June 9, 20 Abstract We consider a general class of second-order iterations
More informationLINEAR AND NONLINEAR PROGRAMMING
LINEAR AND NONLINEAR PROGRAMMING Stephen G. Nash and Ariela Sofer George Mason University The McGraw-Hill Companies, Inc. New York St. Louis San Francisco Auckland Bogota Caracas Lisbon London Madrid Mexico
More informationNumerical optimization
Numerical optimization Lecture 4 Alexander & Michael Bronstein tosca.cs.technion.ac.il/book Numerical geometry of non-rigid shapes Stanford University, Winter 2009 2 Longest Slowest Shortest Minimal Maximal
More informationarxiv: v1 [math.na] 8 Jun 2018
arxiv:1806.03347v1 [math.na] 8 Jun 2018 Interior Point Method with Modified Augmented Lagrangian for Penalty-Barrier Nonlinear Programming Martin Neuenhofen June 12, 2018 Abstract We present a numerical
More informationCE 191: Civil and Environmental Engineering Systems Analysis. LEC 05 : Optimality Conditions
CE 191: Civil and Environmental Engineering Systems Analysis LEC : Optimality Conditions Professor Scott Moura Civil & Environmental Engineering University of California, Berkeley Fall 214 Prof. Moura
More informationConvex Optimization M2
Convex Optimization M2 Lecture 3 A. d Aspremont. Convex Optimization M2. 1/49 Duality A. d Aspremont. Convex Optimization M2. 2/49 DMs DM par email: dm.daspremont@gmail.com A. d Aspremont. Convex Optimization
More informationNumerical Optimization. Review: Unconstrained Optimization
Numerical Optimization Finding the best feasible solution Edward P. Gatzke Department of Chemical Engineering University of South Carolina Ed Gatzke (USC CHE ) Numerical Optimization ECHE 589, Spring 2011
More informationA STABILIZED SQP METHOD: GLOBAL CONVERGENCE
A STABILIZED SQP METHOD: GLOBAL CONVERGENCE Philip E. Gill Vyacheslav Kungurtsev Daniel P. Robinson UCSD Center for Computational Mathematics Technical Report CCoM-13-4 Revised July 18, 2014, June 23,
More informationSolution Methods. Richard Lusby. Department of Management Engineering Technical University of Denmark
Solution Methods Richard Lusby Department of Management Engineering Technical University of Denmark Lecture Overview (jg Unconstrained Several Variables Quadratic Programming Separable Programming SUMT
More informationOptimisation in Higher Dimensions
CHAPTER 6 Optimisation in Higher Dimensions Beyond optimisation in 1D, we will study two directions. First, the equivalent in nth dimension, x R n such that f(x ) f(x) for all x R n. Second, constrained
More information