Multidisciplinary System Design Optimization (MSDO)
|
|
- Elfreda Tucker
- 6 years ago
- Views:
Transcription
1 Multidisciplinary System Design Optimization (MSDO) Numerical Optimization II Lecture 8 Karen Willcox 1 Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox
2 Today s Topics Sequential Linear Programming Penalty and Barrier Methods Sequential Quadratic Programming 2 Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox
3 Technique Overview Steepest Descent UNCONSTRAINED Conjugate Gradient Quasi-Newton Newton Simplex linear CONSTRAINED SLP often not effective SQP nonlinear, expensive, common in engineering applications Exterior Penalty nonlinear, discontinuous design spaces Interior Penalty nonlinear Generalized Reduced Gradient nonlinear Method of Feasible Directions nonlinear Mixed Integer Programming 3 Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox
4 Standard Problem Definition min J( x) s.t. g ( x) 0 j 1,.., m j h ( x) 0 k 1,.., m k u x x x i 1,.., n i i i For now, we consider a single objective function, J(x). There are n design variables, and a total of m constraints (m=m 1 +m 2 ). For now we assume all x i are real and continuous. 4 Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox 1 2
5 Optimization Process x 0, q=0 Calculate J(x q ) Calculate S q q=q+1 Perform 1-D search x q = x q-1 + S q no Converged? yes Done 5 Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox
6 Constrained Optimization Definitions: Feasible design: a design that satisfies all constraints Infeasible design: a design that violates one or more constraints Optimum design: the choice of design variables that minimizes the objective function while satisfying all constraints In general, constrained optimization algorithms try to cast the problem as an unconstrained optimization and then use one of the techniques we looked at in Lecture 7. 6 Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox
7 Linear Programming Most engineering problems of interest are nonlinear Can often simplify nonlinear problem by linearization LP is often the basis for developing more complicated NLP algorithms Standard LP problem: min J( x) c x min J( x) i n 1 a x b j 1,.., m ij i j x 0 i 1,..., n i i n 1 All LP problems can be converted to this form. i i Ax T c x x 0 i 1,..., n 7 Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox i b
8 Linear Programming To convert inequality constraints to equality constraints, use additional design variables: i n a x b becomes a x x b ji i j ji i n 1 j 1 i 1 where x n+1 0 x n+1 is called a slack variable e.g. the constraint can be written x 1 +x 2 1 x 1 +x 2 +x 3 =1 x i 0, i=1,2,3 8 Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox n if x 3 =0, this constraint is active
9 Simplex Method Solutions at the vertices of the design space are called basic feasible solutions. The Simplex algorithm moves from BFS to BFS so that the objective always improves. feasible region basic feasible solution 9 Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox
10 Sequential Linear Programming Consider a general nonlinear problem linearized via first order Taylor series: 0 0 T min J( ) J( ) J( ) x x x x 0 0 T s.t. g j( x) g j( x ) g j( x ) x 0 h h h 0 0 T k( x) k( x ) k( x ) x 0 x x x x u i i i i where 0 x x x This is an LP problem with the design variables contained in x. The functions and gradients evaluated at x 0 are constant coefficients. 10 Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox
11 Sequential Linear Programming 1. Initial guess x 0 2. Linearize about x 0 using first-order Taylor series 3. Solve resulting LP to find x 4. Update: x 1 = x 0 + x 5. Linearize about x 1 and repeat: x q = x q-1 + x where x is the solution of an LP (model linearized about x q-1 ). 11 Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox
12 Sequential Linear Programming Linearization approximation is only valid close to x 0 Need to restrict size of update x Not considered to be a good method 12 Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox
13 Nonlinear vs. Linear Constraints Linear constraints: start from initial feasible point, then all subsequent iterates are feasible can construct search direction & step length so that constraints are satisfied improvement from x k to x k+1 based on J(x) feasible region Nonlinear constraints: not straightforward to generate a sequence of feasible iterates if feasibility is not maintained, then need to decide whether x k+1 is better than x k 13 Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox
14 Merit Function Is objective reduced? Are constraints satisfied? merit function merit function = f(j(x), g(x), h(x)) examples: penalty function methods, barrier function methods 14 Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox
15 Subproblems Many optimization algorithms get to the optimum by generating and solving a sequence of subproblems that are somehow related to the original problem. Typically there are two tasks at each iteration: 1. Calculate search direction 2. Calculate the step length Sometimes, the initial formulation of a subproblem may be defective i.e. the subproblem has no solution or the solution is unbounded A valid subproblem is one for which a solution exists and is well defined The option to abandon should be available if the subproblem appears defective It is possible that a defective subproblem is an accurate reflection of the original problem having no valid solution 15 Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox
16 General Approach: Penalty and Barrier Methods minimize objective as unconstrained function provide penalty to limit constraint violations magnitude of penalty varies throughout optimization called sequential unconstrained minimization techniques (SUMT) create pseudo-objective: ( x, r ) J( x) r P( x) J(x) = original objective function P(x) = imposed penalty function p r p = scalar multiplier to determine penalty magnitude p = unconstrained minimization number p 16 Penalty method approaches useful for incorporating constraints into derivative-free and heuristic search algorithms.
17 Penalty Methods Four basic methods: (i) Exterior Penalty Function Method (ii) Interior Penalty Function Method (Barrier Function Method) (iii) Log Penalty Function Method (iv) Extended Interior Penalty Function Method Effect of penalty function is to create a local minimum of unconstrained problem near x*. Penalty function adds a penalty for infeasibility, barrier function adds a term that prevents iterates from becoming infeasible. 17 Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox
18 Exterior Penalty Function Method ( x, ) J( x) P( x) p p m 1 2 j 1 k 1 2 m 2 P( x) max 0, g ( x) h ( x) j if all constraints are satisfied, then P(x)=0 p = penalty parameter; starts as a small number and increases if p is small, (x, p ) is easy to minimize but yields large constraint violations if p is large, constraints are all nearly satisfied but optimization problem is numerically ill-conditioned if optimization stops before convergence is reached, the design will be infeasible 18 Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox k
19 Quadratic Penalty Function ˆ1 2 m 2 2 ( x, ) J( x) gˆ ( x) h ( x) Q p p j k j 1 k 1 gˆ j( x) contains those inequality constraints that are violated at x. It can be shown that Q(x, p ) is well-defined everywhere m lim x* ( ) x* p p Algorithm: -choose 0, set k=0 -find min Q(x, k ) x k * -if not converged, set k+1 > k, k k+1 and repeat Note: x k * could be infeasible wrt the original problem 19 Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox
20 Quadratic Penalty Function Example Q(x, r p ) min J( x) x 2 s.t. x 1 0 = 10 = 5 = 2 = 1 Q(x, )=x 2 + (1-x) 2 0 * * ** 1 J(x) x 20 Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox
21 Absolute Value Penalty Function ˆ1 2 ( x, ) J( x) gˆ ( x) h ( x) A p p j k j 1 k 1 where gˆ j( x) contains those inequality constraints which are violated at x. A has discontinuous derivatives at points where or h x are zero. k ( ) 21 Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox m m gˆ j( x) The crucial difference between Q and A is that we do not require p for x* to be a minimum of A, so we can avoid ill-conditioning Instead, for some threshold p, x* is a minimum of A for any p p Sometimes called exact penalty functions
22 Absolute Value Penalty Function Example A(x, ) = 10 min J( x) x 2 s.t. x 1 0 = 5 = 2 = 1.5 = 1 (x, )=x x 0 * * 1 J(x) x 22 Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox
23 Interior Penalty Function Method (Barrier Function Method) P( x) ˆ 1 m j 1 1 gˆ ( x) j P j (x) g j (x) ˆ 1 m2 m 1 ( x, r, ) J( x) r h ( x) ˆ ( ) p p p p k j 1g j x k 1 2 r p = barrier parameter; starts as a large positive number and decreases barrier function for inequality constraints only sequence of improving feasible designs (x,r p, p ) discontinuous at constraint boundaries 23 Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox
24 Barrier Function Method m 1 P( x) ln g ( x) often better numerically conditioned than penalty function has a positive singularity at the boundary of the feasible region penalty function is undefined for g i (x)>0 lim x* ( r ) x* r p 0 p j 1 ( x, r, ) J( x) r ln gˆ ( x) L p p p j j 1 m ˆ1 j 1 g ( x) j 24 Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox
25 Barrier Function Example min J( x) x 2 L(x,r) s.t. x 1 0 r = 5 r = 1 10 r = 0.5 L(x,r)=x 2 -r ln(x-1) 0 J(x) r = x 25 Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox
26 Extended Interior Penalty Function Method Combine features of interior/exterior methods P( x) g ( x) 1 j 1 1 where gj( x) if gj( x) g ( x) m j j 2 g ( x) 2 j if g ( x) j -linear extended penalty function - is a small negative number, and marks transition from interior penalty to extended penalty - must be chosen so that has positive slope at constraint boundary -can also use quadratic extended and variable penalty functions 26 Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox
27 Sequential Quadratic Programming Create a quadratic approximation to the Lagrangian Create linear approximations to the constraints Solve the quadratic problem to find the search direction, S Perform the 1-D search Update the approximation to the Lagrangian 27 Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox
28 Sequential Quadratic Programming Create a subproblem with quadratic objective function: min Q( S ) J( x ) J( x ) S S BS 2 and linear constraints k k k T k 1 T k T k k s.t. j( x ) S j( x ) 0 1,, g g j m h h k m k T k k k( x ) S k( x ) 0 1,, Design variables are the components of S B=I initially, then is updated to approximate H (as in quasi-newton) Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox
29 Incompatible Constraints The constraints of the subproblem can be incompatible, even if the original problem has a well-posed solution For example, two linearized constraints could be linearly dependent This is a common occurrence in practice Likelihood of incompatible constraints reduced by allowing flexibility in RHS (e.g. allow scaling factors in front of g j (x k ) term) T g ( x k ) S k g ( x k ) 0 j 1,, m j j j Typically =0.9 if constraint is violated and =1 otherwise Doesn t affect convergence, since specific form of constraint is only crucial when x k is close to x* 1 29 Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox
30 Sequential Quadratic Programming Widely used in engineering applications e.g. NLOpt Considered to be one of the best gradient-based algorithms Fast convergence for many problems Strong theoretical basis MATLAB : fmincon (medium scale) 30 Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox
31 Lecture Summary We have seen the following nonlinear techniques: Penalty and Barrier Methods Sequential Quadratic Programming It is important to understand when it is appropriate to use these methods the basics of how the method works why the method might fail what your results mean (numerically vs. physically) 31 Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox
32 References Practical Optimization, P.E. Gill, W. Murray and M.H. Wright, Academic Press, Numerical Optimization Techniques for Engineering Design, G.N. Vanderplaats, Vanderplaats R&D, Optimal Design in Multidisciplinary Systems, AIAA Professional Development Short Course Notes, September NLOPT: Open-source library for nonlinear optimization 32 Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox
33 MIT OpenCourseWare ESD.77 / Multidisciplinary System Design Optimization Spring 2010 For information about citing these materials or our Terms of Use, visit:
Lecture 13: Constrained optimization
2010-12-03 Basic ideas A nonlinearly constrained problem must somehow be converted relaxed into a problem which we can solve (a linear/quadratic or unconstrained problem) We solve a sequence of such problems
More informationPenalty and Barrier Methods General classical constrained minimization problem minimize f(x) subject to g(x) 0 h(x) =0 Penalty methods are motivated by the desire to use unconstrained optimization techniques
More information5 Handling Constraints
5 Handling Constraints Engineering design optimization problems are very rarely unconstrained. Moreover, the constraints that appear in these problems are typically nonlinear. This motivates our interest
More information2.098/6.255/ Optimization Methods Practice True/False Questions
2.098/6.255/15.093 Optimization Methods Practice True/False Questions December 11, 2009 Part I For each one of the statements below, state whether it is true or false. Include a 1-3 line supporting sentence
More information2.3 Linear Programming
2.3 Linear Programming Linear Programming (LP) is the term used to define a wide range of optimization problems in which the objective function is linear in the unknown variables and the constraints are
More informationAlgorithms for Constrained Optimization
1 / 42 Algorithms for Constrained Optimization ME598/494 Lecture Max Yi Ren Department of Mechanical Engineering, Arizona State University April 19, 2015 2 / 42 Outline 1. Convergence 2. Sequential quadratic
More informationSolution Methods. Richard Lusby. Department of Management Engineering Technical University of Denmark
Solution Methods Richard Lusby Department of Management Engineering Technical University of Denmark Lecture Overview (jg Unconstrained Several Variables Quadratic Programming Separable Programming SUMT
More informationISM206 Lecture Optimization of Nonlinear Objective with Linear Constraints
ISM206 Lecture Optimization of Nonlinear Objective with Linear Constraints Instructor: Prof. Kevin Ross Scribe: Nitish John October 18, 2011 1 The Basic Goal The main idea is to transform a given constrained
More informationCONSTRAINED NONLINEAR PROGRAMMING
149 CONSTRAINED NONLINEAR PROGRAMMING We now turn to methods for general constrained nonlinear programming. These may be broadly classified into two categories: 1. TRANSFORMATION METHODS: In this approach
More informationMS&E 318 (CME 338) Large-Scale Numerical Optimization
Stanford University, Management Science & Engineering (and ICME) MS&E 318 (CME 338) Large-Scale Numerical Optimization 1 Origins Instructor: Michael Saunders Spring 2015 Notes 9: Augmented Lagrangian Methods
More informationScientific Computing: Optimization
Scientific Computing: Optimization Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 Course MATH-GA.2043 or CSCI-GA.2112, Spring 2012 March 8th, 2011 A. Donev (Courant Institute) Lecture
More informationConstrained optimization: direct methods (cont.)
Constrained optimization: direct methods (cont.) Jussi Hakanen Post-doctoral researcher jussi.hakanen@jyu.fi Direct methods Also known as methods of feasible directions Idea in a point x h, generate a
More informationOptimization. Escuela de Ingeniería Informática de Oviedo. (Dpto. de Matemáticas-UniOvi) Numerical Computation Optimization 1 / 30
Optimization Escuela de Ingeniería Informática de Oviedo (Dpto. de Matemáticas-UniOvi) Numerical Computation Optimization 1 / 30 Unconstrained optimization Outline 1 Unconstrained optimization 2 Constrained
More informationNumerical optimization. Numerical optimization. Longest Shortest where Maximal Minimal. Fastest. Largest. Optimization problems
1 Numerical optimization Alexander & Michael Bronstein, 2006-2009 Michael Bronstein, 2010 tosca.cs.technion.ac.il/book Numerical optimization 048921 Advanced topics in vision Processing and Analysis of
More informationE5295/5B5749 Convex optimization with engineering applications. Lecture 8. Smooth convex unconstrained and equality-constrained minimization
E5295/5B5749 Convex optimization with engineering applications Lecture 8 Smooth convex unconstrained and equality-constrained minimization A. Forsgren, KTH 1 Lecture 8 Convex optimization 2006/2007 Unconstrained
More informationNumerical Optimization. Review: Unconstrained Optimization
Numerical Optimization Finding the best feasible solution Edward P. Gatzke Department of Chemical Engineering University of South Carolina Ed Gatzke (USC CHE ) Numerical Optimization ECHE 589, Spring 2011
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY /ESD.77 MSDO /ESD.77J Multidisciplinary System Design Optimization (MSDO) Spring 2010.
16.888/ESD.77J Multidisciplinary System Design Optimization (MSDO) Spring 2010 Assignment 4 Instructors: Prof. Olivier de Weck Prof. Karen Willcox Dr. Anas Alfaris Dr. Douglas Allaire TAs: Andrew March
More information1 Computing with constraints
Notes for 2017-04-26 1 Computing with constraints Recall that our basic problem is minimize φ(x) s.t. x Ω where the feasible set Ω is defined by equality and inequality conditions Ω = {x R n : c i (x)
More informationAM 205: lecture 19. Last time: Conditions for optimality Today: Newton s method for optimization, survey of optimization methods
AM 205: lecture 19 Last time: Conditions for optimality Today: Newton s method for optimization, survey of optimization methods Optimality Conditions: Equality Constrained Case As another example of equality
More informationNumerical optimization
Numerical optimization Lecture 4 Alexander & Michael Bronstein tosca.cs.technion.ac.il/book Numerical geometry of non-rigid shapes Stanford University, Winter 2009 2 Longest Slowest Shortest Minimal Maximal
More informationAM 205: lecture 19. Last time: Conditions for optimality, Newton s method for optimization Today: survey of optimization methods
AM 205: lecture 19 Last time: Conditions for optimality, Newton s method for optimization Today: survey of optimization methods Quasi-Newton Methods General form of quasi-newton methods: x k+1 = x k α
More informationSF2822 Applied Nonlinear Optimization. Preparatory question. Lecture 9: Sequential quadratic programming. Anders Forsgren
SF2822 Applied Nonlinear Optimization Lecture 9: Sequential quadratic programming Anders Forsgren SF2822 Applied Nonlinear Optimization, KTH / 24 Lecture 9, 207/208 Preparatory question. Try to solve theory
More information10 Numerical methods for constrained problems
10 Numerical methods for constrained problems min s.t. f(x) h(x) = 0 (l), g(x) 0 (m), x X The algorithms can be roughly divided the following way: ˆ primal methods: find descent direction keeping inside
More information5.6 Penalty method and augmented Lagrangian method
5.6 Penalty method and augmented Lagrangian method Consider a generic NLP problem min f (x) s.t. c i (x) 0 i I c i (x) = 0 i E (1) x R n where f and the c i s are of class C 1 or C 2, and I and E are the
More informationNONLINEAR. (Hillier & Lieberman Introduction to Operations Research, 8 th edition)
NONLINEAR PROGRAMMING (Hillier & Lieberman Introduction to Operations Research, 8 th edition) Nonlinear Programming g Linear programming has a fundamental role in OR. In linear programming all its functions
More information6.252 NONLINEAR PROGRAMMING LECTURE 10 ALTERNATIVES TO GRADIENT PROJECTION LECTURE OUTLINE. Three Alternatives/Remedies for Gradient Projection
6.252 NONLINEAR PROGRAMMING LECTURE 10 ALTERNATIVES TO GRADIENT PROJECTION LECTURE OUTLINE Three Alternatives/Remedies for Gradient Projection Two-Metric Projection Methods Manifold Suboptimization Methods
More informationApplications of Linear Programming
Applications of Linear Programming lecturer: András London University of Szeged Institute of Informatics Department of Computational Optimization Lecture 9 Non-linear programming In case of LP, the goal
More informationMIT Manufacturing Systems Analysis Lecture 14-16
MIT 2.852 Manufacturing Systems Analysis Lecture 14-16 Line Optimization Stanley B. Gershwin Spring, 2007 Copyright c 2007 Stanley B. Gershwin. Line Design Given a process, find the best set of machines
More informationScientific Computing: An Introductory Survey
Scientific Computing: An Introductory Survey Chapter 6 Optimization Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002. Reproduction permitted
More informationScientific Computing: An Introductory Survey
Scientific Computing: An Introductory Survey Chapter 6 Optimization Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002. Reproduction permitted
More informationAlgorithms for constrained local optimization
Algorithms for constrained local optimization Fabio Schoen 2008 http://gol.dsi.unifi.it/users/schoen Algorithms for constrained local optimization p. Feasible direction methods Algorithms for constrained
More informationNumerical Optimization Professor Horst Cerjak, Horst Bischof, Thomas Pock Mat Vis-Gra SS09
Numerical Optimization 1 Working Horse in Computer Vision Variational Methods Shape Analysis Machine Learning Markov Random Fields Geometry Common denominator: optimization problems 2 Overview of Methods
More informationDetermination of Feasible Directions by Successive Quadratic Programming and Zoutendijk Algorithms: A Comparative Study
International Journal of Mathematics And Its Applications Vol.2 No.4 (2014), pp.47-56. ISSN: 2347-1557(online) Determination of Feasible Directions by Successive Quadratic Programming and Zoutendijk Algorithms:
More informationLecture V. Numerical Optimization
Lecture V Numerical Optimization Gianluca Violante New York University Quantitative Macroeconomics G. Violante, Numerical Optimization p. 1 /19 Isomorphism I We describe minimization problems: to maximize
More informationComputational Optimization. Augmented Lagrangian NW 17.3
Computational Optimization Augmented Lagrangian NW 17.3 Upcoming Schedule No class April 18 Friday, April 25, in class presentations. Projects due unless you present April 25 (free extension until Monday
More informationLecture 3. Optimization Problems and Iterative Algorithms
Lecture 3 Optimization Problems and Iterative Algorithms January 13, 2016 This material was jointly developed with Angelia Nedić at UIUC for IE 598ns Outline Special Functions: Linear, Quadratic, Convex
More informationOptimization Problems with Constraints - introduction to theory, numerical Methods and applications
Optimization Problems with Constraints - introduction to theory, numerical Methods and applications Dr. Abebe Geletu Ilmenau University of Technology Department of Simulation and Optimal Processes (SOP)
More informationCE 191: Civil and Environmental Engineering Systems Analysis. LEC 05 : Optimality Conditions
CE 191: Civil and Environmental Engineering Systems Analysis LEC : Optimality Conditions Professor Scott Moura Civil & Environmental Engineering University of California, Berkeley Fall 214 Prof. Moura
More informationLECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE
LECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE CONVEX ANALYSIS AND DUALITY Basic concepts of convex analysis Basic concepts of convex optimization Geometric duality framework - MC/MC Constrained optimization
More informationOptimization Methods
Optimization Methods Decision making Examples: determining which ingredients and in what quantities to add to a mixture being made so that it will meet specifications on its composition allocating available
More informationLecture 15: SQP methods for equality constrained optimization
Lecture 15: SQP methods for equality constrained optimization Coralia Cartis, Mathematical Institute, University of Oxford C6.2/B2: Continuous Optimization Lecture 15: SQP methods for equality constrained
More information8 Numerical methods for unconstrained problems
8 Numerical methods for unconstrained problems Optimization is one of the important fields in numerical computation, beside solving differential equations and linear systems. We can see that these fields
More informationOutline. Scientific Computing: An Introductory Survey. Optimization. Optimization Problems. Examples: Optimization Problems
Outline Scientific Computing: An Introductory Survey Chapter 6 Optimization 1 Prof. Michael. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002. Reproduction
More informationInterior Point Methods. We ll discuss linear programming first, followed by three nonlinear problems. Algorithms for Linear Programming Problems
AMSC 607 / CMSC 764 Advanced Numerical Optimization Fall 2008 UNIT 3: Constrained Optimization PART 4: Introduction to Interior Point Methods Dianne P. O Leary c 2008 Interior Point Methods We ll discuss
More informationThe Squared Slacks Transformation in Nonlinear Programming
Technical Report No. n + P. Armand D. Orban The Squared Slacks Transformation in Nonlinear Programming August 29, 2007 Abstract. We recall the use of squared slacks used to transform inequality constraints
More informationOptimization Concepts and Applications in Engineering
Optimization Concepts and Applications in Engineering Ashok D. Belegundu, Ph.D. Department of Mechanical Engineering The Pennsylvania State University University Park, Pennsylvania Tirupathi R. Chandrupatia,
More informationAlgorithms for nonlinear programming problems II
Algorithms for nonlinear programming problems II Martin Branda Charles University Faculty of Mathematics and Physics Department of Probability and Mathematical Statistics Computational Aspects of Optimization
More informationWhat s New in Active-Set Methods for Nonlinear Optimization?
What s New in Active-Set Methods for Nonlinear Optimization? Philip E. Gill Advances in Numerical Computation, Manchester University, July 5, 2011 A Workshop in Honor of Sven Hammarling UCSD Center for
More informationOptimization and Root Finding. Kurt Hornik
Optimization and Root Finding Kurt Hornik Basics Root finding and unconstrained smooth optimization are closely related: Solving ƒ () = 0 can be accomplished via minimizing ƒ () 2 Slide 2 Basics Root finding
More information4TE3/6TE3. Algorithms for. Continuous Optimization
4TE3/6TE3 Algorithms for Continuous Optimization (Algorithms for Constrained Nonlinear Optimization Problems) Tamás TERLAKY Computing and Software McMaster University Hamilton, November 2005 terlaky@mcmaster.ca
More informationminimize x subject to (x 2)(x 4) u,
Math 6366/6367: Optimization and Variational Methods Sample Preliminary Exam Questions 1. Suppose that f : [, L] R is a C 2 -function with f () on (, L) and that you have explicit formulae for
More informationCS 542G: Robustifying Newton, Constraints, Nonlinear Least Squares
CS 542G: Robustifying Newton, Constraints, Nonlinear Least Squares Robert Bridson October 29, 2008 1 Hessian Problems in Newton Last time we fixed one of plain Newton s problems by introducing line search
More informationMultidisciplinary System Design Optimization (MSDO)
oday s opics Multidisciplinary System Design Optimization (MSDO) Approimation Methods Lecture 9 6 April 004 Design variable lining Reduced-Basis Methods Response Surface Approimations Kriging Variable-Fidelity
More informationPenalty and Barrier Methods. So we again build on our unconstrained algorithms, but in a different way.
AMSC 607 / CMSC 878o Advanced Numerical Optimization Fall 2008 UNIT 3: Constrained Optimization PART 3: Penalty and Barrier Methods Dianne P. O Leary c 2008 Reference: N&S Chapter 16 Penalty and Barrier
More informationNumerical Optimization: Basic Concepts and Algorithms
May 27th 2015 Numerical Optimization: Basic Concepts and Algorithms R. Duvigneau R. Duvigneau - Numerical Optimization: Basic Concepts and Algorithms 1 Outline Some basic concepts in optimization Some
More informationCHAPTER 2: QUADRATIC PROGRAMMING
CHAPTER 2: QUADRATIC PROGRAMMING Overview Quadratic programming (QP) problems are characterized by objective functions that are quadratic in the design variables, and linear constraints. In this sense,
More informationNonlinear Optimization for Optimal Control
Nonlinear Optimization for Optimal Control Pieter Abbeel UC Berkeley EECS Many slides and figures adapted from Stephen Boyd [optional] Boyd and Vandenberghe, Convex Optimization, Chapters 9 11 [optional]
More informationNonlinear Optimization: What s important?
Nonlinear Optimization: What s important? Julian Hall 10th May 2012 Convexity: convex problems A local minimizer is a global minimizer A solution of f (x) = 0 (stationary point) is a minimizer A global
More informationWritten Examination
Division of Scientific Computing Department of Information Technology Uppsala University Optimization Written Examination 202-2-20 Time: 4:00-9:00 Allowed Tools: Pocket Calculator, one A4 paper with notes
More informationMATH 4211/6211 Optimization Basics of Optimization Problems
MATH 4211/6211 Optimization Basics of Optimization Problems Xiaojing Ye Department of Mathematics & Statistics Georgia State University Xiaojing Ye, Math & Stat, Georgia State University 0 A standard minimization
More informationStructural and Multidisciplinary Optimization. P. Duysinx and P. Tossings
Structural and Multidisciplinary Optimization P. Duysinx and P. Tossings 2018-2019 CONTACTS Pierre Duysinx Institut de Mécanique et du Génie Civil (B52/3) Phone number: 04/366.91.94 Email: P.Duysinx@uliege.be
More informationComputational Finance
Department of Mathematics at University of California, San Diego Computational Finance Optimization Techniques [Lecture 2] Michael Holst January 9, 2017 Contents 1 Optimization Techniques 3 1.1 Examples
More information5.5 Quadratic programming
5.5 Quadratic programming Minimize a quadratic function subject to linear constraints: 1 min x t Qx + c t x 2 s.t. a t i x b i i I (P a t i x = b i i E x R n, where Q is an n n matrix, I and E are the
More informationNumerisches Rechnen. (für Informatiker) M. Grepl P. Esser & G. Welper & L. Zhang. Institut für Geometrie und Praktische Mathematik RWTH Aachen
Numerisches Rechnen (für Informatiker) M. Grepl P. Esser & G. Welper & L. Zhang Institut für Geometrie und Praktische Mathematik RWTH Aachen Wintersemester 2011/12 IGPM, RWTH Aachen Numerisches Rechnen
More informationCE 191: Civil & Environmental Engineering Systems Analysis. LEC 17 : Final Review
CE 191: Civil & Environmental Engineering Systems Analysis LEC 17 : Final Review Professor Scott Moura Civil & Environmental Engineering University of California, Berkeley Fall 2014 Prof. Moura UC Berkeley
More informationAM 121: Intro to Optimization Models and Methods
AM 121: Intro to Optimization Models and Methods Fall 2017 Lecture 2: Intro to LP, Linear algebra review. Yiling Chen SEAS Lecture 2: Lesson Plan What is an LP? Graphical and algebraic correspondence Problems
More informationIE 5531: Engineering Optimization I
IE 5531: Engineering Optimization I Lecture 19: Midterm 2 Review Prof. John Gunnar Carlsson November 22, 2010 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I November 22, 2010 1 / 34 Administrivia
More informationConstrained Optimization
1 / 22 Constrained Optimization ME598/494 Lecture Max Yi Ren Department of Mechanical Engineering, Arizona State University March 30, 2015 2 / 22 1. Equality constraints only 1.1 Reduced gradient 1.2 Lagrange
More informationLecture 3: Lagrangian duality and algorithms for the Lagrangian dual problem
Lecture 3: Lagrangian duality and algorithms for the Lagrangian dual problem Michael Patriksson 0-0 The Relaxation Theorem 1 Problem: find f := infimum f(x), x subject to x S, (1a) (1b) where f : R n R
More informationLINEAR AND NONLINEAR PROGRAMMING
LINEAR AND NONLINEAR PROGRAMMING Stephen G. Nash and Ariela Sofer George Mason University The McGraw-Hill Companies, Inc. New York St. Louis San Francisco Auckland Bogota Caracas Lisbon London Madrid Mexico
More informationQUADRATIC APPROXIMATION METHODS FOR CONSTRAINED PROBLEMS
10 QUADRATIC APPROXIMATION METHODS FOR CONSTRAINED PROBLEMS In the preceding two chapters we considered a number of alternative strategies for exploiting linear approximations to nonlinear problem functions.
More informationNumerical Optimization
Numerical Optimization Unit 2: Multivariable optimization problems Che-Rung Lee Scribe: February 28, 2011 (UNIT 2) Numerical Optimization February 28, 2011 1 / 17 Partial derivative of a two variable function
More informationIE 5531 Midterm #2 Solutions
IE 5531 Midterm #2 s Prof. John Gunnar Carlsson November 9, 2011 Before you begin: This exam has 9 pages and a total of 5 problems. Make sure that all pages are present. To obtain credit for a problem,
More informationIntroduction to Mathematical Programming IE406. Lecture 10. Dr. Ted Ralphs
Introduction to Mathematical Programming IE406 Lecture 10 Dr. Ted Ralphs IE406 Lecture 10 1 Reading for This Lecture Bertsimas 4.1-4.3 IE406 Lecture 10 2 Duality Theory: Motivation Consider the following
More informationPart 4: Active-set methods for linearly constrained optimization. Nick Gould (RAL)
Part 4: Active-set methods for linearly constrained optimization Nick Gould RAL fx subject to Ax b Part C course on continuoue optimization LINEARLY CONSTRAINED MINIMIZATION fx subject to Ax { } b where
More informationREGULARIZED SEQUENTIAL QUADRATIC PROGRAMMING METHODS
REGULARIZED SEQUENTIAL QUADRATIC PROGRAMMING METHODS Philip E. Gill Daniel P. Robinson UCSD Department of Mathematics Technical Report NA-11-02 October 2011 Abstract We present the formulation and analysis
More informationOptimization. Yuh-Jye Lee. March 21, Data Science and Machine Intelligence Lab National Chiao Tung University 1 / 29
Optimization Yuh-Jye Lee Data Science and Machine Intelligence Lab National Chiao Tung University March 21, 2017 1 / 29 You Have Learned (Unconstrained) Optimization in Your High School Let f (x) = ax
More informationCS711008Z Algorithm Design and Analysis
CS711008Z Algorithm Design and Analysis Lecture 8 Linear programming: interior point method Dongbo Bu Institute of Computing Technology Chinese Academy of Sciences, Beijing, China 1 / 31 Outline Brief
More informationConvex Optimization. Ofer Meshi. Lecture 6: Lower Bounds Constrained Optimization
Convex Optimization Ofer Meshi Lecture 6: Lower Bounds Constrained Optimization Lower Bounds Some upper bounds: #iter μ 2 M #iter 2 M #iter L L μ 2 Oracle/ops GD κ log 1/ε M x # ε L # x # L # ε # με f
More information3E4: Modelling Choice. Introduction to nonlinear programming. Announcements
3E4: Modelling Choice Lecture 7 Introduction to nonlinear programming 1 Announcements Solutions to Lecture 4-6 Homework will be available from http://www.eng.cam.ac.uk/~dr241/3e4 Looking ahead to Lecture
More informationOperations Research Lecture 6: Integer Programming
Operations Research Lecture 6: Integer Programming Notes taken by Kaiquan Xu@Business School, Nanjing University May 12th 2016 1 Integer programming (IP) formulations The integer programming (IP) is the
More informationMore First-Order Optimization Algorithms
More First-Order Optimization Algorithms Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye Chapters 3, 8, 3 The SDM
More informationLinear Programming, Lecture 4
Linear Programming, Lecture 4 Corbett Redden October 3, 2016 Simplex Form Conventions Examples Simplex Method To run the simplex method, we start from a Linear Program (LP) in the following standard simplex
More informationAlgorithms for nonlinear programming problems II
Algorithms for nonlinear programming problems II Martin Branda Charles University in Prague Faculty of Mathematics and Physics Department of Probability and Mathematical Statistics Computational Aspects
More informationBindel, Spring 2017 Numerical Analysis (CS 4220) Notes for So far, we have considered unconstrained optimization problems.
Consider constraints Notes for 2017-04-24 So far, we have considered unconstrained optimization problems. The constrained problem is minimize φ(x) s.t. x Ω where Ω R n. We usually define x in terms of
More informationLecture 18: Optimization Programming
Fall, 2016 Outline Unconstrained Optimization 1 Unconstrained Optimization 2 Equality-constrained Optimization Inequality-constrained Optimization Mixture-constrained Optimization 3 Quadratic Programming
More informationMVE165/MMG631 Linear and integer optimization with applications Lecture 13 Overview of nonlinear programming. Ann-Brith Strömberg
MVE165/MMG631 Overview of nonlinear programming Ann-Brith Strömberg 2015 05 21 Areas of applications, examples (Ch. 9.1) Structural optimization Design of aircraft, ships, bridges, etc Decide on the material
More informationChapter 4. Unconstrained optimization
Chapter 4. Unconstrained optimization Version: 28-10-2012 Material: (for details see) Chapter 11 in [FKS] (pp.251-276) A reference e.g. L.11.2 refers to the corresponding Lemma in the book [FKS] PDF-file
More informationSF2822 Applied Nonlinear Optimization. Preparatory question. Lecture 10: Interior methods. Anders Forsgren. 1. Try to solve theory question 7.
SF2822 Applied Nonlinear Optimization Lecture 10: Interior methods Anders Forsgren SF2822 Applied Nonlinear Optimization, KTH 1 / 24 Lecture 10, 2017/2018 Preparatory question 1. Try to solve theory question
More informationGradient Descent. Dr. Xiaowei Huang
Gradient Descent Dr. Xiaowei Huang https://cgi.csc.liv.ac.uk/~xiaowei/ Up to now, Three machine learning algorithms: decision tree learning k-nn linear regression only optimization objectives are discussed,
More informationPrimal-Dual Interior-Point Methods for Linear Programming based on Newton s Method
Primal-Dual Interior-Point Methods for Linear Programming based on Newton s Method Robert M. Freund March, 2004 2004 Massachusetts Institute of Technology. The Problem The logarithmic barrier approach
More informationReview for Exam 2 Ben Wang and Mark Styczynski
Review for Exam Ben Wang and Mark Styczynski This is a rough approximation of what we went over in the review session. This is actually more detailed in portions than what we went over. Also, please note
More informationIE 5531: Engineering Optimization I
IE 5531: Engineering Optimization I Lecture 15: Nonlinear optimization Prof. John Gunnar Carlsson November 1, 2010 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I November 1, 2010 1 / 24
More informationLecture 9 Tuesday, 4/20/10. Linear Programming
UMass Lowell Computer Science 91.503 Analysis of Algorithms Prof. Karen Daniels Spring, 2010 Lecture 9 Tuesday, 4/20/10 Linear Programming 1 Overview Motivation & Basics Standard & Slack Forms Formulating
More informationAn Inexact Newton Method for Nonlinear Constrained Optimization
An Inexact Newton Method for Nonlinear Constrained Optimization Frank E. Curtis Numerical Analysis Seminar, January 23, 2009 Outline Motivation and background Algorithm development and theoretical results
More informationLECTURE 13 LECTURE OUTLINE
LECTURE 13 LECTURE OUTLINE Problem Structures Separable problems Integer/discrete problems Branch-and-bound Large sum problems Problems with many constraints Conic Programming Second Order Cone Programming
More informationA New Penalty-SQP Method
Background and Motivation Illustration of Numerical Results Final Remarks Frank E. Curtis Informs Annual Meeting, October 2008 Background and Motivation Illustration of Numerical Results Final Remarks
More informationStandard Form An LP is in standard form when: All variables are non-negativenegative All constraints are equalities Putting an LP formulation into sta
Chapter 4 Linear Programming: The Simplex Method An Overview of the Simplex Method Standard Form Tableau Form Setting Up the Initial Simplex Tableau Improving the Solution Calculating the Next Tableau
More information1 Newton s Method. Suppose we want to solve: x R. At x = x, f (x) can be approximated by:
Newton s Method Suppose we want to solve: (P:) min f (x) At x = x, f (x) can be approximated by: n x R. f (x) h(x) := f ( x)+ f ( x) T (x x)+ (x x) t H ( x)(x x), 2 which is the quadratic Taylor expansion
More informationPart 2: NLP Constrained Optimization
Part 2: NLP Constrained Optimization James G. Shanahan 2 Independent Consultant and Lecturer UC Santa Cruz EMAIL: James_DOT_Shanahan_AT_gmail_DOT_com WIFI: SSID Student USERname ucsc-guest Password EnrollNow!
More information