4M020 Design tools. Algorithms for numerical optimization. L.F.P. Etman. Department of Mechanical Engineering Eindhoven University of Technology
|
|
- Baldric Neal
- 5 years ago
- Views:
Transcription
1 4M020 Design tools Algorithms for numerical optimization L.F.P. Etman Department of Mechanical Engineering Eindhoven University of Technology Wednesday September 3, / 32
2 Outline 1 Problem formulation: classes and properties 2 Optimization algorithms 2 / 32
3 Problem formulation: classes and properties The mathematical problem formulation is to the heart of the success of design optimization 3 / 32
4 Two-bar truss example d S F S h min x f(x) = C 0 x2 2 S 2 + x1 2 s.t. g 1 (x) = C 1 x g 2 (x) = C 2 S 2 + x1 2 x2 2x ( S 2 + x1 2 g 3 (x) = C 3 x2 4 x 1 χ : x 1, x 2 > 0, ) 3/2 1 0 How can we solve this optimization problem? 4 / 32
5 Mathematical problem formulation Minimize f(x) x = (column) vector of design variables x subject to h j (x) = 0 j = 1,..., m h g k (x) 0 k = 1,..., m g x X R n [Papalambros & Wilde 2000: Principles of optimal design] 5 / 32
6 Formulation examples Design variables: sizing (dimensions) shape (geometry of boundary) topological (material distribution) 6 / 32
7 Formulation examples Design variables x: sizing (dimensions) shape (geometry of boundary) topological (material distribution) Objective function f(x): profit (cost, efficiency, weight,...) Constraint functions h j (x) and g k (x): geometrical (width, length, height,...) structural (stresses, displacements,...) dynamical (accelerations, eigenfrequency,...) physical (temperatures, pressures,...) 7 / 32
8 Formulation classes Design variables: continuous or discrete single-variable or multi-variable Objective function: minimization or maximization single-objective or multi-objective Constraint functions: unconstrained or constrained equality or inequality 8 / 32
9 Formulation classes: linear programming LP temp 2006 Linear programming (LP) problem: Linear objective function Linear constraint functions Continuous design variables g 3 f Example: x 2 g 2 min x R 2 f(x) = 2x 1 x 2 s.t. g 1 (x) = x 1 + 2x g 2 (x) = 2x 1 2x g 3 (x) = 2x g 4 (x) = 2x (Some) design variables discrete: (Mixed-)integer linear programming problem: (M)ILP F x 1 g 4 g 1 9 / 32
10 Formulation classes: quadratic programming QP temp 2006 Quadratic programming (QP) problem: Quadratic objective function Linear constraint functions Continuous design variables f g 3 Example: x 2 min x R 2 f(x) = 3x 2 1 2x 1 + 5x x 2 s.t. g 1 (x) = 2x 1 3x g 2 (x) = 3x 1 + 2x g 3 (x) = x F g 2 g 1 (Some) design variables discrete: (Mixed-)integer quadratic programming problem: (M)IQP x 1 10 / 32
11 Formulation classes: nonlinear programming NLP temp 200 Nonlinear programming (NLP) problem: Non-linear objective function Non-linear constraint functions Continuous design variables Example: min f(x) = 3x 1 + 3x 2 x R 2 s.t. g 1 (x) = 18 x x g 2 (x) = 5.73 x 1 0 g 3 (x) = 7.17 x 2 0 (Some) design variables discrete: (Mixed-)integer nonlinear programming problem: (M)INLP x 2 g 2 g 3 g 1 F f x 1 11 / 32
12 Formulation properties Design variables: Domain: {0, 1}, N, N +, Z, R, R + Objective and constraint functions: Linearity: linear or nonlinear Continuity: none/once/twice-differentiable Optimization problem: Modality: uni-modal or multi-modal 12 / 32
13 Two-bar truss example d S F S h min x f(x) = C 0 x2 2 S 2 + x1 2 s.t. g 1 (x) = C 1 x g 2 (x) = C 2 S 2 + x1 2 x2 2x ( S 2 + x1 2 g 3 (x) = C 3 x2 4 x 1 χ : x 1, x 2 > 0, ) 3/2 1 0 Exercise 1: introduction to the Matlab algorithm fmincon 13 / 32
14 Optimization algorithms Select an optimization algorithm in accordance with the optimization problem class and properties 14 / 32
15 Searching for a zero ( ) 0 15 / 32 Solving a set of m nonlinear equations with m unknowns: Φ(q) = 0 Example: 1 + e q 2 q 2 2 e q 1 = 0 1 q 1 e q 2 + 2q 2 e q 1 = 0
16 Searching for a minimum ( ) Finding the minimum of a function with n variables: Minimize x f(x) Example: Min x f(x) = x 1 + x 2 + x 1 e x 2 + x 2 2 e x 1 16 / 32
17 Zero-search is special case of minimum-search Finding the solution to a set of nonlinear equations: Φ(q) = 0 can be reformulated as a minimization problem: Minimize q e T e with the error (deviations to zero) defined as: e = Φ(q) 17 / 32
18 Analytical versus numerical solution Some optimization problems can be analytically solved: Min x f(x) = 2x 1 + x x 2 + x 2 2 Many optimization problems can only be numerically solved: Min x f(x) = x 1 + x 2 + x 1 e x 2 + x 2 2 e x 1 An optimization algorithm is an iterative procedure to solve an unconstrained or constrained minimization (maximization) problem. [Papalambros & Wilde 2000: Principles of optimal design] 18 / 32
19 Classification of optimization algorithms Four classifiers of an optimization algorithm: unconstrained / constrained search local / global search determistic / stochastic search 0 th / 1 th / 2 nd -order search 19 / 32
20 Unconstrained / constrained search Unconstrained search: aims to minimize a nonlinear, possibly multi-modal, objective function in n-dimensional space Constrained search: aims to minimize a (non)linear objective function in n-dimensional space while accounting for (non)linear equality and/or inequality constraint functions 20 / 32
21 Local / global search Local search: seeks improvement based on the Taylor series expansion; in a sufficiently small region any nonlinear function can be represented by a quadratic approximation: f(x) = f(x )+ n i=1 f(x ) x i (x i x i )+ 1 2 n n i=1 j=1 2 f(x ) x i x j (x i x i )2 Global search: seeks improvement by design space exploration 21 / 32
22 Deterministic / stochastic search Deterministic search: gives exactly the same search path when running the algorithm twice for unchanged algorithmic settings Stochastic search: generates random search paths for every run of the algorithm 22 / 32
23 0 th / 1 th / 2 nd -order search A 0 th -order algorithm uses function evaluations only A 1 th -order algorithm uses function and gradient evaluations A 2 nd -order algorithm uses function, gradient, and Hessian evaluations 23 / 32
24 Newton type of algorithms Characteristics line2 temp 2006/6/26 Matlab Optimization 14:30 trust2 temp page toolbox 1 # unconstrained/constrained fminunc local fmincon deterministic 2 nd -order x 1 x 2 x 0 s 0 x 3 s 1 x s 2 2 s 3 x x 1 x 2 2 s 0 s 1 s2 x 0 x 3 3 Line search x 1 Trust-region x 1 24 / 32
25 Bio-inspired algorithms Characteristics unconstrained/constrained global stochastic 0 th -order Algorithms Particle swarms Genetic algorithms (GAs) Simulated Annealing Matlab GA toolbox 2Dparticleswarm temp ga / 32
26 Direct search algorithms Characteristics unconstrained/constrained local deterministic 0 th -order Algorithms Nelder-Mead simplex search Generalized Pattern Search Matlab Optimization 2Dsimplex temp toolbox fminsearch Matlab Direct Search toolbox patternsearch / 32
27 Two-bar truss example d S F S h min x f(x) = C 0 x2 2 S 2 + x1 2 s.t. g 1 (x) = C 1 x g 2 (x) = C 2 S 2 + x1 2 x2 2x ( S 2 + x1 2 g 3 (x) = C 3 x2 4 x 1 χ : x 1, x 2 > 0, ) 3/2 1 0 Exercise 2: visualization of the optimization search path 27 / 32
28 Gradients Calculating derivatives: Analytically (by hand) Symbolically (e.g. Mathematica, Maple) Finite differencing By the CAE analysis code (e.g. MARC) Automatic differentiation (e.g. ADIFOR, ADIC) Matlab: fmincon and fminunc calculate gradients by finite differencing if the user does not provide them Matlab Symbolic Toolbox Matlab Automatic Differentiation Toolbox 28 / 32
29 Conditions for optimality unc infwell2d temp 2007/4/4 15:12 ineq opt temp page 1 #12006/6/20 10:00 page 1 #1 Karush-Kuhn-Tucker (KKT) conditions x 2 f x 2 F f x 2 g 1 F f 3 = 0 g 3 P 2 g g 1 g 1 2 g 2 g 2 P 1 x 1 x 1 x 1 Unconstrained: gradients objective function zero Constrained: linear combination of gradients objective and gradients active constraints zero (i.e. gradients Lagrange function L zero) 29 / 32
30 Termination criteria Condition on optimality L(x k+1 ) < ε g j (x k+1 ) < ε and h j (x k+1 ) < ε (Matlab: TolFun) (Matlab: TolCon) or a condition on change in x x k+1 x k < ε (Matlab: TolX) or a condition on the number of iterations k k max (Matlab: MaxIter) or a combination of these, with ε > 0 30 / 32
31 Scaling Scale design variables, objective function and constraint functions to avoid numerical difficulties and premature termination 31 / 32
32 Two-bar truss example d F h min x f(x) = m s.t. g 1 (x) = C 1 x g 2 (x) = σ(x) σ y 1 0 g 3 (x) = P(x) P c (x) 1 0 χ : x 1, x 2 > 0, S S Exercise 3: FEM-model in the optimization loop 32 / 32
Numerical Optimization. Review: Unconstrained Optimization
Numerical Optimization Finding the best feasible solution Edward P. Gatzke Department of Chemical Engineering University of South Carolina Ed Gatzke (USC CHE ) Numerical Optimization ECHE 589, Spring 2011
More informationScientific Computing: Optimization
Scientific Computing: Optimization Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 Course MATH-GA.2043 or CSCI-GA.2112, Spring 2012 March 8th, 2011 A. Donev (Courant Institute) Lecture
More informationContents. Preface. 1 Introduction Optimization view on mathematical models NLP models, black-box versus explicit expression 3
Contents Preface ix 1 Introduction 1 1.1 Optimization view on mathematical models 1 1.2 NLP models, black-box versus explicit expression 3 2 Mathematical modeling, cases 7 2.1 Introduction 7 2.2 Enclosing
More informationLecture 18: Optimization Programming
Fall, 2016 Outline Unconstrained Optimization 1 Unconstrained Optimization 2 Equality-constrained Optimization Inequality-constrained Optimization Mixture-constrained Optimization 3 Quadratic Programming
More informationNonlinear Programming (Hillier, Lieberman Chapter 13) CHEM-E7155 Production Planning and Control
Nonlinear Programming (Hillier, Lieberman Chapter 13) CHEM-E7155 Production Planning and Control 19/4/2012 Lecture content Problem formulation and sample examples (ch 13.1) Theoretical background Graphical
More informationConstrained Optimization
1 / 22 Constrained Optimization ME598/494 Lecture Max Yi Ren Department of Mechanical Engineering, Arizona State University March 30, 2015 2 / 22 1. Equality constraints only 1.1 Reduced gradient 1.2 Lagrange
More informationISM206 Lecture Optimization of Nonlinear Objective with Linear Constraints
ISM206 Lecture Optimization of Nonlinear Objective with Linear Constraints Instructor: Prof. Kevin Ross Scribe: Nitish John October 18, 2011 1 The Basic Goal The main idea is to transform a given constrained
More informationECS550NFB Introduction to Numerical Methods using Matlab Day 2
ECS550NFB Introduction to Numerical Methods using Matlab Day 2 Lukas Laffers lukas.laffers@umb.sk Department of Mathematics, University of Matej Bel June 9, 2015 Today Root-finding: find x that solves
More informationOptimization Problems with Constraints - introduction to theory, numerical Methods and applications
Optimization Problems with Constraints - introduction to theory, numerical Methods and applications Dr. Abebe Geletu Ilmenau University of Technology Department of Simulation and Optimal Processes (SOP)
More informationLecture V. Numerical Optimization
Lecture V Numerical Optimization Gianluca Violante New York University Quantitative Macroeconomics G. Violante, Numerical Optimization p. 1 /19 Isomorphism I We describe minimization problems: to maximize
More informationCE 191: Civil and Environmental Engineering Systems Analysis. LEC 05 : Optimality Conditions
CE 191: Civil and Environmental Engineering Systems Analysis LEC : Optimality Conditions Professor Scott Moura Civil & Environmental Engineering University of California, Berkeley Fall 214 Prof. Moura
More informationOptimization Methods
Optimization Methods Decision making Examples: determining which ingredients and in what quantities to add to a mixture being made so that it will meet specifications on its composition allocating available
More informationMVE165/MMG631 Linear and integer optimization with applications Lecture 13 Overview of nonlinear programming. Ann-Brith Strömberg
MVE165/MMG631 Overview of nonlinear programming Ann-Brith Strömberg 2015 05 21 Areas of applications, examples (Ch. 9.1) Structural optimization Design of aircraft, ships, bridges, etc Decide on the material
More informationNatSciLab - Numerical Software Introduction to MATLAB
Outline 110112 NatSciLab - Numerical Software Introduction to MATLAB Onur Oktay Jacobs University Bremen Spring 2010 Outline 1 Optimization with Matlab 2 Least Squares problem Linear Least Squares Method
More informationAM 205: lecture 19. Last time: Conditions for optimality, Newton s method for optimization Today: survey of optimization methods
AM 205: lecture 19 Last time: Conditions for optimality, Newton s method for optimization Today: survey of optimization methods Quasi-Newton Methods General form of quasi-newton methods: x k+1 = x k α
More informationGeneralization to inequality constrained problem. Maximize
Lecture 11. 26 September 2006 Review of Lecture #10: Second order optimality conditions necessary condition, sufficient condition. If the necessary condition is violated the point cannot be a local minimum
More informationSupport Vector Machines: Maximum Margin Classifiers
Support Vector Machines: Maximum Margin Classifiers Machine Learning and Pattern Recognition: September 16, 2008 Piotr Mirowski Based on slides by Sumit Chopra and Fu-Jie Huang 1 Outline What is behind
More informationOptimization: Nonlinear Optimization without Constraints. Nonlinear Optimization without Constraints 1 / 23
Optimization: Nonlinear Optimization without Constraints Nonlinear Optimization without Constraints 1 / 23 Nonlinear optimization without constraints Unconstrained minimization min x f(x) where f(x) is
More information5 Handling Constraints
5 Handling Constraints Engineering design optimization problems are very rarely unconstrained. Moreover, the constraints that appear in these problems are typically nonlinear. This motivates our interest
More informationCONSTRAINED NONLINEAR PROGRAMMING
149 CONSTRAINED NONLINEAR PROGRAMMING We now turn to methods for general constrained nonlinear programming. These may be broadly classified into two categories: 1. TRANSFORMATION METHODS: In this approach
More informationEAD 115. Numerical Solution of Engineering and Scientific Problems. David M. Rocke Department of Applied Science
EAD 115 Numerical Solution of Engineering and Scientific Problems David M. Rocke Department of Applied Science Multidimensional Unconstrained Optimization Suppose we have a function f() of more than one
More informationOptimization Tutorial 1. Basic Gradient Descent
E0 270 Machine Learning Jan 16, 2015 Optimization Tutorial 1 Basic Gradient Descent Lecture by Harikrishna Narasimhan Note: This tutorial shall assume background in elementary calculus and linear algebra.
More informationInstitute of Eng. and Comp. Mechanics Optimization of Mechanical Systems Prof. Dr.-Ing. Prof. E.h. P. Eberhard WT 18/19 A1.1.
Prof. Dr.-Ing. Prof. E.h. P. Eberhard WT 18/19 A1.1 Motivational Example 1 In medieval times catapults where used in the siege of castles. These catapults where capable of throwing heavy stones over a
More informationNONLINEAR. (Hillier & Lieberman Introduction to Operations Research, 8 th edition)
NONLINEAR PROGRAMMING (Hillier & Lieberman Introduction to Operations Research, 8 th edition) Nonlinear Programming g Linear programming has a fundamental role in OR. In linear programming all its functions
More informationCHAPTER 2: QUADRATIC PROGRAMMING
CHAPTER 2: QUADRATIC PROGRAMMING Overview Quadratic programming (QP) problems are characterized by objective functions that are quadratic in the design variables, and linear constraints. In this sense,
More informationminimize x subject to (x 2)(x 4) u,
Math 6366/6367: Optimization and Variational Methods Sample Preliminary Exam Questions 1. Suppose that f : [, L] R is a C 2 -function with f () on (, L) and that you have explicit formulae for
More informationAlgorithms for constrained local optimization
Algorithms for constrained local optimization Fabio Schoen 2008 http://gol.dsi.unifi.it/users/schoen Algorithms for constrained local optimization p. Feasible direction methods Algorithms for constrained
More informationStructural and Multidisciplinary Optimization. P. Duysinx and P. Tossings
Structural and Multidisciplinary Optimization P. Duysinx and P. Tossings 2018-2019 CONTACTS Pierre Duysinx Institut de Mécanique et du Génie Civil (B52/3) Phone number: 04/366.91.94 Email: P.Duysinx@uliege.be
More informationNumerical optimization
Numerical optimization Lecture 4 Alexander & Michael Bronstein tosca.cs.technion.ac.il/book Numerical geometry of non-rigid shapes Stanford University, Winter 2009 2 Longest Slowest Shortest Minimal Maximal
More informationLinear Support Vector Machine. Classification. Linear SVM. Huiping Cao. Huiping Cao, Slide 1/26
Huiping Cao, Slide 1/26 Classification Linear SVM Huiping Cao linear hyperplane (decision boundary) that will separate the data Huiping Cao, Slide 2/26 Support Vector Machines rt Vector Find a linear Machines
More informationA Trust-region-based Sequential Quadratic Programming Algorithm
Downloaded from orbit.dtu.dk on: Oct 19, 2018 A Trust-region-based Sequential Quadratic Programming Algorithm Henriksen, Lars Christian; Poulsen, Niels Kjølstad Publication date: 2010 Document Version
More informationNumerical optimization. Numerical optimization. Longest Shortest where Maximal Minimal. Fastest. Largest. Optimization problems
1 Numerical optimization Alexander & Michael Bronstein, 2006-2009 Michael Bronstein, 2010 tosca.cs.technion.ac.il/book Numerical optimization 048921 Advanced topics in vision Processing and Analysis of
More informationNumerical Optimization: Basic Concepts and Algorithms
May 27th 2015 Numerical Optimization: Basic Concepts and Algorithms R. Duvigneau R. Duvigneau - Numerical Optimization: Basic Concepts and Algorithms 1 Outline Some basic concepts in optimization Some
More informationPattern Classification, and Quadratic Problems
Pattern Classification, and Quadratic Problems (Robert M. Freund) March 3, 24 c 24 Massachusetts Institute of Technology. 1 1 Overview Pattern Classification, Linear Classifiers, and Quadratic Optimization
More informationAM 205: lecture 19. Last time: Conditions for optimality Today: Newton s method for optimization, survey of optimization methods
AM 205: lecture 19 Last time: Conditions for optimality Today: Newton s method for optimization, survey of optimization methods Optimality Conditions: Equality Constrained Case As another example of equality
More informationOptimization. Yuh-Jye Lee. March 21, Data Science and Machine Intelligence Lab National Chiao Tung University 1 / 29
Optimization Yuh-Jye Lee Data Science and Machine Intelligence Lab National Chiao Tung University March 21, 2017 1 / 29 You Have Learned (Unconstrained) Optimization in Your High School Let f (x) = ax
More informationMATHEMATICS FOR COMPUTER VISION WEEK 8 OPTIMISATION PART 2. Dr Fabio Cuzzolin MSc in Computer Vision Oxford Brookes University Year
MATHEMATICS FOR COMPUTER VISION WEEK 8 OPTIMISATION PART 2 1 Dr Fabio Cuzzolin MSc in Computer Vision Oxford Brookes University Year 2013-14 OUTLINE OF WEEK 8 topics: quadratic optimisation, least squares,
More informationNonlinear Optimization: What s important?
Nonlinear Optimization: What s important? Julian Hall 10th May 2012 Convexity: convex problems A local minimizer is a global minimizer A solution of f (x) = 0 (stationary point) is a minimizer A global
More informationConstrained Optimization and Lagrangian Duality
CIS 520: Machine Learning Oct 02, 2017 Constrained Optimization and Lagrangian Duality Lecturer: Shivani Agarwal Disclaimer: These notes are designed to be a supplement to the lecture. They may or may
More informationNumerical Optimization of Partial Differential Equations
Numerical Optimization of Partial Differential Equations Part I: basic optimization concepts in R n Bartosz Protas Department of Mathematics & Statistics McMaster University, Hamilton, Ontario, Canada
More informationICS-E4030 Kernel Methods in Machine Learning
ICS-E4030 Kernel Methods in Machine Learning Lecture 3: Convex optimization and duality Juho Rousu 28. September, 2016 Juho Rousu 28. September, 2016 1 / 38 Convex optimization Convex optimisation This
More informationNumerical Optimization
Constrained Optimization Computer Science and Automation Indian Institute of Science Bangalore 560 012, India. NPTEL Course on Constrained Optimization Constrained Optimization Problem: min h j (x) 0,
More informationIE 5531: Engineering Optimization I
IE 5531: Engineering Optimization I Lecture 12: Nonlinear optimization, continued Prof. John Gunnar Carlsson October 20, 2010 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I October 20,
More informationComputational Optimization. Augmented Lagrangian NW 17.3
Computational Optimization Augmented Lagrangian NW 17.3 Upcoming Schedule No class April 18 Friday, April 25, in class presentations. Projects due unless you present April 25 (free extension until Monday
More informationSolution Methods. Richard Lusby. Department of Management Engineering Technical University of Denmark
Solution Methods Richard Lusby Department of Management Engineering Technical University of Denmark Lecture Overview (jg Unconstrained Several Variables Quadratic Programming Separable Programming SUMT
More informationComputational Finance
Department of Mathematics at University of California, San Diego Computational Finance Optimization Techniques [Lecture 2] Michael Holst January 9, 2017 Contents 1 Optimization Techniques 3 1.1 Examples
More informationMotivation: We have already seen an example of a system of nonlinear equations when we studied Gaussian integration (p.8 of integration notes)
AMSC/CMSC 460 Computational Methods, Fall 2007 UNIT 5: Nonlinear Equations Dianne P. O Leary c 2001, 2002, 2007 Solving Nonlinear Equations and Optimization Problems Read Chapter 8. Skip Section 8.1.1.
More informationMotivation. Lecture 2 Topics from Optimization and Duality. network utility maximization (NUM) problem:
CDS270 Maryam Fazel Lecture 2 Topics from Optimization and Duality Motivation network utility maximization (NUM) problem: consider a network with S sources (users), each sending one flow at rate x s, through
More informationLecture 3. Optimization Problems and Iterative Algorithms
Lecture 3 Optimization Problems and Iterative Algorithms January 13, 2016 This material was jointly developed with Angelia Nedić at UIUC for IE 598ns Outline Special Functions: Linear, Quadratic, Convex
More information2.3 Linear Programming
2.3 Linear Programming Linear Programming (LP) is the term used to define a wide range of optimization problems in which the objective function is linear in the unknown variables and the constraints are
More informationNumerical Optimization Professor Horst Cerjak, Horst Bischof, Thomas Pock Mat Vis-Gra SS09
Numerical Optimization 1 Working Horse in Computer Vision Variational Methods Shape Analysis Machine Learning Markov Random Fields Geometry Common denominator: optimization problems 2 Overview of Methods
More informationMore on Lagrange multipliers
More on Lagrange multipliers CE 377K April 21, 2015 REVIEW The standard form for a nonlinear optimization problem is min x f (x) s.t. g 1 (x) 0. g l (x) 0 h 1 (x) = 0. h m (x) = 0 The objective function
More informationNumerical Optimization Techniques
Numerical Optimization Techniques Léon Bottou NEC Labs America COS 424 3/2/2010 Today s Agenda Goals Representation Capacity Control Operational Considerations Computational Considerations Classification,
More informationLectures 9 and 10: Constrained optimization problems and their optimality conditions
Lectures 9 and 10: Constrained optimization problems and their optimality conditions Coralia Cartis, Mathematical Institute, University of Oxford C6.2/B2: Continuous Optimization Lectures 9 and 10: Constrained
More informationConstrained optimization
Constrained optimization In general, the formulation of constrained optimization is as follows minj(w), subject to H i (w) = 0, i = 1,..., k. where J is the cost function and H i are the constraints. Lagrange
More informationOptimization. Escuela de Ingeniería Informática de Oviedo. (Dpto. de Matemáticas-UniOvi) Numerical Computation Optimization 1 / 30
Optimization Escuela de Ingeniería Informática de Oviedo (Dpto. de Matemáticas-UniOvi) Numerical Computation Optimization 1 / 30 Unconstrained optimization Outline 1 Unconstrained optimization 2 Constrained
More informationSF2822 Applied nonlinear optimization, final exam Saturday December
SF2822 Applied nonlinear optimization, final exam Saturday December 5 27 8. 3. Examiner: Anders Forsgren, tel. 79 7 27. Allowed tools: Pen/pencil, ruler and rubber; plus a calculator provided by the department.
More information8 Numerical methods for unconstrained problems
8 Numerical methods for unconstrained problems Optimization is one of the important fields in numerical computation, beside solving differential equations and linear systems. We can see that these fields
More informationCS-E4830 Kernel Methods in Machine Learning
CS-E4830 Kernel Methods in Machine Learning Lecture 3: Convex optimization and duality Juho Rousu 27. September, 2017 Juho Rousu 27. September, 2017 1 / 45 Convex optimization Convex optimisation This
More informationInverse Problems and Optimal Design in Electricity and Magnetism
Inverse Problems and Optimal Design in Electricity and Magnetism P. Neittaanmäki Department of Mathematics, University of Jyväskylä M. Rudnicki Institute of Electrical Engineering, Warsaw and A. Savini
More informationTopic one: Production line profit maximization subject to a production rate constraint. c 2010 Chuan Shi Topic one: Line optimization : 22/79
Topic one: Production line profit maximization subject to a production rate constraint c 21 Chuan Shi Topic one: Line optimization : 22/79 Production line profit maximization The profit maximization problem
More information2.098/6.255/ Optimization Methods Practice True/False Questions
2.098/6.255/15.093 Optimization Methods Practice True/False Questions December 11, 2009 Part I For each one of the statements below, state whether it is true or false. Include a 1-3 line supporting sentence
More informationAlvaro F. M. Azevedo A. Adão da Fonseca
SECOND-ORDER SHAPE OPTIMIZATION OF A STEEL BRIDGE Alvaro F. M. Azevedo A. Adão da Fonseca Faculty of Engineering University of Porto Portugal 16-18 March 1999 OPTI 99 Orlando - Florida - USA 1 PROBLEM
More informationMerging Optimization and Control
Merging Optimization and Control Bjarne Foss and Tor Aksel N. Heirung September 4, 2013 Contents 1 Introduction 2 2 Optimization 3 2.1 Classes of optimization problems.................. 3 2.2 Solution
More informationCE 191: Civil & Environmental Engineering Systems Analysis. LEC 17 : Final Review
CE 191: Civil & Environmental Engineering Systems Analysis LEC 17 : Final Review Professor Scott Moura Civil & Environmental Engineering University of California, Berkeley Fall 2014 Prof. Moura UC Berkeley
More information6.252 NONLINEAR PROGRAMMING LECTURE 10 ALTERNATIVES TO GRADIENT PROJECTION LECTURE OUTLINE. Three Alternatives/Remedies for Gradient Projection
6.252 NONLINEAR PROGRAMMING LECTURE 10 ALTERNATIVES TO GRADIENT PROJECTION LECTURE OUTLINE Three Alternatives/Remedies for Gradient Projection Two-Metric Projection Methods Manifold Suboptimization Methods
More informationg(x,y) = c. For instance (see Figure 1 on the right), consider the optimization problem maximize subject to
1 of 11 11/29/2010 10:39 AM From Wikipedia, the free encyclopedia In mathematical optimization, the method of Lagrange multipliers (named after Joseph Louis Lagrange) provides a strategy for finding the
More informationOutline Introduction: Problem Description Diculties Algebraic Structure: Algebraic Varieties Rank Decient Toeplitz Matrices Constructing Lower Rank St
Structured Lower Rank Approximation by Moody T. Chu (NCSU) joint with Robert E. Funderlic (NCSU) and Robert J. Plemmons (Wake Forest) March 5, 1998 Outline Introduction: Problem Description Diculties Algebraic
More informationMultiple Bits Distributed Moving Horizon State Estimation for Wireless Sensor Networks. Ji an Luo
Multiple Bits Distributed Moving Horizon State Estimation for Wireless Sensor Networks Ji an Luo 2008.6.6 Outline Background Problem Statement Main Results Simulation Study Conclusion Background Wireless
More informationOptimization Methods for Circuit Design
Technische Universität München Department of Electrical Engineering and Information Technology Institute for Electronic Design Automation Optimization Methods for Circuit Design Compendium H. Graeb Version
More informationAlgorithms for Constrained Optimization
1 / 42 Algorithms for Constrained Optimization ME598/494 Lecture Max Yi Ren Department of Mechanical Engineering, Arizona State University April 19, 2015 2 / 42 Outline 1. Convergence 2. Sequential quadratic
More informationConstrained optimization: direct methods (cont.)
Constrained optimization: direct methods (cont.) Jussi Hakanen Post-doctoral researcher jussi.hakanen@jyu.fi Direct methods Also known as methods of feasible directions Idea in a point x h, generate a
More informationMicroeconomics I. September, c Leopold Sögner
Microeconomics I c Leopold Sögner Department of Economics and Finance Institute for Advanced Studies Stumpergasse 56 1060 Wien Tel: +43-1-59991 182 soegner@ihs.ac.at http://www.ihs.ac.at/ soegner September,
More informationE5295/5B5749 Convex optimization with engineering applications. Lecture 8. Smooth convex unconstrained and equality-constrained minimization
E5295/5B5749 Convex optimization with engineering applications Lecture 8 Smooth convex unconstrained and equality-constrained minimization A. Forsgren, KTH 1 Lecture 8 Convex optimization 2006/2007 Unconstrained
More informationOn Nesterov s Random Coordinate Descent Algorithms - Continued
On Nesterov s Random Coordinate Descent Algorithms - Continued Zheng Xu University of Texas At Arlington February 20, 2015 1 Revisit Random Coordinate Descent The Random Coordinate Descent Upper and Lower
More informationUnit: Optimality Conditions and Karush Kuhn Tucker Theorem
Unit: Optimality Conditions and Karush Kuhn Tucker Theorem Goals 1. What is the Gradient of a function? What are its properties? 2. How can it be used to find a linear approximation of a nonlinear function?
More informationSufficient Conditions for Finite-variable Constrained Minimization
Lecture 4 It is a small de tour but it is important to understand this before we move to calculus of variations. Sufficient Conditions for Finite-variable Constrained Minimization ME 256, Indian Institute
More informationAppendix A Taylor Approximations and Definite Matrices
Appendix A Taylor Approximations and Definite Matrices Taylor approximations provide an easy way to approximate a function as a polynomial, using the derivatives of the function. We know, from elementary
More informationReview of Optimization Basics
Review of Optimization Basics. Introduction Electricity markets throughout the US are said to have a two-settlement structure. The reason for this is that the structure includes two different markets:
More informationLecture 13: Constrained optimization
2010-12-03 Basic ideas A nonlinearly constrained problem must somehow be converted relaxed into a problem which we can solve (a linear/quadratic or unconstrained problem) We solve a sequence of such problems
More informationLecture 7: Convex Optimizations
Lecture 7: Convex Optimizations Radu Balan, David Levermore March 29, 2018 Convex Sets. Convex Functions A set S R n is called a convex set if for any points x, y S the line segment [x, y] := {tx + (1
More informationMultidisciplinary System Design Optimization (MSDO)
Multidisciplinary System Design Optimization (MSDO) Numerical Optimization II Lecture 8 Karen Willcox 1 Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Today s Topics Sequential
More informationAlgorithms for nonlinear programming problems II
Algorithms for nonlinear programming problems II Martin Branda Charles University Faculty of Mathematics and Physics Department of Probability and Mathematical Statistics Computational Aspects of Optimization
More informationUnconstrained Multivariate Optimization
Unconstrained Multivariate Optimization Multivariate optimization means optimization of a scalar function of a several variables: and has the general form: y = () min ( ) where () is a nonlinear scalar-valued
More informationIntroduction to unconstrained optimization - direct search methods
Introduction to unconstrained optimization - direct search methods Jussi Hakanen Post-doctoral researcher jussi.hakanen@jyu.fi Structure of optimization methods Typically Constraint handling converts the
More informationNonlinear Optimization for Optimal Control
Nonlinear Optimization for Optimal Control Pieter Abbeel UC Berkeley EECS Many slides and figures adapted from Stephen Boyd [optional] Boyd and Vandenberghe, Convex Optimization, Chapters 9 11 [optional]
More informationGradient Descent. Dr. Xiaowei Huang
Gradient Descent Dr. Xiaowei Huang https://cgi.csc.liv.ac.uk/~xiaowei/ Up to now, Three machine learning algorithms: decision tree learning k-nn linear regression only optimization objectives are discussed,
More informationIntroduction to Machine Learning Lecture 7. Mehryar Mohri Courant Institute and Google Research
Introduction to Machine Learning Lecture 7 Mehryar Mohri Courant Institute and Google Research mohri@cims.nyu.edu Convex Optimization Differentiation Definition: let f : X R N R be a differentiable function,
More informationOptimal control problems with PDE constraints
Optimal control problems with PDE constraints Maya Neytcheva CIM, October 2017 General framework Unconstrained optimization problems min f (q) q x R n (real vector) and f : R n R is a smooth function.
More informationOptimization. Next: Curve Fitting Up: Numerical Analysis for Chemical Previous: Linear Algebraic and Equations. Subsections
Next: Curve Fitting Up: Numerical Analysis for Chemical Previous: Linear Algebraic and Equations Subsections One-dimensional Unconstrained Optimization Golden-Section Search Quadratic Interpolation Newton's
More informationPrediktivno upravljanje primjenom matematičkog programiranja
Prediktivno upravljanje primjenom matematičkog programiranja Doc. dr. sc. Mato Baotić Fakultet elektrotehnike i računarstva Sveučilište u Zagrebu www.fer.hr/mato.baotic Outline Application Examples PredictiveControl
More informationInterpolation-Based Trust-Region Methods for DFO
Interpolation-Based Trust-Region Methods for DFO Luis Nunes Vicente University of Coimbra (joint work with A. Bandeira, A. R. Conn, S. Gratton, and K. Scheinberg) July 27, 2010 ICCOPT, Santiago http//www.mat.uc.pt/~lnv
More informationNewton-Type Algorithms for Nonlinear Constrained Optimization
Newton-Type Algorithms for Nonlinear Constrained Optimization Angelika Altmann-Dieses Faculty of Management Science and Engineering Karlsruhe University of Applied Sciences Moritz Diehl Department of Microsystems
More informationNonlinear Optimization
Nonlinear Optimization Etienne de Klerk (UvT)/Kees Roos e-mail: C.Roos@ewi.tudelft.nl URL: http://www.isa.ewi.tudelft.nl/ roos Course WI3031 (Week 4) February-March, A.D. 2005 Optimization Group 1 Outline
More informationQuadratic Programming
Quadratic Programming Outline Linearly constrained minimization Linear equality constraints Linear inequality constraints Quadratic objective function 2 SideBar: Matrix Spaces Four fundamental subspaces
More informationSECTION C: CONTINUOUS OPTIMISATION LECTURE 9: FIRST ORDER OPTIMALITY CONDITIONS FOR CONSTRAINED NONLINEAR PROGRAMMING
Nf SECTION C: CONTINUOUS OPTIMISATION LECTURE 9: FIRST ORDER OPTIMALITY CONDITIONS FOR CONSTRAINED NONLINEAR PROGRAMMING f(x R m g HONOUR SCHOOL OF MATHEMATICS, OXFORD UNIVERSITY HILARY TERM 5, DR RAPHAEL
More informationDetermination of Feasible Directions by Successive Quadratic Programming and Zoutendijk Algorithms: A Comparative Study
International Journal of Mathematics And Its Applications Vol.2 No.4 (2014), pp.47-56. ISSN: 2347-1557(online) Determination of Feasible Directions by Successive Quadratic Programming and Zoutendijk Algorithms:
More informationOn the Relationship Between Regression Analysis and Mathematical Programming
JOURNAL OF APPLIED MATHEMATICS AND DECISION SCIENCES, 8(2), 3 40 Copyright c 2004, Lawrence Erlbaum Associates, Inc. On the Relationship Between Regression Analysis and Mathematical Programming DONG QIAN
More informationECE 680 Modern Automatic Control. Gradient and Newton s Methods A Review
ECE 680Modern Automatic Control p. 1/1 ECE 680 Modern Automatic Control Gradient and Newton s Methods A Review Stan Żak October 25, 2011 ECE 680Modern Automatic Control p. 2/1 Review of the Gradient Properties
More informationLecture 3: Basics of set-constrained and unconstrained optimization
Lecture 3: Basics of set-constrained and unconstrained optimization (Chap 6 from textbook) Xiaoqun Zhang Shanghai Jiao Tong University Last updated: October 9, 2018 Optimization basics Outline Optimization
More information