Solving Minimax Problems with Feasible Sequential Quadratic Programming
|
|
|
- Dana Parsons
- 5 years ago
- Views:
Transcription
1 Qianli Deng (Shally) Advisor: Dr. Mark A. Austin Department of Civil and Environmental Engineering Institute for Systems Research University of Maryland, College Park, MD Solving Minimax Problems with Feasible Sequential Quadratic Programming 05/06/2014
2 Constrained minimax problem bl x bu nb g j ( x) 0 j = 1,..., n g ( x) c, x d 0 j = 1,..., n h j ( x) = 0 j = 1,..., n h ( x) a, x b = 0 j = 1,..., n j j j LI j j j LE NI NE 2 5/5/2014
3 Constrained minimax problem * Objectives bl x bu nb g j ( x) 0 j = 1,..., n g ( x) c, x d 0 j = 1,..., n h j ( x) = 0 j = 1,..., n h ( x) a, x b = 0 j = 1,..., n j j j LI j j j LE NI NE 3 5/5/2014
4 Constrained minimax problem * Objectives bl x bu nb g j ( x) 0 j = 1,..., n g ( x) c, x d 0 j = 1,..., n h j ( x) = 0 j = 1,..., n h ( x) a, x b = 0 j = 1,..., n j j j LI j j j LE NI NE * Bounds * Nonlinear inequality * Linear inequality * Nonlinear equality * Linear equality 4 5/5/2014
5 FSQP Algorithm A random point A point fitting all linear constraints A point fitting all constraints An optimal point fitting all constraints 5 5/5/2014
6 FSQP Algorithm A random point A point fitting all linear constraints A point fitting all constraints An optimal point fitting all constraints 6 5/5/2014
7 FSQP Algorithm A random point A point fitting all linear constraints boundary + linear inequality + linear equality A point fitting all constraints An optimal point fitting all constraints 7 5/5/2014
8 FSQP Algorithm A random point A point fitting all linear constraints boundary + linear inequality + linear equality A point fitting all constraints boundary + linear inequality + linear equality + nonlinear inequality + nonlinear equality An optimal point fitting all constraints 8 5/5/2014
9 FSQP Algorithm A random point A point fitting all linear constraints boundary + linear inequality + linear equality A point fitting all constraints boundary + linear inequality + linear equality + nonlinear inequality + nonlinear equality An optimal point fitting all constraints 9 5/5/2014
10 FSQP Algorithm Step 1. Initialization xk a point; H Hessian matrix k Step 2. A search line d k direction Step 3. Line search t k distance Step 4. Updates x k + 1 H k + 1 an update point; an update Hessian matrix 10 5/5/2014
11 Initialization x 0 H 0 = an initial point fitting all constraints = an identity matrix p 0, j k = a constant = 0 { } NE f ( x, p) = max f ( x) p h ( x) m f i j j i I j= 1 j = 1,..., n NE n 11 5/5/2014
12 Computation of a search line min d, H '(,, ) 0 kd + f xk d pk d 2 0 bl xk + d bu s. t. 0 g j ( xk ) + g j ( xk ), d 0, j = 1,..., ti 0 h ( ) ( ), 0, j 1,..., n j xk + hj xk d = e 0 a, j 1,..., te n j xk d b = + = j e η min d d, d d + γ n k k d R, γ R 1 s. t. bl xk d bu 1 f '( xk, d, pk ) γ g x + g x d γ j = n 1 j( k ) j ( k ), 1,..., i c x + d d j = t n 1 j, k j 1,..., i i h x h x d γ j n 1 j ( k ) + j ( k ), = 1,..., e a x d b j t n 1 j, k + = j = 1,..., e e ρ = + k 0 κ 0 κ k dk /( dk vk ) v = d τ 1 1 max(0.5, k ) 12 5/5/2014
13 Line search Armijo line search 0 0 dk, Hkdk ni + ne = 0 No nonlinear constraints δk = f '( xk, dk, pk ) ni + ne 0 NE f '( x, d, p) = max{ f ( x) + f ( x), d } f ( x) p h ( x), d i I f i i f I j i j= 1 n f ( x + t d, p ) f ( x, p ) +αtδ m k k k k m k k k g ( x + td ) 0, j = 1,..., n c, x + td d, j = 1,..., n h ( x + td ) 0, j = 1,..., n j k k NI j k k j LI j k k NE NE = f ( x, p) max{ f ( x)} p h ( x) m f i j j i I j= 1 n 13 5/5/2014
14 Updates x = x + + t d k 1 k k k H k + 1 p k + 1, j updated by BFGS formula with Powell s modification η = x x, γ = L( x, p ) L( x, p ) k + 1 k + 1 k k + 1 x k + 1 k x k k 1, η γ 0.2η H η T θ k + 1 = 0.8η k + 1H kη k + 1 otherwise T T η k + 1H kη k + 1 η k + 1γ k + 1 ξ = θ γ + (1 θ ) H γ H k = k + 1 k + 1 k + 1 k + 1 k + 1 k k + 1 H η η H ξ ξ T T k k + 1 k + 1 k k + 1 k + 1 k + 1 = H k + T T η k + 1H kη k + 1 η k + 1ξ k +1 p = max{ ε 1 µ j, δ pk, j} k, j k, j j 1 p T T k + 1 k + 1 k + 1 k k µ ε otherwise 14 5/5/2014
15 Implementation JFS Initial.java MiniMax.java Direction_d0.java KKT.java Direction_d1.java Arcsearch.java Check.java BFGS_Powell.java MiniMax.java Direction_d0.java KKT.java Direction_d1.java Arcsearch.java Check.java BFGS_Powell.java 15 5/5/2014
16 Implementation JFS Initial.java MiniMax.java Loop 1. Find initial feasible point Direction_d0.java KKT.java Direction_d1.java Arcsearch.java Check.java BFGS_Powell.java MiniMax.java Direction_d0.java KKT.java Direction_d1.java Arcsearch.java Check.java BFGS_Powell.java Loop 2. Find optimal point 16 5/5/2014
17 Implementation JFS Initial.java MiniMax.java Loop 1. Find initial feasible point Direction_d0.java KKT.java Direction_d1.java Arcsearch.java Check.java BFGS_Powell.java MiniMax.java Direction_d0.java KKT.java Direction_d1.java Arcsearch.java Check.java BFGS_Powell.java Loop 2. Find optimal point 17 5/5/2014
18 Implementation JFS Initial.java Find a point fitting all linear constraints MiniMax.java Find point fitting all constraints (linear + nonlinear) Direction_d0.java KKT.java Direction_d1.java Arcsearch.java Check.java BFGS_Powell.java Find the optimal point also fitting all constraints (linear + nonlinear) MiniMax.java Direction_d0.java KKT.java Direction_d1.java Arcsearch.java Check.java BFGS_Powell.java 18 5/5/2014
19 Implementation Three open-source Java libraries were leveraged: 1) JAMA (Version 1.0.3; 11/2012) 2) Apache Commons.Lang (Version 3.3.2; 04/2014) 3) Joptimizer (Version 3.3.0; 04/2014) /5/2014
20 , Validation JFS Initial.java MiniMax.java Direction_d0.java KKT.java Direction_d1.java Arcsearch.java Check.java BFGS_Powell.java 20 5/5/2014
21 , Validation JFS Initial.java MiniMax.java Direction_d0.java KKT.java Direction_d1.java Arcsearch.java Check.java BFGS_Powell.java testjfs testinitial.java test testminimax.java test testkkt.java test testbfgs_powell.java 21 5/5/2014
22 , Validation JFS Initial.java MiniMax.java Direction_d0.java KKT.java Direction_d1.java Arcsearch.java Check.java BFGS_Powell.java testjfs testinitial.java test testminimax.java test testkkt.java test testbfgs_powell.java 22 5/5/2014
23 , Validation JFS Initial.java MiniMax.java Direction_d0.java KKT.java Direction_d1.java Arcsearch.java Check.java BFGS_Powell.java testjfs testinitial.java test testminimax.java test testkkt.java test testbfgs_powell.java 23 5/5/2014
24 , Validation JFS Initial.java MiniMax.java Direction_d0.java KKT.java Direction_d1.java Arcsearch.java Check.java BFGS_Powell.java testjfs testinitial.java test testminimax.java test testkkt.java test testbfgs_powell.java 24 5/5/2014
25 , Validation JFS Initial.java MiniMax.java Direction_d0.java KKT.java Direction_d1.java Arcsearch.java Check.java BFGS_Powell.java testjfs testinitial.java test testminimax.java test testkkt.java test testbfgs_powell.java 25 5/5/2014
26 , Validation testjfs testinitial.java test testminimax.java test testkkt.java test testbfgs_powell.java 26 5/5/2014
27 , Validation testjfs QP with LI, LE consts testinitial.java test testminimax.java QP with LI, LE consts linear consts in QP test testkkt.java QP with linear consts test Same program checking with Matlab testbfgs_powell.java 27 5/5/2014
28 , Validation LP, QP, NP NI, LI, NE, LE consts QP with LI, LE consts Feasible point, NP with consts testjfs testinitial.java test testminimax.java QP with LI, LE consts linear consts in QP test testkkt.java QP with linear consts test Same program checking with Matlab testbfgs_powell.java 28 5/5/2014
29 , Example 1 min f ( x) = x + x x R x1 0 x2 0 x1 1 0 x2 1 0 Known minimizer: x x = ( 3, 2) ; f ( x ) = 5 T 0 0 = (1,0) ; f ( x ) = 1 * T * Reference: Hock, Willi, and Klaus Schittkowski. "Test examples for nonlinear programming codes." Journal of Optimization Theory and Applications 30.1 (1980): /5/2014
30 , Example 1 min f ( x) = x + x x R x1 0 x2 0 x1 1 0 x2 1 0 Known minimizer: x x = ( 3, 2) ; f ( x ) = 5 T 0 0 = (1,0) ; f ( x ) = 1 * T * Reference: Hock, Willi, and Klaus Schittkowski. "Test examples for nonlinear programming codes." Journal of Optimization Theory and Applications 30.1 (1980): /5/2014
31 , Example 2 min f ( x) = ( x + 3 x + x ) + 4( x x ) x R x1, x2, x3 0 3 x1 6x2 4x x1 x2 x3 = 0 Known global minimizer: x = (0, 0,1) ; f ( x ) = 1 * T * Reference: Hock, Willi, and Klaus Schittkowski. "Test examples for nonlinear programming codes." Journal of Optimization Theory and Applications 30.1 (1980): /5/2014
32 , Example /5/2014
33 Example /5/2014
34 Project Schedule October Literature review; Specify the implementation module details; Structure the implementation; November Develop the quadratic programming module; Unconstrained quadratic program; Strictly convex quadratic program; Validate the quadratic programming module; December Develop the Gradient and Hessian matrix calculation module; Validate the Gradient and Hessian matrix calculation module; Midterm project report and presentation; 34 5/5/2014
35 January Develop Armijo line search module; Validate Armijo line search module; Develop the feasible initial point module; February Validate the feasible initial point module; Integrate the program; March Debug and document the program; Validate and test the program with case application; April d ɶ Add arch search variable ; Compare calculation efficiency of line search with arch search methods; May Develop the user interface if time available; Final project report and presentation; 35 5/5/2014
36 , Deliverables Project proposal Algorithm description Well-documented Java codes Test cases: LP, QP, NP with NI, LI, NE, LE constraints Validation results Presentations Project reports 36 5/5/2014
37 Bibliography 37 5/5/2014
Numerical Methods. V. Leclère May 15, x R n
Numerical Methods V. Leclère May 15, 2018 1 Some optimization algorithms Consider the unconstrained optimization problem min f(x). (1) x R n A descent direction algorithm is an algorithm that construct
5 Handling Constraints
5 Handling Constraints Engineering design optimization problems are very rarely unconstrained. Moreover, the constraints that appear in these problems are typically nonlinear. This motivates our interest
Algorithms for constrained local optimization
Algorithms for constrained local optimization Fabio Schoen 2008 http://gol.dsi.unifi.it/users/schoen Algorithms for constrained local optimization p. Feasible direction methods Algorithms for constrained
4F3 - Predictive Control
4F3 Predictive Control - Lecture 3 p 1/21 4F3 - Predictive Control Lecture 3 - Predictive Control with Constraints Jan Maciejowski jmm@engcamacuk 4F3 Predictive Control - Lecture 3 p 2/21 Constraints on
An Active Set Strategy for Solving Optimization Problems with up to 200,000,000 Nonlinear Constraints
An Active Set Strategy for Solving Optimization Problems with up to 200,000,000 Nonlinear Constraints Klaus Schittkowski Department of Computer Science, University of Bayreuth 95440 Bayreuth, Germany e-mail:
Algorithms for Constrained Optimization
1 / 42 Algorithms for Constrained Optimization ME598/494 Lecture Max Yi Ren Department of Mechanical Engineering, Arizona State University April 19, 2015 2 / 42 Outline 1. Convergence 2. Sequential quadratic
E5295/5B5749 Convex optimization with engineering applications. Lecture 8. Smooth convex unconstrained and equality-constrained minimization
E5295/5B5749 Convex optimization with engineering applications Lecture 8 Smooth convex unconstrained and equality-constrained minimization A. Forsgren, KTH 1 Lecture 8 Convex optimization 2006/2007 Unconstrained
A Constraint-Reduced MPC Algorithm for Convex Quadratic Programming, with a Modified Active-Set Identification Scheme
A Constraint-Reduced MPC Algorithm for Convex Quadratic Programming, with a Modified Active-Set Identification Scheme M. Paul Laiu 1 and (presenter) André L. Tits 2 1 Oak Ridge National Laboratory laiump@ornl.gov
6.252 NONLINEAR PROGRAMMING LECTURE 10 ALTERNATIVES TO GRADIENT PROJECTION LECTURE OUTLINE. Three Alternatives/Remedies for Gradient Projection
6.252 NONLINEAR PROGRAMMING LECTURE 10 ALTERNATIVES TO GRADIENT PROJECTION LECTURE OUTLINE Three Alternatives/Remedies for Gradient Projection Two-Metric Projection Methods Manifold Suboptimization Methods
A feasible descent SQP algorithm for general constrained optimization without strict complementarity
Journal of Computational and Applied Mathematics 180 (2005) 391 412 www.elsevier.com/locate/cam A feasible descent SQP algorithm for general constrained optimization without strict complementarity Jin-Bao
Solving l 1 Regularized Least Square Problems with Hierarchical Decomposition
Solving l 1 Least Square s with 1 mzhong1@umd.edu 1 AMSC and CSCAMM University of Maryland College Park Project for AMSC 663 October 2 nd, 2012 Outline 1 The 2 Outline 1 The 2 Compressed Sensing Example
Zero-Order Methods for the Optimization of Noisy Functions. Jorge Nocedal
Zero-Order Methods for the Optimization of Noisy Functions Jorge Nocedal Northwestern University Simons Institute, October 2017 1 Collaborators Albert Berahas Northwestern University Richard Byrd University
Numerical Optimization. Review: Unconstrained Optimization
Numerical Optimization Finding the best feasible solution Edward P. Gatzke Department of Chemical Engineering University of South Carolina Ed Gatzke (USC CHE ) Numerical Optimization ECHE 589, Spring 2011
OPER 627: Nonlinear Optimization Lecture 14: Mid-term Review
OPER 627: Nonlinear Optimization Lecture 14: Mid-term Review Department of Statistical Sciences and Operations Research Virginia Commonwealth University Oct 16, 2013 (Lecture 14) Nonlinear Optimization
Line Search Methods for Unconstrained Optimisation
Line Search Methods for Unconstrained Optimisation Lecture 8, Numerical Linear Algebra and Optimisation Oxford University Computing Laboratory, MT 2007 Dr Raphael Hauser (hauser@comlab.ox.ac.uk) The Generic
School Year Calendar,
School Year Calendar, 0- August 0 0 First day for Start of st quarter September 0 Labor Day 0 ; schools closed for st -quarter progress reports 0 st -quarter progress reports Updated as of May 0; see www.baltimorecityschools.org
An Inexact Newton Method for Nonlinear Constrained Optimization
An Inexact Newton Method for Nonlinear Constrained Optimization Frank E. Curtis Numerical Analysis Seminar, January 23, 2009 Outline Motivation and background Algorithm development and theoretical results
Some new facts about sequential quadratic programming methods employing second derivatives
To appear in Optimization Methods and Software Vol. 00, No. 00, Month 20XX, 1 24 Some new facts about sequential quadratic programming methods employing second derivatives A.F. Izmailov a and M.V. Solodov
MPC Infeasibility Handling
MPC Handling Thomas Wiese, TU Munich, KU Leuven supervised by H.J. Ferreau, Prof. M. Diehl (both KUL) and Dr. H. Gräb (TUM) October 9, 2008 1 / 42 MPC General MPC Strategies 2 / 42 Linear Discrete-Time
Sub-Sampled Newton Methods
Sub-Sampled Newton Methods F. Roosta-Khorasani and M. W. Mahoney ICSI and Dept of Statistics, UC Berkeley February 2016 F. Roosta-Khorasani and M. W. Mahoney (UCB) Sub-Sampled Newton Methods Feb 2016 1
AM 205: lecture 19. Last time: Conditions for optimality, Newton s method for optimization Today: survey of optimization methods
AM 205: lecture 19 Last time: Conditions for optimality, Newton s method for optimization Today: survey of optimization methods Quasi-Newton Methods General form of quasi-newton methods: x k+1 = x k α
An Inexact Newton Method for Optimization
New York University Brown Applied Mathematics Seminar, February 10, 2009 Brief biography New York State College of William and Mary (B.S.) Northwestern University (M.S. & Ph.D.) Courant Institute (Postdoc)
PDE-Constrained and Nonsmooth Optimization
Frank E. Curtis October 1, 2009 Outline PDE-Constrained Optimization Introduction Newton s method Inexactness Results Summary and future work Nonsmooth Optimization Sequential quadratic programming (SQP)
Stochastic Optimization with Inequality Constraints Using Simultaneous Perturbations and Penalty Functions
International Journal of Control Vol. 00, No. 00, January 2007, 1 10 Stochastic Optimization with Inequality Constraints Using Simultaneous Perturbations and Penalty Functions I-JENG WANG and JAMES C.
5.5 Quadratic programming
5.5 Quadratic programming Minimize a quadratic function subject to linear constraints: 1 min x t Qx + c t x 2 s.t. a t i x b i i I (P a t i x = b i i E x R n, where Q is an n n matrix, I and E are the
Global convergence of a regularized factorized quasi-newton method for nonlinear least squares problems
Volume 29, N. 2, pp. 195 214, 2010 Copyright 2010 SBMAC ISSN 0101-8205 www.scielo.br/cam Global convergence of a regularized factorized quasi-newton method for nonlinear least squares problems WEIJUN ZHOU
CE 191: Civil & Environmental Engineering Systems Analysis. LEC 17 : Final Review
CE 191: Civil & Environmental Engineering Systems Analysis LEC 17 : Final Review Professor Scott Moura Civil & Environmental Engineering University of California, Berkeley Fall 2014 Prof. Moura UC Berkeley
2.3 Linear Programming
2.3 Linear Programming Linear Programming (LP) is the term used to define a wide range of optimization problems in which the objective function is linear in the unknown variables and the constraints are
Constrained Optimization
1 / 22 Constrained Optimization ME598/494 Lecture Max Yi Ren Department of Mechanical Engineering, Arizona State University March 30, 2015 2 / 22 1. Equality constraints only 1.1 Reduced gradient 1.2 Lagrange
Lecture 18: Optimization Programming
Fall, 2016 Outline Unconstrained Optimization 1 Unconstrained Optimization 2 Equality-constrained Optimization Inequality-constrained Optimization Mixture-constrained Optimization 3 Quadratic Programming
Nonlinear Optimization: What s important?
Nonlinear Optimization: What s important? Julian Hall 10th May 2012 Convexity: convex problems A local minimizer is a global minimizer A solution of f (x) = 0 (stationary point) is a minimizer A global
On Nesterov s Random Coordinate Descent Algorithms - Continued
On Nesterov s Random Coordinate Descent Algorithms - Continued Zheng Xu University of Texas At Arlington February 20, 2015 1 Revisit Random Coordinate Descent The Random Coordinate Descent Upper and Lower
A Robust Implementation of a Sequential Quadratic Programming Algorithm with Successive Error Restoration
A Robust Implementation of a Sequential Quadratic Programming Algorithm with Successive Error Restoration Address: Prof. K. Schittkowski Department of Computer Science University of Bayreuth D - 95440
A COMPARATIVE STUDY ON OPTIMIZATION METHODS FOR THE CONSTRAINED NONLINEAR PROGRAMMING PROBLEMS
A COMPARATIVE STUDY ON OPTIMIZATION METHODS FOR THE CONSTRAINED NONLINEAR PROGRAMMING PROBLEMS OZGUR YENIAY Received 2 August 2004 and in revised form 12 November 2004 Constrained nonlinear programming
Simulating Solid Tumor Growth Using Multigrid Algorithms
Simulating Solid Tumor Growth Using Multigrid Algorithms Asia Wyatt Applied Mathematics, Statistics, and Scientific Computation Program Advisor: Doron Levy Department of Mathematics/CSCAMM Abstract In
On the convergence properties of the modified Polak Ribiére Polyak method with the standard Armijo line search
ANZIAM J. 55 (E) pp.e79 E89, 2014 E79 On the convergence properties of the modified Polak Ribiére Polyak method with the standard Armijo line search Lijun Li 1 Weijun Zhou 2 (Received 21 May 2013; revised
Optimization Problems with Constraints - introduction to theory, numerical Methods and applications
Optimization Problems with Constraints - introduction to theory, numerical Methods and applications Dr. Abebe Geletu Ilmenau University of Technology Department of Simulation and Optimal Processes (SOP)
Penalty and Barrier Methods General classical constrained minimization problem minimize f(x) subject to g(x) 0 h(x) =0 Penalty methods are motivated by the desire to use unconstrained optimization techniques
Quasi-Newton Methods
Quasi-Newton Methods Werner C. Rheinboldt These are excerpts of material relating to the boos [OR00 and [Rhe98 and of write-ups prepared for courses held at the University of Pittsburgh. Some further references
Convex Optimization. Prof. Nati Srebro. Lecture 12: Infeasible-Start Newton s Method Interior Point Methods
Convex Optimization Prof. Nati Srebro Lecture 12: Infeasible-Start Newton s Method Interior Point Methods Equality Constrained Optimization f 0 (x) s. t. A R p n, b R p Using access to: 2 nd order oracle
MS&E 318 (CME 338) Large-Scale Numerical Optimization
Stanford University, Management Science & Engineering (and ICME) MS&E 318 (CME 338) Large-Scale Numerical Optimization 1 Origins Instructor: Michael Saunders Spring 2015 Notes 9: Augmented Lagrangian Methods
Infeasibility Detection and an Inexact Active-Set Method for Large-Scale Nonlinear Optimization
Infeasibility Detection and an Inexact Active-Set Method for Large-Scale Nonlinear Optimization Frank E. Curtis, Lehigh University involving joint work with James V. Burke, University of Washington Daniel
IE 5531: Engineering Optimization I
IE 5531: Engineering Optimization I Lecture 19: Midterm 2 Review Prof. John Gunnar Carlsson November 22, 2010 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I November 22, 2010 1 / 34 Administrivia
5 Quasi-Newton Methods
Unconstrained Convex Optimization 26 5 Quasi-Newton Methods If the Hessian is unavailable... Notation: H = Hessian matrix. B is the approximation of H. C is the approximation of H 1. Problem: Solve min
A Regularized Interior-Point Method for Constrained Nonlinear Least Squares
A Regularized Interior-Point Method for Constrained Nonlinear Least Squares XII Brazilian Workshop on Continuous Optimization Abel Soares Siqueira Federal University of Paraná - Curitiba/PR - Brazil Dominique
Optimization. Yuh-Jye Lee. March 21, Data Science and Machine Intelligence Lab National Chiao Tung University 1 / 29
Optimization Yuh-Jye Lee Data Science and Machine Intelligence Lab National Chiao Tung University March 21, 2017 1 / 29 You Have Learned (Unconstrained) Optimization in Your High School Let f (x) = ax
Computational Optimization. Augmented Lagrangian NW 17.3
Computational Optimization Augmented Lagrangian NW 17.3 Upcoming Schedule No class April 18 Friday, April 25, in class presentations. Projects due unless you present April 25 (free extension until Monday
Stochastic Optimization Algorithms Beyond SG
Stochastic Optimization Algorithms Beyond SG Frank E. Curtis 1, Lehigh University involving joint work with Léon Bottou, Facebook AI Research Jorge Nocedal, Northwestern University Optimization Methods
2 Chapter 1 rely on approximating (x) by using progressively ner discretizations of [0; 1] (see, e.g. [5, 7, 8, 16, 18, 19, 20, 23]). Specically, such
![2 Chapter 1 rely on approximating (x) by using progressively ner discretizations of [0; 1] (see, e.g. [5, 7, 8, 16, 18, 19, 20, 23]). Specically, such](/thumbs/88/114801350.jpg "2 Chapter 1 rely on approximating (x) by using progressively ner discretizations of [0; 1] (see, e.g. [5, 7, 8, 16, 18, 19, 20, 23]). Specically, such")
1 FEASIBLE SEQUENTIAL QUADRATIC PROGRAMMING FOR FINELY DISCRETIZED PROBLEMS FROM SIP Craig T. Lawrence and Andre L. Tits ABSTRACT Department of Electrical Engineering and Institute for Systems Research
Modeling Multiphase Flow in Porous Media with Complementary Constraints
Background Modeling Multiphase Flow in Porous Media with Complementary Constraints Applied Math, Statistics, and Scientific Computation, University of Maryland - College Park October 07, 2014 Advisors
A GLOBALLY CONVERGENT STABILIZED SQP METHOD
A GLOBALLY CONVERGENT STABILIZED SQP METHOD Philip E. Gill Daniel P. Robinson July 6, 2013 Abstract Sequential quadratic programming SQP methods are a popular class of methods for nonlinearly constrained
Primal-Dual Interior-Point Methods for Linear Programming based on Newton s Method
Primal-Dual Interior-Point Methods for Linear Programming based on Newton s Method Robert M. Freund March, 2004 2004 Massachusetts Institute of Technology. The Problem The logarithmic barrier approach
Lecture 13: Constrained optimization
2010-12-03 Basic ideas A nonlinearly constrained problem must somehow be converted relaxed into a problem which we can solve (a linear/quadratic or unconstrained problem) We solve a sequence of such problems
CSCI 1951-G Optimization Methods in Finance Part 09: Interior Point Methods
CSCI 1951-G Optimization Methods in Finance Part 09: Interior Point Methods March 23, 2018 1 / 35 This material is covered in S. Boyd, L. Vandenberge s book Convex Optimization https://web.stanford.edu/~boyd/cvxbook/.
Scientific Computing: Optimization
Scientific Computing: Optimization Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 Course MATH-GA.2043 or CSCI-GA.2112, Spring 2012 March 8th, 2011 A. Donev (Courant Institute) Lecture
Unconstrained optimization
Chapter 4 Unconstrained optimization An unconstrained optimization problem takes the form min x Rnf(x) (4.1) for a target functional (also called objective function) f : R n R. In this chapter and throughout
Programming, numerics and optimization
Programming, numerics and optimization Lecture C-3: Unconstrained optimization II Łukasz Jankowski ljank@ippt.pan.pl Institute of Fundamental Technological Research Room 4.32, Phone +22.8261281 ext. 428
Non-negative Matrix Factorization via accelerated Projected Gradient Descent
Non-negative Matrix Factorization via accelerated Projected Gradient Descent Andersen Ang Mathématique et recherche opérationnelle UMONS, Belgium Email: manshun.ang@umons.ac.be Homepage: angms.science
Numerical Optimization of Partial Differential Equations
Numerical Optimization of Partial Differential Equations Part I: basic optimization concepts in R n Bartosz Protas Department of Mathematics & Statistics McMaster University, Hamilton, Ontario, Canada
Qing-Juan Xu & Jin-Bao Jian
A nonlinear norm-relaxed method for finely discretized semi-infinite optimization problems Qing-Juan Xu & Jin-Bao Jian Nonlinear Dynamics An International Journal of Nonlinear Dynamics and Chaos in Engineering
Numerical Optimization: Basic Concepts and Algorithms
May 27th 2015 Numerical Optimization: Basic Concepts and Algorithms R. Duvigneau R. Duvigneau - Numerical Optimization: Basic Concepts and Algorithms 1 Outline Some basic concepts in optimization Some
Reduced-Hessian Methods for Constrained Optimization
Reduced-Hessian Methods for Constrained Optimization Philip E. Gill University of California, San Diego Joint work with: Michael Ferry & Elizabeth Wong 11th US & Mexico Workshop on Optimization and its
Nonlinear Model Predictive Control Tools (NMPC Tools)
Nonlinear Model Predictive Control Tools (NMPC Tools) Rishi Amrit, James B. Rawlings April 5, 2008 1 Formulation We consider a control system composed of three parts([2]). Estimator Target calculator Regulator
Support Vector Machines: Maximum Margin Classifiers
Support Vector Machines: Maximum Margin Classifiers Machine Learning and Pattern Recognition: September 16, 2008 Piotr Mirowski Based on slides by Sumit Chopra and Fu-Jie Huang 1 Outline What is behind
Global Convergence of Perry-Shanno Memoryless Quasi-Newton-type Method. 1 Introduction
ISSN 1749-3889 (print), 1749-3897 (online) International Journal of Nonlinear Science Vol.11(2011) No.2,pp.153-158 Global Convergence of Perry-Shanno Memoryless Quasi-Newton-type Method Yigui Ou, Jun Zhang
AM 205: lecture 19. Last time: Conditions for optimality Today: Newton s method for optimization, survey of optimization methods
AM 205: lecture 19 Last time: Conditions for optimality Today: Newton s method for optimization, survey of optimization methods Optimality Conditions: Equality Constrained Case As another example of equality
An Alternative Three-Term Conjugate Gradient Algorithm for Systems of Nonlinear Equations
International Journal of Mathematical Modelling & Computations Vol. 07, No. 02, Spring 2017, 145-157 An Alternative Three-Term Conjugate Gradient Algorithm for Systems of Nonlinear Equations L. Muhammad
Trust-region methods for rectangular systems of nonlinear equations
Trust-region methods for rectangular systems of nonlinear equations Margherita Porcelli Dipartimento di Matematica U.Dini Università degli Studi di Firenze Joint work with Maria Macconi and Benedetta Morini
Numerical Optimization Professor Horst Cerjak, Horst Bischof, Thomas Pock Mat Vis-Gra SS09
Numerical Optimization 1 Working Horse in Computer Vision Variational Methods Shape Analysis Machine Learning Markov Random Fields Geometry Common denominator: optimization problems 2 Overview of Methods
4M020 Design tools. Algorithms for numerical optimization. L.F.P. Etman. Department of Mechanical Engineering Eindhoven University of Technology
4M020 Design tools Algorithms for numerical optimization L.F.P. Etman Department of Mechanical Engineering Eindhoven University of Technology Wednesday September 3, 2008 1 / 32 Outline 1 Problem formulation:
A New Penalty-SQP Method
Background and Motivation Illustration of Numerical Results Final Remarks Frank E. Curtis Informs Annual Meeting, October 2008 Background and Motivation Illustration of Numerical Results Final Remarks
Optimization. Yuh-Jye Lee. March 28, Data Science and Machine Intelligence Lab National Chiao Tung University 1 / 40
Optimization Yuh-Jye Lee Data Science and Machine Intelligence Lab National Chiao Tung University March 28, 2017 1 / 40 The Key Idea of Newton s Method Let f : R n R be a twice differentiable function
Sequential Convex Programming
Sequential Convex Programming sequential convex programming alternating convex optimization convex-concave procedure Prof. S. Boyd, EE364b, Stanford University Methods for nonconvex optimization problems
minimize x subject to (x 2)(x 4) u,
Math 6366/6367: Optimization and Variational Methods Sample Preliminary Exam Questions 1. Suppose that f : [, L] R is a C 2 -function with f () on (, L) and that you have explicit formulae for
CE 191: Civil and Environmental Engineering Systems Analysis. LEC 05 : Optimality Conditions
CE 191: Civil and Environmental Engineering Systems Analysis LEC : Optimality Conditions Professor Scott Moura Civil & Environmental Engineering University of California, Berkeley Fall 214 Prof. Moura
Unconstrained Optimization
1 / 36 Unconstrained Optimization ME598/494 Lecture Max Yi Ren Department of Mechanical Engineering, Arizona State University February 2, 2015 2 / 36 3 / 36 4 / 36 5 / 36 1. preliminaries 1.1 local approximation
MA/OR/ST 706: Nonlinear Programming Midterm Exam Instructor: Dr. Kartik Sivaramakrishnan INSTRUCTIONS
MA/OR/ST 706: Nonlinear Programming Midterm Exam Instructor: Dr. Kartik Sivaramakrishnan INSTRUCTIONS 1. Please write your name and student number clearly on the front page of the exam. 2. The exam is
CHAPTER 2: QUADRATIC PROGRAMMING
CHAPTER 2: QUADRATIC PROGRAMMING Overview Quadratic programming (QP) problems are characterized by objective functions that are quadratic in the design variables, and linear constraints. In this sense,
Nonlinear Programming
Nonlinear Programming Kees Roos e-mail: C.Roos@ewi.tudelft.nl URL: http://www.isa.ewi.tudelft.nl/ roos LNMB Course De Uithof, Utrecht February 6 - May 8, A.D. 2006 Optimization Group 1 Outline for week
Lecture 15 Newton Method and Self-Concordance. October 23, 2008
Newton Method and Self-Concordance October 23, 2008 Outline Lecture 15 Self-concordance Notion Self-concordant Functions Operations Preserving Self-concordance Properties of Self-concordant Functions Implications
Interior Point Algorithms for Constrained Convex Optimization
Interior Point Algorithms for Constrained Convex Optimization Chee Wei Tan CS 8292 : Advanced Topics in Convex Optimization and its Applications Fall 2010 Outline Inequality constrained minimization problems
Constrained Optimization and Distributed Computation Based Car-Following Control of A Connected and Autonomous Vehicle Platoon
Constrained Optimization and Distributed Computation Based Car-Following Control of A Connected and Autonomous Vehicle Platoon Siyuan Gong a, Jinglai Shen b, Lili Du a ldu3@iit.edu a: Illinois Institute
Numerical Methods I Solving Nonlinear Equations
Numerical Methods I Solving Nonlinear Equations Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 MATH-GA 2011.003 / CSCI-GA 2945.003, Fall 2014 October 16th, 2014 A. Donev (Courant Institute)
An interior point type QP-free algorithm with superlinear convergence for inequality constrained optimization
Applied Mathematical Modelling 31 (2007) 1201 1212 www.elsevier.com/locate/apm An interior point type QP-free algorithm with superlinear convergence for inequality constrained optimization Zhibin Zhu *
Inexact Newton Methods and Nonlinear Constrained Optimization
Inexact Newton Methods and Nonlinear Constrained Optimization Frank E. Curtis EPSRC Symposium Capstone Conference Warwick Mathematics Institute July 2, 2009 Outline PDE-Constrained Optimization Newton
Convex Optimization. Problem set 2. Due Monday April 26th
Convex Optimization Problem set 2 Due Monday April 26th 1 Gradient Decent without Line-search In this problem we will consider gradient descent with predetermined step sizes. That is, instead of determining
Constrained optimization: direct methods (cont.)
Constrained optimization: direct methods (cont.) Jussi Hakanen Post-doctoral researcher jussi.hakanen@jyu.fi Direct methods Also known as methods of feasible directions Idea in a point x h, generate a
Minimax Design of Complex-Coefficient FIR Filters with Low Group Delay
Minimax Design of Complex-Coefficient FIR Filters with Low Group Delay Wu-Sheng Lu Takao Hinamoto Dept. of Elec. and Comp. Engineering Graduate School of Engineering University of Victoria Hiroshima University
A derivative-free nonmonotone line search and its application to the spectral residual method
IMA Journal of Numerical Analysis (2009) 29, 814 825 doi:10.1093/imanum/drn019 Advance Access publication on November 14, 2008 A derivative-free nonmonotone line search and its application to the spectral
Optimization. Next: Curve Fitting Up: Numerical Analysis for Chemical Previous: Linear Algebraic and Equations. Subsections
Next: Curve Fitting Up: Numerical Analysis for Chemical Previous: Linear Algebraic and Equations Subsections One-dimensional Unconstrained Optimization Golden-Section Search Quadratic Interpolation Newton's
Convergence of a Two-parameter Family of Conjugate Gradient Methods with a Fixed Formula of Stepsize
Bol. Soc. Paran. Mat. (3s.) v. 00 0 (0000):????. c SPM ISSN-2175-1188 on line ISSN-00378712 in press SPM: www.spm.uem.br/bspm doi:10.5269/bspm.v38i6.35641 Convergence of a Two-parameter Family of Conjugate
Chapter 4. Unconstrained optimization
Chapter 4. Unconstrained optimization Version: 28-10-2012 Material: (for details see) Chapter 11 in [FKS] (pp.251-276) A reference e.g. L.11.2 refers to the corresponding Lemma in the book [FKS] PDF-file
The Alternating Direction Method of Multipliers
The Alternating Direction Method of Multipliers With Adaptive Step Size Selection Peter Sutor, Jr. Project Advisor: Professor Tom Goldstein December 2, 2015 1 / 25 Background The Dual Problem Consider
Static unconstrained optimization
Static unconstrained optimization 2 In unconstrained optimization an objective function is minimized without any additional restriction on the decision variables, i.e. min f(x) x X ad (2.) with X ad R
Lecture V. Numerical Optimization
Lecture V Numerical Optimization Gianluca Violante New York University Quantitative Macroeconomics G. Violante, Numerical Optimization p. 1 /19 Isomorphism I We describe minimization problems: to maximize
1. Introduction. We analyze a trust region version of Newton s method for the optimization problem
SIAM J. OPTIM. Vol. 9, No. 4, pp. 1100 1127 c 1999 Society for Industrial and Applied Mathematics NEWTON S METHOD FOR LARGE BOUND-CONSTRAINED OPTIMIZATION PROBLEMS CHIH-JEN LIN AND JORGE J. MORÉ To John
Interior-Point and Augmented Lagrangian Algorithms for Optimization and Control
Interior-Point and Augmented Lagrangian Algorithms for Optimization and Control Stephen Wright University of Wisconsin-Madison May 2014 Wright (UW-Madison) Constrained Optimization May 2014 1 / 46 In This
Cubic regularization of Newton s method for convex problems with constraints
CORE DISCUSSION PAPER 006/39 Cubic regularization of Newton s method for convex problems with constraints Yu. Nesterov March 31, 006 Abstract In this paper we derive efficiency estimates of the regularized
IE 5531: Engineering Optimization I
IE 5531: Engineering Optimization I Lecture 15: Nonlinear optimization Prof. John Gunnar Carlsson November 1, 2010 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I November 1, 2010 1 / 24
CHARACTERIZATION OF NONLINEAR NEURON RESPONSES
CHARACTERIZATION OF NONLINEAR NEURON RESPONSES Matt Whiteway whit8022@umd.edu Dr. Daniel A. Butts dab@umd.edu Neuroscience and Cognitive Science (NACS) Applied Mathematics and Scientific Computation (AMSC)