MPC Infeasibility Handling
|
|
- Verity Daniel
- 5 years ago
- Views:
Transcription
1 MPC Handling Thomas Wiese, TU Munich, KU Leuven supervised by H.J. Ferreau, Prof. M. Diehl (both KUL) and Dr. H. Gräb (TUM) October 9, / 42
2 MPC General MPC Strategies 2 / 42
3 Linear Discrete-Time Dynamic Systems - Simple Car Model - Quarter Car Model Linear Model Predictive Control Receding Horizon - Receding Horizon MPC General 3 / 42
4 Linear Discrete-Time Dynamic Systems Linear Discrete-Time Dynamic Systems - Simple Car Model - Quarter Car Model Linear Model Predictive Control Receding Horizon - Receding Horizon We consider the following linear discrete-time dynamic system which shall be steered to a setpoint: x k+1 = Ax k + Bu k x and u are referred to as states and inputs of the system. example car and quarter car MPC General 4 / 42
5 - Simple Car Model Linear Discrete-Time Dynamic Systems - Simple Car Model - Quarter Car Model Linear Model Predictive Control Receding Horizon - Receding Horizon (ẋ ) ẍ = ( ) ( ) 0 1 x 0 0 ẋ + ( ) 0 u 1 MPC General 5 / 42
6 - Quarter Car Model Linear Discrete-Time Dynamic Systems - Simple Car Model - Quarter Car Model Linear Model Predictive Control Receding Horizon - Receding Horizon MPC General 6 / 42
7 Linear Model Predictive Control Linear Discrete-Time Dynamic Systems - Simple Car Model - Quarter Car Model Linear Model Predictive Control Receding Horizon - Receding Horizon State-space approach Prediction of system behavior Quadratic cost function Hard input and output constraints Results in a nonlinear control law MPC General 7 / 42
8 Linear Discrete-Time Dynamic Systems - Simple Car Model - Quarter Car Model Linear Model Predictive Control Receding Horizon - Receding Horizon MPC General The linear MPC can be formulated as follows: minimize a cost function Φ(u 0,u 1,...) = k=0,1,... such that the constraints x k+1 x ref,k+1 2 W x + u k 2 W u G u u k b u G x x k+1 b x are satisfied for all future k and the x k are determined by the system dynamics and where x 0 is known. 8 / 42
9 Receding Horizon Linear Discrete-Time Dynamic Systems - Simple Car Model - Quarter Car Model Linear Model Predictive Control Receding Horizon - Receding Horizon MPC General Infinitely many optimization variables Solution Cut off the sum at term N (N = horizon ) Apply only the first control Repeat at every time step The solution can be found by solving a Quadratic Program (QP) and must be available within the sampling time. 9 / 42
10 - Receding Horizon Linear Discrete-Time Dynamic Systems - Simple Car Model - Quarter Car Model Linear Model Predictive Control Receding Horizon - Receding Horizon MPC General 10 / 42
11 MPC - Short Horizon - Car Model - Measurement noise General MPC 11 / 42
12 MPC - Short Horizon - Car Model - Measurement noise General An MPC problem is called infeasible at time step t (t = 0 in the following) if the constraints cannot be satisfied. How can this happen? The problem is ill-posed The states are subject to measurement errors The prediction or control horizon is too short The predictions are not correct (model/plant mismatch) 12 / 42
13 - Short Horizon - Car Model MPC - Short Horizon - Car Model - Measurement noise General 13 / 42
14 - Measurement noise MPC - Short Horizon - Car Model - Measurement noise General 14 / 42
15 MPC of MPC General 15 / 42
16 of MPC MPC By elminating the states, the MPC can be expressed as a parametric Quadratic Program (QP): of MPC General subject to 1 min x 2 xt Qx + gx T 0 x Gx b x0 where the gradient g x0 and b x0 depend linearly on the parameter x / 42
17 MPC General Relaxation Exactness Guaranteeing Exactness 1/2 Guaranteeing Exactness 2/2 Many Objectives General 17 / 42
18 Relaxation MPC General Relaxation Exactness Guaranteeing Exactness 1/2 Guaranteeing Exactness 2/2 Many Objectives Reformulate MPC such that it is guaranteed to be feasible: min Φ(u 0,...,u N ) = N x k+1 x ref,k+1 2 W x + u k 2 W u k=0 such that the relaxed constraints G u u k ub u + ε G x x k+1 ub x + ε are satisfied for all k {0,...,N} and the x k are determined by the system dynamics and where x 0 is known. 18 / 42
19 Exactness MPC General Relaxation Exactness Guaranteeing Exactness 1/2 Guaranteeing Exactness 2/2 Many Objectives Strength of MPC: Hard Constraints. To preserve hard constraints the infeasibility handling strategy must be exact. A strategy is called exact if it only relaxes the constraints when the problem is infeasible. 19 / 42
20 Guaranteeing Exactness 1/2 MPC Embed MPC into a bigger optimization problem: First check for feasibility (e.g. by solving a Linear Program) General Relaxation Exactness Guaranteeing Exactness 1/2 Guaranteeing Exactness 2/2 Many Objectives If feasible: solve original problem Else: solve relaxed problem = 2-step algorithm 20 / 42
21 Guaranteeing Exactness 2/2 MPC General Relaxation Exactness Guaranteeing Exactness 1/2 Guaranteeing Exactness 2/2 Many Objectives Relax original MPC problem: Formulate the MPC problem with soft constraints Introduce a big linear penalty for slack variables that guarantees exactness (Exact Soft Constraints) : Finding lower bound for the linear penalty 21 / 42
22 Many Objectives MPC General Relaxation Exactness Guaranteeing Exactness 1/2 Guaranteeing Exactness 2/2 Many Objectives How to return to feasibility? Regain feasibility as quick as possible Minimize total anticipated duration of constraint violations Minimize the maximal amplitude of constraint violations Minimize quadratic sum of constraint violations etc. 22 / 42
23 MPC General l Penalty l Penalty - l Penalty - Minimum Time l 1 Penalty l 1 Penalty - l 1 Penalty - Other Approaches 23 / 42
24 l Penalty One slack variable ε is introduced and the constraints are relaxed by ε. MPC General l Penalty l Penalty - l Penalty - Minimum Time l 1 Penalty l 1 Penalty - l 1 Penalty - Other Approaches Only one additional optimization variable Minimizes the maximum violation : Can result in large open-/closed-loop mismatch 24 / 42
25 l Penalty - MPC General l Penalty l Penalty - l Penalty - Minimum Time l 1 Penalty l 1 Penalty - l 1 Penalty - Other Approaches Before solving the original MPC problem, the following LP must be solved: subject to min ε x 0 = x 0 ε x k+1 = Ax k + Bu k G u u k b u + ε G x x k+1 b x + ε ε 0 k {0,...,N} 25 / 42
26 l Penalty - MPC General l Penalty l Penalty - l Penalty - Minimum Time l 1 Penalty l 1 Penalty - l 1 Penalty - Other Approaches 26 / 42
27 Minimum Time The minimal time τ min after which the problem is feasible again can be computed (solution of a QP and several LPs). MPC General l Penalty l Penalty - l Penalty - Minimum Time l 1 Penalty l 1 Penalty - l 1 Penalty - Other Approaches only relax constraints before τ min Time-optimal relaxation : Can result in large violations at the start High additional computational cost 27 / 42
28 MPC General l Penalty l Penalty - l Penalty - Minimum Time l 1 Penalty l 1 Penalty - l 1 Penalty - Other Approaches Before solving the original MPC problem, τ min must be computed (QP + several LPs). Then the following LP has to be solved: subject to x 0 = x 0 x k+1 = Ax k + Bu k G u u k b u + ε k G x x k+1 b x + ε k ε k 0 G u u k b u G x x k+1 b x min ε s T ε, s 0 k = 0,...,N k = 0,...,τ min k = 0,...,τ min k = 0,...,τ min k = τ min + 1,...,N k = τ min + 1,...,N 28 / 42
29 MPC General l Penalty l Penalty - l Penalty - Minimum Time l 1 Penalty l 1 Penalty - l 1 Penalty - Other Approaches 29 / 42
30 l 1 Penalty MPC Introduce independent slack variables ε k for each constraint. Minimizes the total sum of constraint violations General l Penalty l Penalty - l Penalty - Minimum Time l 1 Penalty l 1 Penalty - l 1 Penalty - Other Approaches : High additional computational cost 30 / 42
31 l 1 Penalty - MPC General l Penalty l Penalty - l Penalty - Minimum Time l 1 Penalty l 1 Penalty - l 1 Penalty - Other Approaches Before solving the original MPC problem, the following LP must be solved: subject to min ε s T ε, s 0 x 0 = x 0 x k+1 = Ax k + Bu k G u u k b u + ε k G x x k+1 b x + ε k ε k 0 k {0,...,N} 31 / 42
32 l 1 Penalty - MPC General l Penalty l Penalty - l Penalty - Minimum Time l 1 Penalty l 1 Penalty - l1 Penalty - Other Approaches 32 / 42
33 Other Approaches MPC General l Penalty l Penalty - l Penalty - Minimum Time l 1 Penalty l 1 Penalty - l 1 Penalty - Other Approaches priority based approach: remove constraints in hierarchical order only relax constraints which are later in time (e.g. if infeasibility due to model/plant mismatch) 33 / 42
34 MPC General Active Set QP Solver Feasible Parameter Space and Critical Regions Homotopy Infeasible Parameter Relaxed QP Relaxed KKT-System Continuity of the Solution Summary 34 / 42
35 Active Set QP Solver MPC General Active Set QP Solver Feasible Parameter Space and Critical Regions Homotopy Infeasible Parameter Relaxed QP Relaxed KKT-System Continuity of the Solution Summary At the optimal solution, several constraints are active, others inactive. If the set of active constraints is known beforehand, the QP can be reformulated: 1 min x 2 xt Qx + gx T 0 x s.t. G A x = b A,x0 ignoring the inactive constraints. The optimal solution can be found by solving the KKT-system: ( ) ( ) ( ) Q G T A x gx0 = G A 0 λ b A,x0 35 / 42
36 Feasible Parameter Space and Critical Regions MPC General The parameter space can be divided into critical regions. Each critical region has a certain set G CR A of active constraints at the optimal solution. Active Set QP Solver Feasible Parameter Space and Critical Regions Homotopy Infeasible Parameter Relaxed QP Relaxed KKT-System Continuity of the Solution Summary 36 / 42
37 Homotopy MPC If we know the optimal solution at x 0, a homotopy to the new optimal solution at x new 0 can be performed. General Active Set QP Solver Feasible Parameter Space and Critical Regions Homotopy Infeasible Parameter Relaxed QP Relaxed KKT-System Continuity of the Solution Summary At each boundary one constraint is added or removed from the active set. 37 / 42
38 Infeasible Parameter What happens, if a parameter is not inside the feasible set? MPC General Active Set QP Solver Feasible Parameter Space and Critical Regions Homotopy Infeasible Parameter Relaxed QP Relaxed KKT-System Continuity of the Solution Summary 38 / 42
39 Relaxed QP MPC Only the newly active constraint c j is relaxed, the following optimization problem is solved: General Active Set QP Solver Feasible Parameter Space and Critical Regions Homotopy Infeasible Parameter Relaxed QP Relaxed KKT-System Continuity of the Solution Summary subject to Gx b 1 min x 2 xt Qx + g T x h εε 2 c T j x b j + ε row j dropped 39 / 42
40 Relaxed KKT-System MPC General Active Set QP Solver Feasible Parameter Space and Critical Regions Homotopy Infeasible Parameter Relaxed QP Relaxed KKT-System Continuity of the Solution Summary The KKT-system is modified accordingly: h ε 1 1 ε 0 Q G T A c j x λ = g b A µ b j G A c T j ε and µ can be eliminated from the system: ε = c T j x b j, µ = h ε ε = h ε (c T j x b j ) ( Q + hε c j c T j G T A G A 0 ) ( ) x λ = ( ) g + hε b j c j b A 40 / 42
41 Continuity of the Solution MPC General Active Set QP Solver Feasible Parameter Space and Critical Regions Homotopy Infeasible Parameter Relaxed QP Relaxed KKT-System Continuity of the Solution Summary The solutions of the old system and the modified system coincide at the boundary. The homotopy can be continued with a modified Hessian matrix. The Cholesky decomposition of Q can be updated by a rank-1 update, which is computationally cheap. ε will be supervised and removed when it becomes zero. Removal of an infeasible constraint works exactly as addition, but with inverse signs. 41 / 42
42 Summary MPC General Active Set QP Solver Feasible Parameter Space and Critical Regions Homotopy Infeasible Parameter Relaxed QP Relaxed KKT-System Continuity of the Solution Summary The strategy is exact (as long as the problem is feasible) Very low additional computational cost No additional computational cost when the problem is feasible Possible open-/closed-loop mismatch (open question) 42 / 42
Regional Solution of Constrained LQ Optimal Control
Regional Solution of Constrained LQ Optimal Control José DeDoná September 2004 Outline 1 Recap on the Solution for N = 2 2 Regional Explicit Solution Comparison with the Maximal Output Admissible Set 3
More information1 Computing with constraints
Notes for 2017-04-26 1 Computing with constraints Recall that our basic problem is minimize φ(x) s.t. x Ω where the feasible set Ω is defined by equality and inequality conditions Ω = {x R n : c i (x)
More informationCourse on Model Predictive Control Part II Linear MPC design
Course on Model Predictive Control Part II Linear MPC design Gabriele Pannocchia Department of Chemical Engineering, University of Pisa, Italy Email: g.pannocchia@diccism.unipi.it Facoltà di Ingegneria,
More informationNonlinear Model Predictive Control Tools (NMPC Tools)
Nonlinear Model Predictive Control Tools (NMPC Tools) Rishi Amrit, James B. Rawlings April 5, 2008 1 Formulation We consider a control system composed of three parts([2]). Estimator Target calculator Regulator
More informationA Constraint-Reduced MPC Algorithm for Convex Quadratic Programming, with a Modified Active-Set Identification Scheme
A Constraint-Reduced MPC Algorithm for Convex Quadratic Programming, with a Modified Active-Set Identification Scheme M. Paul Laiu 1 and (presenter) André L. Tits 2 1 Oak Ridge National Laboratory laiump@ornl.gov
More information10 Numerical methods for constrained problems
10 Numerical methods for constrained problems min s.t. f(x) h(x) = 0 (l), g(x) 0 (m), x X The algorithms can be roughly divided the following way: ˆ primal methods: find descent direction keeping inside
More information5.6 Penalty method and augmented Lagrangian method
5.6 Penalty method and augmented Lagrangian method Consider a generic NLP problem min f (x) s.t. c i (x) 0 i I c i (x) = 0 i E (1) x R n where f and the c i s are of class C 1 or C 2, and I and E are the
More informationParis'09 ECCI Eduardo F. Camacho MPC Constraints 2. Paris'09 ECCI Eduardo F. Camacho MPC Constraints 4
Outline Constrained MPC Eduardo F. Camacho Univ. of Seville. Constraints in Process Control. Constraints and MPC 3. Formulation of Constrained MPC 4. Illustrative Examples 5. Feasibility. Constraint Management
More informationA New Penalty-SQP Method
Background and Motivation Illustration of Numerical Results Final Remarks Frank E. Curtis Informs Annual Meeting, October 2008 Background and Motivation Illustration of Numerical Results Final Remarks
More informationLecture 13: Constrained optimization
2010-12-03 Basic ideas A nonlinearly constrained problem must somehow be converted relaxed into a problem which we can solve (a linear/quadratic or unconstrained problem) We solve a sequence of such problems
More informationQuadratic Programming
Quadratic Programming Outline Linearly constrained minimization Linear equality constraints Linear inequality constraints Quadratic objective function 2 SideBar: Matrix Spaces Four fundamental subspaces
More informationTheory in Model Predictive Control :" Constraint Satisfaction and Stability!
Theory in Model Predictive Control :" Constraint Satisfaction and Stability Colin Jones, Melanie Zeilinger Automatic Control Laboratory, EPFL Example: Cessna Citation Aircraft Linearized continuous-time
More informationLearning Model Predictive Control for Iterative Tasks: A Computationally Efficient Approach for Linear System
Learning Model Predictive Control for Iterative Tasks: A Computationally Efficient Approach for Linear System Ugo Rosolia Francesco Borrelli University of California at Berkeley, Berkeley, CA 94701, USA
More informationPart 4: Active-set methods for linearly constrained optimization. Nick Gould (RAL)
Part 4: Active-set methods for linearly constrained optimization Nick Gould RAL fx subject to Ax b Part C course on continuoue optimization LINEARLY CONSTRAINED MINIMIZATION fx subject to Ax { } b where
More informationNumerical Optimization. Review: Unconstrained Optimization
Numerical Optimization Finding the best feasible solution Edward P. Gatzke Department of Chemical Engineering University of South Carolina Ed Gatzke (USC CHE ) Numerical Optimization ECHE 589, Spring 2011
More informationAlgorithms for constrained local optimization
Algorithms for constrained local optimization Fabio Schoen 2008 http://gol.dsi.unifi.it/users/schoen Algorithms for constrained local optimization p. Feasible direction methods Algorithms for constrained
More informationCO350 Linear Programming Chapter 6: The Simplex Method
CO50 Linear Programming Chapter 6: The Simplex Method rd June 2005 Chapter 6: The Simplex Method 1 Recap Suppose A is an m-by-n matrix with rank m. max. c T x (P ) s.t. Ax = b x 0 On Wednesday, we learned
More information2.098/6.255/ Optimization Methods Practice True/False Questions
2.098/6.255/15.093 Optimization Methods Practice True/False Questions December 11, 2009 Part I For each one of the statements below, state whether it is true or false. Include a 1-3 line supporting sentence
More informationPenalty and Barrier Methods General classical constrained minimization problem minimize f(x) subject to g(x) 0 h(x) =0 Penalty methods are motivated by the desire to use unconstrained optimization techniques
More information5 Handling Constraints
5 Handling Constraints Engineering design optimization problems are very rarely unconstrained. Moreover, the constraints that appear in these problems are typically nonlinear. This motivates our interest
More informationLecture 11 and 12: Penalty methods and augmented Lagrangian methods for nonlinear programming
Lecture 11 and 12: Penalty methods and augmented Lagrangian methods for nonlinear programming Coralia Cartis, Mathematical Institute, University of Oxford C6.2/B2: Continuous Optimization Lecture 11 and
More informationOptimization. Yuh-Jye Lee. March 21, Data Science and Machine Intelligence Lab National Chiao Tung University 1 / 29
Optimization Yuh-Jye Lee Data Science and Machine Intelligence Lab National Chiao Tung University March 21, 2017 1 / 29 You Have Learned (Unconstrained) Optimization in Your High School Let f (x) = ax
More informationAlgorithms for Constrained Optimization
1 / 42 Algorithms for Constrained Optimization ME598/494 Lecture Max Yi Ren Department of Mechanical Engineering, Arizona State University April 19, 2015 2 / 42 Outline 1. Convergence 2. Sequential quadratic
More informationSupport Vector Machine (continued)
Support Vector Machine continued) Overlapping class distribution: In practice the class-conditional distributions may overlap, so that the training data points are no longer linearly separable. We need
More informationISM206 Lecture Optimization of Nonlinear Objective with Linear Constraints
ISM206 Lecture Optimization of Nonlinear Objective with Linear Constraints Instructor: Prof. Kevin Ross Scribe: Nitish John October 18, 2011 1 The Basic Goal The main idea is to transform a given constrained
More informationNumerical optimization
Numerical optimization Lecture 4 Alexander & Michael Bronstein tosca.cs.technion.ac.il/book Numerical geometry of non-rigid shapes Stanford University, Winter 2009 2 Longest Slowest Shortest Minimal Maximal
More informationOptimization Problems in Model Predictive Control
Optimization Problems in Model Predictive Control Stephen Wright Jim Rawlings, Matt Tenny, Gabriele Pannocchia University of Wisconsin-Madison FoCM 02 Minneapolis August 6, 2002 1 reminder! Send your new
More informationEE C128 / ME C134 Feedback Control Systems
EE C128 / ME C134 Feedback Control Systems Lecture Additional Material Introduction to Model Predictive Control Maximilian Balandat Department of Electrical Engineering & Computer Science University of
More informationNumerical optimization. Numerical optimization. Longest Shortest where Maximal Minimal. Fastest. Largest. Optimization problems
1 Numerical optimization Alexander & Michael Bronstein, 2006-2009 Michael Bronstein, 2010 tosca.cs.technion.ac.il/book Numerical optimization 048921 Advanced topics in vision Processing and Analysis of
More informationOptimality Conditions for Constrained Optimization
72 CHAPTER 7 Optimality Conditions for Constrained Optimization 1. First Order Conditions In this section we consider first order optimality conditions for the constrained problem P : minimize f 0 (x)
More informationminimize x subject to (x 2)(x 4) u,
Math 6366/6367: Optimization and Variational Methods Sample Preliminary Exam Questions 1. Suppose that f : [, L] R is a C 2 -function with f () on (, L) and that you have explicit formulae for
More informationLecture 18: Optimization Programming
Fall, 2016 Outline Unconstrained Optimization 1 Unconstrained Optimization 2 Equality-constrained Optimization Inequality-constrained Optimization Mixture-constrained Optimization 3 Quadratic Programming
More informationNONLINEAR. (Hillier & Lieberman Introduction to Operations Research, 8 th edition)
NONLINEAR PROGRAMMING (Hillier & Lieberman Introduction to Operations Research, 8 th edition) Nonlinear Programming g Linear programming has a fundamental role in OR. In linear programming all its functions
More informationNumerical Optimization Professor Horst Cerjak, Horst Bischof, Thomas Pock Mat Vis-Gra SS09
Numerical Optimization 1 Working Horse in Computer Vision Variational Methods Shape Analysis Machine Learning Markov Random Fields Geometry Common denominator: optimization problems 2 Overview of Methods
More informationSupport Vector Machine (SVM) & Kernel CE-717: Machine Learning Sharif University of Technology. M. Soleymani Fall 2012
Support Vector Machine (SVM) & Kernel CE-717: Machine Learning Sharif University of Technology M. Soleymani Fall 2012 Linear classifier Which classifier? x 2 x 1 2 Linear classifier Margin concept x 2
More informationConvex Optimization and SVM
Convex Optimization and SVM Problem 0. Cf lecture notes pages 12 to 18. Problem 1. (i) A slab is an intersection of two half spaces, hence convex. (ii) A wedge is an intersection of two half spaces, hence
More informationA Warm-start Interior-point Method for Predictive Control
A Warm-start Interior-point Method for Predictive Control Amir Shahzad Eric C Kerrigan George A Constantinides Department of Electrical and Electronic Engineering, Imperial College London, SW7 2AZ, UK
More informationIE 5531: Engineering Optimization I
IE 5531: Engineering Optimization I Lecture 19: Midterm 2 Review Prof. John Gunnar Carlsson November 22, 2010 Prof. John Gunnar Carlsson IE 5531: Engineering Optimization I November 22, 2010 1 / 34 Administrivia
More information4TE3/6TE3. Algorithms for. Continuous Optimization
4TE3/6TE3 Algorithms for Continuous Optimization (Algorithms for Constrained Nonlinear Optimization Problems) Tamás TERLAKY Computing and Software McMaster University Hamilton, November 2005 terlaky@mcmaster.ca
More informationLINEAR PROGRAMMING I. a refreshing example standard form fundamental questions geometry linear algebra simplex algorithm
Linear programming Linear programming. Optimize a linear function subject to linear inequalities. (P) max c j x j n j= n s. t. a ij x j = b i i m j= x j 0 j n (P) max c T x s. t. Ax = b Lecture slides
More informationChapter 2: Linear Programming Basics. (Bertsimas & Tsitsiklis, Chapter 1)
Chapter 2: Linear Programming Basics (Bertsimas & Tsitsiklis, Chapter 1) 33 Example of a Linear Program Remarks. minimize 2x 1 x 2 + 4x 3 subject to x 1 + x 2 + x 4 2 3x 2 x 3 = 5 x 3 + x 4 3 x 1 0 x 3
More informationMultidisciplinary System Design Optimization (MSDO)
Multidisciplinary System Design Optimization (MSDO) Numerical Optimization II Lecture 8 Karen Willcox 1 Massachusetts Institute of Technology - Prof. de Weck and Prof. Willcox Today s Topics Sequential
More informationMath 273a: Optimization The Simplex method
Math 273a: Optimization The Simplex method Instructor: Wotao Yin Department of Mathematics, UCLA Fall 2015 material taken from the textbook Chong-Zak, 4th Ed. Overview: idea and approach If a standard-form
More informationLecture slides by Kevin Wayne
LINEAR PROGRAMMING I a refreshing example standard form fundamental questions geometry linear algebra simplex algorithm Lecture slides by Kevin Wayne Last updated on 7/25/17 11:09 AM Linear programming
More informationMS&E 318 (CME 338) Large-Scale Numerical Optimization
Stanford University, Management Science & Engineering (and ICME) MS&E 318 (CME 338) Large-Scale Numerical Optimization 1 Origins Instructor: Michael Saunders Spring 2015 Notes 9: Augmented Lagrangian Methods
More informationNonlinear Optimization for Optimal Control
Nonlinear Optimization for Optimal Control Pieter Abbeel UC Berkeley EECS Many slides and figures adapted from Stephen Boyd [optional] Boyd and Vandenberghe, Convex Optimization, Chapters 9 11 [optional]
More information4. The Dual Simplex Method
4. The Dual Simplex Method Javier Larrosa Albert Oliveras Enric Rodríguez-Carbonell Problem Solving and Constraint Programming (RPAR) Session 4 p.1/34 Basic Idea (1) Algorithm as explained so far known
More informationPrediktivno upravljanje primjenom matematičkog programiranja
Prediktivno upravljanje primjenom matematičkog programiranja Doc. dr. sc. Mato Baotić Fakultet elektrotehnike i računarstva Sveučilište u Zagrebu www.fer.hr/mato.baotic Outline Application Examples PredictiveControl
More information2.3 Linear Programming
2.3 Linear Programming Linear Programming (LP) is the term used to define a wide range of optimization problems in which the objective function is linear in the unknown variables and the constraints are
More informationCE 191: Civil & Environmental Engineering Systems Analysis. LEC 17 : Final Review
CE 191: Civil & Environmental Engineering Systems Analysis LEC 17 : Final Review Professor Scott Moura Civil & Environmental Engineering University of California, Berkeley Fall 2014 Prof. Moura UC Berkeley
More informationIV. Violations of Linear Programming Assumptions
IV. Violations of Linear Programming Assumptions Some types of Mathematical Programming problems violate at least one condition of strict Linearity - Deterministic Nature - Additivity - Direct Proportionality
More informationHomework Solution # 3
ECSE 644 Optimal Control Feb, 4 Due: Feb 17, 4 (Tuesday) Homework Solution # 3 1 (5%) Consider the discrete nonlinear control system in Homework # For the optimal control and trajectory that you have found
More informationEfficient Numerical Methods for Nonlinear MPC and Moving Horizon Estimation
Efficient Numerical Methods for Nonlinear MPC and Moving Horizon Estimation Moritz Diehl, Hans Joachim Ferreau, and Niels Haverbeke Optimization in Engineering Center (OPTEC) and ESAT-SCD, K.U. Leuven,
More informationAcademic Editor: Dominique Bonvin Received: 26 November 2016; Accepted: 13 February 2017; Published: 27 February 2017
Article Sensitivity-Based Economic NMPC with a Path-Following Approach Eka Suwartadi 1, Vyacheslav Kungurtsev 2 and Johannes Jäschke 1, * 1 Department of Chemical Engineering, Norwegian University of Science
More informationInfeasibility Detection and an Inexact Active-Set Method for Large-Scale Nonlinear Optimization
Infeasibility Detection and an Inexact Active-Set Method for Large-Scale Nonlinear Optimization Frank E. Curtis, Lehigh University involving joint work with James V. Burke, University of Washington Daniel
More informationSupport Vector Machine (SVM) and Kernel Methods
Support Vector Machine (SVM) and Kernel Methods CE-717: Machine Learning Sharif University of Technology Fall 2016 Soleymani Outline Margin concept Hard-Margin SVM Soft-Margin SVM Dual Problems of Hard-Margin
More informationOptimization. Yuh-Jye Lee. March 28, Data Science and Machine Intelligence Lab National Chiao Tung University 1 / 40
Optimization Yuh-Jye Lee Data Science and Machine Intelligence Lab National Chiao Tung University March 28, 2017 1 / 40 The Key Idea of Newton s Method Let f : R n R be a twice differentiable function
More informationAN AUGMENTED LAGRANGIAN AFFINE SCALING METHOD FOR NONLINEAR PROGRAMMING
AN AUGMENTED LAGRANGIAN AFFINE SCALING METHOD FOR NONLINEAR PROGRAMMING XIAO WANG AND HONGCHAO ZHANG Abstract. In this paper, we propose an Augmented Lagrangian Affine Scaling (ALAS) algorithm for general
More informationSupport Vector Machine (SVM) and Kernel Methods
Support Vector Machine (SVM) and Kernel Methods CE-717: Machine Learning Sharif University of Technology Fall 2014 Soleymani Outline Margin concept Hard-Margin SVM Soft-Margin SVM Dual Problems of Hard-Margin
More informationInterior-Point and Augmented Lagrangian Algorithms for Optimization and Control
Interior-Point and Augmented Lagrangian Algorithms for Optimization and Control Stephen Wright University of Wisconsin-Madison May 2014 Wright (UW-Madison) Constrained Optimization May 2014 1 / 46 In This
More informationSupport Vector Machines: Maximum Margin Classifiers
Support Vector Machines: Maximum Margin Classifiers Machine Learning and Pattern Recognition: September 16, 2008 Piotr Mirowski Based on slides by Sumit Chopra and Fu-Jie Huang 1 Outline What is behind
More informationCE 191: Civil and Environmental Engineering Systems Analysis. LEC 05 : Optimality Conditions
CE 191: Civil and Environmental Engineering Systems Analysis LEC : Optimality Conditions Professor Scott Moura Civil & Environmental Engineering University of California, Berkeley Fall 214 Prof. Moura
More informationHot-Starting NLP Solvers
Hot-Starting NLP Solvers Andreas Wächter Department of Industrial Engineering and Management Sciences Northwestern University waechter@iems.northwestern.edu 204 Mixed Integer Programming Workshop Ohio
More informationDecision Procedures An Algorithmic Point of View
An Algorithmic Point of View ILP References: Integer Programming / Laurence Wolsey Deciding ILPs with Branch & Bound Intro. To mathematical programming / Hillier, Lieberman Daniel Kroening and Ofer Strichman
More informationConstrained Optimization
1 / 22 Constrained Optimization ME598/494 Lecture Max Yi Ren Department of Mechanical Engineering, Arizona State University March 30, 2015 2 / 22 1. Equality constraints only 1.1 Reduced gradient 1.2 Lagrange
More informationWritten Examination
Division of Scientific Computing Department of Information Technology Uppsala University Optimization Written Examination 202-2-20 Time: 4:00-9:00 Allowed Tools: Pocket Calculator, one A4 paper with notes
More informationCS711008Z Algorithm Design and Analysis
CS711008Z Algorithm Design and Analysis Lecture 8 Linear programming: interior point method Dongbo Bu Institute of Computing Technology Chinese Academy of Sciences, Beijing, China 1 / 31 Outline Brief
More informationNonlinear Programming (Hillier, Lieberman Chapter 13) CHEM-E7155 Production Planning and Control
Nonlinear Programming (Hillier, Lieberman Chapter 13) CHEM-E7155 Production Planning and Control 19/4/2012 Lecture content Problem formulation and sample examples (ch 13.1) Theoretical background Graphical
More informationInterior Point Methods in Mathematical Programming
Interior Point Methods in Mathematical Programming Clóvis C. Gonzaga Federal University of Santa Catarina, Brazil Journées en l honneur de Pierre Huard Paris, novembre 2008 01 00 11 00 000 000 000 000
More informationECE7850 Lecture 9. Model Predictive Control: Computational Aspects
ECE785 ECE785 Lecture 9 Model Predictive Control: Computational Aspects Model Predictive Control for Constrained Linear Systems Online Solution to Linear MPC Multiparametric Programming Explicit Linear
More informationChapter 3, Operations Research (OR)
Chapter 3, Operations Research (OR) Kent Andersen February 7, 2007 1 Linear Programs (continued) In the last chapter, we introduced the general form of a linear program, which we denote (P) Minimize Z
More informationBenders Decomposition
Benders Decomposition Yuping Huang, Dr. Qipeng Phil Zheng Department of Industrial and Management Systems Engineering West Virginia University IENG 593G Nonlinear Programg, Spring 2012 Yuping Huang (IMSE@WVU)
More informationAnalytic Center Cutting-Plane Method
Analytic Center Cutting-Plane Method S. Boyd, L. Vandenberghe, and J. Skaf April 14, 2011 Contents 1 Analytic center cutting-plane method 2 2 Computing the analytic center 3 3 Pruning constraints 5 4 Lower
More information5.5 Quadratic programming
5.5 Quadratic programming Minimize a quadratic function subject to linear constraints: 1 min x t Qx + c t x 2 s.t. a t i x b i i I (P a t i x = b i i E x R n, where Q is an n n matrix, I and E are the
More informationSupport Vector Machines
Support Vector Machines Hypothesis Space variable size deterministic continuous parameters Learning Algorithm linear and quadratic programming eager batch SVMs combine three important ideas Apply optimization
More informationGeneralization to inequality constrained problem. Maximize
Lecture 11. 26 September 2006 Review of Lecture #10: Second order optimality conditions necessary condition, sufficient condition. If the necessary condition is violated the point cannot be a local minimum
More informationLecture 3. Optimization Problems and Iterative Algorithms
Lecture 3 Optimization Problems and Iterative Algorithms January 13, 2016 This material was jointly developed with Angelia Nedić at UIUC for IE 598ns Outline Special Functions: Linear, Quadratic, Convex
More information4F3 - Predictive Control
4F3 Predictive Control - Lecture 3 p 1/21 4F3 - Predictive Control Lecture 3 - Predictive Control with Constraints Jan Maciejowski jmm@engcamacuk 4F3 Predictive Control - Lecture 3 p 2/21 Constraints on
More informationConvex Optimization and Support Vector Machine
Convex Optimization and Support Vector Machine Problem 0. Consider a two-class classification problem. The training data is L n = {(x 1, t 1 ),..., (x n, t n )}, where each t i { 1, 1} and x i R p. We
More informationWhat s New in Active-Set Methods for Nonlinear Optimization?
What s New in Active-Set Methods for Nonlinear Optimization? Philip E. Gill Advances in Numerical Computation, Manchester University, July 5, 2011 A Workshop in Honor of Sven Hammarling UCSD Center for
More informationEE363 homework 2 solutions
EE363 Prof. S. Boyd EE363 homework 2 solutions. Derivative of matrix inverse. Suppose that X : R R n n, and that X(t is invertible. Show that ( d d dt X(t = X(t dt X(t X(t. Hint: differentiate X(tX(t =
More informationPOWER SYSTEMS in general are currently operating
TO APPEAR IN IEEE TRANSACTIONS ON POWER SYSTEMS 1 Robust Optimal Power Flow Solution Using Trust Region and Interior-Point Methods Andréa A. Sousa, Geraldo L. Torres, Member IEEE, Claudio A. Cañizares,
More informationLecture 9: Large Margin Classifiers. Linear Support Vector Machines
Lecture 9: Large Margin Classifiers. Linear Support Vector Machines Perceptrons Definition Perceptron learning rule Convergence Margin & max margin classifiers (Linear) support vector machines Formulation
More informationConstrained Nonlinear Optimization Algorithms
Department of Industrial Engineering and Management Sciences Northwestern University waechter@iems.northwestern.edu Institute for Mathematics and its Applications University of Minnesota August 4, 2016
More informationCHAPTER 2: QUADRATIC PROGRAMMING
CHAPTER 2: QUADRATIC PROGRAMMING Overview Quadratic programming (QP) problems are characterized by objective functions that are quadratic in the design variables, and linear constraints. In this sense,
More informationNonlinear Optimization for Optimal Control Part 2. Pieter Abbeel UC Berkeley EECS. From linear to nonlinear Model-predictive control (MPC) POMDPs
Nonlinear Optimization for Optimal Control Part 2 Pieter Abbeel UC Berkeley EECS Outline From linear to nonlinear Model-predictive control (MPC) POMDPs Page 1! From Linear to Nonlinear We know how to solve
More informationAn Introduction to Model-based Predictive Control (MPC) by
ECE 680 Fall 2017 An Introduction to Model-based Predictive Control (MPC) by Stanislaw H Żak 1 Introduction The model-based predictive control (MPC) methodology is also referred to as the moving horizon
More informationPart II: Model Predictive Control
Part II: Model Predictive Control Manfred Morari Alberto Bemporad Francesco Borrelli Unconstrained Optimal Control Unconstrained infinite time optimal control V (x 0 )= inf U k=0 [ x k Qx k + u k Ru k]
More informationSupport Vector Machine (SVM) and Kernel Methods
Support Vector Machine (SVM) and Kernel Methods CE-717: Machine Learning Sharif University of Technology Fall 2015 Soleymani Outline Margin concept Hard-Margin SVM Soft-Margin SVM Dual Problems of Hard-Margin
More informationDr. Maddah ENMG 500 Engineering Management I 10/21/07
Dr. Maddah ENMG 500 Engineering Management I 10/21/07 Computational Procedure of the Simplex Method The optimal solution of a general LP problem is obtained in the following steps: Step 1. Express the
More informationN. L. P. NONLINEAR PROGRAMMING (NLP) deals with optimization models with at least one nonlinear function. NLP. Optimization. Models of following form:
0.1 N. L. P. Katta G. Murty, IOE 611 Lecture slides Introductory Lecture NONLINEAR PROGRAMMING (NLP) deals with optimization models with at least one nonlinear function. NLP does not include everything
More informationThe simplex algorithm
The simplex algorithm The simplex algorithm is the classical method for solving linear programs. Its running time is not polynomial in the worst case. It does yield insight into linear programs, however,
More informationSection Notes 9. IP: Cutting Planes. Applied Math 121. Week of April 12, 2010
Section Notes 9 IP: Cutting Planes Applied Math 121 Week of April 12, 2010 Goals for the week understand what a strong formulations is. be familiar with the cutting planes algorithm and the types of cuts
More informationAn Inexact Sequential Quadratic Optimization Method for Nonlinear Optimization
An Inexact Sequential Quadratic Optimization Method for Nonlinear Optimization Frank E. Curtis, Lehigh University involving joint work with Travis Johnson, Northwestern University Daniel P. Robinson, Johns
More informationIntroduction to Model Predictive Control. Dipartimento di Elettronica e Informazione
Introduction to Model Predictive Control Riccardo Scattolini Riccardo Scattolini Dipartimento di Elettronica e Informazione Finite horizon optimal control 2 Consider the system At time k we want to compute
More informationPart 1. The Review of Linear Programming
In the name of God Part 1. The Review of Linear Programming 1.2. Spring 2010 Instructor: Dr. Masoud Yaghini Outline Introduction Basic Feasible Solutions Key to the Algebra of the The Simplex Algorithm
More informationConvex Optimization. Prof. Nati Srebro. Lecture 12: Infeasible-Start Newton s Method Interior Point Methods
Convex Optimization Prof. Nati Srebro Lecture 12: Infeasible-Start Newton s Method Interior Point Methods Equality Constrained Optimization f 0 (x) s. t. A R p n, b R p Using access to: 2 nd order oracle
More informationLecture 10: Linear programming duality and sensitivity 0-0
Lecture 10: Linear programming duality and sensitivity 0-0 The canonical primal dual pair 1 A R m n, b R m, and c R n maximize z = c T x (1) subject to Ax b, x 0 n and minimize w = b T y (2) subject to
More informationMATH4406 (Control Theory) Unit 6: The Linear Quadratic Regulator (LQR) and Model Predictive Control (MPC) Prepared by Yoni Nazarathy, Artem
MATH4406 (Control Theory) Unit 6: The Linear Quadratic Regulator (LQR) and Model Predictive Control (MPC) Prepared by Yoni Nazarathy, Artem Pulemotov, September 12, 2012 Unit Outline Goal 1: Outline linear
More informationOptimization Problems with Constraints - introduction to theory, numerical Methods and applications
Optimization Problems with Constraints - introduction to theory, numerical Methods and applications Dr. Abebe Geletu Ilmenau University of Technology Department of Simulation and Optimal Processes (SOP)
More information