ECE7850 Lecture 7. Discrete Time Optimal Control and Dynamic Programming
|
|
- Dorcas Brooks
- 5 years ago
- Views:
Transcription
1 ECE7850 Lecture 7 Discrete Time Optimal Control and Dynamic Programming Discrete Time Optimal control Problems Short Introduction to Dynamic Programming Connection to Stabilization Problems 1
2 DT nonlinear control system: Discrete Time Optimal Control Problem x(t +1)=f(x(t),u(t)),x X, u U, t Z + (1) For traditional system: X R n, U R m are continuous variables A large class of DT hybrid systems can also be written in (or viewed as) the above form: switched systems: U R m Qwith mixed continuous/discrete control input Discrete Time Optimal Control Problem 2
3 variable structure system (e.g. Piecewise affine system): f defined in terms of partitions: f(x(t),u(t)) = f i (x(t),u(t)), if x(t) Ω i more general hybrid system: X R n Q x and U R m Q u Actual control constraint is state-dependent: U(x) ={u U : f(x, u) X} Discrete Time Optimal Control Problem 3
4 A general way to determine control: time-varying state-feedback law: u(t) =μ t (x(t)) require: μ t (x(t)) U(x(t)) Control policy: N-horizon policy: π N = {μ 0,...,μ N 1 } Π N Infinite-horizon policy: π = {μ 0,μ 1,...} Π Stationary policy: π = {μ,μ,...} Π Discrete Time Optimal Control Problem 4
5 x( ; z, π): Cl-trajectory starting from z under policy π = {μ 0,μ 1,...} x(t +1;z, π) =f(x(t; z, π),μ t (x(t; z, π))) (2) Corresponding control sequence: u( ; z, π) =μ t (x(t; z, π)) Quantify performance through cost functions: Finite Horizon: J N (z, π N )= N 1 t=0 l(x(t; z, π N),u(t; z, π N )) + J f (x(n; z, π N )) Infinite Horizon: J (z, π )= t=0 l(x(t; z, π ),u(t; z, π )) running cost l : X U R + ; terminal cost: J f : X R + Discrete Time Optimal Control Problem 5
6 Optimal control problem: Finite-Horizon: V N (z) =inf πn Π N J N (z, π N ) Infinite-Horizon: V (z) =inf π Π J (z, π ) V N : N-horizon value function; V : infinite horizon value function lim N V N may not exist; even when it exists V V in general Discrete Time Optimal Control Problem 6
7 Short Introduction to Dynamic Programming Bellman s principle of optimality: any segment along an optimal trajectory is also optimal among all the trajectories joining the two end points of the segment Short Introduction to Dynamic Programming 7
8 This leads to the backward value iteration: compute V N iteratively starting from V 0 V 0 = J f : zero-horizon cost is just the terminal cost function Suppose we know V j for some j 0, then V j+1 (z) =min u U(z) {l(z, u)+v j (f(z, u))} u = argmin u U(z) {l(z, u)+v j (f(z, u))}: optimal control when there is j +1-step left and system is at state z Short Introduction to Dynamic Programming 8
9 Example 1 (Shortest Path Problem): Find the shortest path to a 4 X = {a 1,...,a 8 }; U(x): available links to take at x X dynamics: x(k +1)=f(x(k +1),u(k + 1)); running cost: l(x(k),u(k)) = edge length terminal cost: J f (x) = Value iteration: 0 if x = a 4 ow Short Introduction to Dynamic Programming 9
10 Linear Quadratic Regulator: System dynamics: x(t +1)=Ax(t)+Bu(t) Running cost: l(z, u) =z T Qz + u T Ru; Terminal cost: J f () = z T Q f z Suppose j-horizon value function is V j (z) =z T P j z, derive V j+1 (z) Short Introduction to Dynamic Programming 10
11 Derivation (cont.): optimal control u at t = N (j +1): Short Introduction to Dynamic Programming 11
12 Summary of LQR results: j-horizon value function: V j (z) =z T P j z, where P j is generated by Riccati recursion P j+1 = ρ(p j ) ρ(p )=Q + A T PA A T PB(R + B T PB) 1 B T PA (3) To compute LQR controller 1. Start with P 0 = Q f 2. Riccati recursion: P j+1 = ρ(p j ) 3. Compute optimal feedback gain: K j+1 = (R + B T P j B) 1 B T P j A To use LQR controller: 1. initial state: x 0 2. compute control input: u (t) =K N t x (t) 3. state update: x (t +1)=Ax (t)+bu (t) Short Introduction to Dynamic Programming 12
13 Theorem 1 (Infinite-Horizon LQR): If (A, B) is stabilizable and (A, C) is detectable, where Q = C T C, then as j, P j P and K j K with stable cl-system (A + BK ); P satisfies the algebraic Riccati equation: P = ρ(p ) K = (R + B T P B) 1 B T P A K(P ) Short Introduction to Dynamic Programming 13
14 Value Iteration Operator To simplify notations, we define two function spaces: G {g : X R {± }}and G + {g : X R {+ }} Value iteration can be viewed as an operator T : G G T [g](z) = min l(z, u)+g(f(z, u)), z X, g G (4) u U(z) Iteration under a given control law μ : X U with μ(x) X, x X is defined as T μ [g](z) =l(z, μ(z)) + g(f(z, μ(z))), z X, g G (5) Short Introduction to Dynamic Programming 14
15 T k : the composition of T with itself k times: T k+1 [g] =T [T k [g]], with T 0 [g] =g Theorem 2 Key Properties of Value Iteration: 1. Value Iteration: V N = T N (J f ) 2. Bellman Equation: V G + is a fixed point of T, i.e. T [V ]=V 3. Minimum Property: For any g G +,ifg T[g], then g V 4. Monotonicity: If J f 0, then J f = V 0 V 1 V 5. Stationary Policy: A stationary policy π = {μ,μ...} is optimal if and only if T [V ]= T μ [V ] Short Introduction to Dynamic Programming 15
16 Some unfortunate facts: 1. lim N V N may not exist 2. Even when lim N V N exists, we may have V V 3. Bellman equation: T [g] =g may have infinitely many solutions that are not V Counterexamples can be found in Bertsekas book (volume II) These cases won t happen if state space is finite, cost per stage l(z, u) is bounded, and cost function is discounted. However, for control problems, we often don t have the luxury to make these assumptions Short Introduction to Dynamic Programming 16
17 Connection to Stabilization Problems : an arbitrary p-norm raised to an arbitrary positive power, namely, r p, for some p, r R + Definition 1 (Exponential stabilizability): π such that x(t; z, π ) cγ t z Definition 2 (Exponentially Stabilizing Control Lyapunov Function (ECLF)): A PD function g G + is called an ECLF if μ with μ(z) U(z), z X, such that κ 2 z g(z) κ 1 z g(z) g(f(z, μ(z))) κ 3 z (6) Connection to Stabilization Problems 17
18 If condition (6) holds, then CL-system under π = {μ,μ,...} is exponentially stable with x(t; z, π ) κ 2 κ κ 3 /κ 2 t z (7) How does stabilization relate to optimal control? Let us start with a well known result for LQR (see Slide 10 for notations) Theorem 3 If (A, B) is stabilizable, and (A,C) is detectable, where Q = C T C, then 1. Convergence of value iteration: P j P where P j+1 = ρ(p j ) with P 0 =0 2. V (z) =z T P z is an ECLF 3. μ(z) =K(P )z is an stabilizing feedback law Connection to Stabilization Problems 18
19 The theorem indicates: if we choose cost function properly (i.e. to make (A, C) detectable where Q = C T C), then a linear system if stabilizable if and only if it can be stabilized by the LQR controller Question: can we generalize such result, to what extent, and how? Roughly speaking, if a system can be (exponentially) stabilized, then we can always stabilize it by solving a properly defined optimal control problem. Connection to Stabilization Problems 19
20 Main idea: select running cost l : X U R + such that exponentially stabilizable V (z) β z (8) V (z) β z the optimal trajectory exponentially stable (9) This can be achieved by various types of cost functions. To illustrate the key idea, let us choose the simplest one, say l(z, u) = z exponentially stabilizable V (z) β z for some β< Connection to Stabilization Problems 20
21 V (z) β z V (z) is an ECLF Note: always stationary stabilizing policy π = {μ,μ,...}, where T μ [V ]=V : Connection to Stabilization Problems 21
22 can also incorporate nontrivial control cost (e.g. l(z, u) = z + u ). This will not affect condition (9), but condition (8) requires more discussions. For example, if the stabilizing control inputs u does not converge, then V (z) cannot be bounded. All that matters is whether stabilizable with finite control energy For linear systems (and many other systems) stabilizability always means stabilizability with finite control energy thus, adding control cost will not destroy the stability of the optimal trajectory Connection to Stabilization Problems 22
23 In summary, if system is exponentially stabilizable, then with properly selected running cost function l(z, u), V is an ECLS optimal infinite-horizon policy π = {μ,μ,...} is exponentially stabilizing This provides a unified way to construct ECLF and stabilizing controller. However, infinite-horizon optimal control is often more challenging to solve than stabilization One way is to use V N (and μ N ) to replace V (and μ ) Model Predictive Control Connection to Stabilization Problems 23
ECE7850 Lecture 8. Nonlinear Model Predictive Control: Theoretical Aspects
ECE7850 Lecture 8 Nonlinear Model Predictive Control: Theoretical Aspects Model Predictive control (MPC) is a powerful control design method for constrained dynamical systems. The basic principles and
More informationMATH4406 (Control Theory) Unit 6: The Linear Quadratic Regulator (LQR) and Model Predictive Control (MPC) Prepared by Yoni Nazarathy, Artem
MATH4406 (Control Theory) Unit 6: The Linear Quadratic Regulator (LQR) and Model Predictive Control (MPC) Prepared by Yoni Nazarathy, Artem Pulemotov, September 12, 2012 Unit Outline Goal 1: Outline linear
More informationLecture 5 Linear Quadratic Stochastic Control
EE363 Winter 2008-09 Lecture 5 Linear Quadratic Stochastic Control linear-quadratic stochastic control problem solution via dynamic programming 5 1 Linear stochastic system linear dynamical system, over
More informationOPTIMAL CONTROL. Sadegh Bolouki. Lecture slides for ECE 515. University of Illinois, Urbana-Champaign. Fall S. Bolouki (UIUC) 1 / 28
OPTIMAL CONTROL Sadegh Bolouki Lecture slides for ECE 515 University of Illinois, Urbana-Champaign Fall 2016 S. Bolouki (UIUC) 1 / 28 (Example from Optimal Control Theory, Kirk) Objective: To get from
More informationLecture Note 7: Switching Stabilization via Control-Lyapunov Function
ECE7850: Hybrid Systems:Theory and Applications Lecture Note 7: Switching Stabilization via Control-Lyapunov Function Wei Zhang Assistant Professor Department of Electrical and Computer Engineering Ohio
More informationOn Piecewise Quadratic Control-Lyapunov Functions for Switched Linear Systems
On Piecewise Quadratic Control-Lyapunov Functions for Switched Linear Systems Wei Zhang, Alessandro Abate, Michael P. Vitus and Jianghai Hu Abstract In this paper, we prove that a discrete-time switched
More informationCONTROLLER SYNTHESIS FOR SWITCHED SYSTEMS USING APPROXIMATE DYNAMIC PROGRAMMING. A Dissertation. Submitted to the Faculty.
CONTROLLER SYNTHESIS FOR SWITCHED SYSTEMS USING APPROXIMATE DYNAMIC PROGRAMMING A Dissertation Submitted to the Faculty of Purdue University by Wei Zhang In Partial Fulfillment of the Requirements for
More informationA Tour of Reinforcement Learning The View from Continuous Control. Benjamin Recht University of California, Berkeley
A Tour of Reinforcement Learning The View from Continuous Control Benjamin Recht University of California, Berkeley trustable, scalable, predictable Control Theory! Reinforcement Learning is the study
More informationEN Applied Optimal Control Lecture 8: Dynamic Programming October 10, 2018
EN530.603 Applied Optimal Control Lecture 8: Dynamic Programming October 0, 08 Lecturer: Marin Kobilarov Dynamic Programming (DP) is conerned with the computation of an optimal policy, i.e. an optimal
More informationOutline. 1 Linear Quadratic Problem. 2 Constraints. 3 Dynamic Programming Solution. 4 The Infinite Horizon LQ Problem.
Model Predictive Control Short Course Regulation James B. Rawlings Michael J. Risbeck Nishith R. Patel Department of Chemical and Biological Engineering Copyright c 217 by James B. Rawlings Outline 1 Linear
More information4F3 - Predictive Control
4F3 Predictive Control - Lecture 2 p 1/23 4F3 - Predictive Control Lecture 2 - Unconstrained Predictive Control Jan Maciejowski jmm@engcamacuk 4F3 Predictive Control - Lecture 2 p 2/23 References Predictive
More informationOptimal control and estimation
Automatic Control 2 Optimal control and estimation Prof. Alberto Bemporad University of Trento Academic year 2010-2011 Prof. Alberto Bemporad (University of Trento) Automatic Control 2 Academic year 2010-2011
More informationStabilization of Discrete-Time Switched Linear Systems: A Control-Lyapunov Function Approach
Stabilization of Discrete-Time Switched Linear Systems: A Control-Lyapunov Function Approach Wei Zhang 1, Alessandro Abate 2 and Jianghai Hu 1 1 School of Electrical and Computer Engineering, Purdue University,
More informationLecture 10 Linear Quadratic Stochastic Control with Partial State Observation
EE363 Winter 2008-09 Lecture 10 Linear Quadratic Stochastic Control with Partial State Observation partially observed linear-quadratic stochastic control problem estimation-control separation principle
More informationLQR, Kalman Filter, and LQG. Postgraduate Course, M.Sc. Electrical Engineering Department College of Engineering University of Salahaddin
LQR, Kalman Filter, and LQG Postgraduate Course, M.Sc. Electrical Engineering Department College of Engineering University of Salahaddin May 2015 Linear Quadratic Regulator (LQR) Consider a linear system
More informationOptimal Control Theory
Optimal Control Theory The theory Optimal control theory is a mature mathematical discipline which provides algorithms to solve various control problems The elaborate mathematical machinery behind optimal
More informationRobotics. Control Theory. Marc Toussaint U Stuttgart
Robotics Control Theory Topics in control theory, optimal control, HJB equation, infinite horizon case, Linear-Quadratic optimal control, Riccati equations (differential, algebraic, discrete-time), controllability,
More informationECE7850 Lecture 9. Model Predictive Control: Computational Aspects
ECE785 ECE785 Lecture 9 Model Predictive Control: Computational Aspects Model Predictive Control for Constrained Linear Systems Online Solution to Linear MPC Multiparametric Programming Explicit Linear
More informationModel Predictive Control Short Course Regulation
Model Predictive Control Short Course Regulation James B. Rawlings Michael J. Risbeck Nishith R. Patel Department of Chemical and Biological Engineering Copyright c 2017 by James B. Rawlings Milwaukee,
More informationEE C128 / ME C134 Feedback Control Systems
EE C128 / ME C134 Feedback Control Systems Lecture Additional Material Introduction to Model Predictive Control Maximilian Balandat Department of Electrical Engineering & Computer Science University of
More informationOptimal Control. Lecture 18. Hamilton-Jacobi-Bellman Equation, Cont. John T. Wen. March 29, Ref: Bryson & Ho Chapter 4.
Optimal Control Lecture 18 Hamilton-Jacobi-Bellman Equation, Cont. John T. Wen Ref: Bryson & Ho Chapter 4. March 29, 2004 Outline Hamilton-Jacobi-Bellman (HJB) Equation Iterative solution of HJB Equation
More informationAutomatic Control 2. Nonlinear systems. Prof. Alberto Bemporad. University of Trento. Academic year
Automatic Control 2 Nonlinear systems Prof. Alberto Bemporad University of Trento Academic year 2010-2011 Prof. Alberto Bemporad (University of Trento) Automatic Control 2 Academic year 2010-2011 1 / 18
More informationLinear-Quadratic Optimal Control: Full-State Feedback
Chapter 4 Linear-Quadratic Optimal Control: Full-State Feedback 1 Linear quadratic optimization is a basic method for designing controllers for linear (and often nonlinear) dynamical systems and is actually
More informationESC794: Special Topics: Model Predictive Control
ESC794: Special Topics: Model Predictive Control Discrete-Time Systems Hanz Richter, Professor Mechanical Engineering Department Cleveland State University Discrete-Time vs. Sampled-Data Systems A continuous-time
More informationLecture 4 Continuous time linear quadratic regulator
EE363 Winter 2008-09 Lecture 4 Continuous time linear quadratic regulator continuous-time LQR problem dynamic programming solution Hamiltonian system and two point boundary value problem infinite horizon
More informationQuadratic Stability of Dynamical Systems. Raktim Bhattacharya Aerospace Engineering, Texas A&M University
.. Quadratic Stability of Dynamical Systems Raktim Bhattacharya Aerospace Engineering, Texas A&M University Quadratic Lyapunov Functions Quadratic Stability Dynamical system is quadratically stable if
More informationESC794: Special Topics: Model Predictive Control
ESC794: Special Topics: Model Predictive Control Nonlinear MPC Analysis : Part 1 Reference: Nonlinear Model Predictive Control (Ch.3), Grüne and Pannek Hanz Richter, Professor Mechanical Engineering Department
More informationLecture 2: Discrete-time Linear Quadratic Optimal Control
ME 33, U Berkeley, Spring 04 Xu hen Lecture : Discrete-time Linear Quadratic Optimal ontrol Big picture Example onvergence of finite-time LQ solutions Big picture previously: dynamic programming and finite-horizon
More informationLinear-Quadratic-Gaussian (LQG) Controllers and Kalman Filters
Linear-Quadratic-Gaussian (LQG) Controllers and Kalman Filters Emo Todorov Applied Mathematics and Computer Science & Engineering University of Washington Winter 204 Emo Todorov (UW) AMATH/CSE 579, Winter
More informationLinear Quadratic Regulator (LQR) Design I
Lecture 7 Linear Quadratic Regulator LQR) Design I Dr. Radhakant Padhi Asst. Proessor Dept. o Aerospace Engineering Indian Institute o Science - Bangalore LQR Design: Problem Objective o drive the state
More informationProblem 1 Cost of an Infinite Horizon LQR
THE UNIVERSITY OF TEXAS AT SAN ANTONIO EE 5243 INTRODUCTION TO CYBER-PHYSICAL SYSTEMS H O M E W O R K # 5 Ahmad F. Taha October 12, 215 Homework Instructions: 1. Type your solutions in the LATEX homework
More informationCDS 110b: Lecture 2-1 Linear Quadratic Regulators
CDS 110b: Lecture 2-1 Linear Quadratic Regulators Richard M. Murray 11 January 2006 Goals: Derive the linear quadratic regulator and demonstrate its use Reading: Friedland, Chapter 9 (different derivation,
More informationEE363 homework 2 solutions
EE363 Prof. S. Boyd EE363 homework 2 solutions. Derivative of matrix inverse. Suppose that X : R R n n, and that X(t is invertible. Show that ( d d dt X(t = X(t dt X(t X(t. Hint: differentiate X(tX(t =
More informationLecture 9: Discrete-Time Linear Quadratic Regulator Finite-Horizon Case
Lecture 9: Discrete-Time Linear Quadratic Regulator Finite-Horizon Case Dr. Burak Demirel Faculty of Electrical Engineering and Information Technology, University of Paderborn December 15, 2015 2 Previous
More information5. Observer-based Controller Design
EE635 - Control System Theory 5. Observer-based Controller Design Jitkomut Songsiri state feedback pole-placement design regulation and tracking state observer feedback observer design LQR and LQG 5-1
More informationHomework Solution # 3
ECSE 644 Optimal Control Feb, 4 Due: Feb 17, 4 (Tuesday) Homework Solution # 3 1 (5%) Consider the discrete nonlinear control system in Homework # For the optimal control and trajectory that you have found
More informationAsymptotic stability and transient optimality of economic MPC without terminal conditions
Asymptotic stability and transient optimality of economic MPC without terminal conditions Lars Grüne 1, Marleen Stieler 2 Mathematical Institute, University of Bayreuth, 95440 Bayreuth, Germany Abstract
More informationOutline. Linear regulation and state estimation (LQR and LQE) Linear differential equations. Discrete time linear difference equations
Outline Linear regulation and state estimation (LQR and LQE) James B. Rawlings Department of Chemical and Biological Engineering 1 Linear Quadratic Regulator Constraints The Infinite Horizon LQ Problem
More informationComputational Issues in Nonlinear Dynamics and Control
Computational Issues in Nonlinear Dynamics and Control Arthur J. Krener ajkrener@ucdavis.edu Supported by AFOSR and NSF Typical Problems Numerical Computation of Invariant Manifolds Typical Problems Numerical
More informationIEOR 265 Lecture 14 (Robust) Linear Tube MPC
IEOR 265 Lecture 14 (Robust) Linear Tube MPC 1 LTI System with Uncertainty Suppose we have an LTI system in discrete time with disturbance: x n+1 = Ax n + Bu n + d n, where d n W for a bounded polytope
More informationInfinite-Horizon Switched LQR Problems in Discrete Time: A Suboptimal Algorithm With Performance Analysis
Infinite-Horizon Switched LQR Problems in Discrete Time: A Suboptimal Algorithm With Performance Analysis Wei Zhang, Jianghai Hu, and Alessandro Abate Abstract This paper studies the quadratic regulation
More informationEE363 homework 8 solutions
EE363 Prof. S. Boyd EE363 homework 8 solutions 1. Lyapunov condition for passivity. The system described by ẋ = f(x, u), y = g(x), x() =, with u(t), y(t) R m, is said to be passive if t u(τ) T y(τ) dτ
More informationRobotics: Science & Systems [Topic 6: Control] Prof. Sethu Vijayakumar Course webpage:
Robotics: Science & Systems [Topic 6: Control] Prof. Sethu Vijayakumar Course webpage: http://wcms.inf.ed.ac.uk/ipab/rss Control Theory Concerns controlled systems of the form: and a controller of the
More informationGeneralized Input-to-State l 2 -Gains of Discrete-Time Switched Linear Control Systems
Generalized Input-to-State l 2 -Gains of Discrete-Time Switched Linear Control Systems Jianghai Hu, Jinglai Shen, and Vamsi Putta February 9, 205 Abstract A generalized notion of input-to-state l 2 -gain
More information6. Linear Quadratic Regulator Control
EE635 - Control System Theory 6. Linear Quadratic Regulator Control Jitkomut Songsiri algebraic Riccati Equation (ARE) infinite-time LQR (continuous) Hamiltonian matrix gain margin of LQR 6-1 Algebraic
More informationEE363 Review Session 1: LQR, Controllability and Observability
EE363 Review Session : LQR, Controllability and Observability In this review session we ll work through a variation on LQR in which we add an input smoothness cost, in addition to the usual penalties on
More informationOptimal Control. McGill COMP 765 Oct 3 rd, 2017
Optimal Control McGill COMP 765 Oct 3 rd, 2017 Classical Control Quiz Question 1: Can a PID controller be used to balance an inverted pendulum: A) That starts upright? B) That must be swung-up (perhaps
More informationModels for Control and Verification
Outline Models for Control and Verification Ian Mitchell Department of Computer Science The University of British Columbia Classes of models Well-posed models Difference Equations Nonlinear Ordinary Differential
More informationHamilton-Jacobi-Bellman Equation Feb 25, 2008
Hamilton-Jacobi-Bellman Equation Feb 25, 2008 What is it? The Hamilton-Jacobi-Bellman (HJB) equation is the continuous-time analog to the discrete deterministic dynamic programming algorithm Discrete VS
More informationHybrid Control and Switched Systems. Lecture #9 Analysis tools for hybrid systems: Impact maps
Hybrid Control and Switched Systems Lecture #9 Analysis tools for hybrid systems: Impact maps João P. Hespanha University of California at Santa Barbara Summary Analysis tools for hybrid systems Impact
More informationA Globally Stabilizing Receding Horizon Controller for Neutrally Stable Linear Systems with Input Constraints 1
A Globally Stabilizing Receding Horizon Controller for Neutrally Stable Linear Systems with Input Constraints 1 Ali Jadbabaie, Claudio De Persis, and Tae-Woong Yoon 2 Department of Electrical Engineering
More informationFormula Sheet for Optimal Control
Formula Sheet for Optimal Control Division of Optimization and Systems Theory Royal Institute of Technology 144 Stockholm, Sweden 23 December 1, 29 1 Dynamic Programming 11 Discrete Dynamic Programming
More informationLinear Systems. Manfred Morari Melanie Zeilinger. Institut für Automatik, ETH Zürich Institute for Dynamic Systems and Control, ETH Zürich
Linear Systems Manfred Morari Melanie Zeilinger Institut für Automatik, ETH Zürich Institute for Dynamic Systems and Control, ETH Zürich Spring Semester 2016 Linear Systems M. Morari, M. Zeilinger - Spring
More informationValue and Policy Iteration
Chapter 7 Value and Policy Iteration 1 For infinite horizon problems, we need to replace our basic computational tool, the DP algorithm, which we used to compute the optimal cost and policy for finite
More informationEL2520 Control Theory and Practice
EL2520 Control Theory and Practice Lecture 8: Linear quadratic control Mikael Johansson School of Electrical Engineering KTH, Stockholm, Sweden Linear quadratic control Allows to compute the controller
More information6.241 Dynamic Systems and Control
6.241 Dynamic Systems and Control Lecture 24: H2 Synthesis Emilio Frazzoli Aeronautics and Astronautics Massachusetts Institute of Technology May 4, 2011 E. Frazzoli (MIT) Lecture 24: H 2 Synthesis May
More informationUCLA Chemical Engineering. Process & Control Systems Engineering Laboratory
Constrained Innite-Time Nonlinear Quadratic Optimal Control V. Manousiouthakis D. Chmielewski Chemical Engineering Department UCLA 1998 AIChE Annual Meeting Outline Unconstrained Innite-Time Nonlinear
More information1 Markov decision processes
2.997 Decision-Making in Large-Scale Systems February 4 MI, Spring 2004 Handout #1 Lecture Note 1 1 Markov decision processes In this class we will study discrete-time stochastic systems. We can describe
More informationLinear Quadratic Regulator (LQR) I
Optimal Control, Guidance and Estimation Lecture Linear Quadratic Regulator (LQR) I Pro. Radhakant Padhi Dept. o Aerospace Engineering Indian Institute o Science - Bangalore Generic Optimal Control Problem
More informationSuppose that we have a specific single stage dynamic system governed by the following equation:
Dynamic Optimisation Discrete Dynamic Systems A single stage example Suppose that we have a specific single stage dynamic system governed by the following equation: x 1 = ax 0 + bu 0, x 0 = x i (1) where
More information5. Solving the Bellman Equation
5. Solving the Bellman Equation In the next two lectures, we will look at several methods to solve Bellman s Equation (BE) for the stochastic shortest path problem: Value Iteration, Policy Iteration and
More informationTopic # /31 Feedback Control Systems. Analysis of Nonlinear Systems Lyapunov Stability Analysis
Topic # 16.30/31 Feedback Control Systems Analysis of Nonlinear Systems Lyapunov Stability Analysis Fall 010 16.30/31 Lyapunov Stability Analysis Very general method to prove (or disprove) stability of
More informationModule 07 Controllability and Controller Design of Dynamical LTI Systems
Module 07 Controllability and Controller Design of Dynamical LTI Systems Ahmad F. Taha EE 5143: Linear Systems and Control Email: ahmad.taha@utsa.edu Webpage: http://engineering.utsa.edu/ataha October
More informationLyapunov Stability Theory
Lyapunov Stability Theory Peter Al Hokayem and Eduardo Gallestey March 16, 2015 1 Introduction In this lecture we consider the stability of equilibrium points of autonomous nonlinear systems, both in continuous
More informationTime-Invariant Linear Quadratic Regulators Robert Stengel Optimal Control and Estimation MAE 546 Princeton University, 2015
Time-Invariant Linear Quadratic Regulators Robert Stengel Optimal Control and Estimation MAE 546 Princeton University, 15 Asymptotic approach from time-varying to constant gains Elimination of cross weighting
More informationReinforcement Learning and Optimal Control. ASU, CSE 691, Winter 2019
Reinforcement Learning and Optimal Control ASU, CSE 691, Winter 2019 Dimitri P. Bertsekas dimitrib@mit.edu Lecture 8 Bertsekas Reinforcement Learning 1 / 21 Outline 1 Review of Infinite Horizon Problems
More informationStochastic resource allocation Boyd, Akuiyibo
Stochastic resource allocation Boyd, Akuiyibo ACHIEVEMENT DESCRIPTION STATUS QUO Current resource allocation research focus on iterative methods. These automatically adapt to changing data assuming they
More informationTime-Invariant Linear Quadratic Regulators!
Time-Invariant Linear Quadratic Regulators Robert Stengel Optimal Control and Estimation MAE 546 Princeton University, 17 Asymptotic approach from time-varying to constant gains Elimination of cross weighting
More informationSemidefinite Programming Duality and Linear Time-invariant Systems
Semidefinite Programming Duality and Linear Time-invariant Systems Venkataramanan (Ragu) Balakrishnan School of ECE, Purdue University 2 July 2004 Workshop on Linear Matrix Inequalities in Control LAAS-CNRS,
More informationEE363 homework 7 solutions
EE363 Prof. S. Boyd EE363 homework 7 solutions 1. Gain margin for a linear quadratic regulator. Let K be the optimal state feedback gain for the LQR problem with system ẋ = Ax + Bu, state cost matrix Q,
More informationLecture 6. Foundations of LMIs in System and Control Theory
Lecture 6. Foundations of LMIs in System and Control Theory Ivan Papusha CDS270 2: Mathematical Methods in Control and System Engineering May 4, 2015 1 / 22 Logistics hw5 due this Wed, May 6 do an easy
More informationZ i Q ij Z j. J(x, φ; U) = X T φ(t ) 2 h + where h k k, H(t) k k and R(t) r r are nonnegative definite matrices (R(t) is uniformly in t nonsingular).
2. LINEAR QUADRATIC DETERMINISTIC PROBLEM Notations: For a vector Z, Z = Z, Z is the Euclidean norm here Z, Z = i Z2 i is the inner product; For a vector Z and nonnegative definite matrix Q, Z Q = Z, QZ
More informationHybrid Control and Switched Systems. Lecture #11 Stability of switched system: Arbitrary switching
Hybrid Control and Switched Systems Lecture #11 Stability of switched system: Arbitrary switching João P. Hespanha University of California at Santa Barbara Stability under arbitrary switching Instability
More informationNumerical Optimal Control Overview. Moritz Diehl
Numerical Optimal Control Overview Moritz Diehl Simplified Optimal Control Problem in ODE path constraints h(x, u) 0 initial value x0 states x(t) terminal constraint r(x(t )) 0 controls u(t) 0 t T minimize
More information4F3 - Predictive Control
4F3 Predictive Control - Lecture 3 p 1/21 4F3 - Predictive Control Lecture 3 - Predictive Control with Constraints Jan Maciejowski jmm@engcamacuk 4F3 Predictive Control - Lecture 3 p 2/21 Constraints on
More informationNonlinear reference tracking: An economic model predictive control perspective
1 Nonlinear reference tracking: An economic model predictive control perspective Johannes Köhler, Matthias A. Müller, Frank Allgöwer Abstract In this paper, we study the system theoretic properties of
More informationMarkov Decision Processes and Dynamic Programming
Markov Decision Processes and Dynamic Programming A. LAZARIC (SequeL Team @INRIA-Lille) Ecole Centrale - Option DAD SequeL INRIA Lille EC-RL Course In This Lecture A. LAZARIC Markov Decision Processes
More informationEfficient robust optimization for robust control with constraints Paul Goulart, Eric Kerrigan and Danny Ralph
Efficient robust optimization for robust control with constraints p. 1 Efficient robust optimization for robust control with constraints Paul Goulart, Eric Kerrigan and Danny Ralph Efficient robust optimization
More informationInput to state Stability
Input to state Stability Mini course, Universität Stuttgart, November 2004 Lars Grüne, Mathematisches Institut, Universität Bayreuth Part IV: Applications ISS Consider with solutions ϕ(t, x, w) ẋ(t) =
More informationOn the Stability of the Best Reply Map for Noncooperative Differential Games
On the Stability of the Best Reply Map for Noncooperative Differential Games Alberto Bressan and Zipeng Wang Department of Mathematics, Penn State University, University Park, PA, 68, USA DPMMS, University
More informationLinear Quadratic Regulator (LQR) Design II
Lecture 8 Linear Quadratic Regulator LQR) Design II Dr. Radhakant Padhi Asst. Professor Dept. of Aerospace Engineering Indian Institute of Science - Bangalore Outline Stability and Robustness properties
More informationChapter 3 Nonlinear Model Predictive Control
Chapter 3 Nonlinear Model Predictive Control In this chapter, we introduce the nonlinear model predictive control algorithm in a rigorous way. We start by defining a basic NMPC algorithm for constant reference
More informationOptimal control of nonlinear systems with input constraints using linear time varying approximations
ISSN 1392-5113 Nonlinear Analysis: Modelling and Control, 216, Vol. 21, No. 3, 4 412 http://dx.doi.org/1.15388/na.216.3.7 Optimal control of nonlinear systems with input constraints using linear time varying
More informationPostface to Model Predictive Control: Theory and Design
Postface to Model Predictive Control: Theory and Design J. B. Rawlings and D. Q. Mayne August 19, 2012 The goal of this postface is to point out and comment upon recent MPC papers and issues pertaining
More informationStability of Feedback Solutions for Infinite Horizon Noncooperative Differential Games
Stability of Feedback Solutions for Infinite Horizon Noncooperative Differential Games Alberto Bressan ) and Khai T. Nguyen ) *) Department of Mathematics, Penn State University **) Department of Mathematics,
More informationLinearly-Solvable Stochastic Optimal Control Problems
Linearly-Solvable Stochastic Optimal Control Problems Emo Todorov Applied Mathematics and Computer Science & Engineering University of Washington Winter 2014 Emo Todorov (UW) AMATH/CSE 579, Winter 2014
More informationChapter 2 Optimal Control Problem
Chapter 2 Optimal Control Problem Optimal control of any process can be achieved either in open or closed loop. In the following two chapters we concentrate mainly on the first class. The first chapter
More informationMATH4406 (Control Theory) Unit 1: Introduction to Control Prepared by Yoni Nazarathy, August 11, 2014
MATH4406 (Control Theory) Unit 1: Introduction to Control Prepared by Yoni Nazarathy, August 11, 2014 Unit Outline The many faces of Control Theory A bit of jargon, history and applications Focus on inherently
More informationEnlarged terminal sets guaranteeing stability of receding horizon control
Enlarged terminal sets guaranteeing stability of receding horizon control J.A. De Doná a, M.M. Seron a D.Q. Mayne b G.C. Goodwin a a School of Electrical Engineering and Computer Science, The University
More informationAlberto Bressan. Department of Mathematics, Penn State University
Non-cooperative Differential Games A Homotopy Approach Alberto Bressan Department of Mathematics, Penn State University 1 Differential Games d dt x(t) = G(x(t), u 1(t), u 2 (t)), x(0) = y, u i (t) U i
More informationSubgradients. subgradients and quasigradients. subgradient calculus. optimality conditions via subgradients. directional derivatives
Subgradients subgradients and quasigradients subgradient calculus optimality conditions via subgradients directional derivatives Prof. S. Boyd, EE392o, Stanford University Basic inequality recall basic
More informationExtensions and applications of LQ
Extensions and applications of LQ 1 Discrete time systems 2 Assigning closed loop pole location 3 Frequency shaping LQ Regulator for Discrete Time Systems Consider the discrete time system: x(k + 1) =
More informationLinear conic optimization for nonlinear optimal control
Linear conic optimization for nonlinear optimal control Didier Henrion 1,2,3, Edouard Pauwels 1,2 Draft of July 15, 2014 Abstract Infinite-dimensional linear conic formulations are described for nonlinear
More informationThe Inverted Pendulum
Lab 1 The Inverted Pendulum Lab Objective: We will set up the LQR optimal control problem for the inverted pendulum and compute the solution numerically. Think back to your childhood days when, for entertainment
More informationStochastic and Adaptive Optimal Control
Stochastic and Adaptive Optimal Control Robert Stengel Optimal Control and Estimation, MAE 546 Princeton University, 2018! Nonlinear systems with random inputs and perfect measurements! Stochastic neighboring-optimal
More informationEML5311 Lyapunov Stability & Robust Control Design
EML5311 Lyapunov Stability & Robust Control Design 1 Lyapunov Stability criterion In Robust control design of nonlinear uncertain systems, stability theory plays an important role in engineering systems.
More informationObjective. 1 Specification/modeling of the controlled system. 2 Specification of a performance criterion
Optimal Control Problem Formulation Optimal Control Lectures 17-18: Problem Formulation Benoît Chachuat Department of Chemical Engineering Spring 2009 Objective Determine the control
More informationChapter 3. LQ, LQG and Control System Design. Dutch Institute of Systems and Control
Chapter 3 LQ, LQG and Control System H 2 Design Overview LQ optimization state feedback LQG optimization output feedback H 2 optimization non-stochastic version of LQG Application to feedback system design
More informationLinear Quadratic Zero-Sum Two-Person Differential Games Pierre Bernhard June 15, 2013
Linear Quadratic Zero-Sum Two-Person Differential Games Pierre Bernhard June 15, 2013 Abstract As in optimal control theory, linear quadratic (LQ) differential games (DG) can be solved, even in high dimension,
More informationLecture 20: Linear Dynamics and LQG
CSE599i: Online and Adaptive Machine Learning Winter 2018 Lecturer: Kevin Jamieson Lecture 20: Linear Dynamics and LQG Scribes: Atinuke Ademola-Idowu, Yuanyuan Shi Disclaimer: These notes have not been
More information