Lecture 9: Discrete-Time Linear Quadratic Regulator Finite-Horizon Case

Size: px
Start display at page:

Download "Lecture 9: Discrete-Time Linear Quadratic Regulator Finite-Horizon Case"

Transcription

1 Lecture 9: Discrete-Time Linear Quadratic Regulator Finite-Horizon Case Dr. Burak Demirel Faculty of Electrical Engineering and Information Technology, University of Paderborn December 15, 2015

2 2 Previous lectures In the previous lectures, we considered the synthesis problem, solved by pole-placement techniques (i.e., the main design parameter is the location of poles) These techniques are limited to SISO systems.

3 LQR problem: Background Given a discrete-time LTI system x k+1 = Ax k + Bu k y k = Cx k (1) LQR problem: Find the control inputs π = {u 0,, u N 1 } that makes the following criterion as small as possible: where N 1 J(π) = x N Px N + x k Qx k + u k Ru k (2) k=0 Q S 0, P S 0, R S >0, are given state cost, terminal cost, and input cost matrices, respectively.

4 4 Multi-objective interpretation Common form for Q and R: Q = P = C C R = ρi where ρ > 0. Then, the cost is where y k = Cx k. J(π) = N k=0 N 1 y k 2 +ρ k=0 u k 2 ρ gives relative weighting of output and input norm.

5 5 Multi-objective interpretation LQR quadratic cost can be written as: J = J out + ρj in. The term J out = N y k 2 k=0 provides a measure of the output energy, and the term J in = N 1 k=0 u k 2 provides a measurement of the control energy. There are competing objectives; we want both small.

6 Trade-off between conflicting goals π 3 is worse than π 2 on both J out and J in π 1 is better than π 2 in J in but worse in J out J out J out + ρj in = J π 1 π 2 π 3 (Constant) J in When ρ is much larger than 1, the most effective way to decrease J is to employ a small control input, at the expense of a large output. When ρ is much smaller than 1, the most effective way to decrease J is to obtain a very small output, even if this is achieved at the expense of employing a large control input.

7 7 LQR via least-squares LQR can be formulated (and solved) as a least-squares problem X = [x 0,, x N ] is a linear function of x 0 and U = [u 0,, u N 1 ]: 0 x 0 B 0 I u 0. = AB B A. +. x 0 x N.. 0 u N 1 A N 1 B A N 2 A N B B express as X = GU + Hx 0, where G R Nn Nm, H R Nn n

8 8 LQR via least-squares Express linear-quadratic cost as J(U) = diag(q 1/2,, Q 1/2, P 1/2 )(GU + Hx 0 ) 2 + diag(r 1/2,, R 1/2 )U 2 This is only a least-squares problem. This solution method requires solving a least-squares problem with size N(n + m) Nm (see Boyd) Using a naive method (e.g., QR factorization), cost is O(N 3 nm 2 ) (see Boyd)

9 9 Finite-horizon LQR control Theorem Consider the finite horizon LQR problem. The optimal control u k = (R + B S k+1 B) 1 B S k+1 Ax k is a linear function of the state x k. The matrix S k evolves according to the backward Riccati recursion S k = A S k+1 A + Q A S k+1 B(B S k+1 B + R) 1 B S k+1 A with the initial condition S N = P. Furthermore, the optimal control loss is given by J N = x 0 Qx 0.

10 10 Dynamic programming solution gives an efficient, recursive method to solve LQR least-squares problem; cost is O(Nn 3 ) but a less naive approach to solve the LQR least-squares problem will have the same complexity same idea can be applied to many other problems

11 11 Completing the squares Consider the loss function of the form [ ] [ ] [ ] x Qx Q J(x, u) = xu x, (3) u u and minimize with respect to u. Then, there exists a matrix L satisfying Q xu Q u Q u L = Q xu such that the loss function (3) becomes J(x, u) = x (Q x L Q u L)x + (u + Lx) Q u (u + Lx). (4) Since (4) is quadratic in u and both terms are greater or equal zero, (3) is minimized for u = Lx. The minimum is J min = x (Q x L Q u L)x.

12 12 Proof: Via dynamic programming Define the value function V k : R n R by { N 1 } V k = min x u k,,u i Qx i + u i Ru i + x N Px N N 1 i=k subject to x i+1 = Ax i + Bu i for i {k,, N 1}. V k can be interpreted as the loss-to-go from k to N and is a function of x k at time k. We will find that V k is quadratic, i.e., V k (x) = x S k x, where S k S 0 S k can be found recursively, working from k = N the LQR optimal u is easily expressed in terms of S k

13 13 Proof: Via dynamic programming Loss-to-go with no time left is only final state cost: V N = x N S Nx N where S N = P. Now, we will show that V k is quadratic in x k for all k. For k = N 1, we have { } V N 1 = min x u N 1 Qx N 1 + u N 1 Ru N 1 + V N N 1 Using x N = Ax N 1 + Bu N 1 gives V N 1 = min {x u N 1 Qx N 1 + u N 1 Ru N 1 N 1 } + (Ax N 1 + Bu N 1 ) S N (Ax N 1 + Bu N 1 )

14 Proof: Via dynamic programming Then, we get: V N 1 = min u N 1 [ xn 1 u N 1 ] [ Q + A S N A B S N A ] [ ] xn 1 A S N B R + B S N B u N 1 This function is quadratic in u N 1. Completing the squares, we can write: V N 1 = min {x u N 1( A S N A + Q L N 1 (R + ) B S N B)L N 1 xn 1 N 1 } + (u N 1 + L N 1 x N 1 ) (R + B S N B)(u N 1 + L N 1 x N 1 ) (5) where L N 1 = (R + B S N B) 1 B S N A. To minimize (5), we need to select which gives the minimum loss u N 1 = L N 1 x N 1 V N 1 = x N 1 S N 1x N 1

15 DP algorithm for LQR 1. set S N := P 2. for k = N,, 1, compute S k 1 := A S k A + Q A S k B(R + B S k B) 1 B S k A 3. for k = 0,, N 1, compute L k := (R + B S k+1 B) 1 B S k+1 A 4. for k = 0,, N 1, compute u k := L k x k

16 16 LQ-control of the double integrator Consider a double integrator system (with unity sampling interval): x k+1 = y k = [ 1 [ ] [ ] x k + 2 u k ] x k with initial state x 0 = [1 0], horizon N = 20, and weighting matrices: Q = P = C C, R = ρ where ρ > 0.

17 17 LQ-control of the double integrator Optimal trade-off curve of J out and J in : Jout J in Blue circle denotes ρ = 0.3 while red circle denotes ρ = 10.

18 18 LQ-control of the double integrator yk uk ρ = 0.3 ρ = Number of samples [k]

19 Reference 1. S. Boyd, Linear quadratic regulator: Discrete-time finite horizon, Lecture Notes. 2. K. J. Åström and B. Wittenmark, Computer-Controlled Systems, Prentice Hall, D. P. Bertsekas, Dynamic Programming and Optimal Control, Athena Scientific, 2000.

4F3 - Predictive Control

4F3 - Predictive Control 4F3 Predictive Control - Lecture 2 p 1/23 4F3 - Predictive Control Lecture 2 - Unconstrained Predictive Control Jan Maciejowski jmm@engcamacuk 4F3 Predictive Control - Lecture 2 p 2/23 References Predictive

More information

Optimal control and estimation

Optimal control and estimation Automatic Control 2 Optimal control and estimation Prof. Alberto Bemporad University of Trento Academic year 2010-2011 Prof. Alberto Bemporad (University of Trento) Automatic Control 2 Academic year 2010-2011

More information

Hamilton-Jacobi-Bellman Equation Feb 25, 2008

Hamilton-Jacobi-Bellman Equation Feb 25, 2008 Hamilton-Jacobi-Bellman Equation Feb 25, 2008 What is it? The Hamilton-Jacobi-Bellman (HJB) equation is the continuous-time analog to the discrete deterministic dynamic programming algorithm Discrete VS

More information

4F3 - Predictive Control

4F3 - Predictive Control 4F3 Predictive Control - Lecture 3 p 1/21 4F3 - Predictive Control Lecture 3 - Predictive Control with Constraints Jan Maciejowski jmm@engcamacuk 4F3 Predictive Control - Lecture 3 p 2/21 Constraints on

More information

In: Proc. BENELEARN-98, 8th Belgian-Dutch Conference on Machine Learning, pp 9-46, 998 Linear Quadratic Regulation using Reinforcement Learning Stephan ten Hagen? and Ben Krose Department of Mathematics,

More information

Outline. 1 Linear Quadratic Problem. 2 Constraints. 3 Dynamic Programming Solution. 4 The Infinite Horizon LQ Problem.

Outline. 1 Linear Quadratic Problem. 2 Constraints. 3 Dynamic Programming Solution. 4 The Infinite Horizon LQ Problem. Model Predictive Control Short Course Regulation James B. Rawlings Michael J. Risbeck Nishith R. Patel Department of Chemical and Biological Engineering Copyright c 217 by James B. Rawlings Outline 1 Linear

More information

EE C128 / ME C134 Final Exam Fall 2014

EE C128 / ME C134 Final Exam Fall 2014 EE C128 / ME C134 Final Exam Fall 2014 December 19, 2014 Your PRINTED FULL NAME Your STUDENT ID NUMBER Number of additional sheets 1. No computers, no tablets, no connected device (phone etc.) 2. Pocket

More information

Suppose that we have a specific single stage dynamic system governed by the following equation:

Suppose that we have a specific single stage dynamic system governed by the following equation: Dynamic Optimisation Discrete Dynamic Systems A single stage example Suppose that we have a specific single stage dynamic system governed by the following equation: x 1 = ax 0 + bu 0, x 0 = x i (1) where

More information

Optimal Control. Quadratic Functions. Single variable quadratic function: Multi-variable quadratic function:

Optimal Control. Quadratic Functions. Single variable quadratic function: Multi-variable quadratic function: Optimal Control Control design based on pole-placement has non unique solutions Best locations for eigenvalues are sometimes difficult to determine Linear Quadratic LQ) Optimal control minimizes a quadratic

More information

MATH4406 (Control Theory) Unit 6: The Linear Quadratic Regulator (LQR) and Model Predictive Control (MPC) Prepared by Yoni Nazarathy, Artem

MATH4406 (Control Theory) Unit 6: The Linear Quadratic Regulator (LQR) and Model Predictive Control (MPC) Prepared by Yoni Nazarathy, Artem MATH4406 (Control Theory) Unit 6: The Linear Quadratic Regulator (LQR) and Model Predictive Control (MPC) Prepared by Yoni Nazarathy, Artem Pulemotov, September 12, 2012 Unit Outline Goal 1: Outline linear

More information

Lecture 5 Linear Quadratic Stochastic Control

Lecture 5 Linear Quadratic Stochastic Control EE363 Winter 2008-09 Lecture 5 Linear Quadratic Stochastic Control linear-quadratic stochastic control problem solution via dynamic programming 5 1 Linear stochastic system linear dynamical system, over

More information

Theory in Model Predictive Control :" Constraint Satisfaction and Stability!

Theory in Model Predictive Control : Constraint Satisfaction and Stability! Theory in Model Predictive Control :" Constraint Satisfaction and Stability Colin Jones, Melanie Zeilinger Automatic Control Laboratory, EPFL Example: Cessna Citation Aircraft Linearized continuous-time

More information

ESC794: Special Topics: Model Predictive Control

ESC794: Special Topics: Model Predictive Control ESC794: Special Topics: Model Predictive Control Nonlinear MPC Analysis : Part 1 Reference: Nonlinear Model Predictive Control (Ch.3), Grüne and Pannek Hanz Richter, Professor Mechanical Engineering Department

More information

Homework Solution # 3

Homework Solution # 3 ECSE 644 Optimal Control Feb, 4 Due: Feb 17, 4 (Tuesday) Homework Solution # 3 1 (5%) Consider the discrete nonlinear control system in Homework # For the optimal control and trajectory that you have found

More information

Lecture 2: Discrete-time Linear Quadratic Optimal Control

Lecture 2: Discrete-time Linear Quadratic Optimal Control ME 33, U Berkeley, Spring 04 Xu hen Lecture : Discrete-time Linear Quadratic Optimal ontrol Big picture Example onvergence of finite-time LQ solutions Big picture previously: dynamic programming and finite-horizon

More information

Optimal Polynomial Control for Discrete-Time Systems

Optimal Polynomial Control for Discrete-Time Systems 1 Optimal Polynomial Control for Discrete-Time Systems Prof Guy Beale Electrical and Computer Engineering Department George Mason University Fairfax, Virginia Correspondence concerning this paper should

More information

Static and Dynamic Optimization (42111)

Static and Dynamic Optimization (42111) Static and Dynamic Optimization (421) Niels Kjølstad Poulsen Build. 0b, room 01 Section for Dynamical Systems Dept. of Applied Mathematics and Computer Science The Technical University of Denmark Email:

More information

Course Outline. Higher Order Poles: Example. Higher Order Poles. Amme 3500 : System Dynamics & Control. State Space Design. 1 G(s) = s(s + 2)(s +10)

Course Outline. Higher Order Poles: Example. Higher Order Poles. Amme 3500 : System Dynamics & Control. State Space Design. 1 G(s) = s(s + 2)(s +10) Amme 35 : System Dynamics Control State Space Design Course Outline Week Date Content Assignment Notes 1 1 Mar Introduction 2 8 Mar Frequency Domain Modelling 3 15 Mar Transient Performance and the s-plane

More information

OPTIMAL CONTROL. Sadegh Bolouki. Lecture slides for ECE 515. University of Illinois, Urbana-Champaign. Fall S. Bolouki (UIUC) 1 / 28

OPTIMAL CONTROL. Sadegh Bolouki. Lecture slides for ECE 515. University of Illinois, Urbana-Champaign. Fall S. Bolouki (UIUC) 1 / 28 OPTIMAL CONTROL Sadegh Bolouki Lecture slides for ECE 515 University of Illinois, Urbana-Champaign Fall 2016 S. Bolouki (UIUC) 1 / 28 (Example from Optimal Control Theory, Kirk) Objective: To get from

More information

EN Applied Optimal Control Lecture 8: Dynamic Programming October 10, 2018

EN Applied Optimal Control Lecture 8: Dynamic Programming October 10, 2018 EN530.603 Applied Optimal Control Lecture 8: Dynamic Programming October 0, 08 Lecturer: Marin Kobilarov Dynamic Programming (DP) is conerned with the computation of an optimal policy, i.e. an optimal

More information

Roots and Coefficients Polynomials Preliminary Maths Extension 1

Roots and Coefficients Polynomials Preliminary Maths Extension 1 Preliminary Maths Extension Question If, and are the roots of x 5x x 0, find the following. (d) (e) Question If p, q and r are the roots of x x x 4 0, evaluate the following. pq r pq qr rp p q q r r p

More information

Time-Invariant Linear Quadratic Regulators Robert Stengel Optimal Control and Estimation MAE 546 Princeton University, 2015

Time-Invariant Linear Quadratic Regulators Robert Stengel Optimal Control and Estimation MAE 546 Princeton University, 2015 Time-Invariant Linear Quadratic Regulators Robert Stengel Optimal Control and Estimation MAE 546 Princeton University, 15 Asymptotic approach from time-varying to constant gains Elimination of cross weighting

More information

Australian Journal of Basic and Applied Sciences, 3(4): , 2009 ISSN Modern Control Design of Power System

Australian Journal of Basic and Applied Sciences, 3(4): , 2009 ISSN Modern Control Design of Power System Australian Journal of Basic and Applied Sciences, 3(4): 4267-4273, 29 ISSN 99-878 Modern Control Design of Power System Atef Saleh Othman Al-Mashakbeh Tafila Technical University, Electrical Engineering

More information

State Regulator. Advanced Control. design of controllers using pole placement and LQ design rules

State Regulator. Advanced Control. design of controllers using pole placement and LQ design rules Advanced Control State Regulator Scope design of controllers using pole placement and LQ design rules Keywords pole placement, optimal control, LQ regulator, weighting matrixes Prerequisites Contact state

More information

HPMPC - A new software package with efficient solvers for Model Predictive Control

HPMPC - A new software package with efficient solvers for Model Predictive Control - A new software package with efficient solvers for Model Predictive Control Technical University of Denmark CITIES Second General Consortium Meeting, DTU, Lyngby Campus, 26-27 May 2015 Introduction Model

More information

Problem 1 Cost of an Infinite Horizon LQR

Problem 1 Cost of an Infinite Horizon LQR THE UNIVERSITY OF TEXAS AT SAN ANTONIO EE 5243 INTRODUCTION TO CYBER-PHYSICAL SYSTEMS H O M E W O R K # 5 Ahmad F. Taha October 12, 215 Homework Instructions: 1. Type your solutions in the LATEX homework

More information

Model Predictive Control Short Course Regulation

Model Predictive Control Short Course Regulation Model Predictive Control Short Course Regulation James B. Rawlings Michael J. Risbeck Nishith R. Patel Department of Chemical and Biological Engineering Copyright c 2017 by James B. Rawlings Milwaukee,

More information

EE221A Linear System Theory Final Exam

EE221A Linear System Theory Final Exam EE221A Linear System Theory Final Exam Professor C. Tomlin Department of Electrical Engineering and Computer Sciences, UC Berkeley Fall 2016 12/16/16, 8-11am Your answers must be supported by analysis,

More information

Linear-Quadratic Optimal Control: Full-State Feedback

Linear-Quadratic Optimal Control: Full-State Feedback Chapter 4 Linear-Quadratic Optimal Control: Full-State Feedback 1 Linear quadratic optimization is a basic method for designing controllers for linear (and often nonlinear) dynamical systems and is actually

More information

ECE7850 Lecture 7. Discrete Time Optimal Control and Dynamic Programming

ECE7850 Lecture 7. Discrete Time Optimal Control and Dynamic Programming ECE7850 Lecture 7 Discrete Time Optimal Control and Dynamic Programming Discrete Time Optimal control Problems Short Introduction to Dynamic Programming Connection to Stabilization Problems 1 DT nonlinear

More information

Time-Invariant Linear Quadratic Regulators!

Time-Invariant Linear Quadratic Regulators! Time-Invariant Linear Quadratic Regulators Robert Stengel Optimal Control and Estimation MAE 546 Princeton University, 17 Asymptotic approach from time-varying to constant gains Elimination of cross weighting

More information

Control Systems. Design of State Feedback Control.

Control Systems. Design of State Feedback Control. Control Systems Design of State Feedback Control chibum@seoultech.ac.kr Outline Design of State feedback control Dominant pole design Symmetric root locus (linear quadratic regulation) 2 Selection of closed-loop

More information

Numerical Methods for Model Predictive Control. Jing Yang

Numerical Methods for Model Predictive Control. Jing Yang Numerical Methods for Model Predictive Control Jing Yang Kongens Lyngby February 26, 2008 Technical University of Denmark Informatics and Mathematical Modelling Building 321, DK-2800 Kongens Lyngby, Denmark

More information

Extensions and applications of LQ

Extensions and applications of LQ Extensions and applications of LQ 1 Discrete time systems 2 Assigning closed loop pole location 3 Frequency shaping LQ Regulator for Discrete Time Systems Consider the discrete time system: x(k + 1) =

More information

Linear Quadratic Regulator (LQR) Design I

Linear Quadratic Regulator (LQR) Design I Lecture 7 Linear Quadratic Regulator LQR) Design I Dr. Radhakant Padhi Asst. Proessor Dept. o Aerospace Engineering Indian Institute o Science - Bangalore LQR Design: Problem Objective o drive the state

More information

Linear Quadratic Regulator (LQR) I

Linear Quadratic Regulator (LQR) I Optimal Control, Guidance and Estimation Lecture Linear Quadratic Regulator (LQR) I Pro. Radhakant Padhi Dept. o Aerospace Engineering Indian Institute o Science - Bangalore Generic Optimal Control Problem

More information

Quadratic Stability of Dynamical Systems. Raktim Bhattacharya Aerospace Engineering, Texas A&M University

Quadratic Stability of Dynamical Systems. Raktim Bhattacharya Aerospace Engineering, Texas A&M University .. Quadratic Stability of Dynamical Systems Raktim Bhattacharya Aerospace Engineering, Texas A&M University Quadratic Lyapunov Functions Quadratic Stability Dynamical system is quadratically stable if

More information

Linear-Quadratic-Gaussian (LQG) Controllers and Kalman Filters

Linear-Quadratic-Gaussian (LQG) Controllers and Kalman Filters Linear-Quadratic-Gaussian (LQG) Controllers and Kalman Filters Emo Todorov Applied Mathematics and Computer Science & Engineering University of Washington Winter 204 Emo Todorov (UW) AMATH/CSE 579, Winter

More information

Lecture 20: Linear Dynamics and LQG

Lecture 20: Linear Dynamics and LQG CSE599i: Online and Adaptive Machine Learning Winter 2018 Lecturer: Kevin Jamieson Lecture 20: Linear Dynamics and LQG Scribes: Atinuke Ademola-Idowu, Yuanyuan Shi Disclaimer: These notes have not been

More information

Lecture 10 Linear Quadratic Stochastic Control with Partial State Observation

Lecture 10 Linear Quadratic Stochastic Control with Partial State Observation EE363 Winter 2008-09 Lecture 10 Linear Quadratic Stochastic Control with Partial State Observation partially observed linear-quadratic stochastic control problem estimation-control separation principle

More information

Linear Quadratic Regulator (LQR) Design II

Linear Quadratic Regulator (LQR) Design II Lecture 8 Linear Quadratic Regulator LQR) Design II Dr. Radhakant Padhi Asst. Professor Dept. of Aerospace Engineering Indian Institute of Science - Bangalore Outline Stability and Robustness properties

More information

MODERN CONTROL DESIGN

MODERN CONTROL DESIGN CHAPTER 8 MODERN CONTROL DESIGN The classical design techniques of Chapters 6 and 7 are based on the root-locus and frequency response that utilize only the plant output for feedback with a dynamic controller

More information

Structured State Space Realizations for SLS Distributed Controllers

Structured State Space Realizations for SLS Distributed Controllers Structured State Space Realizations for SLS Distributed Controllers James Anderson and Nikolai Matni Abstract In recent work the system level synthesis (SLS) paradigm has been shown to provide a truly

More information

Adaptive linear quadratic control using policy. iteration. Steven J. Bradtke. University of Massachusetts.

Adaptive linear quadratic control using policy. iteration. Steven J. Bradtke. University of Massachusetts. Adaptive linear quadratic control using policy iteration Steven J. Bradtke Computer Science Department University of Massachusetts Amherst, MA 01003 bradtke@cs.umass.edu B. Erik Ydstie Department of Chemical

More information

1 Basics of probability theory

1 Basics of probability theory Examples of Stochastic Optimization Problems In this chapter, we will give examples of three types of stochastic optimization problems, that is, optimal stopping, total expected (discounted) cost problem,

More information

Lecture 7 LQG Design. Linear Quadratic Gaussian regulator Control-estimation duality SRL for optimal estimator Example of LQG design for MIMO plant

Lecture 7 LQG Design. Linear Quadratic Gaussian regulator Control-estimation duality SRL for optimal estimator Example of LQG design for MIMO plant L7: Lecture 7 LQG Design Linear Quadratic Gaussian regulator Control-estimation duality SRL for optimal estimator Example of LQG design for IO plant LQG regulator L7:2 If the process and measurement noises

More information

Linear Quadratic Optimal Control Topics

Linear Quadratic Optimal Control Topics Linear Quadratic Optimal Control Topics Finite time LQR problem for time varying systems Open loop solution via Lagrange multiplier Closed loop solution Dynamic programming (DP) principle Cost-to-go function

More information

LQR, Kalman Filter, and LQG. Postgraduate Course, M.Sc. Electrical Engineering Department College of Engineering University of Salahaddin

LQR, Kalman Filter, and LQG. Postgraduate Course, M.Sc. Electrical Engineering Department College of Engineering University of Salahaddin LQR, Kalman Filter, and LQG Postgraduate Course, M.Sc. Electrical Engineering Department College of Engineering University of Salahaddin May 2015 Linear Quadratic Regulator (LQR) Consider a linear system

More information

Pontryagin s maximum principle

Pontryagin s maximum principle Pontryagin s maximum principle Emo Todorov Applied Mathematics and Computer Science & Engineering University of Washington Winter 2012 Emo Todorov (UW) AMATH/CSE 579, Winter 2012 Lecture 5 1 / 9 Pontryagin

More information

Efficient robust optimization for robust control with constraints Paul Goulart, Eric Kerrigan and Danny Ralph

Efficient robust optimization for robust control with constraints Paul Goulart, Eric Kerrigan and Danny Ralph Efficient robust optimization for robust control with constraints p. 1 Efficient robust optimization for robust control with constraints Paul Goulart, Eric Kerrigan and Danny Ralph Efficient robust optimization

More information

ECE7850 Lecture 9. Model Predictive Control: Computational Aspects

ECE7850 Lecture 9. Model Predictive Control: Computational Aspects ECE785 ECE785 Lecture 9 Model Predictive Control: Computational Aspects Model Predictive Control for Constrained Linear Systems Online Solution to Linear MPC Multiparametric Programming Explicit Linear

More information

Some New Results on Linear Quadratic Regulator Design for Lossless Systems

Some New Results on Linear Quadratic Regulator Design for Lossless Systems Some New Results on Linear Quadratic Regulator Design for Lossless Systems Luigi Fortuna, Giovanni Muscato Maria Gabriella Xibilia Dipartimento Elettrico Elettronico e Sistemistico Universitá degli Studi

More information

Lecture 4 Continuous time linear quadratic regulator

Lecture 4 Continuous time linear quadratic regulator EE363 Winter 2008-09 Lecture 4 Continuous time linear quadratic regulator continuous-time LQR problem dynamic programming solution Hamiltonian system and two point boundary value problem infinite horizon

More information

Stability of Parameter Adaptation Algorithms. Big picture

Stability of Parameter Adaptation Algorithms. Big picture ME5895, UConn, Fall 215 Prof. Xu Chen Big picture For ˆθ (k + 1) = ˆθ (k) + [correction term] we haven t talked about whether ˆθ(k) will converge to the true value θ if k. We haven t even talked about

More information

Formula Sheet for Optimal Control

Formula Sheet for Optimal Control Formula Sheet for Optimal Control Division of Optimization and Systems Theory Royal Institute of Technology 144 Stockholm, Sweden 23 December 1, 29 1 Dynamic Programming 11 Discrete Dynamic Programming

More information

Appendix A Solving Linear Matrix Inequality (LMI) Problems

Appendix A Solving Linear Matrix Inequality (LMI) Problems Appendix A Solving Linear Matrix Inequality (LMI) Problems In this section, we present a brief introduction about linear matrix inequalities which have been used extensively to solve the FDI problems described

More information

A Tour of Reinforcement Learning The View from Continuous Control. Benjamin Recht University of California, Berkeley

A Tour of Reinforcement Learning The View from Continuous Control. Benjamin Recht University of California, Berkeley A Tour of Reinforcement Learning The View from Continuous Control Benjamin Recht University of California, Berkeley trustable, scalable, predictable Control Theory! Reinforcement Learning is the study

More information

On Stochastic Adaptive Control & its Applications. Bozenna Pasik-Duncan University of Kansas, USA

On Stochastic Adaptive Control & its Applications. Bozenna Pasik-Duncan University of Kansas, USA On Stochastic Adaptive Control & its Applications Bozenna Pasik-Duncan University of Kansas, USA ASEAS Workshop, AFOSR, 23-24 March, 2009 1. Motivation: Work in the 1970's 2. Adaptive Control of Continuous

More information

6. Linear Quadratic Regulator Control

6. Linear Quadratic Regulator Control EE635 - Control System Theory 6. Linear Quadratic Regulator Control Jitkomut Songsiri algebraic Riccati Equation (ARE) infinite-time LQR (continuous) Hamiltonian matrix gain margin of LQR 6-1 Algebraic

More information

Linear Systems. Manfred Morari Melanie Zeilinger. Institut für Automatik, ETH Zürich Institute for Dynamic Systems and Control, ETH Zürich

Linear Systems. Manfred Morari Melanie Zeilinger. Institut für Automatik, ETH Zürich Institute for Dynamic Systems and Control, ETH Zürich Linear Systems Manfred Morari Melanie Zeilinger Institut für Automatik, ETH Zürich Institute for Dynamic Systems and Control, ETH Zürich Spring Semester 2016 Linear Systems M. Morari, M. Zeilinger - Spring

More information

ESC794: Special Topics: Model Predictive Control

ESC794: Special Topics: Model Predictive Control ESC794: Special Topics: Model Predictive Control Discrete-Time Systems Hanz Richter, Professor Mechanical Engineering Department Cleveland State University Discrete-Time vs. Sampled-Data Systems A continuous-time

More information

Outline. Linear regulation and state estimation (LQR and LQE) Linear differential equations. Discrete time linear difference equations

Outline. Linear regulation and state estimation (LQR and LQE) Linear differential equations. Discrete time linear difference equations Outline Linear regulation and state estimation (LQR and LQE) James B. Rawlings Department of Chemical and Biological Engineering 1 Linear Quadratic Regulator Constraints The Infinite Horizon LQ Problem

More information

Part II: Model Predictive Control

Part II: Model Predictive Control Part II: Model Predictive Control Manfred Morari Alberto Bemporad Francesco Borrelli Unconstrained Optimal Control Unconstrained infinite time optimal control V (x 0 )= inf U k=0 [ x k Qx k + u k Ru k]

More information

IEOR 265 Lecture 14 (Robust) Linear Tube MPC

IEOR 265 Lecture 14 (Robust) Linear Tube MPC IEOR 265 Lecture 14 (Robust) Linear Tube MPC 1 LTI System with Uncertainty Suppose we have an LTI system in discrete time with disturbance: x n+1 = Ax n + Bu n + d n, where d n W for a bounded polytope

More information

Structure of the Hessian

Structure of the Hessian Structure of the Hessian Graham C. Goodwin September 2004 1. Introduction We saw in earlier lectures that a core ingredient in quadratic constrained optimisation problems is the Hessian matrix H. So far

More information

EE363 Review Session 1: LQR, Controllability and Observability

EE363 Review Session 1: LQR, Controllability and Observability EE363 Review Session : LQR, Controllability and Observability In this review session we ll work through a variation on LQR in which we add an input smoothness cost, in addition to the usual penalties on

More information

The Newton-ADI Method for Large-Scale Algebraic Riccati Equations. Peter Benner.

The Newton-ADI Method for Large-Scale Algebraic Riccati Equations. Peter Benner. The Newton-ADI Method for Large-Scale Algebraic Riccati Equations Mathematik in Industrie und Technik Fakultät für Mathematik Peter Benner benner@mathematik.tu-chemnitz.de Sonderforschungsbereich 393 S

More information

Robust Control 5 Nominal Controller Design Continued

Robust Control 5 Nominal Controller Design Continued Robust Control 5 Nominal Controller Design Continued Harry G. Kwatny Department of Mechanical Engineering & Mechanics Drexel University 4/14/2003 Outline he LQR Problem A Generalization to LQR Min-Max

More information

Module 05 Introduction to Optimal Control

Module 05 Introduction to Optimal Control Module 05 Introduction to Optimal Control Ahmad F. Taha EE 5243: Introduction to Cyber-Physical Systems Email: ahmad.taha@utsa.edu Webpage: http://engineering.utsa.edu/ taha/index.html October 8, 2015

More information

Final Exam. Problem Score Problem Score 1 /10 8 /10 2 /10 9 /10 3 /10 10 /10 4 /10 11 /10 5 /10 12 /10 6 /10 13 /10 7 /10 Total /130

Final Exam. Problem Score Problem Score 1 /10 8 /10 2 /10 9 /10 3 /10 10 /10 4 /10 11 /10 5 /10 12 /10 6 /10 13 /10 7 /10 Total /130 EE103/CME103: Introduction to Matrix Methods December 9 2015 S. Boyd Final Exam You may not use any books, notes, or computer programs (e.g., Julia). Throughout this exam we use standard mathematical notation;

More information

Robotics. Control Theory. Marc Toussaint U Stuttgart

Robotics. Control Theory. Marc Toussaint U Stuttgart Robotics Control Theory Topics in control theory, optimal control, HJB equation, infinite horizon case, Linear-Quadratic optimal control, Riccati equations (differential, algebraic, discrete-time), controllability,

More information

The Uncertainty Threshold Principle: Some Fundamental Limitations of Optimal Decision Making under Dynamic Uncertainty

The Uncertainty Threshold Principle: Some Fundamental Limitations of Optimal Decision Making under Dynamic Uncertainty The Uncertainty Threshold Principle: Some Fundamental Limitations of Optimal Decision Making under Dynamic Uncertainty Michael Athans, Richard Ku, Stanley Gershwin (Nicholas Ballard 538477) Introduction

More information

Topic # Feedback Control Systems

Topic # Feedback Control Systems Topic #17 16.31 Feedback Control Systems Deterministic LQR Optimal control and the Riccati equation Weight Selection Fall 2007 16.31 17 1 Linear Quadratic Regulator (LQR) Have seen the solutions to the

More information

Linear Matrix Inequality (LMI)

Linear Matrix Inequality (LMI) Linear Matrix Inequality (LMI) A linear matrix inequality is an expression of the form where F (x) F 0 + x 1 F 1 + + x m F m > 0 (1) x = (x 1,, x m ) R m, F 0,, F m are real symmetric matrices, and the

More information

Course on Model Predictive Control Part II Linear MPC design

Course on Model Predictive Control Part II Linear MPC design Course on Model Predictive Control Part II Linear MPC design Gabriele Pannocchia Department of Chemical Engineering, University of Pisa, Italy Email: g.pannocchia@diccism.unipi.it Facoltà di Ingegneria,

More information

Control Systems Lab - SC4070 Control techniques

Control Systems Lab - SC4070 Control techniques Control Systems Lab - SC4070 Control techniques Dr. Manuel Mazo Jr. Delft Center for Systems and Control (TU Delft) m.mazo@tudelft.nl Tel.:015-2788131 TU Delft, February 16, 2015 (slides modified from

More information

Convex Optimization: Applications

Convex Optimization: Applications Convex Optimization: Applications Lecturer: Pradeep Ravikumar Co-instructor: Aarti Singh Convex Optimization 1-75/36-75 Based on material from Boyd, Vandenberghe Norm Approximation minimize Ax b (A R m

More information

Networked Sensing, Estimation and Control Systems

Networked Sensing, Estimation and Control Systems Networked Sensing, Estimation and Control Systems Vijay Gupta University of Notre Dame Richard M. Murray California Institute of echnology Ling Shi Hong Kong University of Science and echnology Bruno Sinopoli

More information

Topic # Feedback Control

Topic # Feedback Control Topic #5 6.3 Feedback Control State-Space Systems Full-state Feedback Control How do we change the poles of the state-space system? Or,evenifwecanchangethepolelocations. Where do we put the poles? Linear

More information

UCLA Chemical Engineering. Process & Control Systems Engineering Laboratory

UCLA Chemical Engineering. Process & Control Systems Engineering Laboratory Constrained Innite-time Optimal Control Donald J. Chmielewski Chemical Engineering Department University of California Los Angeles February 23, 2000 Stochastic Formulation - Min Max Formulation - UCLA

More information

Deterministic Kalman Filtering on Semi-infinite Interval

Deterministic Kalman Filtering on Semi-infinite Interval Deterministic Kalman Filtering on Semi-infinite Interval L. Faybusovich and T. Mouktonglang Abstract We relate a deterministic Kalman filter on semi-infinite interval to linear-quadratic tracking control

More information

Optimal Control Theory

Optimal Control Theory Optimal Control Theory The theory Optimal control theory is a mature mathematical discipline which provides algorithms to solve various control problems The elaborate mathematical machinery behind optimal

More information

EE363 homework 2 solutions

EE363 homework 2 solutions EE363 Prof. S. Boyd EE363 homework 2 solutions. Derivative of matrix inverse. Suppose that X : R R n n, and that X(t is invertible. Show that ( d d dt X(t = X(t dt X(t X(t. Hint: differentiate X(tX(t =

More information

Chapter 8 Stabilization: State Feedback 8. Introduction: Stabilization One reason feedback control systems are designed is to stabilize systems that m

Chapter 8 Stabilization: State Feedback 8. Introduction: Stabilization One reason feedback control systems are designed is to stabilize systems that m Lectures on Dynamic Systems and Control Mohammed Dahleh Munther A. Dahleh George Verghese Department of Electrical Engineering and Computer Science Massachuasetts Institute of echnology c Chapter 8 Stabilization:

More information

5. Observer-based Controller Design

5. Observer-based Controller Design EE635 - Control System Theory 5. Observer-based Controller Design Jitkomut Songsiri state feedback pole-placement design regulation and tracking state observer feedback observer design LQR and LQG 5-1

More information

EE C128 / ME C134 Feedback Control Systems

EE C128 / ME C134 Feedback Control Systems EE C128 / ME C134 Feedback Control Systems Lecture Additional Material Introduction to Model Predictive Control Maximilian Balandat Department of Electrical Engineering & Computer Science University of

More information

6.231 DYNAMIC PROGRAMMING LECTURE 9 LECTURE OUTLINE

6.231 DYNAMIC PROGRAMMING LECTURE 9 LECTURE OUTLINE 6.231 DYNAMIC PROGRAMMING LECTURE 9 LECTURE OUTLINE Rollout algorithms Policy improvement property Discrete deterministic problems Approximations of rollout algorithms Model Predictive Control (MPC) Discretization

More information

Chapter 9 Observers, Model-based Controllers 9. Introduction In here we deal with the general case where only a subset of the states, or linear combin

Chapter 9 Observers, Model-based Controllers 9. Introduction In here we deal with the general case where only a subset of the states, or linear combin Lectures on Dynamic Systems and Control Mohammed Dahleh Munther A. Dahleh George Verghese Department of Electrical Engineering and Computer Science Massachuasetts Institute of Technology c Chapter 9 Observers,

More information

Linear Quadratic Zero-Sum Two-Person Differential Games Pierre Bernhard June 15, 2013

Linear Quadratic Zero-Sum Two-Person Differential Games Pierre Bernhard June 15, 2013 Linear Quadratic Zero-Sum Two-Person Differential Games Pierre Bernhard June 15, 2013 Abstract As in optimal control theory, linear quadratic (LQ) differential games (DG) can be solved, even in high dimension,

More information

Riccati Equations and Inequalities in Robust Control

Riccati Equations and Inequalities in Robust Control Riccati Equations and Inequalities in Robust Control Lianhao Yin Gabriel Ingesson Martin Karlsson Optimal Control LP4 2014 June 10, 2014 Lianhao Yin Gabriel Ingesson Martin Karlsson (LTH) H control problem

More information

Module 9: State Feedback Control Design Lecture Note 1

Module 9: State Feedback Control Design Lecture Note 1 Module 9: State Feedback Control Design Lecture Note 1 The design techniques described in the preceding lectures are based on the transfer function of a system. In this lecture we would discuss the state

More information

Lecture 6. Regularized least-squares and minimum-norm methods 6 1

Lecture 6. Regularized least-squares and minimum-norm methods 6 1 Regularized least-squares and minimum-norm methods 6 1 Lecture 6 Regularized least-squares and minimum-norm methods EE263 Autumn 2004 multi-objective least-squares regularized least-squares nonlinear least-squares

More information

Robust LQR Control Design of Gyroscope

Robust LQR Control Design of Gyroscope Robust LQR Control Design of Gyroscope Ashok S. Chandak 1, Anil J. Patil 2 Abstract The basic problem in designing control systems is the ability to achieve good performance in the presence of uncertainties

More information

1 Linear Quadratic Control Problem

1 Linear Quadratic Control Problem 1 Linear Quadratic Control Problem Suppose we have a problem of the following form: { vx 0 ) = max β t ) } x {a t,x t+1 } t Qx t +a t Ra t +2a t Wx t 1) x t is a vector of states and a t is a vector of

More information

Optimal Control. Lecture 18. Hamilton-Jacobi-Bellman Equation, Cont. John T. Wen. March 29, Ref: Bryson & Ho Chapter 4.

Optimal Control. Lecture 18. Hamilton-Jacobi-Bellman Equation, Cont. John T. Wen. March 29, Ref: Bryson & Ho Chapter 4. Optimal Control Lecture 18 Hamilton-Jacobi-Bellman Equation, Cont. John T. Wen Ref: Bryson & Ho Chapter 4. March 29, 2004 Outline Hamilton-Jacobi-Bellman (HJB) Equation Iterative solution of HJB Equation

More information

Linear State Feedback Controller Design

Linear State Feedback Controller Design Assignment For EE5101 - Linear Systems Sem I AY2010/2011 Linear State Feedback Controller Design Phang Swee King A0033585A Email: king@nus.edu.sg NGS/ECE Dept. Faculty of Engineering National University

More information

First-Order Low-Pass Filter!

First-Order Low-Pass Filter! Filters, Cost Functions, and Controller Structures! Robert Stengel! Optimal Control and Estimation MAE 546! Princeton University, 217!! Dynamic systems as low-pass filters!! Frequency response of dynamic

More information

Inverses. Stephen Boyd. EE103 Stanford University. October 28, 2017

Inverses. Stephen Boyd. EE103 Stanford University. October 28, 2017 Inverses Stephen Boyd EE103 Stanford University October 28, 2017 Outline Left and right inverses Inverse Solving linear equations Examples Pseudo-inverse Left and right inverses 2 Left inverses a number

More information

On-off Control: Audio Applications

On-off Control: Audio Applications On-off Control: Audio Applications Graham C. Goodwin Day 4: Lecture 3 16th September 2004 International Summer School Grenoble, France 1 Background In this lecture we address the issue of control when

More information

An LQ R weight selection approach to the discrete generalized H 2 control problem

An LQ R weight selection approach to the discrete generalized H 2 control problem INT. J. CONTROL, 1998, VOL. 71, NO. 1, 93± 11 An LQ R weight selection approach to the discrete generalized H 2 control problem D. A. WILSON², M. A. NEKOUI² and G. D. HALIKIAS² It is known that a generalized

More information