Constrained Optimal Control I

Optimal Control, Guidance and Estimation Lecture 34 Constrained Optimal Control I Pro. Radhakant Padhi Dept. o Aerospace Engineering Indian Institute o Science - Bangalore opics Motivation Brie Summary o Unconstrained Optimal Control Pontryagin Minimum Principle ime Optimal Control o LI Systems ime Optimal Control o Double-Integral System Fuel Optimal Control Energy Optimal Control State Constrained optimal control 2

Motivation Pro. Radhakant Padhi Dept. o Aerospace Engineering Indian Institute o Science - Bangalore Motivation Physical systems are always restricted by constraints on control and state variables. Examples: hrust delection o the rocket engine cannot not exceed a certain designed value Control surace delections are constrained by hard bounds Aircrats cannot climb beyond a certain altitude (else, they will loose lit because o low dynamic pressure) Robotic arms are constrained by physical limits on angular movements Speed o electric motors should not increase beyond a limit (to prevent wear and tear) Current in a circuit must not increase beyond a limit. Else, some component may burn out. 4

Motivation Question: Can these constraints be explicitly handled in the control design? Answer: YES! (optimal control ramework allows that) Ways to handle Sot constrainormulations Hard constrainormulation Problem classiication Control constrained problems State constrained problems Mixed state and control constrained problems 5 Pioneers o Optimal Control 1700s Bernoulli, Newton Euler (Student o Bernoulli) Lagrange...200 years later... 1900s Pontryagin Bellman Kalman Bernoulli Pontryagin Euler Bellman Lagrange Kalman Newton 6

Lev Semyonovich Pontryagin Lev Semyonovich Pontryagin (September 3,1908: May 3, 1988) Moscow Russia. Lost his eyesight when he was about 14 years old due to an explosion. Entered Moscow State University (1925). In 1930s & 1940s signiicant contribution to topology which was translated into several Languages. As a head o Steklov Mathematical Institute he ocused on general theory o singularly perturbed systems o ordinary dierential equations and maximum principle in optimal control theory. 7 L. S. Pontryagin In 1955, he ormulated a general time-optimal control problem or a ith-order dynamical system describing optimal maneuvers o an aircrat with bounded control unctions. o invent a new calculus o variation he spent three consecutive sleepless nights and came up with the idea o the Hamiltonian ormulation or the problem and the adjoint dierential equations. His other contributions include "singular perturbation theory" and "dierential game theory". He and his co-workers were awarded "Lenin Prize" in 1961. 8

Brie Summary o Unconstrained Optimal Control Pro. Radhakant Padhi Dept. o Aerospace Engineering Indian Institute o Science - Bangalore Objective o ind an "admissible" time history o control variable U t, t,, which: ( ) 1) Causes the system governed by Xɺ = t, X, U to ollow an admissible trajectory ( ) 2) Optimizes (minimizes/maximizes) a "meaningul" perormance index = ϕ (, ) + (,, ) J t X L t X U dt 3) Forces the system to satisy "proper boundary conditions". 10

Optimal Control Problem Perormance Index (to minimize / maximize): Path Constraint: Xɺ = t X U Boundary Conditions: X = ϕ (, ) + (,, ) J t X L t X U dt (,, ) t ( ) t 0 = X : Speciied 0 ( ) : Fixed, X t : Free 11 Necessary Conditions o Optimality Augmented PI Hamiltonian First Variation ( ɺ ) J = ϕ + L + λ X dt ( + λ ) H L t ( ɺ ) δ J = δϕ + δ H λ X dt ( ɺ ) = δϕ + δ H λ X dt 12

Necessary Conditions o Optimality First Variation Individual terms δϕ ( t, X ) ( δ X ) ( ɺ ɺ ) δ J = δϕ + δ H δλ X λ δ X dt ϕ = X δ H H H H ( t, X, U, λ ) ( δ X ) ( U ) ( ) X δ U δλ = + + λ 13 Necessary Conditions o Optimality d ( δ X ) λ δ Xɺ dt λ dt ( ) = dt, δ X dλ = λ δ X, δ X 0 t dt 0 0 0 ( ) ɺ 0 X ( X ) ɺ dt δ X dt = λ δ X λ δ X δ X λ dt = λ δ δ λ 14

Necessary Conditions o Optimality First Variation ϕ δ J = ( δ X ) ( δ X ) λ X H H H + ( δ X ) + ( δu ) + ( δλ ) dt X U λ ( δ ) ɺ λ ( δλ ) + X dt Xɺ dt 15 Necessary Conditions o Optimality First Variation ϕ δ J = ( δ X ) λ X = 0 ( δ ) ɺ λ ( δ ) H H + X + dt + U dt X U H + ( δλ ) X ɺ dt λ 16

Necessary Conditions o Optimality: Summary State Equation Costate Equation H Xɺ = = t X U λ ɺ H λ = X (,, ) Optimal Control Equation H U = 0 Boundary Condition λ ϕ = ( ) X X t X Fixed = : 0 0 17 Necessary Conditions o Optimality: Some Comments State and Costate equations are dynamic equations. I one is stable, the other turns out to be unstable! Optimal control equation is a stationary equation Boundary conditions are split: it leads to wo-point- Boundary-Value Problem (PBVP) State equation develops orward whereas Costate equation develops backwards. It is known as Curse o Complexity in optimal control raditionally, PBVPs demand computationally-intensive iterative numerical procedures, which lead to open-loop control structure. 18

Control Constrained Problems: Pontryagin Minimum Principle Reerence: D. S. Naidu: Optimal Control Systems, CRC Press, 2002. Pontryagin Pro. Radhakant Padhi Dept. o Aerospace Engineering Indian Institute o Science - Bangalore Objective o ind an "admissible" time history o control variable U t, t, t, + ( or, component wise, j j j ) where U ( t) U U u ( t) U,which: ( ) 1) Causes the system governed by Xɺ = t, X, U to ollow an admissible trajectory 2) Optimizes (minimizes/maximizes) a "meanigul" perormance index = ϕ (, ) + (,, ) J t X L t X U dt 3) Forces the system to satisy "proper boundary conditions". ( ) 20

Optimum o Control Functional u, u ( ) ( ) ( ) Variation: δ u t = u t u t u ( t) δu( t) u ( t) : Optimum control a b t 21 Optimum o Control Functional ( ) ( ) u( t) u( t) u ( t) A unctional u t is said to have a relative optimum at u t, i ε > 0 such thaor all unctions Ω which satisy < ε, the increment o J has the "same sign". ( ) ( ) ( ) 1) I J = J u J u 0, then J u is a relative (local) "Minimum". ( ) ( ) ( ) 2) I J = J u J u 0, then J u is a relative (local) "Maximum". Note: I the above relationships are satisied or arbitrarily large ε > 0, then ( ) J u is a "global optimum". 22

Pontryagin Minimum Principle U U δu With variations in control = +, ( δ ) ( ) ( ) = δ J ( U δu ) J U, U = J U J U 0 or Minimum, + HO J δu (Neglecting HO) U However, when U ( t) U, δu is no longer arbitrary or all t, 23 Control Constrained Problems Reerence: D. S. Naidu: Optimal Control Systems, CRC Press, 2002. Note: he condition δ J = 0 is valid only i u (t) lies within the boundary (i.e. it has no constraint) or the entire time interval t, 0 t 24

Pontryagin Minimum Principle U ( δ ) Necessary Condition: δ J U ( t), U ( t) 0 First Variation: ( t) : Optimal solution δu t U t ( ) : Allowable variation about ( ) H H H δ J = + ɺ λ δ X + δ U + Xɺ δλ dt X U λ ϕ + λ X δ X 25 Pontryagin Minimum Principle 1) In control constrained problems, variations in costates δλ( t) can be arbitrary. his gives H Xɺ = 0 λ H Xɺ = = ( t, X, U ) (state equation) λ 2) I the costate λ( t) is selected such that the coeicient o δ X ( t) is 0 (i.e. variations in states can be arbitrary), then ɺ H λ + = 0 X ɺ H λ = (costate equation) X 26

Pontryagin Minimum Principle 3) Boundary conditions are not eected by the control constraints. Hence, the ollowing ransversality condition still holds good. ϕ λ = X With the above observations, the necessary condition becomes H δ J ( U, δu ) = δu dt U (, δ, λ ) (,, λ ) = H X U U H X U + dt 0 admissible δu arbitrarily small 27 Pontryagin Minimum Principle Since δu ( t) is arbitrarily small, the integrand 0. his gives us H X, U + δu, λ H X, U, U i. e. (, U, λ ) H ( X, U, λ ) H X U ( t) U ( λ ) "Necessary condition" or constrained optimal control U is given by min H ( X, U, λ ) = H ( X, U, λ ) i. e. the optimal control should minimize the Hamiltonian his is known as the "Pontryagin Minimum Principle". 28

Solution Procedure o a given Problem Hamiltonian : Necessary Conditions : (,, λ) = (, ) + λ (, ) H X U L X U X U H (i) State Equation: Xɺ = = ( t, X, U ) λ H (ii) Costate Equation: ɺ λ = X (ii i) Optimal Control Equation: Minimize H with repect to U ( t) U ( 0) ( λ) H ( X U λ ) i.e. H X, U,,, (iv) Boundary conditions: X ϕ = Speciied, λ = X 29 Some Important Observations 1) he optimality condition (,, λ ) H ( X, U, λ ) H X U is valid or both constrained and unconstrained control system, ( H U ) whereas the control relation / = 0 is valid or unconstrained systems only. 2) he results given above provide the necessary conditions only. 3) he suicient condition or unconstrained control problem is that 2 H 2 should be positive deinite matrix t, U ( X, U, λ ) 30

A Simple Scalar Algebraic Example Problem : 2 Minimize the unction H = u 6u + 7 subject to the constraint relation u 2, i. e. 2 u + 2 Solution : Using the relation or unconstrained control, H = 2u 6 = 0 u u = 3 (Not admissible!) 31 Plot o Hamiltonian Reerence: D. S. Naidu: Optimal Control Systems, CRC Press, 2002. 32

A Simple Scalar Example In this case, the admissible optimal value is u = + 2 can also be obtained rom static optimization results using Kharush-Kuhn-ucker conditions. Note : I the constraint had been u 3, i. e. 3 u + 3, then either o the relation could be used and obtain the optimal value as u = 3. However, unortunately many practical constraints do not admit such solutions! 33 Additional Necessary Conditions (Due to Pontryagin & Co-workers) 1) I the inal time t is " ixed" and the Hamiltonian H does not depend on time t explicitly, then the Hamiltonian must be constant along the optimal trajectory, i.e. H t t t = Constant 0, 2) I the inal time t is " ree" and the Hamiltonian does not depend on time t explicitly, then the the Hamiltonian must be identically zero along the optimal trajectory, i.e. H = 0 t, t 34

Proo or Unconstrained Problem heorem: I the Hamiltonian H is not an expliciunction o time, then H is constant along the optimal path. Proo: dh H H H H H X U But X and X X = + ɺ + ɺ + ɺ λ = ɺ ɺ λ ɺ = ɺ ɺ λ dt t X U λ λ H H H = + Xɺ + ɺ λ + Uɺ t X U 0 0 dh H = ( on optimal path) dt t = 0 i H is not an expliciunction o t. Hence, the result! ( ) 35 Conclusions Physical systems are always restricted by constraints on control and state variables. In this class we studied about Brie Summary o Unconstrained Optimal Control Pontryagin Minimum Principle or Control Constrained Optimal Control (in a generic sense) In the next two classes, we will study about ime Optimal Control o LI Systems Fuel Optimal Control Energy Optimal Control State Constrained optimal control 36

Reerences D. S. Naidu: Optimal Control Systems, CRC Press, 2002. L. M. Hocking: Optimal Control: An Introduction to heory and Applications, Oxord University Press, New York, NY, 1966. L. S. Pontryagin, V. G. Boltyanskii, R. V. Gamkrelidze and E. F. Mishchenko: he Mathematical heory o Optimal Processes, Wiley-Intersciences, New York, NY, 1962 (ranslated rom Russian). 37 hanks or the Attention.!! 38