Lecture 26 Classical Numerical Methods to Solve Optimal Control Problems Dr. Radhakant Padhi Asst. Professor Dept. of Aerospace Engineering Indian Institute of Science - Bangalore
Necessary Conditions of Optimality in Optimal Control State Equation Costate Equation X H = = λ H λ = = X (,, ) f t X U (,, ) g t X U Optimal Control Equation H U = 0 U = ψ, ( X λ ) Boundary Condition λ f ϕ = Xt ( ) X f = X:Fixed 0 0 2
Necessary Conditions of Optimality: Salient Features State and Costate equations are dynamic equations State equation develops forward whereas Costate equation develops backwards Optimal control equation is a stationary equation The formulation leads to Two-Point-Boundary-Value Problems (TPBVPs), which demand computationally-intensive iterative numerical procedures to obtain the optimal control solution 3
Classical Methods to Solve TPBVPs Gradient Method Shooting Method Quasi-Linearization Method 4
Gradient Method Dr. Radhakant Padhi Asst. Professor Dept. of Aerospace Engineering Indian Institute of Science - Bangalore
Gradient Method Assumptions: State equation satisfied Costate equation satisfied Boundary conditions satisfied Strategy: Satisfy the optimal control equation 6
7 Gradient Method ( ) ( ) ( ) ( ) 0 0 0 f f f T f f f t T t t T t t T t J X X H X dt X H U dt U H X dt φ δ δ λ δ λ δ δλ λ = + + + +
Gradient Method After satisfying the state & costate equations and boundary conditions, we have t f T H δj = ( δu) dt U Select δu() t = τ, τ > 0 t 0 H U This leads to δj = τ t f t 0 T H H U U dt 8
Gradient Method We select This lead to 1 δu () t = U () t U () t = τ U i i i H + U () t = U () t τ U i+ 1 i H i i Note: δj t f H H = τ dt U U t 0 T 0 Eventually, δ J H = 0 = 0 U 9
Gradient Method: Procedure Assume a control history (not a trivial task) Integrate the state equation forward Integrate the costate equation backward Update the control solution This can either be done at each step while integrating the costate equation backward or after the integration of the costate equation is complete Repeat the procedure until convergence t t 0 T f H H dt U U γ (a pre-selected constant) 10
Gradient Method: Selection of τ τ Select so that it leads to a certain percentage reduction of J Let the percentage be α t f T Then α This leads to H H τ dt = J t U U 100 0 α J τ = 100 t f T H H dt U U t 0 11
A Real-Life Challenging Problem Objective: Air-to-air missiles are usually launched from an aircraft in the forward direction. However, the missile should turn around and intercept a target behind the aircraft. Aircraft Missile Target To execute this task, the missile should turn around by -180 o and lock onto its target (after that it can be guided by its own homing guidance logic). Note: Every other case can be considered as a subset of this extreme scenario! 12
A Real-Life Challenging Problem MATHEMATICAL PERSPECTIVE: Minimum time optimization problem Fixed initial conditions and free final time problem SYSTEM DYNAMICS: Equations of motion for a missile in vertical plane. The non-dimensional equations of motion (point mass) in a vertical plane are: M S M C T 2 ' = w D sin( γ) + wcos( α) 1 2 γ ' = [ SMC w L + Twsin( α) cos( γ)] M where prime denotes differentiation with respect to the non-dimensional time τ 13
A Real-Life Challenging Problem The non-dimensional parameters are defined as follows: 2 g T ρa S V ; Tw ; Sw ; M τ = = = = at mg 2mg a where M = flight Mach number γ = flight path angle T = thrust m = mass of the missile S = reference aerodynamic area V C D = speed of the missile C = lift coefficient L = drag coefficient g = the acceleration due to gravity a = the local speed of sound ρ = the atmospheric density t = flight time after launch NOTE: C, C are usually functions of α & M (tabulated data) L D 14
A Real-Life Challenging Problem COST FUNCTION: Mathematically the problem is possed as follows to find the control minimizing cost function: J = o Constraints γ (0) = 0, M (0) = initial Mach number t f 0 dt o γ ( t ) = 180, M( t ) = 0.8 f f 15
A Real-Life Challenging Problem Choosing γ as the independent variable the equations are reformulated as follows: 2 ( w D sin( γ) + wcos( α) ) dm SMC T M = dγ S M C cos( γ) + T sin( α) dt = dγ 2 w L w a M ( cos( γ) + Tw sin( α) ) 2 g SwM CL and the transformed cost function is t π π f dt a M J = dt = dγ = dγ dγ g S M C γ T α 0 0 0 2 ( w L cos( ) + wsin( )) A difficult minimum-time problem has been converted to a relatively easier fixed final-time problem (with hard constraint: M ( γ f ) = 0.8)! 16
Task Solve the problem using gradient method. Assume M (0) = 0.5 and engagement height as 5 km. Next, generate the trajectories and tabulate the values of M for various q values. Use the following system parameters (typical for an air-to-air missile): m= 240 kg S = 0.0707 m2 T = 24,000 N C C D L = 0.5 = 3.12 f Use standard atmosphere chart for the atmospheric data. 17
Shooting Method Dr. Radhakant Padhi Asst. Professor Dept. of Aerospace Engineering Indian Institute of Science - Bangalore
Necessary Conditions of Optimality (TPBVP): A Summary State Equation Costate Equation X H = = λ H λ = = X (,, ) f t X U (,,, λ ) g t X U Optimal Control Equation H U = 0 Boundary Condition λ f ϕ = Xt ( ) X f = X:Fixed 0 0 19
Shooting Method 20
Shooting Method 21
Shooting Method 22
Shooting Method 23
Shooting Method 24
Quasi-Linearization Method Dr. Radhakant Padhi Asst. Professor Dept. of Aerospace Engineering Indian Institute of Science - Bangalore
Quasi-Linearization Method Problem: ( ) ( ), ( ) { 1,, T } T T Differential Equation: Z = F Z, t, Z X λ Boundary condition: C t, Z t = C Z = b Assumption: i i i i i i t t i n This vector differential equation has a unique solution over T t t t 0, f Trick: The nonlinear multi-point boundary value problem is transformed into a sequence of linear non-stationary boundary value problems, the solution of which is made to approximate the solution of the true problem. 26
Quasi-Linearization Method ( ) ( = ) N (1) Guess an approximate solution Z t N 1 (it need not satisfy the B.C.) For updating this solution, proceed with the following steps: (2) Linearize the system dynamics about Δ Z N Z N () t F N N N+ 1 N =, where, Z ΔZ ΔZ t Z t Z t N Z To be found () At () Δ = Δ N N Z A t Z N+ 1 N N ( ), ( ) ( ), ( ) ( ) N N ( ), Δ ( ) = ( ), Z ( t ) + b () () () (3) Enforce the boundary with respect to the updated solution C t Z t = C t Z t +Δ Z t = b i i i i i i C t Z t C t i i i i i Z N + 1 () t Philosophy: Solve this linear system and update the solution! 27
Quasi-Linearization Method: Solution by STM Approach (1) From the linearized system dynamics, we can write ()( ) Z = Z + A t Z Z N+ 1 N N+ 1 N + 1 () (, ) () N N N = At Z + F Z t At Z Homogeneous Forcing function N + 1 Z () t () = Φ (, ) ( ) + () (2) The solution to the above equation is given by N+ 1 N+ 1 N+ 1 N+ 1 Z t t t0 Z t0 p t State transition matrix (STM) Particular solution N + 1 (3) The solution for STM Φ ( tt, 0 ) can be obtained from the fact that it satisfies the following differential equation and boundary conditions Φ = Φ t t t I N ( tt, ) At ( ) ( tt, ) N+ 1 + 1 0 0 (, ) N + 1 Φ 0 0 = 28
Quasi-Linearization Method: Solution by STM Approach ( t) N + 1 (4) The particular solution p can be obtained by observing that it satisfies the the following differential equation and boundary condition () t N + 1 Substituting the complete solution i Z n the original equation N+ 1 N+ 1 N+ 1 N+ 1 N+ 1 N+ 1 ( tt, 0) Z ( t0) p ( t) At ( ) ( tt, 0) Z ( t0) p ( t) t Φ + = Φ + N N + F( Z, t) A() t Z N+ 1 N+ 1 N N p () t = A() t p () t + F( Z, t) A() t Z N + 1 (5) The boundary condition p t can be obtained by observing that ( 0 ) ( ) =Φ (, ) ( ) + ( ) Z t t t Z t p t p N+ 1 N+ 1 N+ 1 N+ 1 0 0 0 0 0 N + 1 ( t ) 0 = 0 I 29
Quasi-Linearization Method: Solution by STM Approach N + 1 Z ( t0 ) N + 1 ( ), ( ) (6) The boundary condition can be obtained as follows C t Z t = b i i i N+ 1 N+ 1 N+ 1 ( ), (, ) ( ) ( ) C t Φ t t Z t + p t = b i i 0 0 i i N+ 1 N+ 1 N + 1 ( ), Φ (, ) ( ) = ( ), ( t ) C t t t Z t C t p i i 0 0 i i + b i Solve the above system to obtain Z N + 1 ( t ) 0 N ( 0 ) ( ) N+ 1 N+ 1 N+ 1 N+ 1 Z () t =Φ ( t t ) Z ( t ) + p () t N+ 1 + 1 Once is determined, the solution is available from the Z t Z t STM solution:, STM 0 0 Particular solution 30
Quasi-Linearization Method: Convergence Property Under the assumption that the problem admits a unique solution for t t0, t f it can be shown that the sequence of vectors converge to the true solution. N + 1 { Z () t } Morover, the process can be shown to have "quadratic convergence" in general () () () () ( ) Z t Z t k Z t Z t k f N N+ 1 N N N 1 i.e., it can be shown that, where. Further more, for a large class of systems, it can be shown to have "monotone convergence" as well, i.e. there won't be any over-shooting in the convergence process. Reference : R. Kabala, "On Nonlinear Differential Equations, The Maximum Operation and Monotone Convergence", J. of Mathematics and Mechanics, Vol. 8, 1959, pp. 519-574. 31
A Demonstrative Example 1 1 2 2 2 Problem: Minimize J = ( x u ) dt for the system x x u, x( 0) 10. 2 + = + = 0 Solution: 1 2 2 2 Hamiltonian: H = ( x + u ) + λ ( x + u) 2 2 1) State Equation: x = x + u 2) Optimal Control Equation: u+ λ = 0 u = λ = ( H x) = x+ λx ( ) λ ( ) ( x) 3) Costate Equation: λ / 2 4) Bounary Conditions: x 2 x x x 0 = 10, 1 = Φ/ = 0 Substituting the expression for u in the state equation, we can write ( ) () = λ, 0 = 10 λ = x+ 2 λx, λ 1 = 0 Task : Solve this problem using shooting and quasi-linearization methods. 32
References on Numerical Methods in Optimal Control Design D. E. Kirk, Optimal Control Theory: An Introduction, Prentice Hall, 1970. S. M. Roberts and J.S. Shipman, Two Point Boundary Value Problems: Shooting Methods, American Elsevier Publishing Company Inc., 1972. A. E. Bryson and Y-C Ho, Applied Optimal Control, Taylor and Francis, 1975. A. P. Sage and C. C. White III, Optimum Systems Control (2nd Ed.), Prentice Hall, 1977. 33
Survey of Classical Methods M. Athans (1966), The Status of Optimal Control Theory and Applications for Deterministic Systems, IEEE Trans. on Automatic Control, Vol. AC-11, July 1966, pp.580-596. H. J. Pesch (1994), A Practical Guide to the Solution of Real-Life Optimal Control Problems, Control and Cybernetics, Vol.23, No.1/2, 1994, pp.7-60. 34
35