Theory and Applications of Optimal Control Problems with Time-Delays

Transcription

1 Theory and Applications of Optimal Control Problems with Time-Delays University of Münster Institute of Computational and Applied Mathematics South Pacific Optimization Meeting (SPOM) Newcastle, CARMA, 9 12 February 213

2 EXPECT DELAYS

3 Challenges for delayed optimal control problems Theory and Numerics for non-delayed optimal control problems with control and state constraints are rather complete: 1 Necessary and sufficient conditions, 2 Stability and sensitivity analysis, 3 Numerical methods: Boundary value methods, Discretization and NLP, Semismooth Newton methods, 4 Real-time control techniques for perturbed extremals. Challenge: establish similar theoretical and numerical methods for delayed (retarded) optimal control problems.

4 DC24_Osmolovskii-Maurer_cover.indd 1 1/2/212 9:58:56 AM Book on SSC for regular and bang-bang controls N. P. Osmolovskii H. Maurer This book is devoted to the theory and applications of second-order necessary and sufficient optimality conditions in the calculus of variations and optimal control. The authors develop theory for a control problem with ordinary differential equations subject to boundary conditions of both the equality and inequality type and for mixed state-control constraints of the equality type. The book is distinctive in that necessary and sufficient conditions are given in the form of no-gap conditions, the theory covers broken extremals where the control has finitely many points of discontinuity, and a number of numerical examples in various application areas are fully solved. This book is suitable for researchers in calculus of variations and optimal control and researchers and engineers in optimal control applications in mechanics; mechatronics; physics; chemical, electrical, and biological engineering; and economics. Nikolai P. Osmolovskii is a Professor in the Department of Informatics and Applied Mathematics, Moscow State Civil Engineering University; the Institute of Mathematics and Physics, University of Siedlce, Poland; the Systems Research Institute, Polish Academy of Science; the University of Technology and Humanities in Radom, Poland; and of the Faculty of Mechanics and Mathematics, Moscow State University. He was an Invited Professor in the Department of Applied Mathematics, University of Bayreuth, Germany (2), and at the Centre de Mathématiques Appliquées, École Polytechnique, France (27). His fields of research are functional analysis, calculus of variations, and optimal control theory. He has written fifty papers and four monographs. was a Professor of Applied Mathematics at the Universität Münster, Germany (retired 21) and has conducted research in Austria, France, Poland, Australia, and the United States. His fields of research in optimal control are control and state constraints, numerical methods, second-order sufficient conditions, sensitivity analysis, real-time control techniques, and various applications in mechanics, mechatronics, physics, biomedical and chemical engineering, and economics. For more information about SIAM books, journals, conferences, memberships, or activities, contact: Society for Industrial and Applied Mathematics 36 Market Street, 6th Floor Philadelphia, PA USA Fax siam@siam.org Applications to Regular and Bang-Bang Control Second-Order Necessary and Sufficient Optimality Conditions in Calculus of Variations and Optimal Control Nikolai P. Osmolovskii Applications to Regular and Bang-Bang Control Second-Order Necessary and Sufficient Optimality Conditions in Calculus of Variations and Optimal Control Nikolai P. Osmolovskii ISBN DC24 DC24

5 Overview 1 Formulation of optimal control problems with state and control delays. 2 Example: Two-stage continuous stirred tank reactor (CSTR). 3 Minimum Principle. 4 NLP methods: discretize and optimize. 5 Optimal control of the innate immune response.

6 Delayed Optimal Control Problem with State Constraints State x(t) R n, Control u(t) R m, Delays d x, d u. Dynamics and Boundary Conditions ẋ(t) = f (x(t), x(t d x ), u(t), u(t d u )), a.e. t [, t f ], x(t) = x (t), t [ d x, ], u(t) = u (t), t [ d u, ), ψ(x(t f )) = Control and State Constraints u(t) U R m, S(x(t)), t [, t f ] (S : R n R k ). Minimize tf J(u, x) = g(x(t f )) + f (x(t), x(t d x ), u(t), u(t d u )) dt

7 Two-Stage Continuous Stirred Tank Reactor (CSTR) Time delays are caused by transport between the two tanks.

8 Two Stage CSTR Dadebo S. and Luus R. Optimal control of time-delay systems by dynamic programming, Optimal Control Applications and Methods 13, pp (1992). A chemical reaction A B is processed in two tanks. State and control variables: Tank 1 : x 1 (t) : (scaled) concentration x 2 (t) : (scaled) temperature u 1 (t) : temperature control Tank 2 : x 3 (t) : (scaled) concentration x 4 (t) : (scaled) temperature u 2 (t) : temperature control

9 Dynamics of the Two-Stage CSTR ( ) Reaction term in Tank 1 : R 1 (x 1, x 2 ) = (x 1 +.5) exp 25x2 1+x ( 2 ) Reaction term in Tank 2 : R 2 (x 3, x 4 ) = (x ) exp 25x4 1+x 4 Dynamics: ẋ 1 (t) ẋ 2 (t) =.5 x 1 (t) R 1 (t), = (x 2 (t) +.25) u 1 (t)(x 2 (t) +.25) + R 1 (t), ẋ 3 (t) = x 1 (t d) x 3 (t) R 2 (t) +.25, ẋ 4 (t) = x 2 (t d) 2x 4 (t) u 2 (t)(x 4 (t) +.25) + R 2 (t).25. Initial conditions: x 1 (t) =.15, x 2 (t) =.3, d t, x 3 () =.1, x 4 () =. Delays d =.1, d =.2, d =.4 in the state variables x 1, x 2.

10 Optimal control problem for the Two-Stage CSTR Minimize t f ( x1 2 + x x x u u2 2 ) dt (t f = 2). Hamiltonian with y k (t) = x k (t d), k = 1, 2 : H(x, y, λ, u) = f (x, u) + λ 1 ẋ 1 +λ 2 ( (x ) u 1 (x ) + R 1 (x 1, x 2 ) ) +λ 3 (y 1 x 3 R 2 (x 3, x 4 ) +.25) +λ 4 (y 2 2x 4 u 2 (x ) + R 2 (x 3, x 4 ) +.25) Advanced adjoint equations: λ 1 (t) = H x1 (t) χ [,tf d ] λ 3 (t + d), λ 2 (t) = H x2 (t) χ [,tf d ] λ 4 (t + d), λ k (t) = H xk (t) (k = 3, 4). The minimum condition yields H u = and thus u 1 = 5λ 2 (x ), u 2 = 5λ 4 (x ).

11 Two-Stage CSTR with free x(t f ) : x 1, x 2, x 3, x concentration x 1 d=.1 d=.2 d= temperature x 2 d=.1 d=.2 d= concentration x 3 d=.1 d=.2 d= temperature x 4 d=.1 d=.2 d= Delays d =.1, d =.2, d =.4.

12 Two-Stage CSTR with free x(t f ) : u 1, u 2, λ 1, λ control u 1 d=.1 d=.2 d= control u 2 d=.1 d=.2 d= adjoint variable λ 1 d=.1 d=.2 d= adjoint variable λ 2 d=.1 d=.2 d= Delays d =.1, d =.2, d =.4.

13 Two-Stage CSTR with x(t f ) = : x 1, x 2, x 3, x concentration x 1 d=.1 d=.2 d= temperature x 2 d=.1 d=.2 d= concentration x 3 d=.1 d=.2 d= temperature x 4 d=.1 d=.2 d= Delays d =.1, d =.2, d =.4.

14 Two-Stage CSTR with x(t f ) = : u 1, u 2, λ 1, λ control u 1 d=.1 d=.2 d= control u 2 d=.1 d=.2 d= adjoint variable λ 1 d=.1 d=.2 d= adjoint variable λ 2 d=.1 d=.2 d= Delays d =.1, d =.2, d =.4.

15 Two-Stage CSTR with x(t f ) = and x 4 (t) temperature x 4 d=.1 d=.2 d= control u 1 d=.1 d=.2 d= multiplier µ for x 4 <=.1 d=.1 d=.2 d= control u 2 d=.1 d=.2 d= Delays d =.1, d =.2, d =.4.

16 Delayed Optimal Control Problem with State Constraints State x(t) R n, Control u(t) R m, Delays d x, d u. Dynamics and Boundary Conditions ẋ(t) = f (x(t), x(t d x ), u(t), u(t d u )), a.e. t [, t f ], x(t) = x (t), t [ d x, ], u(t) = u (t), t [ d u, ), ψ(x(t f )) = Control and State Constraints u(t) U R m, S(x(t)), t [, t f ] (S : R n R k ). Minimize tf J(u, x) = g(x(t f )) + f (x(t), x(t d x ), u(t), u(t d u )) dt

17 Literature on optimal control with time-delays State delays and pure control constraints: Kharatishvili (1961), Oguztöreli (1966), Banks (1968), Halanay (1968), Soliman, Ray (197, chemical engineering), Warga (1972, abstract theory, optimization in Banach spaces), Guinn (1976, transform delayed problems to standard problems), Colonius, Hinrichsen (1978), Clarke, Wolenski (1991), Dadebo, Luus (1992), Mordukhovich, Wang (23 ). State delays and pure state constraints: Angell, Kirsch (199). State and control delays and mixed control state constraints: Göllmann, Kern, Maurer (OCAM 29), Göllmann, Maurer: Theory and applications of optimal control problems with multiple time-delays, to appear in JIMO 213.

18 Optimal control problems with state constraints Use the transformation method of Guinn (1976) and transform an optimal control problem with delays and state constraints to a standard non delayed optimal control problem with state constraints. Then apply the necessary conditions for non-delayed problems: Jacobson, Lele, Speyer (1975): KKT conditions in Banach spaces. Maurer (1979) : Regularity of multipliers for state constraints. Hartl, Sethi, Thomsen (SIAM Review 1995): Survey on Maximum Principles. Vinter (2): (Nonsmooth) Optimal Control

19 Hamiltonian Hamiltonian (Pontryagin) Function H(x, y, λ, u, v) :=λ f (t, x, y, u, v) + λf (t, x, y, u, v) y variable with y(t) = x(t d x ) v variable with v(t) = u(t d u ) λ R n, λ R adjoint (costate) variable Let (u, x) L ([, t f ], R m ) W 1, ([, t f ], R n ) be a locally optimal pair of functions. Then there exist an adjoint function λ BV([, t f ], R n ) and λ, a multiplier ρ R q (associated with terminal conditions), a multiplier function (measure) µ BV([, t f ], R k ), such that the following conditions are satisfied for a.e. t [, t f ] :

20 Minimum Principle (i) Advanced adjoint equation and transversality condition: λ(t) = t f t t f ( H x (s) + χ [,tf d x ](t) H y (s + d x ) ) ds + S x (x(s)) dµ(s) + (λ g + ρψ) x (x(t f )) ( if S(x(t f )) < ), where H x (t) and H y (t + d x ) denote evaluations along the optimal trajectory and χ [,tf d x ] is the characteristic function. (ii) Minimum Condition: H(t) + χ [,tf d u](t) H(t + d u ) = min w U [ H(x(t), y(t), λ(t), w, v(t)) + χ [,tf d u](t) H(t + d u )H(x(t + d u ), y(t), λ(t + d u ), u(t + d u ), w) ] (iii) Multiplier condition and complementarity condition: dµ(t), t f S(x(t)) dµ(t) = t

21 Minimum Principle (i) Advanced adjoint equation and transversality condition: λ(t) = t f t t f ( H x (s) + χ [,tf d x ](t) H y (s + d x ) ) ds + S x (x(s)) dµ(s) + (λ g + ρψ) x (x(t f )) ( if S(x(t f )) < ), where H x (t) and H y (t + d x ) denote evaluations along the optimal trajectory and χ [,tf d x ] is the characteristic function. (ii) Minimum Condition: H(t) + χ [,tf d u](t) H(t + d u ) = min w U [ H(x(t), y(t), λ(t), w, v(t)) + χ [,tf d u](t) H(t + d u )H(x(t + d u ), y(t), λ(t + d u ), u(t + d u ), w) ] (iii) Multiplier condition and complementarity condition: dµ(t), t f S(x(t)) dµ(t) = t

22 Minimum Principle (i) Advanced adjoint equation and transversality condition: λ(t) = t f t t f ( H x (s) + χ [,tf d x ](t) H y (s + d x ) ) ds + S x (x(s)) dµ(s) + (λ g + ρψ) x (x(t f )) ( if S(x(t f )) < ), where H x (t) and H y (t + d x ) denote evaluations along the optimal trajectory and χ [,tf d x ] is the characteristic function. (ii) Minimum Condition: H(t) + χ [,tf d u](t) H(t + d u ) = min w U [ H(x(t), y(t), λ(t), w, v(t)) + χ [,tf d u](t) H(t + d u )H(x(t + d u ), y(t), λ(t + d u ), u(t + d u ), w) ] (iii) Multiplier condition and complementarity condition: dµ(t), t f S(x(t)) dµ(t) = t

23 Regularity conditions for dµ(t) = η(t)dt if d u = Boundary arc : S(x(t)) = for t 1 t t 2. Assumption : u(t) int(u) for t 1 < t < t 2. Under certain regularity conditions we have dµ(t) = η(t) dt with a smooth multiplier η(t) for all t 1 < t < t 2. Adjoint equation and jump conditions λ(t) = H x (t) χ [,tf d x ] H y (t + d x ) η(t)s x (x(t)) λ(t k +) = λ(t k ) ν k S x (x(t k )), ν k at each contact or junction time t k, ν k = µ(t k +) µ(t k ) Minimum condition on the boundary H u (t) =. This condition allows to compute the multiplier η = η(x, λ).

24 Model of the immune response Dynamic model of the immune response: Asachenko A, Marchuk G, Mohler R, Zuev S, Disease Dynamics, Birkhäuser, Boston, Optimal control: Stengel RF, Ghigliazza R, Kulkarni N, Laplace O, Optimal control of innate immune response, Optimal Control Applications and Methods 23, (22), Lisa Poppe, Julia Meskauskas: Diploma theses, Universität Münster (26,28). L. Göllmann, H. Maurer: Optimal control problems with multiple time-delays, to appear in JIMO 213.

25 Innate Immune Response: state and control variables State variables: x 1 (t) : concentration of pathogen (=concentration of associated antigen) x 2 (t) : concentration of plasma cells, which are carriers and producers of antibodies x 3 (t) : concentration of antibodies, which kill the pathogen (=concentration of immunoglobulins) x 4 (t) : relative characteristic of a damaged organ ( = healthy, 1 = dead ) Control variables: u 1 (t) : pathogen killer u 2 (t) : plasma cell enhancer u 3 (t) : antibody enhancer u 4 (t) : organ healing factor

26 Innate Immune Response: state and control variables State variables: x 1 (t) : concentration of pathogen (=concentration of associated antigen) x 2 (t) : concentration of plasma cells, which are carriers and producers of antibodies x 3 (t) : concentration of antibodies, which kill the pathogen (=concentration of immunoglobulins) x 4 (t) : relative characteristic of a damaged organ ( = healthy, 1 = dead ) Control variables: u 1 (t) : pathogen killer u 2 (t) : plasma cell enhancer u 3 (t) : antibody enhancer u 4 (t) : organ healing factor

27 Generic dynamical model of the immune response ẋ 1 (t) = (1 x 3 (t))x 1 (t) u 1 (t), ẋ 2 (t) = 3A(x 4 (t))x 1 (t d)x 3 (t d) (x 2 (t) 2) + u 2 (t), ẋ 3 (t) = x 2 (t) ( x 1 (t))x 3 (t) + u 3 (t), ẋ 4 (t) = x 1 (t) x 4 (t) u 4 (t). Immune deficiency function triggered by target organ damage { } cos(πx4 ), x A(x 4 ) = x 4 For.5 x 4 (t) the production of plasma cells stops. State delay d in variables x 1 and x 3 Initial conditions (d = ) : x 2 () = 2, x 3 () = 4/3, x 4 () = Case 1 : x 1 () = 1.5, decay, requires no therapy (control) Case 2 : x 1 () = 2., slower decay, requires no therapy Case 3 : x 1 () = 3., diverges without control (lethal case)

28 Generic dynamical model of the immune response ẋ 1 (t) = (1 x 3 (t))x 1 (t) u 1 (t), ẋ 2 (t) = 3A(x 4 (t))x 1 (t d)x 3 (t d) (x 2 (t) 2) + u 2 (t), ẋ 3 (t) = x 2 (t) ( x 1 (t))x 3 (t) + u 3 (t), ẋ 4 (t) = x 1 (t) x 4 (t) u 4 (t). Immune deficiency function triggered by target organ damage { } cos(πx4 ), x A(x 4 ) = x 4 For.5 x 4 (t) the production of plasma cells stops. State delay d in variables x 1 and x 3 Initial conditions (d = ) : x 2 () = 2, x 3 () = 4/3, x 4 () = Case 1 : x 1 () = 1.5, decay, requires no therapy (control) Case 2 : x 1 () = 2., slower decay, requires no therapy Case 3 : x 1 () = 3., diverges without control (lethal case)

29 Generic dynamical model of the immune response ẋ 1 (t) = (1 x 3 (t))x 1 (t) u 1 (t), ẋ 2 (t) = 3A(x 4 (t))x 1 (t d)x 3 (t d) (x 2 (t) 2) + u 2 (t), ẋ 3 (t) = x 2 (t) ( x 1 (t))x 3 (t) + u 3 (t), ẋ 4 (t) = x 1 (t) x 4 (t) u 4 (t). Immune deficiency function triggered by target organ damage { } cos(πx4 ), x A(x 4 ) = x 4 For.5 x 4 (t) the production of plasma cells stops. State delay d in variables x 1 and x 3 Initial conditions (d = ) : x 2 () = 2, x 3 () = 4/3, x 4 () = Case 1 : x 1 () = 1.5, decay, requires no therapy (control) Case 2 : x 1 () = 2., slower decay, requires no therapy Case 3 : x 1 () = 3., diverges without control (lethal case)

30 Generic dynamical model of the immune response ẋ 1 (t) = (1 x 3 (t))x 1 (t) u 1 (t), ẋ 2 (t) = 3A(x 4 (t))x 1 (t d)x 3 (t d) (x 2 (t) 2) + u 2 (t), ẋ 3 (t) = x 2 (t) ( x 1 (t))x 3 (t) + u 3 (t), ẋ 4 (t) = x 1 (t) x 4 (t) u 4 (t). Immune deficiency function triggered by target organ damage { } cos(πx4 ), x A(x 4 ) = x 4 For.5 x 4 (t) the production of plasma cells stops. State delay d in variables x 1 and x 3 Initial conditions (d = ) : x 2 () = 2, x 3 () = 4/3, x 4 () = Case 1 : x 1 () = 1.5, decay, requires no therapy (control) Case 2 : x 1 () = 2., slower decay, requires no therapy Case 3 : x 1 () = 3., diverges without control (lethal case)

31 Optimal control model: cost functional State x = (x 1, x 2, x 3, x 4 ) R 4, Control u = (u 1, u 2, u 3, u 4 ) R 4 L 2 -functional quadratic in control: Stengel et al. Minimize J 2 (x, u) = x 1 (t f ) 2 + x 4 (t f ) 2 t f + ( x1 2 + x u2 1 + u2 2 + u2 3 + u2 4 ) dt L 1 -functional linear in control Minimize J 1 (x, u) = x 1 (t f ) 2 + x 4 (t f ) 2 t f + ( x1 2 + x u 1 + u 2 + u 3 + u 4 ) dt Control constraints: u i (t) u max, i = 1,.., 4 Final time: t f = 1

32 Optimal control model: cost functional State x = (x 1, x 2, x 3, x 4 ) R 4, Control u = (u 1, u 2, u 3, u 4 ) R 4 L 2 -functional quadratic in control: Stengel et al. Minimize J 2 (x, u) = x 1 (t f ) 2 + x 4 (t f ) 2 t f + ( x1 2 + x u2 1 + u2 2 + u2 3 + u2 4 ) dt L 1 -functional linear in control Minimize J 1 (x, u) = x 1 (t f ) 2 + x 4 (t f ) 2 t f + ( x1 2 + x u 1 + u 2 + u 3 + u 4 ) dt Control constraints: u i (t) u max, i = 1,.., 4 Final time: t f = 1

33 Optimal control model: cost functional State x = (x 1, x 2, x 3, x 4 ) R 4, Control u = (u 1, u 2, u 3, u 4 ) R 4 L 2 -functional quadratic in control: Stengel et al. Minimize J 2 (x, u) = x 1 (t f ) 2 + x 4 (t f ) 2 t f + ( x1 2 + x u2 1 + u2 2 + u2 3 + u2 4 ) dt L 1 -functional linear in control Minimize J 1 (x, u) = x 1 (t f ) 2 + x 4 (t f ) 2 t f + ( x1 2 + x u 1 + u 2 + u 3 + u 4 ) dt Control constraints: u i (t) u max, i = 1,.., 4 Final time: t f = 1

34 L 2 functional, d = : optimal state and control variables State variables x 1, x 2, x 3, x 4 and optimal controls u 1, u 2, u 3, u 4 : second-order sufficient conditions via matrix Riccati equation

35 L 2 functional, d = : state constraint x 4 (t).2 State and control variables for state constraint x 4 (t).2. Boundary arc x 4 (t).2 for t 1 =.398 t t 2 = 1.35

36 L 2 functional, multiplier η(t) for constraint x 4 (t).2 Compute multiplier η as function of (x, λ) : η(x, λ) = λ 2 3π sin(πx 4 )x 1 x 3 λ 1 + 2λ 4 2x 3 x 1 + 2x 4 Scaled multiplier.1 η(t) and boundary arc x 4 (t) =.2

37 L 2 functional, delay d >, constraint x 4 (t) α Dynamics with state delay d > ẋ 1 (t) = (1 x 3 (t))x 1 (t) u 1 (t), ẋ 2 (t) = 3 cos(πx 4 ) x 1 (t d)x 3 (t d) (x 2 (t) 2) + u 2 (t), ẋ 3 (t) = x 2 (t) ( x 1 (t))x 3 (t) + u 3 (t), ẋ 4 (t) = x 1 (t) x 4 (t) u 4 (t) x 4 (t) α.5 Initial conditions x 1 (t) =, d t <, x 1 () = 3, x 3 (t) = 4/3, d t, x 2 () = 2, x 4 () =.

38 L 2 functional : delay d = 1 and x 4 (t).2 State variables for d = and d = 1

39 L 2 functional : delay d = 1 and x 4 (t).2 Optimal controls for d = and d = 1

40 L 2 functional, d = 1 : multiplier η(t) for x 4 (t).2 Compute multiplier η as function of (x, λ) : η(x, y, λ) = λ 2 3π sin(πx 4 )y 1 y 3 λ 1 + 2λ 4 2x 3 x 1 + 2x 4 Scaled multiplier.1 η(t) and boundary arc x 4 (t) =.2 ; η(t) is discontinuous at t = d = 1

41 L 1 functional and state delay d Minimize J 1 (x, u) = p 11 x 1 (t f ) 2 + p 44 x 4 (t f ) 2 t f + ( p 11 x1 2 + p 44x4 2 + q 1 u 1 + q 2 u 2 + q 3 u 3 + q 4 u 4 ) dt Dynamics with delay d and control constraints ẋ 1 (t) = (1 x 3 (t))x 1 (t) u 1 (t), ẋ 2 (t) = 3A(x 4 (t))x 1 (t r)x 3 (t r) (x 2 (t) 2) + u 2 (t), ẋ 3 (t) = x 2 (t) ( x 1 (t))x 3 (t) + u 3 (t), ẋ 4 (t) = x 1 (t) x 4 (t) u 4 (t), u i (t) u max, t t f (i = 1,.., 4)

42 L 1 functional : non-delayed problem d = : u max = 2

43 L 1 functional : delayed problem d = 1 : u max = 2

44 L 1 functional : non-delayed, time optimal control for x 1 (t f ) = x 4 (t f ) =, x 3 (t f ) = 4/3 u max = 1: minimal time t f = , singular arc for u 4 (t)

45 Further applications and future work 1 Optimal oil extraction and exploration : state delay (Bruns, Maurer, Semmler) 2 Biomedical applications: optimal protocols in cancer treatment and immunology 3 Vintage control problems 4 Delayed control problems with free final time 5 Optimal control problems with state-dependent delays 6 Verifiable sufficient conditions

46 Thank you for your attention!