An Introduction to Optimal Control Applied to Disease Models Suzanne Lenhart University of Tennessee, Knoxville Departments of Mathematics Lecture1 p.1/37
Example Number of cancer cells at time (exponential growth) State Drug concentration Control known initial data minimize where the first term represents number of cancer cells and the second term represents harmful effects of drug on body. Lecture1 p.2/37
Optimal Control Adjust controls in a system to achieve a goal System: Ordinary differential equations Partial differential equations Discrete equations Stochastic differential equations Integro-difference equations Lecture1 p.3/37
Deterministic Optimal Control Control of Ordinary Differential Equations (DE) control state State function satisfies DE Control affects DE Goal (objective functional) Lecture1 p.4/37
Basic Idea System of ODEs modeling situation Decide on format and bounds on the controls Design an appropriate objective functional Derive necessary conditions for the optimal control Compute the optimal control numerically Lecture1 p.5/37
Design an appropriate objective functional balancing opposing factors in functional include (or not) terms at the final time Lecture1 p.6/37
Big Idea In optimal control theory, after formulating a problem appropriate to the scenario, there are several basic problems : (a) to prove the existence of an optimal control, (b) to characterize the optimal control, (c) to prove the uniqueness of the control, (d) to compute the optimal control numerically, (e) to investigate how the optimal control depends on various parameters in the model. Lecture1 p.7/37
# # " & Deterministic Optimal Control- ODEs Find piecewise continuous control and associated state variable to maximize $ #! " subject to ' # & % and ( * *) Lecture1 p.8/37
, +,, 0 +, 0,, 0 + Contd. Optimal Control Put / -., / -. / -. / -. achieves the maximum into state DE and obtain corresponding optimal state / -. optimal pair / -. Lecture1 p.9/37
2 2 6 1 7 2 2 6 1 2 2 6 1 Necessary, Sufficient Conditions Necessary Conditions If, are optimal, then the following conditions hold. 5 3 4 5 3 4 Sufficient Conditions If, and (adjoint) satisfy the conditions. then 5 3 4, 5 3 4 5 3 4 5 3 4 are optimal. Lecture1 p.10/37
Adjoint like Lagrange multipliers to attach DE to objective functional. Lecture1 p.11/37
8 < @ 8 < B A < 8 < B < A < E < Deterministic Optimal Control- ODEs Find piecewise continuous control and associated state variable to maximize ; 9 : ; 9 : : ; ;C 9 : ; 9 : 9 : B >? = subject to ; ; 9 : ; 9 : 9 : F B E ; D 9 : ; 9 and ; 9G I IH Lecture1 p.12/37
K K L J M R Q M K Q J P M K Q J Q S P U Q S P M K N P Q J Q S W V T Quick Derivation of Necessary Condition Suppose is an optimal control and corresponding state. variation function,. P N O u*(t)+ ah(t) u*(t) P N O N O T another control. state corresponding to P N O t, P P P N O N O T N O T N O T UO Lecture1 p.13/37
_ \ Z X [ Y \ b _ \ X b c [ ` e _ \ X f _ \ X b \ \ _ \ d X ^ c [ ^ Y a d Z Y ^ ^ Lecture1 p.14/37 Contd. `a Y ^ Z ], At y(t,a) x * (t) x 0 t all trajectories start at same position control Z ] Y ^ when Y _ Z X ] _g _ ^ ] X ] X ]. occurs at w.r.t. Maximum of
m hi v u t h w j v j k u t h w j v j y k t v Contd. nononpnrqs jlk h k m v k { mlz y m~ k j} { mlz j} y hx m m j x { mlz j x jy hx vƒ u m k j} { mlz j} y m ~ v k { m z hx m m j x { m z j x jy hx gives m j k i to our Adding Lecture1 p.15/37
Œ ˆ Contd. Ž Ž Ž Ž Œ Š ˆ l š š Ž Ž œ Ž Ž Š š š l Ÿ Ž lž Ž. ˆ ž here we used product rule and Lecture1 p.16/37
µ «µ ¹ «³ ±»» µ ¾ ¹ ² ± û» Contd. ««Ž ³ ² l± «ª ««ª ««º µ ³ ² «± ½ º¼» ««Ž. Ž ² Ž» Ž» terms are Arguments of ³ o o o ÁÀ µ µ ª ¾ ª ¾ ½ º¼» ««º µ. Ž ² Ž» terms are Arguments of Lecture1 p.17/37
Ô Ô Õ Ó Õ Ó Ò Ò Û Ü Ý Ô Ô Ô Û Õ Ó Õ Ó ã Ò Ò Contd. s.t. Ç Å Æ Ä Choose adjoint equation ÙÌ Ô ÊŽË Ò ÌlØ Ñ ÊÈË Ì Ö Ô ÊŽË Ò ÏÐªÑ Î Ì Í ÉÊŽË È transversality condition Ì Í ÊÚ È ÌàË ÊŽË Ìß È Ø Þ Ö Ê ÐªÞ Û Í arbitrary variation Ì ÊŽË ß Ú â Ë Ûâ for all Ì Í ÊŽË Ò Ì Ø Þ ÊŽË È Ì Ô Ö Ê Ë Ò ÐªÞ á Optimality condition. Lecture1 p.18/37
ê ê ê é è ç è ç è ç ë æ æ æ æ î è è ð ð ë ë ñ ñ è ê ò é ò é ó ë ó ë ç ê ä é ð ë Using Hamiltonian Generate these Necessary conditions from Hamiltonian ä å æ éíì ä å æ ä å æ integrand (adjoint) (RHS of DE) maximize w.r.t. at ï ï é ì optimality eq. ï ï ä ô éíì ô adjoint eq. transversality condition Lecture1 p.19/37
õ ö ü û ú ù õ ö ü û ú ù Converted problem of finding control to maximize objective functional subject to DE, IC to using Hamiltonian pointwise. For maximization öõø at as a function of For minimization öõø at as a function of Lecture1 p.20/37
þ þ ÿ ý þ þ ÿ ý ý þ ý þ þ ÿ ý þ ÿ Two unknowns introduce adjoint and (like a Lagrange multiplier) Three unknowns nonlinear w.r.t., and Eliminate by setting and solve for in terms of Two unknowns and with 2 ODEs (2 point BVP) + 2 boundary conditions. and Lecture1 p.21/37
Pontryagin Maximum Principle are optimal for above problem, then there and If s.t. exists adjoint variable is defined by at each time, where Hamiltonian and transversality condition Lecture1 p.22/37
% ' # # # ) % # $ & + # * 0 " & % # & % # - 2 3 # 1 # ' 3 # # Hamiltonian $ # "!! $ & $ # "!! is linear w.r.t., w.r.t. maximizes $ & "!! $ "!! (, + $. bounded controls, Bang-bang control or singular control "/. Example: 1 cannot solve for & % - and solve for 54, set is nonlinear w.r.t. optimality equation. Lecture1 p.23/37
= K E = E G H Lecture1 p.24/37 Example 1? BDC A >@? ; < 7986 : A >JI A >@? A >@? A F >? E G subject to What optimal control is expected?
P R Q a ] Y Y S W R ] Y c ] ] R [ c g e R R [ h d e R e [ S d R a Tc c [ h h d Y ` c f P j i Q c [ ` l k j i Q c k i ^ ^ Lecture1 p.25/37 c T Example 1 worked WXV SUT V MON L W [ T` Z\[ Y R_^ RHS of DE integrand b [ b d at g R [ f ` c [ ] b Sd b d ` W [ Z c [ Q [ c [ g Y [ ` g m R ` cnm
s y { u z r t y s u { u ~ z Example 2 w sd x v w x v@w p9q o x vj y ƒ x v@w x v w x } v@w u ~ subject to What optimal control is expected? Lecture1 p.26/37
Œ ˆ Œ Š Œ Š Hamiltonian Š ˆ The adjoint equation and transversality condition give Œ @Ž Š Š @Ž Lecture1 p.27/37
š Lecture1 p.28/37 Example 2 continued and the optimality condition leads to Ÿ @ž œ š š The associated state is Ÿ @ž œ
Graphs, Example 2 3 Example 2.2 2 State 1 0 1 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 Time 8 6 Adjoint 4 2 0 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 Time 0 2 Control 4 6 8 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 Time Lecture1 p.29/37
³ ¹ µ ±» ¾ ½ ¹ ½» ¹ µ µ ¼»» ¹ ¹» ¹ Lecture1 p.30/37 Example 3 U±² «ª ± «\µ _ º µ º ¼ ¾ µ at» µ ¹ µ ¼ º ¼ ± «À» ± µ» µ Áµ
É Æ È Â Ç Ç Æ Æ É Ç Ç Ñ Â Ò Ó Ì Ì Â Ü Ä É Æ Ç Æ Ç Ê Ø ÙÚ Lecture1 p.31/37 Æ Ê Contd. ÃÅÄ Â Â/Ê ÃÅÄ É Í Ä ËÐ É ÇÏÎ Í Ä ËÌ. and then get É Ò Â Î Solve for Do numerically with Matlab or by hand Ø ÙÚ ÔÖÛ È ÔÖÕ Æ Ê
Exercise Ý Þß à á â ãåä æ çè é ä ê ë ì í æ î ï ä ãjð ç ë ì ï æ control ñ ë ò ò æ ë ð ó ê ë ó ë ó ã ì ç ë æ ô ë ä ô ë Lecture1 p.32/37
Exercise completed õ ö ø ù ú û ü ýÿþ ü þ ü û þ control û ü þ ü ý þ ý û þ þ þ þ ø ø ü û ü þ û ü û û ý þ û û Lecture1 p.33/37
# Contd. "! There is not an Optimal Control" in this case. Want finite maximum. Here unbounded optimal state unbounded OC Lecture1 p.34/37
& $ ( ) $ ( $ + *, $ * 1 2 ' % ( $ / Opening Example ' %& ' %& Number of cancer cells at time (exponential growth) State Drug concentration Control ' %& ' %& )& ' %.- $0/ known initial data minimize )& ' %& See need for bounds on the control. See salvage term. Lecture1 p.35/37
Further topics to be covered Interpretation of the adjoint Salvage term Numerical algorithms Systems case Linear in the control case Discrete models Lecture1 p.36/37
3 4 more info See my homepage www.math.utk.edu lenhart Optimal Control Theory in Application to Biology short course lectures and lab notes Book: Optimal Control applied to Biological Models CRC Press, 2007, Lenhart and J. Workman Lecture1 p.37/37