Tutorial on Control and State Constrained Optimal Control Pro. Control Problems and Applications Part 3 : Pure State Constraints

Transcription

1 Tutorial on Control and State Constrained Optimal Control Problems and Applications Part 3 : Pure State Constraints University of Münster Institute of Computational and Applied Mathematics SADCO Summer School Imperial College, London, September 5, 20

2 Outline Theory of Optimal Control Problems with Pure State Constraints 2 Academic Example: order q = of the state constraint 3 Van der Pol Oscillator : order q = of the state constraint 4 Example: Immune Response 5 Example: Optimal Control of a Model of Climate Change

3 Optimal Control Problem with Pure State Constraints State x(t) R n, Control u(t) R (for simplicity). All functions are assumed to be sufficiently smooth. Dynamics and Boundary Conditions ẋ(t) = f (x(t), u(t)), t [0, t f ], x(0) = x 0 R n, ψ(x(t f )) = 0 R k, (0 = ϕ(x(0), x(t f )), mixed boundary conditions) Pure State Constraints and Control Bounds Minimize s(x(t)) 0, t [0, t f ], ( s : R n R ) α u(t) β, t [0, t f ]. tf J(u, x) = g(x(t f )) + f 0 (x(t), u(t)) dt 0

4 Hamiltonian and Minimum Principle Hamiltonian H(x, λ, u) = λ 0 f (x, u) + λ f (x, u) λ R n (row vector) Let (u, x) L ([0, t f ], R) W, ([0, t f ], R n ) be a locally optimal pair of functions. Then there exist an adjoint (costate) function λ W, ([0, t f ], R n ) and a scalar λ 0 0, a multiplier function of bounded variation µ BV ([0, t f ], R), a multiplier ρ R associated to the boundary condition ψ(x(t f )) = 0, that satisfy the following conditions for a.a. t [0, t f ], where the argument (t) denotes evaluations along the trajectory (x(t), u(t), λ(t)) :

5 Minimum Principle of Pontraygin et al. (Hestenes) (i) Adjoint integral equation and transversality condition: λ(t) = t f t t f H x (s) ds + s x (x(s)) dµ(s) t + (λ 0 g + ρψ) x (x(t f )) ( if s(x(t f )) < 0 ), (iia) Minimum Condition for Hamiltonian: H(x(t), λ(t), u(t)) = min { H(x(t), λ(t), u) α u β } (iii) positive measure dµ and complementarity condition: dµ(t) 0 and t f 0 s x (t))dµ(t) = 0

6 Order of a state constraint s(x(t)) 0 Define recursively functions s (k) (x, u) by s (0) (x, u) = s(x), s (k+) (x, u) = s(k) x (x, u) f (x, u), (k = 0,,..) Suppose there exist q N with s (k) u (x, u) 0, i.e., s(k) = s (k) (x), (k = 0,,.., q ), s (q) u (x, u) 0. Then along a solution of ẋ(t) = f (x(t), u(t)) we have s (k) (x(t)) = dk dt k s(x(t)) (k = 0,,.., q ), s (q (x(t), u(t)) = dq dt q s(x(t))

7 Regularity conditions to ensure dµ(t) = η(t)dt Regularity assumption on a boundary arc s(x(t)) = 0, t t t 2. s (q) u (x(t), u(t)) 0 t t t 2. Assumption on boundary control There exists a sufficiently smooth boundary control u = u b (x)) with s(x, u b (x)) 0. Assumption: α < u(t) = u b (x(t)) < β t < t < t 2. Regularity of multiplier (measure) µ The regularity and assumption on boundary control imply that there exist a smooth multiplier η(t) with dµ(t) = η(t) dt t < t < t 2

8 Minimum Principle under regularity Augmented Hamiltonian H(x, λ, η, u) = H(x, λ, u) + η s(x) Adjoint equation and jump conditions λ(t) = H x (t) = H x (t) η(t)s x (x(t)) λ(t k +) = λ(t k ) ν k s x (x(t k )), ν k 0 at each contact or junction time t k, ν k = µ(t k +) µ(t k ) Minimum condition H(x(t), λ(t), u(t)) = min { H(x(t), λ(t), u) α u β } H u (t) = 0 on boundary arcs t + < t < t 2

9 Academic Example Minimize J(x, u) = 2 0 ( u 2 + x 2 ) dt subject to ẋ = x 2 u, x(0) =, x(2) =, and the state constraint x(t) a, 0 t state trajectories x(t) "x.dat" "x-a=0.6.dat" "x-a=0.7.dat" "x-a=0.85.dat" Boundary arc x(t) a = 0.7 for t = 0.64 t t 2 =.386.

10 Academic Example: state and control trajectories state trajectories x(t) "x.dat" "x-a=0.6.dat" "x-a=0.7.dat" "x-a=0.85.dat" control u "u.dat" "u-a=0.6.dat" "u-a=0.7.dat" "u-a=0.85.dat"

11 Academic Example: Computation of Multiplier η State constraint: s(x) = a x 0. Order of the state constraint is q =, since s () (x, u) = ẋ = x 2 + u, (s () ) u =. Boundary control u = u b (x) with s () (x, u b (x)) 0 is given by u b (x) = x 2 = a 2. Augmented Hamiltonian and adjoint equation: H(x, λ, η, u) = u 2 + x 2 + λ(x 2 u) + η(a x), λ = H x = 2x 2λx + η. The minimum condition 0 = H u = 2u λ gives u = λ/2. Since u = a 2 holds on a boundary arc, we get λ = 0 and hence the multiplier η on the boundary η(t) 2a( + 2a 2 ) > 0.

12 Academic Example: state and control trajectories state trajectories x(t) "x.dat" "x-a=0.6.dat" "x-a=0.7.dat" "x-a=0.85.dat" multiplier eta

13 Van der Pol Oscillator t f Minimize J(x, u) = ( u 2 + x 2 + x 2 2) dt (t f = 4), 0 subject to ẋ = x 2, ẋ 2 = x + x 2 ( x 2 ) + u, x (0) = x 2 (0) =, x (t f ) = x 2 (t f ) = 0, state constraint x 2 (t) a 0 t t f state trajectories x 2 "x2.dat" "x2-a=-0.5.dat" "x2-a=-0.4.dat" "x2-a=-0.3.dat" Boundary arc x 2 (t) a = 0.4 for t = t t 2 = 2.62.

14 Van der Pol : Computation of Multiplier η Order of the state constraint is q =, since s(x) = a x and s () (x, u) = ẋ 2 = x + x 2 (x 2 ) u, (s () ) u = 0. Boundary control u = u b (x) with s () (x, u b (x)) 0 is given by u b (x) = x + x 2 (x 2 ) = x + a(x 2 ). Augmented Hamiltonian and adjoint equation: H(x, λ, η, u) = u 2 + x 2 + x λ x 2 + λ 2 ( x + x 2 ( x 2) + u) +η(a x), λ = λ 2 = H x = 2x + λ 2 ( + 2x x 2 ), H x2 = 2x 2 λ λ 2 ( x 2) + η. The minimum condition 0 = H u = 2u + λ 2 gives u = λ 2 /2. Hence, x a + ax 2 = λ 2/2 holds on the boundary. Differentiation yields η = η(x, λ) = 4a 2 x + λ + λ 2 ( x 2 ).

15 Van der Pol Oscillator : optimal control state trajectories x 2 "x2.dat" "x2-a=-0.5.dat" "x2-a=-0.4.dat" "x2-a=-0.3.dat" control u Boundary arc x 2 (t) a = 0.4 for t = t t 2 = 2.62.

16 Van der Pol Oscillator : multiplier η state trajectories x 2 "x2.dat" "x2-a=-0.5.dat" "x2-a=-0.4.dat" "x2-a=-0.3.dat" multiplier eta Boundary arc x 2 (t) a = 0.4 for t = t t 2 = 2.62.

17 CASE I : Hamiltonian is regular CASE I : control u appears nonlinearly and U = R Assume that the Hamiltonian is regular, i.e., H(x, λ, u) admits a unique minimum with respect to u, the strict Legendre condition H uu (t) > 0 holds. Let q be the order of the state constraint s(x) 0 and consider a boundary arc with s(x(t)) = 0 t t t 2. Junction conditions q = : The control u(t) and the adjoint variable λ(t) are continuous at t k, k =, 2. q = 2 : The control u(t) is continuous but the adjoint variable may have jumps according to λ(t k +) = λ(t k ) ν k s x (t k )). q 3 and q odd: If the control u(t) is piecewise analytic then there are no boundary arcs but only contact points.

18 Case II : control u appears linearly Dynamics and Boundary Conditions ẋ(t) = f (x(t)) + f 2 (x(t)) u(t), a.e. t [0, t f ], x(0) = x 0 R n, ψ(x(t f )) = 0 R k, Control and State Constraints α u(t) β s(x(t) 0 t [0, t f ]. Minimize J(u, x) = g(x(t f )) + tf 0 ( f 0 (x(t)) + f 02 (x(t) ) u(t) dt

19 Case II : Hamiltonian and switching function Normal Hamiltonian H(x, λ, u) = f 0 (x) + λf (x) + [ f 02 (x) + λf 2 (x) ] u. Augmented Hamiltonian H(x, λ, µ, u) = H(x, λ, u) + µ s(x)). Switching function σ(x, λ) = H u (x, λ, u) = f 02 (x) + λf 2 (x), σ(t) = σ(x(t), λ(t)). On a boundary arc we have s(x(t)) = 0, t t t 2, α < u(t) < β, t < t < t 2. The minimum condition for the control implies 0 = H u (x(t), λ(t), u(t)) = σ(t), t + t t 2. Formally, the boundary control behaves like a singular control.

20 Case II : Boundary Control and Junction Theorem Let q be the order of the state constraint: s (q) (x, u) = d q dt q s(x) = s (x) + s 2 (x) u. The boundary control u = u b (x) is determined from s (q) (x, u) = 0 as u = u b (x) = s (x)/s 2 (x). Junction Theorem for q = Let q = and let a bang-bang arc be joined with a boundary arc at t (0, T ). Claim: If the control is discontinuous at t, then the adjoint variable is continouus at t.

21 Model of the immune response Dynamic model of the immune response: Asachenko A, Marchuk G, Mohler R, Zuev S, Disease Dynamics, Birkhäuser, Boston, 994. Optimal control: Stengel RF, Ghigliazza R, Kulkarni N, Laplace O, Optimal control of innate immune response, Optimal Control Applications and Methods 23, 9 04 (2002), Lisa Poppe, Julia Meskauskas: Diploma theses, Universität Münster (2006,2008).

22 Innate Immune Response: state and control variables State variables: x (t) : concentration of pathogen (=concentration of associated antigen) x 2 (t) : concentration of plasma cells, which are carriers and producers of antibodies x 3 (t) : concentration of antibodies, which kill the pathogen (=concentration of immunoglobulins) x 4 (t) : relative characteristic of a damaged organ ( 0 = healthy, = dead ) Control variables: u (t) : pathogen killer u 2 (t) : plasma cell enhancer u 3 (t) : antibody enhancer u 4 (t) : organ healing factor

23 Generic dynamical model of the immune response ẋ (t) = ( x 3 (t))x (t) u (t), ẋ 2 (t) = 3A(x 4 (t))x (t d)x 3 (t d) (x 2 (t) 2) + u 2 (t), ẋ 3 (t) = x 2 (t) ( x (t))x 3 (t) + u 3 (t), ẋ 4 (t) = x (t) x 4 (t) u 4 (t). Immune deficiency function triggered by target organ damage { } cos(πx4 ), 0 x A(x 4 ) = x 4 For 0.5 x 4 (t) the production of plasma cells stops. State delay d 0 in variables x and x 3 Initial conditions (d = 0) : x 2 (0) = 2, x 3 (0) = 4/3, x 4 (0) = 0 Case : x (0) =.5, decay, requires no therapy (control) Case 2 : x (0) = 2.0, slower decay, requires no therapy Case 3 : x (0) = 3.0, diverges without control (lethal case)

24 Optimal control model: cost functional State x = (x, x 2, x 3, x 4 ) R 4, Control u = (u, u 2, u 3, u 4 ) R 4 L 2 -functional quadratic in control: Stengel et al. Minimize J 2 (x, u) = x (t f ) 2 + x 4 (t f ) 2 t f + ( x 2 + x u2 + u2 2 + u2 3 + u2 4 ) dt 0 L -functional linear in control Minimize J (x, u) = x (t f ) 2 + x 4 (t f ) 2 t f + ( x 2 + x u + u 2 + u 3 + u 4 ) dt 0 Control constraints: 0 u i (t) u max, i =,.., 4 Final time: t f = 0

25 L 2 functional, d = 0 : optimal state and control variables State variables x, x 2, x 3, x 4 and optimal controls u, u 2, u 3, u 4 : second-order sufficient conditions via matrix Riccati equation

26 L 2 functional, d = 0 : state constraint x 4 (t) 0.2 State and control variables for state constraint x 4 (t) 0.2. Boundary arc x 4 (t) 0.2 for t = t t 2 =.35

27 L 2 functional, multiplier η(t) for constraint x 4 (t) 0.2 Compute multiplier η as function of (x, λ) : η(x, λ) = λ 2 3π sin(πx 4 )x x 3 λ + 2λ 4 2x 3 x + 2x 4 Scaled multiplier 0. η(t) and boundary arc x 4 (t) = 0.2

28 L 2 functional, delay d > 0, constraint x 4 (t) α Dynamics with state delay d > 0 ẋ (t) = ( x 3 (t))x (t) u (t), ẋ 2 (t) = 3 cos(πx 4 ) x (t d)x 3 (t d) (x 2 (t) 2) + u 2 (t), ẋ 3 (t) = x 2 (t) ( x (t))x 3 (t) + u 3 (t), ẋ 4 (t) = x (t) x 4 (t) u 4 (t) x 4 (t) α 0.5 Initial conditions x (t) = 0, d t < 0, x (0) = 3, x 3 (t) = 4/3, d t 0, x 2 (0) = 2, x 4 (0) = 0.

29 L 2 functional : delay d = and x 4 (t) 0.2 State variables for d = 0 and d =

30 L 2 functional : delay d = and x 4 (t) 0.2 Optimal controls for d = 0 and d =

31 L 2 functional, d = : multiplier η(t) for x 4 (t) 0.2 Compute multiplier η as function of (x, λ) : η(x, y, λ) = λ 2 3π sin(πx 4 )y y 3 λ + 2λ 4 2x 3 x + 2x 4 Scaled multiplier 0. η(t) and boundary arc x 4 (t) = 0.2 ; η(t) is discontinuous at t = d =

32 L functional : no delays Minimize J (x, u) = x (t f ) 2 + x 4 (t f ) 2 t f + ( x 2 + x u + u 2 + u 3 + u 4 ) dt 0 Dynamics with delay d and control constraints ẋ (t) = ( x 3 (t))x (t) u (t), ẋ 2 (t) = 3A(x 4 (t))x (t)x 3 (t) (x 2 (t) 2) + u 2 (t), ẋ 3 (t) = x 2 (t) ( x (t))x 3 (t) + u 3 (t), ẋ 4 (t) = x (t) x 4 (t) u 4 (t), 0 u i (t) u max, 0 t t f (i =,.., 4)

33 L functional : u max = 2

34 L functional : non-delayed, time optimal control for x (t f ) = x 4 (t f ) = 0, x 3 (t f ) = 4/3 u max = : minimal time t f = 2.25, singular arc for u 4 (t)

35 Dynamical Model of Climate Change A. Greiner, L. Grüne, and W. Semmler, Growth and Climate Change: Threshhold and Multiple Equilibria, Working Paper, Schwartz Center for Economic Policy Studies, The New School, 2009, to appear. State Variables: K(t) : Capital (per capita) M(t) : CO 2 concentration in the atmosphere T (t) : Temperature (Kelvin) Control Variables C(t) : Consumption A(t) : Abatement per capita

36 Dynamical Model of Climate Change Production : Y = K 0.8 D(T T o ), T o = 288 (K) Damage : D(T T o ) = (0.025 (T T o ) 2 + ) Dynamics of per-capita capital K K = Y C A (δ + n)k, K(0) = K 0. ( δ = 0.075, n = 0.03 ) Emission : E = K/A Dynamics of CO 2 concentration M Ṁ = 0.49 E 0. M, M(0) = M 0.

37 Dynamical Model of Climate Change (continued) Albedo (non-reflected energy) at temperature T (Kelvin): α (T ) = ( ) 2 k π(t 293) + k π 2 2, k = , k 2 = Radiative forcing : 5.35 ln (M). Outgoing radiative flux (Stefan-Boltzman-law) : ɛ σ T T 4. Parameters : ɛ = 0.95, σ T = , cth = , Q = 367. Dynamics of temperature T with delay d 0 Ṫ (t) = cth [ ( α (T (t)) Q ɛ σ T T (t) ln (M(t d)) ], T (0) = T 0.

38 Optimal Control Model of Climate Change Control constraints for 0 t t f = 200 : 0 < C(t) C max =, A(t) State constraints of order 2 and 3 : M(t) M max, 0 t t f = 200, T (t) T max, t e = 20 t t f = 200. Maximize consumption t f J(K, M, T, C, A) = e (n ρ)t ln (C(t)) dt 0

39 Stationary points of the canonical system Constant Abatement A = : 3 stationary points T s := , M s = , K s =.44720, T s := , M s =.90954, K s =.34726, T s := , M s = , K s = Control variable abatement A(t) : Social Optimum T s := , M s =.28500, K s =

40 T (0) = 29, M(0) = 2.0, K(0) =.4 Concentration M, Emission E Temperature T (Kelvin) Consumption C Capital K

41 T (0) = 29, M(0) = 2.0, K(0) =.4 T (t f ) = 29, M(t f ) =.8, K(t f ) =.4 Concentration M, Emission E Temperature T (Kelvin) Consumption C Capital K

42 T (0) = 29, M(0) = 2.0, K(0) =.4 T (t f ) = 29, M(t f ) =.8, K(t).2, 0.85 C(t) Concentration M, Emission E Temperature T (Kelvin) Consumption C Capital K

43 d = 0 : T (0) = 29, M(0) =.8, K(0) =.4 T (t f ) = 29, M(t f ) =.8, K(t) =.4 Concentration M, Emission E Temperature T (Kelvin) Consumption C Capital K

44 T (0) = 294.5, M(0) = 2.2, K(0) =.4 T (t f ) = 29, M(t f ) =.8, K(t) =.4, 0.85 C(t) Concentration M, Emission E Temperature T (Kelvin) Consumption C Capital K

45 d = 0 : T (0) = 294.5, M(0) = 2.2, K(0) =.4 T (t f ) = 29, K(t) =.4, 0.85 C(t) State constraint of order two : M(t).9 for 30 t t f = 200 Concentration M, Emission E Temperature T (Kelvin) Consumption C Capital K

46 d = 0 : T (0) = 294.5, M(0) = 2.2, K(0) =.4 T (t f ) = 29, K(t) =.4, 0.85 C(t) State constraint of order two : M(t).9 for 30 t t f = 200 Concentration M, Emission E adjoint variable lambda K adjoint variable lambda M adjoint variable lambda T