Theory and Applications of Constrained Optimal Control Proble

Transcription

1 Theory and Applications of Constrained Optimal Control Problems with Delays PART 1 : Mixed Control State Constraints Helmut Maurer 1, Laurenz Göllmann 2 1 Institut für Numerische und Angewandte Mathematik, Universität Münster, Germany 2 Department of Mechanical Engineering, University of Applied Sciences, Steinfurt, Germany ANOC Spring School and Workshop, ENSTA Paris, April 23 27, 212

2 Overview of Part 1 Constrained Optimal Control Problems with Delays Discretization and NLP methods ( Discretize and Optimize ) Illustrative Example Minimum Principle for Delayed Optimal Control Problems with Mixed Control State Constraints Optimal Exploitation of a Renewable Resource with Investment Delay Optimal Control of Continuous Stirred Tank Reactors (CSTR)

3 Constrained Optimal Control Problems with Delays State variable x R n, Control variable u R m. Dynamics and Boundary Conditions ẋ(t) = f (t, x(t), x(t d x ), u(t), u(t d u )), a.e. t [, t f ], x(t) = x (t), t [ d x, ) (state delay d x ), u(t) = u (t), t [ d u, ) (control delay d u ), ψ(x(t f )) = Mixed control-state constraints and pure state constraints C(x(t), u(t)), S(x(t)), t [, t f ] Minimize J(u, x) = g(x(t f )) + tf f (t, x(t), x(t d x ), u(t), u(t d u )) dt

4 Discretization and NLP For simplicity consider a MAYER-type problem with cost functional J(u, x) = g(x(t f )) Assume: there exists a stepsize h > and integers k, l, N N with d x = k h, d u = l h, t f = N h. For simplicity: EULER discretization at grid points t i := i h, i =, 1,..., N. Approximation at grid points: u(t i ) u i R m (i =,..., N), x(t i ) x i R n (i =,..., N)

5 Discretization and NLP For simplicity consider a MAYER-type problem with cost functional J(u, x) = g(x(t f )) Assume: there exists a stepsize h > and integers k, l, N N with d x = k h, d u = l h, t f = N h. For simplicity: EULER discretization at grid points t i := i h, i =, 1,..., N. Approximation at grid points: u(t i ) u i R m (i =,..., N), x(t i ) x i R n (i =,..., N)

6 Discretization and NLP For simplicity consider a MAYER-type problem with cost functional J(u, x) = g(x(t f )) Assume: there exists a stepsize h > and integers k, l, N N with d x = k h, d u = l h, t f = N h. For simplicity: EULER discretization at grid points t i := i h, i =, 1,..., N. Approximation at grid points: u(t i ) u i R m (i =,..., N), x(t i ) x i R n (i =,..., N)

7 Discretization and NLP For simplicity consider a MAYER-type problem with cost functional J(u, x) = g(x(t f )) Assume: there exists a stepsize h > and integers k, l, N N with d x = k h, d u = l h, t f = N h. For simplicity: EULER discretization at grid points t i := i h, i =, 1,..., N. Approximation at grid points: u(t i ) u i R m (i =,..., N), x(t i ) x i R n (i =,..., N)

8 Large-scale NLP Problem Minimize J(u, x) = g(x N ) subject to x i+1 x i h f (t i, x i, x i k, u i, u i l ) =, i =,.., N 1, ψ(x N ) =, C(x i, u i ), i =,.., N, S(x i ), i =,.., N, x i := x ( ih) (i =,.., k), u i := u ( ih) (i = 1,.., l). Optimization Variable: z := (u, x 1, u 1, x 2,..., u N 1, x N ) R N(m+n)

9 Large-scale NLP Problem Minimize J(u, x) = g(x N ) subject to x i+1 x i h f (t i, x i, x i k, u i, u i l ) =, i =,.., N 1, ψ(x N ) =, C(x i, u i ), i =,.., N, S(x i ), i =,.., N, x i := x ( ih) (i =,.., k), u i := u ( ih) (i = 1,.., l). Optimization Variable: z := (u, x 1, u 1, x 2,..., u N 1, x N ) R N(m+n)

10 Application of NLP-Solvers AMPL : Programming language (Fourer, Gay, Kernighan) IPOPT: Interior point method (A. Wächter et al.) LOQO: Interior point method (B. Vanderbei et al. WORHP : SQP method (C. Büskens, M. Gerdts) Other NLP solvers embedded in AMPL : cf. NEOS server Special feature: solvers provide LAGRANGE-multipliers

11 A non-convex control problem with state delays state x(t) R, control u(t) R, delay d Dynamics and Boundary Conditions ẋ(t) = x(t d) 2 u(t), t [, 2], x(t) = x (t) = 1, t [ d, ], x(2) = 1 Control and State Constraints x(t) α, i.e., S(x(t)) = x(t) + α, t [, 2] Minimize J(u, x) = 2 (x(t) 2 + u(t) 2 ) dt

12 Optimal solutions without state constraints min 2 (x(t) 2 +u(t) 2 ) dt s.t. ẋ(t) = x(t d) 2 u(t), x (t) 1, x(2) = 1 optimal state and control for delays d =., d =.1, d =.2, d =.5, state x d= d=.1 d=.2 d= control u d= d=.1 d=.2 d=

13 Optimal solutions with state constraint x(t) α =.7 Optimal state and control for delays d =., d =.1, d =.2, d = state x d= d=.1 d=.2 d= control u d= d=.1 d=.2 d=

14 Minimum Principle Augmented Hamiltonian: y(t) = x(t d) H(x, y, λ, η, u) = u 2 + x 2 + λ(y 2 u) + η( x + α) Adjoint equation λ(t) = H x (t) χ [,2 d] H y (t + d) = { 2x(t) 2λ(t + d)x(t) + η(t), t 2 d } 2x(t) + η(t), 2 d t 2 Minimum condition H u (t) = u(t) = λ(t)/2

15 Boundary arc x(t) = α =.7 for t 1 t t 2 x(t) α ẋ(t) = x(t d) 2 = u(t) = λ(t)/2 Computation of multiplier η(t) by differentiation η(t) = 2(2x(t d)(x(t 2d) 2 λ(t d)/2)+x(t)+λ(t +d)x(t)) 1 delays d =., d =.1, d =.2, d =.5, d = 1. state x d= d=.1 d=.2 d= multiplier for state constraint x(t) >= alpha

16 Optimal Control Problems with Control-State Constraints State variable x R n, Control variable u R m. Dynamics and Boundary Conditions ẋ(t) = f (t, x(t), x(t d x ), u(t), u(t d u )), a.e. t [, t f ], x(t) = x (t), t [ d x, ) (state delay d x ), u(t) = u (t), t [ d u, ) (control delay d u ), ψ(x(t f )) = Mixed control-state constraints and pure state constraints C(x(t), u(t)), t [, t f ] Minimize J(u, x) = g(x(t f )) + tf f (t, x(t), x(t d x ), u(t), u(t d u )) dt

17 References for Minimum/Maximum Principle State delays and pure control constraints: Kharatishvili (1961), Oguztöreli (1966), Banks (1968), Halanay (1968), Soliman, Ray (197), Colonius, Hinrichsen (1978), Clarke, Wolenski (1991), Dadebo, Luus (1992), Mordukhovich, Wang (23 ) State delays and pure state constraints: Angell, Kirsch (199) State and control delays and mixed control state constraints: L. Göllmann, D. Kern, H. Maurer. Optimal control problems with delays in state and control and mixed control-state constraints. Optimal Control Applications and Methods 3, (29).

18 Hamiltonian for mixed control-state constraints (Augmented) Hamiltonian or Pontryagin Function H(t, x, y, u, v, λ, µ) := λ f (t, x, y, u, v) + λf (t, x, y, u, v) + µc(t, x, u) y represents delayed state v represents delayed control λ R n, λ R adjoint (costate) variables µ R k multiplier for control-state constraint C(x, u)

19 Pontryagin (Type) Minimum Principle Let (u, x) L ([, t f ], R m ) W 1, ([, t f ], R n ) be a locally optimal pair of functions. Regularity assumption: rank ( Cj (x(t),u(t)) u ) j J (t) = # J (t), J (t) := { j {1,.., p} C j (x(t), u(t)) = } Then there exist an adjoint function λ W 1, ([, t f ], R n ) and λ, a multiplier function µ L ([, t f ], R k ), and a multiplier ρ R q such that the following conditions are satisfied for a.a. t [, t f ] :

20 Pontryagin Type Minimum Principle (i) Advanced adjoint ODE and transversality condition: λ(t) = H x (t) χ [,tf d x ](t)h y (t + d x ), λ(t ) = (λ g + ρψ) x (x(t f )), where H x (t) and H y (t + d x ) denote evaluations along the optimal trajectory and χ [,t dx ] is the characteristic function. (ii) Local Minimum Condition for augmented Hamiltonian: = H u (t) + χ [,T du](t)h v (t + d u ) (iii) Non-negativity of multiplier and complementarity condition: µ(t) and µ i (t) C i (x(t), u(t)) =, i = 1,..., k.

21 Pontryagin Type Minimum Principle (i) Advanced adjoint ODE and transversality condition: λ(t) = H x (t) χ [,tf d x ](t)h y (t + d x ), λ(t ) = (λ g + ρψ) x (x(t f )), where H x (t) and H y (t + d x ) denote evaluations along the optimal trajectory and χ [,t dx ] is the characteristic function. (ii) Local Minimum Condition for augmented Hamiltonian: = H u (t) + χ [,T du](t)h v (t + d u ) (iii) Non-negativity of multiplier and complementarity condition: µ(t) and µ i (t) C i (x(t), u(t)) =, i = 1,..., k.

22 Pontryagin Type Minimum Principle (i) Advanced adjoint ODE and transversality condition: λ(t) = H x (t) χ [,tf d x ](t)h y (t + d x ), λ(t ) = (λ g + ρψ) x (x(t f )), where H x (t) and H y (t + d x ) denote evaluations along the optimal trajectory and χ [,t dx ] is the characteristic function. (ii) Local Minimum Condition for augmented Hamiltonian: = H u (t) + χ [,T du](t)h v (t + d u ) (iii) Non-negativity of multiplier and complementarity condition: µ(t) and µ i (t) C i (x(t), u(t)) =, i = 1,..., k.

23 Proof ideas: delay chain arguments Apply Kuhn Tucker conditions for optimization problems in Banach spaces. Simple proof: Rationality assumption for delays: d x d u = k l Q with k, l N coprime integers. This implies d x = k h max, d u = l h max, h max := d u l maximal stepsize Then augment the control system by the number of intervals of length h max in [, t f ] and use continuity of state variables. Apply the standard Minimum Principle to the augmented control system and translate the conditions back to the retarded control problem.

24 Optimal Fishing, Clark, Clarke, Munro Colin W. Clark, Frank H. Clarke, Gordon R. Munro: The optimal exploitation of renewable resource stock: problem of irreversible investment, Econometric 47, pp (1979). State variables and control variables: x(t) : population biomass at time t [, t f ], renewable resource, e.g., fish, K(t) : amount of capital invested in the fishery, e.g., number of standardized fishing vessels available, E(t) : fishing effort (control), h(t) = E(t)x(t) is harvest rate, I (t) : investment rate (control),

25 Optimal Fishing Dynamics in [, t f ] ( parameters a = 1, b = 5, γ =.3 ) ẋ(t) = a x(t) (1 x(t)/b) E(t) x(t), x() = x, K(t) = I (t d) γ K(t), K() = K. Mixed Control-State Constraint and Control Constraint E(t) K(t), I (t) I max, t [, t f ], Maximize benefit ( parameters r =.5, c E = 2, c I = 1.1 ) J(u, x) = tf exp( r t)( p E(t) x(t) c E E(t) c I I (t) ) dt Parameters in Clark et al.: I max =, γ =, d =.

26 Optimal Fishing : Necessary Conditions Controls E and I appear linearly in the Hamiltonian. Switching functions σ E (t) = exp( r t)(px(t) c E ) λ x x, { exp( r t)ci + λ σ I (t) = K (t + d), t < t f d exp( r t)c I, t f d t t f } Optimal controls maximizing the Hamiltonian K(t) for σ E (t) > E(t) = for σ E (t) < singular for σ E (t) = on I s [, t f ] { } Imax for σ I (t) = I (t) > for σ I (t) <

27 Optimal Fishing: x = 1.5, K =.1, I max =.1 fishing rate E, investment I, capital K 1.8 E I K E, K, I time t (years)

28 Optimal Fishing: x = 1.5, K =.1, I max =.1 Switching functions σ E (t) and σ I (t) E, K, σ E rate E, capital K, switching function σ E time t (years) I, σ I Investment I and switching function σ I time t (years) I σ I

29 Optimal Fishing: d =.5, x = 1.5, K =.1, I max =.1 fishing rate E, investment I, capital K 1.8 E I K E, K, I time t (years)

30 Optimal Fishing: d =.5, x = 1.5, K =.1, I max =.1 Switching functions σ E (t) and σ I (t) E, K, σ E rate E, capital K, switching function σ E time t (years) Investment I and function σ I I, σ I time t (years)

31 Optimal Fishing: x = 1.5, K =.5, I max =.1 fishing rate E, investment I, capital K 1.8 E I K E, K, I time t (years)

32 Optimal Fishing: d =.5, x = 1.5, K =.5, I max =.1 fishing rate E, investment I, capital K 1.8 E I K E, K, I time t (years)

33 Optimal Fishing: d =.5, Impulsive control I max = 2 E, K, I fishing rate E, investment I, capital K time t (years) E I K

34 Continuous Stirred Chemical Tank Reactor (CSTR) Seinfeld S.H.: Optimal control of a continuous stirred tank reactor with transportation lag, Intern. Journal of Control 1, (1969). Ray W.H. and Soliman M.A.: The optimal control of processes containing pure time delays I, necessary conditions for an optimum, Chemical Engineering Science 25, (197). Soliman M.A. and Ray W.H.: Optimal control of multivariable systems with pure time delays, Automatica 7, (1971). Dadebo S. and Luus R. Optimal control of time-delay systems by dynamic programming, Optimal Control Applications and Methods 13, pp (1992).

35 Continuous Stirred Chemical Tank Reactor (CSTR)

36 CSTR Model State variables: (deviations from a desired final state) x 1 (t) : concentration of product x 2 (t) : concentration of catalyst, x 3 (t) : temperature inside vessel Control variables: u 1 (t) : temperature control by heat exchanger u 2 (t) : reactant inlet valve,

37 CSTR Model System Dynamics for t [,.2] : ẋ 1 (t) = x 1 (t) R(x 1 (t), x 2 (t), x 3 (t)), ẋ 2 (t) = x 2 (t) +.9u 2 (t d u ) +.1u 2 (t), ẋ 3 (t) = 2x 3 (t) +.25R(t) 1.5u 1 (t)x 3 (t d x ), ( ) 25x3 R(x 1, x 2, x 3 ) := (1 + x 1 )(1 + x 2 ) exp. 1 + x 3 Boundary conditions and control bounds: x 3 (t) =.2, t [ d x, ), d x =.15 u 2 (t) = 1, t [ d u, ), d u =.2 x() = (.49,.2,.2), x(.2) = (,, ), u 1 (t) 5, t [,.2].

38 CSTR Model System Dynamics for t [,.2] : ẋ 1 (t) = x 1 (t) R(x 1 (t), x 2 (t), x 3 (t)), ẋ 2 (t) = x 2 (t) +.9u 2 (t d u ) +.1u 2 (t), ẋ 3 (t) = 2x 3 (t) +.25R(t) 1.5u 1 (t)x 3 (t d x ), ( ) 25x3 R(x 1, x 2, x 3 ) := (1 + x 1 )(1 + x 2 ) exp. 1 + x 3 Boundary conditions and control bounds: x 3 (t) =.2, t [ d x, ), d x =.15 u 2 (t) = 1, t [ d u, ), d u =.2 x() = (.49,.2,.2), x(.2) = (,, ), u 1 (t) 5, t [,.2].

39 CSTR Model Objective functional: quadratic in control u 2 : Minimize J(u, x) =.2 ( x(t) u 2 (t) 2 )dt Numerical solution: discretize and optimize Delays: d x =.15, d u =.2 EULER discretization: N = 16, meshpoints Performance Index computed: J(x, u) = CPU Time: 63, 932 CPU seconds (!) Solver used: IPOPT with AMPL

40 CSTR Model Objective functional: quadratic in control u 2 : Minimize J(u, x) =.2 ( x(t) u 2 (t) 2 )dt Numerical solution: discretize and optimize Delays: d x =.15, d u =.2 EULER discretization: N = 16, meshpoints Performance Index computed: J(x, u) = CPU Time: 63, 932 CPU seconds (!) Solver used: IPOPT with AMPL

41 CSTR: Numerical Results - State and Adjoint Variables

42 CSTR: Numerical Results - State and Adjoint Variables

43 CSTR: Numerical Results - Control

44 CSTR: Numerical Results - Control

45 CSTR: Numerical Results - Control

46 CSTR: Numerical Results - Control

47 Continuous Stirred Chemical Tank Reactor (CSTR) Seinfeld S.H.: Optimal control of a continuous stirred tank reactor with transportation lag, Intern. Journal of Control 1, (1969). Dadebo S. and Luus R. Optimal control of time-delay systems by dynamic programming, Optimal Control Applications and Methods 13, pp (1992).

48 CSTR (Seinfeld): Delay in mixed control-state constraint x 1 (t) : normalized concentration of product, t 4, x 2 (t) : normalized temperature, u(t) : temperature control ( Dynamics: Reaction term R(x 2 ) = exp 25x2 : ( ) ẋ 1 (t) = x 1 (t) (1 + x 1 (t)) k R(x 2 (t)) 1, k = 1, 2, ( ) ẋ 2 (t) = 2x 2 (t) +.25 (1 + x 1 (t)) k R(x 2 (t)) 1 1+x 2 ) u(t)x 2 (t d x )(x 2 (t) +.125). Mixed control state constraint with state delay: 1 u(t)x 2 (t.1) 1

49 CSTR (Seinfeld): Delay in mixed control-state constraint x 1 (t) : normalized concentration of product, t 4, x 2 (t) : normalized temperature, u(t) : temperature control ( Dynamics: Reaction term R(x 2 ) = exp 25x2 : ( ) ẋ 1 (t) = x 1 (t) (1 + x 1 (t)) k R(x 2 (t)) 1, k = 1, 2, ( ) ẋ 2 (t) = 2x 2 (t) +.25 (1 + x 1 (t)) k R(x 2 (t)) 1 1+x 2 ) u(t)x 2 (t d x )(x 2 (t) +.125). Mixed control state constraint with state delay: 1 u(t)x 2 (t.1) 1

50 CSTR : Delay in mixed control-state constraint Initial conditions: Delay d x =.1 Cost functional: Minimize x 1 () =.82298, x 2 (t) =.1286, t [.1, ], J(x, u) = 4 x 1 (t) 2 dt Switching function : σ(t) = λ 2 (t)x 2 (t d x )(x 2 (t) +.125). { } sign (σ(t)), if σ(t) u(t)x 2 (t d x ) = singular, if σ(t) =

51 CSTR : Numerical solution x x u constraint u(t)*x 2 (t-d) Constraint u(t)x 2 (t.1) is bang singular!

52 Two Stage CSTR Dadebo S. and Luus R. Optimal control of time-delay systems by dynamic programming, Optimal Control Applications and Methods 13, pp (1992). A chemical reaction A B is processed in two tanks. State and control variables: Tank 1 : x 1 (t) : (scaled) concentration x 2 (t) : (scaled) temperature u 1 (t) : temperature control Tank 2 : x 3 (t) : (scaled) concentration x 4 (t) : (scaled) temperature u 2 (t) : temperature control

53 Dynamics of the Two-Stage CSTR ( ) Reaction term in Tank 1 : R 1 (x 1, x 2 ) = (x 1 +.5) exp 25x2 1+x ( 2 ) Reaction term in Tank 2 : R 2 (x 3, x 4 ) = (x ) exp 25x4 1+x 4 Dynamics: ẋ 1 (t) ẋ 2 (t) =.5 x 1 (t) R 1 (t), = (x 2 (t) +.25) u 1 (t)(x 2 (t) +.25) + R 1 (t), ẋ 3 (t) = x 1 (t d) x 3 (t) R 2 (t) +.25, ẋ 4 (t) = x 2 (t d) 2x 4 (t) u 2 (t)(x 4 (t) +.25) + R 2 (t).25. Initial conditions: x 1 (t) =.15, x 2 (t) =.3, d t, x 3 () =.1, x 4 () =. Delays d =.1, d =.2, d =.4.

54 Optimal control problem for the Two-Stage CSTR Minimize t f ( x1 2 + x x x u u2 2 ) dt (t f = 2). Hamiltonian with y k (t) = x k (t d), k = 1, 2 : H(x, y, λ, u) = f (x, u) + λ 1 ẋ 1 +λ 2 ( (x ) u 1 (x ) + R 1 (x 1, x 2 ) ) +λ 3 (y 1 x 3 R 2 (x 3, x 4 ) +.25) +λ 4 (y 2 2x 4 u 2 (x ) + R 2 (x 3, x 4 ) +.25) Adjoint equations: λ 1 (t) = H x1 (t) χ [,tf d ] λ 3 (t + d), λ 2 (t) = H x2 (t) χ [,tf d ] λ 4 (t + d), λ k (t) = H xk (t) (k = 3, 4). The minimum condition yields H u = and thus u 1 = 5λ 2 (x ), u 2 = 5λ 4 (x ).

55 Two-Stage CSTR with free x(t f ) : x 1, x 2, x 3, x concentration x 1 d=.1 d=.2 d= temperature x 2 d=.1 d=.2 d= concentration x 3 d=.1 d=.2 d= temperature x 4 d=.1 d=.2 d= Delays d =.1, d =.2, d =.4.

56 Two-Stage CSTR with free x(t f ) : u 1, u 2, λ 1, λ control u 1 d=.1 d=.2 d= control u 2 d=.1 d=.2 d= adjoint variable λ 1 d=.1 d=.2 d= adjoint variable λ 2 d=.1 d=.2 d= Delays d =.1, d =.2, d =.4.

57 Two-Stage CSTR with x(t f ) = : x 1, x 2, x 3, x concentration x 1 d=.1 d=.2 d= temperature x 2 d=.1 d=.2 d= concentration x 3 d=.1 d=.2 d= temperature x 4 d=.1 d=.2 d= Delays d =.1, d =.2, d =.4.

58 Two-Stage CSTR with x(t f ) = : u 1, u 2, λ 1, λ control u 1 d=.1 d=.2 d= control u 2 d=.1 d=.2 d= adjoint variable λ 1 d=.1 d=.2 d= adjoint variable λ 2 d=.1 d=.2 d= Delays d =.1, d =.2, d =.4.

59 Two-Stage CSTR with x(t f ) = and x 4 (t) temperature x 4 d=.1 d=.2 d= control u 1 d=.1 d=.2 d= multiplier µ for x 4 <=.1 d=.1 d=.2 d= control u 2 d=.1 d=.2 d= Delays d =.1, d =.2, d =.4.