Second Order Analysis of Control-affine Problems with Control and State Constraints

Transcription

1 & state constraint Second Order Analysis of Control-affine Problems with Control and State Constraints María Soledad Aronna Escola de Matemática Aplicada, Fundação Getulio Vargas, Rio de Janeiro, Brazil Mathematical Control Theory With a special session in honor of Gianna Stefani Porquerolles, France, June (joint works with A. Dmitruk, J.-F. Bonnans, B.-S. Goh, P. Lotito)

2 & state constraint 1 2 & state constraint

3 & state constraint 1 2 & state constraint

4 & state constraint The problem min J(u) := ϕ(x(0), x(t )) s.t. ẋ(t) = f(x(t), u(t)) := f 0 (x(t)) + m u i (t) f i (x(t)), a.e. [0, T ], i=1 Φ(x(0), x(t )) K Φ = {0} R n 1 R n 2, u i,min u(t) u i,max, i = 1,..., m, a.e. [0, T ]. Control and state spaces: L (0, T ); W 1, ([0, T ]; R n ). Assuming feasibility, one has existence of optimal trajectory (ˆx, û).

5 & state constraint Hamiltonian and Lagrangian functions Let λ := (β, Ψ, p) R + R (n 1+n 2 ) W 1, ([0, T ]; R n ). Let us define the (unmaximized) pre-hamiltonian: the endpoint Lagrangian: H[λ](x, u, t) := p(t) m u i f i (x), i=0 l β,ψ (x(0), x(t )) := βϕ(x(0), x(t )) + Ψ.Φ(x(0), x(t )), and the Lagrangian function T ( ) L[λ](x, u) := l β,ψ (x(0), x(t )) + p(t) f(x(t), u(t)) ẋ(t) dt. 0

6 & state constraint First order optimality conditions (I) Costate equation: for p W 1, ([0, T ]; R n ), ṗ(t) = p(t)f x (x(t), u(t)), a.e. on [0, T ], with endpoint conditions ( p(0), p(t )) = Dl β,ψ (x(0), x(t )). (II) Minimization of Hamiltonian (affine w.r.t. u i ) for a.a. t [0, T ], u i (t) = u i,min, if p(t)f i (x(t)) > 0, u i (t) = u i,max, if p(t)f i (x(t)) < 0, p(t)f i (x(t)) = 0, if u i,min < u i (t) < u i,max.

7 & state constraint First order optimality conditions (III) Final constraints multiplier: Ψ R (n 1+n 2 ) ; Ψ i 0, Ψ i Φ i ( x(0), x(t ) ) = 0, i = n1 + 1,..., n 2. Λ(x, u) = set of Pontryagin multipliers associated to (x, u) : λ := (β, Ψ, p) R + R (n 1+n 2 ) W 1, ([0, T ]; R n ) that satisfy (I)-(III) and β + Ψ = 1. PMP: Λ.

8 & state constraint Singular arcs General definition ([Bryson-Ho 1975]): u has a singular arc on (a, b) [0, T ] if (i) Stationarity condition: H u = 0 a.e. on (a, b), (ii) H uu = 0 a.e. on (a, b). For each component: when u i,min < u i (t) < u i,max, on (a, b) [0, T ], we say that u i has a singular arc on (a, b). If u i (t) = u i,min on (a, b) or u i (t) = u i,max on (a, b) then u i has a bang arc on (a, b).

9 & state constraint Bang and singular sets/arcs Bang sets for the control constraint: B i, := {t [0, T ] : û i (t) = u i,min }, B i,+ := {t [0, T ] : û i (t) = u i,max }, and set B i := B i, B i,+. And the singular set S i := {t [0, T ] : u i,min < û i (t) < u i,max }. B, S arcs are maximal intervals contained in these sets.

10 & state constraint Previous second order conditions Necessary conditions: Kelley, Kopp, Moyer, Goh, Gabasov, Kirillova, Krener, Levitin, Milyutin, Osmolovskii, Agrachev, Sachkov, Gamkredlidze, more recently, Frankowska & Tonon, others... Sufficient conditions: Moyer, Dmitruk, Sarychev, Poggiolini & Stefani, Zezza, Bonnard, Caillau & Trélat, Maurer, Osmolovskii; Sussmann, Schättler, Jankovic, others...

11 & state constraint Assumptions on control structure & Complementarity (i) (ii) (iii) For each i, the interval [0, T ] is (up to a zero measure set) the disjoint union of finitely many arcs of type B and S, the control û i is at uniformly positive distance of the bounds, over S arcs, strict complementarity for the control constraints: B i,+ = { t [0, T ] : max λ Λ H ui [λ] < 0 }, B i, = { t [0, T ] : min λ Λ H ui [λ] > 0 }. These hypotheses are verified in many practical examples: see e.g. aerospace applications by Bonnans, Martinon, Trélat, Maurer; Goh s fishing problem, others aerospace applications; Ledzewicz & Schättler s chemotherapy optimization problems, many other examples... Jumping may be a necessary condition: [McDanell-Powers, Necessary conditions for joining optimal singular and nonsingular arcs, SIAM J. Control, 1971]

12 & state constraint For example: the Goddard problem in 1D (vertical motion of a rocket) [Goddard, 1919] [Seywald & Cliff, 1993] State variables: (h, v, m) are the position, speed and mass. Control variable: u is the normalized thrust (proportion of T max ) Cost: final mass of the rocket (or minimize fuel consumption) max m(t ), s.t. ḣ(t) = v(t), v(t) = 1/h(t) 2 + 1/m(t) ( T max u(t) D(h(t), v(t)) ), ṁ(t) = b T max u(t), 0 u(t) 1, a.e. on [0, T ], h(0) = 0, v(0) = 0, m(0) = 1, h(t ) = 1, T : free final time, b : fuel consumption coefficient, T max is the maximal thrust, D(h, v) is the drag

13 & state constraint Numerical solution for Goddard problem THRUST (NORMALIZED) CONTROL TIME Figure: Goddard Problem

14 & state constraint Critical directions z[v, z 0 ] solution of linearized state equation for (v, z 0 ) L 2 R n Tangent cone to the endpoint constraints: T Φ := {0} R n 1 {η R n 2 : η i 0 if Φ n1 +i(ˆx(0), ˆx(T )) = 0, 1 i n 2 }. The critical cone: { ( z[v, z 0 ], v ) } W 1, L : ϕ (ˆx(0), ˆx(T ))(z(0), z(t )) 0 C := Φ. (ˆx(0), ˆx(T ))(z(0), z(t )) T Φ, v i = 0 a.e. on B i, i

15 & state constraint Second order necessary optimality condition Weak minimum: exists ε > 0 such that ϕ(ˆx(0), ˆx(T )) ϕ(x(0), x(t )) for any feasible (x, u) for which For λ Λ, we set (x, u) (ˆx, û) < ε Q[λ](z, v) := D 2 (x,u) 2 L[λ](ˆx, û)(z, v) 2, for (z = z[v, z 0 ], v) W 1, L, then Q[λ](z, v) = D 2 l β,ψ (z(0), z(t )) 2 + T Theorem (Second order necessary condition) Assume that (ˆx, û) is a weak minimum. Then 0 ( z H xx z + 2v H ux z ) dt. max Q[λ](z, v) 0, for all (z, v) C. λ Λ Q does not contain a term in H uu!

16 & state constraint Second order necessary optimality condition Weak minimum: exists ε > 0 such that ϕ(ˆx(0), ˆx(T )) ϕ(x(0), x(t )) for any feasible (x, u) for which For λ Λ, we set (x, u) (ˆx, û) < ε Q[λ](z, v) := D 2 (x,u) 2 L[λ](ˆx, û)(z, v) 2, for (z = z[v, z 0 ], v) W 1, L, then Q[λ](z, v) = D 2 l β,ψ (z(0), z(t )) 2 + T Theorem (Second order necessary condition) Assume that (ˆx, û) is a weak minimum. Then 0 ( z H xx z + 2v H ux z ) dt. max Q[λ](z, v) 0, for all (z, v) C. λ Λ Q does not contain a term in H uu!

17 & state constraint Goh s Transformation [Goh 1966] (z, v) where B := f x f u d dt f u. ( y := ) v, ξ := z f u y. ż = f x z + f u v ξ = f x ξ + By, Ω Ω P (ξ, y, v) := 1 2g(ξ(0), ξ(t ), y(t )) where T 0 (ξ H xx ξ + 2y Mξ + y Ry + 2v V y)dt, M := f u Hxx d dt Hux Huxfx, R := f u Hxxfu HuxB (HuxB) + d dt (Huxfu + (Huxfu) ), g(ζ 0, ζ T, h) := D 2 l(ζ 0, ζ T + f u(t )h) 2 + h H ux(t )(2ζ T + f u(t )h), V := 1 2 (H uxf u (H ux f u ) ), V ij = 1 2 p[f i, f j ].

18 & state constraint Goh conditions G(co Λ) := {λ co Λ : V ij [λ] = 0 on S i S j, for any 1 i < j m}. Theorem If (ˆx, û) is a weak minimum then max λ G(co Λ) Ω P[λ](ξ, y, v) 0, for (z, v) C. Corollary (Goh condition) If (ˆx, û) is a weak minimum then G(co Λ) and, for all λ G(co Λ), p[f i, f j ] = 0, on S i S j.

19 & state constraint Removing v C Goh P := For λ G(Λ), set (ξ, y, y(t )) : y i = 0 over B i, ξ = fx ξ + By, ϕ i(x(0), x(t ))(ξ(0), ξ(t ) + f u (T )h) 0, η j(x(0), x(t ))(ξ(0), ξ(t ) + f u (T )h) = 0 P 2 := closure of P in W 1,2 L 2 R m Ω P2 [λ](ξ, y, h) := Ω P [λ](ξ, y, h) + Ξ(ξ, y, h). Theorem (Second order necessary condition in new variables) Let (ˆx, û) be a weak minimum, then max Ω P 2 [λ](ξ, y, h) 0, for all (ξ, y, h) P 2. λ G(co Λ)

20 & state constraint Sufficient condition: scalar control case Pontryagin minimum: for any N > 0, ε N > 0 such that (ˆx, û) is optimal on the set { (x, u) feasible : (x, u) (ˆx, û) < ε N, u û 1 < ε N }. Convergence in the Pontryagin sense: v k 1 0, v k < N. γ order: (ξ 0, y, h) R n L 2 R, let T γ(ξ 0, y, h) := ξ y(t) 2 dt + h 2. 0 γ growth condition in the Pontryagin sense: exists ρ > 0, for every v k 0 in the Pontryagin sense, where y k := v k. J(û + v k ) J(û) ργ(y k, y k (T )),

21 & state constraint Sufficient condition Theorem (Characterization of quadratic growth) Suppose that there exists ρ > 0 such that max Ω P 2 [λ](ξ, y, h) ργ(ξ(0), y, h), for all (ξ, y, h) P 2. λ Λ Then (ˆx, û) is a Pontryagin minimum satisfying γ growth. Furthermore, if (ˆx, û) is normal, the converse holds. Remark In case the bang arcs are absent, this theorem reduces to one proved in [Dmitruk, 1977].

22 & state constraint 1 2 & state constraint

23 & state constraint Control bounds & state constraint: the problem We consider the optimal control problem: min ϕ(x(0), x(t )) ẋ(t) = f(u(t), x(t)) = f 0 (x(t)) + u(t)f 1 (x(t)), a.e. on [0, T ], Φ(x(0), x(t )) K Φ = {0} R n 1 R n 2, u min u(t) u max, a.e. on [0, T ], g(x(t)) 0, t [0, T ]. where g : R n R. Assuming feasibility, one has existence of optimal trajectory (ˆx, û).

24 & state constraint Some applications Some control-affine problems with control bounds and state constraints: J.B. Keller, Optimal velocity in a race, Amer. Math. Monthly 81 (1974), A.V. Dmitruk, I.A. Samylovskiy: A simple Trolley-like model in the presence of a nonlinear friction and a bounded fuel expenditure, Disc. Math., Diff. Inclusions, Control and Optim. 33 (2013), A. Aftalion and J.F. Bonnans, Optimization of running strategies based on anaerobic energy and variations of velocity, SIAM J. Applied Math., 2014 P.-A. Bliman, M.S.A., F.C. Coelho, and M.A. da Silva, Global stabilizing feedback law for a problem of biological control of mosquito-borne diseases, Proceedings of the 54th IEEE CDC, 2015 & arxiv preprint March 2015

25 & state constraint Hamiltonian and Lagrangian functions As before, we have the unmaximized Hamiltonian H and the endpoint Lagrangian l. Let λ := (β, Ψ, p, dµ) R + R (n 1+n 2 ) BV n M, and define the Lagrangian function T L[λ](u, x) := l β,ψ (x(0), x(t ))+ p ( f(x, u) ẋ ) dt+ g(x)dµ(t). 0 [0,T ]

26 & state constraint First order optimality conditions (I) Costate equation: for p BV n, dp(t) = p(t)f x (u(t), x(t))dt+g (x(t)) dµ(t), for a.a. t [0, T ], with endpoint conditions ( p(0), p(t )) = Dl β,ψ (x(0), x(t )). (II) Minimization of Hamiltonian (affine w.r.t. u) for a.a. t [0, T ], u(t) = u min, if p(t)f 1 (x(t)) > 0, u(t) = u max, if p(t)f 1 (x(t)) < 0, p(t)f 1 (x(t)) = 0, if u min < u(t) < u max.

27 & state constraint First order optimality conditions (III) Final constraints multiplier: Ψ R (n 1+n 2 ) ; Ψ i 0, Ψ i Φ i (x(0), x(t )) = 0, i = n 1 + 1,..., n 2. (IV) State constraint multiplier: dµ M; dµ 0; T 0 g(x(t))dµ(t) = 0. Λ(x, u) = set of Pontryagin multipliers associated to (x, u) : λ := (β, Ψ, p, dµ) R + R (n 1+n 2 ) BV n M that satisfy (I)-(IV) and β + Ψ = 1. PMP: Λ.

28 & state constraint Bang, contact and singular arcs Bang sets for the control constraint: { B := {t [0, T ] : û(t) = u min }, B + := {t [0, T ] : û(t) = u max }, and set B := B B +. Contact set associated with the state constraint: Singular set C := {t [0, T ] : g(ˆx(t)) = 0}, S := {t [0, T ] : u min < û(t) < u max and g(ˆx(t)) < 0}. C, B, S arcs are maximal intervals contained in these sets.

29 & state constraint Assumptions on control structure & Complementarity (i) (ii) (iii) (iv) the interval [0, T ] is (up to a zero measure set) the disjoint union of finitely many arcs of type B, C and S, the control û is at uniformly positive distance of the bounds, over C and S arcs, the control û is discontinuous at CS and SC junctions, complementarity of control and state constraints, i.e., the corresponding multipliers do not vanish on the active arcs. The state constraint is of first order, i.e. g (ˆx(t))f 1 (ˆx(t)) 0, on C. These hypotheses are verified in some practical examples.

30 & state constraint Regulator problem with state constraint min (x x 2 2)dt; s.t. ẋ 1 = x 2 ; ẋ 2 = u; x 1 (0) = 0, x 2 (0) = 1. subject to the control bounds and state constraint: 1 u(t) 1, x 2 0.2,

31 & state constraint Regulator problem with state constraint min (x x 2 2)dt; s.t. ẋ 1 = x 2 ; ẋ 2 = u; x 1 (0) = 0, x 2 (0) = 1. subject to the control bounds and state constraint: 1 u(t) 1, x 2 0.2,

32 & state constraint Regulator problem with state constraint min (x x 2 2)dt; s.t. ẋ 1 = x 2 ; ẋ 2 = u; x 1 (0) = 0, x 2 (0) = 1. subject to the control bounds and state constraint: 1 u(t) 1, x 2 0.2,

33 & state constraint Numerical solution Figure: x 1 : position, x 2 : speed, x 3 : cost, u : acceleration

34 & state constraint Second order necessary conditions The critical cone: (z[v, z 0 ], v) W 1, L : ϕ (ˆx(0), ˆx(T ))(z(0), z(t )) 0 C := Φ (ˆx(0), ˆx(T ))(z(0), z(t )) T Φ, v(t) = 0 a.e. on B g. (ˆx(t))z(t) = 0 on C For every multiplier λ Λ, we have D 2 (x,u) 2 L[λ](û, ˆx)(z, v) 2 = Q[λ](z, v), for (z[v, z 0 ], v) W 1, L, where Q := D 2 l β,ψ (z(0), z(t )) 2 + T Theorem (Second order necessary condition) Assume that (ˆx, û) is a weak minimum. Then 0 ( z H xx z+2vh ux z ) dt+ z g zdµ(t). [0,T ] max Q[λ](v, z) 0, for all (z, v) C. λ Λ

35 & state constraint Goh s transform (z, v) ( y := ) v, ξ := z f u y. ż = f x z + f u v ξ = f x ξ + By, Write ξ = ξ[y, ξ 0 ]. Set of (strict) primitive critical directions: { } (ξ, y, h) W 1, R W 1, : P :=. y(0) = 0, y(t ) = h, (ẏ, ξ + yf 1 ) C

36 & state constraint Closure of P in H 1 L 2 R Proposition (ξ, y, h) P iff g (ˆx(t))(ξ(t) + y(t)f 1 (ˆx(t))) = 0 on C, φ (ˆx(0), ˆx(T )) ( ξ(0), ξ(t ) + hf 1 (ˆx(T )) ) 0, Φ (ˆx(0), ˆx(T )) ( ξ(0), ξ(t ) + hf 1 (ˆx(T )) ) T Φ, y is constant on each B arc, y is continuous at the BC, CB and BB junctions, y(t) = 0, on B 0± if a B 0± arc exists, y(t) = h, on B T ± if a B T ± arc exists, lim y(t) = h, if T C. t T Set P 2 := P.

37 & state constraint Goh transformation on the Hessian of Lagrangian Set for (ξ, y, h) H 1 L 2 R and λ := (β, Ψ, p, dµ) Λ, Ω := Ω T + Ω 0 + Ω E + Ω g, where Ω T [λ](ξ, y, h) :=2h H ux (T )ξ(t ) + h H ux (T )f 1 (ˆx(T ))h, Ω 0 [λ](ξ, y, h) := T 0 ( ξ H xx ξ + 2yMξ + yry ) dt, Ω E [λ](ξ, y, h) := D 2 l β,ψ (ξ(0), ξ(t ) + f 1 (ˆx(T ))h) 2, Ω g [λ](ξ, y, h) := T 0 (ξ + f 1 (ˆx)y) g (ˆx)(ξ + f 1 (ˆx)y)dµ(t).

38 & state constraint Second order necessary condition after Goh transformation Let (z[v, z 0 ], v) H 1 L 2 and let (ξ[y, z 0 ], y) be defined by Goh transformation. Then, for any λ Λ, Q[λ](z, v) = Ω[λ](y, y T, ξ). Theorem (Second order necessary condition) If (ˆx, û) is a weak minimum, then max Ω[λ](ξ, y, h) 0, for all (ξ, y, h) λ Λ P2.

39 & state constraint Extended critical cone Define P 2 P 2 consisting of (ξ, y, h) H 1 L 2 R m verifying g (ˆx(t))(ξ(t) + y(t)f 1 (ˆx(t))) = 0 on C, y is constant on each B arc, φ (ˆx(0), ˆx(T )) ( ξ(0), ξ(t ) + hf 1 (ˆx(T )) ) 0, Φ (ˆx(0), ˆx(T )) ( ξ(0), ξ(t ) + hf 1 (ˆx(T )) ) T Φ, y is continuous at the BC, CB and BB junctions, y(t) = 0, on B 0± if a B 0± arc exists, y(t) = h, on B T ± if a B T ± arc exists, lim t T y t = h, if T C (if T C, [µ(t )] = 0 for all µ)

40 & state constraint Second order sufficient condition Theorem Suppose that for each λ Λ, Ω[λ]( ) is a Legendre form in {(ξ = ξ[y], y, h) H 1 L 2 R} and there exists ρ > 0 such that max Ω[λ](ξ, y, h) ργ(ξ 0, y, h), for all (ξ, y, h) P 2. λ Λ Then (ˆx, û) is a Pontryagin minimum satisfying γ growth. Remark: (ii) holds if and only if, for every λ Λ, R(ˆx(t)) + f 1 (ˆx(t)) g (ˆx(t))f 1 (ˆx(t))ν(t) > α > 0, on [0, T ], where ν is the density of dµ [Hestenes 1951].

41 & state constraint Applications Application I: analytical proof of local optimality in some examples (see references) Application II: convergence of an associated shooting algorithm Application III: stability under data perturbation

42 & state constraint References M.S.A., J.F. Bonnans, A.V. Dmitruk & P.A. Lotito, Quadratic order conditions for bang-singular extremals, Numerical Algebra, Control Optim., AIMS Journal, 2012 M.S.A., J.F. Bonnans & P. Martinon, A shooting algorithm for optimal control problems with singular arcs, J. Optim. Theory App., 2013 M.S.A., J.F. Bonnans & B.S. Goh, Second order analysis of control-affine problems with scalar state constraint. Math. Programming, 2016

43 & state constraint Open problems Optimal control continuous at the junction times. For instance: B.S. Goh, G. Leitmann, T.L. Vincent, Optimal control of a prey-predator system, Math. Biosci No-gap conditions for the single-input state-constrained case? Note that one may have P 2 P 2 S. Sufficient condition for the vector control case Many, many others..

44 THANK YOU FOR YOUR ATTENTION