Value Function and Optimal Trajectories for some State Constrained Control Problems

Transcription

1 Value Function and Optimal Trajectories for some State Constrained Control Problems Hasnaa Zidani ENSTA ParisTech, Univ. of Paris-saclay "Control of State Constrained Dynamical Systems" Università di Padova, September 25-29, 2017 H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

2 For a given non-empty compact subset U of R k, define the set of admissible controls as: U := { u : (0, T ) R k, measurable, u(t) U a.e }. Consider the following control system: {ẏ(s) := f (y(s), u(s)), a.e s [t, T ], (1) y(t) := x, where f : R d U R d is continuous, and Lipschitz continuous w.r.t x. The set of trajectories: S [t,t ] (x) := {y u t,x W 1,1 (t, T ; R d ), y u t,x satisfies (1) for some u U}, The multi-application: x S [t,t ] (x) is Lipschitz continuous. H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

3 State constrained optimal control problems inf Φ(y u t,x(t )) s.t. u U, y u t,x(s) K s [t, T ]. K is a closed sub-set of R d. The final cost Φ : R d R is a Lipschitz continuous function H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

4 Outline 1 Characterization of the value function under some controllability assumptions 2 A general case where the controllability assumption is not satisfied 3 A numerical example 4 End-point constrained control problem H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

5 Outline 1 Characterization of the value function under some controllability assumptions 2 A general case where the controllability assumption is not satisfied 3 A numerical example 4 End-point constrained control problem H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

6 Set of constrained trajectories Assume that f (x, U) := {f (x, u), u U} is a convex set. Then, by Filippov s theorem, the set of trajectories S [t,t ] (x) is a compact set of W 1,1 ([t, T ]) endowed with the C 0 -topology. H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

7 Set of constrained trajectories Assume that f (x, U) := {f (x, u), u U} is a convex set. Then, by Filippov s theorem, the set of trajectories S [t,t ] (x) is a compact set of W 1,1 ([t, T ]) endowed with the C 0 -topology. The set of feasible trajectories: S K [t,t ](x) := {y S [t,t ] (x) y(s) K s [t, T ]} is a compact subset of W 1,1 ([t, T ])... when it is non-empty! H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

8 Set of constrained trajectories Assume that f (x, U) := {f (x, u), u U} is a convex set. Then, by Filippov s theorem, the set of trajectories S [t,t ] (x) is a compact set of W 1,1 ([t, T ]) endowed with the C 0 -topology. The set of feasible trajectories: S K [t,t ](x) := {y S [t,t ] (x) y(s) K s [t, T ]} is a compact subset of W 1,1 ([t, T ])... when it is non-empty! Inward pointing (IP) condition: Assume K = K and β > 0, x K, min f (x, u) ηx < β. u U Then, for x K, S K [t,t ](x), and x S K [t,t ](x) is Lipschitz. Ref: Arutyunov 84, Soner 86, Rampazzo-Vinter 99, Vinter-Frankowska 00, Clarke-Rifford-Stern H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

9 A State constrained control problem { ϑ(t, x) := min Φ(y(T )) } y S K [t,t ](x), In general, ϑ is only l.s.c. on K. Under suitable controllability assumptions, ϑ is the unique constrained viscosity solution of: tϑ(t, x) + H(x, D xϑ(t, x)) = 0 on (0, T ) K, ϑ(0, x) = Φ(x) on K. Under IP condition, the value function ϑ is Lipschitz continuous on K. Soner 86, Motta 95, Ishii-Koike 96, Frankowska-et-al. 00-,... If K is convex, f (x, u) = Ax + Bu and ( x, v), A x + Bū = 0 Hermosilla-Vinter-HZ 17 If K has a stratified structure + a local controllability assumption: Hermosilla-HZ 15, Hermosilla-Wolenski-HZ 17 H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

10 How can we characterize (and compute) the value function for problems lacking controllability assumptions? H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

11 Outline 1 Characterization of the value function under some controllability assumptions 2 A general case where the controllability assumption is not satisfied 3 A numerical example 4 End-point constrained control problem H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

12 An other alternative... Assume K is a closed nonempty set (no additional requirement) Consider a function g : R d R, Lipschitz continuous, such that x R d, g(x) 0 x K. H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

13 An other alternative... Assume K is a closed nonempty set (no additional requirement) Consider a function g : R d R, Lipschitz continuous, such that x R d, g(x) 0 x K. In particular, [y(s) K, s [t, T ]] max g(y(s)) 0. s [t,t ] H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

14 An other alternative... Assume K is a closed nonempty set (no additional requirement) Consider a function g : R d R, Lipschitz continuous, such that x R d, g(x) 0 x K. In particular, [y(s) K, s [t, T ]] max g(y(s)) 0. s [t,t ] Assume that, for every x R d, f (x, U) := {f (x, u), u U} is a convex set. H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

15 An auxiliary control problem (Altarovici-Bokanowski-HZ 13) Consider the following auxiliary control problem (z R): { (Φ(y u w(t, x, z) := inf t,x(t )) z ) } max g(y u t,x(s)). y S [t,t K ] (x) s [t,t ] H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

16 An auxiliary control problem (Altarovici-Bokanowski-HZ 13) Consider the following auxiliary control problem (z R): { (Φ(y u w(t, x, z) := inf t,x(t )) z ) } max g(y u t,x(s)). y S [t,t K ] (x) s [t,t ] Improvement function (Mifflin 77, Solodov-Sagastizábal 04, Apkarian-et-al. 08,...) (P) min F (X) G(X) 0 The auxiliary optimization problem: ( P) { min (F(X) z) } G(X) X Under Slater condition: X solution of ( P) for z = F( X), X is optimal for (P) { min (F(X) z) } G(X) = 0. X H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

17 An auxiliary control problem (Altarovici-Bokanowski-HZ 13) Consider the following auxiliary control problem (z R): { (Φ(y u w(t, x, z) := inf t,x(t )) z ) } max g(y u t,x(s)). y S [t,t K ] (x) s [t,t ] Theorem For every x K, we have: { } (i) Epi ϑ(t, ) = (x, z) : w(t, x, z) 0, { } (ii) ϑ(t, x) = min z R, w(t, x, z) 0, H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

18 An auxiliary control problem (Altarovici-Bokanowski-HZ 13) Consider the following auxiliary control problem (z R): { (Φ(y u w(t, x, z) := inf t,x(t )) z ) } max g(y u t,x(s)). y S [t,t K ] (x) s [t,t ] Theorem For every x K, we have: { } (i) Epi ϑ(t, ) = (x, z) : w(t, x, z) 0, { } (ii) ϑ(t, x) = min z R, w(t, x, z) 0, (iii) Under IP condition: for every x K we have: ϑ(t, x) = z, s.t. w(t, x, z) = 0. H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

19 Define the Hamiltonian as: H(x, p) := max u U ( f (x, u) p ) x, p R d. Theorem The value function w is the unique Lipschitz continuous viscosity solution of the following Hamilton-Jacobi-Bellman (HJB) equation: ( ) min tw(t, x, z) + H(x, D xw), w(t, x, z) g(x) = 0 [0, T [ R d R, w(t, x, z) = (Φ(x) z) g(x), R d R. H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

20 A particular choice of function g Let η > 0 and define the following extended set K η: K η : K + B(0, η). g(y) := d K(y) the signed distance to K. Consider the following auxiliary control problem : [ ( w(t, x, z) := inf Φ(y(T )) z) ) ] max g(y(s) η, y S [t,t ] (x) s [t,t ] where a b = min(a, b). Theorem Let (t, x, z) [0, T ] K R. The following assertions hold: (i) ϑ(t, x) z 0 w(t, x, z) 0, { } (ii) ϑ(t, x) = min z R, w(t, x, z) 0. H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

21 A particular choice of function g Theorem The function w is the unique Lipschitz continuous viscosity solution of the following HJB equation: ( ) min tw(t, x, z) + H(x, xw), w(t, x, z) g(x) = 0 [0, T [ K η R, w(t, x, z) = Ψ η(x, z), K η R, w(t, x, z) = η, y K η, [ where Ψ η(x, z) = (Φ(x) z) g(y)] η. H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

22 Link with exit time function Define the exit time function: T (y, z) := inf { t [0, T ] ϑ(t, y) z } = inf { t [0, T ] w(t, y, z) 0 } H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

23 Link with exit time function Define the exit time function: T (y, z) := inf { t [0, T ] } ϑ(t, y) z = inf { t [0, T ] } w(t, y, z) 0 Link between ϑ and T : (i) T is the exit time function for Epi(Φ) ( K R d ), (ii) T (y, z) = t w(t, y, z) = 0, (iii) ϑ(t, y) = inf { z T (y, z) t }. H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

24 Reconstruction of optimal trajectories Proposition Let x K such that ϑ(t, x) <. Define z := ϑ(t, x). Let (y, z ) be the optimal trajectory for the auxiliary control problem associated with the initial point (x, z ) K R. Then, the trajectory y is optimal for the original control problem. The converse is also true. H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

25 Reconstruction of optimal trajectories - Algorithm A. For n 1, consider (t 0 = 0, t 1,..., t n 1, t n = T ) a uniform partition of [0, T ] with t = T n. Let {y n ( ), z n ( )} be a trajectory defined recursively on the intervals (t i 1, t i ], with z n ( ) := z = ϑ(0, y) and y n (0) = y. H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

26 Reconstruction of optimal trajectories - Algorithm A. For n 1, consider (t 0 = 0, t 1,..., t n 1, t n = T ) a uniform partition of [0, T ] with t = T n. Let {y n ( ), z n ( )} be a trajectory defined recursively on the intervals (t i 1, t i ], with z n ( ) := z = ϑ(0, y) and y n (0) = y. [Step 1] Knowing yk n = y n (t k ), choose the optimal control at t k s.t.: ( uk n arg min w(t k, y n ( k + tf t y n k, u ) ), z). u U H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

27 Reconstruction of optimal trajectories - Algorithm A. For n 1, consider (t 0 = 0, t 1,..., t n 1, t n = T ) a uniform partition of [0, T ] with t = T n. Let {y n ( ), z n ( )} be a trajectory defined recursively on the intervals (t i 1, t i ], with z n ( ) := z = ϑ(0, y) and y n (0) = y. [Step 1] Knowing yk n = y n (t k ), choose the optimal control at t k s.t.: ( uk n arg min w(t k, y n ( k + tf t y n k, u ) ), z). u U [Step 2] Define u n (t) := u n k, t (t k, t k+1 ] and y n (t) on (t k, t k+1 ] as the solution of ẏ(t) := f (y(t), u n (t)) a.e t (t k, t k+1 ], with initial condition y n (t k ) at t k and z n ( ) := z. H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

28 Theorem Let {y n ( ), z n ( ), u n ( )} be a sequence generated by algorithm A for n 1. Then, the sequence of trajectories {y n ( )} n has cluster points with respect to the uniform convergence topology. For any cluster point ȳ( ) there exists a control law ū( ) such that (ȳ( ), z( ), ū( )) is optimal for the auxiliary control problem. H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

29 Theorem Let {y n ( ), z n ( ), u n ( )} be a sequence generated by algorithm A for n 1. Then, the sequence of trajectories {y n ( )} n has cluster points with respect to the uniform convergence topology. For any cluster point ȳ( ) there exists a control law ū( ) such that (ȳ( ), z( ), ū( )) is optimal for the auxiliary control problem. Let w be a numerical approximate solution such that, w (t, y, z) w(t, y, z) E 1 ( t, y), where E 1 ( t, y) 0 as t, y 0. H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

30 Theorem Let {y n ( ), z n ( ), u n ( )} be a sequence generated by algorithm A for n 1. Then, the sequence of trajectories {y n ( )} n has cluster points with respect to the uniform convergence topology. For any cluster point ȳ( ) there exists a control law ū( ) such that (ȳ( ), z( ), ū( )) is optimal for the auxiliary control problem. Let w be a numerical approximate solution such that, w (t, y, z) w(t, y, z) E 1 ( t, y), where E 1 ( t, y) 0 as t, y 0. Let {Y n (.), u n (.)} be the sequence generated by the algorithm A with w. H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

31 Theorem Let {y n ( ), z n ( ), u n ( )} be a sequence generated by algorithm A for n 1. Then, the sequence of trajectories {y n ( )} n has cluster points with respect to the uniform convergence topology. For any cluster point ȳ( ) there exists a control law ū( ) such that (ȳ( ), z( ), ū( )) is optimal for the auxiliary control problem. Let w be a numerical approximate solution such that, w (t, y, z) w(t, y, z) E 1 ( t, y), where E 1 ( t, y) 0 as t, y 0. Let {Y n (.), u n (.)} be the sequence generated by the algorithm A with w. Then, (Y n ) n converges to an optimal trajectory for the auxiliary control problem. H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

32 Outline 1 Characterization of the value function under some controllability assumptions 2 A general case where the controllability assumption is not satisfied 3 A numerical example 4 End-point constrained control problem H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

33 Abort landing problem in presence of windshear (Miele, Wang and Melvin(1987,1988); Bulirsch, Montrone and Pesch (1991..); Botkin-Turova( )) Consider the flight motion of an aircraft in a vertical plane: ẋ = V cos γ + w x ḣ = V sin γ + w h V = F T m cos(α + δ) F D m g sin γ (ẇ x cos γ + ẇ h sin γ) γ = 1 ( F T V m sin(α + δ) + F L m g cos γ + (ẇ x sin γ ẇ h cos γ)) where and F T := F T (V ) is the thrust force ẇ x = wx wx (V cos γ + wx ) + x h (V sin γ + w h) ẇ h = w h x (V cos γ + wx ) + w h h (V sin γ + w h) F D := F D (V, α) and F L := F L (V, α) are the drag and lift forces w x := w x(x) and w h := w h (x, h) are the wind components m, g, and δ are constants. H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

34 Controlled system Consider the state y(.) = (x(.), h(.), V (.), γ(.), α(.)). The control variable u is the angular speed of the angle of attack α. Let T be a fixed time horizon and let U be the set of admissible controls { } U := u : (0, T ) R, measurable, u(t) U a.e where U is a compact set. The controlled dynamics in this case is: ẋ = V cos γ + w x, ḣ = V sin γ + w h, V = F T m cos(α + δ) F D m g sin γ (ẇ x cos γ + ẇ h sin γ), γ = 1 V ( F T m sin(α + δ) + F L m g cos γ + (ẇ x sin γ ẇ h cos γ)), α = u. H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

35 Formulation of the optimal control problem Aim: Maximize the minimal altitude over a time interval: min h(θ) θ [0,t] while the aircraft stays in a given domain K. H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

36 Formulation of the optimal control problem Aim: Maximize the minimal altitude over a time interval: min h(θ) θ [0,t] while the aircraft stays in a given domain K. Consider the following optimal control problem: { (P) : ϑ(t, y) = inf max Φ(y u y (θ)), } u U, and y u y (s) K, s [0, t] θ [0,t] where Φ(y u y(.)) = H r h(.), H r being a reference altitude, and K is a set of state constraints. H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

37 Figure: Optimal trajectories for different initial conditions H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

38 Outline 1 Characterization of the value function under some controllability assumptions 2 A general case where the controllability assumption is not satisfied 3 A numerical example 4 End-point constrained control problem H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

39 Consider the end-point constrained control problem inf Φ(y u t,x(t )) s.t. u U, g(y u t,x(t )) 0. An associated auxiliary control problem (z R) can be defined as: { (Φ(y u w(t, x, z) := inf t,x (T )) z ) g(y u t,x (T ))}. u U Assume that Φ : R d R and g : R d R are of C 1 H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

40 Relationship between the PMP and HJB Let x o R d and let z := min{z : w(0, x o, z) 0}. If z <, then there exists u U and its associated trajectory y S [0,T ] (x o) such that: g(y (T )) 0, ϑ(0, x o) = z = Φ(y (T )). H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

41 Relationship between the PMP and HJB Denote H(x, u, p) = p, f (x, u). There exists (p x, p z ) satisfying: ṗ x (s) = xh(y (s), u (s), p x (s)) ṗ z (s) = 0 ( ) p x (T ) p z (T ) x,z {(Φ(y (T )) z ) } g(y (T )) H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

42 Relationship between the PMP and HJB Denote H(x, u, p) = p, f (x, u). There exists (p x, p z ) satisfying: ṗ x (s) = xh(y (s), u (s), p x (s)) ṗ z (s) = 0 ( ) ( p x (T ) λo Φ(y (T )) + λ g(y (T )) p = z (T ) λ o ) where λ o, λ [0, 1] and λ 0 + λ = 1. H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

43 Relationship between the PMP and HJB Denote H(x, u, p) = p, f (x, u). There exists (p x, p z ) satisfying: ṗ x (s) = xh(y (s), u (s), p x (s)) ṗ z (s) = 0 ( ) ( p x (T ) λo Φ(y (T )) + λ g(y (T )) p = z (T ) λ o ) where λ o, λ [0, 1] and λ 0 + λ = 1. For a.e s [0, T ], H(y (s), p x (s)) = H(y (s), u (s), p x (s)). H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

44 Relationship between the PMP and HJB Denote H(x, u, p) = p, f (x, u). There exists (p x, p z ) satisfying: ṗ x (s) = xh(y (s), u (s), p x (s)) ṗ z (s) = 0 ( ) ( p x (T ) λo Φ(y (T )) + λ g(y (T )) p = z (T ) λ o ) where λ o, λ [0, 1] and λ 0 + λ = 1. For a.e s [0, T ], H(y (s), p x (s)) = H(y (s), u (s), p x (s)). Moreover, (p x (s), p z (s)) x,zw(s, y (s), z ). H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

45 The same framework can be extended to: final state constraints and time-dependent state constraints impulsive control problems (Forcadel-Rao-HZ 13) supremum running cost problems (Assellaou-Bokanowski-Desilles-HZ 17), stochastic control setting (Bokanowski-Picarelli-HZ 16) The relationship between the PMP and the auxiliary value function can be (easily) derived for control problems with a finite number of state constraints. H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29

46 ...thanks for your attention. H. Zidani (ENSTA ParisTech) State constrained control problems Padova, Sep25-29, / 29