Recent Advances in State Constrained Optimal Control

Transcription

1 Recent Advances in State Constrained Optimal Control Richard B. Vinter Imperial College London Control for Energy and Sustainability Seminar Cambridge University, 6th November 2009

2 Outline of the talk Overview of techniques for solving optimal control problems. The role of Filippov type estimates in state constrained optimal control Linear estimates for 1 state constraint. Linear estimates for multiple state constraints -: Counter-Examples. New Superlinear estimates for 2 state constraints. Optimality of estimate structure. Linear estimates for strictly convex velocity sets. Future directions. Joint work with Piernicola Bettiol and Alberto Bressan

3 The Classical Optimal Control Minimize g(x(1)) over functions u(.) : [0, 1] R m, and trajectories x(.) s.t. ẋ(t) = f(x(t), u(t)) for [0, 1] u(t) U for t [0, 1] and x(0) = x 0. Data: g : R n R, f : R n R m R n, U R m, x 0 R n. Traditional Application Areas 1. Aerospace: flight trajectories for planetary exploration 2. Resource economics: optimal harvesting (fishing, forestry,..) 3. Chemical engineering: optimize yield, purity etc.

4 Modern Day Applications Areas implementation of Predictive Control schemes and and closed loop response analysis path planning in air traffic control (trajories between way stations) hybrid controllers for wind turbines, engine management systems. In applications, optimal controls are calculated by means of numerical schemes bases on discretization. But continuous time optimal control has an important role: Control problems associated with the physical world are continuous Theory can tell us when when problems are degenerate, and computational schemes will be ill-conditioned Basis for high precision shooting methods Theory provides tests of local optimality for controls obtained by numerical methods

5 Hamilton Jacobi Methods (Dynamic Programming) Analyse minimizers via solutions to the Hamilton Jacobi equation { Minimize g(x(1)) P(0, x 0 ) over trajectories x(.) s.t. x(0) = x 0. Embed in family of problems, parameterized by initial data { Minimize g(x(1)) P(τ, ξ) over trajectories x(.) s.t. x(τ) = ξ. Define V(τ, ξ) = Inf(P(τ, ξ)) (Value Function) V(.,.) is a solution to { Vt (t, x) + min v F(x) V x (t, x) v = 0 (t, x) (0, 1) R n V(1, x) = g(x) x R n.

6 First Order Necessary Conditions Take an optimal pair ( x(.), ū(.)). Define H(x, p, u) = p f(x, u) (The Hamiltonian). Maximum Principle: There exists an arc p(.) and λ 0, s.t. (p(.), λ) 0 ṗ(t) = p(t) f x ( x(t), ū(t)) H( x(t), p(t), ū(t)) = max u U p(1) = λ g x ( x(1)) H( x(t), p(t), u) Widely used to solve optimal control problems, either directly or via numerical methods it inspires (Shooting Methods).

7 Enter State Constraints Consider state constrained control system ẋ(t) f(t, x(t), u(t)) for t [0, 1] u(t) U(t) for t [0, 1] x(t) A t [0, 1]. Assume functional inequality representation : A r i=1 {x hi (x) 0}. for some C 2 functions h i : R n R, i = 1,...,r. Given an arc x(.) : [0, 1] R n write h(x(.)) := max max {h 1 (x(t)) 0,..., h r (x(t)) 0} t [0,1] (the index of constraint violation )

8 Dynamic Programming for State Constrained Problems Minimize g(x(1)) over trajectories x(.) s.t. x(t) A x(0) = x 0. How does state constraint affect optimality conditions? Now, value function V(.,.) : [0, 1] R n R {+ } is a lsc solution to { Vt (t, x) + min v F(x) V x (t, x) v = 0 (t, x) (0, 1) int A V(1, x) = g(x) x A (the unique solution, in fact, in some technical sense)

9 State Constrained Max. Principle Take an optimal pair ( x(.), ū(.)). There exists an arc p(.), bounded variation multiplier µ 0 and λ 0, s.t. (p(.), µ, λ) 0 dµ/dt = 0 when x(t) int A ṗ(t) = p(t) f x ( x(t), ū(t) h x ( x(t))dµ/dt H( x(t), p(t), ū(t)) = max u U p(1) = λ g x ( x(1)). H( x(t), p(t), u) (Formally follows from inserting into cost the penalty term h(x(t))dµ/dt).)

10 The Degeneracy Issue for State Constrained Problems Conditions (Constraint Qualifications) must be imposed on the problem data to ensure: The cost multiplier is non-zero The value function is the unique solution to the Bellman equation adjoint variables in the Max. Principle can be interpreted as gradients of the value function. Typically these take the form: for each x A there exists u U s.t. h j (x) f(x, u) < 0 for active j. Inward Pointing Condition. Major theme in state constrained optimal control: Develop tools for analysing (and excluding!) degeneracy, under inward pointing conditions.

11 Standing Hypotheses Minimize g(x(1)) s.t ẋ(t) f(x(t), u(t)) and u(t) U h j (x(t)) 0 for j = 1,...,r. Assume, for some c > 0 such that f and g are C 1 functions U is a closed, bounded set f(x, u) c(1 + x ) and min u U max j I(x) hx(x) j f(x, u) < 0 for all x A. (I(x) is set of active indices.) Constraint Qualification.

12 Filippov Estimates for State Constrained Control Systems Important analytical tool for investigating non-degeneracy of optimality conditions, H-J characterizations of the value function, etc. Theorem 1 (Soner 86,.., Frankowska and Rampazzo 00) Assume standing hypotheses. Then, for any trajectory ˆx(.) : [0, 1] R n s.t. x(0) A(0), there exists a feasible trajectory x(.) such that x(0) = ˆx(0) ˆx(.) x(.) L K h(ˆx(.)), where h(ˆx(.)) := max {h 1 (ˆx(t)) 0,..., h r (ˆx(t)) 0} t [0,1] (K is a constant independent of ˆx(.).) Index of Constraint Violation

13 L 1 Estimates on the Controls Soner s estimate was used to show the value function satisfies the HJ equation. For establishing nondegeneracy of the Max. Principle (general case) we need something stronger: Theorem 2 (L 1 Control Estimates for 1 State Constraint) Assume standing hypotheses and r = 1 (one state constraint) Then, for any pair (ˆx(.), û(.)) s.t. ˆx(0) A(0), there exists a feasible pair (x(.), u(.)) such that x(0) = ˆx(0) and û(.) u(.) L 1 K h(ˆx(.)) (K does not depend on ˆx(.))

14 t ˆx(.) h + W set - modify velocity ˆx(.) here only Figure: Construction of the Feasible Trajectory Feasible trajectory x(.): u(t) = { v for t [ t, t + δ] W ū(t) otherwise where v is inward pointing control, W = {t d/dt h(ˆx(t)) > 0} and δ > 0 is chosen so that meas {[ t, t + δ] W } = c h(ˆx(.))

15 L 1 Control Estimates for 2 or more State Constraints? Conjecture Preceding L 1 control estimate is valid for multiple state constraints. Plausible, but 1 state constraint proof does not adapt! (At least five papers, over the past 12 years claim validity of this conjecture, but are all based on erroneous proofs!)

16 L 1 Control Estimate Counter-Example The set A Velocity Set F=f(x,U) Figure: Example where Linear L 1 Estimate is not Valid

17 L 1 Control Estimate Counter-Example, Cont. x 1 1 x 1 (.) ǫ 2 (0,0) 1 t 1 ˆx 1 (.) 2 Velocity Set F Figure: Example where L 1 control Estimate is not Valid

18 Counter-Example: Details f(x, u) = u, U = co ({(1, +2)}, {(1, 2)}, {(0, 0)}) A = {(y, x) R 2 x y} Then h(ˆx(.)) = ǫ and û(.) u(.) L 1 N (ˆx(t i+1 ) x(t i+1 )) (ˆx(t i ) x(t i )) i=1 2 ǫ N, where N = number of switches: 3 N 1 2 ( 1 ǫ + 1). So û(.) u(.) L 1 const. h(ˆx(.)) log h(ˆx(.)).

19 A Superlinear, L 1 Control Estimate Theorem 3 (h + log(h + ) Estimates For Two State Constraints) Assume r = 2 (Two State Constraints) f(x, U) closed, bounded; active h j (x) s lin. indep. and co f(x, U) int A for x A ( inward pointing ) Then, given any pair (ˆx(.), û(.)) with ˆx(0) A, there exists a feasible (x(, ), u(.)) such that x(0) = ˆx(0) and u(.) û(.) L 1 K h(x(.)) log e h(x(.)).

20 The Strictly Convex Case Theorem 4 (Strictly Convex F ) Assume r = 2 (two state constraints) f(x, U) R n is compact and strictly convex; active h j (x) s lin. indep. and co f(x, U) int A for x A ( inward pointing ) Then, given any (ˆx(.), û(.)) pair with ˆx(0) A, there exists a feasible (x(.), u(.)) such that x(0) = ˆx(0) and u(.) û(.) L 1 K h(ˆx(.)), Linear, L 1 Control Estimate!

21 Earlier Counter Example Recall earlier example: The set F The Set A Figure: Example where Linear L 1 Estimate is not Valid

22 Optimality of Estimates Consider earlier example f(x, U) = co {(1, +2), (1, 2), (0, 0)} A = {(y, x) R 2 x y} We showed: for any ǫ > 0, there exists a (ˆx(.), û(.)) pair s.t. ˆx(0) = 0, h(ˆx(.)) = ǫ and û(.) u(.) L 1 K h(ˆx(.)) log(h(ˆx(.))), for all (x(.), u(.)) pairs with x(0) = ˆx(0).

23 Optimality of Estimates, Cont. Data for example: satisfies hypotheses for Thm. 3 ( h + log h + estimate) violates strict convexity hypothesis of Thm. 4 (linear estimate) Implications Thm. 4 h(ˆx(.)) log h(ˆx(.)) estimate is optimal: for any constant K > 0, ǫ > 0 and modulus θ(.) : (0, ) (0, ) such that lim ǫ 0 θ(ǫ) ǫ log e (ǫ) 0, there exists an F -trajectory ˆx(.) such that such that h(ˆx(.)) ǫ and û(.) u(.) L 1 > K θ(h(ˆx(.)) for all feasible F -trajectories x(.) with left endpoint ˆx(0). Thm. 4 (linear, W 1,1 estimate): strict convexity hypothesis cannot be dropped

24 Concluding Remarks Filippov estimates have an important role in the derivation of optimality conditions for state constrained optimal control (first order necessary conditions and Hamilton Jacobi conditions). Linear estimates valid for single state constraint. It surprised the optimal control community that similar linear estimates are not valid, for multiple constraints. For two constraints (simple edge ) Filippov type estimates can be established involving a superlinear modulus h log(h) function. It remains an open question, what kinds of estimates are valid when there are 3 or more state constraints. The Holy Grail is this area is to validate L 1 estimates for control systems, when the constraint set is a general closed convex set (say) in R n.

25 Some references [1]. P. Bettiol, A. Bressan and R. Vinter, On Trajectories Satisfying a State Constraint: W 1,1 Estimates and Counter-Examples, SIAM J. Control and Optim. (submitted) [2]. P. Bettiol, A. Bressan and R. Vinter, Estimates for Trajectories Confined to a Cone in R n, SIAM J. Control and Optim. (submitted) [3]. P. Bettiol and R. Vinter, Sensitivity Interpretations of the Co-state Variable for Optimal Control Problems with State Constraints, SIAM J. Control and Optim. (in press). [4]. P. Bettiol and R. Vinter, Sensitivity Interpretations of the Co-state Variable for Optimal Control Problems with State Constraints, IEEE TAC (submitted). Richard Vinter (Investigator) and Piernicola Bettiol (RA1.A), Control for Energy and Sustainability Programme (Project Optimal Control ).