Tutorial on Control and State Constrained Optimal Control Problems

Transcription

1 Tutorial on Control and State Constrained Optimal Control Problems To cite this version:. blems. SADCO Summer School Optimal Control, Sep 211, London, United Kingdom. <inria > HAL Id: inria Submitted on 6 Oct 211 HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

2 Tutorial on Control and State Constrained Optimal Control Problems Part 2 : Mixed Control-State Constraints University of Münster, Germany Institute of Computational and Applied Mathematics SADCO Summer School Imperial College London, September 5, 211

3 Outline 1 Optimal Control Problems with Control and State Constraints 2 Numrical Method: Discretize and Optimize 3 Theory of Optimal Control Problems with Mixed Control-State Constraints 4 Example: Rayleigh Problem with Different Constraints and Objectives 5 Example: Optimal Exploitation of Renewable Resources

4 Optimal Control Problem state x(t) R n, control u(t) R m. Dynamics and Boundary Conditions ẋ(t) = f (t, x(t),u(t)), a.e. t [,t f ], x(t) = x, ψ(x(t f )) = (ψ : R n R r ). Control and State Constraints c(x(t),u(t)), t t f, ( c : R n R m R k ) s(x(t)), t t f, ( s : R n R l ) Minimize tf J(u, x) = g(x(t f )) + f (t,x(t),u(t))dt

5 Discretization For simplicity consider a MAYER type problem with cost functional J(u, x) = g(x(t f )). This can achieved by considering the additional state variable x with ẋ = f (x,u), x () =. Then we have x (t f ) = tf f (t,x(t), u(t)). Choose an integer N N, a stepsize h and grid points t i : h = t f /N, t i : = ih, (i =,1,...,N). Approximation of control and state at grid points: u(t i ) u i R m, x(t i ) x i R n (i =,...,N)

6 Large-scale NLP using EULER s method Minimize J(u,x) = g(x N ) subject to x i+1 = x i + h f (t i,x i,u i ), i =,..,N 1, x = x, ψ(x N ) =, c(x i,u i ), i =,..,N, s(x i ), i =,..,N, Optimization variable for full discretization: z := (u,x 1,u 1,x 2,...,u N 1,x N,u N ) R N(m+n)+m

7 NLP Solvers AMPL : Programming language (Fourer, Gay, Kernighan) IPOPT : Interior point method (Andreas Wächter) LOQO : Interior point method (Vanderbei et al.) Other NLP solvers embedded in AMPL : cf. NEOS server NUDOCCCS : optimal control package (Christof Büskens) WORHP : SQP solver (Christof Büskens, Matthias Gerdts) Special feature: solvers provide LAGRANGE-multipliers as approximations of the adjoint variables.

8 Optimal Control Problem with Control-State Constraints State x(t) R n, Control u(t) R m. All functions are assumed to be suffciently smooth Dynamics and Boundary Conditions ẋ(t) = f (x(t),u(t)), a.e. t [,t f ], x() = x R n, ψ(x(t f )) = R k, ( = ϕ(x(), x(t f )) mixed boundary conditions ) Mixed Control-State Constraints α c(x(t),u(t)) β, t [, t f ], c : R n R m R Control bounds α u(t) β are included by c(x, u) = u. Minimize tf J(u,x) = g(x(t f )) + f (x(t),u) dt

9 Hamiltonian Hamiltonian H(x, λ, u) = λ f (x, u) + λ f (x, u) Augmented Hamiltonian λ R n (row vector) H(x, λ,µ, u) = H(x, λ, u) + µ c(x, u) = λ f (x, u) + λ f (x, u) + µ c(x, u), µ R. Let (u, x) L ([, T], R m ) W 1, ([, T], R n ) be a locally optimal pair of functions. Regularity assumption c u (x(t), u(t)) t J a J a := { t [, t f ] c(x(t), u(t)) = α or = β }

10 Minimum Principle of Pontryagin et al. and Hestenes Let (u, x) L ([, t f ], R m ) W 1, ([, t f ], R n ) be a locally optimal pair of functions that satisfies the regularity assumption. Then there exist an adjoint (costate) function λ W 1, ([, t f ], R n ) and a scalar λ, a multiplier function µ L ([, t f ], R), and a multiplier ρ R r associated to the boundary condition ψ(x(t f )) = that satisfy the following conditions for a.a. t [, t f ], where the argument (t) denotes evaluation along the trajectory (x(t), u(t), λ(t)) :

11 Minimum Principle of Pontraygin et al. and Hestenes (i) Adjoint ODE and transversality condition: λ(t) = H x (t) = (λ f + λ f ) x (t) µ(t) c x (t), λ(t f ) = (λ g + ρψ) x (x(t f )), (iia) Minimum Condition for Hamiltonian: H(x(t), λ(t),u(t)) = min { H(x(t), λ(t),u) α c(x(t),u) β } (iib) Local Minimum Condition for Augmented Hamiltonian: = H u (t) = (λ f + λ f ) u (t) + µ(t) c u (t) (iii) Sign of multiplier µ and complementarity condition: µ(t), if c(x(t),u(t)) = α ; µ(t), if c(x(t), u(t)) = β, µ(t) =, if α < c(x(t),u(t)) < β.

12 Evaluation of the Minimum Principle: boundary arc Boundary arc: Let [t 1,t 2 ], t 1 < t 2 < t f, be an interval with c(x(t), u(t)) = α or c(x(t), u(t)) = β t 1 t t 2. For simplicity assume a scalar control, i.e., m = 1. Due to the regularity condition c u (x(t),u(t)) there exists a smooth function u b (x) satisfying c(x,u b (x)) α ( β) x in a neighborhood of the trajectory. The control u b (x) is called the boundary control and yields the optimal control by the relation u(t) = u b (x(t)). It follows from the local minimum condition = H u = H u + µc u that the multiplier µ is given by µ = µ(x, λ) = H u (x, λ,u b (x))/c u (x,u b (x)).

13 Case I : Regular Hamiltonian, u is continuous CASE I : Consider optimal control problems which satisfy the Assumption: The Hamiltonian H(x, λ,u) is regular, i.e., it admits a unique minimum u. The strict Legendre condition holds: H uu (t) > t [,t f ]. (a) Then there exists a free control u = u free (x, λ) satisfying H u (x, λ,u free (x, λ)). (b) The optimal control u(t) is continuous in [,t f ]. Claim (b) follows from the continuity and regularity of H. The continuity of the control implies junctions conditions at junction points t k (k = 1, 2) with the boundary: u free (x(t k ), λ(t k )) = u b (x(t k )), µ(t k ) = (k = 1,2).

14 Rayleigh Problem with Quadratic Control The Rayleigh problem is a variant of the van der Pol Oszillator, where x 1 denotes the electric current. Control problem for the Rayleigh Equation Minimize J(x, u) = subject to t f (u 2 + x 2 1 ) dt (t f = 4.5) ẋ 1 = x 2, x 1 () = 5, ẋ 2 = x 1 + x 2 (1.4.14x 2 2 ) + 4u, x 2() = 5. Three types of constraints: Case (a) : no control constraints. Case (b) : control constraint 1 u(t) 1. Case (c) : mixed control-state constraint α u(t) + x 1 (t)/6, α = 1, 2

15 Case I (a) : Rayleigh problem, no constraint Normal Hamiltonian: H(x, λ,u) = u 2 + x λ 1 x 2 + λ 2 ( x 1 + x 2 (1.4.14x 2 2) + 4u) Adjoint Equations: λ 1 = H x1 = 2x 1 + λ 2 λ 1 (t f ) =, λ 2 = H x2 = λ 1 λ 2 (1.4.42x 2 2 ) λ 2(t f ) =, Minimum condition: = H u = 2u + 4λ 2 u = u free (x, λ) = 2λ 2. Shooting method for solving the boundary value problem for (x, λ): Determine unknown shooting vector s = λ() R 2 that satisfies the terminal condition λ(t f ) = : use Newton s method

16 Case I : Rayleigh problem without constraints x 1, x 2 state variables x 1, x (x 1,x 2 ) phaseportrait (x 1,x 2 ) u optimal control u lambda 1, lambda 2 adjoint variables lambda 1, lambda Note: Hamiltionian is regular, control u(t) is continuous (analytic).

17 Case I (b) : Rayleigh problem, constraint 1 u(t) 1 Hamiltonian H and adjoint equations are as in Case (a). The free control is given by u free (x, λ) = 2λ 2. Structure of optimal control: u(t) = 1 for t t 1 2λ 2 (t) for t 1 t t 2 1 for t 2 t t 3 2λ 2 (t) for t 3 t t f Junction conditions: Continuity of the control implies u(t k ) = 2λ 2 (t k ) = 1 1 1, k = 1,2,3. Shooting method for solving the boundary value problem for (x, λ): Determine shooting vector s = (λ(),t 1,t 2,t 3 ) R 2+3 that satisfies 2 terminal conditions λ(t f ) = and 3 junction conditions.

18 Rayleigh problem with control constraint u(t) 1 x 1, x state variables x 1, x (x 1,x 2 ) phaseportrait (x 1,x 2 ) u optimal control u lambda 1, lambda adjoint variables lambda 1, lambda Note: Hamiltonian is regular, control u(t) is continuous. Junction conditions: 2λ 2 (t k ) = 1 1 1, k = 1,2,3

19 Case I (c) : Rayleigh problem, 1 u + x 1 /6 Augmented (normal) Hamiltonian: H(x, λ,µ,u) = u 2 + x λ 1 x 2 +λ 2 ( x 1 + x 2 (1.4.14x 2 2 ) + 4u) + µ(u + x 1/6) Adjoint Equations: λ 1 = H x1 = 2x 1 + λ 2 µ/6, λ 1 (t f ) =, λ 2 = H x2 = λ 1 λ 2 (1.4.42x 2 2 ) λ 2(t f ) =, Free control : u free (x, λ) = 2λ 2. Boundary control : u b (x) = α x 1 /6 for α { 1,}. Multiplier : µ = µ(x, λ) = H u (x, λ,u b (x))/c u (x,u b (x)) = 2u b (x) + 4λ 2.

20 Case I (c) : Rayleigh problem, 1 u + x 1 /6 u optimal control u C(x,u) = u + x 1 /6 mixed constraint -1 <= u + x 1 /6 <= x 1, x state variables x 1, x mu multiplier for constraint -1 <= u + x 1 /6 <= Hamiltonian is regular, control u(t) is continuous. Junction conditions: 2λ(t k ) = α x 1 (t k )/6, α {, 1}.

21 Case I (c) : Rayleigh problem, structure of optimal control for mixed constraint 1 u + x 1 /6 mixed constraint -1 <= u + x 1 /6 <= C(x,u) = u + x 1 / u(t) = x 1 /6 for t t 1 2λ 2 (t) for t 1 t t 2 1 x 1 /6 for t 2 t t 3 2λ 2 (t) for t 3 t t 4 x 1 /6 for t 4 t t 5 2λ 2 (t) for t 5 t t f

22 Case II : control u appears linearly CASE II : Control appears linearly in the cost functional, dynamics and mixed control-state constraint. Let u be scalar. Dynamics and Boundary Conditions ẋ(t) = f 1 (x(t)) + f 2 (x(t)) u(t), a.e. t [,t f ], x() = x R n, ψ(x(t f )) = R k, Mixed Control-State Constraints α c 1 (x(t)) + c 2 (x(t)) u(t) β t [,t f ]. c 1, c 2 : R n R Minimize J(u, x) = g(x(t f )) + tf ( f 1 (x(t)) + f 2 (x(t) ) u(t)dt

23 Case II : Hamiltonian and switching function Normal Hamiltonian H(x, λ,u) = f 1 (x) + λf 1 (x) + [ f 2 (x) + λf 2 (x) ] u. Augmented Hamiltonian H(x, λ,µ,u) = H(x, λ,u) + µ(c 1 (x) + c 2 (x) u) The optimal control u(t) solves the minimization problem min { H(x(t), λ(t),u) α c 1 (x(t)) + c 2 (x(t)) u β } Define the switching function σ(x, λ) = H u (x, λ,u) = f 2 (x) + λf 2 (x), σ(t) = σ(x(t), λ(t)).

24 Case II : Hamiltonian The minimum condition is equivalent to the minimization problem min { σ(t) u) α c 1 (x(t)) + c 2 (x(t)) u β } We deduce the control law c 1 (x(t))+c 2 (x(t)) u(t) = α, if σ(t) c 2 (x(t)) > β, if σ(t) c 2 (x(t)) < undetermined if σ(t) The control u is called bang-bang in an interval I [, t f ], if σ(t) c 2 (x(t)) for all t I. The control u is called singular in an interval I sing [, t f ], if σ(t) c 2 (x(t)) for all t I sing. For the control constraint α u(t) β with c 1 (x) =, c 2 (x) = 1 we get the classical control law α, if σ(t) > u(t) = β, if σ(t) < undetermined, if σ(t)

25 Bang-Bang and Singular Controls

26 Case II : Rayleigh problem with 1 u(t) 1 Rayleigh problem with control appearing linearly Minimize J(x, u) = subject to t f (x x2 2 )dt (t f = 4.5) ẋ 1 = x 2, x 1 () = 5, ẋ 2 = x 1 + x 2 (1.4.14x 2 2 ) + 4u, x 2() = 5, 1 u(t) 1. Adjoint Equations: λ 1 = H x1 = 2x 1 + λ 2, λ 1 (t f ) =, λ 2 = H x2 = 2x 2 λ 1 λ 2 (1.4.42x 2 2 ), λ 2(t f ) =, The switching function σ(t) = H u (t) = 4λ 2 (t) gives the control law u(t) = sign (λ 2 (t)).

27 Case II : Rayleigh problem, 1 u(t) 1 u optimal control u x 1, x 2 state variables x 1, x u, lambda 2 / control u and (scaled) switching function lambda 1, lambda 2 adjoint variables lambda 1, lambda Control u(t) is bang-bang-singular. Switching conditions: λ(t 1 ) =, λ 2 (t) t [t 2,t f ].

28 Case II : Rayleigh problem, α u + x 1 /6 Minimize J(x, u) = subject to t f (x x2 2 )dt (t f = 4.5) ẋ 1 = x 2, x 1 () = 5, ẋ 2 = x 1 + x 2 (1.4.14x 2 2 ) + 4u, x 2() = 5, and the mixed control-state constraint Adjoint Equations: α u(t) + x 1 (t)/6 t t f. λ 1 = H x1 = 2x 1 + λ 2 µ/6, λ 1 (t f ) =, λ 2 = H x2 = 2x 2 λ 1 λ 2 (1.4.42x 2 2 ), λ 2(t f ) =,

29 Control law for α u + x 1 /6 The switching function is σ(t) = H u (t) = 4λ 2 (t). In view of c 2 (x) 1 we have the control law α <, if λ 2 (t) > u + x 1 /6 =, if λ 2 (t) < undetermined, if λ 2 (t)

30 Case II : Rayleigh problem, 1 u + x 1 /6 u optimal control u x 1, x 2 state variables x 1, x C(x,u) and (scaled) switching function 2 multiplier for constraint -1 <= u + x 1 /6 <=.5 1 u, lambda 2 / mu Note: constraint u(t) + x 1 (t)/6 is bang-bang.

31 Case II : Rayleigh problem, 2 u + x 1 /6 u optimal control u x 1, x 2 state variables x 1, x c(x,u), lambda 2 /1 c(x,u) and (scaled) switching function mu multiplier for constraint -1 <= u + x 1 /6 <= Note: constraint u(t) + x 1 (t)/6 is bang-singular-bang-singular.

32 Optimal Fishing, Clark, Clarke, Munro Colin W. Clark, Frank H. Clarke, Gordon R. Munro: The optimal exploutation of renewable resource stock: problem of irreversible investment, Econometric 47, pp (1979). State variables and control variables: x(t) : population biomass at time t [,t f ], renewable resource, e.g., fish, K(t) : amount of capital invested in the fishery, e.g., number of standardized fishing vessels available, E(t) : fishing effort (control), h(t) = E(t)x(t) is harvest rate, I(t) : investment rate (control),

33 Optimal Fishing: optimal control model Dynamics in [,t f ] ( here: a = 1, b = 5, γ = ) ẋ(t) = a x(t) (1 x(t)/b) E(t) x(t), x() = x, K(t) = I(t) γ K(t), K() = K. Mixed Control-State Constraint and Control Constraint E(t) K(t), I(t) I max, t [,t f ], Maximize benefit (parameters: r =.5, c E = 2, c I = 1.1 ) J(u,x) = tf exp( r t)(p E(t) x(t) c E E(t) c I I(t) )dt

34 Optimal Fishing: x =.5, K =.2, I max =.5 controls E, I and state variables.5*x, K 2 "E.dat" "I.dat" "K.dat" "x-scal.dat"

35 Optimal Fishing: x =.5, K =.6, I max =.5 controls E, I and state variables.5*x, K 2 "E.dat" "I.dat" "K.dat" "x-scal.dat" Fishing rate : E(t) =, E(t) = singular, E(t) = K(t). Investment rate : I(t) =, I(t) = I max, I(t) =.

36 Optimal Fishing: x =.2, K =.1, I max =.1 controls E, I and state variables.5*x, K 2 "E.dat" "I.dat" "K.dat" "x-scal.dat" Fishing rate : E(t) =, E(t) = K(t), E(t) singular, E(t) = K(t). Investment rate : 2 arcs with I(t) = I max.

37 Optimal Fishing: x = 1., K =.5, I max = controls E, I and state variables.5*x, K "E.dat" "I.dat" "K.dat" "x-scal.dat" Fishing rate : E(t) =, E(t) singular E(t) = K(t), Investment rate : 1 impulse with I(t) = I max.