Direct and indirect methods for optimal control problems and applications in engineering

Size: px

Start display at page:

Download "Direct and indirect methods for optimal control problems and applications in engineering"

Lorena Sanders
5 years ago
Views:

Group School of Mathematics The University of Birmingham gerdtsm@maths.

1 Direct and indirect methods for optimal control problems and applications in engineering Matthias Gerdts Computational Optimisation Group School of Mathematics The University of Birmingham M. Gerdts K.U. Leuven, 29/04/ / 49

-200-250 0.7 0.6 0.5 0.4 0.3 0 0.2 0.4 x_1 0.6 0.8 1 0.8 0.6 0.4 x_2 0.2 robot control 0.2 0.1 0 1 0.

2 Optimal Control and Applications test-driving Optimal Control Theory & Algorithms aerospace engineering control x_ x_2 0.2 robot control x y nonlinear programming PDE control M. Gerdts K.U. Leuven, 29/04/ / 49

3 Contents 1 Optimal Control Problems 2 Nonsmooth Newton Method 3 Direct Discretization 4 Parametric Optimization and Real-time Optimization 5 Gear Shifts and Mixed-Integer Optimal Control 6 Numerical Results M. Gerdts K.U. Leuven, 29/04/ / 49

4 Motivation: Robot Control Task: Minimize t f subject to q (t) = v(t) M(q(t))v (t) = f (q(t), v(t), u(t)) for t [0, t f ] and u i (t) u max + additional constraints, e.g. q i (t) q max v i (t) v max M. Gerdts K.U. Leuven, 29/04/ / 49

5 Optimal Control Problem (without pure state constraints) OCP Minimize Z 1 0 f 0 (x(t), u(t), p)dt s.t. x (t) f (x(t), u(t), p) = 0 a.e. in [0, 1] c(x(t), u(t), p) 0 a.e. in [0, 1] ψ(x(0), x(1), p) = 0 Notation differential state: x W 1, ([0, 1], IR nx ) control: u L ([0, 1], IR nu ) parameter: p, no optimization variable! M. Gerdts K.U. Leuven, 29/04/ / 49

6 Solution Paradigms Indirect approach: OCP(p) minimum principle BVP(p) multiple shooting Direct approach: Version 1: discretization approach OCP(p) discretization NLP(p) SQP Version 2: function space approach OCP(p) SQP/Newton discretization LEQ Computation of sensitivities w.r.t. p: linearized KKT conditions existence: regularity assumptions, SSC, strict complementarity, controllability,... M. Gerdts K.U. Leuven, 29/04/ / 49

7 First-Order Necessary Conditions: Minimum Principle Hamiltonian: H(x, u, λ, η) := f 0 (x, u) + λ f (x, u) + η c(x, u) Adjoint equation: λ (t) = H x (x (t), u (t), λ(t), η(t)) a.e. in [0, 1] Transversality conditions: λ(0) = σ ψ x 0, λ(1) = σ ψ x 1 Optimality conditions: H u (x (t), u (t), λ(t), η(t)) = 0 a.e. in [0, 1] Complementarity condition: η(t) 0, η(t) c(x (t), u (t)) = 0 a.e. in [0, 1] M. Gerdts K.U. Leuven, 29/04/ / 49

8 Nonsmooth Reformulation Idea: reformulate minimum principle equivalently as nonlinear equation: F (z) = 0, F : Z Y Z, Y appropriate Banach spaces Apply some version of Newton s method! How? ( complementarity conditions) M. Gerdts K.U. Leuven, 29/04/ / 49

9 Fischer-Burmeister Function [Fischer 92] Fischer-Burmeister function: ϕ(a, b) = a 2 + b 2 a b Properties: convex (hence locally Lipschitzsch) ϕ(a, b) = 0 a 0, b 0, ab = 0 ( 2 ψ(a, b) := ϕ(a, b) 2 = a 2 + b 2 a b) is continuously differentiable! But: ϕ is not differentiable at (0, 0) M. Gerdts K.U. Leuven, 29/04/ / 49

10 Nonsmooth Reformulation Minimum principle is equivalent with nonsmooth equation ( ) F1 (z) F (z) = = 0, z = (x, u, λ, η, σ) F 2 (z) Smooth part: F 1 (z) := x ( ) f (x( ), u( )) λ ( ) + H x (x( ), u( ), λ( ), η( )) ψ(x(0), x(1)) λ(0) + ψ x 0 (x(0), x(1)) σ λ(1) ψ x 1 (x(0), x(1)) σ H u (x( ), u( ), λ( ), η( )) M. Gerdts K.U. Leuven, 29/04/ / 49

11 Nonsmooth Reformulation Nonsmooth part: ( complementarity conditions) F 2 (z) := ϕ( c(x( ), u( )), η( )) ϕ( c 1 (x( ), u( )), η 1 ( )) :=. ϕ( c nc (x( ), u( )), η nc ( )) M. Gerdts K.U. Leuven, 29/04/ / 49

12 Local Nonsmooth Newton s Method Local Nonsmooth Newton s Method (0) Choose z 0. (1) If some stopping criterion is satisfied, stop. (2) Choose an arbitrary V k F(z k ) and compute the search direction d k from the linear equation V k (d k ) = F (z k ). (3) Set z k+1 = z k + d k, k = k + 1, and goto (1). F (z k ) replaces nonexisting Jacobian F (z k )! M. Gerdts K.U. Leuven, 29/04/ / 49

13 Generalized Jacobian [Ulbrich 02] Replacement for Jacobian: Set-valued map with F : Z L(Z, Y ) F (z k )(z) := ( F 1 (z k )(z) S ( c x x + c u u) + Rη ) S = diag(s 1,..., s nc ), R = diag(r 1,..., r nc ), (s i, r i ) φ(z k )a.e. Motivation: chain rule in finite dimensional spaces ( Clarke s generalized Jacobian) M. Gerdts K.U. Leuven, 29/04/ / 49

14 Local Convergence Theorem 1 (G. 07) Assumptions: Let z be a zero of F. Uniform nonsingularity: V 1 L(Y,Z ) C V F(z), z U r (z ) Then: Locally superlinear convergence if f 0, f, c, ψ C 2 Proof: Show semismoothness F(z) F (z ) V (z z ) Y = o( z z Z ) V F(z) Existence & boundedness of V 1 : coercivity, uniform linear independence, controllability M. Gerdts K.U. Leuven, 29/04/ / 49

15 Application to OCP: Search Direction Linear operator equation V k (d) = F (z k ) has following structure: linear BVP of type ξ = Bξ + b q = E 0 ξ(0) + E 1 ξ(1) for ξ = (x, λ, σ) if ( H A := uu ( ) c u Sc u R ) non-singular and A 1 C, linear DAE-BVP of at least index 2, otherwise M. Gerdts K.U. Leuven, 29/04/ / 49

16 Optimal Control of a PDAE Minimize Z 1 2 subject to (0,T ) Ω z z d 2 dxdydt + α 2 Z (0,T ) Ω z t = z p + u, 0 = div(z), z(0, x, y) = 0, (x, y) Ω, z(t, x, y) = 0, (t, x, y) (0, T ) Ω, u u max. u 2 dxdydt M. Gerdts K.U. Leuven, 29/04/ / 49

17 2D-Stokes Equation Method of lines: (5-point-star for, forward FD for, backward FD for div) Minimize 1 2 Z T subject to index-2 DAE 0 z h (t) z d,h (t) 2 dt + α 2 Z T 0 u h (t) 2 dt z h (t) = A hz h (t) + B h p h (t) + u h (t), z h (0) = 0, 0 = Bh z h(t) and u h (t) u max. M. Gerdts K.U. Leuven, 29/04/ / 49

18 2D-Stokes Equation: Collocation Structure of collocation matrix: E 0 E 1 M 1 I......, I Collocation matrix: M nt n t = 25, n x = 18, 10 5 eqns n t = # time steps Per iteration and time step: (13005 eqns, nonzero, SuperLU) M. Gerdts K.U. Leuven, 29/04/ / 49

19 2D-Stokes Equation: Results [M. Kunkel] (T = 2, α = 10 5, n x = 18, n t = 25) 1 ITER α 2 F (zk ) e e e e e e e e e e e e-06 M. Gerdts K.U. Leuven, 29/04/ / 49

20 Limitations/Difficulties problems with pure state constraints problems with singular controls linear dependencies (redundant initial values, constraints) robustness computation of 2nd derivatives algorithmic differentiation M. Gerdts K.U. Leuven, 29/04/ / 49

21 State Constrained Optimal Control Problem OCP Minimize Φ(x(0), x(1), p) s.t. x (t) f (x(t), u(t), p) = 0 a.e. in [0, 1] c(x(t), u(t), p) 0 a.e. in [0, 1] s(x(t), p) 0 in [0, 1] ψ(x(0), x(1), p) = 0 Notation state: x W 1, ([0, 1], IR nx ) control: u L ([0, 1], IR nu ) parameter: p, no optimization variable! M. Gerdts K.U. Leuven, 29/04/ / 49

22 Direct Shooting x h (BDF, RK) x 1 xm u h (B-Splines) u 1 un control grid t 0 t N state grid t 0 t M M. Gerdts K.U. Leuven, 29/04/ / 49

23 Direct Shooting DOCP Structure Minimize Φ(x h (t 0 ; z), x h (t N ; z), p) s.t. c(t i, x h (t i ; z), u h (t i ; z), p) 0, i, s(t i, x h (t i ; z), p) 0, i, ψ(x h (t 0 ; z), x h (t N ; z), p) = 0, x h ( t j ; z) x j = 0, j M = 1 (single shooting) : small & dense; z = (x 1, u 1,..., u N ) M > 1 (multiple shooting) : large-scale & sparse; z = (x 1,..., x M 1, u 1,..., u N ) M. Gerdts K.U. Leuven, 29/04/ / 49

24 Nonlinear Programming Numerical solution: sequential quadratic programming (SQP) Gradient and Jacobian Evaluation: finite Differences Sensitivity ODE (IND) if #constraints #variables Adjoint ODE if #constraints < #variables Automatic Differentiation fixed step-size integrator M. Gerdts K.U. Leuven, 29/04/ / 49

states, 4 algebraic states) direct discretization method

25 Test-Drives: Numerical Results Background: nonlinear optimal control problem with index-1 DAE (41 differential states, 4 algebraic states) direct discretization method numerical solution by SQP M. Gerdts K.U. Leuven, 29/04/ / 49

26 Long Test-Courses Challenges for long test-courses: solution is required soon (within minutes) solution has to be computed automatically and robustly for many different test-courses (without user interaction, without initial guess) optimizing the whole test-course is very difficult (very good initial guess necessary; sensitive; frequent breakdowns of SQP) nonlinear model predictive control (NMPC) M. Gerdts K.U. Leuven, 29/04/ / 49

27 Nonlinear Model Predictive Control z h (t) z i h (t) Ingredients: local time horizon T > 0 discretization parameter h = T /N shifting parameter 0 < M N z i+1 h (t) T i T i + T T i+1 = T i + Mh T i+1 + T M. Gerdts K.U. Leuven, 29/04/ / 49

28 Test-Course of Oschersleben y(t) [m] Center of gravity -250 x=pilon x(t) [m] length: 3650 [m] time horizon: T = 10 [s] grid points: N = 50 shift parameter: M = 2 weights: c 1 = 1, c 2 = 0.01 CPU time: 11m39.948s M. Gerdts K.U. Leuven, 29/04/ / 49

29 Oschersleben: Velocity and Lateral Acceleration v(t) [m/s] Velocity t [s] ay(t) [m/s^2] Lateral acceleration t [s] M. Gerdts K.U. Leuven, 29/04/ / 49

30 Parametric Optimization NLP(p) Minimize f (z, p) s.t. g i (z, p) = 0, i = 1,..., n E, g i (z, p) 0, i = n E + 1,..., n g Sensitivity [Fiacco 83] Let z be a strongly regular local minimum of (NLP(p 0 )) for nominal parameter p 0. Then: NLP(p) has unique strongly regular local minimum z(p), λ(p) in neighborhood of p 0. z(p), λ(p) are continuously differentiable w.r.t. p. M. Gerdts K.U. Leuven, 29/04/ / 49

31 Idea of Real-time Optimization [Büskens,Maurer] Given: ( offline) nominal parameter p 0 strongly regular local solution z(p 0 ) to NLP(p 0 ) sensitivities dz dp (p 0) Real-time update formula by linearization (p: perturbation of p 0 ): ( online) z(p) z(p 0 ) + dz dp (p 0) (p p 0 ) M. Gerdts K.U. Leuven, 29/04/ / 49

32 Computation of Sensitivities At p = p 0 it holds dz dp dλ A(z,p 0 ) dp zz 2 L ( ) z g A(z,p 0 ) = z g A(z,p 0 ) 0 Linear equation! 1 2 zp L p g A(z,p 0 ) M. Gerdts K.U. Leuven, 29/04/ / 49

33 Real-time optimal control: Application to DOCP(p) Open-loop: nominal solution sensitivities measurement: parameter p p real-timecontrol u(t, p) ẋ = f (x, u, p) x(t) Open-loop real-time update formula (p: perturbation of p 0 ): ( online) u i (p) u i (p 0 ) + du i dp (p 0) (p p 0 ) (i = 0,..., N) real-time control approximation: piecewise linear/constant with u i (p) M. Gerdts K.U. Leuven, 29/04/ / 49

34 Emergency Landing Manoeuvre in Realtime Scenario: propulsion system breakdown Goal: maximization of range w.r.t. current position Controls: lift coefficient, angle of bank no thrust available; no fuel consumption (constant mass) M. Gerdts K.U. Leuven, 29/04/ / 49

35 VW Golf Controls: Steering wheel gas pedal/brakes gear shift (integer!) M. Gerdts K.U. Leuven, 29/04/ / 49

36 Variable Time Transformation Method Method: based on variable time transformation Dubovitskii/Milyutin used it to prove global minimum principle Lee et al. 99: Control Parametrization Enhancing Technique (CPET), optimal control problems with discrete control set leads to standard optimal control problem with continuous control set easy and fast M. Gerdts K.U. Leuven, 29/04/ / 49

37 Optimal Control Problem Mixed-Integer OCP Notation Minimize ϕ(x(t 0 ), x(t f )) s.t. ẋ(t) f (t, x(t), u(t), v(t)) = 0 s(t, x(t)) 0 ψ(x(t 0 ), x(t f )) = 0 u(t) U v(t) V state: x W 1, ([t 0, t f ], IR nx ) controls: u L ([t 0, t f ], IR nu ), v L ([t 0, t f ], IR nv ) U IR nu convex, closed, V = {v 1,..., v M } discrete M. Gerdts K.U. Leuven, 29/04/ / 49

38 Discretization Main grid Minor grid G h : t i = t 0 + ih, i = 0,..., N, h = t f t 0 N G h,m : τ i,j = t i + j h, j = 0,..., M, i = 0,..., N 1 M M = number of discrete values in V = {v 1,..., v M } M. Gerdts K.U. Leuven, 29/04/ / 49

39 Idea Replace the discrete control v by a fixed and piecewise constant function on the minor grid according to v Gh,M (τ ) = v j for τ [τ i,j 1, τ i,j ], i = 0,..., N 1, j = 1,..., M v M. v 1 t i 1 τ i 1,j t i τ i,j t i+1 τ i+1, j M. Gerdts K.U. Leuven, 29/04/ / 49

40 Idea: Variable Time Transformation Variable time transformation: and Remark: t = t(τ ), t(τ ) := t 0 + t f t 0 = Z τ Z tf t 0 t 0 w(s)ds, τ [t 0, t f ] w(s)ds w is the speed of running through [t 0, t f ]: dt dτ = w(τ ), τ [t 0, t f ] w(τ ) = 0 in [τ i,j, τ i,j+1 ) [t(τ i,j ), t(τ i,j+1 )] shrinks to {t(τ i,j )} M. Gerdts K.U. Leuven, 29/04/ / 49

41 Time Transformation w(τ ) t i+2 t(τ ) t i+1 t i t i 1 t i 1 τ i 1,j t i τ i,j t i+1 τ i+1, j M. Gerdts K.U. Leuven, 29/04/ / 49

42 New Control Consider w as new control subject to the restrictions: w(τ ) 0 for all τ ( no running back in time) w(τ ) piecewise constant on the minor grid G h,m Major grid points are invariant under time transformation: Z ti+1 Control set: t i w(τ )dτ = t i+1 t i = h, i = 0,..., N 1 W := w L ([t 0, t f ], R) w(τ ) 0, w piecewise constant on G h,m, Z ti+1 t i w(τ )dτ = t i+1 t i i M. Gerdts K.U. Leuven, 29/04/ / 49

43 Backtransformation v Gh,M (τ ): w(τ ): Corresponding control v(s) = v Gh,M (t 1 (s)): t(τ t i 1 t i,1 ) t(τ i t i+1,1 ) = t(τ i+1 i,2 ) = t(τ i+1,2 ) M. Gerdts K.U. Leuven, 29/04/ / 49

44 Transformed Optimal Control Problem TOCP Minimize ϕ(x(t 0 ), x(t f )) s.t. ẋ(τ ) w(τ )f (τ, x(τ ), u(τ ), v Gh,M (τ )) = 0 s(τ, x(τ )) 0 ψ(x(t 0 ), x(t f )) = 0 u(τ ) U w W Remarks: If w(τ ) 0 in [τ i,j, τ i,j+1 ] then x remains constant therein! v Gh,M is the fixed function defined before! TOCP has only continuous controls, no discrete controls anymore! M. Gerdts K.U. Leuven, 29/04/ / 49

45 Result: 20 grid points steering angle velocity w δ : gear shift µ: Braking force: F B (t) 0, Gas pedal position: φ(t) 1 Complete enumeration (1 [s] to solve DOCP): years Branch & Bound: 23 m 52 s, objective value: Transformation: 2 m s, objective value: M. Gerdts K.U. Leuven, 29/04/ / 49

46 Result: 40 grid points Steering angle velocity w δ : Gear shift µ: Braking force: F B (t) 0, Gas pedal position: φ(t) 1 Complete enumeration (1 [s] to solve DOCP): years Branch & Bound: 232 h 25 m 31 s, objective value: Transformation: 9 m s, objective value: M. Gerdts K.U. Leuven, 29/04/ / 49

47 Result: 80 grid points Steering angle velocity w δ : Gear shift µ: Braking force: F B (t) 0, Gas pedal position: φ(t) 1 65 m s, objective value: M. Gerdts K.U. Leuven, 29/04/ / 49

48 Next steps... mechanical multibody systems with contacts and friction: optimal control problems with compl. constraints robust optimal solutions: min max u U p P s.t. ϕ(x(u, p)(t 0 ), x(u, p)(t f )) max s(x(u, p)(t), u(t)) 0. p P optimal solutions w.r.t. perturbations/uncertainties M. Gerdts K.U. Leuven, 29/04/ / 49

49 Thanks for your attention! Questions? Further information: web.mat.bham.ac.uk/m.gerdts M. Gerdts K.U. Leuven, 29/04/ / 49

Trajectory Planning and Collision Detection for Robotics Applications

Trajectory Planning and Collision Detection for Robotics Applications joint work with Rene Henrion, Dietmar Homberg, Chantal Landry (WIAS) Institute of Mathematics and Applied Computing Department of Aerospace