Project Optimal Control: Optimal Control with State Variable Inequality Constraints (SVIC)

Transcription

1 Project Optimal Control: Optimal Control with State Variable Inequality Constraints (SVIC) Mahdi Ghazaei, Meike Stemmann Automatic Control LTH Lund University

2 Problem Formulation Find the maximum of the following cost functional subject to the contraints on states and control signals: J(u) = T 0 F (x(t), u(t), t)dt + S(x(T ), T ) ẋ(t) = f(x(t), t), x(0) = x 0 g(x(t), u(t), t) 0 h(x(t), t) 0 a(x(t ), T ) 0 b(x(t ), T ) = 0

3 Outline State Constraints Direct Adjoining Approach Indirect Adjoining Approach Example: Double Integrator Numerical Solution by Interior-Point Method Direct Approach Conclusion

4 Constraint Qualification for terminal constraints: rank for mixed constraints: [ a/ x ] diag(a) b/ x 0 = l + l a : E n E E l, b : E n E E l rank[ g/ u diag(g)] = s s : E n E m E E s

5 Order of Pure State Constraints h 0 (x, u, t) = h = h(x, t) h 1 (x, u, t) = ḣ = h x(x, t)f(x, u, t) + h t (x, t) h 2 (x, u, t) = ḣ1 = h 1 x(x, t)f(x, u, t) + h 1 t (x, t). h p (x, u, t) = ḣp 1 = h p 1 x (x, t)f(x, u, t) + h p 1 t (x, t) The state constraint is of order p if h i u(x, u, t) = 0 for 0 i p 1, h p u(x, u, t) 0

6 Direct Adjoining approach, Hamiltonian Hamiltonian and Lagrangian (P-form): H(x, u, λ 0, λ, t) = λ 0 F (x, u, t) + λf(x, u, t) L(x, u, λ 0, λ, µ, ν, t) = H(x, u, λ 0, λ, t) + µg(x, u, t) + νh(x, u, t) Control region: Ω(x, t) = {u E m g(x, u, t) 0}

7 Direct Adjoining approach, Theorem Part 1 {x (.), u (.)} an optimal pair for the problem on slide 2 over [0 T ] such that u (.)is right-continuous with left-hand limits constraint qualification holds for every tripple {t, x (t), u}, t [0 T ], u Ω(x (t), u) Assume x (t) has only finitely many junction times Then there exist a constant λ 0 0 a piecewise absolutely continuous costate trajectory λ(.) mapping [0, T ] into E n piecewise continuous multiplyer functions µ(.) and ν(.) mapping [0, T ] into E s and E q, respectively a vector η(τ i ) E q for each point τ i of discontinuity of λ(.) α E l and β E l, γ E q, not all zero, such that the following conditions hold almost everywhere:

8 Direct Adjoining approach, Theorem Part 2 Hamiltonian Maximization u (t) = arg max H(x (t), u, λ 0, λ(t), t) u Ω(x (t),t) L u [t] = H u[t] + µg u[t] = 0 λ(t) = L x [t] and µ(t) 0, µ(t)g [t] = 0 ν(t) 0, ν(t)h [t] = 0 dh [t] /dt = dl [t] /dt = L t [t] L [t]/ t,

9 Direct Adjoining approach, Theorem Part 3 At the terminal time T, the transversality conditions hold. λ(t ) = λ 0 S x [T ] + αa x [T ] + βb x [T ] + γh x [T ] α 0, γ 0, αa [T ] = γh [T ] = 0 For any time τ in the boundary interval and for any contact time τ, the costate trajectory may have a discontinuouty given by the following conditions. λ(τ ) = λ(τ + ) + η(τ)h x [τ] H [ τ ] = H [ τ +] η(τ)h t [τ] η(τ) 0, η(τ)h [τ] = 0

10 Indirect Adjoining approach, Hamiltonian Hamiltonian and Lagrangian (P-form): H(x, u, λ 0, λ, t) = λ 0 F (x, u, t) + λf(x, u, t) L(x, u, λ 0, λ, µ, ν, t) = H(x, u, λ 0, λ, t) + µg(x, u, t) + νh 1 (x, u, t) Control region: Ω(x, t) = { } u E m g(x, u, t) 0, h 1 (x, u, t) 0 if h(x, t) = 0

11 Indirect Adjoining approach, Theorem Part 1 {x (.), u (.)} an optimal pair for the problem on slide 2 such that x (.) has only finitely many junction times strong constraint qualification holds Then there exist a constant λ 0 0 a piecewise absolutely continuous xostate trajectory λ mapping [0, T ] into E n piecewise continuous multiplyer functions µ(.) and ν(.) mapping [0, T ] into E s and E q, respectively a vector η(τ i ) E q for each point τ i of discontinuity of λ(.) α E l and β E l, not all zero, such that the following conditions hold almost everywhere:

12 Indirect Adjoining approach, Theorem Part 2 Hamiltonian Maximization u (t) = arg max H(x (t), u, λ 0, λ(t), t) u Ω(x (t),t) L u [t] = 0 λ(t) = L x [t] µ(t) 0, µ(t)g [t] = 0 ν i is nondecreasing on boundary intervals of h i, i = 1, 2,, q with ν(t) 0, ν(t) 0, ν(t)h [t] = 0 and dh [t] /dt = dl [t] /dt = L t [t], whenever these derivatives exist.

13 Indirect Adjoining approach, Theorem Part 3 At the terminal time T, the transversality conditions λ(t ) = λ 0 S x [T ] + αa x [T ] + βb x [T ] + γh x [T ] α 0, γ 0, αa [T ] = γh [T ] = 0 hold. At each entry or contace time, the costate trajectory λ may have a discontinuouty of the form: λ(τ ) = λ(τ + ) + η(τ)h x [τ] H [ τ ] = H [ τ +] η(τ)h t [τ] η(τ) 0, η(τ)h [τ] = 0 and η(τ 1 ) ν(τ + 1 ) for entry times τ 1.

14 Example: Double Integrator T minimize x T Rx + u T Qu dt 0 ( ) 0 1 subject to ẋ(t) = x(t) u(t) 1 ( ) 0 u(t) 1 x 2 (t) c ( x(0) = x i T 0) x(1) = 0 2 1

15 Numerical Solution No of discretization points: 100 ( ) 1 0 Q = 0.25, R = 0 2 ( ) 0.17 c = 0.24, x I = 0 xv u t (sec) t (sec)

16 Direct Approach: Parameters, Hamiltonian and Lagrangian F (x(t), u(t), t) = x T Rx + u T Qu S(T ) = 0 f 1 (x(t), u(t), t) = x 2 (t) f 2 (x(t), u(t), t) = u(t) H = λ 0 (r 1 x 1 (t) 2 + r 2 x 2 (t) 2 + qu(t) 2 ) + λ 1 (t)x 2 (t) + λ 2 (t)u(t) L = H + µ 1 (t)(1 u(t)) + µ 2 (t)(1 + u(t)) + ν 1 (t)(c x 2 (t)) + ν 2 (t)(c + x 2 (t))

17 Direct Approach: Constraints h(x(t)) = ( ) c x2 (t) g(u(t)) = c + x 2 (t) ( ) 1 u(t) 1 + u(t) b(x(t)) = ( ) x1 (t) x 2 (t)

18 Direct Approach: Hamiltonian Maximization Without constraints, the Hamiltonian is maximized for: û(t) = q λ 2 (t) λ 0 Considering the constraints, the optimal solution is: 1 if λ 2 (t) < 2q u 1 1 λ (t) = 2 (t) 2 q λ 0 if 2q < λ 2 (t) < 2q 1 if λ 2 (t) > 2q Looking at the numerical solution time dependent u : 1 for t < τ λ 2 (t) u 2 q λ 0 for τ 1 < t < τ 2 (t) = 0 for τ 2 < t < τ λ 2 (t) 2 q λ 0 for τ 3 < t < τ 4 1 for τ 4 < t

19 Direct Approach: The Conditions L u[t] = H u[t] + µg u[t] = 0 2qu(t) + λ 2 (t) µ 1 (t) + µ 2 (t) = 0 (1) λ(t) = L x[t] { λ1 (t) = 2λ 0 r 1 x 1 (t) λ 2 (t) = 2λ 0 r 2 x 2 (t) λ 1 (t) + ν 1 (t) ν 2 (t) (2) µ(t) = ( µ 1 (t) µ 2 (t)) T 0 µ(t)g [t] = 0 (µ 1 (t) + µ 2 (t)) (µ 1 (t) µ 2 (t))u (t) = 0 (3) ν(t) = ( ν 1 (t) ν 2 (t)) T 0, ν(t) 0, ν(t)h (t) = 0 (ν 1 (t) + ν 2 (t))c (ν 1 (t) ν 2 (t))x 2 (t) = 0 (4)

20 Direct Approach: The Conditions dh [t] = dl = L t [t] dt dt 2r 1 x 1 (t)x 2 (t) 2r 2 x 2 (t)u(t) 2qu(t) u(t) + λ 1 (t)u(t) + λ 1 (t)x 2 (t) + λ 2 (t) u(t) + λ 2 (t)u(t) = 0 (5) u(t)(µ 1 (t) µ 2 (t)) + ( µ 1 (t) µ 2 (t))u(t) ( µ 1 (t) + µ 2 (t)) + (ν 1 (t) ν 2 (t))u(t) + ( ν 1 (t) ν 2 (t))x 2 (t) c( ν 1 (t) + ν 2 (t)) = 0 (6)

21 Direct Approach: The Conditions Transversality conditions: λ(t ) = βb x[t ] + γh x[t ] ) ( ) β1 = I β 2 ( λ1 [1 ] λ 2 [1 ] ( ) ( ) γ1 γ 0, γh[t ] = 0 (γ 1 + γ 2 )c (γ 1 γ 2 )x 2 (T ) = 0 γ 2 (7) Since x 2 (T ) = 0, we conclude γ 1 = γ 2 = 0. (8)

22 Direct Approach: u (t) = 1, t < τ 1 Assuming that x i (0) > 0 x 1 (t) = 1 2 t2 + x i x 2 (t) = t (9) From (1) follows: 2q + λ 2 (t) µ 1 (t) + µ 2 (t) = 0 (10) From (3) follows: From (4) follows: From (6) follows: µ 1 (t) = 0, µ 2 (t) free (11) ν 1 (t) = t c t + c ν 2(t) (12) 2 µ 1 + (t + c) ν 1 (t c) ν 2 + (ν 1 ν 2 ) = 0 (13)

23 Direct Approach: u (t) = 1, t < τ 1 From (2) follows: { λ1 (t) = r 1 t 2 + 2r 1 x i λ 2 (t) = 2r 2 t λ 1 (t) + ν 1 (t) ν 2 (t) (14) Combined with (5) follows: ν 1 (t) = ν 2 (t) (15) From (12) and (15) follows: ν 1 (t) = ν 2 (t) = 0 From (14) follows: { λ1 (t) = 1 3 r 1t 3 r 1 x i t K 1 λ 2 (t) = 1 12 r 1t 4 (r 1 x i + r 2 )t 2 (16) + tk 1 + K 2

24 Direct Approach: u (t) = 0, τ 2 < t < τ 3 x 1 (t) = ct + K 6 x 2 (t) = c (17) From (1) follows: From (3) follows: From (4) follows: λ 2 (t) µ 1 (t) + µ 2 (t) = 0 (18) µ 1 (t) = µ 2 (t) = 0 (19) ν 1 (t) = 0, ν 2 (t), free (20) (6) provides no new information.

25 Direct Approach: u (t) = 0, τ 2 < t < τ 3 From (2) follows: λ 1 (t) = 2r 1 ct + 2r 1 K 6 (5) gives the same equation for λ 1 (t) λ 2 (t) = λ 1 (t) 2r 2 c ν 2 (t) (21) From (18) and (19) we conclude λ 2 (t) = 0. Considering this, (21) simplifies to: ν 2 (t) = λ 1 (t) 2r 2 c (22) λ 1 (t) = r 1 ct 2 + K 6 t + K 7 (23)

26 Direct Approach: u (t) = 1, t > τ 4 x 1 (t) = 1 2 t2 + K 9 t + K 8 x 2 (t) = t + K 9 (24) From (1) follows: 2q + λ 2 (t) µ 1 (t) + µ 2 (t) = 0 (25) From (3) follows: µ 1 (t) free, µ 2 (t) = 0 (26) From (4) follows: (c t)ν 1 (t) + (c + t)ν 2 (t) (ν 1 (t) ν 2 (t))k 9 = 0 (27)

27 Direct Approach: u (t) = 1, t > τ 4 From (6) follows: ν 1 (t) ν 2 (t) + ( ν 1 (t) ν 2 (t))(t c + K 9 ) = 0 (28) From (2) follows: { λ1 (t) = 2r 1 ( 1 2 t2 + K 9 t + K 8 ) λ 2 (t) = 2r 2 (t + K 9 ) λ 1 (t) + ν 1 (t) ν 2 (t) (29) Combined with (5) follows: ν 1 (t) = ν 2 (t) (30) From (27) and (30) follows ν 1 (t) = ν 2 (t) = 0. From (29) follows: λ 1 (t) = 1 3 r 1t 3 + r 1 K 9 t 2 + 2r 1 K 8 t + K 10 λ 2 (t) = 1 12 r 1t r 1K 9 t 3 + (r 2 r 1 K 8 )t 2 (31) +(2r 2 K 9 K 10 )t + K 11

28 Direct Approach: u (t) = 1 2q λ 2(t), τ 1 < t < τ 2, τ 3 < t < τ 4 From (1) follows: From (3) follows: u(t) = λ 2(t) µ 1 (t) + µ 2 (t) 2q (32) µ 1 (t) = µ 2 (t) (33) µ 1 (t) + µ 2 (t) = 0 (34) Therefore, µ 1 (t) = µ 2 (t) = 0. From (4) follows: From (6) follows: c(ν 1 (t) + ν 2 (t)) (ν 1 (t) ν 2 (t))x 2 (t) = 0 (35) (ν 1 (t) ν 2 (t)) λ 2(t) 2q +( ν 1(t) ν 2 (t))x 2 (t) c( ν 1 (t)+ ν 2 (t)) = 0 (36)

29 Direct Approach: u (t) = 1 2q λ 2(t), τ 1 < t < τ 2, τ 3 < t < τ 4 From (5) follows: ( 2r 2 x 2 (t) + λ 1 (t) + λ ) 2 (t) λ 2 (t) = 0 (37) With λ 2 (t) 0, 2r 2 x 2 (t) + λ 1 (t) + λ 2 (t) = 0. (38) Comparing with (2) and using (4), we conclude that ν 1 (t) = ν 2 (t) = 0. Using (38), system dynamics and u (t) = 1 2q λ 2(t), we can conclude: λ (4) 2 (t) = 1 q (r 2 λ 2 (t) r 1 λ 2 (t)) (39)

30 Direct Approach: u (t) = 1 2q λ 2(t), τ 1 < t < τ 2, τ 3 < t < τ 4 From this follows: λ 2 (t) = C 1 e σ 1t + C 2 e σ 1t + C 3 e σ 2t + C 4 e σ 2t (40) with r 2 + r2 2 σ 1 = 4qr 1 2q r 2 s r2 2 σ 2 = 4qr 1 2q

31 Direct Approach: u (t) = 1 2q λ 2(t), τ 1 < t < τ 2, τ 3 < t < τ 4 From system equations: ẋ 2 (t) =u(t) x 2 (t) = 1 ( C 1 e σ1t + C 2 e σ1t C 3 e σ2t + C ) 4 e σ 2t + κ 1 2q σ 1 σ 1 σ 2 σ 2 (41) ẋ 1 (t) =x 2 (t) x 1 (t) = 1 2q ( C1 σ κ 1 t + κ 2 e σ 1t + C 2 σ 2 1 e σ 1t + C 3 σ 2 2 e σ2t + C ) 4 σ2 2 e σ 2t (42)

32 Direct Approach: u (t) = 1 2q λ 2(t), τ 1 < t < τ 2, τ 3 < t < τ 4 With (38) it follows: ( λ 1 (t) = σ 1 r ) ( 2 ) C 1 e σ1t C 2 e σ 1t σ 1 q ( + σ 2 r ) ( 2 ) C 3 e σ2t C 4 e σ 2t + 2r 2 K 12 (43) σ 2 q

33 Direct Approach: Initial and Final Conditions Initial condition on x used earlier. Final conditions for x: From system equations when u = 1 follows: Continuity u: u(τ + 1 ) = 1 λ 2(τ + 1 ) = 2q x 1 (1) = 0, x 2 (1) = 0 (44) K 8 = 1 2, K 9 = 1 (45) C 1 e σ 1τ 1 + C 2 e σ 1τ 1 + C 3 e σ 2τ 1 + C 4 e σ 2τ 1 = 2q (46) u(τ 2 ) = 0 λ 2(τ 2 ) = 0 C 1 e σ 1τ 2 + C 2 e σ 1τ 2 + C 3 e σ 2τ 2 + C 4 e σ 2τ 2 = 0 (47)

34 Direct Approach: Initial and Final Conditions u(τ + 3 ) = 0 λ 2(τ + 3 ) = 0 D 1 e σ 1τ 3 + D 2 e σ 1τ 3 + D 3 e σ 2τ 3 + D 4 e σ 2τ 3 = 0 u(τ 4 ) = 1 λ 2(τ 4 ) = 2q Continuity of x: x 1 (τ 1 ) = x 1(τ + 1 ) 1 2 τ x i = 1 2q D 1 e σ 1τ 4 + D 2 e σ 1τ 4 + D 3 e σ 2τ 4 + D 4 e σ 2τ 4 = 2q ( C1 σ K 12 τ 1 + K 13 e σ 1τ 1 + C 2 σ 2 1 e σ 1τ 1 + C 3 σ 2 2 e σ 2τ 1 + C ) 4 σ2 2 e σ 2τ 1

35 Direct Approach: Initial and Final Conditions x 2 (τ1 ) = x 2(τ 1 + ) τ 1 = 1 ( C 1 e σ 1τ 1 + C 2 e σ 1τ 1 C 3 e σ 2τ 1 + C ) 4 e σ 2τ 1 + K 12 2q σ 1 σ 1 σ 2 σ 2 x 1 (τ 2 ) = x 1(τ + 2 ) 1 2q ( C1 σ 2 1 = cτ 2 + K 6 e σ 1τ 2 + C 2 σ 2 1 e σ 1τ 2 + C 3 σ 2 2 e σ 2τ 2 + C ) 4 σ2 2 e σ 2τ 2 + K 12 τ 1 + K 13 x 2 (τ2 ) = x 2(τ 2 + ( ) 1 C 1 e σ 1τ 2 + C 2 e σ 1τ 2 C 3 e σ 2τ 2 + C ) 4 e σ 2τ 2 + K 12 = c 2q σ 1 σ 1 σ 2 σ 2

36 Direct Approach: Initial and Final Conditions x 1 (τ 3 ) = x 1(τ + 3 ) cτ 3 + K 6 = 1 2q + K 14 τ 3 + K 15 ( D1 σ 2 1 e σ 1τ 3 + D 2 σ 2 1 e σ 1τ 3 + D 3 σ 2 2 e σ 2τ 3 + D ) 4 σ2 2 e σ 2τ 3 x 2 (τ3 ) = x 2(τ 3 + ) c = 1 ( D 1 e σ 1τ 3 + D 2 e σ 1τ 3 D 3 e σ 2τ 3 + D ) 4 e σ 2τ 3 + K 14 2q σ 1 σ 1 σ 2 σ 2

37 Direct Approach: Initial and Final Conditions x 1 (τ 4 ) = x 1(τ + 4 ) 1 2q ( D1 σ 2 1 e σ 1τ 4 + D 2 σ 2 1 = 1 2 τ τ 4 e σ 1τ 4 + D 3 σ 2 2 e σ 2τ 4 + D ) 4 σ2 2 e σ 2τ 4 + K 14 τ 4 + K 15 x 2 (τ4 ) = x 2(τ 4 + ( ) 1 D 1 e σ 1τ 4 + D 2 e σ 1τ 4 D 3 e σ 2τ 4 + D ) 4 e σ 2τ 4 + K 14 = τ 4 1 2q σ 1 σ 1 σ 2 σ 2

38 Direct Approach: Continuity of λ λ 1 (τ1 ) = λ 1(τ 1 + ) 1 ( 3 r 1τ1 3 r 1 x i τ 1 K 1 = σ 1 r ) 2 (C1 e σ 1τ 1 C 2 e σ 1τ 1 ) σ 1 q ( + σ 2 r ) 2 (C3 e σ 2τ 1 C 4 e σ 2τ 1 ) + 2r2 K 12 σ 2 q λ 2 (τ 1 ) = λ 2(τ + 1 ) 1 12 r 1τ 4 1 (r 1 x i + r 2 )τ τ 1 K 1 + K 2 = C 1 e σ 1τ 1 + C 2 e σ 1τ 1 + C 3 e σ 2τ 1 + C 4 e σ 2τ 1

39 Direct Approach: Continuity of λ λ 1 (τ2 ) = λ 1(τ 2 + ( ) σ 1 r ) 2 (C1 e σ 1τ 2 C 2 e σ 1τ 2 ) σ 1 q ( + σ 2 r ) 2 (C3 e σ 2τ 2 C 4 e σ 2τ 2 ) + 2r2 K 12 σ 2 q = r 1 cτ2 2 + K 6 τ 2 + K 7 λ 2 (τ 2 ) = λ 2(τ + 2 ) C 1 e σ 1τ 2 + C 2 e σ 1τ 2 + C 3 e σ 2τ 2 + C 4 e σ 2τ 2 = 0

40 Direct Approach: Continuity of λ λ 1 (τ3 ) = λ 1(τ 3 + ) ( r 1 cτ3 2 + K 6 τ 3 + K 7 = σ 1 r ) 2 (D1 e σ 1τ 3 D 2 e σ 1τ 3 ) σ 1 q ( + σ 2 r ) 2 (D3 e σ 2τ 3 D 4 e σ 2τ 3 ) + 2r2 K 14 σ 2 q λ 2 (τ 3 ) = λ 2(τ + 3 ) 0 = D 1 e σ 1τ 2 + D 2 e σ 1τ 3 + D 3 e σ 2τ 3 + D 4 e σ 2τ 3

41 Direct Approach: Continuity of λ λ 1 (τ4 ) = λ 1(τ 4 + ( ) σ 1 r ) 2 (D1 e σ 1τ 4 D 2 e σ 1τ 4 ) σ 1 q ( + σ 2 r ) 2 (D3 e σ 2τ 4 D 4 e σ 2τ 4 ) σ 2 q + 2r 2 K 14 = 1 3 r 1τ 3 4 r 1 τ K 10 + r 1 τ 4 λ 2 (τ 4 ) = λ 2(τ + 4 ) D 1 e σ 1τ 2 + D 2 e σ 1τ 4 + D 3 e σ 2τ 4 + D 4 e σ 2τ 4 = 1 12 r 1τ r 1τ (r r 1)τ ( 2r 2 K 10 )τ 4 + K 11

42 Direct Approach: Results These conditions result in 24 unkowns and 20 equations, most of them nonlinear. To determine a solution, we need four more equations. Hence, the solution is calculated by measuring values at the following four points: u(0.2) = , x 2 (0.2) = u(0.8) = , x 2 (0.8) =

43 Direct Approach: Results x t (sec) u t (sec)

44 Conclusion Since H is concave, we can conclude that the solution to the example found by the direct approach is optimal, according to the Mangasarian-Type sufficient condition. The necessary condition by itself does not provide enough information to calculate the solution for the example.

45 References This work is based on the following references: Richard F. Hartl, Suresh P. Sethi and Raymond G. Vickson, "A Survey of the Maximum Principles for Optimal Control Problems with State Constraints," SIAM Review, Vol. 37, No. 2 (Jun., 1995), pp Seierstad, Atle, and Knut Sydsæter. Optimal Control Theory with Economic Applications. Amsterdam: North-Holland, 1987.