Lecture Note 13:Continuous Time Switched Optimal Control: Embedding Principle and Numerical Algorithms

Transcription

1 ECE785: Hybrid Systems:Theory and Applications Lecture Note 13:Continuous Time Switched Optimal Control: Embedding Principle and Numerical Algorithms Wei Zhang Assistant Professor Department of Electrical and Computer Engineering Ohio State University, Columbus, Ohio, USA Spring 217 Lecture 12 (ECE785 Sp17) Wei Zhang(OSU) 1 / 3

2 Outline Switched Optimal Control Problems Embedding Principle and Chattering Lemma Solving Relaxed Switched Optimal Control Switched Optimal Control Problems Lecture 12 (ECE785 Sp17) Wei Zhang(OSU) 2 / 3

3 Switched Optimal Control Problems (1/5) Switched nonlinear systems: ẋ(t) = f σ(t) (t, x(t), u(t)), where σ(t) Q = {1,..., q} (1) Hybrid control: ξ(t) = (u(t), σ(t)) with constraints: u(t) U R m, σ(t) Q, where U bounded and convex State trajectory driven by ξ: x(t; ξ), or simply x(t). A finite time horizon, w.l.g., assume T = [, 1]. State trajectory constraint: h j (x(t)), j J = {1, 2,... n s }, t T. Switched Optimal Control Problems Lecture 12 (ECE785 Sp17) Wei Zhang(OSU) 3 / 3

4 Switched Optimal Control Problems (2/5) Cost function: J(x(1; ξ)) - only penalize terminal state. - problems with running cost can be reduced to this form by introducing additional state. Notations to emphasize dependence on ξ: - φ t(ξ) x(t; ξ), ψ j,t(ξ) h j(x(t; ξ)), J(ξ) J(x(1; ξ)) - Overall constraint functional: Ψ(ξ) max j,t ψ j,t(ξ). Switched Optimal Control Problems Lecture 12 (ECE785 Sp17) Wei Zhang(OSU) 4 / 3

5 Switched Optimal Control Problems (3/5) Assumption 1. - f i(t, x, u), h j(x), J(x) are Lipchitz continuous w.r.t all arguments - f i (t, x, u), f i x arguments (t, x, u), h j u x An equivalent way to write system dynamics: J (x), (x) exist and are Lipchitz continuous w.r.t. all x ẋ = f(t, x, u, d) q d i (t)f i (t, x, u) i=1 where d(t) = [d 1 (t),..., d q (t)] T is a corner of the q-simplex: { } q Σ q p = (d 1,..., d q ) {, 1} q d i = 1 i=1 Switched Optimal Control Problems Lecture 12 (ECE785 Sp17) Wei Zhang(OSU) 5 / 3

6 Switched Optimal Control Problems (4/5) Control Spaces: - We say f : [, 1] F belongs to L 2([, 1], F ) if ( 1 1/2 f L2 = f(t) 2dt) 2 < - Continuous input space: U = L 2([, 1], U) - Discrete input space: D p = L 2([, 1], Σ q p) - Overall optimization space: X = L 2([, 1], R m ) L 2([, 1], R q ) - Pure optimization space: X p = U D p Switched Optimal Control Problems Lecture 12 (ECE785 Sp17) Wei Zhang(OSU) 6 / 3

7 Switched Optimal Control Problems (5/5) (Pure) Switched Optimal Control Problem: { P p : Jp inf ξ J(ξ) = (2) subj. to Ψ(ξ), ξ X p Challenges: space X p = U D is not a vector space due to D, on which gradient of J and Ψ are not well defined. Switched Optimal Control Problems Lecture 12 (ECE785 Sp17) Wei Zhang(OSU) 7 / 3

8 Outline Switched Optimal Control Problems Embedding Principle and Chattering Lemma Solving Relaxed Switched Optimal Control Embedding Principle and Chattering Lemma Lecture 12 (ECE785 Sp17) Wei Zhang(OSU) 8 / 3

9 Relaxed System (1/2) The Key idea for solving P p is to embed the switched systems into a larger class of nonlinear systems for which d takes values inside the entire q-simplex (not just the cornor points) q-simplex: Σ q r = {(d 1,..., d q ) [, 1] q q i=1 d i = 1}. Relaxed System: ẋ(t) = i=q d i (t)f i (t, x(t), u(t)), with x() = x. (3) - d(t) Σ q p original switched systems - d(t) Σ q r relaxed switched systems - The set of all trajectories of the switched system is contained in that of the relaxed system. Embedding Principle and Chattering Lemma Lecture 12 (ECE785 Sp17) Wei Zhang(OSU) 9 / 3

10 Relaxed System (2/2) Relaxed control spaces: - Relaxed discrete input space: D r = L 2 ([, 1], Σ q r) - Relaxed optimization space: X r = U D r Relaxed Switched Optimal Control Problem P r : { P r : Jr inf ξ J(ξ) = (4) subj. to Ψ(ξ), ξ X r Obviously: J r J p Problem P r can be solved using classical optimal control methods Embedding Principle and Chattering Lemma Lecture 12 (ECE785 Sp17) Wei Zhang(OSU) 1 / 3

11 Embedding-Based Approach (1/5) Solution through Embedding: - Solve P r, resulting in ξ r X r - project back to pure space: Γ(ξ r ) ξ p X p Question: can we find a good projection without losing much on performance? - Answer: Yes. - The cost of any relaxed control input ξ r can be approximated arbitrarily well by a pure control input ξ p. - This is known as the Chattering Lemma. Embedding Principle and Chattering Lemma Lecture 12 (ECE785 Sp17) Wei Zhang(OSU) 11 / 3

12 Embedding-Based Approach (2/5) Lemma 1 (Chattering Lemma). ɛ >, ξ X r, ξ p X p s.t. φ t(ξ r) φ t(ξ p) 2 ɛ Proof of chattering lemma - We show the case with M = 2 with no continuous control. The result can be easily extended to the general case. - Given an arbitrary α(t) [, 1]. Let φ t be the solution to ẋ = f(t, x(t)) = α(t)f (t, x(t)) + (1 α(t))f 1(t, x(t)). - We want to construct another α(t) {, 1} so that the corresponding solution φ t to ẋ(t) = f(t, x(t)) = α(t)f (t, x(t)) + (1 α(t))f 1(t, x(t)) satisfies the desired inequality. Embedding Principle and Chattering Lemma Lecture 12 (ECE785 Sp17) Wei Zhang(OSU) 12 / 3

13 Embedding-Based Approach (3/5) - Given partition = t < t 1 < < t n = 1 with t k+1 t k = t. Choose t k (t k, t k+1 ) such that t k(1 α(τ))dτ = t k+1 α(τ)dτ. We propose to t k t k construct { if t [t k, t α(t) = k) 1 if t [t k, t k+1 ) - Now let s derive a bound for φ t φ t. Note that φ t φ t = = t t f(τ, φ τ ) f(τ, φ τ )dτ [ f(τ, φ τ ) f(τ, ] φ τ ) dτ + t Define f (t) = f 1(t, φ t) f (t, φ t), t [, 1]. [ f(τ, φτ ) f(τ, φ τ )] dτ (5) Embedding Principle and Chattering Lemma Lecture 12 (ECE785 Sp17) Wei Zhang(OSU) 13 / 3

14 Embedding-Based Approach (4/5) - first term of (5) = ( t ) k tk+1 (1 α(τ))f (τ))dτ α(τ)f (τ)dτ k t k t k = ( [ ] [ t ] ) f k tk+1 (t k ) (1 α(τ))dτ f (t k ) α(τ)dτ + e k = k t k t k k e k where t k tk+1 e k = (1 α(τ))(f (τ) f (t k ))dτ α(τ)(f (τ) f (t k ))dτ L t 2 t k t k Hence, choose t small enough: first term ɛ 2 Embedding Principle and Chattering Lemma Lecture 12 (ECE785 Sp17) Wei Zhang(OSU) 14 / 3

15 Embedding-Based Approach (5/5) - second term of (5): t [ t f(τ, φτ ) f(τ, φ τ )] dτ L φ τ φ τ dτ t φ t φ t ɛ + L φ τ φ τ dτ by Gronwall inequality ɛ The proof is constructive, but is not the best way to construct pure control input. A more effective projection strategy based on wavelet can be found in [vasudevan213consistent1; vasudevan213consistent2]. Embedding Principle and Chattering Lemma Lecture 12 (ECE785 Sp17) Wei Zhang(OSU) 15 / 3

16 Outline Switched Optimal Control Problems Embedding Principle and Chattering Lemma Solving Relaxed Switched Optimal Control Solving Relaxed Switched Optimal Control Lecture 12 (ECE785 Sp17) Wei Zhang(OSU) 16 / 3

17 How to solve the relaxed problem? (1/3) Now the question comes back to how to solve the relaxed optimal control ξ r? This is a classical optimal control problem. Analytical solution usually does not exist. ξ r can be found using gradient type of algorithms in function space. Solving Relaxed Switched Optimal Control Lecture 12 (ECE785 Sp17) Wei Zhang(OSU) 17 / 3

18 How to solve the relaxed problem? (2/3) The key is to compute the directional derivative: DJ(ξ; η) and Dψ j,t (ξ; η) - If DJ(ξ; η) <, we can decrease cost by moving in η direction - If max j,t Dψ j,t(ξ, η) <, we can reduce infeasibility by moving in η direction - Once we have DJ and Dψ j,t, numerous algorithms are available to find local min (See [polak212optimization]) Since d can be varied continuously for problem P r. We move d into u and deal with typical nonlinear system: ẋ = f(t, x, u). In this case, ξ = u. Solving Relaxed Switched Optimal Control Lecture 12 (ECE785 Sp17) Wei Zhang(OSU) 18 / 3

19 How to solve the relaxed problem? (3/3) Directional derivative of state trajectory Dφ t (u; η) is given by Dφ t (u; η) = t where Φ(t, τ; u) is the unique solution to With Dφ t (u; η), we can easily find ( ) f Φ(t, τ; u) u (τ, φ τ (u), u(τ)) η(τ) dτ (6) Φ f (t, τ) = t x (t, φ t(u), u(t)) Φ(t, τ) (7) DJ(u; η) = J x (φ t f (u))dφ tf (u; η), Dψ j,t (u; η) = h j x (φ t(u))dφ t (u; η) (8) Solving Relaxed Switched Optimal Control Lecture 12 (ECE785 Sp17) Wei Zhang(OSU) 19 / 3

20 Example I: Switching Time Optimization Problem (1/4) Example 1 (Switching Time Optimization Problem). Consider switched linear system 1 t [, u 1 ) ẋ = A σ(t) x(t), σ(t) = 2 t [u 1, u 2 ), 1 t [u 2, 1] A 1 = 1 [ ] 1, A = 1 [ ] [ 1 1 1, x() = where mode sequence is known and the transition time u = [u 1, u 2 ] needs to be optimized with cost function J(u) = x(t) 2 dt. ] Solving Relaxed Switched Optimal Control Lecture 12 (ECE785 Sp17) Wei Zhang(OSU) 2 / 3

21 Example I: Switching Time Optimization Problem (2/4) Solution to Example 1 First note that this problem involves nontrivial running cost. To address this issue, we first augment the state space by introducing z R with dynamics ż(t) = x 2 1(t) + x 2 2(t). It directly follows that z(1) = 1 x 2 dt. Let x(t) = [ x(t) z(t) ], and the augmented vector field f is given as follows [ [ x(t) = f(t, x, u) = [ A 1 x(t) x(t) 2 A 2 x(t) x(t) 2 A 1 x(t) x(t) 2 Augmented initial state is given by x() = as J( x(1)) = 1 2 z(1). ] ] ] f 1 (t, x, u), t [, u 1 ) f 2 (t, x, u), t [u 1, u 2 ) f 1 (t, x, u), t [u 2, 1] 1 1 (9), and the cost is redefined Solving Relaxed Switched Optimal Control Lecture 12 (ECE785 Sp17) Wei Zhang(OSU) 21 / 3

22 Example I: Switching Time Optimization Problem (3/4) From above discussion (6)-(8), we know that in order to obtain Dφ t (u; η), it remains to compute f f u and x. For t (, u 2 ), we have. f(t, x, u) = f 1(t, x, u) + U(t u 1)[f 2(t, x, u) f 1(t, x, u)], where U(a) is a step function. It follows that Similarly, f u 1 = δ(t u 1 )[f 1 (t, x, u 1 ) f 2 (t, x, u 1 )]. f u 2 = δ(t u 2 )[f 2 (t, x, u 2 ) f 1 (t, x, u 2 )] f x = [ A i 2x 1(t) 2x 2(t) ] R 3 3. Solving Relaxed Switched Optimal Control Lecture 12 (ECE785 Sp17) Wei Zhang(OSU) 22 / 3

23 Example I: Switching Time Optimization Problem (4/4) Plug into (6), we have Dφ t (u; η) = 1 t u1 Φ(t, u 1 )[f 1 ( x(u 1 )) f 2 ( x(u 1 ))]η t u2 Φ(t, u 2 )[f 2 ( x(u 2 )) f 1 ( x(u 2 ))]η 2 where 1 A is the indicator function of set A. Therefore, Dφ 1 (u; η) = Φ(t, u 1 )[f 1 ( x(u 1 )) f 2 ( x(u 1 ))]η 1 Therefore DJ(u; η) is given by + Φ(t, u 2 )[f 2 ( x(u 2 )) f 1 ( x(u 2 ))]η 2 DJ(u; η) = 1 2 T Dφ 1 (u; η). Implement the above results with initial condition given by u = [.3,.5] and a simple gradient descent algorithm with fixed step size, we have the optimal transition time is given by [.257,.7159] with a cost of Solving Relaxed Switched Optimal Control Lecture 12 (ECE785 Sp17) Wei Zhang(OSU) 23 / 3

24 Example II: Quadrotor (1/6) Example 2 (Switched optimal control problem: Quadrotor Model). ẋ = f i (x, u) where f 1 (x, u) = f 2 (x, u) = x 4 (t) x 5 (t) x 6 (t) sin x 3 (t) (u(t) + Mg) M cos x 3 (t) M x 4 (t) x 5 (t) x 6 (t) g sin x 3 (t) g cos x 3 (t) g Lu(t) I (u(t) + Mg) g,, f 3 (x, u) = x 4 (t) x 5 (t) x 6 (t) g sin x 3 (t) g cos x 3 (t) g Lu(t) I Cost function: ( t f 5u2 (t)dt + 5(x 1 (t f ) 6) 2 + 5(x 2 (t f ) 1) 2 x3 ) (t + sin f ) 2 constraint: u(t) [, 1 3 ] and x 2 (t) Solving Relaxed Switched Optimal Control Lecture 12 (ECE785 Sp17) Wei Zhang(OSU) 24 / 3

25 Example II: Quadrotor (2/6) Solution to Example 2: Similar to Example 1, we first augment the state space by defining z such that [ ż(t) = ] 5u 2 (t) and z() [ =. ] Let the overall augmented state be x x = and let z f fi i = 5u 2 be the augmented vector field corresponding to each f i. The augmented relaxed control variable is given by ũ = [ u α β ]T, and the augmented vector field is given by f = α f 1 + β f 2 + (1 α β) f 3. Cost function and constraint are then given by and respectively. J = z(t f ) + 5(x 1 (t f ) 6) 2 + 5(x 2 (t f ) 1) 2 + sin( x 3(t f ) ), 2 ψ t (ũ) = h(x(t)) = x 2 (t), Solving Relaxed Switched Optimal Control Lecture 12 (ECE785 Sp17) Wei Zhang(OSU) 25 / 3

26 Example II: Quadrotor (3/6) Let η = [ ] η α β T be another control signal, from above discussion (6)-(8), we know that in order to obtain DJ(ũ; η), it remains to compute Dφ t (ũ; η), J x (φ t f (ũ)) and h x (φ t(ũ)). The last two terms are easy to obtain, which are given by J [ x (φt (ũ)) = f and 1(x 1(t f ) 6) 1(x 2(t f ) 1) h x (φt(ũ)) = [ 1 ] T. 1 cos x 3(t f ) Now, we focus on Dφ t (ũ; η). Similar to the previous example, in order to compute Dφ t (ũ; η), we need to compute f f x and ũ. ] T Solving Relaxed Switched Optimal Control Lecture 12 (ECE785 Sp17) Wei Zhang(OSU) 26 / 3

27 Example II: Quadrotor (4/6) - f x x x x 1 1 f 1 x = 1 u+mg cos x3 M u+mg sin x3 M f 2 x = g cos x 3 g sin x 3 Solving Relaxed Switched Optimal Control Lecture 12 (ECE785 Sp17) Wei Zhang(OSU) 27 / 3

28 Example II: Quadrotor (5/6) - Hence, f x = [ f = f ũ u f 3 x = g cos x 3 g sin x α u+mg cos x3 + (1 α)g cos x3 M α u+mg sin x3 (1 α)g sin x3 M f α f β ], where f u = α f 1 u + β f 2 u + (1 α β) f 3 u. Solving Relaxed Switched Optimal Control Lecture 12 (ECE785 Sp17) Wei Zhang(OSU) 28 / 3

29 Example II: Quadrotor (6/6) - - Hence, f 1 u = sin x 3 M cos x 3 M 1u f ũ =, f 2 u = f α = f 1 f 3, L I 1u, f β = f 2 f 3 f 3 u = α sin x 3 M α cos x 3 M sin x 3 M u M u cos x 3 (1 α 2β) L I Lu I 2 Lu I 1u L I 1u Up to now, we have all the elements for computing the directional derivative DJ(ũ; η). Solving Relaxed Switched Optimal Control Lecture 12 (ECE785 Sp17) Wei Zhang(OSU) 29 / 3

30 References [Pol12] Elijah Polak. Optimization: algorithms and consistent approximations. Vol Springer Science & Business Media, 212. [Vas+13a] [Vas+13b] Ramanarayan Vasudevan et al. Consistent Approximations for the Optimal Control of Constrained Switched Systems Part 1: A Conceptual Algorithm. In: SIAM Journal on Control and Optimization 51.6 (213). Ramanarayan Vasudevan et al. Consistent approximations for the optimal control of constrained switched systems Part 2: An implementable algorithm. In: SIAM Journal on Control and Optimization 51.6 (213). Solving Relaxed Switched Optimal Control Lecture 12 (ECE785 Sp17) Wei Zhang(OSU) 3 / 3