Direct Optimal Control and Costate Estimation Using Least Square Method

Transcription

1 21 American Control Conference Marriott Waterfront, Baltimore, MD, USA June 3-July 2, 21 WeB22.1 Direct Optimal Control and Costate Estimation Using Least Square Method Baljeet Singh and Raktim Bhattacharya Aerospace Engineering Department, Texas A&M University, College Station, Texas 77843, USA. Abstract In this paper, we present a direct method to solve optimal control problems based on the least square formulation of the state dynamics. In this approach, we approximate the state and control variables in a finite dimensional Hilbert space. We impose the state dynamics as a weighted integral formulation based on the least square method to solve initial value problems. We analyze the resulting nonlinear programming problem to derive a set of conditions under which the costates of the optimal control problem can be estimated from the associated Karush-Kuhn-Tucker multipliers. We present numerical examples to demonstrate the applicability of the present method. I. ITRODUCTIO Direct methods to solve optimal control problems have become extremely popular. In a direct method, the optimal control problem (OCP) is discretized by parameterizing the unknowns to transcribe the continuous time OCP into a finite-dimensional nonlinear programming problem (LP)[1]. The P is then solved using numerical optimization techniques. Compared to indirect methods, a direct method requires much less analytical derivation, and the resulting P can be solved with relative ease. However, direct methods do not provide information about the optimality of the solution. The Minimum Principle can be used to check the optimality of a solution resulting from a direct method, however, this requires estimates of the dual variables. Therefore, the equivalence between dualization and discretization, for a direct method needs to be investigated. Direct methods for optimal control problems differ in how they approximate the state equations. Most of these methods are based on collocation techniques, where the constraints are imposed at a set of discrete time instances, popular among these methods are the Hermite- Simpson (HS) and the pseudospectral (PS) methods [2]. In another category of direct methods, known as taumethods [3], [4], [5], [6], [7], global orthogonal polynomials are used to parameterize the state and control Graduate student, b singh@tamu.edu Assistant Professor, raktim@tamu.edu trajectories, and the polynomial coefficients are treated as the optimization variables. For many direct methods the costates can be estimated from the Karush-Kuhn-Tucker (KKT) multipliers of the LP. In this framework, Stryk and Blurisch [8] showed the equivalence between the two for the HS method. Hager [9] presented the convergence analysis for Runge- Kutta based direct methods. Ross and Fahroo [1], [11] presented a similar result for Legendre pseudospectral method, where a set of closure conditions are derived under which the costates can be estimated from the KKT mutipliers. Williams [12] generalizes the same result for the Jacobi pseudospectral method. Benson et al. [13] have shown equivalence between the discrete costates and the KKT multipliers for the Gauss pseudospectral method. More recently, Singh et al. [14] presented the optimality analysis for the method of Hilbert space projection (MHSP). In this paper, we present a direct method based on the least square formulation of the state dynamics. In the least square method for optimal control (LSM oc ), we approximate the state and control variables as linear combinations of a priori selected basis functions of a Hilbert space. The state dynamics is imposed as a weighted integral formulation derived from the least square method to solve initial value problems. The LSM oc is flexible with respect to the choice of approximating functions, where both local and global basis functions can be employed. Another main contribution of this paper is the derivation of a costate estimation estimation procedure for the LSM oc. We examine the KKT conditions associated with the direct optimization solution and the discretized first-order necessary conditions for the optimal control problem to define a set of equivalence conditions under which the two approaches are completely equivalent, in which case the costate estimates can be obtained from the KKT multipliers of the LP. The paper is organized as follows. Section II defines the optimal control problem of interest. The detailed /1/$ AACC 1556

2 description of the LSM oc is presented in Section III. In Section IV, we derive the equivalence conditions and the costate estimates for the LSM oc. In Section V presents numerical examples to demonstrate the applicability of the present method. II. PROBLEM FORMULATIO A. Continuous-time Mayer Problem: M Without loss of generality, we consider an optimal control problem in Mayer form. We consider a continuous time autonomous system with free final time. The objective is to determine the state-control pair {X(τ) R n,u(τ) R m ;τ [,τ f ]} and time instance τ f, that minimize the cost, subject to the state dynamics, J = Ψ(X(τ f ),τ f ), (1) Ẋ(τ) = f(x(τ),u(τ)), (2) and end-point state equality constraints, X() = x ; ψ(x(τ f ),τ f ) = R p. (3) It is assumed that the optimal solution to the above problem exists, and the constraint qualifications required to apply the first-order optimality conditions are implicitly assumed. We consider an autonomous system because an extension to a non-autonomous system is straight forward. Currently, we do not consider any state or control path constraints. However, the present analysis can be extended to include path constraints, which is the scope of future work. III. THE LEAST SQUARE METHOD FOR OPTIMAL COTROL (LSM oc ) In a direct method to solve an optimal control problem, state and control trajectories are first approximated using a known functional form with a set of coefficients to be determined. Then, a differentiation or integration based method is applied to transform the state dynamics into a set of equations in the unknown coefficients [2]. In our approach, we approximate the state dynamics using the least square method based on the following theorem, Theorem 1: Consider an initial value problem (IVP), r(t) = g(z(t)) ż(t) =, t [,1]; z() = a. (4) Let H be an -dimensional Hilbert space equipped with a norm H and an inner-product, H, spanned by a set of linearly independent basis functions {φ j (t),t [,1]}. If z(t) = α jφ j (t) H, then z(t) is the stationary solution of the functional, J = r 2 H ν(z() a), (5) where ν is a lagrange multiplier. Proof: The stationary conditions are given by, r, r α j H 1 2 νφ j() =, j = 1,..,, (6) z() = a, (7) which are trivially satisfied with ν =. In the view of Theorem 1, we describe a direct method to solve problem M. In this method, we approximate the state dynamics as a stationary solution of (5) on a finitine-dimensional subspace V of H. We approximate the state and control trajectories in V := span{φ 1 (t),φ 2 (t),..,φ (t),t [,1]}, with a linearly independent basis set {φ j (t)}, and an inner product defined as, p,q = p T (t)q(t)dt; p(t),q(t) V. (8) We scale problem M appropriately so that the state and control trajectories can be approximated in V. Since V is defined over the time domain [,1], we use the following transformation to map the problem from the physical domain τ [,τ f ] to the computational domain t [,1], τ(t) = τ f t. (9) The state and control trajectories are approximated as x(t),û(t) V, so that, x(t) = X(τ(t)) x(t) = u(t) = U(τ(t)) û(t) = α k φ k (t), (1) β k φ k (t), (11) where, α k R n and β k R m are the unknowns. Differentiating the expression in (1) with respect to τ and using (9), we get, dx(τ(t)) dτ = 1 τ f ẋ(t) 1 τ f x(t) = 1 τ f α k φ k (t), (12) where an overdot denotes the derivative with respect to t. Using (2) and (12), we define the residual error in state dynamics as, r(t) = τ f f( x(t),û(t)) x(t). (13) Using (6), (7) and (8), we approximate the state dynamics as the following weighted integral form, [τ f f T x ( x(t),û(t))φ j (t) I φ j (t)]r(t)dt 1 2 νφ j() =, j = 1,..,, (14) x() x =, (15) 1557

3 with the end-point equality constraint in (3) imposed as, ψ( x(1),τ f ) =. (16) Here ν R n is an unknown to be determined. A. onlinear Programming Problem (LP): M φ Function approximation of state and control trajectories using (1) and (11), combined with the least square formulation as in (14), (15) and (16), transcribe problem M into a finite dimensional nonlinear programming problem, denoted as Problem M φ. For the subsequent treatment, we denote the approximate state dynamics as: f(αk,β k,φ k (t)) = f( x(t),û(t)). (17) Using similar notation for all other functionals, Problem M φ is to determine {α k R n,β k R m }, ν Rn and time instance τ f, that minimize the cost, subject to the constraints, Ĵ = Ψ(α k,φ k (1),τ f ), (18) [τ f ft x (α k,β k,φ k )φ j I φ j ][τ f f(αk,β k,φ k ) α k φ k ]dt 1 2 νφ j() =, (19) α k φ k () x =, ψ(α k,φ k (1),τ f ) =, (2) where j = 1,..,. Problem M φ constituting (18) to (2) can be solved using standard numerical optimization software. Any numerical quadrature scheme can be used to evaluate the integral expressions in (19). For the results presented in this paper, we use SOPT[15] as the optimization solver and MATLAB s Symbolic Math Toolbox for integral evaluations. ext, to facilitate the derivation of the costate estimation procedure and related equivalence conditions, we derive the KKT firstorder necessary conditions associated with Problem M φ. B. KKT conditions for Problem M φ : M φλ The Lagrangian for Problem M φ is formed by adjoining the cost function with the constraint equations. For brevity, we use f to denote f(α k,β k,φ k ). Using similar notation for all other variables, we have, J = γ T j γ T j [τ f ft x φ j I φ j ][τ f f α k φ k ]dt 1 2 νφ j() µ T ( α k φ k () x ) Ψη T ψ, (21) where γ j R n, µ R n and η R p are the KKT multipliers associated with the constraints given by (19) and (2) respectively. The KKT first-order necessary conditions are obtained by setting the derivatives of the Lagrangian J with respect to the unknowns {α i,β i,ν,γ i, µ,η,τ f } equal to zero. We have for i = 1,..,, J α i = [ φ i φ j τ f ft x φ i φ j τ f fx φ j φ i τ f ft x τ f fx φ i φ j ]γ j dt [τ f r T fxx φ i φ j ]γ j dt µφ i () [ Ψ x(1) ψ T x(1) η]φ i(1). (22) Using integration by parts, we write, φ i (t) φ j (t)dt = φ i φ j 1 τ f fx φ j φ i dt = τ f fx φ j φ i 1 φ j (t)φ i (t)dt, (23) (τ f fx φ j )φ i dt. (24) Using (22), (23), (24) and re-arranging, we get, also, = [ γ j φ j τ f ft x γ j φ j (τ f fx γ j φ j ) τ f ft x τ f fx γ j φ j ]φ i dt [τ f r T fxx γ j φ j ]φ i dt [ Ψ x(1) ψ T x(1) η]φ i(1) µφ i () [τ f fx γ j φ j (1) [τ f fx γ j φ j () γ j φ j (1)]φ i (1) γ j φ j ()]φ i (). (25) J µ = J α k φ k () x =, = ψ =, (26) η J ν = γ j φ j () =, (27) J = τ f [τ f ft β u fx i j φ j f γ T u γ j φ j ]φ i dt [τ f r T fxu γ j φ j ]φ i dt =, (28) J = [τ f fx φ i I φ i ][τ f f γ i k φ k ]dt α 1 2 νφ i() =, (29) 1558

4 J = τ f γ T j [τ f ft x φ j I φ j ] fdt γ T j f T x rφ j dt [ Ψ τ f η T ψ τ f ] =. (3) Equations (25) through (3) constitute the KKT conditions for Problem M φ. ext, we derive the first-order optimality conditions for problem M which are then discretized to derive equivalence between the costates of the optimal control problem and the KKT multipliers of the associated LP. IV. COSTATE ESTIMATIO As stated earlier, Singh and Bhattacharya [14] derived a set of conditions under which a linear mapping exists between the costates and the KKT multipliers of the nonlinear programming problem for the MHSP. More rigorously, their derivation is based on the commutative nature of problems B λφ and B φλ under a set of closure conditions, where Problem B φλ is the set of KKT conditions associated with the LP and Problem B λφ is the set of discretized first-order optimality conditions. We adopt a similar approach with a slight modification. We introduce set of auxiliary costates which are derived from the true costates of M. We write the first-order optimality conditions for M in term of the auxiliary costates. These auxiliary first-order optimality conditions are then compared to M φλ to derive equivalence conditions under which these two problems commute. A. First-Order Optimality Conditions for M : M λ Problem M can be solved by applying calculus of variations and Pontryagin s minimum principle. In this framework, the first-order necessary conditions for optimality lead to a two-point boundary value problem derived by using the augmented Hamiltonian H and the terminal cost C defined as, H (X,U,Λ) = Λ T (τ)f(x(τ),u(τ)), (31) C (X(τ f ),τ f,υ,κ) = Ψ(X(τ f ),τ f )υ T ψ(x(τ f ),τ f ) κ T (X() x ), (32) where Λ(τ) R n is the costate, υ R p and κ R n are the lagrange multipliers. Time dependence of state and control trajectories has been dropped for brevity. Problem M seeks to find the functions {X(τ),U(τ),Λ(τ);τ [,τ f ]}, vectors υ,κ and time instance τ f, that satisfy the following conditions, Ẋ = f, X() x =, H u = f T u Λ = ΛH x = Λf T x Λ =, {Λ(),Λ(τ f )} = { κ,c x(τ f )}, ψ(x(τ f ),τ f ) =,, H τ=τ f = C τ f. (33) B. Auxiliary First-Order Optimality Conditions: M ρ Here we introduce a set of auxiliary costates ρ(t) R n, t [,1] which lead to the estimation the actual costates λ(t) by establishing equivalence with the KKT multipliers of the LP Problem M φλ. We define ρ(t) as the solution of the differential equation, Λ(τ(t)) = λ(t) = τ f f x ρ(t) ρ(t); ρ() =. (34) The auxiliary first order optimality conditions are derived from (33) using (34) and (9), ẋ(t) = τ f f(x,u), x() x =, ψ(x(1),τ f ) = ρ τ f (f x ρ) τ f f T x ρ τ2 f f T x f x ρ =, H u = τ f f T u(τ f f x ρ ρ) =, ρ() = τ f f x ρ() ρ() = κ, τ f f x ρ(1) ρ(1) = C x(τ f ), (τ f ρ T f T x ρ T )f t=1 Ψ τ f υ T ψ τ f =. (35) C. Discretized First-Order Optimality Conditions: M ρφ Problem M ρ as defined by equation set (35) must be discretized to obtain conditions for optimality in the functional space V. The auxiliary costates are approximated as, ρ(t) = γ k φ k (t), (36) where γ k R n. Using Eqns. (1), (11), (36) we obtain the following weighted integral equations for equation set (35), = = = = [τ f ft x φ i I φ i ][τ f f α k φ k ]dt 1 2 πφ i(), (37) [ γ k φ k τ f ( f x γ k φ k ) τ f ft x γ k φ k τ 2 f f T x f x γ k φ k ]φ i dt, (38) τ f [ f T u(τ f fx γ k φ k γ k φ k )]φ i dt, (39) α k φ k () x, = ψ(α k,φ k (1),τ f ), (4) κ = τ f fx γ k φ k () C x(1) = τ f fx γ k φ k (1) = (τ f γ T k φ k f T x γ k φ k (), (41) γ k φ k (1), = γ k φ k (), (42) γ T k φ k ) f t=1 C τ f, (43) 1559

5 156

6 1561