Second Order Sufficient Conditions for Optimal Control Problems with Non-unique Minimizers

Transcription

1 2 American Control Conference Marriott Waterfront, Baltimore, MD, USA June 3-July 2, 2 WeA22. Second Order Sufficient Conditions for Optimal Control Problems with Non-unique Minimizers Christos Gavriel and Richard Vinter FIEEE Abstract This paper concerns second order sufficient conditions of optimality, for optimal control problems whose minimizers are not locally unique, i.e. there are distinct admissible controls arbitrarily close to the nominal optimal control which also achieve the minimum cost. Problems of this nature naturally arise in periodic control, shape optimization and other areas. Here, there exist transformations of the feasible control functions, such as a time translation of a periodic control function, that preserve feasibility and the value of the cost. Standard sufficient conditions which, when they apply, give the information not only that a putative minimizer is locally optimal, but also that it is locally unique, can never be directly applied in such circumstances. We provide a framework for deriving sets of sufficient conditions which, unlike the classical ones, cover problems for which optimal controls are not locally unique. We illustrate the application of the new framework with a numerical example. I. INTRODUCTION Computational solvers for optimal control problems are typically based on necessary conditions of optimality. While in most cases it is reasonable to expect that such solvers will provide local minimizers, this is by no means guaranteed. There is, therefore, interest in numerical tests that can be applied to numerically obtained processes to assess whether they are truly locally optimal. A feature of standard tests ( sufficient conditions ) for local optimality is that they cover only situations where the minimizers are locally unique. This is a defect since for a significant class of problems, uniqueness of a solution cannot be taken for granted and the standard sufficient conditions cannot be satisfied. A case in point is optimal periodic control. Here, a small time translation of a given, non-constant, optimal periodic control gives another, distinct, optimal periodic control which is arbitrarily close to the original one. We present a new general framework for studying optimal control problems with minimizers that are not locally unique. It is based on the assumption that the optimal control under consideration can be embedded in a smooth family of optimal controls, each member of which satisfies first order conditions of optimality. An example illustrates how this framework can be employed to generate numerical tests for local optimality, beyond the scope of the traditional theory. There is an extensive literature on sufficient conditions for local optimality of controls, with potential application C. Gavriel is with the Department of Electrical and Electronic Engineering, Imperial College, London, SW7 2BT, christos.gavriel@imperial.ac.uk R. Vinter is with the Department of Electrical and Electronic Engineering, Imperial College, London, SW7 2BT, r.vinter@imperial.ac.uk in situations where minimizers are locally unique. See, 8, 4 and further references therein. Among contributors who have sought numerical tests of local optimality based on sufficient conditions we mention 2 and 6. By contrast, developments concerning sufficient conditions for problems with non-unique minimizers are sparse. One exception is the work of Speyer and his collaborators 3, 7 who have provided sufficient conditions for an important special case of such problems, periodic optimal control. Consider the optimal control problem, min T L(x, u)dt over x W, (, T; R n ) and (P) measurable functions u :, T R m satisfying, ẋ f(x, u) a.e. t, T m(x(), x(t)). The data for (P) comprise a number T > and the functions L : R n R m R, f : R n R m R n, m : R n R n R p, and x :, T R n, u :, T R m. W, (, T; R n ) denotes the space of absolutely continuous R n valued functions on, T. A pair (x( ), u( )) comprising a state trajectory x( ) and a control function u( ) is referred to as a process. The state trajectory is a solution to the differential equation ẋ f(x, u) with a given control function and initial condition. If x( ) satisfies the boundary constraints m(x(), x(t)) then the process (x,u) is called admissible. A process ( x( ),ū( )) that achieves the infimum cost for (P) over all admissible processes is called a minimizer for (P). The control component is called an optimal control. Necessary conditions of optimality for (P) supply candidates for optimal controls, either via analytic or numerical solution. The Maximum Principle (for a recent version of which see, e.g. 5) is the best known condition of this nature. Admissible processes that satisfy the conditions of the Maximum Principle are called extremal processes. These conditions, which are expressed in terms of the the Hamiltonian function, H(x,p,u) : p T f(x,u) + L(x,u) () are as follows: there exists a function p( ) W, (, T; R n ) and λ R p such that p fx T ( x,ū)p+l x ( x,ū) H u ( x,p,u) uū p(t) m T x(t) ( x(), x(t))λ p() m T x() ( x(), x(t))λ //$26. 2 AACC 756

2 Second order sufficient conditions center on the analysis of the accessory problem associated with a given extremal ( x( ),ū( )). Define, T J A (y,v) : y T H xx y + 2y T H xu v 2 +v T H uu vdt, where the subscripts denote partial derivatives of H(,, ), as defined in (), at ( x, p,ū). The accessory problem is then posed as, (A) minj A (y( ), v( )) subject to ẏ Ay + Bv a.e. t, T m i ( x(), x(t)) (y(), y(t)) i,..,p in which A f x ( x,ū) and B f u ( x,ū). A standard test for optimality is to check coercivity of the second variation. Specifically, there exists γ > such that ( J A (y,v) γ y() 2 + T ) v 2 dt for all admissible processes (y,v) of the accessory problem. An illustrative case where the standard sufficient conditions of optimality cannot be satisfied is the following optimal control problem with periodic boundary constraints, (P p ) min T L(x, u)dt subject to ẋ f(x, u) a.e. t, T x() x(t). Take ( x( ),ū( )) to be an admissible process for (P p ), obtained by numerical or analytical solution of optimality conditions. We wish to confirm local optimality of this process. Let us suppose that the control associated with this process is continuous and non-constant. We may obtain a new process by introducing a time delay of τ in both state and control functions, and wrapping around. There results another admissible process ( x( + τ),ū( + τ)). The integral cost of ( x( +τ),ū( +τ)) is the same as that of ( x( ),ū( )), because the integral of a periodic function, over the period, is translationally invariant. Since we have assumed that the control function is continuous, we can arrange that the new control function, obtained by time translation, is arbitrarily close to the original control function by adjusting the size of the time delay τ. Since the control function is assumed nonconstant, the time translation (however small) produces a distinct control function. But the classical sufficient conditions, if satisfied, provide the information that the process under consideration is locally unique. So, in fact, such conditions can never be satisfied. It was shown in 7 that this difficulty can be overcome, by modifying the classical positivity conditions to be satisfied, not over a space associated with the linearized (2) boundary constraints, but over a subspace. We show that this approach extends to a general class of optimal control problems, exhibiting non-uniqueness of optimal controls. An example serves to illustrate how the new conditions can be made the basis of a numerical test for local optimality. II. MAIN RESULTS Consider the following optimal control problem with equality endpoint constraints, minj(x( ), u( )) T L(x, u)dt over pairs (x( ), u( )) that satisfy (P oc ) ẋ f(x, u) a.e. t, T m(x(), x(t)). The data for (P oc ) comprise the functions L(, ), f(, ) which are of class C 2 for each t, T; m(, ) is also of class C 2. Attention focuses on situations in which the optimal process is not locally unique, rather it belongs to an equivalence class of processes, generated by transformations (time shifts, translations etc.) that continue to satisfy the constraints and leave the cost invariant. Such situations are captured by embedding the given process in a family of extremal processes {( x α ( ),ū α ( )) α O}, where O is a ball in k-dimensional space, center the origin, such that each ( x α ( ),ū α ( )) is an extremal with some associated costate p α ( ) and some endpoint constraint multiplier λ α. Hypotheses. (H )α (x α ( ), u α ( )) : O L L is continuous. (H 2 )α (x α (), x α (T)) is of class C. (H 3 )( x α ( ),ū α ( )) is an extremal with multipliers p α ( ) and λ α associated with f and m respectively. Moreover, α p α ( ) : O L is continuous and α λ α is continuous. (H 4 )There exists ǫ > such that 2 uuh( x, p,ū) ǫi for all t, T. (H 5 )(A( ), B( )) (f x ( x( ),ū( )), f u ( x( ),ū( ))) is controllable. Theorem. Take an extremal process ( x,ū). Assume that L(, ), f(, ) and m(, ) are C 2 functions. Assume also that ( x,ū) can be embedded in a family of extremal processes {( x α,ū α ) α O} and accompanying multipliers p α ( ), λ α, such that (H )-(H 5 ) are satisfied, and ( x,ū) ( x,ū ) J( x α,ū α ) J( x,ū) for all α O. Also assume that there exists γ > such that ( T ) J A (y, v) γ y() 2 + v 2 dt 757

3 for all admissible pairs (y,v) of the accessory problem satisfying y T () y T d x (T) α () dα x α. (T) α Then, ( x( ),ū( )) is a local minimizer in the sense that there exists δ > such that J(x( ), u( )) J( x( ),ū( )) for all admissible (x,u) satisfying x x L δ and u ū L δ. The conditions of this theorem are not suitable for numerical testing. Alternative conditions for this purpose are based on consideration of the Hamiltonian system of equations for the accessory problem, namely ( q,ẏ) y,q H A (y, q, v(y, q)) (3) where H A (y, q, v)q (Ay+ bv) + y T v T Hxx H xu y Hxu T H uu v and v(y, q) arg min v {H A (y,q, v)}. The 2n 2n transition matrix of the Hamiltonian system, Φ (t, s) Φ Φ(t, s) 2 (t, s), Φ 2 (t, s) Φ 22 (t, s) with t, s and Φ ij (t, s) an n n matrix, satisfies for each s, T, { d dtφ(t, s) AΦ(t, s) a.e. t, T Φ(s, s) I where A BR A D T BR B Q + DR D T A T + DR B T The general solution (q, y) to this set of equations, satisfies y() Φ (, T) Φ 2 (, T) y(t). q() Φ 2 (, T) Φ 22 (, T) q(t) Define, Q D D T : R Hxx ( x, p,ū) H xu ( x, p,ū) Hxu( x, T p,ū) H uu ( x, p,ū) Now define the 2n 2n matrix, omitting the argument (, T), (see 6) Φ22 Φ W : 2 Φ 2 Φ 22 Φ 2 Φ Φ 2 Φ 2 Φ. Finally, let Z be a matrix with full column rank r, such that y() {(y(), y(t)) m( x(), x(t)) } {Zξ ξ R r }. y(t).. Such a matrix always exists. The following theorem makes reference to the matrix equations: { P +PA+A T P +Q (B T P +D T ) T R (B T P +D T ) P( ) P T (4) ( ) Theorem (Alternative test). Take an extremal process ( x,ū). Assume that L(, ), f(, ) and m(, ) are C 2 functions. Assume also that ( x,ū) can be embedded in a family of extremal processes {( x α,ū α ) α O} and accompanying multipliers p α ( ), λ α, such that (H )-(H 5 ) are satisfied, and ( x,ū) ( x,ū ) J( x α,ū α ) J( x,ū) for all α O. Assume also that (I) detφ 2 (, T) and (II) the Riccati equation (4) has a solution on, T. Suppose that ξ T Z T WZξ > for all non-zero ξ satisfying ξ T Z T d x α () dα x α (T) α. Then, ( x( ),ū( )) is a local minimizer in the sense that there exists δ > such that J(x( ), u( )) J( x( ),ū( )) for all admissible (x,u) satisfying x x L δ and u ū L δ. The conditions of the theorem simplify when considering optimal control problems with periodic boundary conditions. In particular, d dα x α () x α (T) α x() x(t) and Z I I while the last assertion of the theorem reduces to: for all non-zero ξ satisfying ξ T Z T WZξ > ξ T x(). III. EXAMPLE, In this section we illustrate the application of our new conditions, by confirming local optimality of a process obtained by application of a numerical solver to an optimal control problem. We recall that standard sufficient conditions, when applicable, yield local uniqueness as well as local optimality; an exception is the set of conditions in 7, which treats the special class of periodic optimal control problems with possibly non-unique minimizers. The optimal control problem considered here, a variant of Speyer s sailboat problem, has minimizers which are not locally unique. Furthermore, it is 758

4 not periodic, so it is not covered by the sufficient conditions of 7, 3. This is a modified version of a problem considered in 3, to include an integral constraint. Consider, min x2 x x u 2 dt subject to (P M ) x2 ẋ for all t, u x() x() and x dt. A physical interpretation of the problem is that of a sailing boat trying to maximize its average velocity into the direction of a prevailing wind. The state variables x and x 2 are taken to be the lateral position and velocity of the boat respectively. In this scenario the integral constraint represents restrictions on the average lateral position of the boat over its journey. After state augmentation, (P M ) can be written as, min x2 x x u 2 dt subject to (P MA ) ẋ x 2 u for all t, x () x 2 () x x () x 2 () and x 3 (), x 3 (), with x R 3. Problem (P MA ) was passed to a numerical solver IPOPT 9 following discretization of the time variable. This yields state trajectories and control function obtained as shown in Fig. and 2. Control Fig. 2. Control signal Control plot for all t,, p x ṗ p 2 x x 3 2 (5) ū 5p 2 (6) p () p () and p p 2 () p 2 () 3 (), p 3 () free.(7) Plots of the costate variable p, which satisfy numerically the conditions (5)-(7), are shown in Fig. 3. Costate trajectories.2 State trajectories 2 p.8.5 p 2 p 3.6 States.4.2 Costate x x 2.6 x Fig.. State plot Fig. 3. Costate plot A. Necessary Conditions The first step is to confirm that the numerical solution obtained is an extremal by checking the conditions of Pontryagin s Maximum principle in the following special form: B. Second order sufficient conditions Having established that the candidate solution is indeed an extremal, we turn our attention to verifying local optimality of the process ( x,ū) under consideration, using second order sufficient conditions. The accessory problem associated with 759

5 (P MA ) is, minj A (y, v) 2 yt Qy+2y T Dv +Rv 2 dt (P A ) subject to ẏ Ay + bv for all t, y,2 () y,2 () and (y 3 (), y 3 ()) (,), where Q D 2 2 x 2 2 R.2 A b. The Hamiltonian system linked to the accessory problem, with (y, q) R 3 R 3 is, d y A br b T y dt q Q A T. q Computing the transition matrix Φ(,) and taking I 2 2 Z Z 2 I 2 2 (8) we obtain the 2 2 matrix, W Z T WZ (9) The requirement of the extended Riccati condition that ξ T Wξ > for all non-zero ξ is not satisfied since W is not a positive definite matrix. Now apply the new test. We must check the conditions (I) detφ 2 (,) (II) there exists a solution to the Riccati equation (4). For (P A ), the matrix of condition (I) is Φ 2 (,) This has determinant det Φ 2 (,) Furthermore, we found the Riccati equation has a global solution for P() ǫi, ǫ.98. The solution is plotted in Fig. 4 for P().9. It remains to check strict positivity of the W matrix on the required subspace. We must confirm. ξ Wξ > () Solution, P i Solution to Riccati equation Fig. 4. Costate plot for all non-zero ξ satisfying the condition x () x 2 () ξ T Z T x (). x 2 () Noting (8) and x (), x 2 () 4.72, this condition can be expressed as 2 x ξ ξ 2 () 2 x 2 () implying, ξ x () + ξ 2 x 2 () ξ 2 x 2 () and therefore ξ 2. In turn, condition () becomes T ξ ξ W 33.6 ξ 2 > for all non-zero ξ, indicating that the solution obtained numerically is indeed a local minimizer for (P A ). IV. CONCLUSION AND FUTURE WORK A. Conclusion Standard tests for optimality of processes, in the form of second order sufficient conditions, fail when applied to problems with non-unique minimizers. This shortcoming has been addressed in earlier work for periodic optimal control problems. We present new sufficient conditions for local optimality covering a broader class of problems, whose minimizers are possibly non-unique and illustrate their application with reference to a refinement of Speyer s sailboat problem. P P 2 P 3 P 4 P 5 P 6 76

6 B. Future Work Application of the sufficient conditions, presented in this paper, to problems of shape optimization, will be considered in future work. Here, non-uniqueness of minimizers typically arises because the cost is invariant under translations and rotations of the original shape. Generalizations to allow the sub-manifold of constant cost minimizers to be infinite dimensional, too, will be an interesting new departure. Proofs of the conditions reported here will appear in a future paper. REFERENCES M. R. Hestenes, Applications of the Theory of Quadratic Forms in Hilbert Space to the Calculus of Variations, Pacific J. Math,, 95, pp H. Maurer and S. Pickenhahn, Second Order Sufficient Conditions for Optimal Control Problems with Mixed Control-State Constraints, J. Optim. Theory and Applic., 86, 995, pp J. L. Speyer, Periodic Optimal Flight, J. Guidance, Control and Dynamics, 6, pp , G. Stefani and P. Zezza, Constrained Regular LQ-Control Problems, SIAM J. Control and Optim., 35, 997, pp R. B. Vinter. Optimal Control. Birkhäuser, Boston, 2. 6 R. B. Vinter and J. C. Allwright, Second Order Conditions for Periodic Optimal Control Problems, Control and Cybernetics, 34, 25, pp Q. Wang and J. L. Speyer, Necessary and Sufficient Conditions for Local Optimality of a Periodic Process, SIAM J. Control and Optim.,28, 99, pp V. Zeidan, New Second-Order Optimality Conditions for Variational Problems with C 2 -Hamiltonians, SIAM J. Control and Optim., 4, 2, pp A. Wächter, An Interior Point Algorithm for Large-Scale Nonlinear Optimization with Applications in Process Engineering, Ph.D. Thesis, Carnegie Mellon University,