Optimization methods for the verification of second order sufficient conditions for bang bang controls

Transcription

1 OPTIMAL CONTROL APPLICATIONS AND METHODS Optim. Control Appl. Meth. 25; 26:29 56 Published online April 25 in Wiley InterScience ( DOI:.2/oca.756 Optimization methods for the verification of second order sufficient conditions for bang bang controls H. Maurer,n,y,C.Bu skens 2, J.-H. R. Kim and C. Y. Kaya 3 Institut fu r Numerische Mathematik, Westfa lische Wilhelms-Universita t Mu nster, Germany 2 Zentrum fu r Technomathematik, Fachbereich 3, Universita t Bremen, Germany 3 School of Mathematics and Statistics, University of South Australia, Mawson Lakes, SA 595, Australia SUMMARY It has been common practice to find controls satisfying only necessary conditions for optimality, and then to use these controls assuming that they are (locally) optimal. However, sufficient conditions need to be used to ascertain that the control rule is optimal. Second order sufficient conditions (SSC) which have recently been derived by Agrachev, Stefani, and Zezza, and by Maurer and Osmolovskii, are a special form of sufficient conditions which are particularly suited for numerical verification. In this paper we present optimization methods and describe a numerical scheme for finding optimal bang bang controls and verifying SSC. A straightforward transformation of the bang bang arc durations allows one to use standard optimal control software to find the optimal arc durations as well as to check SSC. The proposed computational verification technique is illustrated on three example applications. Copyright # 25 John Wiley & Sons, Ltd. KEY WORDS: bang bang control; second order sufficient conditions; non-linear programming; control of a van der Pol oscillator; control of a batch reactor; robot control. INTRODUCTION In the optimal control design of systems arising from various disciplines, it has been common practice to find controls satisfying Pontryagin s minimum principle, which furnishes first-order necessary conditions for optimality, and then to use these controls assuming that they are (locally) optimal. However to ensure that the control rule is indeed locally optimal, one has to verify certain sufficiency conditions. Second order sufficient optimality conditions (SSC) for optimal control problems, where the control variable appears non-linearly and the strict Legendre condition is satisfied, have been studied extensively in the literature; cf., e.g. References [ 7]. In these control problems, the optimal control usually is continuous. For optimal control problems with control appearing n Correspondence to: H. Maurer, Institut fu r Numerische Mathematik, Einsteinstr. 62, 4849 Mu nster, Germany. y maurer@math.uni-muenster.de Copyright # 25 John Wiley & Sons, Ltd. Received 2 December 24 Revised 2 December 24

2 3 H. MAURER ET AL. linearly, the necessary optimal controls are often characterized by (discontinuous) bang bang type controls. The issue of sufficient optimality conditions for bang bang controls has been addressed in References [8 ] using an extremal field approach. However, the numerical verification of this type of conditions is often tedious and is restricted to control problems of moderate complexity. Recently, second order sufficient conditions (SSC) have been derived in the literature. Based on the abstract theory of higher order conditions in Osmolovskii [], Maurer and Osmolovskii [2, 3] analyse a Lyapunov equation approach for SSC. In these studies, numerical experiments and verification of the SSC presented had to be limited to problems with a scalar control and relatively small number of switchings. Another type of SSC has been given by Agrachev et al. [4]. These authors derive SSC for bang bang controls in terms of SSC for an associated nonlinear programming problem taking the switching points and free initial point as optimization variables. The results in Reference [4] are given for fixed final time. Extensions to the free final time case may be found in Reference [5] on the basis of the theory developed in References [, 3]. In this paper, we consider a fairly general optimal control problem with control appearing linearly. The vector-valued controls have simple lower and upper bounds. The problem is subject to equality constraints for the initial and terminal states. The optimal control components are assumed to be bang bang with a finite number of switchings and known bang bang structure. Moreover, the so-called strict bang bang property is assumed to hold [4, 5]. For convenience of exposition, it is also assumed that only one control component switches its value at a given time. We formulate optimization methods and present a numerical scheme for computing optimal bang bang controls and verifying the SSC in References [4, 5] on problems with larger number of switchings. In the formulation of the optimization method, the durations of the bang bang arcs are used as optimization variables instead of the switching times (or instances). A straightforward transformation of the arc durations results in an optimization scheme [6] that can be handled with the general purpose optimal control software NUDOCCCS [7]. This approach yields the optimal arc durations as well as the Hessian of the Lagrangian and the Jacobian of the equality constraints, which can then be used to check the SSC presented in References [4, 5]. The paper is organized as follows. In Section 2, we give preliminaries, notation and assumptions. In Section 3, the optimization problem associated with the bang bang optimal control problem is formulated and SSC are stated. In Section 4, computational methods for finding optimal bang bang controls are described and the SSC computational verification scheme is given. In Sections 5 7, the computational technique is illustrated on three example applications, one with a scalar control and the other two each with two control components. 2. OPTIMAL CONTROL PROBLEMS WITH CONTROL APPEARING LINEARLY: BANG BANG CONTROLS We consider a general class of optimal control problems with control appearing linearly. Let xðtþ 2R n denote the state variable and uðtþ 2R m the control variable in the time interval t 2 D ¼½; t f Š where the final time t f > is either fixed or free. The following optimal control problem will be denoted by (OC): determine a pair of functions ðxðþ; uðþþ and a final

3 VERIFICATION OF SUFFICIENT CONDITIONS FOR BANG BANG CONTROLS 3 time t f that minimize the cost functional of Mayer type Jðx; u; t f Þ :¼ gðxðþ; xðt f Þ; t f Þ ðþ subject to the constraints on the interval ½; t f Š; xðtþ ¼f ðt; xðtþ; uðtþþ ¼ f ðt; xðtþþ þ F ðt; xðtþþuðtþ ð2þ jðxðþ; xðt f Þ; t f Þ¼ ð3þ uðtþ 2U R m ð4þ For simplicity, the admissible control set U R m is supposed to be the cube U :¼ fu ¼ðu ;...; u m Þ2R m j u min i 4u i 4u max i ; i ¼ ;...; mg ð5þ The techniques developed in this paper also pertain to any polyhedric control set but the exposition would become rather involved and technical. The functions g : R 2nþ! R; f : R R n! R n ; F : R R n! R nm ; and j : R 2nþ! R r ; 4r42n; are assumed to be twice continuously differentiable. A control process T ¼fðxðtÞ; uðtþþ j t 2½; t f Š; t f > g is said to be admissible if xðþ is absolutely continuous, uðþ is measurable and essentially bounded and the pair of functions ðxðtþ; uðtþþ satisfies the constraints (2) (5) on the interval ½; t f Š: The component xðtþ is called the state trajectory. Definition 2. An admissible control process T ¼fðx ðtþ; u ðtþþjt 2½; t f Š; t f > g is said to be a strong (resp., a strict strong) minimum for problem (OC) if there exists e > such that gðxðþ; xðt f Þ; t f Þ5 gðxðþ; xðt f Þ; t f Þ (resp., gðxðþ; xðt f Þ; t f Þ > gðxðþ; xðt f Þ; t f Þ) holds for all admissible control processes T ¼fðxðtÞ; uðtþþ j t 2½; t f Šg (resp., for all admissible control processes T different from T ) with jt f t f j5e and maxfjjxðtþ x ðtþjj; t 2½; t f Š\½; t f Šg5e First order necessary optimality conditions for a strong minimum in problem (OC) are given by Pontryagin s minimum principle. The Pontryagin or Hamiltonian function is defined by Hðt; x; u; lþ :¼ l f ðt; x; uþ ¼l f ðt; xþþlf ðt; xþu ð6þ where l 2 R n is a row vector while x; u; f ; f are column vectors and F ðt; xþ is a ðn mþ-matrix. The factor at u in the Hamiltonian is called the switching vector sðt; x; lþ :¼ ðs ðt; x; lþ;...; s m ðt; x; lþþ :¼ lf ðt; xþ 2R m ð7þ

4 32 H. MAURER ET AL. Furthermore, let us introduce the Lagrange function for the initial and terminal point, lðx ; x f ; t f ; a; rþ :¼ agðx ; x f ; t f Þþrjðx ; x f ; t f Þ x ; x f 2 R n ; a 2 R; r 2 R r ðrow vectorþ ð8þ In the sequel, partial derivatives of functions will be denoted by subscripts referring to the respective variables. If T ¼fðx ðtþ; u ðtþþ j t 2½; t f Š; t f > g provides a strong minimum for (OC) then there exist a pair of functions ðl ðþ; l ðþþ=; a 5; and a multiplier r 2 R r that satisfy the following conditions for a.e. t 2½; t f Š: l ðtþ ¼ H x ðt; x ðtþ; u ðtþ; l ðtþþ ð9þ l ðtþ ¼ H tðt; x ðtþ; u ðtþ; l ðtþþ ðþ l ðþ ¼ l x ðx ðþ; x ðt f Þ; t f ; a ; r Þ ðþ l ðt f Þ¼l x f ðx ðþ; x ðt f Þ; t f ; a ; r Þ ð2þ l ðt f Þ¼l t f ðx ðþ; x ðt f Þ; t f ; a ; r Þ ð3þ Hðt; x ðtþ; u ðtþ; l ðtþþ þ l ðtþ ¼ ð4þ Hðt; x ðtþ; u ðtþ; l ðtþþ ¼ minfhðt; x ðtþ; u; l ðtþþ j u 2 Ug ð5þ Henceforth, we shall use the notations f ðtþ ¼f ðt; xðtþ; uðtþþ; sðtþ ¼sðt; xðtþ; lðtþþ ¼ ðs ðtþ;...; s m ðtþþ etc. Evaluating the minimum condition (5) for the admissible control set U in (5), we get the following control law for the ith control component: 8 u min i if s i ðtþ > >< u i ðtþ ¼ u max i if s i ðtþ5 ð6þ >: undetermined if s i ðtþ ¼ Bang bang controls are characterized by the fact that any switching function s i ðtþ has only isolated zeroes that are the candidates for switching points of the control. To obtain SSC we require the stronger conditions that there are only finitely many zeroes of any switching function; cf. References [3 5, 8].

5 VERIFICATION OF SUFFICIENT CONDITIONS FOR BANG BANG CONTROLS 33 Assumption 2. Let ðx ; u Þ be an admissible pair. Assume that S :¼ft ; t 2 ;...; t s g is a finite set of switching points such that (a) ¼: t 5t 5 5t k 5 5t s 5t sþ :¼ t f ; (b) the control u ðþ is continuous on ½; t f Š\S and is piecewise constant with values of the ith component u i ðtþ 2fumin i ; u max i g; i ¼ ;...; m; (c) for every k ¼ ;...; s there exists a uniquely determined index i ¼ iðkþ such that only the control component u iðkþ is discontinuous at t k : Thus S represents the set of possible switching points for all control components u i ðtþ: Part (c) requires that exactly one control component can switch at a switching point. This assumption can be relaxed [4, 8] but results in a more complicated technical exposition of the material from which we refrain here. By this assumption, there exist vectors u k 2 R m such that the control is given by u ðtþ u k for t k 5t5t k ; k ¼ ;...; s þ ð7þ with ith component u k i 2fu min i ; u max i g; i ¼ ;...; m: The jump of the control u at the point t k is denoted by ½u Š k ¼ u kþ u k ; k ¼ ;...; s ð8þ We shall assume that the necessary conditions (9) (5) are satisfied in normal form with a multiplier a ¼ and multipliers l ðtþ; l ðtþ; r : Let sðtþ be the switching vector function defined in (7). We shall need a further assumption that will be referred to as the strict bang bang Legendre condition or strict bang bang property; cf. References [3 5, 8]. Assumption 2.2 For every k ¼ ;...; s; let i ¼ iðkþ be the uniquely determined index defined in Assumption 2.(c) such that s iðkþ ðt kþ¼: Then the following strict inequality holds: D k ðhþ :¼ sðt k Þ½u Š k ¼ s iðkþ ðt k Þ½u iðkþ Šk > ; k ¼ ;...; s ð9þ The quantity D k ðhþ defined in (9) always satisfies the inequality D k ðhþ5 in view of the minimum condition (5). Note that the derivative s iðkþ exists at t k since it involves only control components u j ðtþ for j=iðkþ that are assumed to be continuous at the switching point t k : 3. SECOND ORDER SUFFICIENT CONDITIONS FOR BANG BANG CONTROLS VIA OPTIMIZATION METHODS In Reference [4], second order sufficient conditions (SSC) for the optimality of the control (7) have been given in terms of SSC of an associated optimization problem. The results are derived for control problems with fixed final time. Extensions to control problems with free final time and an explicit representation of the quadratic form in SSC have recently been given in Reference [5]. In this paper, the SSC in References [, 2] are related to those in Reference [4].

6 34 H. MAURER ET AL. We are going to formulate a finite-dimensional optimization problem involving the optimization vector z :¼ ðx ; t ; t 2 ;...; t s ; t f Þ2R n R s R ¼ R n z ; n z ¼ n þ s þ ð2þ where ¼: t 5t 5 5t k 5t s 5t sþ :¼ t f : The vector z is considered as a column vector. It is chosen in a neighborhood of the vector z ¼ðx ðþ; t ;...; t s ; t f Þ: Of course, if the initial state xðþ ¼x is fixed, resp., if the final time t f is fixed, the vector x ; resp., the component t f ; is deleted from the optimization vector z: For any vector z in (2) we denote by uðt; zþ the bang bang control with uðt; zþ u k for t k 5t5t k ; k ¼ ;...; s þ ð2þ where u k are the values of the control (7). Consider now the absolutely continuous solution of the state equation associated with the bang bang control uðt; zþ; xðtþ ¼f ðt; xðtþ; u k Þ for t k 5t5t k ðk ¼ ;...; s þ Þ; xðþ ¼x ð22þ where the initial condition xðþ ¼x is given by the component x of z: Denote this solution for convenience by xðt; zþ ¼xðt; x ; t ;...; t s Þ though, strictly speaking, this solution does not depend on the component t f of the vector z: It follows from elementary properties of ODEs that the ðt; z k ðt; z Þ; k ¼ ;...; s þ ð23þ exist and can be expressed via the fundamental transition matrix Yðt; tþ of the linearized state equation. Let the ðn nþ-matrix Yðt; tþ with 4t4t4t f be the solution to the matrix differential equation d dt Yðt; tþ ¼f x ðtþyðt; tþ; Yðt; tþ ¼I n ðunity matrixþ ð24þ Then the partial derivatives in (23) have the following ðt; z f ðt f ; z Þ¼ x ðt f k ðt; z Þ¼Yðt; t k Þð x ðt k Þ x ðt k þþþ for t5t k ð26þ x ðt k Þ x ðt k þþ ¼ f ðt k Þ f ðt k þþ ¼ F ðt k Þðuk u kþ Þ

7 VERIFICATION OF SUFFICIENT CONDITIONS FOR BANG BANG CONTROLS 35 The existence of these derivatives follows from elementary results on the dependence of differential equations on parameters. Since readers might not be familiar with formula (26), we shall sketch a derivation here. It follows from the piecewisely defined equation (22) that the following integral equation holds: Z t xðt; zþ ¼xðt k ; zþþ f ðt; xðt; zþ; uðt; zþþ dt for t5t k t k Differentiating this identity w.r.t. t k we obtain t ðt; z Þ¼ x ðt Þ x ðt k þþ þ f x ðt; xðt; z Þ; uðt; z ðt; z Þ dt k t k which implies (26) in view of the definition of the fundamental matrix in (24). Using the solution xðt; zþ of the state equation we arrive at the following optimization problem (OP) w.r.t. the optimization variable z defined in (2): Minimize GðzÞ :¼ gðx ; xðt f ; zþ; t f Þ; z 2 R n z ð27þ subject to FðzÞ :¼ jðx ; xðt f ; zþ; t f Þ¼ ð28þ The functions G and F are twice continuously differentiable in view of (2). The Lagrange function for the optimization problem (OP) is then given by Lðz; pþ :¼ GðzÞþrFðzÞ; r 2 R r ðrow vectorþ ð29þ Any nominal vector z ¼ðx ðþ; t ;...; t s ; t f Þ; for which the necessary conditions (9) (5) of the minimum principle are satisfied with a multiplier r 2 R r ; is a stationary point of the Lagrangian. Let us sketch a short proof. The solution of the adjoint equation (9), l ðtþ ¼ l ðtþfx ðtþ; with terminal value (2), has the explicit representation l ðtþ ¼l ðt f ÞYðt f ; tþ; 4t4t f This implies L tk ðz ; r Þ¼G tk ðz Þþr F tk ðz Þ¼l xf ðx ðþ; x ðt f Þ; t f ; r ðt ; z Þ k ¼ l ðt f ÞYðt f ; t k Þð x ðt k Þ x ðt k þþþ ¼ l ðt k ÞF ðt k Þðuk u kþ Þ¼s iðkþ ðt k Þðuk iðkþ ¼ ukþ iðkþ Þ Here s iðkþ is the component of the switching vector (7) according to Assumption 2.(c) which satisfies s iðkþ ðt k Þ¼: The optimality condition for the optimal initial state x ðþ ¼x holds

8 36 H. MAURER ET AL. in view of L x ðz ; r Þ¼G x ðz Þþr F x ðz Þ¼l ðt f ÞYðt f ; Þþl x ðx ðþ; x ðt f Þ; t f ; r Þ Analogously, one verifies the condition ¼ l ðþþl x ðx ðþ; x ðt f Þ; t f ; r Þ¼ L tf ðz ; r Þ¼Hðt f Þþl ðt f Þ¼ Hence, in summary we have shown that L z ðz ; r Þ¼ 2 R nþsþ holds. Proceeding now to second order conditions, the following SSC have been proved in Reference [4] for problems with fixed final time t f : The extension to control problems with free final time follows from the results in References [3, 5]. Theorem 3.(SSC for the control problem (OC)) Let ðx ; u Þ be an admissible pair for the control problem (OC) with a bang bang control u that satisfies Assumptions 2. and 2.2. Assume that the vector z ¼ðx ðþ; t ;...; t s ; t f Þ and the multiplier r 2 R r are such that the SSC for the optimization problem (OP) in (27), (28) hold, i.e. L z ðz ; r Þ¼G z ðz Þþr F z ðz Þ¼ ð3þ rank F z ðz Þ¼r ð3þ hl zz ðz ; r Þv; vi > 8v 2 R nþsþ ; v=; F z ðz Þv ¼ ð32þ Then the pair ðx ; u Þ provides a strict strong minimum for the control problem (OC). Moreover there exist c > and e > such that the estimate GðzÞ Gðz Þ¼gðx ; xðt f ; zþ; t f Þ gðx ðþ; x ðt f Þ; t f Þ 5 c jjz z jj 2 2 ¼ c jjx x ðþjj 2 2 þ Xsþ k¼ ðt k t k Þ2! holds for all jjz z jj 2 4e with FðzÞ ¼: 4. COMPUTATIONAL METHODS FOR BANG BANG CONTROLS AND THE VERIFICATION OF SECOND ORDER SUFFICIENT CONDITIONS The form of SSC given in Theorem 3. is not yet convenient for numerical verification. Instead of directly optimizing the switching points t k ; k ¼ ;...; s; one determines the arc durations x k :¼ t k t k ; k ¼ ;...; s; s þ ð33þ

9 VERIFICATION OF SUFFICIENT CONDITIONS FOR BANG BANG CONTROLS 37 of the bang bang arcs. This approach has been used previously in References [6, 9, 2]. Thus we replace the optimization variable z ¼ðx ; t ;...; t s ; t sþ Þ; t sþ :¼ t f ; in (2) by the optimization variable *z :¼ ðx ; x ;...; x s ; x sþ Þ2R n R sþ ; x k :¼ t k t k ð34þ The variables z and *z are related by the following linear transformation involving the regular ðn þ s þ ÞðnþsþÞ-matrix R; *z ¼ Rz; R ¼ I! n ; z ¼ R *z; R ¼ I! n ð35þ S S S ¼ ; S ¼ C.. A... C A... Denoting the solution to the ODE (22) by xðt; *zþ; the optimization problem (27) and (28) is obviously equivalent to the following optimization problem ðopþ: Minimize *Gð*zÞ :¼ gðx ; xðt f ; *zþ; t f Þ; t f ¼ Psþ subject to Feð*zÞ :¼ jðx ; xðt f ; *zþ; t f Þ¼ x k k¼ ð36þ The Lagrangian for this problem is given in normal form by Leð*z; rþ ¼ *Gð*zÞþr Feð*zÞ ð37þ with the same multiplier r as in the Lagrangian (42). It is clear that the SSC (3) (32) in the variable z are satisfied if and only if the corresponding SSC for the problem ðopþ hold. This immediately follows from the fact that the Jacobian and Hessian for both optimization problems are related through F z ðzþ ¼ Fe *z ð*zþr; L z ðz; rþ ¼ Le *z ð*z; rþr; L zz ðz; rþ ¼R n Le *z*z ð*z; rþr ð38þ Thus we can express the positive definiteness condition (32) evaluated for the variable *z as follows, h Le *z*z ð*z ; r Þv; vi > 8v 2 R nþsþ ; v=; Fe *z ð*z Þv ¼ ð39þ This condition is equivalent to the condition that the so-called reduced Hessian is positive definite; cf. References [7, 2 22]. Let N be the ðn z ðn z rþþ-matrix, n z ¼ n þ s þ ; with full column rank n z r whose columns span the kernel of Fe *z : Then condition (39) is

10 38 H. MAURER ET AL. reformulated as N n Le *z*z ð*z ; r ÞN > ðpositive definiteþ ð4þ Hence, the task in the following numerical examples will be to check either (39) or (4). The computational method for determining the optimal vector *z 2 R nþsþ is based on a multiprocess approach; cf., e.g. Reference [23]. The time interval ½t k ; t k Š of the kth bang bang arc is mapped to the fixed interval ½; Š by the transformation t ¼ t k þ t x k ; t 2½; Š; x k ¼ t k t k ; k ¼ ;...; s þ ð4þ The state variables are viewed as functions of the new time variable t 2½; Š according to x ðkþ ðtþ :¼ xðt k þ t x k Þ; t 2½; Š ð42þ This family of individual arc functions is a solution to the ODE system x ðkþ ðtþ ¼ dxðkþ dt ¼ x k f ðt k þ t x k ; x ðkþ ðtþ; u k Þ; k ¼ ;...; s þ ð43þ which is concatenated by the continuity condition x ðkþ ðþ ¼x ðkþþ ðþ; k ¼ ;...; s ð44þ Denoting the dependence of solutions on the vector *z by x ðkþ ðt; *zþ and substituting t f ¼ P sþ k¼ x k; the optimization problem (36) takes the form min *z f *Gð*zÞ :¼ gðx ; x ðsþþ ð; *zþ; t f Þj Feð*zÞ :¼ jðx ; x ðsþþ ð; *zþ; t f Þ¼g ð45þ Any optimization method for this problem can be viewed as a shooting procedure that requires a good initial guess not only for the arc durations x k ; k ¼ ;...; s þ ; but also for the values x ðkþ ðþ ¼xðt k Þ; k ¼ ;...; s; at the switching points. Since such initial guesses are often difficult to obtain, we shall employ a slightly different method that has been proposed in Reference [6]. The time interval ½t k ; t k Š is mapped to the fixed interval ½ðk Þ=ðs þ Þ; k=ðs þ ÞŠ by the linear transformation t ¼ a k þ b k t; a k ¼ t k kx k ; b k ¼ðsþÞx k ; t 2 k s þ ; k ð46þ s þ Identifying xðtþ ffixða k þ b k tþ¼xðtþ in the relevant intervals, we obtain the ODE system xðtþ ¼ðs þ Þx k f ðt k þ t x k ; xðtþ; u k Þ for t 2 k s þ ; k ð47þ s þ Thus the solutions on the intervals ½ðk Þ=ðs þ Þ; k=ðs þ ÞŠ are concatenated in such a way that they yield the continuous solution xðtþ ¼xðt; zþ in the unit interval ½; Š: Instead of the

11 VERIFICATION OF SUFFICIENT CONDITIONS FOR BANG BANG CONTROLS 39 optimization problem (45) we solve the problem min f *Gð*zÞ :¼ gðx ; xð; *zþ; t f Þj Feð*zÞ :¼ jðx ; xð; *zþ; t f Þ¼g; t f ¼ Xsþ x k *z k¼ ð48þ This approach can be conveniently implemented using the routine NUDOCCCS of Bu skens [7]. We can then take advantage of the fact that this routine also provides the Jacobian of the equality constraints and the Hessian of the Lagrangian which are needed in the check of the second order condition (39), resp., the positive definiteness of the reduced Hessian in (4). To conclude this section we shall briefly comment on the connections between the optimization approach presented here and the so-called shooting method for computing optimal bang bang controls. The shooting method solves a multipoint boundary value problem (MBVP) that takes into account the state and adjoint equations, the boundary conditions for state and adjoint variables and the switching conditions; cf. Reference [24]. The MBVP can be reduced to a system of 2n þ s þ þ r equations for the unknown shooting vector ðxðþ; lðþ; t ;...; t s ; t f ; rþ 2R 2nþsþþr : To apply Newton s method, one needs the property that the Jacobian matrix of this system of equations is regular. Based on arguments in Section 3, it can easily been seen that this Jacobian reduces to the Kuhn Tucker matrix! L zz ðz ; r Þ F z ðz Þ n F z ðz Þ From well-known arguments in optimization theory [2] we then get the following Corollary to Theorem 3.. Corollary 4. The Jacobian of the shooting method is regular provided that the SSC in (3) (32) hold. For convenience, in the following three numerical examples we shall drop the upper index zero denoting the optimal control and trajectory. 5. TIME-OPTIMAL CONTROL IN THE RAYLEIGH PROBLEM: SCALAR CONTROL SSC for the time-optimal control in the Rayleigh problem have recently been checked in Reference [2] using a second order test based on an abstract quadratic form in a Hilbert space. Here, we treat the same example and check SSC using the optimization approach in Theorem 3.. The control problem is to minimize the final time t f subject to x ðtþ ¼x 2 ðtþ; x 2 ðtþ ¼ x ðtþþx 2 ðtþð:4 :4x 2 ðtþ 2 ÞþuðtÞ ð49þ x ðþ ¼x 2 ðþ ¼ 5; x ðt f Þ¼x 2 ðt f Þ¼ ð5þ juðtþj44 for t 2½; t f Š ð5þ

12 4 H. MAURER ET AL Figure. Rayleigh problem: state x 2 ðtþ and scaled switching function sðtþ; dashed line: control uðtþ: The Hamilton Pontryagin function (6) for this problem is Hðx; u; lþ ¼l x 2 þ l 2 ð x þ x 2 ð:4 :4x 2 2 ÞþuÞ ð52þ The adjoint equations (9) are and the switching function is given by l ¼ l 2 ; l2 ¼ l þ l 2 ð:4 :42x 2 2 Þ ð53þ sðtþ ¼l 2 ðtþ ð54þ The conditions (), (4) yield the transversality condition l 2 ðt f Þuðt f Þþ ¼ ð55þ The switching function sðtþ ¼l 2 ðtþ determines the optimal control via the minimum condition (6), ( uðtþ ¼ 4 if l ) 2ðtÞ5 ð56þ 4 if l 2 ðtþ > It is easy to show that the singular case, where l 2 ðtþ in a time interval ½t ; t 2 Š; can be eliminated. In Reference [2] it was found that the optimal control is composed of three bang bang arcs, for 4t4t >< >= uðtþ ¼ 4 for t 4t4t 2 ð57þ >: >; 4 for t 2 4t4t f

13 VERIFICATION OF SUFFICIENT CONDITIONS FOR BANG BANG CONTROLS 4 This control structure yields the two switching conditions The optimization vector (34) thus becomes l 2 ðt Þ¼; l 2 ðt 2 Þ¼ ð58þ *z ¼ðx ; x 2 ; x 3 Þ; x ¼ t ; x 2 ¼ t 2 t ; x 3 ¼ t f t 2 Using the techniques described in Section 4, the code NUDOCCCS yields the following numerical results for the arc durations, switching times and adjoint variables (Figure ): x ¼ t ¼ :25; x 2 ¼ 2:8954; t 2 ¼ 3:35 x 3 ¼ :3583; t f ¼ 3:6687 l ðþ ¼ :2234; l 2 ðþ ¼ :82652 l ðt Þ¼ :2522; l ðt 2 Þ¼:89992 l ðt f Þ¼:842762; l 2 ðt f Þ¼ :25 ð59þ Assumptions 2. and 2.2 are satisfied in view of the control structure (57) and the relations sðt k Þ¼ l 2 ðt k Þ¼l ðt k Þ; k ¼ ; 2; as well as ½uŠ ¼ 8; ½uŠ 2 ¼ 8; which yield D ðhþ ¼ 8 sðt Þ¼8 :2522 > ; D 2 ðhþ ¼8 sðt 2 Þ¼8 :89992 > For the terminal conditions (5), the Jacobian in the optimization problem is the following ð2 3Þ-matrix:! 4:5376 3:44748 : Fe *z ð*zþ ¼ : : : which is of rank 2. The Hessian of the Lagrangian is found to be the ð3 3Þ-matrix :3735 8: : Le *z*z ð*z ; r Þ¼B 8: : :66867 A 6: :66867 :9743 Note that this Hessian is not positive definite. However, the projected Hessian (4) is computed as the positive number N n Le *z*z ð*z ; r ÞN ¼ :555787: Hence, the second order test (39) holds. We have thus verified the SSC in Theorem 3. which shows that the solution data (59) provide a local minimum w.r.t. the L -norm. Now we consider the modified control problem where the two terminal conditions x ðt f Þ¼x 2 ðt f Þ¼ are substituted by a single terminal condition x ðt f Þ 2 þ x 2 ðt f Þ 2 ¼ :25 ð6þ

14 42 H. MAURER ET AL. The Hamiltonian (52) and the adjoint equations (53) remain the same. In view of the transversality condition (2) we get the terminal condition l i ðt f Þ¼2rx i ðt f Þ ði ¼ ; 2Þ; r 2 R ð6þ It turns out that the control is bang bang with only one switching point t in contrast to (57), Hence, the optimization vector is ( uðtþ ¼ 4 for 4t4t ) 4 for t 4t4t f ð62þ *z ¼ðx ; x 2 Þ; x ¼ t ; x 2 ¼ t f t The code NUDOCCCS then gives the following numerical results for the arc durations and adjoint variables (Figure 2): x ¼ t ¼ :274943; x 2 ¼ : ; t f ¼ 2: l ðþ ¼ :73596; l 2 ðþ ¼ : l ðt Þ¼ :238348; l 2 ðt Þ¼: x ðt f Þ¼:426767; x 2 ðt f Þ¼: l ðt f Þ¼:448259; l 2 ðt f Þ¼: ; r ¼ : ð63þ Assumptions 2. and 2.2 are satisfied in view of the control structure (62) and the relations sðt Þ¼ l 2 ðt Þ¼l ðt Þ; ½uŠ ¼ 8; which gives D ðhþ ¼ 8 sðt Þ¼8 : > For the single terminal condition (6), the Jacobian is the following non-zero row vector: Fe *z ð*zþ ¼ð :97469; :97338Þ while the Hessian of the Lagrangian is the ð2 2Þ-matrix Le *z*z ð*z ; r Þ¼! 28: :3845 9:3845 4:482 which already is positive definite. Hence, the second order test (39) holds. Thus the second order conditions in Theorem 3. hold and the solution characterized by the data (63) provides a local minimum.

15 VERIFICATION OF SUFFICIENT CONDITIONS FOR BANG BANG CONTROLS Figure 2. Rayleigh problem: state x 2 ðtþ and scaled switching function sðtþ for terminal constraint (6), dashed line: control uðtþ: OPTIMAL CONTROL OF A BATCH-REACTION: TWO CONTROL COMPONENTS The following optimal control problem for a batch reactor is taken from References [23, 25]. Consider a chemical reaction A þ B! C and its side reaction B þ C! D which are assumed to be strongly exothermic. Thus, direct mixing of the entire necessary amounts of the reactants must be avoided. The reactant A is charged in the reactor vessel, which is fitted with a cooling jacket to remove the generated heat of the reaction, while the reactant B is added. These reactions result in the product C and the undesired byproduct D: Feed of B Cooling

16 44 H. MAURER ET AL. Let x ¼ðM A ; M B ; M C ; M D ; HÞ 2R 5 denote the vector of state variables, where M i ðtþ [mol] and C i ðtþ ½mol=m 3 Š stand for the molar holdups and the molar concentrations of the components i ¼ A; B; C; D; respectively. HðtÞ ½MJŠ denotes the total energy holdup, T R ðtþ ½KŠ the reactor temperature and VðtÞ ½m 3 Š the volume of liquid in the system. The two-dimensional control vector is given by u ¼ðF B ; QÞ 2R 2 ð64þ where F B ðtþ ½mol=sŠ controls the feed rate of the component B while QðtÞ ½kWŠ controls the cooling load. The objective is to determine a control u that maximizes the molar holdup of the component C: Hence, the performance index is Jðx; uþ ¼gðxðt f ÞÞ ¼ M C ðt f Þ ð65þ which has to be minimized subject to the dynamical equations M A ¼ V r ; M B ¼ F B V ðr þ r 2 Þ M C ¼ V ðr r 2 Þ; M D ¼ V r 2 H ¼ F B h f Q V ðr DH þ r 2 DH 2 Þ with the reaction rates r j ; the corresponding Arrhenius rate constants k j ð j ¼ ; 2Þ: ð66þ of both reactions A þ B! C : r ¼ k C A C B with k ¼ A e E =T R C þ B! D : r 2 ¼ k 2 C B C C with k 2 ¼ A 2 e E 2=T R ð67þ and functions S ¼ X i¼a;b;c;d M i a i ; W ¼ X i¼a;b;c;d M i b i T R ¼ qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi W S þ ðw T ref þ SÞ 2 þ 2 W H C i ¼ M X i M i ; i ¼ A; B; C; D; V ¼ V r i¼a;b;c;d i ð68þ The reference temperature for the enthalpy calculations is given by T ref ¼ 298 ½KŠ and the specific molar enthalpy of the reactor feed stream is h f ¼ 2 ½kJ=molŠ: Initial values are given for all state variables, M A ðþ ¼9; M i ðþ ¼ ði ¼ B; C; DÞ; HðÞ ¼5259:97 ð69þ while there is only one terminal constraint T R ðt f Þ¼3 ð7þ

17 VERIFICATION OF SUFFICIENT CONDITIONS FOR BANG BANG CONTROLS 45 with T R defined in (68). The control vector u ¼ðF B ; QÞ appears linearly in the control system. Bounds for the control components are as follows, 4F B ðtþ4 and 4QðtÞ4 ð7þ The reaction and component data appearing in (66) (68) are given in Table I. Calculations show that for increasing t f the switching structure gets more and more complex. However, the total profit of M C ðt f Þ is nearly constant if t f is greater than a certain value t f 6: For these values one obtains singular controls. We choose the final time t f ¼ 45: We are going to show that the optimal control has the bang bang structure 8 9 ð; Þ for 4t4t >< >= uðtþ ¼ðF B ðtþ; QðtÞÞ ¼ ð; Þ for t 4t4t 2 ð72þ >: >; ð; Þ for t 2 4t4t f with 5t 5t 2 5t f : Obviously, this bang bang control satisfies Assumption 2.. Since the initial point xðþ is specified and the final time t f is fixed, the optimization variable in the optimization problem (45), resp., (48) is *z ¼ðx ; x 2 Þ; x ¼ t ; x 2 ¼ t 2 t Table I. Reaction and component data. Reactions Abbr. j ¼ j ¼ 2 Meaning m 3 A j mol s.8.2 Pre-exponential Arrhenius constants E j ½KŠ 3 p 24 Activation energies kj DH j mol 75 Enthalpies Components Meaning Abbr. i ¼ A i ¼ B i ¼ C i ¼ D r j a i b i mol m 3 kj mol K kj mol K Molar density of pure component i Coefficient of the linear ða i Þ and quadratic ðb i Þ term in the pure component specific enthaply expression

18 46 H. MAURER ET AL. The arc duration of the last bang bang arc is 45 x x 2 : The code NUDOCCCS yields the following arc durations, resp., switching times x ¼ t ¼ 433:698; x 2 ¼ t 2 t ¼ 333:575 t 2 ¼ 767:273; x 3 ¼ 45 t 2 ¼ 682:727 ð73þ We note that for this control the state constraint T R ðtþ452 imposed in Reference [23] does not become active. The adjoint equations are rather complicated and are not given here explicitly. Numerical values for the adjoint functions are also provided by the code NUDOCCCS, e.g. the initial values l MA ðþ ¼ : ; l MB ðþ ¼: l MC ðþ ¼ 2: ; l MD ðþ ¼ : l H ðþ ¼:92489 ð74þ Assumption 2.2 holds since Figure 3 clearly shows that s 2 ðt Þ5; s ðt 2 Þ > For the scalar terminal condition (7), the Jacobian in the optimization problem is Fe *z ð*zþ ¼ð: ; : Þ while the Hessian of the Lagrangian is! : : Le *z*z ð*z ; r Þ¼ : : This matrix is positive definite which implies in particular that the second order test (39) is satisfied. Thus Theorem 3. tells us that the solution (72) (74) provides a local minimum Figure 3. Batch reactor: scaled switching functions s ðtþ and s 2 ðtþ; dashed line: controls u ðtþ ¼F B ðtþ; u 2 ðtþ ¼QðtÞ; resp.

19 VERIFICATION OF SUFFICIENT CONDITIONS FOR BANG BANG CONTROLS 47 Let us consider now a second control problem that consists in maximizing the average gain of the component C in time, maximize M C ðt f Þ t f ð75þ where the final time t f is free. It will be shown that the bang bang control ( uðtþ ¼ðF B ðtþ; QðtÞÞ ¼ ð; Þ for 4t4t ) ð; Þ for t 4t4t f ð76þ with only one switching point 5t 5t f for the control F B ðtþ is optimal. Obviously, this bang bang control satisfies Assumption 2.. Since the initial point xðþ is specified, the optimization variable in the optimization problem (45), resp., (48) is *z ¼ðx ; x 2 Þ; x ¼ t ; x 2 ¼ t f t The code NUDOCCCS yields the following arc durations, resp., switching times x ¼ t ¼ 285:59; x 2 ¼ t f t ¼ 7:399 t f ¼ 456:98 ð77þ Numerical values for the adjoint functions are also provided by the code NUDOCCCS, e.g. the initial values l MA ðþ ¼ : ; l MC ðþ ¼ : ; l H ðþ ¼:298 5 l MB ðþ ¼ :636 3 l MD ðþ ¼: ð78þ Assumption 2.2 is satisfied since we may conclude from Figure 4 that s ðt Þ > The Jacobian for the scalar terminal condition (7) is Fe *z ð*zþ ¼ð: ; 3: Þ while the Hessian of the Lagrangian is the positive definite matrix! : :25766 Le *z*z ð*z ; r Þ¼ 4 :25766 : Hence in particular, the second order test (39) holds. We have thus verified the SSC in Theorem 3. and have shown that the solution data (76) (78) provide a local minimum.

20 48 H. MAURER ET AL Figure 4. Batch reactor: state M C ðtþ and scaled switching function s ðtþ for cost functional (75), dashed line: control u ðtþ ¼F B ðtþ: TIME-OPTIMAL CONTROL OF A TWO-LINK ROBOT: TWO CONTROL COMPONENTS The control of two-link robots has been discussed in many articles; cf., e.g. References [26 29]. In these papers, optimal control policies are determined solely on the basis of first order necessary conditions since SSC were not available. The purpose of this section is to develop a second order test for some of the optimal control models in References [26 29]. First, we discuss the robot model considered in Reference [26]. A side effect of the analysis of SSC will be the fact that the optimal control candidate presented in Reference [26] is not optimal since one can verify that the sign conditions in (6) do not hold. In Figure 5 the robot is schematically represented. The state variables are the angles q and q 2 : Let I and I 2 be the moments of inertia of the upper arm OQ and the lower arm QP w.r.t. the points O and Q; resp. Further, let m 2 be the mass of the lower arm, L ¼jOQj the length of the upper arm, and L ¼jQCj the distance between the second link Q and the center of gravity C of the lower arm. With the abbreviations ð79þ A ¼ I þ m 2 L 2 þ I 2 þ 2m 2 L L cos q 2 ; B ¼ I 2 þ m 2 L L cos q 2 R ¼ u þ m 2 L Lð2 q þ q 2 Þ q 2 sin q 2 ; R 2 ¼ u 2 m 2 L L q 2 sin q 2 D ¼ I 2 ; D ¼ AD B 2 the dynamics of the two-link robot can be described by the ODE system q ¼ o ; o ¼ D ðdr BR 2 Þ q 2 ¼ o 2 ; o 2 ¼ D ðar 2 BR Þ ð8þ

21 VERIFICATION OF SUFFICIENT CONDITIONS FOR BANG BANG CONTROLS 49 x 2 P C q 2 Q q O x Figure 5. Two-link robot. where o and o 2 are the angular velocities. We will use the torques in the two links, u and u 2 ; as control variables to steer the robot from a given initial position to an end position in minimal final time t f : q ðþ ¼; q 2 ðþ ¼; o ðþ ¼; o 2 ðþ ¼ q ðt f Þ¼ :44; q 2 ðt f Þ¼:83; o ðt f Þ¼; o 2 ðt f Þ¼ ð8þ Both control components are bounded: The Hamilton Pontryagin function (6) for this problem is ju ðtþj42; ju 2 ðtþj4; t 2½; t f Š ð82þ H ¼ l o þ l 2 o 2 þ l 3 D ðdr ðu Þ BR 2 ðu 2 ÞÞ þ l 4 D ðar 2ðu 2 Þ BR ðu ÞÞ ð83þ The switching functions are given by s ðtþ ¼H u ðtþ ¼ l 3 D D l 4 D B; s 2ðtÞ ¼H u2 ðtþ ¼ l 4 D A l 3 D B Go llmann [] has found that the following bang bang control structure: 8 9 ð 2; Þ for 4t4t >< ð2; Þ for t 4t4t 2 >= uðtþ ¼ðu ðtþ; u 2 ðtþþ ¼ ð2; Þ for t 2 4t4t 3 >: >; ð 2; Þ for t 3 4t4t f ð84þ ð85þ

22 5 H. MAURER ET AL. where 5t 5t 2 5t 3 5t f ; satisfies the minimum principle. This control structure differs substantially from the one in Reference [26] which violates the switching conditions. Obviously, the bang bang control (85) satisfies Assumption 2.. Since the initial point ðq ðþ; q 2 ðþ; o ðþ; o 2 ðþþ is specified the optimization variable in the optimization problem (45), resp., (48) is For the numerical values *z ¼ðx ; x 2 ; x 3 ; x 4 Þ; x ¼ t ; x 2 ¼ t 2 t ; x 3 ¼ t 3 t 2 ; x 4 ¼ t f t 3 L ¼ ; L ¼ :5; m 2 ¼ ; I ¼ I 2 ¼ 3 the code NUDOCCCS yields the following arc durations and switching times: x ¼ t ¼ : ; x 2 ¼ t 2 t ¼ :335882; t 2 ¼ :3673 x 3 ¼ t 3 t 2 ¼ : ; t 3 ¼ 2: x 4 ¼ t f t 3 ¼ :837667; t f ¼ 3:979 ð86þ The adjoint equations are rather complicated and are not given here explicitly. Numerical values for the adjoint functions are also provided by the code NUDOCCCS, e.g. the initial values l ðþ ¼ : ; l 3 ðþ ¼ 2:953665; l 2 ðþ ¼ : l 4 ðþ ¼ : ð87þ Assumption 2.2 is satisfied since Figure 6 clearly shows that s ðt Þ5; s 2 ðt 2 Þ > ; sðt 3 Þ > Figure 6. Two-link robot: scaled switching functions s ðtþ and s 2 ðtþ; dashed line: controls u ðtþ; u 2 ðtþ; resp.

23 VERIFICATION OF SUFFICIENT CONDITIONS FOR BANG BANG CONTROLS 5 For the terminal conditions in (8), we find the Jacobian :75432 :3566 : :7699 : :24722 Fe *z ð*zþ ¼ : : :22727 :78929 C A : : : : This quadratic matrix has full rank det Fe *z ð*zþ ¼: = Thus, the second order test (39) trivially holds and we have shown that the solution (85) (87) provides a local minimum w.r.t. the L -norm. In the model treated above, some parameters like the mass of the upper arm and the mass of a load at the end of the lower arm enter the system s equations implicitly. The mass m of the upper arm is included in the moment of inertia I 2 and the mass M of a load in the point P can be added to the mass m 2 ; where the point C and therefore the length L have to be adjusted. The length L 2 of the lower arm is hidden in L: The second robot model that we are going to discuss is taken from References [27, 29] where every physical parameter enters the system s equation explicitly. The ODE system is as follows: q ¼ o ; o ¼ D ðai 22 BI 2 cos q 2 Þ q 2 ¼ o 2 o ; o 2 ¼ D ðbi AI 2 cos q 2 Þ ð88þ with the abbreviations A ¼ I 2 o 2 2 sin q 2 þ u u 2 ; B ¼ I 2 o 2 sin q 2 þ u 2 D ¼ I I 22 I 2 2 cos2 q 2 ; I ¼ I þðm 2 þ MÞL 2 ð89þ I 2 ¼ m 2 LL þ ML L 2 ; I 22 ¼ I 2 þ I 3 þ ML 2 2 Here I 3 is the moment of inertia of the load with respect to the point P and o 2 is now the angular velocity of the angle q þ q 2 : For simplicity, we set I 3 ¼ : We will again use the torques in the two links, u and u 2 ; as control variables to steer the robot from a given initial position to a non-fixed end position in minimal final time t f : q ðþ ¼; qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ðx ðt f Þ x ðþþ 2 þðx 2 ðt f Þ x 2 ðþþ 2 ¼ r q 2 ðþ ¼; o ðþ ¼; o 2 ðþ ¼; q 2 ðt f Þ¼ o ðt f Þ¼ o 2 ðt f Þ¼ ð9þ

24 52 H. MAURER ET AL. where ðx ðtþ; x 2 ðtþþ are the Cartesian co-ordinates of the point P: x ðtþ ¼L cos q ðtþþl 2 cosðq ðtþþq 2 ðtþþ x 2 ðtþ ¼L sin q ðtþþl 2 sinðq ðtþþq 2 ðtþþ ð9þ The initial point ðx ðþ; x 2 ðþþ ¼ ð2; Þ is fixed. Both control components are bounded, ju ðtþj4; ju 2 ðtþj4; t 2½; t f Š ð92þ The Hamilton Pontryagin function (6) for this problem is H ¼ l o þ l 2 ðo 2 o Þþ l 3 D ðaðu ; u 2 ÞI 22 Bðu 2 ÞI 2 cos q 2 Þ þ l 4 D ðbðu 2ÞI Aðu ; u 2 ÞI 2 cos q 2 Þ ð93þ The switching functions are given by s ðtþ ¼H u ðtþ ¼ D ðl 3I 22 l 4 I 2 cos q 2 Þ s 2 ðtþ ¼H u2 ðtþ ¼ D ðl 3ð I 22 I 2 cos q 2 Þþl 4 ði þ I 2 cos q 2 ÞÞ ð94þ We will show that the optimal control has the bang bang structure 8 ð ; Þ for 4t4t 9 ð ; Þ for t 4t4t 2 >< >= uðtþ ¼ðu ðtþ; u 2 ðtþþ ¼ ð; Þ for t 2 4t4t 3 ð95þ ð; Þ for t 3 4t4t 4 >: >; ð ; Þ for t 4 4t4t f with 5t 5t 2 5t 3 5t 4 5t f : Obviously, this bang bang control satisfies Assumption 2.. Since the initial point ðq ðþ; q 2 ðþ; o ðþ; o 2 ðþþ is specified the optimization variable in the optimization problem (45), resp., (48) is *z ¼ðx ; x 2 ; x 3 ; x 4 ; x 5 Þ; x ¼ t ; x 2 ¼ t 2 t ; x 3 ¼ t 3 t 2 ; x 4 ¼ t 4 t 3 ; x 5 ¼ t f t 4 For the numerical values L ¼ L 2 ¼ ; L ¼ :5; m ¼ m 2 ¼ M ¼ ; I ¼ I 2 ¼ 3 ; r ¼ 3

25 VERIFICATION OF SUFFICIENT CONDITIONS FOR BANG BANG CONTROLS 53 the code NUDOCCCS yields the following arc durations, resp., switching times x ¼ t ¼ :546742; x 2 ¼ :23573; t 2 ¼ : x 3 ¼ :386655; t 3 ¼ 2:798347; x 4 ¼ :96392 t 4 ¼ 3:743862; x 5 ¼ :8523; t f ¼ 3: ð96þ Numerical values for the adjoint functions are also provided by the code NUDOCCCS, e.g. the initial values l ðþ ¼:8472; l 2 ðþ ¼ :25 ð97þ l 3 ðþ ¼:482636; l 4 ðþ ¼: A look at Figure 7 may convince the reader that Assumption 2.2 is satisfied: s 2 ðt Þ > ; s ðt 2 Þ5; s 2 ðt 3 Þ5; s ðt 4 Þ > For the terminal conditions in (9), the Jacobian in the optimization problem is the 4 5-matrix :8575 2:7462 5:88332 :4995 :9928 2:75 :4555 :9476 4:8387 Fe *z ð*zþ ¼ : :3422 2:3545 2: :9355 C A 9:3685 3:3934 : :458 while the Hessian of the Lagrangian is 7:424 9:763 42:3 8:49889 :5826 9:763 2:544 5:329 :769 :49854 Le *z*z ð*z ; r Þ¼ 42:3 5:329 23:9633 5:243 :3864 8:49889 :769 5:243 :49988 :778 C A :5826 :49854 :3864 :778 : Then the projected Hessian (4) is computed as the positive number N n Le *z*z ð*z ; r ÞN : Hence, the second order test (39) holds which implies that the solution (95) (97) provides a local minimum. 8. CONCLUSION AND PERSPECTIVES FOR FUTURE RESEARCH We have formulated optimization methods for computing optimal bang bang controls and verifying the associated SSC. Essentially, the method is a nonlinear programming approach where the switching points t k ; k ¼ ;...; s; the free final time and unknown components of the

26 54 H. MAURER ET AL Figure 7. Two-link robot: scaled switching functions s ðtþ and s 2 ðtþ; dashed line: controls u ðtþ; u 2 ðtþ; resp. initial state are optimized using a special scaling technique. We illustrated the computational technique on three problems, namely time-optimal control of the Rayleigh system, optimal control of a batch reaction, and time-optimal control of a two-link robot. The first problem has a scalar control, the second and third systems have two control components. One crucial assumption in the second and third control problem was Assumption 2.(c) which requires that only one control component switches at a switching time. Recently, we have studied the minimum time control of an underwater vehicle [3] where two control components switch at the same time. Our algorithm was successful also in this case and we could verify SSC for the underlying optimization problem. However, so far it has not yet been clarified if such SSC would imply already SSC for the control problem. The presented computational methods can easily be adapted to the more general class of control systems where the optimal control in each time interval ½t k ; t k Š is given by a feedback function u k ðt; xþ; i.e. where uðtþ ¼u k ðt; xðtþþ holds for t k 4t4t k : This approach covers two important classes of control problems: () bang-singular control problems where the singular control can be computed as a feedback expression, (2) bang bang control problems with first order state constraints. In both classes, the theory of SSC has not yet been completely developed and is under current investigation. Nonetheless, in various examples we have been able to verify the SSC for the associated optimization problems. Further generalizations of the numerical technique in this paper concern so-called switched control systems. Another important aspect of the verification of SSC is the sensitivity analysis of optimal solutions w.r.t. parameter variations in the system; cf. References [, 2, 2, 22, 3]. Some basic ideas for obtaining sensitivity results of optimal bang bang controls have been sketched in Reference [32]. An outline of numerical methods for computing sensitivity derivatives is given in Reference [23]. The general theory of sensitivity analysis for bang bang controls will be the subject of a future paper. For the special class of time-optimal linear control problems, sensitivity results have recently been given in References [8, 33]. Applications of sensitivity analysis in the optimal control of a semiconductor laser may be found in References [34, 35].

27 VERIFICATION OF SUFFICIENT CONDITIONS FOR BANG BANG CONTROLS 55 ACKNOWLEDGEMENTS C. Y. Kaya gratefully acknowledges the support given by the Institute for Numerical and Applied Mathematics at the University of Mu nster for his visit there. REFERENCES. Malanowski K, Maurer H. Sensitivity analysis for parametric control problems with control-state constraints. Computational Optimization and Applications 996; 5: Malanowski K, Maurer H. Sensitivity analysis for state constrained optimal control problems. Discrete and Continuous Dynamical Systems 998; 4: Maurer H, Augustin D. Sensitivity analysis and real-time control of parametric optimal control problems using boundary value methods. In Online Optimization of Large Scale Systems, Grötschel M et al. (eds). Springer: Berlin, 2; Maurer H, Oberle HJ. Second order sufficient conditions for optimal control problems with free final time: the Riccati approach. SIAM Journal on Control and Optimization 22; 4: Maurer H, Pickenhain S. Second order sufficient conditions for optimal control problems with mixed control-state constraints. Journal of Optimization Theory and Applications 995; 86: Milyutin AA, Osmolovskii NP. Calculus of variations and optimal control. Translations of Mathematical Monographs, vol. 8. American Mathematical Society: Providence, RI, Zeidan V. The Riccati equation for optimal control problems with mixed state-control constraints: necessity and sufficiency. SIAM Journal on Control and Optimization 994; 32: Noble J, Schättler H. Sufficient conditions for relative minima of broken extremals in optimal control theory. Journal of Mathematical Analysis and Applications 22; 269: Sussmann HJ. The structure of time-optimal trajectories for single-input systems in the plane: the C nonsingular case. SIAM Journal on Control and Optimization 987; 25: Sussmann HJ. The structure of time-optimal trajectories for single-input systems in the plane: the general real analytic case. SIAM Journal on Control and Optimization 987; 25: Osmolovskii NP. High-order necessary and sufficient conditions for Pontryagin and bounded-strong minima in the optimal control problems. Doklady Akademii Nauk SSSR. Seriya}Cybernetics and Control Theory 988; 33: English transl., Soviet Physics Doklady 988; 33(2): Maurer H, Osmolovskii NP. Second order sufficient conditions for time-optimal bang bang control problems. SIAM Journal on Control and Optimization 24; 42: Maurer H, Osmolovskii NP. Quadratic sufficient optimality conditions for bang bang control problems. Control and Cybernetics 23; 33: Agrachev AA, Stefani G, Zezza PL. Strong optimality for a bang bang trajectory. SIAM Journal on Control and Optimization 22; 4(4): Osmolovskii NP, Maurer H. Equivalence of second order optimality conditions for bang bang control problems. Part : main results. Control and Cybernetics, 25, in press. 6. Kaya CY, Noakes JL. Computational method for time-optimal switching control. Journal of Optimization Theory and Applications 23; 7: Bu skens C. Optimierungsmethoden und Sensitivitätsanalyse fu r optimale Steuerprozesse mit Steuer- und Zustands- Beschränkungen. Dissertation, Institut fu r Numerische Mathematik, Universität Mu nster, Felgenhauer U. On stability of bang bang type controls. SIAM Journal on Control and Optimization 23; 4: Kaya CY, Noakes JL. Computations of time-optimal controls. Optimal Control Applications and Methods 996; 7: Simakov ST, Kaya CY, Lucas SK. Computations for time-optimal bang bang control using a Lagrangian formulation. Preprints of the 5th World Congress of IFAC, Barcelona, Fiacco AV. Introduction to Sensitivity and Stability Analysis in Nonlinear Programming. Mathematics in Science and Engineering, vol. 65. Academic Press: New York, Bu skens C, Maurer H. Sensitivity analysis and real-time optimization of parametric nonlinear programming problems. In Online Optimization of Large Scale Systems, Gro tschel M et al. (eds). Springer: Berlin, 2; Bu skens C, Pesch HJ, Winderl S. Real-time solutions of bang bang and singular optimal control problems. In Online Optimization of Large Scale Systems, Gro tschel M et al. (eds). Springer: Berlin, 2; Oberle HJ, Grimm W. BNDSCO}a program for the numerical solution of optimal control problems. Report 55-89/22, Institute for Flight Systems Dynamics, DLR, Operpfaffenhofen, Germany, Vassiliadis VS, Sargent RW, Pantelides CC. Solution of a class of multistage dynamic optimization problems. 2. Problems with path constraints. Industrial and Engineering Chemistry Research 994; 33: