The Pontryagin Maximum Principle and a Unified Theory of Dynamic Optimization

Transcription

1 The Pontryagin Maximum Principle and a Unified Theory of Dynamic Optimization Francis Clarke February 10, 2009 Abstract The Pontryagin maximum principle is the central result of optimal control theory. In the half-century since its appearance, the underlying theorem has been generalized, strengthened, extended, reproved and interpreted in a variety of ways. We review in this article one of the principal approaches to obtaining the maximum principle in a powerful and unified context, focusing upon recent results that represent the culmination of over thirty years of progress using the methodology of nonsmooth analysis. We illustrate the novel features of this theory, as well as its versatility, by introducing a far-reaching new theorem that bears upon the currently active subject of mixed constraints in optimal control. 1 Introduction In this introductory section we state the classical Pontryagin maximum principle as it applies to a standard fixed-time optimal control problem in Mayer form, the problem which will serve throughout the Professeur, Institut universitaire de France. Affiliation : Université de Lyon; Université Claude Bernard Lyon 1; CNRS, UMR5208, Institut Camille Jordan. Adresse : 43 blvd du 11 novembre 1918, F Villeurbanne, France ; clarke@math.univ-lyon1.fr Plenary talk delivered at the Pontryagin Centennial Conference, June To appear in the Proceedings of the Steklov Institute. 1

2 article for comparison purposes. We also recall a well-known nonsmooth version of the result. Consider the problem (P C ) that consists of minimizing the cost functional l(x(a), x(b)) subject to the boundary or endpoint conditions and the dynamics (x(a), x(b)) E x (t) = f(t, x(t), u(t)) a.e. t [a, b], where the (measurable) control function u( ) is constrained by u(t) U(t) a.e. t [a, b]. Here x(t) is an absolutely continuous function with values in R n, and U(t) is a measurable subset of R m. The pair (x, u) above is referred to as a process. The data of the problem consists of the functions f and l, the set E, and the multifunction U( ). Let us assume for simplicity that the functions in question are continuously differentiable, that E is a classical manifold (or manifold with boundary), and that U has continuous dependence on t. The issue is to give a set of necessary conditions that an optimal (or locally optimal) process (x, u ) must satisfy. We say that an admissible process (x, u ) is a strong local minimum for the problem (P C ) if, for some ɛ > 0, for any process (x, u) admissible for (P C ) that satisfies x x := max t [a,b] x(t) x (t) < ɛ, we have l(x (a), x (b)) l(x(a), x(b)). Following Pontryagin et alii, we introduce the pseudo-hamiltonian function H(t, x, u, p) := p, f(t, x, u). The classical Pontryagin maximum principle asserts: Theorem 1.1 (Pontryagin et al. 1956) If (x, u ) is a strong local minimum for the problem (P C ), and if the optimal control u is essentially bounded, then there exist an absolutely continuous function 2

3 p( ) on [a, b] together with a scalar λ 0 equal to 0 or 1 satisfying the nontriviality condition: λ 0 + p(t) 0, t [a, b], the transversality condition: (p(a), p(b)) λ 0 l(x (a), x (b)) is normal toe at the point(x (a), x (b)), the adjoint equation: for almost every t [a, b], p (t) = D {H(t,, u (t), p(t))} (x (t)) and the maximum condition: for almost every t [a, b], max H(t, x (t), u, p(t)) = H(t, x (t), u (t), p(t)). u U(t) This theorem was the successful conclusion to a long quest. In the words of L.C. Young: 1 The proof of the maximum principle, given in the book of Pontryagin, Boltyanskii, Gamkrelidze and Mischenko... represents, in a sense, the culmination of the efforts of mathematicians, for considerably more than a century, to rectify the Lagrange multiplier rule. A further, very important contribution of this theory was its formulation of the basic problem, with the focus upon the explicit control aspect. This greatly enhanced its appeal for purposes of modeling, and helped make it a vital tool in a wide range of applications. Given the importance of the maximum principle, it is natural to seek extensions of it. There have been very many such advances in the last fifty years; we refer the reader to [9][16][17][18][19][20] [21][22] and to the references therein, but of course the list could be much longer. One of the persistent themes in developing generalizations of the maximum principle has involved reducing the differentiability requirements in the hypotheses, as well as other regularity of the data. The first versions of the theorem for merely Lipschitz data were proven by Clarke in the early 1970 s; there, the adjoint equation is replaced 1 Lectures on the Calculus of Variations and Optimal Control (1969). 3

4 by an inclusion in terms of the generalized gradient introduced by the author. The nonsmooth maximum principle can now be considered a well-known result; we proceed to state it, in essentially its original form (see Clarke [4, 5, 7]). The hypotheses are the following: f(t, x, u) is L B-measurable with respect to t and (x, u), the multifunction U( ) has L B-measurable graph, the set E is closed, and l is locally Lipschitz. We posit also that f is Lipschitz in x, in the following sense: for each t and u U(t) there exists k(t, u) such that f(t, x 2, u) f(t, x 1, u) k(t, u) x 2 x 1 x 1, x 2 B(x (t), ɛ). Theorem 1.2 (Clarke1975) If (x, u ) is a strong local minimum for the problem (P C ), and if k(t, u (t)) is summable, then there exist an absolutely continuous function p( ) on [a, b] together with a scalar λ 0 equal to 0 or 1 satisfying the nontriviality condition: λ 0 + p(t) 0, t [a, b], the transversality condition: (p(a), p(b)) λ 0 L l(x (a), x (b)) + N L E(x (a), x (b)), the adjoint equation: for almost every t [a, b], p (t) C {H(t,, u (t), p(t))} (x (t)) and the maximum condition: for almost every t [a, b], max H(t, x (t), u, p(t)) = H(t, x (t), u (t), p(t)). u U(t) In the theorem statement, C denotes the generalized gradient (with respect to the x variable), L is the limiting subdifferential, and N L S the limiting normal cone to S. We refer to Clarke [9] for a brief summary of these constructs of nonsmooth analysis, or to Clarke et al. [12] for a detailed presentation. Let us merely remark that Theorem 1.2 strictly subsumes the classical result embodied in Theorem 1.1: under the hypotheses of the latter, the generalized normals and differentials reduce to the classical notions, and the conclusions are identical. 4

5 Certain additional conclusions can be added to the maximum principle, notably when the problem is autonomous, or when the underlying interval [a, b] can vary (see Clarke [9] or Vinter [21] for example). Also, the presence of unilateral state constraints can be considered, necessitating the use of measures in the necessary conditions. We do not discuss such issues here. We remark that the early versions of Theorem 1.2 write the transversality condition using C l and NE C rather than the potentially smaller constructs L l and NE L, but (as several authors have noted) the original proof actually yields this minor improvement without any modifications. We shall now describe a longstanding project that seeks to view the maximum principle as one aspect of a unified approach to dynamic optimization. 2 The unified approach There have been several attempts to develop a unified approach to dynamic optimization. The general theories of Dubovitskiĭ and Milyutin [16], and of Neustadt [20], are well-known examples. Another approach was initiated by Clarke in the early 1970 s; it may be called the nonsmooth analysis approach. To explain the motivation behind this work, we need to recall the classical necessary conditions developed for the basic problem in the calculus of variations in the course of its long history; we do so now, somewhat informally. Consider the problem (P B ) of minimizing the so-called Bolza functional l(x(a), x(b)) + b a L(t, x(t), x (t)) dt over a class of smooth functions satisfying the endpoint constraints (x(a), x(b)) E. The first necessary condition is the Euler equation, which asserts that if x is a solution of the problem, then there is an arc p which satisfies (p (t), p(t)) = x,v L(t, x (t), x (t)), t [a, b] a.e. [E] This is actually an important variant of the equation first proposed by Euler in 1744; it is known as the integral form of the Euler equation, and it was discovered by dubois-reymond in the latter half of 5

6 the nineteenth century. It corresponds to the adjoint equation of the maximum principle, as pointed out by Pontryagin et alii when they first published their result. The second basic necessary condition was proved at about the same time; it is known as the Weierstrass condition: max v R n p(t), v L(t, x (t), v) = p(t), x (t) L(t, x (t), x (t)), t [a, b] a.e. [W] This corresponds to the maximum condition in the maximum principle. The third element we require adds the endpoint information to the necessary conditions. Historically this was rarely expressed in very general form, but we can group the various conditions found in the literature exactly as we have expressed them in Theorem 1.1, in a transversality condition: (p(a), p(b)) l(x (a), x (b)) is normal to E at the point (x (a), x (b)). [T] The attempt to develop a unified theory of dynamic optimization can be described (in part) as the project of obtaining the necessary conditions [E][W][T] for the problem of Bolza (P B ) when the data (L, l, E) are not necessarily smooth, and more particularly when L is extended-valued (sometimes equal to + ). The point of having L extended-valued is that one can implicitly represent additional constraints by defining L to be + when they are violated. For example, if the basic problem above is considered under the additional equality constraint h(t, x(t), x (t)) = 0 a.e. (this is classically referred to as a problem of Lagrange), we are led to redefine L(t, x, v) to equal + when h(t, x, v) 0, and to equal its old value L(t, x, v) when h(t, x, v) = 0. Then (and we emphasize that this is completely rigorous) the minimization of the new functional b a L(t, x, x ) dt (with no added constraint) is equivalent to the minimization of the old one under the equality constraint. It turns out that the standard optimal control problem (P C ) discussed previously, but also a number of less standard problems involving mixed equality and inequality constraints, differential inclusions, or generalized control systems, can be brought under the umbrella of the extended-valued approach. Of course, the expression of 6

7 the basic necessary conditions [E][W][T] will have to change to accommodate nonsmooth data. This requires some constructs of nonsmooth analysis. The ultimate goal, then, would be to prove extended necessary conditions for (P B ) in a sufficiently general setting, and under sufficiently nonrestrictive hypotheses, so that not only the classical cases, but also the maximum principle, its principal extensions, and other cases could be subsumed in a satisfactory manner by the unified theory. Ideally, the new unified theory so obtained would in fact give the state of the art for each of its special cases. This project, initiated in 1973 in the author s doctoral thesis [2] (see also [6]) has required several decades of development and the contributions of many people, principal among which have been Clarke, Ioffe, Ledyaev, Loewen, Rockafellar and Vinter. The outcome is described in the succinct, selfcontained monograph [9]. One of the principal features of the theory is its versatility: it applies to the calculus of variations, to differential inclusions, to various types of optimal control problems. Another important feature is the absence of many hypotheses that have encumbered such results in the past (convexity, boundedness, data regularity, constraint qualififcations). A weaker type of Lipschitz condition (called pseudo- Lipschitz) is postulated, and an exceptionally weak type of local minimum is shown to suffice to derive the strongest forms of the necessary conditions. In addition, and these are the two features we shall stress in the examples below, the results are stratified and expressible in a natural geometric form. The first of these features refers to the fact that hypotheses are made only relative to a certain radius function, and the conclusions are then asserted to hold to precisely the same extent. As we shall see, this is especially useful in deriving multiplier rules in the presence of functional constraints, as it eliminates the need to call upon implicit function theorems. The second feature, the geometrical formulation, allows one to state a simple theorem which specializes easily to a variety of contexts. We stress that the interest of the results obtained is not limited to problems with nonsmoothness: we derive a new state of the art even for problems with smooth data. In order briefly to give an idea of the nature of the unified approach, it is convenient to consider first a control problem phrased in terms of a differential inclusion. 7

8 A differential inclusion problem We are given a multifunction F from [a, b] R n to the subsets of R n. It is assumed that F is measurable and graph-closed. A trajectory of F refers to an absolutely continuous function x on [a, b] satisfying x (t) F (t, x(t)) a.e. We consider the problem (P D ) of minimizing l(x(a), x(b)) over the trajectories x of F satisfying the endpoint constraints (x(a), x(b)) E. A measurable function R : [a, b] (0, + ] is called a radius function. Definition 2.1 The trajectory x is a local W 1,1 minimum of radius R for the problem (P D ) if, for some ɛ > 0, we have l(x (a), x (b)) l(x(a), x(b)) for all trajectories x satisfying the endpoint constraints as well as and x x < ɛ, b a x (t) x (t) dt < ɛ, x (t) x (t) R(t), a.e. t [a, b]. Note that when R is identically + (which is allowed), this reduces to what is usually referred to as a local W 1,1 minimum, which is in turn a weaker assumption than that of a strong local minimum (as in Section 1). When R is a finite constant, we obtain a type of minimum that is known in the calculus of variations as a weak local minimum. In the following, G(t) refers to the graph of the multifunction F (t, ), and NG P refers to the cone of proximal normals. When S is a subset of R n and x S, we say that ζ R n is a proximal normal to S at x (written ζ NS P (x)) provided that, for some σ > 0, we have ζ, x x σ x x 2 x S. This fundamental type of normal vector is a building block which generates all the others, and which coincides with the familiar normal vectors if G happens to be smooth or convex. The following is a geometrical version of a property of F that is known as a pseudo-lipschitz condition: Definition 2.2 We say that F satisfies the bounded slope condition of radius R near x if there exists k L 1 (a, b) and ɛ > 0 such that for almost every t, for every (x, v) G(t) with x B(x (t), ε) and v B(ẋ (t), R(t)), for all (α, β) NG(t) P (x, v), one has α k(t) β. 8

9 The following result, taken from [9], turns out to be a powerful and unifying tool for necessary conditions. Theorem 2.3 (Clarke 2005) Suppose that, for some radius function R, the trajectory x is a local W 1,1 minimum of radius R for the problem (P D ), where F satisfies near x the bounded slope condition of radius R, with R(t)/k(t) η a.e. for some η > 0. Then there exist an absolutely continuous function p( ) on [a, b] together with a scalar λ 0 equal to 0 or 1 satisfying the nontriviality condition and the transversality condition: (λ 0, p(t)) 0 t [a, b] (p(a), p(b)) L λ 0 l(x (a), x (b)) + N L E(x (a), x (b)), the Euler adjoint inclusion [E] : { } p (t) co ω : (ω, p(t)) NG(t) L (x (t), ẋ (t)) a.e. t [a, b], as well as the Weierstrass condition [W R ] of radius R: for almost every t we have p(t), v p(t), ẋ (t) v F (t, x (t)) B(ẋ (t), R(t)). If the above holds for a sequence of radius functions R i (with all parameters ɛ, k, η possibly depending on i) for which lim inf i R i(t) = + a.e., then the conclusions hold for an arc p which satisfies the global Weierstrass condition: p(t), v p(t), ẋ (t) v F (t, x (t)), a.e. t [a, b]. The fact that this theorem gives rise to definitive results in such varied contexts as the calculus of variations, standard control problems, and generalized systems, is amply described in [9]. In particular, the goal described above of finding satisfactory extended necessary conditions for the problem of Bolza (P B ) is achieved through this means. In this article, however, we proceed to develop its applications in a different direction: optimal control problems in which the control set depends upon the state x. This is often referred to as the case of mixed constraints. 9

10 3 A general theorem on mixed constraints We consider now the same problem (P C ) as in Section 1, but with one important difference: the control constraint u(t) U(t) is replaced by the mixed state/control constraint (x(t), u(t)) S(t), where S(t) is a closed subset of R n R m for each t. As always, S and f are assumed to be measurable in a suitable sense, while l is taken to be locally Lipschitz and E closed. The main theorem below features hypotheses directly related to a given pair (x, u ) that is admissible for (P C ). Let R : [a, b] (0, + ] be a given radius function, and ɛ > 0. We set S(t, x) := {u R m : (x, u) S(t)} S ɛ,r (t) := {(x, u) S(t) : x x (t) ɛ, u u (t) R(t)}. We say that (x, u ) is a local minimum of radius R for (P C ) provided that for every pair (x, u) admissible for (P C ) which also satisfies (x(t), u(t)) S ɛ,r (t) a.e., b a x (t) x (t) dt ɛ, we have l(x(a), x(b)) l(x (a), x (b)). This resembles a W 1,1 local minimum, which constitutes a weaker hypothesis than the classical strong local minimum. But it is weaker still, because of the additional restriction stemming from the radius function. The main hypotheses of the theorem are conditioned by the radius R; they concern Lipschitz behavior of f(t, x, u) with respect to (x, u) and a certain bounded slope condition bearing upon the sets S(t). [H1] For almost every t [a, b], the function f(t,, ) is locally Lipschitz on a neighborhood of S ɛ,r (t), and there exist measurable functions k x and k u, with k x summable, such that, for almost every t, (x i, u i ) S ɛ,r (t) (i = 1, 2) = f(t, x 1, u 1 ) f(t, x 2, u 2 ) k x (t) x 1 x 2 + k u (t) u 1 u 2. [H2] There exists a measurable function k S such that k S k u is summable and, for almost every t [a, b], the following bounded slope condition holds: (x, u) S ɛ,r (t), (α, β) NS(t) P (x, u) = α k S(t) β. 10

11 The following theorem asserts necessary conditions under optimality and regularity hypotheses which are imposed only for a radius R, and whose conclusions hold to the same extent; this situation is referred to in [9] as stratified. We stress that the theorem allows the case R +. Theorem 3.1 Let the process (x, u ) be a local minimum of radius R for (P C ), where hypotheses [H1][H2] hold, and where, for some positive constant η, we have R(t) ηk S (t) a.e. Then there exist an arc p and a number λ 0 in {0, 1} satisfying the nontriviality condition and the transversality condition: (λ 0, p(t)) 0 t [a, b] (p(a), p(b)) L λ 0 l(x (a), x (b)) + N L E(x (a), x (b)), and such that p satisfies the adjoint inclusion: for almost every t ( p (t), 0) C {H(t,,, p(t))} (x (t), u (t)) N C S(t) (x (t), u (t)), as well as the Weierstrass condition of radius R: for almost every t, u S(t, x (t)), u u (t) R(t) = H(t, x (t), u, p(t)) H(t, x (t), u (t), p(t)). If the hypotheses hold for a sequence of radius functions R i (with all parameters ɛ, k x, k u, k S, η possibly depending on i) for which lim inf i R i(t) = + a.e., then the conclusions above hold for an arc p which satisfies the global Weierstrass condition: for almost every t u S(t, x (t)) = H(t, x (t), u, p(t)) H(t, x (t), u (t), p(t)). We omit the proof of Theorem 3.1 (see [11]), which consists of a direct appeal to Theorem 2.3. We remark that when the velocity set f(t, x (t), S(t, x (t))) is convex, then the Weierstrass condition of any positive radius implies the global one. It is not possible to eliminate the radius R from the theorem by simply replacing S(t, x) by S(t, x) B(u (t), R), since the latter will 11

12 not generally satisfy suitable hypotheses. In other words, the presence of R gives rise to necessary conditions that make intrinsically new assertions. The lower bound on R is needed: an example in [9] (p. 47) shows that the necessary conditions of Theorem 3.1 fail if (other things being equal) R(t)/k S (t) is not bounded away from 0. Note that [H1] requires some Lipschitz behavior with respect to the control u; this was not the case in Theorems 1.1 or 1.2. This is a price that must be paid in order to consider control sets that depend on the state. 4 Special cases In this section we examine several different contexts in which Theorem 3.1 yields a definitive version of the appropriate necessary conditions. We shall stress the ease of application of Theorem 3.1, and how the stratified nature of the theorem gives rise in a natural way to either local or global versions of the resulting conditions. We claim in particular that this represents an important simplification and unification of the issue of mixed constraints. 4.1 Unilateral control constraints Let S(t) = {(x, u) : u U(t)}. Then the problem (P C ) of the previous section coincides with the classical optimal control problem of Section 1, in which the control constraints are unilateral (unmixed). Let (α, β) belong to NS(t) P (x, u). Then, by definition of proximal normal, for some constant σ, the function (x, u ) α, x β, u + σ { x x 2 + u u 2} has a minimum relative to (x, u ) R n U(t) at (x, u ) = (x, u). It follows that α = 0, so that the bounded slope condition of [H2] is automatically satisfied, with k S = 0. If f satisfies [H1], then Theorem 3.1 applies. When f is locally Lipschitz, [H1] is a consequence of the classically familiar hypothesis that u is bounded. With the radius function taken to be R +, the necessary conditions reduce to familiar ones which include the full Weierstrass (or maximum) condition H(t, x (t), u, p(t)) H(t, x (t), u (t), p(t)) u U(t), a.e. t [a, b], 12

13 as well as the Euler form of the adjoint inclusion: for almost every t ( p (t), 0) C {H(t,,, p(t))} (x (t), u (t)) {0} N C U(t) (u (t)). For smooth data, this is precisely Theorem 1.1. For f merely locally Lipschitz, however, this adjoint inclusion, taken jointly in (x, u), is different from the one obtained in the usual nonsmooth maximum principle, Theorem 1.2, and neither implies the other. The relative merits of these two different forms are discussed and illustrated in [14] and [10], to which we refer for further details. We add only that since k S = 0 here, we can allow any (arbitrarily small) positive time-dependent radius function, and still get the Euler adjoint equation, together with a corresponding Weierstrass condition. Note also that the function k u need not be summable. This goes beyond previous results. 4.2 Calculus of variations: the multiplier rule We consider now the following problem of Lagrange in the calculus of variations: minimize b a L(t, x(t), x (t) dt over the arcs x satisfying the following boundary conditions and pointwise constraint: x(a) = A, x(b) = B, h(t, x(t), x (t)) = 0 a.e. t [a, b]. There is a large literature on such problems (see Hestenes [17] and the references therein). In the classical setting, which we adopt here, it is assumed that L : R R n R n R and h : R R n R n R N (with N n) are continuously differentiable, and that there exists a solution x with piecewise-continuous derivative. The goal is to derive necessary conditions in the form of a multiplier rule. (This project was in fact the century-long quest that L.C. Young refers to in the quotation that appears in Section 1). The problem is framed as a special case of (P) as follows: Minimize y(b) subject to x (t) = u(t) a.e. y (t) = L(t, x(t), u(t)) a.e. (x(t), y(t), u(t)) S(t) := {(x, y, u) : h(t, x, u) = 0} a.e. (x(a), y(a), x(b), y(b)) E := {(A, 0, B)} R 13

14 If R is any constant finite radius function, then [H1] holds, for certain constants k x, k u (depending on R). In order to apply Theorem 3.1, we study the bounded slope condition, suppressing the (irrelevant) y variable for ease of notation. We remark that if (α, β) NS(t) P (x, u), and if the matrix [D x h(t, x, u), D u h(t, x, u)] has maximal rank N, then (α, β) belongs to the classical normal space to S(t) at (x, u), whence (α, β) = λ T [D x h(t, x, u), D u h(t, x, u)] for some vector λ R N. This simple but crucial geometric fact is a consequence of the definition of proximal normal, together with the classical Lagrange multiplier rule. It will allow us to confirm the bounded slope condition, and also to interpret the resulting necessary condition through multipliers (rather than normal vectors). A rank hypothesis is commonly made in the literature: that the matrix D u h(t, x, u) is of maximal rank N, either globally in some prescribed region (as in [17]), or else just along the optimal arc (as in [1]). We now show that both scenarios can be handled by Theorem 3.1. Suppose first that we assume only that D u h(t, x (t), x (t)) is of maximal rank for every t. (This is to be interpreted as holding for both x (t+) and x (t ) if x has a corner at t.) We claim (while omitting the simple proof by contradiction) that under this rank condition, there exist constants ɛ > 0 and k S such that t [a, b], x x (t) ɛ, u x (t) ɛ = λ T D x h(t, x, u) k S λ T D u h(t, x, u) λ R N, λ = 1. This allows us to verify [H2] for a suitably small radius R, and to invoke Theorem 3.1. A straightforward inspection of the resulting necessary conditions reveals the existence of λ 0 {0, 1}, λ L (a, b) N and an arc p such that λ 0 + p > 0 and p (t) = D x {λ 0 L + λ(t), h } (t, x (t), x (t)), p(t) = D u {λ 0 L + λ(t), h } (t, x (t), x (t)). We recognize this as the classical Euler equation for the Lagrangian λ 0 L + λ, h, and the desired multiplier rule is obtained. (The abnormal case λ 0 = 0 can only arise when, for some nonzero λ, x is an extremal for λ, h.) 14

15 Theorem 3.1 also yields, for almost every t, h(t, x (t), u) = 0, u x (t) R = p(t), u λ 0 L(t, x (t), u) p(t), x (t) λ 0 L(t, x (t), x (t)), a local Weierstrass condition which is new in this context. We summarize: Corollary 4.1 Under the hypotheses above, with a local rank condition, we obtain the multiplier rule, together with a local Weierstrass condition. The other approach to the multiplier rule is to require that the matrix D u h(t, x, u) be of maximal rank globally, and not just along the optimal arc. In that case, we can apply Theorem 3.1 as above for a sequence of constant radius functions R i increasing to +, and we obtain the multiplier rule accompanied by the global Weierstrass condition, as in [17]: Corollary 4.2 Under the hypotheses above, with a global rank condition, we obtain the multiplier rule, together with a global Weierstrass condition. Thus we are able to easily obtain via Theorem 3.1 the multiplier rule in either its local or global forms. In fact, the theorem can be used to obtain such multiplier rules under considerably weaker regularity hypotheses than the ones we have posited in this example (both on the data and the solution), and for constraints that are not necessarily of equality type; this theme will be developed in forthcoming work. 4.3 Mixed constraints in optimal control We study next the case in which the constraint set S(t) of problem (P C ) is described as follows: S(t) := {(x, u) : g(t, x, u) 0, h(t, x, u) = 0, u U}, which combines unilateral control constraints with mixed ones defined via functional equalities and inequalities. In order to facilitate comparison with the literature, we limit ourselves here to the smooth setting. Thus, g : R R n R m R M and h : R R n R m R N are taken to be continuously differentiable, as is f. 15

16 We are given (x, u ) admissible for the optimal control problem (P C ). We consider first the case in which u is piecewise continuous, as in the classical calculus of variations. In this setting, we impose a constraint qualification hypothesis of a type familiar in optimization, and referred to here as the local Mangasarian-Fromowitz condition [LMFC] : for all t [a, b], γ 0, γ, g(t, x (t), u (t)) = 0, 0 D u { γ, g + λ, h } (t, x (t), u (t)) + N L U (u (t)) = γ = 0, λ = 0. When t is a point of discontinuity of u, this is to be interpreted as holding with both u (t+) and u (t ). The following result follows easily from the definition of proximal normal: Proposition 4.3 There exist positive constants ɛ, k S such that for any t [a, b] and (x, u) S(t) satisfying (x, u) (x (t), u (t)) < ɛ, for any (α, β) in NS(t) P (x, u), there exist γ 0 with γ, g(t, x, u) = 0 and λ such that α k S β and (α, β) D x,u { γ, g + λ, h } (t, x, u) + {0} N L U (u). This proposition confirms the bounded slope condition of [H2], and also interprets the adjoint inclusion given by Theorem 3.1. We phrase the resulting necessary conditions in terms of the function H(t, x, u, p, γ, λ) := p, f(t, x, u) + γ, g(t, x, u) + λ, h(t, x, u). Corollary 4.4 Let (x, u ) be a local solution of constant positive radius R for (P C ), where u is piecewise continuous and [LMFC] holds along (x, u ). Then we obtain the necessary conditions of Theorem 3.1, where, for certain summable functions λ and γ satisfying γ(t) 0, γ(t), g(t, x (t), u (t)) = 0 a.e., the adjoint inclusion is expressible in the form p (t) = D x {H(t,, p(t), u (t), γ(t), λ(t))} (x (t)) a.e. D u {H(t, x (t),, p(t), γ(t), λ(t))} (u (t)) N C U (u (t)) a.e. 16

17 Furthermore, the following local Weierstrass condition holds for some δ > 0: for almost every t, we have u S(t, x (t)), u u (t) δ = H(t, x (t), u, p(t), γ(t), λ(t)) H(t, x (t), u (t), p(t), γ(t), λ(t)). We remark that, as usual, the Weierstrass condition becomes a global one under suitable convexity hypotheses. We now assume that (x, u ) is a solution of (P C ) with u bounded (instead of piecewise continuous). An essential value of u at t refers to a point u having the property that, for any ɛ > 0, the set {τ [a, b] (t ɛ, t + ɛ) : u u (τ) < ɛ} has positive measure. We use the notation ū (t) for the set of essential values of u at t. Note that ū (t) reduces to {u (t)} whenever t is a point of continuity of u, and reduces to {u (t ), u (t+)} if u is piecewise continuous. In light of this observation, we may extend consistently the definition of [LMFC] to bounded controls u as follows: for all t [a, b], u ū (t), γ 0, γ, g(t, x (t), u) = 0, 0 D u { γ, g + λ, h } (t, x (t), u) + N L U (u) = γ = 0, λ = 0. Then the same path yields the following generalization of the previous result: Corollary 4.5 Let (x, u ) be a local solution of constant positive radius R for (P C ), where u is bounded and [LMFC] holds along (x, u ). Then the necessary conditions of Corollary 4.4 hold. We conclude our examples with a global form of the last result above. We now take U to be compact, and we posit the following global Mangasarian-Fromowitz condition [GMFC] : for all t [a, b], for all (x, u) S(t), γ 0, γ, g(t, x, u) = 0, 0 D u { γ, g + λ, h } (t, x, u) + N L U (u) = γ = 0, λ = 0. 17

18 Corollary 4.6 Let (x, u ) be a local solution of constant positive radius R for (P C ), where [GMFC] holds. Then the necessary conditions of Corollary 4.4 hold, with a global Weierstrass condition: for almost every t, u S(t, x (t)) = H(t, x (t), u, p(t), γ(t), λ(t)) H(t, x (t), u (t), p(t), γ(t), λ(t)). We have stressed in this section the relative ease with which multiplier rules can be derived from Theorem 3.1, together with the versatility of the theorem and the transparency of its use. The unified treatment of both local and global versions is new, along with the assertion of the (reduced) Weierstrass condition in the local case. The discussion has been limited to smooth constraint functions here, but future work will report on the case of nonsmooth data. We mention also that it has not been necessary to make the type of artificial and restrictive structural assumption that has been a common feature of the literature on mixed constraints (see for example [15] and [8]). We mean by this the assumption that the control variable u can be partitioned into two parts (v, w) in such a way that v is completely free of unilateral constraints, and where a maximal rank condition is satisfied with respect to this free part of the control. To be precise, it has been assumed previously that the set S(t) is described by S(t) := {(x, v, w) : g(t, x, v, w) 0, h(t, x, v, w) = 0, w U}, and that the following global constraint hypothesis holds: for all t [a, b], for all (x, v, w) S(t), γ 0, γ, g(t, x, v, w) = 0, 0 D v { γ, g + λ, h } (t, x, v, w) = γ = 0, λ = 0. It is clear that this is a more restrictive hypothesis than the condition [GMFC] required by Corollary 4.6. References [1] G. A. Bliss. Lectures on the Calculus of Variations. University of Chicago Press,

19 [2] F. H. Clarke. Necessary Conditions for Nonsmooth Problems in Optimal Control and the Calculus of Variations. Doctoral thesis, University of Washington, (Thesis director: R. T. Rockafellar). [3] F. H. Clarke. Necessary conditions for nonsmooth variational problems. In Optimal Control Theory and its Applications, volume 106 of Lecture Notes in Econ. and Math. Systems, New York, Springer-Verlag. [4] F. H. Clarke. Le principe du maximum avec un minimum d hypothèses. Comptes Rendus Acad. Sci. Paris, 281: , [5] F. H. Clarke. Maximum principles without differentiability. Bulletin Amer. Math. Soc., 81: , [6] F. H. Clarke. The generalized problem of Bolza. SIAM J. Control Optim., 14: , [7] F. H. Clarke. The maximum principle under minimal hypotheses. SIAM J. Control Optim., 14: , [8] F. H. Clarke. The maximum principle in optimal control. J. Cybernetics and Control, 34: , [9] F. H. Clarke. Necessary Conditions in Dynamic Optimization. Memoirs of the Amer. Math. Soc., 173(816), [10] F. H. Clarke and M. d. R. de Pinho. The nonsmooth maximum principle. Preprint, [11] F. H. Clarke and M. d. R. de Pinho. Optimal control problems with mixed constraints. Preprint, [12] F. H. Clarke, Yu. S. Ledyaev, R. J. Stern, and P. R. Wolenski. Nonsmooth Analysis and Control Theory. Graduate Texts in Mathematics, vol Springer-Verlag, New York, [13] M. d. R. de Pinho. Mixed constrained control problems. J. Math. Anal. Appl., 278: , [14] M. d. R. de Pinho and R. B. Vinter. An Euler-Lagrange inclusion for optimal control problems. IEEE Trans. Automat. Control, 40: , [15] A. V. Dmitruk. Maximum principle for a general optimal control problem with state and regular mixed constraints. Comp. Math. and Modeling, 4: , [16] A. Ya. Dubovitskiĭ and A. A. Milyutin. Theory of the principle of the maximum. (Russian). In Methods of the Theory of Extremal Problems in Economics. Nauka, Moscow,

20 [17] M. R. Hestenes. Calculus of Variations and Optimal Control Theory. Wiley, New York, [18] A. D. Ioffe and V. Tikhomirov. Theory of Extremal Problems. Nauka, Moscow, English translation, North-Holland (1979). [19] A. A. Milyutin and N. P. Osmolovskii. Calculus of Variations and Optimal Control. American Math. Soc., Providence, [20] L. W. Neustadt. Optimization. Princeton University Press, Princeton, [21] R. B. Vinter. Optimal Control. Birkhäuser, Boston, [22] J. Warga. Optimal Control of Differential and Functional Equations. Academic Press, New York,