THE PRINCIPLE OF LEAST ACTION AND FUNDAMENTAL SOLUTIONS OF MASS-SPRING AND N-BODY TWO-POINT BOUNDARY VALUE PROBLEMS

Transcription

1 THE PRINCIPLE OF LEAST ACTION AND FUNDAMENTAL SOLUTIONS OF MASS-SPRING AND N-BODY TWO-POINT BOUNDARY VALUE PROBLEMS WILLIAM M. MCENEANEY AND PETER M. DOWER Abstrat. Two-point boundary value problems for onservative systems are studied in the ontext of the least ation priniple. One obtains a fundamental solution, whereby two-point boundary value problems are onverted to initial value problems via an idempotent onvolution of the fundamental solution with a ost funtion related to the terminal data. The lassial mass-spring problem is inluded as a simple example. The N-body problem under gravitation is also studied. There, the least ation priniple optimal ontrol problem is onverted to a differential game, where an opposing player maximizes over an indexed set of quadratis to yield the gravitational potential. Solutions are obtained as indexed sets of solutions of Riati equations. Key words. Least ation, two-point boundary value problem, differential game, N-body problem, optimal ontrol. MSC. 49LXX, 93E, 93C, 6H, 35G, 35D4.. Introdution. We suppose a onservative system follows a trajetory whih is a stationary point of the ation funtional, this being known as the priniple of least (more orretly, stationary) ation or as Hamilton s priniple (.f., [, ]). This allows the dynamial model to be posed in terms of various optimal ontrol problems. Solution of these ontrol problems allows one to onvert two-point boundary value problems (TPBVPs) for the dynamial system into initial value problems (IVPs). For purposes of illustration, we will onsider a simple mass-spring system, wherein solution of an assoiated Riati equation generates the fundamental solution, and allows one to answer a variety of TPBVPs via a simple max-plus integral (equivalently, a supremum). We will also onsider the N-body problem in orbital mehanis. There, the analysis beomes more tehnial. Nonetheless, one an onstrut mahinery for guaranteed solution of various TPBVPs. We begin with a somewhat formal disussion; speifiation of the exat assumptions will follow in the next setion. Suppose the position omponent of the state at time, t, is denoted by ξ t IR n, where also, we will use x IR n to denote generi positions. Let the potential energy at x IR n be denoted by V (x). The kineti energy at time, t, will be denoted by T ( ξ(t)) =. ξ (t)m ξ(t). If ξ(t) is a point mass, M is simply mi, where m is the mass of the body; in a multi-body system, this is generalized in the obvious way. The ation funtional orresponding to {ξ(r) r [, t]} is F(ξ( )). = V (ξ(r)) + T ( ξ(r)) dr. The original priniple of least ation stated that a system evolves so as to minimize the ation funtional. More reently, it has been understood that systems evolve so as to ahieve a stationary point of the ation funtional (.f., []). One an also interpret this in terms of the harateristi equations orresponding to the Hamiltonian of the system. Let the initial position be ξ() = x IR n, and Researh partially supported by grants from AFOSR and the Australian Researh Counil. Dept. Meh. and Aero. Eng., UC San Diego, La Jolla, CA 993 USA. wmeneaney@usd.edu Dept. Ele. & Eletroni Eng., Univ. Melbourne, Vitoria, Australia. pdower@unimelb.edu.au

2 let the dynamis be ξ(r) = u(r) for all r (, t), where u = u( ) U s,t, with U s,t. = L ([s, t); IR n ). Also let U. = {u : [, ) IR n u [,t) U,t t [, )}, (.) where u [,t) denotes the restrition of the funtion to domain [, t). Define the ontrol formulation payoff, J : [, ) IR n U IR {, + }, as J (t, x, u). = V (ξ(r)) + T (u(r)) dr = V (ξ(r)) + u (r)mu(r) dr, (.) where M is positive-definite symmetri, and the orresponding value funtion as W (t, x) =. inf J (t, x, u). (.3) u U Clearly a solution of this problem yields an ξ( ) satisfying the least ation priniple, and so is the trajetory of the onservative system under potential energy field V, when the stationary ation is the least. Let D =. (, t) IR n., D = [, t] IR n., and Ĉ = C( D) C (D). Under quite reasonable onditions on V, one an expet that W Ĉ, and that on D, W satisfies = { W (r, x) + inf v x W (r, x) + t v IR n v Mv } V (x) (.4) = t W (r, x) V (x) [ xw (r, x)] M x W (r, x). = H ( r, x, t W (r, x), xw (r, x) ).= t W (r, x) H ( r, x, x W (r, x) ). (.5) It is also well-established that under suffiiently strong onditions, first-order Hamilton-Jaobi-Bellman (HJB) partial differential equations (PDEs) suh as (.5) an be solved via the method of harateristis (.f., [7]). The harateristi equations assoiated with (.5) are dr dρ = H q (r, ˆξ, q, ˆp) =, dq dρ = H r (r, ˆξ, q, ˆp) =, These have assoiated initial and terminal onditions dˆξ dρ = H p (r, ˆξ, q, ˆp) = M ˆp(ρ) (.6) dˆp dρ = H x (r, ˆξ, q, ˆp) = x V (ˆξ(ρ)). (.7) ˆξ(t) = x, r(t) =, ˆp() =, q() = V (ˆξ()) (ˆp()) M ˆp() = V (ˆξ()), (.8) where ˆp() = follows from the lak of a terminal ost here. Beause of (.6), we may take r = ρ. Noting (.7) and (.8), we see that q(r) = V (ˆξ()) for all r. Also, in order to return to forward time, we may take s = t r, ξ(s) = ˆξ(t s) and p(s) = ˆp(t s), in whih ase we have or, dξ ds = M p(s), dp ds = xv (ξ(s)), (.9) d ξ ds = M x V (ξ(s)), (.) whih of ourse, is the lassial Newton s seond law formulation. Note that in the above development, the trajetory was not fully speified, as only the initial position,

3 not the initial state (position and veloity), was given. Of ourse, (.9) implies that the additive inverse of the o-state p(r), is the momentum. (One might also note that the optimal veloity in (.4) is attained at v = M x W = M p.) Given both the initial position and initial veloity, forward integration of (.9) is the lassial IVP form for the system dynamis. Suppose however, that one attahes a terminal ost to J yielding, say J(t, x, u) = J (t, x, u) + ψ(ξ(t)), (.) W (t, x) = inf J(t, x, u), (.) u U where U is given by (.). The dynami programing equation (DPE) and harateristi equations (.9) remain unhanged. However, although the initial ondition is still ξ() = x, the terminal ondition is defined by ψ. That is, we have a TPBVP where we ontrol the terminal ondition. TPBVPs are ommon in lassial optimal ontrol theory, where the above harateristi equations appear in Calulus of Variations and Pontryagin Maximum Priniple approahes (.f., [9]). There, one is required to solve the relevant TPBVP to obtain the desired optimal ontrol problem solution. Classial methods used a shooting approah, and more modern methods suh as pseudo-spetral algorithms (.f., [8]) have greatly advaned the state of the art. Here we have a slightly different goal; we desire to solve TPBVPs arising from dynamial systems governed by onservative dynamis. With the addition of terminal ost, ψ, the boundary onditions for (.9) onsist of initial and terminal onditions ξ() = x, p(t) = x ψ(ξ(t)). (.3) If one takes, for example, ψ(x) = x M v for some given v IR n, then the terminal ondition in (.3) beomes p(t) = M v. That is, one has boundary onditions ξ() = x and ξ(t) = v. (.4) Alternatively, if one takes z IR n and ψ(x) = ψ (x) =. δ (x z) where { δ (y) =. if y = + otherwise (.5) (i.e., the min-plus delta funtion,.f., [6, ]), then the solution of ontrol problem (.) yields solution of the onservative system with boundary onditions ξ() = x and ξ(t) = z. (.6) Clearly, other boundary onditions an be generated as well. The goal here will be the development of fundamental solutions for TPBVPs orresponding to onservative systems. These fundamental solutions will generate partiular solutions for boundary onditions suh as ξ(t) = v via a max-plus integration over IR n (.f., [,,, ]). In the ase where the potential energy takes a linear-quadrati form, the fundamental solution will be obtained through solution of an assoiated Riati equation. Here, we will use only a simple mass-spring example to demonstrate the onept, although a ombination of this approah with previously developed mahinery for solution of ertain infinite-dimensional problems [6, 7, 8] is expeted to yield fundamental solutions for ertain TPBVPs for infinite-dimensional systems [5]. 3

4 We will also apply the approah to N-body problems under the gravitational potential. In that ase, the potential does not take a linear-quadrati form. However, we will see that one may take a dynami game approah to gravitation, where the potential is a linear-quadrati form in the position variable. This requires an additional max-plus integral, over the opponent ontrols, beyond that whih is required in the purely linear-quadrati potential ase.. General Theory. We now begin the rigorous development. As indiated above, we onsider onservative systems, and take the least-ation approah. (That is, in this paper, we onentrate on the ase where the stationary ation is least see Lemma 4.6 and [,, 3] as well.).. Optimal ontrol problem. We model the dynamis of position as ξ(r) = u(r), ξ() = x IR n, (.) with u U. Let the potential and kineti energy funtions be denoted by V (x) and T (y) = y My, respetively. Realling (.), we now have J (t, x, u) = L(ξ(r), u(r)) dr. = Throughout this setion, we employ the following assumptions: M is positive-definite and symmetri. T (u(r)) V (ξ(r)) dr. (.) (A.M) There exists D V < suh that V (x) D V for all x IR n. (A.V ) There exists K L, K L < suh that V (x) V (z) K L x z, and V (x) K L ( + x ). (A.V ) (Of ourse, in (A.V ), the existene of suh a K L follows from the existene of K L, but we find it useful to introdue both onstants.) For [, ), let ψ : IR n IR n [, ) be given by ψ (x, z) = x z. (.3) Also let ψ : IR n IR n [, ] (where [, ]. = [, ) {+ }) be given by ψ (x, z) = lim ψ (x, z) = δ (x z), (.4) where δ is given in (.5). Define the finite time-horizon payoffs J : [, ) IR n U IR {, + } by J (t, x, u) = J (t, x, u, z). = J (t, x, u) + ψ (ξ(t), z), (.5) for [, ], where we speifially note that J (t, x, u) = L(ξ(s), u(s)) ds. Also, for [, ], we let W (t, x, z) =. inf J (t, x, u, z). (.6) u U Value funtions where one also notes dependene on terminal state omponents sometimes appear in the literature as generating funtions, speifially in referene to two-point boundary value problems (.f., [4]). As in the introdution, for generi terminal ost, ψ : IR n IR, we ontinue to let J(t, x, u). = J (t, x, u) + ψ(ξ(t)), and W (t, x). = inf u U J(t, x, u). (.7) 4

5 We begin with general theory; results speifi to appliation in mass-spring and N-body systems will follow in later setions. Lemma.. W (t, x, z) D V t for all x, z IR n and all t. Also, suppose there exists D, R < suh that V (y) D for all y B R (). Then W (t, x, z) Dt + min{, M /t} x z Dt + ψ (x, z) for all x, z B R () and t. More generally, for ψ : IR n IR, W (t, x) IR for all t (, ) and all x IR n. Lastly, note that for ĉ, W (t, x, z) W ĉ(t, x, z) for all t and x, z IR n. Proof. To obtain the first assertion, note that for any u U, J (t, x, u, z) V (ξ(r)) dr D V t, where D V is given in Assumption (A.V ). For the seond assertion, let ū(s) = for all s (, t). Then, J (t, x, ū, z) = V (x) dr+ψ (x, z) Dt + ψ (x, z), whih implies W (t, x, z) Dt + ψ (x, z). Alternatively, let ũ(r) = t (z x) for all r (, t). Then, the orresponding trajetory satisfies ξ(r) B R () for all r [, t], and we have J (t, x, ũ, z) = t (z x) M(z x) V ( ξ(r)) dr+ψ (z, z) M t x z +Dt, whih implies W (t, x, z) M t x z +Dt. For the third assertion, simply take ū. The final assertion is immediate by the definition. One expets that W will be a visosity solution on (, ) IR n of = t W (t, x, z) H(r, x, xw (t, x, z)) = H(r, x, t W (t, x, z), xw (t, x, z)) (.8) W (, x, z) = ψ (x, z) x IR n, (.9) where H, H are the Hamiltonians (.5). In fat, we have the following: Theorem.. Let [, ) and z IR n. Value funtion W (,, z) of (.7) is Lipshitz ontinuous on ompat sets, and is the unique visosity solution of (.8),(.9). Proof. This follows immediately from [3], where we speifially use Proposition.3 and Theorems.,. and 3. there... A limit property. In order to haraterize the fundamental solution to the optimal ontrol problem (.7), it is useful to first demonstrate that a speifi limit property holds. In partiular, it is demonstrated via a sequene of lemmas that lim W = W. Lemmas.4 and.5 provide bounds on near-optimal trajetories defined with respet to W, leading to a sandwihing of W using W. The required limit property is then stated via Theorem.6 and Corollary.7. By the positive-definiteness of M, there exists m > suh that T (v) m v, v IR n. (.) Let t >. The straight-line ontrol from x to z is given by u s r = (/t)[z x] for all r [, t], and we let the orresponding trajetory be denoted by ξ s. The resulting ost is W s (t, x, z) =. J (t, x, u s, z) KL( M z x + x + z )t +, t whih for an appropriate hoie of D = D (t) <, D (t)[ + x + z ], x, z IR n. (.) Remark.3. We have W (t, x, z) W s (t, x, z) D (t)[ + x + z ] for all t (, ) and all x, z IR n. Lemma.4. There exists ˆD = ˆD(t) < suh that for any ɛ-optimal trajetory, ξ ɛ (i.e., any trajetory ξ ɛ orresponding to an ɛ-optimal input in the definition (.6)) with ɛ (, ], ξ ɛ (r) ˆD[ + x + z ] for all r t < and x, z IR n. 5

6 Proof. Let t > and x, z IR n. Let u ɛ U be ɛ-optimal in the definition (.6) of W with ɛ (, ], and let ξ ɛ be the orresponding trajetory. Let R. = max{ ξ ɛ (r) r [, t]}, τ argmax{ ξ ɛ (r) r [, t]}. (.) Note that by Hölder s inequality, R = ξ ɛ (τ) τ u ɛ L(,τ) + x t u ɛ L(,t) + x. (.3) Now, using Assumption (A.V ), (.) and (.), J (t, x, u ɛ, z) whih by (.3), K L( + R)t + m t (R x ). V (ξ ɛ (r)) + T (u ɛ (r)) dr K L( + R)t + m ξ ɛ L (,t), Consequently, onsidering the quadrati inequality in R given by KL( + R)t + m t (R x ) [ KL( M z x + x + z )t + + ], t and solving the quadrati equality by lassial methods, we see that there exists ˆD = ˆD(t) < suh that J (t, x, u ɛ, z) > KL(+ x + z )t+ M z x + t W s (t, x, z)+ W (t, x, z)+ɛ if R > ˆD[ + x + z ], whih ontradits the ɛ-optimality of u ɛ. Hene, R ˆD( + x + z ), ompleting the proof. Lemma.5. There exists D = D(t) < suh that for ɛ-optimal ontrols, u,ɛ U, with ɛ (, ], ξ,ɛ (t) z D[+ x + z ], for all, t > and x, z IR n. Proof. Let ɛ (, ],, t > and x, z IR n. By Assmp. (A.V ) and Lemma.4, On the other hand, J (t, x, u,ɛ, z) K L( + ˆD[ + x + z ])t + ξ,ɛ (t) z. (.4) W (t, x, z) W (t, x, z) J (t, x, u,ɛ, z) ɛ J (t, x, u,ɛ, z). (.5) Combining (.4) and (.5) yields whih by (.), ξ,ɛ (t) z W (t, x, z) + K L( + ˆD[ + x + z ])t W s (t, x, z) + K L( + ˆD[ + x + z ])t, D (t)[ + x + z ] + K L( + ˆD[ + x + z ])t. Theorem.6. There exists D = D(t) < suh that W (t, x, z) D [ + x + z ] W (t, x, z) W (t, x, z), for all t (, ), x, z IR n and. 6

7 Proof. Clearly, W (t, x, z) W (t, x, z) for all t, (, ) and x, z IR n. We onentrate on the other bound. Let u,ɛ be ɛ-optimal for W (t, x, z), with ɛ (, ], and let ξ,ɛ denote the orresponding trajetory. Also for r [, t], let û,ɛ (r). = u,ɛ (r) + (/t)[z ξ,ɛ (t)], whih yields ˆξ,ɛ (t) = z. (.6) Further, using Lemma.5, this implies ˆξ,ɛ (r) ξ,ɛ (r) = r t ξ,ɛ (t) z r D[ + x + z ] t, r [, t]. (.7) Next, note that whih implies J (t, x, u,ɛ, z) = V (ξ,ɛ (r)) + T (u,ɛ (r)) dr + ψ (ξ,ɛ (t), z) W (t, x, z) + ɛ W s (t, x, z) +, m u,ɛ L (,t) V (ξ,ɛ (r)) dr + W s (t, x, z) +, whih by Assumption (A.V ), (.) and Lemma.4 K L( + ˆD(t)[ + x + z ])t + D (t)[ + x + z ] +. This implies there exists D = D (t) < suh that u,ɛ L(,t) D (t)[ + x + z ]. (.8) Now, realling that a b a b ( a + b ) for all a, b IR n, one has T (u,ɛ (r)) dr T (û,ɛ (r)) dr M whih by the definition of û,ɛ, M t M t M t u,ɛ (r) û,ɛ (r) ( u,ɛ (r) + û,ɛ (r) ) dr z ξ,ɛ (t) ( u,ɛ (r) + û,ɛ (r) ) dr [ z ξ z ξ,ɛ,ɛ ] (t) (t) + u,ɛ (r) dr t z ξ,ɛ (t) [ z ξ,ɛ (t) + t u,ɛ ] L(,t) (where the last bound follows by Hölder s inequality), whih by Lemma.5 and (.8), D 3(t)[ + x + z ], (.9) for all x, z IR n and all [, ) for proper hoie of D 3 (t) <. Also, by Assumption (A.V ), whih by (.7), V (ξ,ɛ (r)) dr V (ˆξ,ɛ (r)) dr K L 7 ξ,ɛ (r) ˆξ,ɛ (r) dr,

8 K L D[ + x + z ]t (.) By (.6), (.9), (.) (and noting that ψ ), J (t, x, u,ɛ, z) J (t, x, û,ɛ, z) D 3(t)[ + x + z ] D(t)[ + x + z ], K L D[ + x + z ]t for an appropriate hoie of D(t) <. This implies J (t, x, u,ɛ, z) W (t, x, z) D(t)[ + x + z ], and sine this is true for all ɛ (, ], W (t, x, z) W (t, x, z) D(t)[+ x + z ], whih ompletes the proof. Of ourse, Theorem.6 immediately implies: Corollary.7. The value funtions W and W of (.6) satisfy the limit property lim W (t, x, z) = W (t, x, z) for all t (, ), x, z IR n..3. Fundamental solution. A reahability problem of interest is defined via the value funtion W : IR IR n IR n IR, where W (t, x, z) =. { } inf L(ξ(s), u(s)) ds (.) holds with u U. (.) ξ() = x, ξ(t) = z Using W of (.), it is onvenient to define the funtion Ŵ : IR IR n IR by Ŵ (t, x) =. } inf { W (t, x, z) + ψ(z). (.) z IR n Theorem.8. The value funtion W of (.7) and the funtion Ŵ of (.) are equivalent. That is, W (t, x) = Ŵ (t, x) for all t IR and x IR n. Proof. Fix t IR, x IR n, δ IR >. Let u δ U denote a δ-optimal input in the definition (.7) of W (t, x). As u δ U is sub-optimal in the definition (.) of W (t, x, ξ δ (t)), and ξ δ (t) IR n is sub-optimal in the definition (.) of Ŵ (t, x), it follows that W (t, x) + δ L(ξ δ (s), u δ (s)) ds + ψ(ξ δ (t)) W (t, x, ξ δ (t)) + ψ(ξ δ (t)) Ŵ (t, x). As δ IR > is arbitrary, it follows by taking δ + that W (t, x) Ŵ (t, x). (.3) In order to prove the opposite inequality, (.) and (.) together imply that { { } } Ŵ (t, x) = inf inf L(ξ(s), u(s)) ds (.) holds with z IR n u U + ξ() = x, ξ(t) = z ψ(z) { } = inf inf L(ξ(s), u(s)) ds + ψ(ξ(t)) (.) holds with z IR n u U ξ() = x, ξ(t) = z 8

9 inf { inf L(ξ(s), u(s)) ds + ψ(ξ(t)) { L(ξ(s), u(s)) ds + ψ(ξ(t)) z IR n u U = inf u U } (.) holds with ξ() = x } (.) holds with ξ() = x = W (t, x). (.4) Combining (.3) and (.4) ompletes the proof. In view of Theorem.8, W may be regarded as fundamental to the solution of the optimal ontrol problem (.7). In partiular, haraterization of W of (.) admits the evaluation of W of (.7) for any terminal ost ψ via (.). To this end, establishing a relationship between W of (.6) and W of (.) is vital. Theorem.9. The value funtions W of (.6) and W of (.) are equivalent. That is, W (t, x, z) = W (t, x, z), for all t IR and x, z IR n. Proof. Fix t IR and x, z IR n. Let u δ U denotes a δ-optimal input in the definition (.) of W (t, x, z). Note that (by definition) the orresponding trajetory ξ δ satisfies ξ δ () = x and ξ δ (t) = z. Hene, ψ (ξ δ (t), z) =. So, as u δ is sub-optimal in the definition (.6) of W (t, x, ξ δ (t)), W (t, x, z) = L(ξ δ (s), u δ (s)) ds + ψ (ξ δ (t), z) L(ξ δ (s), u δ (s)) ds W (t, x, z) δ. As δ IR > is arbitrary, sending δ + yields that W (t, x, z) W (t, x, z). (.5) In order to prove the opposite inequality, let ũ δ U denote a δ-optimal input in the definition (.6) of W (t, x, z). Let ξ δ denote the orresponding trajetory. First suppose that ξ δ (t) z. Then, (.4) implies that W (t, x, z) =, whih in turn implies that W (t, x, z) = by (.5). That is, W (t, x, z) = W (t, x, z) =, thereby ompleting the proof in this ase. So, alternatively, suppose that ξ δ (t) = z. Then, ψ (ξ δ (t), z) =, while ũ δ U must be sub-optimal in the definition (.) of W (t, x, z). That is, W (t, x, z) + δ = As δ IR > is arbitrary, sending δ + yield that Combining (.5) and (.6) ompletes the proof. L( ξ δ (s), ũ δ (s)) ds + ψ ( ξ δ (t), z) L( ξ δ (s), ũ δ (s)) ds W (t, x, z). W (t, x, z) W (t, x, z). (.6) 3. Appliation: a simple mass-spring system. 3.. Model. We onsider the standard example: A mass M (, ) is fixed to a vertial wall via an elasti spring with spring onstant K (, ), with the mass free to move horizontally. Frition is negleted. Newton s seond law implies that the position ξ satisfies the ordinary differential equation (ODE) 9

10 = ξ(t) + ω ξ(t) (3.) where ω. = K/M is the frequeny of osillation. The potential and kineti energy assoiated with the spring and mass respetively are given by V (x). = K x, T ( ξ). = M ( ξ). (3.) In this ase, our Hamiltonian beomes { H(x, p) = K x inf vp + M v IR p} = K x + M p. (3.3) As the potential energy for this idealized spring is quadrati (with potential energy possibly going to + ), Assumptions (A.V ) and (A.V ) are violated, and we annot employ Lemma. or Theorem.. However, we will have an expliit solution of the HJB PDE, and onsequently, we will use the following instead. Theorem 3.. Let (, ), z IR n, < t < T <. Suppose W C([, T ) IR n IR n ; IR) C ((, T ) IR n IR n ; IR) satisfies (.8),(.9). Then, W (t, x, z) J (t, x, u, z) for all x IR n, u U. Furthermore, W (t, x, z) = J (t, x, u, z) for the input u (s) =. M x W (t s, ξ (s), z), s [, t], where ξ is the solution of dynamis (.), driven by u. Consequently W (t, x, z) = W (t, x, z). Proof. With z IR n fixed, let W denote a solution of (.8), (.9) as per the theorem statement. Fix any t [, T ) and any ū U. Define π(v) =. p v + v Mv, p IR n, and note that by ompletion of squares that π(v) p M p. Selet v = ū(s) and p = x W (t s, ξ(s), z) at eah s [, t], where ξ denotes the trajetory satisfying (.) orresponding to input ū. Then, x W (t s, ξ(s), z) ū(s) + ū(s) Mū(s) [ xw (t s, ξ(s), z)] M x W (t s, ξ(s), z), so that (.8) and (.5) imply that for all s [, t], = t W (t s, ξ(s), z) V ( ξ(s)) [ xw (t s, ξ(s), z)] M x W (t s, ξ(s), z) t W (t s, ξ(s), z) + x W (t s, ξ(s), z) ū(s) + ū(s) Mū(s) V ( ξ(s)) = s W (t s, ξ(s), z) + ū(s) Mū(s) V ( ξ(s)). Integrating with respet to s over [, t] then yields (via the fundamental theorem of alulus and (.9)) that W (t, x, z) L( ξ(s), ū(s)) ds + ψ ( ξ(t), z) = J (t, x, ū, z), (3.4) proving the first assertion. To prove the seond assertion, fix ū. = u, where u is as indiated in the theorem statement. Repeating the above argument yields equality in (3.4), so that W (t, x, z) = J (t, x, u, z) = W (t, x, z) as required. 3.. Fundamental solution of the mass-spring system. Analogues of Theorems.6,.8 and.9 provide a path for solution of the optimal ontrol problem with value funtion W of (.7) assoiated with the priniple of least ation. In partiular, Theorem.8 provides a haraterization of W in terms of W of (.), whih is in turn equivalent to W of (.6) by Theorem.9. However, W may be obtained as the limit ase of W of (.6) as by Theorem.6, when suffiiently smooth,

11 where W may be obtained by solving (.8),(.9). To this end, let T define the time-indexed quadrati funtion W : [, T ) IR IR by. = π/ω, and W (t, x, z) = P t x + Q t x z + R t z, (3.5) where P t, Q t, R t IR satisfy the IVPs on [, T ) given by P t = K M P t, Q t = M P t Q t, Ṙ t = M Q t, (3.6) P =, Q =, R =. (3.7) Theorem 3.. The value funtion W of (.6) and the expliit funtion W of (3.5) are equivalent. That is, W (t, x, z) = W (t, x, z) for all t [, T ), x, z IR. Proof. By inspetion of (3.5), note that t (t, x, z) = P t x + Q t x z + Ṙt z, (3.8) x W (t, x, z) = P t x + Q t z. (3.9) By inspetion of (3.6), (3.7), (3.8) and (3.9), observe that for all t (, T ) and x, z IR, = [ P t x + Q t x z + Ṙt z + ( K ) x + ( M ) (P t x + Q t z) ] = t W (t, x, z) H(x, W (t, x, z)), +π t where H is the Hamiltonian (3.3). That is, (.8) holds for W. Also observe that W (, x, z) = x x z + z = ψ (x, z), where ψ is as per (.3). That is, (.9) also holds for W. Hene, Theorem 3. yields the desired result. Theorem 3. and the unbounded-potential analogue of Corollary.7 may be used to explore the limit ase of W of (.6) as. This limiting ase an be approahed expliitly by solving (3.6), (3.7) for arbitrary fixed IR > followed by taking the aforementioned limit. Applying Theorem.6 then yields W of (.6), and hene W of (.) by Theorem.9. By inspetion of (3.6), first note that it is onvenient to ompute the inverse of P t to failitate omputation of the limiting ase. To this. end, define α π t = P t, or P t π t = α, where α IR > is fixed. Differentiation yields P t π t + P t π t =, or ( π t = α π t P t π t = α π t K M P t ) πt = α K πt + α M = α K ( πt + α K M). For onveniene, selet α =. ( ) K M, so that π t = ω. Let t (, T ). Integra- tion over the interval [, t] yields tan π t t = ω t, or π t = tan ( tan π + ω t ) = tan ( ω t + tan ( α )). As, π t πt, where πt. = tan(ω t). Equivalently, P t P t as. Similarly, one obtains. = α πt = α tan(ω t) Q t = sin tan ( α ) sin ( ω t + tan ( α )) Q t. = α sin ω t as,

12 and ( R t = α + ( α ) ) ( + α + ( α ) ) ot ( ) ω t + α R. t = α tan(ω t) as. Hene, in the ase of the mass-spring system, Theorems.9 and 3. and the unbounded-potential analogue of Corollary.7 imply that for t (, π/ω), W (t, x, z) = P t x + Q t x z + R t z, (3.) where Pt = ( α ) ot(ω t), Q t = ( α )ose(ω t), R t = ( α ) ot(ω t). (3.) 3.3. Usage in a two-point boundary value problem. As an appliation of Theorem.8, onsider the ase where the terminal veloity v is known. As the state of (.) orresponds to the position of the mass, the additive inverse of the o-state defined via the value funtion W of (.7) orresponds to the momentum of the mass. As the final o-state is x ψ(x(t)), knowledge of the final momentum M v implies that x ψ(x(t)) = M v, whih in turn implies a terminal ost of ψ(z) = M v z. (3.) Let t (, π/ω). Applying Theorem.8, and using (.), the terminal position z (t, x) IR orresponding to initial position x IR and terminal veloity v = ẋ(t) is z (t, x, v) =. } argmin { W (t, x, z) M v z z IR { = argmin P t x + Q t x z + R t z M v z }. z IR Hene, by inspetion, = Q t x + R t z (t, x, v) M v, so that z (t, x, v) = M v Q t R t x = ( v ω ) tan(ω t) + se(ω t) x. (3.3) In order to hek (3.3), the dynamis of the mass-spring system may be integrated expliitly. In partiular, using general solution ξ(t) = A os(ω t) + B sin(ω t), and solving for A, B from ξ() = x and ξ(t) = v, one may hek the above solution. 4. The N-body problem. Here, we address the solution of TPBVPs with N bodies ating under gravitational aeleration. In partiular, we obtain a means for onversion of TPBVPs into initial value problems. The key to appliation of our approah to this lass of problems lies in a variation of onvex duality, leading to an interpretation of the least ation priniple as a differential game. Lemma 4.. For ρ (, ), one has ρ = ( ) 3/ 3 max α α (, ) ] [ (αρ) = ( ) 3/ 3 max α α [, /3ρ ] [ (αρ) Proof. Suppose f : (, ) IR is given by f(ˆρ) = ˆρ /. By standard methods of onvex duality (.f., [3, 4, 5]), one has the onvex duality pair [ ] f(ˆρ) = sup ˆβ ˆρ + a( ˆβ) ˆρ (, ), ˆβ< ].

13 a( ˆβ) [ ] = sup ˆβ ˆρ f(ˆρ) ˆρ> ˆβ (, ). Further, a( ˆβ) = 3 ( ˆβ) /3 for all ˆβ (, ). Next, letting β =. ˆβ, this yields [ ] 3 ˆρ / = sup β> (β)/3 β ˆρ, ˆρ >. Letting α = 3 (β)/3 for β >, one finds [ (3 ) 3/ ˆρ / = sup α α ( 3 ) ] 3/ α 3 ˆρ, ˆρ >. Finally, letting ˆρ = ρ for ρ >, one sees that this beomes ( ) 3/ ] 3 ρ = sup α [ (αρ), ρ >. α Lastly, note that the supremum is always attained, and does so at From Lemma 4., one immediately obtains the following. Lemma 4.. Given any δ (, ) and any ρ [ δ, ), one has ρ = while for ρ (, δ), one has max ρ δ ρ ( ) 3/ 3 max α α [, /3 δ ] ( ) 3/ 3 max α α [, /3 δ ] [ (αρ) ], ] [ (αρ) < ρ. Reall that the gravitational potential energy due to two point masses of mass m and m, separated by distane ρ >, is given by G m,m (ρ) = Gm m, ρ where G is the universal gravitational onstant. Of ourse, this is also valid for spherially symmetri bodies when the distane is greater than the sum of the radii of the bodies. Using Lemma 4., we see that this may be represented as G m,m (ρ) = Ĝ m max (α,m ) [ (α,ρ) ], α, where the universal gravitational onstant is replaed by Ĝ =. ( 3 3/ ) G. In the ase of N bodies at loations x i IR 3 for i N =],. N[ (where for integers i < j, we let ]i, j[ denote {i, i +, i +,... j} throughout), the additive inverse of the potential is given by Ṽ (x) = Ĝ m i max α i,j (α i,jm j ) [ (α i,j x i x j ) ] = 3 3 ρ. Gm i m j x i x j, (4.)

14 where I. = {(i, j) ], N[ j > i} and x = {x, x,... x N } IR n. = (IR 3 ) N. In view of Lemma 4., we fix some δ >, and use instead, V (x) = Ĝ m i max (α i,j m j ) α i,j [, /3 δ ] [ (α i,j x i x j ) ]. (4.) Throughout, we will largely suppress the dependene of V on the body masses. It may be worth mentioning that while form (4.) is only valid for point masses and spherially symmetri masses at distanes greater than the sum of the body radii, form (4.) also holds for a point mass within the radius of a uniform density, spherially symmetri body. The next result follows immediately from Lemma 4.. From it, we will see that for the realisti ase where the bodies have positive radii, one may hoose δ suh that V and Ṽ yield idential solutions. Lemma 4.3. Suppose x i x j δ for all (i, j) I. Then V (x) = Ṽ (x). Otherwise, V (x) Ṽ (x). Let A. = { α = {α i,j } α i,j [, /3 δ ] (i, j) I }, (4.3) and note that A IR I where I. = #I. Then (4.) may be written as V (x) = max { ˆV (x, α)}, ˆV (x, α) =. Ĝ m i (α i,j m j ) [ (α i,j x i x j ) ]. α A (4.4) Let ξ( ) be a trajetory of the N-body system satisfying (.). The running ost will again be where now V is given by (4.4). Also, let L(ξ(r), ξ(r)) = T ( ξ(r)) V (ξ(r)), (4.5) M. = diag(m, m, m, m, m, m,... m N ) = diag(m, m,, m N ) I 3 (4.6) (where denotes the Kroneker produt), m. = min i N m i >, and M. = max i N m i. Note that we may write T (y) = y My, y IR n. (4.7) We also ontinue to take ψ as given in Setion. (i.e., by (.3) and (.4)) for [, ]. With these speifi definitions, the least-ation payoff, J given by (.5), beomes J (t, x, u, z) = = As in (.6), we let the value be given by T (u(r)) V (ξ(r)) dr + ψ (ξ(t), z) (4.8) T (u(r)) + max α A { ˆV (ξ(r), α)} dr + ψ (ξ(t), z). (4.9) W (t, x, z) = inf u U J (t, x, u, z). (4.) 4

15 Let J : [, ) IR n U IR n IR and W : [, ) IR n IR n IR be given by Fix δ δ, and let J (t, x, u, z) = T (u(r)) Ṽ (ξ(r)) dr + ψ (ξ(t), z), (4.) W (t, x, z) = inf J (t, x, u, z). (4.) u U D δ. = { x IR n x i x j > δ (i, j) I }. (4.3) Fix t > and x, z D δ. We assume: = (t, x, z) <, ɛ = ɛ(t, x, z) > suh that ɛ-optimal u ɛ U in (4.) with ɛ (, ɛ], and with ξ ɛ denoting the orresponding trajetory, we have (ξ ɛ ) i (r) (ξ ɛ ) j (r) δ (A.N ) r [, t], (i, j) I. Theorem 4.4. Let t (, ) and x, z D δ. Let (t, x, z). Suppose u U minimizes J (t, x,, z). Then u also minimizes J (t, x,, z). Proof. Fix t [, ) and x, z IR n. Let u U minimize J (t, x,, z). Let ũ U. By (4.9), (4.), Lemma 4.3, and then by the hoie of u, J (t, x, ũ, z) J (t, x, ũ, z) J (t, x, u, z), whih by Assumption (A.N) and Lemma 4.3, = J (t, x, u, z). Corollary 4.5. Let t [, ) and x, z D δ. Then, W (t, x, z) = W (t, x, z) for all (t, x, z). Heneforth, we work only with V, J, W, rather than Ṽ, J, W. Let A. = { α : [, ) A K <, {τk } k ],K[ suh that τ =, τ K = t, and τ (k ) < τ k and α [τk,τ k ) C([τ k, τ k ); A) k ], K[ }, (4.4) Ā. = L ([, ); A), (4.5) and we note that, of ourse, C([, ); A) A Ā. Also, we replae the timeindependent potential energy funtion, V ( ), with V α (r, x) =. ˆV (x, α(r)) = Ĝ m i (α i,j (r)m j ) [ (α i,j(r) x i x j ) ]. (4.6) Let J : [, ) IR n U Ā IR n IR be given by J (t, x, u, α, z). = Theorem 4.6. Let t and x, z IR n. Then, T (u(r)) V α (r, ξ(r)) dr + ψ (ξ(t), z). (4.7) and J (t, x, u, z) = max α A J (t, x, u, α, z) = max α Ā J (t, x, u, α, z), u U, (4.8) W (t, x, z) = inf u U max α( ) A J (t, x, u, α, z) = inf u U max J (t, x, u, α, z). (4.9) α( ) Ā 5

16 Proof. Fix t and x, z IR n. Let u U, and reall from (4.4) and (4.9) that J (t, x, u, z) = T (u(r)) + max Ĝ m i (α i,j m j ) [ (α i,j ξ i (r) ξ j (r) ) ] dr α A + ψ (ξ(t), z). (4.) By (4.4), (4.5), (4.7) and (4.), any α(r) is suboptimal in the maximization in (4.) for any r [, t] and any α Ā A, and in partiular, J (t, x, u, z) max J (t, x, u, α, z) max J (t, x, u, α, z), (4.) α( ) Ā α( ) A and we do not inlude the obvious details. Let ᾱ : IR n A be given by ᾱ (x) = {ᾱ i,j (xi, x j )}, where ᾱi,j(x i, x j ) =. argmax α [ (α xi x j ) α [, /3 δ ] = argmax Ĝ m i (αm j ) [ (α xi x j ) α [, /3 δ ] ], (i, j) I, x IR n ], (i, j) I, x IR n. Let ξ denote the state trajetory orresponding to u and ξ = x. Let (4.) α (r) = α (r; u( )) = {α i,j(r) (i, j) I } A, (4.3) where the (i, j) th element of α is given by α i,j(r) = ᾱ i,j(ξ i (r), ξ j (r)), r [, t). (4.4) Note that α A. Also note that by (4.) and (4.4), αi,j(r) = argmax Ĝ m i (αm j ) [ (α ξi (r) ξ j (r) ) ] α [, /3 δ ] Then, by (4.), (4.6) and (4.5), (i, j) I, r [, t). (4.5) By (4.8), (4.7), and (4.6), V α (r, ξ(r)) = V (ξ(r)) r [, t). (4.6) J (t, x, u, z) = J (t, x, u, α, z) max α A J (t, x, u, α, z). (4.7) By (4.) and (4.7), we have (4.8). That, in turn, immediately implies (4.9). We speifially note that the problem of finding the fundamental solution of the TPBVP for the N-body problem has been onverted to a differential game. In a heuristi sense, one may think of the problem now as not only a searh over possible world lines of the bodies, but as also inluding a searh over negotiated potentials between the bodies. Again heuristially, one may think of the potentials, not as fields existing throughout spae but as the opposing player in a game interpretation. The first player minimizes the ation at eah moment, with immediate effet on the kineti 6

17 term and integrated effet on the other terms, while the seond player maximizes the potential term at eah moment. The analytial advantage obtained through the use of this viewpoint is that one may express the potential energy as a quadrati form. Remark 4.7. We note that (4.9) is a non-standard form for dynami games. The inf / sup is neither in terms of non-antiipative strategies (.f., [, 9]), nor in terms of state feedbak ontrols. This is due to the very simple form of the maximizing player, whih is only a representation for the running ost. Remark 4.8. Note that with V given by (4.) and M given by (4.6), V and M satisfy onditions (A.M), (A.V ) and (A.V ) of Setion.. Lemma 4.9. W (t, x, z) [, Dt+ψ (x, z)] for all t and all x, z IR n, where D = (G/ δ) (i,j) I m im j. Proof. The result follows by Remark 4.8 and Lemma.. Lemma 4.. Let ɛ (, ]. Given ɛ-optimal u ɛ in the definition, (4.), of W (t, x, z), we have u L m ( Dt (,t) + ψ (x, z) + ). Proof. Let ɛ (, ], and let u ɛ be as per the lemma statement. Let the orresponding trajetory be denoted by ξ ɛ. Then, using Lemma 4.9, T (u ɛ (r)) V (ξ ɛ (r)) dr + ψ (ξ ɛ (t), z) W (t, x, z) + Dt + ψ (x, z) +. Hene, noting the non-positivity of the potential, one has T (u ɛ (r)) dr Dt + ψ (x, z) + + V (ξ ɛ (r)) dr Dt + ψ (x, z) +. That is, t (uɛ ) (r)mu ɛ (r) dr Dt + ψ (x, z) +. This immediately implies that u ɛ L (/ m)[ Dt (,t) + ψ (x, z) + ]. Lemma 4.. For any t >, W (t, x, z) is semionave in x, uniformly in (t, x, z, ) [t, ) IR n IR n [, ). Proof. Let t >, t [t, ), x, z IR n, [, ) and ɛ (, ]. Let γ IR n, γ < δ/4 where δ, ɛ are as in Assumption (A.N). Let u be an ɛ-optimal input in the definition, (4.), or W (t, x, z). We will obtain an upper bound on seond-order differene, [W (t, x + γ, z) + W (t, x γ, z) W (t, x, z)]/ γ, where this implies the asserted semionavity (.f., [4, ]). Let u + (r). = { u(r) t γ if r [, t ] and u (r). = { u(r) + t γ if r [, t ] u(r) if r (t, t), u(r) if r (t, t). (4.8) By the ɛ-optimality of u with respet to W (t, x, z) and the suboptimality of u ± with respet to W (t, x ± γ, z), W (t, x + γ, z) + W (t, x γ, z) W (t, x, z) < J (t, x + γ, u +, z) + J (t, x γ, u, z) J (t, x, u, z) + ɛ. (4.9) Let ξ, ξ + and ξ be the trajetories resulting from these ontrols with ξ() = x, ξ + () = x + γ and ξ () = x γ, and note that ξ + (r) ξ(r) = ξ (r) ξ(r) γ, r [, t ), (4.3) ξ(r) = ξ + (r) = ξ (r), r [t, t]. (4.3) 7

18 We see that (4.8) and (4.9) imply W (t, x + γ, z) + W (t, x γ, z) W (t, x, z) < T (u + (r)) + T (u (r)) T (u(r)) dr + + ψ (ξ + (t), z) + ψ (ξ + (t), z) ψ (ξ(t), z) + ɛ, whih by (4.8) and (4.3), = T (u + (r)) + T (u (r)) T (u(r)) dr + V (ξ(r)) V (ξ + (r)) V (ξ (r)) dr V (ξ(r)) V (ξ + (r)) V (ξ (r)) dr +ɛ. (4.3) We examine eah of the seond-order differenes separately. A simple alulation (and using notation (4.6)) verifies that T (u + (r)) + T (u (r)) T (u(r)) = t γ Mγ M t γ. Integrating, this yields T (u + (r)) + T (u (r)) T (u(r)) dr M t γ. (4.33) By the hoie of ontrols, Assumption (A.N), and the fat that γ < δ/4, for all (i, j) I, (ξ + ) i (r) (ξ + ) j (r) ξ i (r) ξ j (r) [ (ξ + ) i (r) ξ i (r) + (ξ + ) j (r) ξ j (r) ] δ/, r [, t], (4.34) and similarly for ξ. One may also show that there exists K < suh that V xx (y) K y IR n suh that y i y j δ/ for all (i, j) I. Then, using (4.3), (4.34) and a similar argument to that for T ( ), one finds that there exists K < suh that V (ξ(r)) [V (ξ + (r)) + V (ξ (r))] dr K γ. (4.35) Employing (4.33) and (4.35) in (4.3), one has W (t, x + γ, z) + W (t, x γ, z) W (t, x, z) [ ] M + K γ + ɛ. t As this is true for all suffiiently small ɛ >, we obtain the desired result. The HJB PDE assoiated with our problem here is = t W (t, x, z) H(x, xw (t, x, z)) { }. = tw (t, x, z) + inf sup v IR n v Mv ˆV (x, α) + v x W (t, x, z), (4.36) α A Note that the right-hand side of (4.36) is separated (and in fat, the Isaas ondition is satisfied). Consequently, we may write (4.36) as { } = tw (t, x, z) + min v IR n v Mv + v x W (t, x, z) + sup{ ˆV (x, α)} (4.37) α A 8

19 = t W (t, x, z) ( x W (t, x, z) ) M x W (t, x, z) + sup{ ˆV (x, α)}, (4.38) α A whih by (4.4), = t W (t, x, z) ( x W (t, x, z) ) M x W (t, x, z) V (x). (4.39) The initial onditions, indexed by z IR n, orresponding to value funtion W are For t >, let W (, x, z) = ψ (x, z), x IR n. (4.4) D t. = C([, t] IR n ) C ((, t) IR n ). (4.4) Theorem 4.. Let [, ) and z IR n. Value funtion W (,, z) is Lipshitz ontinuous on ompat sets, and is the unique visosity solution of HJB PDE (4.36) (equivalently, (4.37) (4.39)) and initial ondition (4.4). Let t >, and suppose further that W (,, z) D t and satisfies (4.36) (equivalently, (4.37) (4.39)) and initial ondition (4.4). Let x IR n, and let u be given by u (s) = ũ(s, ξ(s)) where ξ(s) is generated by (.) with feedbak ũ(s, x). = M x W (t s, x, z) and initial ondition ξ() = x. Then, W (t, x, z) = J (t, x, u, z) = W (t, x, z). Proof. By Remark 4.8, onditions (A.M), (A.V ) and (A.V ) of Setion. are satisfied. Consequently, the first assertion follows diretly from Theorem.. (We remark that the loal Lipshitz assertion also follows from Lemma 4..) We turn to the seond assertion. Fix t >. Let, z, W (,, z), u, ũ, ξ be as indiated. Let s (, t). Then, From (4.38) and then (4.4), x W (t s, ξ(s), z) u (s) + u (s) Mu (s) = [ xw (t s, ξ(s), z)] M x W (t s, ξ(s), z). (4.4) = t W (t s, ξ(s), z) + sup{ ˆV ( ξ(s), α)} α A [ xw (t s, ξ(s), z)] M x W (t s, ξ(s), z) = t W (t s, ξ(s), z) + sup{ ˆV ( ξ(s), α)} α A + x W (t s, ξ(s), z) u (s) + u (s) Mu (s) = s W (t s, ξ(s), z) + u (s) Mu { } (s) + sup ˆV ( ξ(s), α). α A Note that u U by definition of D t. Integrating with respet to s over [, t] (noting that the integrand is L by u U, the form of ˆV, (4.7), and Assumption (A.N)) yields = W (, ξ(t), z) W (t, x, z) + = W (, ξ(t), z) W (t, x, z) + whih, by applying (4.4), yields W (t, x, z) = T (u (s)) + sup{ ˆV ( ξ(s), α)} ds α A T (u (s)) V ( ξ(s)) ds, (4.43) T (u (s)) V ( ξ(s)) ds + ψ ( ξ(t), z) = J (t, x, u, z). 9

20 To obtain the last equality, note that by assumption and the definition of a visosity solution, W is also a visosity solution of (4.36),(4.4). Therefore, by the uniqueness obtained in the first assertion, W (t, x, z) = W (t, x, z). We now proeed to onsider the game where the order of infimum and supremum are reversed. Due to the very simple form of this partiular game, with the α ontroller ating only on the running ost and that being in a separated form, an unusual equivalene an be obtained. Realling (4.7), let W (t, x, z). = sup α Ā inf J (t, x, u, α, z). (4.44) u U By the usual reordering inequality, (4.9) immediately implies that W (t, x, z) W (t, x, z) (t, x, z) [, ) IR n IR n. (4.45) It will be helpful to introdue more notation. For [, ] and α Ā, we let W α, (t, x, z). = The orresponding Hamiltonian is Of ourse, one immediately sees that inf J (t, x, u, α, z). (4.46) u U H α (r, x, p). = V α (r, x) + p M p. (4.47) W (t, x, z) = sup α Ā W α, (t, x, z) (t, x, z) [, ) IR n IR n. (4.48) In a similar fashion to verifiation Theorem 4., we have the following. Theorem 4.3. Let (, ), z IR n and α Ā. In partiular, suppose that α is pieewise ontinuous, with possible disontinuities only at < τ < τ <... τ K < t with K <. Let τ =, τ K = t and O t. = k ],K [ (τ k, τ k+ ). Suppose W α (,, z) C(IR IR n ; IR) C (O t IR n ; IR) satisfies = r W α (r, x, z) H α (t r, x, x W α (r, x, z)), (r, x) O t IR n, (4.49) W α (, x, z) = ψ (x, z), x IR n. (4.5) Then, W α (t, x, z) J (t, x, u, α, z) for all x IR n, u U. Further, W α (t, x, z) = J (t, x, u, α, z) where u (s) = ũ(s, ξ(s)) with ξ(s). given by (.) with ũ(s, x) = M x W (t s, x, z) and ξ() = x. Consequently W α (t, x, z) = W α, (t, x, z). Proof. Fix t >, (, ), z IR n and α Ā. Let W α be as asserted, and let ū U. We use indution on k. Let k ], K [, and suppose W α (t τ k+, x, z) J (t τ k+, x, ū, α, z) for all x IR n, whih is ertainly true for k + = K. Define π(v) =. p v + v Mv, p IR n, and note that by ompletion of squares that π(v) p M p. Selet v = ū(s) and p = x W α (t s, ξ(s), z) at eah s (τ k, τ k+ ), where ξ denotes the trajetory satisfying (.) orresponding to input ū. Then, x W α (t s, ξ(s), z) ū(s) + ū(s) Mū(s) [ xw α (t s, ξ(s), z)] M x W α (t s, ξ(s), z), so that (4.47) and (4.49) imply that for all s (τ k, τ k+ ), = t W α (t s, ξ(s), z) V α (s, ξ(s))

21 [ xw α (t s, ξ(s), z)] M x W α (t s, ξ(s), z) t W α (t s, ξ(s), z) + x W α (t s, ξ(s), z) ū(s) + ū(s) Mū(s) V α (s, ξ(s)) = d ds W α (t s, ξ(s), z) + ū(s) Mū(s) V α (s, ξ(s)). Integrating with respet to s over (τ k, τ k+ ) then yields that W α (t τ k+, ξ(τ k+ ), z) W α (t τ k, ξ(τ k ), z) + or equivalently, W α (t τ k, ξ(τ k ), z) whih by supposition, τk+ τk+ τ k τk+ τ k T (ū(s)) V α (s, ξ(s)) ds, T (ū(s)) V α (s, ξ(s)) ds + W α (t τ k+, ξ(τ k+ ), z), τ k T (ū(s)) V α (s, ξ(s)) ds+j (t τ k+, ξ(τ k+ ), ū, α, z) = J (t τ k, ξ(τ k ), ū, α, z). (4.5) By indution, we have the first assertion. To prove the seond assertion, fix ū. = u, where u is as indiated in the theorem statement. Repeating the above argument yields equality in (4.5), so that W α (t, x, z) = J (t, x, u, z) = W α, (t, x, z) as required. Lemma 4.4. Let t (, ) and x, z IR n. Let u U be a ritial point of J (t, x,, z) of (4.8), and let the orresponding state trajetory be denoted by ξ. Let α (r). = ᾱ (ξ (r)) for all r [, t) where ᾱ is given by (4.). Then, u is a ritial point of J (t, x,, α, z), where J is given in (4.7). Proof. Let ν U and δ >. We examine differenes in the diretion ν from u. In partiular, by inspetion of (4.6), (4.7) and (4.7), J (t, x, u + δν, α, z) J (t, x, u, α, z) = = δ[u (r)] Mν(r) δ [ x ˆV (ξ (r), α (r)) ] + δ ( x ψ (ξ (t), z) ) ν(r) dr + O(δ ) r δ[u (r)] Mν(r) δ [ x ˆV (ξ (r), ᾱ (ξ (r))) ] + δ ( x ψ (ξ (t), z) ) ν(ρ) dρ dr r ν(ρ) dρ dr ν(r) dr + O(δ ). (4.5) Now reall from (4.4) that V (x) = max α A [ ˆV (x, α)], where the maximum is uniquely attained at ᾱ (x). Consequently, x V (x) = x ˆV (x, ᾱ (x)), and therefore with α (r). = ᾱ (ξ (r)), we see that (4.5) beomes J (t, x, u + δν, α, z) J (t, x, u, α, z) = δ[u (r)] Mν(r) δ [ x V (ξ (r)) ] + δ ( x ψ (ξ (t), z) ) r ν(r) dr + O(δ ) ν(ρ) dρ dr

22 = J (t, x, u + δν, z) J (t, x, u, z) + O(δ ), and sine u is a ritial point of J (t, x,, z), = O(δ ). (4.53) That is, u is a ritial point of J (t, x,, α, z). Now let u be an optimal ontrol for our original problem (with potential energy funtion, V ( )), that is, we let u argmin u U J (t, x,, z). (4.54) where J is as per (.5). As u is a minimizer of J (t, x,, z), Lemma 4.4 immediately yields the following. Lemma 4.5. Let t (, ) and x, z IR n. Then, u given by (4.54) is a ritial point of J (t, x,, α, z). Lemma 4.6. Let t = t( δ) =. δ 3 G max i ],n[ ( j>i m j). (4.55) Let x, z IR n and t (, t). Then J (t, x,, α, z) is stritly onvex, and further, u given by (4.54) is the minimizer of J (t, x,, α, z). Proof. Let δ (, ) and ũ U, and let u be as per (4.54). Let ˆξ( ) denote the trajetory of (.) orresponding to initial state x IR n and input û =. u +δ ũ U. Note that ξ (r) = x + ˆξ(r) = x + r r u (s) ds u (s) + δ ũ(s) ds = ξ (r) + δ ξ(r), r. ξ(r) = ũ(s) ds. (4.56) In order to demonstrate onvexity of J (t, x, α,, z), it is onvenient to represent V α (r, ) of (4.6) as an expliit quadrati funtion of the vetor x IR n of all initial states. To this end, let E i IR N, i ], N[, denote the i th elementary basis vetor in IR N, and define E i,j IR N N by Similarly, define the matrix E i IR 3 n by E i,j. = (E i E j ) (E i E j ). (4.57) E i = E i I 3, E i,j. = (E i E j ) (E i E j ) = E i,j I 3, (4.58) in whih denotes the Kroneker produt (.f., [5]). quadrati funtion Ψ i,j : IR n IR by Using (4.58), define the Ψ i,j (x). = x E i,j x = (E i E j ) x = xi x j. (4.59) Employing (4.59) in the definition (4.6) of V α (r, ) yields that V α (r, x) = Ĝ m i (α i,j (r) m j ) [ (αi,j (r)) Ψ i,j (x) ]. (4.6) It is evident by inspetion of (4.7), (.3), and (4.6) that the funtions T, ψ (, z), and V α (r, ) are quadrati. In general, a quadrati funtion ψ : IR n IR satisfies ψ(x + h) = ψ(x) + x ψ(x) h + ( xxψ(x) h) h (4.6)

23 for all x, h IR n. Here, x ψ : IR n IR n and xx ψ : IR n IR n n denote respetively the derivative and Hessian of ψ. The first inner-produt term on the right-hand side is the diretional derivative of ψ at x IR n in diretion h IR n. In the speial ase where ψ(x) =. x P x is a quadrati funtion with P IR n n, x ψ(x) = (P + P ) x and xx ψ(x) = (P + P ). So, applying (4.6) to (4.7), (.3), (4.59), T (u (r) + δ ũ(r)) = T (u (r)) + δ (M u (r)) ũ(r) + δ (M ũ(r)) ũ(r), (4.6) Ψ i,j (ξ (r) + δ ξ(r)) = Ψ i,j (ξ (r)) + δ (E i,j ξ (r)) ξ(r) + δ (E i,j ξ(r)) ξ(r), (4.63) ψ (ξ (t) + δ ξ(t), z) = ψ (ξ (t), z) + δ ( (ξ (t) z)) ξ(t) + δ ξ(t). (4.64) In partiular, (4.6) and (4.63) imply that V α (r, ξ (r) + δ ξ(r)) = Ĝ m i (α i,j (r) m j ) = Ĝ m i (α i,j (r) m j ) = V α (r, ξ (r)) δ [ (α i,j (r)) Ψ i,j (ξ (r) + δ ξ(r)) ] [ (α i,j (r)) ( Ψ i,j (ξ (r)) + δ (E i,j ξ (r)) ξ(r) + δ (E i,j ) ξ(r)) ξ(r) ] Ĝ m i m j (α i,j (r)) 3 (E i,j ξ (r)) ξ(r) δ Hene, ombining (4.7), (4.6), (4.64) and (4.65), J (t, x, α, u + δ ũ, z) J (t, x, α, u, z) = Ĝ m i m j (α i,j (r)) 3 (E i,j ξ(r)) ξ(r). (4.65) δ(m u (r)) ũ(r) + δ (M ũ(r)) ũ(r) δ δ Ĝ m i m j (α i,j (r)) 3 (E i,j ξ(r)) ξ(r) dr Ĝ m i m j (α i,j (r)) 3 (E i,j ξ (r)) ξ(r) + δ ( (ξ (t) z)) ξ(t) + δ ξ(t). (4.66) The orresponding expression for J (t, x, α, u δ ũ, z) J (t, x, α, u, z) follows by substituting δ with δ in (4.66). Adding the result of this substitution to (4.66) yields the seond differene of J (t, x, α,, z), namely, J (t, x, α, u + δ ũ, z) + J (t, x, α, u δ ũ, z) J (t, x, α, u, z) (4.67) = δ (M ũ(r)) ũ(r) δ ( Ĝ m i m j α i,j (r) ) 3 (E i,j ξ(r)) ξ(r) dr + δ ξ(t). It remains to bound this seond differene from below. To this end, write ũ(r) = [ ũ (r) ũ N (r) ] and ξ(r) = [ ξ (r) ξn (r) ], in whih ũ i (r), ξ i (r) IR 3 for eah i ], n[ and r [, t]. So, realling the definition of M, (M ũ(r)) ũ(r) = N m i ũ i L. (4.68) [,t] i= 3

24 Similarly, realling the definition of E i,j and the fat that α i,j (r) [, δ ( 3 ) Assumption (A.N ), ) by ( α i,j (r) ) (E i,j ξ(r)) ξ(r) δ3 ( 3 ) 3 ξ i (r) ξ j (r), r [, t]. (4.69) So, in order to bound the summation term in (4.67), note that by (4.56), Hölder s inequality, and a reordering of the summations involved, m i m j ξ i (r) ξ ( j (r) dr m i m j ξ i (r) + ξ j (r) ) dr ( t r r ) = m i m j ũ i (s) ds + ũ j (s) ds dr = t = t N i= i= m i m i m j m i m j N j=, j i ( t ( r ( r ) ) ũ i (s) ds) + ũ j (s) ds dr ( ) ( ) r dr ũ i L + [,t] ũj L [,t] m j ( ũ i L [,t] + ũj L [,t] N N m i ũ i L [,t] m j + t i= j= N N = t m i ũ i L [,t] m j t j= N N m i i= j= N i= ) m j ũ j L [,t] t m i ũ i L [,t] = t N i= N i= m i ũ i L [,t] Combining (4.68) (4.7) in (4.67) (and noting there that [, )), m i ũ i L [,t] N j=, j i J (t, x, α, u + δ ũ, z) + J (t, x, α, u δ ũ, z) J (t, x, α, u, z) δ ( (M ũ(r)) ũ(r) Ĝ m i m j α i,j (r) ) 3 (E i,j ξ(r)) ξ(r) dr N δ m i ũ i L Ĝ δ3 [,t] ( 3 ) 3 t i= N = δ m i ũ i L [,t] ( Ḡ ) t δ 3 i= δ ( Ḡ ) t δ 3 N max m j i ],N[ j=, j i N i= N j=, j i m i ũ i L [,t] m j N j=, j i m j m j. (4.7) N m i ũ i L > (4.7) [,t] i= if δ (, ), ũ L[,t] >, and t (, t). That is, J (t, x, α,, z) is stritly onvex if t (, t), as required. 4

25 Theorem 4.7. Suppose t [, t) where t is as per (4.55). Then one has W (t, x, z) = W (t, x, z) =sup α Ā Wα, (t, x, z) for all x, z IR n. Proof. Let x, z IR n and t [, t). By the hoie of u viz (4.54) and (4.7), we have W (t, x, z) = J (t, x, u, z) = J (t, x, u, α, z), whih by Lemma 4.6, = min u U J (t, x, u, α, z) sup α Ā min J (t, x, u, α, z) = W (t, x, z). u U On the other hand, by (4.45), W (t, x, z) W (t, x, z), and onsequently we have the first equality. The seond follows immediately from (4.48). 4.. The limit property, N-body ase. Reall that the fundamental solution of interest is obtained in the limit. Consequently, we note that we have: Theorem 4.8. W (t, x, z) = lim W (t, x, z) = sup [, ) W (t, x, z), where the onvergene is uniform on ompat subsets of [, t) R N R N, with t as per (4.55). Proof. This follows diretly from Remark 4.8, Theorem.6 and the monotoniity of W (t, x, z) with respet to. Theorem 4.9. W (t, x, z) = sup α Ā Wα, (t, x, z) for all t [, t) and x, z IR n, where t is as per (4.55). Proof. Fix t [, t) and x, z IR n. By Theorems 4.7 and 4.8, W (t, x, z) = sup [, ) sup W α, (t, x, z) = sup α Ā sup α Ā [, ) W α, (t, x, z). (4.7) Now, J (t, x, u, α, z) is monotonially inreasing in for all (t, x, u, α, z) [, t) IR n U Ā IR n. From this one easily sees that W α, (t, x, z) is monotonially inreasing in. (Take ɛ-optimal ontrols, u.) Therefore, W α, (t, x, z) = lim Wα, (t, x, z) = sup W α, (t, x, z), (4.73) [, ) where we note that the finiteness of the supremum follows easily by using onstantveloity trajetories from x to z, and we do not inlude the details. Combining (4.7) and (4.73) yields the result. 4.. Fundamental solution as set of Riati solutions. We will find that the fundamental solution may be given in terms of a set of solutions of Riati equations. We look for a solution, Wα,, of the form W α, (r, x, z) = [x P r x + x Q r z + z R r z + γ r], r [, t], (4.74) where P r, Q r, R r, γ r IR n n depend impliitly on the hoie of α Ā and satisfy P r = P r I 3, Q r = Q r I 3, R r = R r I 3, γ r = γ r. (4.75) Here, P r I 3 denotes the Kroneker produt (.f., [5]) of P r with the identity matrix on IR 3, with P r, Q r, R r IR N N and γ r IR satisfying the respetive initial value problems P r = P r M P r + ν r, P = + I N, (4.76) 5