DISCRETE MECHANICS AND OPTIMAL CONTROL: AN ANALYSIS

ESAIM: COCV 17 (211) 322 352 DOI: 1.151/cocv/2112 ESAIM: Control, Optimisation an Calculus of Variations www.esaim-cocv.org DISCRETE MECHANICS AND OPTIMAL CONTROL: AN ANALYSIS Sina Ober-Blöbaum 1, Oliver Junge 2 an Jerrol E. Marsen 3 Rapie Not Abstract. The optimal control of a mechanical system is of crucial importance in many application areas. Typical examples are the etermination of a time-minimal path in vehicle ynamics, a minimal energy trajectory in space mission esign, or optimal motion sequences in robotics an biomechanics. In most cases, some sort of iscretization of the original, infinite-imensional optimization problem has to be performe in orer to make the problem amenable to computations. The approach propose in this paper is to irectly iscretize the variational escription of the system s motion. The resulting optimization algorithm lets the iscrete solution irectly inherit characteristic structural properties from the continuous one like symmetries an integrals of the motion. We show that the DMOC (Discrete Mechanics an Optimal Control) approach is equivalent to a finite ifference iscretization of Hamilton s equations by a symplectic partitione Runge-Kutta scheme an employ this fact in orer to give a proof of convergence. The numerical performance of DMOC an its relationship to other existing optimal control methos are investigate. Mathematics Subject Classification. 49M25, 49N99, 65K1. Receive October 8, 28. Revise September 17, 29. Publishe online March 31, 21. Introuction In orer to solve optimal control problems for mechanical systems, this paper links two important areas of research: optimal control an variational mechanics. The motivation for combining these fiels of investigation is twofol. Besies the aim of preserving certain properties of the mechanical system for the approximate optimal solution, optimal control theory an variational mechanics have their common origin in the calculus of variations. In mechanics, the calculus of variations is also funamental through the principle of stationary action; that is, Hamilton s principle. When applie to the action of a mechanical system, this principle yiels the equations of motion for that system the Euler-Lagrange equations. In optimal control theory the calculus of variations also plays a funamental role. For example, it is use to erive optimality conitions via the Pontryagin maximum principle. In aition to its importance in continuous mechanics an control theory, the iscrete calculus of variations an the corresponing iscrete variational principles play an important role in constructing efficient numerical schemes for the simulation of mechanical systems an for optimizing ynamical systems. Keywors an phrases. Optimal control, iscrete mechanics, iscrete variational principle, convergence. Research partially supporte by the University of Paerborn, Germany an AFOSR grant FA955-8-1-173. 1 Department of Mathematics, Faculty of Electrical Engineering, Computer Science an Mathematics, University of Paerborn, 3398 Paerborn, Germany. sinaob@math.upb.e 2 Zentrum Mathematik, Technische Universität München, 85747 Garching, Germany. junge@ma.tum.e 3 Control an Dynamical Systems, California Institute of Technology 17-81, Pasaena, CA 91125, USA. jmarsen@caltech.eu Article publishe by EDP Sciences c EDP Sciences, SMAI 21

DISCRETE MECHANICS AND OPTIMAL CONTROL: AN ANALYSIS 323 Discrete optimal control an iscrete variational mechanics. The theory of iscrete variational mechanics has its roots in the optimal control literature of the 196s; see for example [14,33,34]. Specifically, [14] evelope a iscrete calculus of variations theory in the following way: A function is introuce which epens on a sequence of numbers, e.g. a sequence of times. A minimizing sequence necessarily satisfies a secon-orer ifference equation, which is calle the iscrete Euler equation in reminiscence of its similarity with the Euler equation of the classical calculus of variations. An application of the iscrete calculus of variations to an optimal control problem leas to a so calle irect solution metho. In this, one transforms the optimal control problem into a finite imensional equality constraine nonlinear optimization problem via a finite imensional parameterization of states an controls. In contrast, inirect methos (see Sect. 2.3 for an overview) are base on the explicit erivation of the necessary optimality conitions via the Pontryagin maximum principle. On the other han, the theory of iscrete variational mechanics escribes a variational approach to iscrete mechanics an mechanical integrators. The application of a iscrete version of Hamilton s principle results in the iscrete Euler-Lagrange equations. Analogous to the continuous case, near conservation of iscrete energy, iscrete momentum maps relate to the iscrete system s symmetries an the iscrete symplectic form can be shown. This is ue to the iscretization of the variational structure of the mechanical system irectly. Early work on iscrete mechanics was often inepenently one by [15,5,51,58,6 62]. In this work, the role of the iscrete action sum, the iscrete Euler-Lagrange equations an the iscrete Noether s theorem were clearly unerstoo. The variational view of iscrete mechanics an its numerical implementation is further evelope in [85,86] an then extene in [5,6,38,39,66,67]. The route of a variational approach to symplectic-momentum integrators has been taken by [59,8]; see the review by [64] an references therein. In this review a etaile erivation an investigation of these variational integrators for conservative as well as for force an constraine systems is given. Combining optimal control an variational mechanics. The present paper concerns the optimal control of ynamical systems whose behavior can be escribe by the Lagrange- Alembert principle. To numerically solve this kin of problem, we make use of the iscrete calculus of variations only, that means we apply the iscrete variational principle on two layers. On the one han we use it for the escription of the mechanical system uner consieration, an on the other han for the erivation of necessary optimality conitions for the optimal control problem. The application of iscrete variational principles alreay on the ynamical level (namely the iscretization of the Lagrange- Alembert principle) leas to structure-preserving time-stepping equations which serve as equality constraints for the resulting finite imensional nonlinear optimization problem. The benefits of variational integrators are hane own to the optimal control context. For example, in the presence of symmetry groups in the continuous ynamical system, also along the iscrete trajectory the change in momentum maps is consistent with the control forces. Choosing the objective function to represent the control effort, which has to be minimize is only meaningful if the system respons exactly accoring to the control forces. Relate work. A survey of ifferent methos for the optimal control of ynamical systems escribe by orinary ifferential equations is given in Section 2.3. However, to our knowlege, DMOC is the first approach to solutions of optimal control problems involving the concept of iscrete mechanics to erive structure-preserving schemes for the resulting optimization algorithm. Since our first formulations an applications to space mission esign an formation flying [35 37,71], DMOC has been applie for example to problems from robotics an biomechanics [4,46,47,68,73,78] an to image analysis [69]. From the theoretical point of view, consiering the evelopment of variational integrators, extensions of DMOC to mechanical systems with nonholonomic constraints or to systems with symmetries are quite natural an have alreay been analyze in [46,47]. Further extensions are currently uner investigation, for example DMOC for hybri systems [73] anforconstraine multi-boy ynamics (see [56,57,71]). DMOC relate approaches are presente in [52,53]. The authors iscretize the ynamics by a Lie group variational integrator. Rather than solving the resulting optimization problem numerically, they construct the iscrete necessary optimality conitions via the iscrete variational principle an solve the resulting iscrete bounary value problem (the iscrete state an ajoint system). The metho is applie to the optimal control of a rigi boy an to the computation of attitue maneuversof a rigi spacecraft. Rapie Note

324 S. OBER-BLÖBAUM ET AL. Rapie Not Outline. In Sections 1.1 an 1.2 we introuce the relevant concepts from classical variational mechanics an iscrete variational mechanics following the work of [64]. Especially, we focus on the Lagrangianan Hamiltonian escription of control forces for the establishe framework of variational mechanics. Definitions an concepts of the variational principle, the Legenre transform, an Noether s theorem are reaopte for the force case in both the continuous as well as the iscrete setting. In Sections 2.1 an 2.2 we combine concepts from optimal control an iscrete variational mechanics to buil up a setting for the optimal control of a continuous an a iscrete mechanical system, respectively. Section 1.3 escribes the corresponence between the continuous an the iscrete Lagrangian system as basis for a comparison between the continuous an the iscrete optimal control problems in Section 2.3: We link both frameworks viewing the iscrete problem as an approximation of the continuous one. The application of iscrete variational principles for a iscrete escription of the ynamical system leas to structure-preserving time-stepping equations. Here, the special benefits of variational integrators are hane own to the optimal control context. These time-stepping equations serve as equality constraints for the resulting finite imensional nonlinear optimization problem, therefore the escribe proceure can be categorize as a irect solution metho. Furthermore, we show the equivalence of the iscrete Lagrangian optimal control problems to those resulting from Runge-Kutta iscretizations of the corresponing Hamiltonian system. This equivalence allows us to construct an compare the ajoint systems of the continuous an the iscrete Lagrangian optimal control problem. In this way, one of our main results is relate to the orer of approximation of the ajoint system of the iscrete optimal control problem to that of the continuous one. With the help of this approximation result, we show that the solution of the iscrete Lagrangian optimal control system converges to the continuous solution of the original optimal control problem. The proof strategy is base on existing convergence results of optimal control problems iscretize via Runge-Kutta methos [21,28]. Section 3.1 gives a etaile escription of implementation issues of our metho. Furthermore, in Section 3.2 we numerically verify the preservation an convergence properties of DMOC an the benefits of using DMOC compare to other stanar methos to the solution of optimal control problems. 1.1. Variational mechanics 1. Mechanical systems with forcing an control Our aim is to optimally control Lagrangian an Hamiltonian systems. For the escription of their ynamics, we introuce a variational framework incluing external forcing resulting from issipation, friction, loaing an in particular control forces. To this en, we exten the notions in [64] to Lagrangian control forces. Force Lagrangian systems. Consier an n-imensional configuration manifol Q with local coorinates q =(q 1,...,q n ), the associate state space given by the tangent bunle TQ an a C k Lagrangian L : TQ R, k 2. Given a time interval [, T], we consier curves q in the path space C 1,1 ([, T],Q) 4 an the action map G : C 1,1 ([, T],Q) R, T G(q) = L(q(t), q(t)) t. (1.1) To efine control forces for Lagrangian systems, we introuce a control manifol U R m an efine the control path space L ([, T],U)withu(t) U also calle the control parameter 5. With this notation we efine a Lagrangian control force as a map f L : TQ U T Q, which is given in coorinates as f L : (q, q, u) (q, f L (q, q,u)), where we assume that the control forces can also inclue configuration an velocity epenent forces resulting e.g. from issipation an friction. We interpret a Lagrangian control force as a family of Lagrangian forces that are fiber-preserving maps fl u : TQ T Q over the ientity i 6 Q, i.e. in coorinates fl u :(q, q) (q, fu L (q, q)). Whenever we enote f L(q, q, u) as a one-form on TQ, we mean the family 4 C 1,1 ([, T],Q) is the space of functions q :[, T] Q which are continuously ifferentiable on (, T) an whose first erivative is Lipschitz continuous on [, T]. 5 L enotes the space of essentially boune, measurable functions equippe with the essential supremum norm. 6 Amapϕ : T S T Q is calle fiber preserving if f π Q = π S ϕ 1 with π Q : T Q Q, π S : T S S the canonical projections an where f : Q S is efine by f = ϕ 1 Q.

DISCRETE MECHANICS AND OPTIMAL CONTROL: AN ANALYSIS 325 of horizontal one-forms fl u(q, q) ontq inuce by the family of fiber-preserving maps f L u. Given a control path u L ([, T],U), the Lagrange- Alembert principle seeks curves q C 1,1 ([, T],Q) satisfying δ T L(q(t), q(t)) t + T f L (q(t), q(t),u(t)) δq(t)t =, (1.2) where δ represents variations vanishing at the enpoints. The secon integral in (1.2) isthevirtual work acting on the mechanical system via the force f L. Integration by parts shows that this is equivalent to the force Euler-Lagrange equations L q (q, q) ( ) L (q, q) + f L (q, q, u) =. (1.3) t q These equations implicitly efine a family of force Lagrangian vector fiels XL u : TQ [, T] T (TQ)an associate force Lagrangian flows FL u : TQ [, T] TQ (u L ([, T],U)fixe). Remark 1.1. We istinguish between two notations: When we fix u U we always consier a family of Lagrangian control forces fl u. As soon as we consier evolutions given by ifferential equations or integrals, instea of fixing only one u U, wefixanentirecurveu L ([, T],U), such that for each time t we use the force f L that correspons to f L (q(t), q(t),u(t)). In particular, by fixing a control path u L ([, T],U)we obtain a non-autonomous system whose evolution is also epenent on the initial time t, such that the flow FL u woul be efine on TQ [, T] 2 rather than on TQ [, T]. In the following we will fix the initial time to be t = so that we o not nee to keep track on the initial time in the notation. This is no restriction since we consier all possible control paths u L ([, T],U). The one-form Θ L on TQ given in coorinates by Θ L = L q q i is calle the Lagrangian one-form, anthe i Lagrangian symplectic form Ω L = Θ L is given in coorinates by Ω L (q, q) = 2 L q i q q i q j + 2 L j q i q q i q j. j Recall that in the absence of forces, the Lagrangian symplectic form is preserve uner the Lagrangian flow [64]. Force Hamiltonian systems. Consier an n-imensional configuration manifol Q, an efine the phase space to be the cotangent bunle T Q. The Hamiltonian is a function H : T Q R. We will take local coorinates on T Q to be (q, p) withq =(q 1,...,q n )anp =(p 1,...,p n ). Define the canonical one-form Θ on T Q by Θ(p q ) u pq = p q,t πq u pq,wherep q T Q, u pq T pq (T Q), π Q : T Q Q is the canonical projection, T πq : T (T Q) TQ is the tangent map of π Q an, enotes the natural pairing between vectors an covectors. In coorinates, we have Θ(q, p) =p i q i. The canonical two-form ΩonT Q is efine to be Ω = Θ, which has the coorinate expression Ω(q, p) =q i p i. A Hamiltonian control force is amapf H : T Q U T Q ientifie by a family of fiber preserving maps fh u : T Q T Q over the ientity. Given such a control force, we efine the corresponing family of horizontal one-forms f H on T Q by (fh u ) (p q ) w pq = fh u (p q),t πq w pq. This expression is reminiscent of the canonical one-form Θ on T Q, an in coorinates it reas (fh u ) (q, p) (δq, δp) =fh u (q, p) δq. For a given curve u L ([, T],U), the force Hamiltonian vector fiel XH u is now efine by the equation i XH u Ω=H (f H u ) an in coorinates this gives the well-known force Hamilton s equations Rapie Note X u q (q, p) = H (q, p), p Xu H p (q, p) = q (q, p)+f H u (q, p), (1.4) which are the stanar Hamilton s equations in coorinates with the forcing term ae to the momentum equation. This efines the force Hamiltonian flow FH u : T Q [, T] T Q of the force Hamiltonian vector fiel XH u =(Xu q,xu p )(u L ([, T],U)fixe). The Legenre transform with forces. Given a Lagrangian L, we can take the stanar Legenre transform FL : TQ T Q efine by FL(v q ) w q = ɛ L(v q + ɛw q ), ɛ=

326 S. OBER-BLÖBAUM ET AL. where v q,w q T q Q, an which has coorinate form FL : (q, q) (q, p) = (q, L/ q(q, q)), an relate Hamiltonian an Lagrangian control forces by fl u = f H u FL. If we also have a Hamiltonian H relate to L by the Legenre transform as H(q, p) = FL(q, q) q L(q, q), then the force Euler-Lagrange equations an the force Hamilton s equations are equivalent. That is, if XL u an Xu H are the force Lagrangian an Hamiltonian vector fiels, respectively, then (FL) (XH u )=Xu L, cf. [64]. Noether s theorem with forcing. A key property of Lagrangian flows is their behavior with respect to group actions. Assume a Lie group G with Lie algebra g acts on Q by the (left or right) action φ : G Q Q. Consier the tangent lift of this action to φ TQ : G TQ given by φ TQ g (v q ) = T (φ g ) v q. For ξ g the infinitesimal generators ξ Q : Q TQ an ξ TQ : TQ T (TQ) are efine by ξ Q (q) = g (φ g(q)) ξ an ( ξ TQ (v q )= g φ TQ g (v q ) ) ξ, anthelagrangian momentum map J L : TQ g is efine to be J L (v q ) ξ = Θ L ξ TQ (v q ). If the Lagrangian is invariant uner the lift of the action, that is we have L φ TQ g = L for all g G (we also say, the group action is a symmetry of the Lagrangian), the Lagrangian momentum map is preserve of the Lagrangian flow in the absence of external forces. We now consier the effect of forcing on the evolution of momentum maps that arise from symmetries of the Lagrangian. In [64] it is shown that the evolution of the momentum map from time to time T is given by the relation [( ) ] J L (FL u )T (q(), q()) J L (q(), q()) ξ = T f u L (q(t), q(t)) ξ Q(q(t)) t. (1.5) Rapie Not Equation (1.5) shows, that forcing will generally alter the momentum map. However, in the special case that the forcing is orthogonal to the group action, the above relation shows that Noether s theorem will still hol. Theorem 1.2 (force Noether s theorem). Consier a Lagrangian system L : TQ R with control forcing f L : TQ U T Q such that the Lagrangian is invariant uner the lift of the (left or right) action φ : G Q Q an fl u(q, q),ξ Q(q) =for all (q, q) TQ, u U an all ξ g. Then the Lagrangian momentum map J L : TQ g will be preserve by the flow, such that J L (FL u)t = J L for all t. 1.2. Discrete mechanics The iscrete Lagrangian. Again we consier a configuration manifol Q, an efine the ( iscrete ) state space to be Q Q. Rather than consiering a configuration q an velocity q (or momentum p), we now consier two configurations q an q 1, which shoul be thought of as two points on a curve q which are a time step h>apart,i.e. q q() an q 1 q(h). The manifol Q Q is locally isomorphic to TQ an thus contains the same amount of information. A iscrete Lagrangian is a function L : Q Q R, which we think of as approximating the action integral along the exact solution curve segment q between q an q 1 : L (q,q 1 ) h L(q(t), q(t)) t. We consier the gri {t k = kh k =,...,N}, Nh = T, an efine the iscrete path space P (Q) ={q : {t k } N k= Q}. We will ientify a iscrete trajectory q P (Q) with its image q = {q k } N k=,whereq k = q (t k ). The iscrete action map G : P (Q) R along this sequence is calculate by summing the iscrete Lagrangian on each ajacent pair an efine by G (q )= N 1 k= L (q k,q k+1 ). As the iscrete path space P is isomorphic to Q... Q (N + 1 copies), it can be given a smooth prouct manifol structure. The iscrete action G inherits the smoothness of the iscrete Lagrangian L. The tangent space T q P (Q) top (Q) atq is the set of maps v q : {t k } N k= TQ such that τ q v q = q, which we will enote by v q = {(q k,v k )} N k=. To complete the iscrete setting for force mechanical systems, we present a iscrete formulation of the control forces introuce in the previous section. Since the control path u :[, T] U has no geometric interpretation, we have to fin an appropriate iscrete formulation to ientify a iscrete structure for the Lagrangian control force. Discrete Lagrangian control forces. Analogous to the replacement of the path space by a iscrete path space, we replace the control path space by a iscrete one. To this en we consier a refine gri Δ t, generate via a set of control points c 1 <...< c s 1asΔ t = {t kl = t k + c l h k =,...,N 1,l=1,...,s}. With this notation the iscrete control path space is efine to be P (U) ={u :Δ t U}. We efine the intermeiate

DISCRETE MECHANICS AND OPTIMAL CONTROL: AN ANALYSIS 327 control samples u k on [t k,t k+1 ]asu k =(u k1,...,u ks ) U s to be the values of the control parameters guiing the system from q k = q (t k )toq k+1 = q (t k+1 ), where u kl = u (t kl )forl {1,...,s}. With this efinition of the iscrete control path space, we take two iscrete Lagrangian control forces f +,f : Q Q U s T Q, given in coorinates as f + (q k,q k+1,u k )= ( q k+1,f + (q k,q k+1,u k ) ), f (q k,q k+1,u k )= ( q k,f (q k,q k+1,u k ) ), (1.6) also calle left an right iscrete forces 7. Analogously to the continuous case, we interpret the two iscrete Lagrangian control forces as two families of iscrete fiber-preserving Lagrangian forces f u k,± : Q Q T Q in the sense that π Q f u k,± = π ± Q with fixe u k U s an with the projection operators π + Q : Q Q Q, (q k,q k+1 ) q k+1 an π Q : Q Q Q, (q k,q k+1 ) q k. We combine the two iscrete control forces to give a single one-form f u k : Q Q T (Q Q) efine by f u k (q k,q k+1 ) (δq k,δq k+1 )=f u k,+ (q k,q k+1 ) δq k+1 + f u k, (q k,q k+1 ) δq k, (1.7) where f (q k,q k+1,u k ) enotes the family of all one-forms f u k (q k,q k+1 )withfixeu k U s. To simplify the notation we enote the left an right iscrete forces by f ± k := f ± (q k,q k+1,u k ), respectively, an the pair consisting of both by f k := f (q k,q k+1,u k ). We interpret the left iscrete force f + k 1 (an right iscrete force f k, respectively) as the force resulting from the continuous control force acting uring the time span [t k 1,t k ] (uring the time span [t k,t k+1 ], respectively) on the configuration noe q k. The iscrete Lagrange- Alembert principle. As with iscrete Lagrangians, the iscrete control forces also epen on the time step h, which is important when relating iscrete an continuous mechanics. Given such forces, we moify the iscrete Hamilton s principle, following [39], to the iscrete Lagrange- Alembert principle, which seeks iscrete curves {q k } N k= that satisfy L (q k,q k+1 )+ N 1 δ k= N 1 k= [ f (q k,q k+1,u k ) δq k + f + (q k,q k+1,u k ) δq k+1 ] = (1.8) for all variations {δq k } N k= vanishing at the enpoints. This is equivalent to the force iscrete Euler-Lagrange equations Rapie Note D 2 L (q k 1,q k )+D 1 L (q k,q k+1 )+f + (q k 1,q k,u k 1 )+f (q k,q k+1,u k )=, k =1,...,N 1. (1.9) These equations implicitly efine the force iscrete Lagrangian map F u k 1,u k L : Q Q Q Q for fixe controls u k 1,u k U s, mapping (q k 1,q k )to(q k,q k+1 ). The iscrete Lagrangian one-forms Θ + L an Θ L are in coorinates Θ + L (q,q 1 )=D 2 L (q,q 1 )q 1 an Θ L (q,q 1 )= D 1 L (q,q 1 )q. In the absence of external forces, the iscrete Lagrangian maps inherit the properties of symplectic preservation from the continuous Lagrangian flows. That means the iscrete Lagrangian symplectic form Ω L = Θ + L = Θ L (see [64] forthe coorinate expression) is preserve uner the iscrete Lagrangian map as (F L ) (Ω L )=Ω L,ifnoexternal forcing is present. The iscrete Legenre transforms with forces. Although in the continuous case we use the stanar Legenre transform for systems with forcing, in the iscrete case it is necessary to take the force iscrete Legenre transforms F f+ L :(q,q 1,u ) (q 1,p 1 )= ( q 1,D 2 L (q,q 1 )+f + (q,q 1,u ) ), (1.1a) F f L :(q,q 1,u ) (q,p )= ( q, D 1 L (q,q 1 ) f (q,q 1,u ) ). (1.1b) 7 Observe that the iscrete control force is now epenent on the iscrete control path.

328 S. OBER-BLÖBAUM ET AL. Again, we enote with F f± L u the force iscrete Legenre transforms for fixe controls u U s.usingthese efinitions an the force iscrete Euler-Lagrange equations (1.9), we can see that the corresponing force u iscrete Hamiltonian map F L = F f± L u1 F u,u1 L (F f± L u ) 1 u is given by the map F L :(q,p ) (q 1,p 1 ), where p = D 1 L (q,q 1 ) f u, (q,q 1 ), p 1 = D 2 L (q,q 1 )+f u,+ (q,q 1 ), (1.11) which is the same as the stanar iscrete Hamiltonian map with the iscrete forces ae. As in [64] one can show that the following two efinitions of the force iscrete Hamiltonian map F u L = F f± L u1 F u,u1 L ( F f± ) 1 L u, u F L = F f+ L u ( F f ) 1 L u, (1.12) are equivalent with coorinate expression (1.11). Thus from the secon expression in (1.12) it becomes clear, that the force iscrete Hamiltonian map that maps (q,p )to(q 1,p 1 ) epens on u only. The iscrete Noether theorem with forcing. As in the unforce case, we can formulate a iscrete version of the force Noether s theorem (for the erivation see for example [64]). To this en, the iscrete momentum map in presence of forcing is efine as J f+ L (q,q 1 ) ξ = F f+ L u (q,q 1 ),ξ Q (q 1 ), J f L (q,q 1 ) ξ = F f L u (q,q 1 ),ξ Q (q ). Rapie Not The evolution of the iscrete momentum map is escribe by [ J f+ L ( F u ) ] N 1 N 1 L J f L (q,q 1 ) ξ = f u k (q k,q k+1 ) ξ Q Q (q k,q k+1 ). (1.13) Again, in the case that the forcing is orthogonal to the group action we have the unique momentum map J f L : Q Q g an it hols: Theorem 1.3 (force iscrete Noether s theorem). Consier a iscrete Lagrangian system L : Q Q R with iscrete control forces f +,f : Q Q U s T Q such that the iscrete Lagrangian is invariant uner the lift of the (left or right) action φ : G Q Q an f u k,ξ Q Q =for all ξ g an u k U s,k {,...,N 1}. Then the iscrete Lagrangian momentum map J f L : Q Q g will be preserve by the iscrete Lagrangian evolution map, such that J f L F u k,u k+1 L = J f L. 1.3. The iscrete vs. the continuous Lagrangian systems In this section, we relate the continuous an the iscrete Lagrangian system. First, along the lines of [64], we efine expressions for the iscrete mechanical objects that exactly reflect the continuous ones. Base on the exact iscrete expressions, we etermine the orer of consistency concerning the ifference between the continuous an the iscrete mechanical system. Exact iscrete Lagrangian an forcing. Given a regular Lagrangian L : TQ R an a Lagrangian control force f L : TQ U T Q, we efine the exact iscrete Lagrangian L E : Q Q R R an the exact iscrete control forces f E+,f E : Q Q L ([,h],u) R T Q to be k= L E (q,q 1,h)= h L (q(t), q(t)) t, f E+ (q,q 1, u,h)= f E (q,q 1, u,h)= h h f L (q(t), q(t),u(t)) q(t) q 1 t, (1.14a) f L (q(t), q(t),u(t)) q(t) q t, (1.14b) with u k L ([kh, (k +1)h],U)anwhereq :[,h] Q is the solution of the force Euler-Lagrange equations (1.3) with control function u :[,h] U for L an f L satisfying the bounary conitions q() = q

DISCRETE MECHANICS AND OPTIMAL CONTROL: AN ANALYSIS 329 F u,u1 L E (q,q 1 ) (q 1,q 2 ) F f L E,u F f+ L E,u F f L E,u1 F f+ L E,u1 (q,p ) (q 1,p 1 ) (q 2,p 2 ) F u =(F u L E H )h u F 1 =(F u1 L E H )h F L F L F L (q, q ) (q 1, q 1 ) (q 2, q 2 ) (F u L )h (F u1 L )h Figure 1. Corresponence between the exact iscrete Lagrangian an forces an the continuous force Hamiltonian flow. an q(h) =q 1. Observe, that the exact iscrete control forces epen on an entire control path in contrast to the continuous control forces. Consequently, the exact force iscrete Legenre transforms are given by F f+ L E (q,q 1, u,h)= ( q 1,D 2 L E (q,q 1,h)+f E+ (q,q 1, u,h) ), F f L E (q,q 1, u,h)= ( q, D 1 L E (q,q 1,h) f E (q,q 1, u,h) ). As in Section 1.2 F f± L E,u k (q k,q k+1 )anf E,u k,± (q k,q k+1 ) enote the exact iscrete forces an the exact force iscrete Legenre transforms for a fixe u k L ([kh, (k +1)h],U). As in [64] (Lem. 1.6.2) by taking the presence of control forces into account we obtain that a regular Lagrangian L an the corresponing exact iscrete Lagrangian L E have Legenre transforms relate by Rapie Note F f+ L E (q,q 1, u,h)=fl (q,1 (h), q,1 (h)), F f L E (q,q 1, u,h)=fl (q,1 (), q,1 ()), for sufficiently small h an close q,q 1 Q. Hereq,1 enotes the solution of the corresponing Euler-Lagrange equations with q() = q,q(h) =q 1. This also proves that exact iscrete Lagrangians are automatically regular. Combining this result with the relations (1.12) gives the commutative iagram shown in Figure 1 for the exact iscrete Lagrangian an forces. The iagram also clarifies the following observation, that was alreay prove in [64] (Thm. 1.6.3) for unforce systems an can now be establishe for the force case as well: consier the pushforwar of both, the continuous Lagrangian an forces an their exact iscrete Lagrangian an iscrete forces to T Q, yieling a force Hamiltonian system with Hamiltonian H an a force iscrete Hamiltonian map F u k L E, respectively. Then, for a sufficiently small time step h R, the force Hamiltonian flow map equals the pushforwar iscrete Lagrangian map: (F u H )h = u F. L E Orer of consistency. In the previous paragraph we observe that the exact iscrete Lagrangian an forces generate a force iscrete Hamiltonian map that exactly equals the force Hamiltonian flow of the continuous system. Since we are intereste in using iscrete mechanics to reformulate optimal control problems, we generally o not assume that L an L or H are relate by (1.14). Moreover, the exact iscrete Lagrangian an exact iscrete forces are generally not computable. In this section we etermine the error we obtain by using iscrete approximations for the Lagrangian an the control forces an make use of the concept of variational error analysis (introuce in [64]). In this context, it is consiere how closely a iscrete Lagrangian matches the exact iscrete Lagrangian given by the action. Thus, a given iscrete Lagrangian L is of orer r, ifforafixe

33 S. OBER-BLÖBAUM ET AL. Rapie Not curve u L ([, T],U) there exist an open subset V v TQ with compact closure an constants C v > an h v > such that L (q(),q(h),h) L E (q(),q(h),h) C v h r+1 (1.15) for all solutions q(t) of the force Euler-Lagrange equations with initial conition (q, q ) V v an for all h h v, were we assume Q to be a norme vector space equippe with the norm. For force systems, we aitionally take into account how closely the iscrete forces match the exact iscrete forces: Analogously, we efine that a given iscrete force f u,± an the iscrete Legenre transforms F + L u an F L u of a iscrete Lagrangian L are of orer r, ifwehave f u,± (q(),q(h),h) f E,u,± (q(),q(h),h) C w h r+1 an F ± L u (q(),q(h),h) F± L E,u (q(),q(h),h) C f h r+1 for existing open subsets V w,v f TQ with compact closure an constants C w,c f > anh w,h f > as above, respectively, an with fixe u U s an u L ([,h],u). To give a relationship between the orers of a iscrete Lagrangian, iscrete forces, the force iscrete Legenre transforms, an their force iscrete Hamiltonian maps, we efine that L 1 is equivalent to L2 if their iscrete Hamiltonian maps are equal. For the force case, we say analogously, that for fixe u k U s the iscrete pair (L 1,fu k,1 ) is equivalent to the iscrete pair (L 2,fu k,2 ) if their force iscrete Hamiltonian maps are equal, such that F u k = F u L 1 k. With F u L 2 k = F f+ L u k,1 L 1 (F f L u k,1 ) 1, it follows that if (L 1,fu k,1 )an(l 2,fu k,2 ) are equivalent, then their force iscrete Legenre transforms are equal. Thus, equivalent pairs of iscrete Lagrangians an control forces generate the same integrators. As in [64] (Thm. 2.3.1) an [72] wegetthe following equivalent statements: (i) (ii) (iii) the force iscrete Hamiltonian map for (L,f u k,± )isoforerr; the force iscrete Legenre transforms of (L,f u k,± ) are of orer r; (L,f u k,± ) is equivalent to a pair of iscrete Lagrangian an iscrete forces, both of orer r. (1.16) Note that, given a iscrete Lagrangian an iscrete forces, their orer can be calculate by expaning the expressions for L (q(),q(h),h)anf u k,± in a Taylor series in h an comparing these to the same expansions for the exact Lagrangian an the exact forces, respectively. If the series agree up to r terms, then the iscrete objects are of orer r. 2.1. The continuous setting 2. Optimal control of a mechanical system On the configuration space Q we consier a mechanical system escribe by a regular Lagrangian L : TQ R. Aitionally, assume that a Lagrangian control force acts on the system an is efine by a map f L : TQ U T Q with f L :(q, q, u) (q, f L (q, q,u)) an u :[, T] U, the time-epenent control parameter. Note that the Lagrangian control force may inclue both issipative forces within the mechanical system an external control forces resulting from actuators steering the system. The Lagrangian optimal control problem. We now consier the following optimal control problem: During thetimeinterval[,t], the mechanical system escribe by the Lagrangian L istobemoveonacurveq from an initial state (q(), q()) = (q, q ) TQ to a final state. The motion is influence via a Lagrangian control force f L with control parameter u such that a given objective functional J(q, u) = T C (q(t), q(t),u(t)) t +Φ(q(T), q(t)) (2.1) is minimize. Here C : TQ U R an Φ : TQ R (Mayer term) are continuously ifferentiable cost functions. The final state (q(t), q(t)) is require to fulfil a constraint r(q(t), q(t),q T, q T )=withr : TQ TQ R nr

DISCRETE MECHANICS AND OPTIMAL CONTROL: AN ANALYSIS 331 an (q T, q T ) TQ given. The motion of the system is to satisfy the Lagrange- Alembert principle, which requires that δ T L (q(t), q(t)) t + T f L (q(t), q(t),u(t)) δq(t)t = (2.2) for all variations δq with δq() = δq(t) =. In many cases, one encounters aitional constraints on the states an controls given by h(q(t), q(t),u(t)) withh : TQ U R n h,wherev for vectors V R n hols componentwise. To summarize, we are face with the following: Problem 2.1 (Lagrangian optimal control problem (LOCP)). subject to δ T min J(q, u) (2.3a) q C 1,1 ([,T],Q),u L ([,T],U) L(q(t), q(t)) t + T f L (q(t), q(t),u(t)) δq(t) t =, (q(), q()) = (q, q ), (2.3b) (2.3c) h(q(t), q(t),u(t)), t [, T], (2.3) r(q(t), q(t),q T, q T )=. (2.3e) The interval length T may either be fixe, or appear as egree of freeom in the optimization problem. Definition 2.2. Acurve(q, u) C 1,1 ([, T],Q) L ([, T],U)isfeasible, if it fulfills the constraints (2.3b) (2.3e). The set of all feasible curves is the feasible set of Problem 2.1. A feasible curve (q,u )isanoptimal solution of Problem 2.1, ifj(q,u ) J(q, u) for all feasible curves (q, u). An feasible curve (q,u )isalocal optimal solution, ifj(q,u ) J(q, u) in a neighborhoo of (q,u ). The function q is calle (locally) optimal trajectory, anu is the (locally) optimal control. The Hamiltonian optimal control problem. We now formulate the problem using the Hamiltonian variant for the system ynamics. This is equivalent to the Lagrangian formulation as we have seen in Section 1.1 on the Legenre transform with forces. For a set R = {(q, q) TQ g(q, q) } etermine via aconstraintg : TQ R ng on TQ we obtain the corresponing set in the cotangent bunle as R = {(q, p) T Q g(q, p) } with g = g (FL) 1. Analogously, we efine J, C, Φ, h, an r such that the optimal control problem in the Hamiltonian formulation reas as follows: Rapie Note Problem 2.3 (Hamiltonian optimal control problem (HOCP)). min J(q, p, u) = q,p,u T C(q(t),p(t),u(t)) t + Φ(q(T),p(T)) (2.4a) subject to q(t) = p H(q(t),p(t)), (2.4b) ṗ(t) = q H(q(t),p(t)) + f H (q(t),p(t),u(t)), (2.4c) (q(),p()) = (q,p ), (2.4) h(q(t),p(t),u(t)), t [, T] (2.4e) r(q(t),p(t),q T,p T ) =, (2.4f) where (q T,p T )=FL(q T, q T ), p() = D 2 L(q(), q()), p = D 2 L(q, q )an Φ =Φ (FL) 1 etc., an the minimization is over q C 1,1 ([, T],Q)=W 2, ([, T],Q), p W 1, ([, T],T q Q)anu L ([, T],U).

332 S. OBER-BLÖBAUM ET AL. Necessary optimality conitions. In this paragraph, we erive necessary conitions for the optimality of asolution(x,u )withx =(q, p) toproblems2.1 an 2.3. Since we nee (x,u ) W 2, ([, T],T Q) W 1, ([, T],U) later on we use the same smoothness assumption for the erivation. In aition, we restrict ourselves to the case of problems with the controls pointwise constraine to the (nonempty) set U = {u R nu h(u) } an fixe final time T. With f(x, u) =( p H(q, p), q H(q, p)+f H (q, p, u)) we can rewrite (2.4) as min x W 2, ([,T],T Q) u W 1, ([,T],U) J(x, u) = T C(x(t),u(t)) t +Φ(x(T)) (2.5) subject to ẋ = f(x, u), x() = x, u(t) U for t [, T] an r(x(t),x T ) =. Necessary conitions for optimality of solution trajectories η( ) =(x( ),u( )) can be erive base on variations of an augmente cost function, the Lagrangian of the system: L(η, λ) = T C(x(t),u(t)) + λ T (t) (ẋ f(x(t),u(t))) t +Φ(x(T)), (2.6) Rapie Not where λ W 2, ([, T], R nx )istheajoint variable or the costate. Apoint(η,λ )isasale point of (2.6), if L(η, λ ) L(η,λ ) L(η,λ) for all η an λ. The function H(x, u, λ) := C(x, u)+λ T f(x, u) is calle the Hamiltonian of the optimal control problem. When setting variations of L with respect to η an λ to zero, the resulting Euler-Lagrange equations provie necessary optimality conition for the optimal control problem (2.5). Formally, one obtains the following celebrate theorem (cf. [74]): Theorem 2.4 (Pontryagin maximum principle). Let (x,u ) W 2, ([, T],T Q) W 1, ([, T],U) be an optimal solution to (2.5). Then there exists a function λ W 2, ([, T], R nx ) anavectorα R nr such that an λ solves the following initial value problem: H (x (t),u (t),λ(t)) = max H (x(t),u,λ(t)) t [, T], (2.7a) u U 2.2. The iscrete setting λ(t) = x Φ(x (T)) x r ( x (T),x T) α, λ = x H (x,u,λ). (2.7b) For the numerical solution we nee a iscretize version of Problem 2.1. To this en we formulate an optimal control problem for the iscrete mechanical system escribe by iscrete variational mechanics introuce in Section 1.2. In Section 2.3 we show how the optimal control problem for the continuous an the iscrete mechanical system are relate. To obtain a iscrete formulation, we replace each expression in (2.3) byits iscrete counterpart in terms of iscrete variational mechanics. As escribe in Section 1.2, we replace the state space TQ of the system by Q Q an a path q :[, T] Q by a iscrete path q : {,h,2h,...,nh= T } Q with q k = q (kh). Analogously, the continuous control path u :[, T] U is replace by a iscrete control path u :Δ t U (writing u k =(u (kh + c l h)) s l=1 U s ). The iscrete Lagrange- Alembert principle. Base on this iscretization, the action integral in (2.2) is approximate on a time slice [kh, (k +1)h] bytheiscrete Lagrangian L : Q Q R, L (q k,q k+1 ) (k+1)h L(q(t), q(t)) t, an likewise the virtual work by the left an right iscrete forces, f kh k δq k + f + k δq k+1 (k+1)h f kh L (q(t), q(t),u(t)) δq(t)t, wheref k,f+ k T Q. As introuce in equation (1.8), the iscrete version of the Lagrange- Alembert principle (2.2) requires one to fin iscrete paths {q k } N k= such that for all variations {δq k } N k= with δq = δq N =, one has the iscrete Lagrange- Alembert principle (1.8), or, equivalently, the force iscrete Euler-Lagrange equations (1.9).

DISCRETE MECHANICS AND OPTIMAL CONTROL: AN ANALYSIS 333 Bounary conitions. In the next step, we nee to incorporate the bounary conitions q() = q, q() = q an r(q(t), q(t),q T, q T ) = into the iscrete escription. Those on the configuration level can be use as constraints in a straightforwar way as q = q. However, since in the present formulation velocities are approximate in a time interval [t k,t k+1 ] (as oppose to an approximation at the time noes), the velocity conitions have to be transforme to conitions on the conjugate momenta. These are efine at each time noe using the iscrete Legenre transform. The presence of forces at the time noes has to be incorporate into that transformation leaing to the force iscrete Legenre transforms F f L an F f+ L efine in (1.1). Using the stanar Legenre transform FL : TQ T Q, (q, q) (q, p) =(q, D 2 L(q, q)) leas to the iscrete initial constraint on the conjugate momentum D 2 L ( q, q ) + D 1 L (q,q 1 )+f (q,q 1,u )=. (2.8) As shown in the previous section, we can transform the bounary conition from a formulation with configuration an velocity to a formulation with configuration an conjugate momentum. Thus, instea of consiering a iscrete version of the final time constraint r on TQ we use a iscrete version of the final time constraint r on T Q. We efine the iscrete bounary conition on the configuration level to be r : Q Q U s TQ R nr, r ( qn 1,q N,u N 1,q T, q T) = r ( F f+ L (q N 1,q N,u N 1 ), FL ( q T, q T)), i.e. we use (q N,p N )=F f+ L (q N 1,q N,u N 1 )an(q T,p T )=FL(q T, q T ), that is p N = D 2 L (q N 1,q N )+ f + (q N 1,q N,u N 1 )anp T = D 2 L(q T, q T ). Notice that for the simple final velocity constraint q(t) q T =, we obtain for the transforme conition on the momentum level r(q(t),p(t),q T,p T )=p(t) p T the iscrete constraint D 2 L(q T, q T )+D 2 L (q N 1,q N )+f + (q N 1,q N,u N 1 )=, (2.9) which together with equation (2.8) constitute the bounary constraints on momentum level. Discrete path constraints. Oppose to the final time constraint we approximate the path constraint in (2.3) on each time interval [t k,t k+1 ] rather than at each time noe. Thus, we maintain the formulation on the velocity level an replace the continuous path constraint h(q(t), q(t),u(t)) byaiscrete path constraint h : Q Q U s R sn h which suitably approximate the continuous constraint pointwise (see Sect. 2.4) with h (q k,q k+1,u k ), k =,...,N 1. Discrete objective function. Similar to the Lagrangian we approximate the objective functional in (2.1) on the time slice [kh, (k +1)h] byc (q k,q k+1,u k ) (k+1)h C(q(t), q(t),u(t)) t. Analogously to the final time kh constraint, we approximate the final conition via a iscrete version Φ : Q Q U s R yieling the iscrete objective function J (q,u )= N 1 k= C (q k,q k+1,u k )+Φ (q N 1,q N,u N 1 ). The iscrete optimal control problem. In summary, after performing the above iscretization steps, one is face with the following iscrete optimal control problem. Problem 2.5 (iscrete Lagrangian optimal control problem). Rapie Note min (q,u ) (q,u ) P (Q) P (U) (2.1a) subject to q = q, (2.1b) D 2 L ( q, q ) + D 1 L (q,q 1 )+f =, (2.1c) D 2 L (q k 1,q k )+D 1 L (q k,q k+1 )+f + k 1 + f k =, k =1,...,N 1, (2.1) h (q k,q k+1,u k ), k =,...,N 1, (2.1e) ( r qn 1,q N,u N 1,q T, q T) =. (2.1f)

334 S. OBER-BLÖBAUM ET AL. Recall that the f ± k are epenent on u k U s. To incorporate a free final time T as in the continuous setting, the step size h appears as a egree of freeom within the optimization problem. However, in the following formulations an consierations we restrict ourselves to the case of fixe final time T an thus fixe step size h. Special case: fixe bounary conitions. Consier the special case of a problem with fixe initial an final configuration an velocities an without path constraints, i.e. consier Problem 2.1 without the constraint (2.3) an r(q(t), q(t)) = (q(t) q T, q(t) q T ). A straightforwar way to erive initial an final constraints for the conjugate momenta rather than for the velocities from the variational principle irectly is state in the following proposition: Proposition 2.6. With (q,p )=FL(q, q ) an (q T,p T )=FL(q N, q N ) equations (2.3b), (2.3c) an (2.3e) are equivalent to the following principle with free initial an final variation an with augmente Lagrangian ( ) T T δ L(q(t), q(t)) t + p (q() q ) p T (q(t) q T ) + f L (q(t), q(t),u(t)) δq(t) t =. (2.11) Proof. Variations of (2.3b) with respect to q an zero initial an final variation δq() = δq(t) = together with (2.3c), (2.3e) yiel Rapie Not t q L(q(t), q(t)) q L(q(t), q(t)) = f L(q(t), q(t),u(t)), q() = q, q() = q,q(t) = q T, q(t) = q T. (2.12a) (2.12b) On the other han variations of (2.11) with respect to q an λ =(p,p T ) with free initial an final variation lea to t q L(q(t), q(t)) q L(q(t), q(t)) = f L(q(t), q(t),u(t)), (2.13a) q() = q,q(t) = q T, L(q(t), q(t)) q t= = p T, L(q(t), q(t)) q t=t = p. (2.13b) The Legenre transform applie to the velocity bounary equations in (2.12b) gives the corresponing momenta bounary equations (2.13b). On the iscrete level we erive the optimal control problem for fixe initial an final configurations an velocities in an equivalent way. Thus, we consier the iscrete principle with iscrete augmente Lagrangian δ ( N 1 k= L (q k,q k+1 )+p ( q q ) p T ( q N q T)) + N 1 which, with free initial an final variation δq an δq N, respectively, is equivalent to L (q k,q k+1 )+ N 1 δ k= N 1 k= k= [ f k δq k + f + k δq k+1] =, (2.14) [ f k δq k + f + k δq k+1] =, (2.15a) q = q,q N = q T,p + D 1 L (q,q 1 )+f =, pt + D 2 L (q N 1,q N )+f + N 1 =, (2.15b) where the thir an fourth equations in (2.15b) are exactly the iscrete initial an final velocity constraints erive in the remark containing equation (2.9) withp = D 2 L(q, q )anp T = D 2 L(q T, q T ). We note that this erivation of the iscrete initial an final conitions irectly gives the same formulation that we foun before by first transforming the bounary conition on the momentum level T Q an then formulating the corresponing iscrete constraints on Q Q U s.

DISCRETE MECHANICS AND OPTIMAL CONTROL: AN ANALYSIS 335 Necessary optimality conitions. The system (2.1) results in a constraine nonlinear optimization problem, also calle a nonlinear programming problem, that is an objective function has to be minimize subject to algebraic equality an inequality constraints. Let ξ be the set of parameters introuce by the iscretization of an infinite imensional optimal control problem. Then, the nonlinear programming problem (NLP) to be solve is min ϕ(ξ), subject to a(ξ) =,b(ξ), (2.16) ξ where ϕ : R n R, a: R n R m, an b : R n R p are continuously ifferentiable. We briefly summarize some terminology: A feasible point is a point ξ R n that satisfies a(ξ) =anb(ξ). A (strict) local minimum of (2.16) is a feasible point ξ with ϕ(ξ ) ϕ(ξ) (ϕ(ξ ) <ϕ(ξ)) for all feasible points ξ in a neighborhoo of ξ. Active inequality constraints b act (ξ) at a feasible point ξ are those components b j (ξ) ofb(ξ) withb j (ξ) =. Subsuming the equality constraints an the active inequalities at a point ξ (known as active set) in a combine vector function gives the active constraints as ã(ξ) :=(a(ξ),b act (ξ)). The active set may be ifferent at ifferent feasible points. Feasible points ξ that satisfy the conition that the Jacobian of the active constraints, ã(ξ) T, has full rank are enote as regular points. To investigate local optimality in the presence of constraints, we introuce the Lagrangian multiplier vectors λ R m an μ R p, that are also calle ajoint variables, an we efine the Lagrangian function L by L(ξ,λ,μ) :=ϕ(ξ) λ T a(ξ) μ T b(ξ). (2.17) The following variant of the Karush-Kuhn-Tucker necessary conitions for local optimality of a point ξ have been erive first by Karush in 1939 [41] an inepenently by Kuhn an Tucker in 1951 [49]. For brevity, we restrict our attention to regular points only. Theorem 2.7 (Karush-Kuhn-Tucker conitions (KKT)). If a regular point ξ R n is a local optimum of the NLP problem (2.16), then there exist unique Lagrange multiplier vectors λ R m an μ R p such that the triple (ξ,λ,μ ) satisfies the following necessary conitions: ξ L(ξ,λ,μ )=, a(ξ )=, b(ξ ), μ, μ j b j (ξ )=, j =1,...,p. Atriple(ξ,λ,μ ) that satisfies these conitions is calle a Karush-Kuhn-Tucker point (KKT point). Rapie Note 2.3. The iscrete vs. the continuous problem This section gives an interpretation of the iscrete problem as an approximation to the continuous one. In aition, we ientify certain structural properties that the iscrete problem inherits from the continuous one. We etermine the consistency orer of the iscrete scheme an establish a result on the convergence of the iscrete solution as the step size goes to zero. The place of DMOC amongst solution methos for optimal control problems. In Figure 2 we present schematically ifferent iscretization strategies for optimal control problems: In an inirect metho, starting with an objective function an the Lagrange- Alembert principle we obtain via two variations (the first for the erivation of the Euler-Lagrange equations an the secon for the erivation of the necessary optimality conitions) the Pontryagin maximum principle. The resulting bounary value problem is then solve numerically, e.g. by graient methos [11,16,17,43,7, 81], multiple shooting [7,12,18,24,32,42] or collocation [1,2,19]. In a irect approach, starting form the Euler-Lagrange equations we irectly transform the problem into a restricte finite imensional optimization problem by iscretizing the ifferential equation. Common methos like e.g. shooting [48], multiple shooting [8], or collocation methos [82], rely on a irect integration of the associate orinary ifferential equations or on its fulfillment at certain gri points (see also [3,76] for an overview of the current state of the art). The resulting finite imensional nonlinear constraine optimization problem can be solve by stanar nonlinear optimization techniques

336 S. OBER-BLÖBAUM ET AL. Rapie Not Figure 2. Optimal control for mechanical systems: the orer of variation an iscretization for eriving the necessary optimality conitions. like sequential quaratic programming [31,75]. Implementations are foun in software packages like DIRCOL [83], SOCS [4], or MUSCOD [54]. In the DMOC approach, rather than iscretizing the ifferential equations arising from the Lagrange Alembert principle, we iscretize in the earliest stage, namely alreay on the level of the variational principle. Then, we consier variations only on the iscrete level to erive the restricte optimization problem an its necessary optimality conitions. This approach erive via the concept of iscrete mechanics leas to a special iscretization of the system equations base on variational integrators, which are ealt with in etail in [64]. Thus, the iscrete optimal control problem inherits special properties exhibite by variational integrators. In the following, we specify particular important properties an phenomena of variational integrators an try to translate their meaning into the optimal control context. Preservation of momentum maps. If the iscrete system, obtaine by applying variational integration to a mechanical system, inherits the same symmetry groups as the continuous system, the corresponing iscrete momentum maps are preserve. For the force case the same statement hols, if the forcing is orthogonal to the group action (see Thm. 1.3). On the one han, this means for the optimal control problem, that if the control force is orthogonal to the group action, our iscretization leas to a iscrete system, for which the corresponing momentum map is preserve. On the other han, in the case of the forcing not being orthogonal to the group action, the force iscrete Noether s theorem provies an exact coherence between the change