Optimization of a point-mass walking moel using irect collocation an sequential quaratic programming Chris Dembia June 5, 5
Telescoping actuator y Stance leg Point-mass boy m (x,y) Swing leg x Leg uring stance F(t) F(t) l(t) F(t) mg Boy uring stance mg Boy uring flight Figure : The point mass moel of walking from [SR6]. a) Some possible gaits b) Inverte penulum walk c) Impulsive run ) Hybri intermeiate gait: penular run Stance Flight Inverte penulum Bounce Inverte penulum Flight Flight l max Bounce lmax Figure : (a) Possible perioic gaits for the point mass moel. (b-) Optimal gaits for the point mass moel, epening on the prescribe spee an step step length of the gait [SR6]. Introuction It is possible to uncover some funamental principles of human locomotion with a moel that consists of only a point mass an a massless telescoping (compressible) leg. Srinivasan an Ruina foun that such a moel chooses to walk if aske to move at slow spees, an to run if aske to move at high spees [SR6]. The moel they use is shown in Figure. The moel interacts with its environment via a telescoping linear actuator (its leg). The motion consists of two phases: a stance phase, uring which the stance leg is in contact with the groun; an a swing phase uring which the point mass is in free flight. Specifically, their moel chose between three istinct gaits, epening on the spee an step length of the gait: an inverte penulum walk, an impulsive run, an a penular run; these are shown in Figure. Srinivasan an Ruina foun these gaits by solving an optimal control problem in which the objective was to maximize the energy efficiency of the gait. The parameters in the optimization consiste of the initial state of the moel an the force applie by the actuator throughout the stance phase. To evaluate the objective function, they performe a forwar simulation (integration) of the moel s equations of motion, starting from the initial state. Their metho is calle single shooting; the optimizer chooses an initial state an shoots to the final state by integrating. This metho is typically challenging for optimizers, since the motion of a nonlinear ynamical system can be very sensitive to the initial state. Another
approach is to iscretize the state of the moel through time, an to enforce the ynamics at these time points through explicit constraints in the optimization problem. This metho is calle irect collocation [Bet]. When formulate via irect collocation, optimal control problems are far less sensitive to variations in the parameters. Direct collocation problems have many more parameters than single shooting problems, but are extremely sparse. In this work, we replicate the work of Srinivasan an Ruina using irect collocation instea of single shooting. We also solve the problem with a suite of ifferent cost functions, an explore the effect of these ifferent costs on the resulting motion. The problem Here, we escribe the optimal control problem that Srinivasan an Ruina [SR6] solve. The variables consist of the moel s state s(t) = (x(t), y(t), ẋ(t), ẏ(t)), where (x(t), y(t)) is the position of the point mass; the force applie to the groun by the leg F (t); an the uration of the stance phase t s. The step length, spee v, an step uration t step = /v are specifie. The leg length l(t) = x (t) + y (t) has a maximum value l max uring the stance phase, an the leg also has a finite strength. The motion must also be perioic. Formally, the optimal control problem is minimize C = tstep E(s(t), F (t)) t mg cost subject to mẍ = F x/l t t s stance ynamics mÿ = F y/l mg t t s stance ynamics mẍ = t s < t t step swing ynamics mÿ = mg t s < t t step swing ynamics l(t) l max t t s leg length F (t) t t s leg strength < t s t step stance uration x(t step ) x() = istance travele y() y(t step ) ẋ() = ẋ(t step ). perioicity ẏ() ẏ(t step ) () The point mass has mass m, an g is the acceleration ue to Earth s gravity on Earth s surface. For the integran cost E, Srinivasan an Ruina use the positive work performe by the leg, E = [F (t) l(t)] + ; with this E, then C is the cost of transport, which is a imensionless measure of the energetic cost of traveling a certain istance. In their work, the ynamics constraints were enforce via forwar integration. In this work, we use m = 5kg, g = 9.8m/s, l max =.5m, = N. Different gaits are foun by varying the nonimensional step length D = /l max an spee V = v/ gl max. 3
. Methos Direct collocation We iscretize Problem into T step mesh points t, t,..., t Tstep = t step. The transition from stance to swing occurs at time inex i = T s ; that is, t s = t Ts T step. In orer to express the secon orer ynamics of the system using first orer ifferential equations, we introuce the variables w = ẋ an z = ẏ for the horizontal an vertical spee of the point mass. The iscretization yiels state variables s, s,..., s Tstep (with s i = (x i, y i, w i, z i )) an leg forces F, F,..., F Ts. We enforce the ifferential constraints using a numerical metho for solving orinary ifferential equations, namely the implicit Euler metho: for a ifferential equation ẏ = f(y, t), we obtain y via y i+ = y i +h i f(y i+, t i+ ). We use an implicit integrator because they are unconitionally stable (the solution won t blow up ). The resulting finite-imensional optimization problem is minimize C = T h i [E(s i+, F i+ ) + E(s i, F i )] i= subject to x i+ x i = h i w i+ i =,,..., T step y i+ yi = h i z i+ i =,,..., T step x i+ F i+ w i+ w i = h i i =,,..., T s ( m l i+ ) Fi+ y i+ z i+ z i = h i g i =,,..., T s m l i+ w i+ w i = i = T s,..., T step z i+ z i = h i g i = T s,..., T step li lmax i =,,..., T s F i i =,,..., T s < t s t step x Tstep x = y y Tstep w = w Tstep. z z Tstep () The scalars h i = t i+ t i are the integration step sizes. The performance cost is now approximate by the iscretize C, which is compute using the trapezoial rule. This problem has a total n = 4T step + T s + variables (4 states, the leg force uring stance, an t s ) an 4(T step ) ynamics constraints. For all results shown in this report, we use T s =, T step =. The value of T step is chosen to be sufficiently large so that the error in the ODE solution is below some tolerance, typically 3. For all of the constraints, the parameters at only one or two of the mesh points is present; therefore, this problem is very sparse. Since E may be non-convex, an since the thir an fourth constraints in Problem are non-affine, this problem is not convex. 4
Sequential quaratic programming We solve Problem using sequential quaratic programming [GMS94]. At each iteration k of the metho, we approximate the problem as a quaratic program about our current iterate q (k). The solution of this QP, q (k), is use to compute a search irection q (k) = q (k) q (k) along which to perform a line search. The result of this line search is the next iterate q (k+). Within the QP, the quaratic objective is q C (k) (q q(k) ) + (q q(k) ) qc (k) (q q(k) ), where q C (k) an qc (k) are the graient an Hessian evaluate at q (k), respectively. The only constraints that must be approximate for the QP are the thir an fourth from Problem. We efine the following: x i+ F i+ a i = w i+ w i h i, ( m l i+ Fi+ y i+ = z i+ z i h i b i m u i = (w i+, w i, x i+, y i+, F i+ ), v i = (z i+, z i, x i+, y i+, F i+ ). ) g, l i+ The quantities a i an b i are the resiuals in thir an fourth constraints, an u i an v i are the parameters containe in the thir an fourth constraints. The affine approximations to these constraints are a (k) i + ( ui a (k) i ) T (u i u (k) i ) = b (k) i + ( vi b (k) i ) T (v i v (k) i ) =. The graients in these equations are [ ( ( ui a i ) T F i+ x = h i+ i ) F i+ x i+ y i+ h m li+ 3 i [ l i+ ( m li+ 3 ( vi b i ) T F i+ x i+ y i+ F i+ y = h i h i+ m li+ 3 i ) m li+ 3 l i+ h i m h i m x i+ l i+ To solve the SQP, we use the SNOPT package [GMS5]. SNOPT is a package for solving large sparse non-convex problems, an approximates the Hessian using a BFGS upate. For the problems we explore, SNOPT requires between 5 to 35 iterations to converge; on a moern personal computer, this takes about minute. During the first few outer iterations, the QP solver takes potentially hunres of steps. After about 5 outer iterations, when the iterate is nearly feasible, the QP usually only requires two steps. Transcribing the problem The proceure of converting Problem into Problem is calle transcription. We use the PSOPT optimal control package to perform this transcription for us [Bec]. PSOPT also iteratively solves the problem on larger meshes until an ODE error tolerance is satisfie. 5 y i+ l i+ ] ].
b) Penular walk c) Impulsive run ) Penular run Leg-force F Leg-length L Inverte penulum e e e e τ step τ τ s τ step τ τ step τ Bounce Inverte Flight penulum Flight F' max F' max F' max τ τ τ τ s 3 Results Figure 3: Leg length an leg force for the three gaits prouce by [SR6]. Reproucing the results of Srinivasan an Ruina, 6 By varying the values of the non-imensional step length D an non-imensional spee V, Srinivasan an Ruina iscovere three istinct gaits (Figure ). Figure 3 shows the leg force F (t) an leg length l(t) for a specific instance of each of these three gaits. The penular walk emerges from small D an small V, the impulsive run emerges for all D (> ) with V above, an the hybri penular run emerges from D above for V <. We reprouce these results qualitatively using the methos presente in section.; the three gaits are shown in Figure 4. The force F is not smooth as a result of the non-smooth cost E. Notably, the penular walk oes not contain a swing phase, an the force for both penular gaits is U-shape. Exploring ifferent cost functions We explore three ifferent cost functions an analyze their effect on the resulting walking solution, similar to work one by [Sri]. For all three, we use D =.5 an V =.5. Minimize force over the gait cycle. With E = F (t), the solution minimizes the amount of force use throughout the whole motion. The solution shows that this causes the introuction of a swing phase that lasts as long as the stance phase. Minimize square leg force. The cost E = F (t) penalizes large forces. This objective might capture the esire to reuce joint loaing for those with joint pain. With this gait, the moel oes not change its height at all, an the leg force remains close to only what is necessary to counteract gravity; F (t) mg. Minimize gross work. The cost E = F (t) l(t) essentially allows the moel to propel itself using the energy absorbe from breaking. Inee, with this cost function, the net work over the gait is. The resulting motion is spring-like. 6
E =[F L] +, D =.5, V =.5 E =[F L] +, D =.5, V =.5 E =[F L] +, D =.5, V =.5.8 t step.8 t s t step.8 t step t step t s t step t step Figure 4: Three istinct gaits (left to right): penular walk, impulsive run, penular run. E =F, D =.5, V =.5 E =F, D =.5, V =.5 E =F L, D =.5, V =.5.8 t step.8 t step.6 t step t step t step t step Figure 5: Walking with ifferent costs (left to right): leg force, square leg force, gross work. 7
References [Bec] [Bet] V. M. Becerra. Solving complex optimal control problems at no cost with PSOPT. Proc. IEEE Multi-conference on Systems an Control, Yokohama, Japan, pages 39 396,. John Betts. Practical Methos for Optimal Control an Estimation using Nonlinear Programming. Society for Inustrial an Applie Mathematics, secon eition,. [GMS94] Philip E Gill, Walter Murray, an Michael A Sauners. Large-scale sqp methos an their application in trajectory optimization. In Computational optimal control, pages 9 4. Springer, 994. [GMS5] Philip E. Gill, Walter Murray, an Michael A. Sauners. Snopt: An SQP Algorithm For Large-Scale Constraine Optimization. SIAM Review, 47():99 3, 5. [SR6] [Sri] Manoj Srinivasan an Any Ruina. Computer optimization of a minimal bipe moel iscovers walking an running. Nature, 439(77):7 75, 6. Manoj Srinivasan. Fifteen observations on the structure of energy-minimizing gaits in many simple bipe moels. Journal of The Royal Society Interface, 8(54):74 98,. 8