Acrobot stable walking in Hybrid systems notation*

4 UKSim-AMSS 6th International Conference on Computer Modelling and Simulation Acrobot stable walking in Hybrid systems notation* Milan Anderle Institute of Information Theory and Automation Academy of Sciences of the Czech Republic Prague, Czech Republic Email: anderle@utia.cas.cz Sergej Čelikovský Institute of Information Theory and Automation Academy of Sciences of the Czech Republic Prague, Czech Republic Email: celikovs@utia.cas.cz Abstract The Acrobot walking is interesting and still challenging control problem. By virtue of an actuator location, the Acrobot is partially feedback linearizable up to order 3. This property is widely used both in a feedback tracking of a reference trajectory and in the reference trajectory design. Moreover, the walking of the Acrobot consists of continuoustime and discrete-time dynamics, therefore, the Acrobot belongs in a class of hybrid systems. The aim of this paper is to integrate continuous-time and discrete-time dynamics into a general model of hybrid systems and prove hybrid stability of the Acrobot walking by using both the already developed feedback control and the reference multi-step trajectory. Keywords-walking robots; hybrid systems; modelling; I. INTRODUCTION Hybrid dynamical systems combine continuous-time and discrete-time dynamics. Many real systems have this behaviour, e.g. switching systems or various types of mechanical systems. Despite the fact that, the discrete-time part of the hybrid system is hard to control if ever, it is essentially to take both dynamics of the system into account during a control or stability analysis, because, the discrete-time part could improve or worsen the stability of the whole system. Indeed, walking mechanisms and walking robots belong in a class of hybrid systems. One can find in the literature many examples of various kind of walking mechanisms. Underactuated walking robots, i.e. robots with less degree of freedom than number of actuators, form subclass of walking robots. The Acrobot is the simplest underactuated walking mechanism theoreticaly able to walk. It consists from two links and one actuator placed between the links. The Acrobot, alternatively also referred to as the Compass gait walker, have been studied extensively in the control area during past few decades. One of the most important issues in the study of walking robots is an efficient control of these systems in application to reliable and economic walking or running. There are many works in this direction. Dynamics of walking robots have been exploited in order to make passive walkers, which have limited capabilities and walk down a slope, or have used n actuators to execute *This work was supported by the Czech Science Foundation through the research grant P3//794. motion on a flat terrain 3, 4, 5. Recent state of the art is reflected in 6, 7. Dynamics of the Acrobot consists of both continuous-time and discrete-time dynamics, therefore, the Acrobot belongs in hybrid dynamical systems. The movement of the Acrobot with one leg in air is described by continuous-time dynamics whereas instantaneous change of angular velocities when one leg touches the ground is described by discrete-time dynamics. The control of the Acrobot during the discrete-time part, usually called impact, is almost impossible due to the fact that actuators, in general, are not able to generate impulsive forses in order to influence impulsive forses generated by the impact. However, a control of the Acrobot during its continuous-time part is interesting task due to nonlinear Acrobot s dynamics. The control of the Acrobot during one step, i.e. only the continuous-time dynamics is considered, was firstly done in 8 and extended in 9,,. The partial exact feedback linearization of the general n-link chain system is studied in 4. The efect of the impact was firstly taken into account in, 3. An analytical test of the Acrobot s walking stability was already done and presented in 3. However, the whole closed-feedback system in 3 was expected to be continuous. Nevertheless, in a real application, the controller is usually discontinuous. Therefore, in the current paper, the Acrobot s walking simulation and the stability test is done according typical sample-and-hold control approach. It means that the Acrobot is simulated in continuous way whereas the controller samples Acrobot s states and controller s output remains constant during the sampling period. In 4 is shown that the closed-loop system consist of continuous-time dynamics and sampled controller can be modelled as a hybrid system. The multi-step walking trajectory from and its tracking from 5 is used here under the sample-and-hold control approach in order to have a cyclic motion of the Acrobot which enable us to make a proof of the feedback tracking stability by method of Poincaré mappings. Numerical simulations of the first four Acrobot steps and a phase-plane trajectory of 5 Acrobot steps based on the 978--4799-493-6/4 $3. 4 IEEE DOI.9/UKSim.4.4 99

combination of the continuous-time system and sampled feedback controller are demonstrated as well. The paper is organized as follows. The next section describes intergration of the continuous and dicrete-time dynamics into a general model of hybrid systems. The section III briefly reminds previously developed results for the Acrobot s control. The section IV shows results of simulation and final section draws briefly some conclusions. II. LAGRANGIAN HYBRID SYSTEMS Lagrangian hybrid systems are typically represented by mechanical systems with a collision or, in other words, with an impact. In general, the impact causes a discontinuous change in velocities while positions remain unchanged. Walking robots, in our specific case, the Acrobot is a typical representant of Lagrangian hybrid systems, i.e. a mechanical system having continuous-time and discretetime dynamics. The continuous-time dynamics is described by differential equations whereas the discrete-time dynamics is described using an equation. Both dynamics are covered by a general hybrid system model in the form ẋ F(x,u), x C(x), () x + G(x,u), x D(x), () where x R, F(x) is a set-valued mapping, C is a subset of R, G(x) is a set-valued mapping, D is a subset of R and u is an input. The general model (,) describes wide variety of systems not only mechanical systems, but e.g. switching systems, hybrid system automata, discrete events in biological systems etc. The Acrobot model or the Compass-gait model is a special case of n-link chain with n actuators attached by one of its ends to a pivot point through an unactuated rotary joint. Similar mechanical systems are numerous: the single inverted pendulum, the pendubot, etc. Indeed, the Acrobot depicted in Figure has two degrees of freedom and one actuator placed between its rigid links, roughly speaking, the Acrobot has two legs with only one actuator placed between them. Acrobot walking consists of the continuous part, i.e. when one leg, usually called swing leg is in air and of the discrete part which occurs when the swing leg touches the ground. Only for completeness, the second leg which is in contact with the ground is usually called stance leg. A. Continuous-time dynamics of the Acrobot The continuous part, when the swing leg of the Acrobot is in air, is modelled by usual Lagrangian approach. The resulting Euler-Lagrange equation is ddt L q L q = u =, (3) τ d dt L q L q where u stands for vector of external controlled forces. The system (3) is the so-called underactuated mechanical y x q m τ q l l Figure. The Acrobot. system having the degree of the underactuation equal to one, 3. Moreover, the underactuated angle is at the pivot point. Equation (3) leads to a dynamic equation in the form m D(q) q +C(q, q) q + G(q)=u, (4) where D(q) is the inertia matrix which depends on variable q only, C(q, q) contain Coriolis and centrifugal terms, G(q) contain gravity terms and u stands for vector of external forces, see 6. For the Acrobot, these computations lead to a second-order nonholonomic constraint and a kinetic symmetry, i.e. the inertia matrix depends only on the second variable q. It is possible to rewrite the equation of motion of the Acrobot (4) in the following form q q = D (q)c(q, q) q D (q)g(q)+d (q) u, (5) after substitution x = q, x = q, x 3 = q, x 4 = q, the original Acrobot s equation of motion has following standard form of a nonlinear system ẋ = f (x)+g(x)u = F(x,u), (6) where x is state vector x =x x x 3 x 4 T. The form of f (x) and g(x) is obvious. The configuration of the Acrobot is described by generalized coordinates q and is bounded by an one-sided constraint of limitation that, in general, two solid bodies do not penetrate each other. In our case, the limitation means that the Acrobot s swing leg cannot goes under the ground, i.e. the height of the swing leg s end-point has to be h end poin (q) >. B. Discrete-time dynamics of the Acrobot When the swing leg of the Acrobot touches the ground, i.e. h end point (q) =, the impact occurs. The result of this

event is instantaneous jump in angular velocities q while angular positions q remain unchanged. The impact is modelled as a contact between two rigid bodies. Crucial in the impact mapping is extended inertia matrix D e (q e ). In order to obtain the matrix, it is necessary to extend the original model of the Acrobot (4) by Cartesin coordinates of the end of the stance leg x, y and apply the identical procedure to obtain the dynamic Acrobot s equation, especially the matrix D(q). There are different ways the impact can be modelled in the literature 7, 8, 9, 7. The impact is based on following equations D e q + e q e = Fext, (7) E (q e ) q + e =, (8) where D e (q e ) is extend inertia matrix, q + e (respectively, q e ) is extend angular velocity of the Acrobot just after (respectively, just before) impact, F ext is given by forces acting in the end of the swing leg and E (q e )= ϒ(q e) q e, where ϒ represents end point coordinates of the swing leg, i.e. x + l sinq ϒ = + l sin(q + q ). y + l cosq + l cos(q + q ) The right side of equation (7) corresponds to the contact impulse over the impact duration and it is accompanied by (8) which corresponds to condition of no rebound and no slipping of the swing leg. For more details see 7. The impact model from 9 is used here under hypotheses that imply conservation of the total angular momentum a) the impact results from the contact of the swing leg end with the ground b) the impact is instantaneous and results in no rebound and no slipping of the swing leg c) at the moment of impact the stance leg lifts from the ground without interaction d) the external forces during the impact can be represented by impulses e) actuators cannot generate impulses and hence can be ignored during impact f) the impulsive forces may result in an instantaneous change in the robot s velocities, but there is no instantaneous change in the robot s configuration. Following an identical development as in 9, the expression relating the angular velocity of the robot just before to just after the impact q + e and forces acting in the swing leg end point are given as a result of (7),(8), i.e. De (q e ) E (q e ) q + e E (q e ) F = De (q e ) q e. (9) At impact, it is assumed that the swing leg, respectively stance leg, becomes the new stance leg, respectively the new swing leg, and the Acrobot s coordinates q and q are relabeled. To do so, consider Figure where one can see a course of Acrobot s angles in the beginning and in the end of the step. Acrobot s angles in the end of the previous step are denoted q, q whereas Acrobot s angles in the beginning of the new step are denoted q, q. Using a trigonometric law, one can immediately see following dependence between angular positions in the end of the old step and angular positions in the beginning of the new step q = π q q, () q = π q. () Equations () and () represent the change of Acrobot s coordinates due to switch of the legs, i.e. they represent the relabeling of the stance and the swing leg. Furthemore, their time derivative q + = q + q+, () q + = q +, (3) represent the change of Acrobot s angular velocities q, q followed by the impact. As a consequence, equations () and (3) complement impact equation (9). Equations (,) and (,3) as a result of impact equation (9) formulate matrix G(x, u) in (). After substitution x = q, x = q, x 3 = q, x 4 = q, matrix G(x,u) has following form G(x,u)= π x x 3 x + x+ 4 π x 3 x + 4 (4) The notation x,4 + represents angular velocity after the impact. The next step is to develope a controller in order to maintain the Acrobot in a movement reminds a human walk. The controller and reference trajectory design will be described in the next section. III. CONTROL STRUCTURE OF HYBRID ACROBOT MODEL As was shown in previous section, the Acrobot consists of continuous-time and dicrete-time dynamics. Our task is q q -q q Figure. The definition of Acrobot s angles in the beginning, respectively in the end, of the step.

to control the Acrobot in a way resembling a human walk. It means, it is necessary to design a controller and a walking trajectory to be tracked. However, according one of assumptions of the impact model, the actuators cannot generate impulses and influence the impact, therefore, the controller is designed only for continuous-time part of a walking. At the impact it is necessary to reset a walking trajectory and track either the new step or a fixed point in order to keep the walking stability in the time domain. In other words, it is necessary to ensure that the Acrobot neither get ahead nor behing of the reference model. For this purpose, a supervisory control is used. A. Control of the Acrobot based on partial linearization The state feedback linearization method is based on a system transformation into a linear and controllable system thus the well known linear control method could be applied. However, class of nonlinear systems with this property is indeed tenuous, therefore, the method is used occasionally especially in a real aplication. Whereas, the partial feedback linearization method is based on a system transformation into a new system of coordinates that displays linear dependencies between some auxiliary output and new (virtual) input. However, each nonlinear system transformed into a system with inputoutput linear map has an unobservable dynamics called a zero dynamics. The stability of the zero dynamics has to be verified in order to control the nonlinear system in the input-output feedback linearizable form. In general, to use either the state feedback linearization or the partial feedback linearization method it is necessary to find a suitable output function h having an appropriate relative degree r. In 4 it was shown that if the generalized momentum conjugates to the cyclic variable is not conserved (as it is the case of Acrobot) then there exists a set of outputs that defines a one-dimensional exponentially stable zero dynamics. In other words, it means that it is possible to find a function y(q, q) with relative degree 3 that transforms the original 4 dimensional system (4) by a local coordinate transformation z = T (q, q) into the new input/output linear system which has 3 dimensional state plus unobservable nonlinear dynamics of dimension, i.e. the zero dynamics. In the case of the Acrobot, there exist two independent functions with relative degree 3 which transform the original nonlinear system of the Acrobot into the desired partial linearized form with one dimensional zero dynamics σ = L =(θ + θ + θ 3 cosq ) q + (5) q (θ + θ 3 cosq ) q, p = q + q + θ θ θ arctan (θ + θ ) 4θ3 ( ) θ + θ θ 3 tan q θ + θ + θ 3, (6) where model parametrs θ,θ,θ 3 are defined in 6. The stability of zero dynamics was investigated in 8, moreover, there was, as well, shown that using the set of functions with maximal relative degree, the following transformation ξ = T (q, q) : ξ = p, ξ = σ, ξ 3 = σ, ξ 4 = σ (7) can be defined. Notice, that by (5,6) and some straightforward but laborious computations the following relation holds ṗ = d (q ) σ, (8) where d (q ) is the corresponding element of the inertia matrix D(q) in (4). Applying (7), (8) to (4) we obtain Acrobot s dynamics in partial exact linearized form ξ = d (q ) ξ, ξ = ξ 3, ξ3 = ξ 4, ξ 4 = α(q, q)τ + β(q, q)=w, (9) with the new coordinates ξ and the input w being well defined wherever α(q, q). The new input w is a virtual input in ξ coordinates. It is necessary to recompute the virtual input w in ξ coordinates to the real input u in q, q coordinates before applying the virtual input on the real model of the Acrobot. Assume that an open loop control w r (t) generates a suitable reference trajectory in partial exact linearized coordinates (9). In other words, our task is to track the following reference system ξ r = d (qr )ξ, r r ξ = ξ3, r r ξ 3 = ξ4, r r ξ 4 = w r. () Denoting e := ξ ξ r and subtracting () from (9) and the use of the Taylor expansions on the first line one obtains ė = μ (t)e + μ (t)e + μ 3 (t)e 3 + o(e), ė = e 3, ė 3 = e 4, ė 4 = w w r, () where μ (t), μ (t), μ 3 (t) are known smooth time functions and are bounded from bellow and above. Functions μ,,3 (t) are defined in 8. Actually, by (3) σ = d dt L q = L q and therefore by the definition of the V (q) Lagrangian σ = q as D(q) D(q ) by definition of the inertia matrix. In other words, σ has relative degree, i.e. σ has the relative degree 3. Moreover, by the straightforward differentiation holds ṗ = d (q ) σ, i.e. ṗ has relative degree, i.e. p should have relative degree 3 as well.

Several different ways to stabilize error dynamics () were developed. Controllers are based on deeper knowledge of functions μ,,3 (t) including their time derivation, see, 3,, 5 or on range of μ,,3 (t) functions, see 9. In simulations of the hybrid Acrobot control, the controller from 5 is used. B. Multi-step walking trajectory In an algorithm was presented to desing a cyclic walking like trajectory which is crucial for hybrid exponentially stable multi step tracking of the reference trajectory. The cyclic trajectory is a trajectory which after the impact at the end of the swing phase gets into the exactly same initial conditions as at the beginning of the swing phase. It is clear that the key issue is to design proper initial velocities as the positions are naturally continuous even after impact and due to symmetricity of postures (the both legs of the Acrobot should have the same lenghts), the conditions on positions are automatically satisfied. Mathematically, the cyclic walking like trajectory is expressed as follows q (T + ) q (T q (T + = Φ ) Imp (q(t )) ) q () q (T =, () ) q () where Φ Imp (q) as the matrix realizing influence of the impact on angular velocities including their relabeling due to switching the swing and the stance leg, q (T ), q (T ) are velocities just before the impact, while q (T + ), q (T + ) are velocities just after the impact and re-labeling. C. Supervisory control for the Acrobot An typical example of hybrid controllers is a so called supervisory control. It is a common task consists either in selection among controllers according to current state of the system or in generating the reference trajectory based on current and desired next state of the system. Roughly speaking, the supervisory controller is another controller making decision either which controller use in feedback control of the system or which trajectory should be tracked. In our case the supervisory controller is implemented in a form of logic rules in order to maintain a periodic walking of the Acrobot. Therefore, the reference trajectory, generated by the reference model, is repeatedly restarted. Moreover, the controller tracks either the reference trajectory or a fixed point for a while in order to keep the Acrobot simultaneously with the reference one. IV. SIMULATIONS AND TEST OF HYBRID STABILITY Stability of the sample-and-hold control approach was tested numericaly by method of Poincaré mappings. Moreover, phase plane of more than 5 steps is shown. The stability analysis of the walking is done numericaly by the Poincaré method. In Figures 3, 4 one can see a periodic motion of the Acrobot over four steps in q and q coordinates. The hybrid system is stable if and only if the fixed point p of the discrete-time system on the Poincaré section is stable. For more details about Poincaré method, see 6. The corresponding simulations of the reference trajectory tracking with an initial error in angular positions and angular velocities are shown in Figures 3, 4. One can easily see the convergence to the cyclic motion of the real Acrobot after four steps. Figure 5 shows phase-plane plots of variables q and q. The convergence towards a periodic motion is clearly seen from simulation 5 steps. Thin curves correspond to motion of the real Acrobot in the first four steps which are plotted in detail in Figures 3, 4. Thin curves convergence towards a periodic motion, the thick curve. The straight line corresponds to the impact phase, where states of the Acrobot change instantaneously. q rad 4 3.5 3.5.5.5 q q.5.5.5.5 Time s Figure 3. Angular positions q, q and references (dotted line) for 4 steps walking using the sampling controller in feedback tracking. V. CONCLUSIONS This paper presented continuous-time and discrete-time dynamics of the Acrobot in hybrid systems notation. The supervisory control of the Acrobot was introduced and presented. Numerical stability test of Acrobot walking controlled by sampled feedback controller was done. Sample-and-hold control approach was also successfully demonstrated in simulations of exponetial tracking of walking-like trajectory with possibly unlimited number of steps, including impact effect. REFERENCES R. Goebel, R. Sanfelice, and A. Teel, Hybrid dynamical systems, Control Systems Magazine, IEEE, vol. 9, pp. 8 93, apr. 9. T. McGeer, Passive dynamic walking, International Journal of Robotics Research, vol., no., pp. 6 8, 99. 3

q rad/s 3 3 4 5 dq dq 6.5.5.5 Time s Figure 4. Angular velocities q, q and references (dotted line) for 4 steps walking using the sampling controller in feedback tracking. dq rad/s.5.5 q rad (a) dq rad/s 5 5.5 3 3.5 4 q rad Figure 5. Phase-plane plots for (a) q, (b) q. The initial state is represented by a red circle. 3 M. Spong, Underactuated mechanical systems, control problems in robotics and automation. London: Springer Verlag, 998. 4 J. Grizzle, C. Moog, and C. Chevallereau, Nonlinear control of mechanical systems with an unactuated cyclic variable, IEEE Transactions on Automatic Control, vol. 5, pp. 559 576, may 5. 5 J. Zikmund and C. Moog, The structure of -body mechanical systems, in Proceedings of the IEEE Conference on Decision and Control, (San Diego, USA), pp. 6464 6469, 6. 6 E. Westervelt, J. Grizzle, C. Chevallereau, J. Choi, and B. Morris, Feedback control of dynamic bipedal robot locomotion. CRC Press, 7. 7 C. Chevallereau, G. Bessonnet, G. Abba, and Y. Aoustin, (b) Bipedal Robots: Modeling, Design and Walking Synthesis. Wiley-ISTE, 9. 8 S. Čelikovský, J. Zikmund, and C. Moog, Partial exact linearization design for the acrobot walking, in Preprints of the American Control Conference, 8, (Seattle, USA), pp. 874 879, 8. 9 M. Anderle, S. Celikovsky, D. Henrion, and J. Zikmund, Advanced lmi based analysis and design for acrobot walking, International Journal of Control, vol. 83, no. 8, pp. 64 65,. M. Anderle and S. Čelikovský, Analytical design of the acrobot exponential tracking with application to its walking, in Proccedings of the IEEE International Conference on Control and Automation, ICCA 9, pp. 63 68, 9. M. Anderle and S. Čelikovský, Approximate feedback linearization of the acrobot tracking error dynamics with application to its walking-like trajectory tracking, in Proceedings of the th Mediterranean Conference on Control & Automation, MED, (Barcelona, Spain),. M. Anderle and S. Čelikovský, Sustainable acrobot walking based on the swing phase exponentially stable tracking, in Proceedings of the ASME Dynamic Systems and Control Conference, (Cambridge, Massachusetts, USA), pp. /8 8/8, ASME,. 3 M. Anderle and S. Čelikovský, Feedback design for the acrobot walking-like trajectory tracking and computational test of its exponential stability, in Proceedings of the IEEE Multi- Conference on Systems and Control, (Denver, Colorado),. 4 M. Mousa, R. Miller, and A. Michel, Stability analysis of hybrid composite dynamical systems: Descriptions involving operators and differential equations, Automatic Control, IEEE Transactions on, vol. 3, no. 3, pp. 6 6, 986. 5 S. Čelikovský, M. Anderle, and C. Moog, Embedding the acrobot into a general underactuated n-link with application to novel walking design, in Proceedings of the Europen Control Conference, ECC, (Zurich, Switzerland), 3. 6 I. Fantoni and R. Lozano, Non-linear control of underactuated mechanical systems. Heidelberg: Springer Verlag,. 7 Y. Hurmuzlu and D. Marghitu, Rigid Body Collisions of Planar Kinematic Chains With Multiple Contact Points, The International Journal of Robotics Research, vol. 3, no., pp. 8 9, 994. 8 R. Brach, Rigid body collisions, Transaction of the ASME, Journal of Applied Mechanics, vol. 56, pp. 33 38, 989. 9 J. Grizzle, G. Abba, and F. Plestan, Asymptotically stable walking for biped robots: analysis via systems with impulse effects, IEEE Transactions on Automatic Control, vol. 46, pp. 5 64, jan. A. Isidori, Nonlinear Control Systems. New York: Springer Verlag, 996. 4