On control of reaching movements for musculo-skeletal redundant arm model

Transcription

1 Applied Bionics and Biomechanics Vol. 6, No., March 29, On control o reaching movements or musculo-skeletal redundant arm model Kenji Tahara a,c,, Suguru Arimoto b,c, Masahiro Sekimoto b and Zhi-Wei Luo d a Organization or the Promotion o Advanced Research, Kyushu University, 744 Motooka, Nishi-ku, Fukuoka , Japan; b Research Organization o Science and Engineering, Ritsumeikan University, --Nojihigashi, Kusatsu , Japan; c RIKEN-TRI Collaboration Centre or Human-Interactive Robot Research, Riken, Anagahora, Shimoshidami, Moriyama-ku, Nagoya 463-3, Japan; d Department o Computer Science and Systems Engineering, Kobe University, - Rokkodai-cho, Nada-ku, Kobe , Japan (Received 24 June 28; inal version received February 29) Downloaded By: [Tahara, Kenji] At: 5:27 7 May 29 This paper ocuses on a dynamic sensory-motor control mechanism o reaching movements or a musculo-skeletal redundant arm model. The ormulation o a musculo-skeletal redundant arm system, which takes into account non-linear muscle properties obtained by some physiological understandings, is introduced and numerical simulations are peromed. The nonlinear properties o muscle dynamics make it possible to modulate the viscosity o the joints, and the end point o the arm converges to the desired point with a simple task-space eedback when adequate internal orces are chosen, regardless o the redundancy o the joint. Numerical simulations were perormed and the eectiveness o our control scheme is discussed through these results. The results suggest that the reaching movements can be achieved using only a simple task-space eedback scheme together with the internal orce eect that comes rom non-linear properties o skeletal muscles without any complex mathematical computation such as an inverse dynamics or optimal trajectory derivation. In addition, the dynamic damping ellipsoid or evaluating how the internal orces can be determined is introduced. The task-space eedback is extended to the virtual spring-damper hypothesis based on the research by Arimoto et al. (26) to reduce the muscle output orces and heterogeneity o convergence depending on the initial state and desired position. The research suggests a new direction or studies o brain-motor control mechanism o human movements. Keywords: redundancy; musculo-skeletal model; sensory-motor control mechanism. Introduction The natural movements o a human seem to be smooth, dexterous and sophisticated compared with today s robots. Until now, these excellent movements have attracted many physiologists, kinesiologists and robotics researchers who hope to understand and extract them. In paticular, obtaining and mimicking these natural movements in robotics has become a hot topic, especially or robots that will be used in close proximity to humans. Obtaining the undamental elements o how the central nervous system (CNS) generates control strategies to perorm these kinds o movements is one o the ultimate objectives and dreams or understanding the nature o human movements. In robotics, many human movements such as object manipulations, walking, etc., have been mimicked. However, achieving highly natural movements, like human behaviour, is still diicult. There are many reasons or these diiculties. One is the existence o ill-posed problems induced by the joint and muscle redundancies that a human body intrinsically possesses (Bernstein 935). Until now, many solutions or overcoming these ill-posed problems by introducing various optimisation criteria have been proposed not only in physiology but also in robotics (Hogan 984a; Hollerbach and Suh 987; Katayama and Kawato 993; Kawato et al. 987, 99; Nakamura 99). In general, these optimisations require large calculation costs to solve complicated optimal unctions as well as an inverse dynamics, which would be very diicult or a human to perorm in real time. Indeed, do humans perorm actually these complex calculations every time they move? In this paper, we suggest that the sensory-motor control mechanism, which enables many kinds o dexterous human movements by the CNS, may not be so complicated and may even use simple task-space eedback by utilising intrinsic non-linear mechanical characteristics o skeletal muscles. Morasso (98) observed human reaching movements and suggested that the central commands in human movements may be coordinated not in joint space but in task space. We ocus on a simple reaching task by using a three-link planar arm model driven by nine redundant muscles so as to mimic the construction o a human arm in a simple way. We introduce non-linear muscle characteristics that correspond to the indings o physiological studies (Hill 938; Mashima et al. 972) and propose a simple task-space eedback control scheme or the arm model taking into account internal orces Corresponding author. tahara@ieee.org ISSN: print / online Copyright C 29 Taylor & Francis DOI:.8/

2 58 K. Tahara et al. Downloaded By: [Tahara, Kenji] At: 5:27 7 May 29 induced by redundant muscles. These internal orces can be basically generated by the co-contraction o agonist and antagonist muscles. Arimoto et al. (25) suggested that the perormence o the convergence to the desired position or human-like reaching movements by using a joint redundant robotic arm depends upon the damping shaping o each joint. Unortunately, humans cannot directly modulate the rotational joint viscosity because each joint is driven by linear movement o antagonistic muscles. How do humans modulate the rotational joint viscosity? Hogan (984b) remarked and Gribble et al. (23) observed that the co-contraction o agonist and antagonist muscles contributes to the modulation o the joint impedance. In our control scheme presented here, the co-contraction o antagonistic muscles to invoke the internal orces coming rom the redundant muscles can modulate the damping shaping o each joint i we introduce non-linear muscle dynamics based on Mashima et al. s model (972). Basically, a redundant actuation system such as a musculo-skeletal arm can generate internal orces. These internal orces belong to the null-space o the muscle space with respect to the joint space and thereby they are independent o a principal part o the joint torques in relation to generation o movements o the arm end point towards the target. However, it causes an ill-posed problem o how to determine the internal orces uniquely. Until now, many optimisational methods to determine the internal orces uniquely have been proposed by introducing some sort o artiicial indices (Delp and Loan 2; Rasmussen et al. 23). Unlike these methods, our proposed method does not need any optimisation criteria, because the internal orces can aect only part o the joint torques irrelevant to motion o the arm end point towards the target in the task space. Nevertheless, human-like slightly curved trajectories o the end point can be realised by adequately adjusting the internal orces through the null space with respect to the joint space. This point quite diers rom the traditional optimisation methods. In our previous work (Tahara et al. 25), we treated the two-joint six-muscle model which has only muscle redundancy, that is, joint redundancy was not taken into consideration. In this paper, we take into account not only the muscle redundancy but also the joint redundancy by introducing a three-joint nine-muscle planar arm model. The crucial dierence between these two cases is the convergence analysis that uses Lyapunov s direct method to prove that the closed-loop dynamics o the joint redundant model cannot be applied. To prove the stability and convergence o the closed-loop dynamics under the existences o both muscle and joint redundancies, we introduce the novel concept o stability on a maniold originated by Arimoto (25). In what ollows, we ormulate the kinematics and dynamics o the three-joint planar arm model driven by six mono-articular muscles and three bi-articular muscles governed by non-linear behaviour. The stability and convergence o the overall system is given by introducing a stability theory on a maniold. We report some numerical simulations to demonstrate that the simple sensory-motor control scheme makes it possible to make reaching movements even though both joint and muscle redundancies are taken into consideration. To discuss and evaluate how to choose the internal orces, we introduce a dynamic damping ellipsoid. In addition, the task-space eedback manner is extended to the virtual spring-damper hypothesis proposed by Arimoto et al. (25) to reduce the muscle output orces and the heterogeneity o convergence, which depends on the initial state and desired position o the arm. Finally, these results suggest a direction or studies o the brainmotor control mechanism o human movements. 2. Musculo-skeletal redundant system We introduce a musculo-skeletal redundant arm model that consists o three serial links, six mono-articular muscles, and three bi-articular muscles to imitate the coniguration o a human arm. This model is shown in Figure. The three bi-articular muscles enhance the end-point stiness. 2.. Kinematics o muscle-joint space Assume that the overall movements o the system are limited to a horizontal plane so that gravity does not aect Figure. Three-link musculo-skeletal arm model with six mono-articular muscles and three bi-articular muscles.

3 Applied Bionics and Biomechanics 59 Downloaded By: [Tahara, Kenji] At: 5:27 7 May 29 movements at all. Any riction such as Coulomb s riction and static riction is ignored in our model. All skeletal muscles included in the arm can only be linearly contracted and are not curved on the way. Also the mass transer o the muscles during contraction is omitted because its eect on arm movements is not so crucial. The physical parameters and variables are shown in Figure. The lengths o the muscles l i (θ) R 9 are ormulated as the distances between the insertion points o the muscles. They can be expressed as ollows: l(θ) = (l,l 2,l 3,l 4,l 5,l 6,l 7,l 8,l 9 ) T (r 2 + s 2 + 2r s cos θ ) 2 (r s 2 2 2r 2 s 2 cos θ ) 2 (r s r 3 s 3 cos θ 2 ) 2 (r s 2 4 2r 4 s 4 cos θ 2 ) 2 (w w L 2 + 2w 5 L cos θ +2w = 52 L cos θ 2 + 2w 5 w 52 cos (θ + θ 2 )) 2 (w w L 2 2w 6 L cos θ,() 2w 62 L cos θ 2 + 2w 6 w 62 cos (θ + θ 2 )) 2 (r s r 7 s 7 cos θ 3 ) 2 (r s 2 8 2r 8 s 8 cos θ 3 ) 2 (w w L w 9 L 2 cos θ 2 +2w 92 L 2 cos θ 3 + 2w 9 w 92 cos (θ 2 + θ 3 )) 2 where l i (i = 9) are the lengths o the muscles, and θ i (i = 3) are the joint angles. Also, r i (i = 4, 7 8), and s i (i = 4, 7 8) are the distances between the centre o each joint and the insertion point o each mono-articular muscle, and w i (i = 5, 52, 6, 62, 9, 92) are those o bi-articular muscles. Taking the time derivative o Equation () yields l = W T θ, (2) where W T R 9 3 stands or the Jacobian matrix rom joint space to muscle space. It is known that the relation between contractile orces o the muscles F m R 9 and joint torques τ R 3 can be expressed rom the principle o virtual work as τ = WF m. (3) In this study, we assume that the Jacobian matrix W is o row ull-rank during movement. Thereore, we can take the inverse o Equation (3) as F m = W + τ + (I 9 W + W)k e, (4) where W + = W T (WW T ) R 9 3 signiies the pseudoinverse matrix o W, ( I 9 W + W ) k e R 9 stands or the null-space o W, k e R 9 is an arbitrary vector, and I 9 R 9 9 denotes an identity matrix. The physical meaning o the second term o the right-hand side is the internal orce generated by the redundant muscles. This relation does not mean an optimisation, and it expresses only decomposition o the muscle orce space with respect to the joint space into two spaces, one is the image space and the other is the kernel space Kinematics o joint-task space The end-point position o the arm model given by x = (x,y) T in Cartesian coordinates is expressed as x= L cos θ +L 2 cos(θ + θ 2 ) + L 3 cos(θ + θ 2 + θ 3 ) L sin θ +L 2 sin(θ + θ 2 ) + L 3 sin(θ + θ 2 + θ 3 ). (5) Taking the time derivative o Equation (5) yields ẋ = J θ, (6) where J R 2 3 stands or the Jacobian matrix rom joint space to task space. The relation between joint torques τ R 3 and output orces o the end point in Cartesian coordinates F x R 2 can be represented as τ = J T F x. (7) Substituting Equation (7) into Equation (4) yields the relation between contractile orces o the muscles F m and output orces o the end-point F x.itisgivenby F m = W + J T F x + ( I 9 W + W ) k e. (8) 2.3. Non-linear muscle model It is known that the skeletal muscles are governed by strong non-linearity. Here, we introduce a tangible nonlinear muscle model that is based on several physiological results. In the modeling o the muscles, we assume that the masses o all muscles are included in the masses o each link. The model o muscle is basically according to the two-element model which is composed o contractile element (CE) and series elastic element (SE). In this paper, we want to ocus on the dumping eect on each joint generated by CE, and thereby we introduce quite strong assumption that the SE part can be approximated as a rigid element which indicate that its elastic coeicient becomes ininity. Thereore, hereater, we only consider the CE part in the muscle model. In physiology, the orce velocity relation o the skeletal muscle proposed by Hill (938), which is expressed as a simple hyperbolic equation, is known as

4 6 K. Tahara et al. Downloaded By: [Tahara, Kenji] At: 5:27 7 May 29 one o the most undamental muscle eatures. This equation is ( + a)( l + b) = b( + a) (9) where is the output tensile orce o the muscle, l is the contractile velocity o the muscle, is the maximum isometric tensile orce o the muscle which depends on its length, a is the heat constant and b is the rate constant o energy liberation. The shortening directions o and l are deined as positive. In addition, it is known in physiology that a skeletal muscle exerts more tensile orce in the lengthening phase than in the shortening phase. Mashima et al. (972) determined the above eature speciically through some experiments. By considering this eature, we can modiy Equation (8) as ollows: b a l l +b α i l (α, l) = b (2 + a) l α i l <, l +b () where α is the muscle activation level, the coeicients a and b are determined experimentally in Mashima et al. (972) as a =.25, b =.9 l and l is the intrinsic rest length o the muscles. Now, we deine the control input to the muscles as ᾱ = α, in order to set the dimension o the input as that o the muscle orce. Thereore, Equation () can be represented as ollows:.9l.9l + l (ᾱ, l) =.9l.9l + l ( ᾱ.25 ) ᾱ l.9l ( ᾱ 2.25 ) ᾱ l.9l i l i l < () Since a skeletal muscle intrinsically owns some damping actor, which comes rom its component and material, we introduce a muscle intrinsic viscosity c > independently rom ᾱ. Hence, the muscle dynamic model used in this study can be expressed as (ᾱ, l) = p { ᾱ (ᾱc + c ) l }, (2) Muscle output orce (N) α = 2. (N) α = 6. (N) α =. (N) Muscle contractile velocity (m/s) Figure 2. Force-velocity curve o the modiied Hill s muscle property model, which is expressed by Equation (2). as the contractile velocity o the muscle increases and vice versa. Now, it should be remarked that in Equation (2) the parameter p depending on l satisies the inequality Equation <p as long as l is upper bounded. Eventually, the overall muscle dynamics in this study can be expressed as ollows: F m = Pᾱ P {A(ᾱ)C + C } l (3) F m = (, 2,..., 9 ) R 9 P = diag(p,p 2,..., p 9 ) R 9 9 ᾱ = (ᾱ, ᾱ 2,..., ᾱ 9 ) T R 9 A(ᾱ) = diag(ᾱ, ᾱ 2,..., ᾱ 9 ) R 9 9 C = diag(c,c 2,..., c 9 ) R 9 9 C = diag(c,c 2,..., c 9 ) R 9 9 It is remarkable that in Equation (3), the diagonal matrix A(ᾱ) is constructed by the elements o the control input vector ᾱ (Tahara et al. 25, 26). This comes rom the non-linearity o the muscle model. Also, the skeletal muscle generates only a contractile orce, so we deine the control signal ᾱ, which is a saturated unction so as to keep a semi-positive value, as shown in Figure 3. Thereore, A(ᾱ) is semi-positive deinite. Both damping matrices C and C where p =.9l.9l + l, c =.25 >, i l.9l 2.25 >, i l <..9l The orce velocity curve o the muscle property expressed by Equation (2) is shown in Figure 2. As the igure shows, the contractile orce o the muscle model reduces gradually Muscle input (N) + Figure 3. Saturated unction o the input ᾱ. α

5 Applied Bionics and Biomechanics 6 Downloaded By: [Tahara, Kenji] At: 5:27 7 May 29 are also positive deinite. In this paper, the positive coeicient c, which is the intrinsic viscosity o muscle, is deined as c =.2 by considering physiological aspects Control input In physiological studies o the brain-motor control mechanism in human movements, several control strategies or skeletal muscles have been proposed since Fel dman irst proposed a spring-like model called the λ-model (Fel dman 966, 986) which exerts muscle orce by modulating the threshold o tonic stretch relex about muscle length. Mclntyre and Bizzi (993) also proposed the α-model which modulates spring-like characteristics o the muscles by α neuron. These models are based on the equilibrium point (EP) hypothesis proposed by Fel dman and observed by Bizzi et al. Many studies concerned with multijoint reaching movement, which were based on the EP hypothesis, have been done (Bizzi et al. 976, 992; Flash 987; Flash and Hogan 985). Recently, Arimoto et al. (25) have proposed a virtual spring hypothesis that a given potential energy in task space can generate a spring-like orce on the end point and distribute it to each joint torque through the transpose o the Jacobian matrix J T.This task-space control scheme originated rom Takegaki and Arimoto (98). In developmental psychology, it is said that a potential unction that emanates neuro-motor signals emerges rom intentional incentive o a voluntary movement (Killeen 992). The virtual spring hypothesis has emphasised that such a potential-generating neuro-motor signals should be coordinated not in the muscle or joint spaces but in the task space in terms o the distance between a present position o the end point and a desired one such as 2 xt K x. This potential is eventually distributed to each joint through the transposed Jacobian matrix J T which implies the joint coniguration o a human body. In the present study, we introduce a simple task-space eedback control scheme with internal orces, which basically ollows the virtual spring hypothesis. Now, let us consider the diagonal matrix P expressed by Equation (3). Since the boundaries o its components p i are given by <p i, P is positive deinite. Hence, we can obtain a pseudo-inverse matrix W + = (WP) + as long as the muscle Jacobian matrix W does not degenerate during movement. The control input to the muscles ᾱ is expressed by ᾱ = W + J T K x + (I 9 W + W)k e, (4) where W = WP, and x = x x d is the end point position error between the present position x R 2 and the desired position x d R 2 in the task space. The positive deinite diagonal matrix K = k I 2 R 2 2 denotes the position eedback gain, and k signiies the positive stiness coeicient o the virtual spring. It should be noted that in Equation (4), the irst term o the right-hand side denotes an artiicial potential energy in task-space and produces spring-like orces to each muscle through two Jacobian matrices J T and W + and the second term stands or the internal orces generated by the redundant muscles, where k e R 9 is an arbitrary vector. This control signals expressed by Equation (4) can be interpreted more speciically as that K x comes rom intentional incentive o a movement which is generated in the motor cortex, and J T implies the joint coniguration o the arm relative to the arm end-point, W + and (I 9 W + W)k e come rom the coniguration o the related muscles relative to the joints Dynamics o three-link planar arm model The dynamics o a three-link planar arm model can be described by Lagrange s equation o motion as { } H(θ) θ + 2 Ḣ(θ) + S( θ, θ) θ = τ, (5) where H(θ) R 3 3 is the inertia matrix, which is symmetric and positive deinite, and θ R 3, θ R 3, θ R 3 are angular accelerations, angular velocities and angles o joints, respectively. S( θ, θ) R 3 3 is a skew-symmetric matrix coming rom Coriolis and centriugal orces, and τ R 3 are the input torques to the joints. Substituting Equations (2), (3), (3) and (4) into Equation (5) yields { } H(θ) θ + 2 Ḣ(θ) + S(θ, θ) θ + J T K x + WP(AC + C )W T θ =. (6) Equation (6) shows the overall closed-loop dynamics, which is expressed in the joint coordination system. 3. Stability o the closed-loop dynamics This section illustrates the convergence o the overall system by introducing the concepts o stability on a maniold and transerability to a submaniold. Now, it is convenient to rewrite Equation (6) as { } H(θ) θ + 2 Ḣ(θ) + S(θ, θ) θ + J T K x + WP (AC + C )W T θ = u,(7) where u = are the input to the closed-loop dynamics o Equation (6). Taking the inner product between the input as Equation (7) and the output as the joint angular velocity

6 62 K. Tahara et al. Downloaded By: [Tahara, Kenji] At: 5:27 7 May 29 θ yields where θ T u = d dt E(t) + θ T WP (AC + C )W T θ, (8) E(t) = 2 θ T H θ + k 2 x 2. (9) Here, we should consider the damping matrix WP(AC + C )W T R 3 3 such that the control inputs ᾱ i, which are components o the diagonal matrix A, are deined as a saturated unction (see Figure 3) to be semi-positive. Thereore, the matrix (AC + C ) is positive deinite because A, C >, C > and P >. Hence, the damping matrix WP (AC + C )W T is positive deinite as long as the Jacobian matrix W is o row ull-rank. Now, integrating Equation (8) over time t [,T) yields t + θ T udσ = E(t) E() t θ T WP (AC + C )W T θ dσ E(). (2) This inequality means that the closed-loop dynamics o Equation (6) satisies the passivity condition. However, it should be noted that this system is a joint redundant system because the control input o task-space eedback requires only an end-point position (x,y) although the arm consists o three joints. Then, the scalar unction E(t) is non-negative deinite, but not positive deinite or all state variables o (θ, θ). Thereore, to discuss the convergence o this redundant system, we introduce the novel concept o stability theory on a maniold proposed by Arimoto et al. (25). In this proo, we consider that the one-dimensional submaniold M on the spectrum S 6 R 6 such that the transposed Jacobian matrix J T R 3 2 is o column ullrank. Let us introduce the ollowing deinitions to prove the stability and convergence o the closed-loop dynamics. Deinition (Stability on a maniold) For the reerence state (θ = θ, θ = ) M, i or an arbitrarily given ε>, there exist a constant δ(ε) > and another constant r > (r <r ) independent o ε such that a solution trajectory starting rom any initial state (θ(), θ()) contained in N 6 (δ(ε),r ) in the neighbourhood o δ stays on N 6 (ε, r ) in the neighbourhood o ε,the reerence state (θ, ) is said to be stable on a maniold. Deinition 2 (Transerability to a submaniold) I or the reerence state (θ, ) M, there exist a constant ε > and another constant r > (r <r ) such that any solution trajectory starting rom an arbitrary initial state contained in N 6 (δ(ε),r ) stays on N 6 (ε,r ) and converges asymptotically as t to a point on a submaniold M N 6 (ε,r ), the reerence state (θ, ) is said to be transerable to a submaniold o M. The physical meaning o Deinition is that the overall energy o the system does not increase during movement. In addition, the meaning o Deinition 2 is that the desired state ( x =, θ = ) is guaranteed while the system is stable. Taking the inner product between Equation (6) and θ + βkj T x yields { d dt 2 θ T H θ + k } 2 x 2 + θ T WP(AC + C )W T θ { } +βk 2 J T x 2 +βk x T JH θ + βk x T J 2 Ḣ + S θ +βk x T JWP(AC + C )W T θ =, (2) where β is a positive number. We introduce the ollowing equation. x T JH θ = d dt xt JH θ θ T J T JH θ x T JH θ x T J Ḣ θ. (22) Substituting Equation (22) into Equation (2) yields { d dt 2 θ T H θ + k } 2 x 2 + βk x T JH θ where + θ T WP (AC + C )W T θ + βk 2 J T x 2 + βk x T JWP(AC + C )W T θ + βkh( θ, x) =, (23) h( θ, x) = x T J { 2 } Ḣ + S θ θ T J T JH θ x T JH θ. (24) The second, third and sixth terms o the let-hand side o Equation (2) can be rewritten as ollows: θ T WP(AC + C )W T θ βk 2 J T x 2 βk x T JWP(AC + C )W T θ { = γ 2 θ T H θ + k } 2 x 2 + βk x T JH θ

7 Downloaded By: [Tahara, Kenji] At: 5:27 7 May 29 θ T { W(AC + C )W T γ 2 H } θ k x T { βkj T J γ 2 I 2 } x +γβk x T JH θ βk x T JWP(AC + C )W T θ, (25) where γ is a positive number. Also, the ollowing inequality can be obtained by considering the positive deiniteness o the damping matrix WP(AC + C )W T. βk x T JWP(AC + C )W T θ βk x T JWP(AC + C )W T J T x + βk 4 θ T WP(AC + C )W T θ. (26) Similarly, we can obtain the ollowing inequality: γαk x T JH θ γβk x T JHJ T x + γβk 4 θ T H θ. (27) By substituting Equations (26) and (27) into Equation (25), we obtain the ollowing inequality: θ T WP(AC + C )W T θ βk 2 J T x 2 βk x T JWP(AC + C )W T θ { γ 2 θ T H θ + k 2 x 2 + βk x T JH θ {( βk ) WP(AC + C )W T γ 4 2 k x T {βkj T J γ 2 I 2 Applied Bionics and Biomechanics 63 } θ T ( + βk ) } H θ 2 ) } β J (γ H + WP (AC + C )W T J T x. (28) Here, we introduce the scalar unction V (t) such that where {( ( θ, x) = θ T βk 4 γ 2 ) WP(AC + C )W T ( + βk ) } H θ + k x T[ βkj T J γ 2 2 I 2 β J { γ H + WP(AC + C )W T} J T] x. (3) In Equation (25), since Ḣ and S are in linear homogeneous with respect to the angular velocities θ, and the order o their coeicients is less than or equal to the maximum eigenvalue o H.Also, J consists o a trigonometric unction and physical parameters, and they are thereby totally bounded. Thus, as long as x(t) < x(), there exists a positive number ζ or h to satisy the ollowing inequality (Tahara et al. 25, 26): h( θ, x) ζ θ 2. (32) Now, assume that we can choose the numbers k, β, γ and the internal orce vector k e to satisy the ollowing inequalities. I 2 β 2 k J T HJ > (33) ( βk ) WP(AC+C )W T γ ( + βk ) H >βkζ (34) βkj T J γ 2 I 2 β J { γ H+WP(AC+C )W T} J T >. (35) By considering these inequalities, we see that the scalar unction V (t) expressed by Equation (29) is positive, and the inequality βkh( θ, x) ( θ, x) (36) is satisied. Hence, Equation (23) can be reduced to V (t) = 2 θ T H θ + k 2 x 2 + βk x T JH θ = ( θ + βkj T x ) T H ( θ + βkj T x ) 2 + k 2 xt ( I 2 β 2 k J T HJ ) x. (29) Substituting Equations (28) and (29) into Equation (23) yields d dt V (t) γv(t) βkh( θ, x) ( θ, x), (3) d V (t) γv(t). (37) dt Thereore, rom Equation (37) we obtain V (t) e γt V () (38) and conclude that the scalar unction V (t) converges to zero exponentially. The scalar unction V (t) plays the role o a modiied energy unction o E(t) in Equation (9). Thereore, θ and x when t. Q.E.D.

8 64 K. Tahara et al. Table. Physical parameters o the arm. Table 3. Initial and desired points. Length Mass Inertia Mass centre (m) (kg) (kg m 2 ) (m) Upper arm Forearm Wrist Initial point (m) (x, y) = (.,.6) Desired point (m) Short range (x d,y d ) = (.2,.4) Middle range (x d,y d ) = (.4,.2) Downloaded By: [Tahara, Kenji] At: 5:27 7 May Numerical simulation Table shows the physical parameters o the arm model, Table 2 shows the insertions o the muscles and Table 3 shows the initial and desired positions o the end point. In this paper, we treat two types o reaching movements: short-range and middle-range reaching. We also determine the internal orce vector as k e = k e e in order to see the inluence o k e on the dynamics, where k e is a positive number and e = (,,...,) T R 9. Simulation results or the short-range reaching movement ( x.3[m]) with virtual spring coeicient k =. and internal orce coeicient k e =5. are shown in Figure 4. The end-point trajectory moves along a very winding path even though it converges to the desired point. Simulation results or the short-range reaching movement with k =. and k e = 25. are shown in Figure 5. Here, in contrast to Figure 4, the end-point converges to the desired point smoothly and its trajectory is close to a straight line. Figure 6 shows the trajectories o the end-point position (x,y) o a shortrange reaching movement with the virtual spring coeicient k =. when k e is shited rom 5. to 3. by 5. units. These trajectories are made close to a straight line by increasing the internal orce coeicient k e.figure 7 shows the transient responses o the end-point velocity v = ẋ 2 + ẏ 2 o short-range reaching movement with k =. when k e is shited rom 5. to 3. by 5. units. The velocity proiles are made smooth by increasing the internal orce coeicient k e. Figures 8 and 9 show the transient responses o the end-point x- and y-position with k =. when k e is shited rom 5. to 3. by 5. units. We see rom these igures that the shortest settling time o the end-point x-position is about.5 s when k e = 3., and that o the y-position is about 2. s when k e = 2.. On the other hand, Figure shows the simulation result o the middle-range reaching movement with k =. and k e = 5.. The trajectory o the end point is not smooth and describe a more winding route than that o the short-range reaching, and the inal posture o the arm seems to be impossible or a real human. In contrast, Figure shows the simulation result o the middle-range reaching movement with k =. and k e = 25.. The end-point converges to the desired point smoothly, and its trajectory becomes a quasi-straight line even though x.6 m holds or in middle-range reaching. Figure 2 shows trajectories o the end-point position (x,y) o the middle-range reaching movement with k =. when k e is shited rom k e =5. to k e =3. by 5. units. Like the short-range reaching movements shown in Figure 6, the end-point trajectories become smooth and close to straight lines as k e increases. Figure 3 shows transient responses o the end-point velocity v o the middle-range reaching movement with k =. when k e is shited rom k e = 5. to k e = 3. by 5. units. The velocity proiles when k e = 5. and k e =. are oscillatory, and each settling time is more than 3. s. However, those in the case o k e = 25. and k e = 3. are smooth because o the higher k e, and also each settling time is about.5 s. Figures 4 and 5 show the transient responses o the end-point x- and y-position with k =. when k e is shited rom 5. to 3. by 5. units. We see rom these igures that the astest convergence time o the end-point.8.6 Table 2. Muscle Insertion o each muscle. Value (m) l r =.55 s =.8 l 2 r 2 =.55 s 2 =.8 l 3 r 3 =.3 s 3 =.2 l 4 r 4 =.3 s 4 =.2 l 5 w 5 =.4 w 52 =.45 l 6 w 6 =.4 w 62 =.45 l 7 r 7 =.35 s 7 =.22 l 8 r 8 =.5 s 8 =.25 l 9 w 9 =.4 w 92 = Figure 4. Short-range reaching movement with k =. and k e = 5..

9 Applied Bionics and Biomechanics End-point velocity v (m/s).5 k e = 5. k e =. k e = 5. k e = 2. k e = 25. k e = 3. Figure 5. k e = Short-range reaching movement with k =. and Figure 7. Transient responses o the end-point velocity o shortrange reaching movements with k =.. Downloaded By: [Tahara, Kenji] At: 5:27 7 May 29 x-position is about 2. s when k e = 5., and that o the y-position is about 2. s when k e = 25.. These simulation results suggest that the internal orce coeicieint k e are chosen according to the virtual spring coeicient k,the end point o the musculo-skeletal redundant arm converges to the desired point smoothly without any complex mathematical optimization. The beneit o our proposed model is that co-contraction o antagonistic muscles can modulate the damping eect in the joint space. Until now, many control schemes or muscle redundant arm model are used by some optimisation method to determine muscle output orce uniquely because the internal orces cannot aect to the joint torques. Unlike these methods, in our model, internal orces can aect the joint torques, and thereby the trajectory o the end point is changed by varying the internal orce vector k e k e = 5. k e =. k e = 5. k e = 2. k e = 25. k e = Figure 8. Transient responses o the end-point x-position o short-range reaching movements with k = k e = 5. k e =. k e = 5. k e = 2. k e = 25. k e = k e = 5. k e =. k e = 5. k e = 2. k e = 25. k e = Figure 6. Trajectories o the end-point position o short-range reaching movements with k =.. Figure 9. Transient responses o the end-point y-position o short-range reaching movements with k =..

10 66 K. Tahara et al End-point velocity v (m/s) k e = 5. k e =. k e = 5. k e = 2. k e = 25. k e = Figure. k e = 5.. Middle-range reaching movement with k =. and Figure 3. Transient responses o the end-point velocity o middle-range reaching movements with k =.. Downloaded By: [Tahara, Kenji] At: 5:27 7 May Figure. Middle-range reaching movement with k =. and k e = k e = 5. k e =. k e = 5. k e = 2. k e = 25. k e = Figure 4. Transient response o the end-point x-position o middle-range reaching movements with k = k e = 5. k e =. k e = 5. k e = 2. k e = 25. k e = k e = 5. k e =. k e = 5. k e = 2. k e = 25. k e = Figure 2. Trajectories o the end-point position o middlerange reaching movements with k =.. Figure 5. Transient response o the end-point y-position o middle-range reaching movements with k =..

11 Applied Bionics and Biomechanics 67 Downloaded By: [Tahara, Kenji] At: 5:27 7 May Dynamic damping ellipsoid We introduce a dynamic damping ellipsoid, in order to discuss how the internal orce vector k e is determined, and to evaluate it. The damping ellipsoid at the end point o the human arm has been proposed by Tsuji et al. (994). The ellipsoid can express the damping eect at the end point o a human arm when the arm s dynamics can be approximated as being almost governed by only a term related to the velocity. It was derived rom the result o measurement and the evaluation o a real human hand s damping eect in a quasi-static situation. Unlike the above ellipsoid, our dynamic damping ellipsoid expresses the damping eect that can be given to the end point o the arm by modulating the internal orce vector k e included in the control input to the muscles. Figure 6 illustrates the dynamic damping ellipsoid at the end point o a human arm. We consider part o the control input in Equation (4), which can generate the joint torque or only the damping eect as τ D = WP(AC + C )W T θ, (39) where τ D is the vector o the input torques or generating the damping eect. Since the damping orce F D at the end point in task space can be expressed as where F D = Dẋ, (4) D = ( J T ) + WP(AC + C )W T J +, (4) Figure 7. Dynamic damping ellipsoid at the end point when k =. and k e = 25.. ( J T ) + is the pseudo-inverse transposed Jacobian matrix o J T, and J + means the pseudo-inverse Jacobian matrix o J. Thereore, the exertable end-point damping orce when the end-point velocity ẋ satisies the inequality o ẋ can be expressed as F D T (D ) T D F D. (42) This ellipsoid shows the end-point damping orces generated by the control input during movement. By considering this ellipsoid, we can evaluate the internal orces required to modulate the damping eect. Figure 7 shows the damping ellipsoid at the end point o the middle-range reaching movement with k e = 25., and Figure 8 shows that with k e = 5.. We see rom these igures that in the case o a suitable internal orce (k e = 25.), the long axis o Figure 6. Dynamic damping ellipsoid at the end point. Figure 8. Dynamic damping ellipsoid at the end point when k =. and k e =..

12 68 K. Tahara et al. Downloaded By: [Tahara, Kenji] At: 5:27 7 May 29 the damping ellipsoid turns on the shoulder during movements. Thereore, the short axis o the damping ellipsoid turns on the desired point, and the damping eect at the end-point is relatively small in the direction o the desired point during movement. Also at the beginning o the movement, the ellipsoid is relatively large. It becomes gradually small as the end-point velocity increases, and inally the shape o the ellipsoid becomes large again like at the beginning shape. However, in the case o an unsuitable internal orce (k e = 5.), the damping ellipsoid is relatively small: the long and short axes o the ellipsoid are nearly equal. By comparing these results we can conclude that the end point can converge to the desired point smoothly by considering the size and direction o the dynamic damping ellipsoid during movement. The size and direction o dynamic damping ellipsoid are governed by the internal orce vector k e and muscle contractile velocity. However in this paper, we only pointed out a relation between the internal orces and the ellipsoid rom the viewpoint o muscle and joint conigurations as a posteriori reasoning, and could not disclose any dynamic relation between the internal orce vector and the ellipsoid explicitly rom the dynamics viewpoint, because an explicit dynamics o co-activation model is not presented here. Thereore, Table 4. Parameters or a virtual spring-damper. Virtual spring k =. Virtual damper ζ =. (k e = 5.) ζ =. (k e = 25.) a more systematic paradigm or determining k e according to the dynamic damping ellipsoid or a totally dierent modeling o muscle dynamics that may naturally prevent involvement o internal orces is needed in our next work. 6. Extension to virtual spring-damper hypothesis We extend our control stragegy to the virtual springdamper hypothesis proposed by Arimoto et al. (26). This hypothesis made the point that in reaching movements, i we assume the existence o a virtual spring and a virtual damper between the end point and the desired point, the end point o a human arm can converge to the desired point smoothly according to this virtual spring-damper (see Figure 9). Also in physiology, it is known that the cocontraction o agonist and antagonist muscles is gradually Figure 9. Virtual spring-damper between the end point and the desired position.

13 Applied Bionics and Biomechanics Output orce o muscle (N) Figure 2. Middle-range reaching movement with a virtual spring-damper when k =., k e = 5. and ζ = Figure 22. Transient responses o the muscle s output orces o middle-range reaching with a virtual spring-damper. Downloaded By: [Tahara, Kenji] At: 5:27 7 May 29 reduced as the process o learning a movement progresses (Osu et al. 997). Now, we hypothesise that the internal orces generated by the co-contraction o agonist and antagonist muscles are reduced, which enables an adequate damping shape in the task space to be gradually obtained as the learning process or reaching movements progresses. We modiy the control input to the virtual spring-damper model. It is given by ᾱ = W + J T {k x ζ kẋ}+(i 9 W + W)k e. (43).6 Output orce o muscle (N) Figure 23. Transient responses o the muscle s output orces o middle-range reaching with only a virtual spring D C B E A F H G Figure 2. Middle-range reaching movement with only a virtual spring when k =., k e = 25. and ζ =.. Figure 24. End-point trajectories o all range-reaching movements with a virtual spring-damper.

14 7 K. Tahara et al C D B E A H F G Figure 25. End-point trajectories o all range-reaching movements with only a virtual spring. End-point velocity (m/s) Figure 27. End-point velocities o all range-reaching movements with only a virtual spring. A B C D E F G H Downloaded By: [Tahara, Kenji] At: 5:27 7 May 29 Also, the closed-loop dynamics can be expressed as { } H(θ) θ + 2 Ḣ(θ) + S(θ, θ) θ + J T k x +{J T Jζ k + WP(AC+C )W T } θ = u =, (44) where the irst term o the right-hand side o Equation (43) represents the virtual spring-damper and the second term represents the internal orces generated by redundant muscles. Table 4 shows each parameter or a virtual springdamper k and ζ used in the numerical simulations, and these results are shown below. Figure 2 shows the simulation results with a virtual spring-damper, and Figure 2 shows those results without spring-damper. We see rom these igures that the trajectories o the end point in these two cases converge to the desired point smoothly. The inal orientation o the wrist joint in the case o using a virtual spring-damper is more natural than that without one. In contrast, Figures 22 and 23 show the transient responses End-point velocity (m/s) A B C D E F G H Figure 26. End-point velocities o all range-reaching movements with a virtual spring-damper. o the muscle s output orces. We see rom these igures that the output orces in the case o using a virtual springdamper are obviously reduced compared with those without a virtual spring-damper, even though both end points converge to the desired point smoothly. Figures 24 and 25 show the simulation results or reaching movements o all ranges. We see rom these igures that when the desired position is in the extension direction rom the initial position (especially F, G, H in Figure 25), there is excessive overshoot, which depends on the coniguration o the muscles. However, this excessive overshoot can be reduced by introducing a virtual damper (see Figure 24). Figures 26 and 27 show the end-point velocities o all range-reaching movements with and without a virtual spring-damper. We see rom these igures that the velocity proiles become more smooth when a virtual spring-damper is used. In paticular, F, G, H in Figure 26, the oscillation is decreased. These simulation results led us to suppose that reaching movements made beore the movements were learned need a certain level o internal orce to generate the joint damping eect because there is no adequate virtual damper at the end point. Ater learning, the internal orces can be reduced by obtaining pro-prioceptive inormation in the CNS, so an adequate virtual damper can be generated at the end point. However, in order to discuss this more, we need to compare a human s movements and our scheme. 7. Conclusion This paper treated reaching movements by using the musculo-skeletal redundant arm model composed o three joints and nine muscles with physiological non-linear muscle properties to mimic a human arm. Asymptotic convergence o the closed-loop dynamic system was obtained based on the concept o stability theory on a maniold. We inally conirm through numerical simulation results that the end point o the musculo-skeletal redundant arm could

15 Applied Bionics and Biomechanics 7 Downloaded By: [Tahara, Kenji] At: 5:27 7 May 29 converge to the desired point using only task-space eedback control with internal orce adjustment under the existence o both joint and muscle redundancies, without any complex mathematical computation. In addition, we introduced the dynamic damping elipse to determine the internal orce vector and evaluated it. We extended our control scheme to the virtual spring-damper hypothesis proposed by Arimoto et al. (26). By introducing a virtual damper in addition to the virtual spring at the end point, we can reduce the internal orces. Moreover, the heterogeneity o convergence depending on the initial state and desired position is also reduced. These results lead us to the understanding that learning o human movements might be interpreted as acquisition o adequate coordination o a virtual spring and damper in the task coordinates. However, we have not treated the problem o learning o movements in this paper yet. Thereore, in order to claim that our control scheme is reasonable, we should gain a more physical insight into determination o the internal orce vector k e. In addition, we have only modeled co-contraction o agonist and antagonist muscles rom the viewpoint o their conigurations and contractile velocities, but did not present dynamics o muscle co-activation which might be given as a dynamic system o more complex dierential equations. By introducing such a dynamic model o coactivation, the structure o the internal orce space described by (I 9 W + W)k e and the mechanisum o cocontraction would be disclosed in a more explicit orm. Fortunately, since modeling o skeletal muscles is a hot topic in physiology (Houdijk et al. 26; Perreault et al. 23) and many new models will be proposed hereater, we would like to introduce such skeletal muscle models into our control strategy or redundant joint and musculoskeletal systems. On the other hand, one o the advantages o our proposed method is that any planning o end-point trajectories is unnecessary in advance. Similarly, Todorov and Jordan (22) have also proposed an optimal eedback control theory that does not need end-point trajectories. Our control method presented here may be interpreted as one o such optimal eedback control schemes by regarding an artiicial potential energy produced by the virtual spring as one o the possible cost unctions. The equivalence o both methods will be treated in our uture works. We would also like to introduce a learning strategy or the internal orce to investigate human-like multi-body control mechanisms based on our control strategy. Acknowledgements This work was partially supported by the Kyushu University Research Superstar Program (SSP), based on the budget o Kyushu University allocated under the President s initiative, and by the Ministry o Education, Science, Sports and Culture through a Grant-in-Aid or Young Scientists (B), 87625, Reerences Arimoto S, Sekimoto M, Hashiguchi H, Ozawa R. 25. Natural resolution o ill-posedness o inverse kinematics or redundant robots: a challenge to Bernstein s degrees-o-reedom problem. Adv Robot. 9(4): Arimoto S, Sekimoto M. 26. Human-like movements o robotic arms with redundant DOFs: virtual spring-damper hypothesis to tackle the Bernstein problem. In: Proceedings o IEEE International Conerence Robotics and Automation Bernstein N The problem o the interrelation o coordination and localization. Arch Biol Sci. 38:5 59 (reprinted as Bernstein NA The coordination and regulation o movements. London: Pergamon. Bizzi E, Hogan N, Mussa-Ivaldi F, Giszter S Does the nervous system use equilibrium-point control to guide single and multiple joint movements? Behav Brain Sci. 5: Bizzi E, Polit A, Morasso P Mechanism underlying achievement o inal head position. J Neurophysiol. 3(9): Delp S, Loan J. 2. A computational ramework or simulating and analyzing human and animal movement. IEEE Comput Sci Eng. 2: Fel dman A Functional tuning o the nervous system with control o movement or maintenance o steady posture. iii. mechanographic analysis o the execution by man o the simplest robot tasks. Biophysics. : Fel dman A Once more on the equilibrium-point hypothesis (λ model) or motor control. J Mot Behav. 8:7 54. Flash T The control o hand equilibrium trajectories in multi-joint arm movements. Biol Cybern. 57(4 5): Flash T, Hogan N The coordination o arm movements: an experimentally conirmed mathematical model. J Neurosci. 5(7): Gribble P, Mullin L, Cothros N, Mattar A. 23. Role o cocontraction in arm movement accuracy. J Neurophysiol. 89: Hill A The heat o shortening and the dynamic constants o muscles. Proc R Soc B. (975): Hogan N. 984a. An organizing principle or a class o voluntary movements. J Neurosci. 4(): Hogan N. 984b. Adaptive control o mechanical impedance by coactivation o antagonist muscles. IEEE Trans Automat Contr AC-29(8): Hollerbach J, Suh K Redundancy resolution o manipulators through torque optimisation. IEEE Trans Robot Autom. RA-3(4): Houdijk H, Bobbert M, De Haan A. 26. Evaluation o a Hill based muscle model or the energy cost and eiciency o muscular contraction. J Biomech. 39(3): Katayama M, Kawato M Virtual trajectory and stiness ellipse during multijoint arm movement predicted by neural inverse model. Biol Cybern. 69(5 6): Kawato M, Furukawa K, Suzuki R A hierarchical neural network model or control and learning o voluntary movement. Biol Cybern. 57(3): Kawato M, Maeda Y, Uno Y, Suzuki R. 99. Trajectory ormation o arm movements by cascade neural network model based on minimum torque-change criterion. Biol Cybern. 62(4): Killeen P Mechanics o the animate. J Exp Anal Behav. 57(3): Mashima H, Akazawa K, Kushima H, Fujii K The orceload-velocity relation and the viscous-like orce in the rog skeletal muscle. Jap J Physiol. 22:3 2.

16 Downloaded By: [Tahara, Kenji] At: 5:27 7 May K. Tahara et al. Mclntyre J, Bizzi E Servo hypotheses or the biological control o movement. J Mot Behav. 25(3): Morraso P. (98) Spatial control o arm movements. Exp Brain Res. 42: Nakamura Y. 99. Advanced robotics: redundancy and optimisation. Reading (MA): Addison-Wesley. Osu R, Gomi H, Domen K, Yoshioka T, Kawato M Decrease in joint stiness during motor learning revealed rom measured EMG signals. Paper presented at: The Society or Neuroscience 27th Annual Meeting. 23:22. Perreault E, Heckman C, Sandercock T. 23. Hill muscle model errors during movement are greatest within the physiologically relevant range o motor unit iring rates. J Biomech. 36(2):2 28. Rasmussen J, Damsgaard M, Surma E, Christensen S, de Zee M, Vondrak V. 23. AnyBody a sotware system or ergonomic optimisation. Paper presented at: 5th World Congress on Structural and Multidisciplinary Optimisation. May 9 23, 23; Lido di Jesolo, Venice, Italy. Tahara K, Luo ZW, Arimoto S, Kino H. 25. Sensory-motor control mechanism or reaching movements o a redundant musculo-skeletal arm. J Robot Sys. 22(): Tahara K, Luo ZW, Arimotos. 26. On control mechanism o human-like reaching movements with musculo-skeletal redundancy. In: Proceedings o IEEE/RSJ International Conerence Intellerence Robots and Systems 42 49; Edmonton, Canada. Takegaki M, Arimoto S. 98. A new eedback method or dynamic control o manipulators. ASME J Dynam Syst Meas Control. 2:9 25. Tsuji T, Morasso P, Goto K, Ito K Human hand impedance characteristics during maintained posture. Biol Cybern. 72(6): Todorov E, Jordan M. 22. Optimal eedback control as a theory o motor coordination. Nature Neuroscience. 5():