Vision Sensor. Vision. (Phase 1) pre-shaping. Actuator. Tactile Sensor. Vision. (Phase 2) shaping. Actuator. Tactile Sensor.

Similar documents
OBSTACLE DETECTION USING RING BEAM SYSTEM

Extra Examples for Chapter 1

From E.G. Haug Escape Velocity To the Golden Ratio at the Black Hole. Branko Zivlak, Novi Sad, May 2018

Photographing a time interval

(conservation of momentum)

SAMPLE LABORATORY SESSION FOR JAVA MODULE B. Calculations for Sample Cross-Section 2

Eddy Currents and Magnetic Calibrations in LDX using a Copper Plasma. D.P. Boyle, PPPL M.E. Mauel, D.T. Garnier, Columbia J.

AVS fiziks. Institute for NET/JRF, GATE, IIT-JAM, JEST, TIFR and GRE in PHYSICAL SCIENCES

Time Dilation in Gravity Wells

COMPARING MORE THAN TWO POPULATION MEANS: AN ANALYSIS OF VARIANCE

SKP-2 ALGORITHM: ON FORMING PART AND MACHINE CLUSTERS SEPARATELY

Numerical Modeling in Biomedical Systems

Physics 218, Spring March 2004

Suppose you have a bank account that earns interest at rate r, and you have made an initial deposit of X 0

Red Shift and Blue Shift: A realistic approach

Chapter 4. Sampling of Continuous-Time Signals

Experiment 1 Electric field and electric potential

Correspondence Analysis & Related Methods

Discrete-Time Immersion and Invariance Adaptive Control of a Slider-crank Mechanism

4/18/2005. Statistical Learning Theory

dp p v= = ON SHOCK WAVES AT LARGE DISTANCES FROM THE PLACE OF THEIR ORIGIN By Lev D. Landau J. Phys. U.S.S.R. 9, 496 (1945).

Special relativity with clock synchronization

Lab 10: Newton s Second Law in Rotation

How to Obtain Desirable Transfer Functions in MIMO Systems Under Internal Stability Using Open and Closed Loop Control

IMPLEMENTATION OF MUR S ABSORBING BOUNDARIES WITH PERIODIC STRUCTURES TO SPEED UP THE DESIGN PROCESS USING FINITE-DIFFERENCE TIME-DOMAIN METHOD

Modeling of Vector Controlled Induction Motor in Different Reference Frames

Confidence Intervals for the Squared Multiple Semipartial Correlation Coefficient. James Algina. University of Florida. H. J.

On the Optimal Multirate Control of Networked Control Systems

Khmelnik S.I. Mathematical Model of Dust Whirl

PROPAGATION OF PHOTON IN RESTING AND MOVING MEDIUM. J. Zaleśny. Institute of Physics, Technical University of Szczecin, A b s t r a c t

Generalized Vapor Pressure Prediction Consistent with Cubic Equations of State

Determine the Stress Calculating Mode of Sliding Failure of Soil Mass under the Push-Extend Multi-under-Reamed Pile

Non-Ideal Gas Behavior P.V.T Relationships for Liquid and Solid:

effective pinhole, p=(0,c)

Rigid Body Dynamics 2. CSE169: Computer Animation Instructor: Steve Rotenberg UCSD, Winter 2018

An analytic calculation method on air gap flux in permanent magnet. brushless DC motor with ironless rotor

Absorption Rate into a Small Sphere for a Diffusing Particle Confined in a Large Sphere

Revised Newtonian Formula of Gravity and Equation of Cosmology in Flat Space-Time Transformed from Schwarzschild Solution

Chapter 5 Linear Equations: Basic Theory and Practice

STABILITY AND PARAMETER SENSITIVITY ANALYSES OF AN INDUCTION MOTOR

Special Relativity in Acoustic and Electromagnetic Waves Without Phase Invariance and Lorentz Transformations 1. Introduction n k.

Planck Quantization of Newton and Einstein Gravitation

Quaternion Based Inverse Kinematics for Industrial Robot Manipulators with Euler Wrist

Electric Anisotropy, Magnetic Anisotropy, Uniaxial and Biaxial Materials, Bianisotropic Media (Definitions)

Investigation of Magnitude and Phase Errors in Waveguide Samples for the Nicolson-Ross-Weir Permittivity Technique

A Deep Convolutional Neural Network Based on Nested Residue Number System

Chapter 13 Gravitation

A NEW VARIABLE STIFFNESS SPRING USING A PRESTRESSED MECHANISM

Physics 2020, Spring 2005 Lab 5 page 1 of 8. Lab 5. Magnetism

Analytical Solutions for Confined Aquifers with non constant Pumping using Computer Algebra

An Exact Solution of Navier Stokes Equation

Functions Defined on Fuzzy Real Numbers According to Zadeh s Extension

1 Fundamental Solutions to the Wave Equation

Projection Gravitation, a Projection Force from 5-dimensional Space-time into 4-dimensional Space-time

AN ELECTROMAGNETIC LAUNCH SYSTEM FOR UAVs

On the characteristic of projectively invariant Pseudo-distance on Finsler spaces

EXAM NMR (8N090) November , am

C e f paamete adaptation f (' x) ' ' d _ d ; ; e _e K p K v u ^M() RBF NN ^h( ) _ obot s _ s n W ' f x x xm xm f x xm d Figue : Block diagam of comput

8.022 (E&M) Lecture 13. What we learned about magnetism so far

ASTR415: Problem Set #6

Homework # 3 Solution Key

Particle Systems. University of Texas at Austin CS384G - Computer Graphics Fall 2010 Don Fussell

matschek (ccm2548) Ch17-h3 chiu (57890) 1

MASSACHUSETTS INSTITUTE OF TECHNOLOGY Physics Department. Problem Set 10 Solutions. r s

2 x 8 2 x 2 SKILLS Determine whether the given value is a solution of the. equation. (a) x 2 (b) x 4. (a) x 2 (b) x 4 (a) x 4 (b) x 8

Newton s Laws, Kepler s Laws, and Planetary Orbits

Anisotropic 2-D Wavelet Packets and Rectangular Tiling: Theory and Algorithms

Mass- and light-horizons, black holes' radii, the Schwartzschild metric and the Kerr metric

ELECTROSTATICS::BHSEC MCQ 1. A. B. C. D.

working pages for Paul Richards class notes; do not copy or circulate without permission from PGR 2004/11/3 10:50

DARK MATTER AND THE DYNAMICS OF GALAXIES: A NEWTONIAN APPROACH 1. INTRODUCTION

Motithang Higher Secondary School Thimphu Thromde Mid Term Examination 2016 Subject: Mathematics Full Marks: 100

B. Spherical Wave Propagation

Markscheme May 2017 Calculus Higher level Paper 3

7.2.1 Basic relations for Torsion of Circular Members

e sin cos i sin sin j cos k [2 POINTS] (c) Hence, determine expressions for sin sin i sin cos j sin e

Perturbation to Symmetries and Adiabatic Invariants of Nonholonomic Dynamical System of Relative Motion

Physics 2A Chapter 10 - Moment of Inertia Fall 2018

Duality between Statical and Kinematical Engineering Systems

Gain-Scheduled Controller Design: An Analytic Framework Directly Incorporating Non-Equilibrium Plant Dynamics

Related Rates - the Basics

The Kerr-metric, mass- and light-horizons, and black holes' radii.

2.5 The Quarter-Wave Transformer

Analysis of high speed machining center spindle dynamic unit structure performance Yuan guowei

Magnetometer Calibration Algorithm Based on Analytic Geometry Transform Yongjian Yang, Xiaolong Xiao1,Wu Liao

In electrostatics, the electric field E and its sources (charges) are related by Gauss s law: Surface

Motion in One Dimension

MASSACHUSETTS INSTITUTE OF TECHNOLOGY Physics Department Physics 8.07: Electromagnetism II September 15, 2012 Prof. Alan Guth PROBLEM SET 2

PHYS 301 HOMEWORK #10 (Optional HW)

1 Spherical multipole moments

PHYS 110B - HW #7 Fall 2005, Solutions by David Pace Equations referenced as Eq. # are from Griffiths Problem statements are paraphrased

EFFECTS OF FRINGING FIELDS ON SINGLE PARTICLE DYNAMICS. M. Bassetti and C. Biscari INFN-LNF, CP 13, Frascati (RM), Italy

Universal Gravitation

EN40: Dynamics and Vibrations. Midterm Examination Thursday March

1 Fundamental Solutions to the Wave Equation

A Theory of the Podkletnov Effect based on General Relativity: Anti-Gravity Force due to the Perturbed Non-Holonomic Background of Space

On the indirect e ect in the Stokes±Helmert method of geoid determination

transformation Earth V-curve (meridian) λ Conical projection. u,v curves on the datum surface projected as U,V curves on the projection surface

Central Force Motion

17.1 Electric Potential Energy. Equipotential Lines. PE = energy associated with an arrangement of objects that exert forces on each other

Transcription:

Optimal Gasping using Visual and Tatile Feedbak Akio NAMIKI Masatoshi ISHIKAWA Depatment of Mathematial Engineeing and Infomation Physis Univesity of Tokyo Tokyo 3, Japan namik@k.t.u-tokyo.a.jp Abstat Senso feedbak and senso fusion ae indispensable fo woking in the eal wold. In this pape a obot hand gasping method is poposed. This method uses visual and tatile senso infomation. Fist, gasping evaluation funtions is poposed, whih is deived fom the elationships between a hand and an objet. Then a ontol method is poposed whih uses visual and tatile feedbak. In this method pe-shaping and gasping ae exeuted smoothly and optimally. Expeimental esults ae pesented and a \ms Sensoy-Moto System" is intodued as the expeimental system fo senso fusion. Key Wods: gasping, pe-shaping, multisenso fusion, senso feedbak, vision, tatile senso Intodution Consideing obot hand gasping poess in the eal wold, a manipulated objet and envionment ae mostly unknown. Then thee is the possibility that some aidents happen while gasping. A human being an easily solve these poblems. Fo example we an gasp a moving objet by binging ou hand nea it. Then we an hold an objet in dakness using its tatile impession. To exeute these exellent gasp faulties a gasping ontol method using senso feedbak is needed. By using extenal sensoy infomation, the obot system an ope with many poblems. Most ealy wok to solve gasping poblems was based on o-line omputing [, 3]. In these systems gasping is not obust and omputing powe is vey expensive beause all infomation should be obseved befoe gasping. This is not ealisti ondition in the eal wold. Then thee ae some woks based on on-line omputing. Tinkle gives an estimation in fitionless enveloping gasping []. But this is not a method to seah optimal gasping atively. Then Coelho gives a method to seah optimal gasping [5]. But senso feedbak is not suiently onsideed in this method. It is neessay to make a gasping method using eal-time senso feedbak. Then it is moe eetive to use many senso feedbaks and senso fusion algoithms []. We addess the poblem of a gasping method using senso feedbaks, patiulaly visual and tatile feedbak. In the next setion we disuss the elationship between gasping poess and senso feedbaks. Gasping using Senso Feedbak The gasping poess an be lassied into two phases (Figue ). (Phase ) pe-shaping (Phase ) shaping Vision Atuato Tatile Senso Vision Atuato Tatile Senso Vision Senso Tatile Senso Figue : Gasping using Senso Feedbak (Phase ) Pe-shaping This is the opeation duing whih the hand hanges its position and shape befoe the nges touh the objet to be manipulated. This opeation is obseved in human gasping and is alled \pe-shaping" [6]. Speedy and obust gasping is exeuted by this opeation. Visual feedbak is the main ontol method in this phase. (Phase ) Shaping This is the opeation duing whih the hand hanges its position and shape afte ontat. We all this opeation \shaping". Sine both the nges' motion and sensing ae dietly onneted, obsevation eos ae smalle. This opeation mainly onsists of tatile o foe senso feedbak, but visual feedbak is also used to obtain global infomation. 589

High Level Laye Low Level Laye Planning Intenal Model Vision Senso Atuato Tatile Senso O p θ C o O C k Finge n u k u k l k Objet n d k Figue 3: Paametes of Gasping d k k: ontat numbe i: finge numbe j: joint numbe ij k C ij Objet Hand witten as Vision Figue : Gasping System using Senso Feedbaks To integate these two phases, we popose a senso feedbak gasping system. Figue shows an outline of the integated system. This system onsists of two layes. In the high laye the evaluation funtions fo gasping is alulated. This evaluation uses both visual and tatile infomation. In the low laye the motion ontol using senso feedbaks is exeuted. We disuss the evaluation method in Setion, and the motion ontol method in Setion 5. 3 Kinematis of Vitual Contat Model As pepaations, fundamental equations fo gasping ae deived fom ontat elationships. We assume the point-ontat model [7]. Conventional models assume omplete ontat, whih means all nges ae in ontat with an objet. By adding the idea of \vitual ontat", we develop the model so that they inlude non-ontat phase. Consideing a point d k R3 on a nge and a point u k R3 on a manipulated objet, both expessed in the base fame (Figue 3), these points satisfy the equation as d k 0 u k = n u k l k = 0n d k l k; k = ; ; ; m ; () whee k is the numbe of the ontat point. A veto n u k R3 is the nomal veto on the objet, and n d k R 3 is the nomal veto on the nge. The minimum distane between the nge and the objet is expessed as l k R. We all l k the \ontat distane". And we all u k and d k the \vitual ontat points". If l k = 0, the vitual ontat point k is equal to the odinay ontat point. Then linea veloities of vitual ontat points an be _ u k = W k _ + v u k ; () _ d k = J k _ + Hk _p + v d k ; (3) whee R 6 is the wenh veto of the objet gavity ente, R m is the joint angle veto, whee m is the total numbe of joints. A veto p R 6 is the wenh veto of the palm position, v u k R3 and v d k R3 ae the veloity vetos of slip. Eah vaiable is expessed in the base fame. The \wenh veto" is the ombination of a 3-dimensional position veto and a 3-dimensional oientation veto. Thee ae some basi maties, W k R 36 is alled the \Gasp Matix", J k R 3m and H k R 36 ae the Jaobian maties. Substituting Eqn.() and Eqn.(3) to the dieential of Eqn.(), the dieential elation an be witten as 0 W k _ + J k _ + Hk _p = v u k 0 vd k + _n u k l k + n u k _ l k : () We sepaate the nomal oientation pat and the tangential oientation pat at eah vitual ontat point. Multiplying both sides by n u k T, the nomal oientation pat is witten as n u k T (0W k _ + J k _ + Hk _p) = _ lk ; (5) whee it should be notied that n u k T v u k = 0, nu k T v d k = 0 and n u k T _n u k = 0. Juxtaposing all suh expessions fo the m ontat points in matix notation, the equation of nomal pat is witten as N T (0W _ + J _ + H _p) = _ l; (6) whee 3 n u 0 0 0 n u 0 N = 6..... 7. 5 R3m m ; 0 n u m 3 3 W J W W = 6. 7 5 R3m 6 J ; J = 6. 7 5 R3m m ; W m J m 590

H = 6 H H. H m 3 7 5 R3m 6 ; l = 6 l l. l m 3 7 5 R3m : Eqn. (6) is the basi equation of gasping. If l k = 0, the ontat onstaint _ lk 0 is imposed. Satisfying _ lk = 0, the ontat point k is peseved. Evaluation of Gasping Thee ae many woks about evaluation of gasping. These ae onened about stability [, 3], mobility [8], obustness, and othes. But in most woks gasping ontol and senso feedbak ae not onsideed. We popose new funtions fo evaluation whih is deived fom the fundamental ontat model in Setion 3. These funtions and thei dieentials an be easily alulated using obseved values. Fo this eason these an be easily inluded in senso feedbak. This method onsists of two kinds of evaluation funtions. One is about gasp ondition, whih evaluates gasp stability, obustness, and othe popeties. Anothe is about ontat distane, whih evaluates the distane between nges and a manipulated objet.. Evaluation of Gasping Condition We onside evaluation funtions about the gasp ondition. Beause a palm geneally has a lage moment of inetia than nges and a manipulated objet we an ignoe _p, so that the equation is deived fom Eqn. (6) as N T (0W _ + J _ ) = _ l 0; (7) whee we onside the onstaint _ l 0 on ontat points. Eqn. (7) is egaded as elations among _, _ and _ l. Thee ae fou elations shown in Figue among these paametes. Eah elation is expessed by a linea tansfoma- To evaluate this linea tansfomation, we alulate the size of the aea made fom x in the ase when y is given as the unit sphee aea jjyjj. The pojetion of the unit sphee is the ellipsoid as x T (N T BB T N) + x ; x 0; (9) whee the \+" opeato expesses the genealized invese. It is diult to alulate this aea analytially, so we popose anothe method. By using the singula value deomposition, the pojetion matix is deomposed as N T B = U3V T. Consideing the inne podut of the axis vetos of the ellipsoid and the veto, in whih eah element is, this oesponds to the size of the pojeted aea in ental oientation of the onstaint spae. On the othe hand, onsideing the tae of the summation of the squaes of axis lengths, this oesponds to the size of the pojeted aea. By using the podut of these two values, the size of pojeted aea in the onstaint spae an be evaluated. These ae shown as Evaluation = jj3u T jj = T N T BB T N; Evaluation = Tae(3 ) = Tae(N T BB T N): Applying these two equations to ontat elationships, we an deive funtions whih ae desibed by the following equations: () Funtion of Contat Veloity " veloity = m T N T W W T N; (0) " veloity = m Tae(N T W W T N); () () Funtion of Passive Joint Foe/Toque " passive = m T N T JJ T N; () " passive = m Tae(N T JJ T N); (3) Objet (3) () Hand θ (3) Funtion of Contat Foe " foe = m T N T W +T W + N; () () l () Vitual Contat Point Figue : Relations among Paametes tion. Conside a linea tansfomation matix with the onstaint as x = N T By; x 0: (8) " foe = m Tae(N T W +T W + N); (5) () Funtion of Ative Joint Foe/Toque " ative = m T N T J +T J + N; (6) " ative = m Tae(N T J +T J + N); (7) whee m is the total numbe of vitual ontat points. 59

If the funtions " veloity and " veloity ae minimized, gasping is ealized that ontat veloity at eah ontat point beome minimum when the objet is moved. This means that it is diult to sepaate nges fom an objet. Then if thee ae enough ontat points, objet motion is onstained. This oesponds to the \fom losue" without slip. If the funtions " foe and " f oe ae minimized, gasping is ealized that ontat foes beome minimum when extenal foe is applied to the objet. This means that gasp is obust against distubanes. Then if thee ae enough ontat points, objet motion is onstained. This oespond to the \foe losue" without fition. If the funtions " passive and " passive ae minimized, gasping is ealized that passive joint toques ae minimized when ontat foes ae applied to nges. If the funtions " ative and " ative ae maximized, gasping is ealized that joint toques applied to the objet atively ae maximized. By ombining these funtions, the funtions fo desiable gasping is dened as: Funtion of Gasp Condition " gasp = f(" f oe " foe ; " passive " passive ; " veloity " veloity ; " ative " ative );(8) whee funtion f() expesses the linea summation of eah element o eah invese element.. Funtion of Contat Distane We evaluate the ontat distane on the vitual ontat points with the following funtion: Funtion of Contat Distane " ontat = m l T M l l; (9) whee M l R m m is a weight matix. If omplete ontat is ahieved on eah vitual ontat point, the value of this funtion equals zeo. 5 Gasping Algoithm In this pape we dene the \optimal gasping" as the gasping poess, in whih the evaluation funtions about both the gasp ondition " gasp and the ontat distane " ontat ae minimized. In this setion a planning algoithm fo optimal gasping is poposed. This algoithm uses the evaluation funtions in Setion. In this algoithm position and shape of hand is gadually hanged using senso feedbak. 5. Optimal Planning using Evaluations Assuming that the objet does not move duing gasp motion, namely _ = 0. Substituting this into Eqn. (6), the equation of vitual ontat is deived: G _q = _ l; (0) whee G = N T [J H] R m (m +6), q = ( T ; p T ) T R m +6. We all a veto q the \hand paamete", whih is a ontollable paamete. By using the genealized invese of G, the equation of the hand paamete is deived: _q = G + _ l + (I 0 G + G) _x; () whee the st tem is the patiula solution and depends on the deivative of the ontat distane. The seond tem is the solution of G _q = 0 and _x is an abitay veto. Eah ontat distane is invaiable duing motion by the seond tem. We adopt Eqn. () fo planning gasp. By using evaluation funtions " gasp and " ontat in Setion, the planning in eah step is alulated as q k+ = q k + k k ; () k = 0 G + @" ontat 0 g (I 0 G + G) @" gasp @l @q = 0 G + M l l 0 g (I 0 G + G) @" gasp ; (3) m @q whee k is the numbe of step, k, and g ae a suitable sala. In Eqn. () we omit the alulation of @q @" gasp beause this has omplex expession. This an be alulated if objet paametes (nomal vetos and uvatues at ontat points) ae obseved. It should be notied that the st tem is othogonal to the seond tem, so that the planning motion by " gasp is onsistent with it by " ontat. Then planning gasp is smoothly exeuted though both the non-ontat phase and the ontat phase. 5. Compensation fo Objet Motion We onside the ase when a manipulated objet is moving. The elation among the objet oodinate system C o, the palm oodinate system C p and the base oodinate system C b is witten as T b p = T b T p ; () whee T b, T b p and T p ae espetively the homogeneous tansfomation matix fom C b to C, one fom C b to C p and one fom C to C p. 59

C C p p C b o Figue 5: Compensation fo Objet Motion whee eal is the inement by the motion of the eal objet, vitual is the inement by the motion of the vitual objet. Gasp motion fom a emote plae is exeuted by using this method. It is neessay to onside olusion between nges and the objet, but this poblem will be onsideed in futue. Sensing q Intenal Senso Tatile Senso Visual Senso The onstaint of ompensation is that the matix Tp is onstant. Unde this onstaint the deivative of Eqn. () is shown as T_ p b = T_ b T p : (5) The homogeneous maties T_ p b and T_ b an be espetively expessed by the palm wenh veto p and the wenh veto of the ente of gavity. This elationship is shown as _p = V p _: (6) We add this equation to Eqn. (3) to ompensate fo objet motion as q k+ = q k + k k ; (7) k = 0 G + M l l 0 g (I 0 G + G) @" gasp @q ; (8) +V q whee V q = [O m V p T ] T. 5.3 Gasping of Vitual Objet Model If a obot hand is fa fom a manipulated objet, we annot use the poposed method beause vitual ontat points do not exist. This poblem is solved by onsideing the \vitual objet model" (Figue 6). In this method, an objet model is vitually onsideed nea a hand and gasping motion is exeuted to this model. By binging the vitual model to the eal objet while gasping the vitual model, gasping motion and appoahing motion an be simultaneously exeuted. 0 vitual objet model eal objet Figue 6: Gasping of Vitual Objet Model Fist the obit and the veloity of the gavity ente of the vitual objet model ae detemined. Then the inement should be substituted in Eqn. (7) instead of the inement the motion of the eal objet. If the eal objet is moving, the summation is substituted as = eal + vitual ; (9) Paamete Calulation Planning evaluate funtion Contolle Foe/Toque Senso F Gasping Paamete Gasp Condition Evaluation dε dq τ Tatile Senso Objet Afte Contat Befoe Contat vitual obit Estimation Calulation Contat Distane Evaluation dε dl Planning q Contolle τ Atuato Foe Senso Hand Vision Objet Motion Figue 7: Gasping Algoithm 5. Senso Fusion fo Gasping To use the poposed planning method, it is neessay to integate visual and tatile infomation. We adopt the stategy that st we get global infomation about the objet using vision and next we update the loal infomation using the tatile senso. We assume that an objet wenh veto an be alulated using visual infomation. If the model of the objet is known, this an be alulated by measuing moe than 3-points on the objet. Then we assume the ontat points veto k an be alulated using tatile infomation. Unde this assumption, the poedue of senso data fusion is witten as follows.. Befoe ontat, and k, whih is a position of a vitual ontat point, ae alulated using visual infomation. The homogeneous matix T is alulated by using this two equations.. If nge ontats the objet, k, whih is a position of a ontat point is obseved using tatile infomation. The homogeneous matix T is updated using this vetos. 593

C0 Wok Station Netwok (LAN) DSP Laye Wok Station Laye C0 C0 C0 C0 Intefae Laye PIO (pot) DA (8CH) AD (3CH) DA (8CH) DA (8CH) AD (3CH) Video IO PIO (pot) DA (8CH) AD (3CH) Enode (7CH) AC Sevo AC AC AC Sevo 7CH Foe/Toque Senso (6CH) Joint Toque Senso (7CH) DC Sevo DC DC Sevo Sevo DC Sevo CH Foe/Toque Senso Potentio Mete (CH) (CH) SPE DC DC Sevo Sevo CH Lens Lens Contolle Contolle Lens Contolle 3CH Potentio Mete (5CH) 7 Axis Manipulate Finge Dexteous Hand Expeimental System CCD Camea Supe High Speed Ative Vision Figue 8: ms Sensoy-Moto Fusion System 3. Afte ontat if the objet o the vitual objet moves, is obseved using visual infomation. Fom this value T is updated.. Using both T and the objet model, the nomal veto n k an be alulated. Using, n k, k and the joint angle veto, whih is obseved by joint angle senso, eah evaluation funtion is alulated. 5. The hand motion is planned by these evaluation funtions. In this poedue, the homogeneous matix T means tansfomation fom a ontat point k to the objet ente gavity veto. As the method to update T least squaes method an be used if the objet model is known. 5.5 Gasping Algoithm The whole algoithm is shown in Figue 7. This blok diagam onsists of fou pats. In the sensing pat visual infomation, tatile infomation and joint angle infomation is obtained. In the paamete alulation pat gasping paametes ae alulated using eah infomation and senso fusion is exeuted. In the planning pat dieentials of evaluation funtions ae alulated and objet motion is planned using Eqn. (7). In the ontol pat, ontol is exeuted aoding to this planning. These fou pats ae inluded in the senso feedbak loop. One of impotant advantages of this algoithm is that this is obust against distubane and eos, fo example modeling eos o estimated eos, beause senso feedbak ompensates fo these poblems. Anothe is that this method onnets gasping in non-ontat phase to in ontat phase. Then gasping is smoothly and speedily exeuted though all phases by using this method. 6 ms Sensoy-Moto Fusion System In this setion, ms Sensoy-Moto Fusion System is intodued as the expeimental system shown in Figue 8 [9]. This system has two main featues: () high speed senso feedbak In most onventional systems extenal sensos, suh as vision o tatile sensos, ae not inluded in feedbak loops. This is beause these sensos ae too slow and the ontolle does not have enough omputing powe. To solve this poblem, we pepae a high speed vision system and poessos with stong omputing powe. As a esult ou system has the ability to exeute senso feedbak in ms. () multi-senso fusion By using plual sensos, moe auate and moe obust infomation an be obtained []. In the obot system, not only sensos but also atuatos ae neessay. Ou system has many sensos and atuatos. 59

This system onsists of a ontolle and obot systems. In the ontolle, we adopt a DSP (TMS30C0) as the poesso, whih has the ability to exeute high speed alulation and to ommuniate with othe poessos. We onstut a high speed ontolle with a paallel ahitetue using a netwok of DSPs. This netwok is onneted with many intefae pots. Though these pots the ontolle is onneted with the obot systems,, whih onsists of an ative-vision system and a hand-am system. The ative vision system has the high speed visual poessing system SPE (Sensoy Poessing Elements). The expeiment of high speed taget taking is exeuted on this system [0]. The hand-am system onsists of a 7-axes manipulato and a dextous hand. Ou gasping algoithm needs muh omputational powe and high speed feedbak, then ou ms feedbak system is suitable fo this expeiment. 7 Expeiment 7. Expeimental System We use the -ngeed dextous hand in the ms feedbak system. The palm of the hand is xed and only the nges an move. The hand has a potentio mete and a foe senso at eah joint. By using these sensos the position and the ontat foe at eah nge tip an be measued. Beause this hand does not have a tatile senso now, we use these sensos fo obtaining the positions of ontat points instead. Assuming only ngetip ontat, these paametes an be alulated. Fo vision we use a CCD amea beause high esolution is needed in this expeiment. The system is shown in Figue 9. In this expeiment we use the evaluation funtion of ontat distane and the evaluation funtion of ontat veloity. The planning method desibed in Setion 5 is used. Beause it is diult to exeute both ontol and estimation at the same time we adopt the algoithm shown in Figue 0. In this algoithm the time inteval of position feedbak ontol and foe feedbak ontol is within 3ms. But visual feedbak is exeuted only time evey 600 steps and tatile feedbak is exeuted only time evey 300 steps. This is beause the CCD amea fame ate is slow and fo tatile sensing it is neessay that all nges touh the objet one with suient ontat foes. We pepaed a hexahedon as a manipulated objet. The model of the objet was given befoehand and the system estimated paametes of the objet (the wenh veto of gavity, nomal vetos) by using visual and tatile infomation with planning. In this algoithm two estimation methods wee used. One was the estimation of the homogeneous matix T Yes Figue 9: Expeimental System Estimation using Vision Yes Stat i= i> j= j>300 No No Planning using Eqn.() Objet motion is deteted j=j+ No Yes Foe Contol Contat is deteted Yes No Estimation using Tatile Senso i=i+ Figue 0: Algoithm in Expeiment using visual infomation. The seond was the estimation of the ontat plane using tatile infomation. We used least squaes method in both ases. In the st six maked points wee measued on the objet using the CCD amea. By using measued values and the objet model the homogeneous matix was alulated. In the seond paametes of the ontat plane wee estimated using vetos of ngetip position. The objet shape and the matix T wee updated fom elationships between plane paametes and the ente of gavity veto. 7. Expeimental Results Figue shows the hand gasping motion with time. In this gue eah nge goes to the ente of the ontat plane, whih is the neaest point fom the ente of gavity and the most diult point fom whih a nge sepaates. Finges wee moved suh that the objet was 595

(mm) 0-50 afte moving small. This poblem will be onsideed in futue. In this expeiment we annot show the eetiveness of algoithm in high speed gasping beause the sensos ae slow. But now a high speed vision hip is being developed in ou laboatoy. If suh high-speed sensos ae ealized the validity of ou algoithm an be demonstated. 000 3500 3000 500 000 500 000 500 0 50 00 50 00 (mm) -50 Hand fist position optimal position afte objet is moved optimal position befoe objet is moved befoe moving 0 Objet 50 Figue : Results of Hand Planning estimation using visual feedbak objet obseved by vision objet obseved by tatile senso ε ontat ε gasp x0 00 (mm) objet is moved 8 Conlusion A gasping algoithm using visual and tatile feedbak is intodued. This algoithm is obust against distubanes and eetive in all gasping phases, that is both ontat and non-ontat phase. The algoithm onsists of two pats. One is an evaluation funtion of gasping. This an be be easily inluded in the senso feedbak loop and evaluate gasping in both the ontat phase and the non-ontat phase. The seond pat is a planning method of gasping. We popose a new method deived fom the fundamental ontat model. This is also exeuted in all gasping phases. Visual and tatile infomation is integated by using the objet loal model in this method. Expeimental esults ae shown. In the ms Sensoy- Moto Fusion System, whih inludes a -ngeed oboti hand and a CCD amea fo vision, we demonstated the gasping of a hexahedon. The esults show the eetiveness of ou algoithm. 0 0 00 00 600 800 000 00 00 600 (steps) estimation using tatile feedbak Figue : Result of Evaluation Values onstained against the oientation of the nomal vetos of ontat planes. This esult mathes the popety of the evaluation funtion of ontat veloity. Figue shows how evaluation funtions vay with time. In this gue both evaluation funtions go to zeo and these two values ae minimized in paallel. When the objet is moved, the values of evaluation funtions beome tempoaily high. But new paametes ae estimated by visual feedbak so that evaluation funtions go to zeo again. Afte estimation using visual and tatile infomation the values of evaluation funtions also beome high. This is beause of estimated eos. Howeve in this ase optimization is also exeuted and these values go to zeo again. These esults shows that this system is obust against eos and distubanes beause they ae ompensated by senso feedbak. Thee is a poblem that motion of the hand wings if estimated values swing. To solve this poblem it is neessay to use estimation that it an be exeuted in eal-time and whose estimated eos ae Refeenes [] M. Ishikawa. Senso fusion : The state of the at. J. of Robotis and Mehatonis, {:35{, 99. [] V. Nguyen. Constuting foe-losue gasps. Int. J. of Robotis Reseah, 7(3):3{6, 988. [3] X. Makenso and C. H. Papadimitiou. Optimum gip of a polygon. Int. J. of Robotis Reseah, 8():7{9, 989. [] J. C. Tinkle, J. M. Abel, and R. P. Paul. Enveloping, fitionless, plana gasping. Po. IEEE Int. Conf. Robotis and Automation, pages 6{5, 987. [5] J. A. Coelho J. and R. A. Gupen. Optimal multingeed gasp synthesis. Po. IEEE Int. Conf. on Robotis and Automation, pages 937{9, 99. [6] J. Ewet and M. A. Abib, editos. Visuomoto Coodination: Neual Models and Peeptual Robotis. Plenum, 989. [7] J. Ke and B. Roth. Analysis of multingeed hands. Int. J. of Robotis Reseah, ():3{7, 986. [8] A. Bihi, C. Melhioi, and D. Balluhi. On the mobility and manipulability of geneal multiple limb obots. Po. IEEE Int. Conf. Robotis and Automation, ():5{8, 995. [9] A. Namiki, Y. Nakabo, I. Ishii, M. Ishikawa. ms Sensoy- Moto Fusion System. To be appea. [0] I. Ishii, Y. Nakabo, and M. Ishikawa. Taget taking algoithm fo ms visual feedbak system using massively paallel poessing vision. Po. IEEE Int. Conf. on Robotis and Automation, pages 309{3, 996. 596