Shaping Event-Based Haptic Transients Via an Improved Understanding of Real Contact Dynamics

Transcription

1 Shaping Event-Based Haptic Transients Via an Iproved Understanding of Real Contact Dynaics Jonathan P. Fiene Katherine J. Kuchenbecker Telerobotics Lab, Stanford University, USA Abstract Haptic interactions with stiff virtual surfaces feel ore realistic when a short-duration transient is added to the spring force at contact. But how should this event-based transient be shaped? To answer this question, we present a targeted user study on virtual surface realis that deonstrates the iportance of scaling transients correctly and hints at the coplexity of this dynaic relationship. We then present a detailed exaination of the dynaics of tapping on a rigid surface with a hand-held probe; theoretical odeling is cobined with epirical data to deterine the influence of ipact velocity, ipact acceleration, and user grip force on the resulting transient surface force. The derived atheatical relationships provide a forula for generating open-loop, event-based force transients upon ipact with a virtual surface. By incorporating an understanding of the dynaics of real interactions into the re-creation of virtual contact, these findings proise to iprove the perforance and realis of a wide range of haptic siulations. 1 Introduction Realistic excitation of the user s sense of touch during virtual interactions has been shown to decrease task copletion tie, error incidence, and cognitive load [2, 14]. Nuerous applications exist wherein realistic virtual siulations could have significant potential, ranging fro edical training to echanical design. For instance, a sufficiently realistic siulator could be used to ierse surgeons in coplex operative tasks without endangering live patients, thereby providing a safe, repeatable environent for learning and practicing new procedures. Such siulators can be used for both traditional and inially-invasive surgery (MIS), with particular application in the eergent field of robotic MIS. To be a viable alternative to traditional training, systes ust faithfully reproduce the haptic cues that surgeons experience during live procedures, a design goal that requires careful attention to the dynaics of the corresponding real interactions. While today s ipedance-type haptic siulators can adeptly portray interactions with soft environents, rendering realistic rigid contact has been a significantly ore challenging objective. A large body of research over the past decade has sought to iprove the feel of virtual hard contact, and one of the ost proising advances has been recognizing the value of transient contact forces. An overview of this field is presented in Section 2, and Section 3 presents a huan subject study that exaines how transient aplitude affects surface realis. In Section 4, we introduce a set of dynaic odels based on first principles to reveal the relationship between the echanical paraeters of the contacting objects (stylus and hand ass, surface stiffness, and surface daping), the user-controlled ipact paraeters (incoing velocity, incoing acceleration, and grip force), and the shape of the resulting force transient. Understanding the dynaics of contact between a hand-held stylus and a physical hard object will facilitate the process of virtual transient selection and enable ore realistic virtual environents. 2 Background While both slow pressing and discrete taps yield useful inforation about an unknown aterial touched through a stylus, LaMotte showed that active tapping best enables huans to ake accurate aterial property judgents, as the quickly changing contact force provides salient cues about surface copliance [1]. During interactions with rigid surfaces, haptic feedback necessarily becoes the priary sensory channel because the tool s penetration into the surface is too sall to be detected via vision or proprioception. As shown in Fig. 1, tapping a stylus on a fir surface produces a high-frequency transient acceleration, which is known to excite the Pacinian corpuscles that lay deep within the glabrous skin of the hand [1, 17]. The central nervous

2 Acceleration (/s 2 ) Tie (s) 1 Figure 1: The acceleration of a hand-held probe when tapped on a fir surface. syste processes this sensory response and generates a haptic ipression of the event, which depends priarily on the aterial and geoetry of the object and is naturally controlled for changes in contact velocity and grip force [1]. Contact acceleration transients like the one shown in Fig. 1 are often odeled with an exponentially decaying sinusoid, where the aplitude is linearly proportional to the ipact velocity and the frequency and duration depend epirically on the aterial properties of the surface and tool [12, 16]. Unfortunately, the low closed-loop bandwidth of ost existing ipedance-type haptic interfaces restricts the axiu virtual stiffness to that of soft foa [3, 8]. Siulations that attept to portray hard aterials with closed-loop position feedback thus feel far too soft and lack the high-frequency accelerations that users expect when interacting with hard objects. While efforts have been ade to overcoe these liitations, the required 1-fold iproveent in renderable stiffness reains elusive. One proising alternative for rendering stiff contact is the paradig of event-based haptics, wherein traditional position-based feedback is augented with the open-loop display of high-frequency force transients. Wellan and Howe first ipleented such an algorith on a haptic device, providing vibratory feedback via a secondary actuator ounted near the user s fingertips [16]. Shortly thereafter, Salcudean and Vlaar showed that an open-loop pulse at contact iproved the perceived stiffness of virtual walls without causing instability [13], and Okaura et al. deonstrated that overlaying proportional feedback with psychophysically-tuned decaying sinusoids iproved aterial discriination [11]. Hwang et al. continued this line of research through the use of short-duration force pulses to bring the stylus to rest with inial penetration [7]. Kuchenbecker et al. then developed a deterinistic approach to transfor real recorded accelerations into openloop otor current transients to be played at contact, using a full dynaic odel of the haptic syste to ensure accurate atching of the virtual and real accelerations [8]. A copleentary user study of blind tapping on real and virtual surfaces showed that the inclusion of fixed-width pulse, decaying sinusoid, or acceleration atched transients significantly iproved the perceived realis of virtual surfaces over proportional feedback. Further work by Fiene et al. deonstrated that a grip-force sensor could be used to estiate changing hand dynaics, which could then be accounted for in real tie by adjusting the syste s dynaic odel [4]. Such a syste was shown to be capable of producing event-based virtual surfaces that accurately recreated recorded transient accelerations over a wide range of grip force values and incoing velocities. 3 Perception of Transient Aplitude Prior work has shown that event-based transients generally iprove the realis of virtual hard contact, but little is known about the relationship between transient shape and surface feel. We hypothesize that the perceived realis of a virtually rendered surface is strongly dependent on how the aplitude of the acceleration transient copares to that produced during interactions with the corresponding real surface. A brief user study was conducted to test this hypothesis, as presented below. 3.1 Experiental Design For direct user coparison between real and virtual surfaces, the workspace of a desktop Phanto was divided laterally between a virtual rendering and a real object (a layer of wood on a foa substrate, siilar to that used in [8]). A stylus was attached to the distal link of the Phanto to allow the user to tap on both surfaces. An Analog Devices ADXL321 ±18 g acceleroeter was affixed near the tip of the stylus, and a Flexi-force A21-1 force-sensitive resistor was located under the user s index finger to easure grip force. A PC running RTAI Linux was used to render the virtual surface and saple the sensors at 5 khz using a National Instruents PCI-12 card, providing resolutions of.172 /s 2 and.9 N respectively. The user s haptic sense was isolated through the use of a visual barrier and headphones playing cacophonous noise. The virtual surface consisted of a 5 s long, 5 Hz decaying-sinusoid transient overlaid on a 9 N/ spring. For each tap on the virtual surface, the 32 Ns/ transient aplitude was scaled by the incoing velocity and ultiplied by a rando value between and 2 to introduce wide variations in contact response. Users were instructed to alternate tap on the real and virtual surfaces. After each pair of taps, users were asked to rate the feel of the virtual surface on a seven-point bipolar rating scale, where a rating of corresponds to a realistic virtual rendering, +3 is far too strong and -3 is far too weak. 3.2 Results Three users participated in the study, each rating 62 virtual surfaces. To understand the significance of these rat-

3 Aplitude Ratio Too Weak User Rating Too Strong Figure 2: Relationship between realis ratings and the virtual-to-real transient aplitude ratio. ings, we quantitatively copared each virtual contact to the response of the real surface, as follows. First, a linear fit between incoing velocity and peak contact acceleration for taps on the real surface was deterined for each user. Then the aplitude ratio of each virtual tap was calculated by dividing the easured peak acceleration by that predicted using the linear scaling odel. Figure 2 shows the ean and standard deviation of these ratios for each realis rating averaged across subjects; as hypothesized, aplitude ratio correlates well with rated realis. Interestingly, the best realis rating () corresponds to a ean aplitude ratio of approxiately.86, echoing the anecdotal observation of [8] that users prefer virtual transients that are slightly less powerful than real surfaces. Additionally, the wide standard deviations of Fig. 2 show that an aplitude ratio that is corrected for velocity alone does not fully characterize the intricacies of user perception, though it captures a strong trend. As such, the reainder of this work will exaine the dynaics of real interactions to ore thoroughly understand the factors controlling transient aplitude. 4 Modeling and Analysis of Real Tapping Event-based haptic algoriths coonly scale or index the aplitude of the contact transient by the velocity with which the user ipacts the virtual surface [8, 11, 16]. While this relationship significantly iproves the feel of hard virtual surfaces, it does not provide an obvious way to incorporate the effects of other varying paraeters such as incoing acceleration and changes in hand dynaics. To understand how these other paraeters affect the resulting surface force, an ATI Mini-4 force sensor was placed beneath a saple object, and the Phanto setup described above was used to easure position, acceleration, grip force, and surface force for a series of 3 real taps spanning a range of incoing velocities, incoing accelerations, and grip force levels. To elucidate the relationships between the usercontrolled paraeters and the resulting surface force, this section introduces three successive dynaic odels of the stylus/hand syste at contact and tests the against this recorded tap data. In all odels, hand otion (position, velocity, and acceleration) and surface force are defined positive away fro the surface, such that negative incoing velocities cause positive surface forces. 4.1 Moentu Approach The first explored odel treats the stylus/hand syste as a luped ass,, with a constant pre-ipact velocity of v in. Under conservation of oentu, perfectly plastic ipact with a stationary surface will produce zero velocity after tie t 1, and the change in linear oentu, L, of the ass ust equal the integral of the surface force, as follows L 1 L = v in = t1 F s (t) dt (1) Assuing that the contact acceleration follows an exponentially decaying sinusoid, the surface force will take the for F s (t) = β sin(ωt) e αt (2) where the frequency, ω, and decay rate, α, are epirically deterined fro saple contacts. Substituting (2) into (1), solving for β, and rewriting (2) yields F s (t) = (α2 + ω 2 ) v in sin(ωt)e αt (3) ω which provides the previously observed linear correlation between incoing velocity and transient force aplitude. The peak of the decaying sinusoid surface force, F s = v in α2 + ω 2 e (α/ω) tan 1 (ω/α) is therefore also proportional to the incoing velocity. Least squares regression between the incoing velocity and the peak of the surface force for the recorded data yields the linear fit shown in Fig. 3, which has an RMS error of.25 N. While this odel captures the priary trend, the scatter suggests that other factors contribute to the dynaics of a tap, echoing the results of the user study in Sec. 3. Peak Surface Force (N) F s = v in (4) Incoing Velocity (/s) Figure 3: Linear fit between stylus incoing velocity and peak surface force.

4 4.2 Adding a Constant Force Recognizing that the user s hand velocity ay not be constant before contact, we can use a easureent of the pre-ipact acceleration, a in, to augent the odel derived fro the oentu approach. This acceleration stes fro the user applying a force, F in = a in, which we can treat as constant over the short duration of the ipact. The surface force after contact ust oppose this additional hand force, resulting in F s (t) = (α2 + ω 2 ) ω v in sin(ωt) e αt F in (5) F s = v in α2 + ω 2 e (α/ω) tan 1 (ω/α) a in (6) Introducing an acceleration-dependent ter into the leastsquares fitting process reduces the RMS error to.148 N. Figure 4 presents these results graphically, wherein each paraeter is isolated by subtracting off the estiated contribution of the other paraeters. The epirical data shows the predicted linear effect of incoing acceleration, though quantization of the acceleration signal produces significant striping. The resulting vertical spread in the data would collapse soewhat with a higher-resolution acceleration easureent. Analyzing the result of the linear fit in connection with (6), we see that the acceleration scaling ter should provide a direct estiate of the ipacting syste ass. Considering the addition of the stylus and device, the leastsquares result of.335 kg is reasonable when copared with other hand ass estiates [15]. PSF Coponent (N) PSF Coponent (N) F s = v in.335 a in Incoing Velocity (/s) Incoing Acceleration (/s 2 ) Figure 4: Linear fits between incoing velocity, incoing acceleration, and peak surface force. -1 b b s k k s x d x Figure 5: Dynaic Ipact Model. 4.3 Hand and Surface Dynaics It is widely recognized that the dynaics of the hand change with configuration and uscle contraction. By analyzing the surface force recorded fro real taps with a stylus held in a two-finger grasp, Fiene et al. found an approxiately linear relationship between grip force and the effective stiffness, k, and daping, b, of the user s hand [4], a result that atches well with other research [5, 6, 9]. To account for changes in user ipedance, we introduce the odel shown in Fig. 5, which cobines a second-order user odel with a spring-daper representation of the surface. We define the state vector, x = [x ẋ x d ], the input vector u = [ẋ d ], and the full state-space dynaics ẋ = Ax + Bu y = Cx + Du (7) 1 x ẋ = (k+k s) (b+b s) k ẋ + b [ẋ d ] (8) x d 1 y = [ k s b s ] x + [] [ẋ d ] (9) ẋ x d where the output y is the surface force, F s. Assuing the user has a constant desired acceleration, we know the input ẋ d (t)=v in + a in t, and we can include an initial user force, applied via the spring k, by choosing the following initial state vector x = x, ẋ, = v in (1) a in /k x d, The Laplace transfor of the syste output can then be derived with the equation Y(s) = C(sI A) 1 BU(s) + C(sI A) 1 x (11) where U(s) = [v in /s + a in /s 2 ], providing F s (s)= (v ins + a in )(b s s + k s )(s 2 + bs + k) s 3 (s 2 + (b + b s )s + (k + k s )) (12)

5 Solving for the inverse Laplace transfor of F s (s) results in a tie-doain solution of the for F s (t) = β sin(ωt + φ)e αt c 2 t 2 c 1 t c (13) supporting the use of a decaying sinusoid transient in the previous two odels. After coparing the ters in (12) and (13), we can deterine that the paraeters governing the decaying sinusoid are ω = β = where k + k s ( b + bs ) 2 α = b + b s ( c 3 vin 2 b + b s v in a in + ) a 2 in k + k s k + k s c 3 = 4(bb s k s k 2 s kb 2 s) 2 (k + k s ) 2 ((b + b s ) 2 4(k + k s )) (14) (15) (16) and the phase shift, φ, which stes fro the addition of a decaying cosinusoid ter, has the for tan φ = p 1 p 2 a in p 3 v in p 4 a in + p 5 v in (17) where the p n ters are various cobinations of the syste s echanical paraeters. Siulation has shown inial effect fro the phase shift, which is on the order of.2 radians for the paraeters of our syste. The reaining three ters in (13) are c 2 = k e a in c 1 = k e v in + k 2 e d a in (18) c = k 2 e d v in + k2 s + k e (2bb s d(k + k s )) (k + k s ) 2 a in (19) where the series spring stiffness, k e, and the d ter are: k e = kk s and d = b k + k s k 2 + b s ks 2 (2) Exaining the syste for the siplified case of zero incoing acceleration, the tie-doain response reduces to F s (t) = v in c3 sin(ωt + φ)e αt + k e v in (t + k e d) (21) which cobines a velocity-scaled decaying sinusoid with rap and step functions resulting fro oveent of the desired position within the surface. To visualize how changes in hand paraeters affect the peak surface force, a siulation of the syste was created using noinal values for the surface paraeters and the hand ass. Fig. 6(a) shows the resulting change in peak surface force as a function of hand stiffness and daping for zero incoing acceleration. The solid line, which is also Peak Surface Force (N) Stiffness, k (N/) 6 8 (a) Peak Surface Force (N) 1 12 Daping, b (Ns/) Grip Force (N) Figure 6: Siulation of the change in peak surface force as a function of (a) the hand stiffness and daping, and (b) the user-specific grip-force. PSF Coponent (N) PSF Coponent (N) PSF Coponent (N) F s = v in.35 a in.22 g in Incoing Acceleration (/s 2 ) (b) Incoing Velocity (/s) Grip Force (N) Figure 7: Linear fits between incoing velocity, incoing acceleration, grip force, and peak surface force. -1

6 shown in Fig. 6(b), represents the user-dependent relationship between hand paraeters and grip force fro [4]. The flatness of the surface in Fig. 6(a) suggests that user-to-user differences in the grip-force-to-hand-paraeter relationship should not significantly affect the overall correlation to peak surface force. Introducing a linear dependence on grip force into the least-squares paraeter fitting routine reduces the RMS error to.81 N, suggesting that the inclusion of both incoing acceleration and grip force ost accurately accounts for the aplitude of the transient response. The coponents of the peak surface force can be visualized along with the least-squares results in Fig Conclusion This work provides an iproved understanding of the dynaics of real contact and will enable ore realistic virtual rendering of hard objects. The user study presented in Section 3 shows that user realis ratings correlate with the aplitude of event-based virtual surface transients. Interpreting the results of Section 4, we see that oentu analysis provides a justification for the coonly observed approxiate linear relationship between incoing velocity and transient force aplitude. The addition of a constant user-applied force accounts for the ore subtle effect of incoing acceleration. The state-space odel, which includes both hand and surface dynaics, provides a theoretical basis for the coonly assued decaying sinusoid response. It also yields paraetric relationships for the sinusoid frequency and exponential decay rate based on easurable echanical properties of the surface and the user. Finally, this treatent provides a forula for the surface force as a function of tie after contact. Future work will continue to explore the structure and iplications of these odels to understand how they can best be eployed in the real-tie rendering of virtual hard contact, including extension to ore coplex interactions such as three-diensional contact and ulti-body ipacts. To ensure sooth transitions after event-based output, we will develop ethods for cobining the calculated openloop transients, which do not necessarily decay to zero, with traditional proportional feedback. We will also perfor ore extensive user testing to deterine the full effect of transient shape on perception. Acknowledgents The authors thank Professor Günter Nieeyer of the Stanford Telerobotics Lab for his encourageent and support of this work. References [1] K. R. Boff and J. E. Lincoln. Engineering Data Copendiu: Huan Perception and Perforance, volue 3. Arstrong Aerospace Research Laboratory, Wright- Patterson AFB, [2] G. C. Burdea. Force and Touch Feedback for Virtual Reality. John Wiley and Sons, Inc., [3] N. Diolaiti, G. Nieeyer, F. Barbagli, and J. K. Salisbury. Stability of haptic rendering: Discretization, quantization, tie delay, and Coulob effects. IEEE Transactions on Robotics, 22(2): , Apr. 26. [4] J. Fiene, K. J. Kuchenbecker, and G. Nieeyer. Event-based haptic tapping with grip force copensation. In Proc. IEEE Haptics Syposiu, pages , Mar. 26. [5] A. Z. Hajian and R. D. Howe. Identification of the echanical ipedance at the huan finger tip. J. Bioechanical Engineering, 119(1):19 114, Feb [6] C. J. Hasser and M. R. Cutkosky. Syste identification of the huan hand grasping a haptic knob. In Proc. IEEE Haptics Syposiu, pages , Mar. 22. [7] J. D. Hwang, M. D. Willias, and G. Nieeyer. Toward event-based haptics: Rendering contact using open-loop force pulses. In Proc. IEEE Haptics Syposiu, pages 24 31, Mar. 24. [8] K. J. Kuchenbecker, J. Fiene, and G. Nieeyer. Iproving contact realis through event-based haptic feedback. IEEE Trans on Visualization and Coputer Graphics, 12(2):219 23, March/April 26. [9] K. J. Kuchenbecker, J. G. Park, and G. Nieeyer. Characterizing the huan wrist for iproved haptic interaction. In Proc. ASME Int. Mechanical Engineering Congress and Exposition, volue 2, paper 4217, Nov. 23. [1] R. H. LaMotte. Softness discriination with a tool. J. Neurophysiology, 83: , 2. [11] A. M. Okaura, M. R. Cutkosky, and J. T. Dennerlein. Reality-based odels for vibration feedback in virtual environents. IEEE/ASME Trans. on Mechatronics, 6(3): , Sept. 21. [12] A. M. Okaura, J. T. Dennerlein, and R. D. Howe. Vibration feedback odels for virtual environents. In Proc. IEEE Int. Conf. on Robotics and Autoation, volue 3, pages , May [13] S. Salcudean and T. Vlaar. On the eulation of stiff walls and static friction with a agnetically levitated input-output device. ASME J. Dynaic Systes, Measureent and Control, 119: , [14] M. Srinivasan, C. Basdogan, and C.-H. Ho. Haptic interactions in virtual worlds: Progress and prospects. In Proc. Int. Conf. on Sart Materials, Structures, and Systes, [15] C. R. Wagner and R. D. Howe. Mechaniss of perforance enhanceent with force feedback. In Proc. IEEE World Haptics Conf., Mar. 25. [16] P. Wellan and R. D. Howe. Towards Realistic Display in Virtual Environents. In Proc. ASME Dynaic Systes and Control Division, volue 57, pages , [17] G. Westling and R. S. Johansson. Responses in glabrous skin echanoreceptors during precision grip in huans. Experiental Brain Research, 66(1):128 14, 1987.