Shengli Xie Minyue Fu Derong Liu

Transcription

1 2

2 Shengli Xie Minyue Fu Derong Liu 3

3 F.L. Lewis, NAI Moncrief-O Donnell Chair, UTA Research Institute (UTARI) The University of Texas at Arlington, USA and Guest Foreign Professor, Guangdong University of Technology, Guangzhou China Supported by : NSF NRI Initiative ONR Assistive Human-Robot Interaction (HRI) and Dan Popa, University of Louisville and Isura Ranatunga, Reza Modares, Bakur AlQaudi Supported by : China NNSF China Project 111 Talk available online at

4 Multi-Modal Skin and Garments for Healthcare and Home Robots Dan O. Popa 1, Frank L. Lewis 1, Nicoleta Bugnariu 2, Woo Ho Lee 1 and Muthu Wijesundara 3 1 Department of Electrical Engineering, University of Texas at Arlington, 2 University of North Texas Health Science Center, 3 UT Arlington Research Institute Partner Companies: Advanced Arm Dynamics, Hanson Robotics, Inc., National Instruments 1. System Design: where to place sensors on robot? Novel algorithms and methods for optimal placement and data management of such devices on several co-robots. - Statistical adaptive sampling for sensor selection - Sensor fusion based on noise and sensor scaling models - Optimization algorithms for maximizing robot perception - New Collect sensor simulation models and robot control Meshes Wor GAZEBO algorithms Sens ld Sketch-up or Plu Plac gin eme nt Sen O Infrar W sor ut ed o Plu pu Accel rl gins t. SDF d Inf Tem rar p. Model ed Tactil Plugins Ac e Overlay cel Thermal. Temp User Te profile Application m C++/Python p. Diagram of SkinSim, multi-modal skin simulation environment Ta ctil e 4. Co-Robot performance: how does this technology help humans? The impact of the new technology to humans will be assessed, including the safety, level of assistance to several targeted user groups, ease of use, aesthetics, and therapeutic benefits. - Clinical Testing at UNTHSC and UTARI - Collaborative work with Advanced Arm Dynamics Assistance for Upper Limb Amputees PR2 Teach-by-demonstration PR2 Robot, Tactile sensor array and the environment in Gazebo Initial sensor prototypes & Robotic hardware Task requirements Human Interaction Data Collection R o b ot S e ns or H u m a n Measuremen t & Simulation Huma n Chara cter Robot and Skin Simulation phri Model Reference Neuroadaptive Controller Neuroadaptive Impedance Control Fabrication & Integration of Skin/Garment Hardware Iterate Designs & Algorithms Sensors and skin onto PR2 and youbot robots at UTARI Microsensor packaging and interconnects Interaction Learning Perceived Impedance Electro Hydro Dynamical sensor printing (maskless lithography) 2. Control and Learning: both human and robot learn during interaction Learning algorithms and adaptive impedance control for efficient use of multimodal sensors to sense human intent and improve the usability of co-robots. - Online reinforcement learning for phri with co- Robots wearing skin and garments, given humancentric rewards and cost functions - Neuroadaptive Impedance Control with stability and performance guarantees Where, f ( x) M( q)( qm e ) Vm( q, q )( q m e) F( q ) G( q) ˆ T ˆT W ( V x) Kvrvr The robustifying signal vt () is, vt () K ( Zˆ Z ) r 3. Devices: distributed skin sensors Integration of multi-modal, multi-resolution, MEMS skin sensors to include tactile, thermal, pressure, acceleration, and distance IR sensing. - Sensor design tuned for phri - Fabrication on flexible substrates - Robust packaging in Frubber & laminates - Efficient wire interconnect schemes Array of temperature sensors Parylene (polymer substrate) at UTARI z F B Top Kapton layer with electric traces and pads Pressure sensitive adhesive tape (50µm Bottom thick) Kapton layer with electric Concept: Microactuator traces and array using piezo pads actuator University of Texas at Arlington NRI Grant No Program Manager: Dr. Paul Werbos, ECCS, ENG, NSF

5 Fully Automated Robot vs. Assistive Robot

6 PR2 meets Isura

7 Standard Robot Trajectory Tracking Controller Where is the human?

8 Robot dynamics Impedance Control Prescribed Error system Control torque depends on Impedance model parameters

9 Human Performance Factors Studies Human task learning has 2 components: 1. Human learns a robot dynamics model to compensate for robot nonlinearities 2. Human learns a task model to properly perform a task Inner Robot Specific Control Loop INDEPENDENT OF TASK Outer Task Specific Control Loop INDEPENDENT OF ROBOT DETAILS

10 Two-loop HRI Design- Robot Control versus Task Control 1. Inner Robot-Specific Control Loop 2. Outer Task-Specific Control Loop 2A. Adaptive Inverse Filter Task Design 2B. Model Reference Adaptive Control Task Design 2C. Reinforcement Learning Task Control for Minimum Human Effort 3. Experiments

11 Task control outer loop Robot control inner loop

12 1. Inner Loop Robot Specific Controller Model reference Neuro adaptive Control Make the robot behave like the prescribed model There is NO prescribed trajectory in the robot control loop design

13 A Novel Control Objective Using Neuro adaptive Control Techniques F.L. Lewis, S. Jagannathan, and A. Yesildirek, Neural Network Control of Robot Manipulators and Nonlinear Systems, Taylor and Francis, London, F.L. Lewis, D.M. Dawson, and C.T. Abdallah, Robot Manipulator Control: Theory and Practice, 2 nd edition, Revised and Expanded, CRC Press, Boca Raton, 2006.

14 Model following error formulation Robot dynamics M ( qx ) Vqqx (, ) Fq ( ) Gq ( ) f = f f d c h Control torque T = J ( q) f c Prescribed robot impedance model M x D x K x = f m m m m m m h Model following error e= x m x Sliding mode error r = e e Error dynamics There is NO task trajectory here M ( qr ) = Vqqr (, ) f( ) fd fc fh Unknown robot nonlinear function f( )= M( q)( x e ) V( q, q )( x e) F( q ) G( q) m T T T T T T T = e e xm x m xm q q T m

15 Dynamics Robot M ( qx ) Vqqx (, ) Fq ( ) Gq ( ) f = f f d c h Error M ( qr ) = Vqqr (, ) f( ) fd fc fh Unknown nonlinearities parameterized in terms of a function approximator T T f( )= W ( V ) W, V unknown parameters Controller approximation based f = fˆ ( ) K rv( t) f c v h Estimate for unknown nonlinearities ˆ( )= ˆ T ( ˆT f W V ) WV ˆ, ˆ Estimated parameters Closed loop error dynamics M ( qr ) = Vqqr (, ) Kr f ( ) f vt ( ) v d Model following error driven by parameter estimation error ˆ T T ( ) ( ) ( )= ( ) ˆ T ( ˆT f f f W V W V )

16 Adaptive control structure ˆ T = ( ˆT f W V ) K rv( t) f c v h Standard Adaptive Parameter tuning algorithms ˆ = ( ˆT T ) ( ˆT ) ˆT T W F V r F V V r FrWˆ ˆ = ( ( ˆT T ) ˆ T V GV Wr) GrVˆ Model following error e x x = m r = e e Robust control term vt ()= K( Zˆ Z ) r z F B T T T T T T T = e e xm x m xm q q T No task reference trajectory is used here The robot controller makes the model following error e= x m x small The parameters of the admittance model M x D x K x = f are not needed m m m m m m h

17 No task trajectory information is used in this inner loop robot controller The inner loop robot controller makes the model following error small The admittance model parameters are not needed Only the admittance model trajectories x, x, x are needed. m m m

18 2. Outer Task-Specific Control Loop 2A. Adaptive Inverse Filter Task Design 2B. Model Reference Adaptive Control Task Design 2C. Reinforcement Learning Task Control for Minimum Human Effort

19 Task control outer loop Robot control inner loop

20 Three Outer Loop Designs To appear 2016

21 2A. Outer loop Task Specific Design #1 Work of Isura Ranatunga Adaptive Inverse Control and Wiener Filter B. Widrow Adaptive inverse filter Signal to robot controller Want to find M(s) so that Ds () M() shs () with H(s) and D(s) unknown For trajectory following task e.g. point to point motion control

22 Want to find M(s) so that Ds () M() shs () with H(s) and D(s) unknown Wiener Filter Solution in terms of power spectral densities f x () s h d Ds () M()= s = () s H() s f h f h

23 Find Wiener Filter online using adaptive learning f () h t xm () t 1/s 1/s b 1 a 1 1/s 1/s b 2 a 2 1/s 1/s b 3 a 3 d dt f h d dt x m

24 Ideal Filter Wiener filter solution x ()= t H() t () t d x ()= t H() t ˆ () t m Known regression vector Unknown coefficients Kalman Filter = CT RLS x d x m

25 Combined stability analysis of Inner robot control loop and Outer task following loop Lyapunov function L r M q r tr W F W tr V G V t P t t T T 1 T 1 T 1 = ( ) { } { } () () () Robot model following error Outer loop inverse adaptive filter error NN weight estimation errors

26 Shi nian shu mu Bai nian shu ren Keshi- Wu nian shu xuesheng

27 2B. Outer loop Task Specific Design #2 Work of Bakur AlQaudi Model Reference Adaptive Control K. Astrom BUT In standard MRAC, the controller appears before the unknown plant Here, the unknown plant (e.g. human) is BEFORE the controller

28 BUT In standard MRAC, the controller appears before the unknown plant Here, the unknown plant (e.g. human) is BEFORE the controller So we need to add a human dynamics identifier

29 Unknown Human model H () h s b y sa u c Basic muscle response model Nominal Robot impedance model H p () s b y n s a u n p To generate prescribed model trajectory x, x, x m m m Task reference model first order crossover model ideal human + robot system H s b y m m m() sam uc Human factors studies show that AFTER Learning, the human plus robot system obeys A simple first order roll off high bandwidth dynamics

30 Human response estimation error y y yˆ Parameter estimation error a aˆ a b bˆ b

31 Human response estimation error y y yˆ

32

33 Combined stability proof of overall 2 loop robot task system Adaptive tuning parameter errors r M q r tr W F W tr V G V T T 1 T 1 ( ) { } { } Inner model tracking error NN parameter estimation errors

34 PD controller like that provided by cerebellum Basic muscle response

35 2C. Outer loop Task Specific Design #3 Reinforcement Learning for minimum human effort Force exerted by human indicates his discontent A measure of Human Intent Feedforward assistive control term Work of Reza Modares l (.) Human force amplifier xd e + d ( Ks + Ks ) - p d Human 1 f h K h ( Ms + Bs + K) xm Prescribed Impedance Model Find robot impedance model parameters To minimize human force effort f h And task trajectory following error e d M, BK,

36 ( Ks+ K ) f = ke d p h e d Tracking error e = x -x Î d d m n e = [ e e ] = x -x Î T T T 2n d d d d x = [ x x ] Î 2 T T T n m m x = [ x x ] Î T T T 2n d d d

37 Performance index ò J = ( e T Q e + f T Q f + u T Ru ) dt t Then control is d d d h h h e e u = K e + K f e 1 d 2 h Minimize human effort and tracking error How to get human force into PI? ò T T J = ( X QX + u Ru ) dt t e e

38 Robot Impedance Model Unknown Human Model ( Ks+ K ) f= ke d p e d K f + K f = k e d h p h e d f =- K K f + k K e º A f + E e -1 h d p h e d,0 d h h h d Overall Augmented Dynamics Feedback linearization loop

39 Optimal control is an offline method Based on solving ARE Knowing all the plant dynamics We want online method to learn the optimal control without knowing the System Matrix A Optimal Design Always Admits Reinforcement Learning for Real time Optimal Adaptive Control

40 OFF POLICY Reinforcement Learning Needs NO knowledge of the system dynamics Take enough data along the system trajectory To solve this equation using least squares

41 Off Policy Reinforcement Learning Needs NO knowledge of the system dynamics Off policy IRL Bellman equation t+d t t+dt T T T T T Xt () PXt () + ò [2 Xt () PBe] dt = é Xt () QXt () u Ru ù dt Xt ( t) PXt ( t) t ò ê + + +D +D ë e eúû t t Off policy term Finds optimal control gains without using ANY system dynamics

42 3. Experimental Results on PR2

43 3. Experimental Results Work of Isura Ranatunga Sven Cremer

44

45

46 Point to point tracking error Human force effort

47 Future Work

48 Thanks!!

49