F.L. Lewis, NAI. Talk available online at Supported by : NSF AFOSR Europe ONR Marc Steinberg US TARDEC

Transcription

1

2 F.L. Lewis, NAI Moncrief-O Donnell Chair, UTA Research Institute (UTARI) The University of Texas at Arlington, USA and Qian Ren Consulting Professor, State Key Laboratory of Synthetical Automation for Process Industries Northeastern University, Shenyang, China Assistive Human-Robot Interaction (HRI) With Dan Popa, UTA Research Institute Lead Roboticist and Isura Ranatunga, Reza Modares, Bakur AlQaudi Supported by : NSF AFOSR Europe ONR Marc Steinberg US TARDEC Supported by : China NNSF China Project 111 Talk available online at

3

4 Invited by Yunhui Liu

5 PR2 meets Isura

6 Robot dynamics Error system Control torque depends on Impedance model parameters

7 Human Performance Factors Studies Human task learning has 2 components: 1. Human learns a robot dynamics model to compensate for robot nonlinearities 2. Human learns a task model to properly perform a task Inner Robot Specific Control Loop INDEPENDENT OF TASK Outer Task Specific Control Loop INDEPENDENT OF ROBOT DETAILS

8 Two-loop HRI Design- Robot Control versus Task Control 1. Inner Robot-Specific Control Loop 2. Outer Task-Specific Control Loop 2A. Adaptive Inverse Filter Task Design 2B. Model Reference Adaptive Control Task Design 2C. Reinforcement Learning Task Control for Minimum Human Effort 3. Experiments

9 Task control outer loop Robot control inner loop

10 1. Inner Loop Robot Specific Controller Model reference Neuro adaptive Control There is NO prescribed trajectory in the robot control loop design

11 A Novel Control Objective Using Neuro adaptive Control Techniques F.L. Lewis, S. Jagannathan, and A. Yesildirek, Neural Network Control of Robot Manipulators and Nonlinear Systems, Taylor and Francis, London, F.L. Lewis, D.M. Dawson, and C.T. Abdallah, Robot Manipulator Control: Theory and Practice, 2 nd edition, Revised and Expanded, CRC Press, Boca Raton, 2006.

12 Model following error formulation Robot dynamics M ( qx ) Vqqx (, ) Fq ( ) Gq ( ) f = f f d c h Control torque T = J ( q) f c Prescribed robot impedance model M x D x K x = f m m m m m m h Model following error e= x m x Sliding mode error r = e e Error dynamics There is NO task trajectory here M ( qr ) = Vqqr (, ) f( ) fd fc fh Unknown robot nonlinear function f( )= M( q)( x e ) V( q, q )( x e) F( q ) G( q) m T T T T T T T = e e xm x m xm q q T m

13 Dynamics Robot M ( qx ) Vqqx (, ) Fq ( ) Gq ( ) f = f f d c h Error M ( qr ) = Vqqr (, ) f( ) fd fc fh Unknown nonlinearities parameterized in terms of a function approximator T T f( )= W ( V ) W, V unknown parameters Controller approximation based f = fˆ ( ) K rv( t) f c v h Estimate for unknown nonlinearities ˆ( )= ˆ T ( ˆT f W V ) WV ˆ, ˆ Estimated parameters Closed loop error dynamics M ( qr ) = Vqqr (, ) Kr f ( ) f vt ( ) v Model following error driven by parameter estimation error d

14 Adaptive control structure ˆ T = ( ˆT f W V ) K rv( t) f c v h Standard Adaptive Parameter tuning algorithms ˆ = ( ˆT T ) ( ˆT ) ˆT T W F V r F V V r FrWˆ ˆ = ( ( ˆT T ) ˆ T V GV Wr) GrVˆ Robust control term vt ()= K( Zˆ Z ) r z F B T T T T T T T = e e xm x m xm q q T No task reference trajectory is used here The robot controller makes the model following error e= x m x small The parameters of the admittance model M mxm Dmx m Kmxm = fh are not needed

15 No task trajectory information is used in this inner loop robot controller The inner loop robot controller makes the model following error small The admittance model parameters are not needed Only the admittance model trajectories x, x, x are needed. m m m

16 Shi nian shu mu Bai nian shu ren Keshi- Wu nian shu xueshang

17 2. Outer Task-Specific Control Loop 2A. Adaptive Inverse Filter Task Design 2B. Model Reference Adaptive Control Task Design 2C. Reinforcement Learning Task Control for Minimum Human Effort

18 2A. Outer loop Task Specific Design #1 Work of Isura Ranatunga Adaptive Inverse Control and Wiener Filter B. Widrow Adaptive inverse filter Signal to robot controller Want to find M(s) so that Ds () M() shs () with H(s) and D(s) unknown For trajectory following task e.g. point to point motion control

19 Want to find M(s) so that Ds () M() shs () with H(s) and D(s) unknown Wiener Filter Solution in terms of power spectral densities f x () s h d Ds () M()= s = () s H() s f h f h

20 Find Wiener Filter online using adaptive learning f () h t xm () t 1/s 1/s b 1 a 1 1/s 1/s b 2 a 2 1/s 1/s b 3 a 3 d dt f h d dt x d

21 Ideal Filter Wiener filter solution x ()= t H() t () t d x ()= t H() t ˆ () t m Known regression vector Unknown coefficients x d x m

22 Combined stability analysis of Inner robot control loop and Outer task following loop Lyapunov function L r M q r tr W F W tr V G V t P t t T T 1 T 1 T 1 = ( ) { } { } () () () Robot model following error Outer loop inverse adaptive filter error NN weight estimation errors

23 2B. Outer loop Task Specific Design #2 Work of Bakur AlQaudi Model Reference Adaptive Control K. Astrom BUT In standard MRAC, the controller appears before the unknown plant Here, the unknown plant (e.g. human) is BEFORE the controller

24 BUT In standard MRAC, the controller appears before the unknown plant Here, the unknown plant (e.g. human) is BEFORE the controller So we need to add a human dynamics identifier

25 Unknown Human model H () h s b y sa u c Basic muscle response model Nominal Robot impedance model H p () s b y n s a u n p To generate prescribed model trajectory x, x, x m m m Task reference model first order crossover model ideal human + robot system H s b y m m m() sam uc

26 Human response estimation error y y yˆ Parameter estimation error a aˆ a b bˆ b

27 Human response estimation error y y yˆ

28

29 Combined stability proof of overall 2 loop robot task system Adaptive tuning parameter errors r M q r tr W F W tr V G V T T 1 T 1 ( ) { } { } Inner model tracking error NN parameter estimation errors

30 PD controller like that provided by cerebellum Basic muscle response

31 Kung Tz 500 BC Confucius Tian xia da tong Harmony under heaven Archery Chariot driving Music Rites and Rituals Poetry Mathematics Man s relations to Family Friends Society Nation Emperor Ancestors 124 BC - Han Imperial University in Chang-an

32 2C. Outer loop Task Specific Design #3 Work of Reza Modares Reinforcement Learning for minimum human effort Feedforward assistive control term l (.) Human force amplifier xd e + d ( Ks + Ks ) - p d Human 1 f h K h ( Ms + Bs + K) xm Prescribed Impedance Model Find robot impedance model parameters To minimize human force effort f h And task trajectory following error e d M, BK,

33 ( Ks+ K ) f = ke d p h e d Tracking error e = x -x Î d d m n e = [ e e ] = x -x Î T T T 2n d d d d x = [ x x ] Î 2 T T T n m m x = [ x x ] Î T T T 2n d d d

34 Performance index ò J = ( e T Q e + f T Q f + u T Ru ) dt t Then control is d d d h h h e e u = K e + K f e 1 d 2 h Minimize human effort and tracking error How to get human force into PI? ò T T J = ( X QX + u Ru ) dt t e e

35 Robot Impedance Model Unknown Human Model ( Ks+ K ) f= ke d p e d K f + K f = k e d h p h e d f =- K K f + k K e º A f + E e -1 h d p h e d,0 d h h h d Overall Augmented Dynamics Feedback linearization loop

36 Optimal control is an offline method Based on solving ARE Knowing all the plant dynamics We want online method to learn the optimal control without knowing the System Matrix A Reinforcement Learning

37 Off policy IRL Bellman equation t+d t t+dt T T T T T Xt () PXt () + ò [2 Xt () PBe] dt = é Xt () QXt () u Ru ù dt Xt ( t) PXt ( t) t ò ê + + +D +D ë e eúû t t Off policy term

38 Take enough data along the system trajectory To solve this equation using least squares

39 3. Experimental Results on PR2

40 3. Experimental Results Work of Isura Ranatunga Sven Cremer

41

42 Point to point tracking error Human force effort

43 Future Work

44 Thanks!!

45