EXPERIMENTS IN ON-LINE LEARNING NEURO-CONTROL

Transcription

1 From: Proceeings of the welfth International FLAIRS Conference Copyright 1999, AAAI (wwwaaaiorg) All rights reserve EXPERIMENS IN ON-LINE LEARNING NEURO-CONROL A G Pipe, M Ranall, Y Jin Intelligent Autonomous Systems Laboratory, Faculty of Engineering University of the West of Englan, Bristol BS16 1QY, UK AnthonyPipe@uweacuk Abstract We apply our on-line learning neural network approach to control of non-linear systems; first to joint level trajectory control of an inustrial weling robot, an then to one front leg of a hexapo walking robot he first application emonstrates the ability of neuro-control to moel complex non-linear components of a plant, thereby yieling impressive improvements in accuracy compare to the original robot s controller he secon application illustrates the strength of an on-line learning approach in coping with isturbances to a plant s characteristics In orer to set this work in context we briefly review our on-line learning neuro-control metho, use for both sets of experiments It has a strict theoretical basis incluing guarantees of the whole system s stability Introuction he overall aim of the work escribe here is to illustrate the capabilities of an on-line learning neural network approach to improving the performance of controllers operating in complex non-linear omains o achieve this aim one must be able to esign systems that can improve their performance by moelling crucial components of a given plant irectly using the ata normally available as part of the control process; eg control effort, esire an actual set-points, their erivatives an so on Off-line techniues suffer from some limitations If only simulation is use for training, then a sufficiently complex moel is reuire, which coul probably have been use to buil an acceptable traitional controller Alternatively, if real plant ata are use, then the learne transfer function is limite to that information which has been acuire uring the ata gathering phase An on-line approach, wherein the processes of ata gathering an training occur simultaneously with control of the real plant, can overcome these problems However, such an approach is not without ifficulties If the controller characteristics are changing ue to on-line aaptation as the real plant is exercise, then it becomes imperative to provie guarantees of learning algorithm convergence an stability of the whole system he new work presente in this paper is concerne with the application of these techniues to one 3-jointe front leg of a hexapo walking robot However, to set the context, we must first review our previous work on the Yaskawa robot an the neuro-control laws we use Relate Work Over the years much research effort has been put into the esign of neural network applications for manipulator control Albus [1] use the Cerebellum Moel Articulation Controller (CMAC) to control manipulators as early as 1975 Miller et al [11] an Kraft et al [7] extene Albus' results an evelope neural network learning algorithms in 1987 an 1992 respectively Kawato et al [6] ae MLP networks to original manipulator PD control systems as feeforwar compensators in 1988 Iiguni et al [3] combine manipulator linear optimal control techniues with Multi-Layer Perceptron (MLP) neural networks, which were use to compensate for the non-linear uncertainty in 1991 Others have concentrate on linking neural network base approaches to control with learning convergence an stability guarantees Examples are Narenra [12, 13] in 1991 an 1992, Sanner & Slotine [14] in 1991, Suykens, Van e Wall & De Moor [15] in 1995, an Lewis & Parthasarathy [8, 9] in 1996 an 1999 he Yaskawa Robot Application Since 1994 IAS Laboratory researchers have been working on on-line learning neuro-control an its applications he team first concentrate on establishing the necessary stability theory After this, implementations were emonstrate for improving trajectory tracking accuracy on a simulate Puma 56 robot manipulator an on a real eucational Mentor robot [4, 5] More recently we have extene the experimental work to cover a real Yaskawa Motaman robot [2], extracte from aily use as a manufacturing weling robot in a local company One coul consier the three major axes (base, shouler, elbow) of a manipulator as a couple system of links an revolute joints, riven by electric motors he Yaskawa is fitte with a shaft encoer an tachometer on each of these joints so as to provie position an velocity feeback

2 A general euation of motion for a rigi manipulator is H ( ) C(, ) g( ) F(, ) = τ (,, ) (1) where is the n x 1 vector of joint isplacements, τ is the n x 1 vector of applie joint torues, H ( ) is the n x n symmetric positive efinite manipulator inertia matrix, C(, ) is the n x 1 vector of centripetal an Coriolis torues, g( ) is the n x 1 vector of gravitational torues, an F (, ) are the unstructure uncertainties of the ynamics incluing friction an other isturbances On-Line Learning for Neural Networks Although the static performance of the PID controllers commonly use in such situations is goo, the ynamic performance leaves much to be esire By exploiting the neural network universal approximation feature we can assume that the left han sie of the above euation can be approximate by a neural network, ie H C g F= G (,, ) Wξ (,, ), (2) where G is the non-linear mapping matrix of an RBF or CMAC neural network, W is its weight vector, which is initially unknown, an ξ is its approximation error, which shoul be as small as possible by careful esign he main objective of on-line learning is to force the manipulator to follow the esire trajectory, ie In orer to simplify the final control structure, we will use the esire joint values as the neural network inputs, however the actual joint values are use to train the neural network on-line During on-line learning, the neural network is use as a part of the manipulator controller Its output forms part of the control signals which are use to rive manipulator joints he joint values in turn are use to train the neural network herefore the on-line learning algorithm must guarantee that these two couple systems (neural network & manipulator) work in a converging fashion Our main theoretical research result is presente in the following theorem he structure combines a simple two-term linear controller with the neurocontroller he control law guarantees the asymptotic stability of the whole system (neural network an manipulator) heorem: Consier euation 1 with the neuro-control law ~ ~ τ = K p Kv G(,, ) Ŵ (3) an the on-line learning algorithm Ŵ = Γ G(,, )( ~ c ~ ) (4) where is actual position an is esire position, Γ is a constant positive efinite matrix an is calle the learning rate If K p an K v are positive efinite matrices an sufficiently large, an c is a small enough positive constant, then the close-loop manipulator system is asymptotically stable an =, = lim t _ > lim t _ > Proof: he proof is rather complicate We only present the outline here hose intereste in further etails are referre to [5] It is well known in the robotics fiel that, i) the inertia matrix, H, is positive efinite, ii) for one choice of C, H = 1 C C 2 ( ), iii) iv) H an g have boune norms, H an g only contain trigonometric function of hence their partial erivatives with respect to also have boune norms By extening the work of Wen an Bayar [16] it can be prove that the following control law results in exponential stability if K p an K v are sufficiently large τ = p ~ ~ K Kv g( ) F(, ) (5) C(, ) H ( ) he Lyapunov metho [1] is use to prove this result he Lyapunov function is 1 ~ ~ 1 ~ ~ ~ ~ 1 V = H ~ ~ H K p c c Kv (6) he control law of Euation 3 is the control law of Euation 5 ~ with an aitional term - G(,, ) W, where ~ W= W- W his aitional term will prouce aitional terms in the first time erivative of the Lyapunov function hese aitional terms are cancelle by the on-line learning algorithms if we use a new Lyapunov function, ie 1 ~ ~ 1 ~ ~ ~ V = H H ~ K p c 2 2 (7) 1 ~ ~ 1 c K W ~ Γ -1 W ~ v 2 2 herefore the whole system is asymptotically stable he control iagram is shown in Figure 1 he neural network

3 acts as a feeforwar controller he esire joint values (isplacements, velocities, accelerations) are its inputs In the feeback loop, there is a PD controller A linear combination of joint isplacement errors an velocity errors is use to train the neural network, as shown in figure 1 c ~ ~ NN,, ~ K p robot - ~ K v Figure 1: Robot manipulator neuro-control architecture Remark: Although the theorem guarantees that the whole system will be asymptotically stable, the transient performance an the conitions which K p, K v an c must satisfy all epen on the initial Lyapunov function value Unlike conventional aaptive control where the number of ajustable unknown parameters is small, the number of ajustable neural network weights is very large, so the initial Lyapunov function value will be very large In orer to reuce this initial value an improve on-line learning convergence velocity, it is often esirable to train the neural network offline before it is use for on-line control Neuro-Control of the Yaskawa Manipulator hree neuro-controllers are use, one for each of the three major axes In the experimental approach aopte for the Yaskawa robot a CMAC (Cerebellum Moel Articulation Controller) neural network [1] is effectively strappe aroun each existing joint controller rather than replacing it, as shown in figure 2 hey are implemente via a PC-resient exas Instruments DSP boar, base aroun the 4MHz 32C4 processor, which executes the coe for all three neural networks Each neural network utilises the same eman an feeback information available to the existing controller, an the output signals from the existing an neurocontrollers are combine by simple analogue voltage aition just before the power stages of the motor rive circuit Each neural network therefore simply as its control effort to that of the existing controller in orer to help reuce errors he control arrangement for one joint is as efine by euations 3 an 4 above, an epicte in figure 1 Each composite joint controller receives new position, velocity an acceleration emans every 2ms he existing two term linear controller operates on only the position an velocity error signals to erive control actions he neural network takes position, velocity an acceleration emans as inputs an is traine on-line using a combination of velocity an position errors Each CMAC has 32,768 ajustable weights For a particular input set, 12 weights are selecte for each output Between each 2 millisecon sampling perio an the next, the 32C4 must therefore complete one pass of the learning algorithm an a forwar pass of the neural network for each CMAC Over repeate trials, performance improves; though the bulk of this is obtaine in the first five trials of a given trajectory I 32 C4 Neural Network Output - (R /R ) 1 2 R 1 D/A A/D Counter - R 2 he original system Figure 2: he Experiment Yaskawa Setting Yaskawa robot acho Encoer Figure 3 shows an example esire trajectory which the robot shoul follow he minimum ( min ) an maximum ( max ) joint values are -256 an 372 counts of the joint position encoer for joint 1, an 8192 for joint 2, an 6144 for joint 3 36 encoer counts roughly eual 1 egree 35 Encoer Counts X Figure 3: he Desire rajectories (he soli line is for the Base Joint, the ashe line is for the Lower Arm Joint, an the otte line is for the Upper Arm Joint) Figure 4 shows the joint 1 trajectory tracking errors Line (a) shows the errors without the neural network he worst case error is 6 counts Line (b) shows the errors of the neurocontroller in the first trial he worst case is reuce to 46 Lines (c) an () show the errors in the thir an fifth trials Now the worst case errors are further reuce to 17 an 9 respectively he tracking errors are finally reuce by more than six times A similar improvement is also achieve to

4 joint 2 an joint 3 heir tracking errors are shown in figure 5 an figure 6 respectively Encoer Counts (b) (a) Figure 4: he rajectory Following Errors of Joint 1 (a) Using the original controller; (b) Using the neuro-controller in the first trial; (c) Using the neuro-controller in the thir trial; () Using the neuro-controller in the fifth trial Encoer Counts (a) (b) () () (c) (c) Figure 5: he rajectory Following Errors of Joint 2 (a) Using the original controller, (b) Using the neuro-controller in the first trial, (c) Using the neuro-controller in the thir trial, () Using the neurocontroller in the fifth trial Encoer Counts (a) () (c) (b) Figure 6: he rajectory Following Errors of Joint 3 (a) Using the original controller; (b) Using the neuro-controller in the first trial; (c) Using the neuro-controller in the thir trial; () Using the neuro-controller in the fifth trial Neuro-Control of a Hexapo Leg Joint We escribe our initial results from the first phase of an investigation into on-line aaptive neuro-control of a hexapo walking robot s leg trajectories In the intene application area of rugge terrain walking, very accurate trajectories an foot placement are reuire - even in the presence of isturbances Our first experiments are concerne with the neuro-controller s performance in the face of intermittent en-effector force isturbances Below we ientify a simplifie neuro-control structure (relative to the Yaskawa experiments escribe above) which implements control of the coxa joint of one of the hexapo s front legs; in future work this will be extene to control of the other two joints of each leg, an then to all six legs A Raial Basis Function (RBF) neural network is use as the controller [14] In the Yaskawa robot experiments, the neural network contribute irectly to control current supplie to each joint s motor In the case here each motor is package as a servo-system, with on-boar linear position control A esire position is ientifie by Pulse-Position-Moulation (PPM) o generate a trajectory, esire positions are prouce in seuence for each of the three servo-motors of a leg by the on-boar computer (escribe below) In these experiments the RBF network is place between this computer output an the servo-motor input, as etaile in the control iagram of figure 7 NN ' - ~ - servo contro ller servo "plant" coxa motor Figure 7: Leg joint neuro-control architecture An error signal is erive from the position potentiometer alreay containe in the servo as part of the existing position control loop his feeback signal is rea into the on-boar computer via an A/D converter an then use to erive an integer position error signal which is in turn use to train the neural network between one control action an the next his generates a moifie trajectory eman signal which shoul reuce trajectory tracking errors over time an aapt to control isturbances In the Yaskawa experiments velocity feeback was use in aition to position (see euation 4) his will be reuire here also when the experimental platform is extene For j

5 the initial results presente here, however, only one esire trajectory, a curve forwar movement of the leg, is consiere an so velocity variations can be ignore Gaussian basis functions were use for the neural network, evenly space at intervals of 1 units across an integer esire position ( ) input range of to 1 Each basis function ha a variance of 169 he on-boar computer is base aroun the Motorola microcontroller, clocke at a rate of 2MHz he controller possesses an on-chip ime Processor Unit (PU) which allows for conversion between integer values representing joint positions an a PPM signal to be sent to a servo-motor input he controller also possess a Queue Serial Moule (QSM) which provies for synchronous (SPI) an asynchronous (RS232) communications he former is use for communication with some on-boar peripheral evices, whilst the latter is use for transmitting the results of experiments to a host computer he boar is euippe with A/D converters, which are connecte over the SPI line, for measuring the servomotor position feeback signals Software has been evelope in the C language, using a PC-resient Microtec Research XRAY cross-evelopment suite he RBF network is traine initially with an ientity function so that, without on-line learning, it contributes no extra control effort here are three plots on each graph; showing the esire position for the coxa joint, the servomotor position feeback signal, an finally the RBF neural network output servo-controller is performing uite well uner these circumstances an the RBF network is unnecessary A spring was attache to the leg en-effector after the first run, which applie a variable tangential loa isturbance to the system Figure 9 shows the situation at the fourth cycle of the trajectory One can clearly see that the RBF neural network has learne to compensate for tracking inaccuracies over the preceing three cycles, proucing a moifie trajectory eman which is substantially ifferent from the original so as to give the reuire position feeback signal Figure 9: rajectory three cycles after loa applie Vertical axis: position in the integer units of the trajectory planning algorithm Horizontal axis: time steps in units of approximately 3 ms 12 esire RBF NN feeback esire RBF NN esire RBF NN feeback feeback Figure 8: Initial off-loa trajectory Vertical axis: position in the integer units of the trajectory planning algorithm Horizontal axis: time steps in units of approximately 3 ms Figure 8 shows the initial off-loa case on the first run through the trajectory It can be seen that all three signals are uite similar uner these conitions, meaning that the Figure 1: Recovere off-loa trajectory Vertical axis: position in the integer units of the trajectory planning algorithm Horizontal axis: time steps in units of approximately 3 ms After this the spring was remove his prouce a significant initial trajectory overshoot However by the seventh cycle the situation was nearly recovere to the original situation, as shown in figure 1

6 Discussion & Conclusions For the work carrie out on the Yaskawa manipulator it was possible to achieve a six- to ten-fol reuction in joint level trajectory tracking errors on each of the principal three axes of movement compare with using the Yaskawa s existing joint controllers on their own Recently these experimental results have been extene by testing the composite controller in many more parts of the manipulator s operational envelope, enabling an empirical analysis of its learning an generalisation performance across many trajectories he results of this work have been encouraging, incluing recommenations for enhancement of the controller implementation [2] he work on the hexapo walking robot is at a much earlier stage here are a number of problems to overcome Perhaps most significant is the noise generate by the potentiometer base feeback signal from the servosystem, this can be seen clearly in all the iagrams above his reuires either complete replacement, or perhaps some filtering However, espite these ifficulties, the two sets of experimental results escribe in this paper inicate that an aaptive on-line neuro-control approach is not only capable of yieling accurate control performance when applie to complex non-linear plant, it can also provie useful aaptive responses to time varying ynamic loa conitions Although there was not space to print them in full here, these algorithms are base on thorough analyses of system convergence an stability, that we have summarise here an publishe fully elsewhere [4, 5] References [1] Albus, JS, 1975, "A New Approach to Manipulator Control: he Cerebellar Moel Articulation Controller (CMAC)", Journal of Dynamics Systems, Measurement, & Control, Vol97, pp [2] Cherian, R P, 1997, On-Line Learning Neuro- Control of an Inustrial Manipulator, MSc hesis, Engineering Faculty, Univ of the West of Englan [3] Iiguni, Y, Sakai, H, 1991, an okumaru, H, "A Nonlinear Regulator Design in the Presence of System Uncertainties Using Multilayere Neural Networks", IEEE rans on Neural Networks, Vol2, pp [4] Jin Y, Pipe A G, Winfiel A, 1997, Chapter 5: Stable Manipulator rajectory Control Using Neural Networks, Neural Systems for Robotics, Es Omi Omivar & Patrick van er Smagt, Acaemic Press, ISBN [5] Jin Y, 1994, Intelligent Neural Control an Its Applications to Robotics, PhD hesis, Engineering Faculty, Univ of the West of Englan [6] Kawato, M, Uno, Y, Isobe, M, an Suzuki, R, 1988, "Hierarchical Neural Network Moel for Voluntary Movement with Application to Robotics", IEEE Control Sys Magazine, Vol8, No2, pp8-15 [7] Kraft, LG, Miller, W an Dietz, D, 1992, "Development an Application of CMAC Neural Network-Base Control", Hanbook of Intelligent Control, Neural, Fuzzy, an Aaptive Approaches, es: DAWhite an DASofge, New York: Multiscience Press Inc, pp [8] Lewis, F W an Jagannathan, S, 1996, Neural Network Control of Robot Manipulators an Nonlinear Systems, aylor & Francis books [9] Lewis, F W, Jagannathan, S an Yesilirek A, 1999, Neural Network Control of Robot Manipulators an Non-linear Systems, aylor & Francis books [1] Lyapunov, A M, 1992, he General Problem of the Stability of Motion, aylor & Francis books [11] Miller III, W, Glanz, FH, Kraft III, LG, 1987, "Application of a General Learning Algorithm to the Control of Robotic Manipulators", IntJournal of Robotics Research, Vol6, No2 pp84-98 [12] Narenra, K S an Parthasarathy K, 199, Ientification an control of ynamical systems using neural networks, IEEE rans Neural Networks, vol 1, pp4-27 [13] Narenra, K S, 1992, Aaptive Control of ynamical systems using neural networks, in Hanbook of Intelligent Control, e White, D A an Sofge, D A, New York: Van Nostran Reinhol, pp [14] Sanner, RM an Slotine, JJE, 1992, "Gaussian Networks for Direct Aaptive Control", IEEE rans on Neural Networks, Vol3, pp [15] Suykens, Van e Wall & De Moor, 1995, Artificial Neural Networks for Moelling an Control of Non- Linear Systems, Kluwer Acaemic Publishers [16] Wen J, Bayar DS, 1988, "New Class of Control Laws for Robotic Manipulators, Part 1: Non- Aaptive Case", International Journal of Control, Vol47, pp