An Integrated Architecture of Adaptive Neural Network Control for Dynamic Systems

Transcription

1 An Integrated Arhiteture of Adaptive Neural Network Control for Dynami Systems Robert L. Tokar 2 Brian D.MVey2 'Center for Nonlinear Studies, 2Applied Theoretial Physis Division Los Alamos National Laboratory, Los Alamos, NM, Abstrat In this study, an integrated neural network ontrol arhiteture for nonlinear dynami systems is presented. Most of the reent emphasis in the neural network ontrol field has no error feedbak as the ontrol input, whih rises the lak of adaptation problem. The integrated arhiteture in this paper ombines feed forward ontrol and error feedbak adaptive ontrol using neural networks. The paper reveals the different internal funtionality of these two kinds of neural network ontrollers for ertain input styles, e.g., state feedbak and error feedbak. With error feedbak, neural network ontrollers learn the slopes or the gains with respet to the error feedbak, produing an error driven adaptive ontrol systems. The results demonstrate that the two kinds of ontrol sheme an be ombined to realize their individual advantages. Testing with disturbanes added to the plant shows good traking and adaptation with the integrated neural ontrol arhiteture. 1 INTRODUCTION Neural networks are used for ontrol systems beause of their apability to approximate nonlinear system dynamis. Most neural network ontrol arhitetures originate from work presented by Narendra[I), Psaltis[2) and Lightbody[3). In these arhitetures, an identifiation neural network is trained to funtion as a model for the plant. Based on the neural network identifiation model, a neural network ontroller is trained by bakpropagating the error through the identifiation network. After training, the identifiation network is replaed by the real plant. As is illustrated in Figure 1, the ontroller reeives external inputs as well as plant state feedbak inputs. Training proedures are employed suh that the networks approximate feed forward ontrol surfaes that are funtions of external inputs and state feedbaks of the plant (or the identifiation network during training). It is worth noting that in this arhiteture, the error between the plant output and the desired output of the referene model is not fed bak to the ontroller, after the training phase. In other words, this error information is ignored when the neural network applies its ontrol. It is well known in ontrol theory that the error feedbak plays a signifiant role in adaptation. Therefore, when model unertainty or noise/disturbanes are present, a feed forward neural network ontroller with only state feedbak will not adaptively update the ontrol signal. On line training for the neural ontroller has been proposed to obtaip adaptive ability[i)[3). However, the stability for the on line training of the neural network ontroller is unresolved[1][4]. In this study, an additional nonlinear reurrent network is ombined with the feed forward neural network ontroller to form an adaptive ontroller. This added neural network uses feedbak error between the referene model output and the plant output as an input In addition, the system's external

2 132 Liu Ke, Robert L. Tokar, Brian D. MVey inputs and the plant states are also input to the feedbak network. This arhiteture is used in the ontrol ommunity, but not with neural network omponents. The approah differs from a onventional error feedbak ontroller, suh as a gain sheduled PID ontroller, in that the neural network error feedbak ontroller implements a ontinuous nonlinear gain sheduled hypersurfae, and after training, adaptive model referene ontrol for nonlinear dynami systems is ahieved without further parameter omputation. The approah is tested on well-known nonlinear ontrol problems in the neural network literature, and good results are obtained. 2 NEURAL NETWORK CONTROL In this setion, several different neural network ontrol arhitetures are presented. In these strutures, identifiation neural networks, viewed as aurate models for real plants, are used. 2.1 NEURAL NETWORK FEED FORWARD CONTROL The neural network ontrollers are trained by bakpropagation of errors through a well trained neural identifiation network. In this arhiteture, the state variable yet) of the system is sent bak to the neural network, and the external input x(t) also is input to the network. With these inputs, the neural network estabjishes a feed forward mapping from the external input x(t) to the ontrol signal u(t). This ontrol mapping is expressed as a funtion of the external input x(t) and the plant state yet): u(t)==j(x(t), yet» (1) where x(t)=[x(t), x(t-l),.. J, andy(t)=[y(t), y(t-l),..y. This neural network ontrol arhiteture is denoted in this study as feed forward neural ontrol even though it inludes state feedbak. Neural ontrol with error feedbak is denoted as feedbak neural ontrol. x(t) -..:...r-----ref. Modelf , e(t+ 1) x(t) -----Ref. Modell , Control NN y(t+ 1) u(t), , f y(t+ 1) Figure I Neural Network Control Arhiteture. ID NN represents the identifiation network. Ref. Model means referene model, and NN means neural network. Figure 2 Neural Network Feedbak Control Arhiteture During the training phases, based on the assumption that the neural identifiation network provides a model for the plant, the gradient information needed for error bakpropagation is obtained by alulating the Jaobian of the identifiation network. The following equation desribes this proess for the ontrol arhiteture shown in Figure I. If the ost funtion is defined as E, then the gradient of the ost funtion with respet to weight w of the neural ontroller is

3 An Integrated Arhiteture of Adaptive Neural Network Control for Dynami Systems 133 a E a E a u (a E a u a E J a Yt-l a: = a; a w + a u a Yt-l + a Yt-l --a;- (2) where u is tbe ontrol signal and YI-1 is tbe plant feedbak state. After tbe training stage, tbe neural network supplies a ontrol law. Beause neural networks have the ability to approximate any arbitrary nonlinear funtions[5], a feed forward neural network an build a nonlinear ontroller, whih is ruial to tbe use of tbe neural network in ontrol engineering. Also, sine all tbe parameters of the neural network identifiation model and tbe neural network ontroller are obtained from learning through samples, matbematially untraeable features of tbe plant an be extrated from tbe samples and imbedded into tbe ontrol system. However, beause tbe feed forward ontroller has no error feedbak, tbe ontroller an not adapt to tbe disturbanes ourring in tbe plant or tbe referene model. This problem is of substantial importane in tbe ontext of adaptive ontrol. In tbe next subsetion, error feedbak between tbe referene models and tbe plant outputs is introdued into neural network ontrollers for adaptation. 2.2 NEURAL ADAPTIVE CONTROL WITH ERROR FEEDBACK It is known that feedbak errors from the system are important for adaptation. Due to the flexibility of the neural network arhiteture, the error between the referene model and the plant an be sent bak to the ontroller as an extra input. In suh an arhiteture, neural networks beome nonlinear gain sheduled ontrollers with smooth ontinuous gains. Figure 2 shows the arhiteture for the feedbak neural ontrol. With tbis arhiteture, tbe neural network ontrol surfae is not tbe fixed mapping from tbe x(t) to u(t) for eah state y(t), but instead it learns tbe slope or tbe gain referring to tbe feedbak error e(t) for ontrol. This gain is a ontinuous nonlinear funtion of tbe external input x(t) and tbe state feedbak yet). Figure 3 shows tbe reurrent network arhiteture of tbe feedbak neural ontroller. The output node needs to be reurrent beause tbe output witbout tbe reurrent link from tbe neural ontroller is only a orretion to tbe old ontrol signal, and tbe new ontrol signal should be tbe ombination of old ontrol signal and tbe orretion. The otber nodes of tbe network an be feed forward or reurrent. If we denote tbe weight for tbe output node's reurrent link as w., tben tbe output from tbe reurrent link is w.u(t-l). The following equation desribes the feedbak network. u(t) = wbu(t-i )+j(x(t), y(t), e(t» (3) where j(.) is a nonlinear funtion established by tbe network for whih tbe reurrent link output is not inluded and e(t)=[e(t), e(t-i),... f To ompare tbe ontrol gain expression with onventional ontrol theory, onsider tbe Taylor series expansion of tbe network forward mappingj(.), equation (3) beomes u(t) = w.u(t-l) +!'(x(t). yet» e(t)+ j"(x(t), yet» e 2 (t)+... (4) where f'(x(t), y(t»=[ i1j(x(t), y(t), e(t»/ae(t), aj!:x(t), y(t), e(t»ii1e(t-i),...]. ignored and go representsf'o, we get If high order terms are u(t) = wbu(t-i)+ g(x(t), yet» e(t) (5)

4 134 Liu Ke, Robert L. Tokar, Brian D. MVey whih is a gain sheduled ontroller and the gain is the funtion of external input x(/) and the plant state y(/). It is lear that when w.=l.o, g(.) is a onstant vetor and e(/)=[e(t), e(t-l), e(t-2)]t, the feedbak neural network ontroller degenerates to a disrete PID ontrouer. Beause the neural network an approximate arbitrary nonlinear funtions through learning, the neural network feedbak ontroller an generate a nonlinear ontinuous gain hypersurfae. Ref. Model Figure 3 Feedbak Neural Network Controller Figure 4 Integrated NN Control Arhiteture. In the training proess, error bakpropagating through the identifiation network is used. The proess is similar to the training of a feed forward neural ontroller, but the resulting ontrol surfae is ompletely different due to the different inputs. After training, the neural network is able to provide a nonlinear ontrol law, that is, the desired model following response an be obtained with fixed ontroller parameters for nonlinear dynami systems. Traditionally, the ontrol of the nonlinear plant is derived from ontinuous omputing of the ontroller gains. This feedbak ontroller is error driven. As long as an error exists, the ontrol signal is updated aording to the error and the gain. This kind of neural ontroller is an adaptive ontroller in priniple. 2.3 INTEGRATED NEURAL NETWORK CONTROLLER The harateristis of feed forward and error feedbak neural ontrol networks are desribed in the previous subsetions. In this setion. the two ontrollers are ombined. Figure 4 shows the arhiteture. In this arhiteture, we inlude both feed forward and feedbak neural network ontrollers. The ontrol signal is the ombination from these two networks' outputs. In the training stage, it is our experiene that the feed forward network should be trained first. The feedbak network is not inluded while training the feed forward network. After training the feed forward ontroller, the error feedbak network is trained with the feed forward network, but the feed forward networks' weights are unhanged. Bakpropagating the error through the identifiation network is applied for the training of both networks. When training the feedbak ontrol network, the feed forward alulation is u(t) = ujt)+u/b(t). y(t+ 1) = P(x(t), y(t), u(t», where uj/) is the output from the feed forward ontroller network and u,..(t) is the output from the feedbak ontroller network, P(.) is the identifiation mapping. (6) (7)

5 An Integrated Arhiteture of Adaptive Neural Network Control for Dynami Systems CONTROL ON EXAMPLE PROBLEMS In this setion, the ontrol arhiteture desribed above is applied to a well-known problem from the literature[i). The plants and the referene model of the sample problems are desribed by differene equations yet) plant: y(t + 1) = 2 + (u(t) -1.O)u(t)(u(t) + 1.) (II) 1.+ Y (t) referene model: y(t + 1) =.6y(t) + u(t) (12) This is a nonlinear time varying dynami system with no analytial inverse. 3.1 FEED FORWARD CONTROL A feed forward neural network is trained to ontrol the system to follow the referene model. The plant state yet) and external inputx(t) are fed to the ontroller. During the training, the x(t) is randomly generated. After training, the ontroller generates a ontrol signal u(t) suh that the plant an follow the referene model output. Figure 5 shows the testing result of the referene model output and the ontrolled plant output. The input funtion is x(t)=sin(21ttf25)+sin(21tt/1o). The ontroller network arhiteture is (2, 2, 1). 4 OJ Q) 2 1- OJ '" 1J -2 '" ::J 2 2? (j ()... 1 u Figure 5 Traking Result From the Feed Forward NN. Output of referene (solid line) and plant (dash line). Figure 6 Feed Forward Control Surfae The output surfae of the ontroller network is shown in Figure 6. By examining the ontroller output surfae, we an see that the neural network builds a feed forward mapping from x(t) to u(t). This feed forward mapping is also a funtion of the plant state yet). Under eah state, the neural network ontroller aepts input x(t) to produe ontrol signal u(t) suh that the plant follows the referene model reasonably well. In Figure 6, the x axis is the external input x(t) and the y axis is the plant feedbak output yet). The z axis represents the ontrol surfae. The feed forward ontroller lacks the ability to adapt to plant unertainty, noise or hanges in the referene model. As an example, we apply this feed forward ontroller to the disturbed plant with a bias.5 added to the original plant. The traking result is shown in Figure 7. With this slight bias, the plant does not follow the referene model. Clearly, the feed forward ontroller has no adaptive ability to this model bias.

6 136 Liu Ke, Robert L. Tokar, Brian D. MVey 3.2 FEEDBACK CONTROL FtrSt, we ompare the neural network feedbak ontroller with fixed gain PID ontrollers. For many nonlinear systems, the fixed gain PID ontrollers will give poor traking and ontinuous adaptation of the ontroller parameters is needed. The neural network approah offers an alternative ontrol approah for nonlinear systems. Through the training, ontrol gains, imbedded in the neural network, are established as a ontinuous funtion of system external inputs x(t) and plant states yet). The sample problem in the above setion is now employed to desribe how the neural network reates a nonlinear ontrol gain surfae with error feedbak and additional inputs. First, we show one simple ase of neural adaptive feedbak ontroller. This ontroller an only adapt to the system nonlinearity with a fixed linear input pattern. The reason to show this simple adaptation ase first is that its ontrol gain surfae an be illustrated graphially. Figure 8 illustrates, for the system in equations (11) and (12) that a fixed gain PI ontroller fails to trak the referene model, for even one fixed linear input pattern x(t)=.2t-2.5, beause the plant nonlinearity. Figure 9 illustrates the result from a reurrent neural network with feedbak error e(t) and x(t) as inputs. The neural network is trained by bakpropagation error through the identifiation network. Compared to the flxed gain PI ontroller, the neural network improves the traking ability signifiantly.., 4.,.,.. 2 '!! -.::I C -2 6 OJ, <J I C I 3.2 ' t Figure 7 Traking Result for Shifted Plant, plant output (dash line) and referene output (solid line). o t Figure 8 Referene Model Output (solid line) and PID Controlled Plant Output (dashed line) The ontrol surfae of the updating output fl.) is shown in Figure 1, whih is the output from the neural network ontroller without reurrent link (see equation (3». We plot the surfae of the updating output from the ontroller with respet to input x(t) and error feed bak input e(t). The gain of the ontroller is equivalent to the updating output from the network when error=l.o. As shown in the figure, the gain in the neighborhood about x(t)=o hanges largely aording to the diretion of hanges in the plant in the orresponding region. The updating surfae for a PID ontroller is a plane. The neural network implements a nonlinear ontinuous ontrol gain surfae. For a more ompliated ase, we addx(t-i) as another input to the neural network as well as e(t-l), and train by error bakpropagation through the identifiation network. These two inputs, x(t) and x(t-i) add differene information to the network. The network an adapt to not only different operating regions indiated by x(t), but also different input patterns. Figure 11 shows the traking results with two different input patterns. In Figure II (a), input pattern is x(t)=4.sin(ti4.). In Figure 11 (b) input pattern is x(t)=sin(21t1!25)+sin(21t111o).

7 An Integrated Arhiteture of Adaptive Neural Network Control for Dynami Systems 137 Q) u 2 6 3,, I I - 6 o t Figure 9 Referene Model Output (solid line) and Neural Network Controled Output (dashed line) Figure 1 Feedbak Neural Controller Updating Surfae OJ., 4 u u 3 2 OJ 1 l' -1-2 " : - 4 " C (a) Figure 11 Output of the Referene Model (solid line) and the Plant (dash line) 3.3 INTEGRA TED NEURAL CONTROLLER As shown in the above setion, when only error feedbak neural ontroller is used, the ontrol result is not very aurate. Now we ombine feed forward and feedbak ontrol to realize good traking and adaptation. Figure 12 shows the ontrol result from the integrated ontroller when the plant is shifted O.S. Compared to only feed forward ontrol (Figure 7), the integrated ontroller has muh better adaptation to the shifted plant. When the plant hanges, adding an extra feed bak ontroller an avoid on-line training of feed forward network whih may indue potential instability, and the adaptation is ahieved. The output from the feedbak network ontroller is driven by the error between the referene model and the plant. 4 DISCUSSIONS We have emphasized in the above setions that a feed forward ontroller with only state feedbak does not adapt when model unertainties or noise/disturbane are present. The presene of a feed bak ontroller an make the on line training of the feed forward network unneessary, thus avoiding potential instability. The main reason for the instability of on-line training is the inompleteness of sample sets, whih is referred to as a lak of persistent exitation in ontrol theory[6]. First, it leads to an inaurate identifiation network. Training with this network an result in an unstable ontroller. Seond, it makes the training of ontroller away from global representation. With an error feedbak adaptive network, the output from the feedbak network ontroller is driven by the error between the referene model and the plant. In the simplest ase when all the ativity funtions are linear and only the feedbak errors are inputs, this kind of neural network is equivalent to a PID ontroller. However, 5 (b)

8 138 Liu Ke, Robert L Tokar, Brian D. MVey beyond the sope of PID ontrollers, the neural networks are apable to approximating nonlinear time variant ontrol gain surfaes orresponding to different operating regions. Also, unlike a PID ontroller, the oeffiients for the neural adaptive ontroller are obtained through a training proedure. Q) u 1:' Q) ;... >, " C >, 4 o Figure 12 Integrated Network Controller Traking Result for Shifted Plant. Plant Output (dash line) and Referene Output (solid line). The error feedbak network behaves as a gain sheduling ontroller. It has rise time, overshoot onsideration and delay problem. Feed forward ontrol an ompensate for these problems to some degree. For example, the feed forward network an perform a nonlinear mapping with designed time delay. Therefore with the feed forward network, the delay problem maybe overame signifiantly. Also the feed forward ontroller an help to redue rise time ompare to use only feedbak ontroller. With the feed forward network, the feedbak network ontroller an have muh smaller gains ompared to using a feedbak network alone. This inreases the noise rejetion ability. Also this redues the overshoot as well as settle time. The neural network ontrol arhiteture offers an alternative to the onventional approah. It gives a generi model for the broadest lass of systems onsidered in ontrol theory. However this model needs to be onfigured depending on the details of the ontrol problem. With different inputs, the neural network ontrollers establish different internal hyperstates. When plant states are fed bak to the network, a feed forward mapping is established as a funtion of the plant states by the neural network ontroller. When the errors between the referene model and the plant are used as the error feedbak inputs to a dynami neural network ontroller, the network funtions as an assoiative memory nonlinear gain sheduled ontroller. The above two kinds of neural ontroller an be ombined and omplemented to ahieve aurate traking and adaptation. Referenes [1] Kumpati S. Narendra and Kannan Parthasarathy. "Gradient Methods for the Optimization of DynamiCal Systems Containing Neural Networks," IEEE Trans. Neural Networks. vol. 2. pp Mar [2] Psaltis. D. Sideris. A. and Yamamura. A., "Neural ontrollers." Pro. of 1st International Conferene on Neural Networks. Vol. 4. pp San Diego. CA [3) G. lightbody. Q. H. Wu and G. W. Irwin. "Control appliations for feed forward networks." Chapter 4. Neural Networks for Control and Systems. Edited by K.warwih, G. W. Irwin and K. J. Hunt 1992 [4) R. Abikowski and P. 1. Gawthrop. "A survey of neural networks for ontrol" Chapter 3. NeUral Networks for Control and Systems. ISBN Edited by K.warwih. G. W. Irwin and K. 1. Hunt 1992 [5] John Hertz. Anders Krogh and Rihard G. Palmer. "Introdution to the Theory of Neural Computation." [6J Thomas Miller. RiChard S. Sutton and Paul 1. Werbos. "Neural Networks for Control"