Identication and Control of Nonlinear Systems Using. Neural Network Models: Design and Stability Analysis. Marios M. Polycarpou and Petros A.

Transcription

1 Identication and Control of Nonlinear Systems Using Neural Network Models: Design and Stability Analysis by Marios M. Polycarpou and Petros A. Ioannou Report September 1991

2 Identication and Control of Nonlinear Systems Using Neural Network Models: Design and Stability Analysis Marios M. Polycarpou and Petros A. Ioannou Department of Electrical Engineering-Systems University of Southern California, MC-2563 Los Angeles, CA , U.S.A Technical Report September 1991 Abstract The feasibility of applying neural network learning techniques in problems of system identication and control has been demonstrated through several empirical studies. These studies are based for the most part on gradient techniques for deriving parameter adjustment laws. While such schemes perform well in many cases, in general, problems arise in attempting to prove stability of the overall system, or convergence of the output error to zero. This paper presents a stability theory approach to synthesizing and analyzing identication and control schemes for nonlinear dynamical systems using neural network models. The nonlinearities of the dynamical system are assumed to be unknown and are modelled by neural network architectures. Multilayer networks with sigmoidal activation functions and radial basis function networks are the two types of neural network models that are considered. These static network architectures are combined with dynamical elements, in the form of stable lters, to construct a type of recurrent network conguration which is shown to be capable of approximating a large class of dynamical systems. Identication schemes based on neural network models are developed using two dierent techniques, namely, the Lyapunov synthesis approach and the gradient method. Both identication schemes are shown to guarantee stability, even in the presence of modelling errors. A novel network architecture, referred to as dynamic radial basis function network, is derived and shown to be useful in problems dealing with learning in dynamic enviroments. For a class of nonlinear systems, a stable neural network based control conguration is presented and analyzed.

3 1 Introduction Adaptive identication and control of dynamical systems has been an active area of research for the last three decades. Although methods for controlling linear time invariant plants with unknown parameters had been pursued since the 1960's, it was not until the last decade that stable adaptive laws were established [1]-[4]. Recent advances in nonlinear control theory and, in particular, feedback linearization techniques [5, 6], have initiated activity aimed at developing adaptive control schemes for nonlinear plant models [7]-[10]. This area, which came to be known as adaptive nonlinear control, deals with systems where the uncertainty is due to unknown parameters which appear linearly with respect to known nonlinearities. Therefore, adaptive control research so far has been directed towards systems with a special class of parametric uncertainties. The emergence of the neural network paradigm as a powerful tool for learning complex mappings from a set of examples has generated a great deal of excitement in using neural network models for identication and control of dynamical systems with unknown nonlinearities [11, 12]. Due to their approximation capabilities as well as their inherent adaptivity features, articial neural networks present a potentially appealing alternative to modelling of nonlinear systems. Furthermore, from a practical perspective, the massive parallelism and fast adaptability of neural network implementations provide more incentives for further investigating the connectionist approach in problems involving dynamical systems with unknown nonlinearities. The feasibility of applying neural network architectures in identication and control has been demonstrated, through simulations, by several studies, including [13]-[17]. The problem was originally formulated by Narendra et.al. [13], whose work instigated further research in this area. For the most part, these studies are based on rst replacing unknown functions in the dierence equation by static neural networks and then, according to a quadratic cost function, deriving update laws using optimization methods. A particular optimization procedure that has attracted a lot of attention is the steepest descent, or gradient method, which leads to the static backpropagation, or dynamic backpropagation-type algorithms [18]-[20], depending on whether the dynamical behavior of the system is taken into account or not. While such schemes perform well in many cases, in general, problems arise in attempting to prove stability of the overall system, or convergence of the output error to zero. Interestingly, methods based on straightforward application of optimization techniques (such as sensitivity models [21] and the M.I.T. rule [22]), that dominated the early adaptive linear control literature, exhibited similar stability problems. The fact that, even for linear systems, such methods can lead to instability was shown in [23, 24] (see also [25]). In this paper we present a design procedure, based on stability theory, for modelling, identication and adaptive control of continuous-time nonlinear dynamical systems using neural network architectures. Particular emphasis is placed on the synthesis and stability analysis of the proposed schemes. The techniques developed here share some fundamental features with the parametric methods of both adaptive nonlinear control as well as adaptive linear control theory. Therefore, like earlier works [26, 13, 20, 27] this study also serves as 1

4 an attempt to unify learning methods used by connectionists and adaptive control theorists. More specically, one of the objectives of the paper is the development of stable neural network learning techniques for dynamic enviroments. Our approach in this paper is based on continuous-time system representations. In particular, the plant dynamics are considered to evolve in continuous-time and furthermore, adaptive laws based on continuous adjustment of the weights are developed and analyzed. Due to the convenient \tapped delay line" representation, most of the current literature on identication and control of nonlinear systems using neural networks is developed in a discrete-time framework with iterative weight update rules. However, as also pointed out in [28], studying the problem in discrete-time has several drawbacks. It is known that discretization of physical continuous-time nonlinear systems yields highly complex discrete-time models [6]. More precisely, discretization of ane (in the input) continuous-time systems results in non-ane discrete-time models which are almost impossible to analyze. The approach usually taken by nonlinear control theorists is to design a control system based on continuous-time plant model and controller and then study the eect of sampling on the overall system [29]. Furthermore, from a practical perspective, if the neural network approach to control is to achieve its full potential, then it will be through parallel hardware realizations of neural network architectures implemented for real-time control systems. The current trend towards analog realizations of neural networks suggests considering identication and control schemes based on continuous-time systems with continuous-time adaptation. From a theoretic point of view, neural networks can be considered as versatile mappings whose response to a specic input or pattern is determined by the values of adjustable weights. In our analysis we consider two types of neural network architectures: 1) multilayer neural network models with sigmoidal nonlinearities and 2) radial basis function networks with Gaussian activation functions. Multilayer neural networks [30] are by far the most widely used neural network models. They have been applied successfully to a wide range of problems including pattern and speech recognition, image compression, signal prediction and classi- cation [31]. Due to some desirable features such as local adjustment of the weights and mathematical tractability, radial basis function networks have recently also attracted considerable attention especially in applications dealing with prediction and classication [32]-[34]. The importance of radial basis function networks has also greatly benetted from the work of Poggio et.al. [35]-[37], where the relationship between regularization theory and radial basis function networks is explored. The approximation capabilities of static sigmoidal type networks and of radial basis function networks has been studied by several research groups (see for example, [38]-[40]). In Section 2 we use these results to show that a proposed combination of static neural networks with dynamical components, such as stable lters, form a type of recurrent network capable of approximating a large class of dynamical systems. More precisely, it is shown that there exists a set of weights such that for a given input, the outputs of the real system and the proposed recurrent neural network model remain arbitrarily close over a nite interval of time. System identication consists of rst choosing an appropriate identication model and then adjusting the parameters of the model according to some adaptive law such that the 2

5 response of the model to an input signal approximates the response of the real system to the same input. Since a mathematical characterization of a system is often a prerequisite to analysis and controller design, system identication is important not only for understanding and predicting the behavior of the process, but also for designing eective control laws. In Section 3 we develop and analyze two dierent neural network based identication schemes. The rst scheme relies on the Lyapunov synthesis approach while the second is derived based on optimization methods. In this context, we introduce a novel architecture, referred to as dynamic radial basis function network, which is shown to be useful for applying optimization techniques in problems dealing with learning in dynamic enviroments. In Section 4, we consider the problem of controlling simple classes of nonlinear systems. Due to the intricacies of the problem, we rst discuss in detail three types of mechanisms that can lead to instability, namely, the parameter drift, controllability and transient behavior problems, and propose modications to the standard control and adaptive laws for dealing with these problems. Based on these modications, we prove stability of the overall control system. Finally, in Section 5 we discuss the contribution of the paper and draw some nal conclusions. Due to the length of the paper, simulation results are omitted. 2 Modelling of Dynamical Systems by Neural Networks One of the basic requirements in using neural network 1 architectures to represent, identify and control nonlinear dynamical systems is the capability of these architectures to accurately model the behavior of a large class of dynamical systems that are encountered in science and engineering problems. This leads to the question of whether a given neural network conguration is able to approximate, in some appropriate sense, the input-output response of a class of dynamical systems. In order to be able to approximate the behavior of a dynamical system, it is clear that any proposed neural network conguration must have some feedback connections. In the neural network literature such networks are known as recurrent networks. The input-output response of neural networks, whether static or recurrent, is determined by the values of a set of parameters which are referred to as weights. Therefore the representational capabilities of a given network depend on whether there exists a set of weight values such that the neural network conguration approximates the behavior of a given dynamical system. The terms \weights" and \parameters" are used interchangeably throughout the paper. In this section we consider the problem of constructing a neural network architecture that is capable of approximating the behavior of continuous-time dynamical systems, whose input-state-output representation is described by _x(t) = f(x(t); u(t)) x(0) = x 0 y(t) = h(x(t); u(t)) (2:1) 1 The term \neural network" is used very loosely since some of the architectures considered in this paper have very little, if any, relation to biological neurons. In this context, \neural networks" are better interpreted as versatile mappings represented by the composition of many basic functions structured in a parallel fashion. 3

6 Figure 1: A recurrent neural network conguration for modelling the general dynamical system described by (2.1). where u 2 R m is the input, x 2 R n is the state, y 2 R p is the output and t 2 R + is the temporal variable. The input u belongs to a class U of (piecewise continuous) admissible inputs. By adding and subtracting Ax, where A is a Hurwitz or stability matrix (i.e, has all of its eigenvalues in the open left-half complex plane), (2.1) becomes _x = Ax + g(x; u) ; y = h(x; u) (2:2) where g(x; u) := f(x; u)? Ax. Based on (2.2), we construct a recurrent network model by replacing the mappings g and h by feedforward (static) neural network architectures, denoted by N 1 and N 2 respectively. Therefore we consider the model _^x = A^x + ^g(^x; u; g ) ^x(0) = ^x 0 ^y = ^h(^x; u; h ) (2:3) ^x where ^g and ^h are the outputs of the static neural networks N 1 and N 2 respectively, while g and h denote the adjustable weights of these networks. In (2.3), ^x and ^y denote the state and output respectively of the recurrent network model. Corresponding to the Hurwitz matrix A, we let W (s) := (si? A)?1 be an n n matrix whose elements are stable transfer functions and s denotes the dierential (Laplace) operator. Based on this denition of W (s) as a stable lter, a block diagram representation of the recurrent network model described by (2.3) is depicted in Figure 1. This interconnection of static neural nets and dynamic components is proposed for modelling the input-output response of the general dynamical system described by (2.1). If we suppose that the real system and the proposed model are initially at the same state (i.e, ^x 0 = x 0 ), then the natural question to ask is whether there exist weights g, h such that the input-output behavior (u 7! ^y) of the neural network model (2.3) approximates, in some sense, the input-output behavior (u 7! y) of the real system (2.1). This leads to the validity of the proposed model. In examining this question, we will impose the following mild assumptions on the system to be approximated: N 1 g ^ W(s) ^x 4 u N 2 ^ h ^ y

7 Figure 2: Block diagram representation of a two-layer sigmoidal neural network. (S1) Given a class U of admissible inputs, then for any u 2 U and any nite initial condition x 0 the state and output trajectories do not escape to innity in nite time, i.e, for any nite T > 0 we have jx(t )j + jy(t )j < 1. (S2) The vector elds f : R n+m 7! R n and h : R n+m 7! R p are continuous with respect to their arguments. Furthermore, f satises a local Lipschitz condition so that the solution x(t) to the dierential equation (2.1) is unique for any nite initial condition x 0 and u 2 U. The above assumptions are required in order to guarantee that the solution to the system described by (2.1) exists and is unique for any nite initial condition x 0 and any admissible input u 2 U. We will also assume that the static neural network topologies N i, i = 1; 2, that are used to represent the mappings g and h satisfy the following conditions: (N1) Given a positive constant " and a continuous function f : C 7! R p, where C R q is a compact set, there exists a weight vector = such that the output ^f(x; ) of the neural network architecture N i with n nodes (where n may depend on " and f) satises b max X2C ^f(x; )? f(x) " (N2) The output ^f(x; ) of the neural network architecture N i is continuous with respect to its arguments for all nite (X; ). We next describe two popular neural network architectures that satisfy conditions (N1), (N2). (a) Multilayer sigmoidal neural networks: Multilayer neural networks with sigmoidal type of nonlinearities are by far the most widely used neural network models [30]. From a theoretic point of view, multilayer neural networks may be considered as versatile maps whose response to a specic input is determined by the values of adjustable weights. The inputoutput behavior (x 7! y) of a two-layer sigmoidal neural network 2 (shown in Figure 2) with m inputs, n outputs and n hidden units or neurons is described (through the intermediate n -dimensional vectors z, z) by 2 In this paper we use the notational convention that a k-layer network consists of k? 1 hidden layers. 5 z A 1 + σ(z) A y 2 z

8 Figure 3: Block diagram representation of a Radial Basis Function neural network. z = A 1 x + b z i = (z i ) i = 1; 2; : : : n y = A 2 z where x 2 R m is the network input, z 2 R n is the input to the sigmoidal nonlinearities, z 2 R n is the output of the hidden layer and y 2 R n is the network output; the adjustable weights of the network are the elements of A 1 2 R nm, A 2 2 R nn, b 2 R n. The weight vector b is known as the oset or bias weight vector. In order to satisfy (N2), the sigmoidal nonlinearity : R 7! R should be continuous. The hyperbolic tangent and the logistic function are examples of continuous sigmoids that are often used in neural network applications [31]. It is well known (see e.g, [38, 39, 41] and references therein) that if n is large enough then there exist weight values A 1, A 2, b such that the above two-layer neural network can approximate any continuous function f(x) to any degree of accuracy over a compact set. This holds for any function that is nonconstant, bounded and nondecreasing. Therefore multilayer neural networks with as few as two layers satisfy (N1) and hence can be used to model continuous nonlinearities in dynamical systems. Furthermore, it has been shown [42] that three-layer neural networks (i.e, with two hidden layers), are capable of approximating discontinuous functions. Therefore, increasing the number of layers results in a natural enlargement of the class of functions that can be approximated. (b) Radial basis function neural networks: Radial Basis Function (RBF) networks were introduced to the neural network literature by Broomhead et.al. [43] and have since gain signicance in the eld due to several application and theoretical results [32, 33, 35]. Recently, RBF networks have also been considered in adaptive control of nonlinear dynamical systems [28]. The input-output response (x 7! y) of a RBF neural network (shown in Figure 3) with m inputs, n outputs and n hidden, or kernel units, is characterized by i = g (jx? c i j= i ) i = 1; 2; : : : n y = A x where x 2 R m is the input, 2 R n is the output of the hidden layer, y 2 R n is the output of the network; A 2 R nn is the weight matrix, while c i 2 R m and i > 0 are the center and width (or smoothing factor) of the ith kernel unit respectively. The Euclidean or a weighted g ( x - c i / σ i ) ξ 6 A y c i σ i { i = 1, 2,..., n*}

9 Euclidean norm j j is often used. The continuous function g : [0; 1) 7! R is the activation function which is usually chosen to be the Gaussian function g() := e?2. Depending on the application, the centers c i and/or widths i of the network can either be adjustable (during learning) or they can be xed. For example, in control applications it is very crucial for analytical purposes to choose c i, i apriori according to some preliminary training or an ad-hoc procedure and keep these values xed during the learning phase. By doing so, the nonlinearities g() appear linearly with respect to the adjustable weights, which, as we will see later, simplies the analysis considerably. It has recently been shown that under mild assumptions RBF neural networks are capable of universal approximation, i.e approximation of any continuous function over a compact set to any degree of accuracy [40, 44]. Therefore, Gaussian RBF networks also satisfy (N1), (N2) and hence they are candidates for modelling nonlinearities of dynamical systems. Remark 2.1: In addition to sigmoidal and RBF neural networks, there are, of course, other representations that can be used for approximating static maps. Considerable more studies, both of theoretic and of practical nature, need to be performed before it is clear which architecture is best for approximating dierent classes of functions. Practical issues such as cost of hardware implementation, exibility to increasing the number of nodes, speed of weight adjustment, as well as theoretic considerations such as number of parameters required in the approximation and robustness, are very important in choosing an appropriate network design. Using Assumptions (S1)-(S2), (N1)-(N2), the following theorem establishes the capability of the proposed recurrent network architecture depicted in Figure 1 to approximate the behavior of the real system over a nite interval of time. Theorem 1 Suppose x(0) = ^x(0) = x 0 and u 2 U R m where U is some compact set. Then given > 0 and a nite T > 0, there exist weight values g, h such that for all u 2 U the outputs of the real system and the recurrent neural network model satisfy max jy(t)? ^y(t)j t2[0;t ] The proof of Theorem 1 is given in Appendix A and relies on standard techniques from the theory of ordinary dierential equations (see for example [45]). Based on the above result, we will assume in the subsequent sections that the nonlinear dynamical system to be identied and/or controlled is represented by a recurrent network conguration with static neural networks replacing the unknown nonlinearities. In this framework, the real system is parametrized by neural network models with known underlying structure and unknown parameters or weights. In order to accomodate for modelling inaccuracies arising, for example, from having insucient number of adjustable weights, we will allow the presence of modelling errors, which appear as additive disturbances in the dierential equation representing the system model. 7

10 Figure 4: A general conguration for identication of nonlinear dynamical systems based on the series-parallel model. 3 Identication In this section we consider the identication of nonlinear systems of the form _x = f(x) + g(x)u (3:1) u where u 2 R is the input, x 2 R n is the state, which is assumed to be available for measurement, and f, g are smooth vector elds dened on an open set of R n. The above class of continuous-time nonlinear systems are called ane systems because in Equation (3.1) the control input u appears linearly with respect to g. There are several reasons for considering this class of nonlinear systems. First, most of the systems encountered in engineering are by nature (or design) ane systems. Secondly, most of the nonlinear control techniques, including feedback linearization, are developed for ane systems. Finally we note that nonane systems (as described by (2.1)) can be converted to ane systems by passing the input through integrators [6]. This procedure is known as dynamic extension. The problem of identication consists of choosing an appropriate identication model and adjusting the parameters of the model according to some adaptive law such that the response ^x of the model to an input signal u (or a class of input signals) approximates the response x of the real system to the same input. Since a mathematical characterization of a system is x often a prerequisite to analysis and controller design, system identication is important not Real System only for understanding and predicting the behavior of the system, but also for obtaining an eective control law. In this paper we consider identication schemes that are based on the setting shown in Figure 4, which is known as the series-parallel conguration [13]. As is common in identication procedures, we will assume that the state x(t) is bounded for all admissible bounded inputs u(t). Note that even though the real system - is boundedinput bounded-state (BIBS) stable, there is no apriori guarantee that the output ^x of ethe identication model or that the adjustable parameters in the model will remain bounded. x x u Identification Model 8 x^ +

11 Stability of the overall scheme depends on the particular identication model that is used as well as on the parameter adjustment rules that are chosen. This section is concerned with the development of identication models, based on sigmoidal and RBF neural networks, and the derivation of adaptive laws that guarantee stability of the overall identication structure. Following the results of Section 2, the unknown nonlinearities f(x) and g(x) are parametrized by static neural networks with outputs ^f(x; f ) and ^g(x; g ) respectively, where f 2 R n f, g 2 R ng are the adjustable weights and n f, n g denote the number of weights in the respective neural network approximation of f and g. By adding and subtracting the terms ^f and ^gu, the nonlinear system described by (3.1) is rewritten as _x = ^f(x; h f) + ^g(x; g)u + f(x)? ^f(x; i h i f) + g(x)? ^g(x; g) u (3:2) where f, g denote the optimal weight values (in the L 1 -norm sense) in the approximation of f(x) and g(x) respectively, for x belonging to a compact set X R n. For a given class of bounded input signals u, the set X is such that it contains all possible trajectories x(t). While being aware of its existence, in our analysis we do not need to know the region X. Although the \optimal" weights f, g in (3.2) could take arbitrarily large values, from a practical perspective we are interested only in weights that belong to a (large) compact set. Therefore we will consider \optimal" weights f, g that belong to the convex compact sets B(M f ), B(M g ) respectively, where M f, M g are design constants and B(M) := f : jj Mg denotes a ball of radius M. In the adaptive law, the estimates of f, g, which are the adjustable weights in the approximation networks, are also restricted to B(M f ), B(M g ) respectively, through the use of a projection algorithm. By doing so, we avoid any numerical problems that may arise due to having weight values that are too large; furthermore, the projection algorithm prevents the weights from drifting to innity, which, as will be apparent later, is a phenomenon that may occur with standard adaptive laws. To summarize, the optimal weight vector f is dened as the element in B(M f ) that minimizes jf(x)? ^f(x; f )j for x 2 X R n ; i.e, f := arg min f 2B(M f ) ( sup x2x ) f(x)? ^f(x; f ) (3:3) Similarly, g is dened as g := arg min g2b(m g) ( sup jg(x)? ^g(x; g )j x2x ) (3:4) Finally, it is noted that if the optimal weights are not unique then f (and correspondingly g) denotes an arbitrary (but xed) element of the set of optimal weights. Equation (3.2) is now expressed in compact form as _x = ^f (x; f ) + ^g(x; g )u + (t) (3:5) 9

12 where (t) denotes the modelling error, dened as (t) := hf(x(t))? ^f(x(t); i i f ) + hg(x(t))? ^g(x(t); g ) u(t) The modelling error (t) is bounded by a constant 0 where 0 := sup f(x(t))? ^f(x(t); f) + g(x(t))? ^g(x(t); g) u(t) t0 Since by assumption, u(t) and x(t) are bounded, the constant 0 is nite. The value of 0 depends on many factors, such as the type of neural network that is used, the number of weights and layers, as well as the \size" of the compact sets X, B(M f ), B(M g ). For example, the constraint that the optimal weights f, g belong to the sets B(M f ) and B(M g ) respectively, may increase the value of 0. However, if the constants M f and M g are large then any increase will be very small. In general, if the networks ^f and ^g are constructed appropriately then 0 will be a small number. Unfortunately, at present, the factors that inuence how well a network is constructed, such as the number of weights and number of layers, are chosen for the most part by trial and error or other ad-hoc techniques. Therefore, a very attractive feature of our synthesis and analysis procedure is that we do not need to know the value of 0. By replacing the unknown nonlinearities with feedforward neural network models, we have essentially rewritten the system (3.1) in the form (3.5), where the parameters f, g and the modelling error (t) are unknown, but the underlying structure of ^f and ^g is known. Based on (3.5), we next develop and analyze various types of identication schemes using both Gaussian RBF networks and multilayer network models with sigmoidal nonlinearities. 3.1 RBF Network Models We rst consider the case where the network architectures employed for modelling f and g are RBF networks. Therefore the functions ^f and ^g in (3.5) take the form ^f = W (x); ^g = W 1 2 (x) (3:6) where W, W are n n and n n 2 matrices respectively, representing in the spirit of (3.3), (3.4), the optimal weight values, subject to the constraints kw k 1 F M 1, kw k 2 F M 2. The norm kk F denotes the Frobenius matrix norm [46], dened as kak 2 F := P n o ij ja ij j 2 = tr AA T, where trfg denotes the trace of a matrix. The constants n 1, n 2 are the number of kernel units in each approximation and the vector elds (x) 2 R n 1, (x) 2 R n 2, which we refer to as regressors, are Gaussian type of functions, dened element-wise as i (x) = e?jx?c 1ij 2 = 2 1i i = 1; 2; n 1 j (x) = e?jx?c 2jj 2 = 2 2j j = 1; 2; n 2 For analysis, it is crucial that the centers c 1i, c 2j and widths 1i, 2j, i = 1; n 1, j = 1; n 2, are chosen apriori. By doing so, the only adjustable weights are W 1, W 2, which 10

13 appear linearly with respect to the nonlinearities and respectively. Based on \local tuning" training techniques several researchers have suggested methods for appropriately choosing the centers and widths of the radial basis functions [32, 47]. In this paper, we will simply assume that c 1i, c 2j, 1i, 2j are chosen apriori and kept xed during adaptation of W 1, W 2. Generally, in the problem of identication of nonlinear dynamical systems one is usually interested in obtaining an accurate model in a (possibly large) neighborhood N (x 0 ) of an equilibrium point x = x 0 and therefore it is intuitively evident that the centers should be clustered around x 0 in this neighborhood. Clearly, the number of kernel units and the position of the centers and widths will aect the approximation capability of the model and consequently the value of the modelling error (t). Hence, current and future research dealing with eectively choosing these quantities is also relevent to the topics discussed in this paper. By substituting (3.6) in (3.5) we obtain _x = W 1 (x) + W 2 (x)u + (3:7) Based on the RBF network model described by (3.7), we next develop parameter update laws for stable identication using various techniques derived from the Lyapunov synthesis approach and also basic optimization methods Lyapunov Synthesis Method The RBF network model (3.7) is rewritten in the form _x =?x + x + W 1 (x) + W 2 (x)u + (3:8) where > 0 is a scalar (design) constant. Based on (3.8) we consider the identication model _^x =?^x + x + W 1 (x) + W 2 (x)u (3:9) where W 1, W 2 are the estimates of W 1, W 2 respectively, while ^x is the output of the identication model. The identication model (3.9), which we refer to as RBF error ltering model, is similar to estimation schemes developed in [9]. The RBF error ltering model is depicted in Figure 5. As can be seen from the gure, this identication model consists of two RBF network architectures in parallel and n rst order stable lters h(s) = 1=(s + ). If we dene e x := ^x? x, the state error, and 1 := W 1? W 1, 2 := W 2? W 2, the weight estimation errors, then from (3.8), (3.9) we obtain the error equation _e x =?e x + 1 (x) + 2 (x)u? (3:10) The Lyapunov synthesis method consists of choosing an appropriate Lyapunov function candidate V and selecting weight adaptive laws so that the time derivative _ V satises _ V 0. The Lyapunov method as a technique for deriving stable adaptive laws can be traced as far back as the 1960's, in the early literature of adaptive control theory for linear systems [23, 48]. 11

14 Figure 5: A block diagram representation of the error ltering identication model developed using RBF networks. x In our case, an adaptive law for generating the parameter estimates W 1 (t), W 2 (t) is developed by considering the Lyapunov function candidate V (e x ; 1 ; 2 ) = 1 2 je xj k 1 k 2 F + 1 k 2 k 2 F = et x e x + 1 n tr 1 T 1 o+ 1 n o tr 2 T (3:11) where 1, 2 are positive constants. These constants will appear in the adaptive laws and are referred to as learning rates or adaptive gains. Using (3.10), the time derivative of V in (3.11) is expressed as _V =?e T x e x + T T 1 e x + 1 n o tr _1 T 1 + T T 2 e x u + 1 n o tr _2 T 2? e T x 1 2 Identification Model RBF -Network ξ α Using properties of the trace, such as we obtain _V =?je x j 2 + tr T T 1 e x = tr n o n o T T e 1 x = tr e x T T 1 ξ (x) e x T T _ 1 1 W T 1 + tr e x u T T _ 2 T 2? e T x s + oα =?je x j tr n 1 e x T + _ 1 T 1 o 1 + tr n 2 e x u T + _ 2 T 2 2? e T x (3.12) ^x Since W 1, W 2 are constant, we have that W _ 1 = _ 1 and W _ 2 = _ 2. Therefore it is clear from (3.12) that if the parameter estimates W 1, W 2 are generated according to the adaptive laws RBF -Network ζ ζ (x) _W 1 =? 1 e x T ; _W 2 =? 2 e x u T (3:13) W 2 12 u

15 then (3.12) becomes _V =?je x j 2? e T x?je x j je x j (3:14) If there is no modelling error (i.e, 0 = 0), then from (3.14) we have that V _ is negative semidenite; hence stability of the overall identication scheme is guaranteed. However in the presence of modelling error, if je x j < 0 = then it is possible that V _ > 0, which implies that the weights W 1 (t), W 2 (t) may drift to innity with time. This problem, which is referred to as parameter drift [49], is well known in the adaptive control literature. Parameter drift has also been encountered in empirical studies of neural network learning, where it is usually referred to as weight saturation. In order to avoid parameter drift, W 1 (t), W 2 (t) are conned to the sets fw 1 : kw 1 k F M 1 g, fw 2 : kw 2 k F M 2 g respectively, through the use of a projection algorithm [50, 54]. In particular, the standard adaptive laws described by (3.13) are modied to (?1 e x _W T if fkw 1 k F < M 1 g or fkw 1 k F = M 1 and e T x W 1 0g 1 = n o P? 1 e x T if fkw 1 k F = M 1 and e T (3.15) x W 1 < 0g _W 2 = (?2 n e x u T o if fkw 2 k F < M 2 g or fkw 2 k F = M 2 and e T x W 2 u 0g P? 2 e x u T if fkw 2 k F = M 2 and e T (3.16) x W 2 u < 0g where Pfg denotes the projection onto the supporting hyperplane, dened as P n? o 1 e x T :=? 1 e x T e T x + W 1 1 W kw 1 k 2 1 F (3.17) P n? o 2 e x u T :=? 2 e x u T e T x W 2 u + 2 W kw 2 k 2 2 F (3.18) Therefore, if the initial weights are chosen such that kw 1 (0)k F M 1, kw 2 (0)k F M 2 then we have kw 1 (t)k F M 1, kw 2 (t)k F M 2 for all t 0. This can be readily established by noting that whenever kw 1 k F = M 1 (and correspondingly for kw 2 k F = M 2 ) then d dt n kw 1 (t)k 2 F? M 2 1 o 0 which implies that the parameter estimate is directed towards the inside or the surface of the ball fw 1 : kw 1 k F M 1 g. It is worth noting that the projection modication causes the adaptive law to be discontinuous. However, the trajectory behavior on the discontinuity hypersurface is \smooth" and hence existence of a solution, in the sense of Caratheodory [45], is assured. The issue of existence and uniqueness of solutions in adaptive systems is treated in detail in [51]. With the adaptive laws (3.15), (3.16), Equation (3.12) becomes ( ) ( ) _V =?je x j 2? e T x + I e T 1 tr x W 1 W kw 1 k 2 1 T 1 + I e T 2 tr x W 2 u W F kw 2 k 2 2 T 2 F?je x j 2? e T x + I e T x W 1 1 tr n o W kw 1 k 2 1 T 1 + I e T x W 2 u 2 tr n o W F kw 2 k 2 2 T 2 (3.19) F 13

16 where I, 1 I are indicator functions dened as 2 I = 1 if the conditions kw 1 1k F = M 1 and e T x W 1 < 0 are satised and I 1 = 0 otherwise (and correspondingly for I2). The following lemma establishes that the additional terms introduced by the projection can only make V _ more negative, which implies that the projection modication guarantees boundedness of the weights without aecting the rest of the stability properties established in the absence of projection. Lemma 1 Based on the adaptive laws (3.15), (3.16) the following inequalities hold: n o (i) I 1 e T x W 1=kW 1 k 2 F tr W 1 T 1 0. (ii) I 2 e T x W 2 u=kw 2 k 2 F n o tr W 2 T 2 0. The proof of Lemma 1 is given in Appendix B. Now, using Lemma 1, (3.19) becomes _V?je x j 2? e T x?je x j je x j (3:20) Based on (3.20), we next summarize the properties of the weight adaptive laws (3.15), (3.16). It is pointed out that the proof of the following theorem employs well known techniques from the adaptive control literature. In the sequel, the notation z 2 L 2 means R 1 0 jz(t)j2 dt < 1 while z 2 L 1 implies sup t0 jz(t)j < 1. Theorem 2 Consider the error ltering identication scheme (3.9). laws given by (3.15), (3.16) guarantee the following properties: (a) For 0 = 0 (no modelling error), we have The weight adaptive e x ; ^x; 1 ; 2 2 L 1, e x 2 L 2. lim t!1 e x (t) = 0, lim t!1 _ 1 (t) = 0, lim t!1 _ 2 (t) = 0. (b) For sup t0 j(t)j 0 we have e x ; ^x; 1 ; 2 2 L 1. there exist constants k 1, k 2 such that Z t Proof: (a) With 0 = 0, Equation (3.20) becomes 0 Z t je x ()j 2 d k 1 + k 2 j()j 2 d 0 _V?je x j 2 0 (3:21) Hence V 2 L 1, which from (3.11) implies e x ; 1 ; 2 2 L 1. Furthermore, ^x = e x + x is also bounded. Since V is a non-increasing function of time and bounded from below the lim t!1 V (t) = V 1 exists. Therefore by integrating (3.21) from 0 to 1 we have Z 1 0 je x ()j 2 d 1 [V (0)? V 1] < 1 14

17 which implies that e x 2 L 2. By the denition of the Gaussian radial basis function, the regressor vectors (x) and (x) are bounded for all x and by assumption u is also bounded. Hence from (3.10) we have that _e x 2 L 1. Since e x 2 L 2 \ L 1 and _e x 2 L 1, using Barbalat's Lemma [3] we conclude that lim t!1 e x (t) = 0. Now, using the boundedness of (t) and the convergence of e x (t) to zero, we have that _ 1 = W _ 1 also converges to zero. Similarly, _ 2! 0 as t! 1. (b) With the projection algorithm, it is guaranteed that kw 1 k F M 1, kw 2 k F M 2. Therefore the weight estimation errors are also bounded, i.e 1 ; 2 2 L 1. From (3.20) it is clear that if je x j > 0 = then V _ < 0 which implies that e x 2 L 1 and consequently ^x 2 L 1. In order to prove the second part, we proceed to complete the square in (3.20) : _V? 2 je xj 2? je x j et x? 2 je xj jj2 Therefore, by integrating both sides and using the fact that V 2 L 1 we obtain Z t 0 je x ()j 2 d 2 [V (0)? V (t)] + 1 Z t j()j 2 d 2 Z t k 1 + k 2 j()j 2 d where k 1 := 2= V (0)? sup t0 V (t) and k 2 := 1= 2. 2 Remark 3.1: For notational simplicity the above identication scheme was developed with the lter pole and the learning rates 1, 2 being scalars. It can be easily veried that the analysis is still valid if? in (3.9) is replaced by a Hurwitz matrix A 2 R nn and 1, 2 in the parameter update laws are replaced by positive denite learning rate matrices? 1 2 R n 1n 1,? 2 2 R n 2n 2 respectively. Remark 3.2: To ensure robustness of the identication scheme with respect to modelling errors, we have considered a projection algorithm, which prevents the weights from drifting to innity. The stability of the proposed identication scheme in the presence of modelling errors can also be achieved by other modications to the standard adaptive laws, such as the xed, or switching -modication [49, 52], "-modication [53] and the dead-zone [2, 3]. A comprehensive exposition to robust adaptive control theory for linear systems is given in [54]. Remark 3.3: Under the assumptions of Theorem 2 we cannot conclude anything about the convergence of the weights to their optimal values. In order to guarantee convergence, (x), (x)u need to satisfy a persistency of excitation condition. A signal z(t) 2 R n is persistently exciting in R n if there exist positive constants 0, 1, T such that 0 I Z t+t t 0 z()z T ()d 1 I 8t 0 In contrast to linear systems where the persistency of excitation condition has been transformed into a condition on the input signal [2, 3], in nonlinear systems this condition cannot be veried apriori because the regressors are nonlinear functions of the state x. 15 0

18 3.1.2 Optimization Methods In order to derive optimization-based weight adjustment rules that also guarantee stability of the overall scheme, we need to develop an identication model in which the output error e x is related to the weight estimation errors 1, 2 in a simple algebraic fashion. To achieve this objective we consider ltered forms of the regressors, and the state x. We start by rewriting (3.8) in the lter form x = s + [x] + W 1 1 s + [(x)] + W 2 1 s + [(x)u] + 1 [] (3:22) s + where s denotes the dierential (Laplace) operator. 3 As in Section 2, the notation h(s)[z] is to be interpreted as the output of the lter h(s) with z as the input. Equation (3.22) is now expressed in the compact form x = x f + W 1 f + W 2 f + f (3:23) where x f, f, f are generated by ltering x,, u respectively: _x f =?x f + x x f (0) = 0 _ f =? f + f (0) = 0 _ f =? f + u f (0) = 0 Since (t) is bounded by 0, the ltered modelling error f := 1 disturbance signal, i.e j f (t)j 0 =. Based on (3.23), we consider the identication model s+ [] is another bounded ^x = x f + W 1 f + W 2 f (3:24) This model, which will be referred to as the regressor ltering identication model [9], is shown in Figure 6, in block diagram representation. The regressor ltering scheme requires n+n 1 +n 2 rst order lters, which is considerably more than the n lters required in the error ltering scheme. As can be seen from Figure 6, the lters appear inside the RBF networks, forming a dynamic RBF network. This neural network architecture can be directly applied to modelling of dynamical systems in the same way that RBF networks are used as models of static mappings. Using the regressor ltering scheme, the output error e x := ^x? x satises e x = W 1 f + W 2 f + x f? x = 1 f + 2 f? f (3:25) where 1 = W 1?W 1, 2 = W 2?W 2 are the weight estimation errors. Hence by ltering the regressors, we obtain an algebraic relationship between the output error e x and the weight 3 In deriving (3.22) we have assumed (without loss of generality) zero initial condition for the state, i.e x 0 = 0. Note that if x 0 6= 0 then the initial condition will appear in the identication model so that it gets cancelled in the error equation. This is possible because x is available for measurement. 16

19 Figure 6: A block diagram representation of the regressor ltering identication model developed using RBF networks. estimation errors 1, 2. In the framework of the optimization approach [55], adaptive laws for W 1, W 2 are obtained by minimizing an appropriate cost functional with respect to each element of W 1, W 2. Here we consider an instantaneous cost functional with a constraint on the possible values that W 1, W 2 can take. This leads to the following constrained minimization problem: minimize J (W 1 ; W 2 ) = 1 2 et x e x subject to kw 1 k F M 1 (3:26) kw 2 k F M 2 Using the gradient projection method [55], we obtain the following adaptive laws for continuous adjustment of the weights W 1, W 2 : (?1 _W 1 = n e x f T o if fkw 1 k F < M 1 g or fkw α x 1 k F = M 1 and e T x W 1 f 0g P? 1 e x f T if fkw 1 k F = M 1 and f e s + α T (3.27) x W 1 f < 0g (?2 e x f _W T if fkw 2 k F < M 2 g or fkw 2 k F = M 2 and e T x 2 = W 2 f 0g n o P? 2 e x f T if fkw 2 k F = M 2 and e T (3.28) x W 2 f < 0g Identification Model Dynamic RBF-Network where Pfg denotes the projection operation dened in Section The weight adaptive laws (3.27), (3.28) have the ξ (x) same form as (3.15), 1 (3.16), ξ f which were derived by the Lyapunov method, with the ξ exception that the output s + error α e x and the regressor W 1 vectors, are dened in a dierent way. The counterpart of Theorem 2 concerning the stability properties of the regressor ltering identication scheme with the adaptive laws (3.27), (3.28), obtained using the gradient projection method is described by the following result. Dynamic RBF-Network ζ ζ (x) 17 1 s + α ζ f W 2 ^x

20 Theorem 3 Consider the regressor ltering identication scheme described by (3.24). The weight adaptive laws given by (3.27), (3.28) guarantee the following properties: (a) For (t) = 0 (no modelling error), we have e x ; ^x; 1 ; 2 2 L 1, e x 2 L 2. lim t!1 e x (t) = 0, lim t!1 _ 1 (t) = 0, lim t!1 _ 2 (t) = 0. (b) For sup t0 j(t)j 0, we have e x ; ^x; 1 ; 2 2 L 1. there exist constants k 1, k 2 such that Z t 0 Z t je x ()j 2 d k 1 + k 2 j()j 2 d 0 Proof: Consider the Lyapunov function candidate V ( 1 ; 2 ) = k 1 k 2 F k 2 k 2 F 1 = tr 1 T T (3:29) Using (3.25), Lemma 1 and the fact that _ 1 = _W 1, _ 2 = _W 2, the time derivative of V along (3.27), (3.28) can be expressed as _V = tr n o?e x f T T? e 1 xf T T 2 + I 1 o =?tr?tr ne x (a) If (t) = 0 then f T T 1 + f T T 2 o ne x e T x + T f e T x W 1 f kw 1 k 2 F tr n W 1 T 1 o + I 2 e T x W 2 f kw 2 k 2 F tr n W 2 T 2 =?je x j 2? T f e x?je x j je xj (3.30) _V?je x j 2 0 (3:31) Therefore V 2 L 1, which from (3.29) implies that 1 ; 2 2 L 1. Using this, together with the boundedness of f, f in (3.25) gives e x ; ^x 2 L 1. Furthermore, by integrating both sides of (3.31) from 0 to 1 it can be shown that e x 2 L 2. Now by taking the time derivative of e x in (3.25) we obtain _e x = _ 1 f + 1 _ f + _ 2 f + 2 _ f (3:32) Since _ 1 ; f ; 1 ; _ f ; _ 2 ; f ; 2 ; _ f 2 L 1, (3.32) implies that _e x 2 L 1 and thus using Barbalat's Lemma we conclude that lim t!1 e x (t) = 0. Using the boundedness of f and f, it can be readily veried that _ 1 and _ 2 also converge to zero. (b) Suppose sup t0 j(t)j 0. The projection algorithm guarantees that W 1, W 2 are bounded, which implies 1 ; 2 2 L 1. Since f, f are also bounded, from (3.25) we obtain e x 2 L 1 and also ^x 2 L 1. The proof of the second part follows directly along the same lines as its counterpart in Theorem o

21 Remark 3.4: As in the Lyapunov method, the scalar learning rates 1, 2 in the parameter update laws (3.27), (3.28) can be replaced by positive denite matrices? 1,? 2. Remark 3.5: Minimization of an appropriate integral cost using Newton's method yields the recursive least-squares algorithm. In the least-squares algorithm the learning rate matrices? 1,? 2 are time varying and are adjusted concurrently with the weights. Unfortunately, the least-squares algorithm is computationally very expensive and especially in neural network modelling, where the number of units n 1, n 2 are usually large, updating the matrices? 1,? 2, which consist of n 2, 1 n2 2 entries respectively, makes this algorithm impractical. 3.2 Multilayer Network Models In this section we consider the case where the network structures employed in the approximation of f(x) and g(x) are multilayer neural networks with sigmoidal-type of activation functions. Although this is the most commonly used class of neural network models in empirical studies, there are very few analytical results concerning the stability properties of such networks in problems dealing with learning in dynamic enviroments. The main diculty in analyzing the behavior of recurrent network architectures with feedforward multilayer neural networks as subsystems arises due to the fact that the adjustable weights appear non-anely with respect to the nonlinearities of the network structure. Our approach relies on developing an error identication scheme, based on the same procedure as in Section The analysis proceeds through the use of a Taylor series expansion around the optimal weights. The adaptive law is designed based on the rst order (linear) approximation of the Taylor series expansion. In this framework, the RBF error ltering scheme presented earlier, constitutes a special case of the analysis in this section, with the higher order terms not present and the regressor being independent of the weight values. As a consequence of the presence of higher order terms, the results obtained here are weaker, in the sense that, even if there is no modelling error it cannot be guaranteed that the output error will converge to zero. The signicance of this analysis is based on proving that all the signals in the proposed identication scheme remain bounded, as well as developing a unied approach to synthesizing and analyzing stable dynamic learning congurations using dierent types of neural network architectures. Consider the system (3.5), where f, g are the optimal weights in the minimization of jf(x)? ^f(x; f )j and jg(x)? ^g(x; g )j respectively, for x 2 X and subject to the constraints jf j M f, jgj M g, where M f, M g are (large) design constants. The functions ^f and ^g are the outputs of multilayer neural networks with sigmoidal nonlinearities in-between layers. We start by adding and subtracting x in (3.5), where is a positive design constant. This gives _x =?x + x + ^f(x; f ) + ^g(x; g )u + (3:33) Based on (3.33) we consider the error ltering identication model _^x =?^x + x + ^f(x; f ) + ^g(x; g )u (3:34) 19

22 Figure 7: A block diagram representation of the error ltering identication model developed using multilayer sigmoidal neural networks. where f 2 R n f, g 2 R ng are the estimates of the optimal weights f, g respectively. The constants n f, n g are the number of weights in each approximation. The error ltering identication scheme described by (3.34) is shown in block diagram representation in Figure 7. From (3.33), (3.34), the output error e x = ^x? x satises the dierential equation: h _e x =?e x + ^f(x; f )? ^f h i (x; f)i + ^g(x; g )? ^g(x; g) u? (3:35) Identification Model In order to obtain an adaptive law for the weights f it is convenient to consider the rst order approximation of the dierence ^f(x; f )? ^f(x; f ). Using the Taylor series expansion4 of ^f(x; f ) around the point (x; f ) we obtain α ^f(x; f )? ^f(x; f) f (x; f ) f? f + ^f 0 (x; f ) (3:36) x ^ f ^ f (x, θ f ) where ^f 0 (x; f ) represents the higher order terms (with respect to f ) of the expansion. If we dene the weight Multilayer estimationnetwork error as f := f? f then from (3.36) we have that ^f 0 (x; f ) = ^f(x; f )? ^f(x; ^f (x; f ) f 1 s + α ^x 4 Throughout the analysis we require that the network outputs ^f(x; f ), ^g(x; g) are smooth functions of their arguments. This can easily be achieved if the sigmoid used is a smooth function. The logistic function Multilayer Network g^ g ^ (x, θ g ) and the hyperbolic tangent are examples of popular sigmoids that also satisfy the smoothness condition. 20 u