Hopfield Neural Networks for Parametric Identification of Dynamical Systems

Transcription

1 Neural Processing Letters (2005) 21: Springer 2005 DOI /s Hopfield Neural Networks for Parametric Identification of Dynamical Systems MIGUEL ATENCIA 1, GONZALO JOYA 2 and FRANCISCO SANDOVAL 2 1 Departamento de Matemática Aplicada. E.T.S.I. Informática 2 Departamento de Tecnología Electrónica. E.T.S.I. Telecomunicación Universidad de Málaga. Campus de Teatinos, Málaga, Spain. matencia@ctima.uma.es Abstract. In this work, a novel method, based upon Hopfield neural networks, is proposed for parameter estimation, in the context of system identification. The equation of the neural estimator stems from the applicability of Hopfield networks to optimization problems, but the weights and the biases of the resulting network are time-varying, since the target function also varies with time. Hence the stability of the method cannot be taken for granted. In order to compare the novel technique and the classical gradient method, simulations have been carried out for a linearly parameterized system, and results show that the Hopfield network is more efficient than the gradient estimator, obtaining lower error and less oscillations. Thus the neural method is validated as an on-line estimator of the time-varying parameters appearing in the model of a nonlinear physical system. Key words. system identification, Hopfield neural networks, parameter estimation, adaptive control, optimization 1. Introduction System identification can be defined as the characterization of a dynamical system, by observing its measurable behaviour. Identification has been studied since decades [12, 20] from a variety of viewpoints and research communities, such as statistical regression estimation, signal processing filtering, and control engineering adaptive control. When a priori information about the rules that govern a system either do not exist or are too complex, e.g. in electrocardiographic signals, identification techniques are used to build a model by only observing input output data. Then, the system is called a black box since the internal behaviour is unknown. On the contrary, in many physical systems, our knowledge of mechanical, chemical or electrical laws permit us to formulate a model, which is the fundamental tool for studying the system, either analytically or through simulations. However, no matter how deep our physical insight, the parameters of any model present inaccuracies. For instance, measurement devices involve some error, transmission systems produce noise, a robot can take a stone with unknown mass or the friction coefficient of some material can only be approximately known. Due to these perturbations,

2 144 M. ATENCIA ET AL. although the model is still valid, the numerical value of its parameters must be confronted with observed data, and the system is called a gray box since identification must be applied to validate or adapt the a priori formulated model, which is referred to as a parametric model [17]. This paper is focused in on-line identification for gray-box models, i.e. continuously obtaining an estimation of the system parameters. This subject has received increased attention after the successful development of adaptive control methods [13] that consist of an identification module and a controller that would achieve optimal control, should the actual system match the identified one. The mathematical formulation of a dynamical system is a possibly vectorial ordinary differential equation (ODE) ẋ = f(x,u,π) where x(t) is the state vector, u(t) is the input vector, which can be regarded as a control signal, and the parameter vector π(t) may be uncertain and/or time-varying. Notice that, along the paper, we adhere to the notation ẋ for the time derivative. From this formulation, gray box identification is achieved by parameter estimation, i.e. by calculating at each time instant a value ˆπ that approaches the actual value π as much as possible, thus minimizing the estimation error π = ˆπ π. Therefore, the continuous estimation evolves together with the observation of the dynamical system, so the estimation is a dynamical system itself given by an ODE ˆπ = g( ˆπ,x,u). In the above paragraph, we have formulated identification as minimization of the estimation error π. In doing so, we emphasize that the appropriate framework for identification is optimization. Indeed, most proposed techniques for identification are somehow based on optimization methods. On one hand, the simplest method for on-line identification is the gradient method, which is supported by the same rationale as gradient descent optimization: the estimation should evolve in the direction that best minimizes the error, which is the negative gradient of the error function. However, the actual parameter value π is unknown and so it is the estimation error π. Thus, the norm of the output prediction error e = f(x,u, ˆπ) ẋ, or equivalently its square e 2, is instead used as the target function, and the gradient method leads to the estimator dynamics ˆπ = k e( ˆπ) 2. The tuning of the estimation gain k is a key aspect of the estimator design and must be repeated for each particular system. If k is small, the convergence to the actual parameter is slow. If k is too large, oscillations may appear that cause not only larger error but also high frequency noise, which may excite unmodelled dynamics if the estimation is a part of a closed loop controller. On the other hand, several methods have been proposed for batch estimation, i.e. for identification that is performed only once, after the observation of the complete evolution of the system. Most classical batch estimators are based upon minimization of least square error, while heuristic techniques have recently been proposed based upon genetic algorithms [15], which are known to be global optimizers. It is remarkable that most identification techniques have been applied to the simpler linear in the parameters (LIP) systems, with the form f(x,u,π)= A(x,u)π, while identification

3 HOPFIELD NETWORKS FOR PARAMETRIC IDENTIFICATION 145 of nonlinearly parameterized systems is still almost unexplored [14]. In particular, most robotic systems can be cast into the LIP form [18]. In this contribution, we apply the optimization methodology of Tank and Hopfield [19] to on-line identification, by designing a Hopfield neural network whose Lyapunov function is made coincident with the prediction error, so that the network evolution approaches a minimum of the error. However, the prediction error is time-varying, hence the above procedure results in time-varying weights and biases. Therefore, in contrast to standard Hopfield networks, stability cannot be taken for granted, since the neural estimator is a non-autonomous dynamical system [21], thus requiring a specific stability analysis that will be published elsewhere. The Hopfield estimator has already been applied to an epidemiological problem [2], which happens to be a difficult task. In contrast, the method seems to be especially well suited to robotic systems. For this class of models, when compared to the gradient method, the proposed technique is shown to alleviate the problematic tuning of gain, achieving fast convergence of estimations to the actual values of parameters, as well as less oscillations. In contrast to batch estimators, the ability of the Hopfield estimator to track time varying parameters is shown. The only requisite of the proposed methods is the knowledge of a bounded region where the actual values of parameters remain. The organization of this paper is as follows: The well-known gradient method is briefly reviewed in Section 2, together with its usual application to linear in the parameters systems. In Section 3, after describing the application of Hopfield neural networks to optimization, parameter estimation with Hopfield networks is presented. The efficiency of the method is assessed by simulations in Section 4. Finally, conclusions are summarized in Section Gradient Estimation In gray box identification, a system model is known but it includes some unknown, uncertain and/or time-varying parameters. In the simplest and most usual case, the model is in the LIP form ẋ = A (x,u) π. A remark on notation is noticeable: the formulation will be considerably simplified if we consider that the parameter vector π = θ n + θ comprises an expected or nominal known part θ n, as well as the deviation from the nominal θ. The estimation method is designed to provide an estimation ˆθ of the deviation. Thus, we rewrite the system model as: y =ẋ = A(x(t), u(t))(θ n + θ(t)) (1) where y is the output, θ is the unknown possibly time-dependant deviation from the nominal values and A is a matrix that depends on the input u and the state x. Both y and A are assumed to be physically measurable. Identification is accomplished by producing an estimation ˆθ(t) that is intended to continuously minimize the estimation error θ = ˆθ θ. An additional important objective

4 146 M. ATENCIA ET AL. is to reduce the output prediction error that, in the linear in the parameters form, reduces to e = A(θ n + ˆθ) y = A θ. Equivalently, the square of the norm of the prediction error can be chosen as the target function: e 2 = e e = (A θ) (A θ)= θ A A θ (2) A common technique for on-line estimation is the gradient method that, due to its simplicity, presents some advantages over least mean squares algorithms in estimation of time-varying parameters. In the gradient method, the estimation is continuously modified in the direction that best minimizes the prediction error, i.e. the direction of the gradient of e 2 as a function of the estimation vector ˆθ, thus leading to the dynamical equation of the estimator: ˆθ = k (e e)= 2kA e = 2kA [A(θ n + ˆθ) y] = ( 2kA A) ˆθ 2kA (Aθ n y) (3) where the fact e/ ˆθ = e/ θ = A has been used, and k is a design parameter, which must be critically chosen as a trade-off between small values slow convergence and large values oscillations. The last equality emphasizes the fact that the gradient method leads to a linear differential equation, when applied to an LIP system. The asymptotical convergence to zero of the prediction error can be proved if A and y are constant, but the result is also valid as long as A and y change slowly [17], which we assume in the sequel. However, the estimation error may be large even though the prediction error vanishes. This is due to the possible existence of several estimation values that result in zero prediction error, apart from the obvious actual value ˆθ =θ, whereas if the input signal is rich enough then A and y are so complex that the only value of ˆθ that leads to ˆθ = 0 is the actual parameter. This richness of the input signal, called persistent excitation in the control literature [17], occurs when the input signal contains a sufficient number of harmonics, and it will also be assumed. 3. Estimation with Hopfield and Tank Networks The neural network paradigm comprises a variety of computational models, among which Hopfield networks [8, 9] are feedback systems for which a Lyapunov function has been found. This feature guarantees the stability of the network, which can then be useful for the solution of two important problems: associative memory and optimization. In the Abe formulation [1], the evolution of a continuous Hopfield network is governed by the following system of differential equations: µ = net(s); s = tanh(µ/β) (4)

5 HOPFIELD NETWORKS FOR PARAMETRIC IDENTIFICATION 147 where β is a parameter that controls the slope of the hyperbolic tangent function and the net term is a linear function of the state vector: n net i = w ij s i s j I i (5) j=1 The computational ability of Hopfield networks stems from the fact that they are stable dynamical systems, which is proved by the existence of a Lyapunov function: V = 1 2 n n n w ij s i s j + I i s i (6) i=1 j=1 i=1 A significant advantage of the Abe formulation is the multinomial form of the Lyapunov function, as shown in equation (6). In particular, the Lyapunov function has the same structure of the multilinear target function of combinatorial optimization problems. This is in contrast to the original formulation by Hopfield [9] and subsequent generalizations [4], where the presence of an integral term introduces additional difficulties. Note, however, that when neurons are updated synchronously at discrete time intervals, the Lyapunov function is identical to that of the Abe formulation [5,6], see also [7] and references therein. In the sequel, we stick to the Abe formulation in continuous time. For this system, the fact that the function V of equation (6) is indeed a Lyapunov function stems from the identity V / s i = net i, which leads to V = n i=1 V s i ṡ i = n i=1 V s i ds i dµ i µ i = n i=1 1 β (net i) 2 tanh (µ i /β) 0 (7) and the other conditions for V being a Lyapunov function are trivially satisfied (see [10] for a discussion on Hopfield dynamics). The parameters w and I are called weights and biases, respectively. A remarkable property of Hopfield dynamics, is that states remain in a bounded region, namely the hypercube s i [ 1, 1], due to the presence of the bounded function tanh. Therefore, the intended solution should belong to this hypercube, otherwise it will never be attained. The application of the Hopfield model to optimization is a consequence of its stability: since the network seeks a minimum of its Lyapunov function, it can be regarded as a minimization method, as long as the target function can be identified with the Lyapunov function. The parameters w, I are then obtained from the numeric values in the target function [19]. In order to perform system identification, the squared norm of the prediction error, as defined in equation (2), is chosen as the target function. However, since A and y are time-varying, the known stability proofs are no longer valid, hence the existence of a Lyapunov function cannot be taken for granted. Assume that time is frozen so that A c =A(x(t), u(t))

6 148 M. ATENCIA ET AL. is a constant matrix and y c = y(t) is a constant vector and consider the Lyapunov function candidate V = 1/2 e e that, after some algebra, results in: V = 1 2 ˆθ A c A c ˆθ + ˆθ (A c A cθ n A c y c) + V 1 (8) where V 1 does not depend on ˆθ so it can be neglected without changing the position of the minima of V. Therefore, this equation is structurally identical to the standard Lyapunov function of a Hopfield network equation (6) whose states represent the estimation, as long as the weight matrix and the bias vector are appropriately defined: W = A c A c; I = A c A c θ n A c y c (9) Therefore, under the assumption of A and y being constant, the Abe formulation of the Hopfield model provides an estimation that minimizes the prediction error. In the next section, simulation results are presented to show that the assumption that A and y are constants may be relaxed and the prediction error produced by the resulting dynamical system is still reduced. With regard to the exact stability analysis, it has been shown the stability of slowly varying dynamical systems [21], but this result is not directly applicable to the neural estimator, since it first requires proving the exponential stability of every frozen system. Instead, the neural network can be analysed by means of the Barbalat s lemma [17], that guarantees the stability of a system if a Lyapunov function is found, whose temporal derivative is uniformly continuous. Further difficulties arise when the parameter vector θ is time-varying, since then only boundedness of the solution can be proved [11]. The technical details of the stability analysis will be published in a separate paper [3]. It is remarkable that the only difference between the obtained Hopfield dynamics and the already known gradient method given by equation (3) is the presence of the nonlinear sigmoid-like function tanh. A requirement for the application of this methodology is the knowledge of a bounded region around θ n, because the network states can not abandon the hypercube s i [ 1, 1]. However, this region is not required to coincide with the hypercube, since it could be mapped into the hypercube by an appropriate scaling. The proposed method is simulated and its performance is compared to the gradient method in the following section. 4. Simulation Results We will show the efficiency of the proposed method by simulating an idealized single link manipulator (Figure 1), modelled by the following equation: y =ẍ = g l sin x v ml 2 ẋ + 1 ml 2 u (10)

7 HOPFIELD NETWORKS FOR PARAMETRIC IDENTIFICATION 149 Figure 1. Single link manipulator. where x is the angle between the manipulator and the vertical, ẋ is its angular velocity, g is the gravity, l is the manipulator arm length, v is the friction coefficient, m is the mass that is being transported at the arm extreme and u is the torque produced by a motor, which can be regarded as the input or control signal. If the gravity is assumed constant, the physical parameters are l, v and m. In order to cast the model into the linear in the parameters form, we define the formal parameter vector π = (θ n + θ) = ( g/l, v/ml 2, 1/ml 2 ). Then, the matrix A can be defined as A = (sin x,ẋ,u) so that equation (1) holds. The control signal u is defined as a sum of three sinusoids displaced according to the Schroeder phase [12], to achieve persistent excitation without excessive amplitude. The setting of the simulated experiment is as follows: At the beginning, the deviation of the actual parameters from the nominal values is θ(t = 0) = ( ). Since the initial estimation vector ˆθ is null, the estimation error at t = 0 is considerable, in order to determine if the methods are able to converge to the correct values or they get stuck into wrong estimations. Besides, the system is subject to a variable load, the transported mass abruptly changing at t = 15 s. The gains k = 10 and k = 100 have been chosen for the gradient estimator, while no special attention has been paid to the selection of an optimal value for the parameter β of the Hopfield network. The performance of the Hopfield network is graphically compared to the gradient method. In particular, the error in the first component θ 1 is plotted in Figure 2, where we observe a short transient during which all three estimators oscillate. Then, the Hopfield network suddenly converges to the actual value, the gradient estimator with k=100 oscillates wildly and the gradient with k = 10 also converges, but its convergence is so slow that it has not reached the actual value when the parameter change at t = 15 s. occurs. After the load change, both gradient estimations are distorted, while the network response is practically unaffected. It is clear the problematic trade-off of the gradient method between oscillatory answer (k = 100) and slow convergence (k = 10), while the neural network converges quickly and its oscillations are appropriately damped. We also compare the integral of the prediction error of the Hopfield estimator to that of the gradient method in Figure 3. The Hopfield network and the gradient with k = 100 both produce an accurate prediction, but the gradient with k = 10 accumulates significant error, mainly after the load change at t = 15 s.

8 150 M. ATENCIA ET AL Gradient (k=10) Gradient (k=100) Hopfield Figure 2. Estimation error for θ Gradient (k=10) Gradient (k=100) Hopfield Figure 3. Integral of prediction error. 5. Conclusions In this work, the optimization methodology of Hopfield and Tank is applied to parameter estimation, in the context of system identification. When restricting to linear in the parameters systems, simulation results show that this technique presents faster convergence as well as oscillation reduction in contrast to the known gradient method. Furthermore, no parameter exists that must be adjusted ad hoc for each problem. The main requirement for the application of the method is that the parameter values must belong to a known compact interval, which is a mild assumption in robotic systems, where the physical meaning of parameters provide hints of their values. The positive aspect is that there is no need

9 HOPFIELD NETWORKS FOR PARAMETRIC IDENTIFICATION 151 for complicated algorithms, such as normalization or projection, in order to keep the estimations bounded. Simulations have been performed for a simplified example and the proposed method outperforms two instances with different gain choices of the known gradient method. Currently, we are involved in deeper research of the neural estimator, developing several aspects, such as: The application of the neural estimator to more realistic models, such as robots with flexible links. The theoretical study of the convergence and stability of the Hopfield estimator. In the case of sudden parameter changes, the simulations here presented suggest that convergence to the exact parameters is feasible. However, when parameters vary continuously with time, only the convergence of the estimation error towards a bounded neighbourhood can be proved. The extension of the method to non-lip systems, by means of higher-order Hopfield networks [16, 10]. In this case, a general nonlinear function must be approximated, e.g. by its Taylor polynomial, so that it matches the Lyapunov function of higher-order Hopfield networks, which is multilinear. The inclusion of the estimator into an adaptive control system and the stability of the resulting closed-loop system. To summarize, the estimation method based upon Hopfield networks presents favourable properties: it performs on-line estimation and it is able to deal with time-varying parameters. In particular, the neural estimator is well suited to systems in the field of robotics, where physical models are available, and inputs and states can be accurately measured. Acknowledgments This work has been partially supported by the Spanish Ministerio de Ciencia y Tecnología (MCYT), Project No. TIC The careful reading and useful suggestions of the reviewers are gratefully acknowledged. References 1. Abe, S.: Theories on the Hopfield Neural Networks. In: Proceedings IEE International Joint Conference on Neural Networks, Vol. I. (1989) pp Atencia, M. A., Joya, G. and Sandoval, F.: Modelling the HIV-AIDS Cuban Epidemics with Hopfield Neural Networks. In: J. Mira and J. R. Álvarez (eds.): Artificial Neural Nets. Problem Solving Methods. Lecture Notes on Computer Science (IWANN 2003), Vol (2003) pp Atencia, M. A., Joya, G. and Sandoval, F.: Parametric identification of robotic systems with stable time-varying Hopfield networks, Neural Computing & Applications, 13 (2004), In press.

10 152 M. ATENCIA ET AL. 4. Cohen, M. and Grossberg, S.: Absolute stability of global pattern formation and parallel memory storage by competitive neural networks, IEEE Transaction on Systems, Man and Cybernetics, 13(5) (1983), Golden, R. M.: The brain-state-in-a-box neural model is a gradient descent algorithm, Journal of Mathematical Psychology, 30(1) (1986), Golden, R. M.: Stability and optimization analyses of the generalized Brain-State-in-a- Box neural network model, Journal of Mathematical Psychology 37(2) (1993), Golden, R. M.: Mathematical Methods for Neural Network Analysis and Design. Cambridge, MA: MIT Press, Hopfield, J.: Neural Networks and physical systems with emergent collective computational abilities, Proceedings of the National Academy of Scinces USA, 79 (1982), Hopfield, J.: Neurons with graded response have collective computational properties like those of two-state neurons, Proceedings of the National Academy of Sciences USA, 81 (1984), Joya, G., Atencia, M. A. and Sandoval, F.: Hopfield neural networks for optimization: study of the different dynamics, Neurocomputing, 43(1 4) (2002), Khalil, H. K.: Nonlinear Systems. New York: Prentice Hall, Ljung, L.: System Identification. Theory for the User. New York: Prentice Hall, Marino, R.: Adaptive control of nonlinear systems: basic results and applications, Annual Reviews in Control, 21 (1997), Marino, R. and Tomei, P.: Global adaptive output-feedback control of nonlinear systems, Part II: nonlinear parameterization, IEEE Transaction on Automatic Control, 38(1) (1993), Pedroso-Rodríguez, L. M., Marrero, A. and Arazoza, H.: Nonlinear parametric model identification using genetic algorithms. In: J. Mira and J. R. Álvarez (eds.): Artificial Neural Nets. Problem Solving Methods. Lecture Notes on Computer Science (IWANN 2003), Vol. 2687, (2003), Samad, T. and Harper, P.: High-order Hopfield and Tank optimization networks, Parallel Computing, 16 (1990), Slotine, J.-J. and Li, W.: Applied Nonlinear Control. New York: Prentice Hall, Spong, M. W. and Vidyasagar, M.: Robot Dynamics and Control. New York: John Wiley & Sons, Tank, D. and Hopfield, J.: Neural computation of decisions in optimization problems, Biological Cybernetics, 52 (1985), Unbehauen, H.: 1996, Some new trends in identification and modeling of nonlinear dynamical systems, Applied Mathematics and Computation, 78 (1985), Vidyasagar, M.: Nonlinear Systems Analysis. New York: Prentice Hall, 1993.