Intelligent Modular Neural Network for Dynamic System Parameter Estimation

Transcription

1 Intelligent Modular Neural Network for Dynamic System Parameter Estimation Andrzej Materka Technical University of Lodz, Institute of Electronics Stefanowskiego 18, Lodz, Poland Abstract: A technique for dynamic system parameter estimation by using artificialneural-network-like approximators is developed. The technique offers very high speed and high noise immunity. However, in cases where the number of unknown parameters and the estimation accuracy level required are high, the neural network needed to perform the task is rather complex. This causes the training process prohibitevely time-consuming and its results unpredictable. A modular, classifier-approximator neural network architecture is proposed to overcome these deficiences. Results of computer simulation are discussed. INTRODUCTION Parametric modelling is one of the most widely used techniques to represent dynamic systems of interest in engineering and science. It is advantageous both from the data reduction and accuracy improvement points of view [1]. The actual values of the parameters can be used for system diagnosis, eg. in electronic circuit testing [2] or medical investigation [3]. The real problem is to find an efficient parameter estimation methodology given system response to a predetermined stimulus. Least-square- (LSE) modelling is the technique which is most often used for the task [3], [4]. However, it involves iterative solving of a set of nonlinear equations. This takes a substantial amount of computation time and thus makes the LSE technique not suitable for real-time parameter tracking. Moreover, the LSE model tuning can become prematurely terminated due to existence of local minima of respective function. It has been proposed in [5] and further developed in [6]-[8] that feedforward Artificial Neural Networks (ANNs) can be used to perform the approximation of the mapping from the system observation space to the parameter space, provided the ANN has been properly trained. The training process can last long; however, it has to be carried out once only - for a given system and its parameter range. In the recall mode, the ANN does not perform any iterative calculations. Its response to an observation vector is fast, especially should the ANN be hardware-implemented. Depending on the complexity of the System Under Test (SUT) equations, the speedup over the LSE method can be of the order of [7], [8] (on a PC machine). Moreover, it has been proven analytically and confirmed experimentally [8] that the noise-induced of parameter estimation can be reduced compared with LSE approach, provided the ANN has been trained on noisy observations. This property makes the ANN-based method similar to the total-least-squares technique [9]. The ANN-based technique for system parameter estimation seems promising. A significant advance is expected of its application to instrumentation technology, both in terms of higher speed and increased noise immunity it offers. Current work is focused on the optimisation of the ANN architecture for the task.

2 NUMERICAL EXAMPLE Suppose the SUT is an electronic circuit shown in Fig. 1 [7], excited by a negative voltage step: x(t)=, t<, and x(t)=-1, t. The observation noise is neglected, for simplicity. (More complex cases of noisy, nonlinear systems are discussed in detail in [5]-[8].) The response of the circuit can be described as t t y( t) 3 exp exp (1) where 1 1. R 1 C 1, 2 R 2 C 2 and 3 R 1 C 2 / ( R 1 C 1 R 2 C 2 ) are 3 unknown parameters. The response (1) is quite ubiquituous, eg. it may represent biological, multicompartmental systems [3]. It is assumed in this example that the parameters take their values from a cuboid as follows , , (2) A set of example circuit responses calculated for a number of parameter values is shown in Fig. 2. At least 3 observations have to be measured to find 3 unknown parameters. Based on optimal experiment design theory [1], the observation moments were fixed at t ms, t ms and t ms. They give the following observation vector y ( y, y, y )' [ y( t ), y( t ), y( t )]' (3) In the ANN-based approach to parameter estimation, the observation vector y forms an input to a feedforward ANN which produces parameter values at its output [5]-[8]. As an example, for a popular multilayer perceptron (MLP) with sigmoidal neurons nonlinearity [11], the estimate 1 of parameter 1 is obtained as (4) m n 1 u u jv j v ji yi g1( w, y) j 1 i 1 where w=(u,,u m,v j,..,v jn) =(w 1,..,w q) is a weight (connection) vector, m is the number of neurons in the MLP hidden layer, (.) is a sigmoidal single-variable function, and n is the number of ANN inputs (n=3 here). The weights are learned in the course of the training [11] which is basically a numerical minimisation of an function between actual and estimated parameters, over a set of examples [6], [8]. x (t) R2 C2 C1 R1 y (t) Fig. 1 System under test of 3 unknown parameters y(t) 1,8,6,4,2,,5 1, 1,5 2, 2,5 3, 3,5 4, 4,5 5, 5,5 6, 6,5 7, 7,5 8, time, ms 8,5 9, 9,5 1, 1,5 11, 11,5 12, 12,5 13, nom t1- t1+ t2- t2+ t3- t3+ Fig. 2 Circuit step responses for 7 different parameter vectors

3 In general, the larger the number of neurons m, the lower the achievable norm of estimation, k=1,..,n. However, as the decreases linearly, the time of training increases exponentially with m. This is illustrated by Figs 3 and 4, respectively, where minimum, average and maximum values are plotted, obtained from 3 independent training sessions for each m- neuron-mlp. Due to complicated surface and the local minima effect, the result of the training is much dependent on the starting point, so it is unpredictable in general (Fig. 3). The actual mapping approximated by the ANN in the case of the circuit of Fig. 1 is plotted on Fig. 5, for parameter 1. Projections of the observation domain on subspaces (y 1,y 2 ), (y 1,y 3 ) and (y 2,y 3 ) are shown on Fig. 6, to better illustrate the problem at hand. Maximum absolute testing k k y1 1(y1,y3); y2= (y2,y3); y1=.55 y Number of neurons min ave max Fig. 3 Estimation for parameter 1 using an MLP artificial neural network seconds 2 1 Training time Number of neurons min ave max Fig. 4 Training time of the MLP, as a function of m y Fig. 5 Surface plots representing the mapping from the observation space to parameter 1 space for the circuit of Fig. 1 Plots similar to those presented on Figs. 3-5 were obtained for the other 2 parameters of the circuit of Fig. 1. To further complicate the problem, an unknown response time delay was added to (1) to increase the number of parameters to n=4 [7]. Again, all the conlusions drawn based on the 3- parameter case have been confirmed. y3

4 Fig. 6 Projections of the observation domain of the circuit of Fig. 1 on the two-dimensional observation subspaces MODULAR ANN ARCHITECTURE In the proposed modular architecture, the system observation domain is tesselated into a number (p) of non-overlapping regions. A classifier module first takes the decision so as to which region a given observation vector belongs to. An approximator module is then invoked, whose architecture and weights are pre-optimised locally for high accuracy in the region. This is an intelligent approach human operator would follow in a similar situation. The mapping of interest (from the observation space to the parameter space) is less complex within each of the regions, related to its complexity over the whole domain. This reduces the required complexity of the approximator, simplifies the training process, makes it controllable and gives big savings in the time needed for the training. A minimum-distance classifier has been trained in this study to tesselate the observation domain into p non-overlapping regions. Projections of these regions on the two-dimensional subspaces, for p=8, are presented on Fig. 7. A Learning Vector Quantization procedure [11] was used for the classifier training in this example. A number of numerical experiments was carried out to train p=1, 5, 1, 15, and 2 low-complexity approximators. Two approximator structures were given a trial, an MLP with m=2 sigmoidal neurons (q=11) and a Rational Function (RF) neural network [12] with q=13. Similar results were obtained for the two structures; those for the RF network are summarised in Table 1 whose second column contains sum of times needed to train all p approximators of a given network on a PC 486/1 computer. The training was performed by means of a multivariable function minimisation program using a Chebyshev norm. Fig. 7 Projection of p=8 classification regions on the two-dimensional observation subspaces

5 Table 1 Performance of the modular architecture p Training time, s Max abs. RMS The estimation (tested over the whole parameter space) decreases with the number of classification regions p, whereas, interestingly, the total training time tends to decrease with p. In contrast to the global approximation (Figs. 3 and 4), the reduction of approximation in locally optimised modular architecture is not accompanied by the increase of the training time. The minimum value of the maximum absolute obtained with the MLP of m=5 neurons (q=26) equals to.96 (Fig. 3). This is the best result obtained after 3 training sessions that all consumed 12,369 s of time. On the contrary, better performance (max of.83) was obtained with the modular network of p=1 regions, whose training took 181 s only. The [negligible] price is an increase of memory size needed to store the weights, which is q=26 for the MLP and pq=1 13=13 for the modular network, in this example. Short overhead is also attributed to the time needed for the observation vector classification. CONCLUSION A new modular, classifier-approximator ANN architecture has been proposed for accurate, fast estimation of dynamic system parameters. It was demonstrated, using results of computer simulation, that the proposed architecture allows parameter reduction without an increase of the time needed for the ANN training, and makes the training process controllable. A significant advance to instrumentation technology is expected from applying the ANN-based parameter estimation technique to practice. Searching for efficient means of hardware implementation of the modular ANN is one of the current reserach topics. Acknowledgment: Support from Polish Scientific Research Committee Grant 8T11F 11 is much appreciated. REFERENCES 1. A. van den Boss, Measurement - the parametric approach, IEEE Conf. Meas. Instrum, Atlanta, GA, 1991, pp J. Bandler, A. Salama, Fault diagnosis of analog circuits, Proc. IEEE, 37, 1985, pp A. Bahill, Bio-engineering, Prentice -Hall, J. Cadzow, Signal processing via least-squares modeling, IEEE ASSP Magazine, Oct. 199, pp A. Materka, Application of neural networks for dynamic system parameter estimation, 14 th Int. Conf. IEEE Eng. Med. Biol. Soc., Paris, 1992, pp A. Materka, Application of Artificial Neural Networks to Parameter Estimation of Dynamical Systems, IEEE Conf. Meas. Instrum, Hamamatsu, Japan, 1994, pp A. Materka, Neural network technique for parametric testing of mixed-signal circuits, Electronics Letters, vol. 31, no. 3, 1995, pp A. Materka, S. Mizushina, Parametric signal restoration using artificial neural networks IEEE Trans. Biom. Eng., April 1996, pp S. Huffel, J. Vandewalle, The Total Least Squares Problem, SIAM, A. Atkinson, A. Donev, Optimum Experiment Design, Oxford Science, S. Haykin, Neural Networks: A Comprehensive Foundation, Macmillan, H. Leung, S. Haykin, Rational Function Neural Network, Neural Comput., 5, 1993, pp

6 Table 1 Performance of the modular architecture (Euclidean norm) p Training time, s Max abs. RMS