NEURAL NETWORKS BASED SYSTEM IDENTIFICATION TECHNIQUES FOR MODEL BASED FAULT DETECTION OF NONLINEAR SYSTEMS. Afef Fekih

Transcription

1 International Journal of Innovative Computing, Information and Control ICIC International c 2007 ISSN Volume x, Number 0x, x 2005 pp. 0 0 NEURAL NETWORKS BASED SYSTEM IDENTIFICATION TECHNIQUES FOR MODEL BASED FAULT DETECTION OF NONLINEAR SYSTEMS Afef Fekih Department of Electrical and Computer Engineering University of Louisiana at Lafayette P.O. Box 43890, Lafayette, LA 70504, USA afef.fekih@louisiana.edu Hao Xu and Fahmida N. Chowdhury Department of Electrical and Computer Engineering University of Louisiana at Lafayette P.O. Box 43890, Lafayette, LA 70504, USA hxu@louisiana.edu; fnchowdh@louisiana.edu Abstract. Residual generation is an essential part of model-based fault detection schemes. For nonlinear systems, the task of residual generation is sometimes complicated by the size of the problem, or by the lack of a suitable model from where the residual can be generated. This paper develops and implements neural-networks based system identification techniques for nonlinear systems with the specific goal of residual generation for fault detection purposes. Two NN structures were investigated in this paper: a new structure of partially connected neural networks (PCNN), and a conventional, fully connected neural network (FCNN). The two approaches are tested on a Boeing 747 aircraft model. Results of computer experiments are reported. Performance comparisons of the two neural networks are presented. Keywords: Neural networks, Partially connected neural networks, Fully connected neural networks, Identification, Fault detection, Aircraft. 1. Introduction. Fault detection and identification (FDI) are critical issues in the operation of high performance airplanes, space vehicles, and structures where safety, mission satisfaction, and significant material value are important [1,2]. Real-time FDI would insure high performance of the aircraft even with impairments to the actuators, sensors or control surface, and thus increase the aircraft s survivability, and probability of mission success [3]. In model based fault detection, a model (mathematical or heuristic) is employed to describe the nominal behavior of the monitored system. Fault detection is accomplished by using a quality index (residual) to compare the output predicted by the nominal identification map (signals obtained from the model) with the actual measurements (realtime output signals). The residuals are expected to be close to zero in fault-free cases, but are distinguishably different from zero when a component of the system fails [4,5]. The success of the model-based method is heavily dependent on the quality of the model; 1

2 2 A. FEKIH, H. XU AND F. N. CHOWDHURY accurate modeling for complex nonlinear systems is very difficult to achieve in practice [6-9]. Aircraft dynamics are inherently nonlinear and present inertial coupling and aerodynamic nonlinearities [10]. Due to nonlinearities in the aircraft model, two problems are encountered: (1) the full nonlinear model is too slow to simulate in real time, and (2) although the linearized model can be simulated in real time, the model mismatch is too large and thus, the residuals in fault-free situations are not close to zero. To circumvent this problem, we decided to investigate the use of Neural Networks to model the aircraft [11-14]. Indeed, the remarkable learning capability of neural networks is leading to their application in many areas of engineering and science [15-18]. The idea is that after the initial training, the NN model can be run in the feedforward mode to generate the nonlinear model output, which can then be used to calculate residuals that would be usable for fault detection. Two different neural network structures are developed in this paper and tested on a detailed full-scale simulation model of the B747 aircraft [19]. The first is a conventional fully connected neural network (FCNN), where all connections between inputs, hidden layer neurons, and outputs are assumed to exit. The second is a partially connected neural network (PCNN), where the input-output pairs at different time steps are decoupled. The (PCNN) structure proposed in this paper is motivated by the theory developed in [20] proving that the conventional fully-connected NN-based autoregressive models are not realizable in the classical state-space form, while the partially connected NN (PCNN), termed additive NARX model in [20], which make the system identification more general and useful. Compared to (FCNN), (PCNN) has fewer connection weights to adjust and thus a simpler structure which may result in a faster training process. It is an advantage since the NN model might need occasional online adjustment even though it has been trained off-line. Moreover, the (PCNN) structure proposed in this paper may make hardware implementations simpler, since there would be fewer connections in the circuitry. Also, since the hidden layer weight matrix in the proposed (PCNN) structure is guaranteed to have a block-diagonal form, this may motivate newer and faster training algorithms which can make real-time training possible. This paper is organized as follows. Section 2 describes the neural networks scheme for FDI. The NN models for the Boeing 747 aircraft are developed in section 3. Computer simulations showing the implementation of the NN based system identification technique for the B747 aircraft is reported in section 4. Comparison between the two architectures is also conducted in this section. Finally, section 5 gives some concluding remarks. 2. Neural Networks Based Scheme for System Fault Detection. To overcome some of the difficulties of using mathematical models, and make FDI algorithms more applicable to real systems, neural networks can be used to both generate residuals and isolate faults [11,21] Fault Detection via Neural Networks. The approximation abilities of neural networks show a great promise in nonlinear system identification as they can approximate any nonlinear function, given suitable weighting factors and architecture [22]. Traditional methods for dealing with nonlinear systems depend on generating a linear model of the

3 NEURAL NETWORKS BASED SYSTEM IDENTIFICATION 3 system at some operating point [23-24]. No linearization is required for the neural networks. The mathematical model used in traditional methods is very sensitive to modeling errors, parameter variation, noise and disturbance. Online training makes it possible to change the FDI system easily in cases where changes are made in the physical process, control system or parameters. Neural networks have the ability to make intelligent decisions in cases of noisy or corrupted data. They also have a highly parallel structure, which is expected to achieve a higher degree of fault tolerance than conventional schemes. Neural networks can operate simultaneously on qualitative and quantitative data and they are readily applicable to multivariable systems. Generally, a neural network based FDI scheme consists of residual generation and decision making [3]. The general structure of a neural network based scheme for fault diagnosis is displayed in Fig.1. F aults u(k -1) Process y(k) y(k -1) NN based system identification y ( k) Fault Detection r (k) Fault Figure 1. Neural-Networks Based Scheme for Fault Detection For residual generation purposes, the neural network replaces the analytical model describing the process under normal operation. The NN is trained using experimental data, which can be obtained from the process or a full scale simulator. After training, the NN can be used for residual generation; its weights are fixed and used to carry out nonlinear system identification. The difference between the NN output and the plant s output constitute the residual, a good indicator for fault detection. The residuals are calculated as: r i (k) = y i (k) ŷ i (k), i = 1,..., p (1) where y i (k) are the plant measures and ŷ i (k) are the predictions. p is the total number of data samples available from experimental data. The residuals should be independent of the system operating state. In absence of faults, the residuals are only due to noise and disturbance. When a fault occurs in the system, the residuals deviate from zero in characteristic ways. There are a large number of neural network architectures for fault diagnosis. Among them Feedforward neural networks with NARX configuration, recurrent Backpropagation

4 4 A. FEKIH, H. XU AND F. N. CHOWDHURY Networks, Radial Basis Function Networks with NARX configuration, Dynamic Neural Networks. A comparative study of some of these structures can be found in [25]. All methods can be used to learn a nonlinear autoregressive with exogenous NARX system: y(k) = f(y(k 1),..., y(k n y ), u(k 1),..., u(k n u )) + v(k) (2) Where f(.) is a vector-valued nonlinear function. The functions y(k), u(k) and v(k) are the system output, input and noise vectors, respectively. n y is the delay in the outputs and n u is the delay in the inputs. The NN output is: ŷ k = CΦ[W X k ] (3) where C is the weight of the output layer and W is the weight of the hidden layer, and X is the input vector which contains past inputs and outputs of the system being modeled,φ(.) is a saturation type smooth nonlinear activation function for the hidden layer. X k is the input vector to the NN, defined as follows: X k = [(y(k 1),..., y(k n y, u(k 1),..., u(k n u )] (4) This NN model is trained from available experimental data. The dimension of the input vector of the NN depends on the order of the NARMA(X) model, while the dimension of the output vector is the same as the dimension of the plant output. In this paper we are interested in on-line fast fault detection which requires fast conversion of the NN. For this, fewer connections in the NN would be more convenient. Therefore, we are investigating the use of partially connected NN and comparing the results with the fully connected NN Partially Connected NN Versus Fully Connected NN. Neural networks are systems in which computational units analogous to the human brain are interconnected. In conventional fully connected neural networks, all connections between inputs, hidden, and output layers are assumed to exist. Due to the learning method, the structure of the (FCNN) usually have unnecessary connections which often implies high redundancy, complexity of the networks which may result in slow training time, especially for large networks. Partially connected NN are intended to reduce the size of the neural network without producing significant modeling error. In most cases, partially connected neural networks (PCNNs) are constructed based on fully connected neural networks (FCNNs). When training a (FCNN), three methods can be used to reduce the size of the neural network: pruning the connections, pruning the nodes, and pruning the inputs. Several reduction methods can be found in the literature [26-28]. In Kang et al., [27], a (PCNN) is constructed by trimming the inputs based on an input sensitivity analysis test between each input and output. This test is done by analyzing input sensitivity changes while amplifying the magnitude of inputs. Inputs are classified as coupled or uncoupled types according to the correlations between them. That is, the sensitivity changes of the uncoupled inputs are not correlated with the variation on any other input, while those on the coupled inputs are correlated with the variation on any one of the coupled input.

5 NEURAL NETWORKS BASED SYSTEM IDENTIFICATION 5 In this paper, a (PCNN) is constructed by decoupling input-output pairs at different time-steps. This choice is the result of the following considerations:1) It has recently been shown by Kotta, et al. [20] that the conventional fully-connected NN-based autoregressive models are not realizable in the classical state-space form, while the partially connected NN (PCNN), termed additive NARX models in [20], are. This property of the PCNN makes it attractive from a control design point of view, in addition to serving its role as the basis for detecting faults in the system. 2) The simpler structure of the (PCNN), with blocks of zeros in the hidden layer weight matrix, may motivate new, faster and computationally efficient training algorithms in the future. 3) Functional relationships between input-output pairs at different time- steps may actually be unrealistic, therefore, if the (PCNN) outcome is as accurate as the (FCNN), there is no justification for using the (FCNN).4) The (PCNN) structure may make hardware implementations simpler, since there would be fewer connections in the circuitry. Figure 2, illustrates a second order partially connected NN structure constructed by decoupling input-output pairs at different time-steps. x 1 (k-1) x 2 (k-1) x n (k-1) y 1 (k) y 2 (k) x 1 (k-2) x 2 (k-2) y m (k) x n (k-2) Figure 2. 2 nd order partially connected NN structure Now let s investigate the number of weights in fully connected and partially connected NN structures for an n th order nonlinear system. For an n th order nonlinear system: y(k) = f(u(k 1), y(k 1), u(k 2), y(k 2),..., u(k n), y(k n)) + v(k) (5) A fully connected NN for this function can be represented by: ŷ(k) = CΦ[W X(k)] (6) where X(k) is the input vector defined as: where X(k) = [X T k 1, X T k 2,..., X T k n] T (7) X T k i = [u T (k i), y T (k i)] T (8)

6 6 A. FEKIH, H. XU AND F. N. CHOWDHURY Suppose X k i R m, and there are (n h) neurons in the hidden layer, then the W matrix has (n 2 mh) elements. For an (n th ) order nonlinear system with the following particular structure: y(k) = f 1 (u(k 1), y(k 1)) f n (u(k n), y(k n)) + v(k) (9) A partially connected NN is described by: ŷ(k) = C[Φ T (W 1 X k 1 ), Φ T (W 2 X k 2 ),..., Φ T (W n X k n )] T (10) With the same number of hidden neurons, the number of elements in each connection matrix W i is mh, while the total number of connections between the input and hidden layer is nmh, which is 1 of the fully connected NN. Of course, when the partially connected NN n is used to represent the function in (4), the nonlinear correlation among inputs at different moment will be omitted. Moreover, potential advantages of the partially connected neural network (PCNN) compared to the fully connected NN (FCNN) structure are the following: Block diagonal structure of hidden-layer weight matrix, Non-zero weights in the matrix, Fewer Reduced training and recall time, Improved generalization capabilities, Simple structure which might assist the design of controller, Reduced hardware requirements,as well as being a step closer to biological reality. A neural networks based system identification technique will be developed for the B747 Aircraft in the next section. Fully connected and partially connected architectures will be designed and implemented in the B747 aircraft model. 3. NN Model for the B747. In this section, we develop a neural network (NN) model for the Boeing 747 aircraft. The NN accepts the data from the controller and sensors. After training, the NN is planned to accept inputs from controller and the outputs of the sensors to produce the estimate of the future plant outputs. It will be used in real-time to produce the residuals after comparing with the actual measured outputs Fully Connected NN. The NN model is generated using data from FTLAB 747 in MATLAB v 6.1 Simulator, originally developed by a research group at Delft University of Technology, Netherlands [19]. FTLAB provides input data and corresponding output data for different flight conditions (trim). The following trim conditions are reported in this paper: (a) flight condition: straight and level; (b) Flight path angle: zero; (c) Geometric altitude: 7,000 meters; and (d) True air seed: 241 m/sec. The data sets were generated using closed-loop and highfidelity conditions. The following features were also used in the generation of the data sets: 1. Zero wind condition. 2. Zero turbulence condition. 3. Stop time of 20 s. with a step size of 0.01 s. The data generated by FTLAB for the corresponding conditions is then used for our NN Program. Our input data file (actuator input), generated from FTLAB contains seven rows of data and our output data file (sensor output) contains fourteen rows of data. The latitude model of the B747 was tested with MIMO fully connected NARMA-type neural network. For the longitude model of B747, we construct a 2-input 7-output third order model. The inputs are elevator deflection and thrust engine,

7 NEURAL NETWORKS BASED SYSTEM IDENTIFICATION 7 while the outputs are pitch rate about body Y-axis, true air speed, angle of attack, pitch angle, altitude, X-position and true air speed derivative. Training was done off-line. Full model of the B747 was tested with MIMO fully connected NARMA-type neural network. Training was done off-line. Testing was done with simulated data from different flight conditions Partially Connected NN. Due to the complexity of actual aircraft system and changing flight conditions, it is desirable to accomplish on-line adaptation in addition to off-line training of the NN. However, the training process of a fully connected neural network is time-consuming, thus not satisfactory for on-line adaptation. Here, the PCNN have a control-oriented structure. The inputs at the same moment form a fully connected sub-net while the outputs of those sub-nets are combined to be the outputs of the system. With this structure, the order of the neural network can be easily increased and reduced to achieve the balance of the performance and complexity of the network. Furthermore, each sub-net is related to a particular time step and their outputs represent that step s corresponding contribution to the system outputs. It is much easier to design a controller for such a decoupled system. It should also be noted that by doing this, the accuracy of the model might be compromised. We provide a series of computer experiments and a comparison between the two NN structures in the following section. 4. Computer Experiments. MATLAB Neural Networks Toolbox functions were used to implement the network. The data sets used in the NN training were obtained from FTLAB simulator. Different flight conditions were considered for the NN training. The performances of both NN structures are investigated in this section. Both structures are implemented for the B747 Aircraft Model Partially Connected NN (PCNN) versus Fully Connected NN (FCNN). The partially connected NN models of the B747 were tested with MIMO partially connected NARMA (Nonlinear Autoregressive Moving Average) structure which was trained by a recurrent neural network with one hidden layer. The Levenberg-Marquardt algorithm was used for the training. The order of the model used in the NN training is 3. For this 3rd order model, our partially connected neural network has 27 input nodes and 7 output nodes. There are 15 nodes in the hidden layer. The 27 input nodes form 3 groups according to the moment when the data is collected, 5 hidden nodes are assigned to each group. The FCNN has 27 input neurons, 7 output neurons and one hidden layer with 15 neurons. Note that, the B747 physical model has 2 inputs and 7 outputs for each time-step; thus, each block of the NN needs 9 inputs (recall that the input vector to the NN consists of the physical inputs and outputs of the system, and in our PCNN, each block represents one time-step). The order of the model used in the NN training is 3. Note that, the model order was selected by trial and error as is done frequently in many system identification approaches [19,26]. The third order was chosen after several trials with second and fourth order. When we found that no extra accuracy was given using the fourth order, we decided to consider the third order. Since there are three blocks in this third-order model, we have a total of 9x3=27 inputs in the partially connected neural network. For the FCNN, we also have 27 inputs because it is a third-order model with 9 inputs for each time-step. For the hidden layer, we used 15 neurons: in the PCNN, there are 3 blocks with 5 neurons

8 8 A. FEKIH, H. XU AND F. N. CHOWDHURY each, and in the FCNN, for the sake of fair comparison of the results, we chose 15 hidden neurons also. The choice of 5 neurons for each block was the result of trial and error, in the absence of any systematic method for choosing the number of hidden neurons in the general NN literature. The number of output neurons is determined by the number of physical outputs in the B747 model, which is 7. MATLAB Neural Networks Toolbox functions were used to create a custom network to represent the partially connected neural network. The Levenberg-Marquardt algorithm was used for the training. Plots for true air speed, model mismatch error and pitch angle obtained using the partially connected and fully connected neural network are reported in figures 3,4,5 and 6, respectively. The time histories of the true air speed are reported in Figure 3 for both the PCNN output and FCNN output True Air Speed vs. Time True Air Speed (m/sec.) FCNN output Actual output PCNN output Figure 3. Plot for true air speed (Actual output, FCNN output, PCNN output) The time histories of the pitch angle are reported in figure 4. A comparison of modeling errors of FCNN and PCNN is illustrated in figure 5 for true air speed and figure 6 for the pitch angle. We can see from these figures that the NN model provides a close approximation of the nonlinear model. The model mismatch is negligible and very close to zero (Fig. 5). The NN was able to approximate the complicated and highly nonlinear model of the B747 aircraft. The training of the PCNN converges after 12 epochs, similar to FCNN training. However, for each epoch, the training speed is much faster due to fewer adjustable parameters. For training both networks, the outcome is almost the same but PCNN is simpler and fast. Compared to fully connected neural network model, we can see that the model errors of the PCNN and the FCNN are on the same level. Moreover, compared to the FCNN, many zeros are filling the weight matrix of PCNN. Due to the limited space and high dimension of the weight matrices (15x27), we cannot show them in this paper. The following table summarizes the comparison between the FCNN and PCNN training performance.

9 NEURAL NETWORKS BASED SYSTEM IDENTIFICATION Pitch Angle vs. Time Pitch Angle (rad.) FCNN output Actual output PCNN output Figure 4. Plot for pitch angle speed (Actual output, FCNN output, PCNN output) 2 x 10 4 FCNN error PCNN error Error (m/sec.) Figure 5. Comparison of modeling errors of FCNN and PCNN for true air speed Table 1. Comparison between the training performance of FCNN and PCNN F CNN( ) F CNN( ) Training data Target MSE 1e 006 1e 006 Epochs Time (sec) Test MSE e e 007 NN weights

10 10 A. FEKIH, H. XU AND F. N. CHOWDHURY 6 x 10 5 FCNN error PCNN error 5 4 Error (rad.) Figure 6. Comparison of modeling errors of FCNN and PCNN for Pitch angle 4.2. Experiments with Faulty Conditions. In this section, the performances of both NN structures are tested in faulty conditions. Several kinds of faults on Elevator Deflection Actuator were simulated. Results reported here are for two kinds of faults. Fault 1: 0.01 rad. (about 0.6 degree) step increase at 8 sec. Plots for the resulting residuals obtained using the FCNN and PCNN are shown in figures 7 and 8, respectively. 2 x Residual (m/sec.) Figure 7. Residual from FCNN - faulty case Fault 2: 0.01 rad. (about 0.6 degree) step increase developed over 1 sec. period, starting at 8 sec. Plots for the resulting residuals obtained using the (FCNN) and (PCNN) are shown in figures 9 and 10, respectively.

11 NEURAL NETWORKS BASED SYSTEM IDENTIFICATION 11 2 x Residual (m/sec.) Figure 8. Residual from PCNN - faulty case 2 x Residual (m/sec.) Figure 9. Residual from FCNN - faulty case Other faults were also experimented and a similar behavior was observed. Based on several simulation runs, the analysis of the residuals shows that as far as fault detection is concerned, both (FCNN) and (PCNN) models generate quite similar results. Note that, while (PCNN) and (FCNN) are equally accurate in identification for healthy cases, among the several faulty scenarios we investigated, we encountered one case where (FCNN) performed better than (PCNN) in detecting the fault. This is expected, since simpler structure may result in a partial loss of accuracy which is a tradeoff between computational burden and accuracy; the simpler the better but at a price. Though, the (PCNN) structure offers several advantages such as simpler structure, fewer weights, and significant computational time saving.

12 12 A. FEKIH, H. XU AND F. N. CHOWDHURY 2 x Residual (m/sec.) Figure 10. Residual from PCNN - faulty case 5. Conclusions. A neural networks based system identification technique, with the goal of fault detection, was developed and implemented for the Boeing 747 aircraft, using data from a full nonlinear simulation package. Two model structures, partially connected neural network (PCNN) and fully connected neural network (FCNN), were investigated in this paper and trained for the aircraft model. The (PCNN) and (FCNN) required the same number of epochs to train for a given target error, but the (PCNN) required considerably less time per epoch. The (PCNN) possesses fewer connections and weights, which makes its structure simpler and may result in a significant computational time saving. Moreover, the block-diagonal hidden layer weight matrix may motivate newer and faster training algorithms which can make real-time training possible. Acknowledgment. This work is partially supported by Laspace/NASA grant number NNG05GH22H and NASA/LEQSF under award NCC5-573, contract NASA/LEQSF ( )-1. The authors also gratefully acknowledge the helpful comments and suggestions of the reviewers, which have improved the presentation. REFERENCES [1] Frank, D. M., S. X. Ding, B. Koppen-Seliger, Current Developments in the Theory of FDI, Proc. of IFAC Safeprocess, Budapest, Hungary, pp.16-27, [2] Patton, R. J., Fault Detection and Diagnosis in Aerospace Systems Using Analytical Redundancy, Proc. of IEE Coll. on Condition Monitoring and Fault Tolerance, pp.1/1-1/20, [3] Patton, R. J., R. Clark,P. M. Frank, Fault Diagnosis in Dynamic Systems: Theory and application, Englewood Cliffs, N.J.:Prentice-Hall, [4] Gertler, J. J., Fault Detection and Diagnosis in Engineering Systems, Marcel Dekker, New York, [5] Gertler, J. J., Survey of Model Based Failure Detection and Isolation in Complex Plants, IEEE Control Systems Magazine, vol.8, pp.3-11, [6] Basseville, M., I. Nikiforov, Detection of Abrupt Changes-Theory and Application, Englwood Cliffs, NJ: Prentice-Hall, 1993.

13 NEURAL NETWORKS BASED SYSTEM IDENTIFICATION 13 [7] Chen, J., R. J. Patton, Robust Model Based Fault Diagnosis for Dynamic Systems, Kluwer Academic Publishers, New York, [8] Chowdhury, F. N., B., Jiang, C. M., Belcastro, Reduction of False Alarms in Fault Detection Problems, International Journal of Innovative Computing, Information and Control, vol.2, no.3, pp , [9] Iserman, R., Process Fault Diagnosis Based on Modeling and Estimation Methods-A survey, Automatica, vol.20, pp , [10] Stevens, B., F. Lewis,Aircraft Control and Simulation, 2nd edition, John Wiley and Sons, Hoboken, New Jersey, [11] Chen, S., S.A. Billings, P.M. Grant, Nonlinear System Identification Using Neural Networks, International Journal of Control, vol. 51, no.6, pp , [12] Haykin, S., Neural Networks: A Comprehensive Foundation, 2/E., Englewood Cliffs, NJ:Prentice- Hall, [13] Hornik, K., M. Stinchkomb, M. White, Multilayer Feedforward Networks are Universal Approximators, Neural Networks, vol.2, pp , [14] Hagan, M.T., H.B. Demuth, M.H. Beale, Neural Network Design, PWS Publishing, Boston, [15] Norgard, M., O. Ravn, N. K. Poulsen, L.K. Hansen,Neural Networks for Modeling and Control of Dynamic Systems, Springer-Verlag, London, [16] Pham, D.T., X. Liu, Neural Networks for Identification, Prediction, and Control, Springer-Verlag, New-York, [17] Polycarpou, M. M., P.A. Ioannou, Modeling, Identification and Stable Adaptive Control of Continuous-time Nonlinear Dynamical Systems Using Neural Networks, Proc IEEE American Control Conference, pp.36-40, [18] Vachkov, G., Growing Neural Models for Process Identification and Data Analysis, International Journal of Innovative Computing, Information and Control, vol.2, no.1, pp , [19] Linden, V.,D.,C.A.A.M., DASMAT-Delft University Aircraft Simulation Model and Analysis Tool, A Matlab/Simulink Environment for Flight Dynamics and Control Analysis, Report LR-781,Delft University of Technology, Delft, [20] Kotta,U., F.N. Chowdhury, S. Nomma, On Realizability of Neural Networks-based-input-output Models in the Classical State-step Form, Automatica, vol.42, pp , [21] Patton, R.J., C.J. Lopez, F.J. Uppal, Artificial Intelligence Approaches to Fault Diagnosis, IEEE colloquium on condition monitoring, machinery and external structures, pp. 5/1-5/18, [22] Kondo, T., J., Ueno, Revised GMDH-type Neural Network Algorithm with a Feedback Loop Identifying Sigmoid Function Neural Network, International Journal of Innovative Computing, Information and Control, vol.2, no.5, pp , [23] Liu, G.P., Nonlinear Identification and Control, Springer-Verlag, London, [24] Ljung, L., System Identification: Theory for the User, 2/E., Prentice-Hall, Englewood Cliffs, NJ, [25] Zhao, S., H. T. Su, T. J. McAvoy, Comparison of Four Neural Net Learning Methods for Dynamic System Identification,IEEE Trans. on Neural Networks, vol.3, no.1, pp , [26] Elizondo, D., E. Fiesler, A Survey of Partially Connected Neural Networks, International Journal of Neural Systems, vol.8, no. (5-6), pp , [27] Kang, S.,and C. Isik, Partially Connected Feedforward Neural Networks Structured by Input Types, IEEE Tran. Neural Networks, vol.16, no.1, pp , [28] Sietsma, J., R.J.F.Dow, Neural Net Pruning-why and How, Proc. IEEE Int. Conf. Neural Networks, vol.1, pp , 1988.