On-line identification of hybrid systems using an adaptive growing and pruning RBF neural network Gholami, Mehdi

Transcription

1 Aalborg Universitet On-line identification of hybrid systes using an adaptive growing and pruning RBF neural network Gholai, Mehdi Published in: IEEE Conference on Eerging Technologies ≈ Factory Autoation, 7 ETFA DOI (link to publication fro Publisher): 9/EFTA Publication date: 8 Docuent Version Publisher's PDF, also known as Version of record Link to publication fro Aalborg University Citation for published version (APA): Alizadeh, T (8) On-line identification of hybrid systes using an adaptive growing and pruning RBF neural network In IEEE Conference on Eerging Technologies & Factory Autoation, 7 ETFA (pp 57-64) IEEE Press DOI: 9/EFTA General rights Copyright and oral rights for the publications ade accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requireents associated with these rights? Users ay download and print one copy of any publication fro the public portal for the purpose of private study or research? You ay not further distribute the aterial or use it for any profit-aking activity or coercial gain? You ay freely distribute the URL identifying the publication in the public portal? Take down policy If you believe that this docuent breaches copyright please contact us at vbn@aubaaudk providing details, and we will reove access to the work iediately and investigate your clai Downloaded fro vbnaaudk on: septeber 4, 8

2 On-line Identification of Hybrid Systes Using an Adaptive Growing and Pruning RBF Neural Network Tohid Alizadeh *, Kari Salahshoor *, Mohaad Reza Jafari *, Abdollah Alizadeh 4, Mehdi Gholai 5* * Autoation and Instruentation Departent, Petroleu University of Technology, Tehran-Iran PO BOX: Bonab Faculty of Engineering, University of Tabriz, Tabriz-Iran alizadeh_tohid@yahooco, salahshoor@putacir, rjafari@gailco, 4 a_alizadeh@tabrizuacir, 5 etigholai@gailco Abstract This paper introduces an adaptive growing and pruning radial basis function (GAP-RBF) neural network for on-line identification of hybrid systes The ain idea is to identify a global nonlinear odel that can predict the continuous outputs of hybrid systes In the proposed approach, GAP-RBF neural network uses a odified unscented kalan filter (UKF) with forgetting factor schee as the required on-line learning algorith The effectiveness of the resulting identification approach is tested and evaluated on a siulated benchark hybrid syste Introduction In recent years, the interest in hybrid systes has grown widely Hybrid systes arise when the continuous dynaics, driven by physical laws, and discrete dynaics, derived by logical rules, coexist and interact with each other Thus, hybrid odels describe processes that evolve according to dynaic equations and logic rules Most of the literature on hybrid systes has dealt with control, stability analysis, verification and fault detection based on the availability of a odel for the hybrid syste Getting such a odel fro a given input-output data is an identification proble, which does not see to have enough attention in the hybrid syste research counity The existing identification approaches for the hybrid systes can generally be classified into the variants of the Mixed-Integer Prograing (MIP) approach [], the clustering-based approach [], the bounded-error approach [], the Bayesian approach [4] and the algebraic approach [5] [] reforulates the identification proble into subclasses of PieceWise Affine (PWA) systes which require highly coputational coplexity of Mixed- Integer Linear or Quadratic Prograing algoriths to be solved Other identification approaches rely on different odel classes of PieceWise Auto-regressive exogenous (PWARX) systes PWARX odels are suitable when dealing with input-output data, since they provide an input-output description of PWA systes This odel structure partitions the regressor space into polyhedral with ARX sub-odels Thus, it involves the estiation of the nuber of discrete odes, the paraeters of the affine sub-odels and the coefficients of the hyperplanes defining the partitions of the regressor set However the siultaneous optial estiation of all the quantities entioned is a very hard and coputationally intractable proble Hence, the available approaches are heuristic and suboptial based which in ost cases either assue a fixed nuber of discrete odes, or adjust it iteratively For instance, both the clustering-based approach and the Bayesian approach require a priori knowledge of the discrete odes nuber Moreover, soe approaches, such as the clustering-based approach, the boundederror approach and the Bayesian approach require the ARX sub-odels orders to be fixed This can arise two significant liitations First, all the existing identification algoriths concern only the classes of switched linear and PWA odel while the actual hybrid systes are dynaically non-linear Second, it can be practically unacceptable to assue a priori knowledge about the hybrid syste dynaics such as odes and the odel orders To the best of our knowledge, [6] is the first contribution paper which addresses the ore identification challenging case in which the hybrid syste dynaics are assued to be non-linear without the need to consider a priori assuptions about the nuber of odes, the odel paraeters and the switching sequence For the first tie, they used a feed-forward neural network to obtain global paraetric odels for the considered class of hybrid /7/$ 7 IEEE 57 Authorized licensed use liited to: Aalborg Universitetsbibliotek Downloaded on February, 9 at 7: fro IEEE Xplore Restrictions apply

3 d) Utilization of a tie-varying forgetting factor schee in the UKF learning algorith to aintain a desired paraeter tracking capability systes without the need to know the nuber of odes and the current ode However this approach works in an off-line ode and results in a black-box odel with a large nuber of paraeters In this paper, an on-line identification approach based on an adaptive GAP-RBF neural network is presented The UKF learning algorith is utilized to estiate the network paraeters during the identification phase The paper is organized as follows The proposed online identification approach is introduced briefly in section In section, perforance of the developed identification approach is illustrated on a siulated benchark hybrid syste and a well known PWA syste Section 4 suarizes the conclusions fro this research study Modified GAP-RBF (MGAP-RBF) Neural Network The coplete description of the MGAP-RBF learning algorith can be suarized as follows: The learning algorith begins with no initial hidden neurons As each new observation data ( x n, y n ), where x n R, are received the following steps are l perfored: Copute the overall network output: K f ( xn ) = α k φk ( xn ) () k = Adaptive GAP-RBF Neural Network where K is the nuber of hidden neurons, The ability of Neural Networks to identify a global paraetric odel for a class of hybrid systes has been deonstrated in [6] for the first tie in the literature In this contributed paper, two feed-forward neural networks were used to odel a hybrid benchark syste consisting of two interconnected tanks with 64 different nuber of odes In this approach, each neural network was associated with each continuous tank level as the corresponding syste outputs However the proposed identification approach was off-line based which required soe dedicated ethods to choose the optial structure of the neural networks involving the choice of inputs (ie the regression vector) and the nuber of hidden neurons Radial basis functions (RBF) neural networks have been popularly used in any identification applications in recent years due to their ability to approxiate coplex non-linear appings directly fro the inputoutput data with a siple topological dynaic structure Cobining this network with self-organizing network learning algoriths offers an attractive approach to ake efficient adaptive RBF neural network which can adjust its dynaic structure coplexity and paraeters to varying non-linear syste dynaics without requiring a priori knowledge Huang et al [7] have recently proposed a siple sequential learning algorith with network growing and pruning (GAP) capabilities The original GAPRBF neural network algorith has been odified in [8] to enhance it s capabilities for on-line syste identification applications as follows: a) Enhancing the sooth criterion of the neurons b) Enhancing the pruning criterion to prevent probable oscillation in the nuber of created neurons c) Utilization of the unscented kalan filter (UKF) estiation algorith to adjust free network paraeters α k is the connecting weight of the kth hidden neuron to the output neuron φ k ( x n ) denotes response of the kth hidden unit to the input vector x n, defined by the following Gaussian function: x μ φ k ( x n ) = exp n n σk where μk and σk () refer to the center and width of the kth hidden neuron respectively and indicates the Euclidean distance Calculate the paraeters required in the odified growth criterion: ε n = ε in + (ε ax ε in )( e n / τ ) () where τ is the tie constant paraeter that can be used to control the tie rate evolution of ε n ε in and ε ax are iniu and axiu distance thresholds, respectively en = y n f ( x n ) (4) Apply the growth criterion for adding neurons: If en > ein and xn μ nr > ε n ( and 8κ xn μ nr l )e n / S ( X ) > ein (5) (where e in is the expected desired accuracy and μ nr is the center of the nearest neuron to S ( X ) is the estiated size of range the training saples are drawn fro) Allocate a new hidden K+ with: 58 Authorized licensed use liited to: Aalborg Universitetsbibliotek Downloaded on February, 9 at 7: fro IEEE Xplore Restrictions apply x n and ( X ) where

4 α K + = e in μ K + = x n where λ = α ( L + k ) L and scaling paraeters update equations: (6) σ K + = κ x n μ nr Else Adjust the network paraeters γ = L+λ χ k k = f [ χ k, u k ] () L α nr, μ nr, σ nr for xˆ k = Wi ( ) χ i, k k the nearest (nrth) neuron only, using the UKF algorith Check the odified pruning criterion for the nearest (nrth) hidden neuron: () i = L Pk = Wi ( c ) [ χ i, k k xˆ k ][ χ i, k k xˆ k ]T + R w i = (4) If (8σ nr ) α nr / S ( X ) < β ein, (in which a l where {Wi new pruning factor < β has been added), reove the nearest (nrth) hidden neuron and do the necessary changes in the UKF algorith } and {Wi } are sets of scalar weights, R is process noise covariance, and χ k k = [ xˆ k xˆ k + γ Pk xˆ k γ Pk ] (5) Yk k = h[ χ k k ] (6) L (7) i = 4 Measureent update equations: L Py k y k = Wi ( c ) [Yi, k k yˆ k ][Yi, k k yˆ k ]T + R v i = (8) L Pxk y k = Wi ( c ) [ χ i, k k xˆk ][Yi, k k yˆ k ]T (9) i = K k = Px k y k Py ky k k () k xˆk = xˆ + K k ( yk yˆ ) () Pk = ( Pk K k Py k y k K k ) /η k () T where η k behaves as the forgetting factor concept in the usual recursive least-squares (RLS) algorith which undergoes the following tie-varying evolution (8) In the UKF algorith, instead of linearizing the above non-linear syste odel equations using Jacobian atrices in the EKF, a deterinistic sapling approach is used to calculate the ean and variance estiates of Gaussian rando state variables x n with a inial set of L + saple points ( L is the state diension), called as siga points The results are accurate to the third-order (Taylor series expansion) for Gaussian inputs for all nonlinearities The UKF learning algorith can be suarized as follows: Initialize with soe initial guesses for the state estiate ( x n ) and the error covariance atrix ( Po ), defined as: xˆ = E[ x ] (9) η k = η k + ( η k )( e t / τ ), p η k () where t is the recursive tie interval that is spent in the UKF learning algorith to estiate the MGAP-RBF neural network free paraeters with fixed structure Thus, t is reset to zero when any network structural change, ie neuron creation or pruning occurs This schee aintains a desired paraeter adaptive capability in the UKF algorith whenever process dynaics undergo a tie-varying change Identification case studies P = E[( x xˆ )( x xˆ ) T ] () for k {, K, }, ( E[ ] denotes the expected value) Identification of a two tanks flow hybrid benchark syste The proposed on-line identification approach has been applied to the two tanks flow hybrid benchark syste introduced in [6] which has been depicted in Calculate the siga points: χ k = [ xˆ k xˆ k + γ Pk xˆ k γ Pk ] (c ) yk = Wi ( )Yi,k k The UKF Modified Learning Algorith The original GAP-RBF algorith uses the extended kalan filter (EKF) as its network paraeter estiation algorith In practice, however, the use of the EKF has two well-known drawbacks: a Linearization can provide highly unstable filter if the local linearity assuption is violated b The derivations of the Jacobian atrices are nontrivial inn ost applications and often lead to significant ipleentation difficulties To address these liitations, Julier and Uhlann [9] developed the UKF algorith Consider the non-linear syste, described by the following equations: x k + = f ( x k, u k ) + wk (7) ( ) w Endif Endif y k = h( x k ) + v k are () 59 Authorized licensed use liited to: Aalborg Universitetsbibliotek Downloaded on February, 9 at 7: fro IEEE Xplore Restrictions apply

5 figure This syste is considered as a black-box syste equipped with soe sensors that provide the required I/O data for identification purpose The dynaics of the syste can be described by the following equations: Figure The scheatic diagra of two tanks syste Q P = D P, P {,} (4) Q P = D P, P {,} (5) Q = A g h V, V {,} (6) Q = A g h V, V {,} (7) Q = α h h V, V {,} (8) Q4 = α sup( h,5) sup( h,5) V, V4 {,} (9) Sh = Q Q Q4 () S h = Q + Q + () Q4 where α = A gsign( h h ), D is the constant input flow of both pups, QPi, i{,} is the input flow of tank i, Q i, i {,} is the output flow of tank i, Q is the flow in the pipe C and Q4 is the flow in the pipe C In order to avoid either the draining or the overflow of the tanks, the electro valvesv, V and V4 as well as the pup P are driven by an algorith which guarantees the following levels of the fluids in the tanks R and R : h M () h M () with M = 6, M = 75, = 4 and = The pup P and the electro valvev are considered as perturbations which cannot be controlled Hence P andv were opened and closed according to two Nearly Rando Binary Sequences of length: l [, P ] and l [, V 5] The siulated hybrid syste was identified by two MGAP-RBF neural networks using the proposed adaptive approach presented in section Each neural network was used to identify the behavior of the liquid level in each individual tank by considering the following Nonlinear Auto-Regressive with exegenous (NARX) input odel structure: yi ( t) = fi ( xn ( t)) ; i =, (4) where x ( t) = [ P ( t ), P ( t), P ( t ), P ( t), n P ( t ), P ( t), V ( t ), V ( t), ( t), V y ( t ), y ( t )] i V ( t ), V i ( t ), V ( t), (5) Thus, each neural network has 4 inputs which are associated with 6 binary inputs at two latest tie instants of t and t and one continuous liquid level output recorded at of t and t tie instants During the identification, the first data set called as the identification data set, containing N=5 data, generated through the siulation experient done by N Messai et al[6] was used in a sequential anner to eulate a real-tie identification experient As a consequence, each MGAP-RBF neural network started without a priori knowledge about the network structure by setting the free-algorith paraeters at ε = in 9, ε ax =, k = 5 9, e = in The identification results have been illustrated in figures and Figures and show the easured and identified liquid levels The analysis of these figures deonstrates the high accuracy of the identified odels Figures and show the error residuals which do not exceed 4% of the easured levels Figures and illustrate the distributions of the resulting error residuals which can infer the high accuracy of the identified odels Figures (d) and (d) show the tie history of the hidden neuron evolution during the on-line identification for the two MGAP-RBF neural networks As illustrated, the resulting network structures are quite siple and copact with and axiu 4 neurons respectively Coparing the obtained results with those recorded in [6] for the sae hybrid syste (shown in figures 4 and 5 []), deonstrates the superiority of the proposed identification approach based on the following general observations: a The proposed algorith is siple and on-line without a priori knowledge while the approach 6 Authorized licensed use liited to: Aalborg Universitetsbibliotek Downloaded on February, 9 at 7: fro IEEE Xplore Restrictions apply

6 proposed in [6] is off-line and require dedicated ethods to extract the required odel structure b The resulting odels are very siple and copact Each network uses 4 inputs with axiu and 4 hidden neurons while the identified networks in [6] uses input with 8 hidden neurons c The obtained odels have better accuracy in ters of error residuals or residual distributions (d) Figure Predicted and easured liquid level, identification residuals, distribution of the error residuals and hidden neuron evolution (d) for st tank (d) Figure Predicted and easured liquid level, identification residuals, distribution of the error residuals and hidden neuron evolution (d) for nd tank Authorized licensed use liited to: Aalborg Universitetsbibliotek Downloaded on February, 9 at 7: fro IEEE Xplore Restrictions apply

7 Figure 4 Predicted and easured liquid level, identification residuals and distribution of the error residuals for st tank using feed forward NN -5-4 Identification of a PWARX benchark odel The following PWARX odel is considered []: Figure 5 Predicted and easured liquid level, identification residuals and distribution of the error residuals for nd tank using feed forward NN 4 yk + uk ek, if 4 yk + uk + p 5 yk uk 5 + ek, yk = if 4 yk + uk + (6) and 5 yk + uk 6 yk + 5uk 7 + ek, if 5 yk + uk 6 The input uk and the noise ek are white noises yk generated fro unifor distributions over the intervals [ 5, 5] and [, ], respectively data points are available as depicted in figure 6 [] We applied the bounded-error clustering-based identification algorith [] on this exaple In order to copare the results, the predicted output of the identification together with error residuals are presented in figures 7a-7c The predicted output and actual data are depicted in figure 7a, and the identification error with its distribution is shown in figures 7b and 7c, respectively u k- yk- Figure 6 Data identification points 6 Authorized licensed use liited to: Aalborg Universitetsbibliotek Downloaded on February, 9 at 7: fro IEEE Xplore Restrictions apply used for

8 The results show better accuracy in the identified odels with siple network structures Figure 7 Predicted and easured outputs, identification residuals and distribution of the error residuals using Clustering-Based Bounded-Error approach - 4 u k and y k are used as inputs of the MGAP- RBF neural network Identification is perfored using just one hidden neuron and the identification error is very sall copared to previous results (neglecting a few points, identification error is between -7 and 7), as shown in figures 8a-8d 5 5 (d) Figure 8 Predicted and easured outputs, identification residuals, distribution of the error residuals and hidden neuron evolution (d) 4 CONCLUSION In this paper, a new ethod for on-line identification of hybrid syste by using the MGAPRBF neural network has been proposed The proposed approach tested on two exaples and the results are copared with soe existing approaches which leads to the following observations: The proposed approach is an on-line identification ethod, while the considered approaches are offline The MGAP-RBF neural network has an adaptive dynaic structure Acknowledgents The authors would like to thank Dr N Messai (CReSTIC, Université de Reis ChapagneArdenne,) for providing the data used for identification in Section 6 Authorized licensed use liited to: Aalborg Universitetsbibliotek Downloaded on February, 9 at 7: fro IEEE Xplore Restrictions apply

9 References [] Beporad A, Roll J and Ljung L Identification of Hybrid Systes via Mixed-Integer Prograing, 4th IEEE Conference on Decision and Control, Orlando, Florida USA, pp , [] Ferrari-Trecate G, Muselli M, Liberati D and Morari M A clustering technique for the identification of piecewise affine and hybrid systes, Autoatica, vol 9, pp 5-7, [] Beporad A, Garulli A, Paoletti S and Vicino, A Data classification and paraetr estiation for the identification of piecewise affine odels, 4rd IEEE Conference on Decision and Control, Paradise Island, Bahaas pp -5, 4 [4] Juloski A, Weiland S and Heeels W A Bayesian approach to identification of hybrid systes, 4rd IEEE Conference on Decision and Control, Paradise Island, Bahaas pp -9, 4 [5] Vidal R, Soatto S, Ma Y, Sastry S Identification of PWARX hybrid odels with unknown and possibly different orders, IEEE Aerican Control Conference, pp , 4 [6] Messai N, Zaytoon J and Riera B Using Neural networks for the Identification of a Class of Hybrid Dynaic Systes, the nd IFAC Conf on Analysis and Design of Hybrid Systes (Alghero, Italy), 7-9 June 6 [7] Huang G-B, Saratchandran P, and Sundararajan N, An efficient sequential learning algorith for Growing and Pruning RBF (GAP-RBF) networks IEEE Transactions on systes, an, and cybernetics, part B, vol 4, No 6, Deceber 4 [8] K Salahshoor, M R Jafari, On-line Identification of Non-linear Systes Using Adaptive RBF-based Neural Networks, accepted to be published in IJIST journal, 7 [9] Julier S J and Uhlann J K, A New Extension of the Kalan Filter to Non-linear Systes, in Proc of AeroSense: The th Int Syp ADSSC, 997 [] MNørgaard: Neural Network Based Syste Identification Toolbox, Tech Report -E-89, Departent of Autoation, Technical University of Denark, [] A Beporad, A Garulli, S Paoletti, and A Vicino, A greedy approach to identification of piecewise affine odels, in Hybrid Systes: Coputation and Control, LNCS Springer-Verlag, pp 97, [] M Tabatabaei-Pour, K Salahshoor, MMoshiri, A Clustering-Based Bounded-Error Approach for Identification of PWA Hybrid Systes, The 9th International Conference on Control, Autoation, Robotics and Vision, ICARCV, Singapore, Deceber 6 64 Authorized licensed use liited to: Aalborg Universitetsbibliotek Downloaded on February, 9 at 7: fro IEEE Xplore Restrictions apply