White Rose Research Online URL for this paper: Version: Accepted Version

Transcription

1 Thi i a repoitory copy of Identification of nonlinear ytem with non-peritent excitation uing an iterative forward orthogonal leat quare regreion algorithm. White Roe Reearch Online URL for thi paper: Verion: Accepted Verion Article: Guo, Y., Guo, L. Z., Billing, S. A. et al. (1 more author) (2015) Identification of nonlinear ytem with non-peritent excitation uing an iterative forward orthogonal leat quare regreion algorithm. International Journal of Modelling, Identification and Control, 23 (1). pp ISSN Reue Unle indicated otherwie, fulltext item are protected by copyright with all right reerved. The copyright exception in ection 29 of the Copyright, Deign and Patent Act 1988 allow the making of a ingle copy olely for the purpoe of non-commercial reearch or private tudy within the limit of fair dealing. The publiher or other right-holder may allow further reproduction and re-ue of thi verion - refer to the White Roe Reearch Online record for thi item. Where record identify the publiher a the copyright holder, uer can verify any pecific term of ue on the publiher webite. Takedown If you conider content in White Roe Reearch Online to be in breach of UK law, pleae notify u by ing eprint@whiteroe.ac.uk including the URL of the record and the reaon for the withdrawal requet. eprint@whiteroe.ac.uk

2 Identification of Nonlinear Sytem with Non-Peritent Excitation uing an Iterative Forward Orthogonal Leat Square Regreion Algorithm Yuzhu Guo 1, L.Z. Guo 1, 2, S. A. Billing 1, and Hua-Liang Wei 1, 2 1 Department of Automatic Control and Sytem Engineering The Univerity of Sheffield, Mappin Street, Sheffield, S1 3JD, UK. 2 INSIGNEO intitute for in ilico Medicine The Univerity of Sheffield, Mappin Street, Sheffield, S1 3JD, UK. Abtract A new iterative orthogonal leat quare forward regreion (iofr) algorithm i propoed to identify nonlinear ytem which may not be peritently excited. By lightly reviing the claic forward orthogonal regreion (OFR) algorithm, the new iterative algorithm provide earch olution on a global olution pace. Example how that the new iterative algorithm i computationally efficient and capable of producing a good model even when the input i not completely peritently excited. Key word: Model tructure detection, nonlinear ytem identification, non-peritence, orthogonal forward regreion, iterative learning algorithm, OFR algorithm, iofr algorithm 1. Introduction Peritent excitation of the input i a deirable property for ytem identification. An input ignal hould be rich enough to fully excite the dynamic of a ytem o that the ytem can be uniquely determined in the ytem identification proce. Peritent excitation ha been widely tudied for linear ytem identification N A L S where it i well known that the input hould excite over all the frequency range. Non-peritence of excitation may caue a ingularity in the regreion matrix and reult in poor etimation of the parameter. For the identification of nonlinear ytem a rich frequency content i not ufficient. To fully excite a nonlinear ytem, the input mut be adequately rich both in frequency a well a in amplitude o that the full amplitude range of the nonlinearitie i alo excited (Nowak, 2002). 1

3 The problem caued by non-peritent excitation can be olved by experiment deign. Optimal input deign for nonlinear ytem identification ha been tudied (Halmaron and Martenon, 2007, Laron et al., 2010, Hirch, 2010). However, the data ued in many real ytem identification tudie are from real procee where there may be retriction on the input allowed o there i no guarantee of peritently excitation. Input ignal alo cannot be deigned in the identification of an autonomou ytem. Therefore a tudy of the identification of ytem which are not completely peritently excited i important in many practical application. Algorithm which are robut to nonperitent input are needed in practical application. Among the exiting nonlinear ytem identification method, the NARMAX (Nonlinear AutoRegreive Moving Average with exogenou input) model and the aociated Orthogonal Forward Regreion (OFR) algorithm have been widely applied in the modelling of many engineering, chemical, biological, medical, geographical, and economic ytem (Billing, 2013). Variation of thee algorithm have been developed for lumped and ditributed parameter ytem, timeinvariant and rapidly time-varying ytem, and in the time, frequency and patio-temporal domain. The OFR algorithm can efficiently determine a parimoniou model tructure without any a priori knowledge of the nonlinear ytem. The OFR algorithm, which regree the variation of the dependent variable along the path where the um of the ERR (Error Reduction Ratio) value increae at the fatet peed i computationally efficient. The obtained imple model tructure ha many ignificant advantage in application. A model with a imple tructure can uccefully avoid over-fitting in ytem identification and can produce a better etimation of the parameter, wherea a model with redundant term often lead to poor long term prediction and poor qualitative validation. It ha been hown that under ome circumtance, the non-peritence of the excitation may affect term election and a new algorithm baed on imulation error (or model predicted output) ha been propoed (Piroddi and Spinelli, 2003). Alternative olution include the algorithm aided by genetic algorithm (Mao and Billing, 1997), and mutual information (Wei and Billing, 2008, Billing, 2013). However, all thee olution are computationally intenive and hence are difficult to apply in real application where typically a large range of lag, model term, or multivariable ytem ha to be tudied. A iofr (iterative Orthogonal Forward Regreion) algorithm ha recently been propoed to olve the uboptimal olution problem without incurring the exceive proceing required when uing either imulation error or a full optimal earch (Guo et al., 2014). In the iofr algorithm, the claic OFR algorithm i iteratively applied where the next earch i baed on the uboptimal term et obtained at the previou tep. By lightly reviing the claic OFR algorithm, the iofr algorithm earche an 2

4 optimal model on a global olution pace. A more general iofr algorithm i propoed in thi paper and it will be hown that the new iofr algorithm i robut to ome non-peritent input. It i worth emphaiing that it i impractical to provide an ideal algorithm which work for any non-peritent excitation. The example in ubection 4.1 how that when the trength of the input i low and the noie level i high, no algorithm baed on the RSS (reidual um of quare) i likely to be able to give a correct model. The remainder of the paper i organied a follow. Section 2 briefly review the NARMAX model and the claic orthogonal forward regreion algorithm. The new iofr algorithm i introduced in Section T P S efficiency of the new algorithm in Section 4. Concluion are finally drawn in Section Orthogonal forward regreion algorithm A NARMAX model i eentially an expanion of the output with pat input, output and noie term. A wide cla of nonlinear ytem can be repreented by a NARMAX model (Billing, 2013, Leontariti and Billing, 1985) which can be defined a y k 1, y k 2,, y k ny, u k d, u k d 1, yk F e k, u k d nu, ek 1, ek 2,, ek ne (1) where y(k), u(k) and e(k) are the ytem output, input, and noie equence repectively; n y, n u, and n e are the maximum lag for the ytem output, input, and noie; F () i ome nonlinear function; d i a time delay which i often et a d=1. The nonlinear function F () i often written a the uperpoition of a et of bai function a y( k) = å q ( k) + e( k) (2) where ( k) q cient. Collecting N et of obervation yield the matrix form of equation (2) Y (3) 3

5 where i known a the regreion matrix and (1) ( ) T. 1 2 N Sytem identification involve the determination of model tructure { } and the etimation of the aociated parameter. However thee two procee are firmly coupled with each other. Ranking of the ignificance of a term depend on the weight (coefficient) of the term in a model while the etimation of the parameter depend on what term are included in the model. The OFR algorithm decouple the interaction between thee two procee and provide an efficient method for the identification of nonlinear ytem. In the OFR algorithm, the term into the orthogonal term f are orthogonalied tepwie w and the aociated coefficient can then be etimated a g w, y = (4) w, w The ignificance of the term can then be evaluated uing the ERR (Error Reduction Ratio) criterion defined a ( ) ERR w 2 g w, g w w, y = = (5) yy, w, w y, y The term can then be elected into the model according to the ERR criterion. The regreion will top when all the ignificant term have been detected. A commonly ued top condition can be et a ( ) - å ERR w r (6) 1 The um of ERR (denoted a SERR) indicate that a proportion of ERR( w ) output ha been explained by the term { f } which conit the model. å information in the The tandard orthogonal forward regreion algorithm conit of the following tep: (1) Sufficiently excite the ytem and meaure the input and output of the ytem. (2) Specify an initial full model et of candidate term and the value of. 4

6 (3) Compute the value of the ERR for each of the candidate term and elect the term which give the larget value of ERR into the model a the firt term. (4) At the k th ( k 2 ) tage: compute the value of the error reduction ratio for each of the ( k 1) remaining candidate term by auming that each i the k th term in the elected model and perform the correponding orthogonaliation; the term that give the larget value of the error reduction ratio i then elected into the model a the k th term. If condition (6) i atified, finih the proce and go to (5). Otherwie et kk 1 and repeat tep (4). (5) The final model contain term and the parameter etimate can be calculated uing a leat quare formulae. 3. The new iterative orthogonal forward regreion algorithm In the claic OFR algorithm, the term are elected into the model one at a time. At the k-th tep, the remaining term are orthogonalied with the k-1 term which have been elected at the previou tep and the term which produce the maximum ERR will be elected. The claic OFR elect term at each tep to optimize the ERR criterion. However, the elected term in each tep can occaionally produce a uboptimal model. Thi problem i mot noticeable when the ytem are not peritently excited. An iterative orthogonal Forward regreion algorithm ha been introduced to improve the uboptimal problem where a mall modification to the term election procedure ha been made to ignificantly improve the claic OFR algorithm without any ignificant increae in computational cot (Guo et al., 2014). A more general iofr algorithm will be introduced next to olve the problem caued by non-peritent input. The new iofr algorithm comprie two tep, the firt tep i to obtain a uboptimal model et and the econd tep ue a ubet of the term which were obtained in the firt tep a the tarting point of a global earch. The new iterative OFR algorithm can be ummaried in the following tep. i) Preet a tolerance and apply the tandard OFR algorithm on the whole term dictionary to produce a uboptimal term et ii) Select a mall number ; 1 2 a an amendment to the tolerance in the firt tep; 5

7 iii) Select a ubet pre of the term, where 1, 2,,, in a preelected term and earch the other term on the term et \ pre to contruct a uboptimal olution atifying ERR ; 1 i iv) Repeat iii) for different ubet pre of and obtain ome uboptimal model; v) Compare the obtained uboptimal model and chooe the bet one a the final model op. Remark: The ubet pre i often elected a a combination of p term in. There are a total number of k k ( p ) combination. All the combination are evaluated in tep iv) and ( p ) candidate model are obtained. Letting p = 1, the new iofr reduce to the iterative algorithm given in the paper (Guo et al., 2014). 4. Tet example Three example will be ued to how that a claic OFR algorithm may include redundant autoregreive term, even when the data et wa produced from a purely moving average model (Piroddi and Spinelli, 2003). Thee example will be ued in thi paper to tet the efficiency of the new iofr algorithm and to how the iofr algorithm can correctly identify an optimal model even when the ytem are not peritently excited. All the example are from and ue the ame etting in the paper (Piroddi and Spinelli, 2003). 4.1 Example 1 The firt example i given a follow ( 1) ( 2) w k u k u k u k u k u k 1 y( k) w( k) e 1 k 1 0.8z (7) where u repreent the input ignal and y repreent the obervation of the output w. Both the input u(k) and the noie e(k) are Gauian ditributed white noie. It can be hown that the claic OFR algorithm can correctly elect all the term and produce an accurate model when the ytem i peritently excited. However, Piroddi and Spinelli argued that the claic OFR algorithm may incorrectly elect autoregreive term when the input ignal i le rich in frequency component. Piroddi and Spinelli recommended an input which i generated by an AR proce with two real pole 6

8 between 0.75 and 0.9. Repeating P S uing an input ignal which wa generated by the following AR proce u( k) = v( k) (8) z z where v(k) i Gauian noie v(k) ~ N(0,1). The AR proce ha a repeat pole at 0.8 and the coefficient 0.25 i choen to guarantee the input ignal i at a reaonable level. Here the noie ignal e(k) i a Gauian ditributed noie with a variance 0.02, that i, e(k) ~ N(0,0.02). The reult produced by the tandard OFR algorithm are given in Table 1. Table 1 Reult produced by the tandard OFR algorithm for example 1 No. Term ERR Coefficient Standard Deviation 1 y(k-1) y(k-2) u 3 (k-1) u 3 (k-2) u(k-1) u 2 (k-1) u(k-2) SERR Oberve that two incorrect autoregreive term were elected overwhelming the correct term. A correct term u(k-1)u(k-2) wa alo mied in the identification. The new iterative orthogonal Forward regreion algorithm which wa introduced in the previou ection wa employed to overcome the problem by earching the optimal olution on different path. Combination of any two term in the model in Table 1 were elected a the pre-determined two term and the remaining term were elected in a model uing a claic OFR algorithm. In thi example, a total number of ( 7 2) = 21 model were obtained. T ERR F, where the red line indicate the um of ERR value produced by the real model. It can be oberved that two of the 21 model give the maximum SERR value which i equal to the SERR produced by a correct model. The reult how that both model with the maximum SERR value conit of all the correct term in 7

9 (7). Thi mean the optimal model ha been found on two different earch path. The optimal model i given in Table um of ERR' identified model real model model identified in the iterative proce Fig 1 S ERR for example 1 Table 2 Model identified uing the iofr algorithm for example 1 Standard No. Term ERR Coefficient Deviation 1 u 3 (k-1) u(k-1) u (k-1)u(k-2) u(k-2) SERR A reduction of the amplitude of the input caue a decreae of the ignal-to-noie-ratio and conequently under thee condition the identification proce may give an incorrect reult. Conider an input given a 0.2 u( k) = v( k) (9) z z where v(k) i again a equence of Gauian noie v(k) ~ N(0,1). The reult of an iofr proce are given in Fig 2. It can be oberved that ome of the model give a larger SERR value than the correct model did. Thi mean under thi ignal-to-noie-ratio level, any 8

10 ytem identification algorithm baed on a RSS (reidual um of quare) criterion cannot produce a correct model. um of ERR' identified model real model model identified in the iterative proce F S ERR for example 1 with a mall input 4.2 Example 2 Conider the following ytem. 2 2 w k 0.5w k u( k 2) u ( k 1) 0.05w k y( k) w( k) e 1 k 1 0.5z (10) The ytem wa excited by an input defined a 0.16 u( k) = v( k) (11) z z where v(k) i Gauian noie v(k) ~ N(0,1). The AR proce ha a repeat pole 0.8 and the coefficient 0.16 i choen o that the input ignal ha a imilar amplitude a v(k). Here the noie ignal e(k) i a Gauian ditributed noie with a variance 0.05, that i, e(k) ~ N(0,0.05). The reult produced by the tandard OFR algorithm are given in Table 3. Table 3 Reult produced by the tandard OFR algorithm for example 2 No. Term ERR Coefficient Standard Deviation 1 y(k-1) y(k-2)

11 3 u 2 (k-1) u(k-1)u(k-2) u(k-1) y 2 (k-2) contant SERR Oberve that an incorrect autoregreive term y(k-2) wa elected. Ue two of the 7 term in Table 3 a the previou term et and apply the new iofr algorithm. Conidering all the ( 7 2 ) combination T SERR thee model are hown in Fig 3 where the red line indicate the value of SERR of the correct model um of ERR' identified model real model model identified in the iterative proce F S ERR for example 2 Fig 3 how that 3 in 21 model gave the maximum SERR value which i equal to the SERR of the correct value. Actually all the three model are compoed of the correct term with ame etimation of the aociated parameter. The obtained model i hown in Table 4. Table 4 Model identified uing the iofr algorithm for example 2 No. Term ERR Coefficient Standard Deviation 1 y(k-1) y 2 (k-2) u 2 (k-1) u(k-2)

12 5 contant SERR Example 3 Conider ytem (12) ( 2) ( 1) y k y k u k u k y k e k (12) with the input 0.16 u( k) = v( k) (13) z z where v(k) i Gauian noie v(k) ~ N(0,1). The noie ignal e(k) i a Gauian ditributed noie with a variance 0.05, that i, e(k) ~ N(0,0.05). The reult produced by the tandard OFR algorithm are given in Table 5. Table 5 Reult produced by the tandard OFR algorithm for example 3 No. Term ERR Coefficient Standard Deviation 1 y(k-1) y(k-2) u 2 (k-1) u(k-1)u(k-2) u(k-1) y 2 (k-2) contant SERR Uing combination of two of the term in Table 5 a the pre-determined term and applying the iofr algorithm yield SERR are hown in Fig4. 11

13 um of ERR' identified model real model model identified in the iterative proce Fig 4 S ERR for example 3 Fig 4 how that 3 in 21 model gave the maximum SERR which i equal to the SERR given by the real model (the red line in Fig 4). All the three model are of a correct model tructure which i hown in Table 6. Table 6 Model identified uing the iofr algorithm for example 3 No. Term ERR Coefficient Standard Deviation 1 y(k-1) y 2 (k-2) u 2 (k-1) u(k-2) contant SERR Notice that the coefficient given in Table 6 are more accurate than the etimate given in Table 4. Thi happen becaue the noie affect ytem (10) and (12) in a different way though both ytem are of the ame tructure except for the noie model. In ytem (10) the output wa corrupted by obervation noie which doe not involve the dynamic of the ytem. In contrat, ytem (12) wa corrupted by proce noie which affect the whole proce of the ytem. 5. Concluion 12

14 A new iterative forward orthogonal regreion algorithm ha been ued to olve the uboptimal problem caued by non-peritent excitation. By lightly reviing the claic OFR algorithm, the new iofr algorithm i much more robut to non-peritent input. Example howed that the new iofr algorithm i capable of correctly identifying the model and give the optimal reult when the noieto-ignal-ratio i at a reaonable level. The new iofr algorithm, which work under a purely OFR-ERR pirit, inherit the advantage in computational efficiency and univeral applicability. The new iofr algorithm provide a robut and efficient choice for the application of nonlinear ytem identification in real ytem where the input cannot be optimally deigned. Acknowledgement The author gratefully acknowledge upport from the UK Engineering and Phyical Science Reearch Council (EPSRC) and the European Reearch Council (ERC). Reference BILLINGS, S. A. (2013) Nonlinear ytem identification : NARMAX method in the time, frequency, and patio-temporal domain, John Wiley & Son Ltd. GUO, Y., GUO, L. Z., BILLINGS, S. A. & WEI, H. L. (2014) A new iterative orthogonal forward regreion algorithm. Intern. J. Syt. Sci., ubmitted. HIRSCH, M. D. R., LUIGI (2010) Iterative Identification of Polynomial NARX Model for Complex Multi- Input Sytem. IN MARCONI, L. (Ed.) 8th IFAC Sympoium on Nonlinear Control Sytem. Univerity of Bologna, Italy. HJALMARSSON, H. & MARTENSSON, J. (2007) Optimal Input Deign for Identification of Non-linear Sytem: Learning From the Linear Cae. American Control Conference, ACC '07. LARSSON, C. A., HJALMARSSON, H. & ROJAS, C. R. (2010) On optimal input deign for nonlinear FIRtype ytem. 49th IEEE Conference on Deciion and Control (CDC). LEONTARITIS, I. J. & BILLINGS, S. A. (1985) Input-output parametric model for non-linear ytem Part I: determinitic non-linear ytem. International Journal of Control, 41, LJUNG, L. (1987) Sytem Identification: Theory for the Uer, Englewood Cliff, N.J., Prentice-Hall, Inc. MAO, K. Z. & BILLINGS, S. A. (1997) Algorithm for minimal model tructure detection in nonlinear dynamic ytem identification. International Journal of Control, 68, NARENDRA, K. S. & ANNASWAMY, A. M. (1984) Peritent Excitation in Dynamical Sytem. American Control Conference, NOWAK, R. (2002) Nonlinear ytem identification. Circuit, Sytem and Signal Proceing, 21, PIRODDI, L. & SPINELLI, W. (2003) An identification algorithm for polynomial NARX model baed on imulation error minimization. International Journal of Control, 76, S DERSTR M T Sytem Identification, New York; London, Prentice Hall. WEI, H.-L. & BILLINGS, S. A. (2008) Model tructure election uing an integrated forward orthogonal earch algorithm aited by quared correlation and mutual information. International Journal of Modelling, Identification and Control, 3,