Journa of heoretca and Apped Informaton echnoogy th February 3. Vo. 48 No. 5-3 JAI & LLS. A rghts reserved. ISSN: 99-8645 www.jatt.org E-ISSN: 87-395 NONLINEAR SYSEM IDENIFICAION BASE ON FW-LSSVM, XIANFANG WANG, YUANYUAN ZHANG, JIALE DONG, 3 ZHIYONG DU Coege of Computer & Informaton Engneerng, Henan Norma Unversty, Xnxang 4537, Chna Henan Provnce Coeges and Unverstes Engneerng echnoogy Research Center for Computng Integence & Data Mnng, Xnxang 4537, Henan, Chna 3 Henan Mechanca and Eectrca Engneerng Coege, Xnxang 453, Henan, Chna E-ma: xfwang@yahoo.com.cn ABSRAC hs paper proposed a method to dentfy nonnear systems va the fuzzy weghted east squares support machne (FW-LSSVM. At frst, we descrbe the proposed modeng approach n deta and suggest a fast earnng scheme for ts tranng. Because the tranng sampe data of ndependent varabe and dependent varabe has a certan error, and we obtan the sampe whch has a certan fuzzness from measurng, nfuencng the accuracy of mode budng, ths paper woud put the concept of fuzzy membershp nto the east squares support vector machne. Usng the fuzzy weghted east squares agorthm for sampes, each sampe n the vector s ntroduced nto the fuzzy membershp degree, ths mprove the ant nose abty of east squares support vector machne. he effcency of the proposed agorthm was demonstrated by some smuaton exampes. Keywords: Mxed Kerne Functon, FW-LSSVM, Identfcaton,Nonnear Systems.. INRODUCION Most systems encountered n the rea word are nonnear n nature, and snce near modes cannot capture the rch dynamc behavor of mt cyces, bfurcatons, etc. assocated wth nonnear systems, t s mperatve to have dentfcaton technques whch are specfc for nonnear systems[-4]. SVM s empoyed to reaze some compex nonnear desred functons, whch s presented by Vapnk and based on the statstca earnng theory and earnng method has obtaned certan achevements. SVM has strong generazaton abty n sovng the sma sampe, nonnear, hgh dmenson, and oca mnmum of practca probems. SVM use a varety of kerne functon whch can be cassfed nto goba kerne functon and oca kerne functon [5]. Smts and Jordaan use the two kerne functon to make up a knd of mxed kerne functon through the research on two knds of representatve goba kerne functon (poynoma kerne functon and oca kerne functon (RBF kerne functon mappng propertes [6]. In addton, Suykens presents a east squares vector machne (LSSVM method based on SVM. LS-SVM transform the tradtona SVM nequaty constrants nto equaty constrants, n order to avod a quadratc programmng probem sovng tme-consumng, and LS-SVM reduce the run tme, and effectvey mprove the speed of earnng to sove at a arge extent. Athough the agorthm and the structure of the LS-SVM are smpe, and the cacuaton of LS-SVM s fast, t ost the oose and robustness of SVM n a certan extent. hs paper overcome the adverse effects of the tranng sampe nose by seectng the fuzzy weghted east squares support vector machne agorthm and puttng fuzzy weghted membershp on the squared error of each sampe [7-9]. Seectng LS-SVM whch s constructed wth mxed kerne functon not ony has a good earnng abty but aso has better generazaton abty. Smuaton resuts show that the fuzzy weghted east squares support vector machnes hep to mprove the robustness of the mode and have a certan practcaty. Brefy, the paper s organzed as foows: Secton presents bref theory of east squares support vector machne(lssvm. Secton 3 deas wth the descrpton of the fuzzy weght LSSVM. Secton 4 provdes an ustratve exampe to dentfy for the nonnear systems. Fnay, some concusons are obtaned n Secton 5. 336
Journa of heoretca and Apped Informaton echnoogy th February 3. Vo. 48 No. 5-3 JAI & LLS. A rghts reserved. ISSN: 99-8645 www.jatt.org E-ISSN: 87-395. HEORY OF LSSVM LSSVM(Least squares support vector machne [5] fow the prncpe of the structura rsk mnmzaton of the SVM agorthm n tranng process, and the nequaty constrants of the objectve functon n SVM s transformed nto the equaty constrant. Seect a nonnear mappng and nput vector s mapped from the orgna space to a hgh dmensona feature space. And the optma decson functon s constructed by usng the prncpe of the structura rsk mnmzaton n the hgh dmensona feature space. Dot product operaton n hgh-dmensona feature space s repaced by the kerne functon of the orgna space, and the frst power of the emprca rsk s repaced by the square n ths procedure. he nsenstve oss functon s avoded, and the compexty s greaty reduced and that the obtaned externa souton whch s a goba optma souton s ensured. he artce assumes that the sampe tranng set s gven. he sampe tranng set s n {( x, y, ( x, y L ( x, y } R R, and x s the th sampe of the nput mode and y s the th the desred output of the th sampe, and s the tota number for the tranng sampe. he sampes are mapped to the hgh-dmensona feature n space from the orgna space R by usng the nonnear mappng ϕ(. In the feature space, the mode of east square support vector machne s y( x = w ϕ ( x + b ( where ϕ( x s the feature mappng and w represents the weght vector, and b s the offset vector. Based on the structura rsk mnmzaton the recursve mode of LS-SVM for sovng the functon ( s γ mn J ( w, e = w w + ξ ( = s.t. y = w ϕ ( x + b + ξ where γ s caed the adjustabe constant and γ >, ξ s the error term, and ξ. he correspondng Lagrangan functon for ( s as foows γ L( w, b, ξ, α = w + ξ = (3 = { w ( x b y } α ϕ + + ξ where α s Lagrange mutper and [, L, ] α = α α α. Based on KK whch s the optma condtons, the near equatons are obtaned after emnate ξ and w. L = w = αϕ( x w = L = α = b = (4 L = α = γξ ξ L = y = w ϕ ( x + b + ξ α hen the optmzaton probem s transformed nto the foowng near equaton: L b K( x, x + L K( x, x y γ α = M M L M M M K( x, (, x K x x α + y L γ (5 where K ( x, ( k x = ϕ x k ϕ ( x s the kerne functon. Lnear kerne, poynoma kerne and RBF kerne are the common kerne functons. In ths paper, the mxture kernes whch consst of poynoma kerne and RBF kerne are used. he expresson s as foow mx = ρ + + ρ σ K [( x x ] ( exp( x x / (6 where ρ ( ρ s the constant to reguate the effect of the two kerne functons. It s mportant to seect the parameters ρ, σ and the penaty-factor C. In order to get the mode that has the predcted effect, t s necessary to optmze and adjust the parameters [-]. 3. FW- LSSVM LSSVM mproves the norma SVM, but at the same tme t oses the robustness by tsef, and t doesn t ft the fact that the tranng data has the same affecton n the objectve functon. Because the sampe data have dfferent postons, the extent of nfuencng by the nose and the error exsts that have dfferent and mportant nfuence n the measure. So for acqurng a better robustness, each sampe data n the vector s ntroduced nto the fuzzy membershp degree[]. he objectve functon ( s repaced by the foowng functon. 337
Journa of heoretca and Apped Informaton echnoogy th February 3. Vo. 48 No. 5-3 JAI & LLS. A rghts reserved. ISSN: 99-8645 www.jatt.org E-ISSN: 87-395 γ mn J ( w, e = w w + µ e (7 = he correspondng Lagrangan functon for Equaton (7 s as foows: γ L ( w, b, ξ, α = w + µ ξ = (8 = { w ( x b y } α ϕ + + ξ u(k DL u(k-m+... u(k- Nonnear system FW- LSSVM y m(k+ ym(k-n+... y m(k- Z - DL y m(k _ + y(k e(k he near equatons are obtaned for the functon (8: L = w = α ϕ( x w = L = α = (9 b = L = α = γ µ ξ ξ L = y = w ϕ ( x + b + ξ α hen the optmzaton probem of the functon (9 s transformed nto the foowng near equaton: L b K( x, x + L K( x, x γµ α y = ( M M L M M M K( x, (, x K x x α + y L γµ he parameters α and b can be obtaned by the Equaton(, and seectng the weghted coeffcent µ s based on the error ξ = α / γ of the LS-SVM. 4. IDENIFICAION VIA FW-LSSVM Consder the nonnear system y ( k = f ( y ( k, y ( k, L, y ( k n, u ( k, u ( k, L, u ( k m ( where u(k s the system nput, y(k the system output; where m and n are respectvey the deays of u(k and y(k. f(. s a nonnear functon. he dentfcaton of nonnear system based on FW-LSSVM mode s depcted n Fgure. y ( k = f ( y ( k, y ( k, L, y ( k n, M svm M M M u ( k, u ( k, L, u ( k m ( where y M ( k s the output of the dentfcaton mode, ( g s the obtaned mode by tranng FW-LSSVM. f S V M Fgure : he Dagram Identfcaton Of he Nonnear System Based On SVR he detaed process of ths dentfcaton can be descrbed as foowng: Step Chose the nput sgna u(t for the nonnear system Equaton(. Step Chose the sampng tme t s, sampng dots N and wdth of wndow s t = N * t s. Step 3 Intaed the nput and output, and obtaned the vaues of u(k and output y(k at every sampng tme k. Step 4 Choose the deays of nput m and output n ( m n. Construct a sampng set D = { X, Y}, where X ( k m = { y( k, y( k, L, y( k n, u( k, u( k, L, u( k m} Y ( k m = y( k k = ( m +, L, N Step 5 Chose the kerne functon K( x, x determne parameters of the FW-LSSVM (such as γ, µ Step 6 Computng the output Y M when the nput s, and the tranng error E = Y YM. Step 7 If the error E s ess than a threshod,the tranng s over, then go to the next step; ese return to Step 5. Step 8 he parameter vector w woud be obtaned by sovng quadratc reguaton n Equaton (9. Step 9 he mode s obtaned, the dentfcaton process of nonnear system s end. 5. HE SIMULAION EXPERIMEN In order to prove that the opnon put forward n the paper s effectveness, the nonnear system s regarded as the object to emuate. he persona j 338
Journa of heoretca and Apped Informaton echnoogy th February 3. Vo. 48 No. 5-3 JAI & LLS. A rghts reserved. ISSN: 99-8645 www.jatt.org E-ISSN: 87-395 computer and the software of Matab 7. are needed to accompsh the experment. he nonnear mode s as foow [3]: y( k + =.6* y( k /( + y( k* y( k (3 +.3* u( k* y( k +.8* u( k he data need to be traned before test the fuzzy weghted east squares support vector machne. he nput sampe of the tranng data are structured the array and every row of the array represents the nput vector. he tranng sampe s: X ( k = [ y ( k +, y ( k +, u ( k +, u ( k +, u ( k ] Y ( k = y ( k + 3 (4 In the mode, the sampng tme ts =.s, sampng dots N =, the wdth of wndow s. t = N * t = s. he mxture kerne functon s mx K = ρ[( x x + ] + ( ρ exp( x x / σ s used. he parameters of the kerne functon has the stronger nfuence, so the vaue of ρ s. through the experence. he output of the tranng data n the mode are compared wth the actua output unt the error s the mnmum. he vectors of σ and γ are ganed when the error s the mnmum. he vector of σ s.756 and the postve number γ s. Fgure s the output of the smuated nonnear system mode and the Fgure 3 s the output error of modeng. yout 3.5 3.5.5.5..4.6.8..4.6.8 e.35.3.5..5..5 -.5 -. -.5..4.6.8..4.6.8 Fgure 3: he ranng Error Curve Of When he Input Is A Random Sgna In the fuzzy weghted east squares support vector machne of the nonnear system, the random sgna ur ( k = rand(, unt step sgna, snusoda sgna.5 sn(π t +.5 and the square sgna.5 sgn(sn( π t +.5 are used to test the generazaton performance of the mode. he output error curves of the fuzzy weghted LS-SVM nonnear system dentfcaton mode are the fgure from Fgure 4 to Fgure 6. the error of the unt step sgna 8 x -3 7 6 5 4 3..4.6.8..4.6.8 Fgure 4: he Identfcaton Error Curve When he Input Is A Unt Step Sgna Fgure : he Output Of he ranng Mode 339
Journa of heoretca and Apped Informaton echnoogy th February 3. Vo. 48 No. 5-3 JAI & LLS. A rghts reserved. ISSN: 99-8645 www.jatt.org E-ISSN: 87-395 the error of the square sgna the error of the snusoda sgna.5.4.3.. -. -. -.3..4.6.8..4.6.8.35.3.5..5..5 -.5 Fgure 5: he Identfcaton Error Curve When he Input Is A Snusoda Sgna -...4.6.8..4.6.8 Fgure 6: he Identfcaton Error Curve When he Input Is A Square Sgna Fgure 3 to Fgure 6 ndcate that usng the fuzzy weghted LS-SVM avods the over-fttng reduces the ncompactness. he smuaton experment proves that the fuzzy weghted LS-SVM nonnear system has a better robust. 6. CONCLUSIONS In ths paper, we present an mprovement method to dentfy nonnear system. he agorthm s put the fuzzy weghted to use LSSVM. Based on that the sampes have dfferent and mportant nfuence n every mode of the productve process, fuzzy weghted agorthm s ntroduced. he nfuence of the abnorma tranng data are overcame and the over-fttng of the dentfcaton mode s avoded. he smuaton experment ndcates that the fuzzy weghted LS-SVM and the mxture kerne functon avods the agorthm convergent prematurey and mproves the fuzzy earnng precson and the generazaton abty. hen the robustness of the system s mproved and the fuzzy weghted LS-SVM has a better appcaton. he resuts demonstrate that the suggested method gves better error mnmzaton for nonnear system. ACKNOWLEDGEMENS hs work s supported by the Natona Natura Scence Foundaton of Chna (No.6737, the Scence and echnoogy Research Project of Henan Provnce(No.79, the Foundaton and Fronter echnoogy Research Programs of Henan Provnce(No.44387, the Innovaton aent Support Program of Henan Provnce Unverstes (No.HASI, the Doctora Started Project of Henan Norma Unversty(No.39 REFERENCES: [] Musa Ac, Musa H. Asya, Nonnear system dentfcaton va Laguerre network based fuzzy systems, Fuzzy Sets and Systems, Vo. 6, No. 4, 9, pp. 358-359. [] Johan Paduart, Leve Lauwers, Jan Swevers, Krs Smoders, Johan Schoukens, Rk Pnteon, Identfcaton of nonnear systems usng Poynoma Nonnear State Space modes, Automatca, Voume 46, No. 4,, pp. 647-656. [3] Areza Rahrooh, Scott Shepard, Identfcaton of nonnear systems usng NARMAX mode, Nonnear Anayss: heory, Methods & Appcatons, Vo. 7, No., 9, pp. e98- e. [4] Gregor Doanc, Stanko Strmčnk, Identfcaton of nonnear systems usng a pecewse-near Hammersten mode, Systems & Contro Letters, Vo. 54, No., 5, pp. 45-58. [5] Suykens J A K, Vandewae J. Least squares support vector machne cassfers, Neura Processng Letters, Vo. 9, No. 3, 999, pp. 93-3. [6] Mao Zh-ang, Lu Chun-bo, Pan Feng, Parameter Seecton and Appcaton of SVM wth Mxture Kernes Based on IPSO, Journa of Southern Yangtze Unversty, Vo. 8, No. 6, 9, pp. 63-634. [7] Zhang Yng, Su Hong-ye, Chu Jan, Soft sensor modeng based on fuzzy east squares support vector machnes, Contro and Decson, Vo., No. 6, 5, pp.6-64. [8] Zhang Za-fang, Chu Xue-nng, Cheng Hu, Aggregatng Mut-granuarty Decson 34
Journa of heoretca and Apped Informaton echnoogy th February 3. Vo. 48 No. 5-3 JAI & LLS. A rghts reserved. ISSN: 99-8645 www.jatt.org E-ISSN: 87-395 Makng Informaton Based on Fuzzy Weghted Least Squares Method, Journa of Shang Ha Jao ong Unversty. Vo. 43, No. 9, 9, pp. 377-38. [9] L Zheng, Song Bao-we, Mao Shao-yong, Fuzzy Weghted Lnear Regresson on Zero Faure Exponent Dstrbuton, Journa of System Smuaton, Vo. 7, No. 6, 5, pp.373-375. [] Xong We-, Wang Xao, Chen Mn-fang, Xu Bao-guo, Modeng for pencn fermentaton process based on weghted LS- SVM, CIESC Journa, Vo. 63, No. 9,, pp.93-98 [] Wu e-bn, Lu Yun-an, L Xn-jun, Yn Yong-sheng, KPCA-fuzzy Weghted LSSVM Based Predcton Method and ts Appcaton, Computer Measurement and Contro, Vo., No. 3,, pp.67-6. [] Guo Hu, Lu He-png, Wang Lng. Method for Seectng Parameters of Least Squares Support Vector Machnes and Appcaton, Journa of System Smuaton, Vo. 8, No. 7, 6, pp. 33-36. [3] Hu Lang-mou, Cao Ke-qang, Su Xn-bng, L Na, Hou Yan-yan, Research on Interna Mode Contro for Nonnear System Based on Weghted LS-SVM, Mechanca Scence and echnoogy, Vo. 3, No.,, pp. - 6. 34