Nonlinear Time Series Prediction Based on Lyapunov Theory-Based Fuzzy Neural Network and Multiobjective Genetic Algorithm

Transcription

1 Nonlnear Te Seres Predcton Based on Lyapunov Theory-Based Fuzzy Neural Network and Multobectve Genetc Algorth Kah Phoo Seng and Ka Mng Tse School of Engneerng & Scence, Monash Unversty (Malaysa), Bandar Sunway, 4650 PJ, Malaysa School of Mcroelectroncs, Grffth Unversty Kessels Rd, Nathan LD 4, Australa Abstract. Ths paper presents the nonlnear te seres predcton usng Lyapunov theory-based fuzzy neural network and ult-obectve genetc algorth (MOGA). The archtecture eploys fuzzy neural network (FNN) structure and the tunng of the paraeters of FNN usng the cobnaton of the MOGA and the odfed Lyapunov theory-based adaptve flterng algorth (LAF). The proposed schee has been used for a wde range of applcatons n the doan of te seres predcton. An applcaton exaple on sunspot predcton s gven to show the erts of the proposed schee. Sulaton results not only deonstrate the advantage of the neuro-fuzzy approach but t also hghlghts the advantages of the fuson of MOGA and the odfed LAF. Introducton Te seres predcton s a very portant practcal applcaton wth a dverse range of applcatons ncludng econoc and busness plannng, nventory and producton control, weather forecastng, sgnal processng and control []. As a result, there has been consderable nterest n the applcaton of ntellgent technologes such as neural networks (NNs) and fuzzy logc []-[3]. More recently, these two coputatonally ntellgent technques have been vewed as copleentary leadng to developents n fusng the two technologes [4], wth a nuber of several successful neurafuzzy systes reported n the lterature [5]. Such work has deonstrated the superor predcton capabltes of a fuzzy neural network as copared to the conventonal neural network approach [5)-[6]. In ths paper, we eploy the fuzzy neural network (FNN) for nonlnear te seres predcton. A new cobnaton of the odfed Lyapunov theory-base flterng algorth (LAF) and ult-obectve genetc algorth (MOGA) [0] s used to adust the network paraeters. The proposed schee provdes not only the advantages of fuzzy T.D. Gedeon and L.C.C. Fung (Eds.): AI 003, LNAI 903, pp , 003. Sprnger-Verlag Berln Hedelberg 003

2 Nonlnear Te Seres Predcton 89 logc and NN but t also offer addtonal advantages offered by the odfed LAF and MOGA. The weght paraeters of FNN n the consequence part are adaptvely adusted by the odfed LAF. The MOGA s used to tune the paraeters of the ebershp functons (MBFs) n the prese part. Most real world probles requre the sultaneous optsaton of ultple crtera/obectves. In ths case, MOGA can provde the soluton to these probles. In the proposed schee, types of error: nstantaneous and a pror errors defned n later secton are the ultple crtera to be solved by MOGA. The proposed schee has been used for a wde range of applcatons n the doan of te seres predcton. The theoretcal predcton echans of the proposed schee s further confred by the sulaton exaple for real world data such as sunspot forecastng. The paper s organzed as follow: secton brefly descrbes the an features of the proposed FNN and the two crtera. Secton 3 presents the fuzzy neurel network learnng: structure learnng and paraeter learnng. The paraeter learnng the odfed LAF algorth s presented n secton 4. Secton 5 descrbes the MOGA and the transforaton of the ultobectve functon nto a new functon so that sngle obectve optzaton ethods can be used. The predcton results are presented n secton 6. The fnally secton 7 concludes the paper wth a dscusson of the sgnfcance of the results. Fuzzy Neural Network Neural-fuzzy systes have been appled to any felds successfully snce a decade ago. In ths secton, the fuzzy logc nference syste can be pleented as a fvelayer NN (Fg. ). Ths type of archtecture s the ost coon aong neural fuzzy nference systes. Gven the tranng nput data x n, n,... N, and the desred output d,... M, the nference rules of splfed fuzzy reasonng can be establshed by experts or generated based on nuercal data as proposed by Wan and Mendel []. The rule base contans the followng for: R : IF x s A and... x THEN y s w and... y N s A M N, s w M (.) where s a rule nuber, the A N 's are MBF's of the antecedent part and w M's are real nubers of the consequent part. The operaton of the ths syste can be descrbed layer by layer as follows: Layer : Fuzzfcaton Ths layer conssts of lngustc varables. The crsp nputs x n, n,... N are fuzzfed by usng MBFs of the lngustc varables A N. Usually, trangular, trapezod, Gaussan or bell-shaped ebershp functons are used.

3 89 Kah Phoo Seng and Ka Mng Tse Layer. Rule Nodes The second layer contans one node per each fuzzy f-then rule. Each rule node perfors connectve operaton between rule antecedents (f-part). Usually, the nurn or the dot product s used as ntersecton AND. The unon OR s usually done usng axu operaton. In our exaple case the frng strengths, of the fuzzy rules are coputed accordng to A x ) A ( x )... A ( x ) (.) ( N N Layers 3-5:Noralzaton, Consequence & Suaton In the thrd layer, the frng strengths of the fuzzy rules are noralzed. Layer 4 s related to consequent fuzzy labels whch are sngletons n our case. The values of the sngletons are ultpled by noralzed frng strength. The fnal layer coputes the overall output as the suaton of the ncong sgnals. Therefore the output y of the fuzzy reasonng can be represented by the followng equaton: y w (.3) Y [y, y,..., y M ] (.4) After the fuzzy logc rules and network structure have been establshed, the learnng algorth can then appled to adust the paraeters of the MBFs n the prese part and the weghts n the consequence parts. In ths paper, we proposed to use the odfed LAF algorth to adaptvely adust the weghts n the consequence parts and MOGA to tune the paraeters of MBFs. Fg.. The Fuzzy Neural Network Fg. llustrates the overall process of the proposed schee for the predcton proble. The layer 5 conssts of suaton node or output, y (t) whch s defned as

4 Nonlnear Te Seres Predcton 893 y ( t ) w ( t ) (.5) y (t) s not another output node of FNN as shown n Fg. and t s only coputed usng (.6) y ( t ) w ( t ) (.6) Fg.. The block dagra of FNN wth the odfed LAF + MOGA 3 Fuzzy Neural Network Learnng There are two aor learnng n fuzzy neural network: the structure learnng and the paraeter learnng. The structure learnng s to generate ntal fuzzy rules when there s no ntal fuzzy rule establshed by the experts. The ntal fuzzy rules can generated based on nuercal data. Wang and Mendal have proposed a sple and straghtforward algorth []. The an steps of the algorth are descrbed as follows: Frst, we have to deterne the nput and output varables and construct the nuercal data set. The k th nuercal data set can be fored as { x', x',..., x' n } { y', y',..., y' n } where the left hand sde and the rght hand sde specfy the nput and output nuercal data. Second, calculate the ebershp degree for each varable by a row vector denoted by x (t) for nputs and y (t) for outputs as

5 894 Kah Phoo Seng and Ka Mng Tse x y [ A [ B, A, B,..., A N,..., B M ] ] (3.) (3.) N M A, A,..., A ( t ) and B, B ( t ),..., B ( t ) are the ebershp degrees for x' (t) and y' (t). Thrd, calculate the portance degree to each data par as n h ax{ x } (3.3) Fourth, construct the fuzzy rules for all data pars. For nstance, a fuzzy rule has ths for: ( ) R t : IF x s A and. x s A THEN y s B and y s B (3.4) Ffth, delete the conflct fuzzy rules, whle two rules have the sae fuzzy set n IF part but a dfferent set n THEN part. The proper rule s selected accordng to the hghest portance degree. 4 Paraeter Learnng Algorth The Modfed LAF The LAF algorth n [7] has been odfed to adaptvely adusts the weghts of FNN n the consequence part. Those weghts n the consequence part are updated as follow: w w ( t ) + g α (4.) where g (t) s the adaptaton gan and α (t) s defned as w ( t ) α d (4.) The adaptaton s gven by g t) U ( e ( t ) k α (4.3) where 0 < k. U(t) [,,..., ]. It s notceable that the values of U(t) and α n (4.3) ay be zero and rse sngulartes proble. Therefore the adaptaton gan ay be odfed as (4.4)

6 Nonlnear Te Seres Predcton 895 g e ( t ) k U( t) + λ α ( t) + λ (4.4) where 0 < k, and λ, λ are sall postve nubers. 5 Paraeter Learnng Algorth - MOGA The MOGA s used to tune the paraeters of the ebershp functons (MBFs) n the prese part. Wthout the need of lnearly cobnng ultple attrbutes nto a coposte scalar obectve functon, evolutonary algorths ncorporate the concept of Pareto's optalty or odfed selecton schees to evolve a faly of solutons along the tradeoff surface. The anner of Parento optalty s one of the useful approaches to solve utlobectve optzaton probles. In ths paper we eploy the weghted su-based optzaton ethod. 5. Weghted Su Based Optzaton In a weghted su-based optzaton, ultobectve F(f,..., f ) s transfored nto F w k w f so that sngle obectve oprnzaton ethods can be used. Preferences are used for specfyng weghts, Wth reference to Fg., two ftness F, and F can be evaluated fro e l (t) and e (t) t. Thus, these two obectve functons s transfored n an overall ftness functon F w F + w F, where w + w. For exaple, we ay choose w 0.3 and w Coputatonal Algorths The procedure of the MOGA algorth s descrbed as follows:. Intalzaton: Tranng data s clustered to generate 9 centrods based on whch the Gaussan MBFs (ean and varance) are evaluated. 80 potental canddates P(t) are created by varyng ± 0% of the MBF s.. Evaluate the overall ftness F. Select canddates proportonal to ther ftness relatve to the others n P(t) usng the Stochastc Unversal Saplng technque. 3. Applyng genetc operators, whole arthetc: crossover, utaton, and adaptaton wth the best canddate by addng a perturbaton to the relatve best-ft canddate, to reproduce new canddates. 4. Cobne all new canddates wth the P(t) to for the new populaton for the next generaton. 5. Repeat step, 3, 4 untl ternaton condton s satsfed.

7 896 Kah Phoo Seng and Ka Mng Tse 6 Sulaton Exaple In order to deonstrate the perforance of the proposed ethod, we have appled the ethod to predcton of the sunspot data. Sunspot data s used as a benchark for any years by researchers. Data fle of the Sunspot tes seres s download fro [9]. It conssts the sunspot data fro the year 700 to 999 (300 Saples). Fg. 3 shows the plot of the sunspot te seres. Fg. 4 shows the ean squared error of e (t) gvng MSE0.059 at the 30th generaton. Fg. 3 show no dstnct dfference between the y (t) and x(t). Fg. 5 and 6 reveal soe weght paraeters of the FNN. A coputer progra developed n MATLAB software s used to pleent the proposed schee. Fg 3. Noralzed sunspot te seres & the predctor output Fg. 4. MSE of e (t)

8 Nonlnear Te Seres Predcton 897 Fg. 5. Weght paraeters of FNN Fg. 6. Weght para8eters of FNN 7 Concluson Ths paper has presented a new approach n desgnng a FNN wth MOGA and the odfed LAF technques.the prevous secton clearly deonstrate the perforance of the proposed FNN for the predcton of nonlnear te seres. The odfed LAF has provded the fast error convergence to the tranng of FNN. On the other hand, MOGA has added advantage of global optzaton to the FNN tranng based on two crtera/obectves. The results have ephaszed the benefts of the fuson of fuzzy and NN technologes as well as the advantages of the fuson of the odfed LAF and MOGA. Ths ncreases n transparency of the neurofuzzy approach overcoes the

9 898 Kah Phoo Seng and Ka Mng Tse drawback of FNN wth gradent technques and/or GA n the conventonal NNs or FNNs. In general the predcton capablty (accuracy) of ths syste s proportonal to ts granularty (the nuber of fuzzy sets) n the prese part and the nubers of weghts n the consequence part. Future works need to be conducted n ths area. Many ssues need to be addressed regardng sulatons, practcal pleentatons, and the further analyss on the theoretcal parts of the proposed schee. Reference [] Box, GEP; Jenkns, GM: "Te seres analyss: forecastng and control", Oakland CA: Holden-Day, 976. [] Wang, LX,- Mendel. YM: "Generatng fuzzy rules by leang fro exaples", IEEE Trans. Systes Man and Cybernetcs, Vol "'. No. 6, pp , 99. [3] Moody, J; Darken, C: "Fast learnng n networks of locally tuned processng unts", Neural Coputaton, pp. 8-94, Vol., 989~ [4] Brown, M- Harrs, C: "Neurofuzzy adaptve odellng and contro", Prentce- Hall, 994. [5] Jang, Roger JS: "Predctng chaotc te seres wth fuzzy IF-THEN rules" Proc. IEEE nd Int. Conf. on Fuzzy Systes, pp Vol., 993. [6] Jang, Roger J S; Sun, C-T: "Neuro-fuzzy odellng and Control" Proc. of the IEEE, pp , Vol 83, No 3 March 995. [7] Seng Kah Phoo, Zhhong Man, H.R. Wu, Lyapunov Theory-based Radal Bass Functon Networks for Adaptve Flterng, IEEE Transacton on Crcut and Syste, vol 49, no. 8, pp.5-, 00. [8] Slotne, J-J. E. and L, W. Appled. nonlnear control, Prentce-Hall, Englewood Clffs, NJ, 9 9. [9] [0] Deb, K., "Evolutonary Algorths for Mutl-Crteron Optzaton n Engneerng Desgn. Proceedngs of Evolutonary Algorths n Engneerng and Coputer Scence (EUROGEN-99),999. [] L. X. Wang and J. Mendel, Generatng fuzzy rules by learnng for exaples, IEEE transactons on systes, Man, Cybernetcs, vol., pp , 994.