A Neuro-Fuzzy System on System Modeling and Its. Application on Character Recognition

Transcription

1 A Neuro-Fuzzy System on System Modelng and Its Applcaton on Character Recognton C. J. Chen 1, S. M. Yang 2, Z. C. Wang 3 1 Department of Avaton Servce Management Alethea Unversty Tawan, ROC 2,3 Department of Aeronautcs and Astronautcs Natonal Cheng Kung Unversty Tawan, ROC Keywords: system dentfcaton, neuro-fuzzy system, Takag-Sugeno fuzzy system, Mamdan fuzzy system. Abstract It has been known that fuzzy system provde a framework to handle uncertantes and vagueness, however, the Sugeno or Mamdan fuzzy rule often face dffcultes n decdng the number of nference rules and nput/output membershp functons. A fve-layer neuro-fuzzy model s developed for applcatons n system dentfcaton of engneerng system. Smulaton and analyss show that both Sugeno and Mamdan neuro-fuzzy models have good performance n system dentfcaton. A benchmark test s appled to valdate the model accuracy n nonlnear system modelng, and the former s superor n dentfcaton accuracy. Therefore, an experment of pattern recognton s taken by way of the Sugeno fuzzy rules n neuro-fuzzy system, and t consequently gets good works n the applcaton. 321

2 1. Abstract It has been known that fuzzy system provde a framework to handle uncertantes and vagueness, however, the Sugeno or Mamdan fuzzy rule often face dffcultes n decdng the number of nference rules and nput/output membershp functons. A fve-layer neuro-fuzzy model s developed for applcatons n system dentfcaton of engneerng system. Smulaton and analyss show that both Sugeno and Mamdan neuro-fuzzy models have good performance n system dentfcaton. A benchmark test s appled to valdate the model accuracy n nonlnear system modelng, and the former s superor n dentfcaton accuracy. Therefore, an experment of pattern recognton s taken by way of the Sugeno fuzzy rules n neuro-fuzzy system, and t consequently gets good works n the applcaton. 2. Introducton There have been sgnfcant developments n the feld of neural networks and fuzzy system. A neural network s a data processng system that mmcs the bologcal neural network by usng numerous artfcal neurons n order to acqure, optmze and store expermental statstcs. The challenges are the mappng rules nvsble and have lttle physcal meanng, and the tranng s often tme consumng. A fuzzy system s dstngushed by ts ablty of handlng numercal data and naccurate lngustc nformaton. Its performance strongly depends on the selecton of nput/output membershp functons and fuzzy logc rules. The rules and membershp functons are determned by expert s knowledge or experences; however, such decson makng may not be easly transferable. The ntegraton of neural networks and fuzzy system that combnes ther advantages and overcomes ther shortcomngs has been preferable n uncertan and complcated systems. Ln and Cunnngham [1] presented a fuzzy-neuro system modelng. Azam and Vanlandngham [2] also developed an adaptve self-organzng 322

3 neuro-fuzzy technque for system dentfcaton. Palt [3] developed a tranng algorthm for Takag-Sugeno of neuro-fuzzy network. Efe and Kaynak [4] proposed a neuro-fuzzy approach for dentfcaton and control of nonlnear systems. Km and Vachtsevanos [5] also presented a polynomal fuzzy neural network for dentfcaton and control. Fuzzy rule-based system dentfcaton has also been reported by Chakraborty and Pal [6]. Recently, Gorrosteta and Pedraza [7] developed a neuro fuzzy modelng of control systems. Zaheeruddn and Garma [8] proposed a neuro-fuzzy approach for predcton of work effcency. Cheng [9] appled a hybrd learnng-based neuro-fuzzy nference system for system modelng. Banakar and Azeem [10] also presented an dentfcaton and predcton of nonlnear dynamcal plants usng TSK and wavelet neuro-fuzzy models. Yang, et al [11] proposed a development of a self-organzed neuro-fuzzy model for system dentfcaton. Reference [3], [4], [7], [8], [9], [10] and [13] are concerned about the Sugeno fuzzy rules n the neuro-fuzzy system, whle reference [5], [6], [11] and [12] are assocated wth Mamdan fuzzy rules. However, the studes above are concerned wth the system dentfcaton by neuro-fuzzy approach, usng Sugeno or Mamdan fuzzy rules ndvdually. In ths paper, a performance comparson usng the Sugeno and the Mamdan fuzzy rules n the neuro-fuzzy system s conducted. 3. Neuro-fuzzy Model An artfcal neural network can learn complex functonal relatons by generalzng from a lmted tranng data, and thus can serve as a black-box of nonlnear dynamc systems. There are several layers of smple processng elements called the neurons, and the nterconnectons among them by weghts. The fuzzy system s a framework based on the concept of f-then rules to cope wth uncertan and ambguous problems. Two of most commonly used fuzzy nference systems are Mamdan fuzzy model [12] and Sugeno fuzzy model [13]. The former descrbes a 323

4 system by usng the natural language that makes t more ntutve and easy to realze, whle the latter specfes a system by usng the mathematcal relatons that makes t preferable to optmzaton and adapton. 3.1 Mamdan fuzzy rules A neuro-fuzzy system wth fve-layer feed-forward and the fuzzy nference of Mamdan fuzzy model s developed as shown n Fg. 1, whle the frst layer and the least layer s called the nput and output layer, respectvely. The other layers between nput and output layer are named the hdden layers. By determnng the fuzzy logc rules and optmzng the membershp functons through the connectve weghts, a vald neuro-fuzzy system s establshed. Layer 1 defnes the nput nodes and layer 5 ndcates the output nodes. Layer 2 and layer 4 are the term nodes of membershp functon to express the lngustc terms. Layer 3 defnes the nodes representng the fuzzy rules. A seres-parallel dentfcaton model [14] for nonlnear system can be wrtten as $ y( k + 1) = f ( y( k), y( k 1),..., y( k n + 1); u( k), u( k 1),..., u( k m + 1)) (1) where $ y( k + 1) s the estmated output of the neuro-fuzzy model at tme step k+1, [u(k), y(k)] represents the nput-output par f the plant at tme k, and n and m are the maxmum lags n the nput and output respectvely. Equaton (1) ndcates that $ y( k + 1) s a functon of the past n values of the plant output y ( k ), = 0,1 n-1, and the past m values of the nput u( k j ), j = 0,1 m-1. Each node n the frst layer s an nput node n proporton to one nput varable, and there s no computaton n ths layer. Each node drectly transmts sgnals to the next layer, O = 1 x, where O 1 s the output value of the th node n layer 1, and x s the th nput varable. Fuzzfcaton s done n the second layer wth each node correspondng to one 324

5 lngustc term of the nput varables va 2 O 1 m j 2 O 2 = exp j to calculate the σ j 2 membershp value of the fuzzy sets, where m j 2 and σ j 2 are the center (mean) and wdth (varance) of the Gaussan membershp functon of the j th node n layer 2, respectvely. Each node n layer 3 represents ts fuzzy rule whch has the form, R : If x s A and x s A... and x s A, then y s B (2) m m where R denotes the th fuzzy rule, x j, j = 1, 2 m s the lngustc varable of nput, y s the lngustc varable of output of the fuzzy rule R, and A 1, A 2, A m and B are the values of membershp functons. The weght of the lnks are set to unty and the output of node j determned by fuzzy AND operaton, O j3 = mn( O 2), where I j s the set of ndces of the nodes n layer 2 connected to node j n layer 3. Layer 4 represents the consequent part of the fuzzy rules and the performance of fuzzy OR 2 operaton on each node whch can be wrtten as 4 = max ( j3( j ) ) 325 j I I j O O w, where I s the set of ndces of the nodes n layer 3 that are connected to node n layer 4. Weght wj expresses the assocaton between the j th rule and the th output lngustc varable. The nodes of layer 3 and layer 4 are fully connected. Note that the strength of the IF-part and THEN-part rules presented by the node j n layer 3 and node n layer 4 are postve nvarably. The last layer computes the output value of the neuro-fuzzy model va the center of area defuzzfcaton, = ( σ ) / ( σ ) O m O O (3) 5 j 4 j 4 j 4 j 4 j 4 j I j I where I s the set of ndces of the nodes n layer 4 connected to node n layer 5. m j 4 and σ j 4 are the center (mean) and wdth (varance) of the membershp functon of the j th node n layer 4, respectvely. The weght of the lnks between the layer 4 and 5 are set to unty.

6 The desgn of the neuro-fuzzy system s by a three-phase learnng process to locate the ntal membershp functons n phase 1, fnd the fuzzy rules n phase 2, and tune the membershp functons of the nput/output varables n phase 3. In phase 1, the cneter and the wdth of the ntal membershp functon are determned by the feature-map algorthm [15]. x( k) m ( k) = mn{ x( k) m ( k ) } (4) c m ( k + 1) = m ( k) + α ( x( k) m ( k )) (5) c c c m ( k + 1) = m ( k) for m m (6) c where x( k ) and m ( k ) are the nput data and the center of membershp functon, respectvely. The subscrpt c ndcates the assocatve closet value and α s a decreasng scalar learnng rate. Ths adaptve formula runs ndependently for each nput and output lngustc varables. Once m ( k ) s calculated, the wdth σ ( k ) can be determned by the frst-nearest-neghbor heurstc, σ = ( m m ) / r (7) c where r s the overlap parameter. After the membershp functons of σ and m have been calculated, the backpropagaton learnng algorthm s added to fnd the fuzzy rules n phase 2. The output of layer 2 s transmtted to layer 3 to fnd the frng strength of each rule node. Based on the frng strength and the node output n layer 4, the correct consequence-lnk for each node can be determned by usng error backpropagaton learnng method wth the am of mnmzng the error functon = 2 E ( d( k) y( k )) / 2, where d(k) s the desred output and y(k) s the current output. The weght of the lnks from layer 3 to 4 s tuned va the update rule, w ( k + 1) = w ( k) + w ( k ) (8) j j j m 4( σ 4O 4) ( m 4σ 4O 4) where wj ( k) = η( d( k) y( k)) O j3wjσ 4 f j = $ r 2, and ( σ O )

7 w ( k ) = 0, otherwse. j $ r O w and η s the learnng rate. By 2 = Arg max( j3( j ) ) j adjustng the weght, the correct consequent lnk of each rule node s determned, and for every antecedent clause, the centrod of all the possble consequent s calculated. Only the domnant rule whose consequent has the hghest membershp value s selected. After the fuzzy rules have been deduced, a supervsed learnng s appled to tune the membershp functons optmally n phase 3. Intally, the lnks between rule nodes n layer 3 and consequent nodes n layer 4 are fully connected, for the consequence of rule nodes are not decded yet. Startng at the output node, a backward pass s used to compute the gradent of the error functon for all hdden nodes. In layer 5, the center and the wdth of each Gaussan membershp functon are the adjustable parameters. The error propagated to the proceedng layer s δ ( k) = d( k) y( k ) (9) 5 By usng Equaton (3) and the gradent of center m 4, the center parameter s updated va ( 1) ( ) η( ( ) ( )) m 4 k + = m 4 k + d k y k Smlarly, the wdth parameter s σ O / σ O (10) σ ( k + 1) = σ ( k) + η( d( k) y( k)) O 4 4 m ( σ O ) ( m σ O ) ( σ4o 4) (11) The error sgnal n layer 4 s derved as δ ( k) = ( d( k) y( k)) σ 4 4 m ( σ O ) ( m σ O ) ( 2 σ 4O 4) (12) By the same token, only the error sgnal δ 3 s needed and t s dentcal to δ 4 n layer 3. In layer 2, the center and wdth parameter are renovated by ( O m ) m ( k + 1) = m ( k) 2O η q (13) k ( σ 2) k 327

8 ( O m ) σ k σ k O η q (14) ( + 1) = 2( ) k ( σ 2) k where q k = 1 when O 2 = mn(nput of the k th rule node) and q k = 0 for the others. The above learnng algorthm hghlghts the computaton procedures n the neuro-fuzzy model. 3.2 Sugeno fuzzy rules The structure of a neuro-fuzzy system usng Sugeno fuzzy model s smlar to Fg.1 except the fuzzy rules and the defuzzfcaton as shown n Fg. 2. The rules of the Sugeno fuzzy model are descrbed as Ru : If x1 s Au 1 and x2 s Au 2... and p up, then u = u0 + u up p x s A y c c x c x (15) where R u sgnfes the uth fuzzy rule, x v, v =1, 2 p, are the p nputs to the system, y u s the output consequent of the fuzzy rule R u. A u1, A u2,, A up are the varance parameters of membershp functons, and c u0, c u1 c up are the real parameters. In layer 3, the weghts are calculated by takng the average of the ndvdual rule s contrbutons and then transmtted to the next layer, and the layer output s Ov 3 = wu 2 = wu 2 / w u2 (16) Each node n layer 4 s a square mode wth a mode functon O 4 = O 3 y, and v v u the output value as the summaton of all ncomng sgnals by weghted average, O w y w (17) 5 = / v u u u v v Because of the weghted average n calculatng the output of Sugeno neuro-fuzzy model, no membershp functon tunng s requred n ths layer. The neuro-fuzzy system s by a two-phase learnng process. In the frst phase, the antecedent parameters are fxed, and the consequent parameters of fuzzy rules are updated by 328

9 usng gradent descent algorthm smlar to phase 2 n the Mamdan neuro-fuzzy model. Intally, the nput data s fed nto the system and the frng strength of fuzzy rules are computed by Gaussan functon. The weght vector of each frng rule T c = [ c, c,..., c ] s updated accordng to u u0 u1 up T c ( k + 1) = c ( k) + g ( k)α [ y ( k) y( k)][1, x ( k)] (18) u u uo u u where guo( k ) s the decreasng rate, 0 gu0( t ) < 1, and α u s the frng strength of the rule u. The antecedent parameters are tuned by wn = mn{ wn }, and j j wn j mn{ srmj, srnj } max{ slmj, slnj} = c c mj nj (19) where c mj and c nj are the center of the wnner rule and the frst runner-up respectvely. Smlarly, sr mj and sl mj are the rght and left spreads of the wnner fuzzy rules, sr nj and sl nj are the rght and left spreads of the runner-up rules. wn j s the relatve wdth of the wndow and the dmenson wn has the smallest relatve wdth. The spread s rv then s renovated va s ( k + 1) = s ( k) + η( k)( c ( k) s ( k)), sgn( y y ) = sgn( y y ) (20) rv rv rv rv u r l s ( k + 1) = s ( k) η( k)( c ( k) s ( k )), otherwse (21) rv rv rv rv where c rv s the center of wnner rule, η s the learnng rate, y r and y l are the output computed ndependently for each rule. The centers of the fuzzy sets are updated when only a normal fuzzy rule fres. The center c r s moved towards the nput sample x( k) accordng to c ( k + 1) = c ( k) + η( k) α ( k)[ x( k) c ( k )] (22) r r r r The fve-layer networks n Mamdan and Sugeno fuzzy models are to compare ther performance n system dentfcaton. 329

10 4. Modelng and Applcaton It has been proved that the desred accuracy of a nonlnear mappng can be obtaned by nuero-fuzzy systems. A second-order hghly nonlnear dfference equaton s employed to llustrate the neuro-fuzzy model applcatons. y( k 1) y( k 2)( y( k 1) + 2.5) y( k) = + u( k) y( k 1) + y( k 2) The neuro-fuzzy model has three nputs: u( k ), y( k 1), y( k 2) and one output: y( k ). The frst two nputs are parttoned nto seven lngustc spaces {NL, NM, NS, ZE, PS, PM, PL} for ndcatng the negatve large, negatve medum, negatve small, zero, postve small, postve medum, and postve large respectvely, and the nput y( k 2) s parttoned nto fve lngustc spaces {NL, NS, ZE, PS, PL}. The overlap parameter n Eq. (7) s set at r=1.5, the decreasng rate g ( k ) n Eq. (18) s set at 0.9, and the ntal learnng rate s set at η( k ) = Fgure 3 and 4 show the resultant membershp functons of the nput u( k ), y( k 1), and y( k 2) of the neuro-fuzzy model wth Sugeno and Mamdan fuzzy rules. The membershp functons of Segeno fuzzy rules are adjusted by the weghted average, and the way of defuzzfcaton n layer 5 s by the calculaton of lnear relatons. Compared to the Mamdan neuro-fuzzy model, the membershp functons of Sugeno neuro-fuzzy model are not as complcated n calculaton. The performance of the two neuro-fuzzy models are tested by 200 data ponts acqured from the snusod nput sgnal u( k) = sn(2 π k / 25). The outputs of both models are shown n Fg. 5(a) and 3(b). Both takng 300 epochs n the smulaton, and t can be seen that both models perform well n dentfcaton. However, the output of the Sugeno neuro-fuzzy model s so close to the benchmark and has smaller dscrepancy. The dfference between the output of plant and output of neuro-fuzzy models s shown n Fg. 6. It can be seen clearly that the dscrepancy n Mamdan neuro-fuzzy model s hgher than n Sugeno uo (23) 330

11 neuro-fuzzy model, and the later model s adopted as the tool for the followng experment. As for the applcaton, the pattern recognton s taken nto consderaton. Fgure 7 (a) wth 800*600 pxels s vewed as the sample book for the experment, whle t s cut by 207 elements and stored nto a matrx for the desred output, whle the black and whte ponts represent +1 and 0 respectvely. Here some randomly nose ponts are added nto the matrx as the materal of tranng data. After tranng 300 epochs, the result of smulaton s shown as Fg. 7 (b) and (c), whle the former llustrates the correspondng cut elements between desred (red lne) and modelng (blue lne) output, and the later s the mage after smulatng. By the Same token, the lcense plate whch s cut by 539 elements s taken as another example for the experment, and the sample and smulatons are shown n Fg. 8 (a), (b) and (c). It can be clearly seen that t works well and correctly n the pattern recognton. 5. Concluson A performance comparson of the same case between the Sugeno and the Mamdan fuzzy rules n the neuro-fuzzy model whch has a structure of fve-layer s proposed. Both models have the ablty of dentfyng the fuzzy rules and tunng the membershp functons automatcally for the system dentfcaton. The crteron test on a nonlnear system wth three nputs and one output reveals that the membershp functons can be adjusted sensbly and optmally. Instead of needng experts experences and knowledge, the fve-layer neuro-fuzzy n Sugeno and Mamdan fuzzy models can be constructed just from the nput-output tranng data. Fgure 6 shows the error between the two models, the error of Sugeno neuro-fuzzy model s almost equal to zero and the error of Mamdan neuro-fuzzy model has the phenomenon of vbraton. Compared to the dfference by Fg. 5 (a) and (b), both models have good performance n system dentfcaton. However, the error of Sugeno neuro-fuzzy model s smaller than Mamdan neuro-fuzzy model, and the former s closer to the 331

12 benchmark and has a good approach. From Fg. 7 (c) and Fg. 8 (c), on the other hand, the applcaton by the Sugeno neuro-fuzzy model can recognze the mages successfully. References [1] Ln, Y. and Cunnngham, G. A., A New Approach to Fuzzy-Neural System Modelng, IEEE Trans. on Fuzzy Syst., Vol. 3 No. 2, pp , [2] Azam, F. and Vanlandngham, H. F., Adaptve Self Organzng Feature Map Neuro-Fuzzy Technque for Dynamcs System Identfcaton, IEEE Int. Sym. on Intellgent Control - Proceedngs, pp , [3] Palt, A. K., Effcent Tranng Algorthm for Takag- Sugeno Type Neuro-Fuzzy Network, IEEE Int. Conf. on Fuzzy Systems, Vol. 3, pp , [4] Efe, M. O. and Kaynak, O., Neuro-Fuzzy Approach for Identfcaton and Control of Nonlnear Systems, IEEE Int. Sym. on Industral Electroncs, Vol. 1, pp. TU2-TU11, [5] Km, S. and Vachtsevanos, G. A., A Polynomal Fuzzy Neural Network for Identfcaton and Control, Ann. Conf. on New Fronters n Fuzzy Logc and Soft Computng, pp. 5-9, [6] Chakraborty, D. and Pal, N. R., Integrated Feature Analyss and Fuzzy Rule-Based System Identfcaton n a Neuro-Fuzzy Paradgm, IEEE Trans. on Systems, Man, and Cybernetcs, Vol. 31, No. 3, pp , [7] Gorrosteta, E. and Pedraza, C., Neuro Fuzzy Modelng of Control Systems, IEEE Conf. on Electroncs, Communcatons and Computers, pp. 23, [8] Zaheeruddn and Garma, A Neuro-Fuzzy Approach for Predcton of Human Work Effcency n Nosy Envronment, Appled Soft Computng Journal, Vol. 6, No. 3, pp , [9] Cheng, K. H., Hybrd Learnng-Based Neuro-Fuzzy Inference System: A New Approach for System Modelng, Int. J. Systems Scence, Vol. 39, No. 6, pp , [10] Banakar, A. and Azeem, M. F., Identfcaton and Predcton of Nonlnear Dynamcal Plants Usng TSK and Wavelet Neuro-Fuzzy Models, IEEE Conf. 332

13 on Intellgent Systems, pp , [11] Yang, S. M., Chen, C. J., Chang, Y. Y., and Tung, Y. Z., Development of a Self-Organzed Neuro-Fuzzy Model for System Identfcaton, ASME Trans. J. on Vbraton and Acoustcs, Vol. 129, No. 4, pp , [12] Mamdan, E. H. and Asslan, S., Experment n Lngustc Synthess wth a Fuzzy Logc Controller, Int. J. on Man-Machne Studes, Vol. 7, No. 1, pp. 1-13, [13] Takag, T. and Sugeno, M., Fuzzy Identfcaton of Systems and Its Applcatons to Modelng and Control, IEEE Trans. on Systems, Man and Cybernetcs, Vol. SMC-15, No. 1, pp , [14] Narendra, K. S., and Parthasarathy, K., Identfcaton and Control of Dynamc Systems Usng Neural Networks, IEEE Trans. on Neural Networks, Vol. 1, pp. 4-27,

14 Layer 5 : Output node Layer 4 : Output Term Nodes Layer 3 : Rule Nodes Layer 2 : Input Term Nodes Layer 1 : Input Nodes Fg. 1 The structure of the neuro-fuzzy model wth Mamdan fuzzy rule. 334

15 Layer 5 Output Node Layer 4 Output Term Nodes Layer 3 Rule Nodes Layer 2 Input Term Nodes Layer 1 Input Nodes Fg. 2 The structure of the neuro-fuzzy model wth Sugeno fuzzy rule. 335

16 u(k) u(k) y(k-1) y(k-1) y(k-2) Fg. 3 The membershp functons of three nputs of Sugeno neuro-fuzzy model y(k-2) Fg. 4 The membershp functons of three nputsof Mamdan neuro-fuzzy model. 336

17 5 4 Plant Sugeno 3 Output Tme (a) 5 4 Plant Mamdan 3 Output Tme (b) Fg. 5 Modelng of the plant and the neuro-fuzzy system wth a snusod nput sgnal n (a)sugeno fuzzy rules and (b) Mamdan fuzzy rules. 337

18 Sugeno Mamdan 0.2 Error Tme Fg. 6 The dfference between output of plant and neuro-fuzzy model n Sugeno and Mamdan fuzzy rules. Fg. 7 (a) The mage of door plate (b) The result of smulaton (c) The mage after smulatng 338

19 Fg. 7 (a) The mage of lcense plate (b) The result of smulaton (c) The mage after smulatng 339

20 340