Genetic Algorithm Based Recurrent Fuzzy Neural Network Modeling of Chemical Processes

Transcription

1 Journa of Universa Computer Siene, vo. 3, no. 9 (27), submitted: 2/6/6, aepted: 24//6, appeared: 28/9/7 J.UCS Geneti Agorithm Based Reurrent Fuzzy Neura Network odeing of Chemia Proesses Jii Tao, Ning Wang (Nationa aboratory of Industria Contro Tehnoogy, Institute of Advaned Proess Contro, Zhejiang University, Hangzhou 327, China jtao@iip.zju.edu.n, nwang@iip.zju.edu.n) Xuejun Wang (Department of Information Siene and Engineering, Centra South University, Changsha 475, China wxjgya@26.om) Abstrat: A geneti agorithm (GA) based reurrent fuzzy neura network modeing method for dynami noninear hemia proess is presented. The dynami reurrent fuzzy neura network (RFNN) is onstruted in terms of Takagi-Sugeno fuzzy mode. The onsequent part is omprised of the dynami neurons with output feedbak. The number and the parameters of membership funtions in the premise part are optimized by the GA onsidering both the approximation apabiity and struture ompexity of RFNN. The proposed dynami mode is appied to a PH neutraization proess and the advantages of the resuting mode are demonstrated. Key Words: Geneti agorithm, Reurrent fuzzy neura network, odeing, PH neutraization proess Category: H.3.3 Introdution Proess modeing is a very important step in the ontro, diagnosis and optimization of the proess system. The hemia proess modeing is espeiay diffiut beause of its noninear and dynami haraters. The fied of Artifiia Inteigene (AI), suh as Neura Network (NN), Fuzzy Theory, Expert System (ES), Geneti Agorithms (GA), has been rapidy deveoped in reent years. Eah agorithm has their own strengths but there are sti some imitations of them. In order to redue these imitations, the hybrid agorithms were deveoped by ombining two or three AI approahes and then drew on the strength of eah to offset the weakness of the others. Thereinto, fuzzy-neura network (FNN) had emerged as one of the most ative and fruitfu areas of researh in fuzzy ogi and neura networks with the better degree of auray and with the smaer omputationa times [Bukey and Hayashi 994][Rutkowski 24]. However, in the fied of dynami system modeing, the most ommony used modes were the reurrent neura networks (RNN), whih were regarded as osed-oop systems,

2 Tao J., Wang N., Wang X.: Geneti Agorithm Based Reurrent Fuzzy with the feedbak paths introduing dynamis to the mode [Qin et a. 992]. They exhibited greater predition apabiities ompared with the stati ones [astoroostas and Theoharis 22]. In [in et a. 25], a dynami RNN was proposed for system identifiation and derived good resut. The feedbak signa was taken from the output of the ativation funtion, after deayed for severa times, it was fed as input to the neuron. Therefore, the dynami hemia proess modeing in this work ombines the reurrent neura network with Takagi-Sugeno (T-S) fuzzy mode, and a reurrent fuzzy neura network (RFNN) is onstruted, whih is omposed of the premise part and the onsequent part. Though fuzzy systems have been appied suessfuy in the fied of industry, the generation of the fuzzy rues and the adjustment of its membership funtions (Fs) were done by tria and error and/or operator s experiene. Subsequenty, the designers find it diffiut to deveop adequate fuzzy rues and membership funtions to refet the essene of the proess. oreover, some information gets ost or ignored on purpose when human operators artiuate their experiene in the form of inguisti rues. To overome these drawbaks, an eitism GA for optimization of the parameters of Fs in the premise part is proposed. Sine the number of Fs determines the fuzzy rues direty, whih is osey reated to the ompexity of RFNN, a speia fitness funtion onsidering both the approximation apabiity and struture ompexity is designed in this work. One the Fs and rues are deided, the parameters in the onsequent part an be tuned by reursive east-squares (RS) method. The rest of the paper is organized as foows. Setion 2 desribes the basi arhiteture of T-S fuzzy mode, and ombines it with reurrent neura networks. Setion 3 gives a detaied desription of the proposed GA approah to optimize both the number and the parameters of Fs in the premise part. The simuation tests on a ph neutraization proess are made in setion 4. The onusions are summarized at setion 5. 2 T-S Reurrent Fuzzy Neura Networks (RFNN) 2. Basi Struture of the T-S Fuzzy ode Considering a SISO system, we assume the proess to be modeed is desribed as foowing: y(k +)=f(x(k)) () where X(k) =[y(k),..., y(k n),u(k d),...u(k d m)], u is the input, m,n are the orders of the proess whie d is the time deay, respetivey. A T-S type fuzzy mode an be onstruted to mode proess with its fuzzy IF-THEN rues

3 334 Tao J., Wang N., Wang X.: Geneti Agorithm Based Reurrent Fuzzy... expressed as foows: Rue j : If x (k) is A j and x 2 (k) is A 2j and... and x N (k) is A Nj Then y(k +)=B T j X(k),j =, 2...,, N m i (2) i= where B j =[b j,b j2,..., b jn ] T are parameters in the onsequent part, y(k +) is the predition output of the fuzzy mode, m i is the number of Fs for x i, is the number of rues. The output of fuzzy mode an be expressed as foows: y(k +)= j= a j[x(k)]b T j X(k) j= a j[x(k)] (3) where a j [X(k)] is the Gaussion membership funtion of the inferred fuzzy set A j (A j = N i= A ij ), defined by a j [X(k)] = μ i μi2 2...μin N (4) ( ) μ j i = exp x i ij ij 2 (5) where i, 2,..., m ; i 2, 2,..., m 2 ;...; i N, 2,..., m N ; ij is the enter of the Gaussion membership funtion of the fuzzy set A ij, whie ij is the width. Define fuzzy basis funtion (FBF) as Φ j [X(k)] = a j [X(k)] T = a [X(k)] (6) The output y(k + ) an then be expressed as a inear ombination of FBFs in the foowing form. y(k +)= Φ j [X(k)]B T j X(k) (7) j= 2.2 The Struture of Reurrent Fuzzy Neura Networks astoroostas et a. presented a genera struture of the dynami reurrent neuron [astoroostas and Theoharis 22]. Hene, it is easy to onstrut the dynami neuron aording to the onsequent part of the T-S fuzzy mode as shown in Figure. Combining with the premise part, the dynami RFNN an be obtained in Figure 2, where x =[u, y] is the input vetor used to onnet the network to its environment. The whoe RFNN onsists of two parts the premise part and the onsequent part. In Figure 2, the premise part is made up of 4 ayers. The first ayer is the input ayer, the number of the nodes in this ayer is N. Eah node in the seond

4 Tao J., Wang N., Wang X.: Geneti Agorithm Based Reurrent Fuzzy u (k) u ( k ) z z... u( k m) z bi bi b i 2 + b i( m + n) b i ( m+ ) z... z y( k n) y(k) z ( j =,2,, i =,,2 N) (k) y j y( k + ) Figure : A representation of the dynami neuron Consequent part u y Reurrent neuron unit Reurrent neuron unit 2 y y 2 y Reurrent neuron unit μ α y φ μ m α 2 φ 2 μ n m n μ n α φ Premise part Figure 2: Shemati sheme of RFNN ayer deegates the vaue of a anguage variabe, suh as N, PS, et, whihis used to auate the vaue of membership funtion (μ j i ) for eah input variabe. The number of nodes in ayer 2 is N i= m i. The node in the third ayer is utiized to math the premises in every fuzzy rue and ompute the fitness vaue of eah rue (a j ). The number of nodes in the ast ayer is the same as ayer 3, i.e., N 4 = N 3 =, whih is used to figure out the vaue of FBFs (Φ j ). The onsequent part is omposed of 2 ayers. The roe of the first ayer is the same as that in the premise part, and there exist nodes in the seond ayer. Eah node deegate one rue, whih is used to yied y j, j =, 2,...,, and is desribed in Figure. The output ayer is used to produe the resut of defuzzifiation as shown in Eq. (7).

5 336 Tao J., Wang N., Wang X.: Geneti Agorithm Based Reurrent Fuzzy... The suggested RFNN, omprising the rues in the form of Eq. (2), is a quasinoninear fuzzy mode. The rues are not inked with eah other, neither through externa feedbak nor through interna, they are onneted merey via the defuzzifiation part. The premise and defuzzifiation bok are stati whie the onsequent bok is dynami, whih are different from eah other merey in the weighting fators of feedbak. The input vetor of the premise part may be different from that of onsequent part. 3 Tuning the Parameter of RFNN Suppose the input vetor of RFNN is fixed in advane. The parameters need to earn are the weights b jk (j =, 2,...,, k =, 2,..., N) and the enters ( ij ) and widths ( ij )offs. 3. earning Agorithm of the Weights If the membership funtion is deided a priori, i.e., those FBFs Φ j [X(k)] have aready been deided. The parameters in the onsequent part (B j )anthenbe easiy estimated using east-squares (S) agorithm. Denote Θ =[B T B T 2...B T ] T (8) Ψ =[Φ [X(k)]X(k) T,Φ 2 [X(k)]X(k) T,..., Φ [X(k)]X(k) T ] T (9) Substitute Eq.(8) and Eq.(9) into Eq.(7) yieds y(k +)=Ψ(k) T Θ () Aso denote Φ =[Ψ(),Ψ(2),..., Ψ(N n )] () Y d =[y d (2),y d (3),..., y d (N n + )] (2) Rearrange Eq.() for k =, 2,..., N n, yieds Y d = ΦΘ (3) Eq.(3) is in the form of a inear regression mode. If the reverse of matrix (Φ T Φ) exist, the parameters matrix Θ an be uniquey determined by the onventiona S method. Θ =[Φ T Φ] Φ T Y d (4) However, if Φ is poory onditioned, the parameter obtained by Eq.(4) wi hange dramatiay if the eements of Y d are modified sighty and resut in poor preditions of Ŷ d when used Φ with new data [Wiiam and Ronad 26]. This is aways the ase beause suffiient training data are very diffiut to be obtained for most proesses. To sove this probem, the reursive east-squares (RS) method is adopted in this work.

6 Tao J., Wang N., Wang X.: Geneti Agorithm Based Reurrent Fuzzy Optimization of the Numbers and Parameters of Fs For amost every proess, we an get some inguisti information from domain experts or operators. One the fuzzy mode is mapped into the networks shown in Figure 2, and the weights are obtained as desribed in setion 3., there exist 2 N i= m i parameters to be optimized. However, optimization of those parameters annot be easiy soved by the standard optimization methods. An interesting aternative for soving this ompiated probem an be offered by the reenty deveoped evoutionary omputation methods-gas, whih are the most popuar and suessfu strategies among these evoutionary methods [Saomon 996]. In a GA, the popuation is the set of possibe soutions, and eah individua of this popuation is haraterized by hromosome-ike strutures. The possibiities of surviva of eah individua are evauated by the ost funtion. The resut of this evauation is aed fitness and pays an important roe in seetion and reprodution. The evoution is ahieved by the appiation of geneti operators, suh as seetion, rossover and mutation. The optimization of the numbers and parameters of Fs in RFNN is desribed at foowing setions Coding ethod There are totay 2 N i= m i parameters to be optimized in the RFNN, whih means one hromosome shoud be abe to deegate 2 N i= m i rea number. Hene, binary oding hromosome wi beome too ompex, and deima oding hromosome is adopted. The struture of th hromosome is shown as foows m 2... m C = m m... (5) N N2... Nm N N2... Nm where =, 2,..., is the size of the popuation, m i is produed randomy between and D, D is the maximum number of Fs. Without oss of generaity, D is set as, and N as 2. The eements of C are omputed by the foowing equation: ij = x imin + r(x imax x jmin ) i N, j m i (6) ij =. + r( x imax.) i N, j m i (7) 2 where r is a random number between and, x jmin and x jmax is the minimum and maximum vaues of input variabes given in the probem. When premise parts are generated, the orresponding weights of onsequent parts an be obtained using RS method desribed in setion 3.. Thus, the formuation of RFNNs is ompeted, whih an be represented by the pairs (C,Θ ), (C 2,Θ 2 ),..., (C,Θ ).

7 338 Tao J., Wang N., Wang X.: Geneti Agorithm Based Reurrent Fuzzy Fitness Funtion One RFNNs are onstruted, the preditions Ŷd, Ŷd2,..., Ŷd an be obtained and the orresponding errors E, are omputed as foows: E = Y d Ŷd 2 2 (8) In terms of setion 2, the ompexity of RFNNs mainy ies in the number of input (N)) and its number of Fs ( N i= m i). Sine the number of input is deided a priori, the number of Fs is inuded in the fitness funtion to simpify the struture of RFNN. Hene, fitness funtion onsidering both approximation apabiity and struture ompexity is shown as foows. J (C i,θ i )=E + λ N m i (9) This riterion expresses a ompromise between the ost of modeing errors and the ost of the ompexity of network struture, where λ is the weighting fator. i= Popuation Evoution One the odifiation and fitness funtion have finished, the next step wi be the appiation of the evoutionary agorithm. The GA used in this work is a standard geneti agorithm (SGA) with eitism, and its fowhart is shown in Figure 3. The first step of the GA is the generation of an initia random popuation. And after the initiaization, the fitness of every individua in the popuation is obtained through the ost funtion evauation. If the ending ondition is not satisfied, the next step wi be popuation evoution (appying geneti operators). The ending ondition an be the maximum evoution generation, the degree of onvergene, or any other. oreover, Eitism, the inusion of the best urrent set in the next popuation, is used throughout. Seetion. A set of individuas from the previous popuation must be seeted for reprodution. This seetion depends on their fitness vaues. Individuas with better fitness vaues wi more probaby survive. There exist different types of seetion operators, and in this work Rouette whee method is appied. The probabiity of an individua being seeted, P (C ), is given by: P (C )= f(c ) = f(c ) (2) where f(c ) is the fitness vaue of the individua C, whih is obtained by. The rouette whee is paed with equay spaed pointers. A singe J(C,Θ ) spin of the rouette whee wi simutaneousy pik a the members of the next

8 Tao J., Wang N., Wang X.: Geneti Agorithm Based Reurrent Fuzzy m 2 m 2 + m + 2m m 2 m 2 + m + 2m m + 2 m 2 m 2 m m 2 m 2 + m + 2 m 2 ' ( + ) Figure 3: Shemati sheme of the rossover operation popuation. Crossover. The rossover operator is appied after seetion, and this produes new individuas. Not a the seeted individuas are rossed over, this depends on the rossover probabiity, p. Crossover operation is exeuted between the urrent hosen individua C and the next individua C +, and yieds the offspring hromosomes C,C +. Sine the hromosome is omposed of the enters and widths, two points are hosen among the enters and widths, respetivey. The proedure is demonstrated with Figure 3, whih ontains a sheme of two-point rossover. utation. To have a better exporation of the searh spae, mutation operator is arried out. And when the eement of an individua is mutated with a probabiity p m, it is repaed by a new generated eement in terms of Eq. (6) and (7). 4 Simuation Resuts The proposed GA based RFNN modeing approah wi now be appied to a simuated PH neutraization proess [Nahas et a. 992]. The proess mode onsists of three noninear ordinary differentia equations and a noninear output equation: ḣ = q + q 2 + q 3 + C v h.5 W a4 (2) Ẇ a4 = Ah [(W a W a4 )q +(W a2 W a4 )+(W a3 W a4 )] () Ẇ b4 = Ah [(W b W b4 )q +(W b2 W b4 )+(W b3 W b4 )] (23) W a4 + ph ph4 pk2 + W b4 + pk ph4 + = (24) ph4 pk2 ph4 where h is the iquid eve, W a4 and W b4 are the invariants of the effuent stream, q, q 2 and q 3 are the aid, buffer and base fow rate, respetivey. The samping period is hosen as.25 min. It is desired to ontro ph 4 bythebasefowrateq 3. The interesting operation range is defined by ph 4 and q 3 4 m\s The objetive is to buid disrete dynamia modes for prediting the vaue of ph 4

9 34 Tao J., Wang N., Wang X.: Geneti Agorithm Based Reurrent Fuzzy... using as input variabes base fow rate q 3 and previous vaues of ph 4 Aording to the export knowedge of the ph proess, the input of onsequent part is hosen as: X(t) =[u(k),u(k,u(k 2),y FNN (k),y FNN (k ),y FNN (k 2)] T (25) The input of premise part is seeted as the urrent output (ph 4 ). For training and testing the GA based RFNN mode, we reated a set of 5 input-output data by seeting randomy the vaues of the base fow rate q 3 within the spae [,4].The first 3 data points is used for training RFNN, whie the eft is used The Vaue of F The Vaue of F ph(k) ph(k) Figure 4: Fs optimized by GA Figure 5: Fs obtained by k-means to test the performane of the proposed RFNN. The operationa parameters used in the proposed GA are isted as foows: number of hromosomes = 3, maximum number of Fs D =, weighting fator λ =.5, number of generations G = 5, probabiity of rossover p =.8, probabiity of mutation p m =.. After the optimization of RFNN, 5 Fs of the premise part are shown in Figure 4, and the trained T-S type RFNN mode is shown as foowing: If ph 4 (k) isa then y FNN = u(k)+.37u(k ) +.97u(k 2).592y FNN (k).446y FNN (k ).5y FNN (k 2) If ph 4 (k) isa 2 then y FNN = u(k).u(k ).24u(k 2) +.9y FNN (k)+.y FNN (k ) +.34y FNN (k 2) If ph 4 (k) isa 3 then y FNN = u(k)+.93u(k ) +.6u(k 2).394y FNN (k).287y FNN (k ).95y FNN (k 2)

10 Tao J., Wang N., Wang X.: Geneti Agorithm Based Reurrent Fuzzy IfpH 4 (k) isa 4 then y FNN = u(k)+.96u(k ) +.6u(k 2).43y FNN (k).284y FNN (k ).94y FNN (k 2) IfpH 4 (k) isa 5 then y FNN = u(k).27u(k ).33u(k 2) +.873y FNN (k)+.627y FNN (k ) +.2y FNN (k 2) Thus, the RFNN mode is empoyed to give preditions on testing data set. The predition resut on the testing data sequene is ompared with the resut obtained by a genera RFNN. In this method, the number of membership funtions is the same as GA-RFNN, the enters are obtained by k-means method [Darken and oody 99], and the width is set as 2, as shown in Figure 5. And the parameters of the sequent part are obtained by RS. From the modeing errors shown in Figure 6 and Figure 7, we an see that the proposed GA-RFNN mode is superior to genera RFNN mode. For the dynami proess modeing, if the input keeps unhanged, the output of dynami proess mode wi arrive at ertain point if it is stabe. This means that there exists a stati reation between the input and the steady-state vaue of the output. This reation is we known as steady-state response in proess industry. Here we examine the steady-state response of the two methods and the rea proess. The simuation resuts are shown in Figure 8 and Figure 9. From the above omparison, we an see that after the optimization of Fs in the premise part by GA, the struture of RFNN an be simpified whie the approximation apabiity is improved greaty. Sine the training data inude the whoe operating ondition, the GA-RFNN fit the rea proess quite we. Without optimization, the RFNN using the same training data is inferior to GA-RFNN. 5 Conusions The issue of the dynami modeing approah is investigated by GA based RFNN. The onsequent part of RFNN is a inear ombination of dynami reurrent neurons, whose parameters are derived by RS. The GA is designed whih performs both struture and parameter optimization for the number and parameters of Fs in the premise part of RFNN. Based on the GA optimization method, the suggested RFNN modeing approah does not require aurate knowedge of the proess. Performane omparisons on a noninear ph neutraization proess are made with regard to genera RFNN modeing method and demonstrate the effiieny of the proposed method.

11 342 Tao J., Wang N., Wang X.: Geneti Agorithm Based Reurrent Fuzzy....5 x 3.5. Estimation Error.5.5 Estimation Error Time Time Figure 6: odeing errors of GA-RFNN Figure 7: odeing errors of RFNN ph rea proess FNN preditions Base fow q3 ph rea proess RFNN Base fow q3 Figure 8: Predition resuts by GA- RFNN Figure 9: Predition resuts by RFNN Aknowedgements This work is in part supported by the projet of the Nationa Natura Siene Foundation of China (6422). Referenes [Bukey and Hayashi 994] Bukey J., Hayashi Y.: Fuzzy Neura Networks: A Survey ; Fuzzy Set. Syst., 66, (994), -3 [Rutkowski 24] Rutkowski.: New Soft Computing Tehniques for System odein, Pattern Cassifiation and Image Proessing ; Springer Verag, New York (24) [Qin et a. 992] Qin S. Z., Su H. T., Avoy T. J.: Comparison of Four Neura Net earning ethods for Dynami System Identifiatio ; IEEE Trans. Neu. Net., 3, (992), -3

12 Tao J., Wang N., Wang X.: Geneti Agorithm Based Reurrent Fuzzy [astoroostas and Theoharis 22] astoroostas P. A., Theoharis J. B.: A Reurrent Fuzzy-neura ode for Dynami System Identifiatio ; IEEE Trans Syst, an, Cybern Part B: Cybernetis, 32, 2(22), 76-9 [in et a. 25] in C. J., Chen C. H., Chen H. J.: Dynami System Identifiation Using Pseudo-Gaussian-Based Reurrent Compensatory Fuzzy Neura Networks ; J. Chin. Inst. Chem. Eng, 28, (25), [Wiiam and Ronad 26] Wiiam C. H., Ronad. O.: Resuts on the Bias and Inonsisteny of Ordinary east Squares for the inear Probabiity ode ; Eon. ett, 9, 3(26), [Saomon 996] Saomon R.: Re-evauating Geneti Agorithm Performane Under Coordinate Rotation of Benhmark Funtions ; Biosystem, 39, 3(996), [Nahas et a. 992] Nahas E. P., Henson. A.,Seborg D. E.: Noninear Interna ode Contro Strategy for Neura Network odes ; Comp. Chem. Eng, 6, 2(992), [Darken and oody 99] Darken C., oody J.: Fast Adaptive K-means Custering: Some Empiria Resuts ; Neu. Net, IJCNN, (99),