Approximation Bound for Fuzzy-Neural Networks with Bell Membership Function

Transcription

1 Approximation Bound for Fuzzy-Neural Networks with Bell Membership Function Weimin Ma, and Guoqing Chen School of Economics and Management, Tsinghua University, Beijing, 00084, P.R. China {mawm, School of Economics and Management, Xi an Institute of Technology, Xi an, Shaanxi Province, 7003, P.R. China Abstract. A great deal of research has been devoted in recent years to the designing Fuzzy-Neural Networks FNN from input-output data. And some works were also done to analyze the performance of some methods from a rigorous mathematical point of view. In this paper, the approximation bound for the clustering method, which is employed to design the FNN with the Bell Membership Function, is established. The detailed formulas of the error bound between the nonlinear function to be approximated and the FNN system designed based on the input-output data are derived. Introduction FNN systems are hybrid systems that combine the theories of fuzzy logic and neural networks. Designing the FNN system based on the input-output data is a very important problem,,3]. Some Universal approximation capabilities for a broad range of neural network topologies have been established by researchers like Ito 4], and T.P.Chen 5]. Their work concentrated on the question of denseness. Some Approximation Accuracies of FNN have also been established in 3]. More results concerning the approximation of Neural Network can be found in 6,7,8]. In paper ], an approach so called Nearest Neighborhood Clustering was introduced for training of Fuzzy Logic System. The paper 3] studied the relevant approximation accuracy of the clustering method with Triangular Membership Function and Gaussian Membership Function. In this paper, taking advantage of some similar techniques of paper 3], we obtain the upper bound for the approximation by using the clustering method with Bell Membership Function. Clustering Method with Bell Membership Function Before introducing the main results, we firstly introduce some basic knowledge on the designing FNN systems with clustering method, which was proposed in ]. Given the input-out data pairs x q 0,yq 0, q =,,... L. Wang and Y. Jin Eds.: FSKD 005, LNAI 363, pp. 7 77, 005. c Springer-Verlag Berlin Heidelberg 005

2 7 W. Ma and G. Chen where x q 0 U =α,β ]... α n,β n ] R n and y q 0 U y =α y,β y ] R. Ifthe data are assumed to be generated by an unknown nonlinear function y = fx, the clustering method in ] can help us todesignafnnsystemtoapproximate the function fx. For convenience to illustrate the main result of this paper, we describe this method in a brief way as follows. Step. To begin with the first input-output pair x 0,y 0, select a radius parameter r, establish a cluster center with letting x c = x 0,andsety c = y 0, B =. Step. For the kth input-out pair x k 0,yk 0, k =,..., suppose there are M clusters with centers at x c,x c,...,x M c. Find the nearest cluster center x l k c to x k 0 to satisfy x k 0 xl k c =min x k 0 xl c, l =,,...,M. l Then there are two cases Case. If x k 0 xl k c r, establish x k 0 as a new cluster center with xm+ c = x k 0, yc M+ k = y0 k and BM+ k =, and keep yck l = yck l and B l k =B l k for any l. Case. If x k 0 xl k c <r, do the following: y l k c k = yl k c k B l k k + y0 k B l 3 k k + and meanwhile set B l k k =B l k k + 4 y l ck =y l ck, B l k =B l k, for l l k. 5 Step 3. Then the FNN system can be constructed as: ] y lc k ˆf k x = = l= l= yck l l= + x xl c + x xl c j= l= j= + x j xl c,j + x j xl c,j where M = M + for case and M = M for case. Step 4. Repeat by going to Step with k = k +. ] 6 The above FNN system is constructed using singleton fuzzier, product inference engine and center-average defuzzifier, as detailed in ]: the basic idea is to group the data pairs into clusters in terms of their distribution and then use the

3 Approximation Bound for FNN with Bell Membership Function 73 fuzzy IF-THEN rules for one cluster to construct the fuzzy system; the radius parameter r is selected to determine the size of the clusters, namely the smaller of r, the more number of clusters and vice versa. Some intensive simulation results concerning this method for various problems can also be found in that paper. This paper concentrates on the approximation bound for this method with bell membership function. 3 Approximation Bound with Bell Membership Function The following theorem gives the approximation bound of FNN system ˆf k x of 6 which is constructed by using the clustering method with bell membership function. Theorem. Let fx be a continuous function on U that generates the inputoutput pairs in. Then the following approximation property holds for the FNN system ˆf k x of 6: fx ˆf n k x r + d x + n M σ + d x σ f 7 where the infinite norm is defined as dx =sup x U dx and d x is the distance from x to the nearest cluster, i.e., i= d x =min x x l l c = x x lx c 8 Proof. From 6 we have fx ˆf k x fx yc lk l= l= j= j= + x j xl c,j + x j xl c,j ] 9 Paper 3] obtain the following result when the relevant approximation bound for the clustering method with triangular membership function was discussed: n fx yc l k f x i x l c,i + r 0 Combining the 9 and 0, we have n f x fx ˆf i x i x l c,i + r l= i= j= k x i= l= j= + x j xl c,j + x j xl c,j ]

4 74 W. Ma and G. Chen n x i x l c,i f r + l= j= i= l= j= Now, we just focus on analyzing the term x i x l c,i l= l= j= j= + x j xl c,j + x j xl c,j + x j xl c,j + x j xl c,j on the right-hand side of. We only consider the case i = and the proof remains the same for i =,,...,n. Employing the similar method in 3], given any point x =x,x,...,x n U, we divide space U in to n areas and define some sets concerning the cluster center as follows: U x = {x U : x x 0,...,x n x n 0} U x = {x U : x x 0,...,x n x n < 0}... U x n = {x U : x x < 0,...,x n x n 0} U x n = {x U : x x < 0,...,x n x n < 0} And define some sets concerning the cluster centers V x = {x U : x x <d x }, V x = {x U : x x d x }, V x m = V x U x m, m =,...,n. 3 Apparently, there are two cases that we need to consider. Case : x l c V x, which indicates x l c, x <d x,wehave ] x x l c, < d x d x l= j= j= l= j= j= + x j xl c,j + x j xl c,j + x j xl c,j + x j xl c,j 4

5 Approximation Bound for FNN with Bell Membership Function 75 Case : x l c V x = n m= V x. We only consider the case x l c V x. For the cases x l c V x,...,v x n, the same result can be obtained. For any l that satisfied to x l c V x x, according to the definition of V,wehave x l c, x d x and x l c,j x j 0, j =,...,n 5 From, 3 and 5, we have x x l c, n x l c V x j= + xj xl c,j σ = x x l c, = j= x xl c, + xj xl c,j j= M σ The second inequality of 6 holds for 5 and the following reason x x l c, + x xl c, = x x l c, + x xl c, = x x l c, + x xl c, x x l c, x x l c, 6 7 Considering cases x l c V x,...,vx n,wehave x x l c, j= + xj xl c,j n M σ = n M σ 8

6 76 W. Ma and G. Chen On the other hand, it follows from 8 that = l= j= From 8 and 9, we have x x l c, l= j= + xj xl c,j j= + x j xl c,j + x j xl c,j l= + x xl c + d x 9 ] Combining 3, 4 and 0, it can be shown that x i x l c,i l= l= j= j= + x j xl c,j + x j xl c,j n M σ + d x σ 0 ] d x + n M σ + d x σ From and, we get the desired result. In paper 3], by using the same clustering method, an approximation bound, d x + + n n r + n r r + nπσ n ] n n i= f, with Gaussian Membership Function was obtained. However, in fact, by employing the similar techniques of this paper, it can be improved to r + n n M +d x i= f. Namely, we have the following theorem 9]. Theorem. Let fx be a continuous function on U that generates the inputoutput pairs in. If the Gaussian Membership Function is chosen, the following approximation property holds for the FNN system ˆf k x of 6: fx ˆf k x r + n M + d x n f where the infinite } norm is defined as dx = sup x U dx, d x = σ max {d x, and d x is the distance from x to the nearest cluster. The proof of the theorem can be found in 9]. i=

7 Approximation Bound for FNN with Bell Membership Function 77 4 Concluding Remarks In this paper, an upper bound, r + d x + n M σ + d x σ n i= f, concerning the approximation bound of clustering method with Bell Membership Function is proved in a rigorous mathematical way. The techniques employed in the proof of the theorem are expected to be used to obtain or improve other approximation bound of other methods of FNN. Acknowledgements. The work was partly supported by the National Natural Science Foundation of China , and China Postdoctoral Science Foundation References. Wang, L.X. and Mendel, J.M., W.: Fuzzy Basis Functions, Universal Approximation, and orthogonal least squares learning. IEEE Trans. Neural Network Wang, L.X.: Training of Fuzzy Logic System Using Nearest Neighborhood Clustering. Proc. 993 IEEE Int. Conf. Fuzzy Syst. San Francisco, CA, Wang, L.X. and Chen, W.: Approximation Accuracy of Some Neuro-Fuzzy Approches. IEEE Trans. Fuzzy Systems Ito, Y.: Approximation of Continuous Functions on R d by Linear Combination of Shifted Rotations of Sigmoid Function with and without Scaling Neural Networks. Neural Networks Chen, T.P., Chen, H.: Approximation Capability to Functions of Several Variables, Nonlinear Functions, and Operators by Radial Function Neural Networks. IEEE Trans. Neural Networks Maiorov, V., Meir, R.S.: Approximation Bounds for Smooth Functions in CR d by Neural and Mixture Networks. IEEE Trans. Neural Networks Burger, M., Neubauer, A.: Error Bounds for Approximation with Neurnal Networks. J. Approx. Theory Wang, J.J., Xu, Z.B., Xu, W.J.: Approximation Bounds by Neural Networks in L p ω. Proc. of st International Symposium on Neural Networks, F.Yin, J.Wang, and C.Guo Eds.: ISSN 004, LNCS Ma, W.M., Chen, G.Q.: Improvement on the Approximation Bound for Fuzzy- Neural Networks Clustering Method with Gaussian Membership Function. Accepted by the First International Conference on Advanced Data Mining and Applications ADMA005.