Approximation Capability of SISO Fuzzy Relational Inference Systems Based on Fuzzy Implications

Transcription

1 Approximation Capability of SISO Fuzzy Relational Inference Systems Based on Fuzzy Implications Sayantan Mandal and Balasubramaniam Jayaram Department of Mathematics Indian Institute of Technology Hyderabad Yeddumailaram, A.P {ma10p002, Abstract In this work, we show that fuzzy inference systems based on Fuzzy Relational Inference (FRI) with implicative interpretation of the rule base are universal approximators under suitable choice of operations for the rest of the components of the fuzzy system. The presented proofs make no assumption on the form or representations of the considered fuzzy implications and hence show that a much larger class of fuzzy implications other than what is typically considered in the literature can be employed meaningfully in FRIs based on implicative models. Keywords Fuzzy Relational Inference, Fuzzy implications, Universal approximation. I. INTRODUCTION The term approximate reasoning (AR) refers to methods and methodologies that enable reasoning with imprecise inputs to obtain meaningful outputs [6]. Fuzzy Inference Systems (FIS) form one particular type of AR scheme involving fuzzy sets and are one of the best known applications of fuzzy logic in the wider sense. FIS have many degrees of freedom, viz., the underlying fuzzy partition of the input and output spaces, the fuzzy logic operations employed, the fuzzification and defuzzification mechanism used, etc. This freedom gives rise to a variety of FIS with differing capabilities. The best known types of FIS are the Fuzzy Relational Inference (FRI) systems [12], [25], Similarity Based Reasoning (SBR) systems [11], [18], [5] and the Takagi-Sugeno fuzzy systems [15], [16]. In this work, we focus only on the FRI systems. A. Motivation for this work One of the important factors considered while employing an FIS is its approximation capability. While many studies have appeared on this topic, most of them deal with FRIs where the rules are interpreted in a non-conditional way or as just aggregation of possibile configurations of the data (see Sections III and III-A for details). When an implicative or a conditional interpretation of the rules are considered, there are only a few works dealing with their approximation properties. While one of the earliest studies on this topic was that of Castro [2], [3], it was later on shown by Li et al. [10], [9] that some of the operations considered by Castro led to vacuous outputs. Li et al. further went on to present their own results on it. However, it can be shown - as we will see later in Section V-A - that the proof they have presented is not valid for mixed monotonic functions. Recently, Štěpnička et al. [20] have discussed the same in a slightly more general setting. Once again, the assumptions they make on some components of the FRIs are not desirable - for instance, the requirements on the input partition make it non-ruspini. In this work, we provide a constructive proof of the universal approximation property of FRIs when an implicative model of the rule base is employed, i.e., where the operation between the antecedents and consequents is taken as a fuzzy implication. The proof makes no assumption on the form or representation of the considered fuzzy implications and hence is applicable for a large class of fuzzy implications. B. Outline Of the Work After presenting some important definitions from both fuzzy set theory and fuzzy logic connectives in Section II, we describe the two main types of fuzzy rule bases used in fuzzy systems in Section III. Following this, Section IV is devoted to a full description of fuzzy relational inference (FRI) mechanisms. In Section V we present a short survey on the works and results related to universal approximation of fuzzy relational inference systems. Further we clearly specify the main contributions and scope of our work. Relaxing the often insisted coherence of an implicative model suitably to the context of function approximation, Section VI-A investigates the class of fuzzy implications that can be used in FRIs to ensure this form of weak coherence. Finally, Sections VII and VIII contain the major contributions of this work, which shows that FRIs employing a rather large class of fuzzy implications - which include the R- implications - are universal approximators. In Section IX some concluding remarks are given. II. PRELIMINARIES We assume that the reader is familiar with the classical results concerning fuzzy set theory and basic fuzzy logic connectives, but to make this work more self-contained, we introduce some notations, concepts and results employed in the rest of the work. A. Fuzzy Sets Definition 2.1: If X is a non-empty set then F(X) is the fuzzy power set of X, i.e., F(X) = {A A : X [0, 1]}. Definition 2.2: A fuzzy set A is said to be

2 normal if there exists an x X such that A(x) = 1, convex if X is a linear space and for any λ [0, 1], x, y X, A(λx + (1 λ)y) min{a(x), A(y)}. Definition 2.3: For an A F(X), the Support, Height, Kernel and Ceiling of A are denoted, respectively, as Supp A, Hgt A, Ker A and Ceil A and are defined as: Supp A = {x X A(x) > 0}, Hgt A = sup{a(x) x X}, Ker A = {x X A(x) = 1}, Ceil A = {x X A(x) = Hgt A}. A is said to be bounded if Supp A is a bounded set. Note that for a normal fuzzy set Ker A = Ceil A. Definition 2.4: Let P be an arbitrary collection of fuzzy sets of X, i.e, P = {A k } n k=1 F(X). P is said to form a n fuzzy partition on X if X Supp A k. k=1 In literature, a partition P of X as defined above is also called a complete partition. Definition 2.5: A fuzzy partition P = {A k } n k=1 F(X) is said to be consistent if A k (x) = 1 then A j (x) = 0 when j k. n Ruspini Partition if A k (x) = 1 for every x X. B. Defuzzification k=1 Often there is a need to convert a fuzzy set to a crisp value, a process which is called Defuzzification. This process of defuzzification can be seen as a mapping g : F(X) X. There are many types of defuzzification techniques available in the literature, see [14] for a good overview. In this work, we use the following defuzzifier extensively. Example 2.6: For an A F(X), the Mean of Maxima (MeOM) defuzzifier gives as output the mean of all those values in X with the highest membership value, which can be mathematically expressed as MeOM(A) = x CeilA x Ceil(A) depending on whether Ceil A is finite or not. C. Fuzzy Logic Connectives or x CeilA dx Ceil(A). (1) Definition 2.7 ([8]): A binary operation T : [0, 1] 2 [0, 1] is called a t-norm, if it is increasing in both variables, commutative, associative and has 1 as the neutral element. Definition 2.8 ([1]): A function I : [0, 1] 2 [0, 1] is called a fuzzy implication if it is decreasing in the first variable, increasing in the second variable and I(0, 0) = 1, I(1, 1) = 1, I(1, 0) = 0. The set of all fuzzy implications will be denoted by I. Definition 2.9 ([1]): A fuzzy implication I : [0, 1] 2 [0, 1] is called a positive fuzzy implication if I(x, y) > 0, for all x, y (0, 1). Definition 2.10 ([1]): A fuzzy implication I is said to satisfy the ordering property, if I(x, y) = 1 x y, x, y [0, 1]. (OP) Definition 2.11: A function N : [0, 1] [0, 1] is called a fuzzy negation if N(0) = 1, N(1) = 0 and N is decreasing. Example 2.12: One such fuzzy negation is the Gödel negation { 1, if x = 0, N D1 (x) = x [0, 1]. (2) 0, if x > 0, Definition 2.13 ([1]): Let I I be any fuzzy implication. Then the function N I : [0, 1] [0, 1] defined by N I (x) = I(x, 0) is called the natural negation of I. III. FUZZY IF-THEN RULE BASE Given two non-empty classical sets X, Y R, a Single- Input Single-Output (SISO) fuzzy IF-THEN rulebase consists of rules of the form: IF x is A i THEN ỹ is B i, (3) where x,ỹ are the linguistic variables and A i, B i, i = 1, 2,... n are the linguistic values taken by the linguistic variables. These linguistic values are represented by fuzzy sets in their corresponding domains, i.e., A i F(X), B i F(Y ). A fuzzy rule base (3) can be viewed in two different ways, as explained in [7]. When each of the rules is viewed as a constraint, i.e., when the rules in (3) are combined together as IF x is A 1 THEN ỹ is B 1,... AND... IF x is A n THEN ỹ is B n, (4) we have the conditional form (IF-THEN) of the rules. On the other hand, each of the rules can also be viewed as just pieces of data giving possible configurations or positive information, in which case they are combined as follows: IF x is A 1 AND ỹ is B 1,... OR... IF x is A n AND ỹ is B n. (5) A. Fuzzy Relations that model the Rule Base (3) In fuzzy relational inference mechanisms (see Section IV below), fuzzy relations R : X Y [0, 1] are employed to represent the rule base (3). Two of the commonly employed fuzzy relations are the following: For any x X, y Y, n ˆR (x, y) = (A i (x) B i (y)), (Imp- ˆR ) i=1 Ř (x, y) = n (A i (x) B i (y)), (Conj-Ř ) i=1

3 where is taken as an implication and as a t-norm. Note that the fuzzy relation ˆR captures the conditional form (4) of the given rules, while the relation Ř captures the Cartesian product form (5) of the rules. For more on the semantics of Ř and ˆR we refer the readers to [7]. IV. FUZZY RELATIONAL INFERENCE MECHANISM Given a rule base of the form (3) and an input x is A, the main objective of a fuzzy inference mechanism is to find a meaningful B such that ỹ is B. While many types of fuzzy inference mechanisms have been proposed in the literature we restrict this study only to Fuzzy Relation based inference mechanisms. The inference mechanism in a fuzzy relational inference (FRI) can be expressed as follows: B = R (A ) = (FRI-R) where A F(X) is the input, the relation R : X Y [0, 1] i.e, R F(X Y ) represents or models the rule base, B is the obtained output is called the composition operator, which is a F(X) F(X Y ) F(Y ). A. Two main types of FRIs One of the two main FRIs is the Compositional Rule of Inference (CRI) proposed by Zadeh [24] and given as: B (y) = fr(a ) = A (x) R(x, y) = [A (x) R(x, y)], y Y, (CRI-R) x X where is a t-norm. The operator is also known as the sup T composition where T is a t-norm. Later Pedrycz [12] proposed another FRI mechanism based on the Bandler- Kohout Subproduct composition given as: B (y) = fr(a ) = A (x) R(x, y) = [A (x) R(x, y)], y Y, (BKS-R) x X with interpreted as a fuzzy implication. The operator is also known as the inf I composition where I is a fuzzy implication. Note that f R (A ) is also known as the direct image of A over R, while f R (A ) is called the sub-direct image of A over R [20]. B. FRI with Singleton input : R f R Often one needs to deal with crisp inputs x 0 X. In such a case the given input is suitably fuzzified, i.e., a fuzzy set A F(X) is suitably constructed from x 0. Commonly, the following singleton fuzzifier is employed: { 1, x = A x0, (x) = 0, x x 0. With the above input A, the FRI mechanism (FRI-R) reduces to B (y) = R(x 0, y), y Y. (FRI-R-Singleton) Thus in the case of a singleton input both the (CRI-R) and (BKS-R) are essentially the same and the output is fully dependent on the model of the rule base R. In other words, R f R and hence the does not play any role. C. Scope of this work ( We denote an) FRI with singleton input as a quadruple F = {A i, B j }, R, g, where A i s and B j s correspond to the input and output fuzzy partitions on X and Y, respectively, R is the fuzzy relation modeling the rule base and g is the defuzzifier used to obtain a crisp output from the obtained B. In this paper we deal only with the implicative form of the rule base, i.e., where rules are considered as conditions to be satisfied and hence fix R = ˆR in the sequel. Further, we consider the following generalised form of ˆR, where T is any t-norm, not necessarily the minimum t-norm: R T (x, y) = T n i=1(a i (x) B i (y)), (Imp-R T ) ( Thus this work ) deals with FRIs of the form F T = {A i, B j }, R, T g F. V. FRIS AS UNIVERSAL APPROXIMATORS A. Universal Approximation results with FRI In this subsection, we only briefly recall some of the important works dealing with the approximation properties of FRIs and refer the readers to the excellent review of Tikk et al. [17] for more details and the other recent works, for instance [13], [23] and the references therein. The earliest works to appear on this topic dealt with FRIs where R = Ř and hence can be considered to have assumed a Cartesian product interpretation of the fuzzy rules, see Wang [22] and Zeng & Singh [26]. It was Castro [2] who was the first to deal with the approximation properties of FRIs that employed ˆR. However, as was already pointed out by Li et al. Remark 2.4, [10], Castro has considered an FRI as given below: B (y) = j (B j(y)) = j (A j (x 0 ) B j (y)), which is clearly not an appropriate model to work with, since under most practical settings for any given x 0 X there will always exist a rule with an antecedent A i0 such that A i0 (x 0 ) = 0 and since 0 b = 1 for any b [0, 1], when the maximum t-conorm is used to aggregate the individual outputs one always obtains that B (y) = 1 for all y Y. Note that this is the case when the input partitions are of the Ruspini type - a property that is normally both practical and desirable. In the same work, after pointing out the above, Li et al. (see Theorem 3.4, [10]) have given a constructive proof of the approximation capability of an FRI with R = ˆR. The proof of their result rests on the construction of an open cover {U i } N+1 i=1 of the input space [a, b] that depends on the inverse of a continuous f on [a, b]. However, one can see that f 1 may not exist for any continuous function on [a, b], e.g., when f is any mixed monotonic function. Thus the proof is valid only for strictly monotonic functions either of the increasing or the decreasing type. Of course, the function can be approximated separately on each of the piecewise monotonic parts, but care must be taken to seamlessly integrate these different rule bases and to ensure the continuity at the points of inflexion - a

4 situation that does not seem to have been considered in their work. Recently, Štěpnička et al. in [20] considered an FRI with R = R where is the Łukasiewicz t-norm T LK (x, y) = max(0, x + y 1) and is any residuated implication obtained from a left-continuous t-norm, which is different from T LK. They have shown that the FRIs F = F T LK ( ) = {A i, B j }, R T LK, g = MeOM are universal approximators. However, the A i s do not form a Ruspini partition which is normal and desirable in practical settings. B. Main contribution of this work ( In this work, ) we show that FRIs of the form F T = {A i, B j }, R, T g are universal approximators. The proof is general enough for a large class of fuzzy implications and is valid for any continuous function, not necessarily monotonic and the partitions used are of the Ruspini type. Thus, we believe that these results are very much applicable in most of the practical and desirable contexts [7], [21], [20]. VI. WEAK COHERENCE AND IMPLICATIVE MODELS A. A Weaker form of Coherence Dubois et al. [7] defined the concept of coherence for an implicative model ˆR ( see (Imp- ˆR )) of a rule base as follows, which is suitably modified to fit into our notation. Definition 6.1 ([4], [7]): For a given rule base (4) and a fuzzy relation R T (x, y), as in (Imp-R T ) modelling this rule base, is coherent if for any x X there exist y Y such that R T (x, y) = 1. The coherence property states that for any x the final fuzzy output B should be normal, i.e., Ker B. Coherence of an implicative model of a rule base is very much dictated by the semantics involved [7]. Further, it is essential when using defuzzification techniques that are dependent on the kernel to be non-empty. However, there exist other reasonable defuzzification methods that do not depend on the kernel of the output fuzzy set and, further, in the setting of function approximation, as is the case here, perhaps there is an arguable justification to not to insist on this otherwise extremely important property. Relaxing this property we define the following weaker form of coherence. Definition 6.2: For a given rule base (4), a fuzzy relation R T (x, y), as in (Imp-R T ) modeling this rule base is said to be weakly coherent if for any x X there exist y Y such that R T (x, y) > 0. From (FRI-R-Singleton) and (Imp-R T ), we have the following: B (y) = R T (x 0, y) = T n i=1(a i (x 0 ) B i (y)) = T ( A 1 (x 0 ) B 1 (y), A 2 (x 0 ) B 2 (y),..., A n (x 0 ) B n (y) ). Now if the antecedent fuzzy sets form a Ruspini partition, then x 0 intersects atmost two fuzzy sets say, A m, A m+1. Then the above reduces to B (y) = T (A m (x 0 ) B m (y), A m+1 (x 0 ) B m+1 (y)), = T (B m(y), B m+1(y)), where B m and B m+1 are the fuzzy sets B m and B m+1 modified by the fuzzy implication. It is clear that for B to be non-empty the supports of B m and B m+1 should intersect, i.e., Supp B m(y) Supp B m+1. While coherence insists that the kernels of B m and B m+1 should intersect, the weak coherence defined above relaxes this to a mere intersection of their supports. It should be noted that while relaxing coherence to weak coherence does expand the set of fuzzy implications that can be considered in ˆR, it still does not encompass the whole set of fuzzy implications I. In the following, we discuss the class of fuzzy implications that can be considered for an FRI with R T to be at least weakly coherent. This leads us to study the effect of using fuzzy implications to modify fuzzy sets. B. Fuzzy Sets modified by Fuzzy Implications From the above section it is clear that to ensure weak coherence, we need to deal with fuzzy sets that are modified by a fuzzy implication. Thus studying the properties of such modified fuzzy sets is important and we proceed to do this in the section. Definition 6.3: Let C F(X) and I I be any fuzzy implication. We say that a C F(X) is the modification or modified fuzzy set of C by I at a given α [0, 1] if C (x) = I(α, C(x)), x X. (6) Since in this work we consider modification only by a fuzzy implication, we often use the simpler term modified fuzzy set without any explicit mention of either I or the α [0, 1]. The following results show that modification by an I I preserves convexity and also gives some relations between the supports of the original and modified fuzzy sets when an I I is used. We omit the proofs of the following results due to space constraints. Proposition 6.4: For a convex fuzzy set C, a fuzzy implication I and any α [0, 1], I(α, C) is also convex. Proposition 6.5: Let C be a bounded, normal, continuous convex fuzzy set, I I and α (0, 1). Consider the following inclusion relating the supports of C and its modified set C : (i) (ii) Supp C = Supp I(α, C) Supp C. (7) If I is a non-positive fuzzy implication, then there exists an α (0, 1) such that (7) is not valid. For a given I I, let A I = {x I(x, 0) = 0} and let δ = inf A I. a) If α < δ, then (7) is valid always. b) If α > δ, then (7) is valid only if I is positive. c) Let α = δ. If δ A I, then (7) is valid only if I is positive, while (7) holds for any I I if δ / A I.

5 Remark 6.6: Note that, from Proposition 6.5 we see that whenever δ < 1, to ensure that (7) holds we need an I I that is positive. If an I I which is positive and whose N I = N D1 is used to modify C above then the supports of C, C are equal, i.e., Supp C = Supp I(α, C) = Supp C, for all α (0, 1]. C. Types of Fuzzy Implications Considered From Section VI-A above we know that for an R T to ensure weak coherence, we need the support of the output fuzzy sets B m and B m+1 - which are the modified fuzzy sets of B m, B m+1 using a fuzzy implication I I - to intersect. Also, it can be seen from Section VI-B that when we use a non-positive fuzzy implication the supports of these modified fuzzy sets can shrink and hence there is a possibility that the intersection of their supports is empty, which is not desirable. Hence to ensure weak coherence at the least, we see that the class of implications I that can be considered should be restricted. Since in most practical settings we deal only with fuzzy sets that are bounded, continuous, convex and that which often form a Ruspini partition, it is sufficient to consider fuzzy implications I I that either satisfy the ordering property (OP), in which case often we can ensure even coherence [20], or are positive, in which case we can ensure at least a weak coherence. Thus, in the following sections we will deal with rules modeled by fuzzy relations R T where the fuzzy implication either satisfies (OP) or is positive with or without (OP) but whose natural negation N I = N D1, the Gödel negation (2). Remark 6.7: Note that the properties (OP) and N I = N D1 are not mutually exclusive. The Goguen implication I GG (x, y) = min(1, y x ) satisfies both, while the Lukasiewicz implication I LK (x, y) = min(1, 1 x + y) satisfies only (OP), whereas the Yager implication I YG (x, y) = min (1, y x ) does not satisfy (OP) but N IYG = N D1, and finally, the Reichenbach implication I RC (x, y) = 1 x + xy does not satisfy either of them. In fact, many sub-families of very well known families of fuzzy implications fall in either of the above two classes. For instance, every residuated implication obtained from a border continuous t-norm satisfies (OP), while the f-implications generated from generators such that f(0) = and the family of g-implications are such that their natural negation is the Gödel negation, N D1. It is interesting to note that all the above fuzzy implications are positive. However, { if we consider the Rescher implication 1, if x y I RS (x, y) = 0, if x > y, then it satisfies (OP), N I RS = N D1 and is not positive. Further, let us denote by R T OP the fuzzy relation where the fuzzy implication satisfies the ordering ( property (OP) ) and the corresponding FRI by F T OP = {A i, B j }, R T OP, g. Similarly, we denote by R T D1 the fuzzy relation where the fuzzy implication is positive and such that its natural negation is the Gödel ( negation N D1 ) and the corresponding FRI by F T D1 = {A i, B j }, R T D1, g. VII. F T OP ARE UNIVERSAL APPROXIMATORS ( ) Let us consider the FRI F T OP = {A i, B j }, R T OP, g. Recall from (FRI-R-Singleton), for any y Y, B (y) = R T (x 0, y) = T n i=1(a i (x 0 ) OP B i (y)) = T ( A 1 (x 0 ) OP B 1 (y), A 2 (x 0 ) OP B 2 (y),..., A n (x 0 ) OP B n (y) ). Note that to get a final crisp output y Y, we need to defuzzify the above B F(Y ) using g. ( In this section, we ) show that, FRI of the type F T OP = {A i, B j }, R T OP, g with g taken as the MeOM defuzzification (1) are universal approximators, i.e., they can approximate any continuous function over a compact set to arbitrary accuracy. Theorem 7.1: For any continuous function h: [a, b] R over a closed interval ( and an arbitrary given ɛ ) > 0, there is an FRI F T OP = {A i, B j }, R T OP, g = MeOM with {A i, B j } being Ruspini partitions such that max h(x) g(x) < ɛ. x [a,b] Proof: We prove this result in the following steps. Step I : Choosing the points of normality Since h is continuous over a closed interval [a, b], h is uniformly continuous on [a, b]. Thus for a given ɛ > 0 there exists δ > 0 such that w w < δ = h(w) h(w ) < ɛ 2. Step I (a): A Coarse Initial Partition With the δ = δ(ɛ) defined above and taking l = b a δ we now choose w i X, i = 1, 2,... l, such that w i w i+1 < δ. Let z i = h(w i ), the value h takes at the above chosen w i, for i = 1, 2,... l. We call these points w i and z i the points of normality on the input space and the output space, respectively. In Fig. 1, the points w 1, w 2,..., w 11 and the points z 1, z 2,... z 8 (in paranthesis) are the points of normality in the input and the output spaces, respectively. Step I (b): Redundancy Removal and Reordering Let us choose the distinct z i s from the above and sort them in ascending order. Let σ : N l N k denote the above permuation map such that z i = u σ(i), for i = 1, 2,... l and u j, j = 1, 2,..., k are in ascending order. By rearranging the z i s in ascending order and renaming them we obtain: u 1 = z 1 < u 2 = z 8 < u 3 = z 6 < u 4 = z 5 < u 5 = z 7 < u 6 = z 2 < u 7 = z 4 < u 8 = z 3. Step I (c): Refinement of the input space partition: Thus for each i = 1, 2,..., l we have h(w i ) = z i = u σ(i). However, note that consecutive points of normality w i, w i+1

6 Fig. 1. An Illustrative Example for Step I in the proof of Theorem 7.1. in the input space need not be mapped to consecutive points of normality u σ(i), u σ(i)+1 or u σ(i), u σ(i) 1. In Fig. 1, h(w 1 ) = u 1 and h(w 2 ) = u 6. Thus for the consecutive points w 1 and w 2 the function values are u 1 and u 6, which are not consecutive. To ensure the above, we further refine the input space partition. To this end, we refine every sub-interval [w i, w i+1 ], for i = 1, 2,... l 1 as follows. Note that h(w i+1 ) = u σ(i+1). Refinement Procedure: For every i = 1, 2,... l 1 do the following: (i) If u σ(i+1) = u σ(i)+1 or u σ(i) 1 then we do nothing. (ii) Let u σ(i+1) = u σ(i)+p, where p 2. For every u {u σ(i)+1, u σ(i)+2,..., u σ(i)+p 1 } we find a point v [w i, w i+1 ] such that h(v) = u. Note that the existence of such a v [w i, w i+1 ] is guaranteed by the continuity - essentially the ontoness - of the function h. If u = u σ(i)+q, for some 1 q p 1, then we denote the point v as w (q) i,i+1. (iii) Similarly, let u σ(i+1) = u σ(i) p, where p 2. For every u {u σ(i) 1, u σ(i) 2,..., u σ(i) p+1 } we find a v [w i, w i+1 ] such that h(v) = u. Once again, if u = u σ(i) q, for some 1 q p 1, then we denote v as w (q) i,i+1. From Fig. 1, it can be seen that we have inserted points w (1) 1,2, w(2) 1,2, w(3) 1,2, w(4) 1,2 [w 1, w 2 ]. Proceeding similarly, the following sub-intervals, shown in Fig. 1, have been refined: [w 2, w 3 ], [w 4, w 5 ], [w 8, w 9 ] and [w 9, w 10 ]. Step I (d): Final Points of Normality: Once the above process is done, we again rename the points of normality in the input space, viz., w (q) i,i+1 as x 1, x 2,..., x n (n l) and the u σ(i) s of the the output space as y 1, y 2,... y k. Step II : Construction of the Fuzzy Partitions In the next step, we construct fuzzy sets on both the input and output spaces with the above obtained x i s and y j s as the points of normality, as given below. Step II (a): Fuzzy Partition on the input space We construct l fuzzy sets such that each A i is centered at x i, Supp A i = (x i 1, x i+1 ) for i = 2,..., l 1, while Supp A 1 = [x 1, x 2 ) and Supp A l = (x l 1, x l ], each A i is normal at x i, i.e., A i (x i ) = 1, each A i is a continuous convex fuzzy set, {A i } l i=1 form a Ruspini partition. For instance, if each of the A i s (i = 2,..., l 1) is a triangular fuzzy set and A 1, A l are half-triangular with all of them attaining normality at x i then clearly {A i } l i=1 partitioning the input space X form a Ruspini partition and are continuous, convex, of finite support and A i (x i ) = 1. Step II (b): Fuzzy Partition on the output space Now we have the output space partition points as y 1, y 2,... y k. We partition the output space such that B 1, B 2,... B k form a Ruspini partition (as above) with B j (y j ) = 1, j = 1, 2,... k. Here obviously, y j y j 1 < ɛ 2, j = 1, 2,... k. We construct the fuzzy sets {B j } k j=1 partitioning the output space Y as continuous, convex and of finite support along the same lines as the A i s above, i.e., Supp B 1 = [y 1, y 2 ), Supp B j = (y j 1, y j+1 ), j = 2, 3,... k 1, Supp B k = (y k 1, y k ]. Step III: Construction of the smooth rule base We construct the rule base with l rules of the form: IF x is A i THEN ỹ is B i, i = 1, 2,... l, (8) where the consequent B i in the i-th rule is chosen such that i = j is the index of that y j = h(x i ), where x i is the point at which A i attains normality. Note that, since h is continuous, by the above assignment of the rules, we have that rules whose antecedents are adjacent also have adjacent consequents, i.e., for any i = 1, 2,... l 1 we have Supp B i Supp B i+1. Thus the constructed rule base is smooth [19]. Step IV : Approximation capability of the output Let x X be the arbitrary given input. Clearly, x [x m, x m+1 ] for some m l 1. Once again, by our construction, x belongs to atmost two adjacent A i s, w.l.o.g. let them be A m, A m+1. Thus, B (y) = T [A m (x ) OP B m (y), A m+1 (x ) OP B m+1 (y)] = T [s m OP B m (y), s m+1 OP B m+1 (y)] where s m = A m (x ) and s m+1 = A m+1 (x ). Note that since A i s form a Ruspini partition, we have that s m + s m+1 = 1. Further, note that by the construction of {A i, B j }, B m, B m+1 are adjacent fuzzy sets. Consider the kernel of B, i.e., Ker B = {y : B (y) = 1}. We choose the defuzzified output y such that it belongs to

7 Ker B. In fact, as we show below, by the construction of {A i, B j } we see that Ker B is a singleton and this becomes the defuzzified output. Since T is a t-norm, we know that T (p, q) = 1 if and only if p = 1 and q = 1. Further, note that p OP q = 1 p q and s m + s m+1 = 1 and hence we have Ker B = {y : B (y) = 1} = {y : s m OP B m (y) = 1} {y : s m+1 OP B m+1 (y) = 1} = {y : s m B m (y)} {y : s m+1 B m+1 (y)}. Let α m = inf{α : s m OP α = 1} and β m+1 = inf{β : s m+1 OP β = 1}. Since OP has (OP), clearly α m = s m and β m+1 = s m+1. By the continuity and convexity of B m, B m+1 there exist a m, b m, a m+1, b m+1 such that B m (a m ) = B m (b m ) = s m and B m+1 (a m+1 ) = B m+1 (b m+1 ) = s m+1. By the monotonicity of the implication in the second variable, for every y [a m, b m ] we have that s m B m (y) = 1 and for every y [a m+1, b m+1 ] we have that s m+1 B m+1 (y) = 1. Thus, {y : s m B m (y)} = [a m, b m ], {y : s m+1 B m+1 (y)} = [a m+1, b m+1 ], and Ker B = {y : B (y) = 1} = [a m, b m ] [a m+1, b m+1 ]. Claim: Ker B = {a m+1 } = {b m }. Firstly, note that for any s m [0, 1] by the normality of B m we have that B m (y m ) = 1 and hence y m {y : s m B m (y)} = y m [a m, b m ]. Similarly, y m+1 [a m+1, b m+1 ]. It suffices to show that a m+1 b m from whence Ker B = [a m+1, b m ]. Note that since m < m + 1 and B m, B m+1 are adjacent fuzzy sets, either y m < y m+1 or y m > y m+1. WLOG, let us assume y m < y m+1. Now, from a m+1 Supp B m+1 we have that y m a m+1 y m+1. Similarly, y m b m y m+1. Hence, y m a m+1, b m y m+1. Since, s m + s m+1 = 1 = B m+1 (a m+1 ) + B m (b m ) = 1, = B m+1 (a m+1 ) = 1 B m (b m ), = B m+1 (a m+1 ) = B m+1 (b m ), = b m [a m+1, b m+1 ], i.e., a m+1 b m. Now, to see that b m = a m+1, note that since {B j } form a Ruspini partition and B m, B m+1 are adjacent fuzzy sets, we have B m+1 (a m+1 ) = 1 B m (a m+1 ) and hence B m (a m+1 ) = s m = B m (b m ). Since b m, a m+1 Supp B m Supp B m+1 on which both B m, B m+1 are strictly monotonic ( but of opposite types) we have that b m = a m+1. Since g is the MeOM defuzzification, we get that y = g(x ) = a m+1 = b m [y m, y m+1 ]. Clearly, now, y m g(x ) < ɛ 2 or y m+1 g(x ) < ɛ 2. WLOG, let y m g(x ) < ɛ 2, i.e., y m y < ɛ 2. Further, since x [x m, x m+1 ] we have h(x ) y m < ɛ 2. Putting them all together, we have g(x ) h(x ) = y h(x ) y y m + y m h(x ) < ɛ 2 + ɛ 2 < ɛ. Since x is arbitrary we have, max h(x) g(x) < ɛ. x [a,b] VIII. F T D1 ARE UNIVERSAL APPROXIMATORS While in the previous section, we dealt with fuzzy implications satisfying (OP), this class of fuzzy implications is rather limited. In this section, we consider those positive implications whose natural negations are Gödel negation. Once again, recall that from (FRI-R-Singleton), with R = for any y Y, we have R T D1 B (y) = R T D1 (x 0, y) = Ti=1(A n i (x 0 ) D1 B i (y)) = T ( A 1 (x 0 ) D1 B 1 (y), A 2 (x 0 ) D1 B 2 (y),..., A n (x 0 ) D1 B n (y) ). ( We now show that ) the FRIs F T D1 = {A i, B j }, R T D1, g = MeOM are universal approximators, i.e., they can approximate any continuous function over a compact set to arbitrary accuracy. Theorem 8.1: For any continuous function h: [a, b] R over a closed interval ( and an arbitrary given ɛ ) > 0, there is an FRI F T D1 = {A i, B j }, R T D1, g = MeOM with {A i, B j } being Ruspini partitions such that max h(x) g(x) < ɛ. x [a,b] Proof: Once again the proof is given in many steps. Steps I III dealing with the construction of the input and output partitions and the rule base are done in exactly the same way as in Steps I III of the proof of Theorem 7.1. Step IV: Approximation capability of the output Once again, let x X be the arbitrary given input. Clearly, x [x m, x m+1 ] for some m l 1. Once again, by our construction, x belongs to atmost two adjacent A i s, viz., A m, A m+1. Thus, B (y) = T [A m (x ) D1 B m (y), A m+1 (x ) D1 B m+1 (y)] = T [s m D1 B m (y), s m+1 D1 B m+1 (y)] = T [ B m(y), B m+1(y) ] where s m = A m (x ) and s m+1 = A m+1 (x ). Note that since A i s form a Ruspini partition, we have that s m + s m+1 = 1. Now since D1 is positive and such that x D1 0 = 0 for any x (0, 1], we have from Remark 6.6 that the supports of both the modified fuzzy sets B m, B m+1 are the same as those of B m, B m+1, i.e., Supp B m=supp B m and Supp B m+1 = Supp B m+1. Hence, Supp B = Supp B m Supp B m+1 = Supp B m Supp B m+1 = Supp (B m B m+1 ) = [y m, y m+1 ]. (9)

8 Now since (9) holds we have, y = g(x ) = MeOM(B (y)) Supp (B m B m+1 ) = [y m, y m+1 ]. So y m g(x ) < ɛ and y m+1 g(x ) < ɛ 2 2. Let y m g(x ) < ɛ i.e, y m y < ɛ. Once again, since 2 2 x [x m, x m+1 ] we have h(x ) y m < ɛ. Putting them all 2 together, we have g(x ) h(x ) = y h(x ) y y m + y m h(x ) < ɛ. IX. CONCLUDING REMARKS In this work, we provide a constructive proof of the universal approximation properties of FRIs when implicative rules are employed, i.e., where the operation between the antecedents and consequents is taken as a fuzzy implication. ( ) We show that FRIs of the form F T = {A i, B j }, R, T g are universal approximators. The proof is general enough for a large class of fuzzy implications I - without making any assumptions about the form or representation of the considered fuzzy implications - and is valid for any continuous function, not necessarily monotonic and the partitions used are of the Ruspini type. We believe that even the implicative models that employ fuzzy implications which are positive but whose natural negation is not the Gödel negation do possess the universal approximation property, since the convexity of the modified output fuzzy sets is always preserved. This study will be taken up in the near future. In the literature, it is typical to consider ˆR and an R- implication obtained from a left-continuous t-norm for the fuzzy relation R that models the rule base. However, in this work, the R that is considered is much more general, viz., instead of using the min operation which is commonly employed, we take a general t-norm T to aggregate the rules and further the fuzzy implication considered is a much larger set than just the R-implications obtained from a left-continuous t-norm, which are both positive and satisfy (OP). Thus, we believe that these results are very much applicable in most of the practical and desirable contexts as discussed in these works [7], [21], [20] and show that a much larger class of fuzzy implications other than what is typically considered in the literature can be employed meaningfully in FRIs based on implicative models. REFERENCES [1] M. Baczyński and B. Jayaram, Fuzzy Implications, ser. Studies in Fuzziness and Soft Computing. Springer, 2008, vol [2] J. Castro, Fuzzy logic controllers are universal approximators, Systems, Man and Cybernetics, IEEE Transactions on, vol. 25, no. 4, pp , apr [3] J. Castro and M. Delgado, Fuzzy systems with defuzzification are universal approximators, Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on, vol. 26, no. 1, pp , [4] D. Coufal, Coherence of radial implicative fuzzy systems, in Fuzzy Systems, 2006 IEEE International Conference on, 2006, pp [5] V. Cross and T. Sudkamp, Fuzzy implication and compatibility modification, in Fuzzy Systems, 1993., Second IEEE International Conference on, 1993, pp vol.1. [6] D. Driankov, H. Hellendoorn, and M. Reinfrank, An introduction to fuzzy control (2nd ed.). London, UK, UK: Springer-Verlag, [7] D. Dubois, H. Prade, and L. Ughetto, Checking the coherence and redundancy of fuzzy knowledge bases, IEEE Trans. Fuz Sys., vol. 5, no. 3, pp , [8] E. P. Klement, R. Mesiar, and E. Pap, Triangular Norms, ser. Trends in Logic. Dordrecht: Kluwer Academic Publishers, 2000, vol. 8. [9] Y.-M. Li, Z.-K. Shi, and Z.-H. Li, Approximation theory of fuzzy systems based upon genuine many-valued implications: MIMO cases, Fuzzy Sets Syst., vol. 130, pp , [10], Approximation theory of fuzzy systems based upon genuine many-valued implications: SISO cases, Fuzzy Sets Syst., vol. 130, pp , [11] P. Magrez and P. Smets, Fuzzy modus ponens: A new model suitable for applications in knowledge-based systems, Int. J. Intell. Syst., vol. 4, no. 2, pp , June [12] W. Pedrycz, Applications of fuzzy relational equations for methods of reasoning in presence of fuzzy data, Fuzzy Sets Syst., vol. 16, no. 2, pp , [13] I. Perfilieva and V. Kreinovich, A new universal approximation result for fuzzy systems, which reflects CNF DNF duality, International Journal of Intelligent Systems, vol. 17, no. 12, pp , [14] S. Roychowdhury and W. Pedrycz, A survey of defuzzification strategies, International Journal of Intelligent Systems, vol. 16, no. 6, pp , [15] M. Sugeno and G. T. Kang, Structure identification of fuzzy model, Fuzzy Sets Syst., vol. 28, no. 1, pp , Oct [16] T. Takagi and M. Sugeno, Fuzzy identification of systems and its applications to modeling and control, Systems, Man and Cybernetics, IEEE Transactions on, vol. SMC-15, no. 1, pp , jan.-feb [17] D. Tikk, L. T. Kóczy, and T. D. Gedeon, A survey on universal approximation and its limits in soft computing techniques, Int. J. Approx. Reasoning, vol. 33, no. 2, pp , [18] I. Turksen and Z. Zhong, An approximate analogical reasoning approach based on similarity measures, IEEE Transactions on Systems, Man and Cybernetics, vol. 18, no. 6, pp , nov/dec [19] M. Štěpnička and B. D. Baets, Monotonicity of implicative fuzzy models, in FUZZ-IEEE 2010, IEEE International Conference on Fuzzy Systems, Barcelona, Spain, July, 2010, Proceedings, 2010, pp [20] M. Štěpnička, B. De Baets, and L. Nosková, Arithmetic fuzzy models, IEEE Trans. Fuz Sys., vol. 18, no. 6, pp , [21] M. Štěpnička and B. Jayaram, On the suitability of the bandler-kohout subproduct as an inference mechanism, IEEE Trans. Fuz Sys., vol. 18, no. 2, pp , Apr [22] L.-X. Wang and J. Mendel, Fuzzy basis functions, universal approximation, and orthogonal least-squares learning, Neural Networks, IEEE Transactions on, vol. 3, no. 5, pp , sep [23] R. R. Yager and V. Kreinovich, Universal approximation theorem for uninorm-based fuzzy systems modeling, Fuzzy Sets and Systems, vol. 140, no. 2, pp , [24] L. A. Zadeh, Outline of a new approach to the analysis of complex systems and decision processes, IEEE Transactions on Systems, Man and Cybernetics, vol. SMC-3, no. 1, pp , jan [25], The concept of a linguistic variable and its application to approximate reasoning-iii, Information Sciences, vol. 9, pp , [26] X.-J. Zeng and M. Singh, Approximation theory of fuzzy systems- SISO case, Fuzzy Systems, IEEE Transactions on, vol. 2, no. 2, pp , may 1994.