Fuzzy Rough Sets with GA-Based Attribute Division

Transcription

1 Fuzzy Rough Sets with GA-Based Attribute Division HUGANG HAN, YOSHIO MORIOKA School of Business, Hiroshima Prefectural University 562 Nanatsuka-cho, Shobara-shi, Hiroshima , JAPAN Abstract: Rough set theory is a powerful tool to extract classification rules from a database that usually is given in the form of information system in which information is expressed by attributes and their values. In this paper, we first evaluate every example data (tuple) using, not a singleton but, a fuzzy number (or, suppose the original information system is along with such fuzzy numbers), due to the fact that the fuzzy numbers are easily set in comparison with the singletons. Then, based on the information system with the fuzzy numbers, we give a new definition of fuzzy rough set. As a result, the traditional rough set proposed by Z. Pawlak is a special case of the fuzzy rough set. Consequently, the upper/lower approximation, which is corresponding with the negative/positive rules, is varying based on the system uncertainties. At the same time, the possible rules can be reduced whereas the negative/positive rules are increased. It means that the approximation precision is improved. In addition, the genetic algorithm is adopted to divide the attribute values that have continuous values into the most proper discrete values in order to improve the approximateness of the system. Key-Words: Rough sets, information system, rule extraction, fuzzy sets, fuzzy rough sets, genetic algorithm. 1 Introduction The effective use of computers in various realms of human activities strongly depends on the efficiency of algorithms implemented in these computers. So far, many theoretical foundation stones for the algorithm have been set up, in which the rough set theory [1, 2] is a powerful tool to extract classification rules from a database. In general, such a database regarding the knowledge we are interested in is given in the form of information system (IS), which actually indicates an approximation space. In the traditional IS,as pointed out by some researchers [3, 4, 5], the approach for rule extraction implements the fully correct or certain classification rules without considering other factors such as uncertain class labeling, importance of examples. The limitations above severely reduce the applicability of the rough set approach to problems which are more probabilistic than deterministic in nature. In order to deal with the defects above and improve the reality of IS, some new concepts, such as variable precision rough model (VP-model) [3], uncertain information system (UIS) [5], have been suggested. In an UIS, considering data s noise tolerance degrees, two classification factors, which are corresponding with the positive region, and the negative region, respectively, must firstly be set up for whole system, then, the certainty and importance for each tuple need to be given. However, when we extract rules based on the UIS, there are some tough tasks to be encountered. The following are some of them. (1) It is difficult to set up some singleton values for the classification factors and the every certainty and importance. For example, you say the importance factor for a condition contribute set is 0.85, and I may say the one is Such a little difference 0.01 may lead to a completely different classification rule. Therefore, a nature way to avoid the problem above is adopting fuzzy numbers, say about 0.85 or about What s more, when the classification and each example data s factors, where no matter their values are singleton or fuzzy, are involved in the IS, the traditional rough set [1] is no longer capable of giving the upper/lower approximation; (2) How to divide the attribute values which are continuous into some discrete intervals? Obviouly, each discrete attribute value is one of the most important setting in an UIS or IS. Because the different discrete attribute value may make the corresponding object enter into a different elementary class, further, that leads to a different classification rule. For example, say age, the candidate sets of the attribute (discrete) value could be {0, 1}, {0, 1, 2} and so on where 0 expresses young, 1 expresses old, and 2 expresses middle. Which one should be taken? Even we can choose one, then, what interval age is middle? You say year old, and I may argue that is year old. Who is correct? We do not have a clue. Here in this paper, the genetic algorithm (GA), is adopted to obtain the most proper division. In this paper, we first evaluate every example data using, not a singleton but, a fuzzy number (or, suppose the original information system is along with such fuzzy numbers), due to the fact that the fuzzy numbers are easily set in comparison with the singletons. Then, based on the information system with the fuzzy numbers, we give a new definition of fuzzy rough set. As a result, the traditional rough set proposed by Z. Pawlak is a special case of the fuzzy rough set. Consequently, the upper/lower approximation, which is corresponding with the negative/positive rules, is varying based on the system uncertainties. At the same time, the possible rules can be reduced whereas the negative/positive rules are increased. It means that the approximation precision is improved. In addition, the genetic algorithm is adopted to divide the attribute values that have continuous values into the

2 most proper discrete values in order to improve the approximateness of the system. The remainder of this paper is arranged as follows. Section 2 describes some basic definitions of rough set and classification rule extraction in an IS. In section 3, we focus our attention on the full explanation of fuzzy rough set in an UIS. At the same time, the relation between the traditional rough set and the fuzzy rough set will be made clear. Section 4 describes the GA division method. Finally, the conclusion in this paper is given in section 5. 2 Rough Sets The database regarding the experts know-how is generally given in the form of the information system. The definition of the traditional information system is given by Pawlak [2]. Definition 1 An information systems (IS) is an ordered quadruple IS =(U, Q, V, ρ) (1) where U is the universe which is a non-empty finite set of objects x; Q is a finite set of attributes q; V = U q Q V q, and V q is the domain of attribute q; ρ is a mapping function such that ρ(x, q) V q for every q Q and x U. Q is composed of two parts: a set of condition attributes (C) and a decision attribute (D), i.e., Q = C D. ρ also is called a decision function. If we introduce function ρ x : Q V such that ρ x (q) =ρ(x, q) for every q Q and x U, ρ x is called decision rule in IS, and x is called a label of the decision rule ρ x. Let IS =(U, Q, V, ρ) be an information system, and let q Q, x, y U. Ifρ x (q) =ρ y (q), then we say x, y are indistinguishable, in symbols xr q y where R q is an equivalence relation. Also, objects x, y U are indistinguishable with respect to P Q in IS, in symbols xr P y,ifxr p y for every p P. In particular, if P = Q, x, y are indistinguishable in IS, in symbols xry instead of xr Q y. Therefore each information system IS =(U, Q, V, ρ) defines uniquely an approximation space A =(U, R), where R is an equivalence relation generated by the information system IS. The equivalence relation R partitions U into a family of disjoint subsets which are called Q-elementary sets. Likewise, R C leads to C-elementary sets, and R D leads to D-elementary sets. Given an arbitrary set X U, in general it may not be possible to describe X precisely in A. One may characterize X by a pair lower and upper approximations. Definition 2 Let R be an equivalence relation on a universe U. For any set X U, the lower approximation apr(x) and the upper approximation apr(x) are defined by as follows: apr(x) = {x U [x] R X} (2) apr(x) = {x U [x] R X 0} (3) where [x] R = {y xry} (4) is the equivalence class containing x. The lower approximation apr(x) is the union of elementary sets which are subsets of X, and the upper approximation apr(x) is the union of elementary sets which have a non-empty intersection with X. The set bnd(x) = apr(x) apr(x) is called boundary of X in A. If bnd(x) is empty, then subset X is exactly definable. Note that rough set is a set (pair) of lower and upper approximation. An accuracy measure of set X in the approximation space A =(U, R) is defined as α(x) = apr(x) apr(x) (5) where denotes the cardinality of a set. Clearly, it is true that 0 α(x) 1. Besides, X is called definable in A if α(x) = 1, and X is called undefinable in A if α(x) < 1. Now, let us consider the issue of rule extraction from an information system. A natural way to extract rules, or represent experts knowledge, is to construct a set of conditional productions, each of them having the form IF { set of conditions} THEN { set of decisions} Such a form can be easily induced by taking the advantage of rough set. In an approximation space A =(U, R), regarding a subset X of U, the whole universe U is partitioned into three regions: Positive region pos(x) = apr(x); Negative region neg(x) = U apr(x); Boundary region bnd(x) = apr(x) apr(x) which lead to the following classification rules: Describing pos(x) positive rules; Describing neg(x) negative rules; Describing bnd(x) possible rules. Regarding the classification rule extraction as above, a simple illustration example is shown as follows. [Example] Suppose that there is an uncertain information system IS =(U, C, D, V ), which is a medical database about the diagnosis of influenza (Tab.1). In the information system, U = {p 1,p 2,...,p 6 } in which each object (element) expresses a patient; Q = C D = {temp, sneeze, headache, flu}, V temp = {0, 1, 2} in which 0 expresses normal, 1 expresses high and 2 expresses very high ; V sneeze = V headache = V flu = {0, 1} in which 0 expresses no and 1 expresses yes. Also, the mapping function ρ is given in the table. Clearly, such an approximation space yields the following elementary sets with respect to attributes temp, sneeze and headache : E 1 = {p 1,p 5 }, E 2 = {p 2 }, E 3 = {p 3 }, E 4 = {p 4 }, E 5 = {p 6 } i.e., C-elementary sets = {E 1,E 2,...,E 5 }.

3 Table 1: Influenza data Q U C D temp sneeze headache flu p p p p p p Now, let us consider to approximate a subset X = {p 1,p 2,p 4 } which is a set of patients who are catching a cold. Based on the concepts of regular IS above, we have, apr(x) ={p 2,p 4 } apr(x) ={p 1,p 5,p 2,p 4 } pos(x) ={p 2,p 4 } (6) neg(x) ={p 3,p 6 } (7) bnd(x) ={p 1,p 5 } (8) Therefore, pos(x) follows the positive rules below: (1) IF temp=1 sneeze=1 handache=0 THEN flu= yes ; (2) IF temp=1 sneeze=1 handache=1 THEN flu= yes ; where denotes and. bnd(x) follows the possible rules below: (3) IF temp=2 sneeze=0 handache=0 THEN flu= possible ; we can see that in boundary region bnd(x) ={p 1,p 5 }, though they have the same condition: temp=2 sneeze=0 handache=0, the decisions are different: one is yes and another one is no in the datebase. It means in such a case (condition), you are probably catching a cold as we shown in rule (3). Furthermore, the negative rules are obtained by describing neg(x) as follows: (4) IF temp=1 sneeze=0 handache=1 THEN flu= no. (5) IF temp=0 sneeze=1 handache=1 THEN flu= no. Form (16), the approximation accuracy α(x) = 2/4 = Fuzzy Rough Sets As mentioned previously, we can use the theory of rough set to extract (classification) rules form an information system which leads to an approximation space. However, such a classification is based on the facts such as noise-free, importance-identical for each example (or tuple), and error-free for the final rules. Therefore, in this section we first propose an uncertain information system (UIS) in an attempt to relax the traditional information system. Definition 3 An uncertain information system (UIS) is defined as follows: UIS =(U, C, D, V, ρ, W ) (9) where U is the universe which is a non-empty finite set of objects x; C is a finite condition set of attributes; D is a finite decision set of attributes; V = q C D V q, and V q is the domain of attribute q; ρ is a mapping function such that ρ(x, q) V q for every q C D and x U; W = x U w x, and w x is a fuzzy number defined by membership function µ wx [0, 1], which assigns each tuple an importance (weight) factor to represents how important (weighty) is for the corresponding decision. As in above definition, we added the weight factor into the traditional IS. The main purpose is to give a evaluation to each tuple. It means, for a same decision, there maybe are several tuples which have either same or different conditions, but considering their respective situation such as noise, confidence and so on, they do have different weights. For example, in a medical database there are two patients p 1, p 2 who have different conditions which means they have different condition attribute values, but the decisions, say catching a cold are same. In such diagnosis, which condition is easier to lead to the decision? Naturally, the difference between them occurs. Therefore, if there is a strong causal relationship between the condition and decision in the case of p 1, the weight will be big, say 1; likewise, if the causal relationship in the case of p 2 is weak, the weight will be small, say 0.4. As a result, every object x appears with its own weight in the universe U. The images are depicted in Fig.1 and Fig.2. In the UIS (Fig.2), each object has different weight which is imaged by its size of circle, whereas each object has same size of circle in traditional IS (Fig.1). Let E,X be a non-empty elementary set, and a Figure 1: Image of the traditional IS non-empty subset in the approximation space, respectively. First, similarly in [3], we define a concept which is called relative degree of classification of the set E with respect to set X as follows (Fig.3): c(e,x) = x I w x ; I = E X (10) x E w x Considering the system situation such as admissible level of misclassification, noise, and approximation precision, one can set up two thresholds β P, β N,

4 U X 2QUKVKXG4GIKQP $QWPFCT[4GIKQP 0GICVKXG4GIKQP Figure 2: Image of the UIS E I X Figure 4: The three regions in the rough set U X Figure 3: E, X, and their intersection I which are called positive threshold, negative threshold, respectively. We say that E is included in X, if c(e,x) β P, and E is connected nothing with X, if c(e,x) β N. Based on the relative degree of classification (10), the lower approximation, and upper approximation of a subset X with respect to thresholds β P and β N, in symbols apr βp (X), apr βn (X) respectively, are defined as where, apr βp (X) =pos βp (X) (11) apr βn (X) =U neg βn (X) (12) pos βp (X) = {E R C c(e,x) β P } (13) neg βn (X) = {E RC c(e,x) β N } (14) and RC = {E 1,E 2,...,E N } is the C-elementary sets. Similarly, the boundary region bnd βp, β N (X) ofx is composed of those elementary sets, which are neither in the positive region pos βp (X), nor negative region neg βn (X) ofx, bnd βp, β N (X) = {E RC β N < c(e,x) < β P, } (15) In this way, the accuracy measure of set X in the approximation space A =(U, R) is given by x apr (X) w x βp α(x) = x apr βn (X) w (16) x The difference between the two rough sets can be shown in Figs.4 and 5. Let s pay attention on the boundary regions. Compared with the one in Fig.4, its corresponding part is great reduced in Fig.5, where the elementary sets with arrows outward subset X go to the negative region while the others with arrows inward X go to the positive region. Regarding the fuzzy rough set, we would like to give the following remarks: 2QUKVKXG4GIKQP $QWPFCT[4GIKQP 0GICVKXG4GIKQP Figure 5: The three regions in the fuzzy rough set The traditional IS is a special case of the UIS, as well as rough sets. Namely, IS =(U, C, D, V, ρ, W ) (17) where W = x U w x, and w x = 1. In other words, in the traditional IS, all data have equal evaluations (weights). Besides, positive region pos(x), and negative region neg(x) are the special cases of pos βp (X), and neg βn (X), respectively, where, c(e,x) = pos(x) =pos 1 (X) = {E RC c(e,x) =1} (18) neg(x) =neg 0 (X) = {E RC c(e,x) =0} (19) x I w x x E w x ; I = E X (20) Compared with the UIS proposed in [5], there are two main differences in this paper: (1) For each tuple, there are two parameters (d, g) that should be set up in which d is a importance function: U [0, 1] corresponding to the condition attribute set C, and g is a certainty function: U [0, 1] corresponding to the decision attribute set D. How to exactly set up the parameters? Of course, you can say g = 0.67,d = 0.70, however, there is not a clue to set up the parameters like that. In this paper, for every tuple

5 (example) data, there is only one parameter to be set up based on the situation such as noise, decision confidence and so on, and to improve the flexibility, (2) fuzzy number w x is adopted. In this way, the fuzzy membership function can cover the inaccurate parameter setting to a large extent. For example, you are difficult to say that w is 0.78, but it is easer to say that w is about Even though the real value w is exactly 0.78, and you set as w is about 0.8, such a missetting surely is covered by the membership function. For the fuzzy number setting, we can consider to use triangular fuzzy numbers (TFN) from 0 to 1 as shown in Fig.6. Namely, fuzzy number 1 is employed if the weight is very high, whereas fuzzy number 0 is employed if the weight is very low. Also, according to the decision situation, the width l of the membership function flexibly varies. In other words, for a decision, if the corresponding noises is larger or confidence is weaker, l is set bigger, and vice versa. Besides, there is another reason for employing such a TFN, i.e., it is easier to perform the four rules of arithmetic, compared with other kind of fuzzy membership functions like the Gaussion one. In addition, in order to give the order of two fuzzy sets, a socalled removal method [8] is available. 1 µ w ~ x w ~ x 0 x0 l Figure 6: Triangular fuzzy membership function 4 GA-Based Attribute Division In the example in section 2, regarding the attribute values of temp, we used that V temp = {0, 1, 2} ={ normal, high, very high }. Actually, the temperatures of patients are continuous. For example, temperatures of six patients (temp(p 1 ) temp(p 6 )) are given in Tab.2. In order to divide the continuous Table 2: Real temperatures ( o C) patient p 1 p 2 p 3 p 4 p 5 p 6 temp attribute values into a discrete (or digital) attribute values like (0, 1, 2), one of the most common method is to give some appropriate intervals, each of which represents one discrete value like 1 or 2. Here, one case is shown in Fig.7. Obviously, the discrete attribute values of temp in Tab.2 matches the division x Figure 7: One case of division shown in Fig.1. However, if we change such a division like Fig.7 Figure 8: Another case of division Table 3: Influenza data with different conversion Q U C D temp sneeze headache flu p p p p p p to Fig.8, then Tab.1 will become Tab.3 in which the (discrete) attribute values of temp are different form Tab.1. In this case, regarding the same subset X =(p 1,p 2,p 4 ), apr(x) = {p 1,p 2,p 4 } apr(x) = {p 1,p 2,p 4 } α(x) = 1 Clearly, the approximation accuracy is improved. Therefore, even with a same original database in which continuous attribute values are contained, the different division has a different approximation accuracy, which directly influence the rules extracted from the database. Consequently, when we consider such a division for the originally continuous attribute values, there are two problems we have to answer: (1) How to divide the continuous values into some intervals, each of which corresponds to a discrete number? (2) How many intervals should be taken? In this paper, to solve the problems above, here we employ GA, which has been widely used in various problems as a robust search method, especially in optimum seeking. As an important branch of evolutionary computation, GA is characterized by its current effectiveness, strong robustness, and simple implementation. It also has the advantage of not being restrained by certain restrictive factors of search space. GA simulates

6 the evolutionary process of a set of genomes over time. Genome is a biological term that is corresponding with as a set of genes and gene is the basic building block of any living entity. For our use now, genome represents two-figure hexadecimal such as A8, which finally can be transferred to the division points like 37.5 and 38.5 in Fig.8, and gene represents a binary digit in the binary coded hexadecimal code (BCHC) such as , which is the BCHC of A8. A GA starts with a set of genomes, which is referred to as a generation, created randomly and then the evolutionary process of the survival of the fittest genomes takes place. The unfitted genomes are removed and the remaining genomes reproduce a new set of genomes. Reproduction of the genomes is accomplished by applying the simulation of the two well-known genetic processes: mutation and crossover. This process is repeated and in each repetition a fitter generation is created. To fit the use in this paper, our GA is composed of the following steps: Step 1. Find the biggest value x max and the smallest value x min in the continuous attribute values x to be considered, and evenly divide [x min,x max ] into seven interval so that 8 points correspond to a BCHC. Actually, the 8 points are candidates of division points. Binary digit 1 in BCHC means that it is a division point, and 0 means that it is not a division point. For example, a hexadecimal 4C leads to a division shown in Fig xmin xmax Figure 9: Division of a BCHC Step 2. Create a set of N random BCHC (first generation of genomes). Step 3. Calculate the fitness of each BCHC. Each BCHC leads to an information system like Tab.1 or Tab.3, therefore for a subset X to be approximate, there are different approximation accuracy α(x). Here we use α(x) as the fitness. Naturally, a better accuracy presents a better fitness. In Fig.9, the interval number is 3, which equals the sum of binary digit 1 in BCHC(4C). We may suppose that the maximum number Itvl max of intervals is given. So when we calculate the fitness of each BCHC, we will give the worst fitness, say 0, if the sum of binary digit 1 in a BCHC exceeds Itvl max. GA goes to the end if the desired fitness is obtained. At the same time, the problem (2) described in previous sub-section is resolved, namely, the sum of binary digit 1 in a BCHC with the best fitness is interval number we should take. Step 4. Sort the BCHCs based on their fitness in descending order. Step 5. Keep M (M < N) fitter BCHCs and remove the rest of the BCHCs. Step 6. Create the next generation by making N M BCHCs out of M BCHCs using crossover and mutation operations. Step 7. Go to step 3. 5 Conclusion In this paper, an uncertain information system (UIS), in which fuzzy sets are employed to cover the certainties, importance, and classification factors, is proposed. Afterward, a new fuzzy rough model is led out based on the UIS. Consequently, the possible rules can be reduced whereas the negative/positive rules are increased in the rule extraction from the UIS. As a result, the approximation precision can be improved. In addition, the genetic algorithm is adopted to divide the attribute values that have continuous values into the most proper discrete values in order to improve the approximateness of the system. References: [1] Z. Pawlak, Rough sets, Int. J. of Information and Computer Sciences, vol. 11, no. 5, pp , [2] Z. Pawlak, Rough classification, Int. J. of Man-Machine Studies, vol. 20, pp , [3] W. Ziarko, Variable precision rough set model, Journal of Computer and System Sciences, no. 46, p , [4] W. Ziarko, Rough Sets, Fuzzy Sets and Knowledge Discovery, Springer-Verlag, 1994 [5] J. Han, X. Hu, and N. Cercone, Supervised learning: a generalized rough set approach, Proceedings of the Second International Conference on Rough Sets and Current Trends in Computing (RSCTC 2000), pp , Banff, Canada, October, [6] Z. Pawlak, S. K. M. Wong, and W. Ziarko, Rough sets: probabilistic versus deterministic approach, Int. J. Man-Machine Studies, vol. 29, pp , [7] H. Han, Y. Morioka, and K. Takano, Rule extraction using GA-based fuzzy modeling, Advances in Intelligent Systems, Fuzzy Systems, Evolutionary Computation, A. Grmela, and N. E. Mastorakis (eds), pp , WSEAS, [8] A. Kaufmann, and M. M. Gupta, Fuzzy Mathematical Models in Engineering and Management Science, Elsevier Science Publishers B.V., 1988.