Analysis of Classification in Interval-Valued Information Systems

Transcription

1 Analysis of Classification in Interval-Valued Information Systems AMAURY CABALLERO, KANG YEN, EDUARDO CABALLERO Department of Electrical & Computer Engineering Florida International University W. Flagler Street, Miami, Florida USA Abstract. The problem of classification has been studied by many authors, and different methods have been developed. Frequently the data for the different attributes is obtained from sources, where the obtained values for each attribute may change due to noise, equipment imprecision, etc. Under this condition becomes useful to define intervals, where those parameters may change values. Using these intervals, and applying rough sets, it is possible in many cases to optimize or reduce the number of attributes that will be used in the classification. When the object can not be completely determined from the rules obtained from the rough set analysis, fuzzy logic results a useful approach. This paper analyzes the conditions for which the classification is possible using a combination of rough sets and fuzzy logic. Examples are included for clarification purposes. Key Words: Fuzzy Logic, Rough Sets, Classification, Interval-valued Information Systems 1 Introduction When analyzing an information system or a database, frequently we face problems lie attributes redundancy, missing or diffuse values, which are due in general to noise and missing partial data. Rough set theory is a very useful approach for minimizing the number of attributes necessary to represent the desired category structure by eliminating redundancy. The lac of data or complete nowledge of the system maes developing a model a practically impossible tas using conventional means. This lac of data can be attributed to sensors failure, or simple due to incomplete system information. At last, diffuse values could be related to noise or imprecise measurements from sensors. In many applications, the information is collected from different sensors, which are corrupted by noise and outliers. The present wor analyzes the use of combined rough sets and fuzzy logic in the classification tas. Limitations in the process of classification due to the imposed data input conditions are presented. Examples have been included to demonstrate the concepts of using rough and fuzzy sets, in classification applications. Rough sets theory, developed by Pawla [1, 2], can be used as a tool to recover data dependencies and to reduce the number of attributes contained in a given data set by using the data alone, without additional information [3,4]. In many practical cases lie for example, when we are getting information from a group of sensors in robotics, image processing, etc, the received information, representing the same value, may vary in some interval due to external causes. For dealing with interval-valued information systems one frequently used procedure is the discretization. Yee Leung et al. [5] have presented a very useful method for obtaining rules, based on rough sets for using the minimum number of attributes and giving a first approach in the objects classification. Their method can be extracted in the following steps: ISSN: ISBN:

2 2 Attribute Reduction Using Rough Sets 1. Table preparation: From the original table, a new one presenting the minimum and maximum values for each parameter and for each object is generated. 2. Define the misclassification rates (α ij ), where ij α denote the misclassification error where l i, l j are the minimum values and u i, u j are the maximum values. For clarifying the concept, a numerical example is presented in Fig.1. The region of intersection of the values of the two objects represents the zone, where the classification becomes problematic. For the presented case, the misclassification error between the classes i and j for attribute is given by α ij = min{( , )/( )} = 0.57 Note that in general α ij α ji. 3. Define α ij as the error that class u i being misclassified into class u j in the system. α ij = min { α ij : m}, where m is the maximum number of attributes 4. Find out the maximum mutual classification error between classes (β ij ) β ij = max { α ij, α ji } where β ij = β ji. 5. For each pair of classes, find out the permissible misclassification rate between classes u i & u j in the system β ij = min β ij for 1 m Let s define the permissible misclassification rate between classes as α. If β ij α, there must exist an attribute A so that, by using A, the two classes U i and U j can be separated within the permissible misclassification rate α. between the classes i and j for attribute, given by α ij = 0 if [l i, u i ] I [l j, u j ] = 0; α ij = min{(u i l j, u j l i )/(u i l i )},1}, if [l i, u i ] I [l j, u j ] 0, 6. Prepare a table with α-tolerance relations, locating: 1- when U i & U j can not be separated (all β ij > α); 0 - when U i & U j can be separated (all β ij α). This is the α-tolerance matrix. 7. Find out the discernibility matrix: For each U ij, locate the parameters for which β ij α. 8. Find out the minimal implicants: f i α = { D ij α } 9. Write the rules: For each class, find out the minimal implicants. These are the rule antecedents. The consequents will be given by the classes (U ij ) that presented 1 in their correspondent position of the α-tolerance matrix. 3 Possible Situations In this tas solution it can appear three different situations: Case # 1.- All the misclassification errors α ij are smaller than the permissible misclassification rate between classes α or are equal to zero in the limit. In this case each object is classified without any possible error or with an error given by α. Step 1: One example reflecting this situation is given in Table 1 that represents the minimum and maximum values of five parameters (A ) for five different objects (U i ). ISSN: ISBN:

3 l i = 3.2 u i = 3.9 l j = 2.8 u j = 3.6 Fig.1. Region of Coincidence of Classes i and j for the Attribute Step 2: The obtained values of the misclassification rate of objects U1 and U2 for the parameters A for 1 5 (α 12 ) are calculated as follows: α 1 12 = 0.5; α 2 12 = 0.07; α 3 12 = 0.21; α 4 12 = 0.46; α 5 12 = 0. From the obtained results, it is clear that the minimum value is α 5 12 = 0. Following the same procedure with all the objects, it is possible to get α ij Step 3: Definition of α ij. These values are obtained in Table 2. Step 4: β ij = max { α ij, α ji } = 0 Step 5: β ij = min βij = 0 Step 6: α-tolerance Relation Matrix: As all the β ij = 0, it is possible to differentiate each object from all the others so, this matrix will have 1 in the positions, where i = j similarly to Table 2. Step 7: Discernibility Matrix: the parameters for which β ij = 0 are presented in Table 3. Step 8: Minimal Implicants: Obtained from each column for I α i = { D α ij } I 1 = (A 1 A 5 ) (A 2 A 5 ) I 2 = (A 1 A 5 ) (A 2 A 5 ) I 3 = A 1 (A 2 ) (A 2 A 4 ) I 4 = (A 1 A 5 ) (A 2 A 5 ) I 5 = A 1 (A 2 ) (A 2 A 4 ) Step 9: The following rules can be selected.. Rule # 1: If A 1 [4.1, 4.9] and A 5 [4.8, 5.6], Then the object can be classified as U 1 Rule # 2: If A 1 [4.5, 5.2], A 3 [3.3, 4.8] and A 5 [3.9, 4.7], Then the object can be classified as U 2 Rule # 3: If A 1 [2.1, 2.8], Then the object can be classified as U 3 Rule # 4: If A 1 [3.1, 5.4], A 3 [1.9, 2.8] and A 5 [2.4, 4.0], Then the object can be classified as U 4 Rule # 5: If A 1 [1.5, 2.0], Then the object can be classified as U 5 The objects are uniquely classified using only three parameters: A 1, A 3, and A 5. A 2 and A 4 are not necessary for the classification. Case # 2: Not all the misclassification errors α ij are smaller than the permissible misclassification rate between classes α. One example of application of this situation is the iris classification problem. R. Fisher [6] presents the classes Setosa, Versicolor, and Virginica, defined by: SL sepal length, SW sepal width, PL petal length, and PW petal width. The same classes will be used in our example,which was previously presented in [7]. We calculate the mean, as well as the minimum and maximum for each class included. The results are presented in Table 4 ISSN: ISBN:

4 Table 1. Example of an Interval-Valued Information System A 1 A 2 A 3 A 4 A 5 U 1 4.1, , , , , 5.6 U 2 4.5, , , , , 4.7 U 3 2.1, , , , , 5.4 U 4 3.1, , , , , 4.0 U 5 1.5, , , , , 4.8 Table 2. Error of Misclassification of Object U i into U j α ij U 1 U 2 U 3 U 4 U 5 U U U U U Table 3. Discernibility Matrix U 1 U 2 U 3 U 4 U 5 U 1 1 {A 5 } {A 1, A 2 } {A 5 } {A 1, A 2, A 4, A 5 } U 2 {A 5 } 1 {A 1, A 2 } {A 3 } {A 1, A 2 } U 3 {A 1, A 2} {A 1, A 2 } 1 {A 1, A 2 } {A 1, A 3, A 4 } U 4 {A 5 } {A 3 } {A 1, A 2 } 1 {A 1, A 2 } U 5 {A 1, A 2, A 4, A 5 } {A 1, A 2 } {A 1, A 3, A 4 } {A 1, A 2 } 1 Table 4. Attributes for Each Class Attributes Setosa Versicolor Virginica x av Min Max x av Min Max x av Min Max SL (A 1 ) SW (A 2) PL (A 3) PW (A 4 ) Following the steps in a similar way to the previous case, it is obtained that the error that objects in class U i being misclassified into class U j in the system is defined as A ij = min{a ij : m} and are given in Table 5. Table 5. Error of Misclassification of Object U i into U j α ij U 1 U 2 U 3 U U U Selecting α = 0.2, the permissible misclassification rate for the present example is shown on Table 6. Table 6. Permissible Misclassification Rate between Classes U i and U j β ij U 1 U 2 U 3 U U U ISSN: ISBN:

5 0.2 The matrix T A for the α-tolerance relations, where all β ij > α is represented in Table 7 Table 7. α-tolerance Relations U 1 U 2 U 3 U U U T A 0.2 From Table 7, it is clear that object U 1 Setosa can be uniquely defined from the given attributes, but objects U 2 Versicolor and U 3 Virginica may not be separated. This situation can be expressed by S A 0.2 (U 1 ) = {U 1 } S A 0.2 (U 2 ) = S A 0.2 (U 3 ) = {U 2, U 3 } where S A 0.2 (U) denotes that these are the sets of objects which are possible indiscernible by A within U, within the misclassification rate α = 0.2. The 0.2-discernibility set is given on Table 8. Table 8. Discernibility Set U 1 U 2 U 3 U 1 1 A 3, A 4 A 3, A 4 U 2 A 3, A 4 1 U 3 A 3, A 4 1 The obtained function is f = A 3 A 4 Using rough sets, it has been demonstrated that the important attributes for the classification are: A 3 : PL-petal length, and A 4 : PW-petal width. From the previous results, the following rules can be extracted: Rule # 1: IF A 3 [1, 1.9] or A 4 [0.1,0.6] THEN it is U 1 Setosa. Rule # 2: IF A 3 [3.0, 5.1] or A 4 [1.0, 1.8] THEN it can be U 2 Versicolor or U 3 Virginica. Rule # 3: If A 3 [4.5, 6.9] or A 4 [1.4, 2.5] THEN it can be U 2 Versicolor or U 3 Virginica. Rule 1 is clear for Setosa classification. Note that from Rule # 2 and Rule # 3, it is possible to develop other rules that substitute them: Rule # 4: IF A 3 [3.0, 4.5] or A 4 [1.0, 1.4] THEN it is U 2 Versicolor. Rule # 5: If A 3 [5.1, 6.9] or A 4 [1.8, 2.5] THEN it is U 3 Virginica. Rule # 6: If A 3 [4.5, 5.1] or A 4 [1.4, 1.8] THEN it can be U 2 Versicolor or U 3 Virginica. In order to select between U 2 and U 3, into the coincident interval, one possibility is to use fuzzy logic. E. D. Cox has presented a useful method for dealing with this situation using the Compatibility Index (CI) [8]. The authors propose the following procedure: 1) For each of the objects U i in the fired rule, find the degree of membership with the imposed conditions, for each of the participating attributes A. 2) Find the compatibility index (CI) for each object. 3) Compare the different compatibility indexes and select the object with the greater one. For the example, it was proposed for each interval a triangular membership function with the maximum value coincident with the mean value for the interval, and defining the domain from the minimum to maximum values in the interval. The compatibility indexes are calculated using for this case, a single measurement, which is given by: SL (sepal length) = 5.6 SW (sepal width) = 3.0 PL (petal length) = 4.5 PW (petal width) = 1.5 ISSN: ISBN:

6 Rule Rule # 6 has been used taing into consideration only the petal length (A 3 ) and petal width (A 4 ), as stated by the rule. In this case the obtained compatibility indexes for versicolor and virginica are respectively 0.7 and 0.1. From the two compatibility indexes, it is clear that the selection is versicolor. A test was made to all the objects in the table used in the example [6]. In this case, there are 50 virginica and 50 versicolor examples. The algorithm fails in one versicolor and one virginica. This gives an average classification rate of 98% for the analyzed table. Case # 3: All the misclassification errors α ij are bigger than the permissible misclassification rate between classes α. In this case the only possibility is to use fuzzy logic or any other method that will permit, using all the parameters, compare the different objects and based on the comparison, define the one, which presents the smaller differences with the imposed parameters. 4 Conclusions The attribute minimization using rough set theory is extremely useful when dealing with large databases. If the number of possible values for the attributes is large, the selection of interval values is mandatory. Fuzzy logic can be used together with the rough theory for obtaining a unique response, in case where this is not possible using the rough theory alone. Three possible situations are presented based on the misclassification error. In the first two, the number of attributes can be minimized and it is possible as well, to classify the classes using rough sets and fuzzy logic. In the third case, it is not possible to reduce the number of parameters for some fixed permissible misclassification rate between classes α. And the only possibility is to apply fuzzy logic or any other method looing for similarity between the received information and the different classes. References [1] Z. Pawla, Rough Set, Int l J. of Computer & Information Sciences, No. 11, 1982, pp [2] Z. Pawla, Rough Sets: Theoretical Aspects of Reasoning About Data, Dordrecht: Kluwer, [3] E. A. Rady, et Al. A Modified Rough Set Approach to Incomplete Information Systems, Journal of Applied Mathematics and Decision Sciences, Volume 2007, Article ID [4] Feng-Hsu Wang, On Acquiring Classification Knowledge from Noisy Data Based on Rough Sets, Elsevier, Expert Systems with Applications 29(2005) [5] Y. Leung, et Al., A Rough Set Approach for the Discovery of Classification Rules in Interval-valued Information Systems, Int l J. of Approximate Reasoning, No. 47, 2007, pp [6] R. Fisher, The Use of Multiple Measurement in Taxonomic Problem, Ann. Eugenic, No. 7, 1936, pp [7] A. Caballero, K. Yen,Y. Fang. Classification with Diffuse or Incomplete Information. WSEAS Transactions on Systems and Control, Issue 6, Vol. 3, June pp [8] E.D. Cox, Fuzzy Logic for Business and Industry, Rocland: Charles River Media, ISSN: ISBN: