A Possibilistic Decision Logic with Applications

Transcription

1 Fundamenta Informaticae 46 (2001) IOS Press A Possibilistic Decision Logic with Applications Churn-Jung Liau C Institute of Information Science Academia Sinica Taipei, Taiwan liaucj@iis.sinica.edu.tw Duen-Ren Liu Institute of Information Management National Chiao-Tung University Hsinchu, Taiwan dliu@iim.nctu.edu.tw Abstract. In this paper, we investigate a knowledge representation formalism in the context of fuzzy data tables. A possibilistic decision logic incorporating linguistic terms is proposed for representing and reasoning about knowledge in fuzzy data tables. Two applications based on the logic are described. The first is the extraction of fuzzy rules from general fuzzy data tables. In this application, the knowledge in the tables may be made explicit by the formulas of the logic or used implicitly in decision-making. The second is for the fuzzy quantization problem of precise data tables. It can be viewed as a special case of the first, however, due to some special properties of the problem, a polynomial time rule extraction process can be obtained. Finally, the relationship of the logic with some works for handling uncertain information in data tables is also discussed. Keywords: Data table, decision table, decision logic, possibilistic decision logic, data analysis, rough set theory. Some preliminary results of the paper have appeared in [8]. C Corresponding author

2 200 C.J. Liau, D.R. Liu / A Possibilistic Decision Logic with Applications 1. Introduction Theory of knowledge has been a commonly important topic of many academic branches such as philosophy, psychology, economics, and artificial intelligence, whereas the storage and retrieval of data is the main concern of information science. In the modern experimental science, knowledge is usually acquired from observed data. The data can provide the causal-effect or associational relationship between attributes of the observed objects. However, when the amount of data is large, it becomes a difficult task to analyze the data and extract knowledge from them. With the aid of computers, the large amount of data stored in relational data tables can be transformed into symbolic knowledge automatically. To represent and reason about the knowledge from the data tables, a decision logic based on rough set theory is proposed in [13]. The semantics of the logic is defined in a Tarskian style through the notions of models and satisfaction in the context of data tables. A strong assumption about data tables is that each object takes exactly one value with respect to an attribute. However, in practice, we may have only incomplete information about the values of an object s attributes. Thus, more general data tables and decision logics are needed to represent and reason about incomplete information. For example, set-valued or interval set-valued data tables are respectively introduced in [5, 6, 7, 10] and [20] and a generalized decision logic accompanied with the latter is also proposed in [20]. In these formalisms, the attribute values of an object may be a subset or an interval set in the domain. Since crisp subsets and interval sets are both special cases of fuzzy sets, the further generalization to fuzzy data tables is desirable for the representation of uncertain information. In fuzzy data tables, an object can take a fuzzy subset of values for each attribute. Thus, the decision logic also have to be further generalized. The aim of the paper is to provide such a logic for reasoning about knowledge in fuzzy data tables. Though the logic is proposed for fuzzy data tables, it is also useful for summarizing the information in precise data tables. The notion of partition and equivalence relation is central to the analysis of data tables. The data items are partitioned into equivalence classes according to their attribute values. However, when some attributes have large number of possible values (e.g. numerical values), the number of generated classes is inevitably enormous. To cope with the difficulty, some quantization or discretization step must be adopted at the data preprocessing stage. In general, a value is replaced by a subset of the domain of values in the quantization process. For example, a number is replaced by an interval. However, the intervals seldom have linguistic meaning, so the induced rules described by them are not very understandable to the human user. As indicated by Zadeh[22], human usually compute with words instead of numbers, so if we can replace the values by some linguistically meaningful terms, then the induced rules may be more useful to the human decision-makers. The logic proposed in this paper provides a representation formalism for expressing the more natural rules. In what follows, we will first review the basic notions of data tables, decision logic, and possibility theory. A possibilistic decision logic is then proposed as the knowledge representation formalism for fuzzy data tables. The syntax, context, and semantics of the logic will be described. Two applications of the logic will be considered. In the first one, the logic is used in the representation of rules extracted from fuzzy data tables. Some factors influencing the efficiency of the rule extraction process are discussed. Though a large amount of rules can be extracted from a fuzzy data table, there may be only a small part of them useful in a real situation. Thus we also consider the use of the logic in a decision-making environment where we do not have to extract all rules in advance. Instead, we will use the knowledge in the fuzzy data tables implicitly, so it will be more efficient. In the second application, we consider the

3 C.J. Liau, D.R. Liu / A Possibilistic Decision Logic with Applications 201 special case when the data tables are precise but with large domains of values for some or all attributes. In such application, a quantization process is in general needed. The fuzzy quantization process is to replace a precise value with some linguistic labels describing it appropriately. Since more than one linguistic labels may be appropriate for replacing a value, we will keep the data table unchanged but try to extract fuzzy rules from the data tables by using the logic. In such special case, a polynomial time complexity can be obtained for the extraction process. Finally, the relationship of the logic with some works for handling uncertain information in data tables is also discussed. 2. Preliminaries 2.1. Data tables and decision logic A data table is usually used as a regular approach for storage of data. A data table (DT) or knowledge representation system (KRS)[13] is a pair S = (U, A), where U is a nonempty, finite set of objects (the universe) and A is a nonempty, finite set of primitive attributes. Every a A is a total function a : U V a, where V a denotes possible values of a. In data analysis, a special kind of data tables, called decision table, is particularly useful. A decision table is a data table S = (U, A), where A can be partitioned into two sets C and D, called condition attributes and decision attributes respectively. By data analysis, decision rules relating the condition and the decision attributes can be derived from the table. The resultant rules are represented as implication formulas in the so-called decision logic (DL). An atomic formula of DL is a descriptor (a, v), where a C D is an attribute and v V a, the domain of values for a. The well-formed formulas (wff) of DL is the smallest set containing the atomic formulas and closed under the Boolean connectives,,, and. A (deterministic) decision rule is then an implication in DL of the form (a, v a ) (d, v d ). (1) a C Intuitively, each rule says that if the condition attributes fulfill some values, then some particular decisions specified in the consequent will be adopted. More formally, given a DL and a DT S = (U, A), the satisfaction relation = S between x U and wffs of DL is defined inductively as follows. 1. x = S (a, v) iff a(x) = v 2. x = S ϕ iff x = S ϕ 3. x = S ϕ ψ iff x = S ϕ and x = S ψ 4. x = S ϕ ψ iff x = S ϕ or x = S ψ 5. x = S ϕ ψ iff x = S ϕ ψ 6. x = S ϕ ψ iff x = S (ϕ ψ) (ψ ϕ) d D

4 202 C.J. Liau, D.R. Liu / A Possibilistic Decision Logic with Applications We will say that a wff ϕ is true in the table S, denoted by S = ϕ, if x = S ϕ for all x U. For any wff ϕ, the truth set of ϕ in S is defined as Then ϕ is true in S iff ϕ S = U. ϕ S = {x U x = S ϕ} Possibility theory Possibility theory is developed by Zadeh from fuzzy set theory[21]. Given a universe W, a possibility distribution on W is a function π : W [0, 1]. A possibility distribution π is called normalized if sup w W π(w) = 1. Obviously, π is a characteristic function of a fuzzy subset of W. Two measures on W can be derived from π. They are called possibility and necessity measures and denoted by Π and N respectively. Formally, Π, N : 2 W [0, 1] are defined as Π(A) = sup π(w), w A N(A) = 1 Π(A), where A is the complement of A with respect to W. These two measures correspond to our uncertainty about the crisp event A when a piece of vague information π is available, and they can be extended to measure the uncertainty about fuzzy events[2]. Let F(W ) denote the set of all fuzzy subsets of W, then the extended measures, still denoted by Π and N, are defined as Π, N : F(W ) [0, 1], Π(X) = sup min(µ X (w), π(w)), w W N(X) = inf w W max(µ X(w), 1 π(w)), where µ X is the membership function of X. Furthermore, in fuzzy set theory, two measures for the degrees of intersection and inclusion between fuzzy sets are defined as follows. Let X and Y be two fuzzy subsets of W, then CON(X, Y ) = sup µ X (w) µ Y (w), w W INC(X, Y ) = inf w W µ X(w) µ Y (w), where : [0, 1] [0, 1] [0, 1] is a t-norm 1 and : [0, 1] [0, 1] [0, 1] is the residuated implication function defined as a b = sup{x x a b}. Note that if X is a crisp singleton subset of W, i.e. X = {x} for some x W, then CON(X, Y ) = INC(X, Y ) = µ Y (x). 1 A binary operation is a t-norm iff it is associative, commutative, and increasing in both places, and 1 a = a and 0 a = 0 for all a [0, 1].

5 C.J. Liau, D.R. Liu / A Possibilistic Decision Logic with Applications Possibilistic Decision Logic In a DT, it is assumed that a(x) is exactly known for each object x and attribute a. However, in many real situations, we have only incomplete information about a(x) for some a and x. Thus we have to generalize the DT to fuzzy data tables. A fuzzy data table (FDT) is a pair S = (U, F (A)), where U is a finite set of objects and F (A) = {f a : U F(V a ) a A}. Intuitively, f a (x) denotes the uncertain value of attribute a for object x. Thus f a (x) = V a when the value is missing and f a (x) is a singleton when the value is precise. This means FDT s can represent both precise and imprecise data in a uniform framework. As DL is appropriate for representing knowledge in DT s, we also need a logic for fuzzy data tables. In this section, we propose a possibilistic decision logic(pdcl) for the purpose. The linguistic terms used in the logic are fixed in advance and their meaning is given by a context. Once the context is determined, the semantics of wffs of the logic can be defined via possibility theory Syntax The alphabet of PDCL consists of 1. A finite set of attributes A = {a 1, a 2,...}, 2. for each a A, a finite set of linguistic terms or labels L a, 3. a finite set of linguistic hedges H (typical elements of H are very, fairly, moderately, etc.), 4. and the logical symbols π, ν,,,, and. The symbols π and ν correspond to possibly and certainly respectively. Punctuation symbols like (, ), and, are used conveniently. Without loss of generality, we can assume that L a and L b are disjoint if a b. Let L = a A L a, then a function type : L A assigning each linguistic term with its type is defined as type(l) = a if l L a. The set of well-formed formulas is defined inductively in the following way. 1. If l L, type(l) = a, and τ H, then (a, πl), (a, νl), (a, τ πl), and (a, τ νl) are atomic formulas. 2. If ϕ and ψ are wffs, so are ϕ, ϕ ψ, ϕ ψ, ϕ ψ, and ϕ ψ For example, (t, νhigh), (t, πhigh), (t, veryνhigh), and (t, veryπhigh) denote respectively the temperature is certainly high, the temperature is possibly high, it is very certain the temperature is high, and it is very possible the temperature is high Context It is well-known that many natural language terms are highly context-dependent. For example, the term tall may have quite different meanings between a tall basketball player and a tall child. To model the context-dependency, we associate a context with each PDCL. The context determines the domain of values of each attributes and assigns appropriate meaning to each linguistic term and hedge. Formally, a context associated with a PDCL is a triple ({V a } a A, m 1, m 2 ), where V a is a domain of values for each

6 204 C.J. Liau, D.R. Liu / A Possibilistic Decision Logic with Applications a A, m 1 is a function on L such that m 1 (l) F(V a ) if type(l) = a, and m 2 : H ([0, 1] [0, 1]) is a function mapping each hedge to a function from [0, 1] to [0, 1]. While the domains V a and m 1 are totally determined by the users or linguistic experts to reflect the intended meaning of these attributes and linguistic terms, there exist some common definitions for the linguistic hedges in the literature[11]. Some typical examples are as follows. 1. m 2 (very)(x) = x 2, for all x [0, 1], 2. m 2 (fairly)(x) = x, for all x [0, 1], 3. m 2 (absolutely)(x) = { 1, ifx = 1, 0, otherwise Semantics Let L denote the set of wffs of a PDCL with set of attributes A, set of linguistic terms L, set of linguistic hedges H, and a context ({V a } a A, m 1, m 2 ), and S = (U, F (A)) be an FDT,then we can define the truth valuation function E S : U L [0, 1] as follows: 1. E S (x, (a, πl)) = sup v Va µ m1 (l)(v) µ fa(x)(v) 2. E S (x, (a, νl)) = inf v Va µ fa(x)(v) µ m1 (l)(v) 3. E S (x, (a, τπl)) = m 2 (τ)(e S (x, (a, πl))) 4. E S (x, (a, τνl)) = m 2 (τ)(e S (x, (a, νl))) 5. E S (x, ϕ) = 1 E S (x, ϕ) 6. E S (x, ϕ ψ) = E S (x, ϕ) E S (x, ψ) 7. E S (x, ϕ ψ) = E S (x, ϕ) E S (x, ψ) 8. E S (x, ϕ ψ) = (1 E S (x, ϕ)) E S (x, ψ) 9. E S (x, ϕ ψ) = E S (x, ϕ ψ) E S (x, ψ ϕ) where is a t-conorm defined by a b = 1 (1 a) (1 b) Note that E S (x, (a, πl)) = CON(m 1 (l), f a (x)) and E S (x, (a, νl)) = INC(f a (x), m 1 (l)) according to the definition. We define [ ϕ ] S = x U E S(x, ϕ) as the truth degree of ϕ with respect to an FDT S. Sometimes, we will omit the subscript S when no confusion occurs. For any wff ϕ, let π ϕ denote the possibility distribution on U such that π ϕ (x) = E(x, ϕ) and ϕ denote the fuzzy subset of U with membership function µ ϕ (x) = E(x, ψ). ϕ is also called the truth set of ϕ.

7 C.J. Liau, D.R. Liu / A Possibilistic Decision Logic with Applications Application to Rule Representation 4.1. Rule evaluation and selection Any wffs of the PDCL can represent knowledge in an FDT, however, we will be in particular interested in some kind of decision rules. Let p a = (a, πl), (a, νl), (a, τπl), or (a, τνl) be an atomic formula, called a-basic formula, then a CD-decision rule for C, D A is a wff of the form p a p d. (2) a C When ϕ is a CD-decision rule, [ ϕ ] will be the strength of the rule according to our semantics. The left-hand side of (2) is called the antecedent of the rule and the right-hand side the consequent. Theoretically speaking, we can evaluate the strength of any rules in the form (2). However, since the number of such rules may be enormous, we would like to seek some rules which may be especially interesting from the data analysis viewpoint. First, we are in particular interested in the case where D = 1. In other words, we will evaluate the rules in the following form d D p a p d. (3) a C The rationale is based on the assumption that the decision attributes are usually independent, so a multiple attributes decision task can be decompose into some single attribute ones. Furthermore, we can take [ ϕ p d1 ] [ ϕ p d2 ] as an approximation of [ ϕ (p d1 p d2 ) ]. When the t-norm = min, the decomposition is exact since E(x, ϕ (ψ 1 ψ 2 )) = min(e(x, ϕ ψ 1 ), E(x, ϕ ψ 2 )). Thus the number of possible rules would depend on that of possible antecedents. Let ϕ and ψ denote respectively the antecedent and consequent of a CD-decision rule, then when = min, it can be seen that [ ϕ ψ ] = N ϕ ( ψ ) = N(ψ ϕ), where N ϕ is the necessity measure corresponding to π ϕ. A wff ϕ is said to be normalized if sup x U π ϕ (x) = 1. In other words, ϕ is normalized iff it induces a normalized possibility distribution on U. If ϕ(resp. π ϕ ) is not normalized, then it induces a kind of partial inconsistency according to possibilistic logic[1]. The inconsistency degree of ϕ(resp. π ϕ ) is defined as 1 sup x U π ϕ (x). The more inconsistent the antecedent is, the less significant the induced rule will be. When the inconsistency degree of ϕ is equal to 1, it does not make sense to extract the rule ϕ ψ from an FDT since it is always vacuously true. Thus we will restrict our attention to the extraction of rules with antecedent not inconsistent to a degree more than some threshold α < 1 2. When α is set to 0, only rules with normalized antecedent will be extracted. The restriction is somewhat a minimal requirement to the support of a rule. In a classical DL rule like (1), the support is defined as the number of elements in U that satisfy the antecedent. Analogously, when ϕ ψ is a CD-decision rule in PDCL, we can define its β-support for β > 0 as the cardinality of the set {x U E(x, ϕ) β}. Thus the restriction above is equivalent to extracting only rules with nonzero (1 α)-support. In what follows, we will assume the threshold for the inconsistency degree of the antecedents is given as α and let β denote 1 α. To find all wffs qualified as the antecedents of extracted rules, we have to search through the set of all wffs of the form a C p a. Let m denote the cardinality of C, then the number of wffs in this form is

8 206 C.J. Liau, D.R. Liu / A Possibilistic Decision Logic with Applications (2 H ) m a C L a since for each a C, the number of a-basic formulas is equal to 2 H L a. At the worst case, all these wffs have inconsistency degree not more than α, then we will have an exponential explosion of extracted rules. Nevertheless, if we can set the inconsistency threshold α appropriately(e.g. let α = 0), then the number of possible antecedents will have a significant reduction. In the next section, we will show a special case where the number of possible antecedents is bound by the cardinality of U. However, we will first consider the search in general case now. To describe the search process, let us fix an enumeration of the set of condition attributes a 1, a 2,..., a m. The search is conducted systematically on a labeled tree of which each node is labeled with a wff of PDCL and each edge is labeled with an a-basic wff. The root of the tree is labeled with. A node is a dead end if the truth set of its labeling wff ϕ has empty β-cut (i.e. ϕ β = ), otherwise the node is open. Each open node labeled with ϕ at level i 1 has 2 H L ai children, each branch to the children is labeled with a distinct a i -basic formula p ai, and the corresponding child node is labeled with ϕ p ai. The search process continues until all attributes are explored. Then all open terminal nodes correspond to a rule antecedent meeting our requirement. More precisely, the labeling wff of an open terminal node is a possible rule antecedent. The search process is summarized in figure 1. Furthermore, the truth sets of the labeling wffs which have been computed in the tree construction process can be kept and used in the evaluation of the strength of the rules Efficiency consideration To carry out the complexity analysis of the rule extraction process, let us define U = n and assume there exists an integer k such that 2 H L a k for all a C {d}. Also recall the number of condition attributes is m. Note that k is the maximal degree of the search tree. Thus the search tree has at most k m+1 nodes except the root. Since at each node, an intersection operation must be executed which takes O(n) time(for the computation of label(n i 1 ) p ai in evaluating the if-condition of the innermost loop), the total time spent in the search process is at most the order O(n k m+1 ). The number of rule antecedents output by the search process is bound by the number of the terminal nodes, which is at most k m, so the number of rules of the form ϕ p d is at most k m+1. Once the truth set of the antecedent is available, the evaluation time for the strength of a rule is O(n), so the rule evaluation phase takes O(n k m+1 ) time in total. When the PDCL language is fixed, the integer k can be viewed as a constant, so the whole rule extraction process will have time complexity O(nk m ). When m is large, the algorithm is obviously infeasible at the worst case. However, it must be noticed that the term k m may be over-estimated in most practical cases. In fact, some further heuristic approaches may be taken to reduce the complexity of the search phase and the number of the resultant rule antecedents. These approaches depends on some factors influencing our search process. First, the complexity is due to the richness of the language. For example, if linguistic hedges are not allowed in the wffs of PDCL, then the number k will be smaller. Furthermore, we can also restrict our attention to the certainty qualified a-basic formulas, i.e., only a-basic formulas of the form (a, νl) are considered. Second, a more important factor is our requirement on the β-support of the extracted rules. We have seen that increasing the value of β may significantly reduce the number of rule antecedents satisfying the requirement. However, for large data tables, the only requirement of nonzero β-support may be too weak. An extracted rule may be meaningful only when its antecedent occurs very frequently. Thus we can set another threshold γ (γ may be an integer or a real number in (0, 1], the former is called an

9 C.J. Liau, D.R. Liu / A Possibilistic Decision Logic with Applications 207 Procedure SEARCH Input: 1. A decision table S = (U, C {d}). 2. Let C = {a 1, a 2,..., a m } and Φ i = {p ai } be the set of all a i -basic formulas for 1 i m. 3. A real number β ( 1 2, 1]. Output: a search tree where each node is labeled with or a wff and each arc with an a i -basic formula for some i. Begin N 0 := the unique node at the level 0 (i.e. the root node) of the output tree label(n 0 ) := for i = 1 to m do for each node N i 1 at level i 1 do endfor endfor End if label(n i 1 ) then for each p ai Φ i do endif Generate a new node N i at level i Add an arc from N i 1 to N i and label the arc with p ai if label(n i 1 ) p ai β then label(n i ) := label(n i 1 ) p ai else label(n i ) := endif endfor Figure 1. The generic search process for possible rule antecedents

10 208 C.J. Liau, D.R. Liu / A Possibilistic Decision Logic with Applications absolute threshold, whereas the latter is relative one) and modify the definition of dead-end nodes so that a node is dead-end if the β-cut of the truth set for its labeling wff has less than γ or n γ elements. That is, the if-condition in the innermost loop of figure 1 is replaced by if γ is an integer and Card( label(n i 1 ) p ai β ) γ Card( label(n i 1 ) p ai β ) n γ if γ (0, 1]. By this way, we can prune a large part of the search tree effectively. This approach is analogous to the use of high frequency tables in [9]. Third, the uncertainty of the data tables may also have influence on the search process. An object with highly uncertain attribute values may satisfy many a-basic formulas and so reduce the possibility of pruning the search tree. For example, in the extreme case, an object with missing values in all of its condition attributes satisfy all a-basic formulas of the form (a, πl), however, the object does not have any supporting force to the extracted rules since it contains no information. Thus we should delete the objects with low information value in advance. To measure the information value of an object, we can use the specificity measures of fuzzy sets proposed in ([17, 18, 19]). Assume X is a fuzzy subset of V, a specificity measure of X, denoted by Sp(X), is a number in [0, 1] such that the following three properties are satisfied 1. Sp(X) = 1 if X is a crisp singleton subset of V, i.e. X = {v} for some v V. 2. Sp( ) = Sp(V ) = If X and Y are two normalized fuzzy subsets of V, then Sp(X) Sp(Y ) if X Y. A very simple manifestation of specificity measure for finite domains is given in [18] as follows. Let v be an element with maximal membership grade in X, i.e., µ X (v ) = max v V µ X (v), then it is defined Sp(X) = µ X (v v v ) µ X(v) V 1 i.e., the specificity measure of X is the maximal membership grade in X minus the average of the membership grades of other elements. The information value of an object u is then defined as IV (u) = max a C Sp(f a (u)), i.e., the maximal specificity of the attribute values of the object. Thus we can set a number θ < 1 2 and delete all objects u such that IV (u) < θ before the search process starts. In summary, the three parameters (β, γ, θ) have influence on the efficiency of the search process. The larger the three parameters are, the more effectively the search tree can be pruned. Finally, if the rule extraction process is too time-consuming after all, an alternative option to the use of the fuzzy data tables is possible. Instead of extracting the rules explicitly, we can use them implicitly in the decision-making environment. Let us consider an FDT S = (U, F (C {d})) such that for each u U, f d (u) is a singleton subset of V d, i.e., each object has precise value in its decision attribute. Each object in the FDT describes a past case in the decision-making environment. The condition attribute values correspond to the situation description of the case and the decision attribute value is the decision taken in that case. Now, if the decision-maker faces a new situation which is described by the linguistic terms

11 C.J. Liau, D.R. Liu / A Possibilistic Decision Logic with Applications 209 a 1 is l 1, a 2 is l 2,, and a m is l m where each a i is a condition attribute and l i L ai is a linguistic term of type a i. Then there are two possible interpretations of the situation description in our logic. By the certainty interpretation, the description is translated into a PDCL wff ϕ 1 = m i=1 (a i, νl i ) and by the possibility one, ϕ 2 = m i=1 (a i, πl i ). Depending on the interpretation the decision-maker chooses, he can take the decision as l = arg max l L d [ ϕ (d, νl) ], where ϕ is ϕ 1 or ϕ 2. Note that according to our assumption on the FDT, the d-basic formulas (d, νl) and (d, πl) has the same evaluation with respect to each object, so we do not have to distinguish them. In fact, in most practical cases, L d is just V d if fuzzy decisions are not allowed, then each element of V d can be viewed as a classical proposition which is true or false in each object. Recall that ϕ will induce a possibility distribution π ϕ on U, then the decision problem is reduced to finding the proposition p in V d with maximal necessity measure N ϕ (p). 5. Application to Fuzzy Quantization There are in general two kinds of attributes in a data table, the nominal ones and the numerical ones. The former is usually with finite domains. For example, the status of a switch may be on or off, the sex of a person may be male or female, etc. On the other hand, numerical attributes often have an infinite domain of values. Even though the domain is finite, its cardinality may be very large. For example, the temperature may be a subset of real numbers. Due to the continuity of the numerical domains, the objects possess proximate values may behave similarly at their decision attributes. For example, two persons have proximate ages may have the similar shopping behavior. Since the data tables are finite, not all possible values of the attributes appear in a table, so we should be able to extrapolate or interpolate the extracted rules to the values not appearing in the table. To solve the data interpolation problem, many quantization techniques has been adopted[12]. The most direct one is the crisp quantization approach. By the technique, for an attribute a, we can partition V a into n a mutually disjoint subsets D 1, D 2,..., D na, and in the decision table, for each x U, f a (x) is replaced by D i if f a (x) D i for 1 i n a. Although the quantization process may reduce the precision of the data, it also effectively hides irrelevant details of the data, so it is useful in summarizing the data. Example 1. Let us consider the following data table with condition attribute t and decision attribute s, where t means the temperature in centigrade degree at a room, and s represents the status of the

12 210 C.J. Liau, D.R. Liu / A Possibilistic Decision Logic with Applications air-conditioner. Let V t = [0, 40] and V s = {0(for OFF), 1(for ON)}. U t s Without quantization, we may extract rules as follows. 1. If t = 10, then the air-conditioner is off. 2. If t = 15, then the air-conditioner is off If t = 40, then the air-conditioner is on. Or in more compact form, the following two rules are obtained. 1. If t = 10 t = t = 24, then the air-conditioner is off. 2. If t = t = 40, then the air-conditioner is on.. If V t is quantized into [0, 20] (20, 25] (25, 40], then we have the following two rules. 1. If t [0, 25], then the air-conditioner is off. 2. If t (25, 40], then the air-conditioner is on. Obviously, the temperature values not appearing in the table (e.g. t = 16) is not covered by the original rules but can be treated in the new ones. Furthermore, the number of disjuncts in the antecedent of the new rules is also less than that for the original ones. The example shows the usefulness of quantization process in data analysis. However, since the intervals do not necessarily correspond to natural language terms, the extracted rules lack a colloquial reading when we try to explain them. According to [4], one of the main challenges to information mining is to close the semantic gap between computer representation of knowledge and human comprehensible concepts. To obtain more meaningful quantization, we may in advance stipulate some linguistic terms as the labels of the resultant classes of the partition, and then the values in the domain are assigned to the respective classes according to the meaning of these linguistic terms. For example, in the example

13 C.J. Liau, D.R. Liu / A Possibilistic Decision Logic with Applications 211 above, we may want to partition V t into three sets, HIGH, MEDIUM, and LOW, then it will be more reasonable to assign 10 into LOW instead of HIGH. Thus semantics of natural language may guide the quantization process. However, even some linguistic terms are given in advance, it is sometimes still difficult to decide the membership of some values. For example, in the example above, we can not easily determine whether 28 should belong to HIGH or MEDIUM. This is due to the fuzziness of these terms, so it is natural to interpret these terms as fuzzy sets instead of the crisp ones. This means that fuzzy quantization approach may be more appropriate for the problem. However, if we use fuzzy sets in the quantization process, then for a quantized attribute a and an object x, we can not simply replace f a (x) by some linguistic term since f a (x) may satisfy more than one linguistic terms to some degrees, so the rule extraction process presented in the preceding section can be applied to find fuzzy rules while the data tables are kept unchanged. In this section, we describe this application as a special case of the rule extraction process in the preceding section. In this application, we assume an FDT S = (U, F (C {d})) is given such that for each a C {d} and u U, f a (u) is an element (or identically a singleton subset) of V a, i.e., each object has precise attribute values. Though the data table is precise, we want to extract some fuzzy rules from it due to the quantization reason. For simplicity, we first assume that the set of linguistic hedges is empty. It is also assumed that for each a C, the linguistic terms in L a satisfy the disjointness property. That is, the context ({V a } a C {d}, m 1, m 2 ) is given such that for l 1, l 2 L a, CON(m 1 (l 1 ), m 1 (l 2 )) < 1. It can be further required that the set L a forms a fuzzy partition of V a, however, it is not necessary for the current presentation. Since the data table is precise, according to the semantics of PDCL, the wffs (a, πl) and (a, νl) has the same truth values in each object, so we can write both of them as (a, l). We are now interested in extracting rules with normalized antecedents. More specifically, these are rules of the form (a, l a ) (d, l d ) (4) a C such that sup u U E(u, a C (a, l a)) = 1. A typical problem matching the above situation is the fuzzy controller synthesis from observed samples[16]. The condition attributes correspond to the input variables of the controller and the decision attribute the controllable variable. Assume some past samples of input-output variables have been observed and recorded in a data table, then the problem is to find a set of fuzzy rules matching with the observation. The rule extraction process in the last section can be applied to the problem directly and the time complexity is again O(n k m ) with n = U, m = C, and k = max a C {d} L a. However, due to the special property of the problem, we have another bound to the time complexity. Let us say that a wff ϕ of the form a C (a, l a) is a characteristic formula of an object u if E(u, ϕ) = 1. Due to the disjointness property, each object has at most one characteristic formula. Furthermore, the wff ϕ is normalized iff it is the characteristic formula of some object in U. Thus the set of normalized antecedents is equal to the set of characteristic formulas. For each object u, at most O(m k) time is needed to find its characteristic formula (if it has one) since for each attribute a, we can search through the set L a and test whether µ m1 (l)(f a (u)) = 1 for each l L a. Thus all normalized rule antecedents can be found in O(n m k) time and there are at most n such different antecedents, so there are at most O(n k) rules to be extracted by connecting the antecedents to the possible consequents. For each rule, it needs O(n m) time to compute the strength. Thus the total time complexity of the rule extraction process is bounded also by O(n 2 m k) which is a polynomial bound.

14 212 C.J. Liau, D.R. Liu / A Possibilistic Decision Logic with Applications The rule extraction approach in figure 1 can be named as a formula-oriented search process since we search through the set of all possible antecedents. On the other hand, the process described in the last paragraph can be called an object-oriented search since we search the characteristic formulas of all objects. When the number of attributes is sufficiently small, the formula-oriented approach is obviously more efficient, whereas when n k m, the object-oriented search will be more appropriate. Example 2. (continued) Since this is a small example (n = 9 and m = 1), it is indifferent to use formula or object-oriented search. Let us assume L t = {low, medium, high} and the context has the meaning function m 1 which defines the three linguistic terms by trapezoid membership functions as follows: 1, if t 15, µ m1 (low)(t) = 1 t 15 10, if 15 < t 25, 0, if t > 25. µ m1 (medium)(t) = t 10 10, if 10 t 20, 1, if 20 < t < 25, 1 t 25 10, if 25 t 35, 0, otherwise. 1, if t > 30, µ m1 (high)(t) = t 20 10, if 20 < t 30, 0, if t 20. According to these membership functions, the truth sets of the three t-basic wffs are listed in the following table. Each column of the table corresponds to a truth set of a t-basic wff. U (t low) (t medium) (t high) By the formula-oriented approach, since all the three truth sets have nonempty 1-cuts, the three t-basic wffs are qualified as the rule antecedents. If by object-oriented approach, it can also be seen that the three wffs are respectively the characteristic formulas of objects 1-2, 4, and 6-9. Let L s = {off(which is

15 C.J. Liau, D.R. Liu / A Possibilistic Decision Logic with Applications 213 true in objects 1-4), on(which is true in objects 5-9)}, then the following rules with nonzero strength can be extracted: (t low) (s off) 1 (t medium) (s off) (t high) (s on) 0.6 In the example, if for each object with characteristic formula, we replace its condition attribute value with the corresponding linguistic term, and delete the objects without characteristic formulas, then we get a classical data table. U t s 1 low 0 2 low 0 4 medium 0 6 high 1 7 high 1 8 high 1 9 high 1 It can be seen that the same fuzzy rules as above can be extracted from the reduced table by decision logic approach[13] if the strength of the rules are not considered. This is not by accident. In fact, if L d is given such that for each l L d, m 1 (l) is a crisp subset of V d (as it is in the example) and we define a DT S = (U, C {d}) from the original FDT S such that 1. U is the subset of objects in U which have a characteristic formula, 2. for a C and u U, a(u) = the unique l L a such that µ m1 (l)(f a (u)) = 1, then we have the following proposition. Proposition 1. Let be a t-norm satisfying a b > 0 for any a, b (0, 1], then for any fuzzy rules ϕ of the form (4), 2 [ ϕ ] S > 0 iff ϕ is true in S. Proof: Let ϕ = ψ (d, l d ), then according to the construction of S, for any x U, x = S ψ iff E S (x, ψ) = 1 and for any x U U, E S (x, ψ) < 1. Since m 1 (l d ) is a crisp subset of V d, then E S (x, (d, l d )) = 1 or 0 for any x U, so [ ϕ ] S = x:e S (x,(d,l d ))=0 1 E S (x, ψ). By the semantics of DL, ϕ is true in S iff x = S ψ implies E S (x, (d, l d )) = 1 iff for any x U, E S (x, (d, l d )) = 0 implies E S (x, ψ) < 1 iff [ ϕ ] S > 0 by the assumption on the t-norm. 2 Note that ϕ can be seen as a DL wff with respect to the DT.

16 214 C.J. Liau, D.R. Liu / A Possibilistic Decision Logic with Applications 6. Related Works There have been many existing works in the extraction of fuzzy rules from precise or uncertain data, however, many of them use a more restricted rule representation language. For example, in [16], the traditional fuzzy rules of the form IF X 1 is A i,, X m is A m, THEN Y is B where X i s and Y are variables and A i s and B are linguistic terms, are induced from numerical data tables. Since only precise values appears in the numerical data sets, it is unnecessary to distinguish the certainty or possibility-qualified formula, so syntactically, the rules are equivalent to wffs in equation (4). However, when the data may be uncertain, this kind of rules is obviously insufficient in expressive power. Furthermore, in the rule induction process of [16], a degree is assigned to each rule by product. In other words, if the rule is rewritten as a wff ϕ = a C p a p d, then for each x U, the degree is defined as Deg(x, E S (x, p a )) E S (x, p d ), a C p a p d ) = ( a C so in their framework, the semantics for conjunction and implication are both given by the product t- norm. Finally, unlike in our logic, they do not evaluate the truth value of ϕ in the whole table. Instead, for each object x, only rules obtaining the maximal degree in the object are induced, so the corresponding rule of x is r(x) = arg max Deg(x, ϕ). ϕ If r(x) and r(y) are two rules with same antecedents but different consequents, then only the rule with higher degree is remaining in the final rule base. Thus the degree of a rule r(x) is semantically distinct with its truth value in PDCL. An recent approach for fuzzy data mining in [3] also use the traditional fuzzy rules as the representation language. However, their semantics is more in the form of conditional probability (just like the QST measure which would be defined in (5)). Furthermore, instead of returning a rule with its strength, the numerical strength of a rule is transformed into a linguistic fuzzy quantifier, such as most, many, a few, etc. according to a predefined mapping. In [15], an approach for uncertain data analysis is proposed to discover rules from generalized decision table. In that approach, an FDT is effectively transformed into a generalized decision table with multiple descriptors. Although they do not explicitly define a language for representation of the rules, a set of predefined linguistic terms are indeed used in the descriptors for each attribute. Therefore, we can use the above-defined notations in the comparison. Assume an FDT S = (U, F (A)) is given and A = C {d}, where C is the set of condition attributes and d is the decision attribute, then the rules to be induced from the table are of the form 3 (a, πl a ) (d, πl d ), a C where l a L a for all a C and l d L d. Let ϕ be a rule of such form, then the strength of ϕ, denoted by ξ S (ϕ), can be effectively computed by the following formulas. 3 In [15], the linguistic terms for the decision attribute are restricted to the domain V d, however, in principle, it seems not exclusive to use the general linguistic terms.

17 C.J. Liau, D.R. Liu / A Possibilistic Decision Logic with Applications 215 Let Φ(C) = { a C (a, πl a) l a L a, a C} and Φ(d) = {(d, πl d ) l d L d }, then for any ϕ 1 Φ(C) and ϕ 2 Φ(d), we can define ξ S (x, ϕ 1 ϕ 2 ) = E S (x, ϕ 1 ) ψ Φ(C) E S(x, ψ) E S (x, ϕ 2 ) ψ Φ(d) E S(x, ψ), and ξ S (ϕ) = x U ξ S (x, ϕ) From the formulas, it can be easily seen that the main difference between the approach of [15] and ours lies on the semantics of the rules. If ϕ = a C (a, πl a) (d, πl d ) is a rule, then [ ϕ ] S is evaluated according to a Tarskian-style semantics, whereas ξ S (ϕ) is evaluated more in the probabilistic spirit. [ ϕ ] S denote the degree of the data table S verifying the rule ϕ, so it is a relationship between the data and the rule and is independent of the strength of other rules. On the other hand, the value of ξ S (ϕ) strongly depends on other rules available on the language since ξ S (x, ϕ) is computed as a ratio relative to the truth values of other relevant wffs. The logical approach used here is thus more qualitative than the quantitative evaluation in [15]. This means that if appropriate t-norms, t-conorms, and implications are chosen, then our approach can be used in uncertain data analysis where the uncertainty degree is modeled by more general lattices instead of [0, 1]. Though we can compute the strength of rules in [15] by ξ S (ϕ), it is not really appropriate for our enriched representation language, since there are more wffs to be considered than those in Φ(C) and Φ(d). For example, wffs of the form a C (a, νl a) may appear on the antecedent of some rules, then according to the distribution scheme of the [15], should it be considered in the process of computing ξ S (ϕ)? Instead of trying to answer the question, we suggest the following alternative quantitative definition of rule strength analogous to the ordinary conditional probability may be more appropriate in our framework. x U QST (ϕ ψ) = E S(x, ϕ ψ) x U E (5) S(x, ϕ) Analyzing uncertain data in incomplete information systems has also received much attention in the past ([5, 6, 7, 10]). An incomplete information system is a data table in which the attributes may have multiple or missing values, so it can be seen as a special case of FDT s. More specifically, an incomplete information system(iis) is a pair S = (U, F (A)), where U is a finite of objects, A is a finite set of attributes, and F (A) = {f a : U 2 Va { } a A}. Since each crisp subset of V a is also its fuzzy subset, and the missing value can be replaced by V a 2 Va F(V a ), an IIS is just a special kind of FDT. In [7], a decision table is defined as an IIS S = (U, F (C {d})) with the decision attribute d satisfying f d (x), for all x U, and the expected decision rules from the decision table are of the form (a, v a ) (d, v), where v a V a, v d V d. v v d a C According to the semantics given in [7], the rule is equivalent to the following PDCL wff (a, πl a ) (d, νl d ) a C

18 216 C.J. Liau, D.R. Liu / A Possibilistic Decision Logic with Applications with a context ({V a } a C {d}, m 1, m 2 ) such that m 1 (l a ) = v a for all a C and m 1 (l d ) = v d (i.e. l a and l d are respectively the labeling terms corresponding to the subsets of attributes values v a and v d ). This shows that the decision rules induced from an IIS can also be represented in our logical language. 7. Concluding Remarks The management of uncertainty has been a long-standing requirement in intelligent data analysis. In this paper, we present a logical framework for handling fuzzy data tables. In the framework, uncertain attribute values are represented as fuzzy subsets of the domains. Linguistic terms, which are interpreted as fuzzy subsets of respective domains, are taken as the basic building blocks of a possibilistic decision logic. Then, the information contained in each item of data determines the truth values of wffs in the language. A rule in our framework is an implication formula of the language and the aggregated truth degree of the formula on all data items is taken as the strength of the rule. The formulas of decision logic are called information pre-granules in [14], so our wffs of PDCL can be analogously called fuzzy information pre-granules. Therefore, our logical framework can be seen as a formal instance of fuzzy granular information processing approach. Compared to the existing approach of extracting traditional fuzzy rules from precise or uncertain data tables, our logical language is more expressive. However, the search of interesting rules from the richer language is more costly. Thus some heuristic must be incorporated to improve the efficiency of the search process. These include the restriction of the language for rule representation, the β-support of the rules, and the deletion of object with less information values in advance. It is also shown that when the data table is precise, we can have a polynomial time algorithm for rule induction. Nevertheless, witnessing the high complexity of the general search process for the full language, the main contribution of the paper should be more in the representational aspect of the logic than in the algorithmic aspect for the data analysis. Therefore, seeking more efficient data analysis procedures would be the most important further works for the logic. References [1] Dubois, D., Lang, J., Prade, H.: Possibilistic logic, in: Handbook of Logic in Artificial Intelligence and Logic Programming, Vol 3 : Nonmonotonic Reasoning and Uncertain Reasoning (D. Gabbay, C. Hogger, J. Robinson, Eds.), Clarendon Press - Oxford, 1994, [2] Dubois, D., Prade, H.: An introduction to possibilistic and fuzzy logics, in: Non-Standard Logics for Automated Reasoning (P. Smets, A. Mamdani, D. Dubois, H. Prade, Eds.), Academic Press, 1988, [3] Klein, Y., Pery, R., Komem, J., Kandel, A.: Fuzzy data mining, in: Intelligent Systems and Interfaces (H.-N. Teodorescu, D. Mlynek, A. Kandel, H.-J. Zimmermann, Eds.), Kluwer Academic Publishers, 2000, [4] Kruse, R., Borgelt, C., Nauck, D.: Fuzzy data analysis: challenges and perspectives, Proceedings of the 8th IEEE International Conference on Fuzzy Systems, IEEE, San Francisco, CA, [5] Kryszkiewicz, M.: Properties of incomplete information systems in the framework of rough sets, in: Rough Sets in Knowledge Discovery (L. Polkowski, A. Skowron, Eds.), Physica-Verlag, 1998, [6] Kryszkiewicz, M., Rybiński, H.: Reducing information systems with uncertain attributes, Proceedings of the 9th ISMIS (Z. W. Raś, M. Michalewicz, Eds.), LNAI 1079, Springer-Verlag, 1996.

19 C.J. Liau, D.R. Liu / A Possibilistic Decision Logic with Applications 217 [7] Kryszkiewicz, M., Rybiński, H.: Reducing information systems with uncertain real value attributes, Proceedings of the 6th International Conference on Information Processing and Management of Uncertainty in Knowledge-based Systems(IPMU), [8] Liau, C., Liu, D.: A logical approach to fuzzy data analysis, Proceedings of the Third European Conference on Principles of Data Mining and Knowledge Discovery (J. Zytkow, J. Rauch, Eds.), LNAI 1704, Springer- Verlag, [9] Lin, T.: Rough set theory in very large databases, Proceedings of the IMACS Symposium on Modeling, Analysis and Simulation(CESA 96), [10] Lipski, W.: On databases with incomplete information, Journal of the ACM, 28, 1981, [11] de Mántaras, R. L., Godo, L.: From fuzzy logic to fuzzy truth-valued logic for expert systems: A survey, Proceedings of the 2nd IEEE International Conference on Fuzzy Systems, IEEE, San Francisco, CA, [12] Nguyen, H., Nguyen, S.: Discretization methods in data mining, in: Rough Sets in Knowledge Discovery (L. Polkowski, A. Skowron, Eds.), Physica-Verlag, 1998, [13] Pawlak, Z.: Rough Sets Theoretical Aspects of Reasoning about Data, Kluwer Academic Publishers, [14] Polkowski, L., Skowron, A.: Towards Adaptive Calculus of Granules, Proceedings of the 7th IEEE International Conference on Fuzzy Systems, [15] Słowiński, R., Stefanowski, J.: Rough-Set Reasoning about Uncertain Data, Fundamenta Informaticae, 27(2-3), 1996, [16] Wang, L., Mendel, J.: Generating fuzzy rules by learning from examples, IEEE Transcations on Systems, Man, and Cybernetics, 22(6), 1992, [17] Yager, R.: Measuring tranquility and anxiety in decision making: An application of fuzzy sets, International Journal of General Systems, 8, 1982, [18] Yager, R.: On the specificity of a possibility distribution, Fuzzy Sets and Systems, 50, 1992, [19] Yager, R.: On measures of specificity, in: Computational Intelligence: Soft Computing and Fuzzy-Neuro Integration with Applications (O. Kaynak, L. Zadeh, B. Turksen, I. Rudas, Eds.), Springer-Verlag, 1998, [20] Yao, Y., Liu, Q.: A generalized decision logic in interval-set-valued information tables, in: New Directions in Rough Sets, Data Mining, and Granular-Soft Computing (N. Zhong, A. Skowron, S. Ohsuga, Eds.), LNAI 1711, Springer-Verlag, 1999, [21] Zadeh, L.: Fuzzy sets as a basis for a theory of possibility, Fuzzy Sets and Systems, 1(1), 1978, [22] Zadeh, L.: Fuzzy logic = computing with words, IEEE Transactions on Fuzzy Systems, 4, 1996,