The size of decision table can be understood in terms of both cardinality of A, denoted by card (A), and the number of equivalence classes of IND (A),

Transcription

1 Attribute Set Decomposition of Decision Tables Dominik Slezak Warsaw University Banacha 2, Warsaw Phone: +48 (22) Fax: +48 (22) ABSTRACT: Approach to attribute set decomposition of decision tables is proposed. It enables to combine nondeterministic decision rules based on generalized decision functions for dierent subsets of conditions. Optimal distribution onto such subsets is presented as a causal network. Computational complexity of searching for such a network is discussed, in terms of such factors like exactness of synthesis of information or memory required for storing decomposed data. 1. INTRODUCTION In recent years rough set approach ([7]) turned out to be very eective as applications to data mining and decision support. However, real life problems require reconsidering possessed tools in view of large data bases. For this purpose a great eort has been spent on initial decomposition of information systems and decision tables with large numbers of objects and attributes (see e.g. [4], [5]). The aim of such a decomposition is to store the knowledge within smaller subtables with their local rules of reasoning. Such rules can be applicable for a case being tested after examining its relevance to particular subsystems. Final decision is made by synthesis of answers from subsystems adequate for given case. We propose a new approach to decomposition which enables to combine non-deterministic decision rules derived from dierent subsets of conditional attributes. Tools for nding optimal distribution onto such subsets are discussed with respect to their computational complexity. Optimality is understood in terms of such factors like exactness of synthesis of information (compare with [10]) or the decrease of memory required for storing decomposed data tables with respect to initial situation, what is related to minimal description length principle (see e.g. [13]) The paper is organized as follows. In Section 2 we introduce criteria for decomposition based on generalized decision functions. Section 3 contains conditional independence model for decomposition and its potential applications. In Section 4 the complexity of searching for optimal decomposition is discussed. Section 5 closes the paper with nal remarks and directions for further research. 2. ATTRIBUTE SET DECOMPOSITION The purpose of decomposition of large decision tables is to state a distribute system of cooperative agents able to combine their decision knowledge if necessary. By a decision table we understand tuple A = (U; A [ fdg), where each conditional attribute a 2 A is identied with function a : U! V a onto the set of possible values for a. In case of qualitative decision attribute d with discrete V d, decision conditioned by arbitrary B A is expressed by generalized decision B : U! 2 V d dened by for indiscernibility equivalence B (u) = fv d 2 V d : 9 u0 2U d (u 0 ) = v d ^ (u; u 0 ) 2 IND (B)g IND (B) = f(u; u 0 ) : 8 a2b a (u) = a (u 0 )g The assumption about existence of indiscernibility relations is natural for introducing any rough set based model. In our case, e.g., for continuously-valued conditional attributes, such relations can be obtained by considering intervals or hyperplane cuts ([3], [6]). It can be also the case for decision attribute, where considering generalized decisions may be easily modied to handling intervals.

2 The size of decision table can be understood in terms of both cardinality of A, denoted by card (A), and the number of equivalence classes of IND (A), denoted by card (U=A). For each nite family A = fa i g i2i of subsets of A, parameters card (U=A) U = Q i2i card (U=A i) card (A) and A = P i2i card (A i) (1) reect the average degrees of decrease of rows' and columns' number, respectively. If S i2i A i = A, then U ; A 2 [0; 1] can be compared with coecients related to minimal description length principle (compare e.g. with [13]). Such a decomposition causes potential damage to joint information about decision attribute after synthesis. Denition 2.1. We will call A as decomposable onto A = fa i g i2i if and only if the following is satised: 8 A (u) = \ Ai (u) (2) Normally, only inclusion is satised instead of the above equality. In this case it may happen that intersection of local generalized decision value sets for foregoing objects contain more possible decision values than it is actually. In fact, keeping the decision knowledge on initial level requires satisfying inclusion in (2). 3. DECOMPOSITION BASED ON CAUSAL NETWORKS Although intuitively valid, presented criterion for decomposability is too poor tool itself. The lack of knowledge about the nature of connections among elements of considered family A turns out to be serious disadvantage while combining the decision knowledge and solving possible conicts for new objects. The structure basing on the following result leads to more concrete representation, what enables to discuss aims and complexity of searching for optimal decomposition. Denition 3.1. Given decision table A = (U; A [ fdg), for subsets A1; A2; A3 A, we say that A1 and A2 are conditionally independent under A3 (denoted by I A (A1; A2=A3)) if and only if Proposition 3.1. For any A1; A2; A3; A4 A, 8 A1[A 2[A 3 (u) A1[A 3 (u) A2[A 3 (u) (3) I A (A1; A2 [ A3=A4) ) I A (A1; A2=A4) ^ I A (A1; A2=A3 [ A4) I A (A1; A3=A4) ^ I A (A1; A2=A3 [ A4) ) I A (A1; A2 [ A3=A4) Decision tables can be understood as conditional independence models with respect to the notion of conditional independence introduced above. It means that each A = (U; A [ fdg) induces the set of triples of subsets of A for which condition (3) is satised. Such models are claimed to satisfy the above, semi-graphoidal properties, similarly as in case of e.g. chaining conditional probability distributions (see e.g. [9]) or generalized decision functions ([16]) in causal networks. It turns out that concerning such networks leads to interesting results also in our case. Denition 3.2. For A = (U; A [ fdg) consider a directed acyclic graph (DAG) D = A; E ~. For any A1; A2; A3 A, A3 is said to d-separate A1 and A2 (denoted by ha1; A2=A3i D ) if and only if there is no undirected path between a node in A1 and a node in A2, along which (1) every node with converging arrows is in A3 or has a descendant in A3 and (2) every other node is outside A3. Proposition 3.2. (Proof is based on [17]; see [9] for further references) Consider A = (U; A [ fdg) with linearly ordered set A = fa1; ::; a n g and D = A; E ~, such that ~E = [ i=1;::;n f(b; a i ) : b 2 B i g

3 where, for each i = 1; ::; n, B i is a boundary of a i infa1; ::; a i?1g, i.e. minimal (in sense of inclusion) subset of fa1; ::; a i?1g, satisfying I (fa i g ; fa1; ::; a i?1g nb i = B i ). Then, for any u 2 U, A (u) Bi[fa ig (u) (4) Moreover, for each three sets A1; A2; A3 A we have i=1;::;n and deleting any arrow from ~ E destroys such a property. ha1; A2=A3i D ) I A (A1; A2=A3) (5) The above result states concrete tool for constructing DAG-representation for decision tables, which later can be used e.g. as a basis of algorithm testing decomposability due to criterion for d-separation and (5). Eciency of such an algorithm, completely irrelevant to the number of objects in U, depends just on the number of arrows in previously found graph. 4. DERIVING THE OPTIMAL DECOMPOSITION FROM DATA Proposition 3.2., due to equality (5), gives us possibility of attribute decomposition by sequential deleting conditional attributes from A, with boundaries being copied. Although it is not, obviously, the only method for decomposition, its additional features suggest to keep focusing on it. Analysis of time complexity of such an algorithm for nding minimal DAG satisfying conditions (4) and (5) can be performed in two steps, by 1. searching for optimal linear ordering over attributes in A and 2. searching for minimal (in sense of cardinality) boundaries consistent with xed ordering. Finding minimal (with respect to card ~E ) appropriate DAG is important in view of both minimum description length principle corresponding to parameters (1) and the idea of testing algorithm mentioned at the end of previous section. Unfortunately, even if the ordering over conditional attributes is already xed, then, unlike in case of strictly positive probabilistic distributions (see e.g. [9]) or, e.g., tolerance dependency models ([15]), boundaries understood in terms introduced in this paper are not uniquely determined. The following result shows, actually, NP-hardness of the problem of nding minimal boundary. On the other hand, however, its formulation suggests simple heuristics for nding satisfactory network representation. Proposition 4.1. For any a i 2 A, consider discernibility table (U U; fa1; ::; a i g) ([6], [11], [14]), where, for each (u; u 0 ) 2 U U and j = 1; ::; i, 1 iff a j ((u; u 0 aj (u) 6= a )) = j (u 0 ) 0 otherwise Then the problem of nding minimal (in sense of cardinality) boundary of a i in fa1; ::; a i?1g is equivalent to the problem of nding minimal column covering in discernibility table (W; fa1; ::; a i?1g) (U U; fa1; ::; a i g) where W is obtained by sequential performing the following operations 1. Take into account only such pairs (u; u 0 ) that d (u) 6= d (u 0 ). 2. Consider only such pairs (u; u 0 ) that there is (w; w 0 ) satisfying: (a) d (u) = d (w) and d (u 0 ) = d (w 0 ), (b) (w; w 0 ) has record (0; ::; 0; 1) over fa1; ::; a i g in (U U; fa1; ::; a i g). 3. Remove all pairs with record (0; ::; 0; 1) over fa1; ::; a i g and forget about column corresponding to a i. 4. Remove all (u; u 0 ) such that there is (w; w 0 ) satisfying:

4 (a) d (u) = d (w) and d (u 0 ) = d (w 0 ), (b) (w; w 0 ) has record (0; ::; 0) over fa1; ::; a i?1g in (U U; fa1; ::; a i?1g). As mentioned before, the above result is a source of some heuristics for nding optimal decomposition. For instance, optimal ordering over conditional attributes can be searched for by examining cardinalities of W for particular attributes deleted as rst. Attribute corresponding to the largest W becomes the last in ordering as that with boundary potentially hardest to nd. In parallel with constructing the ordering, column covering can be performed for foregoing discernibility tables. Here we suggest to combine common method of choosing attributes corresponding to the largest numbers of 1's over W with examining their potential inuence on the increase of parameter U from (1). 5. CONCLUSIONS AND DIRECTIONS FOR FURTHER RESEARCH In this paper we presented foundations for large decision tables decomposition with respect to generalized decision function representation. Obtained structure of conditional independence model and causal network should be further developed and implemented, where tools developed for both probabilistic network (see e.g. [2], [12]) and rough set (see e.g. [1], [5], [18]) applications can be adopted. The main topics of research should be now focused on improving eciency of proposed distributive reasoning. Proposed DAG-structure should be searched for in terms of obtaining possibly best classication of new objects. Special attention ought to be payed to new objects consistent with distributed information only locally. From such a point of view presented framework must be supported by a system of resolving possible decision conicts (see e.g. [8]). Finally, proposed criterion for decomposition can be generalized. Handling with rough inclusion leads to approximate version of decomposition and synthesis (compare with [10]), where the degree of approximation should be derived with respect to particular applications. Another direction of generalization is to adopt introduced framework for continuously-valued decision attributes and refer the idea of searching for optimal ordering for network decomposition to sequential algorithms for deriving minimal decision rules from continuously-valued conditional attributes ([3]). References [1] Bazan J.G., Skowron A., Synak P., Discovery of Decision Rules from Experimental Data in Proceedings of the Third International Workshop on Rough Sets and Soft Computing RSSC'94, November 10-12, San Jose University, CA, pp , [2] Bouckaert R.R., Properties of Bayesian Belief Network Learning Algorithms in Proceedings of the 10- th Conference on Uncertainty in AI, de Mantarnas R.L., Poole D.(eds.), the University of Washington, Seattle; Morgan Kaufmann, San Francisco, CA, pp , [3] Nguyen S.H., Nguyen H.S., Skowron A., Searching for Features dened by Hyperplanes in Proceedings of the Ninth International Symposium on Methodologies for Information Systems ISMIS'96, Z.W. Ras, M. Michalewicz (eds.), June, Zakopane, Poland; Lecture Notes in AI 1079, Berlin, Springer Verlag, pp , [4] Nguyen S.H., Nguyen T.T., Polkowski L., Skowron A., Synak P., Wroblewski J., Decision Rules for Large Data Tables in Proceedings of Symposium on Modelling, Analysis and Simulation vol 1, Computational Engeneering in Systems Applications CESA'96, July 9-12, Lille, France, pp , [5] Nguyen S.H., Polkowski L., Skowron A., Synak P., Wroblewski J., Searching for Approximate Description of Decision Classes in Proceedings of the Fourth International Workshop on Rough Sets, Fuzzy Sets and Machine Discovery RSFD'96, November 6-8, Tokyo, Japan; the University of Tokyo, pp , [6] Nguyen H.S., Skowron A., Quantization of Real Value Attributes in Proceedings of Second Joint Annual Conference on Information Sciences, September 28 - October 1, Wrightsville Beach, NC, pp.34-37, [7] Pawlak Z., Rough Sets. Theoretical Aspects of Reasoning about Data, Kluwer Academic Publishers, Dortrecht, 1991.

5 [8] Pawlak Z., Anatomy of Conicts in Bulletin of the European Association for Theoretical Computer Science 50, pp [9] Pearl J., Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, Morgan Kaufmann, [10] Polkowski L., Skowron A., Rough mereology: a new paradigm for approximate reasoning in International Journal of Approximate Reasoning, in print. [11] Rauszer C., Skowron A., The Discernibility Matrices and Functions in Information Systems in Intelligent Decision Support. Handbook of Applications and Advances of the Rough Sets Theory, S lowinski R.(ed.), Kluwer, Dortrecht, pp , [12] Singh M., Valtorta M., Construction of Bayesian Network Structures from Data: A Brief Survey and an Ecient Algorithm in International Journal of Approximate Reasoning 12, Elsevier Science Inc., pp , [13] Skowron A., Grzymala-Busse J., From Rough Set Theory to Evidence Theory in Advances in the Dempster- Shafer Theory of Evidence, Yager R.R., Fedrizzi M., Kacprzyk J.(eds.), John Wiley & Sons, New York, pp , [14] Skowron A., Stepaniuk J., Decision Rules based on Discernibility Matrices and Decision Matrices in Proceedings of The Third International Workshop on Rough Sets and Soft Computing RSSC'94, November 10-12, San Jose University, CA, [15] Slezak D., Tolerance dependency model for decision rules generation inproceedings of the Fourth International Workshop on Rough Sets, Fuzzy Sets and Machine Discovery RSFD'96, November 6-8, Tokyo, Japan; The University of Tokyo, pp , [16] Slezak D., Rough Set Reduct Networks in Proceedings of Joint Conference of Information Sciences JCIS'97, vol 3, Duke University, Elsevier Publishing Company, pp.77-80, [17] Verma T.S., Causal networks: Semantics and expressiveness, Technical Report R-65, Cognitive Systems Laboratory, University of California, Los Angeles, [18] Wroblewski J., Finding minimal reducts using genetic algorithms inproceedings of the Second Annual Joint Conference on Information Sciences, September 28 - October 1, Wrightsville Beach, NC, pp , 1995.