202-207. Neighborhoods Systems: Measure, Probability and Belief Functions T. Y. Lin 1 * tylin@cs.sjsu.edu Y, Y, Yao 2 yyao@flash.lakeheadu.ca 1 Berkeley Initiative in Soft Computing, Department of Electrical Engineering and Computer Science, University of California, Berkeley, California 94720 2 Department of Computer Science, Lakehead Univeristy Thunder Bay, Ontario, Canada P7b 5E1 Abstract: The notion of neighborhood system is a mathematical formalism for negligible quantity. It formulate the mathematical concept of neighborhoods in the context of advanced computing. Neighborhood systems, by definition, include topology (topological neighborhood systems), rough sets (S 5 -neighborhood systems) and binary relations (basic neighborhood systems). In this paper, real valued functions based on neighborhood systems are studied. The study covers many important quantities in uncertainty, such as belief functions, measure, and probability; fuzzy sets are not included here, because it was reported elsewhere. It seems that neighborhood systems are an effective underlying data structure for managing uncertainty. Keywords: binary relation, measure, neighborhood, probability, qualitative fuzzy set, rough set, topology. 1. Introduction Representing and measuring uncertainty are critical in advanced computing. Various theories have been proposed. The most notable theories are Zadeh s fuzzy theories [24] and Pawlak s rough set theory [16]. In this paper, we discuss one of the most encompassing notions of uncertainty from a geometric point of view, namely, neighborhood systems. Roughly, a neighborhood system assigns each object a (possibly empty, finite, or infinite) family of non-empty subsets. Such subsets, called neighborhoods, represent the semantics of negligible, which is the essence of uncertainty handling: neglect the negligible. Using neighborhoods, one can define open sets, hence the interior and closure of any subset (as in topological spaces) [6], [19], [20]. By taking equivalence classes as neighborhoods, the lower and upper approximations are precisely the interior and closure, respectively. Rough set approximation is a special form of neighborhood theory; it was called category [2]. Generalized rough sets based on various modal logic [21] are all special forms of neighborhood systems, called basic neighborhood systems. The notion of neighborhood systems is one of the correct mathematical formalisms for expressing the semantics of approximation and uncertainty in the context of advanced computing. Our interests stemmed from database retrieval and mining [15], [3], [7], [8] [9], [2], [10], [21]. One may view it as a first step toward the granulate mathematics described by Lotfi Zadeh [23]. Some basic notions and its application to qualitative fuzzy theory was reported in [13], [14]. In this paper, we turn out attention to the quantitative aspect of uncertainty. One could develop a full fledge measure and probability theory as in classical mathematics [5]. At this early stage, we focus on its applications; belief functions are formulated using the notion of neighborhood systems [18]. 2. Neighborhood Systems Since the systematic study of neighborhood system is relatively recent, we recall some of our motivation from [14]. Neighborhood systems are abstracted from numerical analysis. In any standard procedure of finding approximate solutions, the very first step is to choose a small number ε, or equivalently, an ε-neighborhood for each point on the real line. During the process of finding ---------------- * This research is partially supported by Electric Power Research Institute, Palo Alto, California * On leave from San Jose State University (tylin@cs.sjsu.edu)
approximate solutions, this particular family of neighborhoods never changes. In other words, the only relevant notion of the real line topology [6] is this particular family of chosen neighborhoods. We can view the first step as the step of setting up a proper context for discussions, that is, one step up the standard for what it means by near for this special circumstance. Such a family of chosen neighborhoods, not the full topology, is the essential formalism for approximation. The neighborhood is a fundamental notion in mathematical analysis. It is also a common notion in many other areas. It appears in logic [1], in a text of genetic algorithm [4], rough sets [16], generalized rough sets [21], and databases. A systematic study in the context of advanced computing was started by the first author and his students. The study was motivated from database retrieval and mining [15], [3], [7], [8] [9], [2], [10]. 2.1 Definitions and Properties In this section, we recall some notions of neighborhood system from [14]. Let U be the universe of discourse and p be an object or point in U. 1. A neighborhood, denoted by N(p), or simply N, of p is a non-empty subset of U, which may or may not contain the object p. Any subset that contains a (non-empty) neighborhood is a neighborhood. 2. A neighborhood system of an object p, denoted by NS(p), is a maximal family of neighborhoods of p. If p has no neighborhood, then NS(p) is an empty family; in this case, we simply say that p has no neighborhood. 3. A neighborhood system of U, denoted by NS(U) is the collection of NS(p) for all p in U. Such a neighborhood system may also be called F-topology (read as finite type topology). For simplicity a set U together with NS(U) is called a neighborhood system space or a neighborhood system. A neighborhood system is called a Frechet(V) space, if every NS(p) is non-empty [19]. 4. N is open, if N is a neighborhood of every object in N. 5. More generally, a subset X of U is open if for every object in X, there is a neighborhood N(p) X. A subset X is closed if its complement is open. 6. NS(p) and NS(U) are open if every neighborhood is open. NS(U) is topological, if U is the usual topological space [6]. Both NS(U) and the collection of open sets is called topology if U is a topological space. 7. Let X be a subset of U. The lower approximation of X can be defined by I[X] = { p: there is a N(p) X} = interior of X, that is, I[X] is the largest open set contained in X. 8. Similarly, the upper approximation of X can be defined by ( 0 is the empty set) C[X] = {p: N(p), X N(p) 0}= closure of X. that is, C[X] is the smallest closed set contains X. I[X] and C[X] are precisely the lower and upper approximation in rough set theory. 9. A topological space is a neighborhood system space, but not the converse. 10. Intersections and finite unions of closed sets are closed. 11. In topological spaces, unions and finite intersections of open sets are open. In neighborhood systems, unions is open, but intersections may not be open. 2.2 Basic Neighborhoods and Binary Relations 13. A minimal neighborhood of p, denoted by MN(p), is a minimal member of NS(p) in the sense that MN(p) contains no member of N(p) as proper subsets. In general such MN(p) may or may not exist. The maximal family of all MN(p) at p will be denoted by MNS(p). The family of MNS(p) for all p will be denoted by MNS(U). Let n(p) be the number of (distinct) MNS(p) s at p. If, for all p, n(p) = n is a constant integer, MNS(U) is an n-minimal neighborhood system, and denoted by n-mns(u); we will be interested in 1-minimal neighborhood systems, called basic (binary) neighborhood system BS(U). BS(U) can be defined by a binary relation and vice versa - see below. So B in BS(U) may be referred to as a basic neighborhood or a binary neighborhood. 14. Let R be a binary relation defined on U, then B(p) = {x : prx} is a neighborhood of p. So a binary relation R gives rise to a basic (binary) neighborhood system. Conversely, one can use the basic neighborhoods to define the binary relation. From the implementation point of view, we can rephrase basic neighborhood systems as follows:
15. A basic neighborhood system BS(U) is a data structure that assigns to each datum a list of data. 16. Given a neighborhood system NS(p) at p. A minimal member of NS(p) may or may not exit. For example, a neighborhood system of a real number has no minimal neighborhood. 17. A binary relation on U defines one and only basic (binary) neighborhood system; they are summarized in the table below [3]. Binary Relations Relationships Basic (Binary) Neighborhoods serial serial reflexive reflexive symmetric symmetric symmetric, open Euclidean transitive transitive Euclidean Euclidean reflexive, Browersche, B symmetric reflexive, S 4, (topological) transitive equivalence clopen topology, S 5, Similarly the n-graded binary relations [21] correspond to n-minimal neighborhood systems. 3. Measure and Probability Let U be the universe, we will be interested in the following notions [5]. 1. A ring (or Boolean ring) of sets is a non-empty class R of sets such that if E R and F R, then E F R and E - F R In other words, a ring is a non-empty class of sets which is closed under the formation of unions and differences. Let E be a class of sets. It is not difficult to show that there exists a unique ring R(E), the smallest ring containing E; it will called the ring generated by E. 2. An algebra (or Boolean algebra) of sets is a nonempty class R of sets such that (a) if E R and F R, then E F R (b) if E R, then E Since E-F = (E F) ), it follows that every algebra is a ring. 3. A σ-ring(or σ-boolean Ring) of sets is a non-empty class S of sets such that (a) if E S and F S, then E - F S (b) if E i S then i {E i I =1, 2, } S A σ-algebra is σ-ring containing U. We are interested in finite universes, σ-ring (σ- Algebra) is the same as Ring and Algebra. 4. A non-empty class H of sets is hereditary if, whenever E S and F E, then F S. The power set of X is a hereditary class. For every ring R, H(R) is the smallest hereditary ring generated by R. 5. A measure is an extended real valued, non-negative, and countably additive set function µ, defined on a ring, and such that µ(0)=0. If µ is a measure on a ring R and if, for every E in H(E), µ *(E ) inf {Σ n µ ( E n ) E n E n and E n R}, then µ * is an outer measure on H(R); if µ is finite or σ-finite so is µ *, where inf is the least upper bound. We are interested in finite universes only, so the infinite sum is in fact a finite sum. 6. Let µ be a measure on a ring R. For every E in H(E), we define µ * (E ) sup {Σ n µ ( E n ) E n E n and E n R}, Then µ * is called an inner measure induced by µ, where sup is greatest lower bound. Infinite sum is finite; see item 4. 7. From [5], pp. 50, item 5 can be improved to µ *(E ) inf { µ ( F ) E F and F R}, Since it is finite S(R) = R and extended measure is µ itself. 4. Borel Sets for Neighborhood Systems 8. Traditionally a Borel set is defined on a topological space, we will extend it to a neighborhood system space. Let C be the class of all compact and closed sets. As usual the Borel set is the σ-ring generated
by the class of all compact and closed sets; it will be denoted by BOrel(U). 9. Since we consider finite neighborhood systems only, all closed sets are compact; and σ-ring is a ring; a σ-algebra is an algebra. 10. BOrel(U) is an algebra generated by closed sets; BOrel(U) is an algebra generated by open sets. 11. If U is S 5 -neighborhood system space, then BOrel(U) is the collections of all definable sets, namely, finite unions of equivalence classes; definable sets are the clopen sets. Proposition 1. Let U be a finite S 5 -neighborhood system space and µ is a measure (for example, the counting measure that is the cardinal number of a finite set) on BOrel(U). Then the outer measure and inner measure µ*(e ) = µ (C(E )) µ * (E) = µ (I(E)) are the measure of lower and upper approximation. Corollary 2. U is a finite S 5 -neighborhood system space and µ is a measure. ρ(e)=µ(e)/total, where TOTAL= µ(u). Then P is a probability measure and its outer and inner probability measure are the probability of upper and lower approximation; they are belief and plausibility functions respectively; see [17] ρ is important measure, so we will give a formal definition in next. 12. Let U be a finite set and µ is the counting measure. For simplicity, the probability measure ρ=µ(e)/total will be called counting probability measure. 13. If U is S 4 -neighborhood system space, then BOrel(U) is the class of all finite, disjoint unions of proper differences of sets of closed sets, and BOrel(U) is the class of all finite, disjoint unions of proper differences of sets of closed sets. Proposition 3. U is a finite S 4 -neighborhood system space and µ is a measure (for example, the counting measure that is the cardinal number of a finite set) on BOrel(U). Then the outer measure and inner measure µ * (E) = sup { µ ( F ) E F and F are closed}, µ*(e ) = inf { µ * (F ) E F and Fare open}, 5. Belief Functions Let us recall some notions from [18]. Let U be a finite set and POwer(U) be its power set. If we use a full word as a notation, we cap the first two characters; so that one can distinguish between notations and words. 14. Belief function: A unit interval valued function Bel : POwer(U) [0, 1] is called a belief function if (a) Bel (0 )=0 (b) Bel(U)=1 (c) For every finite collection E j, j=1,2, n of subsets of U, Bel( j n E j ) Σ s n (-1) t Bel ( E s ) where s represent all possible finite subsets of {1, 2,..n} and t is the n- s +1, where s denote the cardinal number of s. Bel can be constructed from basic probability (see next item): Bel (A) = Σ m(b), where B varies through all subsets of A. 15. Basic probability: A unit interval valued function m : POwer(U) [0, 1] is called basic probability if (a) m(0) =0 (0 is also used to denote empty set) (b) Σ n m( E n ) =1, where E n varies through POwer(U). This definition is somewhat deceiving, what we really have here is a generalization of probability mass function from points to subsets. If we assign basic probability to each basic neighborhood (and zero to all other sets), we get immediately a belief function on U. More generally if we assign basic probability to all minimal neighborhoods (and zero to all other sets), we again get a belief function on U. Neighborhood systems (of finite space) are the most natural underlying structure for belief functions. Conversely, if a space has a belief function, then there is a very natural neighborhood system associated to the belief function: Given a
belief function, there is a basic probability. The collection of sets on which the basic probability are non-zero is a neighborhood system. Namely, we have the following: Propositon 4. U is a finite space. U has a belief function iff there is a neighborhood system on U. In the next few paragraphs, we describe some specific examples on previous theorem. 16. In [11], Lin and Hadjimichael studied nonclassificatory learning. It is a multilevel learning. Mathematically, it is a sequence of mappings. At first level, it maps each point (a base concept) to its unique basic neighborhood, called a concept of level 1. The family of such basic neighborhoods is denoted by COncept(1) or simply Concept. In general step, it maps each point in COncept(n) to an element in COncept(n+1), where COncept(n+1) is called concept of level (n+1) and is the family of basic neighborhoods of COncept(n). Implicitly the level one learning is also in [12]; the soft rules in level 0 are hard rules in level 1. 17. Let COncept ={C 1, C 2,, C i,...,c n } be the distinct list of basic neighborhoods (note that two distinct points may have the same neighborhood.) For example, in rough set theory, COncept is the set of equivalence classes. Let µ be the external sum measure. Recall that is the cardinal number. µ (COncept)= C 1 + C 2 + + C i + + C n µ (C i )= C i Next, we consider the following basic probability m : POwer(2 COncept ) [0, 1] defined by the equations, (a) m(c i ) = µ (C i )/µ(concept) (b) m(a) = 0 if A Concept. m induces a belief function on U; we call it the external counting belief function, denoted by e_c_bel of the basic neighborhood system. The "same" m, as a probability mass function, induces a probability measure P_m on Power(COncept) 18. Now we will consider the learning map LEarn : U COncept defined by LEarn(x) = C i, where C i is the unique basic neighborhood of x, or equivalently the concept learned by x. Some comments are in order. For each x there is a unique basic neighborhood, so LEarn is a well defined map (we also consider multi-valued learning [11]). The map LEarn gives rise to a partition on U (its quotient set is isomorphic to COncept). The probability measure P_m on POwer(COncept) induces a probability measure ρ_m on U as follows: ρ_m (LEarn -1 (X))=P_m(X), X POwer(COncept). Note that the collection of all inverse image, LEarn -1 (X)), X POwer(COncept) is the σ-ring generated by the equivalence classes of U. So we have results similar to the Corollary 2. Theorem 5. U is a finite basic neighborhood system. COncept is the distinct list of basic neighborhoods. Then, the inner probability of ρ_m is the external counting belief function i.e., ρ_m * = e_c_bel (item 17) 19. This is a generalization of Pawlak's results. The outer and inner probability measures of ρ_m are the probability of lower and upper approximation of the equivalence relation LEarn. This theorem is related to, but different from [22]. 20. We will call this σ-algebra the canonical algebra of the basic neighborhood system; ρ_m the canonical probability; the belief function the canonical belief function. 6. Conclusions Neighborhood systems were introduced by first author into the arena of advanced computing for modeling approximate retrievals. It turns out to be a very effective notion in handling uncertainty. In this paper, we examine the "uncertainty functions," measure, probability, and belief functions in terms of neighborhood systems; fuzzy sets were treated in [14]. The study conclude that neighborhoods may be a right mathematical formalism for uncertainty. It seems one of the effective formalism to granulate information [24]. Acknowledgment This author would like to express his deepest thank to Professor Zadeh for his kind guidance and warm invitation to join the Berkeley Initiative in Soft Computing group (BISC). Our deepest thanks also go to
Dr. Martin Wildberger at EPRI, Electric Power Research Institute, for his generous sponsorship. References 1. Back, T., Evolutionary Algorithm in Theory and Practice, Oxford University Press, 1996 2. Bairamian, S., Goal Search in Relational Databases, Thesis, California State University at Northridge, 1989. 3. Chellas, B., Modal Logic, an Introduction, Cambridge University Press, 1980. 4. Engesser, K., Some connections between topological and Modal Logic, Mathematical Logic Quarterly, 41, 49-64, 1995. 5. Halmos, P., Measure Theory, Van Nostrand, 1950. 6. Kelly, J., General Topology, Van Nostrand, 1955. 7. Lin, T.Y., Neighborhood Systems and Relational Database. Proceedings of CSC 88, February, 1988. 8. Lin, T.Y., Neighborhood Systems and Approximation in Database and Knowledge Base Systems, Proceedings of the Fourth International Symposium on Methodologies of Intelligent Systems, Poster Session, October 12-15, 1989. 9. Lin, T.Y., Rough Sets, Neighborhood Systems and Approximation, Fifth International Symposium on Methodologies of Intelligent Systems, Selected Papers, Oct. 1990. (Q. Liu, K. J. Huang and W. Chen). 10. Lin, T.Y., Topological Data Models and Approximate Retrieval and Reasoning, Proceedings of Annual ACM Conference, February, 1989. 11. Lin., T. Y. and Hadjimichael, M., Nonclassificatory Generalization in Data Mining, Proceedings of The Fourth Workshop on Rough Sets, Fuzzy Sets and Machine Discovery, Tokyo, Japan, November 8-10,1996 12. Lin T. Y., and Yao, Y. Y., Mining Soft Rules Using Rough Sets and Neighborhoods, Symposium on Modeling, Analysis and Simulation, CESA 96 IMACS Multiconference (Computational Engineering in Systems Applications), Lille, France, July 9-12, 1996, Vol. 2 of 2,pp.1095-1100. (Coauthor Yao) 13. Lin, T. Y. A Set Theory for Soft Computing, Proceedings of 1996 IEEE International Conference on Fuzzy Systems, New Orleans, Louisiana, September 8-11, 1996. 14. Lin T. Y., Neighborhood Systems -Applications to Qualitative Fuzzy and Rough Sets, Advances in Information Sciences, Volume IV. Ed. Paul Wang. 15. Motro, A., Supporting goal queries in relational databases, Expert Database Systems, Proceedings of the First International Conference, L. Kerschberg, Institute of Information Management, Technology and Policy, University of S. Carolina, 1986. 16. Pawlak, Z., Rough sets. Theoretical Aspects of Reasoning about Data, Kluwer Academic Publishers, 1991 17. Pawlak, Z., Rough Probability, Bull. Pol. Acd. Sci., Math 32, 607-615, 1984 18. Schaffer, G., A Mathematical theory of Evidence, Princeton University, 1976. 19. Sierpenski W. and Krieger, C., General Topology, University of Torranto press, 1956 20. Yao, Y., "Two View of Rough Sets on Finite Universes," Journal of Approximate Reasoning. To appear 21. Yao, Y., and Lin, T.Y., Generalization of Rough Sets using Modal Logics, Intelligent Automation and Soft Computing, an International Journal, to appear, 1996 22. Yao and Lingras, Belief Function in Rough Set Models, Proceedings of Second Annual Joint Conference on Information Science, Wrightsville Beach, North Carolina, Sept. 28-Oct. 1, 1995, pp. 190-193. 23. Zadeh L., The Key Roles of Information Granulation and Fuzzy logic in Human Reasoning, 1996 IEEE International Conference on Fuzzy Systems, September 8-11, 1996 24. Zadeh L., Fuzzy Sets, Information and Control, 8, 1965, pp. 338-353. Tsau Young (T. Y.) Lin received his Ph.D from Yale University, and now is a Professor at San Jose State University and Visiting Scholar in BISC, University of California-Berkeley. He has been chairs and members of program committees in various conferences and workshops, associate editors and members of editorial boards of several international journals. He is the president of International Rough Set Society. His interests include approximation in database and knowledge-base systems, data mining, data security, fuzzy sets, intelligent control, Petri nets, and rough sets (alphabetical order). Yiyu (Y.Y.) Yao received his Ph.D from Univeristy of Regina and now is an associate professor at Lkehead University. He served in the program committees of several conferences and workshops, also is an associate editor of an international journal. He is the secretary of rough control group. His interests include fuzzy sets, information retrieval, and rough sets (alphabetical order).