On Submodular and Supermodular Functions on Lattices and Related Structures

Transcription

1 On Submodular and Supermodular Functions on Lattices and Related Structures Dan A. Simovici University of Massachusetts Boston Department of Computer Science Boston, USA Abstract We give single-operations characterizations for submodular and supermodular functions on lattices that have monotonicity properties. We associate to such functions metrics on lattices and we investigate corresponding metrics on the sets of partitions. Keywords-lattice; semilattice; submodularity; entropy; I. Introduction Submodular functions are useful in combinatorial optimization problems [3], [4], [6]. They are defined as functions of the form f : P(S R, where S is a finite set and P(S is the set of subsets of S, which satisfy the submodular inequality, that is, f(x Y+f(X Y f(x + f(y for X,Y P(S. This inequality is equivalent to the diminishing return property of these functions which means that for every X,Y P(S such that X Y and x S Y, we have f(x {x} f(x f(y {x} f(y. In this note we study submodular functions and their duals (known as supermodular functions defined on lattices and we link these functions with a generalization of conditional entropy in lattices and with certain classes of metrics defined on these structures. The characterization of submodular or supermodular functions that have a monotonicity-linked property (obtained in Section II is formulated using only one of the lattice operations, which opens the possibility of extending the notion of modularity to semilattices. Section III is dedicated to submodular and supramodular functions defined on partition lattices. The extension of semimodularity to functions defined on semilattices is of interest for the generalization of the notion of entropy (and of metrics derived from this notion from partition lattices to other algebraic structures that play a role in designing data mining algorithms. II. Submodular Functions on Lattices A semilattice is a semigroup (S,, where is a commutative and idempotent operation. This is a pervasive algebraic structure, with numerous applications in mathematics and computer science. A lattice is an algebraic structure (L,, such that both (L, and (L, are semilattices and the two operations and satisfy the absorption laws (x y y = y and (x y y = y, for x,y L. Every semilattice (S, generates a partial order on S defined by x y if and only if x y = y. Thus, a (L,, generates two partial orders on L, 1 and 2 defined by x 1 y if x y = y, and x 2 y if x y = y for x,y L. Note that, by the absorption laws, we have u 1 v if and only if v 2 u for u,v L. The partial orders 1 and 2 are said to be dual of each other. Unless stated explicitly otherwise, we shall use the partial order 1 on lattices and will denote it simply by. If (P, and (Q, are two partially ordered sets (posets, then f : P Q is a monotonic function if u v implies f(u f(v for u,v P. If u < v implies f(u < f(v, then f is said to be strictly monotonic. The set of monotonic (strictly monotonic functions from P to Q is denoted by MON(P,Q (smon(p,q, respectively. If u v implies f(u f(v for u,v S, then f is an anti-monotonic function; the function f is strictly anti-monotonic if u < v implies f(u > f(v for all u, v P. The set of anti-monotonic (strictly

2 anti-monotonic functions is denoted by A-MON(P, Q (sa-mon(p, Q, respectively. Let (L,, be a lattice. A function f : L R is submodular if f(x y+f(x y f(x+f(y, for every x,y L. The function f : L R is supermodular if f(x y+f(x y f(x+f(y, for every x,y L. The classes of submodular functions and supermodular functions on (L,, are denoted by SUBM(L,, and by SUPM(L,,, respectively. When the lattice is clear from context we denote these classes by SUBM and SUPM. If a function f : L R is both submodular and supermodular, then f satisfies the equality f(x y + f(x y = f(x + f(y for x,y L. Such functions are known as modular functions or as lattice valuations. The last term is used in [1], where lattices that have strictly monotonic valuations are referred to as metric lattices. We show here that metrics can be introduced on lattices using submodular or supermodular functions and that the presence on metrics that have certain monotonicity properties imply the existence of submodular or supermodular functions. EXAMPLE 2.1. Many submodular functions can be naturally associated to finite graphs (see [5], [2]. Let G = (V,E be a graph having V as its set of vertices and E as its set of edges. For a set of edges K of G let v(k be the number of vertices incident with an edge in K. It is easy to see that v : P(E R is both monotonic and submodular. Let S be a set of vertices of G and let t(s be the number of edges whose endpoints are in S. Then, t is monotonic and supermodular. Similarly, if l(s is the number of edges that have at least one end point in S, then l is a monotonic and submodular function. The next theorem allows the introduction of two subtypes of submodular functions. THEOREM 2.1. Let (L,, be a lattice. If f : L R is a function such that f(t+f(u v f(t u+f(t v, (2.1 for t,u,v L, or f(t+f(u v f(t u+f(t v (2.2 for t,u,v L, then f is a submodular function on L. Proof: Suppose that f(t + f(u v f(t u + f(t v for t,u,v L. Substituting u v for t we obtain f(u v+f(u v f((u v u+f((u v v = f(u+f(v, by the absorption property of L, which shows that f is submodular. If f(t+f(u v f(t u+f(t v for t,u,v L, by substituting u v for t we obtain f(u v+f(u v f((u v u+f((u v v = f(u+f(v, so f is again, submodular. Theorem 2.1 allows us to introduce two classes of functions on a lattice(l,,. Namely, SUBM(L,, consists of those function that satisfy Inquality (2.1 and SUBM(L,, consists of those function that satisfy Inquality (2.2. THEOREM 2.2. If f : L R is a function such that f(t+f(u v f(t u+f(t v for t,u,v L, or f(t+f(u v f(t u+f(t v for t,u,v L, then f is a supermodular function on L. Proof: The proof of this theorem that refers to supermodularity has an entirely similar argument. We denote the classes of functions SUBM(L,,, SUBM(L,,, by SUBM, SUBM, respectively, when the lattice L is clear from context. THEOREM 2.3. Let (L,, be a lattice. Any function f SUBM is anti-monotonic and any function in SUBM is monotonic. Proof: Let f : L R be a function in SUBM and let t,u be two elements of L such that t u. Choosing v = u in the definition of SUBM yields f(t+f(u f(t+f(t u = 2f(t, which implies f(u f(t. Thus, f is anti-monotonic. Similarly, choosing v = u in the definition of SUBM we obtain f(t+f(u 2f(u, so f(t f(u, which shows that f is monotonic. THEOREM 2.4. If f SUBM and f is anti-monotonic, then f SUBM ; if f SUBM and f is monotonic, then f SUBM.

3 Proof: Let f be a submodular and anti-monotonic function. The submodularity implies f((t u (t v+f((t u (t v f(t u+f(t v for every t,u,v L. Since (t u (t v t it follows that f(t f((t u (t v. By the subdistributive inequality that holds in any lattice (see [1] or [7] we have (t u (t v t (u v, hence f(t (u v f((t u (t v because f is anti-monotonic. Again, by the anti-monotonicity of f, f(u v f(t (u v, so f(u v f((t u (t v. Consequently, we have f(t+f(u v f((t u (t v +f((t u (t v f(t u+f(t v. The second statement of the theorem has a similar argument. COROLLARY 2.2. For any lattice (L,, we have SUBM = SUBM MON, SUBM = SUBM A-MON, SUPM = SUPM MON, SUPM = SUPM A-MON. For a function f : L R define the functions κ f : L 2 R 0 and λ f : L 2 R 0 by κ f (x,y = f(x y f(y, λ f (x,y = f(x y f(y for x,y L. Note that f is submodular (supermodular if and only if κ f (x,y+λ f (x,y 0 (κ f (x,y+λ f (x,y 0. The functions κ f and λ f are intended as abstract counterparts of conditional entropy. The next result shows that the monotonicity properties of κ f and λ f in their first argument imply monotonicity properties for f, while monotonicity properties of κ f and λ f in their second argument imply modularity properties for f. THEOREM 2.5. Let(L,, be a lattice. Iff SUBM, then κ f is anti-monotonic in its first argument and monotonic in the second. If f SUBM, then λ f is monotonic in its first argument and anti-monotonic in the second. Conversely, if κ f is anti-monotonic in its first argument, then f is anti-monotonic; if κ f is monotonic in its second argument, then f is submodular. Also, if λ f is monotonic in its first argument, thenf is monotonic; ifλ f is anti-monotonic in its second argument, then f is supermodular. Proof: The anti-monotonicity of κ f in its first argument follows directly from the anti-monotonicity of f. To prove the monotonicity of κ f in its second argument let y,z L be such that y z. The definition of SUBM allows us to write f(z+f(x y f(z x+f(z y = f(z x+f(y (because y z, which translates into κ f (x,y κ f (x,z. Similarly, the monotonicity of λ f in its first argument follows from the monotonicity of f. To prove the antimonotonicity of λ f in its second argument, let y,z L be such that y z. Since f SUBM we have f(y+f(x z f(y x+f(y z = f(y x+f(z (because y z, which amounts to λ f (x,z λ f (x,y. For the converse implications suppose that κ f is antimonotonic in its first argument, so x 1 x 2 implies κ f (x 1,y κ f (x 2,y, that is f(x 1 y f(x 2 y for every y L. Choosing y = x 2 it follows that f(x 1 f(x 2, that is, f is anti-monotonic. Suppose now that κ f is monotonic in its second argument, that is y 1 y 2 implies f(x y 1 f(y 1 f(x y 2 f(y 2. Choosing y 2 = x y 1 yields f(x y 1 +f(x y 1 f(y 1 +f(x, which shows the submodularity of f. Similar arguments can be made for the last part of the theorem involving the function δ f. THEOREM 2.6. Let (L,, be a lattice. If f SUBM, then κ f (u,v+κ f (v,w κ f (u,w for u,v,w L. If f SUBM, then λ f (u,v+λ f (v,w λ f (u,w for u,v,w L. Proof: It is easy to see that by expressingκ f in terms of f the inequality that we need to prove is equivalent to f(u v+f(v w f(u w+f(v, which holds by the definition of SUBM. The proof of the second part is similar. Note that κ f (x,x = 0 for f SUBM and x L. Similarly, λ f (x,x = 0 for f SUBM and x L. For f SUBM define the mapping d f : L 2 R as d f (x,y = κ f (x,y+κ f (y,x = 2f(x y f(x f(y

4 for x,y L. Similarly, for g SUBM, let δ f : L 2 R be given by δ g (x,y = λ g (x,y+λ g (y,x = 2g(x y f(x f(y for x,y L. It follows immediately from Theorem 2.6 that d f is a semimetric on L, when f belongs to SUBM and that δ g has the same property if g SUBM. If the functions involved are in smon (in the first case or in sa-mon (in the second, then d f or δ g are metrics. Conversely, if d f is a semimetric on L, where d f (x,y = 2f(x y f(x f(y, thenf SUBM. Indeed, in this case, the triangular inequality of d f amounts to 2f(x y f(x f(y +2f(y z f(y f(z 2f(x z f(x f(z, for x,y,z L, which is clearly equivalent to the defining equality of SUBM. Similarly, if d g is a semimetric on L, g belongs to SUBM. III. Submodular Functions on the Lattice of Partitions A partition of a non-empty set S is a collection of nonempty subsets of S, π = {B i i I} such that i,j I and i j implies B i B j = and i I B i = S. The subsetsb i are referred as blocks of π. The set of partitions of a set S is denoted by PART(S. If π,τ PART(S we write π τ if every block of π is included in a block of τ. This relation between partitions is a partial order. The largest element of the partially ordered set (PART(S, is the one-block partition ω S = {S}, while the smallest element is the partition α S = {{x} x S}. We assume from now on that all partitions are considered over finite sets. The partial ordered set (PART(S, is actually a lattice. The meet of two partitions π τ is the partition of S whose blocks are the non-empty intersections of the form B C, where B π and C σ. Consider the bipartite graph G π,σ having {B 1,...,B m,c 1,...,C n } as its vertices, where π = {B 1,...,B m } and σ = {C 1,...,C n }. An edge exists betweenb i andc j if and only ifb i C j. The blocks of the partition π σ consist of non-empty sets of the form B i C j and correspond to the edges of G π,σ. Let C 1,...,C k be the connected components of G π,σ. For every connected component C we have {Bi B i C} = {C j C j C} and that the blocks of the partition π σ have the form C. A partition π covers a partition µ in (PART(S, if π can be obtained from µ by fusing two blocks of µ. Partition lattices are prototypical for the so called upper semimodular lattices [1], characterized by the following property: if π 1 π 2 and both π 1,π 2 cover a partition σ, the π 1 π 2 covers both π 1 and π 2. If π PART(S and C S, we denote by π C the partition π C = {B C B π}. This is the trace of π on C. Note that if π,σ PART(S, π = {B 1,...,B n }, and σ = {C 1,...,C n }, then we have π σ = π C1 + +π Cn = σ B1 + +σ Bm. (3.3 Let S be a finite set such that 2 and let β be a number, β > 1. For a partitionπ = {B 1,...,B m } PART(S define the function f S : PART(S R as m ( β Bj f S (π = b 1, where β > 1. Then, f S (ω S = 0 and f S (α S = b(1 β 1. Let S 1,...,S l be l non-empty and pairwise disjoint sets and let S = l k=1 S k. Assume that π k = {B k1,...,b kmk } is a partition on S k for 1 k l. Then, the collection of sets {B kj 1 k l,1 j m k } is a partition of the set S denoted by π 1 + +π l. We have l ( β Sk f S (π 1 + +π l = f (π Sk k+f S ({S 1,...,S l }. k=1 (3.4 Therefore, taking into account Equalities (3.3 we can write f S (π σ = ( β Cj f Cj(π Cj+f S (σ. By the definition of κ fs we have κ fs (π,σ = f S (π σ f S (σ = ( β Cj f Cj(π Cj. (3.5 LEMMA 3.1. Let φ : [0,1] R be a convex function such that φ(x x for x [0,1], w 1,...,w n be n positive numbers such that n w i = 1, and let

5 a 1,...,a n [0,1]. We have ( n 1 φ w i a i φ w i (1 a i φ(w i (1 φ(a i φ(1 a i. Proof: By Jensen s inequality applied to φ we have φ w i a i w i φ(a i, φ w i (1 a i w i φ(1 a i. Taking into account that n w i = 1 we have ( n 1 φ w i a i φ w i (1 a i w i (1 φ(a i φ(1 a i φ(w i (1 φ(a i φ(1 a i because w i φ(w i for 1 i n. LEMMA 3.2. Let π PART(S and let C,D be two nonempty disjoint subsets of S. We have C β f C (π C + D β f D (π D ( C + D β f C D (π C D. Proof: Let π = {B 1,...,B n }. Define w i = B i (C D,a i = C D B i C B i (C D for 1 i n, so 1 a i = Bi D B. i (C D By Lemma 3.1 applied to the function φ(x = x β, that is convex on [0,1] when β > 1 we have: β ( n β B i C B i D 1 C D C D ( ( β ( Bi (C D 1 C D ( β Bi D, B i (C D B i C B i (C D which is equivalent to the inequality of the lemma. THEOREM 3.1. The function f S : PART(S R is anti-monotonic and submodular. Proof: To prove that f S is anti-monotonic it suffices to show that if π τ such that τ covers π, then f(π f(τ. β Suppose that π = {B 1,...,B m }; without loss of generality we may assume that τ results from π by fusing the blocks B m 1 and B m. Since B m 1 and B m are nonempty sets we have B m 1 1 and B m 1 which implies B m 1 β + B m β B m 1 B m β. Therefore, m ( β Bj f S (π = b 1 b 1 = f S (τ, m 2 ( β Bj ( β Bm 1 B m which allows us to conclude that f S is indeed antimonotonic. To prove that f S is submodular we shall use the second part of Theorem 2.5 and show that the function κ fs (π,σ is monotonic in its second argument. Let π,σ,τ PART(S such that σ τ and τ covers σ. Again, we may assume without loss of generality that σ = {C 1,...,C n } and τ is obtained from σ by fusing C n 1 and C n. By Equality (3.5 it suffices to show that C n 1 β f Cn 1 (π Cn 1 + C n β f Cn (π Cn C n 1 C n β f Cn 1 C n (π Cn 1 C n, which holds by Lemma 3.2. IV. Further Work The characterization of submodular (or supermodular monotonic and anti-monotonic functions provided by Corollary 2.2 makes use only of one of the operations of the lattice. This makes it possible to extend the notions of submodularity and supermodularity to functions defined on semilattices. This extension is relevant to defining entropies for set covers and metrics on the space of covers of a set. In turn, metrics on set covers can help extending well-known data mining algorithms that make use of the metric space of partitions in feature selection and classification to multi-valued attributes. Acknowledgement This research was supported by a S&T grant from the University of Massachusetts. References [1] G. Birkhoff. Lattice Theory. American Mathematical Society, Providence, Rhode Island, third edition, [2] A. Frank. Submodular functions in graph theory. Discrete Mathematics, 111: , [3] B. Korte and J. Vygen. Combinatorial Optimization: Theory and Algorithms. Springer, Heidelberg, fifth edition, 2012.

6 [4] J. Lee. A First Course in Combinatorial Optimization. Cambridge University Press, Cambridge, [5] H. Narayanan. Submodular Functions and Electrical Networks. North Holland, [6] A. Schrijver. Combinatorial Optimization. Springer, New York, [7] D. A. Simovici and C. Djeraba. Mathematical Tools for Data Mining. Springer, London, second edition, 2014.