Submodular function optimization: A brief tutorial (Part I)

Transcription

1 Submodular function optimization: A brief tutorial (Part I) Donglei Du (ddu@unb.ca) Faculty of Business Administration, University of New Brunswick, NB Canada Fredericton E3B 9Y2 Workshop on Combinatorial Optimization with Applications in Transportation and Logistics, Beijing July 28-August 2, 2015 Donglei Du (UNB) Submodular 1 / 130

2 Table of contents 1 Why submodular? 2 Submodular function 3 Convex closure 4 Lovász extension 5 Multilinear extension 6 Operations preserving submodularity 7 Examples of submodular function 8 Unconstrained submodular optimization Minimization Maximization 9 Constrained submodular optimization Minimization Maximization 10 Multi-criteria submodular maximization Donglei Du (UNB) Submodular 2 / 130

3 Section 1 Why submodular? Donglei Du (UNB) Submodular 3 / 130

4 Why submodular? According to Lovász [Lovász, 1983a] Submodular functions occur in many mathematical models in Economics, Engineering, Science, Management Science, and other sciences. Submodularity is a very natural property of various functions occurring in such models; quite often the only non-trivial property which can be stated in general. Submodularity can be preserved under many operations and transformations, and thereby the effective range of results can be extended, elegant proof techniques can be developed and unforeseen applications of certain results can be given. Submodularity functions exhibit sufficient structure so that a mathematically beautiful and practically useful theory can be developed. There are theoretically and practically (reasonably) efficient methods to find the minimum of a submodular function. Donglei Du (UNB) Submodular 4 / 130

5 A few more reasons on why submodular? Recent advances on submodular maximization with/without constraints and minimization with constraint provide practically efficient methods to approximately solve these problems. It is mysterious and arouse curiosity. It provides a thread to unify/simplify/demystify existing results/models/techniques. Donglei Du (UNB) Submodular 5 / 130

6 Section 2 Submodular function Donglei Du (UNB) Submodular 6 / 130

7 Submodular set function Submodular set function: S, T N, a set function f : 2 N R is submodular iff f (S) + f (T ) f (S T ) + f (S T ). (1) Donglei Du (UNB) Submodular 7 / 130

8 Submodular function on a lattice (In preparation as Part II) Submodular function on any lattice L := (N,, ): x, y L, a function f : L R is submodular iff f (x) + f (y) f (x y) + f (x y). A lattice is a partially ordered set (poset) (N, ) (namely, is reflexive, anti-symmetric, and transitive) in which every two elements x and y have a supremum x y (also called a least upper bound or join), and an infimum x y (also called a greatest lower bound or meet). Donglei Du (UNB) Submodular 8 / 130

9 Examples of lattices (2 N, =, = ) or equivalently ({0, 1} N, = max, = min): reduce to the submodular set function: for any x = (x 1,..., x N ), y = (y 1,..., y N ) {0, 1} N : f (x 1,..., x N ) + f (y 1,..., y N ) f (max{x 1, y 1 },..., max{x N, y N }) (R N, = max, = min); Here max and min are component-wise. + f (min{x 1, y 1 },..., min{x N, y N }). Donglei Du (UNB) Submodular 9 / 130

10 Distributive lattice Note: Many results for the submodular set function can be extended to submodular function on a distributive lattice. A lattice (L,, ) is distributive if the following additional identity holds for all x, y, and z in L: or equivalently (its dual) x (y z) = (x y) (x z). x (y z) = (x y) (x z) A lattice is distributive iff it is isomorphic to a lattice of sets (closed under set union and intersection). Donglei Du (UNB) Submodular 10 / 130

11 Examples of distributive lattices Every totally ordered set (a.k.a., chain) is a distributive lattice with max as join and min as meet. Or equivalently, a special lattice, where {a b, a b} = {a, b} a, b. The set of real numbers ordered by the usual less than (<) or greater than (>) relations is totally ordered. The lexicographical order on the Cartesian product of a set of totally ordered sets indexed by an ordinal, is itself a total order: (a, b) (c, d) iff a < c or (a = c and b d). The usual element-wise order is only a partial order (a, b) (c, d) iff a c and b d. Donglei Du (UNB) Submodular 11 / 130

12 (Individual) First order condition: (weakly) decreasing For each fixed i S =, f i (S) := f (S i) f (S) is (weakly) decreasing of S. Namely, S T, i T =, f i (S) = f (S i) f (S) f (T i) f (T ) = f i (T ). (2) This translates to the economic interpretation for diminishing returns (a.k.a. economies of scale, economies of scope). This definition reminds us of the concavity of continuous function; decreasing first-order derivative. Donglei Du (UNB) Submodular 12 / 130

13 Proof: (1) (2) (1) (2) is easy by letting S = S i and S T : f (S i) + f (T ) f (S) + f (T i). (2) (1): Let (relabeling if necessary): S T = [r] = {1,..., r} From (2) we obtain f i (S T [i 1]) f i (T [i 1]), i = 1,..., r. Summing these r inequalities yields (1): f (S) f (S T ) f (S T ) f (T ). Donglei Du (UNB) Submodular 13 / 130

14 (Group) First order condition: (weakly) decreasing For each fixed A S =, f A (S) := f (S A) f (S) is (weakly) decreasing of S. Namely, S T, A T =, f A (S) = f (S A) f (S) f (T A) f (T ) = f A (T ). Donglei Du (UNB) Submodular 14 / 130

15 Second order condition: (weakly) negative For each {i, j} S = f i,j (S) := [f (S {i, j}) + f (S)] [f (S i) + f (S j)] 0 f j (S i) f j (S) (3) Again, this definition reminds us of the concavity of continuous function; (weakly) negative second-order derivative. Donglei Du (UNB) Submodular 15 / 130

16 Proof: (1) (3) (1) (3) is easy by letting S = S i and T = S j: f (S i) + f (S j) f (S {i, j}) + f (S). (3) (2): Take s T, j / T and let (relabeling if necessary): T S = [r] := {1,..., r} From (3) we obtain f j (S) f j (S [1]) f j (S [2])... f j (S [r]) = f j (T ) Donglei Du (UNB) Submodular 16 / 130

17 Equivalent definition of submodularity More equivalent definitions exist (e.g. [Nemhauser et al., 1978]). For example: f (T ) f (S) + f j (S), S T N j T S Donglei Du (UNB) Submodular 17 / 130

18 Section 3 Convex closure Donglei Du (UNB) Submodular 18 / 130

19 Convex closure of any set function Let w [0, 1] N be sorted such that 1 w 1... w n 0 The convex closure of any set function f is defined as f (w) = max{x 0 + w T x : x 0 + x(a) f (A), A N} (4) = min f (A)y A : y A = w i, i N, (5) A N i A N y A = 1, y R 2n + A N Donglei Du (UNB) Submodular 19 / 130

20 An example of convex closure for n = 2: primal Assume f ( ) = 0, f ({1}) = f ({2}) = f ({1, 2}) = 1. Or equivalently, f (0, 0) = 0, f (1, 0) = f (0, 1) = f (1, 1) = 1 via lattice terminology ({0, 1} 2, max, min). (4) becomes f L (w 1, w 2 ) = max w 0 x 0 + w 1 x 1 + w 2 x 2 s.t. x 0 f ( ) = f (0, 0) = 0 x 0 + x 1 f ({1}) = f (1, 0) = 1 x 0 + x 2 f ({2}) = f (0, 1) = 1 x 0 + x 1 + x 2 f ({1, 2}) = f (1, 1) = 1 Donglei Du (UNB) Submodular 20 / 130

21 An example of convex closure for n = 2: dual (5) becomes f L (w 1, w 2 ) = min f (0, 0)y 00 + f (1, 0)y 10 + f (0, 1)y 01 + f (1, 1)y 11 s.t. y 10 + y 11 = w 1 y 01 + y 11 = w 2 y 00 + y 10 + y 01 + y 11 = w 0 = 1 y 0 Donglei Du (UNB) Submodular 21 / 130

22 The optimal solution of the primal and dual for the example For 1 w 1 w 2 0, Primal solution x 1 = 1, x 0 = x 2 = 0 with value w 1. Dual solution y 00 = 1 w 1, y 10 = w 1 w 2, y 01 = w 2 and y 11 = 0 with value w 1. Therefore both are optimal. Symmetrically, when w 2 w 1, the optimal value is w 2. The convex closure for any w = (w 1, w 2 ) [0, 1] 2 is f (w 1, w 2 ) = max{w 1, w 2 } Donglei Du (UNB) Submodular 22 / 130

23 z Illustration of f (w 1, w 2 ) = max{w 1, w 2 } w w Donglei Du (UNB) Submodular 23 / 130

24 Interpretation of convex closure from primal point of view From (4), f is the (point-wise) largest convex function below f. Donglei Du (UNB) Submodular 24 / 130

25 Interpretation of convex closure from dual point of view (X 1, X 2 ) y 00 y 01 1 w 1 1 y 10 y 11 w 1 1 w 2 w 2 Let y x1,...,x n {0, 1} N be a feasible solution of (5). For each index x i {0, 1} (i = 1,..., n), define a Bernoulli random variable X i such that P(X i = 1) = w i. y = y x1,...,x n is a joint probability distribution with X i as marginal distributions. Then convex closure is to find such a joint distribution y such that the expectation of f is minimized: f (w) = min y E y [f (X 1,..., X n )], Donglei Du (UNB) Submodular 25 / 130

26 Section 4 Lovász extension Donglei Du (UNB) Submodular 26 / 130

27 Lovász extension of any set function [Lovász, 1983b, Choquet, 1954] Let w [0, 1] N be sorted such that 1 = w 0 w 1... w n w n+1 = 0 The Lovász extension of any set function f is defined as f L (w) = = n w i (f ([i]) f ([i 1])) (6) i=0 n f ([i])(w i w i+1 ) (7) i=0 Here [i] = {1,..., i}, i = 0,..., n, [0] = and f ([ 1]) = 0. The equality follows by integration by parts, or Abel summation formula Donglei Du (UNB) Submodular 27 / 130

28 An example of Lovász extension for n = 2 Assume f ( ) = 0, f ({1}) = f ({2}) = f ({1, 2}) = 1. For w 1 w 2, from (6)-(7), we have f L (w 1, w 2 ) = f ( )(1 w 1 ) + f (1)(w 1 w 2 ) + f (1, 2)w 2 = f ( ) + (f (1) f ( ))w 1 + (f (1, 2) f (1))w 2 = w 1 For this example, the convex closure and Lovász extension are one of the same! This equivalence is not coincidental as f in this example is submodular (please verify f is indeed submodular!). Donglei Du (UNB) Submodular 28 / 130

29 Lovász extension convex closure for submodular set function If f is submodular, then Proof: Greedy algorithm + LP duality f L (w) = f (w), w [0, 1] N. (8) Donglei Du (UNB) Submodular 29 / 130

30 Lovász extension as Choquet expectation I f L (w) = E λ U[0,1] [f (i : w i λ)] (9) Donglei Du (UNB) Submodular 30 / 130

31 Proof I (7) + F (X ) U[0, 1] for any random variable with distribution function F : Define the following random variable X with probability mass function (pmf) as follows: P(X = f ([i])) = w i w i+1, i = 0,..., n. Therefore the probability distribution function and its inverse (quantile) function are respectively F (X f ([i])) = w 0 w i+1 = 1 w i+1, i = 0,..., n F 1 (1 w i+1 ) = f ([i]), i = 0,..., n Donglei Du (UNB) Submodular 31 / 130

32 Proof II Now the Lovász Extension f L (w) = E[X ] = xdf (x) = 1 = E µ U[0,1] [f (i : w i 1 µ)] = E 1 λ U[0,1] [f (i : w i λ)] 1 λ U[0,1] λ U[0,1] {}}{ = E λ U[0,1] [f (i : w i λ)]. 0 F 1 (µ)dµ Donglei Du (UNB) Submodular 32 / 130

33 A set function f is submodular its Lovász extension f L is convex I If f is submodular, then f L is convex from (8). If f L is convex, then we show that f is submodular (the proof borrows the idea from [Bach, 2010]). For any A, B N, the vector 1 A B + 1 A B = 1 A + 1 B has components equal to 0 on A B, 2 on A B and 1 on A B = (A\B) (B\A). Donglei Du (UNB) Submodular 33 / 130

34 A set function f is submodular its Lovász extension f L is convex II From (9), we have f ( A B A B ) = = = = 1 f ( i : A B + 1 ) 2 1 A B λ dλ ( f i : 1 ) A B 2 1 A B λ dλ ( f i : A B + 1 ) 2 1 A B λ dλ f (A B)dλ f (A B)dλ = 1 (f (A B) + f (A B)). (10) 2 Donglei Du (UNB) Submodular 34 / 130

35 A set function f is submodular its Lovász extension f L is convex III f L being convex implies that ( 1 f L 2 1 A + 1 ) 2 1 B 1 2 f L (1 A ) f L (1 B ) = 1 2 f (A) + 1 f (B). (11) 2 (10)-(11) together gives the desired result: ( 1 f (A B) + f (A B) = 2f L 2 1 A B + 1 ) 2 1 A B ( 1 = 2f L 2 1 A + 1 ) 2 1 B f (A) + f (B). Donglei Du (UNB) Submodular 35 / 130

36 Properties of Lovász extension f L (w) is piece-wise linear convex f L (w) is positively homogeneous: f L (λw) = λf L (w), w R + f L (w) coincides with f at all 0-1 points: f (S) = f L (1 S ), S N f L (w) can be evaluated in P-time via the greedy algorithm (even strongly P-time O(N 4 ) [?, McCormick, 2005]). f L (w) has its minimum over the unit cube [0, 1] N attained at a vertex and hence can be minimized over the unit cube in P-time (by the Ellipsoid method [Grötschel et al., 1981] weakly or [?, McCormick, 2005] strongly). Donglei Du (UNB) Submodular 36 / 130

37 Lovász extension of any set (Pseudo-Boolean) function: alternative view Any set function f can be viewed as a pseudo-boolean function f : {0, 1} n R, which can be represented as f (x 1,..., x n ) = A N ˆf (A) i A x i, where the Möbius transform is given by ˆf (A) = S A( 1) A S f (1 S ) Then the Lovász Extension is given by f (x 1,..., x n ) = A N ˆf (A) min i A x i Donglei Du (UNB) Submodular 37 / 130

38 Lovász extension of any set function as Choquet integral [Choquet, 1954] I Given a set-function F on ground set N such that F ( ) = 0 and F (N) = 1, the Lovász extension f L : R n R is defined as follows: for any w R n : f L (w) = = + min{w 1,...,w n} + 0 f (i : w i λ)dλ + F (N) min{w 1,..., w n } f (i : w i λ)dλ + 0 (f (i : w i λ) 1)dλ Donglei Du (UNB) Submodular 38 / 130

39 Lovász extension of any set function as Choquet integral [Choquet, 1954] II Compare it to the regular expectation: E[X ] = = = 0 0 xdf (x) 0 F (x)dx + P(X x)dx + ( F (x) 1)dx 0 (P(X x) 1)dx In contrast to linear expectation, Choquet expectation is a kind of nonlinear expectation, which offers one way to resolve the Allais paradox and the Ellsberg paradox in Economics. Note that these definitions do not depend on the order of the w s. Donglei Du (UNB) Submodular 39 / 130

40 Lovász extension of any set function as Choquet integral [Choquet, 1954] III However, we can show that these definitions are equivalent to the previous definitions (6)-(7). Donglei Du (UNB) Submodular 40 / 130

41 Section 5 Multilinear extension Donglei Du (UNB) Submodular 41 / 130

42 Multilinear extension of any set function Let R(x) be a random set obtained by rounding each x i to 1 with probability x i : then the multilinear extension of f is given by: F (x) = E[f (R(x))] = f (S) x i (1 x i ) S N i S i / S Donglei Du (UNB) Submodular 42 / 130

43 Properties of multilinear extension f monotone implies that F (x) monotone. f submodular implies that F (x + tu) is concave of t if u 0: up-concave. f submodular implies that F (x + t(e i e j )) is convex of t for any i j: cross-convex. Donglei Du (UNB) Submodular 43 / 130

44 Relationship between multilinear and Lovász extension I f submodular implies that F (x) f L (x). Donglei Du (UNB) Submodular 44 / 130

45 Proof ([Vondrák, 2013](Page 32)) I Assume 1 = x 0 x 1... x n x n+1 = 0. F (x) = E x [f (R(x))] n f L (x) = x k (f ([k]) f ([k 1])) k=1 Donglei Du (UNB) Submodular 45 / 130

46 Proof ([Vondrák, 2013](Page 32)) II Note that E x [f (R(x))] = E x [ n k=0 (f (R(x) [k]) f (R(x) [k 1])) ] [ n = E x [f ((R(x) [k 1]) (R(x) k)) k=0 f (R(x) [k 1])]] [ n ] = E x R(x) k f (R(x) [k 1]) k=0 = n [ E x R(x) k f (R(x) [k 1]) ]. k=0 Donglei Du (UNB) Submodular 46 / 130

47 Proof ([Vondrák, 2013](Page 32)) III Note further [k 1] (R(x) [k 1]) and (R(x) k) [k 1] = for all k = 0,..., n. First order difference submodularity implies that k = 0,..., n: R(x) k f (R(x) [k 1]) R(x) k f ([k 1]) Therefore = f ([k 1] (R(x) k)) f ([k 1]) { f ([k]) f ([k 1]), with prob. x k = 0, with prob. 1 x k E x [ R(x) k f (R(x) [k 1]) ] E [f ([k 1] (R(x) k)) f ([k 1])] = x k (f ([k]) f ([k 1])). Donglei Du (UNB) Submodular 47 / 130

48 Proof ([Vondrák, 2013](Page 32)) IV Finally n E x [f (R(x))] x k (f ([k]) f ([k 1])) = f L (x) k=0 Donglei Du (UNB) Submodular 48 / 130

49 Section 6 Operations preserving submodularity Donglei Du (UNB) Submodular 49 / 130

50 Summation, Complement, Restriction, and Contraction Summation: f and g submodular = f + g submodular. Complement: f submodular = f (A) = f (N A) submodular. Restriction: f submodular = Fix S N, f (A S) = f (A S) submodular. Contraction: f submodular = Fix S N, f (A/(N S)) = f (A S) f (S) submodular, A N S. Donglei Du (UNB) Submodular 50 / 130

51 Maximization Given a fixed vector w R n = f (A) = max w j submodular: j A Proof: ( max min max w j, max w j j A j B ( max w j, max w j j A j B ) ) = max j A B w j; max w j. j A B Summing up these two relationships leads to the desired result Donglei Du (UNB) Submodular 51 / 130

52 Composition Given a nonnegative modular function m and a concave function g = f (A) = g(m(a)) is submodular. Proof: From (3), S and i, j / S, f is submodular iff g(m(a) + m i ) g(m(a)) g(m(a) + m i + m j ) g(m(a) + m j ), which is true because g is concave iff g(a + t) g(t) is (weakly) decreasing for any a 0. Donglei Du (UNB) Submodular 52 / 130

53 Minimization with monotone difference Monotone difference: f and g submodular, and f g is monotonic = h := min{f, g} submodular = min{f, k} submodular for any monotone f and constant k. Donglei Du (UNB) Submodular 53 / 130

54 Proof I Due to symmetry, we only prove the case where f g is (weakly) increasing. Case 1. For any S and T, h(s) = f (S) and h(t ) = f (T ) (or symmetrically h(s) = g(s) and g(t ) = f (T )). Then h(s) + h(t ) = f (S) + f (T ) Case 2. h(s T ) + h(s T ). Submodularity {}}{ f (S T ) + f (S T ) Donglei Du (UNB) Submodular 54 / 130

55 Proof II Case 2.1. For any S and T, h(s) = f (S) and h(t ) = g(t ). Then h(s) + h(t ) = f (S) + g(t ) = f (S) + f (T ) f (T ) + g(t ) Submodularity {}}{ f (S T ) + (f (S T ) f (T ) + g(t )) f g {}}{ f (S T ) + g(s T ) h(s T ) + h(s T ) Case 2.2. For any S and T, h(s) = g(s) and h(t ) = f (T ). Then h(t ) + h(s) = f (T ) + g(s) = f (T ) + f (S) f (S) + g(s) Submodularity {}}{ f (S T ) + (f (S T ) f (S) + g(s)) f g {}}{ f (S T ) + g(s T ) h(s T ) + h(s T ) Donglei Du (UNB) Submodular 55 / 130

56 Value of optimization problem Value of optimization problem: maximizing a nonnegative linear function over a submodular polyhedron has submodular optimal values [Nemhauser et al., 1978, Schulz and Uhan, 2010, He et al., 2012]. v(s) = max w 0 + w j x j : x 0 + x(a) f (A), A S j S is submodular over ground set N = {1,..., n}. Proof: (Lovász extension + restriction + summation) v(s) = E λ U[0,1] [f (i : w i S λ)] = E λ U[0,1] [f (S A λ )], where A λ = {i N : w i λ} for any given λ [0, 1] The above is extended to maximizing a separable concave function (i.e., j S f j(x j ), each f j is concave) over a submodular polyhedron (a.k.a. polymatroid) [He et al., 2012]. Donglei Du (UNB) Submodular 56 / 130

57 Section 7 Examples of submodular function Donglei Du (UNB) Submodular 57 / 130

58 Examples Cut function Rank function of matriod Coverage function Facility location problem Entropy of multiple information sources Scheduling problem Influence in a social network... Donglei Du (UNB) Submodular 58 / 130

59 Cut function Fix a graph G = (V, E), directed or undirected, with nonnegative edge weights c : E R +. For any subset of nodes S V, define the cut function: f (S) = f (S) is submodular. Proof: i S,j / S c ij = f (S) + f (T ) f (S T ) f (S T ) c ij + c ij 0 (since c 0). i S T,j T S i T S,j S T f (S) is symmetric for undirected graph. Donglei Du (UNB) Submodular 59 / 130

60 Rank function of matroid A pair (N, I) is a matroid if N is a finite set and I is a nonempty collection of subsets of N satisfying: if I J and J I, then I I, if I, J I and I < J, then I + z I for some z J I. The rank function of matroid (N, I): for any subset S N, r(s) = max{ Y : Y S, Y I} The rank function of any matroid is characterized by three properties: 0 r(s) S and is integer valued for all S N r(s) is increasing, r(s) is submodular Donglei Du (UNB) Submodular 60 / 130

61 Rank function of matroid is submodular For any S T and k / T, if f (S + k) f (S) = 1, then f (S + k) f (S) = 1 f (T + k) f (T ). if f (S + k) f (S) = 0, then S + k / I and hence T + k / I by down-closure, implying further f (T + k) f (T ) = 0. Donglei Du (UNB) Submodular 61 / 130

62 again is increasing and submodular Application: sensor placement in sensor network Donglei Du (UNB) Submodular 62 / 130 Coverage function Fix a ground set N and A 1,..., A m are subsets of N. For any index subset S {1,..., m}, define the coverage function: f (S) = A j f (S) is increasing and submodular. Proof: For any S T and A k / T, f (S + k) f (S) = A k A j j S A k A j = f (T + i) f (T ). j T More generally, for any nondecreasing submodular set function g on N, the function f defined by f (S) = g A j j S j S

63 Maximum facility location There are a set of facilities F = {1,..., n}, and a set of customers D = {1,..., m}. Facility i provides a service of value v ij to customer j. Assuming each customer chooses the opened facility with highest value, the objective is to open no more than k facilities to maximize the total values provided to all customers. Assume we open S F, then the objective function can be written as f (S) = j D max i S v ij. f (S) is increasing and submodular: maximization + summation. Donglei Du (UNB) Submodular 63 / 130

64 Entropy is submodular Consider a random vector X = (X 1,..., X n ) with joint density/mass function p(x) := p(x 1,..., x n ). The entropy function of X is given by H(X ) = E[ log p(x )] Let N = {1,..., n} be the ground set. Denote X A as a random vector whose indices are from A. Define a set function f : 2 N R + : f (A) = H(X A ) Claim: f (A) is submodular. Proof: conditioning reduces entropy. With A B and i / B, H(X B+i ) H(X B ) = H(X i X B ) H(X i X A ) = H(X A+i ) H(X A ) Donglei Du (UNB) Submodular 64 / 130

65 The scheduling problem 1 w j C j This problem can be formulated as the following linear program [Queyranne, 1993, Queyranne and Schulz, 1994]: min w j C j s.t. j N p j C j f (S), S N. j S Here f (S) = 1 2 j S p j pj 2 := 1 ( p(s) 2 + p 2 (S) ). 2 2 j S f (S) is supermodular. Proof: S, T N: assuming nonegeative processing time p, f (S) + f (T ) f (S T ) f (S T ) = p(s T )p(t S) 0. (12) Donglei Du (UNB) Submodular 65 / 130

66 Influence in a social network: the independent cascade model [Kempe et al., 2003, Kempe et al., 2005] Problem description Given a network G = (V, E), there is an initial active set S V. Whenever there is an edge (u, v) E such that u is active and v is not, the node u is given a chance to activate v with probability p uv. Define the influence function f (S) to be the expected number of active nodes at the end of the processing, assuming S is the set of nodes that are initially active. Donglei Du (UNB) Submodular 66 / 130

67 Proof Claim: f (S) is submodular. Proof: (summation + coverage function) Suppose m = E. There are 2 m different configurations of the networks, leading to a random network. Fix one such realization G = (V, E ) (where E E) of the random network. Let f (S, G ) be the eventual number of activated nodes, starting from a set of initial active nodes S V. For any s S, let R(G, s) V be the set of nodes that are reachable from s in G by paths. So f (S, G ) = R(G, s) ; s S f (S) = E G [f (S), G )]. Donglei Du (UNB) Submodular 67 / 130

68 Section 8 Unconstrained submodular optimization Donglei Du (UNB) Submodular 68 / 130

69 Unconstrained submodular optimization min / max S N f (S) Minimization: P Maximization: NP-hard Donglei Du (UNB) Submodular 69 / 130

70 Subsection 1 Minimization Donglei Du (UNB) Submodular 70 / 130

71 Non-combinatorial algorithm via the Lovász Extension Theorem [Grötschel et al., 1981] min f (S) = min f L (w). (13) S N w [0,1] n discrete continuous, submodular convex [Murota, 1998, Murota, 2003] has developed a theory of discrete convexity based on submodularity, in which many of the classic theorems of convexity find analogues. Donglei Du (UNB) Submodular 71 / 130

72 Proof of (13) Since f L is an extension of f, evidently, LHS RHS. For the other inequality, for any w [0, 1] n, sorted as 1 = w 0 w 1... w n w n+1 = 0. Then by (7) f L (w) = n f ([i])(w i w i+1 ) min f ([i]) min f (S), i=0,...,n S N i=0 implying RHS LHS. Donglei Du (UNB) Submodular 72 / 130

73 Algorithms for minimizing a submodular function Assume value oracle from now on: each value oracle call takes γ unit of time. This convex program (13) is solvable via Ellipsoid method in (weakly) polynomial time [Grötschel et al., 1981]. A strongly polynomial implementations of the above algorithm are also available [?, McCormick, 2005]. Donglei Du (UNB) Submodular 73 / 130

74 Combinatorial algorithm in strongly polynomial time Time complexity Credit O(n 8 γ + n 9 ) [Iwata et al., 2001] O(n 7 γ + n 8 ) [Schrijver, 2000, Vygen, 2003] [Fleischer and Iwata, 2000] O((n 6 γ + n 7 ) log n) [Iwata, 2003] O(n 5 γ + n 6 ) [Orlin, 2009] For symmetric function: finding a proper set minimizing a symmetric submodular function in O(n 3 ) value oracle calls [Queyranne, 1998]. The above is extended to posi-modular function (a more general condition than symmetric submodular) [Nagamochi and Ibaraki, 1998]: f (A) + f (B) f (A B) + f (B A), A, B N. Donglei Du (UNB) Submodular 74 / 130

75 Subsection 2 Maximization Donglei Du (UNB) Submodular 75 / 130

76 Submodular function maximization and Double Greedy Theorem [Buchbinder et al., 2012] There is a randomized 1 2-approximation for max f (S). S N Moreover this is the best possible, meaning that ɛ-approximation would require exponentially value oracle calls [Feige et al., 2011]. ( ɛ) -approximation for certain explicitly represented submodular function would imply NP = RP [Dobzinski and Vondrák, 2012]. Donglei Du (UNB) Submodular 76 / 130

77 The double-greedy algorithm of [Buchbinder et al., 2012] Initialization (A, B) = (, N). Main loop Choose an arbitrary order of the elements 1,..., n and process each element one by one in this selected order as follows: for each i: { A = A + i with prob. p := α + α + +β +, B = B i with prob. 1 p := β+ α + +β +, where Invariant: A B. α = f (A + i) f (A) = f i (A), β = f (B i) f (B) = f i (B i). Donglei Du (UNB) Submodular 77 / 130

78 An illustration of the algorithm Consider the order 1, 2, 3: A B 1,2,3 A 1 B 1,2,3 A B 2,3 A 1,2 B 1,2,3 A 1 B 1,3 A 2 B 2,3 A B 3 A 1,2,3 B 1,2,3 A 1,2 B 1,2 A 1,3 B 1,3 A 1 B 1 A 2,3 B 2,3 A 2 B 2 A 3 B 3 B Donglei Du (UNB) Submodular 78 / 130 A

79 A claim Claim 1. α + β 0 at any step of the algorithm. i B\A and A B, α + β = f (A + i) f (A) + f (B i) f (B) = (f (A + i) + f (B i)) (f (A) + f (B)) f ((A + i) (B i)) + f ((A + i) (B i)) (f (A) + f (B)) = (f (A) + f (B)) (f (A) + f (B)) = 0, where the inequality follows from the submodularity. Donglei Du (UNB) Submodular 79 / 130

80 Main idea of the proof and notations There are three cases: Case 1. α 0 and β 0; Case 2. α 0 and β 0; Case 3. α 0 and β 0. Let S be the optimal solution, define the evolving optimum: O := A (B S ) = (S A) B( since A B) Donglei Du (UNB) Submodular 80 / 130

81 Main idea of the proof and notations Define the potential function π at any step: π = E[f (A) + f (B) + 2f (O)] Initially: A =, B = N and O = S : At the end: A = B = O: π = E[f ( ) + f (N) + 2f (S )] 2OPT. π = E[f (A) + f (B) + 2f (O)] = 4E[ALG]. So the desired bound follows if π never decreases over the course of the algorithm. Namely, at any stage of the algorithm: π := π π 0 Donglei Du (UNB) Submodular 81 / 130

82 Case 1 Case 1. α 0 and β 0 = p = α α+0 = 1. So Ā : = A + i; B : = B; Ō : = O + i. Claim 2: f i (O) β if i / O. Proof: If i / O, then the definition of O implies that O B i. So by submodularity, f i (O) f i (B i) = β. By Claim 2, π = f i (A) + f (B) f (B) + 2f i (O) { α 0 if i O; = α + 2f i (O) α 2β 0 if i / O; Donglei Du (UNB) Submodular 82 / 130

83 Case 2 Case 2. α 0 and β 0 = 1 p = β 0+β = 1. So Ā : = A B : = B i Ō : = O i Claim 3: f i (O i) α if i O. Proof: Evidently, in this case i / A. If i O, then the definition of O implies that A O i. So by submodularity, f i (O i) f i (A) = α. By Claim 3, π = f (A) f (A) f i (B i) 2f i (O i) { β 0 if i / O; = β 2f i (O i) β 2α 0 if i O; Donglei Du (UNB) Submodular 83 / 130

84 Case 3 Case 3. α 0 and β 0 = p = with probability p, with probability 1 p, α α+β = 1. So Ā : = A + i; B : = B; Ō : = O + i. Ā : = A; B : = B i; Ō : = O i. From Cases 1 and 2 and Claim 2 and 3, π = p(α + 2f i (O)) + (1 p)(β 2f i (O i)) pα + (1 p)β 2αβ = (α β)2 α + β 0. Donglei Du (UNB) Submodular 84 / 130

85 Section 9 Constrained submodular optimization Donglei Du (UNB) Submodular 85 / 130

86 Constrained submodular optimization Minimization: some P, most NP-hard Maximization: still NP-hard Donglei Du (UNB) Submodular 86 / 130

87 Subsection 1 Minimization Donglei Du (UNB) Submodular 87 / 130

88 Minimization min f (S) S F P: Lattices, odd/even sets, T-odd/even sets NP-hard: min f (S): n-hardness even for monotone submodular function S k [Svitkina and Fleischer, 2011]. But min 0< S k<n f (S): 2-approx. for nonnegative symmetric submodular function [Dughmi, 2009]. min S is a shortest path f (S): n2/3 -hardness even for monotone submodular function [Goel et al., 2009] min f (S): Ω(n)-hardness even for monotone S is a spanning tree/perfect matching submodular function [Goel et al., 2009] min S is a vertex cover f (S): 2-approx. [Koufogiannakis and Young, 2009, Iwata and Nagano, 2009]: submodular vertex cover (Rounding). Donglei Du (UNB) Submodular 88 / 130

89 Minimization min f (S) S F NP-hard: min f (S): f -approx. S is a set cover [Koufogiannakis and Young, 2009, Iwata and Nagano, 2009]: submodular cost set cover (rounding or primal-dual). min j S [ j S c j : f (S) = f (N)] : H(max j f (j))-approx. for monotone submodular function [Wolsey, 1982, Fujito, 2000]: submodular constraint set cover problem submodular minimization subject to constraints with two variables per inequality: 2-approx. [Hochbaum, 2010] submodular multiway partition; (2 2/k)-approx. [Chekuri and Ene, 2011] Main technique: Greedy (dual fitting or primal-dual) Convex programming rounding via Lovász Extension Donglei Du (UNB) Submodular 89 / 130

90 Submodular vertex cover [Iwata and Nagano, 2009] For any nonnegative submodular function f, the submodular vertex cover problem: min [f (x) : x i + x j 1, (i, j) E, x {0, 1}] Convex relaxation [ ] min f L (x) : x i + x j 1, (i, j) E, x 0 A 2-approximation algorithm: Given any optimal fractional solution x, choose λ [0, 0.5] uniformly random and let S = {i : x i λ} Evidently S is a feasible vertex cover because x 1 + x 2 1 min{x 1, x 2 } 0.5 The expected value of this random vertex cover is E[f (S)] = f (S)dλ f (S)dλ = 2f L (x). Donglei Du (UNB) Submodular 90 / 130

91 Submodular set cover problem [Wolsey, 1982, Fujito, 2000] For any c 0, and a nonnegative increasing submodular function (assuming integer valued for simplicity): min c j : f (S) = f (N) (14) S N j S The above problem can be formulated as the following ILP [Wolsey, 1982] min c j x j s.t. j N j N S x j {0, 1}, j N, where F X (S) = f (S X ) f (S). f j (S)x j f N S (S), S N (15) Donglei Du (UNB) Submodular 91 / 130

92 Special cases minimum matroid base problem: here f (S) = r(s) The integer cover (IC) problem (with set cover as a special case): min c j x j : Ax b, x {0, 1} N, j N where A R m n and b R m have nonnegative integers. Define m f (S) = min a ij, b i, S N. i=1 j S f (S) = f (N) iff Ax S b, assuming that (IC) is feasible. f (S) is nonnegative, monotone and submodular. Donglei Du (UNB) Submodular 92 / 130

93 Special cases T 1... T n 1 a a 1n b 1 2 a a 2n b m a m1... a mn b m c 1... c n The set cover problem: A ground set M = {1,..., m} and a collection of subsets of M, N = {T 1,..., T n }, and a cost function c : N R +, find a minimum cost subcolletion of N that covers all elements of M. f (S) = j S T j, S N = {1,..., n}.. Donglei Du (UNB) Submodular 93 / 130

94 Why (14) and (15) equivalent? For any T N, let x T {0, 1} be the characteristic function of T. If f (T ) = f (N), j N S f j (S)x T j = j T S f j (S) f T S (S) = f (T ) f (S) = f (N) f (S) = f N S (S), where the inequality is from submodularity. If x T is feasible to the ILP, then consider the constraint corresponding to T : 0 = f j (S)x j f N T (T ) = f (N) f (T ), j N T where the inequality is from feasibility of x T. So f (T ) = F (N) due to monotonicity. Donglei Du (UNB) Submodular 94 / 130

95 Two greedy approximation algorithms: dual-infeasible and dual-feasible greedy The dual of the LP relaxation of ILP: max f N S (S) s.t. S N S;j / S f j (S)y S c j, y S 0, S N. j N This formulation is useful in both the performance analysis and algorithm design. Donglei Du (UNB) Submodular 95 / 130

96 The main idea of dual-infeasible greedy dual-infeasible greedy (a.k.a., dual fitting): Using the LP relaxation and dual, one shows that the primal integral solution found by the algorithm is at most the cost of the fractional infeasible-dual computed by the algorithm. Then shrink the dual fractional solution by a factor to make it dual-feasible. The approximation ratio is this shrinking factor. In the set cover case, it is the simple greedy. Donglei Du (UNB) Submodular 96 / 130

97 The main idea of dual-feasible greedy dual-feasible greedy (a.k.a., primal-dual): Start with a primal infeasible integral solution to the ILP, and a dual fractional feasible solution to the Dual LP relaxation. Alliteratively improve the feasibility of the primal solution integrally, and the optimality of the dual fractional solution, ensuring in the end a primal feasible integral solution is obtained and the cost of the last dual fractional solution is sued as lower bound on OPT. More details on dual-fitting and primal-dual as generic algorithm design techniques can be found in [Vazirani, 2001]. Donglei Du (UNB) Submodular 97 / 130

98 dual-infeasible greedy Theorem [Wolsey, 1982] Assuming f is nonnegative, increasing, submodular and integer-valued with f ( ) = 0, the following ( dual-infeasible ) greedy has an (harmonic) approximation ratio of H max f ({j}). j N Primal greedy Initialization : S = ; Main loop : if f (S) < f (N), S := S { arg min j N S c j f j (S) }. Donglei Du (UNB) Submodular 98 / 130

99 Special cases When applied to minimum weight matriod base problem, dual-infeasible greedy is just the standard greedy algorithm. The bound above reduces to H(max j N r({j})) = 1. When applied to set cover: f (S) = j S T j. The bound is H(max j N T j ) = H(m). Donglei Du (UNB) Submodular 99 / 130

100 dual-feasible greedy Theorem [Fujito, 2000] Assuming f is nonnegative, increasing, submodular and integer-valued with f ( ) = 0, the following dual-feasible greedy has an (frequency-type) approximation ratio of max (S,X ):S N,X N S and X is a minimal SSC in (N S,f ( /(N S)) j X f j(s) f (N) f (S). Donglei Du (UNB) Submodular 100 / 130

101 Special cases When applied to minimum weight matriod base problem, dual-feasible greedy also reduces to the standard greedy algorithm. The bound of 1 follows from the above theorem since in matroid, a minimal spanning set is always a maximum independent set. When applied to set cover: f (S) = j S T j. The bound is for any j X T j m set cover X, max i M {T j : i T j } = f max, where f max = max j=1,...,n f j is maximum frequency among all elements in N, namely f j is the number of sets that contains j. Donglei Du (UNB) Submodular 101 / 130

102 Dual greedy Dual greedy Initialization : S =, y = 0 and l = 0; Main loop : if f (S) < f (N), l l + 1. Increase y S until for some j / S the dual constraint corresponding to j becomes binding. Let c j f j (T )y T F F {j l } j l = arg min j N S c jl y S = T :j l / T S f jl (S) T :j / T S f j (S) f jl (T )y T Reverse deletion : For k = l,..., 1, F F {j k } as long as the resultant solution is feasible.. Donglei Du (UNB) Submodular 102 / 130

103 Subsection 2 Maximization Donglei Du (UNB) Submodular 103 / 130

104 Maximization max f (S) S F max[f (S) : S k] where f (S) is nonnegative and monotonic: Greedy algorithm: (1 1/e)-approx. [Nemhauser et al., 1978]. max[f (S) : w(s) k] where f (S) is nonnegative and monotonic: Greedy algorithm+partial enumeration: (1 1/e)-approx. [Sviridenko, 2004]. max[f (S) : w 1 (S) k 1,..., w d (S) k d ] where f (S) is nonnegative and monotonic vs nonmonotonic: multilinear extension: (1 1/e)-approx. vs 1/e-approx. (d is a constant) [Kulik et al., 2011] max[f (S) : S I] where f (S) is nonnegative and monotonic and I is the independence set of a matriod: Multilinear extension + Pipage rounding (1 1/e)-approx. [Calinescu et al., 2011] Donglei Du (UNB) Submodular 104 / 130

105 The greedy algorithm for max[f (S) : S k] [Nemhauser et al., 1978] Initialization S =. Main loop Choose an element i N S such that Let S = S + i. i = arg max f (S + i) f (S). i N S Donglei Du (UNB) Submodular 105 / 130

106 Proof sketch Let O be the optimal solution, then at any step when i is added: f (S + i) 1 ( k f (O) ) f (S) k Proof: O S = {i 1,..., i p }, p k. f (O) f (O S) (monotone) p = f (S) + [f (S {i 1,..., i j }) f (S {i 1,..., i j 1 })] f (S) + f (S) + j=1 p [f (S + i j ) f (S)] (submodularity) j=1 p [f (S + i) f (S)] (algorithm) j=1 = f (S) + p(f (S + i) f (S)) f (S) + k(f (S + i) f (S)). Donglei Du (UNB) Submodular 106 / 130

107 Proof sketch Let S t be the greedy solution after t iterations. S 0 = and S k = ALG. f (ALG) = f (S k ) 1 k f (O) + ( 1 1 ) f (S k 1 ) k 1 ( k f (O) k ( 1 k f (O) ) ( 1 k f (O) k + ( 1 1 k ) ( 1 1 ) ) f (S k 2 ) k = 1 k f (O)1 ( 1 1 ) k ( ( k 1 ( 1 1 ) = ) ) k f (O) k k ( 1 1 ) ( f (O) 1 x e x ). e ( 1 1 ) ) k 1 k Donglei Du (UNB) Submodular 107 / 130

108 ( 1 1 e ) -approx. for monotone submodular function maximization subject to matriod constraint Matriod base polytope P = {x(a) r(a), x(n) = r(n), A N, x {0, 1} n } along with its relaxation RP by replacing x {0, 1} n with x [0, 1] n. We are interested in the problem max f (S) S I Donglei Du (UNB) Submodular 108 / 130

109 Multilinear extension relaxation Claim: max S I f (S) = max x P f (x) = max F (x) = max F (x). x P x RP The last equality follows from pipage rounding (round any fractional solution without losing factor), implied by the cross-convexity of the multilinear extension when f is submodular. Donglei Du (UNB) Submodular 109 / 130

110 How to solve max x RP F (x)? Solve the relaxation approximately: Continuous greedy algorithm (up-concave) ( ) 1 1 e -approx. for monotone submodular function and solvable polytope [Vondrak, 2008, Calinescu et al., 2011]. 1 e -approx. for nonnegative submodular function and downward-closed solvable polytope [Feldman et al., 2011]. Donglei Du (UNB) Submodular 110 / 130

111 Section 10 Multi-criteria submodular maximization Donglei Du (UNB) Submodular 111 / 130

112 The k-criteria submodular function maximization problem [Du et al., 2012] The k-criteria submodular function maximization problem: max S 2 X {f 1(S),..., f k (S)} Here each f j : 2 X R + (j = 1,..., k) on ground set X is a (nonnegative) submodular function. Donglei Du (UNB) Submodular 112 / 130

113 Solution concept We distinguish two scenarios. Let Sj (j = 1,..., k) be a mono-criterion optimal solution for maximizing f j. Pure solution: any subset S 2 X. The subset S is an (α 1,..., α k )-pure solution if f j (S) α j f j (S j ), j = 1,..., k; Randomized solution (or mixed solution): any random subset set T 2 X. The random set T is an (α 1,..., α k )-randomized solution if E[f j (T )] α j f j (S j ), j = 1,..., k. Donglei Du (UNB) Submodular 113 / 130

114 Two fundamental questions to be answers characterize α 1,..., α k such that an (α 1,..., α k )-pure (or randomized) solution exist; how to find a good simultaneous approximation solution in polynomial time. Donglei Du (UNB) Submodular 114 / 130

115 Main results For symmetric submodular functions, there exists a ( 1 2, 2) 1 -pure solution, and this is best possible for k = 2 ; however, no simultaneously ( bounded ) pure solution exist for k 3; 2 there exists a k 1 2 k 1,..., 2k 1 -randomized solution for any k 2, 2 k 1 and this bound is best possible. For general submodular functions, no simultaneously ( bounded { pure } solution{ exist for}) k 2; 1 there exists a max k, 2k 2 1,..., max 2 k 1 k, 2k 2 -randomized 2 k 1 solution for any k 2, and this bound is best possible for k = 2 and k = 3. Donglei Du (UNB) Submodular 115 / 130

116 Conclusion Many topics not covered! More can be found in Books: [Fujishige, 2005, Murota, 2003, Schrijver, 2003] etc. Surveys: [Lovász, 1983a, McCormick, 2005, Dughmi, 2009] etc. You will be surprised how often submodularity pops up like mushrooms (or bamboo shoots in Chinese idioms) after the rain if you pay attention... Donglei Du (UNB) Submodular 116 / 130

117 References I Bach, F. (2010). Convex analysis and optimization with submodular functions: a tutorial. arxiv preprint arxiv: Buchbinder, N., Feldman, M., Naor, J. S., and Schwartz, R. (2012). A tight linear time (1/2)-approximation for unconstrained submodular maximization. In Foundations of Computer Science (FOCS), 2012 IEEE 53rd Annual Symposium on, pages IEEE. Calinescu, G., Chekuri, C., Pál, M., and Vondrák, J. (2011). Maximizing a monotone submodular function subject to a matroid constraint. SIAM Journal on Computing, 40(6): Donglei Du (UNB) Submodular 117 / 130

118 References II Chekuri, C. and Ene, A. (2011). Approximation algorithms for submodular multiway partition. In Foundations of Computer Science (FOCS), 2011 IEEE 52nd Annual Symposium on, pages IEEE. Choquet, G. (1954). Theory of capacities. In Annales de l institut Fourier, volume 5, pages Institut Fourier. Dobzinski, S. and Vondrák, J. (2012). From query complexity to computational complexity. In Proceedings of the 44th symposium on Theory of Computing, pages ACM. Donglei Du (UNB) Submodular 118 / 130

119 References III Du, D., Li, Y., Xiu, N., and Xu, D. (2012). Simultaneous Approximation of Multi-criteria Submodular Functions Maximization. Faculty of Business Administration, University of New Brunswick. Dughmi, S. (2009). Submodular functions: Extensions, distributions, and algorithms. a survey. arxiv preprint arxiv: Feige, U., Mirrokni, V. S., and Vondrak, J. (2011). Maximizing non-monotone submodular functions. SIAM Journal on Computing, 40(4): Donglei Du (UNB) Submodular 119 / 130

120 References IV Feldman, M., Naor, J. S., and Schwartz, R. (2011). Nonmonotone submodular maximization via a structural continuous greedy algorithm. In Automata, Languages and Programming, pages Springer. Fleischer, L. and Iwata, S. (2000). Improved algorithms for submodular function minimization and submodular flow. In Proceedings of the thirty-second annual ACM symposium on Theory of computing, pages ACM. Fujishige, S. (2005). Submodular functions and optimization, volume 58. Elsevier. Donglei Du (UNB) Submodular 120 / 130

121 References V Fujito, T. (2000). Approximation algorithms for submodular set cover with applications. IEICE Transactions on Information and Systems, 83(3): Goel, G., Karande, C., Tripathi, P., and Wang, L. (2009). Approximability of combinatorial problems with multi-agent submodular cost functions. In Foundations of Computer Science, FOCS th Annual IEEE Symposium on, pages IEEE. Grötschel, M., Lovász, L., and Schrijver, A. (1981). The ellipsoid method and its consequences in combinatorial optimization. Combinatorica, 1(2): Donglei Du (UNB) Submodular 121 / 130

122 References VI He, S., Zhang, J., and Zhang, S. (2012). Polymatroid optimization, submodularity, and joint replenishment games. Operations Research, 60(1): Hochbaum, D. S. (2010). Submodular problems-approximations and algorithms. arxiv preprint arxiv: Iwata, S. (2003). A faster scaling algorithm for minimizing submodular functions. SIAM Journal on Computing, 32(4): Iwata, S., Fleischer, L., and Fujishige, S. (2001). A combinatorial strongly polynomial algorithm for minimizing submodular functions. Journal of the ACM (JACM), 48(4): Donglei Du (UNB) Submodular 122 / 130

123 References VII Iwata, S. and Nagano, K. (2009). Submodular function minimization under covering constraints. In Foundations of Computer Science, FOCS th Annual IEEE Symposium on, pages IEEE. Kempe, D., Kleinberg, J., and Tardos, É. (2003). Maximizing the spread of influence through a social network. In Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining, pages ACM. Kempe, D., Kleinberg, J., and Tardos, É. (2005). Influential nodes in a diffusion model for social networks. In Automata, languages and programming, pages Springer. Donglei Du (UNB) Submodular 123 / 130

124 References VIII Koufogiannakis, C. and Young, N. E. (2009). Greedy {\ ensuremath {\ Delta}}-approximation algorithm for covering with arbitrary constraints and submodular cost. In Automata, Languages and Programming, pages Springer. Kulik, A., Shachnai, H., and Tamir, T. (2011). Approximations for monotone and non-monotone submodular maximization with knapsack constraints. arxiv preprint arxiv: Lovász, L. (1983a). Submodular functions and convexity. In Mathematical Programming The State of the Art, pages Springer. Donglei Du (UNB) Submodular 124 / 130

125 References IX Lovász, L. (1983b). Submodular functions and convexity. In Mathematical Programming The State of the Art, pages Springer. McCormick, S. T. (2005). Submodular function minimization. Handbooks in operations research and management science, 12: Murota, K. (1998). Discrete convex analysis. Mathematical Programming, 83(1-3): Murota, K. (2003). Discrete convex analysis. Number 10. SIAM. Donglei Du (UNB) Submodular 125 / 130

126 References X Nagamochi, H. and Ibaraki, T. (1998). A note on minimizing submodular functions. Information Processing Letters, 67(5): Nemhauser, G. L., Wolsey, L. A., and Fisher, M. L. (1978). An analysis of approximations for maximizing submodular set functionsi. Mathematical Programming, 14(1): Orlin, J. B. (2009). A faster strongly polynomial time algorithm for submodular function minimization. Mathematical Programming, 118(2): Queyranne, M. (1993). Structure of a simple scheduling polyhedron. Mathematical Programming, 58(1-3): Donglei Du (UNB) Submodular 126 / 130

127 References XI Queyranne, M. (1998). Minimizing symmetric submodular functions. Mathematical Programming, 82(1-2):3 12. Queyranne, M. and Schulz, A. S. (1994). Polyhedral approaches to machine scheduling. Citeseer. Schrijver, A. (2000). A combinatorial algorithm minimizing submodular functions in strongly polynomial time. Journal of Combinatorial Theory, Series B, 80(2): Schrijver, A. (2003). Combinatorial optimization: polyhedra and efficiency, volume 24. Springer Verlag. Donglei Du (UNB) Submodular 127 / 130

128 References XII Schulz, A. S. and Uhan, N. A. (2010). Sharing supermodular costs. Operations research, 58(4-Part-2): Sviridenko, M. (2004). A note on maximizing a submodular set function subject to a knapsack constraint. Operations Research Letters, 32(1): Svitkina, Z. and Fleischer, L. (2011). Submodular approximation: Sampling-based algorithms and lower bounds. SIAM Journal on Computing, 40(6): Vazirani, V. V. (2001). Approximation algorithms. springer. Donglei Du (UNB) Submodular 128 / 130

129 References XIII Vondrak, J. (2008). Optimal approximation for the submodular welfare problem in the value oracle model. In Proceedings of the 40th annual ACM symposium on Theory of computing, pages ACM. Vondrák, J. (2013). Symmetry and approximability of submodular maximization problems. SIAM Journal on Computing, 42(1): Vygen, J. (2003). A note on schrijver s submodular function minimization algorithm. Journal of Combinatorial Theory, Series B, 88(2): Donglei Du (UNB) Submodular 129 / 130

130 References XIV Wolsey, L. A. (1982). An analysis of the greedy algorithm for the submodular set covering problem. Combinatorica, 2(4): Donglei Du (UNB) Submodular 130 / 130