Submodular Maximization Over Multiple Matroids via Generalized Exchange Properties

Transcription

1 Submodular Maxmzaton Over Multle Matrods va Generalzed Exchange Proertes Jon Lee Maxm Svrdenko Jan Vondrák June 25, 2009 Abstract Submodular-functon maxmzaton s a central roblem n combnatoral otmzaton, generalzng many mortant NP-hard roblems ncludng Max Cut n dgrahs, grahs and hyergrahs, certan constrant satsfacton roblems, maxmum-entroy samlng, and maxmum faclty-locaton roblems. Our man result s that for any k 2 and any ε > 0, there s a natural local-search algorthm whch has aroxmaton guarantee of 1/k + ε) for the roblem of maxmzng a monotone submodular functon subject to k matrod constrants. Ths mroves a 1/k + 1)-aroxmaton of Nemhauser, Wolsey and Fsher, obtaned more than 30 years ago. Also, our analyss can be aled to the roblem of maxmzng a lnear objectve functon and even a general non-monotone submodular functon subject to k matrod constrants. We show that n these cases the aroxmaton guarantees of our algorthms are 1/k 1 + ε) and 1/k /k + ε), resectvely. Introducton In ths aer, we consder the roblem of maxmzng a non-negatve submodular functon f, defned on a fnte) ground set N, subject to matrod constrants. A functon f : 2 N R s submodular f for all S, T N, fs T ) + fs T ) fs) + ft ). Throughout, we assume that our submodular functon f s gven by a value oracle;.e., for a gven set S N, an algorthm can query an oracle to fnd the value fs). Furthermore, all submodular functons that we deal wth are assumed to be non-negatve. Wthout loss of generalty, we take the ground set N to be [n] := {1, 2,, n}. We assume some famlarty wth matrods [16] and assocated algorthmcs [18]. Brefly, we denote a matrod M by an ordered ar N, I), where N s the ground set of M and I s the set of ndeendent sets of M. For a gven matrod M, the assocated matrod constrant s S IM). In our usage, we deal wth k matrods M = N, I ), = 1,..., k, on the common ground set N. IBM T.J. Watson Research Center, jonlee@us.bm.com IBM T.J. Watson Research Center, svr@us.bm.com Prnceton Unversty, jvondrak@math.rnceton.edu 1

2 It s no concdence that we use N for the ground set of our submodular functon f as well as for the ground set of our matrods M = N, I ), = 1,..., k. Indeed, our otmzaton roblem s } max {fs) : S k =1I. Where necessary, we make some use of other standard matrod notaton: For a matrod M = N, I), we denote ts rank functon by r M and ts dual by M. A base of M s a J I havng cardnalty r M N). For a set S N, we let M\S, M/S, and M S denote deleton of S, contracton of S, and restrcton to S, resectvely. We recall: r M S) := max { J : J S, J I}. M has ground set N and ndeendent sets {X N : N \ X contans a base of M} M\S M delete S) has ground set N \ S and ndeendent sets {X N \ S : X I} Let J be a maxmal subset of S that s ndeendent n M Then M/S M contract S) has ground set N \ S and ndeendent sets {X N \ S : X J I} M S M restrcted to S) s smly M\N/ \ S) M delete N \ S) Prevous Results. Otmzng submodular functons s a central toc n otmzaton [12]. Whle submodular mnmzaton s a olynomally solvable roblem [10, 19], the maxmzaton varants are usually NP-hard because they nclude ether Max Cut, some varant of faclty locaton, or set ackng roblems. There are essentally two algorthmc technques that work for submodular maxmzaton. The greedy algorthm was frst aled to a wde range of submodular maxmzaton roblems n the late-70 s and early-80 s [3, 4, 5, 13, 14]. The most relevant result for our uroses s the roof that the greedy algorthm has aroxmaton guarantee of 1/k + 1) for the roblem of maxmzng a monotone submodular functon subject to k matrod constrants [14]. Untl recently, ths algorthm had the best establshed erformance guarantee for the roblem wth general matrod constrants. Recently, Vondrák [20] desgned the contnuous greedy algorthm that acheves a 1 1/e)-aroxmaton for the roblem wth k = 1,.e. subject to a sngle matrod constrant. Ths result s otmal n the oracle model even for the case of a unform matrod constrant [15], and also otmal unless P = NP for the secal case of the maxmum coverage roblem [6]. The second algorthmc technque that works for submodular maxmzaton s local search. Cornuéjols et al. [4] show that a local-search algorthm acheves a constant-factor aroxmaton guarantee for the maxmum uncaactated faclty-locaton roblem whch s a secal case of submodular maxmzaton. Note that n ths case the greedy aroach has a better erformance guarantee. The maxmum k-dmensonal matchng roblem s a roblem of maxmzng a lnear objectve functon subject to k secal artton matrod constrants. The aroxmaton guarantees for maxmum k-dmensonal matchng are establshed wth local-search algorthms and have erformance guarantees 2/k + ε) for lnear functon wth {0, 1}-coeffcents, and 2/k ε) for a general lnear functon, even n the 2

3 more general settngs of set ackng [9] and ndeendent set roblems n k + 1)-claw free grahs [1]. Unfortunately, general matrod constrants seem to comlcate the matter; the best aroxmaton guarantee for the roblem of maxmzng a lnear functon subject to k general-matrod constrants s 1/k [17]. The result of [14] can be mroved n the case when all k constrants corresond to artton matrods. A smle local-search algorthm has aroxmaton guarantee of 1/k + ε) [11] for ths varant of the roblem. The analyss strongly uses the roertes of artton matrods. It s based on the realtvely smle augmentng-aths exchange roertes of artton matrods that do not hold n general. Local-search algorthms were also desgned for non-monotone submodular maxmzaton [7, 11]. The best aroxmaton guarantee known for unconstraned submodular maxmzaton s 2/5 ε [7]. For the roblem wth k general matrods the best known aroxmaton s 1/k /k + ε) [11]. Our Results and Technques. In ths aer we analyze a natural local-search algorthm: Gven a feasble soluton,.e. a set S that s ndeendent n each of the k matrods, our local-search algorthm tres to add at most elements and delete at most k elements from S. If there s such a local move that generates a feasble soluton that mroves the objectve value, our algorthm reeats the local-search rocedure wth that new soluton, untl no mrovement s ossble. Our man result s that for k 2, every locally-otmal feasble soluton S satsfes the nequalty k + 1 ) fs) fs C) + k ) fs C), for every feasble soluton C. We also rovde an aroxmate varant of the local-search rocedure that fnds an aroxmate locally-otmal soluton n olynomal tme, whle losng a factor of 1 + ε n the left-hand sde of the above nequalty Lemma 3.3) for arbtrary ε > 0. Therefore, we obtan a olynomal-tme local-search algorthm wth aroxmaton guarantee 1/k + ε) for the roblem of maxmzng a monotone ncreasng submodular functon subject to k matrod constrants. Ths algorthm gves a 1/k 1 + ε)-aroxmaton n the case when the objectve functon s lnear. The man techncal contrbutons of ths aer are two new exchange roertes for matrods. One s a generalzaton of the classcal Rota Exchange Proerty Lemma 2.8) and another s a generalzaton of the exchange roerty for artton matrods based on augmentng aths Lemma 2.4). We beleve that both roertes and roofs are nterestng n ther own rght. In the case of a general non-monotone submodular objectve functons, one round of local search s not enough, but alyng the local search teratvely, as n [11], one can obtan an aroxmaton algorthm wth erformance guarantee of 1/k +1+1/k 1) + ε). In 1, we establsh some useful roertes concernng exact k-covers and submodular functons. In 2, we establsh some exchange roertes for matrods. In 3, we descrbe and analyze our local-search algorthm. 3

4 1 Some Useful Proertes of Submodular Functons Lemma 1.1. Let f be a non-negatve submodular functon on N. Let S S N, and let {T l } t l=1 be a collecton of subsets of S \ S such that each element of S \ S aears n exactly k of these subsets. Then t [fs) f S \ T l )] k fs) fs ) ). l=1 Proof. Let s = S and c = S. Wthout loss of generalty, we can assume that S = {1, 2,, c} = [c] and S = {1, 2,, s} = [s] Then for any T S \ S, by submodularty: fs) fs \ T ) [f[]) f[ 1])]. Usng ths we obtan T t [fs) f S \ T l )] l=1 t [f[]) f[ 1])] l=1 T l s = k [f[]) f[ 1])] =c+1 = k fs) fs ) ). The second equalty follows from S \ C = {c + 1,, s} and the fact that each element of S \ C aears n exactly k of the sets {T l } t l=1. The last equalty s due to a telescong summaton. Lemma 1.2. Let f be a non-negatve submodular functon on N. Let S N and C N, and let {T l } t l=1 be a collecton of subsets of C \ S such that each element of C \ S aears n exactly k of these subsets. Then t [fs T l ) f S)] k fs C) fs)). l=1 Proof. Let s = S and c = C \ S. Wthout loss of generalty, we can assume that S = {1, 2,, s} and that C \ S = {s + 1, 2,, c}. Then for any T l C \ S, by submodularty: fs T l ) fs) T l [fs {}) fs)]. Usng ths we obtan t [fs T l ) f S)] l=1 t [fs {}) f S)] l=1 T l = k [fs {}) f S)] C\S k [fs C) f S)]. The last nequalty follows from submodularty. 4

5 2 New Exchange Proertes of Matrods 2.1 Two matrods An exchange dgrah s a well-known construct for devsng effcent algorthms for exact maxmzaton of modular functons on a ar of matrods on a common ground set for examle, see [18]). We are nterested n submodular maxmzaton, k matrods and aroxmaton algorthms; nevertheless, we are able to make use of such exchange dgrahs, once we establsh some new roertes of them. Let M 1 = N, I 1 ) and M 2 = N, I 2 ) be a ar of matrods on ground set N. For I I 1 I 2, we defne a ar of dgrahs D M1 I) and D M2 I) on node set N as follows: For each I, j N \ I such that I + j I 1, we have an arc, j) of D M1 I); For each I, j N \ I such that I + j I 2, we have an arc j, ) of D M2 I). The arcs n D M1 I) encode vald swas n M 1, and the arcs n D M2 I) encode vald swas n M 2. In what follows, we assume that I s our current soluton and J s the otmal soluton. We also assume that I = J. If not, we extend I or J by dummy elements so that we mantan ndeendence n both matrods more detals later). We use two known lemmas from matrod theory. Lemma 2.1 [18, Corollary 39.12a]). If I = J and I, J I = 1 or 2), then D M I) contans a erfect matchng between I \ J and J \ I. Lemma 2.2 [18, Theorem 39.13]). Let I = J, I I, and assume that D M I) has a unque erfect matchng between I \ J and J \ I. Then J I. Next, we defne a dgrah D M1,M 2 I) on node set N as the unon of D M1 I) and D M2 I). A dcycle n D M1,M 2 I) corresonds to a chan of feasble swas. However, observe that t s not necessarly the case that the entre cycle gves a vald exchange n both matrods. If I = J and I, J I 1 I 2, ths means we have two erfect matchngs on I J whch together form a collecton of dcycles n D M1,M 2 I). However, only the unqueness of a erfect matchng assures us that we can legally erform the exchange. Ths motvates the followng defnton. Defnton 2.3. We call a dcycle C n D M1,M 2 I) rreducble f C D M1 I) s the unque erfect matchng n D M1 I) and C D M2 I) s the unque erfect matchng n D M2 I), coverng the vertex set V C). Otherwse, we call C reducble. The followng, whch s our man techncal lemma, allows us to consder only rreducble cycles. The roof follows closely the deas of matrod ntersecton see [18, Lemma 41.5α]). 5

6 Lemma 2.4. Let M 1 = N, I 1 ) and M 2 = N, I 2 ) be two matrods on ground set N. Suose that I, J I 1 I 2 and I = J. Then there s s 0 and a collecton of rreducble dcycles {C 1,..., C m } ossbly wth reetton) n D M1,M 2 I), usng only elements of I J, so that each element of I J aears n exactly 2 s of the dcycles. Proof. Consder D M1,M 2 I) = D M1 I) D M2 I). By Lemma 2.1, there exsts a erfect matchng between I \ J and J \ I, both n D M1 I) and D M2 I). Let us denote these two erfect matchngs by M 1, M 2. The unon M 1 M 2 forms a subgrah of out-degree 1 and n-degree 1 on I J. Therefore, t decomoses nto a collecton of dcycles C 1,..., C m. If they are all rreducble, we are done. If C s not rreducble, t means that ether M 1 = C D M1 I) or M 2 = C D M2 I) s not a unque erfect matchng on V C ). Let us assume, wthout loss of generalty, that there s another erfect matchng M 1 n D M 1 I). We consder the dsjont unon M 1 + M 1 + M 2 + M 2, dulcatng arcs where necessary. Ths s a subgrah of out-degree 2 and n-degree 2 on V C ), whch decomoses nto dcycles C 1,..., C t, coverng each vertex of C exactly twce: V C 1 ) + V C 2 ) V C t ) = 2V C ). Because M 1 M 1, we have a chord of C n M 1, and we can choose the frst dcycle so that t does not cover all of V C ). So we can assume that we have t 3 dcycles, and at most one of them covers all of V C ). If there s such a dcycle among C 1,..., C t, we remove t and dulcate the remanng dcycles. Ether way, we end u wth a collecton of dcycles C 1,..., C t such that each of them s shorter than C and together they cover each vertex of C exactly twce. We reeat ths rocedure for each reducble dcycle C. For rreducble dcycles C, we just dulcate C to obtan C 1 = C 2 = C. Ths comletes one stage of our rocedure. After the comleton of the frst stage, we have a collecton of dcycles {C j } coverng each vertex n I J exactly twce. As long as there exsts a reducble dcycle n our current collecton of dcycles, we erform another stage of our rocedure. Ths means decomosng all reducble dcycles and dulcatng all rreducble dcycles. In each stage, we double the number of dcycles coverng each element of I J. To see that ths cannot be reeated ndefntely, observe that every stage decreases the sze of the longest reducble dcycle. All dcycles of length 2 are rreducble, and therefore the rocedure termnates after a fnte number of stages s. Then, all cycles are rreducble and together they cover each element of I J exactly 2 s tmes. We remark that of course the rocedure n the roof of Lemma 2.4 s very neffcent, but t s not art of our algorthm t s only used for ths roof. Next, we extend ths Lemma 2.4 to sets I, J of dfferent sze, whch forces us to deal wth daths as well as dcycles. Defnton 2.5. We call a dath or dcycle A feasble n D M1,M 2 I), f I V A) I 1 I 2, and 6

7 For any sub-dath A A such that each endont of A s ether an endont of A or an element of I, we also have I V A ) I 1 I 2. Frst, we establsh that rreducble dcycles are feasble. Lemma 2.6. Any rreducble dcycle n D M1,M 2 I) s also feasble n D M1,M 2 I). Proof. An rreducble dcycle C conssts of two matchngs M 1 M 2, whch are the unque erfect matchngs on V C), n D M1 I) and D M2 I) resectvely. Therefore, we have I V C) I 1 I 2 by Lemma 2.2. Consder any sub-dath A C whose endonts are n I. C has no endonts, so the other case n Defnton 2.5 does not aly.) Ths means that A has even length. Suose that a 1 V A ) s the endont ncdent to an edge n M 1 A and a 2 V A ) s the other endont, ncdent to an edge n M 2 A. Note that any subset of M 1 or M 2 s agan a unque erfect matchng on ts resectve vertex set, because otherwse we could roduce a dfferent erfect matchng on V C). We can vew I V A ) n two ossble ways: I V A ) = I a 1 ) V A ) a 1 ); because V A ) a 1 has a unque erfect matchng M 2 A n D M2 I), ths shows that I V A ) I 2. I V A ) = I a 2 ) V A ) a 2 ); because V A ) a 2 has a unque erfect matchng M 1 A n D M1 I), ths shows that I V A ) I 1. Fnally, we establsh the followng roerty of ossble exchanges between arbtrary solutons I, J not necessarly of the same sze). Lemma 2.7. Let M 1 = N, I 1 ) and M 2 = N, I 2 ) be two matrods and let I, J I 1 I 2. Then there s s 0 and a collecton of daths/dcycles {A 1,..., A m } ossbly wth reetton), feasble n D M1,M 2 I), usng only elements of I J, so that each element of I J aears n exactly 2 s daths/dcycles A. Proof. If I = J, we are done by Lemmas 2.4 and 2.6. If I = J, we extend the matrods by new dummy elements E, ndeendent of everythng else n both matrods), and add them to I or J, to obtan sets of equal sze Ĩ = J. We denote the extended matrods by M 1 = N E, Ĩ1), M2 = N E, Ĩ2). We consder the grah D M1, M 2 Ĩ). Observe that the dummy elements do not affect ndeendence among other elements, so the grahs D M1,M 2 I) and D M1, M 2 Ĩ) are dentcal on I J. Alyng Lemma 2.4 to Ĩ, J, we obtan a collecton of rreducble dcycles {C 1,..., C m } on Ĩ J such that each element aears n exactly 2 s dcycles. Let A = C \ E. Obvously, the sets V A ) cover I J exactly 2 s tmes. We clam that each A s ether a feasble dcycle, a feasble dath, or a collecton of feasble daths n the orgnal dgrah D M1,M 2 I)). Frst, assume that C E =. Then A = C s an rreducble cycle n D M1,M 2 I) the dummy elements are rrelevant). By Lemma 2.6, we know that A = C s a feasble dcycle. Next, assume that C E. C s stll a feasble dcycle, but n the extended dgrah D M1, M Ĩ). We remove the dummy elements from C 2 to obtan A = C \ E, a dath or 7

8 a collecton of daths. Consder any sub-dath A of A, ossbly A = A, satsfyng the assumtons of Defnton 2.5. A does not contan any dummy elements. If both endonts of A are n I, t follows from the feasblty of C that Ĩ V A ) Ĩ1 Ĩ2, and hence I V A ) = Ĩ V A )) \ E I 1 I 2. If an endont of A s outsde of I, then t must be an endont of A. Ths means that t has a dummy neghbor n Ĩ C that we deleted. Note that ths case can occur only f we added dummy elements to I,.e. I < J.) In that case, extend the ath to A, by addng the dummy neghbors) at ether end. We obtan a dath from Ĩ to Ĩ. By the feasblty of C, we have Ĩ V A ) Ĩ1 Ĩ2, and therefore I V A ) = Ĩ V A )) \ E I 1 I A generalzed Rota-exchange roerty Next, we establsh a very useful exchange roerty for a ar of bases of a sngle matrod. Lemma 2.8. Let M = N, I) be a matrod and A, B bases n M. Let A 1,..., A m be subsets of A such that each element of A aears n exactly q of them. Then there are sets B 1,..., B m B such that each element of B aears n exactly q of them, and for each A B \ B ) I. Remark 2.9. A very secal case of Lemma 2.8, namely when m = 2 and q = 1, attracted sgnfcant nterest when t was conjectured by G.-C. Rota and roved n [2, 8, 21]; see [18, 39.58)]. Proof. Lemma 2.8) We can assume for convenence that A and B are dsjont otherwse we can make {B } equal to {A } on the ntersecton A B and contnue wth a matrod where A B s contracted). For each, we defne a matrod N = M/A ) B, where we contract A and restrct to B. In other words, S B s ndeendent n N exactly when A S I. The rank functon of N s r N S) = r M A S) r M A ) = r M/A S). Let N be the dual matrod to N. Recall that the ground set s now B. By defnton, T B s a sannng set n N f and only f B\T s ndeendent n N,.e. f A B\T ) I. The bases of N are mnmal such sets T ; these are the canddate sets for B, whch can be exchanged for A. The rank functon of the dual matrod N s by [18, Theorem 39.3)]) r N T ) = T r N B) + r N B \ T ) = T r M A B) + r M A B \ T )) = T B + r M A B \ T )) = r M/B\T ) A ). Observe that the rank of N s r N B) = A. 8

9 Now, let us consder a new ground set ˆB = B [q]. We vew the elements {, j) : j [q]} as arallel coes of. For a set T ˆB, we defne ts rojecton to B as πt ) = { B j [q] wth, j) T }. A natural extenson of N to ˆB s a matrod ˆN where a set T s ndeendent f πt ) s ndeendent n N. The rank functon of ˆN s r ˆN T ) = r N πt )) = r M/B\πT )) A ). 1) The queston now s whether ˆB can be arttoned nto B 1,..., B m so that B s a base n ˆN. If ths s true, then we are done, because each B = πb ) would be a base of N and each element of B would aear n q sets B. To rove ths, consder the unon of our matrods, ˆN := ˆN 1 ˆN 2... ˆN m. By the matrod unon theorem [18, Corollary 42.1a)]), ts rank functon s ) r ˆN ˆB) = ˆB m \ T + r ˆN T ). We clam that for any T ˆB, m r ˆN T ) = =1 mn T ˆB =1 m r M/B\πT )) A ) q r M/B\πT )) A) = q πt ). 2) =1 The frst equalty follows from our rank formula 1), and the last equalty holds because both A and B are bases of M. To rove the nequalty, wthout loss of generalty, assume that A = {1, 2,..., n} = [n], and let gs) := r M/B\πT )) S), whch s a monotone submodular functon wth g ) = 0. We have m ga ) = =1 m =1 [ga [j]) ga [j 1])] j A m [g[j]) g[j 1])] j A =1 = q j A[g[j]) g[j 1])] = q fa), usng submodularty and the fact that each element of A aears n exactly q sets A. Ths roves 2), and we get m =1 r ˆN T ) q πt ) T for any T ˆB. We conclude that the rank functon of ˆN s ) ˆr ˆB) = mn T ˆB ˆB \ T + 9 m r ˆN T ) =1 = ˆB.

10 Ths means that ˆB can be arttoned nto sets B 1,..., B m where B s ndeendent n ˆN. However, the ranks of ˆB n the ˆN sum u to m =1 r ˆN ˆB) = m =1 A = ˆB, so the only way ths can be ossble s f each B s a base of ˆN. Then, each B = πb ) s a base of N and these are the sets demanded by the lemma. Fnally, we resent a verson of Lemma 2.8 where the two sets are not necessarly bases. Lemma Let M = N, I) be a matrod and I, J I. Let I 1,..., I m be subsets of I such that each element of I aears n at most q of them. Then there are sets J 1,..., J m J such that each element of J aears n at most q of them, and for each I J \ J ) I. Proof. We reduce ths statement to Lemma 2.8. Let A, B be bases such that I A and J B. Also, we extend I arbtrarly to A, I A A, so that each element of A aears n exactly q of them. By Lemma 2.8, there are sets B B such that each element of B aears n exactly q of them, and A B \ B ) I for each. We defne J = J B. Then, each element of J aears n at most q sets J, and 3 Local-Search Algorthm I J \ J ) A B \ B ) I. As s usual, our local-search algorthm works n teratons. At each teraton, gven a current feasble soluton S k j=1 I j our algorthm looks for an mroved soluton by lookng at a olynomal number of otons to change S. If the algorthm fnds a better soluton t moves to the next teraton, otherwse the algorthm stos. Secfcally, gven a current soluton S k j=1 I j, the local moves that we consder are: -exchange oeraton: If there s S such that ) S \ S, S \ S k, and ) fs ) > fs), then S S. The -exchange oeraton for S S s called a delete oeraton. Our man result s the followng lower bound on the value of the locally-otmal soluton. Lemma 3.1. For every k 2 and every C k j=1 I j, a locally-otmal soluton S under -exchanges, satsfes k + 1 ) fs) fs C) + k ) fs C). 10

11 Proof. Our roof s based on the new exchange roertes of matrods: Lemmas 2.7 and By alyng Lemma 2.7 to the ndeendent sets C and S n matrods M 1 and M 2, we obtan a collecton of daths/dcycles {A 1,..., A m } ossbly wth reetton), feasble n D M1,M 2 S), usng only elements of C S, so that each element of C S aears n exactly 2 s aths/cycles A. We would lke to defne the sets of vertces corresondng to the exchanges n our localsearch algorthm, based on the sets of vertces n aths/cycles {A 1,..., A m }. The roblem s that these aths/cycles can be much longer than the maxmal cardnalty of a set allowable n a -exchange oeraton. To handle ths, we ndex vertces of the set of C \ S n each ath/cycle A for = 1,..., m, n such a way that vertces along any ath or cycle are numbered consecutvely. The vertces of S\C reman unlabeled. Because one vertex aears n 2 s aths/cycles, t mght get dfferent labels corresondng to dfferent aearances of that vertex. So one vertex could have u to 2 s dfferent labels. We also defne +1 coes of the ndex sets {A 1,..., A m }. For each coy q = 0,, of labeled {A 1,..., A m }, we throw away aearances of vertces from C \ S that were labeled by q modulo +1 from each A. By throwng away some aearances of the vertces, we are changng our set of aths n each coy of the orgnal sets {A 1,..., A m }. Let {A q1,..., A qmq } be the resultng collecton of aths for q = 0,...,. Now each ath A q contans at most vertces from C \ S and at most + 1 vertces from S \ C. Because our orgnal collecton of aths/cycles was feasble n D M1,M 2 S) see defnton 2.5), each of the aths n the new collectons corresond to feasble exchanges for matrods M 1 and M 2,.e. S V A q ) I 1 I 2. Consder now the collecton of aths {A q q = 0,...,, = 1,..., m q }. By constructon, each element of the set S \ C aears n exactly +1)2 s aths, and each element of C \S aears n exactly 2 s aths, because each vertex has 2 s + 1) aearances n total, and each aearance s thrown away exactly once. Let L q = S V A q ) denote the set of vertces n the ath A q belongng to the locally-otmal soluton S, and let W q = C V A q ) denote the set of vertces n the ath A q belongng to the set C. For each matrod M for = 3,..., k, ndeendent sets S I and C I, and collecton of sets {W q q = 0,..., ; = 1,..., m q } note that some of these sets mght be emty), we aly Lemma For convenence, we re-ndex the collecton of sets {W q q = 0,...,, = 1,..., m q }. Let W 1,..., W t be that collecton, after re-ndexng, for t = q=0 m q. By Lemma 2.10, for each = 3,..., k there exst a collecton of sets X 1,..., X t such that W j S \ X j ) I. Moreover, each element of S aears n at most 2 s of the sets from collecton X 1,..., X t. We consder the set of -exchanges that corresond to addng the elements of the set W j to the set S and removng the set of elements Λ j = L j k =3 X j ) for j = 1,..., t. Note that, Λ j + 1) + k 2) = k 1) + 1 k. By Lemmas 2.7 and 2.10, the sets W j S \ Λ j ) are ndeendent n each of the matrods M 1,, M k. By the fact that S s a locally-otmal 11

12 soluton, we have fs) f S \ Λ j ) Wj ), j = 1,..., t. 3) Usng nequaltes 3) together wth submodularty for j = 1,..., t, we have fs W j ) fs) f S \ Λ j ) W j ) f S \ Λ j ) fs) f S \ Λ j ). 4) Moreover, we know that each element of the set C \ S aears n exactly 2 s sets W j, and each element e S \ C aears n n e + 1)2 s + k 2)2 s sets Λ j. Consder the sum of t nequaltes 4), and add + 1)2 s + k 2)2 s n e nequaltes fs) fs \ {e}) 5) for each element e S \ C. These nequaltes corresond to the delete oeratons. We obtan t [fs W j ) fs)] j=1 e S\C t [fs) f S \ Λ j )] + j=1 + 1)2 s + k 2)2 s n e ) [fs \ {e}) fs)]. 6) Alyng Lemma 1.1 to the rght-hand sde of the nequalty 6) and Lemma 1.2 to the left-hand sde of the nequalty 6), we have 2 s [fs C) fs)] + 1)2 s + k 2)2 s ) [fs) fs C)], whch s equvalent to k + 1 ) fs) fs C) + k ) fs C). The result follows. A smle consequence of Lemma 3.1 mles bounds on the value of the locally-otmal solutons n the cases when the submodular functon f has addtonal structure. Corollary 3.2. For k 2, a locally-otmal soluton S, and any C k j=1 I j, the followng nequaltes hold: ) 1. fs) fc)/ k + 1 f functon f s monotone, ) 2. fs) fc)/ k f functon f s lnear. 12

13 Inut: Fnte ground set N := [n], value-oracle access to submodular functon f : 2 N R, and matrods M = N, I ), for [k]. 1. Set v arg max{fu) u N} and S {v}. 2. Whle the followng local oeraton s ossble, udate S accordngly: Outut: S. -exchange oeraton. If there s S such that ) S \ S, S \ S k, and ) fs ) 1 + ɛ n 4 )fs), then S S. Fgure 1: The aroxmate local-search rocedure. The local-search algorthm defned n the begnnng of ths secton could run for an exonental amount of tme untl t reaches a locally-otmal soluton. To ensure olynomal runtme, we follow the standard aroach of an aroxmate local search under a sutable small) arameter ɛ > 0, as descrbed n Fgure 1. The followng Lemma 3.3 s a smle extenson of Lemma 3.1 for an aroxmate local otmum. Lemma 3.3. For an aroxmate locally-otmal soluton S and any C k j=1 I j, 1 + ɛ) k + 1 ) fs) fs C) + k ) fs C), where ɛ > 0 s the arameter used n the rocedure of Fgure 1. Proof. The roof of ths lemma s almost dentcal to the roof of the Lemma 3.1 the only dfference s that left-hand sdes of nequaltes 3) and nequaltes 5) are multled by 1 + ɛ. Therefore, after followng the stes n the roof of Lemma 3.1, we obtan the n 4 nequalty: k ɛ ) n 4 λ 2 s fs) fs C) + k ) fs C), where λ = t + e S\C [ + 1)2s + k 2)2 s n e ] s the total number of nequaltes 3) and 5). because t C 2 s we obtan that λ n + k)2 s. Assumng that n 4 >> n + k, we obtan the result. Lemma 3.3 mles the followng: 13

14 Theorem 3.4. For any k 2 and fxed constant δ > 0, there exsts a 1 k+δ -aroxmaton algorthm for maxmzng a non-negatve non-decreasng submodular functon subject to k 1 matrod constrants. Ths bound mroves to k 1+δ for lnear functons. Remark 3.5. Combnng the technques from ths aer and the teratve local-search technque from [11], we can mrove the erformance guarantees of the aroxmaton algorthms for maxmzng a general non-monotone) submodular functon subject to k 2 matrod constrants from k k + δ to k k 1 + δ for any δ > 0. References [1] P. Berman, A d/2 aroxmaton for maxmum weght ndeendent set n d-claw free grahs, Nordc J. Comut ), no. 3, [2] T.H. Brylawsk. Some roertes of basc famles of subsets, Dscrete Mathematcs ), [3] M. Confort and G. Cornuéjols. Submodular set functons, matrods and the greedy algorthm: tght worst-case bounds and some generalzatons of the Rado-Edmonds theorem, Dscrete Al. Math ), no. 3, [4] G. Cornuéjols, M. Fscher and G. Nemhauser. Locaton of bank accounts to otmze oat: An analytc study of exact and aroxmaton algorthms, Management Scence, ), [5] G. Cornuéjols, M. Fscher and G. Nemhauser. On the uncaactated locaton roblem, Annals of Dscrete Math ), [6] U. Fege. A threshold of ln n for aroxmatng set cover. Journal of ACM ), [7] U. Fege, V. Mrrokn and J. Vondrák. Maxmzng non-monotone submodular functons, FOCS [8] C. Greene. A multle exchange roerty for bases, J Proc. Amer. Math. Soc ), [9] C. Hurkens and A. Schrjver, On the sze of systems of sets every t of whch have an SDR, wth an alcaton to the worst-case rato of heurstcs for ackng roblems. SIAM J. Dscrete Math ), no. 1, [10] S. Iwata, L. Flescher and S. Fujshge. A combnatoral, strongly olynomal-tme algorthm for mnmzng submodular functons, Journal of ACM 48:4 2001),

15 [11] J. Lee, V. Mrrokn, V. Nagarajan and M. Svrdenko. Maxmzng Non-Monotone Submodular Functons under Matrod and Knasack Constrants, submtted for ublcaton, relmnary verson aeared n Proceedngs of STOC [12] L. Lovász. Submodular functons and convexty. In: A. Bachem et. al., eds, Mathematcal Programmmng: The State of the Art, [13] G.L. Nemhauser, L.A. Wolsey and M.L. Fsher. An analyss of aroxmatons for maxmzng submodular set functons I. Mathematcal Programmng ), [14] G.L. Nemhauser, L.A. Wolsey and M.L. Fsher. An analyss of aroxmatons for maxmzng submodular set functons II. Mathematcal Programmng Study ), [15] G. L. Nemhauser and L. A. Wolsey. Best algorthms for aroxmatng the maxmum of a submodular set functon, Math. Oer. Research, 33): , [16] J.G. Oxley. Matrod theory, Oxford Scence Publcatons. The Clarendon Press, Oxford Unversty Press, New York, [17] J. Rechel and M. Skutella. Evolutonary algorthms and matrod otmzaton roblems, n Proceedngs of the 9th Genetc and Evolutonary Comutaton Conference GECCO 07), , [18] A. Schrjver. Combnatoral Otmzaton: Polyhedra and Effcency. Srnger, [19] A. Schrjver. A combnatoral algorthm mnmzng submodular functons n strongly olynomal tme, Journal of Combnatoral Theory, Seres B ), [20] J. Vondrák. Otmal aroxmaton for the submodular welfare roblem n the value oracle model. In STOC, [21] D.R. Woodall. An exchange theorem for bases of matrods, J. Combnatoral Theory Ser. B ),