Greedy can also beat pure dynamic programming

Transcription

1 Electronc Colloquum on Computatonal Complexty, Report No. 49 (208) Greedy can also beat pure dynamc programmng Stasys Jukna Hannes Sewert Insttut für Informatk, Goethe Unverstät Frankfurt am Man, Germany Abstract Many dynamc programmng algorthms are pure n that they only use mn or max and addton operatons n ther recurson equatons. The well known greedy algorthm of Kruskal solves the mnmum weght spannng tree problem on n-vertex graphs usng only O(n 2 log n) operatons. We prove that any pure DP algorthm for ths problem must perform 2 Ω(n) operatons. Snce the greedy algorthm can also badly fal on some optmzaton problems, easly solvable by pure DP algorthms, our result shows that the computatonal powers of these two types of algorthms are ncomparable. Keywords: Spannng tree; arborescence; arthmetc crcut; tropcal crcut; lower bound Introducton and result A dynamc programmng (DP) algorthm s pure f t only uses mn or max and addton operatons n ts recurson equatons, and the equatons do not depend on the actual values of the nput weghts. Notable examples of such DP algorthms nclude the Bellman-Ford-Moore algorthm for the shortest s-t path problem [8, 5, ], the Floyd-Warshall algorthm for the all-pars shortest paths problem [6, 8], and the Held-Karp algorthm for the Travellng Salesman Problem [9]. It s well known and easy to show that, for some optmzaton problems, already pure DP algorthms can be much better than greedy algorthms. Namely, there are a lot of optmzaton problems whch are easly solvable by pure DP algorthms (exactly), but the greedy algorthm cannot even acheve any fnte approxmaton factor: maxmum weght ndependent set n a path, or n a tree, the maxmum weght smple s-t path n a transtve tournament problem, etc. In ths note, we show that the converse drecton also holds: on some optmzaton problems, greedy algorthms can also be much better than pure dynamc programmng. So, the computatonal powers of greedy and pure DP algorthms are ncomparable. We wll show that the gap occurs on the undrected mnmum weght spannng tree problem, by frst dervng an exponental lower bound on the monotone arthmetc crcut complexty of the correspondng polynomal. Let K n be the undrected graph on {,..., n}. Assume that edges e have ther assocated weghts x e, consdered as formal varables. Let T n be the famly of all T n = n n 2 spannng trees n K n, Research supported by the DFG grant JU 305/- (German Research Foundaton). Afflated wth the Insttute of Data Scence and Dgtal Technologes, Faculty of Mathematcs and Informatcs of Vlnus Unversty, Lthuana. E-mal: stukna@gmal.com E-mal: sewert@th.cs.un-frankfurt.de ISSN

2 each vewed as ts set of edges. It s well known that T n s the famly of bases of a matrod, known as graphc matrod; so, on ths famly of feasble solutons, both optmzaton problems (mnmzaton and maxmzaton) can be solved by standard greedy algorthms. On the other hand, the theorem below states that the polynomal correspondng to T n has exponental monotone arthmetc crcut complexty. The spannng tree polynomal (known also as the Krchhoff polynomal of K n ) s the followng homogeneous, multlnear polynomal of degree n : f n (x) = x e. T T n e T For a multvarate polynomal f wth postve coeffcents, let L(f) denote the mnmum sze of a monotone arthmetc (+, ) crcut computng f. The goal of ths note s to prove that L(f n ) s exponental n n. Theorem. L(f n ) 2n ( ) 9 n. 5 A drected verson of f n s the arborescences polynomal fn. An arborescence (known also as a branchng or a drected spannng tree) on the vertex-set [n] = {,..., n} s a drected tree wth edges orented away from vertex such that every other vertex s reachable from vertex. Let T n be the famly of all arborescences on [n]. Jerrum and Snr [] have shown that L( f n ) n ( ) 4 n 3 holds for the arborescences polynomal f n (x) = x e. T T n Note that here varables x, and x, are treated as dstnct, and cannot both appear n the same monomal. Ths dependence on orentaton was crucally utlzed n the argument of [, p. 892] to reduce a trval upper bound (n ) n on the number of monomals n a polynomal computed at a partcular gate tll a non-trval upper bound (3n/4) n. So, ths argument does not apply to the undrected verson f n (where x, and x, stand for the same varable). Ths s why we use a dfferent argument to handle the undrected case; our argument works also n the drected case. 2 Some consequences Every pure DP algorthm s ust a specal (recursvely constructed) tropcal (mn, +) or (max, +) crcut, that s, a crcut usng only mn (or max) and addton operatons as gates; each nput gate of such a crcut holds ether one of the varables x or a nonnegatve real number. So, lower bounds on the sze of tropcal crcuts yeld the same lower bounds on the number of operatons used by pure DP algorthms. For optmzaton problems, whose feasble solutons all have the same cardnalty, ths latter task can be solved by provng lower bounds of the sze of monotone arthmetc crcuts. Recall that a multvarate polynomal s monc f all ts nonzero coeffcents are equal to, multlnear f no varable occurs wth degree larger than, and homogeneous f all monomals have the same degree. Every monc and multlnear polynomal f(x) = S F S x defnes two optmzaton problems: compute the mnmum or the maxmum of S x over all S F. e T 2

3 a 2 3 A b b 2 3 B 2 3 (A \ {a}) {b} 2 3 (A \ {a}) {b } Fgure : Two arborescences A and B on [3] = {, 2, 3}. The only edges n B \ A are b and b. But we cannot add any of them to A \ {a} to get an arborescence. So, the bass exchange axom fals. Reducton Lemma ([, 2]). If a polynomal f s monc, multlnear and homogeneous, then every tropcal crcut solvng the correspondng optmzaton problem defned by f must have at least L(f) gates. Ths fact was proved by Jerrum and Snr [, Corollary 2.0]; see also [2, Theorem 9] for a smpler proof. The proof dea s farly smple: havng a tropcal crcut, turn t nto a monotone arthmetc (+, ) crcut, and use the homogenety of f to show that, after removng some of the edges enterng +-gates, the resultng crcut wll compute our polynomal f. Greedy can beat pure DP The (weghted) mnmum spannng tree problem MST n (x) s, gven an assgnment of nonnegatve real weghts to the edges of K n, compute the mnmum weght of a spannng tree of K n, where the weght of a graph s the sum of weghts of ts edges. So, ths s exactly the mnmzaton problem defned by the spannng tree polynomal f n : MST n (x) = mn x e. T T n Snce the famly T n of feasble solutons of ths problem s the famly of bases of the (graphc) matrod, the problem can be solved by the standard greedy algorthm. In partcular, the well known greedy algorthm of Kruskal [4] solves MST n usng only O(n 2 log n) operatons. On the other hand, snce the spannng tree polynomal f n s monc, multlnear and homogeneous, our theorem together wth the Reducton lemma mples that any (mn, +) crcut solvng the problem MST n must use at least L(f n ) = 2 Ω(n) gates and, hence, at least so many operatons must be performed by any pure DP algorthm solvng MST n. Ths gap between pure DP and greedy algorthms s our man result. Remark (Drected versus undrected spannng trees). The arborescences polynomal f n s also monc, multlnear and homogeneous, so that the Reducton lemma, together wth the above mentoned lower bound on L( f n ) of Jerrum and Snr [], also yelds the same lower bound on the sze of (mn, +) crcuts solvng the mnmzaton problem on the famly T n of arborescences. But ths does not separate DP from greedy, because the downward closure of T n s not a matrod (see Fg. for a smple counter-example). As observed by Edmonds [4], ths famly s an ntersecton of two matrods, meanng that unlke for MST n, the greedy algorthm can only approxmate the mnmzaton problem on T n wthn the factor 2. Polynomal tme algorthms solvng ths problem exactly were found by several authors, startng from Edmonds [5]. The fastest algorthm for the problem s due to Taran [6], and solves the problem n tme O(n 2 log n), that s, wth the same tme complexty as Kruskal s greedy algorthm for undrected graphs [4]. But these are no more greedy algorthms. So, T n does not separate greedy and pure DP algorthms. 3 e T

4 Subtracton can speed up (+, ) crcuts There s a well-known determnantal formula, due to Krchhoff [3] (see, e.g., [2, Theorem II.2]), known as the (weghted) matrx tree theorem. Ths formula provdes a way to compute the spannng tree polynomal f n by an arthmetc (+,, ) crcut of polynomal sze. Together wth our theorem, ths gves yet another explct example where the use of subtracton n arthmetc crcuts leads to exponental savng n ther sze; that already one subtracton gate can lead to such savng was shown already by Valant [7] usng another polynomal correspondng to perfect matchngs n planar graphs. Dvson can speed up (+, ) crcuts Fomn, Grgorev and Koshevoy [7] have recently proved that the spannng tree polynomal f n can be computed by an arthmetc (+,, ) crcut usng only O(n 3 ) gates. Ther recurson (gven by Lemma 8.3 n [7]) turns to the followng procedure for computng f n : n order to compute f n (x) on the vertex-set {,..., n} wth edge-weghts x,, frst compute the sum X n := x,n + + x n,n of the weghts of edges ncdent to the last vertex n, then compute f n (x ) on the frst n vertces under the new edge-weghts x, := x, + x,n x,n X n. Lemma 8.3 n [7] shows that then f n (x) = X n f n (x ) holds. Together wth our theorem, ths shows that, lke subtracton gates, dvson gates n monotone arthmetc crcuts can also lead to exponental savngs. Actually, t s shown n [7] that also the arborescences polynomal f n can be computed by an arthmetc (+,, ) crcut usng O(n 3 ) gates. So, the same gap between (+,, ) and (+, ) crcuts follows also from the aforementoned lower bound of Jerrum and Snr [] for f n. Subtracton can speed up pure DP algorthms Snce dvson n tropcal (mn, +) and (max, +) semrngs corresponds to subtracton, the result of [7] also mples that the mnmum weght spannng tree problem can be solved usng only O(n 3 ) mn, +, gates. So, subtracton operaton can exponentally speed-up pure DP algorthms: n order to compute MST n (x) on the vertexset {,..., n} wth edge-weghts x,, frst compute the mnmum X n := mn {x,n,..., x n,n } of weghts of edges ncdent to the last vertex n, then compute MST n (x ) on the frst n vertces under the new edge-weghts and output MST n (x) = X n + MST n (x ). 3 Proof of the theorem x, := mn {x,, x,n + x,n X n }, We wll use the followng well-known decomposton result, frst proved by Hyafl [0, Theorem ] and Valant [7, Lemma 3]. A polynomal s nonnegatve f t has no negatve coeffcents. Decomposton Lemma (Hyafl, Valant). Let f be a nonnegatve homogeneous polynomal of degree m. If L(f) t, then f can be wrtten as a sum f = g h + + g t h t of products of nonnegatve homogeneous polynomals, each of degree at most 2m/3. Proof. (Due to Valant [7].) Inducton on the crcut sze t. Snce the polynomal f s homogeneous, the polynomal g v computed at each gate v must be also homogeneous. Startng from the output 4

5 gate, we walk backwards, by always choosng that of two nput gates whose polynomal has larger degree. Proceedng n ths way, we wll arrve at a gate v wth m/3 deg(g v ) 2m/3. Let f v be the polynomal computed by the crcut, after the nputs of gate v are replaced by zeroes. Then f = g v h + f v for some polynomal h of degree deg(h) = m deg(g v ) 2m/3. Snce the (also homogeneous) polynomal f v s computable by a crcut of sze t (the need of the gate v s already elmnated), we can apply the nducton hypothess to f v. The combnatoral core of our argument s the followng property of forests, vewed as sets of ther edges. Let A, B be two nonempty famles of forests n K n such that A contans a forest wth a edges, and B contans a forest wth b edges. Forest Lemma. If the ntersecton of any two forests A A and B B s empty, and ther unon s a spannng tree of K n, then the connected components of all forests n A have the same sets of vertces, and the same holds for forests n B. Moreover, then A B a a b b and a + b = n hold. Proof. To prove the frst clam, t s enough to show that every par of vertces s ether connected n all forests n A, or s dsconnected n all forests n A. To show ths, suppose contrarwse that some two vertces x and y are dsconnected n one forest A A, but are connected n some other forest A A. Take an arbtrary forest B B. Snce A B must be a spannng tree of K n, and vertces x and y are dsconnected n the forest A, there must be a path between x and y n the forest B. But then the graph A B contans a cycle throughout vertces x and y, a contradcton wth ths graph beng a tree. Ths shows the clam for the forests n A. The argument for the forests n B s the same. So, let U,..., U p [n] be the sets of vertces of the connected components of forests n A, and V,..., V q [n] be the sets of vertces of the connected components of forests n B. Let also u = U and v = V be the numbers of vertces n these components. Then every forest n A has a = p = (u ) edges, and every forest n B has b = q = (v ) edges. Snce the forests are dsont and ther unons must be spannng trees (wth n edges), we also have that a + b = n. Every forest n A conssts of spannng trees of complete graphs on the sets U,..., U p, and smlarly for forests n B. By a classcal result of Cayley [3], the number of trees on n labeled vertces s n n 2. Ths mples that there are only u u 2 spannng trees of the complete graph on each U, and only v v 2 spannng trees of the complete graph on each V. So, usng the nequaltes r r 2 (r ) r and r r s s (r + s) r+s for ntegers r, s 2, we obtan: A B u u 2 v v 2 (u ) u (v ) v [ (u )] (u ) [ (v )] (v ) = aa b b. Proof of the theorem. Let t = L(f n ). The spannng tree polynomal f n s multlnear and homogeneous of degree m = n : every spannng tree T of K n has T = n edges. By the Decomposton lemma, we can express ths polynomal as a sum of at most t products g h of nonnegatve homogeneous polynomals, where the degree of g and of h s at most 2m/3. The polynomal g h must be a homogeneous polynomal of degree m. So, f we denote the degrees of g and h by a and b, then a + b = m and m/3 a, b 2m/3 must hold. 5

6 Every monomal of f n s a multlnear monomal of the form e T x e for some spannng tree T. Snce the polynomals g and h n the decomposton of f n are nonnegatve, there can be no cancellatons. Ths mples that all the monomals of g h must be also monomals of f, that s, must correspond to spannng trees. So, the monomals of g and h correspond to forests. Let A be the famly of forests correspondng to monomals of the polynomal g, and B the famly of forests correspondng to monomals of the polynomal h. Snce all monomals of g h must also be monomals of f n, the par A, B fulflls the condtons of the Forest lemma for some a between m/3 and 2m/3 (the dsontness property follows from the multlnearty of g h). So, the number of monomals n the polynomal g h cannot exceed ( ) m A B a a (m a) m a (m/3) m/3 (2m/3) 2m/3 = m m 2 2/3 /3 < m m where the second nequalty holds because the functon x x ( x) x s convex n the nterval (0, ). Snce the polynomal f n has n n 2 monomals, the desred lower bound on t = L(f n ) follows: t nn 2 A B mm A B ( ) 9 m = ( ) 9 n ( ) 9 n. m 5 n 5 2n 5 Fnally, note that the same argument works also for the arborescences polynomal f n : n ths case, the (drected) forests n A and B must fulfll an addtonal restrcton: also the drectons of dockng edges must be consstent; so, the number A B of monomals n each polynomal g h can only be smaller. References [] R. Bellman. On a routng problem. Quarterly of Appl. Math., 6:87 90, 958. [2] B. Bollobás. Modern graph theory, Sprnger. Sprnger, 998. [3] A. Cayley. A theorem on trees. Quart. J. Pure Appl. Math., 23: , 889. [4] J. Edmonds. Optmum branchngs. J. of Res. of the Nat. Bureau of Standards, 7B(4): , 967. [5] J. Edmonds. Edge-dsont branchngs. In B. Rustn, edtor, Combnatoral Algorthms, pages Academc Press, 973. [6] R.W. Floyd. Algorthm 97, shortest path. Comm. ACM, 5:345, 962. [7] S. Fomn, D. Grgorev, and G. Koshevoy. Subtracton-free complexty, cluster transformatons, and spannng trees. Found. Comput. Math., 5: 3, 206. [8] L.R. Ford. Network flow theory. Techncal Report P-923, The Rand Corp., 956. [9] M. Held and R.M. Karp. A dynamc programmng approach to sequencng problems. SIAM J. on Appl. Math., 0:96 20, 962. [0] L. Hyafl. On the parallel evaluaton of multvarate polynomals. SIAM J. Comput., 8(2):20 23, 979. [] M. Jerrum and M. Snr. Some exact complexty results for straght-lne computatons over semrngs. J. ACM, 29(3): , 982. [2] S. Jukna. Lower bounds for tropcal crcuts and dynamc programs. Theory of Comput. Syst., 57():60 94, 205. [3] G. Krchhoff. Über de Auflösung der Glechungen, auf welche man be der Untersuchungen der lnearen Vertelung galvanscher Ströme geführt wrd. Ann. Phys. Chem., 72: , 847. [4] J.B. Kruskal. On the shortest spannng subtree of a graph and the travelng salesman problem. Proc. of AMS, 7:48 50, 956. [5] E.F. Moore. The shortest path through a maze. In Proc. Internat. Sympos. Swtchng Theory, volume II, pages , 957. [6] R.E. Taran. Fndng optmum branchngs. Networks, 7():25 35, 977. [7] L.G. Valant. Negaton can be exponentally powerful. Theor. Comput. Sc., 2:303 34, 980. [8] S. Warshall. A theorem on boolean matrces. J. ACM, 9: 2, 962. ( 5 9 ) m, 6 ECCC ISSN