A Parallel Approximation Algorithm for Positive Linear Programming Michael Luby Noam Nisan y Abstract We introduce a fast parallel approximation algorithm for the positive linear programming optimization problem, i.e. the special case of the linear programming optimization problem where the input constraint matrix and constraint vector consist entirely of positive entries. The algorithm is elementary, and has a simple parallel implementation that runs in polylog time using a linear number of processors. 1 Introduction The positive linear programming optimization problem (hereafter referred to as the positive problem) is the special case of the linear programming optimization problem where the input constraint matrix and constraint vector consist entirely of non-negative entries. We introduce an algorithm that takes as input the description of a problem and an error parameter and produces both a primal feasible solution and a dual feasible solution, where the values of these two solutions are within a multiplicative factor of 1 + of each other. Because the opti- International Computer Science Institute and UC Berkeley. Research supported in part by NSF Grant CCR-9016468 and grant No. 89-00312 from the United States-Israel Binational Science Foundation (BSF), Jerusalem, Israel. yhebrew University, Jerusalem, Israel. Supported by USA-Israel BSF 89-00126 and by a Wolfson research award. Research partially done while visiting the International Computer Science Institute. mal values for the primal and dual problems are equal, this implies that the primal and dual feasible solutions produced by the algorithm have a value within (with respect to relative error) of an optimal feasible solution. Let N be the number of non-zero coecients associated with an instance of the problem. Our algorithm can be implemented on a parallel machine using O(N) processors with a running time polynomial in log(n)=. The algorithm is elementary and has a simple parallel implementation. Note that the problem of approximating the value of a general linear program to within a constant factor is P -complete. This can be shown by a reduction from the circuit value problem to a linear programming problem where all coecients are small constants and the linear programming problem has exactly one feasible solution with value either 0 or 1 depending upon the answer to the circuit value problem (see, e.g., [5, Ja Ja]). Previously, [8, Plotkin, Shmoys, Tardos] have developed fast sequential algorithms for both the primal and dual versions of the positive problem which they call fractional packing and covering problems, (as well as for some generalizations of this problem). They introduce algorithms that are much simpler and far superior in terms of running times than known algorithms for the general linear programming optimization problem. However, the algorithms in [8] do not have fast parallel implementations. The algorithm we introduce is competitive with their algorithms in terms of running times when implemented se- 1
quentially. We rst introduce an elementary (but unimplementable) continuous algorithm that produces optimal primal and dual feasible solutions to the problem, and based on this we describe a fast parallel approximation algorithm. We use ideas that were previously employed in similar contexts by [1, Berger, Rompel, Shor], [8, Plotkin, Shmoys, Tardos] and [2, Chazelle, Friedman]. We use the general idea also used in [1] of incrementing the values of many variables in parallel, and we use the general idea also used in [8] and [2] of changing the weight function on the constraints by an amount exponential in the change in the variables. The overall method we introduce is novel in several respects, including the details of how to increment the values of the variables at each iteration (this choice reects a carefully chosen tradeo between the overall running time of the algorithm and the quality of the solution it produces) and the way to normalize the solution output at the end. Positive linear programs are strong enough to represent several combinatorial problems. The rst example is matching in a bipartite graph. From matching theory we know that relaxing the f0; 1g program that denes the largest matching to a linear program does not change the optimal value. This program is positive and thus our algorithm can be used to approximate the size of the largest matching in a bipartite graph. This essentially matches the results of [3, Cohen], except that we don't know how to get the matching itself without using some of Cohen's techniques. The second example is that of set-cover. In this case it is known that relaxing the 0? 1 program that denes the minimum set-cover to a linear program can decrease the optimum by at most a factor of log(), where is the maximum degree in the set system. This program is positive and thus our algorithm can be used to approximate the size of the set cover to within a factor of (1 + ) log(). This is essentially optimal (up to NP-completeness [7, Lund, Yannakakis]) and matches results of [1, Berger, Rompel, Shor]. In this case nding the set cover itself is also possible using e.g. ideas found in [9, Raghavan]. 2 The problem Throughout this paper, n is the number of variables and m is the number of constraints (not including constraints of the form x i 0). We use i to index variables, and whenever it is unspeci- ed we assume that i ranges over all f1; ; ng. We use j to index the non-zero constraints, and whenever it is unspecied we assume that j ranges over all f1; ; mg. Unless otherwise specied, if x = hx 1 ; ; x n i then sum(x) is de- ned as P i x i. (In a few places, sum(x) is de- ned as a weighted sum.) We consider linear problems in the following standard form The Primal Problem The objective is to nd z = hz 1 ; ; z n i that minimizes sum(z) = i d i z i subject to the following constraints For all i, z i 0. For all j, P i c i;j z i b j. We say z is primal feasible if z satises all the constraints. Let opt(z) be an optimal solution, i.e. opt(z) is a primal feasible solution such that sum(opt(z)) = min fsum(z)g z is primal feasible We consider also the dual of such problems The Dual Problem The objective is to nd q = hq 1 ; ; q m i that maximizes sum(q) = j b j q j Our results are a slight improvement over those in [1] in the sense that our multiplicative constant is 1 +, where is an input parameter, whereas their multiplicative constant is xed to something like 2. 2
subject to the following constraints For all j, q j 0. For all i, P j c i;j q j d i. We say q is dual feasible if q satises all the constraints. Let opt(q) be an optimal solution, i.e. opt(q) is a dual feasible solution such that sum(opt(q)) = max fsum(q)g q is dual feasible Description of Positive Problem We say that a linear program is positive if all the coef- cients are non-negative. I.e. For all i, d i > 0. For all j, b j > 0. For all i and j, c i;j 0 (It can easily be sees that restricting d i and b j to be strictly positive instead of non-negative causes no loss of generality.) We can look at the primal problem as trying to put weights on the x i 's such that each j is covered with weight at least b j, where each unit of weight on x i puts c i;j weight on each j. Similarly, we can look at the dual case as trying to put weights on the q j 's such that we pack at most d i weight into each i. For these reasons a positive linear program in the primal form is sometimes called a fractional covering problem, and in the dual case a fractional packing problem. We will sometimes use this terminology in our analysis. In this paper we develop an approximation algorithm for both these problems with the following properties. On input > 0 and the description of the problem, the algorithm produces a primal feasible solution z and a dual feasible solution q such that sum(z) sum(q) (1 + ) The algorithm consists of O log(n) log(m=) 4 iterations. Each iteration can be executed in parallel using O(N) processors in time O(log(N)) on a EREW PRAM, where N is the number of entries hi; ji where c i;j > 0. 2.1 A special form In the appendix we show that without loss of generality we may assume that the linear program is in the following special form Input to special form For all hi; ji, the input is a i;j, such that either a i;j = 0 or 1 a i;j 1=, where = m2 2 Special Form Primal Problem The objective is to nd z = hz 1 ; ; z n i that minimizes sum(z) = i subject to the following constraints For all i, z i 0. For all j, P i a i;j z i 1. Special Form Dual Problem The objective is to nd q = hq 1 ; ; q m i that maximizes sum(q) = j z i q j subject to the following constraints For all j, q j 0. For all i, P j a i;j q j 1. 3
2.2 The algorithm Given a problem instance in special form, the algorithm we develop below has the following properties. Let a i;j be the coecients for the input problem, and let = minfsum(z) z is primal feasibleg = maxfsum(q) q is dual feasibleg On input > 0 and the a i;j, the output is a primal feasible solution z = hz 1 ; ; z n i and a dual feasible solution such that q = hq 1 ; ; q m i sum(z) sum(q) (1 + ) Since sum(z) sum(q), this immediately implies that sum(z) (1 + ) and that sum(q) =(1 + ). The parallel algorithm we present below can be viewed as a parallel discretization of the following simple (but unimplementable) continuous algorithm. Based on the analysis of the parallel algorithm given below, it is not hard to see that the continuous algorithm produces optimal primal and dual solutions. Continuous Algorithm The values of all variables are driven by x = hx 1 ; ; x n i, and this vector is initially all zeroes. For all j, dene j = Pi a i;j x i, = P min j f j g and y j = e?j. For all i, dene D i = j a i;j y j, D = max i fd i g and B = fi D i = Dg. The continuous algorithm increases the values of all x i for all i 2 B in the direction which makes all D i for i 2 B decrease at the same rate. Continue the process forever. The optimal primal solution is z i, where z i is the limiting ratio of x i =. The optimal dual solution is q j, where q j = ^y j = ^D and ^y and ^D are dened as the y and D that maximize sum(y)=d over all time. We now describe the parallel (and implementable) version of this continuous algorithm. Initialization For all i, initialize x i = 0. For all j, initialize y j = 1. We now describe a phase of the algorithm. Each phase is indexed by an integer k and the index of the next phase is one smaller than the index of the previous phase. The values for the index of the start phase k s and the index of the nal phase k f are xed later. Denition 0 = 1 = 2 = 3 = 4 = =5. Phase k Each phase consists of a sequence of iterations. Each iteration consists of the following. For all i, let D i = P j a i;j y j Let B = fi D i (1 + 0 ) k g. If B = ; then the phase ends. For all > 0, dene { E() = fj P i2b a i;j = 1 g. { S() = fj P i2b a i;j < 1 g. { L() = fj P i2b a i;j > 1 g. { D? i () =P j2e()[s() a i;j y j { D + i () =P j2e()[l() a i;j y j Choose 0 such that (a) For at least a fraction 1? 2 of the i 2 B, D? i ( 0) (1? 3 ) D i (b) For at least a fraction 2 of the i 2 B, D + i ( 0) 3 D i For all i 2 B, x i = x i + 0. For all j, y j = y j e?0 Pi2B ai;j 4
Denition D = max i fd i g Start Phase The index of the start phase is the smallest positive integer k s such that the initial value of D satises (1 + 0 ) ks D < (1 + 0 ) ks+1 Final Phase The nal phase executed is the rst phase where sum(y) 1 is true at the m 1= 4 end of the phase. The following invariants are maintained throughout the course of the algorithm. Degree invariant At the beginning of an iteration within the execution of phase k it is the case that, for all i 2 B, (1 + 0 ) k D i < (1 + 0 ) k+1 Coverage invariant Dene Caveat In the analysis given below, for simplicity we are slightly inaccurate in the following sense. We say for example that (1+ 0 )(1+ 1 ) = 1 (1 + 0 + 1 ) and 1? 0 = 1 + 0. Each time we do this we introduce a small error, but we only do this a constant number of times overall. 2.3 Running time analysis We rst determine an upper bound on the number of phases. After the initialization, D m. This is because a i;j 1 for all hi; ji and because initially y j = 1 for all j. Let k s be the index of the rst phase. To guarantee that D < (1 + 0 ) ks+1 at the start of this phase, it is sucient that k s satisfy and this is true when (1 + 0 ) ks+1 > m; j = i a i;j x i k s = O(log(m)=) and let = min j f j g Then, j is the amount that j is covered by the current value of x, and is the minimal coverage of any j. Conventions We use the convention that x i is the value assigned to i at the beginning of an iteration within a phase, x 0 i is the value after the end of the iteration, and x i is the value after termination of the nal phase. This same convention applies to all the other variables, i.e. D, y j, j,, etc. Consider the iteration of the algorithm when sum(y)=d is maximum among all iterations of the algorithm. Let ^y = h^y 1 ; ; ^y m i be y from this iteration and let ^D be D from this iteration. Primal Feasible Solution Output output is, for all i, z i = x i The out- Dual Feasible Solution Output put is, for all j, q j = ^yj^d The Note that P D sum(y)= because each j contributes at least y j = to i D i. Thus, if D 1 then sum(y) m 1= 1. The algorithm 4 m 1= 4 never reaches phase?k if (1 + 0 )?k 1. m 1= 4 Thus, the last phase occurs before phase?k f where k f = O log(m=) 2 Thus, the total number of phases is O log(m=) 2 We now analyze the number of iterations per phase. At least a fraction 2 of the i 2 B have the property that D + i ( 0) 3 D i. For each j 2 E( 0 ) [ L( 0 ), y j goes down by at least a factor of e?1 1? 1. Let Di 0 be the value of D i at the end of the iteration. >From the above, a fraction of at least 2 of the i 2 B have the property that D 0 i D i (1? 1 3 ) 5
Note that after the value of D i drops by a factor of at least 1? 0, i is removed from B. >From this it follows that the number of iterations during phase k is at most O log(jbk j) 2 ; where B k is the set B at the beginning of phase k. Since, jb k j n for all k, it follows that the total number of iterations overall is O log(n) log(m=) 4 2.4 Computing 0 The most dicult part in each iteration is the computation of 0. This can be done as follows. For each j 2 f1; ; mg, compute j = 1 Pi2B a i;j and nd a permutation of f1; ; mg that satises by sorting the set (1) (m) f( j ; j) j 2 f1; ; mgg For xed j, let j? is the smallest index and j + is the largest index for which (j? ) = (j) = (j + ) >From these denitions, it follows that E( (j) ) = f(j? ); ; (j + )g S( (j)) = f(1); ; (j?? 1)g L( (j)) = f(j + + 1); ; (m)g For each i 2 B, for each j 2 f1; ; mg, compute D? i ( (j)) = P kj + a i;(j) y (j) D + i ( (j)) = P kj? a i;(j) y (j) using a parallel prex computation. Note that D? i ( (1)) D? i ( (m)) D + i ( (1)) D + i ( (m)) Then, for each i 2 B, compute an index v(i) 2 f1; ; mg for which D? i ( (v(i))) (1? 3 ) D i D + i ( (v(i))) 3 D i using binary search on j. There is such an index v(i) because, for all i; j, D? i ( (j)) + D + i ( (j+1)) D i Then, nd an ordering hb 1 ; ; b jbj i of B that satises by sorting the set (v(b 1)) (v(b jbj )) f (v(i)) i 2 Bg Note that, for all 1 i < i 0 jbj, D? i ( (v(b i 0 ))) D? i ( (v(b i))) D + i ( (v(b i 0 ))) D + i ( (v(b i))) Compute i 0 so that i0 jbj 1? 2 jbj?i0+1 jbj 2 using binary search. Finally, set 0 = (v(b i 0 )) It is not hard to verify that 0 satises the conditions specied in the description of the algorithm. Each iteration consists of a constant number of parallel prex, parallel sorting, and parallel binary search operations. By the results described in [6, Ladner, Fischer] and [4, Cole], each iteration can be executed in parallel using O(N) processors in time O(log(N)) on a EREW PRAM. 6
2.5 Feasibility Primal We prove that the solution z output by the algorithm is a primal feasible solution. Recall that yj = e? j. Because sum(y ) < 1 at termination, ln(1=sum(y )) > 0. Since z i = x i = it follows that each j is covered at least j = 1 times at the end of the algorithm. This implies that z is a primal feasible solution. Dual Because ^D is the maximum over all i of ^Di, it follows that each i is covered a total of ^D i = ^D 1 with respect to q. It follows that q is a dual feasible solution. 2.6 Optimality Let = 0 + 1 + 2 + 3 Lemma 1 sum(y 0 ) sum(y)? 0 D jbj (1? ) PROOF Let r j = y j? y 0 j = y j (1? e?0 Pi2B ai;j ) We want to show that Note that j j r j 0 D jbj (1? ) r j For all j 2 E( 0 ) [ S( 0 ), j2e( 0)[S( 0) 1? e?0 Pi2B ai;j 0 i2b r j a i;j (1? 1 ) This inequality is because of the following For all 1, 1? e? (1? ) Letting = 0 i2b a i;j ; and noting that j 2 E( 0 ) [ S( 0 ) implies that 1, this implies that Thus, 1? e? (1? ) (1? 1 ) j2e( 0)[S( 0) j2e( 0)[S( 0) i2b j2e( 0)[S( 0) But, i2b j2e( 0)[S( 0) r j = y j (1? e?0 Pi2B ai;j ) a i;j y j 0 (1? 1 ) a i;j y j = i2b D? i ( 0) For at least a fraction (1? 2 ) of the i 2 B, D? i ( 0) (1? 3 ) D i Since D i D=(1 + 0 ), it follows this sum is at least (1? 2 ) (1? 3 ) D jbj 1 + 0 Putting all this together yields j Lemma 2 r j 0 D jbj (1? ) sum(x) sum(q) ln(m=sum(y)) 1? PROOF >From Lemma 1 it follows that sum(y 0 ) sum(y) 1? 0 D jbj (1? ) sum(y) e? 0 DjBj(1?) sum(y) e? 0 jbj(1?) sum(q) The last inequality is because sum(q) sum(y)=d 7
Because i x 0 i? x i = 0 jbj; i and because initially sum(y) = m, it follows that at any point in time, This implies that sum(x) Theorem?Pi x i (1?) sum(y) m e sum(q) sum(q) ln(m=sum(y)) 1? sum(z) sum(q) (1 + ) PROOF >From Lemma 2 and the denition of z it follows that sum ( z) sum(q) ln(m=sum(y )) (1? ) Since ln(1=sum(y )) it follows that sum(z) sum(q) (1 + ) ln(m=sum(y )) ln(1=sum(y )) Since sum(y ) 1 m 1=4 at the termination of the algorithm, the theorem follows. 3 Acknowledgments The authors would like to thank Richard Karp, Se Naor, Serge Plotkin and Eva Tardos for pointing out relationships between this work and previous work. The rst author would like to thank Chu-Cheow Lim for discussions which claried the description of the implementation. References [1] Berger, B., Rompel, J., Shor, P., \Ecient NC Algorithms for Set Cover with Applications to Learning and Geometry", 30 th Annual Symposium on Foundations of Computer Science, pp. 54-59, 1989. [2] Chazelle, B., Friedman, J., \A Deterministic View of Random Sampling and Its Use in Geometry", Princeton Technical Report No. CS-TR-436, September 1988. A preliminary version appears in FOCS 1988. [3] Cohen, E., \Approximate max ow on small depth networks", FOCS, 1992, pp. 648-658. [4] Cole, R., \Parallel merge-sort", SIAM J. Comp., 17(4), pp. 770-785, 1988. [5] Ja Ja, J., An Introduction to Parallel Algorithms, Addison Wesley, 1992 [6] Ladner, R., Fischer, M., \Parallel prex computation", JACM, 27(4), pp. 831-838, 1980 [7] Lund, C., Yannakakis, M., \On the Hardness of Approximating Minimization Problems", preprint. [8] Plotkin, S., Shmoys, D., Tardos, E., \Fast Approximation Algorithms for Fractional Packing and Covering Problems", Stanford Technical Report No. STAN-CS-92-1419, February 1992. [9] Raghavan, P., \Probabilistic construction of deterministic algorithms approximating packing integer programs", JCSS, October 1988, Vol. 37, pp 130-143. 4 Appendix Transforming to special form Given an instance of a positive problem, we rst perform a normalization step to eliminate the weights d i and the constraint bounds b j. Normalized problem For all hi; ji, let c 0 i;j = ci;j b jd i. For all i, dene variable z 0 i = d i z i. 8
The objective is to nd z 0 = hz1 0 ; ; z0 n i that minimizes sum(z 0 ) subject to the following constraints For all i, z 0 i 0. For all j, P i c0 i;j z0 i 1. There is a one-to-one correspondence between primal feasible solutions z 0 to the normalized problem and primal feasible solutions z to the primal positive problem with the property that sum(z) = sum(z 0 ). The next step to transforming the problem to the special form is to limit the range of the coecients c 0 i;j. This step will introduce an error of at most. This turns out to be important for the analysis of the approximation algorithm we develop. For all j, let and let Fact j = maxfc 0 i;j g; i = min j f j g m= sum(opt(z 0 )) 1= The transformation consists of forming the new set of coecients c 00 i;j as follows. For all hi; ji, if c 0 i;j > m then c 00 i;j = m. For all hi; ji, if c 0 i;j < m then c00 i;j = 0. For all hi; ji, if m c0 i;j m then c 00 c 0 i;j. i;j = The objective is to nd z 00 = hz 00 1 ; ; z00 n i that minimizes sum(z 00 ) subject to the following constraints For all i, z 00 i 0. For all j, P i c00 i;j z00 i 1. Let and let Fix t = max i;j fc00 i;j g b = min i;j fc00 i;j c 00 i;j > 0g = m2 2 The main point of this transformation is that the ratio t=b is not too large, i.e. t=b. Lemma A (1) Any primal feasible solution z 00 to the transformed problem is also a primal feasible solution to the normalized problem. (2) Let opt(z 00 ) be an optimal solution to the transformed problem. Then, PROOF sum(opt(z 00 )) sum(opt(z)) (1 + ) (of part (1)) This follows immediately because, for all hi; ji, c 00 i;j c0 i;j. Thus, if z00 covers each j at least once with respect to c 00, then z 00 covers each j at least once with respect to c 0. (of part (2)) Let opt(z 0 ) be an optimal solution to the normalized problem, i.e. sum(opt(z 0 )) = sum(opt(z)). Let = sum(opt(z 0 )). For all j, let m j 2 f1; ; ng be an index such that c 0 m j ;j is maximal, i.e. for all i, c 0 m j;j c0 i;j. Let Dene z 00 as follows. I = fm j j 2 f1; ; mgg For all i 62 I, let z 00 i = opt(z 0 ) i For all i 2 I, let z 00 i = opt(z0 ) i + m Let jij denote the cardinality of I. Since jij m, it is easy to see that sum(z 00 ) (1 + ) 9
We now verify that z 00 is a primal feasible solution to the transformed problem. The only concern with respect to feasibility is that some j is no longer covered at least once by z 00, and the only reason this can happen is because of the lowering of the coecients from c 0 to c 00. We show that this loss in coverage is compensated for by the increase in z 00 above opt(z 0 ). Suppose j is such that for at least one index i it is the case that c 0 i;j > m. By denition of m j and c 00 m j;j, it follows that c 00 m j ;j = m. Since m j 2 I, zm 00 j ;j = opt(z 0 ) mj ;j + m m By the fact on page 9, m m, and thus the coverage of j is at least zm 00 j c 00 m j ;j 1 Suppose j is such that for all indices i it is the case that c 0 i;j m. The decrease in the coverage of j is caused by hi; ji where c 0 i;j < m. Because c00 i;j is set to 0 and because sum(opt(z 0 )) =, the total loss in coverage at j is at most m On the other hand, by denition of and m j and c 00 m j ;j, c00 m j ;j, and z00 m j ;j is larger than opt(z 0 ) mj ;j by m Thus, the coverage of j increases by at least m Note that max i;j fa i;j g = 1 and min i;j fa i;j a i;j > 0g 1= There is a one-to-one correspondence between primal feasible solutions x to the special form problem and primal feasible solutions z 00 to the transformed problem with the property that sum(z 00 ) = sum(x)=t. Lemma B is the culmination of the above development. Lemma B Let x be a primal feasible solution to a special form problem derived as described above with error parameter from a primal positive problem. A primal feasible solution z to the primal positive problem can be easily derived from x with the property that sum(z) (1 + ) sum(x) sum(opt(x)) sum(opt(z)) A similar transformation is possible to preserve the quality of the the dual positive problem, but is omitted from this paper due to lack of space. For convenience, we normalize the largest coef- cient as follows. Special form For all hi; ji, let a i;j = c00 i;j t. For all i, dene variable x i = z 00 i t. The objective is to nd x = hx 1 ; ; x n i that minimizes sum(x) subject to the following constraints For all i, x i 0. For all j, P i a i;j x i 1. 10