Chpter 1 Comprison-Sorting nd Selecting in Totlly Monotone Mtrices Nog Alon Yossi Azr Abstrct An m n mtrix A is clled totlly monotone if for ll i 1 < i 2 nd j 1 < j 2, A[i 1, j 1] > A[i 1, j 2] implies A[i 2, j 1] > A[i 2, j 2]. We consider the complexity of comprison-bsed selection nd sorting lgorithms in such mtrices. Although our selection lgorithm counts only comprisons its dvntge on ll previous work is tht it cn lso hndle selection of elements of different (nd rbitrry) rnks in different rows (or even selection of elements of severl rnks in ech row), in time which is slightly better thn tht of the best known lgorithm for selecting elements of the sme rnk in ech row. We lso determine the decision tree complexity of sorting ech row of totlly monotone mtrix up to fctor of t most log n by proving qudrtic lower bound nd by slightly improving the upper bound. No nontrivil lower bound ws previously known for this problem. In prticulr for the cse m = n we prove tight Ω(n 2 ) lower bound. This bound holds for ny decision-tree lgorithm, nd not only for comprison-bsed lgorithm. The lower bound is proved by n exct chrcteriztion of the bitonic totlly monotone mtrices, wheres our new lgorithms depend on techniques from prllel comprison lgorithms. 1 Introduction. 1.1 Bckground nd previous work. Let A = A[i, j] be n m n mtrix. A is clled totlly monotone if for ll i 1 < i 2 nd j 1 < j 2, A[i 1, j 1 ] > A[i 1, j 2 ] implies A[i 2, j 1 ] > A[i 2, j 2 ]. Totlly monotone mtrices were introduced by Aggrwl, Klwe, Morn, Shor nd Wilber [4]. These mtrices rise nturlly in the study of vrious problems in computtionl geometry, in the nlysis of certin dynmic progrmming lgorithms, nd in other combintoril problems relted to VLSI Deprtment of Mthemtics, Rymond nd Beverly Sckler, Fculty of Exct Sciences, Tel Aviv University, Tel-Aviv, Isrel, nd Bellcore, Morristown, NJ, 07960, USA. Supported in prt by U.S.A.- Isreli BSF Grnt nd by Bergmnn Memoril Grnt DEC Systems Reserch Center, 130 Lytton Ave. Plo-Alto, CA 94301. A portion of this work ws done while the uthor ws in the deprtment of Computer Science, Stnford University, CA 94305-2140, nd supported by Weizmnn fellowship nd contrct ONR N00014-88-K-0166. circuit design. A wide vriety of pplictions tht use totlly monotone mtrices cn be found in [4], [5], [9], [10] nd their references. In most of the pplictions the problems re reduced to selection or sorting problem in ech row in n pproprite totlly monotone mtrix. The bsic problem considered ws row mxim (or row minim), i.e., the problem of finding the mximum (or minimum) element in ech row. An optiml lgorithm for this problem ws given in [4]. This lgorithm, usully referred to s the SMAWK lgorithm (see, e.g., [12]), runs in Θ(n) steps for n m nd in Θ(n log(2m/n)) steps for n < m. This improves significntly the obvious Θ(nm)-time lgorithm tht solves the row mxim problem for generl mtrices, nd hs been used in mny pplictions. The next nturl problem considered ws selecting the k th element in ech row. Krvets nd Prk [9] gve n lgorithm which runs in O(k(m+n)) steps. Mnsour, Prk, Schieber nd Sen [10] designed n lgorithm which runs in time O(m 1/2 n log n log m+m log n) for ny k nd thus yields better complexity for the cse of generl k, nd in prticulr for tht of selecting the medin in ech row. For the typicl cse m = n the first lgorithm is better when k n 1/2 (log n) 2 nd the second is better for ll the remining rnge. Note tht both lgorithms require tht k will be the sme for ll the rows, thus mking rther restrictive ssumption. Krvets nd Prk [9] lso considered the problem of row-sorting, i.e., the problem of sorting ech row in totlly monotone mtrix. They designed n lgorithm which runs in time O(mn+n 2 ), improving the complexity of the trivil lgorithm tht sorts ech row independently for the rnge n = O(m log m). They rised the open problem of improving their lgorithm or estblishing lower bound for this problem. Note, for exmple, tht for the cse m = n the SMAWK lgorithm reduces the time for finding the row mxim from qudrtic to liner by utilizing the specil structure of totlly monotone mtrix, wheres for sorting the time remins qudrtic (the lgorithm of [9] sves only logrithmic fctor). Observe tht the fct tht the size of the output of row-sorting lgorithm is Ω(nm) does not necessr-
2 N. Alon nd Y. Azr ily provide lower bound on the time required to sort the rows, since the output cn possibly hve smll representtion bsed on the fct tht totlly monotone mtrices re structured. Thus, it seems interesting to either improve significntly the time for row-sorting or to prove tht this is impossible. Prllel lgorithms for finding the row mxim were lso considered. The uthors of [5] gve n lgorithm tht runs on CREW PRAM in O(log n log log n) steps using n/ log log n processors (for the cse m = n). A better lgorithm, which runs on n EREW PRAM in O(log n) steps with n processors is given in [6]. 1.2 Our results. In the present pper we consider two min problems for totlly monotone mtrices. The first is selecting elements of desired rnks in the rows (where the rnks my differ, nd we my look for vrious different rnks in some rows) nd the second is rowsorting. We consider both problems minly in the comprison model. Recll tht in this model the complexity of n lgorithm is determined by the number of comprisons performed, nd the other steps in the computtion re given for free. Such model is relistic when the comprisons cost more thn the rest of the computtion. It is lso interesting in the study of lower bounds. Although our lgorithms re sequentil, some of the techniques re bsed on ides tht rise in prllel comprison lgorithms, nd minly these tht pper in the study of pproximtion problems. For the selection problem (in the specil cse of one required element in row) we ssume tht sequence of rnks r i, i = 1,... m is given nd we should find for ny i the element of rnk r i in row i. We design comprison lgorithm tht performs O(nm 1/2 log n(log m) 1/2 ) comprisons for ny given sequence r i. Note tht the complexity of our lgorithm is slightly better thn the complexity of the lgorithm of [10] which is the best known lgorithm for selecting elements with the sme rnk in ech row. In fct, the improvement is more significnt for m which is much bigger thn n. Moreover, our lgorithm hs the dvntge of being ble to del with distinct rnks (lthough it lso hs the disdvntge of being comprison lgorithm, i.e., only comprisons re counted in its complexity). Our comprison lgorithm cn be esily prllelized to run in O(log m + log log n) rounds with no penlty in the number of processors, i.e., with number of processors whose product with the bove time is equl, up to constnt fctor, to the bove mentioned totl sequentil running time. The sorting problem we consider is row-sorting, i.e., the problem of sorting ech row of given totlly monotone m by n mtrix. We prove tight lower nd upper bounds which determine the complexity of this sorting problem up to fctor of t most log n in ll cses. This settles n open problem rised in [9]. In prticulr, for the interesting specil cse m = n we prove tight lower bound of Ω(n 2 ). Here is summry of the complexity of the rowsorting problem. For m n/logn the best known lgorithm ws to sort ech row independently in totl running time O(mn log n). For m > n/ log n the best known upper bound ws O(mn + n 2 ) s shown by [9]. No nontrivil lower bounds were known. We first observe tht one cn esily design comprison row-sorting lgorithm tht runs in O(n 2 log m) time, which is much smller thn the bove mentioned bound when m is much bigger thn n. By pplying similr methods to these used in our selection lgorithm we re lso ble to (slightly) improve the complexity for the row-sorting problem when m n nd n/m = 2 o(log n). In prticulr, if m = n/(log n) O(1) our improved lgorithm replces logrithmic fctor by double logrithmic one. However, our min result for row-sorting is n lmost tight lower bound in generl decision tree model. Specificlly, we show tht ny lgorithm tht sorts ech row of totlly monotone mtrix requires Ω(min(mn, n 2 log(2 + m/n 2 ))) steps. In prticulr, for m = n the Ω(n 2 ) bound is tight. For m > n 2+ɛ the Ω(n 2 log n) bound is lso tight. For ll the remining rnge the lower bound is of the sme order of mgnitude s the upper bound up to fctor of t most log n. Recll tht in the decision tree the lgorithm is llowed t ech step to brnch into two possibilities ccording to ny computtion on the input (nd not merely comprisons), nd hence this lower bound is vlid in very generl setting. We believe tht similr techniques my be useful in estblishing lower bounds for other sorting nd serching problems deling with totlly monotone mtrices. We complete this section by the proof of the esy observtion mentioned bove. Observtion 1.1. Sorting ech row in totlly monotone m by n mtrix cn be done in O(n 2 log m) comprisons. Proof. Consider ny two columns j 1, j 2, (j 1 < j 2 ). Let l be the minimum i for which A[i, j 1 ] > A[i, j 2 ] (if such row does not exits then define l = m + 1). By the definition of the minimum nd the definition of totlly monotone mtrix, for ll i < l, A[i, j 1 ] < A[i, j 2 ] nd if i l, A[i, j 1 ] > A[i, j 2 ]. Hence, by strightforwrd binry serch one cn find, for ny specific pir of columns, this brekpoint l in O(log m) steps. Thus, the brekpoints for ll pirs cn be found in O(n 2 log m) steps. It is esy to see tht the knowledge of this
Sorting nd Selecting in Totlly Monotone Mtrices 3 informtion yields the exct order for ech row. 2 Selection. The min result in this section is the following theorem; Theorem 2.1. Let A be n m by n totlly monotone mtrix, nd let S = {(i 1, r 1 ), (i 2, r 2 ),..., (i s, r s )} be set of pirs, where 1 i j m nd 1 r j n for ll j. There exists comprison lgorithm tht finds, using T (n) = O(ns 1/2 log n(log m) 1/2 ) comprisons, the element whose rnk in row number i j is r j, for ll 1 j s. In prticulr, when s = m nd ll the numbers i j re distinct this is comprison lgorithm tht finds, using O(nm 1/2 log n(log m) 1/2 ) comprisons, n element of desired rnk in ech row. The proof is bsed on combintion of some of the techniques which hve been used in prllel comprison lgorithms with the resoning in the proof of the esy Observtion 1.1. The min prt of the lgorithm is bsed on comprisons performed ccording to the edges of ppropritely chosen rndom grphs. These cn be replced by explicit expnders, with some increse in the totl number of comprisons performed. We describe here only the version bsed on rndom grphs. We need the following two lemms, first proved in [3], which hve been pplied to vrious prllel comprison lgorithms in [11] nd in [2] s well. Lemm 2.1. For every n 1 there exists grph G(n, ) with n vertices nd t most 2n2 log n edges in which ny two disjoint sets of + 1 vertices ech re joined by n edge. Lemm 2.2. Let G = G(n, ) be grph s in Lemm 2.1, nd suppose n elements re compred ccording to the edges of G, i.e., we ssocite ech element with vertex of G nd compre pir of elements iff the corresponding vertices re djcent in G. Then, for every possible result of the comprisons, for every rnk ll but t most 7 log n from the elements with smller rnk will be known to be too smll to hve tht rnk. A symmetric sttement holds for the elements with bigger rnk. Proof of Theorem 2.1 (sketch) Given totlly monotone m by n mtrix A nd set S of pirs s in the theorem, let G = G(n, ) be grph s in Lemm 2.1, where is prmeter to be chosen lter. Let {1, 2,... n} be the set of vertices of G. For ech edge {j 1, j 2 } of G (where j 1 < j 2 ) find, by binry serch using log m comprisons, the minimum i such tht A[i, j 1 ] > A[i, j 2 ]. Altogether this costs O( n2 log n log m ) comprisons, fter which we know the results of compring the elements in ech row of the mtrix A ccording to the edges of G. Clim: In ech row seprtely, for ech rnk r, 1 r n, one cn find the element of rnk r in the row by performing t most O( log n) dditionl comprisons. Let S be the set of elements whose rnk is not known to be smller thn r nor lrger thn r. Denote by l the number of elements whose rnk is known to be smller thn r. Clerly, the element of rnk r is exctly the element of rnk r l in S. Since Lemm 2.2 implies tht S 15 log n, we conclude tht O( log n) dditionl comprisons suffice to select tht element. This completes the proof of the clim. Returning to the proof of the theorem, we conclude tht for ech vlue of it is possible to find ll the required s elements by performing t most T (n) comprison where T (n) = O( n2 log n log m ) + O(s log n). In the trivil cse s n 2 log m we cn sort, by Observtion 1.1, ll the rows of A in time T (n) = O(n 2 log m) O(ns 1/2 log n(log m) 1/2 ) s needed. Otherwise, tke nd conclude tht = n log m s T (n) = O(ns 1/2 log n(log m) 1/2 ). This completes the proof. Remrks 1). Using similr resoning we cn show tht one cn sort ll the rows in totlly monotone m by n mtrix using ( ) n 2 O log n log m + mn log( log n) comprisons, for ech choice of, n 1. This slightly improves the trivil O(nm log n) upper bound (obtined by sorting ech row seprtely) for vlues of m n which re quite close to n. For exmple, for m = n/(log n) O(1) this gives n lgorithm tht sorts the rows using O(mn log log n) comprisons. we omit the detils but mention tht s shown in the next section this is tight up to the log log n fctor (even for generl decision-tree lgorithms).
4 N. Alon nd Y. Azr 2). The rgument used in the first prt of the proof, together with some of the known results on lmost sorting lgorithms ([3], [1], [8]) implies tht by using only O(n log n log log n log m) comprisons one cn know lmost ll the order reltions between pirs of elements shring the sme row of n m by n totlly monotone mtrix. We cn lso get n pproximtion lgorithm for the row selection problem, i.e., find, for ech pir of row nd rnk in the input, n element in tht row whose rnk is close to the desired rnk. This yields trdeoff between the number of comprisons performed to the qulity of the pproximtion, (which stnds for the mening of close ). 3). The lgorithm cn be esily prllelized. The first step cn be esily done in O(log m) rounds with no penlty in the number of processors. The rest of the lgorithm cn be done in O(log log n) rounds, gin, with no loss in the totl number of opertions performed, by using the prllel selection lgorithm of [7] which is bsed on the one of [3]. 3 Comprison lower bound for row-sorting. In this section we prove the lower bound for the problem of sorting ll the rows in totlly monotone mtrices. Theorem 3.1. Any decision tree lgorithm tht sorts ech row in totlly monotone m by n mtrix requires Ω(min(mn, n 2 log(2 + m/n 2 ))) steps. In prticulr, for m n the lower bound is Ω(mn), for m n it is Ω(n 2 ), nd for m n 2+ɛ it is Ω(n 2 log m). Proof. We construct lrge fmily F of m n totlly monotone mtrices such tht ny two mtrices in the fmily differ in the order of t lest one corresponding row. Thus, ny lgorithm tht sorts ech row in the mtrix should hve different output on ech mtrix in the fmily. Hence, lower bound for ny decision tree lgorithm is the logrithm of the size of the fmily. We strt with chrcteriztion of the bitonic totlly monotone mtrices, i.e., the totlly monotone mtrices in which ech row is bitonic (=unimodl). This is done by ssociting with ech such mtrix certin tbleu T [i, j] in one-to-one mnner. We note tht similr tbleux re known s Young-tbleux nd pper in the study of the Representtions of the symmetric group, but here we re merely interested in some simple combintoril properties of T = T [i, j], described below. 1. Let l i, 1 i m be integers such tht n 1 l 1 l 2... l m 0. 2. T [i, j] is defined only for i = 1,.., m, j = 1,.., l i. 3. For ll 1 i m nd 1 j l i, T [i, j] is n integer nd 1 T [i, j] n 1. 4. Ech row of T is monotone incresing sequence. I.e., for ll 1 i m nd 1 j 1 < j 2 l i we hve T [i, j 1 ] < T [i, j 2 ]. 5. Ech column of T is monotone non-decresing sequence. I.e., for ll 1 i 1 < i 2 m nd j l j2 ( l i1 ), T [i 1, j] T [i 2, j]. We next show how to mp these tbleux into fmily F of totlly monotone mtrices such tht different tbleux would mp to different order-type mtrices. (Here, of course, the order-type of mtrix refers to the sequence of liner orders of its rows). To this end, construct the mtrix A corresponding to the tbleu T s follows. Ech row of A strts with the elements in the corresponding row in the tbleu T. Note tht row in T contins subset S of the set {1,..., n 1}. Put N = {1,..., n}. To complete the row we put to the right of S the number n (i.e. A[i, l i + 1] = n) nd then the set N S where the numbers in the set pper in decresing order. We need the following Lemm; Lemm 3.1. For ny tbleu T s bove the construction yields totlly monotone m by n mtrix. Moreover, different tbleux yield different order-type mtrices. Proof. First note tht the length of ech row in the mtrix is precisely n since ny row consists of some permuttion of the set N. Thus the construction yields, indeed, n m n mtrix. Let us show tht the resulting mtrix is totlly monotone. It is cler tht ech row is bitonic. More precisely, the first l i + 1 elements in row i form monotone incresing sequence nd the elements from l i + 1 to the end of the row form monotone decresing sequence. It is lso not too difficult to verify tht our construction nd property 5 imply tht ech column in the complement of the tbleu is monotone non-incresing sequence. Formlly, for ll 1 i 1 < i 2 m nd j > l i1 ( l i2 ), A[i 1, j] A[i 2, j]. This property is clled property 5. Suppose i 1 < i 2 nd j 1 < j 2. We hve to show tht if A[i 1, j 1 ] > A[i 1, j 2 ] then A[i 2, j 1 ] > A[i 2, j 2 ]. We consider severl possible cses. If j 2 l i1 + 1 then clerly A[i 1, j 1 ] nd A[i 1, j 2 ] re in the monotone incresing prt of the row i 1 nd thus A[i 1, j 1 ] < A[i 1, j 2 ] nd there is nothing to prove. If j 1 l i2 + 1 then clerly A[i 2, j 1 ] nd A[i 2, j 2 ] re in the monotone decresing prt of the row i 2 nd thus A[i 2, j 1 ] > A[i 2, j 2 ] which lso leves nothing to prove. Thus, we cn ssume tht j 1 l i2 l i1 < l i1 + 2 j 2.
Sorting nd Selecting in Totlly Monotone Mtrices 5 Therefore, we conclude tht A[i 2, j 1 ] A[i 1, j 1 ] > A[i 1, j 2 ] A[i 2, j 2 ] where the first inequlity uses property 5 of the tbleu (j 1 l i2 ), the middle inequlity is the ssumption nd the lst inequlity tkes dvntge of property 5 (since j 2 > l i1 ). Hence, the mtrix is totlly monotone. It is left to show tht for ny two different tbleux the construction yields mtrices of different order types. Note tht the vlue of ech element in the tbleu is in fct precisely its rnk in its row. Let the sequence l i, i = 1,..., m denote the shpe of the tbleu. Thus, if two different tbleux hve the sme shpe then they must hve different vlue in some entry. Hence, they hve different order types. On the other hnd, if they differ in the shpes, then, there exists n i for which l i in one mtrix is, sy, smller thn l i of the other. Then, the element in row i nd column l i + 1 hs different rnk in its row in the two mtrices since in the first one it hs rnk n, wheres in the second its rnk is smller thn n. This completes the proof of the Lemm. In fct, it is lso true tht ny totlly monotone mtrix A in which ech row is bitonic sequence hs the sme order type s one of the mtrices constructed in the bove wy from n pproprite tbleu T. To see this, construct T s follows. Replce ech element in A by its rnk to obtin mtrix A. Ech row in T will be the prt of the corresponding row of A tht strts from the leftmost entry in the mtrix up to the element n (the mximum) excluding this element. It is esy to verify tht this T hs ll the required properties 1-5 bove. Since this is not essentil for our results, we omit the detils. We continue the proof of Theorem 3.1 by estblishing lower bound on the size of the fmily F. First we observe tht ech row in the tbleu T hs t most 2 n 1 possibilities (ech row is subset of {1,..., n 1}). Since there re m such rows, the size of the fmily is t most 2 mn, so we cnnot expect better lower bound using bitonic mtrices. Consider first the cse m n/2. For this cse we show tht the number of such tbleux A is 2 Ω(nm), nd thus Ω(mn) steps re required in ny decision-tree rowsorting lgorithm. To this end we even restrict ourselves to subfmily of the tbleux where l i = n/2 i for 1 i n/2. Moreover, we consider only the following subfmily; T [i, j] will be either 2 (i + j 1) or 2 (i + j 1) 1. One cn esily check tht ny such ssignment produces legl tbleu T, since ll the elements re in the dmissible rnge, ech row is monotone incresing nd ech column is monotone incresing s well. Moreover, the number of such ssignments is simply 2 to the power of the number of plces in the tbleu which is 2 Ω(mn) nd we re done. For 2n 2 m > n/2 n Ω(n 2 ) lower bound follows from the bound for m = n/2. We re left with the cse m > 2n 2. In order to prove the Ω(n 2 log(m/n 2 )) lower bound for this cse we construct k = Θ(n 2 ) rows R 1,..., R k with the properties described below, nd consider the subfmily of ll tbleux of the following type: R 1 ppers x 1 times; below tht R 2 ppers x 2 times nd so forth while x 1 +... + x k = m nd x i 0. The rows R i will be defined in such wy tht ny such tbleu T will be legl. The number of tbleux tht cn be constructed in such wy is exctly the number of wys to prtition m blls in k cells which is ( ) m+k 1 k 1. The lower bound follows from the fct tht ( ) m + k 1 > ((m + k 1)/(k 1)) k 1 k 1 = 2 Ω(n2 log(m/n 2 )). It is left to show how to define the rows R 1,..., R k. Let ( ) n k = 1 + (n 1) + (n 2) +... + 1 = + 1 2 The first row is the sequence {1, 2,..., n 1}. Assume inductively tht we hve lredy constructed ll the rows up to the row which consists of the sequence {i, i+1,..., n 1}. Denote this row by S i (nd note tht it is not the i th row for i > 1). The next n i rows re constructed s follows. For 1 j n i the j th row below S i is defined to be the row S i without the number n j. The lst row in this group is exctly S i+1 = {i + 1,..., n 1}, hence we cn continue inductively from S i+1. Note tht the lst row (which is S n ) is n empty row, which is legl row (by our definitions). It is not difficult to check tht the tbleu which consists of these k rows is legl, i.e, stisfies ll the required properties. Moreover, omitting nd duplicting rows do not ffect the leglity of the tbleu. Thus, these rows cn be used for the construction described bove, completing the proof of the theorem. 4 Acknowledgement. We would like to thnk Don Coppersmith for simplifying the proof of Theorem 2.1 nd M. Klwe for helpful remrks. References
6 N. Alon nd Y. Azr [1] N. Alon nd Y. Azr, Sorting, pproximte sorting nd serching in rounds, SIAM J. Discrete Mth. 1 (1988) pp. 269 280. [2], Prllel comprison lgorithms for pproximtion problems, Proc. 29th Annul IEEE Symp. on Foundtions of Computer Science, White Plins, New York, 1988, pp. 194 203. Also: Combintoric, in press. [3] M. Ajti, J. Komlós, W.L. Steiger nd E. Szemerédi, Deterministic selection in O(log log n) prllel time, Proc. 18th Annul ACM Symp. on Theory of Computing, Berkeley, CA, 1986, pp. 188 195. [4] A. Aggrwl, M. Klwe, S. Morn, P.Shor nd R. Wilber, Geometric pplictions of mtrix serching lgorithm, Algorithmic 2 (1987) pp. 195 208. [5] A. Aggrwl nd J. Prk, Notes on serching in multidimensionl monotone rrys, Proc. 29th Annul IEEE Symposium on Foundtions of Computer Science 1988 pp. 497 512. [6] M. Attllh nd R. Kosrju, An efficient Prllel Algorithm for the row minim of totlly monotone mtrices, Proc. 2nd Annul ACM-SIAM Symposium on Discrete Algorithms, 1991 pp. 394 403. [7] Y. Azr nd N. Pippenger, Prllel selection, Discrete Applied Mth 27 (1990) pp. 49 58. [8] B. Bollobás nd G. Brightwell, Grphs whose every trnsitive orienttion contins lmost every reltion, Isrel J. Mth., 59 (1987), 112 128. [9] D. Krvets nd J. Prk, Selection nd sorting in totlly monotone rrys, Proc. 1st Annul ACM-SIAM Symposium on Discrete Algorithms, pp. 494 502. [10] Y. Mnsour, J. Prk, B. Schieber nd S. Sen, Improved selection in totlly monotone rrys, My 1991. [11] N. Pippenger, Sorting nd selecting in rounds, SIAM J. Computing 6 (1987) pp. 1032 1038. [12] R. Wilber, The concve lest-weight subsequence problem revisited, Journl of Algorithms 9 (1988) pp. 418 425.