Analysis of the Expected Number of Bit Comparisons Required by Quickselect

Size: px

Start display at page:

Download "Analysis of the Expected Number of Bit Comparisons Required by Quickselect"

Sarah Harris
6 years ago
Views:

1 Aalysis of the Expected Number of Bit Comparisos Required by Quickselect James Alle Fill Takéhiko Nakama Abstract Whe algorithms for sortig ad searchig are applied to keys that are represeted as bit strigs, we ca quatify the performace of the algorithms ot oly i terms of the umber of key comparisos required by the algorithms but also i terms of the umber of bit comparisos. Some of the stadard sortig ad searchig algorithms have bee aalyzed with respect to key comparisos but ot with respect to bit comparisos. I this exteded abstract, we ivestigate the expected umber of bit comparisos required by Quickselect also kow as Fid. We develop exact ad asymptotic formulae for the expected umber of bit comparisos required to fid the smallest or largest key by Quickselect ad show that the expectatio is asymptotically liear with respect to the umber of keys. Similar results are obtaied for the average case. For fidig keys of arbitrary rak, we derive a exact formula for the expected umber of bit comparisos that usig ratioal arithmetic requires oly fiite summatio rather tha such operatios as umerical itegratio ad use it to compute the expectatio for each target rak. 1 Itroductio ad Summary Whe a algorithm for sortig or searchig is aalyzed, the algorithm is usually regarded either as comparig keys pairwise irrespective of the keys iteral structure or as operatig o represetatios such as bit strigs of keys. I the former case, aalyses ofte quatify the performace of the algorithm i terms of the umber of key comparisos required to accomplish the task; Quickselect also kow as Fid is a example of those algorithms that have bee studied from this poit of view. I the latter case, if keys are represeted as bit strigs, the aalyses quatify the performace of the algorithm i terms of the umber of bits compared util Supported by NSF grat DMS 4614, ad by The Johs Hopkis Uiversity s Acheso J. Duca Fud for the Advacemet of Research i Statistics. Departmet of Applied Mathematics ad Statistics at The Johs Hopkis Uiversity. Departmet of Applied Mathematics ad Statistics at The Johs Hopkis Uiversity. it completes its task. Digital search trees, for example, have bee examied from this perspective. I order to fully quatify the performace of a sortig or searchig algorithm ad eable compariso betwee key-based ad digital algorithms, it is ideal to aalyze the algorithm from both poits of view. However, to date, oly Quicksort has bee aalyzed with both approaches; see Fill ad Jaso [3]. Before their study, Quicksort had bee extesively examied with regard to the umber of key comparisos performed by the algorithm e.g., Kuth [11], Régier [16], Rösler [17], Kessl ad Szpakowski [9], Fill ad Jaso [], Neiiger ad Rüschedorf [15], but it had ot bee examied with regard to the umber of bit comparisos i sortig keys represeted as bit strigs. I their study, Fill ad Jaso assumed that keys are idepedetly ad uiformly distributed over,1 ad that the keys are represeted as bit strigs. [They also coducted the aalysis for a geeral absolutely cotiuous distributio over,1.] They showed that the expected umber of bit comparisos required to sort keys is asymptotically equivalet to l lg as compared to the lead-order term of the expected umber of key comparisos, which is asymptotically l. We use l ad lg to deote atural ad biary logarithms, respectively, ad use log whe the base does ot matter for example, i remaider estimates. I this exteded abstract, we ivestigate the expected umber of bit comparisos required by Quickselect. Hoare [7] itroduced this search algorithm, which is treated i most textbooks o algorithms ad data structures. Quickselect selects the m-th smallest key we call it the rak-m key from a set of distict keys. The keys are typically assumed to be distict, but the algorithm still works with a mior adustmet eve if they are ot distict. The algorithm fids the target key i a recursive ad radom fashio. First, it selects a pivot uiformly at radom from keys. Let k deote the rak of the pivot. If k m, the the algorithm returs the pivot. If k > m, the the algorithm recursively operates o the set of keys smaller tha the pivot ad returs the rak-m key. Similarly, if k < m, the the algorithm recursively oper-

2 ates o the set of keys larger tha the pivot ad returs the k m-th smallest key from the subset. Although previous studies e.g., Kuth [1], Mahmoud et al. [13], Grübel ad U. Rösler [6], Let ad Mahmoud [1], Mahmoud ad Smythe [14], Devroye [1], Hwag ad Tsai [8] examied Quickselect with regard to key comparisos, this study is the first to aalyze the bit complexity of the algorithm. We suppose that the algorithm is applied to distict keys that are represeted as bit strigs ad that the algorithm operates o idividual bits i order to fid a target key. We also assume that the keys are uiformly ad idepedetly distributed i, 1. For istace, cosider applyig Quickselect to fid the smallest key amog three keys k 1, k, ad k 3 whose biary represetatios are , , ad , respectively. If the algorithm selects k 3 as a pivot, the it compares each of k 1 ad k to k 3 i order to determie the rak of k 3. Whe k 1 ad k 3 are compared, the algorithm requires bit comparisos to determie that k 3 is smaller tha k 1 because the two keys have the same first digit ad differ at the secod digit. Similarly, whe k ad k 3 are compared, the algorithm requires 4 bit comparisos to determie that k 3 is smaller tha k. After these comparisos, key k 3 has bee idetified as smallest. Hece the search for the smallest key requires a total of 6 bit comparisos resultig from the two key comparisos. We let µm, deote the expected umber of bit comparisos required to fid the rak-m key i a file of keys by Quickselect. By symmetry, µm, µ1 m,. First, we develop exact ad asymptotic formulae for µ1, µ,, the expected umber of bit comparisos required to fid the smallest key by Quickselect, as summarized i the followig theorem. Theorem 1.1. The expected umber µ1, of bit comparisos required by Quickselect to fid the smallest key i a file of keys that are idepedetly ad uiformly distributed i, 1 has the followig exact ad asymptotic expressios: µ1, H 1 c 1 l l B 1 11 l 1 l O1, H ad B deote harmoic ad Beroulli umbers, respectively, ad, with χ k : πik l ad γ : Euler s costat..577, we defie 1.1 c : γ 9 l 4 l k Z\{} ζ1 χ k Γ1 χ k Γ4 χ k 1 χ k The costat c ca alteratively be expressed as k 1. c 1 k l. k 1 It is easily see that the expressio 1.1 is real, eve though it ivolves the imagiary umbers χ k. The asymptotic formula shows that the expected umber of bit comparisos is asymptotically liear i with the lead-order coefficiet approximately equal to Hece the expected umber of bit comparisos is asymptotically differet from that of key comparisos required to fid the smallest key oly by a costat factor the expectatio for key comparisos is asymptotically. Complex-aalytic methods are utilized to obtai the asymptotic formula; i a future paper, it will be show how the liear lead-order asymptotics µ1, c [with c give i the form 1.] ca be obtaied without resort to complex aalysis. A outlie of the proof of Theorem 1.1 is provided i Sectio 3. We also derive exact ad asymptotic expressios for the expected umber of bit comparisos for the average case. We deote this expectatio by µ m,. I the average case, the parameter m i µm, is cosidered a discrete uiform radom variable; hece µ m, 1 m1 µm,. The derived asymptotic formula shows that µ m, is also asymptotically liear i ; see More detailed results for µ m, are described i Sectio 4. Lastly, i Sectio 5, we derive a exact expressio of µm, for each fixed m that is suited for computatios. Our prelimiary exact formula for µm, [show i.7] etails ifiite summatio ad itegratio. As a result, it is ot a desirable form for umerically computig the expected umber of bit comparisos. Hece we establish aother exact formula that oly requires fiite summatio ad use it to compute µm, for m 1,...,,,..., 5. The computatio leads to the followig coectures: i for fixed, µm, [which of course is symmetric about 1/] icreases i m for m 1/; ad ii for fixed m, µm, icreases i asymptotically liearly. Space limitatios o this exteded abstract force us to omit a substatial portio of the details of our study. We refer the iterested reader to our full-legth paper [4]. k

3 Prelimiaries To ivestigate the bit complexity of Quickselect, we follow the geeral approach developed by Fill ad Jaso [3]. Let U 1,..., U deote the keys uiformly ad idepedetly distributed o, 1, ad let U i deote the rak-i key. The, for 1 i < assume,.1 P {U i ad U are compared} if m i m 1 if i < m < i 1 m i 1 if m. To determie the first probability i.1, ote that U m,..., U remai i the same subset util the first time that oe of them is chose as a pivot. Therefore, U i ad U are compared if ad oly if the first pivot chose from U m,..., U is either U i or U. Aalogous argumets establish the other two cases. For < s < t < 1, it is well kow that the oit desity fuctio of U i ad U is give by f Ui,U s, t : i 1, 1, i 1, 1,. s i 1 t s i 1 1 t. Clearly, the evet that U i ad U are compared is idepedet of the radom variables U i ad U. Hece, defiig P 1 s, t, m, :.3 P s, t, m, :.4 P 3 s, t, m, :.5 m i< 1 i<m< 1 i< m m 1 f U i,u s, t, i 1 f U i,u s, t, m i 1 f U i,u s, t, P s, t, m, : P 1 s, t, m, P s, t, m,.6 P 3 s, t, m, [the sums i.3.5 are double sums over i ad ], ad lettig βs, t deote the idex of the first bit at which the keys s ad t differ, we ca write the expectatio µm, of the umber of bit comparisos required to fid the rak-m key i a file of keys as µm, k l1 βs, tp s, t, m, dt ds s k l 1 k l k l 1 k l 1 k k 1 P s, t, m, dt ds; i this expressio, ote that k represets the last bit at which s ad t agree. 3 Aalysis of µ1, I Sectio 3.1, we outlie a derivatio of the exact expressio for µ1, show i Theorem 1.1; see the full paper [4] for the umerous suppressed details of the various computatios. I Sectio 3., we prove the asymptotic result asserted i Theorem Exact Computatio of µ1, Sice the cotributio of P s, t, m, or P 3 s, t, m, to P s, t, m, is zero for m 1, we have P s, t, 1, P 1 s, t, 1, [see.4 through.6]. Let x : s, y : t s, z : 1 t. The 3.1 P 1 s, t, 1, z 1 i< x i 1 y i 1 z 1 t. From.7 ad 3.1, µ1, i 1, 1, i 1, 1, k k 1 k [l 1 l 1 1 ]. k l1 To further trasform 3., defie B r 1 if r r r a,r 1 if r 1 1 if r,

4 B r deotes the r-th Beroulli umber. Let S, : l1 l 1. The S, 1 r a,r r see Kuth [11], ad 1 µ1, k 1 k k a,r k r 1 r r1 H 1 t, H deotes the -th harmoic umber ad [ B ] 3.5 t : Asymptotic Aalysis of µ1, I order to obtai a asymptotic expressio for µ1,, we aalyze t i The followig lemma provides a exact expressio for t that easily leads to a asymptotic expressio for µ1, : Lemma 3.1. Let γ deote Euler s costat..577, ad defie χ k : πik l. The t H 1 a 1 [ H H 7 ] l γ 1 l 1 H 3 b Σ, a : γ 9 18 l l b : Σ : k Z\{} k Z\{} k Z\{} ζ1 χ k Γ1 χ k Γ4 χ k 1 χ k, ζ1 χ k Γ χ k l 1 χ k Γ3 χ k, ζ1 χ k Γ χ k Γ 1 l 1 χ k Γ 1 χ k, ad H deotes the -th Harmoic umber of order, i.e., H : i1 1 i. The proof of the lemma ivolves complex-aalytic techiques ad is rather legthy, so it is omitted i this exteded abstract; see our full-legth paper [4]. From 3.4, the exact expressio for t also provides a alterative exact expressio for µ1,. Usig Lemma 3.1, we complete the proof of Theorem 1.1. We kow H l γ O 4, H π O 3. Combiig with 3.4 ad Lemma 3.1, we obtai a asymptotic expressio for µ1, : µ1, a 1 l l l 1 l O The term O1 i 3.8 has fluctuatios of small magitude due to Σ, which is periodic i log with amplitude smaller tha.11. Thus, as asserted i Theorem 1.1, the asymptotic slope i 3.8 is c a γ 9 l 4 l k Z\{} ζ1 χ k Γ1 χ k Γ4 χ k 1 χ k. The alterative expressio 1. for c is established i a forthcomig revisio to our full-legth paper [4]; this was also doe idepedetly by Graber ad Prodiger [5]. As described i their paper, suitable use of Stirlig s formula with bouds allows oe to compute c very rapidly to may decimal places. 4 Aalysis of the Average Case: µ m, 4.1 Exact Computatio of µ m, Here we cosider the parameter m i µm, as a discrete radom variable with uiform probability mass fuctio P {m i} 1/, i 1,,...,, ad average over m while the parameter is fixed. Thus, usig the otatio defied i Sectio, µ m, µ 1 m, µ m, µ 3 m,,, for l 1,, 3, µ l m, s βs, t 1 P l s, t, m, dt ds. m1 Here µ 1 m, µ 3 m, by a easy symmetric argumet we omit, ad so 4. µ m, µ 1 m, µ m, ; we will compute µ 1 m, ad µ m, exactly i Sectio

5 4.1.1 Exact Computatio of µ m, We use the followig lemma i order to compute µ 1 m, exactly: Lemma s βs, t 1 P 1 s, t, m, dt ds m 1 1 B Space limitatios o this exteded abstract do ot allow us to prove this lemma here; we give the proof i our full-legth paper [4]. Sice µ 1 m, 1 µ1, 1 1 s βs, t 1 P 1 s, t, m, dt ds, m From , we obtai 4.5 µ m, B 4 B [1 1 ] 1. We rewrite or combie some of the terms i 4.5 for the asymptotic aalysis of µ m, described i the ext sectio. We defie it follows from 3.4 ad Lemma 4.1 that µ 1 m, B B F 1 : F : F 3 : F 4 : F 5 : The , B B [ [1 1 ]. 3 ] [ 1 1, 1 ], Similarly, after laborious calculatios, oe ca show that µ m, [1 1 ] µ m, 1 8 F 1 4 F 4 9 F F 4 8 F Asymptotic Aalysis of µ m, We derive a asymptotic expressio for µ m, show i 4.6.

6 Routie argumets show that F l 4 γ 4.7 l γ 1 O1, 1 F 3 l γ 1 l O1, 4.8 F l ã 1 9 γ 1 8 l O1, F 5 1 l 3 l γ l 1 ã : l O, 1 l l 7 36 l 41 7 γ 1 l ζ1 χ k Γ1 χ k l χ k Γ4 χ k. k Z\{} Sice F is equal to t, which is defied at 3.5 ad aalyzed i Sectio 3., we already have a asymptotic expressio for F. Therefore, from , we obtai the followig asymptotic formula for µ m, : µ m, 41 l ã 4 l l l 1 l O1. The asymptotic slope 41 l ã is approximately We have ot yet sought a alterative form for ã like that for c i Derivatio of a Closed Formula for µm, The exact expressio for µm, obtaied i Sectio [see.7] ivolves ifiite summatio ad itegratio. Hece it is ot a preferable form for umerically computig the expectatio. I this sectio, we establish aother exact expressio for µm, that oly ivolves fiite summatio. We also use the formula to compute µm, for m 1,...,,,...,. As described i Sectio, it follows from equatios.6.7 that µm, µ 1 m, µ m, µ 3 m,,, for q 1,, 3, µ q m, : 5.1 k k l1 l 1 k sl 1 k l k tl 1 k k 1 P q s, t, m, dt ds. The same techique ca be applied to elimiate the ifiite summatio ad itegratio from each µ q m,. We describe the techique for obtaiig a closed expressio of µ 1 m,. First, we trasform P 1 s, t, m, show i.3 so that we ca elimiate the itegratio i µ 1 m,. Defie C 1 i, : I{1 m i < } 5. m 1 i 1, 1, i 1, 1, I{1 m i < } is a idicator fuctio that equals 1 if the evet i braces holds ad otherwise. The 5.3 P 1 s, t, m, C f, h : f1 fm 1 fh im f f h s f t h C f, h, i 1 C 1 i, f i 1 1 h i 1. h f Thus, from 5.1 ad 5.3, we ca elimiate the itegratio i µ 1 m, ad express it usig polyomials i l: 5.4 µ 1 m, fm 1 k l1 f h C 3 f, h k 1 k kfh [l h1 l 1 h1 ] [l 1 f1 l 1 f1 ], C 3 f, h : 1 1f 1 C f, h.,

7 Oe ca show that [ l h1 l 1 ] [ h1 l 1 ] f1 l 1 f1 5.5 fh1 1 1 h C 4 f, h, l 1, h 1 C 4 f, h, : 1 fh 1 1 f f 1 h 1 1 [ 1 From , we obtai f1 ] 1 1. µ 3 m,. I order to derive the aalogous exact formula for µ m,, oe eed oly start the derivatio by chagig the idicator fuctio i C 1 i, [see 5.] to I{1 i < m < } ad follow each step of the procedure; similarly, for µ 3 m,, oe eed oly start the derivatio by chagig the idicator fuctio to I{1 i < m }. Usig the closed exact formulae of µ 1 m,, µ m,, ad µ 3 m,, we computed µm, for, 3,..., ad m 1,,...,. Figure 1 shows the results, which suggest the followig: i for fixed, µm, [which of course is symmetric about 1/] icreases i m for m 1/; ad ii for fixed m, µm, icreases i asymptotically liearly. Expectatio of bit comparisos µ 1 m, fm 1 f h fh1 1 C 5 f, h, k k 1 kfh l 1, k C 5 f, h, : C 3 f, h C 4 f, h,. Here, as described i Sectio 3.1, l1 μm, k 1 l 1 a,r k r, l1 r a,r is defied by 3.3. Now defie 5 m C 6 f, h,, r : a,r C 5 f, h,. The 5.6 C 7 a : fm 1 µ 1 m, C 7 a1 a, f hα fh1 β a1 C 6 f, h,, a f h, i which α : a f 1 ad β : 1 f h a. The procedure described above ca be applied to derive aalogous exact formulae for µ m, ad Figure 1: Expected umber of bit comparisos for Quickselect. The closed formulae for µ 1 m,, µ m,, ad µ 3 m, were used to compute µm, for 1,,..., represets the umber of keys ad m 1,,..., m represets the rak of the target key. 6 Discussio Our ivestigatio of the bit complexity of Quickselect revealed that the expected umber of bit comparisos required by Quickselect to fid the smallest or largest key from a set of keys is asymptotically liear i with the asymptotic slope approximately equal to

8 Hece asymptotically it differs from the expected umber of key comparisos to achieve the same task oly by a costat factor. The expectatio for key comparisos is asymptotically ; see Kuth [1] ad Mahmoud et al. [13]. This result is rather cotrastive to the Quicksort case i which the expected umber of bit comparisos is asymptotically l lg as the expected umber of key comparisos is asymptotically l see Fill ad Jaso [3]. Our aalysis also showed that the expected umber of bit comparisos for the average case remais asymptotically liear i with the lead-order coefficiet approximately equal to Agai, the expected umber is asymptotically differet from that of key comparisos for the average case oly by a costat factor. The expected umber of key comparisos for the average case is asymptotically 3; see Mahmoud et al. [13]. Although we have yet to establish a formula aalogous to 3.4 ad 4.6 for the expected umber of bit comparisos to fid the m-th key for fixed m, we established a exact expressio that oly requires fiite summatio ad used it to obtai the results show i Figure 1. However, the formula remais computatioally complex. Writte as a sigle expressio, µm, is a seve-fold sum of rather elemetary terms with each sum havig order terms i the worst case; i this sese, the ruig time of the algorithm for computig µm, is of order 7. The expressio for µm, does ot allow us to derive a asymptotic formula for it or to prove the two ituitively obvious observatios described at the ed of Sectio 5. The situatio is substatially better for the expected umber of key comparisos to fid the m-th key from a set of keys; Kuth [1] showed that the expectatio ca be writte as [31H mh m 3 mh 1 m ]. I this exteded abstract, we cosidered idepedet ad uiformly distributed keys i,1. I this case, each bit i a bit-strig key is 1 with probability.5. I ogoig research, we geeralize the model ad suppose that each bit results from a idepedet Beroulli trial with success probability p. The more geeral results of that research will further elucidate the bit complexity of Quickselect ad other algorithms. Ackowledgmet. We thak Philippe Flaolet, Svate Jaso, ad Helmut Prodiger for helpful discussios. [] J. A. Fill ad S. Jaso. Quicksort asymptotics. Joural of Algorithms, 44:4 8,. [3] J. A. Fill ad S. Jaso. The umber of bit comparisos used by Quicksort: A average-case aalysis. Proceedigs of the ACM-SIAM Symposium o Discrete Algorithms, pages 93 3, 4. [4] J. A. Fill ad T. Nakama. Aalysis of the expected umber of bit comparisos required by Quickselect [5] P. J. Graber ad H. Prodiger. O a costat arisig i the aalysis of bit comparisos i Quickselect. Preprit, 7. [6] R. Grübel ad U. Rösler. Asymptotic distributio theory for Hoare s selectio algorithm. Advaces i Applied Probability, 8:5 69, [7] C. R. Hoare. Fid algorithm 65. Commuicatios of the ACM, 4:31 3, [8] H. Hwag ad T. Tsai. Quickselect ad the Dickma fuctio. Combiatorics, Probability ad Computig, 11: ,. [9] C. Kessl ad W. Szpakowski. Quicksort algorithm agai revisited. Discrete Mathematics ad Theoretical Computer Sciece, 3:43 64, [1] D. E. Kuth. Mathematical aalysis of algorithms. I Iformatio Processig 71 Proceedigs of IFIP Cogress, Lublaa, 1971, pages North- Hollad, Amsterdam, 197. [11] D. E. Kuth. The Art of Computer Programmig. Volume 3: Sortig ad Searchig. Addiso-Wesley, Readig, Massachusetts, [1] J. Let ad H. M. Mahmoud. Average-case aalysis of multiple Quickselect: A algorithm for fidig order statistics. Statistics ad Probability Letters, 8:99 31, [13] H. M. Mahmoud, R. Modarres, ad R. T. Smythe. Aalysis of Quickselect: A algorithm for order statistics. RAIRO Iformatique Théorique et Applicatios, 9:55 76, [14] H. M. Mahmoud ad R. T. Smythe. Probabilistic aalysis of multiple Quickselect. Algorithmica, : , [15] R. Neiiger ad L. Rüschedorf. Rates of covergece for Quickselect. Joural of Algorithm, 44:51 6,. [16] M. Régier. A limitig distributio of Quicksort. RAIRO Iformatique Théorique et Applicatios, 3: , [17] U. Rösler. A limit theorem for Quicksort. RAIRO Iformatique Théorique et Applicatios, 5:85 1, Refereces [1] L. Devroye. O the probablistic worst-case time of Fid. Algorithmica, 31:91 33, 1.

Analysis of Execution Costs for QuickSelect. Takéhiko Nakama

Analysis of Execution Costs for QuickSelect. Takéhiko Nakama Aalysis of Executio Costs for QuickSelect by Takéhiko Nakama A dissertatio submitted to The Johs Hopkis Uiversity i coformity with the requiremets for the degree of Doctor of Philosophy. Baltimore, Marylad