On the Best Case of Heapsort

Transcription

1 JOURNAL OF ALGORITHMS 20, ARTICLE NO. 00 On the Best Case of Heapsort B. Bollobas Department of Pure Mathematcs, Unersty of Cambrdge, Cambrdge CB2 TN, Unted Kngdom T. I. Fenner Department of Computer Scence, Brkbeck College, Unersty of London, London WCE 7HX, Unted Kngdom and A. M. Freze Department of Mathematcs, Carnege Mellon Unersty, Pttsburgh, Pennsylana 523 Receved August 4, 99; revsed July 24, 994 Although dscovered some 30 years ago, the Heapsort algorthm s stll not completely understood. Here we nvestgate the best case of Heapsort. Contrary to clams made by some authors that ts tme complexty s On,.e., lnear n the number of tems, we prove that t s actually Onlog Ž n. and s, n fact, approxmately half that of the worst case. Our proof contans a constructon for an asymptotcally best-case heap. In addton, the proof and constructon provde the worst-case tme complexty and an asymptotcally worst-case example for Bottom-up versons of Heapsort. 996 Academc Press, Inc.. INTRODUCTION In spte of ts age, there are stll some aspects of Heapsort Ždscovered by Wllams. whch have not been completely sorted out. Its worst-case performance s reasonably well understood, but the average-case performance remans a mystery Žsee Knuth 7, pp for some emprcal A prelmnary verson of ths paper was frst presented at the Cambrdge Combnatoral Conference n honour of the 75th brthday of Professor Paul Erdos, March 988. Supported by NSF Grant RG Supported by NSF Grant CCR $8.00 Copyrght 996 by Academc Press, Inc. All rghts of reproducton n any form reserved.

2 206 BOLLOBAS, FENNER, AND FRIEZE data on ths subject.. In ths paper, we examne another aspect of Heapsort whch has no obvous answer,.e., the best case. We prove an asymptotcally tght bound on the mnmum number of operatons taken by ths algorthm. Some authors, e.g., Lorn 8, have clamed erroneously that t s lnear, and Wrth 2 makes the comment that Heapsort seems to lke sequences whch are n nverse sorted order. As we wll see, the best case of the algorthm s rather more complcated than ths. Note that we are restrctng our attenton to the case n whch all of the elements are dstnctotherwse, t s easy to see that the best case s when all the elements are dentcal, and then Heapsort runs n lnear tme. ŽWe menton n passng that heap buldng has attracted some attenton lately, e.g., Bollobas and Smon, Freze 3, Gonnet and Munroe 4, Hayward and McDarmd 5, and McDarmd and Reed. 9. We now establsh our notaton. A Ž max. heap s an array H..n of ntegers satsfyng HH 2 for n. We wll for smplcty assume that n 2 k for some postve nteger k, although our results can be generalzed to arbtrary n. As usual, we magne H as representng a Ž complete. bnary tree Tn n whch poston s the parent of postons 2 and 2. In order to be precse, we wll gve a descrpton of Heapsort. ALGORITHM HEAPSORT. begn BUILDHEAP; for n step - untl 2 do begn A: nterchange H and H ; B: HEAPIFY Ž. end end PROCEDURE HEAPIFYŽ w.. begn ; whle w2 do begn let HjmaxH l : l w and l 2, 244; f HH jthen begn C: nterchange H and Hj ; end end j end else w

3 ON THE BEST CASE OF HEAPSORT 207 Snce BUILDHEAP can be mplemented to run n On tme 7, p. 45, we wll not need to dwell on ths aspect of the algorthm. We wll measure the executon tme on a partcular nstance by the number of executons of Statement C. As stated, ths seems to be about half of the number of comparsons needed, because t s necessary to make two comparsons pror to each nterchange at Statement C. There are, however, versons of HEAPIFY whch attempt to make the number of comparsons roughly equal to the number of executons of Statement C, assumng that the nserted element goes down to near the level of the leaves; see Knuth 7, p. 58, ex. 8, also Carlsson 2, and McDarmd and Reed 9. It appears that the nserted element usually does ths n the average case. Wegener 0 dscusses one such verson whch he calls Bottom-up-Heapsort. Our example n Secton 3 provdes an asymptotcally worst-case example for ths and smlar versons of the algorthm. The basc dea s to assume that the nserted element wll become a leaf and dentfy where t would be nserted, movng the larger chld up at each level as before. Ths takes one nterchange and only one Ž nstead of two. comparsons per level. The actual fnal poston of the element wll be somewhere on the path from ths leaf to the root. Fndng ths poston and nsertng the element there s accomplshed by lnear search up ths path from the leaf. Ths takes one comparson and one nterchange per level from the leaf to the fnal poston. The fnal poston s the same for both versons; f ths s at level d and k s as defned n the followng theorem, then the numbers of comparsons and nterchanges to nsert the element are approxmately 2 d and d for the standard algorthm, whereas both are 2k d for the modfed verson. Thus mnmzng d s best-case and worst-case, respectvely. Let Ž H. denote the number of executons of Statement C, startng wth heap H, and let Ž n. denote the mnmum of Ž H. over all heaps of sze n. Our man result s: THEOREM. If n 2 k for some nteger k then Ž n. n lg n OŽ nlg lg n. 2 lg denotes logarthm to the base 2. It s well known Žsee Knuth 7, p. 49. that the maxmum of Ž H. over all heaps of sze n s n lg n On.

4 208 BOLLOBAS, FENNER, AND FRIEZE 2. A LOWER BOUND ON n The lower bound of our theorem, although easy to prove, does not seem to be well known. Ths was dscovered by the authors, and essentally the same result was obtaned ndependently by Wegener 0. We gve a proof here for completeness. When we execute Statement A, the largest element of the heap s put nto ts fnal poston. We wll refer to ths as the value n H beng deleted from the heap and the heap decreasng n sze by one. Round removes the Ž k. th level from the heap. Thus, for example, the Ž n. 2 largest elements are deleted n Round. We wll show that Round requres at least 2 k Ž k 3. executons of Statement C. Ž. Hence, for k 4, k4 n j2 j2 Ý j Ž k52. k 8 5 nlg n n. Ž Clearly, we need only prove Ž. for. Assume now, w.l.o.g., that H, H2,..., Hn s a permutaton of n, 2,..., n 4. We say that s small f Ž n. 2 and large otherwse. Let now ½ 5 n n L t : t,h t 2 2 postons of large elements n levels, 2,..., k 4;.e., L s the set of postons of large elements whch are not leaves. We wll also say that a node n the tree s large when t contans a large element. Now the elements whch are ntally placed n Ht for t L are large, so they are deleted n Round. To accomplsh ths they must be brought to the top of the heap by nterchanges at Statement C. Hence the number of exchanges n Round s at least Ý tl depthž t., Ž. where depth t the number of arcs n the path from t to the root of H.

5 ON THE BEST CASE OF HEAPSORT 209 Observe next that the postons correspondng to L correspond to a subtree rooted at, snce L mples that the parent of s n L. Thus from Knuth 6, pp , t s easy to show that Ý tl depthž t. Llg L 2L. Ž. Ž. Thus and 2 follow once we have shown that k2 L 2. Ž 3. To do ths let,ž. 2,...,Ž n. be the sequence of nodes vsted n an n-order traversal of T Žsee Knuth 6, pp n. Note that f Ž j., j 2, s a leaf of T then Ž j. n s not. In partcular, f Ž j. s a large leaf then Ž j. L. Thus the number of large leaves cannot exceed L, even f Ž. s a large leaf. Snce the k total number of large elements s 2, nequalty Ž. 3 now follows. 3. AN UPPER BOUND ON n We wll now descrbe an example where the number of exchanges Žat Statement C. n Round n lg n, and then Ž nductvely. 4 the number of exchanges overall Ž. 4 8 n lg n 2n lg n. Snce the number of exchanges nvolvng elements n postons n L s already 4n lg n, we must fnd an example where most of the large elements of the lowest level do not fall very far after they are placed at the top of the heap n Statement A. Note that BUILDHEAP wll generally do nothng f the ntal permutaton s n heaporder; so any partcular heap can be constructed by BUILDHEAP. Consder Fg., whch gves some dea of the ntal heap. Here p 0 lg lg n. We assume the element postons are numbered from left to rght and we magne the bottom p levels of Tn dvded nto 2 kp kp subtrees,,...,, M2 Ž where s the rghtmost subtree. 2 M, each subtree havng 2 p elements of whch 2 p are leaves. So the leaves of are frst placed at the top of the heap n Statement A, then those of 2, etc. The labels L or S nsde each trangle ndcate that the correspondng subtree s flled wth Ž mostly. large or Ž all. small elements. The subtrees,,,..., are alternately Ž mostly m flled wth large or flled wth small elements, where k kp 2 2 m. p 2

6 20 BOLLOBAS, FENNER, AND FRIEZE FIG.. Intal heap. The remanng subtrees 2 m,..., M can be flled arbtrarly n a heap-consstent fashon. We further assume that all of the 2 kp elements n the frst k p levels of the tree are large. We now come to the purpose of the subtree labeled W Žthe watng room.. The leaves of the subtrees,,,..., are Ž almost all m large. The leaves of the trees 2, 4,...,2m are, of course, all small. We wll show how to desgn a heap such that Ž n essence. n Round large elements contaned n the leaes of 2, m, drop nto W Ž 4. and the leaes of drop nto the frst p leels of m. Ž , Ths mples that no leaf of,j2m, wll nterfere wth or dsplace j any leaf n a subtree to the left of tself. Gven ths, we see that no matter what happens to the remanng leaves, we wll have acheved the objectve of makng most of the large elements at the lowest level fall a short dstance only.

7 ON THE BEST CASE OF HEAPSORT 2 We do not need to gve a complete descrpton of the ntal heap. It wll suffce for us to gve enough nformaton about ts structure to verfy Ž. 4 and Ž We use a partton of n nto sets M, M, M, M,, 2,..., m 4 0, M, whch contan the large elements, and S Ž n. 2 m, whch contans the small elements. All we need to specfy s that xm, ym,,,0,4 mples x y, Ž 6. xm, ym 0, zm mples x y z. Ž 7. We now fll n more detals n Fg. n order to gve Fg. 2. Frst of all, observe the contents of,, 2,..., m; except for the bottom level, 2 FIG. 2. Intal heap more detaled.

8 22 BOLLOBAS, FENNER, AND FRIEZE these are all n M ; the contents of the bottom levels are revealed n Fg. 3b below. The other trees 2 m,...,m are flled wth elements from S and M m. The contents of W are n M 0, whch mples H 2 M0 for 0,,..., 2 p, as well. Notce that x k p 4 of the bottom postons are not consdered to be part of W. For any poston t n the remander of the tree we specfy that HtMj where j s as large as possble consstent wth heap order. Thus H 3 M, H 5 M M4, H6M M8,H7M, etc. Notce that ths determnes the subsets of our partton. We do not need to be more specfc about the actual contents, of say, other than requrng that consstency wth heap order s mantaned. Havng descrbed the ntal poston, we now descrbe the th poston,, 2,..., m Ž s the ntal poston.. Fgure 3 deals wth odd and Fg. 4 deals wth even. The frst thng to be checked s that Fg. 3a wth s consstent wth Fg. 2. Ž Note that q.. Assume nductvely that Fg. 3 correctly represents the state of the heap p after 2 2 executons of Statements A, B of Heapsort, where s odd. Consder the nserton of the next 2 p elements at the top of the heap Ž see Fg. 3b.; these are the leaves of 2. It should be clear that Ž. 0 The frst k p q g 2 elements n M wll fall all of the way nto 2 and then all the elements on the path XY wll be n M.If any element n M 0 falls to the bottom level then the element t s swapped wth must also be n M 0 so ths does not affect the partton n Fg. 3b. p The next 2 x 2 p q elements n M wll fall nto W and fll t along wth the path QR. 0 The next q g elements n M wll fall down nto 2 and, afterwards, the path from to the root wll contan elements n M only. 2 At ths pont all of the elements n M 0 and M have been deleted from the heap. ŽWe see that the purpose of the x mssng elements n W s to make room for steps Ž. and Ž. above.. p v Then the 2 small elements n the bottom level of 2 wll fall down nto 2 and make all of ts elements small, along wth the element n poston P. The state of the heap s now as n Fg. 4, wth ncreased to. We now consder what happens when s even. It s, n fact, very smlar to the prevous case. The only dfference s that we nsert h small elements nto the bottom row of. Ther purpose s to make sure that the path UV s 2

9 ON THE BEST CASE OF HEAPSORT 23 Ž. Ž. FIG. 3. a Odd ; b contents of the bottom level of, odd. 2

10 24 BOLLOBAS, FENNER, AND FRIEZE Ž. Ž. FIG. 4. a Even ; b contents of the bottom level of, even. 2

11 ON THE BEST CASE OF HEAPSORT 25 flled wth small elements after the bottom levels of 2, 2 have been nserted. By arrangng the M elements of wth the larger ones to the rght, we can assume that any small element that falls to the bottom level wll always be swapped wth another small element. The next pont to consder s what happens at those ponts at whch q 2 ncreases by,.e., when q q. In ths case, h k p q 2 and g k p q 2, so the rghtmost blocks of M 0 n Fgs. 4b and 3b, for and, respectvely, wll be empty. Increasng q also causes x to decrease by,.e., x x. Ths can be acheved by lettng one of the x mssng elements n Fg. 4a be of sze M nstead of small. Observe that f m then q 2 p so that the node R s always above W. The fnal pont to consder n the nserton of level k s what happens to the elements n m,..., M. Bascally, we handle these n a worst-case scenaro as they only contrbute to the error term. Now let the number of executons of Statement C caused by level k. Then m2 p 3 p Ž 2 kp m. 2 p k mk 2. kp2 k2p3 Ž k3p. However, m 2 2 O 2 and so np 2 k 2 k2 OŽ k2 kp. k p 2 nlg n np O 2 n Ž lg n.. 4 We have sad nothng yet about the dsposton of the small elements. At frst sght, t would seem that, snce we have gnored the relatonshp between them, we can assume that after the frst round the heap looks lke a slghtly smaller verson of what has been descrbed and that we can proceed nductvely Žndeed, we proceeded under ths deluson untl t was tme to wrte the paper.. As t turns out, Heapsort s just a lttle bt more complcated. Even though we would lke to gnore t, we have, n fact, learned somethng about the small elements. We know for example that the small elements that were leaves of subtree are smaller on average than the nonleaves of, but they have been placed to the rght n the next subtree. Ths s not how we would lke the heap to be. To fx ths we wll have to assume that after Round the heap looks as n Fg. except that the labels L and S n the trees,, 2,..., M are nterchanged and that p s replaced by p and k s replaced by k. Of course L and S now refer to those elements whch leave the heap n Round 2 and those whch do not. The reader should convnce hmherself

12 26 BOLLOBAS, FENNER, AND FRIEZE that such a structure s consstent wth what we have assumed about Round. In the th step of the analyss of Round 2 we wll concentrate on 2 and 2. Ths does not nclude. Now we have not made any assumptons about the small elements that were n M at the start of Round. It wll be convenent now to assume that they were, n fact, the largest of the small elements, and t s possble to arrange that at the start of Round 2 the 2 p largest elements le on the leftmost path of the heap and n the leftmost part of M. Then durng the frst 2 p nsertons of Round 2 the elements n the leaves of wll drop down the left-hand sde of the heap. We can assume that after these elements have been nserted that the heap looks as n Fg. 2, except that nsde the trangles representng,, 2,..., M we have S n, M n, S n, M n, etc The partton nto M, M 0, M, etc., has the same ntent as that of Round, but of course the elements are smaller than n the prevous partton. The elements n the Ž smaller. watng room come from the left-hand sde of the orgnal heap. Ths s consstent wth what we have assumed about the left-hand sde of the orgnal heap. Thngs wll now work out much as before. The mportant thng s that the elements n 2 are larger than those n 2. Round 2 progresses n an almost dentcal manner to Round, the contents of the bottom rows of the s beng as n Fgs. 3b and 4b but wth p, k replaced by p, k. The next queston s what about Round 3? Fortunately ths s dentcal to Round. Ths s because there s nothng to stop us makng ths assumpton about the small elements of Round 2. Let us, for example, compare the contents of what remans of, 2. The contents of 2 come from what were the leaves of 3 n Round 2. There s nothng to stop us assumng that they are all smaller than those left n. There s also no reason why we cannot assume that what s now n s greater than j, j 2. The elements of that caused the complcaton for Round 2 have 2 now all gone. The above analyss can be made to hold for the frst, say p2, levels. After whch there are only onlg Ž n. elements left and these can be treated n a worst-case manner. Consequently, p 2 p2 n Ý n lg n 2np OŽ 2 n lg n. O 2 n lg n 2 2 nlg n OŽ nlg lg n., 2 as clamed.

13 ON THE BEST CASE OF HEAPSORT 27 ACKNOWLEDGMENT The authors thank the referee for helpful comments and careful readng of the paper. Note added n proof. The average case performance has recently been obtaned by Schaffer and Sedgewck ŽJournal of Algorthms 5, Ž They also obtaned a constructon for an asymptotcally best-case heap whch s of a smlar type to that contaned n ths paper. REFERENCES. B. Bollobas and I. Smon, Repeated random nserton nto a prorty queue, J. Algorthms 6 Ž 985., S. Carlsson, Average case results on heapsort, BIT 27 Ž 987., A. M. Freze, On the random constructon of heaps, Inform. Process. Lett. 27 Ž 988., G. H. Gonnet and J. I. Munroe, Heaps on heaps, SIAM J. Comput. 5 Ž 986., R. Hayward and C. J. H. McDarmd, Average case analyss of heap buldng by repeated nserton, J. Algorthms 2 Ž 99., D. E. Knuth, The Art of Computer Programmng, Volume, Fundamental Algorthms, AddsonWesley, Readng, MA, D. E. Knuth, The Art of Computer Programmng, Volume 3, Sortng and Searchng, AddsonWesley, Readng, MA, H. Lorn, Sortng and Sort Systems, AddsonWesley, Readng, MA, C. J. H. McDarmd and B. Reed, Buldng heaps fast, J. Algorthms 0 Ž 989., I. Wegener, Bottom-up-Heapsort, a new varant of Heapsort beatng on average Qucksort Ž f n s not too small.., Theoret. Comput. Sc. 8 Ž 993., J. W. J. Wllams, Algorthm 232, Comm. ACM 7 Ž 964., N. Wrth, Algorthms Data Structures Programs, PrentceHall, Englewood Clffs, NJ, 976.