Rates of Covergece for Quicksort Ralph Neiiger School of Computer Sciece McGill Uiversity 480 Uiversity Street Motreal, HA 2K6 Caada Ludger Rüschedorf Istitut für Mathematische Stochastik Uiversität Freiburg Eckerstr. 7904 Freiburg Germay February 5, 2002 Abstract The ormalized umber of key comparisos eeded to sort a list of radomly permuted items by the Quicksort algorithm is kow to coverge i distributio. We idetify the rate of covergece to be of the order Θl)/) i the Zolotarev metric. This implies several l)/ estimates for other distaces ad local approximatio results as for characteristic fuctios, for desity approximatio, ad for the itegrated distace of the distributio fuctios. AMS subject classificatios. Primary: 60F05, 68Q25; secodary: 68P0. Key words. Quicksort, aalysis of algorithms, rate of covergece, Zolotarev metric, local approximatio, cotractio method. Itroductio ad mai result The distributio of the umber of key comparisos X of the Quicksort algorithm eeded to sort a array of radomly permuted items is kow to coverge after ormalizatio i distributio as ; see Régier [9], Rösler [0]. Recetly, some estimates for the rate were obtaied by Fill ad Jaso [4], who roughly speakig get upper estimates O /2 ) for the covergece i the miimal L p -metrics l p, p, ad O /2+ε ) for the Kolmogorov metric for all ε > 0 as well as the lower estimates Ωl)/) for the l p metrics, p 2, ad Ω/) for the Kolmogorov metric. After presetig their results at The Seveth Semiar o Aalysis of Algorithms o Tatihou i July, 200, some idicatio was give at the meetig that Θl)/) might be the right order of the rate of covergece for may metrics of iterest. I this ote we cofirm this cojecture for the Zolotarev metric ζ. Sice ζ serves as a upper boud for several other distace measures this implies l)/ bouds as well for some local metrics, for characteristic fuctios, ad for weighted global metrics. For the proof we use a form of the cotractio method as developed i Rachev ad Rüschedorf [8] ad Cramer ad Rüschedorf []. We establish explicit estimates to idetify the rate of covergece. The paper is orgaized as follows: I this sectio we recall some kow properties of the sequece X ), itroduce the Zolotarev metric ζ, ad state our mai theorem, which is proved i sectio 2. Research supported by NSERC grat A450 ad the Deutsche Forschugsgemeischaft.
I the last sectio implicatios of the ζ covergece rate are draw based o several iequalities betwee probability metrics. The sequece of the umber of key comparisos X ) eeded by the Quicksort algorithm to sort a array of radomly permuted items satisfies X 0 = 0 ad the recursio X D = XI + X I +,, ) where = D deotes equality i distributio, X k ), X k ), I are idepedet, I is uiformly distributed o {0,..., }, ad X k X k, k 0, where also deotes equality of distributios. The mea ad variace of X are exactly kow ad satisfy E X = 2 l) + 2γ 4) + Ol)), VarX ) = σ 2 2 2 l) + O), where γ deotes Euler s costat ad σ := 7 2π 2 / > 0. We itroduce the ormalized quatities Y 0 := 0 ad Y := X E X,, which satisfy, see Régier [9], Rösler [0], a limit law Y Y i distributio as. Rösler [0] showed that Y satisfies the distributioal fixed-poit equatio Y D = UY + U)Y + gu), 2) where Y, Y, U are idepedet, Y Y, U is uiform [0, ] distributed, ad gu) := + 2u lu) + 2 u) l u), u [0, ]. Moreover this idetity, subject to E Y = 0, characterizes Y, ad covergece ad fiiteess of the momet geeratig fuctios hold see Rösler [0] ad Fill ad Jaso [2]). We will use subsequetly that VarY ) = σ 2 ad Y <, where Y p := E Y p ) /p, p <, deotes the L p -orm. The purpose of the preset ote is to estimate the rate of the covergece Y Y. Our basic distace is the Zolotarev metric ζ give for distributios LV ), LW ) by ζ LV ), LW )) := sup f F E fv ) E fw ), where F := {f C 2 R, R) : f x) f y) x y } is the space of all twice differetiable fuctios with secod derivative beig Lipschitz cotiuous with Lipschitz costat. We will use the short otatio ζ V, W ) := ζ LV ), LW )). It is well kow that covergece i ζ implies weak covergece ad that ζ V, W ) < if E V = E W, E V 2 = E W 2, ad V, W <. The metric ζ is ideal of order, i.e., we have for T idepedet of V, W ) ad c 0 ζ V + T, W + T ) ζ V, W ), ζ cv, cw ) = c ζ V, W ). For geeral referece ad properties of ζ we refer to Zolotarev [2] ad Rachev [6]. Our mai result states: Theorem. The umber of key comparisos X ) eeded by the Quicksort algorithm to sort a array of radomly permuted items satisfies ) ) X E X l) ζ VarX ), X = Θ, ), where X := Y/σ is a scaled versio of the limitig distributio give i 2). For related results with respect to other distace measures see sectio. 2
2 The proof I the followig lemma we state two simple bouds for the Zolotarev metric ζ, for which we do ot claim origiality. The upper boud ivolves the miimal L -metric l give by l p LV ), LW )) := l p V, W ) := if{ V W p : V V, W W }, p. ) Lemma 2. For V, W with idetical first ad secod momet ad V, W <, we have E V E W ζ V, W ) V 2 6 2 + V W + W 2 ) l V, W ). Proof: The left iequality follows from the fact that we have f F for fx) := x /6, x R. For the right iequality we use the estimate ζ V, W ) /2)κ V, W ), see Zolotarev [, p. 729], where κ deotes the third differece pseudomomet, which has the represetatio see Rachev [6, p. 27]) κ V, W ) = if { E V W : V V, W W }. From V W = V 2 + V W + W 2 V W ad Hölder s iequality we obtai E V W V 2 + V W + W 2 /2 V W V 2 + V W + W 2 ) V W. Takig the ifimum we obtai the assertio. Proof of Theorem.: First we prove the easier lower boud, where oly iformatio o the momets of X ) is eeded. Throughout we use costats σ) 0 defied by σ 2 ) := VarY ) = σ 2 2 l) ) + O. 4) Lower boud: By Lemma 2. we have the basic estimate ) X E X ζ VarX ), X ) ) 6 E σ) Y E σ Y. The third momet of Y satisfies E Y = E X E X ) = κ X ) = M + O ), with M = E Y = 6ζ) 9 > 0, where we use the expasio of the third cumulat κ X ) of X give by Heequi [5, p. 6]. From 4) we obtai σ ) = σ + ) l) σ 5 + O, thus 6 E which gives the lower estimate of the theorem. ) ) σ) Y E σ Y = M l) 2σ 5 + O ),
Upper boud: The scaled variates Y satisfy the modified recursio D I Y = Y I + I Y I + g I ),, 5) where, as i ), Y k ), Y k ), I are idepedet, Y k Y k for all k 0, ad g k) := µk) + µ k) µ) + ), with µ) := E X, 0. Furthermore, we defie Z 0 := Z 0 := 0 ad Z := σ) σ Y, Z := σ) σ Y,, where Y, Y are idepedet copies of the limit distributio also idepedet of I. Fially, we defie the accompayig sequece Z) by Z0 := 0, Z D := I Z I + I Z I + g I ),. 6) Note that Y, Z, Z have idetical first ad secod momet ad fiite third absolute momet for all 0, thus ζ -distaces betwee these quatities are fiite. We will show ) l) ζ Y, Z ) = O. 7) From this estimate the upper boud follows immediately sice we have X E X )/ VarX ) = Y /σ), X Z /σ), ad therefore ) X E X ζ VarX ), X = ) l) σ ) ζ Y, Z ) = O, sice σ)) has a ozero limit. For the proof of 7) we use the triagle iequality: ζ Y, Z ) ζ Y, Z ) + ζ Z, Z ). 8) To estimate the first summad ote that for ay radom variables V, W, T we obtai E fv ) E fw ) E E fv ) T ) E fw ) T ) ad that for V, W ) idepedet of S, T ) we have ζ V + S, W + T ) ζ V, W ) + ζ S, T ). This implies usig 5),6), that ζ is ideal of order, ad coditioig o I, ζ Y, Z ) = = 2 k=0 k=0 k ζ k ζ Y k, k ) Z k k=0 k= Y k + k ) k ζ Y k, Z k ) + Y k + g k), k Z k + k ) Z k + g k) + ζ k k Y k, k )) Z k ) ζ Y k, Z k)) ) k ζ Y k, Z k ). 9) 4
We will show below that ζ Z, Z ) = Ol)/). Thus otig that ζ Z, Z ) = 0) there exists a costat c > 0 with The we prove 7) by iductio usig the costat c from 0): ζ Z, Z ) c l),. 0) ζ Y, Z ) c l),. ) Assertio ) holds for =. With 8),9),0) ad the iductio hypothesis we obtai ζ Y, Z ) 2 ) k c lk) + c l) k k= k 2 6c l) + cl) k= l) 6c ) + c = c l). The proof is completed by showig 0): Sice Y has a fiite third absolute momet ad σ)) is bouded, we obtai that the third absolute momets of Z ), Z ) are uiformly bouded, thus by Lemma 2. there exists a costat L > 0 with ζ Z, Z ) Ll Z, Z ),. 2) By defiitio of Z ad the fixed-poit property of Y we obtai the relatio D Z = UZ + U)Z + σ) gu), ) σ with U idepedet of Z, Z ) ad U uiform [0, ] distributed. We may choose I = U ; hece it holds that I / U / poitwise. Replacig Z, Z by their represetatios ) ad 6) respectively we have l Z, Z) I Z I + I I Z I UZ + Z I + g I ) I UZ + U)Z + σ) gu)) σ Z I U)Z + g I ) σ) σ gu). 4) The first ad secod summad are idetical. We have I Z I UZ = σi ) I σ Y σ) σ UY = Y σ σi ) I σ)u ad σi ) I σ)u σi ) σ)) I + σ) I U. 5) 5
The secod summad i 5) is O/) sice σ)) is bouded ad I / U /. For the estimate of the first summad we use ) l) σ 2 ) = σ 2 + R), R) = O, ad obtai for sufficietly large such that σ) σ/2 > 0 σi ) σ)) I = σ 2 I ) σ 2 ) ) I σ) + σi ) 2 σ σ 2 I ) σ 2 ) ) I = 2 I σ 2 + RI ) σ 2 R) ) σ ) l) = O. For the proof of the latter equality we use the triagle iequality for the L -orm as well as the fiiteess of l U. This gives the Ol)/) bouds for the first ad secod summad i 4). The third summad i 4) is estimated by g I ) σ) σ gu) g I ) gu) + σ) σ gu). We have g I ) gu) = Ol)/) sice the maximum orm satisfies g I ) gu) = Ol)/), see, e.g., Rösler [0, Prop..2]. Fially, gu) < sice gu) is bouded ad σ) σ σ2 ) σ 2 = 2 ) l) σ 2 + O. Thus we have l Z, Z ) = Ol)/) which by 2) implies ζ Z, Z ) = Ol)/). Related distaces I the followig we compare several further distaces to ζ ad obtai similar covergece rates for these distaces. We deote the ormalized versio of X by X := X E X VarX ),, ad X as i Theorem.. Furthermore let C > 0 be a costat such that, by Theorem., ζ X, X) C l)/ for.. Desity approximatio Let ϑ be a radom variable with support o [0, ] or [ /2, /2] ad with a desity f ϑ beig three times differetiable o the real lie ad suppose C ϑ, := sup f ) ϑ x) <. x R 6
For radom variables V, W with desities f V, f W let the sup-metric l of the desities be deoted by lv, W ) := ess sup f V x) f W x). x R For ay distributios of V ad W, the radom variables V + hϑ ad W + hϑ have desities with bouded third derivative. The smoothed sup-metric µ ϑ,4 V, W ) := sup h 4 lv + hϑ, W + hϑ), h R with ϑ idepedet of V, W, is ideal of order ad µ ϑ,4 V, W ) C ϑ, ζ V, W ), see Rachev [6, p. 269]. Therefore, from Theorem. we obtai the estimate µ ϑ,4 X, X) CC ϑ, l),. This implies the followig local approximatio results for the desities of the smoothed radom variates: Corollary. For ay sequece h ) of positive umbers ad ay we have ess sup x R f X+h x) f l) ϑ X+h ϑx) CC ϑ,. I particular for h we obtai a l)/ approximatio boud. For a related approximatio result for the desity f X see Theorem 6. i Fill ad Jaso [4]. A global desity approximatio result holds i the followig form. Assume C ϑ,2 := f 2) ϑ := f 2) h 4 ϑ x) dx < 6) for some radom variable ϑ with desity f ϑ twice differetiable o the lie ad with support of legth bouded by oe, which is idepedet of X, X. The the followig holds: Corollary.2 For ay sequece h ) of positive umbers ad ay we have f X+h f ϑ X+h ϑ C C l) ϑ,2. 7) h Proof: Cosider the smoothed total variatio metric ν ϑ, V, W ) := sup h f V +hϑ f W +hϑ, h R with ϑ idepedet of V, W, which is a probability metric, ideal of order, satisfyig ν ϑ, V, W ) C ϑ,2 ζ V, W ), see Rachev [6, p. 269]. Therefore, Theorem. implies the estimate 7). I particular, we obtai a l)/ covergece rate for h. Note that the left-had side of 7) is the total variatio distace betwee the smoothed variables X + h ϑ, X + h ϑ. 7
.2 Characteristic fuctio distaces For a radom variable V deote by φ V t) := E expitv ), t R, its characteristic fuctio ad by χv, W ) := sup φ V t) φ W t) t R the uiform distace betwee characteristic fuctios. We obtai the followig approximatio result. Corollary. For all t R ad for ay we have Proof: We defie the weighted χ-metric χ by φ X t) φ X t) Ct l). 8) χ V, W ) := sup t φ V t) φ W t). t R The χ is a probability metric, ideal of order, satisfyig χ ζ, see Rachev [6, p. 279]. Therefore, 8) follows from Theorem... Approximatio of distributio fuctios I this sectio we cosider the local ad global approximatio of the smoothed) distributio fuctios. We deote by F V the distributio fuctio of a radom variable V. Note that for itegrable V, W we have the well-kow represetatio of the l -metric as defied i ) due to Dall Aglio see Rachev [6, p. 5]) The Kolmogorov metric is deoted by l V, W ) = F V F W. ϱv, W ) := sup F V x) F W x). x R Let ϑ be a radom variate, idepedet of X, X, with desity f ϑ twice cotiuously differetiable ad support of legth bouded by oe, ad C ϑ,2 as i 6). It is kow that X has a bouded desity, see Fill ad Jaso []. We obtai: Corollary.4 For ay sequece h ) of positive umbers we have for ay l X + h ϑ, X + h ϑ) C C l) ϑ,2 h 2, 9) ϱ X + h ϑ, X + h ϑ) C C ϑ,2 + f X ) l) h 2. 20) Proof: Note that ζ = l by the classical Katorovich-Rubistei duality theorem see Rachev [6, p. 09]). Furthermore, betwee ζ = l ad ζ we have the relatio ζ V + ϑ, W + ϑ) C ϑ,2 ζ V, W ), 8
see Zolotarev [2, Theorem 5], if V, W have idetical first ad secod momets. This implies that for all h 0 l V + hϑ, W + hϑ) C hϑ,2 ζ V, W ) = C ϑ,2 h 2 ζ V, W ). 2) The iequality i 2) implies that the smoothed l metric 2) l V, W ) := sup h 2 l V + hϑ, W + hϑ) h R is bouded from above by l 2) V, W ) Cϑ,2 ζ V, W ). With Theorem. this implies 9). For the proof of 20) first ote that f X+hϑ f X < for all h 0. With the stop loss metric d V, W ) := sup E V t) + E W t) + t R we obtai from Rachev ad Rüschedorf [7, 2.0),2.26)] ad Rachev [6, p. 25] which implies the assertio. Cocludig remark ϱx + hϑ, X + hϑ) + f X ) d X + hϑ, X + hϑ) C hϑ,2 + f X ) ζ X, X) = C ϑ,2 h 2 + f X ) ζ X, X), Our results idicate that l)/ is the relevat rate for the covergece Y Y for several atural distaces. We do however have o argumet to decide the order of the rate of covergece i the Kolmogorov metric ϱy, Y ) without smoothig) or i the l p -metrics as cosidered i Fill ad Jaso [4]. Refereces [] Cramer, M. ad L. Rüschedorf 996). Aalysis of recursive algorithms by the cotractio method. Athes Coferece o Applied Probability ad Time Series Aalysis 995, Vol. I, 8. Spriger, New York. [2] Fill, J. A. ad S. Jaso 2000) A characterizatio of the set of fixed poits of the Quicksort trasformatio. Electro. Comm. Probab. 5, 77 84. [] Fill, J. A. ad S. Jaso 2000) Smoothess ad decay properties of the limitig Quicksort desity fuctio. Mathematics ad computer sciece Versailles, 2000), 5 64. Birkhäuser, Basel. [4] Fill, J. A. ad S. Jaso 200) Quicksort asymptotics. Techical Report #597, Departmet of Mathematical Scieces, The Johs Hopkis Uiversity. Available at http://www.mts.jhu.edu/ fill/papers/quick asy.ps 9
[5] Heequi, P. 99) Aalyse e moyee d algorithme, tri rapide et arbres de recherche. Ph.D. Thesis, Ecole Polytechique, 99. Available at http://pauillac.iria.fr/algo/aofa/research/src/heequi.these.ps [6] Rachev, S. T. 99). Probability Metrics ad the Stability of Stochastic Models. Joh Wiley & Sos Ltd., Chichester. [7] Rachev, S. T. ad L. Rüschedorf 990). Approximatio of sums by compoud Poisso distributios with respect to stop-loss distaces. Adv. i Appl. Probab. 22, 50 74. [8] Rachev, S. T. ad L. Rüschedorf 995). Probability metrics ad recursive algorithms. Adv. i Appl. Probab. 27, 770 799. [9] Régier, M. 989). A limitig distributio for quicksort. RAIRO Iform. Théor. Appl. 2, 5 4. [0] Rösler, U. 99). A limit theorem for Quicksort. RAIRO Iform. Théor. Appl. 25, 85 00. [] Zolotarev, V. M. 976). Approximatio of distributios of sums of idepedet radom variables with values i ifiite-dimesioal spaces. Theor. Probability Appl. 2, 72 77. [2] Zolotarev, V. M. 977). Ideal metrics i the problem of approximatig distributios of sums of idepedet radom variables. Theor. Probability Appl. 22, 4 449. 0