A Probabilistic Analysis of Quicksort

Size: px

Start display at page:

Download "A Probabilistic Analysis of Quicksort"

Morgan Baker
5 years ago
Views:

1 A Probabilistic Aalysis of Quicsort You are assumed to be familiar with Quicsort. I each iteratio this sortig algorithm chooses a pivot ad the, by performig comparisios with the pivot, splits the remaider of the iput ito those elemets less tha the pivot ad those elemets greater tha the pivot. It the recurses o two subproblems, sortig the elemets less tha the pivot ad those greater tha the pivot separately. I the worst case, the pivot is always the largest of the elemets to be sorted, the size of the problem decreases by oe i each iteratio, ad there are ( ) comparisios. I the best case, the algorithm always split the iput ito two sets whose size differs by at most oe, ad the algorithm taes fewer tha log comparisios. As you will be ased to show i the assigmet, actually i the best case the algorithm taes at least ( + o()) log comparisios. We are iterested i the variat of quicsort i which each pivot is chose uiformly at radom. I.e. if i each subproblem, each elemet i the list curretly beig sorted is equally liely to be the pivot. As discussed above, the radom variable X which couts the umber of comparisios made by this algorithm ca tae values as as large as ( ) ad smaller tha log. We are iterested i computig E(X), the expected umber of comparisios tae.. Our approach exploits the fact that X is actually equal to the sum of ( ) simple radom variables. Spefically, if for i < j, we let Xi,j be the umber of comparisios the algorithm maes betwee the ith smallest elemet z i ad jth smallest elemet z j of the iput the X = X i,j. Now, the expectatio of the sum is the sum of the expectatios, so E(X i,j ). We ote that we compare two elemets of the list to be sorted i a iteratio, precisely if oe of them is the pivot. I this case, the pivot is ot compared i ay future iteratio ad the two elemets are compared exactly oce. Thus each X i,j is either 0 or ad E(X i,j ) is the probability that z i ad z j are compared.

2 We ote further that while the pivot is either equal to or lies betwee z i ad z j, all of z i, z i+,..., z j remai i the same subproblem. Thus, if o z with i < < j is chose as a pivot before either of z i or z j the they will be compared. O the other had if some z with i < < j is chose as a pivot before either of z i or z j the they will ot be compared. I the first iteratio, we ca choose the pivot uiformly as follows. We first expose whether the pivot is oe of z i, z i+,..., z j or ot (it is with probability j i+ ). If so, we choose it to be z for each i j with probability. j i+ Otherwise we chose it to be each z with < i or > j with probablitiy. I the latter case, we use the same procedure i the subproblem j+i cotaiig z i ad z ad the elemets betwee them. I the aalysis, is replaced by where is the umber of elemets to be sorted i this subproblem. We cotiue i this maer util we use z as a pivot for some i j. We see that the probability that = i or = j is exactly j i+ regardless of the size of the subproblem ad so the probability we compare z i ad z j is exactly j i+. Thus, i= = j i + i = + i= = i= = + Now, it is well ow that = is log + O() we see that E(X) log + O(). This meas that o average quicsort performs o more tha + O() times as may comparisios as it performs i the best case. = =

3 Radomized Algorithms for Selectio The ith order statistic of a set of distict is its ith smallest elemet. Thus the miimum is the first order statistic ad the maximum of a set of elemets is the th order statistic. if is odd the the media of a set of elemets is its + order statistic while if is eve the a set of N elemets has two medias. Its + order statistic is the lower media ad its order statistic is the higher media. The ra of the ith order statistic is i. The selectio problem taes as iput a set A of (distict) elemets ad a iteger r betwee ad = A. Its output is the rth order statistic(r here stads for ra). QuicSelectio Oe way of solvig selectio is to sort the iput set usig Quicsort ad to the simply traverse the ordered set to fid the desired elemet. However, this is much too much wor, as i each iteratio we really eed oly recurse o oe of the two subproblems ot both. We ow describe a algorithm QUICKSELECT which chooses pivots ad splits the problem ito two parts i each iteratio just lie QUICKSORT, but oly recurses o at most oe of the subproblems. Specifically if the pivot is the jth order statistic the if j = r we simply retur the pivot, if j > r we simply retur the rth order statistic i the set of elemets which are less tha the pivot, ad if j < r we simply retur the r jth order statistic i the set of elemets which are greater tha the pivot. Defiiig X ad X i,j as i the aalysis of quicsort, we oce agai have that E(X i,j ) ad that X i,j is the probability p i,j that z i ad z j are compared. But the value of p i,j differs from that calculated i the quicsort aalysis. Specifically, if the elemet we are looig for lies o oe side of the pivot, ad z i ad z j both lie o the other side, the either z i or z j remais i the active problem ad so they will ot be compared. Thus, we eed to cosider the first time that there is a pivot lyig betwee the miimum mi i,j of z i, z r, z j ad the maximum max i,j of these 3 values. The probability p i,j is precisely the probability that whe this occurs the pivot is oe of z i or z j. It 3

4 follows that p i,j is always at most. So i calculatig our sum, max( r i, r j )+ we reidex usig i,j = max( r i, r j ) ad ote that p i,j < i,j + i,j. Now, we ow every i,j lies betwee ad. Furthermore, if i,j = the oe of i or j is r or r + ad the other lies betwee r ad r +. Thus there are at most 4 choices for such a pair. It follows that:. E(X) The Two Pivot Algorithm 4 8 = The expected umber of comparisios made by our ext algorithm for radomized selectio is +o(). It employs a subroutie to fids two pivots p low ad p high with p low < p high usig o() comparisios. It the use comparisios with the pivots to split A p low p high ito three sets: Low = {z z < p low }, High = {z z > p high }, ad Middle = {z p low < z < p + high}. This ca be doe with 4 comparisios, by comparig each o-pivot with each pivot. Now, if z r is oe of the pivots we retur it. Otherwise we proceed as follows. If r is less tha the ra r low of p low We sort Low, ad retur its rth order statistic. If r exceeds the ra r high of p high we sort High ad retur its (r r high )th order statistic. If r low < r < r high we sort Middle ad retur its (r r low )th order statistic. We claim that the probability p fail of the evet that our subroutie does ot retur pivots such that the elemet of {Low, Middle, High} we eed to sort has size at most is o( ). log Now, we ca sort a set of m elemets i O(mlog m) steps. The set we sort has at most elemets. So, the total umber of steps we eed is o() O(( p fail ) log( )) + O(p log log faillog ). If our claim is true, this is + o(). To complete the descriptio ad aalysis of our algorithm, it remais to describe a subroutie which chooses the two pivots i o() time for which the claim holds. We do so ow. The subroutie chooses a subset A of elemets of uiformily at radom (by repeatedly choosig a uiform elemet from the uchose set to add to the chose set util it has the right size), ad sorts A. It sets i low to be r 3 8 ad i high to be r Provided r is at least 7 8 ad at most 7 8 It sets p low to be the ith order statistic of A ad p high to be the i high th order statistic of A. As will be 4

5 discussed i class, well ow results the yield that the probability that the evet r low < r < r high < r low + fails is o( ). Provided this evet holds, log we search Middle which has at most elemets. So the claim holds i log this case. If r is less tha 7 8 It sets p low ad p high both to be the to be the i high th order statistics. As will be discussed i class, similar well ow results the yield that the probability that the evet r < r low < fails is o( ). log Provided this evet holds, we search Low which has at most elemets. log So the claim holds i this case. If r is more tha 7 8 It sets p low ad p high both to be the to be the i low th order statistics. A symmetrical argumet yields that the probability that the evet r > r high > fails is o( ). Provided this evet holds, log we search High which has at most elemets. So the claim holds i this log case. 5

Sorting Algorithms. Algorithms Kyuseok Shim SoEECS, SNU.

Sorting Algorithms. Algorithms Kyuseok Shim SoEECS, SNU. Sortig Algorithms Algorithms Kyuseo Shim SoEECS, SNU. Desigig Algorithms Icremetal approaches Divide-ad-Coquer approaches Dyamic programmig approaches Greedy approaches Radomized approaches You are ot