1 Quick Sort LECTURE 7. OHSU/OGI (Winter 2009) ANALYSIS AND DESIGN OF ALGORITHMS

Transcription

1 OHSU/OGI (Winter 2009) CS532 ANALYSIS AND DESIGN OF ALGORITHMS LECTURE 7 1 Quick Sort QuickSort 1 is a classic example of divide and conquer. The hard work is to rearrange the elements of the array A[1..n] so that all elements in A[1..k 1] are less than all elements in A[k..n]. QuickSort(A, i, j) if i < j k Partition(A, i, j) QuickSort(A[i..k 1]) QuickSort(A[k..j]) Partition(A, i, j) Choose a pivot position p such that i p j swap A[j] A[p] x i 1 y j while x < y repeat x x + 1 until x = y or A[x] A[j] repeat y y 1 until x = y or A[y] A[j] if x < y swap A[x] A[y] if x j swap A[x] A[j] return x i _ items < A[j] items yet to be processed x y items > A[j] j 1 C. A. R. Hoare 1

2 2 Complexity of Quick Sort The time to sort the array A[1..n] is T(n) = T(k 1) + T(n k) + Θ(n), where k is the result of Partition(A, 1, n). Best Case If we can could choose the pivot to be the median of the array, we would have and have T(n) = Θ(n log n). T(n) = 2T(n/2) + Θ(n) Worst Case This happens when the partition produces k = 1 or k = n. It gives T(n) = T(n 1) + Θ(n) and implies T(n) = Θ(n 2 ). Bad Case? Suppose the pivot is chosen so that it always splits the input array into parts the sizes of which are in proportion 1:9. Then we have T(n) = T(n/10) + T(9n/10) + Θ(n) The largest array to be processed after k recursive calls has size (9/10) k, so there can be at most log 9/10 n nested recursive calls. It follows that T(n) = O(n log 9/10 n). Not bad! 2

3 Average Case Assuming all partition outcomes are equaly probable (the probability of each is 1/n), we get i.e. T ave (n) = 1 n n (T ave (k 1) + T ave (n k) + Θ(n)) k=1 T ave (n) = Θ(n) + 2 n 1 T ave (k). n k=0 This is t quite easy to solve, but it gives the result T ave (n) = Θ(n logn). 3 Randomization An algorithm is randomized if its behaviour depends t only on its input, but also on the rundom number generator. Example. Suppose the behaviour (but t the result) of your algorithm depends on some parameter k. Suppose you kw that for the random choice of k the algorithm will finish in time t with probability 1/2. Then - with probability 1 (1/2) 2 = 3/4, in two tries, at least one will finish in time t; with probability 1 (1/2) m, in m tries, at least one will finish in time t. For QuickSort, the pivot should be chosen at random. (Often, three random pivot candidates are taken, and the median of the three chosen.) Alternatively, we can modify the input array by a random permutation. Note that randomization does t change the worst case performance. It ensures that you ll end up with the worst case only by pure bad luck. For example, some sorting 3

4 algorithms may perform badly on arrays that are already partially sorted, so it s good to destroy the undesirable structure by a random permutation. 4 Lower Bounds for Sorting The questions is: Are there any sorting algorithms that run in time asymptotically better than O(n log n)? Comparison sorts are algorithms that examine the input data only to make a comparison between a pair of objects. They work the same for integers, reals, strings, etc. All sorting algorithms we ve seen are of this kind. Consider w all possible runs of a comparison sort algorithm on an input set (say array) of fixed size n. When we unwind all loops and recursive calls and focus on the program points when the comparisons are made, we get a picture of a binary decision tree. a _< b a,b,c b _< c b,a,c a _< c a _< c b _< c a,c,b c,a,b b,c,a c,b,a The tree describes all possibilities for the execution of the algorithm on an input of n elements. At the leaves, we have the original input in sorted order. All permutations the input are possible, so there must be at least n! leaves in this tree. A complete tree of height h has 2 h leaves, so our decision tree must have height log 2 (n!). We ve seen in a hw problem that log n! is Ω(n log n). Thus, the number of comparisons in the worst case is Ω(n log n) and we have proved the following. Theorem. The runtime complexity of any comparison 4

5 sort is Ω(n log n). 5 The Zero-One Principle A sorting algorithm takes as input a sequence of comparable objects and outputs a permutation of the same sequence in sorted order. Suppose P is an algorithm that for any input sequence of comparable objects produces a permutation of that sequence and suppose that it examines the objects in the sequence only to make a comparison between a pair of them. Theorem. If P correctly sorts all sequences of 0s and 1s, then it is a correct sorting algorithm for any linearly ordered type (integers, strings,...). Proof. Suppose a = [a 1, a 2,..., a n ] is a sequence (of some type, say integers) that P does t sort correctly. Look at the path in the decision tree corresponding to the execution of P on the input sequence a. Paths from root to leaf determine a permutation of the original sequence, so there is a permutation σ of the set {1, 2,..., n} such that P(a) = [a σ(1), a σ(2),..., a σ(n) ]. Moreover, for any input sequence x = [x 1, x 2,..., x n ] (of any ordered type) we must have P(x) = [x σ(1), x σ(2),...,x σ(n) ] if x satisfies the condition for every i, j : a i a j = x i x j (1) Indeed, the condition ensures that executing P on the input x will follow the same path in the decision tree as the path taken on the input a. Since P does t sort a correctly, there exist indices p, q such that p < q and a σ(p) > a σ(q). Let s define a 0-1 sequence x as follows: { 0 if ai a x i = σ(q) 1 otherwise Note that x σ(p) = 1 and x σ(q) = 0. Also, the sequence x satisfies the condition (1) (why?). Therefore, the values x σ(p) and x σ(q) will occur in the p th and the q th place in 5

6 the sequence P(x), and this sequence will t be in the sorted order. Contradiction. 6