Sorting Algorithms. Algorithms Kyuseok Shim SoEECS, SNU.

Transcription

1 Sortig Algorithms Algorithms Kyuseo Shim SoEECS, SNU.

2 Desigig Algorithms Icremetal approaches Divide-ad-Coquer approaches Dyamic programmig approaches Greedy approaches Radomized approaches

3 You are ot goig to wi at everythig i life. Go out there ad do your best. Whe it is over, cogratulate the wier-- if it's ot you! By Gail Devers's father

4 You are ot goig to wi at everythig i life. Go out there ad do your best. Whe it is over, cogratulate the wier-- if it's ot you! By Gail Devers's father (Gail wo a goldmedal at the 99 Olympic)

5 Icremetal Approach Sortig: Permutig a sequece of umbers ito ascedig order Isertio Sort Algorithm Wors the way may people sort a had of playig cards Start with a empty left had ad cards face dow o the table Remove oe card at a time from the table ad isert it ito the correct positio i the left had To fid a correct positio for a card, we compare it with each of the cards already i the had from right to left At all times, the cards held i the left had are sorted 5

6 Isertio Sort Algorithm Sortig: Permutig a sequece of umbers ito ascedig order Cosists of N- passes For pass p = through N-, it esures that the elemets i positio 0 through p are i sorted order. Use the fact that the elemets 0 through p- are already ow to be i sorted order. It uses a icremetal approach! 6

7 Isertio Sort Algorithm INSERTION-SORT(A) SORT(A) for j<- to legth[a] do ey <- A[j] Isert A[j] ito the sorted sequece A[..j- ] i <- j - 5 while i>0 ad A[i]>ey 6 do A[i+] <- A[i] 7 i <- i- 8 a[i+] <- ey 7

8 Isertio Sort Algorithm Origial After j = After j = After j = After j = 5 After j = Positio Moved 0 8

9 Order of Growth Rate of growth of the ruig time really iterests us Thus, we oly cosider the leadig term of a formula sice the lower-order terms are relatively isigificat for large We also igore the leadig term's costat coefficiet sice costat factors are less sigificat tha the arte of growth Thus, we write that isertio sort has a worst case ruig time of Θ( ) 9

10 Order of Growth We usually cosider oe algorithm to be more efficiet tha aother if its worstcase ruig time has a lower order of growth Due to costat factors ad lower-order terms, this evaluatio may be i error for small iputs But for large eough iputs, a Θ( ) algorithm, for example, will ru quicly i the worst case tha a Θ( ) algorithm 0

11 Iversios Give, 8, 6, 5,, We have 9 iversios: (, 8), (, ), (, ), (6, 5), (6, ), (6, ), (5, ), (5, ), (, ) Swappig two adjacet elemets (that are out of place) removes oe iversio Thus, this is exactly the umber of swaps that eed too be (implicitly) performed by isertio sort

12 Isertio Sort Algorithm THEOREM 7. The average umber of iversio i a array of N distict elemets is N(N-)/. Proof:

13 Isertio Sort Algorithm THEOREM 7. The average umber of iversio i a array of N distict elemets is N(N-)/. Proof: For ay list L, cosider L, the list i reverse order. Cosider ay pair of two elemets i the list (x,y), with y > x. I exactly oe of L ad L, this ordered pair represets a iversio The total umber of these pairs i a list L ad its reverse L is N(N-)/. Thus, a average list has half this amout.

14 Isertio Sort Algorithm THEOREM 7. Ay algorithm that sorts by exchagig adjacet elemets requires Ω( ) Proof: Each swap removes oly oe iversio, so Ω( ) swaps are required.

15 Divide ad Coquer This is more tha just a military strategy It is also a method of algorithm desig that has created such efficiet algorithms as Merge Sort, Quic Sort I terms or algorithms, this method has three distict steps: Divide: If the iput size is too large to deal with i a straightforward maer, divide the data ito two or more disjoit subsets. Recurse: Use divide ad coquer to solve the subproblems associated with the data subsets. Coquer: Tae the solutios to the subproblems ad merge these solutios ito a solutio for the origial problem. 5

16 Merge Sort Divide: If S has at least two elemets, remove all the elemets from S ad put them ito two sequeces, S ad S, each cotaiig about half of the elemets of S. (i.e. S cotais the first / elemets ad S cotais the remaiig / elemets. Recurse: Recursive sort sequeces S ad S. Coquer: Merge the sorted sequeces S ad S ito a uique sorted sequece S. 6

17 Merge(A,p,q,r) <- q p + <- r q create arrays L[.. +] ad R[.. +] for i <- to 5 do L[i] <- A[p+i-] 6 for j <- to 7 do R[j] <- A[q+j] 8 L[ +] <- 9 R[ +] <- 7

18 Merge(A,p,q,r) 0 i <- j <- for <- p to r do if L[i] <= R[j] the A[] <- L[I] 5 i <- i + 6 else A[] <- R[j] 7 j <- j + Loop Ivariat: At the start of each iteratio for the for-loop above, the subarray A[p..-] cotais the -p smallest elemets of L[.. +] ad R[.. +], i sorted order. Moreover, L[I] ad R[j] are the smallest elemets of their arrays that have ot bee copied bac ito A. 8

19 9 ) b ( ) a ( L A i R j 5 L A I R j 5

20 0 L A 8 i ( d ) R 5 j L A 8 i ( 5 c ) R j 5

21 ) f ( ) e ( L A 8 9 i 5 0 R 5 j L A 8 i R 5 j

22 A 5 ` L 5 R i j ( g )

23 Merge Sort Tree Recursively Divide Merge 85 Merge Q: How deep is this tree? Q: How much memory is eeded for merge sort?

24 Merge Sort MergeSort(A,p,r) if p < r the q = floor((p+r)/) MergeSort(A,p,q) MergeSort(A,q+,r) 5 Merge(A,p,q,r) Merge() is the procedure to merge two sorted lists.

25 Merge Sort Aalysis ) log ( log ) ( log () = ) ( ) ( = ) ( = () : equatio Recurrece O T T T T T T = + = + +

26 Merge Sort Mergig two half arrays S, S ito a full array S requires three poiters, oe for S, aother for S, ad the other for S. The formal aalysis result coicides with the ituitive cout of the big Oh, amely, the area tae by the merge sort tree. The amout of memory eeded for merge sort A extra array 6

27 Sortig Algorithms i Geeral Sortig: Permutig a sequece of umbers ito ascedig order O( ) Sortig Algorithms: Isertio Sort, Bubble Sort O(log) Sortig Algorithms Heap Sort: Based o Heap data structure Quic Sort: Widely regarded as the fastest algorithm Merge Sort: Stable algorithm; if two elemets have the same value, the their relative positio after sortig is the same Is it possible to sort faster tha O(log) time? Ay compariso-based sortig must mae at least O(log) Comparisos i the worst-case Liear-Time sortig algorithms for SMALL itegers

28 Quic Sort(cot.) Give a array A[...r] Divide: The array A[...r] is partitioed ito two oempty subarrays A[...p-] ad A[p+...r] aroud the pivot A[p] such that all elemets i A[...p-] <= A[p] <= all elemets i A[p+...r]

29 Partitio(A,p,r) x A[r] i p for j p to r do if A[j] x 5 the i i + 6 exchage A[i] A[j] 7 exchage A[i+] A[r] 8 retur i + 9

30 Quic Sort Coquer: Each of A[...p] ad A[p+...r] are sorted by recursive calls to Quic sort Quicsort(A,,r) if ( >= r) retur; p Partitio(A,,r); Quicsort(A,,p-); Quicsort(A,p+,r); 0

31 Quic Sort: Partitio Shaded regio: ot yet partitioed, white regio: Partitioed First, choose the pivot somehow, let s say, it is A[0]=5. Secod, Move the pivot at the ed of the array. Move i to the right util fidig the elemet > the pivot, ad Move j to the left util fidig the elemet < the pivot. A[...r] Pivot i j i j i j A[p+...r] A[...p-] i j i i j j Fially, swap the pivot with the i-th elemet

32 Performace of Quic Sort T( ) = T( i) + T( i ) + ( Y( 0)) = T( ) = 0) Performace depeds o the selectio of pivot worst- case partitioig : divide - ad elemet T( ) = T( - ) + = T( - ) + ( - ) + = T( ) + i + i = O( ) = T( ) = T( ) + = T( ) + = O( log ) best - case partitioig : divide ad elemets

33 Performace of Quic Sort- Cot. Average-case partitioig: Assume that the size of a partitio is equally liely ( that is probability is The average value of T( i) of T( - i - ) is T( j) T( ) = [ T( j)] + j= 0 ) j= 0 We already ow T( ) = O( log) from the average case aalysis of ubalaced biary search tree This average performace requires good selectio of pivot! Media-of-Partitioig: tae the media of the left, right, ad ceter elemets i A[l...r]

34 Selectio Problem Iput: A set of distict umbers ad a umber i with <= i <= x A Output: The elemet that is larger tha exactly i- other elemets of A O( log ) Ca be solved i time by sortig

35 Quic Selectio Algorithm Fid the -th smallest elemet Pic a pivot v i S. Partitio S {v} ito S ad S If <= S, the -th smallest elemet must be i S If = + S, we got the aswer Otherwise, the -th smallest elemet lies i S ad it is (- S -)st smallest elemet i S. 5

36 RadomizedSelect(A,p,r,i) if p = r the the retur A[p] q <- RadomizedPartitio(A,p,r) <- q p + if i = the retur A[q] else if i < else retur RadomizedSelect(A,p,q-,i) retur RadomizedSelect(A,q+,r,i-) 6

37 RadomizedSelect(A,p,r,i) Worst-case ruig time is Average case: RadomizedSelect is equally liely to retur ay elemet as the pivot For each s.t. <= <=, the subarray A[p..q] has elemets with probability / X = I {subarray A[p..q] has exactly elemets E[X] = / Θ( ) T ( ) = X ( T (max(, )) + ) = = X T (max(, )) + ) 7

38 8 RadomizedSelect(A,p,r,i) ) / / ( / / )] ( [ ) ( / ))], (max( [ ) (/ ))], (max( [ ] [ ))], (max( [ ] )), (max( [ )] ( [ / a c c c a c c E T E T E T X E T X E T X E E T = = + = + = + = + = = = = =

39 Quic Selectio Algorithm Oe recursive call cotrast to the quicsort algorithm Worst case: Θ( ) Average time complexity: O() 9