This Lecture. Divide and Conquer. Merge Sort: Algorithm. Merge Sort Algorithm. MergeSort (Example) - 1. MergeSort (Example) - 2

Transcription

1 This Lecture Divide-ad-coquer techique for algorithm desig. Example the merge sort. Writig ad solvig recurreces Divide ad Coquer Divide-ad-coquer method for algorithm desig: Divide: If the iput size is too large to deal with i a straightforward maer, divide the problem ito two or more disjoit subproblems Coquer: Use divide ad coquer recursively to solve the sub-problems Combie: Take the solutios to the subproblems ad merge these solutios ito a solutio for the origial problem 5//004 5//004 Merge Sort Algorithm Merge Sort: Algorithm Divide: If S has at least two elemets (othig eeds to be doe if S has zero or oe elemets), remove all the elemets from S ad put them ito two sequeces, S ad S, each cotaiig about half of the elemets of S. (i.e. S cotais the first / elemets ad S cotais the remaiig / elemets). Coquer: Sort sequeces S ad S usig Merge Sort. Combie: Put back the elemets ito S by mergig the sorted sequeces S ad S ito oe sorted sequece 5//004 Merge-Sort(A, p, p, r) r) if if p < r the the q (p+r)/ Merge-Sort(A, p, p, q) q) Merge-Sort(A, q+, q+, r) r) Merge(A, p, p, q, q, r) r) Merge(A, p, p, q, q, r) r) Take Take the the smallest of of the the two two topmost elemets of of sequeces A[p..q] ad ad A[q+..r] ad ad put put ito ito the the resultig sequece. Repeat this, this, util util both both sequeces are are empty. Copy Copy the the resultig sequece ito ito A[p..r]. 5//004 4 MergeSort (Example) - MergeSort (Example) - 5// //004 6

2 MergeSort (Example) - MergeSort (Example) - 4 5// //004 8 MergeSort (Example) - 5 MergeSort (Example) - 6 5// //004 0 MergeSort (Example) - 7 MergeSort (Example) - 8 5//004 5//004

3 MergeSort (Example) - 9 MergeSort (Example) - 0 5//004 5//004 4 MergeSort (Example) - MergeSort (Example) - 5// //004 6 MergeSort (Example) - MergeSort (Example) - 4 5// //004 8

4 CHOROCHRONOS Midter Review MergeSort (Example) - 5 5//004 MergeSort (Example) MergeSort (Example) - 7 5//004 Timos Sellis 0 MergeSort (Example) - 8 MergeSort (Example) - 9 5//004 5//004 5//004 MergeSort (Example) - 0 5//

5 MergeSort (Example) - MergeSort (Example) - 5// //004 6 Merge Sort Revisited To sort umbers if doe! recursively sort lists of umbers / ad / elemets merge sorted lists i Θ() time Strategy break problem ito similar (smaller) subproblems recursively solve subproblems combie solutios to aswer 5//004 7 Recurreces Ruig times of algorithms with Recursive calls ca be described usig recurreces A recurrece is a equatio or iequality that describes a fuctio i terms of its value o smaller iputs solvig_trivial_problem if T ( ) um_pieces T ( / subproblem_size_factor) + dividig + combiig if > Example: Merge Sort Θ () if T ( ) T ( /) +Θ ( ) if > 5//004 8 Solvig Recurreces Repeated substitutio method Expadig the recurrece by substitutio ad oticig patters Substitutio method guessig the solutios verifyig the solutio by the mathematical iductio Recursio-trees Master method templates for differet classes of recurreces 5//004 9 Repeated Substitutio Method Let s fid the ruig time of merge sort (let s assume that b, for some b). if T( ) T( /) + if > T ( ) T ( / ) + substitute ( T( / 4 ) + / ) + expad T ( / 4) + substitute + + ( T ( /8) / 4) expad + T ( /8) observe the patter T ( ) i i T ( / ) + i T( / ) + lg + lg lg 5//004 0

6 Repeated Substitutio Method The procedure is straightforward: Substitute Expad Substitute Expad Observe a patter ad write how your expressio looks after the i-th substitutio Fid out what the value of i (e.g., lg) should be to get the base case of the recurrece (say T()) Isert the value of T() ad the expressio of i ito your expressio 5//004 Substitutio method Solve T ( ) 4 T ( / ) + ) Guess that T ( ) O ( ), i.e., that Tof the form c Tk ck k ) Assume ( ) for / ad T c ) Prove ( ) by iductio T ( ) 4 T ( / ) + (recurrece) + Thus ( ) ( )! 4c(/) (id. hypoth.) c + (simplify) c c (rearrage) c if c ad (satisfy) T O Subtlety: Must choose c big eough to hadle T ( ) Θ () for < for some 0 0 5//004 Substitutio Method Achievig tighter bouds Try to show T ( ) O ( ) Assume Tk ( ) ck T ( ) 4 T ( /) + + 4( c/) c + > c for o choice of c 0. Substitutio Method () The problem? We could ot rewrite the equality as: T ( ) c+ (somethig positive) T ( ) c i order to show the iequality we wated Sometimes to prove iductive step, try to stregthe your hypothesis T() (aswer you wat) - (somethig > 0) 5//004 5//004 4 Substitutio Method () Corrected proof: the idea is to stregthe the iductive hypothesis by subtractig lower-order terms! Assume Tk ( ) ck ck for k< T ( ) 4 T ( /) + 4( c ( / ) c ( / )) + c c+ c c ( c ) c cif c Recursio Tree A recursio tree is a coveiet way to visualize what happes whe a recurrece is iterated Costructio of a recursio tree T ( ) T ( /4) + T ( /) + 5// //004 6

7 CHOROCHRONOS Midter Review Recursio Tree () Recursio Tree () T () T ( / ) + T ( / ) + 5// //004 Master Method 8 Master Method () The idea is to solve a class of recurreces that have the form T () at ( / b) + f ( ) a > ad b >, ad f is asymptotically positive! Abstractly speakig, T() is the rutime for a algorithm ad we kow that a subproblems of size /b are solved recursively, each i time T(/b) f() is the cost of dividig the problem ad combiig the results. I merge-sort T () T ( / ) + Θ( ) 5//004 Split problem ito a parts at levels. There are a a leaves 9 5//004 Master Method () Number of leaves: a a Iteratig the recurrece, expadig the tree yields T () f () + at ( / b) f () + af ( / b) + a T ( / b ) f () + af ( / b) + a T ( / b ) a f ( / b ) + a T () T ( ) log b a j f ( / b j ) + Θ( a ) The first term is a divisio/recombiatio cost (totaled across all levels of the tree) log a The secod term is the cost of doig all b subproblems of size (total of all work pushed to leaves) 5// Three commo cases: j 0 Master Method Ituitio Thus, Timos Sellis 4 Ruig time domiated by cost at leaves Ruig time evely distributed throughout the tree Ruig time domiated by cost at root Cosequetly, to solve the recurrece, we eed oly to characterize the domiat term I each case compare f () with O(log a ) b 5//

8 Master Method Case a ε f( ) O( ) for some costat ε > 0 f() grows polyomially (by factor ε ) slower log tha b a The work at the leaf level domiates a Summatio of recursio-tree levels O ( ) a Cost of all the leaves Θ( ) a Thus, the overall cost Θ( ) Master Method Case f ( ) Θ( a lg ) log b a f ( ) ad are asymptotically the same The work is distributed equally a throughout the tree T ( ) Θ( lg ) (level cost) (umber of levels) 5// // Master Method Case ( ) ( a +ε f Ω ) for some costat ε > 0 Iverse of Case log f() grows polyomially faster tha b a Also eed a regularity coditio c< ad > 0 such that af( / b) cf( ) > 0 0 The work at the root domiates T ( ) Θ( f ( )) 5// Master Theorem Summarized Give a recurrece of the form T ( ) atb ( / ) + f ( ) a ε. f( ) O( ) a T ( ) Θ( ) a. f( ) Θ( ) a T ( ) Θ( lg) a+ε. f ( ) Ω( ) ad af( / b) cf( ), for some c<, > 0 T ( ) Θ( f ( )) The master method caot solve every recurrece of this form; there is a gap betwee cases ad, as well as cases ad 5// Strategy Extract a, b, ad f() from a give recurrece Determie log b a Compare f() ad asymptotically Determie appropriate MT case, ad apply Example merge sort T ( ) T ( /) +Θ( ) a b Θ, log b a ; log ( ) Also f( ) Θ( ) log b a ( a ) ( ) Case : T ( ) Θ lg Θ lg 5// Examples T ( ) T ( /) + a b log, ; also f( ), f( ) Θ() Case : T ( ) Θ(lg ) T ( ) 9 T ( /) + a 9, b ; f f O ε log 9 ε ( ), ( ) ( ) with Case : T ( ) Θ ( ) Biary-search(A, p, p, r, r, s): s): q (p+r)/ if if A[q]s the the retur q else else if if A[q]>s the the Biary-search(A, p, p, q-, q-, s) s) else else Biary-search(A, q+, q+, r, r, s) s) 5//004 48

9 Examples () Examples () T ( ) T ( /4) + lg a, b 4; log f f log4 +ε ( ) lg, ( ) Ω( ) with ε 0. Case : Regularity coditio af ( / b) ( /4)lg( / 4) (/ 4) lg cf ( ) for c / 4 T ( ) Θ( lg ) T ( ) T ( /) + lg log a, b ; +ε f ( ) lg, f( ) Ω( ) with ε? also l g / lg either Case or Case! T( ) 4 T( /) + a 4, b ; log 4 f f Ω ( ) ; ( ) ( ) Case : T( ) Θ ( ) Checkig the regularity coditio 4 f( / ) cf( ) 4 /8 c / c c /4< 5// // Next: More sortig algorithms QuickSort HeapSort 5//004 5