Next. Covered bsics of simple desig techique (Divided-coquer) Ch. of the text.. Next, Strsse s lgorithm. Lter: more desig d coquer lgorithms: MergeSort. Solvig recurreces d the Mster Theorem. Similr ide to multiplictio i N, C Divide d coquer pproch provides uexpected improvemets 5/5/00 CSE 0 Lecture Nïve mtrix multiplictio SimpleMtrixMultiply (A,B). N A.rows. C CreteMtrix(,). for i to 4. for j to 5. C[i,j] 0 6. for k to 7. C[i,j] C[i,j] + A[i,k]*B[k,j] 8. retur C Argue tht the ruig time is θ( ) First ttempt d Divide & Coquer Divide A,B ito 4 / x / mtrices C A B + A B C A B + A B C A B + A B C A B + A B Simple Recursive implemettio. Ruig time is give by the followig recurrece. T() C, d for > T() 8T(/) + θ( ) θ( ) time-complexity Strsse s lgorithm Avoid oe multiplictio (detils o pge 80) (but uses more dditios) Recurrece: T() C, d for > T() 7T(/) + θ( ) How c we solve this? Will see tht T() θ( lg 7 ), lg 7.807. The mximum-subrry problem Give rry of itegers, fid cotiguous subrry with the mximum sum. Very ïve lgorithm: Brute force lgorithm: At best, θ( ) time complexity
C we do divide d coquer? Wt to use swers from left d right hlf subrrys. Problem: The swer my ot lie i either! Key questio: Wht iformtio do we eed from (smller) subproblems to solve the big problem? A divide d coquer lgorithm Algorithm i Ch 4.: Recurrece: T() C, d for > T() T(/) + θ() Relted questio: how do we get this iformtio? T() θ( log ) More divide d coquer : Merge Sort Divide: If S hs t lest two elemets (othig eeds to be doe if S hs zero or oe elemets), remove ll the elemets from S d put them ito two sequeces, S d S, ech cotiig bout hlf of the elemets of S. (i.e. S cotis the first / elemets d S cotis the remiig / elemets). Coquer: Sort sequeces S d S usig Merge Sort. Combie: Put bck the elemets ito S by mergig the sorted sequeces S d S ito oe sorted sequece Merge Sort: Algorithm Merge-Sort(A, p, p, r) r) if if p < r the the q (p+r)/ Merge-Sort(A, p, p, q) q) Merge-Sort(A, q+, q+, r) r) Merge(A, p, p, q, q, r) r) Merge(A, p, p, q, q, r) r) Tke Tke the the smllest of of the the two two topmost elemets of of sequeces A[p..q] d d A[q+..r] d d put put ito ito the the resultig sequece. Repet this, this, util util both both sequeces re re empty. Copy Copy the the resultig sequece ito ito A[p..r].
Merge Sort: summry To sort umbers if doe! recursively sort lists of umbers / d / elemets merge sorted lists i Θ() time Strtegy brek problem ito similr (smller) subproblems recursively solve subproblems combie solutios to swer Recurreces Ruig times of lgorithms with Recursive clls c be described usig recurreces A recurrece is equtio or iequlity tht describes fuctio i terms of its vlue o smller iputs Exmple: Merge Sort solvig_trivil_problem if T ( ) um_pieces T ( / subproblem_size_fctor) + dividig + combiig if > Θ () if T ( ) T ( /) +Θ ( ) if > Solvig recurreces Repeted substitutio method Expdig the recurrece by substitutio d oticig ptters Substitutio method guessig the solutios verifyig the solutio by the mthemticl iductio Recursio-trees Mster method templtes for differet clsses of recurreces Repeted Substitutio Method Let s fid the ruig time of merge sort (let s ssume tht b, for some b). if T( ) T( /) + if > T( ) T( / ) + substitute ( T( /4 ) + / ) + expd T( / 4) + substitute ( T( /8) + / 4) + expd T( /8) + observe the ptter i i T( ) T( / ) + i lg T( / ) + lg + lg
Repeted Substitutio Method The procedure is strightforwrd: Substitute Expd Substitute Expd Observe ptter d write how your expressio looks fter the i-th substitutio Fid out wht the vlue of i (e.g., lg ) should be to get the bse cse of the recurrece (sy T()) Isert the vlue of T() d the expressio of i ito your expressio Substitutio method Solve T( ) 4 T( / ) + ) Guess tht T( ) O( ), i.e., tht T of the form c ) Assume ( ) for / d T k ck k ) Prove ( ) by iductio T c T( ) 4 T( / ) + (recurrece) + Thus T( ) O( )! 4c(/) (id. hypoth.) c + (simplify) c c (rerrge) c if c d (stisfy) Subtlety: Must choose c big eough to hdle T( ) Θ () for < for some 0 0 Substitutio method Achievig tighter bouds T Try to show ( ) ( ) Assume Tk ( ) ck T ( ) 4 T ( /) + + 4( c/) + c O > c for o choice of c 0. Substitutio method The problem: We could ot rewrite the equlity s: T ( ) c+ (somethig positive) T ( ) c i order to show the iequlity we wted Sometimes to prove iductive step, try to stregthe your hypothesis T() (swer you wt) - (somethig > 0) Substitutio method Corrected proof: the ide is to stregthe the iductive hypothesis by subtrctig lower-order terms! Assume Tk ( ) ck ck for k< T ( ) 4 T ( /) + 4( c( / ) c( / )) + c c + c c ( c ) c cif c Recursio Tree A recursio tree is coveiet wy to visulize wht hppes whe recurrece is iterted Costructio of recursio tree T ( ) T ( /4) + T ( /) +
Recursio Tree Recursio Tree T ( ) T ( /) + T( /) + Mster Method The ide is to solve clss of recurreces tht hve the form T ( ) Tb ( / ) + f ( ) Mster method d b >, d f is symptoticlly positive! Abstrctly spekig, T() is the rutime for lgorithm d we kow tht subproblems of size /b re solved recursively, ech i time T(/b) f() is the cost of dividig the problem d combiig the results. I merge-sort T ( ) T ( /) +Θ( ) Split problem ito prts t log b levels. There re leves Mster method Number of leves: Itertig the recurrece, expdig the tree yields log log b T ( ) f ( ) + Tb ( / ) b f + f b + T b ( ) ( / ) ( / ) f + f b + T b + ( ) ( / ) ( / )... + f( / b ) + T() log log log b b b Thus, j j ( ) ( / ) ( ) T fb +Θ j 0 The first term is divisio/recombitio cost (totled cross ll levels of the tree) log b The secod term is the cost of doig ll subproblems of size (totl of ll work pushed to leves) Mster method ituitio Three commo cses: Ruig time domited by cost t leves Ruig time evely distributed throughout the tree Ruig time domited by cost t root Cosequetly, to solve the recurrece, we eed oly to chrcterize the domit term I ech cse compre f ( ) with O ( )
Mster method Cse ( ) ( ε f O ) for some costt ε > 0 f() grows polyomilly (by fctor ε ) slower th log b The work t the lef level domites Summtio of recursio-tree levels O ( ) Cost of ll the leves Thus, the overll cost Θ( ) Θ( ) Mster method Cse f ( ) Θ( lg ) log f ( ) d b re symptoticlly the sme The work is distributed eqully throughout the tree T ( ) Θ( lg ) (level cost) (umber of levels) Mster method Cse ( ) ( +ε f Ω ) for some costt Iverse of Cse f() grows polyomilly fster th log b Also eed regulrity coditio c< d > 0 such tht f( / b) cf( ) > The work t the root domites T ( ) Θ( f ( )) ε > 0 0 0 Mster Theorem Summrized Give recurrece of the form T ( ) Tb ( / ) + f ( ) ε. f( ) O( ) T ( ) Θ( ). f( ) Θ( ) T ( ) Θ( lg) +ε. f ( ) Ω( ) d f( / b) cf( ), for some c<, > 0 T ( ) Θ( f ( )) The mster method cot solve every recurrece of this form; there is gp betwee cses d, s well s cses d Usig the Mster Theorem Extrct, b, d f() from give recurrece log Determie b log b Compre f() d symptoticlly Determie pproprite MT cse, d pply Exmple merge sort T ( ) T ( /) +Θ( ) log b log, b ; Θ( ) Also f( ) Θ( ) Cse : T ( ) Θ lg Θ lg ( ) ( ) Exmples T () T (/) + b log, ; lso f (), f () Θ() Cse : T () Θ(lg) T ( ) 9 T ( /) + 9, b ; f f O ε log 9 ε (), () ( ) with Cs e : T () Θ ( ) Biry-serch(A, p, p, r, r, s): s): q (p+r)/ if if A[q]s the the retur q else else if if A[q]>s the the Biry-serch(A, p, p, q-, q-, s) s) else else Biry-serch(A, q+, q+, r, r, s) s)
Exmples Exmples T ( ) T ( /4) + lg, b 4; log4 0.79 f f log4 +ε ( ) lg, ( ) Ω( ) with ε 0. Cse : Regulrity coditio f ( / b) ( /4)lg( / 4) (/ 4) lg cf ( ) for c / 4 T ( ) Θ( lg ) T ( ) T ( /) + lg log, b ; +ε f ( ) lg, f( ) Ω( ) with ε? lso l g / lg either Cse or Cse! T( ) 4 T( /) + 4, b ; log 4 f f Ω ( ) ; ( ) ( ) Cse : T( ) Θ ( ) Checkig the regulrity coditio 4 f( / ) cf( ) 4 /8 c / c c /4< A quick review of logrithms Properties to remember. log (b) log + log b. log (/b) log - log b. log (/) - log 4. log log 5. log Next. Covered bsics of simple desig techique (Divided-coquer) Ch. 4 of the text.. Next, more sortig lgorithms. It follows tht :. log. + log. log (log )