CS583 Lecture 02 Jaa Kosecka some materials here are based o E. Demaie, D. Luebke slides
Previously Sample algorithms Exact ruig time, pseudo-code Approximate ruig time Worst case aalysis Best case aalysis
Rules of thumb Multiplicative costats ca be omitted domiates if ; e.g. domiates a b a>b 2 Ay expoetial domiates ay polyomial E.g. domiates Ay polyomial domiates ay logarithm E.g. 3 5 domiates (log ) 3
Today s topics Solvig recurreces Substitutio method Iteratio methods Recursio tree Masters s theorem
Recurrece Methods for solvig recurreces Some examples last time Expadig the reccurrece Recursio tree Techical issues; assume that 2 k
Solvig Recurreces Aother optio is iteratio method - Expad the recurrece - Work some algebra to express as a summatio - Evaluate the summatio We will show several examples
s() c + s(-1) c + c + s(-2) 2c + s(-2) 2c + c + s(-3) 3c + s(-3) kc + s(-k) ck + s(-k) So far for > k we have s() ck + s(-k) What if k? s( ) s() c + s(0) c c + 0 s( 1) > 0 0
s( ) c + 0 s( 1) > 0 0 Thus i geeral s() c
s() s( ) + 0 s( 1) > 0 0 + s(-1) + -1 + s(-2) + -1 + -2 + s(-3) + -1 + -2 + -3 + s(-4) + -1 + -2 + -3 + + -(k-1) + s(-k)
s() + s(-1) + -1 + s(-2) + -1 + -2 + s(-3) + -1 + -2 + -3 + s(-4) + -1 + -2 + -3 + + -(k-1) + s(-k) > + 0 1) ( 0 0 ) ( s s ) ( 1 k s i k i + +
s( ) + 0 s( 1) > 0 0 So far for > k we have i k + 1 i + s( k)
So far for > k we have What if k? > + 0 1) ( 0 0 ) ( s s ) ( 1 k s i k i + +
So far for > k we have What if k? > + 0 1) ( 0 0 ) ( s s ) ( 1 k s i k i + + 2 1 0 (0) 1 1 + + + i s i i i
So far for > k we have What if k? Thus i geeral > + 0 1) ( 0 0 ) ( s s ) ( 1 k s i k i + + 2 1 0 (0) 1 1 + + + i s i i i 2 1 ) ( + s
" c 1 T() # $ 2T( /2) + c >1 T() 2T(/2) + c 2(2T(/2/2) + c) + c 2 2 T(/2 2 ) + 2c + c 2 2 (2T(/2 2 /2) + c) + 3c 2 3 T(/2 3 ) + 4c + 3c 2 3 T(/2 3 ) + 7c 2 3 (2T(/2 3 /2) + c) + 7c 2 4 T(/2 4 ) + 15c. 2 k T(/2 k ) + (2 k - 1)c
So far we have - T() 2 k T(/2 k ) + (2 k - 1)c What if k lg? - T() 2 lg T(/2 lg ) + (2 lg - 1)c T(/) + ( - 1)c T(1) + (-1)c c + (-1)c (2-1)c " c 1 T() # $ 2T( /2) + c >1
o-recursive algorithms - set up a sum for the umber of times the basic operatio is executed - simplify the sum ad determie the order of growth (usig asymptotic otatio) Boudig Fuctios
Substitutio Method Most geeral method for solvig recurreces Guess the form of solutio Verify by iductio Solve for costats Iductio method of mathematical proof to establish a fact for all atural umbers
Iductio Review Show the fact holds for base case, e.g. P(0) is true Form iductive hypothesis: Show that if P(k) holds the it also holds for P(k+1) > this implies that P() holds Example: Show that ( +1) 0 +1+ 2 + + 2
Example Assume that Guess Assume that Prove Example T() 4T( /4) + 4 O( 3 ) T() c 3 T(1) Θ(1) T(k) ck 3 for by iductio k <
Example of substitutio T() 4T( /2) + 4c( /2) 3 + (c /2) 3 + Wheever for example If c 3 ((c /2) 3 ) c 3 (c /2) 3 0 c 2; 1 desired - residual desired
Example cot Hadle iitial coditios, to groud the iductio with the base case Base case 1 0 For, we have if we pick c big eough T(1) Θ(1) for all < 0 Θ(1) c 3 This boud is ot tight!
Tighter upper boud Prove that T() O( 2 ) T() 4T( /2) + 4c( /2) 2 + c 2 + O( 2 ) Wrog!must prove iductive hyp. c 2 ( ) c 2 For o choice of costat
Tighter upper boud Stregthe iductio hypothesis T(k) c k 2 c k 1 2 T() 4T( /2) + 4(c ( /2) 2 c ( /2)) + 1 2 c 2 2c + 1 2 c 2 c (c ) 1 2 2 c 2 c 1 2
Substitutio we ca also guess that Aother strategy: chage of variables T() 2T( ) + lg
Recursio Tree Recursio tree is good for make a iitial guess of the boud Build a recursio tree for
Recursio Tree Example T() T( /4) + T( /2) + 2
Recursio Tree
Masters Method Cookbook method for solvig recurreces of the type T() at( /b) + f ()
Master Theorem If Idea compare the rate of growth of f () with f () grows polyomialy slower the Solutio is T() at( /b) + f () T() Θ( log b a ) log b a log b a
Masters Theorem Idea compare the rate of growth of with log b a f () grows at similar rate the Solutio is T () Θ( log b a lg) log b a f ()
If Master Theorem Idea compare the rate of growth of f () grows polyomialy faster the Solutio is T() at( /b) + f () T() Θ( f ()) f () with log b a log b a Regularity coditio: costat c <1 af ( /b) cf () for some
Master Theorem If T() at( /b) + f ()
Merge Sort Example CASE 2 T() 2T( /2) + c a 2,b 2 log b a log 2 2 k 0 T() Θ( lg )
Examples T() 4T( /2) +
Examples T() 4T( /2) + 2
Examples T() 4T( /2) + 3
Asymptotic Bouds for Some Commo Fuctios Polyomials. a 0 + a 1 + + a d d is Θ( d ) if a d > 0. Polyomial time. Ruig time is O( d ) for some costat d idepedet of the iput size. Logarithms. O(log a ) O(log b 0. ca avoid specifyig the base ) for ay costats a, b > Logarithms. For every x > 0, log O( x ). log grows slower tha every polyomial Expoetials. For every r > 1 ad every d > 0, d O(r ). every expoetial grows faster tha every polyomial
Masters Theorem via recursio tree
Masters Theorem via recursio tree
Masters Theorem via recursio tree
Masters Theorem via recursio tree
Masters Theorem via recursio tree
Biary Search Fid a elemet i the sorted array Divide ad coquer algorithm 1. Divide: Check the middle elemet 2. Coquer: Recursively search oe subarray 3. Combie: Trivial
Biary Search Fid 9 i sorted array 3 5 7 8 9 12 15
Biary Search Recurrece equatio T() 1T( /2) + Θ(1) # of subproblems work dividig ad combiig subproblem size
Biary Search Recurrece equatio T() 1T( /2) + Θ(1) # of subproblems work dividig ad combiig subproblem size Aalysis
Fiboacci Numbers Recursive defiitio F 0 if 0; 1 if 1; F 1 + F 2 if 2 0 1 1 2 3 5 8 13 21 34
Probabilistic Aalysis use of probability theory i the aalysis of algorithms To perform a probabilistic aalysis, we have to make assumptios o the distributio of iputs After such assumptio, we compute a expected ruig time that is computed over the distributio of all possible iputs We will retur to it later
Sortig Cotiued So far we ve talked about two algorithms to sort a array of umbers - What is the advatage of merge sort? - What is the advatage of isertio sort? Next o the ageda: Heapsort - Combies advatages of both previous algorithms
Heaps A heap ca be see as a complete biary tree: 16 14 10 8 7 9 3 2 4 1 What makes a biary tree complete? Is the example above complete?
Heaps A heap ca be see as a complete biary tree: 16 14 10 8 7 9 3 2 4 1 1 1 1 1 1 The book calls them early complete biary trees; ca thik of ufilled slots as ull poiters
Heaps I practice, heaps are usually implemeted as arrays: 16 14 10 A 16 14 10 8 7 9 3 2 4 1 8 7 9 3 2 4 1
Heaps To represet a complete biary tree as a array: - The root ode is A[1] - Node i is A[i] - The paret of ode i is A[i/2] (ote: iteger divide) - The left child of ode i is A[2i] - The right child of ode i is A[2i + 1] 16 14 10 A 16 14 10 8 7 9 3 2 4 1 8 7 9 3 2 4 1
So Referecig Heap Elemets Paret(i) { retur i/2 ; } Left(i) { retur 2*i; } right(i) { retur 2*i + 1; } A aside: How would you implemet this most efficietly? Aother aside: Really?
The Heap Property Heaps also satisfy the heap property: A[Paret(i)] A[i] for all odes i > 1 - I other words, the value of a ode is at most the value of its paret - Where is the largest elemet i a heap stored? Defiitios: - The height of a ode i the tree the umber of edges o the logest dowward path to a leaf - The height of a tree the height of its root
Heap Height What is the height of a -elemet heap? Why? This is ice: basic heap operatios take at most time proportioal to the height of the heap