Lecture 4 Recursive Algorithm Alysis Merge Sort Solvig Recurreces The Mster Theorem
Merge Sort MergeSortA, left, right) { if left < right) { mid = floorleft + right) / 2); MergeSortA, left, mid); MergeSortA, mid+, right); MergeA, left, mid, right); } } // Merge) tes two sorted surrys of A d // merges them ito sigle sorted surry of A // how log should this te?) CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov
Alysis of Merge Sort Sttemet MergeSortA, left, right) { if left < right) { mid = floorleft + right) / 2); MergeSortA, left, mid); MergeSortA, mid+, right); MergeA, left, mid, right); } } So T) = ) whe =, d 2T/2) + ) whe > So wht more succictly) is T)? Effort T) ) ) T/2) T/2) ) CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov
Recurreces The expressio: T ) 2T c 2 c is recurrece. Recurrece: equtio tht descries fuctio i terms of its vlue o smller fuctios CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov
CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov Recurrece Exmples ) ) s c s ) ) s s 2 2 ) c T c T ) c T c T
Solvig Recurreces Itertio method Mster method Sustitutio method CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov
Solvig Recurreces itertio method Expd the recurrece Wor some lger to express s summtio Evlute the summtio We will show severl exmples CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov
Eg. - Lier Serch Recursively Loo t elemet costt wor, c), the serch the remiig elemets T) = T - ) + c The cost of serchig elemets is the cost of looig t elemet, plus the cost of serchig - elemets CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov
Lier Serch cot.) We ll uwid few of these T) = T-) + c ) But, T-) = T-2) + c, from ove Sustitutig c i: T) = T-2) + c + c Gtherig lie terms T) = T-2) + 2c 2) CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov
Lier Serch cot.) Keep goig: T) = T-2) + 2c T-2) = T-3) + c T) = T-3) + c + 2c T) = T-3) + 3c 3) Oe more: T) = T-4) + 4c 4) CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov
Looig for Ptters Note, the itermedite results re eumerted We eed to pull out ptters, to write geerl expressio for the th uwidig This requires prctise Be creful while simplifyig fter sustitutio CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov
Eg. list of itermedites Result t i th uwidig i T) = T-) + c T) = T-2) + 2c 2 T) = T-3) + 3c 3 T) = T-4) + 4c 4 CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov
Lier Serch cot.) A expressio for the th uwidig: T) = T-) + c We hve 2 vriles, d, ut we hve reltio Td) is costt c e determied) for some costt d we ow the lgorithm) Choose y coveiet # to stop. CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov
Lier Serch cot.) Let s decide to stop t T). Whe the list to serch is empty, you re doe is coveiet, i this exmple Let - = => = Now, sustitute i everywhere for : T) = T-) + c T) = T) + c = c + c = O) T) is some costt, c ) CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov
Biry Serch T) T/2) T/4) T/8) Algorithm split rry, the serch lower ½ or upper ½ T) = T/2) + c where c is some costt, the cost of fidig the middle for the split CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov
Biry Serch cot) Let s do some quic sustitutios: T) = T/2) + c ) ut T/2) = T/4) + c, so T) = T/4) + c + c T) = T/4) + 2c 2) T/4) = T/8) + c T) = T/8) + c + 2c T) = T/8) + 3c 3) CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov
Biry Serch cot.) Result t i th uwidig i T) = T/2) + c T) = T/4) + 2c 2 T) = T/8) + 3c 3 T) = T/6) + 4c 4 CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov
Biry Serch cot) We eed to write expressio for the th uwidig i & ) Must fid ptters, chges, s i=, 2,, This c e the hrd prt Do ot get discourged! Try somethig else We ll re-write those equtios We will the eed to relte d CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov
Biry Serch cot) Result t i th uwidig i T) = T/2) + c =T/2 ) + c T) = T/4) + 2c =T/2 2 ) + 2c 2 T) = T/8) + 3c =T/2 3 ) + 3c 3 T) = T/6) + 4c =T/2 4 ) + 4c 4 CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov
Biry Serch cot) After uwidigs: T) = T/2 ) + c Need coveiet plce to stop uwidig eed to relte & Let s pic T) = c So, /2 = => = Hmm. Esy, ut ot useful CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov
Biry Serch cot) Oy, let s cosider T) = c So, let: /2 = => = 2 => = log 2 = lg CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov
Biry Serch cot.) Sustitutig c i gettig rid of ): T) = T) + c lg) = c lg) + c = O lg) ) CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov
T ) Itertio Method T ) f ) for for The itertio method or uwidig the recurrece): T) = f) + T ) = f) + f-) + T 2) = f) + f-) + f-2) + T 3) = = f) + f-) + f-2) + + f3) + f2) + T) = f) + f-) + f-2) + + f3) + f2) + f) + T) = f) + f-) + f-2) + + f3) + f2) + f). f) is the drivig fuctio of the recurrece Exmple: f) = ) T) = 2 ). f) = 2 ) T) = 2 ). CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov
Itertio Method T ) 2T for for 2 T) = + 2T/2) = + 2 [/2 + 2T/2 2 )] = 2 + 2 2 T/2 2 ) = 2 + 2 2 [/2 2 + 2 T/2 3 )] = 3 + 2 3 T/2 3 ) = 3 + 2 3 [/2 3 + 2T/2 4 )] = 4 + 2 4 T/2 4 ) = = + 2 T/2 ) te = + log, T/2 ) = = + log ) = log ) CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov
Itertio Method cotiued) for T ) 2 2T for T) = 2 + 2T/2) = 2 + 2 [/2) 2 + 2T/2 2 )] = 2 + ½) + 2 2 T/2 2 ) = 2 + ½) + 2 2 [/2 2 ) 2 + 2 T/2 3 )] = 2 + ½ + ½ 2 ) + 2 3 T/2 3 ) = 2 + ½ + ½ 2 ) + 2 3 [/2 3 ) 2 + 2T/2 3 )] = 2 + ½ + ½ 2 + ½ 3 ) + 2 4 T/2 4 ) = 2 = 2 + ½ + ½ 2 + ½ 3 + + ½ ) + 2 + T/2 + ) = 2 + ½ + ½ 3 + ½ 5 + + ½ ) te = log, T/2 + ) = = 2 ) geometric decresig = 2 ) CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov
Recursio Tree Method T ) 2T for f ) for 2 T) f) f) T/2) f/2) T/2) f/2) 2f/2) T/4) T/4) T/4) T/4) f/4) f/4) f/4) f/4) T/8) T/8) T/8) T/8) T/8) T/8) T/8) T/8) f/8) f/8) f/8) f/8) f/8) f/8) f/8) f/8) 4f/4) 8f/8) T/6) T ) f ) 2 f log 2 i f 2i 2 ) 4 f 4 ) 8 f ) i CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov 8 T/6)
log 4 log 2 Recursio Tree Method T ) T for T for 4 2 4 4 CLAIM: T) = ). Lower oud: T) W) ovious T) T/4) /4 T/2) /2 3/4 T/6) T/8) T/8) T/4) /6 /8 /8 /4 T/64) T/32) T/32) T/6) T/32) T/6) T/6) T/8) /64 /32 /32 /6 /32 /6 /6 /8 3/4) 2 3/4) 3 T/256) T/6) T ) 3 3 2 3 4 4 4 2 3 3 3 3 4 O ). 4 4 4 3 Upper oud CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov
FACT: Recursio Tree Method T ) T for T for 4 2 This is specil cse of the followig recurrece: T) = T) + T) + ) where < d < re rel costt prmeters. ) T) = ) [lier] 2) T) = log ) 3) T) = d ) [super-lier poly] where d > is the uique costt tht stisfies the equtio d + d =. 4 4 CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov
Solvig Recurreces exmple) s ) s) = c s ) c + s-) c + c + s-2) 2c + s-2) 2c + c + s-3) 3c + s-3) c + s-) = c + s-) T ) T c c CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov
Solvig Recurreces exmple) s ) c s ) So fr for >= we hve s) = c + s-) Wht if =? s) = c + s) = c CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov
Solvig Recurreces exmple) s ) c s ) So fr for >= we hve s) = c + s-) Wht if =? s) = c + s) = c So s ) c s ) Thus i geerl s) = c CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov
Solvig Recurreces exmple) s) s ) = + s-) = + - + s-2) s ) = + - + -2 + s-3) = + - + -2 + -3 + s-4) = = + - + -2 + -3 + + --) + s-) CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov
Solvig Recurreces exmple) s ) s ) CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov s) = + s-) = + - + s-2) = + - + -2 + s-3) = + - + -2 + -3 + s-4) = = + - + -2 + -3 + + --) + s-) = i s ) i
CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov ) ) s s So fr for >= we hve ) s i i Solvig Recurreces exmple)
CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov ) ) s s So fr for >= we hve Wht if =? ) s i i Solvig Recurreces exmple)
CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov ) ) s s So fr for >= we hve Wht if =? ) s i i 2 ) i s i i i Solvig Recurreces exmple)
CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov ) ) s s So fr for >= we hve Wht if =? Thus i geerl ) s i i 2 ) i s i i i 2 ) s Solvig Recurreces exmple)
Solvig Recurreces exmple) T) = T ) 2T c 2 c 2T/2) + c 22T/2/2) + c) + c 2 2 T/2 2 ) + 2c + c 2 2 2T/2 2 /2) + c) + 3c 2 3 T/2 3 ) + 4c + 3c 2 3 T/2 3 ) + 7c 2 3 2T/2 3 /2) + c) + 7c 2 4 T/2 4 ) + 5c 2 T/2 ) + 2 - )c CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov
Solvig Recurreces exmple) T ) 2T So fr for > 2 we hve T) = 2 T/2 ) + 2 - )c Wht if = lg? c 2 c T) = 2 lg T/2 lg ) + 2 lg - )c = T/) + - )c = T) + -)c = c + -)c = 2 - )c CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov
Solvig Recurreces exmple) T) = T/) + c T//) + c/) + c 2 T/ 2 ) + c/ + c 2 T/ 2 ) + c/ + ) 2 T/ 2 /) + c/ 2 ) + c/ + ) 3 T/ 3 ) + c 2 / 2 ) + c/ + ) 3 T/ 3 ) + c 2 / 2 + / + ) T ) T c c T/ ) + c - / - + -2 / -2 + + 2 / 2 + / + ) CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov
Solvig Recurreces exmple) So we hve T) = T/ ) + c - / - +... + 2 / 2 + / + ) For = log = T ) T c c T) = T) + c - / - +... + 2 / 2 + / + ) = c + c - / - +... + 2 / 2 + / + ) = c + c - / - +... + 2 / 2 + / + ) = c / + c - / - +... + 2 / 2 + / + ) = c / +... + 2 / 2 + / + ) CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov
Solvig Recurreces exmple) T ) So with = log T) = c / +... + 2 / 2 + / + ) Wht if =? T) = c + ) = clog + ) = log ) T c c CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov
Solvig Recurreces exmple) T ) So with = log T) = c / +... + 2 / 2 + / + ) Wht if <? T c c CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov
Solvig Recurreces exmple) T ) So with = log T) = c / +... + 2 / 2 + / + ) Wht if <? T c c Recll tht x + x - + + x + ) = x + -)/x-) CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov
CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov So with = log T) = c / +... + 2 / 2 + / + ) Wht if <? Recll tht x + x - + + x + ) = x + -)/x-) So: ) c T c T Solvig Recurreces exmple)
CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov So with = log T) = c / +... + 2 / 2 + / + ) Wht if <? Recll tht x + x - + + x + ) = x + -)/x-) So: T) = c ) = ) ) c T c T Solvig Recurreces exmple)
Solvig Recurreces exmple) T ) So with = log T) = c / +... + 2 / 2 + / + ) Wht if >? T c c CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov
CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov So with = log T) = c / +... + 2 / 2 + / + ) Wht if >? ) c T c T Solvig Recurreces exmple)
CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov So with = log T) = c / +... + 2 / 2 + / + ) Wht if >? T) = c / ) ) c T c T Solvig Recurreces exmple)
CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov So with = log T) = c / +... + 2 / 2 + / + ) Wht if >? T) = c / ) = c log / log ) = c log / ) ) c T c T Solvig Recurreces exmple)
CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov So with = log T) = c / +... + 2 / 2 + / + ) Wht if >? T) = c / ) = c log / log ) = c log / ) recll logrithm fct: log = log ) c T c T Solvig Recurreces exmple)
CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov So with = log T) = c / +... + 2 / 2 + / + ) Wht if >? T) = c / ) = c log / log ) = c log / ) recll logrithm fct: log = log = c log / ) = c log / ) ) c T c T Solvig Recurreces exmple)
CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov So with = log T) = c / +... + 2 / 2 + / + ) Wht if >? T) = c / ) = c log / log ) = c log / ) recll logrithm fct: log = log = c log / ) = c log / ) = log ) ) c T c T
CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov So ) c T c T T log log ) Solvig Recurreces exmple)
The Mster Theorem Give: divide d coquer lgorithm A lgorithm tht divides the prolem of size ito suprolems, ech of size / Let the cost of ech stge i.e., the wor to divide the prolem + comie solved suprolems) e descried y the fuctio f) The, the Mster Theorem gives us coooo for the lgorithm s ruig time: CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov
CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov The Mster Theorem if T) = T/) + f) the W lrge for ) ) / AND ) ) ) ) log ) log log log log log c cf f f f O f f T
Usig The Mster Method T) = 9T/3) + =9, =3, f) = log = log 3 9 = 2 ) Sice f) = O log 3 9 - ), where =, cse pplies: T ) log log whe f ) O Thus the solutio is T) = 2 ) CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov
Solvig Recurreces The sustitutio method CLR 4.) A... the mig good guess method Guess the form of the swer, the use iductio to fid the costts d show tht solutio wors Exmples: T) = 2T/2) + ) T) = lg ) T) = 2T/2) +??? CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov
Solvig Recurreces The sustitutio method CLR 4.) A... the mig good guess method Guess the form of the swer, the use iductio to fid the costts d show tht solutio wors Exmples: T) = 2T/2) + ) T) = lg ) T) = 2T/2) + T) = lg ) T) = 2T/2 )+ 7) +??? CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov
Solvig Recurreces The sustitutio method CLR 4.) A... the mig good guess method Guess the form of the swer, the use iductio to fid the costts d show tht solutio wors Exmples: T) = 2T/2) + ) T) = lg ) T) = 2T/2) + T) = lg ) T) = 2T/2+ 7) + lg ) CS 447, Algorithms, Uiversity College Cor, Gregory M. Prov