Reurrees: Methods ad Examples CSE 30 Algorithms ad Data Strutures Alexadra Stefa Uiversity of exas at Arligto Updated: 308
Summatios Review Review slides o Summatios
Reurrees Reursive algorithms It may ot be lear what the omplexity is, by just lookig at the algorithm. I order to fid their omplexity, we eed to: Express the ruig time of the algorithm as a reurree formula. E.g.: f = + f- Fid the omplexity of the reurree: Expad it to a summatio with o reursive term. Fid a oise expressio or upper boud, E, for the summatio. Fid Θ,ideally, or O big-oh for E. Reurree formulas may be eoutered i other situatios: Compute the umber of odes i ertai trees. Express the omplexity of o-reursive algorithms e.g. seletio sort. 3
Commo Reurrees Loal ost Θ Θ 3 Θ 4 Θ 5 Θ Number of subproblems Size of subproblem Desriptio Example - Redue the pb size by i ost time. - Redue the pb size by i liear time. Halve problem i ost time Halve problem i liear time Break ad put bak together the problem ito halves i ost time. 6 Θ Break ad put bak together the problem ito halves i liear time. 4
Solvig Reurrees he Master heorem he Reursio-ree Method Useful for guessig the boud. I will also aept this method as proof for the give boud if doe orretly. he Idutio Method Guess the boud, use idutio to prove it. Note that the book alls this the substitutio method, but I prefer to all it the idutio method 5
he Master Method - ite: CLRS Note that log b a - ε = log b a - ε ad log b a + ε = log b a + ε Solve: 9 3 3 3 4 lg Not appliable for: See CLRS, page 95, for solutio. lg 6
he Master heorem Method 7 3 9 0.5, 9 log log 3 9 3 9 0.5 log 3 ase O f a b a a b b lg lg 3 0 log log 3 3 0 0 0 log 3 ase f a b a a b b Note that log b a - ε = log b a - ε ad log b a + ε = log b a + ε
he Master heorem Method Note that log b a - ε = log b a - ε ad log b a + ε = log b a + ε 3 4 lg a 3 b 4 log 4 f lg 3 0.8 0.80. log, 0. ase3 eed : af b f 3 4lg 4 3 4 lg lg, true 3 4. Need, b 3 4 3 4 lg lg a 0.8 lg a b 3 log f lg lg B.., 0,lg log b a but ot by O lg O * O 8 => Caot apply the Master theorem
Reurrees: Reursio-ree Method. Build the tree & fill-out the table. Compute ost per level 3. Compute umber of levels 4. Compute total over levels. * Fid losed form of that summatio. Example : Solve 3 4 Example : Solve 3 3 O 9
N N- ime omplexity tree: N = N- + fatn it fatit N { if N <= retur ; retur N*fatN-; } ime omplexity of fatn? N = N = N- + = 0 = Levels: N Eah ode has ost => N = *N = ΘN 0
N = N-4 + N N-4 ime omplexity tree: 4 it fat4it N { if N <= retur ; if N == retur ; if N == 3 retur 6 retur N*N-*N-*N-3*fat4N-; } ime omplexity of fat4n? N = N = N-4 + 3 = = = 0 = 0 Levels: N4 Eah ode has ost => N = *N4 = ΘN
N N- ime omplexity tree: - N = N- + N seletio_sort_ren it fatit N{ if N <= retur; idx = mi_idexa,st,n; ΘN-st A[st] <-> A[idx] retur sel_sort_rea,st+,n; } N = N- + N = 0 = Levels: N Node at level i has ost N-i => N = N+N-+i+.. = NN+ = ΘN
N = N + N ime omplexity tree: N N = N + = 0 = Levels: lgn from base ase: N k = => k=lgn Eah ode has ost => N = *lgn = ΘlgN 3
N = N + N N ime omplexity tree: N N N N = N + N = 0 = Levels: lgn from base ase: N k = => k=lgn Node at level i has ost N i => N = N + N + N + N i + + N k = = N + + + i + + k = = N[ + + ½ + ½ i + + ½ k ] = = N*ostat = ΘN 4
Reurree - Reursio ree = a* b + Number of subproblems => Number of hildre of a ode i the reursio tree. => Affets the umber of odes per level. At level i there will be a i odes. Affets the level ost. Relatioship he loal ost at the ode b b a b Size of a subproblem => Affets whe the reursio stops the k for whih we get : Last level, k, will be: k = log b assumig the base ase is for. b b 5
Reursio ree for: = + 4 Base ase: = 4..................... 4 Level Arg pb size Nodes per level ode ost 0 4 4 4 Level ost i i i i... Stop at level k, whe the subtree is. => he problem size is, but the geeral formula for the problem size, at level k is: k => k = => k = lg k=lg = k k = k ree ost = ++ + 3 ++ i ++ k = k+ - = k = = Θ 6
Reursio ree for: = + 4 Base ase: = 4 4 4........................ 4 4 4 Stop at level k, whe the subtree is. => he problem size is, but the geeral formula for the problem size, at level k is: k => k = => k = lg Level Arg pb size Nodes per level ode ost Level ost 0 * * * ** =* 4 4 *4 4**4 =* i i i * i i ** i k=lg = k k = =*= * k =* k ** k =* ree ost * k * lg lg lg 7
Reursio ree for = 3+ 4 4 Base ase: = 4 4 4 4..................... 4 4 4 Level Arg pb size Nodes per level ode ost 0 * * Level ost 3 * 3** =3** 4 9 *4 3 ** i i 3 i * i 3 i **... Stop at level k, whe the subtree is. => he problem size is, but the geeral formula for the problem size, at level k is: k => k = => k = lg k=lg = k 3 k =*= * k 3 k ** 8
otal ree Cost for = 3+ Closed form 3 3 *[ 3 3 lg 3 * 3 use : lg 3* lg lg3 3 [3 *3 lg 3...3... 3 lg3 lg3 lg i ] lg lg3lg 3...3 ] 3*3 lg3 lg i 0 lg3 3 lg i lg3 lg Explaatio: sie we eed Θ, we a elimiate the ostats ad odomiat terms earlier after the losed form expressio: lg 3 lg... * *3 *3 *3 3 use : lg * lg lg3 3 lg3 lg lg3 lg3lg lg3 lg 9
5 Reursio ree for: = 5+ 5 5 5 5 5........................ 5 5 5 5 5 Stop at level k, whe the subtree is. => he problem size is, but the geeral formula for the problem size, at level k is: 5 k => 5 k = => k = log 5 Level Arg pb size Nodes per level ode ost Level ost 0 * * 5 *5 **5 =5* 5 4 *5 4** =5 i i 5 i i *5 i i **5 i k=lg =5 k k = =*= *5 k =5 i k **5 k =5 k ree ost derivatio similar to ost for = 3+ 0
otal ree Cost for = 5+ 5 5... 5 i... 5 log 5 *[ 5 5 log 5 i0 5 * 5 3 5 i... 5 i0 5 O ] Also... i log 5
Other Variatios = 73+ = 73+ 5 Here istead of 73 we will use 73 5 = + he tree beomes a hai oly oe ode per level
Reursio ree for: = 75+ 3 5 Base ase: = 5 5 3 5 3 3... 5 3........................ 5 5 3 5 5 3 Stop at level k, whe the subtree is. => he problem size is, but the geeral formula for the problem size, at level k is: 5 k => 5 k = => k = log 5 Level Arg pb size Nodes per level ode ost 0 3 * 3 Level ost 5 7 5 3 7**5 3 = 3 75 3 5 7 5 3 7 **5 3 = 3 75 3 i 5 i 7 i 5 i 3 7 i **5 i 3 k= log 5 =5 k 7 k =*= 5 k 3 = 3 75 3 i 7 k **5 k 3 = 3 75 3 k Where we used: 7 i * 5 i 3 = 7 i * 3 5 i 3 = 7 i * 3 5 3 i = 3 3 75 3 i
ree Method Draw the tree, otie the shape, see legth of shortest ad logest paths. Notie that: as log as the levels are full all odes have hildre the level ost is the sum of osts of the hildre equals the paret: 3*p_ost+3 *p_ost otal ost for those: *log 3 = Θlg he umber of iomplete levels should also be a multiple of lg ad the ost for eah of those levels will be less tha => Guess that = Olg Use the substitutio method to show = Olg If the reurree was give with Θ istead of O, we ould have show = Θlg with O, de oly kow that: 3+3+ he loal ost ould eve be ostat: = 3+3 + Exerise: Solve = 3+ Θ = 3+ Θ useful: lg3.59 Use them to boud. How does that ompare to the aalysis i this slide? he bouds are looser. 3 3 O 4
Reurrees: Idutio Method. Guess the solutio. Use idutio to prove it. 3. Chek it at the boudaries reursio base ases Example: Fid upper boud for:. Guess that = Olg =>. Prove that = Olg usig <= lg for some. Assume it holds for all m<, ad prove it holds for. 3. Assume base ase boudary: =. Pik ad 0 s.t. it works for suffiiet base ases ad applyig the idutive hypotheses. 5
Reurrees: Idutio Method. Prove that = Olg, usig the defiitio: fid ad 0 s.t. *lg here: f =, g = lg Show with idutio: *lg for some >0 * * *lg lg lg lg : lg * * *lg wat 0 lg 0 lg Pik = the largest of both ad. lg 3. Base ase boudary: Assume = Fid 0 s.t. the idutio holds for all 0. =: = **lg =*0 =0 FALSE. => 0 aot be. =: = * + = +=4 Wat *lg=, rue for: =3: 3=*+3=+3=5 Wat 5=3 *3*lg3 rue for: 6 Pik 0 = 4 s.t. we have 0 whe applyig the idutive hypothesis
Reurrees: Idutio Method Various Issues Makig a good guess Solve: 7 Fid a similar reursio Use looser upper ad lower bouds ad gradually tighte them Subtleties stroger oditio eeded with Use a stroger oditio: off by a ostat, subtrat a ostat Avoidig pitfalls Solve: ad 0 = Wrog: I the above example, stop at + ad olude that =O See also book example of wrog proof for is O Chagig variables Reommeded readig, ot required page 86 7
Stroger Hypothesis for Show = O usig the defiitio: fid ad 0 s.t. * here: f =, g =. Use idutio to show * Idutive step: assume it holds for all m<, show for : We re stuk. We CANNO say that =O at this poit. We must prove the hypothesis exatly: ot: +. Use a stroger hypothesis: prove that -d, for some ost d>0: 8 0 : d d d wat d d d d d
Solve: Use the tree method to make a guess for: Use the idutio method for the origial reurree with roudig dow: 4 3 9 4 3 4 3
Additioal materials 30
Commo Reurrees Review. Redue the problem size by i ostat time: = + Θ N. Redue the problem size by i liear time: = - + Θ N 3. Halve problem i ostat time : = + Θ lgn 4. Halve problem i liear time : = + ΘN ~N 5. Break ad put bak together the problem ito halves i ostat time: = + ΘN ~N 6. Break ad put bak together the problem ito halves i liear time: = + Θ N lgn 3
PratieStregthe uderstadig Look ito the derivatio if we had: = d. I geeral, at most, it affets the ostat for the domiat term. 3
Aswer Look ito the derivatio if we had: = d. At most, it affets the ostat for the domiat term. Level Arg pb size Nodes per level ode ost Level ost 0 * * * ** =* 4 4 *4 4**4 =* i i i * i i ** i =* k=lg = k k = d =d* ree ost k d lg d lg 33
Permutatios without repetitios Harder Example Coverig this material is subjet to time availability ime omplexity ree, ituitio for movig the loal ost i the reursive all ost, math justifiatio idutio 34
More Reurrees Extra material M. Redue the problem size by i logarithmi time E.g. Chek lgn items, elimiate M. Redue the problem size by i N time E.g. Chek N pairs, elimiate item M3. Algorithm that: takes Θ time to go over N items. alls itself 3 times o data of size N-. takes Θ time to ombie the results. M4. ** Algorithm that: alls itself N times o data of size N. takes Θ time to ombie the results. his geerates a diffiult reursio. 35