3. RECURSIVE ALGORITHMS AND RECURRENCE RELATIONS

Transcription

1 . RECURSIVE ALGORITHMS AND RECURRENCE RELATIONS I dscussg the example of fdg the determat of a matrx a algorthm was outled that defed det(m) for a x matrx terms of the determats of matrces of sze (-)x(-). If D() s the wor requred to evaluate the determat of a x matrx usg ths method the D()=. D(-) To solve ths -- the sese of edg up wth a expresso for D() that does ot have referece to other occurreces of the fucto D() o the rght had sde -- we ca use progressve substtutos as follows: D()=. D(-) =. (-). D(-2) =. (-). (-2). x x 2 x D() D() O(!)! a costat, the cost of returg a umber COMP004: Part

2 Now cosder ALGORITHM Factoral() // Recursvely calculates! for postve teger f = retur else retur *Factoral(-) The oly possble choce for the elemetary operato here s multplcato, so we choose as a cost fucto F() = umber of multplcatos to evaluate Factoral() The algorthm as defed the mples that F() = 0 F() = F(-) +, > The geeral- form ca be rewrtte as F()-F(-)= Lettg - the above, we also have F(-)-F(-2)= We ca cotue to do ths utl the row begg wth F(2), whch cotas the frst meto of the base case (- = 2- = ). F() - F(-) = + F(-) F(-2) = + F(-2) F(-) = - equatos.. + F(2) - F() = F() - F() = - ('Method of dffereces or ladder method'.) COMP004: Part

3 Addg up the rugs of the ladder there are cacellatos o the left had sde for every term except F() ad F(), ad for the rght had sde the sum s just -. F() = F() + - F() = -, sce F()=0 F() O(). Asde: Is ths reasoable, s evaluatg! really oly O()? The aalyss gored the fact that large values wll buld up qucly ad so t was ot ths case deal to cout teger multplcatos as elemetary. However t was a smple example that showed the techques that ca also be used much more complex cases. * * * Expressos le D() = D(-) F() = F(-) + are ow as recurrece relatos. A recurrece relato s a formula that allows us to compute the members of a sequece oe after the other, gve oe or more startg values. These examples are frst order recurrece relatos because a referece s made to oly oe smaller szed stace. A frst order recurrece requres oly oe startg value -- the frst of these cases D(), the secod F() -- order to obta a uque soluto for geeral. Recurrece relatos arse aturally the aalyss of recursve algorthms, where the startg values are the wor requred to compute base cases of the algorthm. COMP004: Part

4 Cosder as aother example of a frst order recurrece relato f(0) = (the base case s for =0) f() = c. f(-) > 0 (It does't matter here what hypothetcal algorthm may have geerated the defto of f(). Most of the examples ths secto wll be startg drectly from a recurrece relato ad wll be focused prmarly o developg the mathematcal tools for solvg such relatos.) By specto, f() = c. f(-) = c 2. f(-2) = c. f(0) = c f()=c s the soluto of ths recurrece relato. It s purely a fucto of the put sze varable, t does ot mae referece o the rght had sde to the orgal cost fucto appled to ay other put szes. The expresso D()=. D(-) s clearly also a frst order recurrece of the same d, but here the multplyg coeffcet depeds o. Ths s the ext step up complcato, where for a geeral -depedet multpler b() f() = b()xf(-) () (defed for > a wth base case value f(a) gve) Aga by specto f() = b()xf(-) = b()xb(-)xf(-2) = b()xb(-)x...xb(a+)xf(a) COMP004: Part 2.2 0

5 (ote t was ot possble to go ay further because there would the be a referece to f(a-), whch s ot defed) gvg as a geeral soluto to () " $ f() = # %$! =a+ & $ b() ' ($ xf(a) the determat example a=, f()=, b()=, gvg f() =! b() = 2xx...x =! =a+ () s the most geeral case of a frst order (t maes referece to oly oe smaller szed stace), homogeeous (there s othg o the r.h.s. that does't multply a smaller szed stace, such as a polyomal ) recurrece relato. What about the factoral fucto recurrece F() = F(-) +? Ths s a smple example of a frst order, homogeeous recurrece relato, for whch the geeral form (wth o-costat coeffcets b(), c()) s f() = b()xf(-) + c() ( > a, f(a) gve) (2) If we try to do ths oe by progressve substtuto as for the determat case we qucly get to dffcultes: f() = b()xb(-)x f(-2) + b()xc(-) + c() = b()xb(-)xb(-2)xf(-) + b()xb(-)xc(-2) + b()xc(-)+ c() =... COMP004: Part 2.2

6 I cases le ths we eed to defe stead a ew fucto g() by f() = b(a+)xb(a+2)x xb()x g(), >a f(a) g(a) (the two fuctos agree o the base case) Substtutg ths way for f() (2): f() = b()xf(-) + c() b(a+)xb(a+2)x xb()x g() = b()xb(a+)xb(a+2)x xb(-)x g(-) + c() same coeffcet The dvdg by the product b(a+)xb(a+2)x xb(), the commo multpler of both staces of the fucto g(), gves g() = g(-) + d() () where d() = c() b(a +)xb(a + 2)x...xb(). () s ow a much smpler form (the mportat thg s that the multpler of the smaller-put stace of the fucto o the r.h.s. s ow ) ad ca be solved by the ladder method we used the case of F() (where d() =, depedet of ) to gve the soluto " $ & $ f() = ) b() # f(a) +! d(j) ' =a+ %$ j=a+ ($ the factoral example a=, f()=0, b()=, d(j) =, gvg f() = - (as! d(j) =! = ") j=2 j=2 COMP004: Part 2.2 2

7 It's possble, ad ot correct, to memorse ad quote the above formula ad substtute to t order to solve a gve recurrece relato, as the formula ca be appled to homogeeous frst order recurreces of ay degree of complexty. However t s more structve, ad ofte easer, to cosder homogeeous recurreces o a case-by-case bass, otg that the examples you would be gve the base case put a wll ormally be 0 or, ad the d-summato above -- whch could prcple be hard ad requre a approxmato techque -- wll use just the seres summato formulae you have already see. Example: f(0)=0 (base case a=0) f()= f(-) +, > 0 Chage varable: f() = g() (b() = for all, so! b() = ) Ths gves g() =. - g(-) + = g() = g(-) +/ Now use the method of dffereces (ladder method) to obta g () = = COMP004: Part 2.2

8 I detal: g() g(-) = / g(-) g(-2) = / g() g(0) = / g() g(0) = = (Whe you are famlar wth these techques you do t eed to show all the detals.) Multplyg both sdes of the soluto for g() by gves the soluto for f() f () = = We evaluate the sum usg the formula for a geometrc seres: Hece = = = a( a = a a ) = 2 a= a= " f() = 2! % $ ' = # & 2! ( ) ( O( ) COMP004: Part 2.2 4

9 SOLVING HIGHER ORDER RECURRENCES USING THE CHARACTERISTIC EQUATION Ths s a techque whch ca be appled to lear recurreces of arbtrary order but s less geeral tha some of the techques llustrated for frst order recurreces that we wll heceforth assume that aythg multplyg a stace(for some put sze) of the fucto s a costat the recurrece relatos are homogeeous -- they do t cota ay term that s t a multple of a stace of the fucto we are tryg to obta a 'closed' soluto for Hece, we are here tryg to solve th order recurreces of the geeral form a f() + a - f(-) + + a 0 f(-) = 0 where the {a } are costats. (Puttg everythg o the l.h.s. ad equatg t to zero rather tha wrtg 'f() =...' s a coveece, as wll be see later.) Try the soluto f() = α (see Brassard ad Bratley p.65 for a justfcato of ths choce): a α + a - α a 0 α - = 0 α - ca be see to be a commo factor: α - (a α + a - α a 0 ) = 0 characterstc equato We assume that we are ot terested the trval soluto α=0, as ths would mply f()=0 -- a algorthmc cotext that the wor eeded for a put of ay sze was zero -- oly the o-trval solutos of the characterstc equato above. COMP004: Part 2.2 5

10 The characterstc equato s a polyomal of degree ad would geeral be expected to have dstct roots (the case that some of these may ot be dstct wll be cosdered later). Assumg that the roots ρ, ρ 2,... ρ of the characterstc equato are all dstct, ay lear combato of {ρ } f() c = = ρ (Brassard ad Bratley p.65) solves the homogeeous recurrece, where the {c } are costats determed by the tal codtos that defe the base cases of the recurrece. Example (=2): f(0) = 0 f() = f() = f(-) 2f(-2), > Try a soluto of the form f() = α : α - α - + 2α -2 = 0 Dvde by α -2 to get the characterstc equato α 2 α + 2 = 0 (α - )(α - 2) = 0 The roots ρ =, ρ 2 = 2 gve a geeral soluto of the form f() = c. + c 2.2 Usg the tal codtos: c + c 2 = 0 (=0) c = -, c 2 = c + 2c 2 = (=) f() = 2 COMP004: Part 2.2 6

11 Example (=): f(0) = f() = 5 f(2) = 9 f() = 7f(-) 4f(-2) + 8f(-), > 2 The characterstc equato s α 7α 2 + 4α - 8 = 0 (α - ) (α - 2) (α - 4) = 0 Roots ρ =, ρ 2 = 2, ρ = 4 gve a geeral soluto of the form f() = c. + c c.4 Usg the tal codtos: c + c 2 + c = (=0) c = -, c 2 =, c = c + 2c 2 + 4c = 5 (=) c + 4c 2 + 6c = 9 (=2) f() = O( 4 ) (Note that uder exam codtos you would ot be expected to solve > 2 examples. Ths s because t s oly for =2 that there s a smple formula for fdg the roots of the (the quadratc) characterstc equato; for > 2 the roots mght be cosderably more dffcult to fd.) COMP004: Part 2.2 7

12 If the characterstc equato has a root ρ wth multplcty m, the (Brassard ad Bratley p.67) x = ρ, x = ρ,, x = m- ρ are all possble solutos of the recurrece. I ths case the geeral soluto s a lear combato (wth costat coeffcets to be determed by the tal codtos) of these terms together wth those cotrbuted by the other roots of the characterstc equato. Example (=2): f(0) = f() = 5 f() = 2f(-) f(-2), > The characterstc equato s α 2 2α + = 0 (α - ) 2 = 0 The repeated root ρ = gves a geeral soluto of the form f() = c. + c 2.. Usg the tal codtos: c = (=0) c =, c 2 = 4 c + c 2 = 5 (=) f() = 4 + COMP004: Part 2.2 8

13 Example (=): f(0) = 2 f() = 9 f(2) = 29 f() = 5f(-) 8f(-2) + 4f(-), > 2 The characterstc equato s α 5α 2 + 8α - 4 = 0 (α - ) (α - 2) 2 = 0 Roots ρ = ad ρ 2 = 2 (m=2) gve a geeral soluto of the form f() = c. + c c..2 Usg the tal codtos: c + c 2 = 2 (=0) c =, c 2 =, c = c + 2c 2 + 2c = 9 (=) c + 4c 2 + 8c = 29 (=2) f() = O( 2 ) COMP004: Part 2.2 9

14 Example: Fboacc umbers The Fboacc umbers are a famous sequece 0,,, 2,, 5, 8,, 2,... defed by the recurrece Fb() = Fb(-) + Fb(-2) wth Fb(0) = 0 Fb() = Patters corporatg ths sequece appear to be wdespread ature, for example the florets the head of a suflower or the growth of pe coes. The sequece was frst troduced to Europe a treatse o the growth of rabbt populatos by Leoardo Fboacc 202 (though t may have bee ow Ida for much loger). 2 The sze of the Fboacc umbers grows expoetally -- the 49th umber s 7,778,742,049, more tha the total umber of people the world. Helathus flower, L Shyamal 2006 Youg coes of a Colorado spruce, COMP004: Part

15 We ca show the expoetal growth of the Fboacc umbers by solvg the recurrece relato for the seres values Fb() usg the characterstc equato method. The geeral- case s Fb() Fb(-) Fb(-2) = 0 wth assocated characterstc equato α 2 α = 0 Ufortuately ths does't factorse eatly wth teger solutos le prevous examples ad t's ecessary to use the formula for the roots of a quadratc equato to gve the solutos!,2 = ± 5 2 ad hece a geeral soluto for the Fboacc seres values of the form Fb() = c ρ + c 2 ρ 2 Usg the base case values Fb(0)=0, Fb()=: c + c 2 = 0 (=0) c = c ρ + c 2 ρ 2 = (=) ad hece Fb() =! + 5 # 5 " 2 $ & % '! ' 5 # 5 " 2 $ & % 5, c 2 =! 5 = ( 5 ( ' (! ) whch! = (+ 5) / 2 ".680,! = "/! # "0.680 COMP004: Part 2.2 4

16 It ca easly be see that Fb()! O(" ) sce as!! < the cotrbuto from the oscllatg secod part of the soluto falls to zero very rapdly as creases. Rabbts really do breed le rabbts! (Though Fboacc assumed, amog other thgs, that they were also mmortal ad hece hs model of rabbt populato growth would ot owadays be thought very realstc.) But does ths ecessarly mea that computg the Fboacc umbers wll tae a expoetally growg amout of tme? Cosder frst a aïve procedure based drectly o the defto: ALGORITHM Fb() // Calculates the th Fboacc umber for teger 0 f retur else retur Fb(-)+Fb(-2) Tag addto here as the elemetary operato the cost F() terms of the umber of addtos ca be see to be F(0) = 0 F() = 0 F() = F(-) + F(-2) + Ufortuately ths s a homogeeous secod order recurrece, ad the characterstc equato method preseted here does't exted to these cases. (I fact t does fall wth the slghtly expaded rage of fuctoal forms cosdered Brassard ad Bratley (p.68) but we wll ot cover these more geeral methods of soluto.) COMP004: Part

17 But there's a trc: otce that f we defe G() = F()+ the the recurrece relato for the umber of addtos becomes G(0) = G() = G() = G(-) + G(-2) whch s homogeeous ad so soluble usg our usual methods. Eve better, t's ot actually ecessary to solve ths recurrece as t ca also be observed that G() = Fb(+) ad hece that F() = G()! = Fb(+)! = ( 5 "+! "! + )! So t s the case that F()! O(" ) -- the amout of wor requred by the defto-derved algorthm grows expoetally the same way as the sequece values Fb() themselves. However t's possble to do better. It mght have bee predcted that Fb was ot a effcet algorthm from specto of ts structure. Worg through some examples for small (such as the =5 case o p.82 of Levt) wll covce you there s massve effcecy the recomputato of already-ow values -- for example the case of =5 Fb(4) s computed four tmes over. There s a smple teratve alteratve that taes oly lear tme as measured by the umber of teger addtos, storg alreadyow values a array A[0..]: ALGORITHM Fb2() // Calculates the th Fboacc umber for teger 0 A[0] < 0 A[] < for < 2 to do A[] < A[-]+A[-2] retur A[] COMP004: Part 2.2 4

18 Ths clearly does oly - addtos, so s O(). BUT -- be careful -- though the secod algorthm wll certaly be preferable, cosderg addtos ether case as havg ut cost may ot be realstc because the the sze of the umbers the Fboacc sequece geerates. It's the same stuato as for the factoral fucto recurrece, foud to be O() terms of teger multplcatos but where the tegers questo would qucly become very large. Realstcally oe would eed to th about sgle bt operatos, the sze of a stace the beg gve as +! " log 2 # $. CHANGE OF VARIABLE Cosder the problem of rasg a umber, a, to some power. The aïve algorthm for dog ths s ALGORITHM Exp(a,) // Computes a for postve teger f = retur a else retur a*exp(a,-) Exp has the same geeral structure as the factoral fucto we looed at earler ad by a smlar argumet ca be demostrated also to be O() (coutg multplcatos at ut cost). Ca we mprove o ths smple expoetato algorthm? Cosder the followg sequece for computg a 2 : a a 2 a 4 a 8 a 6 a 2 Each term s obtaed by squarg the prevous oe, so oly 5 multplcatos are requred, ot. COMP004: Part

19 It's possble to base a alterate algorthm o ths successve squarg dea: ALGORITHM Exp2(a,) // Computes a for postve teger f = retur a else f eve() retur [Exp2(a,/2)] 2 else retur a*[exp2(a,(-)/2)] 2 (dvsos each case gve teger result) Exp2(a,) gets broe to smaller (Exp2) staces ad the squared to gve the requred result. Let E() be the wor requred to compute Exp2(a,) ad use multplcato as the ut-cost elemetary operato: E() = 0 squarg of smaller-put stace E() = E(/2) + f eve E((-)/2) + 2 f odd squarg of smaller-put stace ad multplcato of result by a Suppose for the momet that s ot oly eve but s a power of 2. Mae the chage of varable 2 COMP004: Part

20 The the eve- recurrece taes the form E(2 ) = E(2 /2) + = E(2 - ) + or more compactly, wrtg E(2 ) E E E - = Ths ca ow easly be solved by the ladder method: E E - = + E - E -2 = equatos + + E - E 0 = E - E 0 = So E = E 0 + where E 0 E(2 0 ) = E() = 0 E(2 ) = Sce = 2 = log 2 E() = log 2 Thus E() O(log s a power of 2) (remember that bases of logs ca be dropped uder O ) codtoal asymptotc otato COMP004: Part

21 It ca be show (Brassard ad Bratley, p.46) that f() O( g() s a power of 2 ) f() O(g()) f (a) g s evetually o-decreasg (b) g(2) O( g() ) -- 'smoothess property' log s a creasg fucto for all, ad sce log(2) = log 2 + log O(log ) we ca say fally that for all E() O(log ) Exp2 s thus sgfcatly more effcet, as grows, tha the aïve algorthm Exp -- though t s always advsable to chec that the computatoal overheads mplemetg a more sophstcated algorthm do t outwegh the beefts for the sze of put staces you are lely to ecouter. Example : f() = f() = 4 f(/2) + 2, > Chage varables 2 to obta the recurrece f = 4f Mae the further substtuto (because of the multpler 4 frot of f - ): f = 4 g, f 0 = g 0 4 g = g COMP004: Part

22 Dvdg by 4 ad costructg the ladder g - g - = + g - g -2 = + + g - g 0 = g - g 0 = g = g 0 + (where ad are related by = 2, = log 2 ) Multplyg the soluto for g by 4 : f = 4 f f() = 2 f() + 2 log 2 (f 0 = f(2 0 ) = f(); 4 = (2 ) 2 = 2 ) f() O( 2 log s a power of 2) Sce 2 log s a o-decreasg fucto for all > (t has a turg pot betwee 0 ad ), ad (2) 2 log(2) = 4 2 (log2) (log ) O( 2 log ) t ca be cocluded that for all f() O( 2 log ) COMP004: Part

23 Example: f()= f(2)=2 f()=4f(/2) 4f(/4), >2 Chage varables 2 (otg that as /2 2 -, /4 2-2 ) to obta the recurrece for the ew varable f = 4 f - 4 f -2 Ths s a homogeeous 2 d order recurrece, ad should be solved usg the characterstc equato method. The characterstc equato for the above recurrece (varable ) s α 2 4α + 4 = 0 (α - 2) 2 = 0 Repeated root ρ=2, so the geeral soluto for s f = c 2 + c 2 2 log 2 I terms of the orgal varable, ths geeral soluto s f() = c + c 2 log 2 Is ths O( log )? It depeds o the base cases, whch determe c ad c 2. f() = c + c 2 log 2 = c = 0 f(2) = 2c + 2 c 2 log 2 2 = 2(c + c 2 ) = 2 c =, c 2 = 0 f()= O() ot O( log ) COMP004: Part

24 Strasse s algorthm for multplcato We ow loo for ways to mprove the effcecy of multplcato of two -bt umbers -- the methods we have see so far are O( 2 ). We ca wrte the two -bt umbers whch are to be multpled le ths: a b c d /2-bts /2-bts /2-bts /2-bts ( a. 2 /2 + b ) ( c. 2 /2 + d ) = a. c. 2 + (a. d + b. c). 2 /2 + b. d ths s just puttg zeros after the value a.c, so s O() ths s puttg /2 zeros after (a.d + b.c), so s O() Base 0 (decmal) example for =6 dgts: = = a b Let M() be the cost of multplyg two -bt umbers usg Strasse s algorthm. M() = a 0 M() = 4 M(/2) + a + a 2 (the cost of multplyg 2 bts) all other arthmetc operatos apart from the 4 /2-bt multplcatos -- ote that addto of two -bt umbers s O() 4 pars of /2-bt multplcatos ( a.c, b.d, a.d ad b.c ) COMP004: Part

25 Ths recurrece s smlar to the oe o p.47 ad we proceed smlarly: Frst, set = 2, M(2 ) = M to get M = 4 M - + a 2 + a 2 Now set M = 4 N, M 0 = N 0 (ths s just dog both ecessary substtutos oe step, a shorteed form of the method the prevous example): 4 N = N - + a 2 + a 2 Dvde by 4 ad costruct ladder: N - N - = a /2 + a 2 /4 + N - - N -2 = a /2 - + a 2 / N - N 0 = a /2 + a 2 /4 N - N 0 = ( a / 2 + a / 4 ) 2 = Multplyg by 4 (wth M 0 = M() = a 0 ): M = 4 a + a + a 0 2 = 2 = 4 COMP004: Part 2.2 5

26 From the earler lecture o geometrc seres, = 2 ( ( / 2) ) / 2 = / 2 = 4 = < 2 ( ( / 4) ) / 4 = / 4 4 = < / Hece M(2 ) < 4 (a 0 + a + a 2 /) M() < 2 (a 0 + a + a 2 /) Sce 2 s o-decreasg, ad (2) 2 O( 2 ), we ca say that M() O( 2 ). But ths s the same tme-complexty as shft-ad-add multplcato ad the à la Russe method), so at frst sght Strasse s algorthm loos o better tha our prevous oes. COMP004: Part

27 COMP004: Part However f we use the followg observato, due to Strasse (a - b). (d - c) = (a. d + b. c) a. c b. d we ca compute a.c, b.d ad (a.d + b.c) just three multplcatos: a.c, b.d ad (a-b).(d-c) (we assume subtractos are of eglgble (O()) cost compared to multplcatos). Repeatg the above aalyss wth M() = M(/2) + a + a 2 we ed up wth + + = = = 2 0 a 2 a a ) (2 M where = = < 2 = = 2 < ½ ad hece + + < 2 a 2a a ) (2 M < 2 a 2a a () M 2 0 log 2 usg log log log ) (2 log log 2 2 ) (2 = log 2 =

28 log 2 Therefore M() O( s a power of 2 ) log 2 So, sce s o-decreasg, ad ( 2) log 2 O( result ca be geeralsed to all, ad hece log 2 M() O( ) = O(.59 ) log 2 ), ths Ths s a tme-complexty termedate betwee the O( 2 ) of the elemetary algorthms ad the O() of the lower boud o the sgle-bt-operato cost of multplyg two -bt umbers. How much further could a Strasse-le method be pushed toward a O() performace? Dvdg to parts of / bts, ad usg a smlar -- but more complcated -- trc to reduce the umber of multplcatos ths case to get just 5 (as opposed to the 9 that would be ävely requred) leads by smlar techques to log 5 M() O( )=O(.46 ) But relatvely speag ths s a much smaller mprovemet tha was gaed by gog from the elemetary algorthms to breag the umbers to two /2-bt parts, ad the assocated O() costs (from the 'trc') are greater. If we try to brea the umbers to /4-bt, /5-bt...parts ad use a Strasse-le method the the relatve mprovemets each tme get lesseed stll further, ad the O() overheads become stll greater, so that such elaborate algorthms would be foud better practce oly for such large- cases that they would ot really be of use eve to cryptographers. The moral s that the sglemded pursut of a algorthm wth the best possble asymptotc ( ) behavour may ot always practce be a sesble thg. COMP004: Part