Itroductio to Algorithms 6.046J/8.40J LECTURE 3 Divide ad coquer Biary search Powerig a umber Fiboacci umbers Matrix multiplicatio Strasse s algorithm VLSI tree layout Prof. Charles E. Leiserso
The divide-ad-coquer desig paradigm. Divide the problem (istace) ito subproblems. 2. Coquer the subproblems by solvig them recursively. 3. Combie subproblem solutios. September 5, 2004 Itroductio to Algorithms L3.2
Merge sort. Divide: Trivial. 2. Coquer: Recursively sort 2 subarrays. 3. Combie: Liear-time merge. September 5, 2004 Itroductio to Algorithms L3.3
Merge sort. Divide: Trivial. 2. Coquer: Recursively sort 2 subarrays. 3. Combie: Liear-time merge. T() = 2 T(/2) + Θ() # subproblems subproblem size work dividig ad combiig September 5, 2004 Itroductio to Algorithms L3.4
Master theorem (reprise) T() = at(/b) + f () CASE : f () = O( log ba ε ), costat ε > 0 T() = Θ( log ba ). CASE 2: f () = Θ( log ba lg k ), costat k 0 T() = Θ( log ba lg k+ ). CASE 3: f () = Ω( log ba + ε ), costat ε > 0, ad regularity coditio T() = Θ( f ()). September 5, 2004 Itroductio to Algorithms L3.5
September 5, 2004 Master theorem (reprise) T() = at(/b) + f () CASE : f () = O( log ba ε ), costat ε > 0 T() = Θ( log ba ). CASE 2: f () = Θ( log ba lg k ), costat k 0 T() = Θ( log ba lg k+ ). CASE 3: f () = Ω( log ba + ε ), costat ε > 0, ad regularity coditio T() = Θ( f ()). Merge sort: a = 2, b = 2 log ba = log 22 = CASE 2 (k = 0) T() = Θ( lg ). Itroductio to Algorithms L3.6
Biary search Fid a elemet i a sorted array:. Divide: Check middle elemet. 2. Coquer: Recursively search subarray. 3. Combie: Trivial. September 5, 2004 Itroductio to Algorithms L3.7
Biary search Fid a elemet i a sorted array:. Divide: Check middle elemet. 2. Coquer: Recursively search subarray. 3. Combie: Trivial. Example: Fid 9 3 5 7 8 9 2 5 September 5, 2004 Itroductio to Algorithms L3.8
Biary search Fid a elemet i a sorted array:. Divide: Check middle elemet. 2. Coquer: Recursively search subarray. 3. Combie: Trivial. Example: Fid 9 3 5 7 8 9 2 5 September 5, 2004 Itroductio to Algorithms L3.9
Biary search Fid a elemet i a sorted array:. Divide: Check middle elemet. 2. Coquer: Recursively search subarray. 3. Combie: Trivial. Example: Fid 9 3 5 7 8 9 2 5 September 5, 2004 Itroductio to Algorithms L3.0
Biary search Fid a elemet i a sorted array:. Divide: Check middle elemet. 2. Coquer: Recursively search subarray. 3. Combie: Trivial. Example: Fid 9 3 5 7 8 9 2 5 September 5, 2004 Itroductio to Algorithms L3.
Biary search Fid a elemet i a sorted array:. Divide: Check middle elemet. 2. Coquer: Recursively search subarray. 3. Combie: Trivial. Example: Fid 9 3 5 7 8 9 2 5 September 5, 2004 Itroductio to Algorithms L3.2
Biary search Fid a elemet i a sorted array:. Divide: Check middle elemet. 2. Coquer: Recursively search subarray. 3. Combie: Trivial. Example: Fid 9 3 5 7 8 9 2 5 September 5, 2004 Itroductio to Algorithms L3.3
Recurrece for biary search T() = T(/2) + Θ() # subproblems subproblem size work dividig ad combiig September 5, 2004 Itroductio to Algorithms L3.4
Recurrece for biary search T() = T(/2) + Θ() # subproblems subproblem size work dividig ad combiig log ba = log 2 = 0 = CASE 2 (k = 0) T() = Θ(lg ). September 5, 2004 Itroductio to Algorithms L3.5
Powerig a umber Problem: Compute a, where N. Naive algorithm: Θ(). September 5, 2004 Itroductio to Algorithms L3.6
Powerig a umber Problem: Compute a, where N. Naive algorithm: Θ(). Divide-ad-coquer algorithm: a = a /2 a /2 a ( )/2 a ( )/2 a if is eve; if is odd. September 5, 2004 Itroductio to Algorithms L3.7
Powerig a umber Problem: Compute a, where N. Naive algorithm: Θ(). Divide-ad-coquer algorithm: a = a /2 a /2 a ( )/2 a ( )/2 a if is eve; if is odd. T() = T(/2) + Θ() T() = Θ(lg ). September 5, 2004 Itroductio to Algorithms L3.8
Fiboacci umbers Recursive defiitio: 0 if = 0; F = if = ; F + F 2 if 2. 0 2 3 5 8 3 2 34 L September 5, 2004 Itroductio to Algorithms L3.9
Fiboacci umbers Recursive defiitio: F = 0 if = 0; if = ; F + F 2 if 2. 0 2 3 5 8 3 2 34 L Naive recursive algorithm: Ω(φ ) (expoetial time), where φ = ( + is the golde ratio. 5)/2 September 5, 2004 Itroductio to Algorithms L3.20
Computig Fiboacci umbers Bottom-up: Compute F 0, F, F 2,, F i order, formig each umber by summig the two previous. Ruig time: Θ(). September 5, 2004 Itroductio to Algorithms L3.2
Computig Fiboacci umbers Bottom-up: Compute F 0, F, F 2,, F i order, formig each umber by summig the two previous. Ruig time: Θ(). Naive recursive squarig: F = φ / 5 rouded to the earest iteger. Recursive squarig: Θ(lg ) time. This method is ureliable, sice floatig-poit arithmetic is proe to roud-off errors. September 5, 2004 Itroductio to Algorithms L3.22
Recursive squarig F + F Theorem: = F F. 0 September 5, 2004 Itroductio to Algorithms L3.23
Recursive squarig F + F Theorem: = F F. 0 Algorithm: Recursive squarig. Time = Θ(lg ). September 5, 2004 Itroductio to Algorithms L3.24
September 5, 2004 Itroductio to Algorithms L3.25 Recursive squarig F F F F = + 0 Theorem:. Proof of theorem. (Iductio o.) Base ( = ):. 0 0 2 = F F F F Algorithm: Recursive squarig. Time = Θ(lg ).
September 5, 2004 Itroductio to Algorithms L3.26 Recursive squarig.. Iductive step ( 2): F F F F F F F F = = = + 0 0 0 0 2
September 5, 2004 Itroductio to Algorithms L3.27 Matrix multiplicatio = b b b b b b b b b a a a a a a a a a c c c c c c c c c L M O M M L L L M O M M L L L M O M M L L 2 2 22 2 2 2 2 22 2 2 2 2 22 2 2 = = k kj ik ij b a c Iput: A = [a ij ], B = [b ij ]. Output: C = [c ij ] = A B. i, j =, 2,,.
Stadard algorithm for i to do for j to do c ij 0 for k to do c ij c ij + a ik b kj September 5, 2004 Itroductio to Algorithms L3.28
Stadard algorithm for i to do for j to do c ij 0 for k to do c ij c ij + a ik b kj Ruig time = Θ( 3 ) September 5, 2004 Itroductio to Algorithms L3.29
Divide-ad-coquer algorithm IDEA: matrix = 2 2 matrix of (/2) (/2) submatrices: r=ae+bg s=af+bh t =ce+dg u=cf+dh r t s u = a c e g September 5, 2004 Itroductio to Algorithms L3.30 b d C = A B 8 mults of (/2) (/2) submatrices 4 adds of (/2) (/2) submatrices f h
Divide-ad-coquer algorithm IDEA: matrix = 2 2 matrix of (/2) (/2) submatrices: r=ae+bg s=af+bh t =ce+dh u=cf+dg r t s u = a c e g September 5, 2004 Itroductio to Algorithms L3.3 b d C = A B recursive 8 mults of (/2) (/2) submatrices ^ 4 adds of (/2) (/2) submatrices f h
Aalysis of D&C algorithm T() = 8 T(/2) + Θ( 2 ) # submatrices submatrix size work addig submatrices September 5, 2004 Itroductio to Algorithms L3.32
Aalysis of D&C algorithm T() = 8 T(/2) + Θ( 2 ) # submatrices submatrix size work addig submatrices log ba = log 2 8 = 3 CASE T() = Θ( 3 ). September 5, 2004 Itroductio to Algorithms L3.33
Aalysis of D&C algorithm T() = 8 T(/2) + Θ( 2 ) # submatrices submatrix size work addig submatrices log ba = log 2 8 = 3 CASE T() = Θ( 3 ). No better tha the ordiary algorithm. September 5, 2004 Itroductio to Algorithms L3.34
Strasse s idea Multiply 2 2 matrices with oly 7 recursive mults. September 5, 2004 Itroductio to Algorithms L3.35
Strasse s idea Multiply 2 2 matrices with oly 7 recursive mults. P = a ( f h) P 2 = (a + b) h P 3 = (c + d) e P 4 = d (g e) P 5 = (a + d) (e + h) P 6 = (b d) (g + h) P 7 = (a c) (e + f ) September 5, 2004 Itroductio to Algorithms L3.36
Strasse s idea Multiply 2 2 matrices with oly 7 recursive mults. P = a ( f h) P 2 = (a + b) h P 3 = (c + d) e P 4 = d (g e) P 5 = (a + d) (e + h) P 6 = (b d) (g + h) P 7 = (a c) (e + f ) r = P 5 + P 4 P 2 + P 6 s = P + P 2 t = P 3 + P 4 u = P 5 + P P 3 P 7 September 5, 2004 Itroductio to Algorithms L3.37
Strasse s idea Multiply 2 2 matrices with oly 7 recursive mults. P = a ( f h) P 2 = (a + b) h P 3 = (c + d) e P 4 = d (g e) P 5 = (a + d) (e + h) P 6 = (b d) (g + h) P 7 = (a c) (e + f ) r = P 5 + P 4 P 2 + P 6 s = P + P 2 t = P 3 + P 4 u = P 5 + P P 3 P 7 7 mults, 8 adds/subs. Note: No reliace o commutativity of mult! September 5, 2004 Itroductio to Algorithms L3.38
Strasse s idea Multiply 2 2 matrices with oly 7 recursive mults. P = a ( f h) P 2 = (a + b) h P 3 = (c + d) e P 4 = d (g e) P 5 = (a + d) (e + h) P 6 = (b d) (g + h) P 7 = (a c) (e + f ) r = P 5 + P 4 P 2 + P 6 =(a + d)(e + h) + d (g e) (a + b) h + (b d)(g + h) = ae + ah + de + dh + dg de ah bh + bg + bh dg dh = ae + bg September 5, 2004 Itroductio to Algorithms L3.39
Strasse s algorithm. Divide: Partitio A ad B ito (/2) (/2) submatrices. Form terms to be multiplied usig + ad. 2. Coquer: Perform 7 multiplicatios of (/2) (/2) submatrices recursively. 3. Combie: Form C usig + ad o (/2) (/2) submatrices. September 5, 2004 Itroductio to Algorithms L3.40
Strasse s algorithm. Divide: Partitio A ad B ito (/2) (/2) submatrices. Form terms to be multiplied usig + ad. 2. Coquer: Perform 7 multiplicatios of (/2) (/2) submatrices recursively. 3. Combie: Form C usig + ad o (/2) (/2) submatrices. T() = 7 T(/2) + Θ( 2 ) September 5, 2004 Itroductio to Algorithms L3.4
Aalysis of Strasse T() = 7 T(/2) + Θ( 2 ) September 5, 2004 Itroductio to Algorithms L3.42
Aalysis of Strasse T() = 7 T(/2) + Θ( 2 ) log ba = log 2 7 2.8 CASE T() = Θ( lg 7 ). September 5, 2004 Itroductio to Algorithms L3.43
Aalysis of Strasse T() = 7 T(/2) + Θ( 2 ) log ba = log 2 7 2.8 CASE T() = Θ( lg 7 ). The umber 2.8 may ot seem much smaller tha 3, but because the differece is i the expoet, the impact o ruig time is sigificat. I fact, Strasse s algorithm beats the ordiary algorithm o today s machies for 32 or so. September 5, 2004 Itroductio to Algorithms L3.44
Aalysis of Strasse T() = 7 T(/2) + Θ( 2 ) log ba = log 2 7 2.8 CASE T() = Θ( lg 7 ). The umber 2.8 may ot seem much smaller tha 3, but because the differece is i the expoet, the impact o ruig time is sigificat. I fact, Strasse s algorithm beats the ordiary algorithm o today s machies for 32 or so. Best to date (of theoretical iterest oly): Θ( 2.376L ). September 5, 2004 Itroductio to Algorithms L3.45
VLSI layout Problem: Embed a complete biary tree with leaves i a grid usig miimal area. September 5, 2004 Itroductio to Algorithms L3.46
VLSI layout Problem: Embed a complete biary tree with leaves i a grid usig miimal area. W() H() September 5, 2004 Itroductio to Algorithms L3.47
VLSI layout Problem: Embed a complete biary tree with leaves i a grid usig miimal area. W() H() H() = H(/2) + Θ() = Θ(lg ) September 5, 2004 Itroductio to Algorithms L3.48
VLSI layout Problem: Embed a complete biary tree with leaves i a grid usig miimal area. W() H() H() = H(/2) + Θ() = Θ(lg ) W() = 2W(/2) + Θ() = Θ() September 5, 2004 Itroductio to Algorithms L3.49
VLSI layout Problem: Embed a complete biary tree with leaves i a grid usig miimal area. W() H() H() = H(/2) + Θ() W() = 2W(/2) + Θ() = Θ(lg ) = Θ() Area = Θ( lg ) September 5, 2004 Itroductio to Algorithms L3.50
H-tree embeddig L() L() September 5, 2004 Itroductio to Algorithms L3.5
H-tree embeddig L() L() L(/4) Θ() L(/4) September 5, 2004 Itroductio to Algorithms L3.52
H-tree embeddig L() L() L() = 2L(/4) + Θ() = Θ( ) Area = Θ() L(/4) Θ() L(/4) September 5, 2004 Itroductio to Algorithms L3.53
Coclusio Divide ad coquer is just oe of several powerful techiques for algorithm desig. Divide-ad-coquer algorithms ca be aalyzed usig recurreces ad the master method (so practice this math). The divide-ad-coquer strategy ofte leads to efficiet algorithms. September 5, 2004 Itroductio to Algorithms L3.54