CS473- Algorithms I Lecture 4 The Divide-and-Conquer Design Paradigm CS473 Lecture 4 1
The Divide-and-Conquer Design Paradigm 1. Divide the problem (instance) into subproblems. 2. Conquer the subproblems by solving them recursively. 3. Combine subproblem solutions. CS473 Lecture 4 2
Example: Merge Sort 1. Divide: Trivial. 2. Conquer: Recursively sort 2 subarrays. 3. Combine: Linear- time merge. T(n) = 2 T(n/2) + O(n) # subproblems subproblem size work dividing and combining CS473 Lecture 4 3
Master Theorem (reprise) T(n) = at(n/b) + f(n) a CASE 1: f(n) = O(n - ) => T(n) = (n log a ). b b CASE 2: f(n) = (n log a lg k n) => T(n) = (n log a lg k+1 b b n). CASE 3: f(n) = Ω(n log ε log => T(n) = (f(n)). Merge Sort: a = 2, b = 2 => n a b ε + ) and af(n/b) cf(n) a log b = n => Case 2 ( k = 0 ) => T(n) = ( n lg n ). CS473 Lecture 4 4
Binary Search Find an element in a sorted array: 1. Divide: Check middle element. 2. Conquer: Recursively search 1 subarray. 3. Combine: Trivial. Example: Find 9 3 5 7 8 9 12 15 CS473 Lecture 4 5
Binary Search Find an element in a sorted array: 1. Divide: Check middle element. 2. Conquer: Recursively search 1 subarray. 3. Combine: Trivial. Example: Find 9 3 5 7 8 9 12 15 CS473 Lecture 4 6
Binary Search Find an element in a sorted array: 1. Divide: Check middle element. 2. Conquer: Recursively search 1 subarray. 3. Combine: Trivial. Example: Find 9 3 5 7 8 9 12 15 CS473 Lecture 4 7
Binary Search Find an element in a sorted array: 1. Divide: Check middle element. 2. Conquer: Recursively search 1 subarray. 3. Combine: Trivial. Example: Find 9 3 5 7 8 9 12 15 CS473 Lecture 4 8
Binary Search Find an element in a sorted array: 1. Divide: Check middle element. 2. Conquer: Recursively search 1 subarray. 3. Combine: Trivial. Example: Find 9 3 5 7 8 9 12 15 CS473 Lecture 4 9
Binary Search Find an element in a sorted array: 1. Divide: Check middle element. 2. Conquer: Recursively search 1 subarray. 3. Combine: Trivial. Example: Find 9 3 5 7 8 9 12 15 CS473 Lecture 4 10
Recurrence for Binary Search T(n) = 1 T(n/2) + Ө(1) # subproblems subproblem size work dividing and combining n log a = n log 1 = n 0 =1 => CASE 2 (k = 0) b 2 => T(n) = (lgn). CS473 Lecture 4 11
Powering a Number Problem: Compute a n, where n is in N. Naive algorithm: (n) Divide-and-conquer algorithm: a n = a n/2. a n/2 a (n-1)/2. a (n-1)/2. a if n is even if n is odd T(n) = T(n/2) + (1) => T(n) = (lgn). CS473 Lecture 4 12
Matrix Multiplication Input : A = [a ij ], B = [b ij ]. Output: C = [c ij ] = A. B. i, j = 1, 2,..., n. c 11 c 12... c 1n a 11 a 12... a 1n b 11 b 12... b 1n c 21 c 22... c 2n = a 21 a 22... a 2n. b 21 b 22... b 2n............... c n1 c n2... c nn a n1 a n2... a nn b n1 b n2... b nn............ c ij = 1 k n a ik.b kj CS473 Lecture 4 13
Standard Algorithm for i 1 to n do for j 1 to n do c ij 0 for k 1 to n do c ij c ij + a ik. b kj Running time = (n 3 ) CS473 Lecture 4 14
Divide-and-Conquer Algorithm IDEA: n x n matrix = 2 x 2 matrix of (n/2) x (n/2) submatrices c 11 c 12 a 11 a 12 b 11 b 12 c 21 c=. 22 a 21 a 22 b 21 b 22 C = A. B c 11 = a 11 b 11 + a 12 b 21 c 12 = a 11 b 12 + a 12 b 22 c 21 = a 21 b 11 + a 22 b 21 8 mults of (n/2)x(n/2) submatrices 4 adds of (n/2)x(n/2) submatrices c 22 = a 21 b 21 + a 22 b 22 CS473 Lecture 4 15
Analysis of D&C Algorithm T(n) = 8 T(n/2) + (n 2 ) # submatrices submatrix size work dividing submatrices a log b log 2 8 n = n = n 3 => CASE 1 => T(n) = (n 3 ). No better than the ordinary algorithm! CS473 Lecture 4 16
Strassen s Idea Multiply 2x2 matrices with only 7 recursive mults P 1 = a 11 x (b 12 - b 22 ) c 11 = P 5 + P 4 P 2 + P 6 P 2 = (a 11 + a 12 ) x b 22 c 12 = P 1 + P 2 P 3 = (a 21 + a 22 ) x b 11 c 21 = P 3 + P 4 P 4 = a 22 x (b 21 - b 11 ) c 22 = P 5 + P 1 P 3 P 7 P 5 = (a 11 + a 22 ) x (b 11 + b 22 ) P 6 = (a 11 - a 22 ) x (b 21 + b 22 ) P 7 = ( a 11 - a 21 ) x (b 11 + b 12 ) 7 mults 18 adds/subs. Does not rely on commutativity of mult. CS473 Lecture 4 17
Strassen s Idea Multiply 2x2 matrices with only 7 recursive mults P 1 = a 11 x (b 12 - b 22 ) P 2 = (a 11 + a 12 ) x b 22 c 12 = P 1 + P 2 P 3 = (a 21 + a 22 ) x b 11 = a 11 (b 12 b 22 )+(a 11 +a 12 )b 22 P 4 = a 22 x (b 21 - b 11 ) = a 11 b 12 -a 11 b 22 +a 11 b 22 +a 12 b 22 P 5 = (a 11 + a 22 ) x (b 11 + b 22 ) = a 11 b 12 +a 12 b 22 P 6 = (a 11 - a 22 ) x (b 21 + b 22 ) P 7 = ( a 11 - a 21 ) x (b 11 + b 12 ) CS473 Lecture 4 18
Strassen s Algorithm 1. Divide: Partition A and B into (n/2) x (n/2) submatrices. Form terms to be multiplied using + and. 2. Conquer: Perform 7 multiplications of (n/2) x (n/2) submatrices recursively. 3. Combine: Form C using + and on (n/2) x (n/2) submatrices. T(n) = 7 T(n/2) + Ө(n 2 ) CS473 Lecture 4 19
Analysis of Strassen T(n) = 7 T(n/2) + (n 2 ) n log b a = n log 2 7 = n 2.81 => CASE 1 => T(n) = (n lg7 ). The number 2.81 may not seemmuch smaller than 3, but because the difference is in the exponent, the impact on running time is significant. In fact, Strassen s algorithm beats the ordinary algorithmon today s machines for n 30 or so. Best to date (of theoretical interest only) : (n 2.376... ). CS473 Lecture 4 20
VLSI Layout Problem: Embed a complete binary tree with n leavesin a grid using minimal area. W(n) H(n) H(n) = H(n/2) + Ө(1) W(n) = 2 W(n/2) + Ө(1) = Ө(lg n) = Ө(n) Area = Ө(nlg(n)) CS473 Lecture 4 21
H-tree Embedding L(n) L(n) L(n) = 2 L(n/4)+ (1) = ( n) Area = (n) CS473 Lecture 4 22
Conclusion Divide and conquer is just one of several powerful techniques for algorithm design. Divide-and-conquer algorithms can be analyzed using recurrences and the master method (so practice this math). Can lead to more efficient algorithms CS473 Lecture 4 23