Tutorials. Algorithms and Data Structures Strassen s Algorithm. The Master Theorem for solving recurrences. The Master Theorem (cont d)

Transcription

1 DS 2018/19 Lecture 4 slide 3 DS 2018/19 Lecture 4 slide 4 Tutorials lgorithms and Data Structures Strassen s lgorithm Start in week week 3 Tutorial allocations are linked from the course webpage DS 2018/19 Lecture 4 slide 1 The Master Theorem for solving recurrences Theorem Let n 0 N, k N 0 and a, b R with a > 0 and b > 1, and let T : N R satisfy the following recurrence: { Θ1 if n < n 0, T n a T n/b + Θn k if n n 0. Let c log b a; we call c the critical exponent. Then Θn c if k < c I, T n Θn c lgn if k c II, Θn k if k > c III. Theorem also holds if we replace a T n/b above by a 1 T n/b + a 2 T n/b for any a 1, a 2 0 with a 1 + a 2 a. The Master Theorem cont d DS 2018/19 Lecture 4 slide 2 We don t have time to prove the Master Theorem in class. You can find the proof in Section 4.6 of [CLRS]. Section 4.4 of [CLRS], 2nd ed. Their version of the M.T. is a bit more general than ours. Consider the following examples: T n 4T n/2 + n, T n 4T n/2 + n 2, T n 4T n/2 + n 3. Could alternatively unfold-and-sum to prove the first and third of these and to get an estimate for the second. CLSS EXERCISE

2 DS 2018/19 Lecture 4 slide 6 DS 2018/19 Lecture 4 slide 7 Matrix Multiplication Matrix Multiplication Recall The product of two n n-matrices b2j a i1 a i2 a in a ij 1 i,j n and b ij 1 i,j n is the n n-matrix C where C 1 i,j n with entries n a ik b kj. k1 The Matrix Multiplication Problem Input: n n-matrices and Output: the n n-matrix Matrix Multiplication DS 2018/19 Lecture 4 slide 5 straightforward algorithm DS 2018/19 Lecture 4 slide 6 - n multiplications and n additions for each. - there are n 2 different entries. b2j a i1 a i2 a in lgorithm MatMult, 1. n number of rows of 2. for i 1 to n do 3. for j 1 to n do for k 1 to n do 6. + a ik b kj 7. return C 1 i,j n Requires Θn 3 arithmetic operations additions and multiplications.

3 DS 2018/19 Lecture 4 slide 9 DS 2018/19 Lecture 4 slide 10 näive divide-and-conquer algorithm näive divide-and-conquer algorithm Observe If a i1 a i2 a in b 2j and for n/2 n/2-submatrices ij and ij then note: We are assuming n is a power of 2. DS 2018/19 Lecture 4 slide 8 näive divide-and-conquer algorithm a i1 a i2 a in b 2j Suppose i n/2 and j > n/2. Then n/2 n n a ik b kj a ik b kj + a ik b kj k1 k1 kn/ DS 2018/19 Lecture 4 slide 9 näive divide-and-conquer algorithm cont d ssume n is a power of 2. lgorithm D&C-MatMult, 1. n number of rows of 2. if n 1 then return a 11 b else 4. Let ij, ij for i, j 1, 2 be n/2 n/2-submatrices s.th and Recursively compute 11 11, 12 21, 11 12, 12 22, 21 11, 22 21, 21 12, Compute C , C , 7. return C , C C11 C 12 C 21 C 22

4 DS 2018/19 Lecture 4 slide 12 DS 2018/19 Lecture 4 slide 13 nalysis of D&C-MatMult T n is the number of operations done by D&C-MatMult. Lines 1, 2, 3, 4, 7 require Θ1 arithmetic operations Line 5 requires 8T n/2 arithmetic operations Line 6 requires 4n/2 2 Θn 2 arithmetic operations. Remember! Size of matrices is Θn 2, NOT Θn We get the recurrence T n 8T n/2 + Θn 2. Since log 2 8 3, the Master Theorem yields T n Θn 3. nalysis of D&C-MatMult T n is the number of operations done by D&C-MatMult. Lines 1, 2, 3, 4, 7 require Θ1 arithmetic operations Line 5 requires 8T n/2 arithmetic operations Line 6 requires 4n/2 2 Θn 2 arithmetic operations. Remember! Size of matrices is Θn 2, NOT Θn We get the recurrence T n 8T n/2 + Θn 2. Since log 2 8 3, the Master Theorem yields T n Θn 3. No improvement over MatMult... why? CLSS?... Strassen s algorithm 1969 DS 2018/19 Lecture 4 slide 11 ssume n is a power of 2. Let and We want to compute C11 C 12. C 21 C 22 Strassen s algorithm uses a trick in applying Divide-and-Conquer. Let Strassen s algorithm cont d P P P P P P P DS 2018/19 Lecture 4 slide 11

5 Let Strassen s algorithm cont d P Strassen s algorithm computes C 11 P1 + P4 P5 + P7. We have P P P P P P P P P P Then C 11 P 1 + P 4 P 5 + P 7 C 12 P 3 + P 5 C 21 P 2 + P 4 C 22 P 1 + P 3 P 2 + P 6 DS 2018/19 Lecture 4 slide 13 Strassen s algorithm computes C 11 P1 + P4 P5 + P7. We have P P P P Then P1 + P DS 2018/19 Lecture 4 slide 14 Strassen s algorithm computes C 11 P1 + P4 P5 + P7. We have P P P P Then P1 + P Then P1 + P4 P DS 2018/19 Lecture 4 slide 14 DS 2018/19 Lecture 4 slide 14

6 DS 2018/19 Lecture 4 slide 15 DS 2018/19 Lecture 4 slide 16 Strassen s algorithm computes C 11 P1 + P4 P5 + P7. We have P P P P Then P1 + P Then P1 + P4 P Then P1 + P4 P5 + P , which is C11. Strassen s algorithm computes C 11 P1 + P4 P5 + P7. We have P P P P Then P1 + P Then P1 + P4 P Then P1 + P4 P5 + P , which is C11. homework: check other 3 equations. Strassen s algorithm cont d DS 2018/19 Lecture 4 slide 14 Crucial Observation Only 7 multiplications of n/2 n/2-matrices are needed to compute. lgorithm Strassen, 1. n number of rows of 2. if n 1 then return a 11 b else 4. Determine ij and ij for i, j 1, 2 as before 5. Compute P 1,..., P 7 as in 6. Compute C 11, C 12, C 21, C 22 as in C11 C return C 21 C 22 nalysis of Strassen s algorithm DS 2018/19 Lecture 4 slide 14 Let T n be the number of arithmetic operations performed by Strassen. Lines 1 4 and 7 require Θ1 arithmetic operations Line 5 requires 7T n/2 + Θn 2 arithmetic operations Line 6 requires Θn 2 arithmetic operations. remember. We get the recurrence T n 7T n/2 + Θn 2. Since log > 2, the Master Theorem yields T n Θn log 2 7.

7 DS 2018/19 Lecture 4 slide 19 reakthroughs on matrix multiplication Coppersmith & Winograd 1987 came up with an improved algorithm with running time of... many years of silence... Θn Then in his 2010 PhD thesis, ndrew Stothers from the School of Maths, at the University of Edinburgh got an algorithm with Θn c for c < Coppersmith/Winograd not optimal. ut Stothers didn t publish. Remarks on Matrix Multiplication In practice, the school MatMult algorithm tends to outperform Strassen s algorithm, unless the matrices are huge. The best known lower bound for matrix multiplication is Ωn 2. This is a trivial lower bound need to look at all entries of each matrix. mazingly, Ωn 2 is believed to be the truth! Open problem: Can we find a On 2+o1 -algorithm for Matrix Multiplication of n n matrices? In December 2011, Virginia Vassilevska Williams of Stanford, came up with a Θn c algoithm, for c < partly, but not only, making use of some of Stothers ideas Reading ssignment DS 2018/19 Lecture 4 slide 17 [CLRS] 3rd ed Section 4.5 The Master method for solving recurrences Section 4.3 Using the Master method of [CLRS], 2nd ed [CLRS] 3rd ed Section 4.2 Section 28.2 of [CLRS], 2nd ed DS 2018/19 Lecture 4 slide 18 Problems 1. Exercise of [CLRS] 3rd ed Exercise of [CLRS], 2nd ed. 2. Exercise of [CLRS], 3rd ed. Exercise [CLRS], 2nd ed. 3. Week 3 tutorial sheet :-