Divide-and-Conquer a technique for designing algorithms decomposing instance to be solved into subinstances of the same problem solving each subinstance combining subsolutions to obtain the solution to the original instance 981 1234 0981 1234 Multiply Shift Result 09 09 81 81 12 34 12 34 n n/2 n/2 4 2 2 0 108 306 972 2754 1210554 1
Multiplying Large Integers Multiplying two n-figure integers classic algorithm taes Θ(n 2 ) time divide-and-conquer: no improvement 4 Θ(n 2 / 4) = Θ(n 2 ) reducing original multiplication to three half-size multiplicantions pad the shorter operand 981 0981 split each operand into two halves 0981 w = 09, x = 81 1234 y = 12, z = 34 981 1234 = (10 2 w + x) (10 2 y + z) = 10 4 wy + 10 2 (wz + xy) + xz let p = wy, q = xz, r = (w + x) (y + z) = wy + (wz + xy) + xz 981 1234 = 10 4 p + 10 2 (r - p - q) + q three half-size multiplications 2
Three Half-Sized Multiplications Is it worth? we perform four more additions to save one multiplication it is worthwhile when the numbers to be multiplied are large the time needed classic algorithm taes h(n) = n 2 time if each multiplication is carried out by the classic algorithm 3h(n/2) + g(n) = 3/4 h(n) + g(n) for additions, shifts, etc. using our new algorithm recursively to solve subinstances t(n)= 3t(n/2)+g(n)= Θ(n lg3 n is a power of 2) in fact the new algorithm is slower than the classic one on instances that are too small 3
The General Template function DC(x) if x is sufficiently small or simple then return adhoc(x) decompose x into smaller instances x 1, x 2,, x l for i 1 to l do y i DC(x i ) recombine the y i s to obtain a solution y for x return y The basic subalgorithm adhoc: a simple algorithm that can solve small instances efficiently l is usually small and independent of the particular instance to be solved l = 1 simplification algorithm 4
More on Divide-and- Conquer Three conditions must be met the decision when to use the basic subalgorithm must be taen must be possible to decompose an instance into subinstances possible to recombine subsolutions fairly efficiently Running-time analysis the size of each subinstance is n/b t(n) = l t(n b) + g(n) if there exists integer s.t. g(n)= Θ(n ) t( n ) = Θ ( n Θ ( n Θ ( n ) log log b l ) n ) if if if l < l = l > b b b.. (7.1) 5
When to Use the Basic Subalgorithm? Usually when instance size does not exceed a threshold n 0 The running time for multiplying large integers h( n) if n n0 t( n) = 3t ( n / 2 ) + g( n) otherwise where h(n)=θ(n 2 ) and g(n)=θ(n) Given 5000-figure numbers the classic algorithm taes 25 sec if n 0 =1, it taes more than 41 sec if n 0 =64, it taes just over 6 sec the threshold can be determined empirically 6
Binary Search An application of simplification function binsearch(t[1..n], x) if n = 0 or x > T[n] then return n + 1 else return binrec(t[1..n], x) function binrec(t[i..j], x) if i = j then return i (i + j) 2 if x T[] then return binrec(t[i..], x) else return binrec(t[+1..j], x) 1 2 3 4 5 6 7 8 9 10 11-5 -2 0 3 8 8 9 12 12 26 31 i i i i j ij j j j x = 12 x T[]? no yes yes no i = j : stop 7
Binary Search: Time Analysis Let t(m) be the time required for a call on binrec(t[i..j], x) m = j - i + 1 When m > 1 t(m) = t(m / 2) + g(m) g(m)=o(1)=o(m 0 ) t(n) = l t(n b) + g(n) if there exists integer s.t. g(n)= Θ(n ) Θ( n ) if l < b t( n) = Θ( n logn) if l = b logb l Θ( n ) if l > b l = 1, b = 2, = 0 t(m) = Θ(log m) 8
Sorting by Merging To sort elements in T[1..n] into ascending order A divide-and-conquer approach separating T into two parts sorting these parts by recursive calls merging the solutions for each part need an efficient algorithm for merging sorted arrays U and V procedure merge(u[1..m+1],v[1..n+1],t[1..m+n]) i, j 1 U[m+1], V[n+1] for 1 to m + n do if U[i] < V[j] then T[] U[i]; i i + 1 else T[] V[j]; j j + 1 9
3 1 Merge Sort Algorithm Use insertion sort as the basic subalgorithm procedure mergesort(t[1..n]) if n is sufficiently small then insert(t) else array U[1..1 + n/2 ], V[1..1 + n/2 ] U[1.. n/2 ] T[1.. n/2 ] V[1..1 + n/2 ] T[1 + n/2..n] mergesort(u[1.. n/2 ]) mergesort(v[1..1 + n/2 ] ) merge(u, V, T) 3 1 4 1 5 9 2 6 5 3 5 8 9 1 4 1 5 9 sorted 1 3 4 5 9 divide 2 6 5 3 5 8 9 sorted 2 3 5 5 6 8 9 merge 1 1 2 3 3 4 5 5 5 6 8 9 9 10
Mergesort Algorithm: Time Analysis Let t(n) be the time taen to sort n elements separating T into U and V taes linear time merge(u, V, T) also taes linear time t(n)=t( n/2 )+t( n/2 )+g(n) t(n)=2t(n/2)+g(n) when n is even Eq. 7.1 applies with l=2, b=2, and =1 t(n) = Θ(n log n) This is due to the fact that mergesort creates subinstances of roughly equal size what happens if it doesn t? 11
Mergesort: A Bad Dividing procedure badmergesort(t[1..n]) if n is sufficiently small then insert(t) else array U[1..n], V[1..2] U[1..n - 1] T[1.. n - 1] V[1] T[n] badmergesort(u[1.. n - 1) badmergesort(v[1..1] ) merge(u, V, T) ˆ( t n) = ˆ( t n 1) + ˆ(1) t + ˆ( g n) 2 ˆ( t n) = Θ( n ) n n - 1 12
Quicsort the pivot Most of the wor to be done is on constructing the subinstances rather than combining subsolutions General description choose a pivot element partition the array on either side of the pivot those greater then the pivot are to its right, whereas the others are to its left both sides of the pivot are sorted by recursive calls of the algorithm merge step is unnecessary 3 1 4 1 5 9 2 6 5 3 5 8 9 pivoting 2 1 3 1 3 9 5 6 5 4 5 8 9 13
The Pivoting Algorithm the pivot procedure pivot(t[i..j]; var l) p T[i] i; l j + 1 repeat + 1 until T[] > p or j repeat l l - 1 until T[l] p while < l do swap T[] and T[l] repeat + 1 until T[] > p repeat l l - 1 until T[l] p swap T[i] and T[l] 3 1 4 1 5 9 2 6 5 3 5 8 9 3 1 4 1 5 9 2 6 5 3 5 8 9 3 1 3 1 5 9 2 6 5 4 5 8 9 3 1 3 1 5 9 2 6 5 4 5 8 9 3 1 3 1 2 9 5 6 5 4 5 8 9 3 1 3 1 2 9 5 6 5 4 5 8 9 2 1 3 1 3 9 5 6 5 4 5 8 9 l 14
The Quicsort Algorithm procedure quicsort(t[i..j]) if j - is sufficiently small then insert(t[i..j]) else pivot(t[i..j], l) quicsort(t[i..l - 1]) quicsort(t[l + 1..j]) If T is already sorted we get l = i each time quicsort taes a time of Ω(n 2 ) use the median element as the pivot If T is initially in random order assume that all elements of T are distinct each of the n! possible permutations of the elements is equally liely average time: O(n log n) 15
Algorithm Pivotbis Quicsort taes quadratic time in the worst case even if the median is chosen as pivot occurs if all elements of T are equal procedure pivotbis(t[i..j], p; var, l) partitions T into three sections i j < p = p > p i +1 l-1 l j to be sorted sorting an array of equal elements taes linear time worst-case time: O(n log n) to be sorted 16
Finding the Median The s-th smallest element of T is in the s-th position if T were sorted median: the n/2 -th smallest element selection problem finding the s-th smallest element of T an algorithm for selection problem can be used to find the median taes Θ(n log n) time if sort T extracting its s-th entry finding the s-th smallest element d Let p be median(t[1..n]) pivot T around p using pivotbis(t, p,, l) we are done if < s < l if s, d is the s-th smallest element of T[1..] if s l, d is the (s - l + 1)-th smallest element of T[l..n] 17
Selection Algorithm Example: s = 4 p function selection(t[1..n], s) i 1; j n repeat p median(t[i..j]) pivotbis(t[i..j], p,, l) if s then j else if s l then i l else return p 3 1 4 1 5 9 2 6 5 3 5 8 9 l pivotbis 3 1 4 1 2 3 5 5 5 9 6 8 9 3 1 j 1 4 1 2 3 l pivotbis 1 2 3 4 3 i 3 4 3 l pivotbis 3 3 4 18
Need to Choose the Median as Pivot? Not necessary selection wors regardless of which element is chosen as pivot using the median is only for efficiency at least halved each time round the loop simply choosing T[i] as pivot? quadratic time in the worst case linear time on the average good approximation to the median divide n elements into n/5 groups of 5 elements each find m i, the median of each group i find mm, the median of {m i 1 i n/5 } mm is an approximation of the median 19
Median Approximation function pseudomed(t[1..n]) if n 5 then return adhocmed(t) z n / 5 array Z[1..z] for i 1 to z do Z[i] adhocmed(t[5i-4..5i]) return selection(z, z/2 ) Z[i]: the median of T[5i-4..5i] at least 3 are less than or equal to it.at least z/2 elements of Z are less than or equal to mm at least 3z/2 elements of T are less than or equal to mm.z = n / 5 (n - 4) / 5 at least (3n - 12)/10 elements of T are less than or equal to mm 20
21 Matrix Multiplication Let A and B be two n n matrices, and let C be their products classic matrix multiplication Assuming scalar addition and multiplication are elementary Θ(n 3 ) time = = n j i ij B A C 1 = nj j j in i i ij B B B A A A C M L L M L M M L L M 2 1 2 1
Strassen s Algorithm Let a A= a Consider m 1 = (a 21 + a 22 - a 11 )(b 22 - b 12 + b 11 ) m 2 = a 11 b 12 m 3 = a 12 b 21 m 4 = (a 11 - a 21 )(b 22 - b 12 ) m 5 = (a 21 + a 22 )(b 12 - b 11 ) m 6 = (a 12 - a 21 + a 11 - a 22 ) b 22 m 7 = a 22 (b 11 + b 22 - b 12 - b 21 ) We have 11 21 a a 12 22 m2 + m3 C = m1 + m2 + m4 m7 b11 and B = b21 use only 7 scalar multiplications b b 12 22 m1 + m2 + m5 + m6 m1 + m2 + m4 + m5 22
Strassen s Algorithm: Time Analysis A divide-and-conquer algorithm replacing each entry of A and B by an n n matrix multiply two 2n 2n matrices by only 7 multiplications of n n matrices t(n): time needed to multiply two n n matrices (n is a power of 2) t(n) = 7t(n/2) + g(n) g(n) = Θ(n 2 ) the time needed for matrix addition and subtraction Eq. 7.1 applies with l=7,b=2, and =2 t(n) = Θ(n lg7 ) = O(n 2.81 ) fastest matrix multiplication nown: O(n 2.376 ) 23
Exponentiation Compute the exponentiation x =a n function exposeq(a, n) r a for i 1 to n - 1 do r a r return r this algorithm taes Θ(n) time provided the multiplications are counted as elementary operations However, even small values of n and a cause integer overflow 15 17 does not fit in a 64-bit integer we must tae account of the time required for each multiplication 24
Notation Time for Each Multiplication M(q, s): the time needed to multiply two integers of sizes q and s in decimals, in bits, or in any other basis m: the size of a r i and m i : value and size of r at the beginning of the i-th loop iteration T(m, n): total time on computing a n product of two integers of sizes i and j is of size at least i + j - 1 and at most i + j 1 r 1 = a m 1 = m 1 r i+1 = ar i m + m i - 1 m i+1 m + m i im - i + 1 m i im for all i Σ M(m, im- i+1) T(m, n) Σ M(m, im) classic multiplication divide-and-conquer M(q, s) = Θ(qs) M(q, s) = Θ(sq lg(3/2) ) T(m, n) = Θ(m 2 n 2 ) T(m, n) = Θ(m lg3 n 2 ) 25
Improving exposeq Key observation a n = (a n/2 ) 2 when n is even. a n/2 can be computed about four times faster than a n with exposeq. plus a single squaring (multiplication) a if n = 1 n n / 2 2 a = ( a ) if n is even n / 2 2 a ( a ) otherwise a 29 = aa 28 = a(a 14 ) 2 = a((a 7 ) 2 ) 2 = function expodc(a, n) if n = 1 then return a if n is even then return [expodc(a, n/2)] 2 return a expodc(a, n - 1) 26
Time Analysis: expodc N(n): the number of multiplication 0 N( n) = N( n / 2) + 1 N( n 1) + 1 N(n) is Θ(log n) if n = 1 if n is even otherwise T(m, n): time spent multiplying by a call on expodc(a, n) multiplication classic D&C exposeq Θ(m 2 n 2 ) Θ(m lg3 n 2 ) expodc Θ(m 2 n 2 ) Θ(m lg3 n lg3 ) 27