Divide-and-Conquer and Recurrence Relations: Notes on Chapter 2, Dasgupta et.al. Howard A. Blair

Transcription

1 Divide-and-Conquer and Recurrence Relations: Notes on Chapter 2, Dasgupta et.al. Howard A. Blair 1 Multiplication Example 1.1: Carl Friedrich Gauss ( ): The multiplication of two complex numbers depends on just three multiplications involving complex numbers. (a + bi)(c + di) = (ac bd) + (bc + ad)i = (}{{} ac }{{} bd ) + ((a + b)(c + d) ac bd)i }{{} Note 1.1: (The recursive procedure for Divide-and-Conquer Multiplication.) Let x and y be two nonnegative integers. Express each of x and y in binary. By including leading 0 s if necessary, we can assume that the binary representations of x and y have the same number of bits, n, and that n is a power of 2. x L be the integer represented by the leftmost n 2 bits of x and let x R be the integer represented by the rightmost n 2 bits of x. Let y L and y R be similarly defined. Then xy = (x L 2 n/2 + x R )(y L 2 n/2 + y R ) = x L y L 2 n + (x L y R + x R y L )2 n/2 + x R y R (by Gauss s trick) = x L y L 2 n + ((x L + x R )(y L + y R ) x L y L x R y R )2 n/2 + x R y R Example 1.2: Let x = ( ) 2, y = ( ) 2. Then x L = (1101) 2, x R = (0011) 2, y L = (1001) 2, y R = (0110) 2. Thus, x = 211, y = 150, x L = 13, x R = 3, y L = 9, y R = = = [ ] = =

2 Note 1.2: (The runtime of Divide-and-Conquer Multiplication.) Let T (n) be the runtime of Divide-and-Conquer Multiplication on multiplication of two n-bit integers. (If n is not a power of 2, in fact if n is odd, let x L be the integer represented by the n/2 leftmost bits of the binary representation of x, let x R be the integer represented by the n/2 rightmost bits of the binary representation of x.) If we don t use Gauss s trick, then T (n) satisfies T (n) = 4 T ( n 2 ) + O(n) since there are 4 multiplications to perform on integers whose bitlength is n/2, and and the resulting additions and subtractions then altogether take O(n) time. If we use Gauss s trick there are only 3multiplications to perform and then T (n) satisfies T (n) = 3 T ( n 2 ) + O(n) We will see from the Master Theorem, given below, that the reduction of the coefficient in the equation for T (n) is a big deal. 2 The Master Theorem Theorem 2.1: Let T : N N such that T (n) = a T ( n b ) + O(nd ) for some real number constants a > 0, b > 0, and d 0. Then O(n d ) if d > log b a T (n) = O(n d ln n) if d = log b a O(n log b a ) if d < log b a 2

3 Example 2.1: Solve the following recurrence: T (n) = 49 T (n/25) + n 3/2 ln n Notice that if we replace the above equation, also called a recurrence relation, by T (n) = 49 T ( n/25 ) + n 3/2 ln n then T (n) T (n) but, since solutions by the Master Theorem are expressed in terms of Big-O, any solution of the second recurrence relation is also a solution of the first. Therefore, we can effectively ignore the ceiling notation when applying the Master Theorem. Next, we have seen that for any ɛ > 0, n d ln n = O)n d+ɛ ). Consider, therefore, T (n) = 49 T (n/25) + n 3/2+ɛ To apply the Master Theorem, note that a = 49, b = 25, d = ɛ and i.e. log = ln 49 ln = 1.25 < ɛ d > log b a Therefore, for any ɛ > 0, T (n) = O(n 3/2+ɛ ) At this point we can guess that T (n) = O(n 3/2 ln n) It it is trivial to verify this solution and we could replace the exponent 3/2 by any smaller positive exponent and drop the ln n factor, but the point is to get the fastest growth rate allowed by the Master Theorem. We want the fastest growth rate because we cannot, based on the runtime recurrence relation and the Master Theorem, assure the growth rate of the runtime function is any smaller than the maximum allowed. 3

4 Note 2.1: (Applying the Master Theorem to the runtime recurrence relation of Divide-and-Conquer Multiplication.) Without Gauss s trick: a = 4, b = 2, d = 1, log 2 4 = 2 > 1. Therefore, T (n) = O(n log 2 4 ) = O(n 2 ) With Gauss s trick: a = 3, b = 2, d = 1, log 2 3 > 1. Therefore, T (n) = O(n log 2 3 ) O(n ) 3 Sorting 3.1 Upper bounds Note 3.1: (MergeSort.) function mergesort(a[1.. n]) { if (n > 1) return merge(mergesort(a[1.. (n div 2)]), mergesort(a[(n div 2) n]); else return a[1]; } function merge(x[1.. k], y[1.. l]) { if (k == 0) return y[1... l]; if (l == 0) return x[1... k]; if (x[1] <= y[1]) return cons(x[1], merge(x[2.. k], y[1.. l])); else return cons(y[1], merge(x[1.. k], y[2.. l])); } 4

5 The work in the MergeSort algorithm is done by the Merge algorithm after MergeSort recursively works its way down to singleton arrays. Example 3.1: numbers (Adaped from Dasgupta et.al) Input to mergesort the array of [10, 2, 5, 3, 7, 13, 1, 11, 6, 8, 3] At the deepest level of recursion, [3], [7] will be merged into [3, 7] and [1], [11] will be merged into [1, 11] and [8] and [3] will be merged into [3, 8]. Then returning up through the levels of recursion, at the next level up, [10] and [2] are merged, [5] and [3, 7] are merged, [13] and [1, 11] are merged, [6] and [3, 8] are merged. Then, at the next level: [2, 10] and [3, 5, 7] are merges, [1, 11, 13] and [3, 6, 8] are merged. Finally, [2, 3, 5, 7, 10] and [1, 3, 6, 8, 11, 13] are merged. Note 3.2: (Runtime recurrence relation of MergeSort.) To ease the analysis, assume that n is a power of 2. Let T (n) be the runtime of MergeSort. Recursively, T (n) = 2 T (n/2) + Time to merge the recursive results = 2 T (n/2) + O(n) When we apply the Master Theorem, we note a = b = 2, d = 1, log 2 2 = 1. Hence, d = log b a. Therefore, T (n) = O(n ln n) 3.2 Lower bounds Definition 3.1: Suppose we have an algorithm to sort an ordered collection of items. The sorted ordering of the items is to be determined as follows: item i precedes item j iff key(item i ) key(item j ). The key of an item is some index, such as a numerical index, serial number, or element of an enumeration type, such that the keys can 5

6 be directly compared to ascertain whether one key is less than or equal to another. Sorting is to be accomplished in some manner that depends on comparing keys. Any such sorting algorithm is called a key-comparison sorting algorithm, or just a key-comparison-sort. Theorem 3.1: All sequential key-comparison sorting algorithms have runtime obeying T (n) = Ω(n ln n) Proof: Any sequential key-comparison sorting algorithm has to be able to sort ordered sets of keys; i.e. all of the keys to be sorted are distinct form each other: no repetitions. Let K = {k 1,..., k n } be an ordered set of keys to be sorted. An algorithm that sorts a set of keys finds a permutation of the original ordered set that is in sorted order. We are looking for the unique permutation π of S such that {k π(1),..., k π(n) } is in sorted order. There are n! permutations in the set P of permutations of K. Let Q be a subset of P, i.e. a set of permutations of K. Let k i be compared with k j. The comparison splits Q into two sets Q = {τ Q k τ(i) < k τ(j) } {τ Q k τ(i) > k τ(j) } Let key i be compared with key j. The comparison splits P into two sets: P 1 is the subset of P Let S be a set of 2 n items to be sorted. Consider comparing any key with key i. 6