CSCI Honor seminar in algorithms Homework 2 Solution

Transcription

1 CSCI Honor seminar in algorithms Homework 2 Solution Saad Mneimneh Visiting Professor Hunter College of CUNY Problem 1: Rabin-Karp string matching Consider a binary string s of length n and another one t of length m where m n. We would like to find all occurrences of t in s. Here s a naive pseudocode which requires O(nm) time. for i 0 to n m if s[i, i + m 1] = t[0, m 1] then output i O(n) O(m) Rabin and Karp described a more efficient algorithm. The idea is simple: treat t and s[i, i + m 1] for every i = 0... n m as m-bit integers. Call these τ, σ 0,..., σ n m. We need O(m) time to obtain τ and σ 0 : τ = t 0 + t t t m 1 2 m 1 σ 0 = s 0 + s s s m 1 2 m 1 Then each σ i can be obtained in O(1) from σ i 1 as follows (shifting): σ i = (σ i 1 s i )/2 + s i+m 1 2 m 1 Finally, compare τ to all σ i in O(n) time. The total running time of this algorithm is O(n + m). (a) The analysis above assumes constant time arithmetic operations; however, if m is large, operations on m-bit numbers cannot be assumed to take constant time. The actual Rabin-Karp algorithm computes all above modulo some prime p. This can be considered as a form of hashing. The prime p has to be comparable to n so that arithmetic operations can be assumed to take constant time. Argue why choosing a random prime in [1, mn log mn] is good. Solution: The number of bits required to represent the prime number is Θ(log(mn log mn)) = Θ(log mn+log log mn) = Θ(log mn) = Θ(log m+log n) = Θ(log n). (b) If τ σ i, what is the probability that τ σ i 0 mod p? Hint: τ σ i has O(m) bits and, therefore, cannot have more than m prime factors.

2 Solution: Since τ σ i has at most m prime factors, the probability that τ σ i 0 mod p is at most m π(mn log mn) where π(mn log mn) is the number of primes less or equal to mn log mn. From the prime number theorem, we know that this number is asymptotically mn log mn Θ( log(mn log mn) ) = Θ( mn log mn log mn + log log mn) ) = Θ(mn) Therefore, the probability is O( 1 n ). (c) The Rabin-Karp algorithm is modified to explicitly check s[i, i + m 1] and t[0, m 1] only if τ σ i 0 mod p. What is the expected running time of the Rabin-Karp algorithm in terms of n, m, and v the number of times t occurs in s? Solution: Since the probability of a false positive is O( 1 n ), one can say that this is also the expected number of false positives for position i. By the linearily of expectation, the expected number of false positives is at most n O( 1 n ) = O(1). The expected number of checks is therefore v + O(1). The running time will be O(n + m(v + O(1))) = O(n + mv). Problem 2: Fibonacci revisited Perhaps one of the classical examples used to exhibit recursion for computer scientists is the Fibonacci sequence: { n n 1 F ib(n) = F ib(n 1) + F ib(n 2) n > 1 which leads to a direct recursive implementation; for instance, in C: int fib(int n) { if (n<=1) return n; else return fib(n-1)+fib(n-2); (a) What often fails to be mentioned is that this is a terrible way of computing Fibonacci numbers. Show that the running time of the above algorithm is exponential in n. Hint: F ib(n) = Θ(φ n ), where φ = Solution: The base case is defined for n 1 for which the Fibonacci number is 1. Therefore, the n th Fibonacci number is computed by altimately adding 1 s, which means we have Ω(F ib(n)) additions, i.e. Ω(φ n ) time.

3 A better way of computing Fibonacci numbers is the following, again using C syntax: int fib(int n, int a, int b) { while (n>1) { b=a+b; a=b-a; n=n-1; return b; int fib(int n) { if (n<=1) return n; else return fib(n, 0, 1); (b) Explain in words how this algorithm works, and show that it has a Θ(n 2 ) running time. Hint: we assume here that adding two β-bit numbers takes Θ(β) time. Solution: This algorithm simulates the Fibonacci sequence by keeping the last two values of the sequence, a and b. Initially a = 0 and b = 1. Iteratively, a and b are updated as follows: b becomes a + b, and a becomes b (before the update). After n 1 iterations, b will be F ib(n) by definition of the sequence. The algorithm makes Θ(n) additions. Since F ib(n) = Θ(φ n ), the last addition addition involves adding two Θ(n)-bit numbers (because the numbers are exponential in n). Therefore, the running time is Θ(n) + Θ(n 1) + Θ(n 2) Θ(n 0 ), where n 0 is large enough constant. Therefore, we have Θ(n 2 ). (c) In this part of the problem, you are asked to break the Θ(n 2 ) bound to obtain yet a faster way of computing Fibonacci numbers. Consider the following matrix [ ] 0 1 F = 1 1 and its powers F, F 2, F 3... Again assume that adding two β-bit numbers takes Θ(β) time. Solution: It can be easily seen that F11 n is the n th Fibonacci number. We can multiply two 2x2 matrices in Θ(β 1.59 ) time where β is the number of bits, because we need 8 multiplications and 4 additions. We can use the divide-andconquer approach for multiplying two β-bit numbers. Now F n requires only Θ(log n) matrix multiplications using repeated squaring. Therefore, our running time is O(n 1.59 log n) because our Fibonacci numbers have eventually O(n) bits. This is already better than Θ(n 2 ) because n 1.59 log n = o(n 2 ). A more careful analysis, however, will show that the running time is actually Θ(n 1.59 ). With repeated squaring, the number of bits doubles each time. Therefore, our running time is actually Θ(n 1.59 )+Θ(( n 2 )1.59 )+Θ(( n 4 )1.59 )+.... This is Θ(n 1.59 ) because the geometric series converges to a constant. This analysis is not perfectly accurate because repeated squaring guarantees that the number of bits

4 doubles at least every other step, but this can only double the amount of work. For the purpose of Θ notation, this is correct. Problem 3: Practice recurrences This is just to get more familiar with the Master method, do not hand in. Use the master method to solve asymptotic bounds for the following recurrences: (a) Merge sort: T (n) = 2T (n/2) + Θ(n) (b) n-bit multiplication: T (n) = 3T (n/2) + Θ(n) (c) Strassen: T (n) = 7T (n/2) + Θ(n 2 ) (d) Binary search: T (n) = T (n/2) + Θ(1) (e) T (n) = 4T (n/2) + Θ(n 3 ) Problem 4: Made up example Can the master method be applied to the recurrence T (n) = 4T (n/2)+n 2 / log n? Why or why not? Give an asymptotic bound for this recurrence. Solution: Let s compare g(n) = n 2 / log n to n log b a = n 2. n 2 / log n n 2 = 1/ log n = O(n ɛ ), Θ(log k n), Ω(n ɛ ), ɛ > 0? i.e. log n = Ω(n ɛ )? No k 0? No ɛ > 0? No Therefore, no case applies. The Bazzi methods solves that. First we solve for p, ab p = p = 1 p = 2. Then we solve the integral: n u 2 / log u n 1 c u 3 du = c u log u du Let s make the change of variable v = log u, dv = du/u. log n log c Therefore, the answer is Θ(n 2 log log n). Problem 5: The missing integer dv v = log v log n = Θ(log log n) log c In the spirit of bits and divide-and-conquer, so I will make it optional Assume we are given an array a[1... n] containing all the integers from 0 to n except one of them. We are asked to find the missing integer. The only type of operation allowed on a is to fetch the j th bit of the i th integer a[i]. Design an algorithm to find the missing integer in Θ(n) time. Hint: try to discover the missing integer bit by bit and think about a way to yield the recurrence T (n) = T (n/2) + Θ(n).

5 Solution: The idea is to discover the missing integer bit by bit. We maintain a list of candidates, originally C = {0, 1, 2,..., n, and a list of indices of a, originally I = {1, 2,... n. Assume all integers are represented using k bits and consider the last bit, bit k. We compute how many 0 s and 1 s we have for that bit in a[i] for i I, and how many 0 s and 1 s we have for that bit in the list of candidates C. The missing integer will result in a missing 0 or a missing 1. This will help us shorten the lists C and I for another round. For instance, if a 0 bit is missing, all integers ending with 1 are not candidates for the missing integer anymore. We remove those from both lists and replace every element e remaining in C with e/2 (integer division), and update k to k 1. All operations on the lists can be done in linear time, including the removal of elements (one could simply copy the remaining elements into a new list). This is repeated until the list of candidates has only one element, the missing integer. Note that in each time, the lists of candidates and indices are almost halved in size, leading to the recurrence: T (n) = T (n/2) + Θ(n) Therefore, we find the missing integer in Θ(n) + Θ(n/2) + Θ(n/4) +... = Θ(n) time. missing-integer(c, I, k) if C = 1 then return the singleton element of C if e C e mod 2 > i I a[i][k] (bit 1 is missing) then b = 0 else b = 1 C = C {e : e mod 2 = b I = I {i : a[i][k] = b missing-integer({e/2 : e C, I, k 1) Problem 6: Modular FFT The DFT requires the use of complex numbers, which can result in a loss of precision due to round-off errors. Bounding the propagation of error can be tricky; an alternative is to utilize a variant of FFT based on modular arithmetic. (a) Instead of 1, w = e i2π/n, w 2,..., w n 1 consider the following integers modulo 2 tn/2 + 1: 1, w = 2 t, w 2,..., w n 1 where t is an arbitrary positive integer. Show that these are the n th roots of 1 modulo 2 tn/2 + 1, i.e. ( 2 it ) n 1 mod 2 tn/2 + 1 for i = 0... n 1 and they are distinct. Furthermore, describe the FFT and its inverse based on these roots. Solution: w n = 2 tn = (2 tn/2 1)(2 tn/2 + 1) + 1 = k(2 tn/2 + 1) + 1

6 Therefore, w is an n th root of 1. The same is true for w k, k = 0,... n 1, because (w k ) n = (w n ) k. They wrap around because w i+n = w i w n w i mod 2 tn/ To show they are distinct assume w i w j mod 2 tn/2 + 1, i > j, and i j < n. Then w j (w i j 1) 0 mod 2 tn/ This means w j (w i j 1) is a multiple of 2 tn/2 + 1, but since w j and 2 tn/2 + 1 are relatively prime (one is a power of 2 and the other is odd), then (w i j 1) = 2 t(i j) 1 = t(i j) 1 must be a multiple of 2 tn/ Therefore, we need an integer k such that when added to itself shifted left by n/2 bits (i.e. multiplied by 2 tn/2 + 1), we get a pattern of all 1 s. The smallest such k 0 is 2 n/2 1, which means the smallest i j 0 is n, a contradiction. FFT and inverse FFT work the same way, where w 1 = w n 1 = 2 t(n 1) (all modulo 2 tn/ (b) Let k be such that p = kn + 1 is prime. Argue intuitively using the prime number theorem that k = O(log n). Solution: We are considering odd numbers n + 1, 2n + 1, 3n + 1,..., i.e. we are jumping by n. Therefore, we are exploring 1/n of the odd numbers in a given interval. Since a prime number must be odd, we expect to find one this way if we have about n of them. By the prime number theorem, n log n log(n log n) n (c) Consider the set {1, 2,..., p 1. Because p is prime, this is a multiplicative group. Furthermore, the group is cyclic, i.e. there exists a g such that g 0, g 1, g 2,... g p 2 are all distinct modulo p (that s why g is called a generator of the group, it generates all the elements). Consider the sequence (k from part (b) above): 1, w = g k mod p, w 2,..., w n 1 Show that these are the n th roots of 1 modulo p and describe the FFT and its inverse in terms of them. Remark: the advantage of this scheme over the one described in part (a) is that the roots in part (a) require O(n) bits because they are integers < 2 tn/2 + 1, but the roots here require O(log n) bits because they are integers < p. Solution: w n = (g k ) n = g nk = g p 1. By Fermat s theorem, since g {1, 2,..., p 1 and p is prime, g p 1 1 mod p. Note that 1, g k, (g 2 ) k, (g 3 ) k,..., (g n 1 ) k modulo p are distinct because n 1 p 2. FFT and inverse FFT work in the same way, where w 1 = w n 1 = g kn k = g p 1 k (all modulo p).