Algorithms and data structures

Transcription

1 Algorithms and data structures Amin Coja-Oghlan LFCS

2 Complex numbers Roots of polynomials A polynomial of degree d is a function of the form p(x) = d a i x i with a d 0. i=0 There are at most d numbers ζ R such that p(ζ) = 0; they are called roots of p. Example: p(x) = x has no roots in R. Reminder: complex numbers Therefore, let s make up a new number i such that i 2 = 1. Then C = {x + i y : x,y R} is the field of complex numbers. A. Coja-Oghlan (LFCS) Algorithms and data structures 2 / 15

3 Complex numbers (ctd.) Reminder: complex numbers Therefore, let s make up a new number i such that i 2 = 1. Then C = {x + i y : x,y R} is the field of complex numbers. Manipulating complex numbers Addition: (x + iy) + (q + ir) = (x + q) + i(y + r). Multiplication: (x + iy) (q + ir) = (x q y r) + i(x r + y q). Conjugate: x + iy = x iy. Absolute value: x + iy = x 2 + y 2. A. Coja-Oghlan (LFCS) Algorithms and data structures 3 / 15

4 Complex numbers (ctd.) Theorem ( fundamental theorem of algebra ) Any polynomial p(x) = d i=0 a ix i with a i C and a d = 1 can be decomposed as p(x) = d (x ζ j ), where each ζ j C is a root of p. j=1 A. Coja-Oghlan (LFCS) Algorithms and data structures 4 / 15

5 Complex numbers (ctd.) Theorem ( fundamental theorem of algebra ) Any polynomial p(x) = d i=0 a ix i with a i C and a d = 1 can be decomposed as p(x) = The roots of unity d (x ζ j ), where each ζ j C is a root of p. j=1 For example, let p(x) = x d 1. Then d 1 p(x) = x ωd i, where i=0 ω d = cos(2π/d) + i sin(2π/d) = exp(2πi/d), where z j exp(z) = for all z C. j! j=0 A. Coja-Oghlan (LFCS) Algorithms and data structures 4 / 15

6 Complex numbers (ctd.) The roots of unity (ctd.) The complex number ω d = cos(2π/d) + i sin(2π/d) = exp(2πi/d) is called a d th principal root of unity. Its inverse is ω d = exp( 2πi/d) = ω d 1 d = ω d. ω d = 1. If d is even, then ω d/2 d = 1. A. Coja-Oghlan (LFCS) Algorithms and data structures 5 / 15

7 Complex numbers (ctd.) The roots of unity (ctd.) 2 w 8 = i w 8 = e i*2pi/8 = (cos (2 pi/8), i*sin(2 pi/8) = (1+i)/sqrt(2) 2pi/8 w 0 = 1 8 Wheel representation of 8th roots of unity in the complex plane. Same wheel structure for any n, with ω n sitting at angle 2π/n. A. Coja-Oghlan (LFCS) Algorithms and data structures 6 / 15

8 The Discrete Fourier Transform (DFT) Computational problem Input: a vector (a 0,...,a n 1 ) C n. Output: the vector (A(1),A(ω n ),A(ωn),... 2,A(ωn n 1 )), where n 1 A(x) = a j x j. Think of this vector as a fingerprint of the polynomial A(x)! j=0 A. Coja-Oghlan (LFCS) Algorithms and data structures 7 / 15

9 The Discrete Fourier Transform (DFT) Computational problem Input: a vector (a 0,...,a n 1 ) C n. Output: the vector (A(1),A(ω n ),A(ωn),... 2,A(ωn n 1 )), where n 1 A(x) = a j x j. Think of this vector as a fingerprint of the polynomial A(x)! Why bother? j=0 DFT is fundamental in signal processing, where it is usually implemented in hardware. Given a signal (e.g., a tune), DFT yields a decomposition by frequencies. This allows removing noise, amending the signal, etc. Further application: fast multiplication of polynomials (next lecture). A. Coja-Oghlan (LFCS) Algorithms and data structures 7 / 15

10 The Fast Fourier Transformation (FFT) Naive approach Compute A(1),A(ω n ),...,A(ωn n 1 ) separately Θ(n 2 ) multiplications. A. Coja-Oghlan (LFCS) Algorithms and data structures 8 / 15

11 The Fast Fourier Transformation (FFT) Naive approach Compute A(1),A(ω n ),...,A(ωn n 1 ) separately Θ(n 2 ) multiplications. A divide and conquer approach Consider a polynomial A(x) = n 1 j=0 a jx j, where n = 2 j for some j > 0. Define A even (y) = a 0 + a 2 y + a 4 y a n 2 y n 2 1, A odd (y) = a 1 + a 3 y + a 5 y a n 1 y n 2 1 ; A(x) = A even (x 2 ) + x A odd (x 2 ). so that Observation: reduction from degree n problem to two degree n 2 problems. Suggests T(n) = 2T(n/2) + O(n) T(n) = Θ(n lnn). A. Coja-Oghlan (LFCS) Algorithms and data structures 8 / 15

12 The FFT (ctd.) How do we deal with A even (x 2 ), A odd (x 2 )? Goal: compute A even (ω 2k n ), A odd (ω 2k n ) for 0 k < n. Key observation: ω 2 n = exp(2πi/n)2 = exp(2πi/(n/2)) = ω n/2. A. Coja-Oghlan (LFCS) Algorithms and data structures 9 / 15

13 The FFT (ctd.) How do we deal with A even (x 2 ), A odd (x 2 )? Goal: compute A even (ω 2k n ), A odd (ω 2k n ) for 0 k < n. Key observation: ω 2 n = exp(2πi/n)2 = exp(2πi/(n/2)) = ω n/2. Therefore, ω 2k n = ω k n/2 for all k. Hence, A(1) = A even (1) + 1 A odd (1), A(ω n ) = A even (ω n/2 ) + ω n A odd (ω n/2 ), A(ω 2 n) = A even (ω 2 n/2 ) + ω n A odd (ω 2 n/2 ),. A(ω n 2 1 n ) = A even (ω n 2 1 n/2 ) + ω n A odd (ω n 2 1 n/2 ). A. Coja-Oghlan (LFCS) Algorithms and data structures 9 / 15

14 The FFT (ctd.) How do we deal with A even (x 2 ), A odd (x 2 )? Goal: compute A even (ω 2k n ), A odd (ω 2k n ) for 0 k < n. Key observation: ω 2 n = exp(2πi/n)2 = exp(2πi/(n/2)) = ω n/2. Therefore, A(ωn) k = A even (ωn/2 k ) + ω n A odd (ωn/2 k ) for all k. Furthermore, as ω n/2 n = 1 and ω n n = 1, we can simplify for n 2 k < n: A(ω n 2 n ) = A even (1) 1 A odd (1), A(ω n 2 +1 n ) = A even (ω n/2 ) ω n A odd (ω n/2 ),. A(ωn n 1 ) = A even (ω n 2 1 n/2 ) ω n 2 1 n A odd (ω n 2 1 n/2 ). A. Coja-Oghlan (LFCS) Algorithms and data structures 10 / 15

15 The FFT (ctd.) How do we deal with A even (x 2 ), A odd (x 2 )? Goal: compute A even (ωn 2k), A odd(ωn 2k ) for 0 k < n. Summary: use the relations for 0 k < n 2. A(ω k n) = A even (ω k n/2 ) + ω n A odd (ω k n/2 ), A(ω n 2 +k n ) = A even (ω k n/2 ) ω n A odd (ω k n/2 ) A. Coja-Oghlan (LFCS) Algorithms and data structures 11 / 15

16 The FFT (ctd.) How do we deal with A even (x 2 ), A odd (x 2 )? Goal: compute A even (ωn 2k), A odd(ωn 2k ) for 0 k < n. Summary: use the relations for 0 k < n 2. Thus, all we need to know is A(ω k n) = A even (ω k n/2 ) + ω n A odd (ω k n/2 ), A(ω n 2 +k n ) = A even (ω k n/2 ) ω n A odd (ω k n/2 ) i.e., the DFTs of A even and A odd. A even (ω k n/2 ) for 0 k < n 2, A odd (ω k n/2 ) for 0 k < n 2, A. Coja-Oghlan (LFCS) Algorithms and data structures 11 / 15

17 The FFT (ctd.) Sketch of the algorithm Goal: compute the DFT of A(x), i.e., (A(ω k n)) 0 k<n, where n = 2 j. If n = 1, the result is just A(1) = n 1 i=0 a i. A. Coja-Oghlan (LFCS) Algorithms and data structures 12 / 15

18 The FFT (ctd.) Sketch of the algorithm Goal: compute the DFT of A(x), i.e., (A(ω k n)) 0 k<n, where n = 2 j. If n = 1, the result is just A(1) = n 1 i=0 a i. Suppose n > 1. Set up A even and A odd. Recurse to compute the DFTs of A even and A odd, i.e., (A even (ω k n/2 )) 0 k< n 2, (A odd(ω k n/2 )) 0 k< n 2. A. Coja-Oghlan (LFCS) Algorithms and data structures 12 / 15

19 The FFT (ctd.) Sketch of the algorithm Goal: compute the DFT of A(x), i.e., (A(ω k n)) 0 k<n, where n = 2 j. If n = 1, the result is just A(1) = n 1 i=0 a i. Suppose n > 1. Set up A even and A odd. Recurse to compute the DFTs of A even and A odd, i.e., (A even (ω k n/2 )) 0 k< n 2, (A odd(ω k n/2 )) 0 k< n 2. Since for 0 k < n 2 A(ω k n ) = A even(ω k n/2 ) + ω n A odd (ω k n/2 ), A(ω n 2 +k n ) = A even (ω k n/2 ) ω n A odd (ω k n/2 ) we can complete the task in O(n) operations. A. Coja-Oghlan (LFCS) Algorithms and data structures 12 / 15

20 The FFT (ctd.) Algorithm FFT(a 0,...,a n 1 ) Input: a vector (a 0,...,a n 1 ) C n. Output: (A(1),...,A(ω n 1 n )) C n. 1 If n = 1, return a 0. Otherwise do the following. 2 Let ω n = exp(2πi/n) and ω = 1. 3 Let (y 0,...,y n/2 1 ) = FFT(a 0,a 2,a 4,...,a n 2 ) // even part 4 Let (z 0,...,z n/2 1 ) = FFT(a 1,a 3,a 5,...,a n 1 ) // odd part A. Coja-Oghlan (LFCS) Algorithms and data structures 13 / 15

21 The FFT (ctd.) Algorithm FFT(a 0,...,a n 1 ) Input: a vector (a 0,...,a n 1 ) C n. Output: (A(1),...,A(ω n 1 n )) C n. 1 If n = 1, return a 0. Otherwise do the following. 2 Let ω n = exp(2πi/n) and ω = 1. 3 Let (y 0,...,y n/2 1 ) = FFT(a 0,a 2,a 4,...,a n 2 ) // even part 4 Let (z 0,...,z n/2 1 ) = FFT(a 1,a 3,a 5,...,a n 1 ) // odd part 5 For k = 0,..., n 2 1 do 6 let â k = y k + ωz k and â k+ n 2 = y k ωz k. 7 let ω = ω ω n. 8 Return (â 0,...,â n 1 ). We may assume that n = 2 j (if not we just pad the input with zeros). A. Coja-Oghlan (LFCS) Algorithms and data structures 13 / 15

22 The FFT (ctd.) Running time of FFT The running time satisfies the recurrence T(n) = 2T(n/2) + Θ(n). Master theorem T(n) = Θ(n lnn). By comparison, the naive algorithm takes time T(n) = Θ(n 2 ). Thus, we (almost) save a factor of n. Numerical issue: computing with complex numbers this can be avoided. In fact, FFT is easy to implement in hardware. A. Coja-Oghlan (LFCS) Algorithms and data structures 14 / 15

23 Reading assignment Check out... Fast Fourier Transform by Mary Cryan (see course website). [CLRS] Section A. Coja-Oghlan (LFCS) Algorithms and data structures 15 / 15