Why Fourier Transform? Fast Fourier Transform Jordi Cortadella and Jordi Petit Department of Computer Science Polynomials: coefficient representation Divide & Conquer Dept. CS, UPC Polynomials: point-value representation Fundamental Theorem (Gauss): A degree n polynomial with complex coefficients has exactly n complex roots. Corollary: A degree n 1 polymonial A(x) is uniquely identified by its evaluation at n distinct values of x. Divide & Conquer Dept. CS, UPC 3 Divide & Conquer Dept. CS, UPC
Polynomials: point-value representation Conversion between both representations representation addition multiplication evaluation coefficient O(n) O(n ) O(n) point-value O(n) O(n) O(n ) evaluation a 0, a 1,, a n 1 Coefficient representation interpolation x 0, y 0,, x n 1, y n 1 Point-value representation Could we have an efficient algorithm to move from coefficient to point-value representation and vice versa? Divide & Conquer Dept. CS, UPC 5 From coefficients to point-values Divide & Conquer Dept. CS, UPC 6 Credits: based on the intuitive explanation by Dasgupta, Papadimitriou and Vazinari, Algorithms, McGraw-Hill, 008. We want to evaluate A(x) at n different points. Let us choose them to be positive-negative pairs: ±x 0, ±x 1,, ±x Τ n 1 The computations for A(x i ) and A( x i ) overlap a lot. Split the polynomial into odd and even powers 3 + x + 6x + x 3 + x + 10x 5 = 3 + 6x + x + x + x + 10x The terms in parenthesis are polynomials in x : A x = A e x + xa o (x ) Divide & Conquer Dept. CS, UPC 7 Divide & Conquer Dept. CS, UPC 8
The calculations needed for A x i computing A x i. A x i = A e x i + x i A o x i A x i = A e x i x i A o x i Evaluating A x at n paired points ±x 0, ±x 1,, ±x nτ 1 can be reused for reduces to evaluating A e x and A o (x) at just n/ points: x 0,, x n/ 1 Evaluate: A x degree n 1 Evaluate: A e x and A o (x) degree n/ 1 +x 0 x 0 +x 1 x 1 +x n/ 1 x n/ 1 x 0 x 1 If we could recurse, we would get a running time: But can we recurse? x n/ 1 T n = T n/ + O n = O(n log n) Divide & Conquer Dept. CS, UPC 9 Divide & Conquer Dept. CS, UPC 10 Evaluate: A x degree n 1 +x 0 x 0 +x 1 x 1 +x n/ 1 x n/ 1 +1 1 +i i + i i + i i +x 0 x 0 +x 1 x 1 +x x +x 3 x 3 Evaluate: A e x and A o (x) degree n/ 1 x 0 x 1 x n/ 1 x 0 x 1 x +i x 3 +1 1 i The problem:? We need x 0 and x 1 to be a plus-minus pair. But a square cannot be negative! x 0 x +1 1 x 0 8 +1 Note: i = ± 1 1 + i i = ± 1 (1 i) Divide & Conquer Dept. CS, UPC 11 Divide & Conquer Dept. CS, UPC 1
Complex numbers: review Complex numbers: multiplication Imaginary b z = a + bi z = r(cos θ + i sin θ) = re iθ Polar coordinates: (r, θ) (r, θ ) (r 1, θ 1 ) r Length: r = a + b r 1, θ 1 r, θ = (r 1 r, θ 1 + θ ) θ a Real Angle θ 0,π : cos θ = a r, sin θ = b r θ can always be reduced modulo π (r 1 r, θ 1 + θ ) For any z = r, θ : Some examples: z = (r, θ + π), since 1 = 1, π Number 1 i 5 + 5i Polar coords (1, π) (1, π/) (5, Τ π ) If z is on the unit circle, then z n = (1, nθ) Divide & Conquer Dept. CS, UPC 13 Complex numbers: the nth roots of unity Divide & Conquer Dept. CS, UPC 1 Divide-and-conquer step Solutions to the equation z n = 1 (n = 16) π/n Solutions are z = 1, θ, for θ a multiple of π/n Angle π/n All roots are plus-minus paired: π n + π 1, θ = (1, θ + π) Evaluate A(x) at nth roots of unity Evaluate A e (x) and A o x at nτ nd roots of unity Divide & Conquer Dept. CS, UPC 15 Divide & Conquer Dept. CS, UPC 16
Divide-and-conquer steps Roots of unity for n = 8 +i i + i 1 +1 i + i i Divide & Conquer Dept. CS, UPC 17 Fast Fourier Transform Divide & Conquer Dept. CS, UPC 18 Fast Fourier Transform: example with n = function FFT(A,ω) Inputs: A = (a 0, a 1,, a n 1 ), for n a power of ω: A primitive nth root of unity Output: A 1, A ω, A ω,, A ω n 1 if ω=1: return A function FFT((a 0, a 1, a, a 3 ),ω) A e (ω 0 ), A e (ω ) = FFT( a 0, a, ω ) A o ω 0, A o (ω ) = FFT( a 1, a 3, ω ) ω 0 = 1 ω 1 = i ω = 1 ω 3 = i A e ω 0, A e ω,, A e (ω n ) = FFT(A e, ω ) A o ω 0, A o ω,, A o (ω n ) = FFT(A o, ω ) for k = 0 to n 1: A ω k = A e ω k + ω k A o (ω k ) return A 1, A ω, A ω,, A(ω n 1 ) A ω 0 = A e ω 0 + ω 0 A o ω 0 A ω 1 = A e ω + ω 1 A o ω A ω = A e ω + ω A o ω = A e ω 0 ω 0 A o ω 0 A ω 3 = A e ω 6 + ω 3 A o ω 6 = A e ω ω 1 A o ω return A 1, A ω, A ω, A(ω 3 ) Divide & Conquer Dept. CS, UPC 19 Divide & Conquer Dept. CS, UPC 0
Fast Fourier Transform function FFT(a,ω) Inputs: a = (a 0, a 1,, a n 1 ), for n a power of ω: A primitive nth root of unity Output: a 1, a ω, a ω,, a ω n 1 if ω=1: return a s 0, s 1,, s nτ 1 = FFT( a 0, a,, a n, ω ) s 0, s 1,, s nτ 1 = FFT( a 1, a 3,, a n 1, ω ) for k = 0 to nτ 1: r k = s k + ω k s k r k+ nτ = s k ω k s k return (r 0, r 1,, r n 1 ) FFT: asymptotic complexity The runtime of the FFT can be expressed as: T n = T Τ n + O n Using the master theorem we conclude: Runtime FFT n = O(n log n) Gilbert Strang (MIT, 199): the most important numerical algorithm of our lifetime. Reference: Cooley, James W., and John W. Tukey, 1965, An algorithm for the machine calculation of complex Fourier series, Math. Comput. 19: 97-301. Divide & Conquer Dept. CS, UPC 1 Unfolding the FFT Divide & Conquer Dept. CS, UPC Unfolding the FFT (butterfly diagram) 000 a 0 A(ω 0 ) 000 a 0 a a n FFT nτ + ω k r k 100 010 110 a a a 6 6 A(ω 1 ) A(ω ) A(ω 3 ) 001 010 011 a 1 a 3 a n 1 FFT nτ ω k+ nτ r k+ nτ Divide & Conquer Dept. CS, UPC 3 + 001 a 1 101 a 5 011 a 3 111 a 7 1 5 3 6 6 A(ω ) A(ω 5 ) A(ω 6 ) A(ω 7 ) Divide & Conquer Dept. CS, UPC 7 100 101 110 111
Why is it called a butterfly diagram? Conversion between both representations representation addition multiplication evaluation coefficient O(n) O(n ) O(n) point-value O(n) O(n) O(n ) a 0, a 1,, a n 1 Coefficient representation values = FFT( coefficients, ω) evaluation O(n log n) interpolation x 0, y 0,, x n 1, y n 1 Point-value representation Divide & Conquer Dept. CS, UPC 5 From point-values to coefficients Divide & Conquer Dept. CS, UPC 6 From point-values to coefficients Divide & Conquer Dept. CS, UPC 7 Divide & Conquer Dept. CS, UPC 8
Conversion between both representations Polynomial multiplication Input: Coefficients of two polynomials A(x) and B(x), of degree d A and d B, respectively. Let d = d A + d B. representation addition multiplication evaluation coefficient O(n) O(n ) O(n) point-value O(n) O(n) O(n ) values = FFT( coefficients, ω) evaluation Output: The product C = A B. 1. Selection: Pick ω = (1, πτn), such that n d + 1 and n is a power of two.. Evaluation (FFT): Compute A 1, A ω, A ω,, A ω n 1. Compute B 1, B ω, B ω,, B ω n 1. a 0, a 1,, a n 1 Coefficient representation O(n log n) interpolation x 0, y 0,, x n 1, y n 1 Point-value representation 3. Multiplication: Compute C ω k = A ω k B(ω k ), for all k = 0,, n 1. coefficients = 1 n FFT( values, ω 1 ). Interpolation (inverse FFT): Recover C x = c 0 + c 1 x + c x + + c d x d. Divide & Conquer Dept. CS, UPC 9 FFT application in Signal Processing Divide & Conquer Dept. CS, UPC 30 Distinguishing instruments Same note (frequency). Different timbre (spectral envelope). Converting a signal: time domain frequency domain Divide & Conquer Dept. CS, UPC 31 Divide & Conquer Dept. CS, UPC 3
Speech Spectrogram Tone: distance between sidelobes (vocal cords). Sound: spectral envelope. Pronouncing veintisiete Divide & Conquer Dept. CS, UPC 33 Exercises Divide & Conquer Dept. CS, UPC 3 1. Consider the polynomials 1 + x x + x 3 and 1 + x : Choose an appropriate power of two to execute the FFT for the polynomial multiplication. Find the value of ω. Give the result of the FFT for x 1 (no need to execute the FFT).. Consider the polynomials 1 + x + x and 1 + x: Choose an appropriate power of two to execute the FFT. Find the value of ω. Calculate their point-value representation using the FFT (execute the FFT algorithm manually). Calculate the product of the point-value representations. Execute the inverse FFT to obtain the coefficients of the product. Divide & Conquer Dept. CS, UPC 35