Computational Methods CMSC/AMSC/MAPL 460 Fourier transform Balaji Vasan Srinivasan Dept of Computer Science Several slides from Prof Healy s course at UMD
Last time: Fourier analysis F(t) = A 0 /2 + A 1 Cos( t) + A 2 Cos(2 t) + A 3 Cos(3 t) + + B 1 Sin( t) + B 2 Sin(2 t) + B 3 Sin(3 t) + = A 0 /2 + A 1 Cos( t) + i (-ib 1 ) Sin( t) + A 2 Cos(2 t) + i (-ib 2 ) Sin(2 t) + = F 0 + F 1 e -it + F 2 e -2it + F k = (1/p) 0 p f(t) e - 2 π i k t/p dt
Last time: Discrete Fourier Transform f(t) φ[n] ; F(k) Φ[k] Discrete Fourier Transform: p F k = (1/p) 0 f(t) e - 2 π i k t/p dt N-1 Φ[k] =1/N Σ φ[n] e -2 π i k n h /N Inverse Discrete Fourier Transform: N-1 n =0 φ[n] = Σ Φ[k] e 2 π i k n /N k=0
Discrete time Numerical Fourier Analysis DFT is really just a matrix multiplication! Φ[0] Φ[1] Φ[2] Φ[N-1] Φ [κ] = 1/Ν = Freq index ex60 50 40 30 20 10 0 N-1 Σ m =0 e time index Φ = F N -2 π i k m/n φ[m] 0 10 20 30 40 50 60 φ[0] φ[1] φ[2] φ[n-1] φ
Numerical Harmonic Analysis FFT: Symmetry Properties permits Divide and Conquer Sparse Factorization 15000 12500 10000 7500 5000 2500 60 50 40 30 20 10 0 0 10 20 30 40 50 60 F n Naive FFT 20 40 60 80 100 120
Structured matrices Fast algorithms have been found for many dense matrices Typically the matrices have some structure Definition: A dense matrix of order N N is called structured if its entries depend on only O(N) parameters Most famous example the fast Fourier transform
Fourier Matrices
i = 1 Primitive Roots of Unity A number ω is a primitive n-th root of unity, for n>1, if ω n = 1 The numbers 1, ω, ω 2,, ω n-1 are all distinct Example: The complex number e 2pi/n is a primitive n-th root of unity, where Check: if properties are satisfied w 4 w 3 w 2 Imaginary 2πi w 2 45 o cos(π/4) w 1 =cos(π/4) + i w 8 Real 1 n 1 ω = e 1 n 2 i n π n 2πi 2 ω = e = e = cos 2π + i sin 2π = 1 n 1 jp 0 j 2 j 3 j j( n 1) 3 S= ω = ω + ω + ω + ω + + ω = 0 p= 0 w 5 w 6 w 7 n complex roots of unity equally spaced around the circle of unit radius centered at the origin of the complex plane
Roots of Unity: Properties Property 1: Let ω be the principal n th root of unity If n > 0, then ω n/2 = -1 Proof: ω = e 2π i / n ω n/2 = e π i = -1 (Euler's formula) Reflective Property: Corollary: ω k+n/2 = -ω k Property 2: Let n > 0 be even, and let ω and ν be the principal n th and (n/2) th roots of unity Then (ω k ) 2 = ν k Proof: (ω k ) 2 = e (2k)2π i / n = e (k) 2π i / (n / 2) = ν k Reduction Property: If ω is a primitive (2n)-th root of unity, then ω 2 is a primitive n-th root of unity
L3: Let n > 0 be even Then, the squares of the n complex n th roots of unity are the n/2 complex (n/2) th roots of unity Proof: If we square all of the n th roots of unity, then each (n/2) th root is obtained exactly twice since: L1 ω k + n / 2 = - ω k thus, (ω k + n / 2 ) 2 = (ω k ) 2 L2 both of these = ν k ω k + n / 2 and ω k have the same square Inverse Property: If ω is a primitive root of unity, then ω -1 =ω n-1 Proof: ωω n-1 =ω n =1
Fast Fourier Transform Presented by Cooley and Tukey in 1965, but invented several times, including by Gauss (1809) and Danielson & Lanczos (1948) Danielson Lanczos lemma
So far we have seen what happens on the right hand side How about the left hand side? When we split the sums in two we have two sets of sums with N/2 quantities for N points So the complexity is N 2 /2 + N 2 /2 =N 2 So there is no improvement Φ[0] Φ[1] Φ[2] Φ[N-1] = 50 40 30 Freq index60 20 10 0 time index 0 10 20 30 40 50 60 φ[0] φ[1] φ[2] φ[n-1]
Need to reduce the number of sums on the left hand side We need to reduce the number of sums computed from 2N to a lower number Notice that the values corresponding to k and k+n/2 will be the same The transforms F ek and F ok are periodic in k with length N/2 So we need only compute half of them! 1 08 06 04 02 1 2 3 4 5 6 7 8 9 10 1112 1314 1516 1718 1920 21
FFT So DFT of order N can be expressed as sum of two DFTs of order N/2 evaluated at N/2 points Does this improve the complexity? Yes (N/2) 2 +(N/2) 2 =N 2 /2< N 2 But we are not done Can apply the lemma recursively Finally we have a set of one point transforms One point transform is identity
FFT Algorithm FFT (n, a 0, a 1, a 2,, a n-1 ) if (n == 1) // n is a power of 2 return a 0 ω e 2π i / n (e 0,e 1,e 2,,e n/2-1 ) FFT(n/2, a 0,a 2,a 4,,a n-2 ) (d 0,d 1,d 2,,d n/2-1 ) FFT(n/2, a 1,a 3,a 5,,a n-1 ) for k = 0 to n/2-1 y k e k + ω k d k y k+n/2 e k - ω k d k O(n) complex multiplies if we pre-compute ω k return (y 0,y 1,y 2,,y n-1 ) T ( n) = 2T ( n / 2) + O( n) T ( n) = O( nlogn)
Complexity Each F k is a sum of log 2 N transforms and (factors) There are N F k s So the algorithm is O(N log 2 N) This is a recursive algorithm
function y = ffttx(x) %FFTTX Textbook Fast Finite Fourier Transform % FFTTX(X) computes the same finite Fourier transform as FFT(X) % The code uses a recursive divide and conquer algorithm for % even order and matrix-vector multiplication for odd order % If length(x) is m*p where m is odd and p is a power of 2, the % computational complexity of this approach is O(m^2)*O(p*log2(p)) x = x(:); n = length(x); omega = exp(-2*pi*i/n); FFTtx if rem(n,2) == 0 % Recursive divide and conquer k = (0:n/2-1)'; w = omega ^ k; u = ffttx(x(1:2:n-1)); v = w*ffttx(x(2:2:n)); y = [u+v; u-v]; else % The Fourier matrix j = 0:n-1; k = j'; F = omega ^ (k*j); y = F*x; end
Scrambled Output of the FFT a 0, a 1, a 2, a 3, a 4, a 5, a 6, a 7 a 0, a 2, a 4, a 6 a 1, a 3, a 5, a 7 a 0, a 4 a 2, a 6 a 1, a 5 a 3, a 7 a 0 a 4 a 2 a 6 a 1 a 5 a 3 a 7 "bit-reversed" order
FFT and IFFT The FFT has revolutionized many applications by reducing the complexity by a factor of almost n Can relate many other matrices to the Fourier Matrix
Circulant Matrices Toeplitz Matrices Hankel Matrices Vandermonde Matrices
Modern signal processing very strongly based on the FFT One of the defining inventions of the 20 th century Applications: digital telephones, digital TV, MP3 players, etc