Sequential Fast Fourier Transform

Size: px

Start display at page:

Download "Sequential Fast Fourier Transform"

Florence O’Brien’
5 years ago
Views:

Sequential Fast Fourier Transform Departement Computerwetenschappen Room 03.

1 Sequential Fast Fourier Transform Departement Computerwetenschappen Room (Celestijnenlaan 200A) Includes material from slides by Prof. dr. Rob H. Bisseling

2 Applications of Fourier analysis Fourier analysis studies the decomposition of functions into their frequency components. Piano Concerto no. 9 by Mozart: enhance high frequencies. Chest picture by Computerised Tomography: reconstruct your interior without slicing you up. Star picture by pre-repair Hubble Space Telescope: remove blur. Sequential FFT 2 / 18

3 Fourier Series General idea: every signal is built out of harmonic parts. (For example, a linear combination of various sinusoids each with varying frequency and amplitude.)

4 Fourier Series Given a T -periodic function f : R C, with f piecewise continuous. Then the Fourier-coefficients {ˆf k } k Z are defined as: ˆf k = 1 T The Fourier series are given as: S n = T 0 f (t)e 2πik t T dt. n k= n ˆf k e 2πik t T. In accordance with the general idea, would like that lim S n = f. n

5 Fourier Inverse theorems f : R C is T -periodic and lim n S n = f, if

6 Fourier Inverse theorems f : R C is T -periodic and lim n S n = f, if f is piecewise differentiable, ( x R, 1 2 (lim y x f (y) + lim y x f (y)) = lim n S n )

7 Fourier Inverse theorems f : R C is T -periodic and lim n S n = f, if f is piecewise differentiable, ( x R, 1 2 (lim y x f (y) + lim y x f (y)) = lim n S n ) f is two-times differentiable (f C 2 ), ( x R, f (x) = lim n S n ; convergence in norm)

8 Fourier Inverse theorems f : R C is T -periodic and lim n S n = f, if f is piecewise differentiable, ( x R, 1 2 (lim y x f (y) + lim y x f (y)) = lim n S n ) f is two-times differentiable (f C 2 ), ( x R, f (x) = lim n S n ; convergence in norm) f is in L 1 T (i.e., T 0 f (t) dt converges) as well as in C 1, or (lim n f S n = 0)

9 Fourier Inverse theorems f : R C is T -periodic and lim n S n = f, if f is piecewise differentiable, ( x R, 1 2 (lim y x f (y) + lim y x f (y)) = lim n S n ) f is two-times differentiable (f C 2 ), ( x R, f (x) = lim n S n ; convergence in norm) f is in L 1 T (i.e., T 0 f (t) dt converges) as well as in C 1, or (lim n f S n = 0) f is in L 2 T (i.e., T 0 f (t) 2 dt converges). (lim n f S n 2 = 0) (Being piecewise continuous is not enough.)

10 Fourier Inverse theorems for non-periodic input If f lives on a bounded domain (f : [0, T ] C), as for example with a time-finite input signal, we can copy the signal to form an aritificial T -periodic function g: g : R C, with g(t) = f (t T /2) (for t [ T /2, T /2], and repeated elsewhere). The inverse is correct if g meets any of the previous conditions.

11 Fourier Inverse theorems for non-periodic input Let f, ĝ : R C (unbounded in time) bounded in 2-norm (f, ĝ L 2 ), (Ff ) s : k s (F g) s : t then s s f (t)e 2πitk dt, and s ĝ(t)e 2πitk dt. ˆf L 2 such that lim s ˆf (Ff ) s 2 = 0, and g L 2 such that lim s g (F ˆf ) s 2 = 0, and F Fg = g. (Depending on f, (F Ff ) s may converge uniformly or point-wise as s ).

12 It s a discrete world One second of audio on a compact disc contains 44,100 function values f (t j ) in regularly spaced sample points t j = jt n, 0 j < n. Sequential FFT 5 / 18

13 Approximation of Fourier coefficients Trapezoidal rule on interval [t j, t j+1 ] = tj+1 On the whole interval [0, T ]: c k = 1 T f (t) dt f (t j) + f (t j+1 ) t j 2 T 0 1 T T n [ ] jt n, (j+1)t n : T n. f (t)e 2πikt/T dt f (0) n f (t j )e 2πikt j /T + f (T ) 2 j=1 = 1 n 1 f (t j )e 2πijk/n (since f (0) = f (T ) = f (t 0 )). n Sequential FFT 6 / 18

14 Discrete Fourier Transform We define the discrete Fourier transform (DFT): Definition. DFT : C n C n with y = DFT (x) such that n 1 y k = x j e 2πijk/n, for each k [0, n 1] (x, y C n ). (Note this skips the factor 1/n from the repeated trapezoidal rule.)

15 Inverse discrete Fourier transform Definition. invdft : C n C n with y = invdft (x) such that x j = 1 n 1 y k e 2πijk/n, for each k [0, n 1] (x, y C n ). n k=0 (Note the factor 1/n reappears here.) We have that for all x C n, invdft (DFT (x)) = x. But how close is DFT (x) to the true Ff?

16 Matrix vector multiplication Define the n n Fourier matrix F n by Hence F n x = DFT (x): (F n ) jk = ω n jk, for 0 j, k < n. n 1 n 1 (F n x) j = (F n ) jk x k = x k ω jk n = (DFT (x)) j. k=0 k=0 Because ω 4 = e 2πi/4 = e πi/2 = i: ω 0 4 ω 0 4 ω 0 4 ω 0 4 F 4 = ω 0 4 ω 1 4 ω 2 4 ω 3 4 ω 0 4 ω 2 4 ω 4 4 ω 6 4 = ω 0 4 ω 3 4 ω 6 4 ω i 1 i i 1 i. Sequential FFT 10 / 18

17 Roots of unity ω 5 ω 6 = i ω 7 ω 4 ω 0 = 1 ω 3 ω 2 ω 1 ω 8 = e 2πi/8 = e πi/4 = i. ω n n = e 2πin/n = e 2πi = 1. ω n n/2 = e 2πi(n/2)/n = e πi = 1. ω n 2 = e 4πi/n = e 2πi/(n/2) = ω n/2. Sequential FFT 9 / 18

18 Cost of straightforward DFT Complex multiplication (a + bi)(c + di) = (ac bd) + (ad + bc)i requires 1 real addition, 1 real subtraction, 4 real multiplications, hence a total of 6 flops. Complex addition (a + bi) + (c + di) = (a + c) + (b + d)i requires 2 real additions. To compute y k, we need n complex multiplications and n 1 complex additions, so 6n + 2(n 1) = 8n 2 flops. To compute the n components of y, we need 8n 2 2n flops. Sequential FFT 11 / 18

19 Towards FFT: divide... Reducing the DFT flop-cost: y k = n 1 x jω jk n, split the sum in odd and even parts: y k = n/2 1 n/2 1 x 2j wn 2jk + x 2j+1 ω (2j+1)k n. Since ωn 2jk = ω jk n/2 (roots of unity: the 2c th root out of n roots is the c th root out of n/2 roots): y k = n/2 1 n/2 1 x 2j w jk n/2 + ωk n x 2j+1 ω jk n/2. These are DFTs of half length each!

20 Towards FFT: divide... Reducing the DFT flop-cost: y k = n 1 x jω jk n, split the sum in odd and even parts: y k = n/2 1 n/2 1 x 2j wn 2jk + x 2j+1 ω (2j+1)k n. Since ωn 2jk = ω jk n/2 (roots of unity: the 2c th root out of n roots is the c th root out of n/2 roots): y 0 k<n/2 = DFT (x even )+Ω n DFT (x odd ), with Ω n = diag(ω 0 n,..., ω n/2 1 n ). These are DFTs of half length each!

21 Towards FFT: divide..., divide... Split up y in two parts of length n/2 as well: y top = DFT (x even ) + Ω n DFT (x odd ), with Ω n = diag(ω 0 n,..., ω n/2 1 n ). Substitute k = k n/2 for the remaining half of y: n/2 1 y0 k bottom <n = n/2 1 x 2j w j(n/2+k ) n/2 + ωn k Note 1 that ω jn/2 n/2 = (ωn/2 n/2 )j = 1 and k = k + n/2, x 2j+1 ω j(n/2+k ) n/2. 1, roots of unity: the n th root out of n roots equals 1.

22 Towards FFT: divide..., divide... Split up y in two parts of length n/2 as well: y top = DFT (x even ) + Ω n DFT (x odd ), with Ω n = diag(ω 0 n,..., ω n/2 1 n ). Substitute k = k n/2 for the remaining half of y: n/2 1 y0 k bottom <n = n/2 1 x 2j w j(n/2+k ) n/2 + ωn k x 2j+1 ω j(n/2+k ) n/2. Note 1 that ω jn/2 n/2 = (ωn/2 n/2 )j = 1 and k = k + n/2, hence: n/2 1 y0 k bottom <n = n/2 1 x 2j w jk n/2 + +n/2 ωk n x 2j+1 ω jk n/2 ; 1, roots of unity: the n th root out of n roots equals 1.

23 Towards FFT: divide..., divide... Split up y in two parts of length n/2 as well: y top = DFT (x even ) + Ω n DFT (x odd ), with Ω n = diag(ω 0 n,..., ω n/2 1 n ). Substitute k = k n/2 for the remaining half of y: n/2 1 y0 k bottom <n = n/2 1 x 2j w jk n/2 + +n/2 ωk n Note 2 that ω k +n/2 n = ω n/2 n ω k n = ω k n, x 2j+1 ω jk n/2. 2, roots of unity: the n/2 th root out of n roots always ends up halfway the circle, at 1.

24 Towards FFT: divide..., divide... Split up y in two parts of length n/2 as well: y top = DFT (x even ) + Ω n DFT (x odd ), with Ω n = diag(ω 0 n,..., ω n/2 1 n ). Substitute k = k n/2 for the remaining half of y: n/2 1 y0 k bottom <n = n/2 1 x 2j w jk n/2 + +n/2 ωk n Note 2 that ω k +n/2 n = ω n/2 n ω k n = ω k n, hence: n/2 1 y0 k bottom <n = n/2 1 x 2j w jk n/2 ωk n x 2j+1 ω jk n/2. x 2j+1 ω jk n/2. 2, roots of unity: the n/2 th root out of n roots always ends up halfway the circle, at 1.

25 Towards FFT: divide..., divide..., combine... Splitting up y in two parts thus yields: y top = DFT (x even ) + Ω n DFT (x odd ) y bottom = DFT (x even ) Ω n DFT (x odd ), with Ω n = diag(ω 0 n,..., ω n/2 1 n ).

26 Towards FFT: divide..., divide..., combine... Splitting up y in two parts thus yields: y top = DFT (x even ) + Ω n DFT (x odd ) y bottom = DFT (x even ) Ω n DFT (x odd ), with Ω n = diag(ωn, 0..., ωn n/2 1 ). Thus a DFT of size n is split in two half-size DFTs plus n/2 multiplications and n additions (Ω-twiddling). a DFT of size n/2 takes 8n 2 /4 n flops, two half-size DFTs take 4n 2 2n flops, n/2 complex multiplications take 3n flops, n complex additionals take 2n flops, so the above divide & combine strategy costs 4n 2 + 3n flops! (Compared to the regular cost of 8n 2 n/2.)

27 FFT: divide, combine, and conquer! Algortihm FFT (x); input: x C n with n a power of two, output: y C n with y = DFT (x): if n = 1 then return y = x else ( y t Let y = y b ) be a vector of size n (x e, x o ) = odd-even-sort(x) τ = Ω n FFT (x o ) y t = FFT (x e ) + τ y b = FFT (x o ) τ return y end

28 How fast is fast? Cost of the recursive FFT: τ = Ω n FFT (x o ): n/2 complex multiplications. y t = FFT (x e ) + τ: n/2 complex additions, y b = FFT (x o ) τ: n/2 complex additions; totals to 5n flops per call to the FFT.

29 Sequential FFT Recursive computation of DFT The problem is split repeatedly, until the problem size is / 18

30 How fast is fast? Cost of the recursive FFT: 5n flops per call to the FFT. Two recursive FFT calls if n > 1: T (n) = 2T (n/2) + 5n. This is linear: for n > 8, T (n) =

31 How fast is fast? Cost of the recursive FFT: 5n flops per call to the FFT. Two recursive FFT calls if n > 1: T (n) = 2T (n/2) + 5n. This is linear: for n > 8, T (n) = 2T (n/2) + 5n =

32 How fast is fast? Cost of the recursive FFT: 5n flops per call to the FFT. Two recursive FFT calls if n > 1: T (n) = 2T (n/2) + 5n. This is linear: for n > 8, T (n) = 2T (n/2) + 5n = 4T (n/4) + 2 5n =

33 How fast is fast? Cost of the recursive FFT: 5n flops per call to the FFT. Two recursive FFT calls if n > 1: T (n) = 2T (n/2) + 5n. This is linear: for n > 8, T (n) = 2T (n/2) + 5n = 4T (n/4) + 2 5n = 8T (n/8) =... ;

34 How fast is fast? Cost of the recursive FFT: 5n flops per call to the FFT. Two recursive FFT calls if n > 1: T (n) = 2T (n/2) + 5n. This is linear: for n > 8, T (n) = 2T (n/2) + 5n = 4T (n/4) + 2 5n = 8T (n/8) =... ; that is, each level of the recursion tree contains a total of 5n flops. The tree height is exactly 1 + log 2 n,

35 How fast is fast? Cost of the recursive FFT: 5n flops per call to the FFT. Two recursive FFT calls if n > 1: T (n) = 2T (n/2) + 5n. This is linear: for n > 8, T (n) = 2T (n/2) + 5n = 4T (n/4) + 2 5n = 8T (n/8) =... ; that is, each level of the recursion tree contains a total of 5n flops. The tree height is exactly 1 + log 2 n, but T (1) = 0 (no flops, just copying of a single complex value).

36 How fast is fast? Cost of the recursive FFT: 5n flops per call to the FFT. Two recursive FFT calls if n > 1: T (n) = 2T (n/2) + 5n. This is linear: for n > 8, T (n) = 2T (n/2) + 5n = 4T (n/4) + 2 5n = 8T (n/8) =... ; that is, each level of the recursion tree contains a total of 5n flops. The tree height is exactly 1 + log 2 n, but T (1) = 0 (no flops, just copying of a single complex value). Hence the total cost of the recursive FFT is 5n log 2 n.

Summary The fast Fourier transform (FFT) idea was discovered by Gauss (1805), rediscovered by Danielson and Lanczos (1942), and is commonly attributed to Cooley and Tukey (1965), who rediscovered it

37 Summary The fast Fourier transform (FFT) idea was discovered by Gauss (1805), rediscovered by Danielson and Lanczos (1942), and is commonly attributed to Cooley and Tukey (1965), who rediscovered it in the digital era. The FFT is the computational workhorse in many applications, from weather forecasting to signal and image processing. Without the FFT, modern medicine would be impossible. The cost of an FFT of length n is 5n log 2 n flops. We have derived a recursive FFT algorithm, i.e., an algorithm that calls itself with a smaller problem size. Sequential FFT 18 / 18

Sequential Fast Fourier Transform (PSC ) Sequential FFT

Sequential Fast Fourier Transform (PSC 3.1 3.2) 1 / 18 Applications of Fourier analysis Fourier analysis studies the decomposition of functions into their frequency components. Piano Concerto no. 9 by