Queens College, CUNY, Department of Computer Science Numerical Methods CSCI 361 / 761 Spring 2018 Instructor: Dr. Sateesh Mane.

Size: px

Start display at page:

Download "Queens College, CUNY, Department of Computer Science Numerical Methods CSCI 361 / 761 Spring 2018 Instructor: Dr. Sateesh Mane."

Christian Ezra Wood
5 years ago
Views:

1 Queens College, CUNY, Department of Computer Science Numerical Methods CSCI 361 / 761 Spring 018 Instructor: Dr. Sateesh Mane c Sateesh R. Mane Lecture 30 April 7, 018 Fourier Series In this lecture, we continue our study of Fourier series. We shall treat functions of one variable only. We shall study algorithms to perform computations for Fourier series. We shall study the Discrete Fourier Transform (DFT). A particularly important concept is the Fast Fourier Transform (FFT). This lecture requires knowledge of complex numbers. 1

2 30.1 Fourier series using sines and cosines This is a review from a previous lecture, reproduced here for ease of reference. Let f(θ) be a periodic function of and angle θ with period π. We assume f is sufficiently well behaved to justify the relevant calculations or algorithms. 1. For example, the function is absolutely integrable around a circle: π 0 f(θ) dθ <. (30.1.1). The function also has at most a finite number of discontinuities in the interval 0 θ < π. Then f can be expressed (or expanded) in a Fourier series as follows: f(θ) = 1 a 0 + a 1 cos(θ) + a cos(θ) + a 3 cos(3θ) + + b 1 sin(θ) + b sin(θ) + b 3 sin(3θ) + = 1 a [ 0 + aj cos(jθ) + b j sin(jθ) ]. j=1 (30.1.) The coefficients a j and b j are obtained from the function f(θ) via a j = 1 π b j = 1 π π 0 π 0 f(θ) cos(jθ) dθ (j 0), f(θ) sin(jθ) dθ (j > 0). (30.1.3)

3 30. Representation using complex exponentials In all the examples we have studied, the function f(θ) is real, hence all the Fourier coefficients a j amd bj are real. However, this is not the best formulation for efficient numerical computation. Let us reexpress eq. (30.1.) using complex exponentials. Note that e ijθ + e ijθ e ijθ e ijθ a j cos(jθ) + b j sin(jθ) = a j + b j i = 1 [ ] (a j ib j ) e ijθ + (a j + ib j ) e ijθ. (30..1) Let us therefore define complex Fourier harmonics via F 0 = a 0, F j = a j ib j, F j = a j + ib j (j > 0). (30..) The Fourier series using complex Fourier harmonics is given by f(θ) = j= F j e ijθ. (30..3) This is a discrete sum, not an integral, but has the appearance of an inverse Fourier transform. Note the following orthogonality and normalization relations: 1 π e i(m n)θ = δ mn ( < m, n < ). (30..4) π 0 Then the value of F j is obtained from the function f(θ) via F j = 1 π π 0 f(θ) e ijθ dθ ( < j < ). (30..5) The integral is over a finite domain, not the entire real line, but has the appearance of a Fourier transform. 3

4 30.3 Example using complex Fourier harmonics Let us study the same example treated previously for a Fourier series. Let us examine the results when we employ complex Fourier harmonics. The example function is f ex (θ) = 1 A 0 + A 1 cos(θ) + B 1 sin(θ) + A cos(θ) + B sin(θ). (30.3.1) We employ n = 4 points. Hence θ k = kπ/4 = kπ/, for k = 0, 1,, 3. Hence we evaluate the integral in eq. (30..5) using a discrete sum with n points: F j = 1 n n 1 f(θ k ) e ijθ k (j = 0,..., n 1). (30.3.) k=0 For n = 4 and the example function in eq. (30.3.1), this yields F j = 1 [ ] f ex (0) + f ex ( 1 4 π)e ijπ/ + f ex (π)e ijπ + f ex ( 3 π)e ijπ/ = 1 [ ( 1 4 A 0 + A 1 + A ) + ( 1) j ( 1 A 0 + B 1 A ) ] + ( 1) j ( 1 A 0 A 1 + A ) + i j ( 1 A 0 B 1 A ). (30.3.3) We obtain the following results for F 0,..., F 3 : F 0 = 1 4 F 1 = 1 4 F = 1 4 F 3 = 1 4 [ ] ( 1 A 0+A 1 +A )+( 1 A 0+B 1 A )+( 1 A 0 A 1 +A )+( 1 A 0 B 1 A ) [ ( 1 A 0+A 1 +A ) i( 1 A 0+B 1 A ) ( 1 A 0 A 1 +A )+i( 1 A 0 B 1 A ) [ ] ( 1 A 0+A 1 +A ) ( 1 A 0+B 1 A )+( 1 A 0 A 1 +A ) ( 1 A 0 B 1 A ) [ ( 1 A 0+A 1 +A )+i( 1 A 0+B 1 A ) ( 1 A 0 A 1 +A ) i( 1 A 0 B 1 A ) ] ] = 1 A 0, (30.3.4a) = A 1 ib 1, (30.3.4b) =A, (30.3.4c) = A 1 + ib 1. (30.3.4d) The values of F 0 and F 1 are as expected. We calculated the Fourier coefficient F 3 as opposed to F 1, but we obtained the value of F 1 anyway: F 1 = F 3 = A 1 + ib 1. (30.3.5) In general, for any value of n (at least if n is even), the Fourier coefficients wrap around because of the periodicity of the function: F j = F n j. (30.3.6) The Fourier coefficient F n/, i.e. F in our example, yields F n/ = A n/ not 1 A n/. Is this a problem? Not necessarily. 4

5 30.4 Consequences of employing complex Fourier harmonics Let us sum the Fourier series obtained in the example in Sec We obtain the following sum, using eq. (30..3): f sum (θ) = F 0 + F 1 e iθ + F 1 e iθ + F e iθ = 1 A 0 + A 1 ib 1 e iθ + A 1 + ib 1 e iθ + A e iθ = 1 A [ ] 0 + A 1 cos(θ) + B sin(θ) + A cos(θ)+i sin (θ). (30.4.1) We obtain an extra term ia sin(θ) which does not exist in the original function. As we know, we do not obtain the term B sin(θ), which does exist in the original function. Clearly the sum of the series is inaccurate for the j = n/ harmonic, i.e. θ in this example. However, are we making unrealistic demands? We evaluated the function f ex (θ) at only four points. It is unreasonable to expect that the resulting Fourier series will match the original function exactly for all values of θ. Let us evaluate the sum of the series only at the input points θ k = kπ/n, i.e. only at the input values of θ used to derive the series. Then sin(θ) = 0 at all the points θ k = kπ/n, hence the sum of the series equals the original function exactly: f sum (0) = 1 A 0 + A 1 + A =f ex (0), (30.4.a) f sum ( 1 π) = 1 A 0 + B 1 A =f ex ( 1 π), (30.4.b) f sum (π) = 1 A 0 A 1 + A =f ex (π), (30.4.c) f sum ( 3 π) = 1 A 0 B 1 A =f ex ( 3 π). (30.4.d) We therefore deduce the following important fact: In general, for any value of n, the sum of the Fourier series, using complex Fourier harmonics, exactly equals the original function at the n input points θ k = kπ/n. The claim exactly equals the original function must however be interpreted with care. 1. If a continuous signal is sampled using a finite number of values, the input data will in general contain aliased information.. Recall the statements in previous lectures about the Nyquist frequency and aliasing. The statement the sum of the Fourier series,..., exactly equals the original function at the n input points θ k = kπ/n is correct, but the original data may contain aliased information. Using complex Fourier coefficients does not solve the aliasing problem. 5

6 30.5 Convolution of discrete series In the continuous case, the convolution of two functions f(x) and g(x) is given by (f g)(x) = For periodic functions, the convolution is given by a discrete sum. Let the periodic functions be f(θ) and g(θ). f(u)g(x u) du. (30.5.1) We calculate both functions at n points θ k = kπ/n, where k = 0,..., n 1. Denote the resulting function values by f k = f(θ k ) and g k = g(θ k ), respectively. Note also the wrap around relations f k = f n k and h k = g n k. Then for k = 0,..., n 1, the convolution of two discrete series h = f g is given by { n 1 k l (k l 0) h k = (f g) k = f l g κ where κ = (30.5.) n + k l (k l < 0). l=0 In the same way, the convolution H = F G of two Fourier series F j and G j is given by n 1 H j = (F G) j = F l G κ where κ = l=0 { j l (j l 0) n + j l (j l < 0). (30.5.3) 6

7 30.6 Cross-correlation & autocorrelation of discrete series The same ideas in Sec also apply to cross correlation and autocorrelation. In the continuous case, the cross correlation of two functions f(x) and g(x) is given by (f g)(x) = f (u)g(x + u) du. (30.6.1) Employing the same definitions as in Sec. 30.5, the cross correlation of two discrete series h = f g is given by { n 1 h k = (f g) k = fl g k + l (k + l < n) κ where κ = (30.6.) k + l n (k + l n). l=0 The cross correlation H = F G of two Fourier series F j and G j is given by { n 1 H j = (F G) j = Fl G j + l (j + l < n) κ where κ = j + l n (j + l n). l=0 (30.6.3) For autocorrelation we simply set f = g or F = G. Parseval s theorem for discrete series states 1 n n 1 f k = k=0 n 1 F j. (30.6.4) j=0 Let us verify eq. (30.6.4) for the example function f ex. 1. The sum of the absolute squares of the Fourier coefficients is 3 F j = A 0 4 j=0 + A 1 + B 1 + A. (30.6.5). The sum of the absolute squares of the function values is 3 k=0 f k = 1 A 0 + A 1 + A + 1 A 0 + B 1 A + 1 A 0 A 1 + A + 1 A 0 B 1 A (30.6.6) = A 0 + A 1 + B A. 3. Hence Parseval s theorem eq. (30.6.4) is satisfied for the example function f ex : f k = k=0 3 F j = A 0 4 j=0 + A 1 + B 1 + A. (30.6.7) 7

8 30.7 Change of notation Since we shall eventually want to write programming code, it is confusing to use the notation f[k] for the function array and F[j] for the Fourier coefficients array. We change notation to X(θ) for the function. Hence we write X k = X(θ k ). Recall θ k = kπ/n, where k = 0,..., n 1. We continue to denote the Fourier coefficients by F j, where j = 0,..., n 1. 8

9 30.8 Discrete Fourier Transform (DFT) We now define the Discrete Fourier Transform (DFT). We employ n equally spaced data points around the circle θ k = kπ/n, where k = 0,..., n 1. The function values are X k = X(θ k ). Let ω be a primitive n th root of unity, i.e. ω n = 1 and choose ω = e iπ/n. The complex Fourier coefficients are denoted by (F 0,..., F n 1 ). Both (X 0,..., X n 1 ) and (F 0,..., F n 1 ) are complex, in general. The Discrete Fourier Transform (DFT) is given by the following sum: n 1 F j = X(θ k ) e ijθ k = k=0 n 1 ω jk X k (j, k = 0,..., n 1). (30.8.1) k=0 The inverse Discrete Fourier Transform computes (f 0,..., f n 1 ) from (F 0,..., F n 1 ): X k = 1 n 1 F j e ikθ j = 1 n 1 (ω ) jk F j (j, k = 0,..., n 1). (30.8.) n n j=0 j=0 The inverse transform sums the series only at the input points θ = kπ/n, hence it returns the original data values X k exactly. The inverse transform is obtained by the same procedure as the discrete Fourier transform, replacing ω by ω and dividing the sum by n. To avoid the asymmetry of 1/n between eqs. (30.8.1) and (30.8.), many authors prefer to divide by 1/ n in both formulas and write F j = 1 n 1 ω jk X k, X k = 1 n 1 (ω ) jk F j (j, k = 0,..., n 1). (30.8.3) n n k=0 The inverse transform in eq. (30.8.3) is given by simply replacing ω by ω. The standard convention in physics is to employ eqs. (30.8.1) and (30.8.). j=0 We shall employ eqs. (30.8.1) and (30.8.) in this course. Naïvely, the calculations in eqs. (30.8.1) and (30.8.) require O(n ) computations. The Fast Fourier Transform (FFT) can calculate the sums in eqs. (30.8.1) and (30.8.) using O(n log n) computations. The breakthrough was published by Cooley and Tukey in With hindsight, it is now known that others had published O(n log n) algorithms for discrete Fourier transforms before, going back to Gauss in 1805, but the significance of those earlier algorithms was not recognized until after Cooley and Tukey s publication. We shall study the FFT algorithm later in this lecture. 9

10 30.9 FFT: warmup exercise n = 4 Let us consider a Discrete Fourier Transform (DFT) with n = 4 points. Set ω = e iπ/n = e iπ/4. The initial data values are (X 0,..., X n 1 ) = (X 0, X 1, X, X 3 ). The transformed data values are (F 0,..., F n 1 ) = (F 0, F 1, F, F 3 ). We write out the equations in eq. (30.8.1) explicitly to obtain F 0 = X 0 + X 1 + X + X 3, F 1 = X 0 + ωx 1 + ω X + ω 3 X 3, F = X 0 + ω X 1 + ω 4 X + ω 6 X 3, F 3 = X 0 + ω 3 X 1 + ω 6 X + ω 9 X 3. Collect terms on the right hand side in odd/even pairs. Also use ω n = ω 4 = 1. F 0 = X 0 + X + X 1 + X 3, F 1 = X 0 + ω X + ω(x 1 + ω X 3 ), F = X 0 + X + ω (X 1 + X 3 ), F 3 = X 0 + ω X + ω 3 (X 1 + ω X 3 ). (30.9.1) (30.9.) Notice that the X k come in only four combinations. Use E and O for even and odd: E 0 = X 0 + X, O 0 = X 1 + X 3, E 1 = X 0 + ω X = X 0 X, O 1 = X 1 + ω X 3 = X 1 X 3. (30.9.3) This can clearly be formalized, so as to generalize to larger values of n. Define Ω via ( ) 1 0 Ω =. (30.9.4) 0 ω Then note that, using ω n/ = ω = 1, ( ) ( ) ( ) ( F0 E0 1 0 O0 = + F 1 E 1 0 ω O 1 ( ) ( ) ( ) ( F E0 1 0 O0 = 0 ω F 3 E 1 O 1 ) = ) = ( E0 E 1 ( E0 E 1 ) ( O0 + ωo 1 ) ), ). ωo 1 ( O0 (30.9.5) We define F + and F via F + = ( F0 F 1 ), F = ( F F 3 ). (30.9.6) Then we see from above that F + = E + ΩO, F = E ΩO. (30.9.7) 10

11 30.10 FFT: second exercise n = 8 Next let us see how this works for for n = 8 points. Set ω = e iπ/n = e iπ/8. The initial data values are (X 0,..., X n 1 ) = (X 0,..., X 7 ). The transformed data values are (F 0,..., F n 1 ) = (F 0,..., F 7 ). Write this out explicitly (in even/odd blocks) to obtain F 0 F 1 F F 3 F 4 = F 5 F 6 F 7 X 0 +X +X 4 +X 6 +(X 1 +X 3 +X 5 +X 7 ) X 0 +ω X +ω 4 X 4 +ω 6 X 6 +ω(x 1 +ω X 3 +ω 4 X 5 +ω 6 X 7 ) X 0 +ω 4 X +ω 8 X 4 +ω 1 X 6 +ω (X 1 +ω 4 X 3 +ω 8 X 5 +ω 1 X 7 ) X 0 +ω 6 X +ω 1 X 4 +ω 18 X 6 +ω 3 (X 1 +ω 6 X 3 +ω 1 X 5 +ω 18 X 7 ) X 0 +ω 8 X +ω 16 X 4 +ω 4 X 6 +ω 4 (X 1 +ω 8 X 3 +ω 16 X 5 +ω 4 X 7 ) X 0 +ω 10 X +ω 0 X 4 +ω 30 X 6 +ω 5 (X 1 +ω 10 X 3 +ω 0 X 5 +ω 30 X 7 ) X 0 +ω 1 X +ω 4 X 4 +ω 36 X 6 +ω 6 (X 1 +ω 1 X 3 +ω 4 X 5 +ω 36 X 7 ) X 0 +ω 14 X +ω 8 X 4 +ω 4 X 6 +ω 7 (X 1 +ω 14 X 3 +ω 8 X 5 +ω 4 X 7 ). ( ) Use ω n = ω 8 = 1 and ω n/ = ω 4 = 1 to simplify some exponents (but maintain a pattern): F 0 X 0 +X +X 4 +X 6 X 1 +X 3 +X 5 +X 7 X 0 +ω X +ω 4 X 4 +ω 6 X 6 X 0 +ω 4 X +ω 8 X 4 +ω 1 X 6 + ω(x 1 +ω X 3 +ω 4 X 5 +ω 6 X 7 ) ω (X 1 +ω 4 X 3 +ω 8 X 5 +ω 1 X 7 ) X 0 +ω 6 X +ω 1 X 4 +ω 18 X 6 ω 3 (X 1 +ω 6 X 3 +ω 1 X 5 +ω 18 X 7 ) F 1 F F 3 F 4 = F 5 F 6 F 7 X 0 +X +X 4 +X 6 X 0 +ω X +ω 4 X 4 +ω 6 X 6 X 0 +ω 4 X +ω 8 X 4 +ω 1 X 6 X 0 +ω 6 X +ω 1 X 4 +ω 18 X 6 X 1 +X 3 +X 5 +X 7 ω(x 1 +ω X 3 +ω 4 X 5 +ω 6 X 7 ) ω (X 1 +ω 4 X 3 +ω 8 X 5 +ω 1 X 7 ) ω 3 (X 1 +ω 6 X 3 +ω 1 X 5 +ω 18 X 7 ) (30.10.) As before, we observe there are only two independent sets of terms that we need to compute.. We define two blocks E and O as follows: E = O = X 0 +X +X 4 +X 6 X 0 +ω X +ω 4 X 4 +ω 6 X 6 X 0 +ω 4 X +ω 8 X 4 +ω 1 X 6 X 0 +ω 6 X +ω 1 X 4 +ω 18 X 6 X 1 +X 3 +X 5 +X 7 X 1 +ω X 3 +ω 4 X 5 +ω 6 X 7 X 1 +ω 4 X 3 +ω 8 X 5 +ω 1 X 7 X 1 +ω 6 X 3 +ω 1 X 5 +ω 18 X 7,. ( ) Notice that each block is the same as in eq. (30.9.1), with ω = e iπ/4, which is exactly the primitive root of unity in eq. (30.9.1). 11

12 Therefore we can calculate each block E and O usng the algorithm for n = 4. To complete the procedure, we define Ω as follows: 1 Ω = ω ω. ( ) ω 3 We define F + and F (with four components) via F + = F 0 F 1 F F 3, F = F 4 F 5 F 6 F 7. ( ) Then we see from above that once again F + = E + ΩO, F = E ΩO. ( ) 1

13 30.11 FFT: general case n = s The pattern established above for n = 4 and 8 clearly generalizes to n = s for any k 1. We define the primitive root of unity ω s = e iπ/n = e iπ/s. We group the X k into even/odd combinations with n/ components each: X 0 +X +X 4 + +X n X 0 +ωsx +ωsx ωs n X n E k = X 0 +ωsx 4 +ωsx ωs (n ) X n,. X 0 +ωs n X +ωs (n ) X 4 + +ω (n ) / s X n X 1 +X 3 +X 5 + +X n 1 X 1 +ωsx 3 +ωsx ωs n X n 1 O k = X 1 +ωsx 4 3 +ωsx ωs (n ) X n 1.. X 1 +ωs n X 3 +ωs (n ) X 5 + +ω (n ) / s X n 1 ( ) Each block is a Discrete Fourier Transform of n/ points sampled at equal intervals, with ω s = e iπ/(n/) = e iπ/k 1, which is exactly the root of unity required for n/ = k 1. Therefore we calculate each block E k and O k recursively using the algorithm with n/. To complete the procedure, we define Ω k as follows: 1 Ω k = ω s... ω (n/) 1 s. (30.11.) We define F k+ and F k (with n/ components each) via F k,0 F k,1 F k+ =., F k = F k,(n/) 1 Then as before we have F k,n/ F k,(n/)+1. F k,n 1. ( ) F k+ = E k + Ω k O k, F k = E k Ω k O k. ( ) 13

14 30.1 Bit reversal The FFT implementation in Sec is a bit simplistic. At every step of the recursion, we have to collect the points into even/odd sets. Let us see how this works for n = 8, hence we have eight data points (X 0,..., X 7 ). At each stage of the recursion, we want to group the points into sets as follows: ( ) ) X 0 X 0 (X 0 ) X 0 X X 4 (X X 1 X 4 4 ( ) ) X X (X X 3 X ) 6 X 6 (X X 4 6 ( ) ) X 5 X 1 X 1 (X 1 ) X 6 X 3 X 5 (X X 7 X 5 5 ( ) ) X 3 (X 3 X ) 7 (X 7 X 7 (30.1.1) It would therefore be helpful if we sorted the points in the following order: X 0 X 1 X X 3 X 4 X 5 X 6 X 7 sort The sorting procedure is given by bit reversal. X 0 X 4 X X 6 X 1. (30.1.) X 5 X 3 X 7 Write the indices (0, 1,..., 7) in binary and reverse the binary digits. The result is the desired sort. j binary bit reverse sorted

15 What are the sums we obtain? X 0 + X 4 + (X + X 6 ) F 0 X 0 X 4 + (X X 6 )ω F 1 X 0 + X 4 (X + X 6 ) F F 3 X 0 X 4 (X X 6 )ω F 4 F 5 X 1 + X 5 + (X 3 + X 7 ) F 6 X 1 X 5 + (X 3 X 7 )ω F 7 X 1 + X 5 (X 3 + X 7 ) X 1 X 5 (X 3 X 7 )ω ( ) ) X 0 + X 4 (X 0 ) X 0 X 4 (X 4 ( ) ) X + X 6 (X ) X X 6 (X 6 ( ) ) X 1 + X 5 (X 1 ) X 1 X 5 (X 5 ( ) ) X 3 + X 7 (X 3 ) X 3 X 7 (X 7 (30.1.3) Do we obtain the correct answer? Let us check. Note that ω = ω 3 and ω4 3 = 1. F 0 = X 0 + X 4 + (X + X 6 ) + [X 1 + X 5 + (X 3 + X 7 )] = X 0 + X 1 + X + X 3 + X 4 + X 5 + X 3 + X 7, (30.1.4a) F 1 = X 0 X 4 + ω (X X 6 ) + ω 3 [X 1 X 5 + ω (X 3 X 7 )] = X 0 + ω 3 X 1 + ω 3X + ω 3 3X 3 + ω 4 3X 4 + ω 5 3X 5 + ω 6 3X 6 + ω 7 3X 7, (30.1.4b) F = X 0 + X 4 (X + X 6 ) + ω 3[X 1 + X 5 (X 3 + X 7 )] = X 0 + ω 3X 1 + ω 4 3X + ω 6 3X 3 + ω 8 3X 4 + ω 10 3 X 5 + ω 1 3 X 6 + ω 14 3 X 7, (30.1.4c) F 3 = X 0 X 4 ω (X X 6 ) + ω 3 3[X 1 X 5 ω (X 3 X 7 )] = X 0 + ω 3 3X 1 + ω 6 3X + ω 9 3X 3 + ω 1 3 X 4 + ω 15 3 X 5 + ω 18 3 X 6 + ω 1 3 X 7, (30.1.4d) F 4 = X 0 + X 4 + (X + X 6 ) [X 1 + X 5 + (X 3 + X 7 )] = X 0 + ω 4 3X 1 + ω 8 3X + ω 1 3 X 3 + ω 16 3 X 4 + ω 0 3 X 5 + ω 4 3 X 6 + ω 8 3 X 7, (30.1.4e) F 5 = X 0 X 4 + ω (X X 6 ) ω 3 [X 1 X 5 + ω (X 3 X 7 )] = X 0 + ω 5 3X 1 + ω 10 3 X + ω 15 3 X 3 + ω 0 3 X 4 + ω 5 3 X 5 + ω 30 3 X 6 + ω 35 3 X 7, (30.1.4f) F 6 = X 0 + X 4 (X + X 6 ) ω 3[X 1 + X 5 (X 3 + X 7 )] = X 0 + ω 6 3X 1 + ω 1 3 X + ω 18 3 X 3 + ω 4 3 X 4 + ω 30 3 X 5 + ω 36 3 X 6 + ω 4 3 X 7, (30.1.4g) F 7 = X 0 X 4 ω (X X 6 ) ω 3 3[X 1 X 5 ω (X 3 X 7 )] = X 0 + ω 7 3X 1 + ω 14 3 X + ω 1 3 X 3 + ω 8 3 X 4 + ω 35 3 X 5 + ω 4 3 X 6 + ω 49 3 X 7. (30.1.4h) Hence if we sort (or rearrange) the input data (X 0,..., X n 1 ) using bit-reversal, the algorithm in Sec can be implemented straightforwardly. Computational complexity. Observe that the number of computations (multiplications) is about = 8 3 = 8 log 8 = O(n log n). 15

16 30.13 Is bit reversal necessary? Recursion? Extra memory storage? Neither bit reversal, recursion nor extra memory storage are necessary. There exist FFT implementations which do not require bit reversal. Recursion: the FFT algorithm can be formulated using a set of nested loops. Extra memory storage: study the calculations in Sec At every step of the recursion (three steps because n = 3 = 8), we have n computed values. For example in the first calculation we have (X 0 + X 4, X 0 X 4, X + X 6, X X 6, X 1 + X 5, X 1 X 5, X 3 + X 7, X 3 X 7 ). The values can be held in an array of length n, which is (F 0,..., F n 1 ) itself. All the results of computations can be stored in place in the array (F 0,..., F n 1 ). The even/odd arrays E and O can be held in (F 0,..., F n 1 ) itself. A few temporary variables may be required, but no extra memory of length n or n/, etc. 16

17 30.14 FFT review Main routine Let us review the the steps of the FFT algorithm. We consider only inputs where the number of points n is a power of. There are FFT algorithms for other values of n but we do not consider them. We begin with a bit reversal of the input data (X 0,..., X n 1 ). 1. Some authors perform the bit reversal in place and overwrite the input data.. We store the bit reversed data in a separate array and do not overwrite the input data. 3. Let us denote the bit reversed array by R = (R 0,..., R n 1 ). We compute the primitive root of unity ω = e iπ/n. For the inverse we compute ω = e iπ/n. We call a lower level function to compute the FFT recursively, with inputs n, ω and R. For the inverse FFT, after the recursion is over, we divide (F 0,..., F n 1 ) by n Recursion If n = 1, we set F 0 = R 0 and return. If n =, we set F 0 = R 0 + R 1 and F 1 = R 0 R 1 and return. Note in this context that F and R are arrays which have been passed to a recursive function call. They are not the array indices F[0], F[1], R[0], R[1] of the original arrays in the top level FFT routine. If n >, we proceed as follows. First define a temporary variable w = ω. Declare temporary arrays Even and Odd of length n/. The arrays Even and Odd do not require additional memory. They can be implemented as pointers to sections of the array F. Call the FFT routine recursively twice, once to compute Even and once for Odd. 1. The inputs to the recursive calls are n/, w and suitable sections of the array R.. Note that in a recursive function call, the parameters n and ω do not necessarily have the same values as in the top level routine. 3. The compiler will take care of the details. After both recursive calls have completed, compute F + and F from Even and Odd. Both F + and F can be stored in place in the array F. 17

18 30.15 FFT code Bit reversal There are multiple ways to code the bit reversal. int bit_reverse(int num_bits, int n) { int r = 0; for (int i=0; i < num_bits; ++i) { int rem = (n & 1); r = r* + rem; n /= ; return r; int bit_reverse(int num_bits, int n) { const int nmax = 1 << (num_bits-1); // ^(num_bits-1) int r = 0; for (int i = 0; i < num_bits; ++i) { int j = (1 << i); if (n & j) { int k = nmax >> i; r = k; return r; 18

19 FFT recursion static void FFT_recursion(int npts, const std::complex<double> & omega, const std::complex<double> * R, std::complex<double> * F) { if (npts == 1) { F[0] = R[0]; return; if (npts == ) { F[0] = R[0] + R[1]; F[1] = R[0] - R[1]; return; const std::complex<double> w = omega*omega; const int nhalf = npts/; std::complex<double> * Even = F; std::complex<double> * Odd = F + nhalf; FFT_recursion(nhalf, w, R, Even); // use array f, no extra storage required FFT_recursion(nhalf, w, R+nhalf, Odd); // use array f, no extra storage required std::complex<double> wtmp(1.0, 0.0); for (int i = 0; i < nhalf; ++i) { std::complex<double> Fplus = Even[i] + wtmp*odd[i]; // Even + Omega*Odd std::complex<double> Fminus = Even[i] - wtmp*odd[i]; // Even - Omega*Odd F[i] = Fplus; F[i+nhalf] = Fminus; wtmp *= omega; 19

20 FFT top level function static void FFT_top(bool inverse, int num_bits, int npts, const std::complex<double> * X, std::complex<double> * F) { // bit reversal std::complex<double> R[npts]; for (int i = 0; i < npts; ++i) { int i_rev = bit_reverse(num_bits, i); R[i_rev] = X[i]; // primitive root of unity const double pi = 4.0*atan(1.0,1.0); double ang =.0*pi/npts; if (inverse) ang = -ang; std::complex<double> omega(cos(ang), -sin(ang)); // note the minus sign // FFT recursion FFT_recursion(npts, omega, R, F); if (inverse) { double ndbl = double(npts); for (int i = 0; i < npts; ++i) { F[i] /= ndbl; 0

Computational Methods CMSC/AMSC/MAPL 460

Computational Methods CMSC/AMSC/MAPL 460 Fourier transform Balaji Vasan Srinivasan Dept of Computer Science Several slides from Prof Healy s course at UMD Last time: Fourier analysis F(t) = A 0 /2 + A