Fall 2011, EE123 Digital Signal Processing

Transcription

1 Lecture 6 Miki Lustig, UCB September 11, 2012 Miki Lustig, UCB

2 DFT and Sampling the DTFT X (e jω ) = e j4ω sin2 (5ω/2) sin 2 (ω/2) 5 x[n] 25 X(e jω ) n ω 5 reconstructed x[n] 25 X(e jω ) n Miki Lustig UCB Based on Course otes by JM Kahn ω

3 Circular Convolution as Matrix Operation Circular convolution: h[n] x[n] = = H c x h[0] h[ 1] h[1] h[1] h[0] h[2] h[ 1] h[ 2] h[0] x[0] x[1] x[] H c is a circulant matrix The columns of the DFT matrix are Eigen vectors of circulant matrices Eigen vectors are DFT coefficients How can you show? Miki Lustig UCB Based on Course otes by JM Kahn

4 Circular Convolution as Matrix Operation Diagonalize: W H c Wn 1 = Right-multiply by W W H c = Multiply both sides by x W H c x = H[0] H[1] 0 0 H[ 1] H[0] H[1] 0 0 H[ 1] H[0] H[1] 0 0 H[ 1] W W x Miki Lustig UCB Based on Course otes by JM Kahn

5 Miki Lustig UCB Based on Course otes by JM Kahn

6 Fast Fourier Transform Algorithms We are interested in efficient computing methods for the DFT and inverse DFT: X [k] = x[n] = 1 n=0 1 k=0 x[n]w kn, k = 0,, 1 X [k]w kn, n = 0,, 1 where W = e j( 2π ) Miki Lustig UCB Based on Course otes by JM Kahn

7 Recall that we can use the DFT to compute the inverse DFT: DFT 1 {X [k]} = 1 (DFT {X [k]}) Hence, we can just focus on efficient computation of the DFT Straightforward computation of an -point DFT (or inverse DFT) requires 2 complex multiplications Miki Lustig UCB Based on Course otes by JM Kahn

8 Fast Fourier transform algorithms enable computation of an -point DFT (or inverse DFT) with the order of just log 2 complex multiplications This can represent a huge reduction in computational load, especially for large 2 log 2 2 log , ,024 1,048,576 10, ,192 67,108, , * 6Mp image size Miki Lustig UCB Based on Course otes by JM Kahn

9 Most FFT algorithms exploit the following properties of W kn : Conjugate Symmetry W k( n) Periodicity in n and k: W kn = W kn = W k(n+) = (W kn ) = W (k+)n Power: W 2 = W /2 Miki Lustig UCB Based on Course otes by JM Kahn

10 Most FFT algorithms decompose the computation of a DFT into successively smaller DFT computations Decimation-in-time algorithms decompose x[n] into successively smaller subsequences Decimation-in-frequency algorithms decompose X [k] into successively smaller subsequences We mostly discuss decimation-in-time algorithms here Assume length of x[n] is power of 2 ( = 2 ν ) If smaller zero-pad to closest power Miki Lustig UCB Based on Course otes by JM Kahn

11 Decimation-in-Time Fast Fourier Transform We start with the DFT X [k] = 1 n=0 x[n]w kn, k = 0,, 1 Separate the sum into even and odd terms: X [k] = x[n]w kn + x[n]w kn n even n odd These are two DFT s, each with half of the samples Miki Lustig UCB Based on Course otes by JM Kahn

12 Decimation-in-Time Fast Fourier Transform Let n = 2r (n even) and n = 2r + 1 (n odd): X [k] = = (/2) 1 r=0 (/2) 1 r=0 x[2r]w 2rk x[2r]w 2rk (/2) 1 + r=0 + W k x[2r + 1]W (2r+1)k (/2) 1 r=0 x[2r + 1]W 2rk ote that: W 2rk ( ) = e j( 2π )(2rk) = e j 2π rk /2 = W rk /2 Remember this trick, it will turn up often Miki Lustig UCB Based on Course otes by JM Kahn

13 Decimation-in-Time Fast Fourier Transform Hence: X [k] = (/2) 1 r=0 (/2) 1 x[2r]w/2 rk + W k r=0 = G[k] + W k H[k], k = 0,, 1 where we have defined: x[2r + 1]W rk /2 G[k] H[k] = = (/2) 1 r=0 (/2) 1 r=0 x[2r]w rk /2 DFT of even idx x[2r + 1]W rk /2 DFT of odd idx Miki Lustig UCB Based on Course otes by JM Kahn

14 Decimation-in-Time Fast Fourier Transform An 8 sample DFT can then be diagrammed as Even Samples Odd Samples x[0] x[2] x[4] x[6] x[1] x[3] x[5] x[7] /2 - Point DFT /2 - Point DFT G[0] G[1] G[2] G[3] H[0] H[1] H[2] H[3] 0 W 1 W 2 W 3 W 4 W 5 W 6 W W 7 X[0] X[1] X[2] X[3] X[4] X[5] X[6] X[7] Miki Lustig UCB Based on Course otes by JM Kahn

15 Decimation-in-Time Fast Fourier Transform Both G[k] and H[k] are periodic, with period /2 For example so G[k + /2] = = = (/2) 1 r=0 (/2) 1 r=0 (/2) 1 r=0 = G[k] x[2r]w r(k+/2) /2 x[2r]w rk /2 W r(/2) /2 x[2r]w rk /2 G[k + (/2)] = G[k] H[k + (/2)] = H[k] Miki Lustig UCB Based on Course otes by JM Kahn

16 Decimation-in-Time Fast Fourier Transform The periodicity of G[k] and H[k] allows us to further simplify For the first /2 points we calculate G[k] and W k H[k], and then compute the sum X [k] = G[k] + W k H[k] {k : 0 k < 2 } How does periodicity help for 2 k <? Miki Lustig UCB Based on Course otes by JM Kahn

17 Decimation-in-Time Fast Fourier Transform X [k] = G[k] + W k H[k] {k : 0 k < 2 } for 2 k < : W k+(/2) =? X [k + (/2)] =? Miki Lustig UCB Based on Course otes by JM Kahn

18 Decimation-in-Time Fast Fourier Transform X [k + (/2)] = G[k] W k H[k] We previously calculated G[k] and W k H[k] ow we only have to compute their difference to obtain the second half of the spectrum o additional multiplies are required Miki Lustig UCB Based on Course otes by JM Kahn

19 Decimation-in-Time Fast Fourier Transform The -point DFT has been reduced two /2-point DFTs, plus /2 complex multiplications The 8 sample DFT is then: x[0] G[k] X[0] Even Samples Odd Samples x[2] x[4] x[6] x[1] x[3] x[5] x[7] /2 - Point DFT /2 - Point DFT 0 W 1 W 2 W 3 W H[k] W k X[1] X[2] X[3] X[4] X[5] X[6] X[7] Miki Lustig UCB Based on Course otes by JM Kahn

20 Decimation-in-Time Fast Fourier Transform ote that the inputs have been reordered so that the outputs come out in their proper sequence We can define a butterfly operation, eg, the computation of X [0] and X [4] from G[0] and H[0]: G[0] X[0] =G[0] + W 0 H[0] H[0] W 0 X[4] =G[0] - W 0 H[0] This is an important operation in DSP Miki Lustig UCB Based on Course otes by JM Kahn

21 Decimation-in-Time Fast Fourier Transform Still O( 2 ) operations What shall we do? x[0] G[k] X[0] Even Samples Odd Samples x[2] x[4] x[6] x[1] x[3] x[5] x[7] /2 - Point DFT /2 - Point DFT 0 W 1 W 2 W 3 W k H[k] W X[1] X[2] X[3] X[4] X[5] X[6] X[7] Miki Lustig UCB Based on Course otes by JM Kahn

22 Decimation-in-Time Fast Fourier Transform We can use the same approach for each of the /2 point DFT s For the = 8 case, the /2 DFTs look like x[0] x[4] x[2] x[6] /4 - Point DFT /4 - Point DFT W /2 0 W /2 1 G[0] G[1] G[2] G[3] *ote that the inputs have been reordered again Miki Lustig UCB Based on Course otes by JM Kahn

23 Decimation-in-Time Fast Fourier Transform At this point for the 8 sample DFT, we can replace the /4 = 2 sample DFT s with a single butterfly The coefficient is W /4 = W 8/4 = W 2 = e jπ = 1 The diagram of this stage is then x[0] x[0] + x[4] x[4] 1 x[0] - x[4] Miki Lustig UCB Based on Course otes by JM Kahn

24 Decimation-in-Time Fast Fourier Transform Combining all these stages, the diagram for the 8 sample DFT is: x[0] X[0] x[4] x[2] x[6] x[1] x[5] x[3] x[7] 0 W /2 1 W /2 W 0 /2 W 1 /2 W 0 W 1 W 2 W 3 X[1] X[2] X[3] X[4] X[5] X[6] X[7] This the decimation-in-time FFT algorithm Miki Lustig UCB Based on Course otes by JM Kahn

25 Decimation-in-Time Fast Fourier Transform In general, there are log 2 stages of decimation-in-time Each stage requires /2 complex multiplications, some of which are trivial The total number of complex multiplications is (/2) log 2 The order of the input to the decimation-in-time FFT algorithm must be permuted First stage: split into odd and even Zero low-order bit first ext stage repeats with next zero-lower bit first et effect is reversing the bit order of indexes Miki Lustig UCB Based on Course otes by JM Kahn

26 Decimation-in-Time Fast Fourier Transform This is illustrated in the following table for = 8 Decimal Binary Bit-Reversed Binary Bit-Reversed Decimal Miki Lustig UCB Based on Course otes by JM Kahn

27 Decimation-in-Frequency Fast Fourier Transform The DFT is X [k] = 1 n=0 x[n]w nk If we only look at the even samples of X [k], we can write k = 2r, X [2r] = 1 n=0 x[n]w n(2r) We split this into two sums, one over the first /2 samples, and the second of the last /2 samples X [2r] = (/2) 1 n=0 x[n]w 2rn (/2) 1 + n=0 x[n + /2]W 2r(n+/2) Miki Lustig UCB Based on Course otes by JM Kahn

28 Decimation-in-Frequency Fast Fourier Transform But W 2r(n+/2) = W 2rnW = W 2rn We can then write = W rn /2 X [2r] = = = (/2) 1 n=0 (/2) 1 n=0 (/2) 1 n=0 x[n]w 2rn x[n]w 2rn (/2) 1 + n=0 (/2) 1 + n=0 (x[n] + x[n + /2]) W rn /2 x[n + /2]W 2r(n+/2) x[n + /2]W 2rn This is the /2-length DFT of first and second half of x[n] summed Miki Lustig UCB Based on Course otes by JM Kahn

29 Decimation-in-Frequency Fast Fourier Transform X [2r] = DFT {(x[n] + x[n + /2])} 2 X [2r + 1] = DFT {(x[n] x[n + /2]) W n } 2 (By a similar argument that gives the odd samples) Continue the same approach is applied for the /2 DFTs, and the /4 DFT s until we reach simple butterflies Miki Lustig UCB Based on Course otes by JM Kahn

30 Decimation-in-Frequency Fast Fourier Transform The diagram for and 8-point decimation-in-frequency DFT is as follows x[0] X[0] x[1] x[2] x[3] x[4] x[5] x[6] x[7] W 0 1 W 2 W 3 W W 0 /2 W 1 /2 W 0 /2 W 1 /2 X[4] X[2] X[6] X[1] X[5] X[3] X[7] This is just the decimation-in-time algorithm reversed! The inputs are in normal order, and the outputs are bit reversed Miki Lustig UCB Based on Course otes by JM Kahn

31 on-power-of-2 FFT s A similar argument applies for any length DFT, where the length is a composite number For example, if = 6, a decimation-in-time FFT could compute three 2-point DFT s followed by two 3-point DFT s x[0] x[3] 2-Point DFT W Point DFT X[0] X[2] x[1] x[4] 2-Point DFT W 6 1 X[4] X[1] x[2] x[5] 2-Point DFT W Point DFT X[3] X[5] Miki Lustig UCB Based on Course otes by JM Kahn

32 on-power-of-2 FFT s Good component DFT s are available for lengths up to 20 or so Many of these exploit the structure for that specific length For example, a factor of W /4 = e j 2π (/4) = e j π 2 = j Why? just swaps the real and imaginary components of a complex number, and doesn t actually require any multiplies Hence a DFT of length 4 doesn t require any complex multiplies Half of the multiplies of an 8-point DFT also don t require multiplication Composite length FFT s can be very efficient for any length that factors into terms of this order Miki Lustig UCB Based on Course otes by JM Kahn

33 For example = 693 factors into = (7)(9)(11) each of which can be implemented efficiently We would perform 9 11 DFT s of length DFT s of length 9, and 7 9 DFT s of length 11 Miki Lustig UCB Based on Course otes by JM Kahn

34 Historically, the power-of-two FFTs were much faster (better written and implemented) For non-power-of-two length, it was faster to zero pad to power of two Recently this has changed The free FFTW package implements very efficient algorithms for almost any filter length Matlab has used FFTW since version 6 Miki Lustig UCB Based on Course otes by JM Kahn

35 001 run time [ms] Miki Lustig UCB Based on Course otes by JM Kahn

36 FFT as Matrix Operation X [0] X [k] X [ 1] = W 00 W 0n W 0( 1) W k0 W kn W k( 1) W ( 1)0 W ( 1)n W ( 1)( 1) x[0] x[n] x[ 1] W is fully populated 2 entries Miki Lustig UCB Based on Course otes by JM Kahn

37 FFT as Matrix Operation X [0] X [k] X [ 1] = W 00 W 0n W 0( 1) W k0 W kn W k( 1) W ( 1)0 W ( 1)n W ( 1)( 1) x[0] x[n] x[ 1] W is fully populated 2 entries FFT is a decomposition of W into a more sparse form: [ I/2 D F = /2 I /2 D /2 ] [ W/2 0 0 W /2 ] [ Even-Odd Perm Matrix ] I /2 is an identity matrix D /2 is a diagonal with entries 1, W,, W /2 1 Miki Lustig UCB Based on Course otes by JM Kahn

38 FFT as Matrix Operation Example: = 4 F 4 = W W Miki Lustig UCB Based on Course otes by JM Kahn

39 Miki Lustig UCB Based on Course otes by JM Kahn