EE518 Digital Signal Processing University of Washington Autumn 2001 Dept of Electrical Engineering Lecture 20: Discrete Fourier Transform and FFT Dec 10, 2001 Prof: J Bilmes <bilmes@eewashingtonedu> TA: Mingzhou Song <msong@uwashingtonedu> 201 Discrete Fourier Transform It is basically the same thing as DFS Start with finite length x[n] of length and associate x[n] r x[n r] otation x[n] x[(n modula )] x[((n)) ] OK when the length of x[n] is less than DFT is defined as X[k] x[n]w kn (201) x[n] 1 X[k]W kn (202) k0 Computation of point DFT or inverse DFT using this definition has a time complexity of O( 2 ) - basically saying we care not about the rest of x[n], since it is zero Pretend that it is periodic for analysis purpose since for the DFT it makes no difference - Defined only for 0 n,k 1 The rest is zero This means the inherent periodicity is not represented - otation x[n] DFT X[k] - Lots of properties (similar to DFS) - circular convolution is important Given x 1 [n] and x 2 [n], form x 1 [n] and x 2 [n] Periodic convolution is x 3 [n] x 1 [m] x 2 [n m] (0 n 1) x 1 [((m)) ]x 2 [((n m)) ] x 1 [n] x 2 [n] x 2 [n] x 1 [n] DFT 1 {X 1 [k]x 2 [k]} 20-1
20-2 Discrete Fourier transform is the same thing just working with finite signals otes: defined only for 0 n 1 must ignore rest of x[n] since it is zero similar properties to DFS Q: x[n] M points, DFT {x[n]} X[k] X[k] x[n]w kn (203) x[n] 1 X[k]W kn (204) k0 Zero pad > M, DFT 1 {X[k]}?, ie, in matlab what do we get? Ex: take x[n] and sample at > 2Ω FT?? (manuscript p1) take IFT, get back?? (manuscript p1) signal Ex: circular convolution otation x[n] x[(n modula )] x[((n)) ] n ((n)) n when both are positive?? Circular convolution is equivalent to periodic convolution of corresponding periodic signals So we can implement convolution by 1) taking the DFT of two signals 2) multiply them together 3) taking inverse DFT x 3 [n] x 1 [m] x 2 [n m] (0 n 1) x 1 [((m)) ]x 2 [((n m)) ] x 1 [n] x 2 [n] x 2 [n] x 1 [n] DFT 1 {X 1 [k]x 2 [k]} To have linear convolution, need to make sure that DFT length is long enough, ie, if x 1 [n] has length of L and x 2 [n] has length of P, the DFT length need to satisfy because x 1 [n] x 2 [n] has a length of L + P 1 L + P 1 For very long signals, ie, streaming media, with a relatively short filter, eg, stream Filter h output
20-3 can break stream into chunks of short length then do the combination There are two ways to combine 1 Overlap-add (very useful) use fully computed results accumulate partially computed results 2 Overlap-save only use fully computed results throw away partially computed results input windows need to overlap ote on final since we don t have time, but for future, these are very useful if you implement this system 202 Fast Fourier Transform Computation of X[k] x[n]w kn for k 0,, 1 need O( 2 ) operations Can we do better than this? Yes By divide and conquer approach, we can turn this into O( log 2 ()) operations Motivation from polynomial evaluation (polynomials of order ) Consider two -th order polynomials: A(x) j0 B(x) j0 a j x j a 0 + a 1 x + a 2 x 2 + + a x b j x j Evaluation of any one of them takes O() operations using Horner s rule A(x 0 ) a 0 + x 0 (a 1 + x 0 (a 2 + x 0 ( + x 0 (a 2 + x 0 a ) ))) Adding is also easily done in O() operations C(x) A(x) + B(x) (a 0 + b 0 ) + (a 1 + b 1 )x + + (a + b )x c 0 + c 1 x + + c x Multiplication is hard A(x)B(x) a 0 B(x) + a 1 xb(x) + a 2 x 2 B(x) + + a x B(x) There are a total of O( 2 ) operations This is like a convolution (time point corresponds to the degree of x) For multiplication, an alternate representation of the polynomial is better, ie, point-value representation {(x 0,y 0 ),(x 1,y 1 ),,(x 2,y 2 )} when y k A(x k ), k ote we need 2 1 points since degree after multiplication will be 2 1 For B(x), {(x 0,y 0),(x 1,y 1),,(x 2,y 2)}
20-4 So for x i, C(x i ) A(x i )B(x i ) y i y i takes one operation For C(x), {(x 0,y 0 y 0),(x 1,y 1 y 1),,(x 2,y 2 y 2)} So we just need an easy way to get from x 0,x 1,,x 2 to y 0,y 1,,y 2 The naive way is to evaluate every point A(x 0 ),A(x 1 ),,A(x 2 ) For each A(x), operations is needed So there has to be O( 2 ) operations We have gained nothing so far ote that any distinct values will work (not unique), but there is a unique polynomial for given point-value pairs Why? Matrix multiplication 1 x 0 x0 2 x0 1 x 1 x1 2 x 1 1 x x 2 x a 0 a 1 a The matrix in above equation is called Vandermonde matrix and we use V (x 0,x 1,,x ) to denote the matrix ote we require det(v ) 0 so V has inverse From above equation, point-values can determine the polynomial There are many possible point-values for a polynomial So we can choose any values of x 0,x 1,,x to form our point-value representation y 0 y 1 y How about we choose the complex roots of unity, ie, x 1 {x 0,x 1,,x }? We already have a name for this W e j2π/ (W k ) e j2π 1, k starting from A(x) in coefficient form So to do multiplication A(x) a n x n y k A(x k ) A(W) k a n W kn DFT {a 0,a 1,,a } a 0,a 1,,a b 0,b 1,,b Ordinary multiplication, or convolution O( 2 ) c 0,c 1,,c 2 2 O( log) Evaluation at points using DFT or FFT Inverse DFT O( log ) A(W2 0 ),B(W 2 0 ) A(W2 1 ),B(W 2 1 ) A(W 2 2 ),B(W 2 2 ) Pointwise multiplication O() C(W2 0 ) C(W2 1 ) C(W2 2 ) So if we can find a fast DFT then this is good FFT is O( log) At wwwfftworg there lots of FFT info, tutorials, various algorithms, Java demo, matlab code, etc There are many FFT algorithms A good text is Advanced Topics in Signal Processing (Lim & Oppenheim) ext, we assume that 2 l for integer l
20-5 Decimation in time FFT algorithm (Divide and Conquer) X[k] x[n]w nk x[n]w nk + x[n]w nk even n odd n G[k] +W k H[k] x[2r]w 2rk + x[2r](w 2 ) rk +W k x[2r]w/2 rk +W k x[2r + 1]W (2r+1)k x[2r + 1](W) 2 rk x[2r + 1]W rk /2 ote: this is the sum of two /2-point DFT s and some multiplications Original number of multiplications is 2 ow 2(/2) 2 + ( 2 /2) + < 2 (for > 2) See flow diagram on O&S pg 636 ote this can be represented for G[k] and H[k] Ex: G[k] (/4) 1 (/4) 1 g[r]w rk /2 g[2l]w 2lk g[r] x[2r] /2 + (/4) 1 (/4) 1 g[2l]w/4 lk +W /2 k g[2l + 1]W (2l+1)k /2 The same can be done for H[k] Show flow graphs on O & S pg 638, fig 95 Computation: O( log) vs O( 2 ) is a huge saving Ratio of speedup 2 2(/2) 2 + g[2l + 1]W lk /4 2[2(/4) 2 + /2] + 2 /4 + + 2[2[2(/8) 2 + /4] + /2] + 2 /8 + + + 1 + + + O( log) old time new time log
20-6 So the bigger gets, the better the speedup ote basic operation Butterfly is W r But W r+/2 W r+/2 WW r /2 We r j2π//2 We r jπ W r ow we get the optimal Butterfly which reduces the computation by a factor of 2 Ex: Recursive FFT algorithm: W r 1 Algorithm 1 RFFT(x) 1: length(x) 2: if 1 then 3: return x 4: end if 5: W e j2π/ 6: W 1 7: x 0 {x[0],x[2],,x[ 2]} 8: x 1 {x[1],x[3],,x[ 1]} 9: y 0 RFFT{x 0 } 10: y 1 RFFT{x 1 } 11: for k 0 to /2 1 do 12: y[k] y 0 [k] +Wy 1 [k] 13: y[k + /2] y 0 [k] Wy 1 [k] 14: W W W 15: end for 16: return y otes: (O&S) Fig 910 on page 640: input must be in bit reversed order storing them in arrays (O&S) Fig 914 on: output in bit reversed order Bit reverse order makes in place computation possible Bit reverse order:
20-7 bit bit reversed 0 000 000 0 1 001 100 4 2 010 010 2 3 011 110 6 4 100 001 1 5 101 101 5 6 110 011 3 7 111 111 7 Go over O&S Fig 915 & 916 on page 645 Decimation in Frequency Start with X[k] Compute X[2r] and X[2r + 1] See O&S Fig 917 x[n]w kn k 0,, 1 203 Final Exam Follow syllabus Open book / open note On topics that are covered in class be prepared to think