Announcements EE Digital Signal Processing otes posted HW due Friday SDR give away Today! Read Ch 9 $$$ give me your names Lecture based on slides by JM Kahn M Lustig, EECS UC Berkeley M Lustig, EECS UC Berkeley Last Time Discrete Fourier Transform Properties of the Linear convolution with Overlap and add Overlap and save Today Fast Fourier Transform vs DTFT (revisit) Back to moving average example: (e j! ) e j!n n e j! sin(!) sin(! ) w M Lustig, EECS UC Berkeley M Lustig, EECS UC Berkeley
and Sampling the DTFT Circular Convolution as Matrix Operation (e j! )e j! sin (!/) sin (!/) x[n] n reconstructed x[n] (e jω ) ω (e jω ) Circular convolution: h[n] x[n] H c x h[] h[ ] h[] h[] h[] h[] h[ ] h[ ] h[] x[] x[] x[] H c is a circulant matrix The columns of the matrix are Eigen vectors of circulant matrices Eigen vectors are coe cients How can you show? n ω Fall, EE Digital Signal Processing Circular Convolution as Matrix Operation Diagonalize: W H c Wn Right-multiply by W W H c Multiply both sides by x W H c x H[] H[] H[ ] H[] H[] H[ ] H[] H[] H[ ] W W x 8
Fast Fourier Transform Algorithms We are interested in e and inverse : where [k] x[n] n k cient computing methods for the x[n]w kn, k,, kn [k]w, n,, W e j( ) Recall that we can use the to compute the inverse : Hence, we can just focus on e { [k]} ( { [k]}) cient computation of the Straightforward computation of an -point (or inverse ) requires complex multiplications 9 Fast Fourier transform algorithms enable computation of an -point (or inverse ) with the order of just log complex multiplications This can represent a huge reduction in computational load, especially for large log log 8,8 89 8,,8,, 8,9,8,8,9 * Mp image size Most FFT algorithms exploit the following properties of W kn : Conjugate Symmetry Periodicity in n and k: Power: k( n) W W kn (W kn ) W kn W k(n+) W W / W (k+)n
Most FFT algorithms decompose the computation of a into successively smaller computations Decimation-in-time algorithms decompose x[n] into successively smaller subsequences Decimation-in-frequency algorithms decompose [k] into successively smaller subsequences We mostly discuss decimation-in-time algorithms here Assume length of x[n] is power of ( ) If smaller zero-pad to closest power We start with the [k] n x[n]w kn, k,, Separate the sum into even and odd terms: [k] x[n]w kn + x[n]w kn n even n odd These are two s, each with half of the samples Let n r (n even) and n r +(n odd): Hence: [k] (/) r (/) r x[r]w rk x[r]w rk (/) + r + W k x[r + ]W (r+)k (/) r x[r + ]W rk [k] (/) r (/) x[r]w/ rk + W k r G[k]+W k H[k], k,, where we have defined: x[r + ]W rk / ote that: W rk e j( )(rk) e j rk / W rk / Remember this trick, it will turn up often G[k] H[k] (/) r (/) r x[r]w rk / ) of even idx x[r + ]W rk / ) of odd idx
An 8 sample can then be diagrammed as Even Samples Odd Samples x[] x[] x[] x[] x[] x[] x[] / - Point / - Point G[] G[] G[] G[] H[] H[] H[] H[] W W W W W W W W [] [] [] [] [] [] [] [] Both G[k] and H[k] are periodic, with period / For example so G[k + /] (/) r (/) r (/) r G[k] x[r]w r(k+/) / x[r]w rk / W r(/) / x[r]w rk / G[k +(/)] G[k] H[k +(/)] H[k] 8 The periodicity of G[k] andh[k] allowsustofurthersimplify For the first / points we calculate G[k] andw k H[k], and then compute the sum [k] G[k]+W k H[k] 8{k :applek < } How does periodicity help for apple k <? for [k] G[k]+W k H[k] 8{k :apple k < } apple k < : W k+(/)? [k +(/)]? 9
[k +(/)] G[k] We previously calculated G[k] andw k H[k] W k H[k] ow we only have to compute their di erence to obtain the second half of the spectrum o additional multiplies are required The -point has been reduced two /-point s, plus / complex multiplications The 8 sample is then: Even Samples Odd Samples x[] x[] x[] x[] x[] x[] x[] / - Point / - Point W W W W G[k] H[k] W k [] [] [] [] [] [] [] [] ote that the inputs have been reordered so that the outputs come out in their proper sequence We can define a butterfly operation, eg, the computation of [] and [] from G[] and H[]: G[] H[] W This is an important operation in DSP [] G[] + W H[] [] G[] - W H[] Still O( ) operations What shall we do? Even Samples Odd Samples x[] x[] x[] x[] x[] x[] x[] / - Point / - Point W W W W G[k] k H[k] W [] [] [] [] [] [] [] []
We can use the same approach for each of the / point s For the 8case,the/ s look like At this point for the 8 sample, we can replace the / sample s with a single butterfly The coe cient is x[] x[] x[] / - Point / - Point W / W / *ote that the inputs have been reordered again G[] G[] G[] G[] W / W 8/ W e j The diagram of this stage is then x[] x[] + x[] - Combining all these stages, the diagram for the 8 sample is: x[] [] In general, there are log stages of decimation-in-time W / x[] W / x[] x[] W W x[] W / W x[] W / W x[] This the decimation-in-time FFT algorithm [] [] [] [] [] [] [] Each stage requires / complex multiplications, some of which are trivial The total number of complex multiplications is (/) log The order of the input to the decimation-in-time FFT algorithm must be permuted First stage: split into odd and even Zero low-order bit first ext stage repeats with next zero-lower bit first et e ect is reversing the bit order of indexes 8
Decimation-in-Frequency Fast Fourier Transform This is illustrated in the following table for 8 Decimal Binary Bit-Reversed Binary Bit-Reversed Decimal The is [k] n x[n]w nk If we only look at the even samples of [k], we can write k r, [r] n x[n]w n(r) We split this into two sums, one over the first / samples, and the second of the last / samples [r] (/) n x[n]w rn (/) + n x[n + /]W r(n+/) 9 Decimation-in-Frequency Fast Fourier Transform Decimation-in-Frequency Fast Fourier Transform But W r(n+/) W rnw W rn We can then write W rn / [r] (/) n (/) n (/) n x[n]w rn x[n]w rn (/) + n (/) + n (x[n]+x[n + /]) W rn / x[n + /]W r(n+/) x[n + /]W rn This is the /-length of first and second half of x[n] summed [r] {(x[n]+x[n + /])} [r + ] {(x[n] x[n + /]) W n } (By a similar argument that gives the odd samples) Continue the same approach is applied for the / s, and the / s until we reach simple butterflies
Decimation-in-Frequency Fast Fourier Transform The diagram for and 8-point decimation-in-frequency is as follows x[] x[] x[] x[] x[] x[] x[] W W W W W / W / W / W / This is just the decimation-in-time algorithm reversed! The inputs are in normal order, and the outputs are bit reversed [] [] [] [] [] [] [] [] on-power-of- FFT s A similar argument applies for any length, where the length is a composite number For example, if, a decimation-in-time FFT could compute three -point s followed by two -point s x[] x[] x[] x[] x[] -Point -Point -Point W W W -Point -Point [] [] [] [] [] [] on-power-of- FFT s Good component s are available for lengths up to or so Many of these exploit the structure for that specific length For example, a factor of W / e j (/) e j j Why? just swaps the real and imaginary components of a complex number, and doesn t actually require any multiplies Hence a of length doesn t require any complex multiplies Half of the multiplies of an 8-point also don t require multiplication Composite length FFT s can be very e cient for any length that factors into terms of this order For example 9 factors into ()(9)() each of which can be implemented e 9 s of length s of length 9, and 9 s of length ciently We would perform
Historically, the power-of-two FFTs were much faster (better written and implemented) For non-power-of-two length, it was faster to zero pad to power of two Recently this has changed The free FFTW package implements very e cient algorithms for almost any filter length Matlab has used FFTW since version 8 FFT as Matrix Operation FFT as Matrix Operation B @ [] [k] [ ] C A B @ W n ( ) W k kn k( ) ( ) W ( )n ( )( ) B C @ A x[] x[n] x[ ] C A B @ [] [k] [ ] C A B @ W n ( ) W k kn k( ) ( ) W ( )n ( )( ) B C @ A x[] x[n] x[ ] C A W is fully populated ) entries W is fully populated ) entries FFT is a decomposition of W into a more sparse form: apple I/ D F / I / D / apple W/ W / apple Even-Odd Perm Matrix I / is an identity matrix D / is a diagonal with entries, W,, W / 9
FFT as Matrix Operation Beyond log What if the signal x[n] has a k sparse frequency Example: F W W A Gilbert et al, ear-optimal sparse Fourier representations via sampling H Hassanieh et al, early Optimal Sparse Fourier Transform Others O(K Log ) instead of O( Log ) From: http://groupscsailmitedu/netmit/sfft/paperhtml M Lustig, EECS UC Berkeley