Lecture 5 Miki Lustig, UCB September 14, 211 Miki Lustig, UCB
Motivatios for Discrete Fourier Trasform Sampled represetatio i time ad frequecy umerical Fourier aalysis requires a Fourier represetatio that is sampled i time ad frequecy Samplig i oe domai correspods to periodicity i the other domai Hece, we wat a represetatio that is discrete ad period i both time ad frequecy This is the discrete-time Fourier series or, equivaletly, the discrete Fourier trasform However, we are ofte dealig with sigals that are ot periodic, but still usig the DFT This requires special cosideratios
Motivatios for Discrete Fourier Trasform Efficiet Implemetios Very efficiet implemetatios of the discrete Fourier trasform exist Direct evaluatio of DFT: O( 2 ) Fast Fourier Trasform (FFT) algorithms: O( log 2 ) FFT algorithms are most straightforward for = 2 m MATLAB commads: X=fft(x) x=ifft(x) Covolutio ca be implemeted efficietly usig the FFT Direct covolutio: O( 2 ) FFT-based covolutio: O( log 2 )
Discrete Fourier Series Defiitio Cosider -periodic sigal: x[ + ] = x[] ad its frequecy-domai represetatio, which is also -periodic: X [k + ] = X [k] k The ~ will idicate a periodic sigal or spectrum
Discrete Fourier Series Defie W = e j2π/ x[] ad X [k] are related by the discrete Fourier series: x[] = 1 X [k] = 1 = 1 k= X [k]w k x[]w k
Discrete Fourier Trasform By covetio, work with oe period of x[] ad X [k]: { x[] 1 x[] X [k] = = otherwise { X [k] k 1 otherwise From these, if desired, we ca recover the periodic represetatios: x[] = x[(()) ] = x[ r] X [k] = X [((k)) ] = where (()) = mod r= r= X [k r]
Discrete Fourier Trasform DFT The discrete Fourier trasform relates x[] ad X [k]: x[] = 1 X [k] = 1 = 1 k= X [k]w k x[]w k Iverse DFT, sythesis DFT, aalysis Although ot stated it explicitly, it is uderstood that x[] = outside 1 X [k] = outside k 1
Alterative trasform ot i the book: Orthoormal DFT x[] = X [k] = 1 1 X [k]w k k= 1 1 x[]w k = Iverse DFT, sythesis DFT, aalysis Why use this or the other?
DFS vs DFT This figure compares the periodic sigal ad its DFS, x[] DFS X [k], to the correspodig oe-period sigal ad its DFT, x[] DFT X [k] x~!" # 1 X ~!" k #1 k x!" X!" k # 1 #1 k
DFT Cotiued Example: Take = 5 X [k] = { 4 = W k 4 k =, 1, 2, 3, 4 otherwise = 5δ[k] What if we take = 1? 5-poit DFT
DFT Cotiued Q: What if we take = 1? A: X [k] = X [k] where x[] is a period-1 sequece Ca show: X [k] = { 9 = W k 9 k =, 1, 2, 3, 4 otherwise 4π j = e 1 k si( π 2 k) si( π 1 k) 1-poit DFT
DFT vs DTFT The DFT ad the DTFT The -poit DFT of x[] is X [k] = 1 = x[]w k The DTFT of x[] is 1 = = x[]e j(2π/)k k 1 X (e jω ) = 1 = x[]e jω < ω < Comparig these two, we see that the DFT X [k] correspods to the DTFT X (e jω ) sampled at equally spaced frequecies betwee ad 2π: X [k] = X (e jω ) ω=k 2π k 1
DFT vs DTFT Back to example: X (e jω ) = 4 = e jω = e j2ω si( 5 2 ω) si( ω 2 )
DFT ad IDFT ote that the DFT ad the iverse DFT are computed i very similar fashio If we write x[] as the iverse DFT of X [k], multiply by ad take the complex cojugate: x [] = = ( 1 k= However, we also kow that 1 1 k= X [k]w k = DFT {X [k]} X [k]w k x [] = ( DFT 1 {X [k]} ) )
DFT ad IDFT Combiig these two expressios DFT {X [k]} = ( DFT 1 {X [k]} ) or DFT 1 {X [k]} = 1 (DFT {X [k]}) We ca evaluate the iverse DFT by Takig the complex cojugate, Takig the DFT, Multiplyig by 1, ad Takig the complex cojugate
DFT as Matrix operator ote that the defiitio of the DFT ad its iverse are equivalet to matrix equatios X [] X [k] X [ 1] = W W W ( 1) W k W k W k( 1) W ( 1) W ( 1) W ( 1)( 1) x[] x[] x[ 1] x[] x[] x[ 1] = 1 W W k W ( 1) W W k W ( 1) W ( 1) W ( 1)k W ( 1)( 1) X [] X [k] X [ 1] This shows that straightforward computatio of the -poit DFT or iverse DFT requires 2 complex multiplies
DFT as Matrix operator We ca write this much more compactly as a matrix equatio, X = W x x = 1 W X W is the DFT coefficiet matrix, ad x ad X are colum vectors cotaiig x[] ad X [k], ad is the cojugate traspose ote that sice the colums ad rows of W are orthogoal, ad W W = W W = I where I is the idetity matrix The as we would expect x = 1 W X = 1 W W x = 1 (I) x = x
Properties of Discrete Fourier Trasform These are iherited from the discrete-time Fourier series (EE12) ad eed ot be proved (1) Liearity α 1 x 1 [] + α 2 x 2 [] α 1 X 1 [k] + α 2 X 2 [k] (2) Circular Time Shift x[(( m)) ] X [k]e j(2π/)km = X [k]w km
Properties of Discrete Fourier Trasform Cot This figure compares a shift of the periodic sequece, x[ m], to a circular shift of the oe-period sequece, x[(( m)) ] x~!" ~ x! # m" # 1 m # 1 x!" x! $ $ # m% % " # 1 m # 1
Properties of Discrete Fourier Trasform Cot (3) Circular Frequecy Shift x[]e j(2π/)l = x[]w l X [((k l)) ] (4) Complex Cojugatio x [] X [(( k)) ] (5) Time Reversal ad Complex Cojugatio x [(( )) ] X [k] (6) Cojugate Symmetry for Real Sigals x[] = x [] X [k] = X [(( k)) ]
Properties of Discrete Fourier Trasform Cot (7) Parseval s Idetity 1 = x[] 2 = 1 1 k= X [k] 2 Proof This is particularly easy usig the matrix otatio ( ) 1 ( ) 1 x x = W X W X = 1 2 X W W X = 1 }{{} X X I
Circular Covolutio Sum Let x 1 [] ad x 2 [] be of legth The circular covolutio betwee x 1 [] ad x 2 [] is defied as: x 1 [] x 2 [] = 1 m= x 1 [m]x 2 [(( m)) ] ote that circular covolutio is commutative: x 2 [] x 1 [] = x 1 [] x 2 []
Circular Covolutio Sum The circular covolutio betwee x 1 [] ad x 2 [] is same as oe period of the periodic covolutio betwee the correspodig periodic sequeces x 1 [] ad x 2 []: { 1 m= x 1 [] x 2 [] = x 1[m] x 2 [ m] 1 otherwise This is illustrated i below: ~! " % ~ x! " x1 2 $ 1 # $ 1 m% 1 2 ~ x! m" ~ x! $ m" $ 1! " x! " x % 1 2 $ 1 x! " x! " 1 O 2 $ 1
Properties of Discrete Fourier Trasform Cot (8) Circular Covolutio Let x 1 [] ad x 2 [] be of legth x 1 [] x 2 [] X 1 [k] X 2 [k] This property is very useful for DFT-based covolutio (9) Multiplicatio Let x 1 [] ad x 2 [] be of legth x 1 [] x 2 [] 1 X 1[k] X 2 [k]
Liear Covolutio usig the DFT We start with two operiodic sequeces: x[] L 1 h[] P 1 We ca thik of x[] as a sigal, ad h[] as a filter ipulse respose We wat to compute the liear covolutio: y[] = x[] h[] = L 1 m= x[m] h[ m] = P 1 m= x[ m]h[m] y[] = x[] h[] is ozero oly for L + P 2, ad is of legth L + P 1 = M
Liear Covolutio usig the DFT We will look at two approaches for computig y[]: (1) Direct Covolutio Evaluate the covolutio sum directly This requires L P multiplicatios (2) Usig Circular Covolutio
Liear Covolutio usig the DFT (2) Usig Circular Covolutio Zero-pad x[] by P 1 zeros: { x[] L 1 x zp [] = L L + P 2 Zero-pad h[] by L 1 zeros: { h[] P 1 h zp [] = P L + P 2 Both zero-padded sequeces x zp [] ad h zp [] are of legth M = L + P 1
Liear Covolutio usig the DFT Both zero-padded sequeces x zp [] ad h zp [] are of legth M = L + P 1 We ca compute the liear covolutio x[] h[] = y[] by computig circular covolutio x zp [] M h zp []: Liear covolutio via circular { x zp [] M h zp [] M 1 y[] = x[] y[] = otherwise
Liear Covolutio usig the DFT Example L = P = 4 M = L + P 1 = 7 1 x 1 2 3 1 h 1 2 3 1 x zp 1 2 3 4 5 6 1 h zp 1 2 3 4 5 6 For 6, x zp [] 7 h zp [] = x[] * h[] 4 3 3 2 2 1 1 1 2 3 4 5 6
Liear Covolutio usig the DFT I practice, the circular covolutio is implemeted usig the DFT circular covolutio property: x[] h[] = x zp [] M h zp [] = DFT 1 {DFT x zp [] DFT {h zp []}} for M 1, M = L + P 1 Advatage: This ca be more efficiet tha direct liear covolutio because the FFT ad iverse FFT are O(M log 2 M) Drawback: We must wait util we have all of the iput data This itroduces a large delay which is icompatible with real-time applicatios like commuicatios
Liear Covolutio usig the DFT I practice, the circular covolutio is implemeted usig the DFT circular covolutio property: x[] h[] = x zp [] M h zp [] = DFT 1 {DFT x zp [] DFT {h zp []}} for M 1, M = L + P 1 Advatage: This ca be more efficiet tha direct liear covolutio because the FFT ad iverse FFT are O(M log 2 M) Drawback: We must wait util we have all of the iput data This itroduces a large delay which is icompatible with real-time applicatios like commuicatios Approach: Break iput ito smaller blocks Combie the results usig 1 overlap ad save or 2 overlap ad add
Block Covolutio Problem A iput sigal x[] has very log legth, which ca be cosidered ifiite A impulse respose h[] has legth P We wat to compute the liear covolutio y[] = x[] h[] usig block legths shorter tha the iput sigal legth
Block Covolutio Example: Impulse Respose, Legth P = 6 5 h[] -5 1 2 3 Iput Sigal, Legth 33 Liear Covolutio, Legth 38 5 5 x[] y[] -5-5 1 2 3 1 2 3
Overlap-Add Method We decompose the iput sigal x[] ito o-overlappig segmets x r [] of legth L: { x[] rl (r + 1)L 1 x r [] = otherwise The iput sigal is the sum of these iput segmets: x[] = x r [] The output sigal is the sum of the output segmets x r [] h[]: r= y[] = x[] h[] = x r [] h[] (1) r= Each of the output segmets x r [] h[] is of legth = L + P 1
Overlap-Add Method We ca compute each output segmet x r [] h[] with liear covolutio DFT-based circular covolutio is usually more efficiet: Zero-pad iput segmet x r [] to obtai x r,zp [], of legth Zero-pad the impulse respose h[] to obtai h zp [], of legth (this eeds to be doe oly oce) Compute each output segmet usig: x r [] h[] = DFT 1 {DFT {x r,zp []} DFT {h zp []}} Sice output segmet x r [] h[] starts offset from its eighbor x r 1 [] h[] by L, eighborig output segmets overlap at P 1 poits Fially, we just add up the output segmets usig (1) to obtai the output
Overlap-Add Method Overlap-Add, Iput Segmets, Legth L = 11 Overlap-Add, Output Segmets, Legth L+P-1 = 16 5 5 x [] y [] -5-5 1 2 3 1 2 3 5 5 x 1 [] y 1 [] -5-5 1 2 3 1 2 3 5 5 x 2 [] y 2 [] -5-5 1 2 3 Overlap-Add, Sum of Iput Segmets 1 2 3 Overlap-Add, Sum of Output Segmets 5 5 x[] y[] -5-5 1 2 3 1 2 3
Overlap-Save Method Basic Idea We split the iput sigal x[] ito overlappig segmets x r [] of legth L + P 1 Perform a circular covolutio of each iput segmet x r [] with the impulse respose h[], which is of legth P usig the DFT Idetify the L-sample portio of each circular covolutio that correspods to a liear covolutio, ad save it This is illustrated below where we have a block of L samples circularly covolved with a P sample filter
Overlap-Save Method Overlap-Save, Iput Segmets, Legth L = 16 Overlap-Save, Output Segmets, Usable Legth L - P + 1 5 5 Usable (y []) Uusable x [] y p [] -5-5 1 2 3 1 2 3 5 5 Usable (y 1 []) Uusable x 1 [] y 1p [] -5-5 1 2 3 1 2 3 5 5 Usable (y 2 []) Uusable x 2 [) y 2p [] -5-5 1 2 3 1 2 3 Overlap-Save, Cocateatio of Usable Output Segmets 5 y[] -5 1 2 3