Discrete Fourier Transform

Discrete Fourier Transform Virtually all practical signals have finite length (e.g., sensor data, audio records, digital images, stock values, etc). Rather than considering such signals to be zero-padded to infinite length, it makes sense to think of them as vectors with either real or complex coordinates. This approach leads to a number of particularly efficient methods of implementation and spectral analysis of LTI systems. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 6 1/1

Discrete Fourier Transform (cont.) As our first step, we will need to adapt our definition of Fourier transform for discrete, finite length sequences. Given samples x[n], 0 n < of an -length sequence, its Discrete Fourier Transform (DFT) X[k] is defined by X[k] = 1 n=0 x[n]e j 2π kn, 0 k <. Given DFT coefficients X[k], 0 k <, their related time- domain samples x[n], 0 n <, can be recovered by the inverse DFT (IDFT) given by x[n] = 1 1 k=0 X[k]e j 2π kn, 0 n <. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 6 2/1

Discrete Fourier Transform (cont.) ote that X[k] is a function of the discrete frequency index k, which corresponds to normalized circular frequencies ω k = 2π k, k = 0, 1,..., 1, that are spread uniformly in the interval [0, 2π). In summary, we have DFT pair X[k] = 1 n=0 x[n]w kn DFT x[n] = 1 1 k=0 X[k]W kn W e j 2π Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 6 3/1

Discrete Fourier Transform (cont.) The correctness of the DFT formulas can be validated through X[k] = 1 n=0 x[n]w nk = 1 n=0 [ 1 = 1 m=0 1 m=0 X[m]W mn X[m] [ 1 1 n=0 ] W nk = W (k m)n ], and the fact that 1 1 n=0 W (k m)n = 1 W k, W m = { 1, k m = r 0, otherwise. ote that the complex numbers {W k } 1 k=0 are, in fact, the roots of unity (i.e., the solutions of z 1 = 0), since they all satisfy ( ) W k = e j2πk = 1. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 6 4/1

Example Consider the DFT of δ[n], which is given by X[k] = 1 n=0 δ[n]w nk = W 0k = 1, k. Substituting X[k] in the inverse DFT formula, we have x[n] = 1 1 k=0 W nk = 1 1 k=0 W (0 n)k = { 1, n = 0 0, 1 n 1. according to the orthogonality of the roots of unity. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 6 5/1

Matrix formulation of DFT The equations for the DFT coefficients can be expressed in matrix form as X[0] X[1]... X[k] =... X[ 1] }{{} X W 00 W 01... W 0( 1) W 10 W 11... W 1( 1).................. W nk............... W ( 1)1... W ( 1)( 1) W ( 1)0 } {{ } W x[0] x[1]... x[n]... x[ 1]. } {{ } x Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 6 6/1

Matrix formulation of DFT (cont.) Thus, we have X = W x. The matrix W is called the DFT matrix. ote that W is symmetric (W = W T ) and orthogonal, viz. W H W = I = W 1 = 1 WH = 1 W. Therefore, x can be recovered from X according to x = W 1 X = 1 W X, which is nothing else but a matrix representation of IDFT. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 6 7/1

Inherent periodicity of DFT and IDFT The twiddle factor W is periodic in both k and n with fundamental period, namely W (k+)n = W kn and W k(+n) = W kn. Given DFT coefficients {X[k]} 1 k=0 and letting k Z results in the Discrete Fourier Series (DFS) { X[k]} k Z, which is an -periodic sequence. Similarly, given IDFT coefficients {x[n]} 1 n=0 and letting n Z results in the inverse Discrete Fourier Series (IDFS) { x[n]} n Z, which is an -periodic sequence as well. These periodicities are an inherent property of DFT, which stem from the discrete nature of time and frequency variables. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 6 8/1

Sampling of DTFT Recall that the samples of an infinite-length sequence x[n], n Z, can be uniquely recovered from its DTFT X(e jω ) according to x[n] = 1 2π π π X(e jω )e jωn dω. Suppose that we sample X(e jω ) at equally spaced frequencies X[k] = X(e jω ) = x[n]e j 2π kn, k = 0, 1,..., 1. ω=2πk/ n= How many samples of the sequence x[n] can be recovered from the samples of its DTFT? Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 6 9/1

Sampling of DTFT (cont.) To answer the above question, we note that X[k] = n= = 1 n=0 x[n]e j 2π kn = [ l= l= x[n l] ] l+ 1 n=l e j 2π kn = where x[n] is a periodization of x[n] given by x[n] x[n l]. l= x[n]e j 2π kn = 1 n=0 x[n]e j 2π kn, Thus, if x[n] is zero outside 0 n L 1 and L, the shifted replicas of x[n] do not overlap, in which case { x[n], 0 n 1, x[n] = 0, otherwise. Otherwise, we have time-domain aliasing. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 6 10/1

Time-domain aliasing Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 6 11/1

Reconstruction of DTFT So, how is X(e jω ) related to its sampled version X(e j2πk/ )? be a finite-length se- To address this question, let {x[n]} 1 n=0 quence and X(e jω ) be its DTFT. Given a rectangular window p of length, we note that [ 1 x[n] = 1 k=0 X(e j 2π k )e j 2π kn ] } {{ } x[n] = 1 p [n] = 1 k=0 [ ] X(e j 2π k ) e j 2π kn p [n]. Thus, x[n] is a linear combination of windowed exponentials e j 2π kn p [n]. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 6 12/1

Reconstruction of DTFT (cont.) ow, the DTFT of p [n] is known as the Dirichlet function given by P (e jω ) = sin(ω/2) sin(ω/2) e jω( 1)/2, which suggests that the DTFTs of {e j 2π kn p [n]} 1 k=0 are its shifted versions. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 6 13/1

Reconstruction of DTFT (cont.) Consequently, by the linearity of DTFT, we obtain X(e jω ) = 1 1 k=0 X(e j 2π k 2πk j(ω )P (e ) ). Therefore, X(e jω ) is given by a linear combination of periodically shifted Dirichlet kernels, with the coefficients of the linear combination equal to 1 X(e j 2π k ). ote that P (e jω ) crosses zero at ω k = 2πk/, 0 k <, and as a result, it appears as δ[k], when restricted to these points. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 6 14/1

Zero-padding Given {x[n]} 1 n=0, one can make the plot of X(ej 2π k ) appear to be almost continuous by computing with K. X(e j 2π K k ) = 1 n=0 x[n]e j 2π K kn, 0 k < K, It is straightforward to check that X(e j 2π K k ), k = 0, 1,..., K 1, is given by the DFT of x[n] padded by zeros to the length of K. The resulting {X[k]} K 1 k=0 is usually a better representation of the DTFT X(e jω ) for display purposes. However, it does not contain additional information that can be exploited by signal processing algorithms. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 6 15/1

Example: Rectangular pulse To illustrate the idea of zero-padding, let x[n] = u[n] u[n ] for = 4 and different values of K. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 6 16/1

Example: Rectangular pulse (cont.) To explain the above results, we first note that X(e jω ) = 1 n=0 e jωn = 1 e jω sin(ω/2) = 1 e jω sin(ω/2) e jω( 1)/2, which is, obviously, a Dirichlet kernel. The K-point DFT of the zero-padded sequence yields X[k] = X(e j 2π K k ) = sin(πk/k) sin(πk/k) e jπ( 1)k/K. If we select K =, the DFT becomes X[k] = δ[k], 0 k < K, which corresponds to sampling X(e jω ) at its zero crossings. When K >, zero-padding makes the shape of the DTFT more evident by evaluating samples over a denser frequency grid. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 6 17/1

CTFT vs DTFT vs DFT Suppose that we are given a continuous-time signal x c (t) with continuous-time Fourier transform (CTFT) X c (jω). The DTFT X(e jω ) of its samples x[n] = x c (nt ) is given by X(e jωt ) = n x c (nt )e jωt n = 1 T m Z X c ( j ( Ω 2π T m)). The -point DFT X[k] is obtained by sampling X(e jω ) at 2π k (or, alternatively, by sampling X(e jωt ) at 2π T k), namely X[k] = 1 T where k = 0, 1,..., 1. k ( 2π X c (j T k 2π )) T m, Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 6 18/1

CTFT vs DTFT vs DFT (cont.) Sampling the DTFT of x[n] results in periodization of x[n] with period or, equivalently, of x c (nt ) with period T. Specifically, x[n] = k x c (nt T k) Therefore, we have the following -point DFT pair x c (nt T k) 1 ( 2π X c (j T T k 2π )) T m, k where 0 n 1 and 0 k 1. The above relation reveals a frequency-domain aliasing caused by time-domain sampling and a time-domain aliasing caused by frequency-domain sampling. This explains why periodicity is built-in in the whole concept of discrete Fourier transformations. k Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 6 19/1

CTFT vs DTFT vs DFT (cont.) Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 6 20/1

Properties of DFT Linearity y[n] = a 1 x 1 [n] + a 2 x 2 [n] Y [k] = a 1 X 1 [k] + a 2 X 2 [k] Time-reversal circular symmetry x[ n ] X [k] where n = (n mod ) and { x[0], n = 0, x[ n ] = x[ n], 1 n <. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 6 21/1

Properties of DFT (cont.) An -point sequence x[n] is called circularly even if x[n] = x[ n ]. An -point sequence x[n] is called circularly odd if x[n] = x[ n ]. Then, one can show that: Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 6 22/1

Properties of DFT (cont.) Consider a sequence {x[n]} 1 n=0 What is the inverse DFT of X[k]W mk? z[n] = 1 1 k=0 which suggests that and its DFT {X[k]} 1 k=0. ( ) X[k]W mk W nk = 1 1 k=0 z[n] = x[ n m ], n = 0, 1,..., 1. X[k]W k(n m), This operation is called a circular shift. Thus, we have Circular shift property x[ n m ] W mk X[k] Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 6 23/1

Circular convolution Let s compute the IDFT of Y [k] = X[k] H[k]. y[n] = 1 = 1 1 k=0 1 k=0 H[k]X[k]W kn = [ 1 m=0 = h[m]w mk 1 m=0 ] [ 1 1 h[m] l=0 l=0 x[l] h[l]w lk [ 1 ] 1 k=0 ow, using the orthogonality condition, we obtain y[n] = 1 m=0 h[m]x[ n m ], 0 n <. W kn = W k(m+l n) This operation is known as circular convolution, denoted as y[n] = (x h)[n]. ]. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 6 24/1

Circular convolution (cont.) Thus, we have Convolution property of DFT y[n] = (x h)[n] Y [k] = X[k] H[k] Let s compute y[n] for the case of = 4. We have y[0] x[0] x[3] x[2] x[1] y[1] y[2] = x[1] x[0] x[3] x[2] x[2] x[1] x[0] x[3] y[3] x[3] x[2] x[1] x[0] } {{ } X h[0] h[1] h[2] h[3] ote that the columns of X are generated by circularly shifting x[n]. A matrix generated in this way is called a circulant matrix. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 6 25/1

Circular correlation The circular correlation of two -point sequences x[n] and y[n] is defined by r xy [l] = 1 n=0 x[n]y[ n l ], 0 l <. Comparison with circular convolution reveals that circular correlation requires circular shifting but not circular folding. Thus, r xy [n] = (x ỹ)[n] R[k] = X[k]Y [k], where ỹ[n] = y[ n ]. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 6 26/1

Circular correlation (cont.) The operation of correlation is commonly used to detect patterns in input sequences via matching filtering. Given an input signal x[n] and a kernel h[n] (zero-padded to the length of x), we compute: r=ifft(fft(x).*conj(fft(h)). ote the phase shift. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 6 27/1

Circular correlation (cont.) To eliminate the phase shift, the kernel h[n] has to be centred around the origin, i.e., around n = 0. ote that now the maximum of the autocorrelation sequence correctly pinpoints the location of the required pattern. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 6 28/1

Summary of the properties of DFT Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 6 29/1

Linear convolution via DFT The linear convolution of two finite-length sequences {x[n]} n=0 L 1 and {h[n]} M 1 n=0 is a sequence y[n] of length L + M 1, given by y[n] = k= h[k]x[n k], 0 n < L + M 1. The DTFT of y[n] is given by Y (e jω ) = X(e jω ) H(e jω ), and y[n] can be recovered uniquely from {Y (e j2πk/ )} 1 k=0, as long as we have L + M 1. On the other hand, the IDFTs of H(e j 2πk ) and X(j 2πk ) yield the sequences h[n] and x[n] padded with ( M) and ( L) zeros, respectively. As a result, y zp [n] = x zp [n] h zp [n] Y [k] = X[k]H[k]. ote that if L + M 1, y[n] = y zp [n], 0 n < L + M 1, that is, circular convolution is identical to linear convolution. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 6 30/1

Linear convolution via DFT (cont.) Thus, linear convolution can be implemented by means of DFT as shown below. Whether the direct computation of linear convolution is more efficient than its DFT-based computation depends on the values of M and L. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 6 31/1

Fourier analysis of signals via DFT Through its efficient implementation by means of Fast Fourier Transform (FFT) algorithms, DTF provides the fundamental computational tool for practical Fourier analysis. In practice, the operation of selecting a finite number of samples is equivalent to multiplying the actual sequence x[n], n Z, by a finite-length window w[n]. For instance, { 1, 0 n < L, w[n] = 0, otherwise, which is known as a rectangular window of size L. Thus, the DFT provides samples of the DTFT of the windowed signal ˆx[n] = w[n]x[n], not of the original signal x[n]. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 6 32/1

Effects of time-windowing Consider the following signal x[n] = A 1 cos(ω 1 n + φ 1 ) + A 2 cos(ω 2 n + φ 2 ), n Z, whose windowed version ˆx[n] is given by ˆx[n] = w[n]x[n] = A 1 2 w[n]ejφ1 e jω1n + A 1 2 w[n]e jφ1 e jω1n + + A 2 2 w[n]ejφ2 e jω2n + A 2 2 w[n]e jφ2 e jω2n. Consequently, the DTFT of ˆx[n] is given by ˆX(e jω ) = A 1 2 ejφ1 W (e j(ω ω1)n ) + A 1 2 e jφ1 W (e j(ω+ω1)n )+ + A 2 2 ejφ2 W (e j(ω ω2)n ) + A 2 2 e jφ2 W (e j(ω+ω2)n ), where W is a Dirichlet kernel in the case when w[n] is a rectangular window. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 6 33/1

Effects of time-windowing (cont.) Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 6 34/1

Effects of time-windowing One can see that W (e jω ) has one large peak ( mainlobe ) and a series of subsidiary peaks of decreasing magnitude ( sidelobes ). The mainlobe has zero-crossings at multiples of 2π/ and an approximate 3 db bandwidth of 2π/ rads. Therefore, to resolve two frequency components ω rad apart, should satisfy ω = ω 2 ω 1 2π. The above effect is known as spectral spreading or smearing. One the other hand, the non-local support of W (e jω ) creates the effect of spectral leakage, which causes transfer of power from one band to another. Leakage creates false peaks, that is, peaks at wrong frequencies, nonexisting peaks, or changes the amplitude of existing peaks. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 6 35/1

Window choices In general, the DTFT of ˆx[n] = x[n]w[n] is given by ˆX(e jω ) = 1 π X(e jθ )W (e j(ω θ) )dθ. 2π π Thus, a good window should have a narrow mainlobe (to minimize the effect of spectral spreading) and low sidelobes (to minimize the effect of spectral leakage). A simple way to reduce the level of sidelobes of the rectangular window by a factor of 2 (in db) is based on squaring its DTFT. Indeed, let w R [n] be a rectangular window. Then, w B [n] = w R [n] w R [n] W B (e jω ) = W 2 R(e jω ). is a triangular (aka Bartlett) window, whose sidelobes decay two times faster (in db) as compared to the Dirichlet kernel. Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 6 36/1

Window choices (cont.) Alternatively, one can reduce the level of the sidelobes by means of destructive superposition, i.e., by defining W (e jω ) = aw R (e jω ) + bw R (e j(ω 2π/) ) + bw R (e j(ω+2π/) ). Particularly, setting a = 0.5 and 0.25 results in the Hann window. w Han [n] = 0.5 0.5 cos(2πn/( 1)), n = 0, 1,..., 1. Alternatively, setting a = 0.54 and 0.23 results in the Hamming window. w Ham [n] = 0.54 0.46 cos(2πn/( 1)), n = 0, 1,..., 1 Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 6 37/1

Window choices (cont.) Prof. O. Michailovich, Dept of ECE, Winter 2017 ECE 413: Digital Signal Processing / Section 6 38/1