Seminar presentation at the Politecnico di Milano, Como, November 12, 2012 Fundamentals of the Discrete Fourier Transform Michael G. Sideris sideris@ucalgary.ca Department of Geomatics Engineering University of Calgary
Contents (1) From Fourier Series to the Continuous Fourier Transform Sinusoids Fourier Series The Continuous Fourier Transform (CFT) Important elementary functions and their CFTs The impulse function The CFT of sine and cosine The sampling function The rectangle and sinc functions Properties of the CFT Convolution and Correlation Properties of convolution
Contents (2) From the CFT to the DFT (Discrete Fourier Transform) Aliasing Leakage Periodicity The Discrete Fourier Transform Circular convolution Scale change Phase shifting Correlation, Covariance and Power Spectra Density functions The Fast Fourier Transform (FFT) The two-dimensional CFT and DFT
From Fourier Series to the Continuous Fourier Transform
Sinusoids s(t) = A 0 cos(ω 0 t +φ 0 ) : Sinusoid of frequency ω 0 A 0... amplitude ω 0... cyclic frequency t... time (or distance) φ 0... phase angle ω 0 = 2π T 0 = 2π f 0 T 0... period f 0...(linear) frequency Expansion s(t) = acosω 0 t + bsinω 0 t where a = A 0 cosφ 0, b = A 0 sinφ 0 A 0 = (a 2 + b 2 ) 1/2 φ 0 = arctan( b a ) Sinusoids in complex form s c (t) = acosω 0 t ± iasinω 0 t = ae ±iω 0 t Real sinusoids in complex form s(t) =A 0 cos(ω 0 t + φ 0 ) = A 0 e i(ω 0 t +φ 0 ) + e i(ω 0 t +φ 0 ) 2 = A 0 2 eiφ 0 e iω 0t + A 0 2 e iφ 0 e iω 0t
Fourier Series If g(t) = g(t +T ); T g(t)dt = g(t)dt, then 0 t 0 +T g(t) = (a n cos 2πn T t + b n sin 2πn T t ) n=0 a n = 2 T t 0 +T t 0 g(t)cosnt dt ; b n = 2 T t 0 t 0 +T t 0 g(t)sin nt dt Provided that: g(t) has a finite numbers of maxima and minima in a period and a finite number of finite discontinuities (Dirichlet s conditions) Complex form : g(t) = 1 T T /2 G n e iωnt, ω n = 2πn n= T G n = g(t)e iωnt dt, G n = 1 2 (a n ib n ), n = 0,±1, ± 2,... T /2 Call Δω = 2π T ) ω n = nδω + * 1 T = Δω, + 2π g(t) = G n n= 2π eiωnt Δω
The Continuous Fourier Transform (1) g(t) = 1 2π G(ω) = G(ω)e iωt dω Inverse CFT g(t) e iωt dt Direct CFT Since ω = 2πf g(t) = G( f )e i2π ft df = F -1 G( f ) { } G( f ) = g(t) e i2π ft dt = F g(t) { } G( f ) = G R ( f )+ ig I ( f ) = G( f ) e iθ ( f ) G( f ) is complex Amplitude: G( f ) = " # G 2 R ( f )+ G 2 I ( f ) $ % 1/2 Phase angle: θ( f ) = Arg{ G( f )} = arctan G ( f ) I G R ( f )
The Continuous Fourier Transform (2) Conditions for Existence : The integral of g(t) from to + exists (it is < ) g(t) has only finite discontinuities If g(t) is periodic or impulse, G( f ) does not exist
Important elementary functions and their CFTs
The Impulse Function (1) Definition : δ(t t 0 ) = 0, t t 0 ' ) ( δ(t t 0 )dt =1 ) * Definition as a distribution δ(t) = lim f (t,a) δ( t t ) φ( t )dt = a 0 0 φ ( t 0 )
The Impulse Function (2) Definition as a generlized limit: If lim f n (t)φ(t)dt = φ(0) δ(t) = n Properties: δ(t 0 )h(t) = h(t 0 )δ(t 0 ) δ(at) = a 1 δ(t) F{ Kδ(t) } = K sin at e.g. If we define it as δ(t) = lim, a πt then cos(2π ft) df = e i2π ft df = δ(t) lim f n (t) n Used, as a distribution, for the (otherwise nonexistent) CFT of periodic functions
The CTF of cosine and sine Acos(2πf o t) A 2 δ( f f o ) + A 2 δ( f + f o ) Asin(2πf o t) i A 2 δ( f + f o ) i A 2 δ( f f o )
The Sampling Function III(t) = δ(t nt ) F III(t) n= { } = 1 T n= δ( f n T ) III(t) f (t) = n= f (nt )δ(t nt ) Digitization Note that denser sampling in one domain transforms to sparser sampling in the other
The Rectangle and sinc Functions h(t) = " A, t = T 0 /2 $ # A /2, t = ±T 0 /2 $ % 0, t > T 0 /2 H( f ) = 2AT o sinc(2t o f ), sinc( f ) = sin(πf ) πf
Properties of the CFT
Properties of the CFT (1) Linearity ah(t) + bg(t) ah( f ) + bg( f ) Symmetry H(t ) h( f ) Time scaling h(at) 1 a H( f a ) Time shifting Differentiation Integration h(t t o ) H( f )e i 2πft o n h(t) t n (i 2πf ) n H( f ) t h(x )dx 1 i 2πf H( f ) + 1 2 H(0)δ( f )
Properties of the CFT (2) DC-value h(t)dt = H(0) Even function h E (t) H E ( f ) = R E ( f ) Odd function h O (t ) H O ( f ) = ii O ( f ) Real function h(t) = h R (t) H( f ) = R E ( f ) + ii O ( f ) Imaginary function h(t) = ih I (t) H( f ) = R O ( f ) + ii E ( f )
Convolution and Correlation
Convolution and Correlation (1) x(t) = g(t')h(t t')dt'= g(t) h(t) = h(t) g(t) = h(t')g(t t')dt' y(t) = g(t')h(t + t')dt'= g(t) h(t) h(t) g(t) Convolution theorem X( f ) = F{ g(t) h(t) } = F{ g(t) }F{ h(t) } = G( f )H ( f ) Convolution is the operation of filtering one of the functions by the other Correlation theorem Y ( f ) = F{ g(t) h(t) } = G( f )H * ( f )
Pictorial Representation of Convolution and Correlation
Convolution and Correlation (3) Properties: a) If either g(t) or h(t) is even, then g(t) h(t) = g(t) h(t) b) g(t) h(t) = h(t) g(t) convolution is commutative c) [g(t) h(t)] q(t) = h(t) [g(t) q(t)] convolution is associative d) k(t) [ag(t) bq(t)] = a[k(t) g(t)]+ b[k(t) q(t)] convolution is linear e) δ(t +τ ) h(t) = h(t +τ ), δ(t) h(t) = h(t) δ(t) is the identity operator in convolution f) x'(t) = (g(t) h(t))' = g'(t) h(t) = g(t) h'(t) the derivative of the g) F{ h(t)g(t) } = F{ h(t) } F{ g(t) } = H ( f ) G( f ) convolution
Convolution and Correlation (4) h) If h(t) and g(t) are time limited functions, i.e., non zero in the domain T 0 t T 0, then x(t) = h(x) g(t) is time limitedwith twice the support of h(t) or g(t), i.e., non zero in the domain 2T 0 t 2T 0 i) Parseval's thereom: h 2 (t)e 2πσ t dt = H ( f )H (σ f )df with σ = 0 and for h(t) real: h 2 (t)dt = H ( f ) 2 df
From the CFT to the DFT (Discrete Fourier Transform)
From the Continuous to the Discrete Fourier Transform aliasing leakage periodicity
The Discrete Fourier Transform
The Discrete Fourier Transform (1) H(mΔf ) = h(kδt) = N 1 h(kδt)e i2πkδtmδf Δt = h(kδt)e i2πkm / N Δt N 1 k= 0 N 1 k= 0 H(mΔf )e i2πkδtmδf Δf = H(mΔf )e i2πkm / N Δf m= 0 N 1 m= 0 h(kδt) H (mδf ) or h(t k ) H ( f m ) or h(k) H (m) T o = 1 Δf = NΔt, F o = 1 Δt = NΔf f N = F o 2 = 1 2Δt Nyquist frequency
The Discrete Fourier Transform (2) Illustration of Nyquist Frequency Highest frequencies existing in the original signal cannot be reproduced from the samples In this example, a sinusoid of lower frequency is recovered (the higher frequencies cannot be recovered; they are aliased due to the under-sampling)
Linear Convolution Discrete Convolution (1) For g(k) and h(k) defined in 0 k < N: x(k) = g(k) h(k) = g(l)h(k l)δt = g(l)h(k l)δt, 0 k 2N 1 l= M-point Circular Convolution k l=0 x c (k) = g(k) h(k) = M 1 l=0 g(l)h(k l) M Δt DFT && N F{g(k)}F{h(k)} Circularly shifted within M positions x c (k) = M 1 g(l)h(k l) M Δt = g(l)h(k l)δt + g(l)h(n + k l)δt l=0 k l=0 M 1 l=k+1 Linear convolution plus Aliasing
Discrete Convolution (2) Linear Convolution via FFT using Circular Convolution M 1 x c (k) = x(k) iff g(l)h(m + k l)δt = 0 l=k+1 which happens when M 2N 1 Zero-padding Procedure To compute the linear convolution of two functions given in 0 k N-1: 1. Append at least N-1 zeros to each function so that they are now 2N-1 samples long (preferably, 2 p 2N-1) each 2. Use FFT to compute their spectra 3. Use FFT to compute the Inverse DFT of the product of the two spectra. This, i.e., the resulting values for 0 k 2N-1, is the correct linear convolution of the two functions
Circular Convolution as Linear Convolution Plus Aliasing
The DFT in Computers Subroutines usually assume Δt = 1 and also ignore T 0 This requires rescaling as follows: H( f m ) = T o H c (m) = NΔtH c (m) x(t k ) = g(t k ) h(t k ) = T o x c (t k ) = T o F c 1 {G c (m)h c (m)} It also yields: H c (0) = h Subroutines also assume the origin at the left of the record This requires changing the phase of the spectrum by e i2π mδft o/2 = e iπ m = cos(mπ ) = ( 1) m h(t k T o / 2) ( 1) m H( f m ) End point of a period must be omitted (assumed due to periodicity)
Correlation, Covariance and Power Spectral Density functions
Definitions: CR, CV and PSD Functions R gh (t k ) = E{g(t l )h(t k + t l )} = lim 1 g(t N N l )h(t k + t l ) N 1 l=0 = lim To 1 T o g(t k ) h(t k ) C gh (t k ) = E{{g(t l ) g][h(t k + t l ) h ]} = lim 1 [g(t N N l ) g][h(t k + t l ) h ] 1 = lim g(t k ) h(t k ) gh = R gh (t k ) gh To T o 1 P gh ( f m ) = F{R gh (t k )} = lim G( f m )H ( f m ) To T o Computation by FFT: ˆP gh ( f m ) = 1 νt o ν λ=1 ˆR gh (t k ) = F 1 { ˆP gh ( f m )} G λ ( f m )H λ ( f m ) Ĉ gh (t k ) = F 1 { ˆP gh ( f m ) ghδ( f m )} N 1 l=0
The Fast Fourier Transform
The FFT - Flow Graph of Operations for N=4 H = Wh FFT achieves its speed by factorizing W and exploring its symmetries in a way that the required number of multiplications and additions is drastically reduced
Computational efficiency: FFT vs DFT 1D case DFT: O(N 2 ) FFT: O(NlogN) Bergland (1969)
The 2D Continuous and Discrete Fourier Transform
The Two-dimensional CFT H (u,v) = h(x, y) = h(x, y)e i2π (ux+vy) dx dy H (u,v)e i2π (ux+vy) du dv The Fourier Transform is a separable transform Very important for practical applications H (u,v) = h(x, y)e i2π (ux+vy) dx dy = h(x, y)e i2πux e i2πvy dx dy = ( h(x, y)e i2πux dx)e i2πvy dy = F x {h(x, y)}e i2πvy dy = F y {F x {h(x, y)}}
The Two-dimensional DFT M 1 N 1 H(u m,v n ) = h(x k, y l ) e i2π (mk / M +nl / N ) ΔxΔy k= 0 1= 0 M 1 N 1 h(x k, y l ) = H(u m,v n ) e i2π (mk / M +nl / N ) ΔuΔv k= 0 1= 0 SPACE DOMAIN FREQUENCY DOMAIN Δu = 1 T y = 1 MΔx, Δv = 1 T y = 1 NΔy Δx = 1 F u = 1 MΔu = 1 2u N, Δy = 1 F v = 1 NΔv = 1 2v N
Concluding Remarks
Concluding Remarks Fourier Transforms are important in geodetic applications for, e.g., Spectral analysis of data Can decide on the spectral content, appropriate sampling interval, and/or best computational method Digital signal/image processing Filtering of data Filter design Noise minimization Efficient evaluation of convolutions and of covariance functions Fourier Transforms are subject to errors due to Aliasing (sampling) Leakage (truncation of domain) Periodicity (digitization) Data noise which should be understood and minimized in practical applications