A523 Signal Modeling, Statistical Inference and Data Mining in Astrophysics Spring 2011 Lecture 6 PDFs for Lecture 1-5 are on the web page Problem set 2 is on the web page Article on web page A Guided Tour of the FFT (Bergland 1969) as much about the DFT as it is about the FFT algorithm Today: Properties of the DFT and how to use the DFT Examples of DFT/FFT applications
DFT Properties Frequency mapping: x(t) =e iω 0t = e 2πif 0t in an N-point DFT, in which bin does the signal fall (mostly)? T = Nδt δf = 1/T define f j = jδf for j = 0, N/2 Nyquist frequency = (N-j) δf for j = N/2+1, N-1 f N = Nδf/2 = N/2T = N/2Nδt = 1/2δt For f 0 f N we have j 0 = f 0 Nδt The frequency bin j 0 : varies with N for fixed f 0 and δt varies with δt for fixed f 0 and N
ASTRONOMY 6523 Spring 2011 Problem Set 2 Due 22 Feb 2011 1. This problem is to help develop your intuition on the relationship of discrete sampling in one domain and periodicity in the other. Consider a CT function that is periodic with period P. f(t) = j g(t jp), where g(t) isanarbitrarypulseshapebutcouldbeagaussianfunction,forexample. TheFTof g(t) is G(f). In each of the steps below, sketch the results in both domains (t and f), assuming a gaussian function. Label the results appropriately to indicate separations, widths, etc. (a) Show that the FT of f(t) canbewrittenintheform F (f) = G(f) m δ(f m/p ) where m is an integer. You may think of FT as being a sampled version of the FT of the pulse shape, G(f) withsamplesinfrequencyspacedby1/p. (b) Now sample the time-domain function by multiplying by the bed of nails function with samples spaced by the interval t, wherek is an integer: f s (t) =f(t) k δ(t k t) Show that the FT of this periodic, DT function is periodic in frequency with spacing l/ t, where l is an integer. (c) Interpret your results. 2. In Lecture 5 we considered a sampled complex sinuosid with noise added. Here you will analyze just the sinusoidal part of the signal. (a) Show that the N-point DFT of a complex, DT sinusoid x n = Ae iωo nδt is N 1 X k = N 1 n=0 X n e 2πink/N = AN 1 e iφn sin N 2 (ω 0δt 2πk/N) sin 1 2 (ω 0δt 2πk/N). (1) Hint: consider a finite sum geometric series. (b) The integer index k implies that the DFT is sampled at a discrete set of frequencies; however, the true signal frequency, f 0 = ω 0 /2π, generallywillnotcoincidewiththenearest discrete frequency. By considering values of k that yield frequencies nearest f 0,investigate the dependence of the maxium of the DFT on f 0 to find the range of amplitudes that
you will see as a function of f 0. You will thus quantify the so-called scalloping effect. Hint: plot the magnitude of Xk vs. f 0 for the case where f 0 equals one of the discrete frequencies. Then plot the magnitude when f 0 falls exactly in between two of the discrete frequencies. 3. This problem explores the effects of aliasing that arise in numerical integration. Estimate the value of the integral by using a summation I = Î = k= where x k = k x and the hat denotes estimate. dx e x2 cos 2πx e x2 k cos 2πxk i. Without explicitly doing any integral (i.e. by using Fourier transform theorems), determine the value of I. Hint: multiply the integrand e x2 k cos 2πx k by e 2πift and calculate the FT, then set f =0togetthevalueoftheoriginalintegral. ii. Estimate the value of I using the summation, Î, byconsideringthefouriertransform of the integrand, by using the convolution theorem, and by considering aliasing. Consider x =0.5 inthesummationforî. Sketch the FT of the integrand. By what percentage will the estimate Î for I be in error? You can obtain this answer without explicitly doing any integrals and certainly without writing code! iii. Same as ii except for x =0.4. iv. To see the effects of aliasing further, you could plot the value of Î vs. x. v. Explain your results. 4. Numerically investigate a sinusoid + noise, x n = Ae i2πf0nδt + n n, n =0,...,N 1, sampled at times t n = n t. For specificity, consider f 0 =50.1367 Hz, t =0.005 sec and a total number of samples, N =1024. Forthenoiseyoucanuseforeach of the real and imaginary parts of n n the pseudo noise, (6)(u n 1/2), where u n is a random number between 0 and 1 generated with a standard random number generator; the normalization is such that n n has zero mean and unit variance, σ 2 n = n n 2 =1. Investigatethespectralanalysisofthissignalbydoing the following: (a) Generate a realization of x n for the choice (S/N) t A/σ n =0.25. (b) Calculate X k, the N-point DFT of x n, using an FFT algorithm. Note that x n is complex. (c) Calculate the squared magnitude of the DFT, X k 2 as an estimator for the spectrum, S k. (d) Consider the detection scheme consisting of finding the peak of the spectrum, S k,and identifying the k value and the amplitude. Calculate S/N for the peak as (S/N) DFT =
S kmax /σ S,whereσ S is the rms noise of the spectrum, which you can compute from the spectrum while excluding the spectral line. (e) Repeat steps (a)-(d) for a large number of realizations (10 or 100, say) and investigate the statistics of the S/N and of kmax. How much do they vary? What are their means and standard deviations? (f) Now develop a new detection scheme that takes into account that the true frequency of the sinusoid does not fall neatly upon one of the DFT s frequency samples. For example, you might consider combining 2 or more samples that include the peak. Develop a scheme without customizing it to the particular frequency of this example. I.e. develop a scheme that could be applied if you do not know f 0 and you don t know aprioriwhether the line is split between one or more bins of the DFT. (g) Apply your scheme to a large number of realizations and evaluate its performance as you did the detection scheme in (d). (h) You can consider other values of (S/N) t also, e.g. a factor of two larger or smaller.
1 Sampling Theorem, Interpolation and Aliasing If samples are obtained according to the sampling theorem, then no information is lost. If so, it must be possible to reconstruct the original, continuous time-domain signal from the discrete samples. To get the interpolation relation we substitute into the inverse Fourier transform (IFT) f(t) = df F (f) e 2πift (1) = f f df F (f) e 2πift (2) = f f df F p (f) }{{} e2πift (3) But since we assume f(t) is bandlimited, we can write the periodic extension of F (f) asa Fourier series: F p (f) C m e 2πi m W f (4) So f(t) = = f f 1 2 f = df f m m= 1 2 f m= f( m 2 f ) m e 2πi 2 f f (5) {}}{ [ 1 f( m 2 f 2 f ) m f] e 2πi 2 f e 2πift (6) ( m 2 f m= ) f f df e 2πif(t m 2 f ). (7) Use Thus, f f m df e 2πif(t m 2 f ) = e2πi f(t 2δf ) e 2πi f(t m 2 f ) 2πi(t m ) 2 f sin 2π f(t m 2 f ) π(t m 2 f ) (8) f(t) = m= f ( m 2 f ) sin 2π f(t m 2 f ) 2π f(t m 2 f ) Note that this holds for Nyquist ( critical ) sampling only.
2 We can rewrite the interpolation formula as f(t) =f s (t) sin 2π ft 2π ft where f s (t) δt f(t) S(t, δt) with δt = 1 2 f The interpolation relation shows how the CT signal can be reconstructed from discrete samples that were obtained by sampling at the Nyquist rate f N =2 f. The importance of this result is that appropriately sampled functions contain all the information in the bandlimited function. Rarely does one actually perform the interpolation if the aim is to perform time-series analysis. In other contexts, however, interpolation is often necessary (e.g. in using tabulated data or in taking data with unequal sample intervals and projecting them onto a uniform grid. Interpolation applications will be treated later.).
3 Sampling at Non-Nyquist Rates Suppose the sample interval is δt. Then the (implied) period in f space is W =1/δt and we can write the periodic extension of F (f) as a Fourier series (again) F p (f) = m C m e 2πimf/W (9) C m = 1/W W/2 W/2 Now we have gaps if 1/δt f > f, δt <1/2 f f 1 n df F p (f) e 2πimf/W (10) 1/2δt = δt df F p (f) e 2πimf/W (11) 1/2δt So long as δt < 1 2 f we can truncate the integral for C m so that C m = δt f f df F p (f) e 2πimδtf (12) = δt f f df F (f) e 2πmδtf δt< 1 2 f (13) = δtf(t) t=mδt (14) C m = δtf(mδt) (15)
4 Now, again plugging into the IFT for f(t) using the F.S. representation we have f(t) = = f f f f df F (f) }{{} e2πift (16) {}}{ [ ] df δtf(mδt) e 2πimδtf e 2πift (17) m This gives us the interpolation function for faster-than-nyquist sampling: f(t) =2δt f m f(mδt) sin 2π f(t mδt) 2π f(t mδt) Thus, the same interpolation relation holds for sampling at any rate faster than the Nyquist rate. The interpolation function again has sin x/x form: sin 2πt f 2πt f has zeroes at 2t f = integer or t = integer 2 f For critical sampling (Nyquist sampling) the zeroes of sin 2πt f 2πt f occur at the sample locations Thus, for critical sampling, sin x/x acts as a Kronecker delta at the sample times In general, for fast sampling, the situation appears as
5 Verify that the interpolation function is bandlimited: The interpolation relation manifestly has a FT that is bandlimited in [ f, f]: Use Π(t) = By duality { 1 t 1 sin πf 2 0 otherwise πf sin πt πt By the scale theorem π( f) =Π(f) sin 2 fπt 2 fπt sinc (f) 1 a Π ( f a ) a =2 f = 1 2 f Π ( f 2 f ) By the shift theorem each term gets multiplied by e 2πimδtf. Thus, each term has a shape in frequency space Π( f 2 f )
6 Aliasing Aliasing occurs if the sample interval is too large. If δt > 1 2 f then the FT F (f) cannotbe written in terms of its periodic extension F (f). In this case Overlap of the different pieces implies that the interpolation relation no longer applies to this case, i.e. sampling has not occurred often enough so the CT signal cannot be reconstructed. As before, the largest frequency that can be represented with a certain sample interval is f max = 1. Sampling of a sinusoid with frequency f > f 2δt max will be aliased to a lower frequency f a (f) = f max +f for f max f 3 with additional folding of f into fa for other 2δt frequency ranges. The mapping of actual frequency to aliased frequency behaves as:
7 Aliasing of a Complex Exponential: Where does the amplitude appear in the transform? Consider f(t) =e 2πift Sample at intervals δt f(mδt) =e 2πifmδt DFT: N 1 F k = N 1 f(mδt) e 2πim k N (18) m=0 = N 1 m e 2πi (fδt k N )m (19) Use m=0 N 1 = N 1 m=0 a m with a e 2πi (fδt k N ) (20) N 1 a m = 1 an 1 a = an 1 a 1 = an/2 (a N/2 a N/2 ) a 1/2 (a 1/2 a 1/2 ) F k = N 1 e N 2 2πi(fδt k N ) e 1 2 2πi(fδt k N ) sin Nπ(fδt k N ) sin π(fδt k N ) F k = 1 sin Nπ(fδt k ) N, k=0,..., N 1 N sin π(fδt k ) N Check that F k is periodic ( F k+pn? = F k ): p =integer Note sin π(fδt (k+pn) N ) = sin π(fδt k N p) Let α = π(fδt k/n). Then sin(α pπ) = cos α sin pπ + sin α cos pπ =+( 1) p sin α Therefore, ( sin Nπ fδt (k + pn) ) N = sin(nα Npπ) (21) = cos Nα sin Npπ + sin Nα cos Npπ (22) = +( 1) Np sin Nα (23) F k+ p = N 1 sin Nα ( 1)Np N = sin α ( 1) p Fk
8 Folding frequency aka Nyquist frequency: Expect f fold = 1 2δt Rewrite the transform as F k = 1 f sin Nπ( k ) f fold 2 N N f sin π( k ) f fold 2 N Let f = Mf fold + δf >0 (consider positive frequencies for now) or δf = fmodf fold. The location of the maximum of F k is determined by the denominator, so look at ( f D = sin π k ) [ = sin M π ( δf 2f fold N 2 + π = cos M π 2 sin π ( δf 2f fold k N )] k 2f fold N ) + sin M π ( δf 2 cos π k 2f fold N ) (24) case a: M =even D = π ( δf } cos {{ M} 2 sin π k ) 2f fold N =±1 (25) D = ( δf sin π k ) 2f fold N (26) Now D =0 k = N δf 2f fold Note that δf <f fold by definition, so k< N 2 for M =even
9 case b: M =odd Now or D = sin Mπ ( δf 2 cos π k ) 2f fold N ( ) δf D = cos π 2f fold k N (27) ( δf + D =0 if π k ) = ± π 2f fold N 2 k N = 1 2 + δf 2f fold (28) k = n 2 + δf N f fold 2 (29) use + sign: k = N 2 ( 1+ δf f fold ) As a function of f/f fold = M + δf/f fold Thus we could redraw this picture as
10 Net conclusion: If a sinusoid with f>f max is undersampled then it is aliased to a frequency in the range [ f fold,f fold ]. Because of the multivalued relationship of f k, a priori one does not know the true frequency of the sampled waveform. It is known only modf fold. Prognosis: Avoid undersampling signals if one is interested in the shape of the transform! Can apply an anti-aliasing (low pass) filter to the signal before sampling. Anti-aliasing Filters: x(t) H(f) h(t) }{{} sampling signal n δ(t nδt) x s (nδt) Low pass filter A LPF always smooths the signal, decreasing its variance.
11 Where do negative frequencies go in k space? Let f f with f <f fold (no aliasing). Then the denominator of F k is ( f sin π k ) 2f fold N ( f = sin π k ) 2f fold N ( f = sin π + k ) 2f fold N (30) k N = f 2f fold (31) k = N f 2f fold (32) but k is periodic in n, so let k k + N ( k = N 1 f ) 2f fold The more negative f is, the smaller k is: Negative frequencies go into the top half of [0, N 1]. Positive times go into the lower half of [0, N 1]
12 Spectral Leakage The response of the DFT to a pure sinusoid is F k = 1 sin Nπ(fδt k ) N N sin π(fδt k ) N I. Suppose fδt = integer/n = k 0 /N,then F k δ k,k0 II. Generally, of course we will not be so lucky and there will be non-zero values in adjacent frequency bins This is called leakage because energy at the true frequency leaks into other frequency channels.
13 Aliasing and Parseval s Theorem (DFT approach) Using the DFT pair f n F n and the fact that Parseval s theorem holds for the DFT N 1 f n 2 = n=0 = N 1 n=0 N 1 n=0 = k N 1 k=0 k F k e 2πin k N 2 (33) k k Fk F k Fk F k e 2πin (k k ) N (34) N 1 n=0 e 2πin (k k ) N } {{ } N δ kk (35) = N k F k 2 (36) An improperly sampled signal [in the sampling theorem sense] has a distorted DFT but all the power in the time domain signal is represented in the frequency domain.
14 Summary and Comments on Aliasing of Sampled Signals Undersampling causes distortion of the DFT but in such a way as to conserve energy [cf. Parseval s theorem]. In practice, aliasing always occurs because we analyze signals of finite duration t. But finite duration corresponds to sharp cutoffs in the t-domain function (by the uncertainty principle) that the f-domain function is very broad. But aliasing can be minimized by applying an appropriate low pass (anti-aliasing) filter so that the sampling theorem is explicitly satisfied. We found for bandlimited functions that (for bandwidth f) f(t) = m= ( m f 2 f ) sin 2π f(t m 2 f ) 2π f(t m 2 f ) for Nyquist sampling while for arbitrary sampling at intervals δt 1 2 f f(t) =2δt f m= f(mδt) sin 2π f(t mδt). 2π f(t mδt) that Aliasing is just the familiar stroboscopic effect wherein positive frequency corresponds to one sense of rotation, negative frequencies to the opposite sense. For obvious reasons, the frequency f max = 1 2δt is often called the folding frequency. By duality, we can also consider a time limited function. If a function g(t) is non-zero only in the interval t T then sampling of its F.T. G(f) at intervals of 1 is sufficient to reconstruct 2T G(f): Thus, G(f) = k ( k ) sin 2πT (f k G ) 2T 2T 2πT (f k ) 2T We can consider a function that is both time limited and frequency limited into intervals T and f. To determine the time domain function we need to sample at intervals δt = 1 2 f.
15 Thus the number of samples needed is N = T =2T f. δt Note that a bandlimited function cannot be strictly time limited because bandlmited G(f) = G e (f) Π( f ) IFT = g sin π ft f e(t), where the subscript e denotes that G π ft e is a function that is arbitrarily extended in frequency, with g e its time-domain counterpart.
Symmetry Properties of DFT DFT: x n = e 2πink/N Xk k=0 X k = 1 N N 1 n=0 e 2πink/N x n periodic in N in both domains discrete functions in time and frequency δt and δf T = time series length = Nδt δf = 1/T Nyquist frequency = Nδf/2 = 1/2δt
Symmetry Properties of DFT Hermitian Show by substituting N-k into DFT If x n is real then x n X N k x n X N k X N k = Xk The symmetry properties tell us how to fill an array with data to achieve specific results What are the symmetry properties of the 2D DFT? X kl = 1 NM n x nm e 2πi(nk+ml)/N M m
How do we fill an array to get a real signal in the other domain? XN X0 X 1 X N 1 X 2 X N 2 0 1 2 3 N/2-1 N/2 N/2+1 N-2 N-1 N unique values k = 0, 1,, N-1 Need to know N/2-1+2 = N/2+1 unique values
Gaussian Example How do we insert a Gaussian function into an array in order to get a real DFT? Recall that a Gaussian function centered on t=0 has a real FT: e πt2 e πf 2 What is the equivalent placement for the DFT? Answer: see previous page!
Efficacy of FFT
Cyclical vs Non-cyclical Convolution & Correlation
Noncyclical Procedures