1.4 Fourier Analysis. Random Data 17

Size: px

Start display at page:

Download "1.4 Fourier Analysis. Random Data 17"

Nickolas Morrison
5 years ago
Views:

1 Random Data Fourier Analysis In the physical world we sample (and acquire) signals in either the temporal domain or the spatial domain. However, we are frequently interested in thefrequencycontentof theses signals. For example consider a tidal gauge sitting in coastalwaterswhichwe sample every 15 minutes or so. We expect that the mean water surface will rise and fall twice a day (semi-diurnal tide) with a period of just over halfaday. Whatarethe driving forces behind the tides? The primary forces are the gravitational forces of the moon and the sun. However, the moon and the sun have slightly different orbital periods (the moon s is 5 minutes or so longer). Furthermore, the sun is much more massive but much further away and it turns out that the moon accounts for something like 7% of tides. How can we extract information on the frequency content of our signal? One method would be to look at the autocorrelation as this will identifydominantperiodic length scales as the signal will correlate with itself strongly over its natural periods. The dominant frequencies are simply the inverse of these periodic time scales. However, asomewhatmoredirectmethodistoconvertoursignaltothefrequency or spectral domain. Infrequencyspacewewillgetlargeamplitudesatthe dominant frequencies in our signal. Consider our sine wave example back in Section If we represent this signal, a pure sine wave, in the frequency domain we get a single non-zero amplitude at frequency f =1/T while all of the other frequencies have zero amplitude. In fact, if we have a complex periodic signal we will get non-zero amplitudes at the frequency components in the signal where the ratios of these amplitudes give us quantitative information about the relative contribution of the various frequencies to the variance of the signal. O.K, but how do we transform our signal into the frequency domain? Fourier transform. Through the The Fourier transform can be written G(f) = g(x)e i2πxf dx (1.4) where i = 1, f is the frequency, andg(f) isthefourier transform of g(x). The c 215 Edwin A. Cowen

2 18 Random Data inverse Fourier transform is defined as g(x) = G(f)e i2πxf df (1.41) Note the only change is that the argument of the exponent term is of opposite sign. Now, as we will be working with discrete data we will work with the discrete form of the Fourier transform (e.g., we will operate on discrete data notcontinuousdata). The discrete form of the Fourier transform, known as the DFT, is given by G k = N 1 n= g n e i2π(n/n)k k =, 1, 2,...,N 1 (1.42) where k is the index of the sample record in the frequency domain and n is the index of the sample record in the spatial/temporal domain. A few remarks on the DFT: As with the discrete autocorrelation function, the DFT is cyclic - e.g., g n is assumed periodic. G()= N 1 n= g n The resolution in the frequency domain is set by the length of the sample record. I.e., if we are working in the time domain with a signal sampled atf s =1Hz(1 samples per second) and the signal is 1 samples long (T =1seconds)thenour t between samples is.1 s. Now, our f (frequency resolution) is f =1/T and our frequency domain is f k = k/t. From the above it should be clear that the maximum frequency inthefrequency domain is essentially the sample rate, f s =1/ t (more precisely it is f s f/2if we take the discrete frequency of each point to be the average frequency across the interval, hence the lowest resolved frequency is f/2). For real data (which is what we will always be interested in) the DFT is symmetric -hencethemaximumfrequencyatwhichwehaveanyindependentinformationis at f N/2 = f s /2 thisisknownasthenyquist frequency. c 215 Edwin A. Cowen

3 Random Data 19 What happens if the frequency content of the sampled signal is higherthanthe Nyquist frequency based on the sample rate? (e.g., the signal containsfrequency components up to 6 Hz but we only sample at 4 Hz) aliasing! We will discuss this further shortly. The cost of calculating the DFT is O(N 2 )(read ordern 2 ). The exact definition of the DFT and FFT (discussed below) can vary slightly the sign of the terms of the exponent may vary, the 2π term may vary, and the location of the 1/N term (in our case in the inverse DFT) may vary. It is critically important that you verify how the DFT is being implemented on any software packages you choose to use. I have presented the form that is consistent with Matlab (note however in Matlab the indices run from k= 1 to N as Matlab does not allow indices but it is generally more common to think of the lowest index as soihaveretainedthatnotationhere)butihavenotverifiedthis with any other software (e.g., Excel, MathCAD, Python) - if you are using canned software other than Matlab you should! The inverse DFT is g n = 1 N N 1 k= G k e i2π(k/n)n n =, 1, 2,...,N 1 (1.43) The penultimate item above is significant - the cost of a DFT is not insignificant, particularly for large N. Fortunately for us there are algorithms to speed up the calculation. The best known of these is the Fast Fourier Transform (FFT), first published in 1965 (Cooley, J.W.; Tukey, J.W.). The algorithm takes advantage of factorizations in the DFT when N is a power of 2 (e.g., log 2 N is an integer). It is not important that we understand the algorithm precisely (See Bracewell, 1986 for avarietyofexplanations) other than to say that it is an exact calculation of the DFT we have defined. Importantly the cost of the calculation is reduced to O(N log N) -afactoroflogn/n less costly than the DFT. Clearly for large N this savings can become significant quickly! For example with N =124,stillrelativelysmall,thecostoftheFFTislessthan1%ofthecostof the DFT. c 215 Edwin A. Cowen

4 2 Random Data You should also be aware that the restriction to multiples of 2isreallynolongerrelevant. AvarietyofalgorithmsforacceleratedDFTcalculationforvectors of almost arbitrary lengths now exist. In particular a group known as FFTW.ORG ( has assembled an incredibly fast and broad set of these tools such that the DFT of vectors of essentially arbitrary length can be calculated in an accelerated manner. The Matlab FFT functions take advantage of the FFTW library so if you are using Matlab don t worry about your vector lengths! Note due to the historical importance of the FFT algorithm and the fact that computers were once a lot slower a lot of legacy algorithms require vector lengths to be powers of 2. This is easy to upgrade if you are working with such packages. One final note, however, that for multi-dimensional data sets on parallel computer architecture there may still be advantages to working with FFTs that have lengths that are powers of Aliasing To get a feel for the spectral domain lets look at some simple sine waves. If we sample three separate sine waves at 1 Hz (Note: the actual frequencies differ slightly from the integer values shown below to ensure zero side-lobe energy, yet another form of aliasing we will talk about later. The actual frequencies are exact multiples of the reciprocal of the record length, which is 124 samples at 1 Hz or 12.4 s, so multiples of 1/12.4 Hz): a =1sin(2.π2.t); b =2sin(6.π2.t); c =4sin(23.π2.t) for 124 samples (12.4 seconds) and we construct the linear superposition of these three as d = a + b + c we can then look at a plot of the magnitude of their Fourier transforms ( G(k) ) versusf yielding the following: c 215 Edwin A. Cowen

5 Random Data t (s) f (Hz) c 215 Edwin A. Cowen

6 22 Random Data If we sample the same signals for only 128 samples (12.8 seconds) we get t (s) f (Hz) c 215 Edwin A. Cowen

7 Random Data 23 Things to notice in general about the spectra: The spectra of both the 12.4 second and 12.8 second sets (sampled at the same rate) have the same maximum frequency, 1 Hz, the sample frequency. The information from half the sampling rate, known as the Nyquist frequency, on is redundant and is the mirror image of the first half of the frequency spectrum. The resolution of the 12.4 second spectrum is finer ( f =1/(12.4 s)=.98 Hz) than that of the 12.8 second spectrum ( f =1/(128 s) =.781 Hz). The amplitude of the spectral peaks vary in a linear manner with the amplitude of the original signal (note this is only true when sampled such that no energy leaks into the side lobes due to the frequency of the signal (f i )beinganon-integer multiple of the frequency resolution (e.g., f i / f integer),however,ingeneral the area under a peak (which may cross several frequencies) is proportionaltothe amplitude of the temporal component. Now, looking at the 12.4 second spectra, A is N a where N is the number of samples (124). 2 For A vs f the only non-zero components are at 2., the frequency of a(t), and 8. Hz which we can think of as 2. Hz. B is N 2 b. For B vs f the only non-zero components are at 4. Hz and 6. Hz (which we think of as 4. Hz)butwehavenotemporalsignalwiththisfrequency,what happened? Aliasing! A 6 Hz signal sampled at 1 Hz looks like a 4Hzsignal. In fact, if a signal being sampled has frequency components greater than the Nyquist frequency (f Nyq )youwillgetaliasing.ifthesignalbeingsampledhasafrequency component (f i )suchthatf Nyq <f i <f s the peak will simply appear at f i with its mirror appearing at f s f i. c 215 Edwin A. Cowen

8 24 Random Data C is N 2 c. For C vs f the only non-zero components are at 3. Hz and 7. Hz (which we think of as 3. Hz).Thisisaliasingagain-butabitmorecomplexversionthan we just encountered. Our original signal (c(t)) has a single frequency component of 23. Hz. Now, since f i >f s the peak folds back into the spectrum. Thus had f i =11Hzwewouldgetapeakat9Hzandhadf i =12Hzwewouldgetapeak at 8 Hz, etc. This works all the way back to Hz which is equivalent to an aliased 2 Hz signal and then the signal folds again meaning a 21 Hz signal would land at 1Hzandour23Hzsignallandsat3Hz! Thus looking at frequency spectrum like that shown for D can be challenging to interpret unless you know that you have sampled such that the signal contains no frequencies greater than the Nyquist frequency. Hence a general rule for spectral analysis is either set your sample frequency at or greater than twice the maximum frequency in the signal or low-pass filter this signal at the Nyquist frequency. If somebody just handed us these spectra, unless we were also given information about the original signals, we would have no choice but to assume that the spectra are telling us that there is energy at frequencies of 2, 3, and 4Hz. Note aliasing is not limited to the frequency domain. As we have indicated there are two methods for determining the autocorrelation function - circular and non-circular. One approach to the non-circular method assumes that all non-specified values are zero (others use data that may exist outside the bounds of the data record length of the calculation, as shown in section 1.3.4). Thus instead of wrapping the signal back onto itself you simply assume the value is zero. If we take our signals a and b (Eqs 1.2 and 1.29) the non-circular autocorrelation functions are: R aa = [ ] R bb = [ ] (1.44) where I have only showed the positive lags (the system is symmetric about the first component). If we compare the circular and non-circular results you see that the circular c 215 Edwin A. Cowen

9 Random Data 25 results are actually an aliased version of the non-circular results where the terms other than R ξξ () in the circular autocorrelation are the sum of two terms in thenon-circular correlation. For example R aa (2) = in the non-circular calculation while R aa (2) = 1.5 inthecircularcalculationwhichisthesumofr aa (2) and R aa (N 2) in the noncircular calculation. In Matlab you can calculate the circular correlations using cxcorr, whichfor1-dcorrela- tions, i.e., vectors, is an order N 2 calculation cost as you must run through an inner and an outer loop of length N to do all the calculations. As we will see in the next section there is a faster way to do the circular correlation. Matlab calculates the non-circular correlation using xcorr, whichhasfouroptionsthatareworthdiscussingastheyshed light on the non-circular correlation. The options each involve what you divide (normalize) each of the calculated R uv (m) inequation1.19with. Aswrittenthenormalizationis N, thiscorrespondstotheoption biased. Thedefaultoptionis none meaning you do not normalize the coefficients - each is just the sum of the N terms at the indicated lag. The most interesting option is unbiased so let s talk about this a bit. The choice of N for the normalization comes from the fact that we have N values being summed in the calculation of each correlation at a specific lag. However, in the noncircular calculation we only truly have N values at a lag of. At a lag of 1 we will have one of the N required products in the summation being simply a multiplication by zero and hence only N 1actualnon-zerocomponentsmakingupthesum.Similarlywith a lag of 2 there will be two multiplications by zero and hence only N 2actualnon-zero components making up the sum; and so on and so on. Thus by normalizing by N we are including values in the summation that we know will be zero and thisunder-biases the correlation value at non-zero lags. This downward bias gets more significant for larger and larger lags. How do we solve it? By normalizing by the actual number of non-zero components in the sum of each autocorrelation at a specific lag, i.e., N 1atalagof1, N 2atalagof2,andsoon. I.e.,theunbiasednon-circularcorrelation is calculated as: c 215 Edwin A. Cowen

10 26 Random Data N 1 1 R uv (m) = u n v n+m (1.45) N m For the positive lags only. for positive and negative lags this becomes: R uv (m) = 1 N m N 1 (N 1) u n v n+m (1.46) The fourth option for xcorr is coeff which is simply the biased non-circular correlation coefficient, that is, none or biased normalized by their zero-lag values Auto and Cross-Correlation via the FFT As mentioned earlier one of the powers of Fourier analysis is that we can compute correlations more inexpensively in the spectral domain then in the temporal or spatial domain. First for simplicity let us define some nomenclature for correlations. We can write autocorrelation in a short-hand form as: 1 R gg (τ) = lim T T T and similarly we can write the cross-correlation as: 1 R gh (τ) = lim T T T g(t)g(t + τ) dt = g g (1.47) g(t)h(t + τ) dt = g h (1.48) Now we can write down the auto and cross-correlation theorems, respectively, as F [g g] = GG = G(s) 2 (1.49) F [g h] = GH (1.5) where F []means Fouriertransformof and() means complex conjugate. Thus we can find the auto and cross-correlations simply by Fourier transforming the sample records, taking a complex conjugate, performing a multiplication in the spectral domain, and inverse Fourier transforming back. In other words R gg = g g= F 1 [ G(s) 2] (1.51) R gh = g h= F 1 [GH ] (1.52) c 215 Edwin A. Cowen

11 Random Data 27 where F 1 []means inversefouriertransformof. Exercise for the student: choose your favorite analysis package, determine how it is performing FFT s and use the package to determine R aa, R bb,andr ab using the FFT. Note, you can determine the non-circular correlation functions in the spectral domain by padding the original sample records with zeros. How many zeros? If you look back at our discussion of aliasing in the previous section clearly the minimum (and hence most efficient) is N -thelengthoftherecord.henceyoumustdoubletherecordlength. Try this out too! Power Spectrum Power spectrum is a common term used by experimentalists which is more specifically known as a power spectral density (PSD) function which intern statisticians will refer to as either autospectral density functions or cross-spectral density functions. What does this all mean? Let s begin with the definitions S uu (s) = S vv (s) = S uv (s) = R uu (τ)e i2πτs dτ = F [R uu ]= U(s) 2 (1.53) R vv (τ)e i2πτs dτ = F [R vv ]= V (s) 2 (1.54) R uv (τ)e i2πτs dτ = F [R uv ]= U(s)V (s) (1.55) and of course the inverse relationships exist, namely R uu (τ) = R vv (τ) = R uv (τ) = S uu (s)e i2πτs ds = F 1 [S uu ] (1.56) S vv (s)e i2πτs ds = F 1 [S vv ] (1.57) S uv (s)e i2πτs ds = F 1 [S uv ] (1.58) Where S ξξ is our nomenclature for spectral density functions. Forrealfunctions(andour sampled signals will be real data) it can be shown that S uu and S vv are real-valued even c 215 Edwin A. Cowen

12 28 Random Data functions of s while S uv is a complex valued function of s. Sincetheautospectraldensity functions are real valued they are frequently written as one-sided spectra, i.e. G uu (s) = 2 G vv (s) = 2 R uu (τ)e i2πτs dτ (1.59) R vv (τ)e i2πτs dτ (1.6) where the factor of two accounts for the symmetric energy in the negative frequencies. Now, based on our definition of the Fourier transform it should be clear that R ξξ () = S ξξ (s) ds = G ξξ (s) = 1 T T ξ 2 (t) dt (1.61) or in other words the integral of the autospectral density function of a signal is the mean squared value of that signal. Further, if we remove the mean of thesignalfromthe original signal (e.g., ζ = ξ ξ where ξ signifies the mean value of ξ) wefindthat R ζζ () = S ζζ (s) ds = G ζζ (s) ds = 1 T T ζ 2 (t) dt = σ 2 ζ (1.62) or in other words the integral of the autospectral density function of a signal is the variance of that signal. O.K., now that we have the definitions, what do spectral density functions tell us? They tell us the portion of the total signal variance that is accounted for at a specific frequency. If we translate our continuous expressions into the discrete domainwefindthatthe spectral density functions tell us the portion of the total signal variance that is accounted for in a specific frequency band, f. If we consider our three sine waves in Section sampled over 12.4 seconds we can determine the autospectral density functions shown at the top of the next page. How did I calculate these in the discrete domain? It is left to the student to verify that S aa = 1 f s N AA = 1 f s N A2 (1.63) f N 1 k= S aa (k) =σ 2 a (1.64) c 215 Edwin A. Cowen

13 Random Data 29 5 S AA 2 S BB 1 5 S CC 5 S DD f (Hz) Note, these are sine waves so the mean of the signals are zero hence the area under the autospectral density function is the variance and the mean squared value. Now we have a method for determining the frequency content of asignal. Realsignals will be considerably more noisy than the above. For example, if we consider just signal a with the addition of uniform noise (known as white noise) withameanamplitudeof 1 we get the signal and spectra shown on the next page. The final spectrum in the series of plots is the average of the spectrum of 16 sample records. Since the noise is random it has infinite frequency content. Thus by averaging the spectrum of sample records we smooth the noise spectrum towards its true spectral shape (a flat line for white noise) which allows our signal of interest to develop an improved signal-to-noise ratio. Ultimately this is how we determine spectra in real signals where noise and/or smooth spectra are of interest. c 215 Edwin A. Cowen

14 3 Random Data a(t) + noise S EE t (s) <S EE > f (Hz) Summarizing our approach to determining spectra (given an ergodic process): Sample data at a rate twice the rate of the maximum relevant frequency (Nyquist criteria). Sample long enough such that the record can be broken up into N data sets to allow individual spectra to be averaged. Keep in mind that the lengthofthese individual records must be long enough to achieve the desired frequencyresolution. Determine the spectrum of each of the N records and average to get a final estimate of S ξξ. This gives us what we need for now to determine spectra. There remains one additional consideration relevant to our work when determining spectra that we will look at later, side-lobe leakage due to the finite length of the sample records. c 215 Edwin A. Cowen

Before we consider two canonical turbulent flows we need a general description of turbulence.

Chapter 2 Canonical Turbulent Flows Before we consider two canonical turbulent flows we need a general description of turbulence. 2.1 A Brief Introduction to Turbulence One way of looking at turbulent