1. Fourier Methods In many diverse ields physical data is collected or analysed as Fourier components. In this section we briely discuss the mathematics o Fourier series and Fourier transorms. 1. Fourier Series (x) Any well-behaved unction can be expanded in terms o an ininite sum o sines and cosines. The expansion takes the orm: ( x) 1 a + a cosnx+ b sin nx n n 1 n 1 n Joseph Fourier (1768-183) SUPA Advanced Data Analysis Course, 18
The Fourier coeicients are given by the ormulae: 1 a ( x) dx a n 1 ( x) cosnxdx b n 1 ( x) sin nxdx These ormulae ollow rom the orthogonality properties o sin and cos: sin mxsin nxdx δ mn cosmxcosnxdx δ mn sin mxcosnxdx SUPA Advanced Data Analysis Course, 18
Some examples rom Mathworld, approximating unctions with a inite number o Fourier series terms
The Fourier series can also be written in complex orm: ( x) n A n e inx where 1 An ( x) e inx dx and recall that e inx e inx cos nx+ isin nx cosnx isin nx Fourier's Theorem is not only one o the most beautiul results o modern analysis, but it is said to urnish an indispensable instrument in the treatment o nearly every recondite question in modern physics
Fourier Transorm: Basic Deinition The Fourier transorm can be thought o simply as extending the idea o a Fourier series rom an ininite sum over discrete, integer Fourier modes to an ininite integral over continuous Fourier modes. Consider, or example, a physical process that is varying in the time domain, i.e. it is described by some unction o time. Alternatively we can describe the physical process in the requency domain by deining the Fourier Transorm unction. h (t) H( ) H( It is useul to think o and as two dierent representations o the same unction; the inormation they convey about the underlying physical process should be equivalent. ) SUPA Advanced Data Analysis Course, 18
We deine the Fourier transorm as H( ) it h( t) e dt and the corresponding inverse Fourier transorm as h( t) t H( ) e i d I time is measured in seconds then requency is measured in cycles per second, or Hertz. SUPA Advanced Data Analysis Course, 18
In many physical applications it is common to deine the requency domain behaviour o the unction in terms o angular requency This changes the previous relations accordingly: ω H( ω) iωt h( t) e dt h( t) 1 ω i t H( ω) e dω Thus the symmetry o the previous expressions is broken. SUPA Advanced Data Analysis Course, 18
Fourier Transorm: Further properties The FT is a linear operation: (1) the FT o the sum o two unctions is equal to the sum o their FTs () the FT o a constant times a unction is equal to the constant times the FT o the unction. I the time domain unction is a real unction, then its FT is complex. However, more generally we can consider the case where is also a complex unction i.e. we can write h( t) h ( t) ih ( t) R + I Real part Imaginary part may also possess certain symmetries: even unction odd unction h( t) h( t) h( t) h( t)
The ollowing properties then hold: See umerical Recipes, Section 1. ote that in the above table a star (*) denotes the complex conjugate, i.e. i z x + i y then z* x i y SUPA Advanced Data Analysis Course, 18
For convenience we will denote the FT pair by It is then straightorward to show that h( t) H( ) 1 h( at) H( a) time scaling a 1 b h( t b) H( b ) requency scaling h( t t ) H( ) e i t time shiting ( ) i t h t e H( ) requency scaling SUPA Advanced Data Analysis Course, 18
Suppose we have two unctions and Their convolution is deined as g (t) ( h) ( t) g g( s) h( t s) ds We can prove the Convolution Theorem ( g h) ( t) G( ) H( ) i.e. the FT o the convolution o the two unctions is equal to the product o their individual FTs. SUPA Advanced Data Analysis Course, 18
Example application: the observed spectrum o light rom a galaxy is a convolution o stellar spectral templates smeared out by the line o sight velocity distribution and the G( u) F d ( vlos) S( u vlos) vlos Galaxy spectrum LOSVD stellar spectra This is a particular type o integral equation: g( y) ( x) s( y x)dx Data unction Source unction Kernel unction
So by the convolution theorem: ~ G ( k) ~ F( k) ~ S( k) and F ~ ( ) ( 1 G k v F LOS ~ ~ ) S( k) Inverse Fourier transorm Hence, we can in principle invert the integral equation and ( ) v LOS reconstruct the LOSVD,. F In practice, this method is vulnerable to noise on the observed G (u) S(u) galaxy spectrum,, and uncertainties in the kernel. eed to ilter out high requency (k) noise SUPA Advanced Data Analysis Course, 18
Filter, denoting range o wavenumberswhich give reliable inversion Ratio o two small quantities: very noisy SUPA Advanced Data Analysis Course, 18
Suppose we have two unctions and Their convolution is deined as g (t) ( h) ( t) g g( s) h( t s) ds We can prove the Convolution Theorem ( g h) ( t) G( ) H( ) i.e. the FT o the convolution o the two unctions is equal to the product o their individual FTs. Known as the lag Also their correlation, which is also a unction o t, is deined as Corr( g, h) g( s+ t) h( s) ds SUPA Advanced Data Analysis Course, 18
We can prove the Correlation Theorem Corr( g, h) G( ) H ( ) i.e. the FT o the irst time domain unction, multiplied by the complex conjugate o the FT o the second time domain unction, is equal to the FT o their correlation. The correlation o a unction with itsel is called the auto-correlation In this case Corr( g, g) G( ) G( ) The unction is known as the power spectral density, or (more loosely) as the power spectrum. Hence, the power spectrum is equal to the Fourier Transorm o the auto-correlation unction or the time domain unction g(t) SUPA Advanced Data Analysis Course, 18
The power spectral density The power spectral density is analogous to the pd we deined in previous sections. In order to know how much power is contained in a given interval o requency, we need to integrate the power spectral density over that interval. The total power in a signal is the same, regardless o whether we measure it in the time domain or the requency domain: Total Power dt - - H() d Parseval s Theorem SUPA Advanced Data Analysis Course, 18
Oten we will want to know how much power is contained in a requency interval without distinguishing between positive and negative values. In this case we deine the one-sided power spectral density: P h ( ) H( ) + H( ) < And Total Power P h ( ) d When is a real unction P h ( ) H( ) With the proper normalisation, the total power (i.e. the integrated area under the relevant curve) is the same regardless o whether we are working with the time domain signal, the power spectral density or the one-sided power spectral density.
From umerical Recipes, Chapter 1. Time domain One-sided PSD Two-sided PSD
Examples (1) h( t) const. Dirac Delta unction H( ) δd() H( ) t () i t h( t) e H( ) δd( ) H( ) t Imaginary part Real part
H( ) sinc( ) (3) 1 h( t) or 1 t otherwise 1 H( ) Imaginary part t y sincx The sinc unction occurs requently in many areas o physics y1 sinc x sin x x 1 st zeroat x 1 st zeroat x x x ±m The unction has a maximum at and the zeros occur at or positive integer m x
(4) h( t) ( t) H( ) sinc ( ) H( ) 1 1-1/ 1/ t Re [ H( ) ] a a + 4 (5) h( t) e at Im [ H( ) ] a ( a + 4 ) H( ) Real part t Imaginary part
(6) h( t) exp( a t ) ( / ) H( ) exp a t i.e. the FT o a Gaussian unction in the time domain is also a Gaussian in the requency domain. SUPA Advanced Data Analysis Course, 18
Question : I the variance o a Gaussian is doubled in the time domain A the variance o its Fourier transorm will be doubled in the requency domain B the variance o its Fourier transorm will be halved in the requency domain C the standard deviation o its Fourier transorm will be doubled in the requency domain D the standard deviation o its Fourier transorm will be halved in the requency domain
(6) h( t) exp( a t ) ( / ) H( ) exp a t i.e. the FT o a Gaussian unction in the time domain is also a Gaussian in the requency domain. The broader the Gaussian is in the time domain, then the narrower the Gaussian FT in the requency domain, and vice versa. SUPA Advanced Data Analysis Course, 18
Discrete Fourier Transorms Although we have discussed FTs so ar in the context o a continuous, analytic unction,, in many practical situations we must work instead with observational data which are sampled at a discrete set o times. Suppose that we sample in total times at evenly spaced time intervals, i.e. (or even) +1 h k h ( t k ) where t k k, k,...,,..., [ I is non-zero over only a inite interval o time, then we +1 suppose that the sampled points contain this interval. Or i has an ininite range, then we at least suppose that the sampled points cover a suicient range to be representative o the behaviour o ]. SUPA Advanced Data Analysis Course, 18
We thereore approximate the FT as H( ) k t h( t) e i dt h k k e i t k + 1 Since we are sampling at discrete timesteps, in view o the symmetry o the FT and inverse FT it makes sense also to compute +1 only at a set o discrete requencies: H( ) n n, n,...,,..., c 1 (The requency is known as the yquist (critical) requency and it is a very important value. We discuss its signiicance later). SUPA Advanced Data Analysis Course, 18
Then ote that Hence, there are only independent values. Also, note that So we can re-deine the Discrete FT as: n k ik n k n H e h H 1 / ) ( in in e e k in ikn in ikn e e e )/ ( / / + + / ) ( k k ikn k k k t i k n e h e h H k n Discrete Fourier Transorm o the k h SUPA Advanced Data Analysis Course, 18
The discrete inverse FT, which recovers the set o rom the set o H n 's is h k 1 1 k H n e ikn/ h k 's Parseval s theorem or discrete FTs takes the orm 1 k h k 1 1 n H n There are also discrete analogues to the convolution and correlation theorems. SUPA Advanced Data Analysis Course, 18
Fast Fourier Transorms Consider again the ormula or the discrete FT. We can write it as H n 1 k e ikn/ h k 1 k W nk h k This is a matrix equation: we compute the ( 1) vector o H n ' s nk by multiplying the ( ) matrix W by the ( 1) vector o. [ ] h k 's In general, this requires o order multiplications (and the may be complex numbers). h k ' s 8 16 1 1 e.g. suppose. Even i a computer can perorm (say) 1 billion multiplications per second, it would still require more than 115 days to calculate the FT.
Fortunately, there is a way around this problem. Suppose (as we assumed beore) is an even number. Then we can write where So we have turned an FT with points into the weighted sum o two FTs with points. This would reduce our computing time by a actor o two. + + + 1 / 1 / 1) ( 1 / / ) ( 1 / j j n j i j j n j i k k ikn n h e h e h e H Even values o k Odd values o k + + 1 1 / 1 / M j j M jn i n M j j M jn i h e W h e / M /
Why stop there, however?... M I is also even, we can repeat the process and re-write the FTs o M M / length as the weighted sum o two FTs o length. I is a power o two (e.g. 14, 48, 148576 etc) then we can repeat iteratively the process o splitting each longer FT into two FTs hal as long. The inal step in this iteration consists o computing FTs o length unity: H ik e hk h k i.e. the FT o each discretely sampled data value is just the data value itsel. SUPA Advanced Data Analysis Course, 18
This iterative process converts multiplications into O ( log ) operations. O( ) This notation means o the order o 1 9.7 1 16 So our operations are reduced to about operations. The Fast Fourier Transorm (FFT) has revolutionised our ability to tackle problems in Fourier analysis on a desktop PC which would otherwise be impractical, even on the largest supercomputers. SUPA Advanced Data Analysis Course, 18
Data Acquisition Earlier we approximated the continuous unction and its FT H( ) +1 by a inite set o discretely sampled values. How good is this approximation? The answer depends on the orm o H( ) and. In this short section we will consider: 1. under what conditions we can reconstruct and exactly rom a set o discretely sampled points? h (t) H( ). what is the minimum sampling rate (or density, i is a spatially varying unction) required to achieve this exact reconstruction? h h (t) H( ) 3. what is the eect on our reconstructed and i our data acquisition does not achieve this minimum sampling rate? SUPA Advanced Data Analysis Course, 18
The yquist Shannon Sampling Theorem Suppose the unction is bandwidth limited. This means that the FT o is non-zero over a inite range o requencies. i.e. there exists a critical requency c such that H( ) or all c The yquist Shannon Sampling Theorem (SST) is a very important result rom inormation theory. It concerns the representation o by a set o discretely sampled values h k h ( t k ) where t k k, k...,, 1,,1,,... SUPA Advanced Data Analysis Course, 18
The SST states that, provided the sampling interval satisies 1 c or less then we can exactly reconstruct the unction rom the discrete { } h k samples. It can be shown that h( t) sin [ c( t k ) ] ( t k ) + hk k - c is also known as the yquist requency and is known as the yquist rate. 1 c SUPA Advanced Data Analysis Course, 18
We can re-write this as h( t) sin [ ( t k ) ] [( t k ) ] + hk k - So the unction is the sum o the sampled values, weighted by the normalised sinc unction, scaled so that its zeroes lie at those sampled values. h k h (t) { } ormalised sinc unction sinc(x) (compare with previous section) x ( t ) k t k h ( t) since sinc ( ) 1 ote that when then h k Sampled values SUPA Advanced Data Analysis Course, 18
The SST is a very powerul result. We can think o the interpolating sinc unctions, centred on each sampled point, as illing in the gaps in our data. The remarkable act is that they do this job perectly, provided is bandwidth limited. i.e. the h (t) H( ) discrete sampling incurs no loss o inormation about and. Suppose, or example, that. Then we need only sample twice every period in order to be able to reconstruct the entire unction exactly. h( t) sin ( t) c (ote that ormally we do need to sample an ininite number o { h } { } discretely spaced values,. I we only sample the over a k inite time interval, then our interpolated will be approximate). h k SUPA Advanced Data Analysis Course, 18
y sin ( t) c x c t 1 c Sampling at (ininitely many o) the red points is suicient to reconstruct the unction or all values o t, with no loss o inormation. SUPA Advanced Data Analysis Course, 18
Aliasing There is a downside, however. I is not bandwidth limited (or, equivalently, i we don t sample 1 < c requently enough i.e. i the sampling rate ) then our h (t) H( ) reconstruction o and is badly aected by aliasing. This means that all o the power spectral density which lies outside the < < H ( ) range is spuriously moved inside that range, so that the FT o will be computed incorrectly rom the discretely sampled data. c c Any requency component outside the range is alsely translated ( aliased ) into that range. (, ) c c SUPA Advanced Data Analysis Course, 18
Consider as shown. Suppose is zero outside the range T. This means that extends to ±. H( ) sampled at regular intervals The contribution to the true FT rom outside the ( ) range 1,1 gets aliased into this range, appearing as a mirror image. Thus, at our computed value o is equal to twice the true value. ± 1 H( ) From umerical Recipes, Chapter 1.1
How do we combat aliasing? c > o Enorce some chosen e.g. by iltering to remove the high requency components. (Also known as anti-aliasing) 1 c o Sample at a high enough rate so that - i.e. at least two samples per cycle o the highest requency present c 1 To check or / eliminate aliasing without pre-iltering: o Given a sampling interval, compute lim 1 o Check i discrete FT o is approaching zero as lim o I not, then requencies outside the range 1,1 are probably being olded back into this range. o Try increasing the sampling rate, and repeat ( )
And inally. Mock Data Challenge Part 1 1 (x,y) data pairs, generated rom an unknown model plus Gaussian noise. y Data posted on my.supa and on Moodle (or MSc course) Three-stage challenge: x 1. Fit a linear model to these data, using ordinary least squares;. Compute Bayesian credible regions or the model parameters, using a bivariate normal model or the likelihood unction; 3. Write an MCMC code to sample rom the posterior pdo the model parameters, and compare their sample estimates with the LS its;
And inally. Mock Data Challenge Part 1 (x,y) newdata pairs, generated rom an unknown model plus Gaussian noise. y Data posted on my.supa and on Moodle (or MSc course) 1. Fit a quadraticmodel to these data, using e.g. an MCMC code to sample rom the posterior pd o the model parameters,presenting plots o the marginal posterior or each pair o parameters; x