A6523 Signal Modeling, Statistical Inference and Data Mining in Astrophysics Spring 2011

Transcription

1 A6523 Signal Modeling, Statistical Inference and Data Mining in Astrophysics Spring 2011 Reading: Chapter 10 = linear LSQ with Gaussian errors Chapter 11 = Nonlinear fitting Chapter 12 = Markov Chain Monte Carlo Lectures 1-24 are on the web page Lecture 24 Matched Filtering Localization Web page: look for PS 4 + article on genetic algorithms

2 Notions of Best Basis Orthogonal basis functions: allows new terms to be added to fitting function without altering the previous terms parameters (coefficients) have diagonal covariance matrix Compactification: in detection problems it is useful to transform the data to concentrate the signal variance into as few basis vectors as possible Example: DFT of sinusoidal signal = least squares fit; ideally the variance in a single sinusoid accounted for in the coefficient of a single basis vector (apart from binning and leakage effects). Underlying model (e.g. physics) may suggest a particular basis (e.g. monopole, dipole, quadrupole terms in a 3D angular distribution)

3 Notions of Best Basis II Coifman & Wickerhauser Entropy Based Algorithms for Best Basis Selection The Karhunen-Loéve basis is the minimum-entropy orthonormal basis for an ensemble of vectors. They develop a technique useful for a single vector that is based on minimum entropy. Minimum entropy is consistent with compactification.

4 Matched Filtering (Template Fitting) Premise: U(t) = at(t-t 0 ) + n(t) a = scale factor, t 0 = arrival time T(t) = template (assumed known) n(t) = additive noise (arbitrary statistics) Template fitting yields a, t 0 Optimal estimation: cross correlate U(t) with T(t) C UT (τ) = U(t) * T(t+τ) = ac TT (τ-t 0 ) + T(t +τ) * n(t) C UT (τ) maximizes at τ max =t 0 in the mean error in τ max due solely to n(t) Practicality: easier to find τ max in the Fourier domain (sampling issues) If shape of U(t) shape of T(t), there are additional errors in the TOA estimate

5 1 Matched Filtering Matched filtering is an optimal method for detecting a signal of known shape in the presence of additive noise. Consider the model where A is deterministic and known and n is additive WSS noise with arbitrary spectrum: x(t) =a 0 A(t)+n(t). We want a filter that whose output maximizes the signal-to-noise ratio of the output. Note matched filtering is different from Wiener filtering, which yields an estimate for a signal that has a minimum least-squares error from the true signal.

6 Matched filter: We want the linear filter h(t) thatmaximizesthesnroftheoutput and thus maximizes the detection probability. Usually we would convolve the filter with the data but it is convenient to define h so that we cross correlate it with the data instead. Also, assume that h(t) is aligned with the signal A(t) sothatweonlyneedconsiderthesnrof the output when h and A are aligned to give the maximum. Thus while we would generally consider the full correlation function, y(τ) =h(t) x(t) = dt A(t)h(t + τ) (1) (where * means correlation), we consider the special time τ = 0 where by construction we say that y is maximized. The mean and variance of y are y = a 0 dt A(t)h(t) σy 2 = dtdt n(t)n(t ) h(t)h(t )= Define the SNR as S = y = a 0 dt A(t)h(t). σ y σ y We want the filter function that maximizes S (or S 2 ): h S 2 δh(t) =0. 2 dtdt R n (t t )h(t)h(t ). This yields (where h means partial derivative with respect to the filter) 2σ 2 y y h y y 2 h σ 2 y =0.

7 3 Keeping only terms that are first order in δh(t): 2σy 2 a 0 dt A(t)δh(t) 2 y dtdt R n (t t )h(t )=0 or [ dt δh(t) σya 2 0 A(t) y ] dt R n (t t )h(t ) =0. If the solution h(t) yieldsamaximumsnrthentheintegrandfactor that multiplies δh(t) iszeroforanyδh(t), so dt R n (t t )h(t )= a 0σy 2 A(t). (2) y White Noise: This case, the simplest, has R n (t t )=σn 2δ(t t ) and gives dt h 2 (t) h(t) =A(t) dt A(t)h(t) for which h(t)=a(t)isasolution,asisanythingproportionaltoa(t). It is now obvious why this approach is called matched filtering.

8 Low-pass Noise: Non-white noise with a short correlation time (relative to the width of the signal) can be treated in the same way. Let R n (t t )=σ 2 nρ n (t t )whereρ n (0) = 1 and its width width of A. Returning to our original expression for S, wehave ( ) a0 dt A(t)h(t) S = σ [ n dtdt ρ n (t t )h(t)h(t ) ] 1/2 ( ) a0 dt A(t)h(t) [ σ n dτρn (τ) dt h 2 (t) ] 1/2 ( ) a0 dt A(t)h(t) [ Wn dt h2 (t) ], 1/2 σ n where a characteristic time scale W n = dτρ n (τ) hasbeendefined. We can now apply the Cauchy-Schwarz inequality for the numerator [ 2 dt A(t)h(t)] dt A 2 (t) dt h 2 (t) which means that S = ( a0 σ n ( a0 σ n ( a0 σ n ) dt A(t)h(t) [ Wn dt h2 (t)] ] 1/2 ) [ dt A 2 (t) dt h 2 (t) ] 1/2 [ Wn dt h2 (t) ] 1/2 ) [ dt A 2 (t) ] 1/2 Wn We have equality when h(t) = A(t), i.e. when the filter matches the signal. 4

9 Simplification and Interpretation For the matched filter case (equality) we can consider A(t) to be dimensionless with A(0) = 1. Then the integral dt A 2 (t) W A defines the characteristic time scale W A.Wecanalsodefinethesignal to noise of the time series as ( ) a0 SNR t and then rewrite the SNR of the correlation function as ( ) 1/2 WA S =SNR t. W n As we have seen in other contexts, the ratio W A /W n represents the number of independent noise fluctuations that have been averaged to get the correlation amplitude. We then see that the correlation function enhances the SNR in the time series by a factor N eff where N eff is the effective number of independent fluctuations. σ n 5

10 Arbitrary WSS Noise: Generally, the solution is gotten by Fourier transforming Equation 2. Let the noise spectrum be R n (τ) S n (f) anddefinetheftsofa and h as à and { h. Thenwehave } FT dt R n (t t )h(t ) = S n (f) h(f), σ 2 y = dtdt R n (t t )h(t)h(t ) = df S n (f) h(f) 2 6 and y = a 0 dt A(t)h(t) =a 0 df à (f) h(f). so the expression for h becomes h(f) = Ã(f) a 0 σy 2 S n (f) y = Ã(f) S n (f) df Sn (f) h(f) 2 df à (f) h(f). and a solution is h(f) Ã(f) S n (f). For white noise, S n (f) =constantandwegetourpreviousresult. More generally, the matched filter favors frequencies where the ratio of signal to noise is larger. Note again that the amplitude of the signal a 0 does not appear in the solution. It can be considered to be part of the proportionality constant (normalization). Note also that the actual noise variance cancels in the solution for h(f). Thus the amplitudes of both the signal and the noise factor out ofthe solution.

11 Practical Applications in Signal Detection: Example: we know the shape of the function A(t) andtheautocorrelation function of the noise (or its spectrum). The optimal detection scheme is to construct the filter using this information and investigate the cross correlation function Eq. 1. Output amplitudes can be tested against a threshold y T that is some multiple of σ y. Over an ensemble one can then define the detection probability P d and the false-alarm probability P fa. By changing the threshold both P d and P fa will change. Ideally one would like P d =1withP fa but in reality there is a tradeoff between the two. Periodic Signals: The matched filter for a periodic signal is simply aperiodictrainofthepulseshape. Whencorrelatedwiththemeasurements, the output is the same thing as folding the data with the underlying period. 6

12 Fig. 1. Matched filtering of Gaussian pulse with itself as a template. One realization of the template and pulse are shown while ten realizations of the CCF are shown. The SNR of the pulse is 5 (peak to rms noise). The pulse width 1/e) is 25.3 samples. 8

13 Fig. 2. Matched filtering of a narrower Gaussian pulse. One realization of the template and pulse are shown while ten realizations of the CCF are shown. The SNR of the pulse is 5 (peak to rms noise). The pulse width 1/e) is 10.3 samples. 9

14 Fig. 3. Matched filtering of a narrow Gaussian pulse with a broader template. One realization of the template and pulse are shown while ten realizations of the CCF are shown. The SNR of the pulse is 5 (peak to rms noise). The pulse width 1/e) is 10.3 samples while the template is 45.3 samples wide. 10

15 Fig. 4. Matched filtering of Gaussian pulse with a narrower template. The SNR of the pulse is 5 (peak to rms noise). The pulse width 1/e) is 10.3 samples while the template is 4.3 samples wide. 11

16 ROC Curves: AplotofP d vs. P fa is called a receiver operating characteristics curve, named after radar detection schemes. 7

17 ROC Curves ROC = receiver operating characteristic The ROC curve originated during World War II for using radar to detect objects in battle fields Now used in all fields where detection of a signal or classification of events and outcomes is done (biology, medicine, astronomy, etc.)

18 h"p://upload.wikimedia.org/ wikipedia/commons/3/36/ ROC_space- 2.png

19 Fig. 5. ROC plot for a pulse with SNR 5 (peak to rms noise) and width of 10.3 samples. Matched filtering is used. 13

20

21 Evaluating three different HIV epitope predictors. h"p:// upload.wikimedi a.org/ wikipedia/ commons/6/6b/ Roccurves.png

22 12 Localization Using Matched Filtering This handout describes localization of an object in a parameter space. For simplicity we consider localization of a pulse in time. The same formalism applies to localization of a spectral feature in frequency or to an image feature in a 2D image. The results can be extrapolated to a space of arbitrary dimensionality. I. First consider finding the amplitude of a pulse when the shape and location are known. Let the data be I(t) =aa(t)+n(t), where a =theunknownamplitudeandn(t) iszeromeannoise. The known pulse shape is A(t). Let the model be Î(t) =âa(t). Define the cost function to be the integrated squared error, [ ] 2 Q = dt I(t) Î(t).

23 Taking a derivative, we can solve for the estimate of the amplitude, â: ] âq = 2 dt [I(t) Î(t) âî(t) =0 â ] dt [I(t) Î(t) A(t) =0 dtî(t)a(t) = dta 2 (t) = dti(t)a(t) dti(t)a(t) 14 â = dti(t)a(t) dta2 (t). Note that: a. The model is linear in the sole parameter, â b. The numerator is the zero lag of the crosscorrelation function (CCF) of I(t) anda(t). c. The denominator is the zero lag of the autocorrelation function (ACF) of A(t).

24 II. Now consider the case where we don t know the location of the pulse in time (the time of arrival, TOA) and that it is the TOA we wish to estimate. We still know the pulse shape, apriori. Let the data, model and cost function be I(t) = aa(t t 0 )+n(t) 15 Î(t) = âa(t ˆt 0 ). Q = dt [ I(t) Î(t) ] 2. Note that the model is linear in â but is nonlinear in ˆt 0. Minimizing Q with respect to â, wehave ] âq = 2 dt [I(t) Î(t) âî(t) =0 ] dt [I(t) Î(t) A(t ˆt 0 )=0 â dtî(t)a(t ˆt 0 )= dta 2 (t ˆt 0 )= dti(t)a(t ˆt 0 ) dti(t)a(t ˆt 0 ) (3) â = dti(t)a(t ˆt 0 ) dta2 (t ˆt 0 ). This last equation has the same form as in I. except that the estimate for the arrival time ˆt 0 is involved.

25 16 Now, minimizing Q with respect to ˆt 0,wehave ] ˆt 0 Q = 2 dt [I(t) Î(t) ˆt 0 Î(t) =0 â â dt Î(t) }{{} âa(t ˆt 0 ) A (t ˆt 0 )= â dt I(t)A (t ˆt 0 ) dt A(t ˆt 0 )A (t ˆt 0 )= dt I(t)A (t ˆt 0 ). (4) Grid Search: One approach to finding the arrival time is to search over a 2D grid of â, ˆt 0 to find the values that satisfy equations 3 and 4. This approach is inefficient. Instead, one can search over a 1D space for the single nonlinear parameter, ˆt 0,andthensolveforâ using either equation 3 or 4. Linearization + Iteration: Another method is to find solutions for â and ˆt 0,wecanlinearize the equations in ˆt 0 t 0 by using Taylorseries expansions for A(t ˆt 0 )anda (t ˆt 0 ). Let ˆt 0 = t 0 + δˆt 0.Then,tofirstorderinδˆt 0 : A(t ˆt 0 ) A(t t 0 ) A (t t 0 )δˆt 0 A (t ˆt 0 ) A (t t 0 ) A (t t 0 )δˆt 0 A 2 (t ˆt 0 ) A 2 (t t 0 ) 2A (t t 0 )A(t t 0 )δˆt 0.

26 Now equations (3) and (4) become â â dt [A 2 (t t 0 ) 2δˆt 0 A(t t 0 )A (t t 0 )] = dt I(t)[A(t t 0 ) δˆt 0 A (t t 0 )] dt [A(t t 0 )A (t t 0 ) δˆt 0 A(t t 0 )A (t t 0 ) δˆt 0 A 2 (t t 0 )] = dt I(t)[A (t t 0 ) δˆt 0 A (t t 0 )]. 17 Consider the integral The integrand may be written as dt A(t t 0 )A (t t 0 ). A(t t 0 )A (t t 0 )= 1 d 2 dt A2 (t t 0 ) and so the integral equals 1 2 A2 (t t 0 ) t 2 t 1 0 in the limit of (e.g.) t 1,2 = T/2withT pulse width. We then have δˆt 0 â â dt A 2 (t t 0 ) = dt I(t)[A(t t 0 ) δˆt 0 A (t t 0 )] dt [A(t t 0 )A (t t 0 ) + A 2 (t t 0 )] = dt I(t)[A (t t 0 ) δˆt 0 A (t t 0 )].

27 18 Solving for â in both cases we have â = â = dt [I(t)A(t t0 ) δˆt 0 I(t)A (t t 0 )] dt A2 (t t 0 ) [ dt I(t) A (t t 0 )+δˆt 0 A (t t 0 ) ] δˆt 0 dt [ A(t t0 )A (t t 0 )+A 2 (t t 0 ) ]. Using the notation i 0 dt I(t)A(t t 0 ) i 1 dt I(t)A (t t 0 ) i 2 dt I(t)A 2 (t t 0 ) i 3 dt I(t)A (t t 0 ) [ ] dt I(t) A(t t 0 )A (t t 0 )+A 2 (t t 0 ). i 4 (5) (6) (7) we have â = i 0 δˆt 0 i 1 i 2 â = i 1 + δˆt 0 i 3 δˆt 0 i 4. Solving for δˆt 0 (to first order) we have δˆt 0 = i 1i 2 i 0 i 4 + i 2 i 3.

28 19 Iterative Solution for ˆt 0 This equation can be solved iteratively for δˆt 0 : 0. choose a starting value for ˆt calculate δˆt 0 using the linearized equations. 2. is δˆt 0 =0? 3a. if yes, stop. 3b. if no, update ˆt 0 ˆt 0 + δˆt 0 and go back to step 1. For the best fit value for ˆt 0,thechangeiszero,δˆt 0 =0(topofthe hill) and â can be calculated using one of the equations 5 or 6. Correlation Function Approach The iterative solution for ˆt 0 is similar to the following procedure that uses a crosscorrelation approach more directly: 1. cross correlate the template A(t) withi(t) togetaccf. 2. find the lag of peak correlation as an estimate for the arrival time, ˆt 0 = τ max. 3. calculate â if needed. Subtleties of the Cross Correlation Method The CCF is calculated using sampled data and therefore is itself a discrete quantity. Often one wants greater precision on the arrival time than is given by the sample interval. I.e. we want a floating point

29 20 number for ˆt 0,notanintegerindex. Thereforewewanttocalculate the peak of the CCF by interpolating near its peak. The interpolation should be done properly by using the appropriate interpolation formula for sampled data (using the sinc function). Using parabolic interpolation yields excessive errors for the arrival time. In practice, the proper interpolation is effectively done in the frequency domain by calculating the phase shift of the Fourier transform of the CCF, which is the product of the Fourier transform of the template and the Fourier transform of the data.

30 Timing Error from Radiometer Noise rms TOA error from template fitting with additive noise: Gaussian shaped pulse: Low-DM pulsars: DISS (and RISS) will modulate SNR N 6 = N / 10 6 Interstellar pulse broadening, when large, increases Δt S/N in two ways: SNR decreases by a factor W / [W 2 +τ d2 ] 1/2 W increases to [W 2 +τ d2 ] 1/2 Large errors for high DM pulsars and lowfrequency observations 23 June 2010 Jim Cordes IPTA2010 Leiden 30

31 Timing Error from Pulse- Phase Jitter f ϕ = PDF of phase variation a(ϕ) = individual pulse shape N i = number of independent pulses summed m I = intensity modulation index 1 f J = fraction jitter parameter = ϕ rms / W 1 Gaussian shaped pulse: N 6 = N i / June 2010 Jim Cordes IPTA2010 Leiden 31

32 Matched Filtering Pulse shape Templat e Correlation function Template too wide Convoluti on Template too narrow Template matched TO A