A6523 Signal Modeling, Statistical Inference and Data Mining in Astrophysics Spring 2015 http://www.astro.cornell.edu/~cordes/a6523 Lecture 4 See web page later tomorrow Searching for Monochromatic Signals in Noise We derived the spectrum of a time series containing a complex exponential and additive noise The shape of the spectral line is a sinc function. For continuous time and frequency this is sinc(x) = sin(πx)/πx For the discrete case it is slightly different The sinc function underlies many of the problems associated with spectral analyis based on the Fourier transform The sinc function is the response of the Fourier transform to a sinusoid. Any function or stochastic process can be represented as a sum of sinusoids à its power spectrum is convolved with the appropriate sinc function. 1
sinc(x) sinc(x) 1.2 1.0 0.8 0.6 0.4 0.2 0.0 0.2 0.4 10 5 0 5 10 1.2 1.0 0.8 0.6 0.4 0.2 0.0 0.2 0.4 sidelobes main lobe Sinc func-on aligned so that its zeros fall on integer values of x. If we plot only the black dots, we get a Kronecker delta func-on Sinc func-on mis- aligned from integer x values. We no longer get a Kronecker delta 10 5 0 5 10 x So what? Generally a monochroma-c signal will not be in integer mul-ple of the frequency resolu-on δf = 1 / T so power in sinc 2 will leak into nearby (main lobe) and distant (sidelobe) frequencies. The envelope of sidelobe amplitudes ~ 1 / f 2 2
Searching for Monochromatic Signals in Noise We derived the spectrum of a time series containing a complex exponential and additive noise In the no-signal limit: The PDF of the spectrum is exponential (one sided) The false-alarm probability is e -η for a threshold for detection of η x spectral mean The spectral mean = spectral rms for an exponential PDF If we find a spectral line that exceeds the threshold, we would say that the line is real at 100e -η % confidence. Relevant PDFs Gaussian or Normal: N(μ, σ 2 ) f X (x) = 1 2πσ e (x µ)2 /2σ 2 Random variable argument Exponential: Chi 2 : X = N j=1 f X (x) = f X (x) =X 1 e x/x H(x) x 2 j with x j = = i.i.d GRV: N(0,1) 1 Γ(N/2)2 N/2 x(n 2)/2 e x/2 3
hsp://en.wikipedia.org/wiki/normal_distribu-on hsp://en.wikipedia.org/wiki/exponen-al_distribu-on hsp://en.wikipedia.org/wiki/chi- squared_distribu-on hsp://upload.wikimedia.org/wikipedia/commons/a/a9/empirical_rule.png 4
Properi-es of a Gaussian or Normal RV χ 2 5
Detection Probability The exponential PDF applies to the no-signal case But for the frequency bin in the spectrum that has a signal the PDF is different: What is the relevant PDF? Need to consider the PDF of phasor + noise From the PDF we can calculate the probability of detection (true positive) and false negatives. 6
PDF of Phasor Magnitude s = 0, 3, 5, 10 sigma_n = 1 PDF of Intensity s = 0, 1, 3, 5 sigma_n = 1 Detec-on probability P det (I min )= I min di f I (I) 7
ROC Curves Receiver operating characteristic Relative operating characteristic In a so-called detection problem, we try to establish whether a signal of some assumed type is present in data that include noise This is a universal problem that applies to many laboratory and observational contexts. In astronomy, ROC curves apply to finding sources/signals in images, spectra, time series, etc. An ROC curve = P d vs P fa (detection vs falsealarm probability) Binary classifier used in physics, biometrics, machine learning, data mining, hsp://en.wikipedia.org/wiki/receiver_opera-ng_characteris-c 8
Accuracy of Ground Hog Day Predictions Meteorological accuracy (from wikipedia): According to Groundhog Day organizers, the rodents' forecasts are accurate 75% to 90% of the time. However, a Canadian study for 13 cities in the past 30 to 40 years found that the weather patterns predicted on Groundhog Day were only 37% accurate over that time period. According to the StormFax Weather Almanac and records kept since 1887, Punxsutawney Phil's weather predictions have been correct 39% of the time. The National Climatic Data Center has described the forecasts as on average, inaccurate and stated that the groundhog has shown no talent for predicting the arrival of spring, especially in recent years. And what about the superbowl predictor for whether the stock market will be up or down? Etc. etc. 9
Estimation Error: For any estimation procedure, we are interested in the estimation error, which we quantify with the variance of the estimator: Var{S k } S 2 k S k 2. This requires that we calculate the fourth moment of the DFT: X k 4 = A δ kk0 + Ñk 4 (3) = A 4 δ kk0 + A 2 δ kk0 Ñk + Ñ k 2 (4) + Ñk 4 (5) + 2A 2 δ kk0 Ñk 2 (6) + 2A 3 δ kk0 (N k + N k ) (7) + 2A δ kk0 Ñk 2 (Ñk + Ñ k ). The last two terms vanish because they involve odd order moments. The third term is Ñk 4 = 2 Ñk 2 2 because Ñk is complex Gaussian noise by the Central Limit Theorem. Thus, the first and fourth terms and half of the third terms are just the square of X k 2, so X k 4 = X k 2 2 + Ñk 2 2 +2A 2 δ kk0 Ñk 2 8 or Var{ X k 2 } = X k 4 X k 2 2 (8) The fractional error in the spectrum is thus k [Var { X k 2 }] 1/2 X k 2 = Ñk 2 2 +2A 2 δ kk0 Ñk 2 (9) = Ñk 2 1+ 2 2A2 δ kk0 Ñk 2 = (σ 2 n/n ) 2 1+ 2A2 N δ kk0 σ 2 n = (1 + 2A2 N δ kk0 /σn) 2 1/2. 1+A 2 N δ kk0 /σn 2 Thus, for frequency bins off the line (k = k 0 ) we have k 1. On the line we have 1 A 2 N/σn 2 0 k = (1 + 2A2 N/σ 2 n) 1/2 1+A 2 N/σ 2 n = 1 1 2 A 2 N σ 2 n 2 N σ n 2 A 2 N/σ 2 n 1 A A 2 N/σ 2 n 1 Thus, as the signal-to-noise A/σ n gets very large, the error in the spectral estimate 0, as expected. (10) 9 10
Frequentist à Bayesian The approach we have taken is classic frequentism: it appeals to the notion of repeated trials and frequency of occurrence; also to an underlying ensemble. Point estimates are given of e.g. the signal strength and the frequency. The Bayesian alternative: using the one realization of data in hand, what is the PDF of the frequency and amplitude? The relevant PDF is the posterior PDF and it is derived from the product of a prior PDF and a likelihood function. Instead of a power spectrum one gets a PDF. The PDF ends up depending on the periodogram. Basic Probability Tools Random variables, event space: ζ = event à X = random var. PDF, CDF, characteristic function Median, mode, mean Conditional probabilities and PDFs Bayes theorem Comparing PDFs Moments and moment tests Sums of random variables and convolution theorem Central Limit Theorem Changes of variable Functions of random variables Sequences of random variables Stochastic processes = sequences of random variables vs. t, f, etc. Power spectrum, autocorrelation, autocovariance, and structure functions Bispectra Random walks, shot noise, autoregressive, moving average, Markov processes 11
I. Ensemble vs. Time Averages Experimentally/observationally we are forced to use sample averages of various types Our goal is often, however, to learn about the parent population or statistical ensemble from which the data are conceptually drawn In some circumstances time averages converge to good estimates of ensemble averages In others, convergence can be very slow or can fail (e.g. red-noise processes) I(t, ζ) 12
I(t, ζ) I(t, ζ) As data span length T à time average à ensemble average Ergodic 13
2/3/15 Types of Random Processes Goodman Sta$s$cal Op$cs Example: the Universe Measurements of the CMB and large-scale structure are on a single realization The goal of cosmology is to learn about the (notional) ensemble of conditions that lead to what we see Quantitatively these are cast in questions like what was the primordial spectrum of density fluctuations? and that spectrum is usually parameterized as a power law Perhaps the multiverse = the ensemble Are all universes the same (statistically)? Do measurements on our universe typify all universes? (Conventional wisdom says no) 14
2/3/15 Basis functions: spherical harmonics TCMB = 2.7 K ΔT/TCMB ~ 10-5 Wilkinson Microwave Anisotropy Probe 15
I(t, ζ) Nonstationary Case I(t, ζ) 16
I(t, ζ) Random walk in spin phase I(t, ζ) Random walk in spin frequency 17
I(t, ζ) Random walk in spin frequency derivative White noise RW 1 RW 2 RW 3 18
19
Random Walk Examples Spinning objects: Earth, neutron stars Steps in torque or spin rate Observable = spin phase Scattered photon propagation (diffusion) Step = mean-free path Observable = propagation time Cosmic-ray propagation in the Galaxy Step = scattering off of small-scale magnetic field variations Observable = `grammage of interaction based on isotopic content (typically ~ 5 g cm -2 ) Orbital perturbations Asteroid belt objects à Near Earth Objects Motions of planetesimals in protoplanetary disks Galactic orbits of stars from gravitational potential granularity (molecular clouds, spiral arms) à diffusion of stellar populations 20
Other Random Walk Examples MCMC: random walk in parameter space Brownian motion of a dust particle Molecular diffusion Diffusion of biological populations Options pricing in financial markets Step = transaction Observable = price Black-Scholes equation = Fokker-Planck equation 21