A6523 Modeling, Inference, and Mining Jim Cordes, Cornell University

Transcription

1 A6523 Modeling, Inference, and Mining Jim Cordes, Cornell University Lecture 23 Birthday problem comments Construc<ng likelihood func<ons Markov processes Reading: Ch 10, 11 and 3 in Gregory (from before) Chapter 29 of Mackay (Monte Carlo Methods) hjp:// book.pdf An Introduc<on to MCMC for Machine Learning (Andrieu et al. 2003, Machine Learning, 50, 5 hjp://link.springer.com/ar<cle/ %2fa %3A Gene<c Algorithms: Principles of Natural Selec<on Applied to Computa<on (Stephanie. Forrest) hjp://science.sciencemag.org/content/ 261/5123/872/tab- pdf Webpage: Projects! Abstract Paper Presenta<on Ques%on: how do we construct a likelihood func<on if (a) we do not know the data PDF and (b) we cannot invoke Gaussian sta<s<cs? 1 Birthday problem Data set from class Facebook friends FormaJed and python reading code from Ross Jennings Data = phases of birthday in cycles 2 1

2 3 4 2

3 PDF Models PDF Phase (cycles) 5 Models and priors What I have ajempted: Model 1: flat, no parameters (self = prior) Model 2: constant + sine with 1- yr period A + B cos(φ φ offset ) Prior(φ offset ) = flat Prior(B) = (1/B) x ln(b max /B min ) Model 3: constant + tophat H = height of tophat B = ini<al phase of tophat C = width of tophat (held fixed at 0.25) Prior(B) = flat Prior(H) = 1- (H/H max )^2 x (3/2H max ) 6 3

4 7 8 4

5 9 10 5

6 Constructing Likelihood Functions for Unknown Underlying PDFs Likelihood Func<ons for Non- analy<cal Cases Can t argue for Gaussian sta<s<cs Other analy<cal forms are too ad hoc 6

7 Likelihood Functions for Unknown PDF Forms General situation: Given data and modeling intentions when is it best to do the inverse problem of transforming the data to estimate the model as compared to doing the forward problem of transforming the model to data space and compare model and data there? Issues: 1. Instrumental responses typically involve convolutions so the inverse problem requires the ability to deconvolve. 2. Data sets may have selection biases that have to be removed. It may be easier to apply the selection biases to simulated model data than to correct the data. 3. The likelihood approach is in the spirit of doing the forward problem on the model. Population Data and Models Measurements of M objects in a survey that has known selec<on biases e.g. a sensi<vity- limited survey covering only a frac<on of the sky Data consist of N ajributes per object e.g. (flux, spectrum, sky posi<on, distance, proper mo<on, shape parameters, <me dependences, ) Model has K parameters We want the best popula<on model e.g. Maximum likelihood parameter values + errors The sample may be stochas<c more because of sta<s<cal varia<ons in the popula<on and not because of addi<ve measurement errors How do we calculate the likelihood func<on if we don t know the mathema<cal form to describe differences between the observa<onal sample and the model? 7

8 Suppose we know the parameters of interest in a model but we do not know the function form of the underlying PDF? How do we construct a likelihood function? 1. Invoke Gaussian statistics if possible. 2. Use a parameterized function for the PDF that may allow flexibility in kurtosis and skewness as well as mean and variance. 3. Use the data to define bins in which simulated values can be accumulated, where simulations are based on physical models. A good example is a population analysis (people, stars, etc.) where we may want to know quantities like the birth rate and spatial distribution, for which we do not have an a priori PDF form. 2 Examples A galaxy survey with strong wavelength- dependent selec<on effects Possible model parameters: Number density of galaxies vs. redshin Luminosity func<on Frac<on of galaxy types (ellip<cal, spiral, irregular) Pulsar popula<ons in the Milky Way Possible model parameters: Number density vs. loca<on in Galaxy Luminosity dependence on magne<c field and spin Distribu<on of pulsar spins and magne<c fields at birth Space veloci<es (runaway popula<on) 8

9 Using MC points to evaluate the likelihood function Compare data and model in observa1on space by doing the forward model (no deconvolu1on of observa1onal filters) and using data- defined bins D D MC D 9

10 Toy example: Consider a simple case: M data points, where {y i,i=1,...,m} are drawn from a N( y, σ) PDF(butwepretendwedonotknowthis!). Assume the data are statistically independent. Then the likelihood function would be M L(θ) = f y (y j ; θ). j=1 We would maximize L as usual to get the parameters θ =(ˆµ, ˆσ). Now suppose we can generate pseudo data by Monte Carlo using some scheme based on a set of population parameters. This might involve some trial function for the PDF, for example, or it might be based on a physical model. An example is the distribution of stars in the Galaxy. How do we calculate L? Formation of a likelihood function when the underlying PDF is not known A toy example consists of MC-ing data points, using them to define bins, and then forgetting that we know the underlying PDF. By other means we develop a model for our data and use it to MC a large number of points that we bin according to our observed data points. From the combined data and MC points we calculate the likelihood function using the formalism given above. By investigating a series of models as a function of their parameters, we can find the model parameters that maximize the likelihood function over the set of models and parameter space. One can go further by calculating posterior PDFs for the parameters and finding odds ratios for pairs of models. Figure 1: Left: ten MC points selected from an N(0, 1) distribution. Middle: Bins defined by midpoints between data points. Right: Bins portrayed in the context where we do not know the underlying PDF. 6 10

11 1. Define bins in y into which MC values can be summed. The actual data range can be divided into equal-sized bins, for example, or bins can be constrained to have 1 data points. Consider for generality bins with unequal widths y k for each bin centered on y k. 2. For M total data points the occupancy in each bin m k satisfies N bins M = m k. 3. MC a large number of points for trial values of model parameters. 4. Calculate the frequencies of occurrence f k = n k /N MC for the k th bin. Then the PDF value at y k is estimated as ˆf y (y k )= f k n k =. y k y k N MC This is an appropriate estimate because the mean number of MC points expected in the k th bin is n k = N MC f y (y k ) y k so the mean estimate is ˆf y (y k ) = k=1 n k y k N MC = f y (y k ). 5. Calculate the likehood function as a product over data points, M M n k(j) L f k(j) =, y k(j) N MC j=1 where k(j) is the bin of the j th data point. 4 j=1 Note if there are mul<ple actual data points in a bin, the product can be redefined over bins instead of data points 6. Alternatively, we could calculate L as a product over bins, taking into account the occupancy m k of actual data points: L N bins k=1 mk N f k bins = y k k=1 n k y k N MC mk. 7. The model used to generate the MC points can be varied until the likelihood is maximized, which allow point estimates of model parameters and their errors. 8. Note that we use because there are Poisson counting statistics in each bin, implying that there is a statistical error in L. 9. The number of MC points must be large enough so that each bin has MC points in them; otherwise f k =0and the likelihood function would vanish! 10. For a bad model, some of the bins will be unoccupied by either data or MC points, either case yielding lower or zero likelihood. 5 11

12 Binning and Likelihood Estimates Case 1: Small N = 20, 10 6 MC points 12

13 13

14 14

15 Binning and Likelihood Estimates Case 2: Large N = 100, 10 6 MC points 15

16 16

17 17

18 Binning and Likelihood Estimates Case 3: Large N = 100, Larger 10 7 MC points 18

19 19

20 20

21 Another Case 3: Binning and Likelihood Estimates Large N = 100, Larger 10 7 MC points N = 100 data points, N MC = 10 7 points Bins defined by midpoints between data points Shown with parent Gaussian PDF Bins shown without reference to parent popula<on 21

22 N = 100 data points, N MC = 10 7 points Bins shown without reference to parent popula<on Crude es<mate of probability based on one data point per cell Scargle approach: Can merge cells to improve the es<mate Histogram of MC points using true values of parameters 22

23 Likelihood contours for N=100 and N MC = 10 7 A more realistic model: TBD: generate figures like those on previous page but with a selection function applied. Suppose we have measurements that involve selection effects of the actual quantities of interest. An example is where f x (x; ) is the actual PDF that we would like to learn about but measurements are in effect selected from s(x)f x (x; ) where s(x) is a selection function that favors some values of x over others. The selection function is known. Observations would correspond to samples drawn from a PDF proportional to s(x)f x (x; ) and the goal of a MC analysis using a sequence of models would be to infer f x (x; ). The following procedure can be used: 1. We know the selection function s(x) and we have data points that are selected from s(x)f x (x; 0). 2. Use the data points to define bins in x. 3. A sequence of models might consist of the form f x (x; ) but with a sequence of parameter vectors,. 4. For a given parameter vector, MC a large number of points, apply the selection function s(x) (this is the forward problem), and bin the points. 5. Find the model that maximizes the likelihood function. 6. Another situation may involve not knowing the form f x (x; ) but rather a family of forms f x (j) (x; (j) ), j =1,,N models. For each of the N models models, the likelihood function can be maximized and the models compared to see which one gives the largest likelihood, as part of an exploratory analysis. 7. A proper analysis would compare pairs of models, e.g. the i th and j th models, by computation 7 of the odds ratio. 23

24 Binning and Likelihood Estimates Cases with selec<on effect applied to data and MC points Large N = 100, 10 6 MC points Excise values < σ 24

25 25

26 26

27 Binning and Likelihood Estimates Cases with selec<on effect applied to data and MC points Large N = 100, 10 6 MC points Excise values < 0. 27

28 28

29 29

30 THE VELOCITY DISTRIBUTION OF ISOLATED RADIO PULSARS Z. Arzoumanian 1 NASA Goddard Space Flight Center, Laboratory for High-Energy Astrophysics, Code 682, Greenbelt, MD 20771; zaven@milkyway.gsfc.nasa.gov D. F. Chernoff CenterforRadiophysicsandSpaceResearch, CornellUniversity, 612 SpaceSciences Building, Ithaca, NY 14853; chernoff@astro.cornell.edu and J. M. Cordes Department of Astronomy and NAIC, 512 Space Sciences Building, Cornell University, Ithaca, NY 14853; cordes@astro.cornell.edu The Astrophysical Received 2001Journal,568: ,2002March20 June 8; accepted November 21 # The American Astronomical Society. All rights reserved. Printed in U.S.A. ABSTRACT We infer the velocity distribution of radio pulsars based on large-scale 0.4 GHz pulsar surveys. We do so by modeling the evolution of the locations, velocities, spins, and radio luminosities of pulsars, calculating pulsed flux according to a beaming model and random orientation angles of spin and beam, applying selection effects of pulsar surveys, and comparing model distributions of measurable pulsar properties with survey data using a likelihood function. The surveys analyzed have well-defined characteristics and cover 95% of the sky. We maximize the likelihood in a six-dimensional space of observables P, _P,DM, b, l,and F (period, period derivative, dispersion measure, Galactic latitude, proper motion, and flux density, respectively). The models we test are described by 12 parameters that characterize a population s birth rate, luminosity, shutoff of radio emission, birth locations, and birth velocities. We infer that the radio beam luminosity (1) is comparable to the energy flux of relativistic particles in models for spin-driven magnetospheres, signifying that radio emission losses reach nearly 100% for the oldest pulsars, and (2) scales approximately as _E 1=2,whichinmagnetosphere models is proportional to the voltage drop available for acceleration of particles. We find that a two-component velocity distribution with characteristic velocities of 90 and 500 km s 1 is greatly preferred to any one-component distribution; this preference is largely immune to variations in other population parameters, such as the luminosity or distance scale or the assumed spin-down law. We explore some consequences of the preferred birth velocity distribution: (1) roughly 50% of pulsars in the solar neighborhood will escape the Galaxy, while 15% have velocities greater than 1000 km s 1 ; (2) observational bias against high-velocity pulsars is relatively unimportant for surveys that reach high Galactic z distances but is severe for spatially bounded surveys; (3) an important low-velocity population exists that increases the fraction of neutron stars retained by globular clusters and is consistent with the number of old objects that accrete from the interstellar medium; (4) under standard assumptions for supernova remnant expansion and pulsar spin-down, 10% of pulsars younger than 20 kyr will appear to lie outside of their host remnants. Finally, we comment on the ramifications of our birth velocity distribution for binary survival and the population of inspiraling binary neutron stars relevant to some GRB models and potential sources for LIGO. Subject headings: methods: statistical pulsars: general stars: neutron 30

31 Binning with Voronoi Tessellation For points on a plane, draw a line that is equidistant from a pair of neighboring points (seeds) and perpendicular to the line connec<ng the points. All points on the lines are equidistant to the nearest two or more seed points. Voronoi cells can be used as bins for data analysis purposes (Scargle) e.g. hjp://en.wikipedia.org/wiki/voronoi_diagram 31