BAYESIAN CROSS-IDENTIFICATION IN ASTRONOMY

Transcription

1 BAYESIAN CROSS-IDENTIFICATION IN ASTRONOMY / The Johns Hopkins University

2 Sketch by William Parsons (1845) 2 Recording Observations Whirlpool Galaxy M51 Discovered by Charles Messier (1773)

3 Multicolor Universe 3

4 Eventful Universe 4

5 5 Trends in Astronomy Exponential growth of data Moore s law in detectors CCDs Glass

6 Sloan Digital Sky Survey 6 Cosmic Genome Project M rows, 400+ cols 18TB 30TB of images from 30 CCDs Software revolution in astro Astronomers learn SQL Cannot look at the data anymore

7 The New Generations 7 Large Synoptic Survey Telescope [OPTICAL] 3 trillion rows, 200+ attributes, 100+ tables ~ 30PB 60PB of images, 3.2 Gpix cam Square Kilometer Array [RADIO] Processing limited

8 8

9 9 Keeping Up? Image processing Catalog extraction O(n) What is difficult? O(n log n) O(n 2 ),

10 10 More is Different Lots of opportunities Lots of challenges Lots of problems New approaches New algorithms New tools New computers

11 11 Do More at the Data Statistics on remote resources Fine-tune SQL Server for astronomy Analyses in and driven by SQL SDSS catalog archive and more GALEX, HLA, UKIDSS, PanSTARRS,

12 12

13 13

14 Storing Simulations 14 Millennium Run (MPA) 10 billion particles, 64 snapshots FoF groups and merger trees Millennium XXL 300 billion particles MultiDark Bolshoi Turbulence simulations (JHU) grid, 27TB

15 Storing Simulations 15 Millennium Run (MPA) 10 billion particles, 64 snapshots FoF groups and merger trees Millennium XXL 300 billion particles MultiDark Bolshoi Turbulence simulations (JHU) grid, 27TB

16 Observing Simulations 16 Comparison to real observations Lots of spatial searches In the database! Lemson, TB & Szalay (2011)

17 Fast Searches 17 Map space-filling curves to database indices Hierarchical Triangular Mesh on the unit sphere Peano-Hilbert / Morton curves in 3D Combine with query shapes Build from primitives

18 Fast Searches 18 Map space-filling curves to database indices Hierarchical Triangular Mesh on the unit sphere Peano-Hilbert / Morton curves in 3D Combine with query shapes Build from primitives

19 Millennium XXL 19

20 Understanding Subtleties 20 Interactive visualization of 27TB of turbulence sim Kai Bürger (TUM, JHU)

21 21 The SDSS Genealogy TerraServer By Alex Szalay SDSS SkyServer Onco Space Life Under Your Feet CASJobs MyDB GalaxyZoo Turbulence DB Super COSMOS Hubble Legacy Arch SkyQuery JHU 1K Genomes GALEX Pan-STARRS Millennium Palomar QUEST UKIDDS VO Services Open SkyQuery INDRA Simulation Potsdam Milky Way Laboratory MHD DB VO Footprint VO Spectrum

22 22 SkyQuery Dynamical federation of archives

23 23 Cross-Identification One of the most fundamental analysis steps

24 24 What is the Right Question? Cross-identification is a hard problem Computationally, Scientifically & Statistically Need symmetric n-way solution Need reliable quality measure Same or not? Distance threshold? Maximum likelihood?

25 Modeling the Astrometry 25 Astrometric precision A simple function Where on the sky? Anywhere really

26 NOT SAME OR Same or Not? 26 The Bayes factor H: all observations of the same object at m K: might be from separate objects at {m i }

27 NOT SAME OR Same or Not? 27 The Bayes factor H: all observations of the same object at m K: might be from separate objects at {m i }

28 NOT SAME OR Same or Not? 28 The Bayes factor H: all observations of the same object at m K: might be from separate objects at {m i }

29 NOT SAME OR Same or Not? 29 The Bayes factor H: all observations of the same object at m On the sky K: might be from separate objects at {m i } Astrometry

30 NOT SAME OR Same or Not? 30 The Bayes factor H: all observations of the same object at m On the sky K: might be from separate objects at {m i } Astrometry

31 Normal Distribution 31 Astrometric precision: Fisher distribution: Analytic results: For high accuracies:

32 32 Analytic Results Normal distribution Flat and spherical Gauss and Fisher 2-way results

33 Wikipedia: Interpretation 33

34 34 From Priors to Posteriors Bayes factor is the connection

35 From Priors to Posteriors 35 Posterior probability from prior & Bayes factor Prior probability of a match Like dice in a bag: 1/N and N 1 n In general?

36 36 From Priors to Posteriors Different selections Nearby / Distant Red / Blue But only 1 number

37 37 Self-Consistent Estimates Prior has an unknown fudge-factor Educated guess Or solve for it: TB & Szalay (2008)

38 38 Simulations Mock objects With correct clustering U 01 values as properties 0 1 Simulated sources Subsets: N 1 N 2 Overlap: N

39 39 Simulations Mock objects With correct clustering U 01 values as properties 0 1 Simulated sources Subsets: N 1 N 2 Overlap: N

40 Simulations 40 Quality Multiple matches Explained by simple model of point sources! Heinis, TB, Szalay (2009)

41 41 SkyQuery Almost pure standard SQL

42 42 SkyQuery Almost pure standard SQL

43 43 SkyQuery Almost pure standard SQL Added XMATCH Verifiable Flexible

44 44 Matching Events Streams of events in time and space E.g., thresholded peaks in signal-to-noise (1) (2) (x) TB (2011)

45 45 Matching Events Streams of events in time and space E.g., thresholded peaks in signal-to-noise (1) (2) (x) TB (2011)

46 46 Proper Motion Same hypotheses but different parameters Just need prior to integrate Sources from SDSS

47 Proper Motion 47 Same hypotheses but different parameters Just need prior to integrate Kerekes, TB+ (2010) Sources from SDSS

48 48 Radio Morphology

49 Example 49 Core match But lobes? overlay0113.png

50 Simplest Model for Radio Sources 50 A core and two symmetric lobes Core direction parameter m Lobe direction parameter m Other lobe is at 2m m The prior is some p(m,m ) = p(m) p(m m) Parameter m anywhere on the sky: p(m) But m is seen certain separations away: p(m m)

51 Simplest Model for Radio Sources 51 A core and two symmetric lobes Core direction parameter m Lobe direction parameter m Other lobe is at 2m m The prior is some p(m,m ) = p(m) p(m m) Parameter m anywhere on the sky: p(m) But m is seen certain separations away: p(m m)

52 Likelihood of Same 52 Data: { x 0 ; y 0 y 1 y 2 } directions 2-way matching using the new model

53 Likelihood of Same 53 Data: { x 0 ; y 0 y 1 y 2 } directions 2-way matching using the new model

54 Example 54 Core match But lobes? overlay0113.png

55 Example 55 Core match But lobes? overlay0113.png

56 Example 56 Core match But lobes? overlay0113.png

57 Competing Hypotheses 57 Many to choose from Which radio detection is the core? (closest ;-) Which are separate objects?

58 Example 58 Competing hypotheses w/ 2 different tophat priors None None None : (F) Core None None : (F) Core Lobe None : (F) Core Lobe Lobe : (F) None Lobe Lobe : (F) None None Lobe : (F)

59 59 Hubble Source Catalog Crossmatching the Hubble Legacy Archive

60 60 Uncertain Positioning Exceptional accuracy with images Large uncertainty in positioning Due to few standards in the tiny field of view Solve for relative corrections Across overlapping observations

61 61 Astrometric Correction Rotation in 3D Describes both translation and rotation locally Need constrained numerical optimization: R T R=I

62 62 Infinitesimal Rotation Optimization without constraints Displacement by the cross product operator Fast computation for the rotation vector ω Just sum up a 3 3 matrix and vector Solve the linear equation:

63 Hubble Source Catalog 63 SQL pipeline Astrometric correction Subpixel precision RELEASE ON MAST TB & Lubow (2012) Available now!

64 64 Summary Bayesian approach to cross-identification Places former heuristics on a firm statistical basis Enables us to properly include Physics, geometry, etc Naturally extends to time-domain Events, proper motion, lightcurves Opens the door for next-generation methods