How to Win With Poisson Data: Imaging Gamma-Rays. A. Connors, Eureka Scientific, CHASC, SAMSI06 SaFDe

Transcription

1 How to Win With Poisson Data: Imaging Gamma-Rays A. Connors, Eureka Scientific, CHASC, SAMSI06 SaFDe 1

2 All-Sky Map > 1 GeV Photons 9 years of CGRO/EGRET 2

3 3

4 Model of Diffuse Emission along Galactic Plane: Strong, Moskalenko, Reimer (GALPROP) 4

5 What is that excess glow around the Milky Way? (Dixon, Hartman, Kolaczyk, et al: Poisson-tailored Haar Wavelet Thresholding) 5

6 How We Began (SAMSI06 SaFDe): Dixon, Hartman, Kolaczyk, et al 1998: New Astronomy 3 (1998) 539. `The immediate question arises as to the statistical significance of this feature. Though we are able to make rigorous statements about the coefficient-wise and level-wise FDR, similar quantification of object-wise significance (e.g., "this blob is significant at the n sigma level") are difficult.' 6

7 How to Win With Low-Count Poisson Data: OVERVIEW 1/ Motivation 2/ Opinion/Discussion: Four Old Rules + One New 3/ Framework: A Single Pixel 4/ Worked Example: All-Sky > 1GeV Gamma-Rays 7

8 FOUR RULES: RESPECT THE DATA (i.e. Respect the underlying distribution) No "binning up" - you lose information and you don't need to (see Scargle etc). No (or minimal) filtering or pre-processing No subtracting ("model out") Cut the cuts DATA EXPLORATION/VISUALIZATION - DIFFERENT THAN INFERENCE And both are useful LIKELIHOOD-BASED IF you want to know uncertainties BUT beware model incompleteness RESPECT YOUR UNKNOWNS i.e. Respect your uncertain background and calibration 'constants' (EffArea, etc.) They have a distribution, too. NEW ONE: What About Goodness of Fit? Try testing for "goodness of fit" and residuals by using flexible model (e.g. non- or semi-parametric) 8

9 FRAMEWORK: A SINGLE BIN y y x x1 Left: Poisson Distribution, mu = 0.2 Right: Poisson Distribution, mu = 10. 9

10 1. For a Single Pixel: 1.1/Examples: bkg = 0.2 cts, Measurement = 2 counts bkg = 10 cts, measured 2 cts Likelihood Distribution: Tails or Evidence Look At Likelihood of unknown flux s Use your favorite summaries: Marginalize; CL/CR; etc etc. 1.2/ Additionally bkg counts/effarea can all vary 1.3/ Assumed SOLVED (i.e. there are good procedures). 10

11 WORKED EXAMPLE: All-Sky >1 GeV Gamma-Rays: Galactic Diffuse

12 WORKED EXAMPLE: All-Sky >1 GeV Gamma-Rays: Galactic Diffuse

13 Background Known from Physics 2E-07 4E-07 6E-07 8E-07 1E-06 13

14 On Top Of One-Pixel Procedure, add: SHAPE Like Dixon, Hartman, Kolaczyk et al, use very flexible non-physics model We use a multiscale (Haar wavelet-like) model In Addition: In Full Likelihood Framework (Also: Instrument Smearing, i.e. PSF, included) Work in progress - Esch et al EMC2 Re-work by D. van Dyk, A. Roy, SAMSI06 CHASC Mistakes: probably mine 14

15 FLEXIBLE MODEL: EMC Four-Way Splits Smoothing parameters for flux ratios fit Hyperparameter (for smoothing kappas) set via MC 15

16 DOES THIS WORK? Yes. AS HYPOTHESIS TEST? Qualified yes. AS RESIDUALS? Qualified yes Simulations - 16

17 Simulation 1: GALPROP Gamma-rays from Galactic Diffuse Gas: Cosmic rays impinging on gas clouds (Gas, particles measured separately) Cosmic rays impinging on photon field 2E-07 4E-07 6E-07 8E-07 1E-06 17

18 Simulation 1: GALPROP Gamma-rays from Galactic Diffuse Gas Convolved with Exposure Map: 18

19 Simulation 1: GALPROP Gamma-rays from Galactic Diffuse Gas * EGRET Exposure Simulated Poisson Counts:

20 * Physical Model is Background It is also the Best Fit * Multiscale Model is Residual * If Residual is SIGNIFICANT, then the model is not a good enough fit. * Shows residual at the same time * Because it is in a likelihood framework, can say something about uncertaintes. 20

21 Results of Procedure for Simulated Data = Model Mode Mean Sigma Skew 21

22 Results of Procedure for Simulated Data = Model Histogram of log10(expecttotalcount) Frequency log10(expecttotalcount) 22

23 Simulation 1I: GALPROP + New Comp 2E-07 4E-07 6E-07 8E-07 1E

24 Simulation II: GALPROP Gamma-rays from Galactic Diffuse Gas * EGRET Exposure Simulated Poisson Counts:

25 Results of Procedure for Simulated Data Unknown Bright Unknown Component Mode Mean Sigma Skew

26 Results of Procedure for Simulated Data With Unknown Bright Unknown: See DS 9 Movie 26

27 Results of Procedure for Simulated Data With Unknown Histogram of log10(expecttotalcount) Frequency log10(expecttotalcount) 27

28 Results of Procedure for Simulated Data With Unknown (new component ~ 8 X fainter than previous example)

29 Results of Procedure for Simulated Data = Model Histogram of log10(expecttotalcount) Frequency log10(expecttotalcount) 29

30 Results of Procedure for Simulated Data Unknown Faint Unknown Component Mode Mean Sigma Skew

31 Conclusion: Many related methods not mentioned here, e.g. John Rice s SAMSI Intro Workshop talk, HC, etc Many challenges in making it work more automatically See me for All-Sky Challenge Data (See Alex Young for Solar Data) EVEN SO - Very encouraging. 31