Accounting for Calibration Uncertainty in Spectral Analysis. David A. van Dyk

Transcription

1 Accounting for in Spectral Analysis David A van Dyk Statistics Section, Imperial College London Joint work with Vinay Kashyap, Jin Xu, Alanna Connors, and Aneta Siegminowska IACHEC March 2013 David A van Dyk Accounting for in Spectral Analysis

2 Outline 1 Bayesian Statistical Methods Components of a Statistical Model Statistical Computation 2 The Calibration Sample The Effect of 3 Pragmatic and Fully Bayesian Solutions Empirical Illustration David A van Dyk Accounting for in Spectral Analysis

3 Outline Bayesian Statistical Methods Components of a Statistical Model Statistical Computation 1 Bayesian Statistical Methods Components of a Statistical Model Statistical Computation 2 The Calibration Sample The Effect of 3 Pragmatic and Fully Bayesian Solutions Empirical Illustration David A van Dyk Accounting for in Spectral Analysis

4 Components of a Statistical Model Statistical Computation Bayesian Statistical Analyses: Likelihood Likelihood Functions: The distribution of the data given the model parameters Eg, Y dist Poisson(λ S ): likelihood(λ S ) = e λ S λ Y S /Y! Maimum Likelihood Estimation: Suppose Y = 3 likelihood lambda Can estimate λ S and its error bars The likelihood and its normal approimation David A van Dyk Accounting for in Spectral Analysis

5 Components of a Statistical Model Statistical Computation Bayesian Analyses: Prior and Posterior Dist ns Prior Distribution: Knowledge obtained prior to current data Bayes Theorem and Posterior Distribution: posterior(λ) likelihood(λ)prior(λ) Combine past and current information: prior lambda posterior lambda Bayesian analyses rely on probability theory David A van Dyk Accounting for in Spectral Analysis

6 Multi-Level Models Components of a Statistical Model Statistical Computation A Poisson Multi-Level Model: LEVEL 1: Y Y B, λ S dist Poisson(λ S ) Y B, LEVEL 2: Y B λ B dist Pois(λ B ) and X λ B dist Pois(λ B 24), LEVEL 3: specify a prior distribution for λ B, λ S Each level of the model specifies a dist n given unobserved quantities whose dist ns are given in lower levels prior lams posterior lams lamb joint posterior with flat prior lams David A van Dyk Accounting for in Spectral Analysis

7 Components of a Statistical Model Statistical Computation Multi-Level Models: X-ray Spectral Analysis counts per unit counts per unit counts per unit energy (kev) energy (kev) absorbtion and submaimal effective area instrument response pile-up background counts per unit counts per unit energy (kev) energy (kev) energy (kev) PyBLoCXS: Bayesian Low-Count X-ray Spectral Analysis David A van Dyk Accounting for in Spectral Analysis

8 Components of a Statistical Model Statistical Computation Embedding into a Statistical Model We aim to include calibration uncertainty as a component of the multi-level statistical model fit in PyBLoCXS Build a multi-level model In addition to spectral parameters, calibration products are treated as unknown quantities Calibration products are high dimensional unknowns Calibration scientists provide valuable prior information about these quantities We must quantify this information into a prior distribution Computation becomes a real issue! David A van Dyk Accounting for in Spectral Analysis

9 Bayesian Statistical Methods Components of a Statistical Model Statistical Computation Markov Chain Monte Carlo Eploring the posterior distribution via Monte Carlo prior lams posterior lams joint posterior with flat prior lams lamb David A van Dyk Accounting for in Spectral Analysis

10 Components of a Statistical Model Statistical Computation Model Fitting: Comple Posterior Distributions Highly non-linear relationship among parameters David A van Dyk Accounting for in Spectral Analysis

11 Components of a Statistical Model Statistical Computation Model Fitting: Comple Posterior Distributions The classification of certain stars as field or cluster stars can cause multiple modes in the distributions of other parameters David A van Dyk Accounting for in Spectral Analysis

12 Outline Bayesian Statistical Methods The Calibration Sample The Effect of 1 Bayesian Statistical Methods Components of a Statistical Model Statistical Computation 2 The Calibration Sample The Effect of 3 Pragmatic and Fully Bayesian Solutions Empirical Illustration David A van Dyk Accounting for in Spectral Analysis

13 The Basic Statistical Model The Calibration Sample The Effect of Computer Models Background Basic Physics Parametric Models Photon Absorption Observed Data (mean=λ) Multi-Scale Models High-Energy Telescope Effective Area (ARF) Blurring (RMF) Optical Telescope Gaussian Measurement Errors Embed physical models into multi-level statistical models Must account for compleities of data generation State of the art computational techniques enable us to fit the resulting highly-structured model David A van Dyk Accounting for in Spectral Analysis

14 Calibration Products The Calibration Sample The Effect of Analysis is highly dependent on Calibration Products: Effective Area Cuves Energy Redistribution Matrices Eposure Maps Point Spread Functions In this talk we focus on uncertainty in the effective area ACIS S effective area (cm 2 ) E [kev] A Chandra effective area Sample Chandra psf s (Karovska et al, ADASS X) EGRET eposure map (area time) David A van Dyk Accounting for in Spectral Analysis

15 Calibration Sample The Calibration Sample The Effect of Calibration sample of 1000 representative curves 1 Sample ehibits comple variability Requires storing many high dimensional calibration products ACIS S effective area (cm 2 ) default subtracted effective area (cm 2 ) E [kev] E [kev] 1 Thanks to Jeremy Drake and Pete Ratzlaff David A van Dyk Accounting for in Spectral Analysis

16 The Calibration Sample The Effect of Generating Calibration Products on the Fly We use Principal Component Analysis to represent uncertainty: A A 0 δ m e j r j v j, j=1 A 0 : default effective area, δ: mean deviation from A 0, r j and v j : first m principle component eigenvalues & vectors, e j : independent standard normal deviations Capture 99% of variability with m = 18 David A van Dyk Accounting for in Spectral Analysis

17 Checking the PCA Emulator The Calibration Sample The Effect of A rep! A o (cm 2 ) E (kev) kev A (cm 2 ) kev A (cm 2 ) kev A (cm 2 ) David A van Dyk Accounting for in Spectral Analysis

18 The Simulation Studies The Calibration Sample The Effect of Simulated Spectra Spectra were simulated using an absorbed power law, f (E j ) = αe N Hσ(E j ) E Γ j, accounting for instrumental effects; E j is the energy of bin j Parameters (Γ and N H ) and sample size/eposure times: Effective Area Nominal Counts Spectal Model Default Etreme Harder Softer SIM 1 X X X SIM 2 X X X SIM 3 X X X An absorbed powerlaw with Γ = 2, N H = /cm 2 An absorbed powerlaw with Γ = 1, N H = /cm 2 David A van Dyk Accounting for in Spectral Analysis

19 The Calibration Sample The Effect of 30 Most Etreme Effective Areas in Calibration Sample A! A (cm 2 ) 50 o E (kev) 15 largest and 15 smallest determined by maimum value David A van Dyk Accounting for in Spectral Analysis

20 The Calibration Sample The Effect of The Effect of SIMULATION 1 SIMULATION N H(10 cm "2 ) ! ! N H(10 cm "2 ) ! ! Columns represent two simulated spectra True parameters are horizontal lines Posterior under default calibration is plotted in black The posterior is highly sensitive to the choice of effective area! N H(10 22 cm "2 ) N H(10 22 cm "2 ) David A van Dyk Accounting for in Spectral Analysis

21 The Effect of Sample Size The Calibration Sample The Effect of counts Γ counts Γ N H(10 22 cm 2 ) N H(10 22 cm 2 ) The effect of is more pronounced with larger sample sizes David A van Dyk Accounting for in Spectral Analysis

22 Outline Bayesian Statistical Methods Pragmatic and Fully Bayesian Solutions Empirical Illustration 1 Bayesian Statistical Methods Components of a Statistical Model Statistical Computation 2 The Calibration Sample The Effect of 3 Pragmatic and Fully Bayesian Solutions Empirical Illustration David A van Dyk Accounting for in Spectral Analysis

23 Pragmatic and Fully Bayesian Solutions Empirical Illustration Accounting for We consider inference under: A PRAGMATIC BAYESIAN TARGET: π 0 (A, θ) = p(a)p(θ A, Y ) THE FULLY BAYESIAN POSTERIOR: π(a, θ) = p(a Y )p(θ A, Y ) Here: p(θ A, Y ): PyBLoCXS posterior with known effective area, A p(a Y ): The posterior distribution for A p(a): The prior distribution for A Should we let the current data inform inference for calibration products? David A van Dyk Accounting for in Spectral Analysis

24 Two Possible Target Distributions Pragmatic and Fully Bayesian Solutions Empirical Illustration We consider inference under: A PRAGMATIC BAYESIAN TARGET: π 0 (A, θ) = p(a)p(θ A, Y ) THE FULLY BAYESIAN POSTERIOR: π(a, θ) = p(a Y )p(θ A, Y ) Concerns: Statistical Fully Bayesian target is correct Cultural Astronomers have concerns about letting the current data influence calibration products Computational Both targets pose challenges, but pragmatic Bayesian target is easier to sample Practical How different are p(a) and p(a Y )? David A van Dyk Accounting for in Spectral Analysis

25 Bayesian Statistical Methods Pragmatic and Fully Bayesian Solutions Empirical Illustration Using Multiple Imputation to Account for Uncertainty N H (10 22 cm "2 ) ! ! N H (10 22 cm "2 ) N H (10 22 cm "2 ) MI is a simple, but approimate, method: ! ! Fit the model M 1000 times, each with random effective area curve from the calibration sample Estimate parameters by averaging the M fitted values N H (10 22 cm "2 ) Estimate uncertainty by combining the within and between fit errors David A van Dyk Accounting for in Spectral Analysis

26 Densi Pragmatic and Fully Bayesian Solutions Empirical Illustration The Multiple Imputation Combining Rules B = 1 M 1 N H (10 22 cm "2 ) ˆθ = 1 M ! M ˆθ m, m= N H (10 22 cm "2 )! W = 1 M Densi 0 20 N H (10 22 cm "2 ) M Var(ˆθ m ), m=1 M (ˆθ m ˆθ)(ˆθ m ˆθ), T = W m= ( 1 1 M ! ! ) B The total variance combines the variances within and between the M analyses N H (10 22 c David A van Dyk Accounting for in Spectral Analysis

27 Pragmatic and Fully Bayesian Solutions Empirical Illustration Accounting for with MCMC When using MCMC in a Bayesian setting we can: Sample a different effective area from the calibration sample at each iteration according to: Pragmatic Bayes: p(a) Fully Bayes: p(a Y ) Computational challenges arise in both cases We focus on comparing the results of the two methods A Simulation Sampled 10 5 counts from a power law spectrum: E 2 A true is 15σ from the center of the calibration sample David A van Dyk Accounting for in Spectral Analysis

28 Sampling From the Full Posterior Pragmatic and Fully Bayesian Solutions Empirical Illustration Default Effective Area Pragmatic Bayes Fully Bayes θ θ θ θ 1 Spectral Model (purple bullet = truth): θ 1 θ 1 f (E j ) = θ 1 e θ 1σ(E j ) E θ 2 j Pragmatic Bayes is clearly better than current practice, but a Fully Bayesian Method is the ultimate goal David A van Dyk Accounting for in Spectral Analysis

29 Bayesian Statistical Methods Pragmatic and Fully Bayesian Solutions Empirical Illustration The Effect in an Analysis of a Quasar Spectrum Default Effective Area 1000θ 1 θ Pragmatic Bayes 1000θ 1 θ Fully Bayes 1000θ 1 θ Default Effective Area θ 3 θ Pragmatic Bayes θ 3 θ Fully Bayes θ 3 θ 2 (obs ID = 866) David A van Dyk Accounting for in Spectral Analysis

30 Pragmatic and Fully Bayesian Solutions Empirical Illustration Results: 95% Intervals Standardized by Standard Fit σ fi(γ) σ fi(γ) µ^prag(γ) µ^full(γ) For large spectra calibration uncertainty swamps statistical error In large spectra fully Bayes identifies A and shifts interval David A van Dyk Accounting for in Spectral Analysis

31 Thanks Bayesian Statistical Methods Pragmatic and Fully Bayesian Solutions Empirical Illustration Collaborative work with: Vinay Kashyap Jin Xu Alanna Connors Hyunsook Lee Aneta Siegminowska California-Harvard Astro-Statistics Collaboration If you want more details Lee, H, Kashyap, V, van Dyk, D, Connors, A, Drake, J, Izem, R, Min, S, Park, T, Ratzlaff, P, Siemiginowska, A, and Zezas, A Accounting for Calibration Uncertainties in X-ray Analysis: Effective Area in Spectral Fitting The Astrophysical Journal, 731, , 2011 David A van Dyk Accounting for in Spectral Analysis