Accounting for Calibration Uncertainty in Spectral Analysis. David A. van Dyk

Transcription

1 Bayesian Analysis of Accounting for in Spectral Analysis David A van Dyk Statistics Section, Imperial College London Joint work with Vinay Kashyap, Jin Xu, Alanna Connors, and Aneta Siegminowska ISIS May 2015 David A van Dyk

2 Bayesian Analysis of Outline 1 Bayesian Statistical Methods Components of a Statistical Model Statistical Computation 2 The Calibration Sample The Effect of 3 Bayesian Analysis of Pragmatic and Fully Bayesian Solutions Empirical Illustration David A van Dyk

3 Outline Bayesian Statistical Methods Bayesian Analysis of Components of a Statistical Model Statistical Computation 1 Bayesian Statistical Methods Components of a Statistical Model Statistical Computation 2 The Calibration Sample The Effect of 3 Bayesian Analysis of Pragmatic and Fully Bayesian Solutions Empirical Illustration David A van Dyk

4 Bayesian Analysis of Components of a Statistical Model Statistical Computation Bayesian Statistical Analyses: Likelihood Likelihood Functions: The distribution of the data given the model parameters Eg, Y dist Poisson(λ S ): likelihood(λ S ) = e λ S λ Y S /Y! Maximum Likelihood Estimation: Suppose Y = 3 likelihood lambda Can estimate λ S and its error bars The likelihood and its normal approximation David A van Dyk

5 Bayesian Analysis of Components of a Statistical Model Statistical Computation Bayesian Analyses: Prior and Posterior Dist ns Prior Distribution: Knowledge obtained prior to current data Bayes Theorem and Posterior Distribution: posterior(λ) likelihood(λ) prior(λ) Combine past and current information: prior lambda posterior lambda Bayesian analyses rely on probability theory David A van Dyk

6 Bayesian Analysis of Multi-Level Models Components of a Statistical Model Statistical Computation A Poisson Multi-Level Model: LEVEL 1: Y Y B, λ S dist Poisson(λ S ) Y B, LEVEL 2: Y B λ B dist Pois(λ B ) and X λ B dist Pois(λ B 24), LEVEL 3: specify a prior distribution for λ B, λ S Each level of the model specifies a dist n given unobserved quantities whose dist ns are given in lower levels prior lams posterior lams lamb joint posterior with flat prior lams David A van Dyk

7 Bayesian Analysis of Components of a Statistical Model Statistical Computation Multi-Level Models: X-ray Spectral Analysis counts per unit counts per unit counts per unit energy (kev) energy (kev) absorbtion and submaximal effective area instrument response pile-up background counts per unit counts per unit energy (kev) energy (kev) energy (kev) PyBLoCXS: Bayesian Low-Count X-ray Spectral Analysis David A van Dyk

8 Bayesian Analysis of Components of a Statistical Model Statistical Computation Embedding into a Statistical Model We aim to include calibration uncertainty as a component of the multi-level statistical model fit in PyBLoCXS Build a multi-level model In addition to spectral parameters, calibration products are treated as unknown quantities Calibration products are high dimensional unknowns Calibration scientists provide valuable prior information about these quantities We must quantify this information into a prior distribution Computation becomes a real issue! David A van Dyk

9 Bayesian Statistical Methods Bayesian Analysis of Components of a Statistical Model Statistical Computation Markov Chain Monte Carlo Exploring the posterior distribution via Monte Carlo prior lams posterior lams joint posterior with flat prior lams lamb David A van Dyk

10 Bayesian Analysis of Components of a Statistical Model Statistical Computation Model Fitting: Complex Posterior Distributions Highly non-linear relationship among parameters David A van Dyk

11 Bayesian Analysis of Components of a Statistical Model Statistical Computation Model Fitting: Complex Posterior Distributions The classification of certain stars as field or cluster stars can cause multiple modes in the distributions of other parameters David A van Dyk

12 Outline Bayesian Statistical Methods Bayesian Analysis of The Calibration Sample The Effect of 1 Bayesian Statistical Methods Components of a Statistical Model Statistical Computation 2 The Calibration Sample The Effect of 3 Bayesian Analysis of Pragmatic and Fully Bayesian Solutions Empirical Illustration David A van Dyk

13 Bayesian Analysis of The Basic Statistical Model The Calibration Sample The Effect of Computer Models Background Basic Physics Parametric Models Photon Absorption Observed Data (mean=λ) Multi-Scale Models High-Energy Telescope Effective Area (ARF) Blurring (RMF) Optical Telescope Gaussian Measurement Errors Embed physical models into multi-level statistical models Must account for complexities of data generation State of the art computational techniques enable us to fit the resulting highly-structured model David A van Dyk

14 Bayesian Analysis of Calibration Products The Calibration Sample The Effect of Analysis is highly dependent on Calibration Products: Effective Area Cuves Energy Redistribution Matrices Exposure Maps Point Spread Functions In this talk we focus on uncertainty in the effective area ACIS S effective area (cm 2 ) E [kev] A Chandra effective area Sample Chandra psf s (Karovska et al, ADASS X) EGRET exposure map (area time) David A van Dyk

15 Bayesian Analysis of Calibration Sample The Calibration Sample The Effect of Calibration sample of 1000 representative curves 1 Sample exhibits complex variability Requires storing many high dimensional calibration products ACIS S effective area (cm 2 ) default subtracted effective area (cm 2 ) E [kev] E [kev] 1 Thanks to Jeremy Drake and Pete Ratzlaff David A van Dyk

16 Bayesian Analysis of The Calibration Sample The Effect of Generating Calibration Products on the Fly We use Principal Component Analysis to represent uncertainty: A A 0 δ m e j r j v j, j=1 A 0 : default effective area, δ: mean deviation from A 0, r j and v j : first m principle component eigenvalues & vectors, e j : independent standard normal deviations Capture 99% of variability with m = 18 David A van Dyk

17 Bayesian Analysis of Checking the PCA Emulator The Calibration Sample The Effect of A rep! A o (cm 2 ) E (kev) Density kev A (cm 2 ) Density kev A (cm 2 ) Density kev A (cm 2 ) David A van Dyk

18 Bayesian Analysis of The Simulation Studies The Calibration Sample The Effect of Simulated Spectra Spectra were simulated using an absorbed power law, f (E j ) = αe N Hσ(E j ) E Γ j, accounting for instrumental effects; E j is the energy of bin j Parameters (Γ and N H ) and sample size/exposure times: Effective Area Nominal Counts Spectal Model Default Extreme Harder Softer SIM 1 X X X SIM 2 X X X SIM 3 X X X An absorbed powerlaw with Γ = 2, N H = /cm 2 An absorbed powerlaw with Γ = 1, N H = /cm 2 David A van Dyk

19 Bayesian Analysis of The Calibration Sample The Effect of 30 Most Extreme Effective Areas in Calibration Sample A! A (cm 2 ) 50 o E (kev) 15 largest and 15 smallest determined by maximum value David A van Dyk

20 Bayesian Analysis of The Calibration Sample The Effect of The Effect of SIMULATION 1 SIMULATION 2 Density N H(10 cm "2 ) Density ! ! Density N H(10 cm "2 ) Density ! ! Columns represent two simulated spectra True parameters are horizontal lines Posterior under default calibration is plotted in black The posterior is highly sensitive to the choice of effective area! N H(10 22 cm "2 ) N H(10 22 cm "2 ) David A van Dyk

21 Bayesian Analysis of The Effect of Sample Size The Calibration Sample The Effect of Density counts Γ Density counts Γ Density N H(10 22 cm 2 ) N H(10 22 cm 2 ) The effect of is more pronounced with larger sample sizes Density David A van Dyk

22 Outline Bayesian Statistical Methods Bayesian Analysis of Pragmatic and Fully Bayesian Solutions Empirical Illustration 1 Bayesian Statistical Methods Components of a Statistical Model Statistical Computation 2 The Calibration Sample The Effect of 3 Bayesian Analysis of Pragmatic and Fully Bayesian Solutions Empirical Illustration David A van Dyk

23 Bayesian Analysis of Pragmatic and Fully Bayesian Solutions Empirical Illustration Accounting for We consider inference under: A PRAGMATIC BAYESIAN TARGET: π 0 (A, θ) = p(a)p(θ A, Y ) THE FULLY BAYESIAN POSTERIOR: π(a, θ) = p(a Y )p(θ A, Y ) Here: p(θ A, Y ): PyBLoCXS posterior with known effective area, A p(a Y ): The posterior distribution for A p(a): The prior distribution for A Should we let the current data inform inference for calibration products? David A van Dyk

24 Bayesian Analysis of Two Possible Target Distributions Pragmatic and Fully Bayesian Solutions Empirical Illustration We consider inference under: A PRAGMATIC BAYESIAN TARGET: π 0 (A, θ) = p(a)p(θ A, Y ) THE FULLY BAYESIAN POSTERIOR: π(a, θ) = p(a Y )p(θ A, Y ) Concerns: Statistical Fully Bayesian target is correct Cultural Astronomers have concerns about letting the current data influence calibration products Computational Both targets pose challenges, but pragmatic Bayesian target is easier to sample Practical How different are p(a) and p(a Y )? David A van Dyk

25 Density Bayesian Statistical Methods Bayesian Analysis of Pragmatic and Fully Bayesian Solutions Empirical Illustration Using Multiple Imputation to Account for Uncertainty N H (10 22 cm "2 ) Density ! ! N H (10 22 cm "2 ) Density N H (10 22 cm "2 ) MI is a simple, but approximate, method: Density ! ! Fit the model M 1000 times, each with random effective area curve from the calibration sample Estimate parameters by averaging the M fitted values N H (10 22 cm "2 ) Estimate uncertainty by combining the within and between fit errors David A van Dyk

26 Bayesian Analysis of Densi Pragmatic and Fully Bayesian Solutions Empirical Illustration The Multiple Imputation Combining Rules B = 1 M 1 N H (10 22 cm "2 ) ˆθ = 1 M Density ! M ˆθ m, m= N H (10 22 cm "2 )! W = 1 M Densi 0 20 N H (10 22 cm "2 ) M Var(ˆθ m ), m=1 M (ˆθ m ˆθ)(ˆθ m ˆθ), T = W m=1 Density ( 1 1 M ! ! ) B The total variance combines the variances within and between the M analyses N H (10 22 c David A van Dyk

27 Bayesian Analysis of Pragmatic and Fully Bayesian Solutions Empirical Illustration Accounting for with MCMC When using MCMC in a Bayesian setting we can: Sample a different effective area from the calibration sample at each iteration according to: Pragmatic Bayes: p(a) Fully Bayes: p(a Y ) Computational challenges arise in both cases We focus on comparing the results of the two methods A Simulation Sampled 10 5 counts from a power law spectrum: E 2 A true is 15σ from the center of the calibration sample David A van Dyk

28 Bayesian Analysis of Sampling From the Full Posterior Pragmatic and Fully Bayesian Solutions Empirical Illustration Default Effective Area Pragmatic Bayes Fully Bayes θ θ θ θ 1 Spectral Model (purple bullet = truth): θ 1 θ 1 f (E j ) = θ 1 e θ 1σ(E j ) E θ 2 j Pragmatic Bayes is clearly better than current practice, but a Fully Bayesian Method is the ultimate goal David A van Dyk

29 Bayesian Statistical Methods Bayesian Analysis of Pragmatic and Fully Bayesian Solutions Empirical Illustration The Effect in the Analysis of a Binary System T 1 T 2 Standard T 1 T 2 Pragmatic Bayes T 1 T 2 Fully Bayes T T T 1 Fitting ζ Ori with a Multithermal Spectral Model David A van Dyk

30 Bayesian Analysis of Learning the Effective Area Pragmatic and Fully Bayesian Solutions Empirical Illustration A [cm 2 ] energy [kev] A A 0 [cm 2 ] energy [kev] David A van Dyk

31 Bayesian Analysis of Pragmatic and Fully Bayesian Solutions Empirical Illustration Results: 95% Intervals Standardized by Standard Fit photon counts photon counts (CI pb Γ^std) σ^std(γ) (CI fb Γ^std) σ^std(γ) With high counts calibration uncertainty swamps statistical error and fully Bayes identifies A and shifts interval David A van Dyk

32 Thanks Bayesian Statistical Methods Bayesian Analysis of Pragmatic and Fully Bayesian Solutions Empirical Illustration Collaborative work with: Vinay Kashyap Jin Xu Alanna Connors Hyunsook Lee Aneta Siegminowska California-Harvard Astro-Statistics Collaboration If you want more details Lee, H, Kashyap, V, van Dyk, D, Connors, A, Drake, J, Izem, R, Min, S, et al Accounting for Calibration Uncertainties in X-ray [Spectral] Analysis The Astrophysical Journal, 731, , 2011 Xu, J, van Dyk, D, Kashyap, V, Siemiginowska, A, Connors, A, Drake, J, et al A Fully Bayesian Method for Jointly Fitting Calibration and X-ray Spectral Models The Astrophysical Journal, 794, 97, 2014 David A van Dyk