Improved Astronomical Inferences via Nonparametric Density Estimation

Transcription

1 Improved Astronomical Inferences via Nonparametric Density Estimation Chad M. Schafer, InCA Group Department of Statistics Carnegie Mellon University Work Supported by NSF, NASA-AISR Grant January

2 The Core Collaborators Susan M. Buchman Peter E. Freeman Ann B. Lee Joseph W. Richards 2

3 Motivation Theory predicts the distribution of observables as a function of cosmological parameters. 3

4 Motivation For example, Ω m = total matter density 0.40 Ω b = baryonic matter density Ω Λ = dark energy density 0.60 H 0 = the Hubble parameter 64.6 km/s/mpc τ = the optical depth n s = spectral index of initial spectrum 0.99 A = amplitude of initial spectrum 0.79 parameterize the power spectrum of the CMB anisotropy. 4

5 Motivation For example, Ω m = total matter density 0.40 Ω b = baryonic matter density Ω Λ = dark energy density 0.60 H 0 = the Hubble parameter 64.6 km/s/mpc τ = the optical depth n s = spectral index of initial spectrum 0.99 A = amplitude of initial spectrum 0.79 parameterize the power spectrum of the CMB anisotropy. 5

6 Motivation l(l + 1)Cl/2π (µk) Angular Frequency (l) 6

7 Motivation Image courtesy of WMAP Science Team. 7

8 Motivation The key role of Density estimation, i.e., estimating the distribution from which a sample of data were drawn Assuming a parametric form is convenient, but often difficult to justify. Nonparametric density estimation drops these restrictions 8

9 Nonparametric Density Estimation density redshift Histogram of 1,425 galaxy redshifts. 9

10 Nonparametric Density Estimation density redshift Compared with best fitting gamma distribution. 10

11 Nonparametric Density Estimation Data Estimate Assumptions Parametric case: Fixed contribution of assumptions. 11

12 Nonparametric Density Estimation Data Estimate Assumptions λ n Nonparametric case: Contribution of assumptions is controlled by λ n. Optimally, λ n = o(n 1/(4+d) ), where d = dimension of data. 12

13 Nonparametric Density Estimation Kernel density estimation puts a smooth mass at each data point. λ n controls the width of the bumps. 13

14 Nonparametric Density Estimation density redshift Parametric versus nonparametric estimate (kernel density estimate). 14

15 Nonparametric Density Estimation density redshift λ n chosen too small, i.e. too much weight on data 15

16 Nonparametric Density Estimation density redshift λ n chosen too large, i.e. too much weight on assumptions 16

17 Nonparametric Density Estimation f(x) x True Density Closest Gaussian Truth is not quite a Gaussian distribution. 17

18 Nonparametric Density Estimation Error (Mean Integrated Squared Error) 5.0e e e 05 Assuming Gaussian Nonparametric Estimator sample size (n) Even at moderate sample sizes, nonparametric estimator superior. 18

19 Bivariate Density Estimation x x Absolute Magnitude Sample of 15,057 SDSS quasars. (Richards, et al. 2006) Redshift 19

20 Bivariate Density Estimation x x Absolute Magnitude Bivariate luminosity function estimate (Schafer (2007)) Redshift 20

21 Bivariate Density Estimation Φ(M i ) (Mpc 3 mag 1 ) z = 0.49 z = 1.25 z = 2.01 Φ(M i ) (Mpc 3 mag 1 ) z = 2.8 z = 3.75 z = M i M i M i Cross-sections, compared with standard approach. 21

22 Working in Higher Dimensions Wavelength (Angstroms) SDSS galaxy spectrum. 22 Line Flux (10^ 17 erg/cm^2/s/angstrom)

23 Working in Higher Dimensions 5 x ψ 3 λ 3 t x t λ ψ λ 1 t 0 ψ x ,846 galaxy spectra, colored by redshift (Richards, Freeman, Lee, Schafer (2009a)) 23

24 Working in Higher Dimensions Examples of galaxy image data. 24

25 Working in Higher Dimensions Third Coordinate 4e 04 2e 04 0e+00 2e 04 4e Second Coordinate First Coordinate 200 galaxies, colored by eccentricity. 25

26 The Big Picture Distribution Space Encoding Space Data Space Component 3 Confidence/Credible Region Component 2 Physically Possible Distributions Component 1 Once represented in low-dimensional space encoding space, nonparametric density estimation useful for comparing observations and theory 26

27 References Buchman, Lee, and Schafer (2009). To appear in Statistical Methodology. arxiv: Richards, et al. (2006) ApJ Richards, Freeman, Lee, and Schafer (2009a). ApJ Schafer (2007). ApJ Schafer and Stark (2009). J. Amer. Stat. Assoc. 27