Making precise and accurate measurements with data-driven models

Transcription

1 Making precise and accurate measurements with data-driven models David W. Hogg Center for Cosmology and Particle Physics, Dept. Physics, NYU Center for Data Science, NYU Max-Planck-Insitut für Astronomie, Heidelberg Flatiron Institute, Simons Foundation, New York City Hunstead Lecture 2017 November 9

2 conclusions A data-driven model measures physical parameters of stars with better quality than any physics-driven pipeline. The Cannon no physics is harmed Connections to other physical systems and models. Connections to extra-solar-planet and Milky-Way science. Criticisms of vanilla machine learning. Everything open-source or public-domain. Melissa Ness (MPIA), Andy Casey (Cambridge), Anna Y. Q. Ho (Caltech), Lauren Anderson (Flatiron), Hans-Walter Rix (MPIA)

3 De-noising (Anderson)

4 Annie Jump Cannon O B A F G K M temperature sequence! alphabetical order is hydrogen-line-strength order Cannon understood the temperature sequence of stars without the benefit of physical models data-driven non-linear dimensionality reduction manifold learning (using a huge amount of prior knowledge) namesake of The Cannon

5 the paradoxes of contemporary physics models are incredibly explanatory QCD, ΛCDM, helioseismology and yet...

6 the paradoxes of contemporary physics models are incredibly explanatory QCD, ΛCDM, helioseismology and yet... models are wrong (ruled out) in detail χ 2 ν The χ 2 statistic is a measure of the size of your data! data are abundant and precise

7 the paradoxes of contemporary physics models are incredibly explanatory QCD, ΛCDM, helioseismology and yet... models are wrong (ruled out) in detail χ 2 ν The χ 2 statistic is a measure of the size of your data! data are abundant and precise missing physics, approximation, gastrophysics models are fundamentally computational

8 context: Galactic archaeology stars populate orbits in the Milky Way conserved actions (or chaotic equivalents) stars are formed from particular gas clouds stars have conserved surface abundances the combined action-chemical space will be far more informative than either taken independently

9 context: Galactic archaeology top priority for many new projects Gaia & Gaia-ESO HERMES & GALAH SDSS-III APOGEE terrifying inconsistencies in current approaches models of stars are amazingly good but chemical signatures are incredibly tiny

10 context: extra-solar planets planets are measured relative to their host stars transits radial-velocity signals astrometric signals some planet measurements are now very precise need stellar properties for accuracy

11 definition: physics-driven models (my usage) put in everything you know gravity, atomic and molecular transitions, radiation make approximations to make things computable sub-grid models, mixing length, etc. compute like hell

12 definition: machine learning (my usage) the most extreme of data-driven models the data is the model none of your knowledge is relevant learn (fit) an exceedingly flexible model explain or cluster the data transformation from data to labels concept of non-parametrics concept of train, validate, and test many packages and implementations (and outrageous successes)

13 When does (vanilla) machine learning help you? train & test situation training data are statistically identical to the test data same noise amplitude same distance or redshift distribution same luminosity distribution never true! training data have accurate and precise labels therefore, we can t use vanilla machine learning! (physicists rarely can)

14 definition: data-driven models (my usage) make use of things you strongly believe noise model & instrument resolution causal structure (shared parameters) capitalize on huge amounts of data exceedingly flexible model concept of train, validate, and test every situation will be bespoke

15 label transfer for stars a few of your stars have good labels (from somewhere) can you use this to label the other stars? why would you want to do this?

16 label transfer for stars a few of your stars have good labels (from somewhere) can you use this to label the other stars? why would you want to do this? you don t have good models at your wavelengths? you want two surveys to be on the same system? you have some stars at high SNR, some at low SNR? you spent human time on some stars but can t on all?

17 stellar spectra stars are very close to black-bodies to first order, a stellar spectrum depends on effective temperature T eff, surface gravity log g, and metallicity [Fe/H]

18 stellar spectra stars are very close to black-bodies to first order, a stellar spectrum depends on effective temperature T eff, surface gravity log g, and metallicity [Fe/H] to second order, tens of chemical abundances, rotation, turbulence, activity

19 stellar spectra all chemical information is in absorption lines corresponding to atomic and molecular transitions some 30 elements are visible in the best stars spectroscopy at is the primary tool R λ > 20, 000 λ

20 stellar astrophysics A B Teff = 4750, log g = 3.0, [Fe/H] = 0.15 Teff = 4849, log g = 2.2, [Fe/H] = -1.0 normalized flux f Teff = 3614, log g = 0.4, [Fe/H] = Teff = 5003, log g = 2.8, [Fe/H] = wavelength λ (Å)

21 stellar astrophysics

22 SDSS-III APOGEE Galactic archaeology APOGEE DR12 & DR13: 156,000 stars (98,000 giants) R = 22, 500 spectra in 1.5 < λ < 1.7 µm precise RVs and stellar parameters abundances per star (our own home-built and special continuum normalization; ask me!) all data completely public!

23 SDSS-III APOGEE A B Teff = 4750, log g = 3.0, [Fe/H] = 0.15 Teff = 4849, log g = 2.2, [Fe/H] = -1.0 normalized flux f Teff = 3614, log g = 0.4, [Fe/H] = Teff = 5003, log g = 2.8, [Fe/H] = wavelength λ (Å)

24 train, validate, and test split the data into three disjoint subsets in the training step you set the parameters of your model using the training set the validation set is used to set hyperparameters or model complexity in the test step you apply the model to the test set new data to make predictions or deliver results

25 The Cannon: Experiment 1: training set 543 stars (too few) from 19 clusters (too few) T eff, log g, [Fe/H] labels from APOGEE calling parameters and abundances labels slight adjustments to labels to get them onto possible isochrones terrible coverage of the main sequence only the Pleiades home-made Pleiades labels (by Ness) no [Fe/H] spread at high log g.

26 The Cannon: Experiment 1; training set log g (dex) log g (dex) log g (dex) log g (dex) Gyr ( 2.35) 12 Gyr ( 2.33) 12.7 Gyr ( 2.06) 13 Gyr ( 1.98) 13 Gyr ( 1.78) M92 M15 M53 N5466 N Gyr ( 1.66) 11.7 Gyr ( 1.58) 11.5 Gyr ( 1.5) 13 Gyr ( 1.33) 13 Gyr ( 003) M2 M13 M3 M5 M Gyr ( 0.82) 1 Gyr ( 0.28) 2 Gyr ( 0.20) 5 Gyr ( 0.03) 3.2 Gyr ( 0.01) M71 N2158 N2420 N188 M Gyr (0.02) 0.15 Gyr (+0.03) 2.5 Gyr (+0.09) 5 Gyr (+0.47) N7789 Pleiades N6819 N Teff (K) Teff (K) Teff (K) Teff (K) Teff (K)

27 The Cannon: Experiment 1; training set log g (dex) log g (dex) log g (dex) log g (dex) Gyr ( 2.35) 12 Gyr ( 2.33) 12.7 Gyr ( 2.06) 13 Gyr ( 1.98) 13 Gyr ( 1.78) M92 M15 M53 N5466 N Gyr ( 1.66) 11.7 Gyr ( 1.58) 11.5 Gyr ( 1.5) 13 Gyr ( 1.33) 13 Gyr ( 003) M2 M13 M3 M5 M Gyr ( 0.82) 1 Gyr ( 0.28) 2 Gyr ( 0.20) 5 Gyr ( 0.03) 3.2 Gyr ( 0.01) M71 N2158 N2420 N188 M Gyr (0.02) 0.15 Gyr (+0.03) 2.5 Gyr (+0.09) 5 Gyr (+0.47) N7789 Pleiades N6819 N Teff (K) Teff (K) Teff (K) Teff (K) Teff (K)

28 The Cannon: model a generative model of the APOGEE spectra given label vector l, predict flux vector f probabilistic prediction p(f l, θ) use every spectral pixel s uncertainty variance σλn 2 responsibly details: spectral expectation is quadratic in the labels every wavelength λ treated independently an intrinsic Gaussian scatter s 2 λ at every wavelength λ 80,000 free parameters in θ!

29 The Cannon: model ln p(f n l n, θ) = L ln p(f λn l n, θ λ, s 2 λ) λ=1 ln p(f λn l n, θ λ, s 2 λ) = 1 [f λn θ λt l n] 2 2 σλn ln(σ s2 λn 2 + s 2 λ) λ l T {1, T eff, log g, [Fe/H], Teff, 2 T eff log g,, [Fe/H] 2} θ T { θ λ, s 2 } L λ λ=1

30 The Cannon: model ln p(f n l n, θ) training step: optimize w.r.t. parameters θ at fixed labels l using training-set data linear least squares every wavelength λ treated independently test step: optimize w.r.t. labels l at fixed parameters θ using test-set (survey) data non-linear optimization every star treated independently

31 The Cannon: model training

32 The Cannon: model training

33 The Cannon: model training cross-validation

34 The Cannon: results The Cannon is far faster than physical modeling model trains in seconds (thousands of fits) The Cannon labels 10 5 stars per hour (pure Python on a laptop) labels appear sensible The Cannon labels lie near sensible isochrones scatter against APOGEE labels consistent with APOGEE precision successfully puts labels on dwarfs

35 The Cannon: Experiment 1: comparison with ASPCAP labels

36 The Cannon: Experiment 1: label veracity

37 The Cannon: Experiment 1: label veracity

38 The Cannon: works at low signal-to-noise

39 The Cannon: works at low signal-to-noise

40 The Cannon: results The Cannon is far faster than physical modeling model trains in seconds (thousands of fits) The Cannon labels 10 5 stars per hour (pure Python on a laptop) labels appear sensible The Cannon labels lie near sensible isochrones scatter against APOGEE labels consistent with APOGEE precision successfully puts labels on dwarfs

41 The Cannon: shortcuts and choices no Bayes; no partial or noisy labels quadratic order replacing polynomial with a Gaussian process continuous model complexity; non-parametric spectral representation too-small training set only three labels age, [α/fe] splitting the giant branch how to go to many elements?

42 The Cannon: label transfer from APOGEE to LAMOST (Casey, Ho)

43 The Cannon: masses and ages for red giants (Ness)

44 The Cannon: masses and ages for red giants (Ness)

45 The Cannon: detailed abundances (Casey) T eff log g [Al/H] [Ca/H] 0.0 [C/H] 0.0 [Fe/H]

46 The Cannon: detailed abundances (Casey) 0.0 [Ca/H] 0.0 [C/H] 0.0 [Fe/H] [K/H] 0.0 [Mg/H] 0.0 [Mn/H] [Na/H] 0.0 [Ni/H] 0.0 [N/H]

47 The Cannon: detailed abundances (Casey) [Na/H] [Ni/H] [N/H] [O/H] [Si/H] [S/H]

48 The Cannon: detailed abundances (Casey) 0.0 [O/H] 0.0 [Si/H] 0.0 [S/H] [Ti/H] 0.0 [V/H]

49 The Cannon: detailed abundances (Ness)

50 The Cannon: detailed abundances (Ness)

51 The Cannon: detailed abundances (Ness)

52 lessons learned regressions are different from density estimators value of convex regularization

53 The Cannon: identification of lines (Casey) 1.0 [Al/H] [S/H] [K/H] 0.5 θ/max θ λ (Å)

54 The Cannon: discovery of outliers (Ho)

55 The Cannon: chemical tagging

56 The future: Unsupervised (Anderson)

57 The future: Unsupervised (Anderson)

58 read more original paper on The Cannon and APOGEE: Ness et al., arxiv: labeling LAMOST, RAVE: Ho et al., arxiv: , Casey et al., arxiv: chemical abundances: Casey et al., arxiv: , Ness et al., arxiv: red-giant masses and ages: Ness et al., arxiv: ; Ho et al., arxiv: chemical tagging: Hogg et al., arxiv: de-noising Gaia: Anderson et al., arxiv: Eilers in prep: working with missing and noisy labels Price-Whelan in prep: modeling spectroscopic binaries Bedell in prep: extreme-precision radial-velocity

59 conclusions A data-driven model measures physical parameters of stars with better quality than any physics-driven pipeline. The Cannon no physics is harmed Connections to other physical systems and models. Connections to extra-solar-planet and Milky-Way science. Criticisms of vanilla machine learning. Everything open-source or public-domain. Melissa Ness (MPIA), Andy Casey (Cambridge), Anna Y. Q. Ho (Caltech), Lauren Anderson (Flatiron), Hans-Walter Rix (MPIA)