Bayesian Model Selection & Extrasolar Planet Detection

Transcription

1 Bayesian Model Selection & Extrasolar Planet Detection Eric B. Ford UC Berkeley Astronomy Dept. Wednesday, June 14, 2006 SCMA IV SAMSI Exoplanets Working Group: Jogesh Babu, Susie Bayarri, Jim Berger, Floyd Bullard, David Chernoff, Merlise Clyde, Pablo de la Cruz, Andrew Cumming, Gauri Datta, Peter Driscoll, Eric Feigelson, Debra Fischer, Eric Ford, Phil Gregory, Bill Jefferys, Tom Jeffreys, Michael Last, Hyunsook Lee, Jaeyong Lee, Tom Loredo, Barbara McArthur, Raman Narayan, Jeff Scargle, Alex Wolszczan

2 Observed Planetary Systems 1543: Copernicus: Revolutionibus 1576: Digges: Universe infinite? 1600: Bruno burned 1604: Kepler's Supernova 1609: Galileo's telescope 1618: Kepler's 3 rd law 1687: Newton: Principia 1698: Huygens: Distance to Sirius 1755: Kant on planet formation 1781: Herschel: Uranus 1796: Laplace on planet formation 1838: Parallax measured 1846: Adams & Le Verrier: Neptune 1925: Hubble: Cepheids in nebulae 1926: Eddington: Sun's energy 1930: Tombaugh: Pluto NASA

3 Observed Planetary Systems 1993: Wolszczan: PSR : First planets around a pulsar! 1995: Mayor & Queloz: 51 Pegasi: First planet around a solar-type star 1999: Marcy: Upsilon Andromedae: First multiple planet system around a solar-type star 2000: ~50 Planetary Systems 2006: ~155 Planetary Systems

4 Motivation Are planetary systems like our own common/rare? Are Giant Planets like Jupiter & Saturn common? Are Terrestrial Planets common? In the Habitable Zone? With atmospheres suitable for life? Understand Formation of Planetary Systems What is the influence of star & environment? Did planets form from accretion of small bodies? What is the role of planet-disk interactions? What is the role of multiple planet systems?

5 Outline Motivation How to Detect Extrasolar Planets Example Radial Velocity Observations Identifying Potential Orbital Periods Parameter Estimation (MCMC) Value of Bayesian Methods Model Selection: Several Estimators Theory Performance on Test Case Conclusions Artwork copyright Lynette Cook

6 Direct Detection Close et al. Chauvin et al. Charbonneau et al. Neuhauser et al.

7 Motion of the Sun Parallax & Proper Motion Solar System Arcsec 0 1 NASA

8

9 Radial Velocity of the Sun Marcy

10 California & Carnegie Planet Search Keck Lick Obs. Anglo-Aus. Aus. Tel. Marcy

11 Starlight From Telescope High-Resolution Echelle Spectrometer Echelle Spectrometer CCD Echelle Grating Collimator Marcy

12 Spectrum of Star: Doppler Effect 4096 Pixels Doppler Precision: v / c ~ 10-9 Δλ / λ ~ 10-9 Marcy

13 Example Sections of Stellar Spectra (Template) Intensity Fischer & Valenti 2005

14 Wavelength Calibration 8 Significant Digits! Echelle Spectrometer Resolution: 60,000 Iodine Abs. Cell. Superimpose I 2 lines Wavelength Calibration Marcy

15 Visible Spectrum of Iodine Chapman

16 Current Doppler Precision: ~1.0 m/s Aug 2004 Marcy

17 Sources of Noise Measurement Uncertainty Undetected Planets Stellar Oscillations Stellar Activity (e.g., spots, flares, convection)

18

19 Radial Velocity Signature of a Planet (assuming unperturbed Keplerian orbit) This proceedings

20

21 The Current State of the Art Observational Data Identifying Possible Orbital Periods Parameter Estimation Value of Bayesian Approach Model Selection: Several Estimators Theory Performance on Test Case Conclusions

22 Example Short-Period Planet: HD Fischer et al. 2005

23 Example Short-Period Planet: HD This proceedings

24 Example Short-Period Planet: HD Period Amplitude Phase Constant Jitter Eccentricity Orientation This proceedings

25 Challenge of Long Period Orbits Points Best-fit Orbital Solution HD Published orbital solution & error bar Updated best-fit orbits ( Contours (1, 2, 3-σ) Bootstrap Markov chain Monte Carlo Conclusion: Large degeneracies when observations span < 1-2 orbital periods. Butler et al Orbital Eccentricity Posterior Probability Contours HD Ford 2005

26 Value of Bayesian Parameter Estimation Points Best-fit Orbital Solution HD Published orbital solution & error bar Updated best-fit orbits ( Contours (1, 2, 3-σ) Bootstrap Markov chain Monte Carlo Add Just One Observation! Conclusion: Large degeneracies when observations span < 1-2 orbital periods. Orbital Eccentricity Posterior Probability Contours HD Ford 2005

27 Value of Bayesian Parameter Estimation Points Best-fit Orbital Solution HD Published orbital solution & error bar Updated best-fit orbits ( Contours (1, 2, 3-σ) Bootstrap Markov chain Monte Carlo Several More Observations Conclusion: Large degeneracies when observations span < 1-2 orbital periods. Orbital Eccentricity Posterior Probability Contours HD Ford 2005

28 Why Bootstrap Fails for Long-Period Planets: Example χ 2 Surfaces Ford 2005

29 Why Model Selection? Rigorous Bayesian Planet Detection When Detect Single Planet? When Detection 2, 3, 4 Planets? Improve Sensitivity (Relative to frequentist tests): To Low Mass Planets Long Period Planets With First Few Observations Adaptive Scheduling Increase evidence for planets with fewer extra observations Improve precision of estimates of model parameters (important for studying dynamics of multi-planet systems)

30 Bayesian Model Selection Posterior Odds Ratio: p(m 2 v,i)/p(m 1 v,i) = (Prior Odds Ratio) (Bayes Factor) Prior Odds Ratio = p(m 2,I) /p(m 1,I) Bayes Factor = m(m 2 )/m(m 1 ) = p(v M 2,I)/p(v M 1,I) Includes Occam s Razor Factor: p(θ 2 M 2,I) / p(θ 1 M 1,I) Want to estimate marginal posterior probability

31 Choice of Priors This proceedings

32 Estimators for Marginal Posterior Basic Monte Carlo: Restricted Monte Carlo: This proceedings

33 Test Case for Model Selection Algorithms HD 88133b Fischer et al This proceedings

34 Estimators for Marginal Posterior Basic Monte Carlo: Restricted Monte Carlo: This proceedings

35 Comparing Estimators of Marginal Restricted Monte Carlo Ford This proceedings

36 Estimators for Marginal Posterior Basic Monte Carlo: Restricted Monte Carlo: Linearized Model + Laplace Approximation This proceedings

37 Comparing Estimators of Marginal Restricted Monte Carlo Linearized Laplace Aprox. Ford This proceedings

38 Estimators for Marginal Posterior Basic Monte Carlo: Restricted Monte Carlo: Weighted Harmonic Mean: This proceedings

39 Comparing Estimators of Marginal Harmonic Mean Restricted Monte Carlo Linearized Laplace Aprox. Ford This proceedings

40 Comparing Estimators of Marginal Harmonic Mean Linearized Weighted Harmonic Mean Restricted Monte Carlo Linearized Laplace Aprox. Ford This proceedings

41 Estimators for Marginal Posterior Standard Importance Sampling: Unimodal: g(θ) ~ N(θ o, Σ)or g(θ) ~ T 4 (θ o, Σ) This proceedings

42 Comparing Estimators of Marginal Harmonic Mean Linearized Weighted Harmonic Mean Importance Sampling Restricted Monte Carlo Linearized Laplace Aprox. Ford This proceedings

43 Comparing Estimators of Marginal Importance Sampling Ford This proceedings

44 Example Short-Period Planet: HD Period Amplitude Phase Constant Jitter Eccentricity Orientation This proceedings

45 Estimators for Marginal Posterior Standard Importance Sampling: Unimodal: g(θ) ~ N(θ o,σ)or g(θ) ~ T 4 (θ o,σ) Mixture for Sampling Distribution: E.g., components centered on samples from posterior: Defensive Importance Sampling: This proceedings

46 Comparing Estimators of Marginal Importance Sampling Unimodal Importance Sampling Mixture Ford This proceedings

47 Comparing Estimators of Marginal Importance Sampling Mixture Weighted Harmonic Mean Estimator Importance Sampling Mixture Ford This proceedings

48 Estimators for Marginal Posterior Ratio Estimator: This proceedings

49 Comparing Estimators of Marginal Ratio Estimator Importance Sampling Mixture Weighted Harmonic Mean Estimator Importance Sampling Mixture Ford This proceedings

50 Estimators for Marginal Posterior Ratio Estimator: Parallel Tempering (Gregory 2005) π β (θ) = p(θ M 1 ) L(v θ,m 1 ) β, 0 < β < 1 log[ m(θ) ] = dβ π β (θ)

51 Comparing Estimators of Marginal Multiple Runs with Parallel Tempering (34 tempering levels) Gregory This proceedings

52 Internal Error Estimates Linearized Laplace Aprox. Restricted Monte Carlo Mixture Weighted Harmonic Mean Estimator Importance Sampling (Mixture) Ratio Estimator Importance Sampling (Unimodal) Ford This proceedings

53 Future Research Importance Sampling: Performs well when Appropriate Sampling Density Improve Algorithms for Constructing Importance Sampling Densities Automatically from Posterior Sample Notable Untested Algorithms: Nested Sampling (Skilling 2005; Bullard & Clyde) Trans-dimensional MCMC This proceedings

54 Conclusions Rapid progress in observations revolutionizing study of planetary systems and planet formation Many Bayesian methods already useful: Identifying orbital periods (Linearizing models & Laplce approximation replacing periodogram) Parameter estimation (MCMC replacing Bootstrap) Adaptive scheduling (Predictive distribution replacing by eye scheduling) Model selection remains a major challenge Importance sampling very good option when good importance sampling density How to construct sampling density?