Bayesian Model Selection & Extrasolar Planet Detection Eric B. Ford UC Berkeley Astronomy Dept. Wednesday, June 14, 2006 SCMA IV SAMSI Exoplanets Working Group: Jogesh Babu, Susie Bayarri, Jim Berger, Floyd Bullard, David Chernoff, Merlise Clyde, Pablo de la Cruz, Andrew Cumming, Gauri Datta, Peter Driscoll, Eric Feigelson, Debra Fischer, Eric Ford, Phil Gregory, Bill Jefferys, Tom Jeffreys, Michael Last, Hyunsook Lee, Jaeyong Lee, Tom Loredo, Barbara McArthur, Raman Narayan, Jeff Scargle, Alex Wolszczan
Observed Planetary Systems 1543: Copernicus: Revolutionibus 1576: Digges: Universe infinite? 1600: Bruno burned 1604: Kepler's Supernova 1609: Galileo's telescope 1618: Kepler's 3 rd law 1687: Newton: Principia 1698: Huygens: Distance to Sirius 1755: Kant on planet formation 1781: Herschel: Uranus 1796: Laplace on planet formation 1838: Parallax measured 1846: Adams & Le Verrier: Neptune 1925: Hubble: Cepheids in nebulae 1926: Eddington: Sun's energy 1930: Tombaugh: Pluto NASA
Observed Planetary Systems 1993: Wolszczan: PSR 1257+12: First planets around a pulsar! 1995: Mayor & Queloz: 51 Pegasi: First planet around a solar-type star 1999: Marcy: Upsilon Andromedae: First multiple planet system around a solar-type star 2000: ~50 Planetary Systems 2006: ~155 Planetary Systems www.exoplanets.org
Motivation Are planetary systems like our own common/rare? Are Giant Planets like Jupiter & Saturn common? Are Terrestrial Planets common? In the Habitable Zone? With atmospheres suitable for life? Understand Formation of Planetary Systems What is the influence of star & environment? Did planets form from accretion of small bodies? What is the role of planet-disk interactions? What is the role of multiple planet systems?
Outline Motivation How to Detect Extrasolar Planets Example Radial Velocity Observations Identifying Potential Orbital Periods Parameter Estimation (MCMC) Value of Bayesian Methods Model Selection: Several Estimators Theory Performance on Test Case Conclusions Artwork copyright Lynette Cook
Direct Detection Close et al. Chauvin et al. Charbonneau et al. Neuhauser et al.
Motion of the Sun Parallax & Proper Motion Solar System Arcsec 0 1 NASA
www.exoplanets.org
Radial Velocity of the Sun Marcy
California & Carnegie Planet Search Keck Lick Obs. Anglo-Aus. Aus. Tel. Marcy
Starlight From Telescope High-Resolution Echelle Spectrometer Echelle Spectrometer CCD Echelle Grating Collimator Marcy
Spectrum of Star: Doppler Effect 4096 Pixels Doppler Precision: v / c ~ 10-9 Δλ / λ ~ 10-9 Marcy
Example Sections of Stellar Spectra (Template) Intensity Fischer & Valenti 2005
Wavelength Calibration 8 Significant Digits! Echelle Spectrometer Resolution: 60,000 Iodine Abs. Cell. Superimpose I 2 lines Wavelength Calibration Marcy
Visible Spectrum of Iodine Chapman
Current Doppler Precision: ~1.0 m/s Aug 2004 Marcy
Sources of Noise Measurement Uncertainty Undetected Planets Stellar Oscillations Stellar Activity (e.g., spots, flares, convection)
www.exoplanets.org
Radial Velocity Signature of a Planet (assuming unperturbed Keplerian orbit) This proceedings
www.exoplanets.org
The Current State of the Art Observational Data Identifying Possible Orbital Periods Parameter Estimation Value of Bayesian Approach Model Selection: Several Estimators Theory Performance on Test Case Conclusions
Example Short-Period Planet: HD 88133 Fischer et al. 2005
Example Short-Period Planet: HD 88133 This proceedings
Example Short-Period Planet: HD 88133 Period Amplitude Phase Constant Jitter Eccentricity Orientation This proceedings
Challenge of Long Period Orbits Points Best-fit Orbital Solution HD 72659 Published orbital solution & error bar Updated best-fit orbits (www.exoplanets.org) Contours (1, 2, 3-σ) Bootstrap Markov chain Monte Carlo Conclusion: Large degeneracies when observations span < 1-2 orbital periods. Butler et al. 2003 Orbital Eccentricity Posterior Probability Contours HD 72659 Ford 2005
Value of Bayesian Parameter Estimation Points Best-fit Orbital Solution HD 72659 Published orbital solution & error bar Updated best-fit orbits (www.exoplanets.org) Contours (1, 2, 3-σ) Bootstrap Markov chain Monte Carlo Add Just One Observation! Conclusion: Large degeneracies when observations span < 1-2 orbital periods. www.exoplanets.org Orbital Eccentricity Posterior Probability Contours HD 72659 Ford 2005
Value of Bayesian Parameter Estimation Points Best-fit Orbital Solution HD 72659 Published orbital solution & error bar Updated best-fit orbits (www.exoplanets.org) Contours (1, 2, 3-σ) Bootstrap Markov chain Monte Carlo Several More Observations Conclusion: Large degeneracies when observations span < 1-2 orbital periods. www.exoplanets.org Orbital Eccentricity Posterior Probability Contours HD 72659 Ford 2005
Why Bootstrap Fails for Long-Period Planets: Example χ 2 Surfaces Ford 2005
Why Model Selection? Rigorous Bayesian Planet Detection When Detect Single Planet? When Detection 2, 3, 4 Planets? Improve Sensitivity (Relative to frequentist tests): To Low Mass Planets Long Period Planets With First Few Observations Adaptive Scheduling Increase evidence for planets with fewer extra observations Improve precision of estimates of model parameters (important for studying dynamics of multi-planet systems)
Bayesian Model Selection Posterior Odds Ratio: p(m 2 v,i)/p(m 1 v,i) = (Prior Odds Ratio) (Bayes Factor) Prior Odds Ratio = p(m 2,I) /p(m 1,I) Bayes Factor = m(m 2 )/m(m 1 ) = p(v M 2,I)/p(v M 1,I) Includes Occam s Razor Factor: p(θ 2 M 2,I) / p(θ 1 M 1,I) Want to estimate marginal posterior probability
Choice of Priors This proceedings
Estimators for Marginal Posterior Basic Monte Carlo: Restricted Monte Carlo: This proceedings
Test Case for Model Selection Algorithms HD 88133b Fischer et al. 2005 This proceedings
Estimators for Marginal Posterior Basic Monte Carlo: Restricted Monte Carlo: This proceedings
Comparing Estimators of Marginal Restricted Monte Carlo Ford This proceedings
Estimators for Marginal Posterior Basic Monte Carlo: Restricted Monte Carlo: Linearized Model + Laplace Approximation This proceedings
Comparing Estimators of Marginal Restricted Monte Carlo Linearized Laplace Aprox. Ford This proceedings
Estimators for Marginal Posterior Basic Monte Carlo: Restricted Monte Carlo: Weighted Harmonic Mean: This proceedings
Comparing Estimators of Marginal Harmonic Mean Restricted Monte Carlo Linearized Laplace Aprox. Ford This proceedings
Comparing Estimators of Marginal Harmonic Mean Linearized Weighted Harmonic Mean Restricted Monte Carlo Linearized Laplace Aprox. Ford This proceedings
Estimators for Marginal Posterior Standard Importance Sampling: Unimodal: g(θ) ~ N(θ o, Σ)or g(θ) ~ T 4 (θ o, Σ) This proceedings
Comparing Estimators of Marginal Harmonic Mean Linearized Weighted Harmonic Mean Importance Sampling Restricted Monte Carlo Linearized Laplace Aprox. Ford This proceedings
Comparing Estimators of Marginal Importance Sampling Ford This proceedings
Example Short-Period Planet: HD 88133 Period Amplitude Phase Constant Jitter Eccentricity Orientation This proceedings
Estimators for Marginal Posterior Standard Importance Sampling: Unimodal: g(θ) ~ N(θ o,σ)or g(θ) ~ T 4 (θ o,σ) Mixture for Sampling Distribution: E.g., components centered on samples from posterior: Defensive Importance Sampling: This proceedings
Comparing Estimators of Marginal Importance Sampling Unimodal Importance Sampling Mixture Ford This proceedings
Comparing Estimators of Marginal Importance Sampling Mixture Weighted Harmonic Mean Estimator Importance Sampling Mixture Ford This proceedings
Estimators for Marginal Posterior Ratio Estimator: This proceedings
Comparing Estimators of Marginal Ratio Estimator Importance Sampling Mixture Weighted Harmonic Mean Estimator Importance Sampling Mixture Ford This proceedings
Estimators for Marginal Posterior Ratio Estimator: Parallel Tempering (Gregory 2005) π β (θ) = p(θ M 1 ) L(v θ,m 1 ) β, 0 < β < 1 log[ m(θ) ] = dβ π β (θ)
Comparing Estimators of Marginal Multiple Runs with Parallel Tempering (34 tempering levels) Gregory This proceedings
Internal Error Estimates Linearized Laplace Aprox. Restricted Monte Carlo Mixture Weighted Harmonic Mean Estimator Importance Sampling (Mixture) Ratio Estimator Importance Sampling (Unimodal) Ford This proceedings
Future Research Importance Sampling: Performs well when Appropriate Sampling Density Improve Algorithms for Constructing Importance Sampling Densities Automatically from Posterior Sample Notable Untested Algorithms: Nested Sampling (Skilling 2005; Bullard & Clyde) Trans-dimensional MCMC This proceedings
Conclusions Rapid progress in observations revolutionizing study of planetary systems and planet formation Many Bayesian methods already useful: Identifying orbital periods (Linearizing models & Laplce approximation replacing periodogram) Parameter estimation (MCMC replacing Bootstrap) Adaptive scheduling (Predictive distribution replacing by eye scheduling) Model selection remains a major challenge Importance sampling very good option when good importance sampling density How to construct sampling density?