Hierarchical Bayesian Modeling with Approximate Bayesian Computation

Size: px

Start display at page:

Download "Hierarchical Bayesian Modeling with Approximate Bayesian Computation"

Gyles Richard
5 years ago
Views:

1 Hierarchical Bayesian Modeling with Approximate Bayesian Computation Jessi Cisewski Carnegie Mellon University June 2014

2 Goal of lecture Bring the ideas of Hierarchical Bayesian Modeling (HBM), Approximate Bayesian Computation (ABC), importance sampling, and sequential ABC together in an astronomy-motivated application to exoplanets.

3 Collaborators Eric Ford Megan Shabram Chad Schafer SAMSI ExoStat Group

Planet detection Figure: http://en.wikipedia.

4 Planet detection Figure:

5 Kepler field of view

6 Kepler field of view (pre )

7 Each exoplanet, i, has the following parameters: w i (R sa, R pa, κ i, Π i, φ i, e i, ω i ) R si R pi = star radius = planet radius κ i = amplitude of velocity model Π i = orbital period φ i = orbital phase at fiducial time e i = orbital eccentricity ω i = longitude of perihelion (orientation of ellipse relative to the sky)

8 Each exoplanet, i, has the following parameters: w i (R sa, R pa, κ i, Π i, φ i, e i, ω i ) R si R pi = star radius = planet radius κ i = amplitude of velocity model Π i = orbital period φ i = orbital phase at fiducial time e i = orbital eccentricity ω i = longitude of perihelion (orientation of ellipse relative to the sky)

9 Orbital eccentricity Planet Eccentricity Mercury Venus Earth Mars Jupiter Saturn Uranus Neptune

10 Orbital eccentricity

11 Eccentricity: observables There is some true (h i, k i ) for planet i such that ( ) (( ) ( hi 0 σ 2 [some parameters] N, ci 0 k i 0 0 σc 2 i )) where h i = e i cos( ω i ), k i = e i sin( ω i ) = hi 2 + ki 2 ω i U[0, 2π) But we observe (h i, k i ) with measurement error ( ) (( ĥi hi [some parameters] N ˆk i k i ) ( σ 2, h 0 0 σk 2 = e 2 i ))

12 Eccentricity: [some parameters] Given a sample of exoplanets p 1,..., p n, we assume the following 1 N m populations (assume N m = 1, 2, or 3) 2 Classify each population as c 1,..., c Nm 3 Each population has a certain true proportion f j, j = 1,..., N m and N m j=1 f j = 1

13 Bayesian hierarchical model for eccentricity ( hi k i N m = # populations (fixed and known ) f Dirichlet(1 Nm ) c i f iid multinomial(f ) ) c i, σ 2 c i N ( ĥi ˆk i ) c i, σ 2 c i, h i, k i N iid σ ci U(0, 1) (( 0 0 (( hi k i ), ), ( σ 2 ci 0 0 σc 2 i ( ˆσ 2 h 0 0 ˆσ 2 k )) ))

14 3D Dirichlet Distribution Image:

15 Beta Distribution f (y α, β) = Γ(α + β) Γ(α)Γ(β) y α 1 (1 y) β 1 Image:

16 Recall the Basic ABC algorithm For the observed data y 1:n, prior π(θ) and distance function ρ: Algorithm 1 Sample θ from prior π(θ) 2 Generate x 1:n from forward process f (y θ ) 3 Accept θ if ρ(y 1:n, x 1:n ) < ɛ 4 Return to step 1 Generates a sample from an approximation of the posterior: f (x 1:n ρ(y 1:n, x 1:n, θ) < ɛ) π(θ) f (y 1:n θ)π(θ) π(θ y 1:n )

17 Summary of basic ABC Decisions that need to be made: 1 Distance function (ρ) 2 Summary statistic(s) 3 Tolerance (ɛ) Finding the right ɛ can be inefficient we end up throwing away many of the theories proposed from the selected priors use sequential sampling to improve efficiency

18 Each exoplanet, i, has the following parameters: w i (R sa, R pa, κ i, Π i, φ i, e i, ω i ) R si R pi = star radius = planet radius κ i = amplitude of velocity model Π i = orbital period φ i = orbital phase at fiducial time e i = orbital eccentricity ω i = longitude of perihelion (orientation of ellipse relative to the sky)

19 The observations The observations are simulated from the forward process (i.e. the hierarchical model) Two populations of exoplanets (N m = 2) True parameter values: 1 Standard deviations of h s and k s for each population: (σ 1, σ 2 ) = (0.05, 0.30) 2 Weights of each population: (f 1, f 2 ) = (.7,.3) n = 500 observations

20 for(i in 1:N){ while(d>epsilon0[t]) { # prior on (f1,f2) ~ Dirichlet(1) gamma0 = rgamma(nm,shape = alpha0,scale = 1) f.proposed0 = gamma0/sum(gamma0) # prior on (sig1, sig2) ~ Uniform(0,1) sigma.proposed0 = runif(nm) index0 = sample.int(nm, size = n, replace = TRUE, prob = f.proposed0) sigma.proposed = sigma.proposed0[index0] # generate the h s and k s theta1.h = sigma.proposed*rnorm(n) theta1.k = sigma.proposed*rnorm(n) # add measurement error theta1.hhat = theta1.h + sigma.hhat*rnorm(n) theta1.khat = theta1.k + sigma.khat*rnorm(n) # check the distance x[,1] = theta1.hhat x[,2] = theta1.khat d = distance.function(x, data0) } # if distance <= epsilon, then keep d0[i,t] = d sigma.final[i,,t]= sigma.proposed0 f.final[i,,t]= f.proposed0 gamma.sum[i,t] = sum(gamma0) }

21 Draw from the prior # prior on (f1,f2) ~ Dirichlet(1) gamma0 = rgamma(nm,shape = alpha0,scale = 1) f.proposed0 = gamma0/sum(gamma0) # prior on (sig1, sig2) ~ Uniform(0,1) sigma.proposed0 = runif(nm) index0 = sample.int(nm, size = n, replace = TRUE, prob = f.proposed0) sigma.proposed = sigma.proposed0[index0]

22 Draw from the prior # prior on (f1,f2) ~ Dirichlet(1) gamma0 = rgamma(nm,shape = alpha0,scale = 1) f.proposed0 = gamma0/sum(gamma0) # prior on (sig1, sig2) ~ Uniform(0,1) sigma.proposed0 = runif(nm) index0 = sample.int(nm, size = n, replace = TRUE, prob = f.proposed0) sigma.proposed = sigma.proposed0[index0]

23 Draw from the prior # prior on (f1,f2) ~ Dirichlet(1) gamma0 = rgamma(nm,shape = alpha0,scale = 1) f.proposed0 = gamma0/sum(gamma0) # prior on (sig1, sig2) ~ Uniform(0,1) sigma.proposed0 = runif(nm) index0 = sample.int(nm, size = n, replace = TRUE, prob = f.proposed0) sigma.proposed = sigma.proposed0[index0]

24 Forward model # generate the h s and k s theta1.h = sigma.proposed*rnorm(n) theta1.k = sigma.proposed*rnorm(n) # add measurement error theta1.hhat = theta1.h + sigma.hhat*rnorm(n) theta1.khat = theta1.k + sigma.khat*rnorm(n) # check the distance x[,1] = theta1.hhat x[,2] = theta1.khat d = distance.function(x, data0) }

25 Distance function: Kolmogorov-Smirnov distance

26 Distance function: Kolmogorov-Smirnov distance

27 Distance function: Kolmogorov-Smirnov distance

28 Does the distance function really matter?

29 KS distance on e 2 (left); mean difference of e 2 (right) Drawing 100 particles

30 KS distance on e 2 (left); mean difference of e 2 (right) Drawing 100 particles

31 KS distance on e 2 (left); mean difference of e 2 (right) Drawing 100 particles

32 KS distance on e 2 (left); mean difference of e 2 (right) Drawing 100 particles

33 N = 500 particles Mean difference of e 2

34 N = 500 particles Mean difference of e 2 KS distance

35 Decreasing tolerances ɛ 1 ɛ T ABC - Population Monte Carlo algorithm (ABC - PMC) 1 At t = 1 For i = 1,..., N particles Generate θ (1) i π(θ) and x f (y θi 1) until ρ(y, x) < ɛ 1 Set w (1) i = N 1 2 At t = 2,..., T ( ) Set τt 2 = 2 var θ (t 1) 1:N For i = 1,..., N particles Draw θ i multinomial ( ) θ (t 1) 1:N, w (t 1) 1:N Generate θ (t) i θi N(θi, τ t 2 ) and x f (y θ (t) i ) until ρ(y, x) < ɛ t Set w (t) i π(θ (t) i )/ N j φ[τt 1 (θ (t) i θ (t 1) j )] φ( ) is the density function of a N(0, 1) From Beaumont et al. (2009) j=1 w (t 1)

36 KS -distance with N = 500 particles

37 KS -distance with N = 500 particles

38 KS -distance with N = 500 particles

39 KS -distance with N = 500 particles

40 KS -distance with N = 500 particles

41 KS -distance with N = 500 particles

42 KS -distance with N = 500 particles

43 KS -distance with N = 500 particles

44 KS -distance with N = 500 particles

45 KS -distance with N = 500 particles

46 KS -distance with N = 500 particles

47 Transition kernels.

48 Moving particles: { } N f (J) 1,..., f (J) Nm J=1

49 Moving particles: { } N f (J) 1,..., f (J) Nm J=1 f old = (f 1, f 2,..., f Nm ) (original mixture weights) f old Dir(α = (α 1, α 2,..., α Nm )) ξ i Gamma(α i, 1) = ξ + = ξ ξ Nm Gamma(α + = α i, 1) ( ξ1 f old =,..., ξ ) Nm Dir(α) ξ + ξ +

50 Moving particles: { } N f (J) 1,..., f (J) Nm J=1 Given weighting parameter p B i Beta(pα i, (1 p)α i ) ζ i Gamma((1 p)α i, 1) ξ i where ξ + = Nm i=1 ξ i. = ξ i B i + ζ i, i = 1,..., N m = ξi Gamma(α i, 1) ( ) ξ = 1 ξ+,..., ξ Nm ξ+ Dir(α)

51 Moving particles: { } N σ (J) 1,..., σ(j) Nm J=1

52 Recall: In a nutshell The basic idea behind ABC is that using a representative (enough) summary statistic η coupled with a small (enough) tolerance ɛ should produce a good (enough) approximation to the posterior... Marin et al. (2012)

53 Concluding remarks We considered examples related to Type Ia Supernova, Stellar IMF, Exoplanet Eccentricity Three main ABC decisions: summary statistic, distance function, tolerance Sequential ABC: need to specify a transition kernel and determine importance weights Approximate Bayesian Computation could be a useful tool in astronomy, but it should be used with care.

54 Concluding remarks We considered examples related to Type Ia Supernova, Stellar IMF, Exoplanet Eccentricity Three main ABC decisions: summary statistic, distance function, tolerance Sequential ABC: need to specify a transition kernel and determine importance weights Approximate Bayesian Computation could be a useful tool in astronomy, but it should be used with care. THANK YOU!!!

55 Additional useful resources Csilléry et al. (2010): Approximate Bayesian Computation (ABC) in practice Csillery et al. (2012): abc: an R package for approximate Bayesian computation (ABC) Jabot et al. (2013): EasyABC: performing efficient approximate Bayesian computation sampling schemes (R package)

56 Bibliography Beaumont, M. A., Cornuet, J.-M., Marin, J.-M., and Robert, C. P. (2009), Adaptive approximate Bayesian computation, Biometrika, 96, Csilléry, K., Blum, M. G., Gaggiotti, O. E., and François, O. (2010), Approximate Bayesian Computation (ABC) in practice, Trends in ecology & evolution, 25, Csillery, K., Francois, O., and Blum, M. G. B. (2012), abc: an R package for approximate Bayesian computation (ABC), Methods in Ecology and Evolution. Jabot, F., Faure, T., and Dumoullin, N. (2013), EasyABC: performing efficient approximate Bayesian computation. Marin, J.-M., Pudlo, P., Robert, C. P., and Ryder, R. J. (2012), Approximate Bayesian computational methods, Statistics and Computing, 22,

Bayes via forward simulation. Approximate Bayesian Computation

Bayes via forward simulation. Approximate Bayesian Computation : Approximate Bayesian Computation Jessi Cisewski Carnegie Mellon University June 2014 What is Approximate Bayesian Computing? Likelihood-free approach Works by simulating from the forward process What