An adaptive SMC scheme for ABC. Bayesian Computation (ABC)

Transcription

1 An adaptve SMC scheme for Approxmate Bayesan Computaton (ABC) (ont work wth Prof. Mke West) Department of Statstcal Scence - Duke Unversty Aprl/2011

2 Approxmate Bayesan Computaton (ABC) Problems n whch lkelhood s ntractable but we can smulate the underlyng stochastc model So-called mplct statstcal models Allow great flexblty to model complex systems Applcatons n evolutonary bology, epdemology, systems bology.

3 ABC Algorthm 1 Draw θ from pror π(θ) 2 Smulate x f (x θ) 3 Accept θ f ρ(x, x obs ) < ε The resultng dstrbuton s π(θ ρ(x, x obs ) < ε) Exact posteror when ε = 0

4 ABC Illustraton

5 Approxmate Bayesan Computaton (ABC) Accuracy of the approxmaton controlled by the tolerance level ε Ideally, ε should be very small, but that mples low acceptance rate Two knds of methods proposed to mprove the effcency: automatc and post-samplng

6 Automatc e.g., ABC-MCMC, ABC-SMC Inputs before smulaton steps Post-samplng e.g., ABC-REG, ABC-GLM Analyss after smulaton steps Automatc methods rely on more effcent schemes to sample from π(θ ρ(x, x obs ) < ε) Post-samplng methods are based on some sort of regresson to correct sampled values and approxmate π(θ x obs )

7 A post-samplng approach: ABC-MIX Margnal data n bonetwork models: toggle swtch model (Bonass, You and West, 2011) Model Observaton y u = u T + μ + μση u/u γ T du dt = αu (1+v t βu ) dv dt = αv (1+u t βv ) (κu + δuut) + τuξu,t (κv + δvvt) + τvξv,t Independent nose processes η., ξ.

8 ABC-MIX for the toggle swtch model Massve pror:model smulaton large sample of (θ, y) Data characterzaton and dmenson reducton by means of sgnatures S(y) over a set of reference dstrbutons Constran the sample {θ, S} keepng the 5% closest synthetc datasets to S obs

9

10 ABC-MIX for the toggle swtch model Ft mxture model to the constraned sample {θ, S} (Suchard et al. 2010, Cron and West, 2011) Condtonal mxture g(θ S obs ) yelds approxmate posteror dstrbuton

11

12 ABC-SMC Automatc ABC approach based on Sequental Monte Carlo Man goal: mprove the acceptance rate of ABC by dvdng the problem nto subproblems (Ssson et al, 2007, Beaumont et al, 2009) In each step t obtan π(θ ρ(x, x obs ) < ε t ) for a decreasng tolerance schedule {ε 1,, ε T }

13 ABC-SMC Algorthm S1 Intalze ε 1 > > ε T S2 t = 1 Smulate θ (1) π(θ) and x f (x θ (1) ) untl ρ(x, x obs ) < ε 1 Set w = 1/N S3 t = 2,..., T Pck θ Generate θ (t) from the θ (t 1) K t(θ (t) s wth probabltes w (t 1) θ ) and x f (x θ (t) ) untl ρ(x, x obs ) < ε t Set w (t) w (t 1) π(θ (t) ) K t(θ (t) θ (t 1) )

14 Toy Example θ Unf ( 10, 10) Lkelhood: f (x θ) = 0.5 N(θ, 1) N(θ, 1/100) Goal: Approxmate posteror of θ for x obs = 0;

15 ABC SMC for tolerance schedule: ε 1 = 5, ε 2 = 1, ε 3 = 0.01 π(θ ρ(x, x obs ) < ε t ) where ρ s the eucldean dstance:

16 For N=5,000 partcles, number of data-generaton smulatons (n 10 3 ) n each step Step t ε t ABC-SMC ABC ,424 Total 770 4,424 The most expensve computatonal step s generally the model smulaton Beaumont et al. (2009) report 95% of tme spent n model smulaton for ther applcaton of ABC-SMC

17 Idea behnd ABC-SMC w (t 1) K t (θ (t) θ (t 1) for π(θ ρ(x, x obs ) < ε t 1 ) ) can be seen as a mxture approxmaton Ths approxmaton s then used as a proposal for π(θ ρ(x, x obs ) < ε t ) n order to acheve a better approxmaton In some sense, t follows the same deas of adaptve mportance samplng of West (1993)

18 Extendng the mxture approxmaton dea, we can approxmate π(x, θ ρ(x, x obs ) < ε t 1 ) by: g(x, θ) w (t 1) K t,x (x (t) x (t 1) )K t,θ (θ (t) θ (t 1) ) Ths s a more complete representaton of the ont dstrbuton of (x, θ), whch should nduce better proposals and better effcency

19 The new nduced approxmaton wll be: g(θ x obs ) K t,x (x obs x (t 1) )w (t 1) K t,θ (θ (t) θ (t 1) ) Whereas n the ABC-SMC t was: g(θ x obs ) w (t 1) K t,θ (θ (t) θ (t 1) )

20 Mxture approxmaton at step one (ε 1 = 5) usng ABC-SMC (blue) and ABC-SMC wth adaptve weghts (red)

21 ABC-SMC wth Adaptve Weghts S2 t = 1 Smulate θ (1) π(θ) and x f (x θ (1) ) untl ρ(x, x obs ) < ε 1 Set w = 1/N S3 t = 2,..., T Set weghts v (t 1) Normalze new weghts v (t 1) w (t 1) K t,x(x obs x (t 1) ) Pck θ from the θ (t 1) s wth probabltes v (t 1) Generate θ (t) K t,θ (θ (t) θ ) and x f (x θ (t) ) untl ρ(x, x obs ) < ε t Set w (t) v (t 1) π(θ (t) ) K t,θ (θ (t) θ (t 1) )

22 Comparson for the Normal toy example For N=5,000 partcles, number of data-generaton smulatons (n 10 3 ) n each step Step t ε t ABC-SMC ABC-SMC wth AW Total

23 ABC-SMC wth AW for the Toggle Swtch Problem Observaton

24 ABC-SMC wth AW for the Toggle Swtch Problem Each step: 10K of model smulaton steps and selecton of 10% closest datasets. Resultng tolerance schedule: ε 1:5 = (4.4, 3.5, 0.8, 0.4, 0.2) Ths way, the total number of data-generaton steps was 50K. For the prevous analyss, ABC-MIX, some dstance quantles (n 10 5 ) for a sample of 200K: q(10%) = 3.45 q(5%) = 2.95 q(1%) = 0.77

25 ABC-MIX: ABC-SMC wth AW:

26

27 Data-smulaton steps n ABC-MIX: 200K. Data-smulaton steps n ABC-SMC wth AW: 50K For the regular ABC-SMC, wth the same tolerance schedule, the number of generaton steps was: (10K, 10K, 23K, 23K, 27K, 35K) Total: 128K

28 ABC-SMC wth AW resulted n fnal effectve sample sze of 456 Generaton steps n ABC-SMC wth AW depend on the partcular real dataset. Then, the algorthm should be run separately for each one of the 10 real datasets In ABC-MIX, all 200K generatons are the same used for every real dataset

29 Another applcaton (from Ton et al. (2009)) Common-cold outbreak n the sland Trstan da Cunha (1967) day I(t) R(t)

30 SIR Model: Susceptble (S), Infected (I) and Recovered (R) For ths case S s unobserved. SIR Model Dfferental equatons: S = γsi I = γsi υi R = υi Pror specfcaton: γ U(0, 3), υ U(0, 3), S(0) Unf {37,, 100} Runge-Kutta method to approxmate soluton for ODE. ρ used was the eucldean dstance based on the observed tme-ponts.

31 Usng ABC-SMC wth the tolerance schedule: ε 1:4 = (100, 70, 40, 20) For N=1,000 partcles, number of data generatons n each step (n 10 3 ): ABC-SMC: Step 1: 29 Step 2: 49 Step 3: 706 Step 4: 63 ABC-SMC wth AW: Step 1: 29 Step 2: 40 Step 3: 116 Step 4: 11

32 Summary ABC methods: nterestng tool to model problems descrbed by complex systems Improvement of effcency can be obtaned by automatc and post-samplng methods As an applcaton and llustraton of post-samplng approach, Toggle Swtch model was studed usng ABC-MIX New extenson of ABC-SMC was proposed, whch s based on adaptve weghts. It presented better effcency than regular ABC-SMC Choce of the most advantageous ABC approach s stll a problem-specfc queston

33 M.A. Beaumont, J.M. Cornuet, J.M. Marn, and C.P. Robert, Adaptve approxmate bayesan computaton, Bometrka 96 (2009), no. 4, 983. M.A. Beaumont, W. Zhang, and D.J. Baldng, Approxmate bayesan computaton n populaton genetcs, Genetcs 162 (2002), no. 4, F.V. Bonass, L.You, and M. West, Bayesan learnng from margnal data n bonetwork models, Department of Statstcal Scence, Duke Unversty: Dscusson Paper (2011). S. A. Ssson, Y. Fan, and M. M. Tanaka, Sequental Monte Carlo wthout lkelhoods, Proceedngs of the Natonal Academy of Scences USA 104 (2007), T. Ton and M. P. H. Stumpf, Smulaton-based model selecton for dynamcal systems n systems and populaton bology, Bonformatcs 26 (2010), T. Ton, D. Welch, N. Strelkowa, A. Ipsen, and M.P.H. Stumpf, Approxmate Bayesan computaton scheme for parameter nference and model selecton n dynamcal systems, Journal of the Royal Socety Interface 6 (2009), no. 31, 187. M. West, Approxmatng posteror dstrbutons by mxtures, Journal of the Royal Statstcal Socety (Ser. B) 54 (1993),