Bayesian Sequential Design under Model Uncertainty using Sequential Monte Carlo

Transcription

1 Bayesian Sequential Design under Model Uncertainty using Sequential Monte Carlo, James McGree, Tony Pettitt October 7, 2

2 Introduction Motivation Model choice abundant throughout literature Take into account model uncertainty in design Motivation for Sequential Monte Carlo (SMC) Efficient algorithm Compute design utilities Application Design for model discrimination

3 SMC For One STATIC Model m Sample from sequence of targets Data annealing here π t (θ m y t,d t ) = f (y t θ m,d t )π(θ m )/Z m,t, for t =,...,T. () y t (independent) data up to t, d t design points up to t, θ m parameter for model m. f is likelihood, π prior, π t posterior f (y t m,d t ) = Z m,t = f (y t θ m,d t )π(θ m )dθ m. θ m SMC: Generate a weighted sample (particles) for each target in the sequence via stems Reweight: particles as data comes in (efficient) Resample: when ESS small Mutation: diversify duplicated particles (can be efficient)

4 SMC For One STATIC Model m (Algorithm) Chopin (22) Have current particles {W i t, θ i t }N i= Re-weight step to included y t+ W i t+ W i t f (y t+ θ i t, d t+), Check effective sample size: ESS = / N i= (W i t+ )2 If not small go back to re-weight step for next observation If small do the following Resample proportional to weights. Duplicates good particles Mutation: Move all particles via MCMC kernel say R times (adaptive proposal)

5 SMC Estimate of Evidence Del Moral et al (26) It can be shown Z t+ /Z t = f (y t+ y t, d t+ ) = Using particles SMC gives estimator θ f (y t+ θ, d t+ )π(θ y t,d t )dθ. Z t+ /Z t N i= W i t f (y t+ θ i t, d t+). Can obtain then approximation of Z t through Z t+ = Z t+ Z t Z. Z Z t Z t Z Also gives estimate of posterior predictive probability of y t+

6 The Algorithm (Using SMC to design under model uncertainty) Effectively run an SMC algorithm for each model m =,...,K Have set of N particles for each model {W i m,t, θ i m,t }N i=. ESS for each model m resampling and within-model updates when required Design part: use data up to t, y t, and particles of all models to compute d t+ (see later)

7 Information Theory Entropy of a random variable X H(X) = x f (x)log f (x), Conditional Entropy of X given knowledge of Y H(X Y ) = y f (y) x f (x y)log f (x y). Mutual information between X and Y I(X; Y ) = H(X) H(X Y ),

8 Utility for Model Discrimination Want some utility U(d y t ) for all d D Consider mutual information between model indicator M and predicted observation Z for y t+ Box and Hill (967). I(M; Z y t, d) = H(M y t ) H(M Z;y t, d). Therefore U(d y t ) = H(M Z;y t, d) which is equal to U(d y t ) = K m= π(m y t ) z S Next design point f (z m,y t, d)log π(m y t, z, d). (2) d t+ = arg max d D U(d y t).

9 Estimating the Utility U(d y t ) = K m= π(m y t ) z S f (z m,y t, d)log π(m y t, z, d). (3) Borth (975) notes difficult computation SMC to rescue. Potential observation z at potential design point d. Pretend this the observation for y t+. Estimate predictive probability using weights w i m,t(d, z) = W i m,tf (z θ i m,t, d), ˆf (z m,y t, d) = N wm,t(d, i z). i=

10 Estimating the Utility (cont...) Denote current evidence for model m, Z m,t Estimate evidence including (d, z) Z m,t (d, z) using log Ẑ m,t (d, z) = log Ẑ m,t + log ˆf (z m,y t, d). Convert to post probs, now have ˆπ(m y t, d, z) Therefore estimate of utility is K Û(d y t ) = ˆπ(m y t ) ˆf (z m,y t, d)log ˆπ(m y t, z, d). m= z S

11 Example - Memory Retention Cavagnaro et al (2) s given words to remember After certain time asked to recall words Keyword recalled (yes/no binary data) Design to choose the times to discriminate between memory retention models D = {, 2,...,} The models: p t = θ (d t + ) θ, power model (θ, θ ) = (.925,.486) p t = η e η d t, exponential model (η, η ) = (.73,.833) There is strong prior information about the parameters, θ beta(2, ), θ beta(, 4), η beta(2, ) and η beta(, 8)

12 Example - Memory Retention - Results Compare random design against mutual information design (a) power, random (b) power, mutual info (c) exp, random (d) exp, mutual info

13 Example - Dose-Response Relationships Clinical Trials etc The models: logit(p t ) = θ + θ d t, (linear model) (θ, θ ) = ( 4, 2). logit(p t ) = η + η log(d t ), (log model) (η, η ) = (, 2). d t logit(p t ) = β + β, (emax model) (β, β, β 2 ) = ( 5, 7,.5). β 2 + d t Priors θ, η, β N(, ), θ, η, β, β 2 HN(, ) D = {.,.2,...,5}.

14 Example - Dose-Response - Results Compare random design against mutual information design (a) linear, random (b) linear, mutual info (c) log, random (d) log, mutual info

15 Example - Dose-Response - Results Here D = {.,.2,...,5} (a) emax, random (b) emax, mutual info Figure: Results for the dose-response example (emax model true).

16 Example - Dose-Response - Results Add in large design point D = {.,.2,...,5, } (a) random (b) mutual info Figure: Results for the dose-response example where the list of models consists of the log and emax models.

17 Key References Borth, D. (975). A total entropy criterion for the dual problem of model discrimination and parameter estimation. JRSS: Series B, 37(): Box, G. E. P. and Hill, W. J. (967). Discrimination among mechanistic models. Technometrics, 9():57-7. Cavagnaro, D. R. et al (2). Adaptive design optimiza- tion: A mutual information-based approach to model discrimination in cognitive science. Neural Computation, 22(4): Chopin, N. (22). A sequential particle filter method for static models. Biometrika, 89(3): Del Moral, P et al (26). Sequential Monte Carlo samplers. JRSS: Series B, 68(3):4-436.