Pseudo-marginal Metropolis-Hastings: a simple explanation and (partial) review of theory

Transcription

1 Pseudo-arginal Metropolis-Hastings: a siple explanation and (partial) review of theory Chris Sherlock Motivation Iagine a stochastic process V which arises fro soe distribution with density p(v θ ). Iagine noisy observations y of this stochastic process with conditional density p(y θ 2, v). Set θ = (θ, θ 2 ) and suppose that p(y θ) = p(v θ )p(y v, θ 2 ) dv is intractable. Let the paraeters have a prior, π 0 (θ). We wish to obtain a saple fro the posterior π(θ). Ideally, we would run a Metropolis-Hastings algorith targeting π(θ), but the intractability of the likelihood prevents this. Unbiased estiators Whilst p(y θ) is intractable, we can create an estiate, ˆp(y θ, u) := p(y θ 2, v), where v has a density of p(v θ ). The corresponding estiator is unbiased since E [ˆp(y θ, U)] = E [ˆp(y θ 2, V )]) = p(v θ )p(y θ 2, v) dv = p(y θ). Clearly, an average of such estiators is also unbiased. Unbiased estiators ay also be obtained, for exaple, fro iportance sapling (i.e. not sapling fro p(v θ ), but then reweighting) or, for hidden Markov odels, by a particle filter. Fro now on we siply assue that we have an unbiased estiator of the likelihood ˆp(y θ, U) where auxiliary variable U is sapled fro soe density q(u θ). This leads to the following unbiased (up to a fixed constant) estiator of the posterior, π(θ): ˆπ(θ U) = π 0 (θ)ˆp(y θ, U).

2 Algorith Start with θ, ˆπ(θ u) and at each iteration:. Propose θ fro soe q(θ θ). 2. Propose u fro soe q(u θ ) and hence create ˆπ(θ u ). 3. Accept (θ, ˆπ(θ u )) with probability α(θ, u; θ, u ) = ˆπ(θ u )q(θ θ ) ˆπ(θ u)q(θ θ). Aazingly (Beauont, 2003; Andrieu and Roberts, 2009), the stationary distribution of the resulting Markov chain has a arginal density of π(θ). Extended target In the final section we show that the chain actually targets π(θ, u) := ˆπ(θ u) q(u θ) = π 0 (θ) q(u θ)ˆp(y θ, u). Since ˆp(y θ, u) is unbiased, the arginal for this is then π 0 (θ) q(u θ)ˆp(y θ, u) = π 0 (θ)p(y θ) π(θ), as required, Detailed balance The chain targets π(θ, u) because detailed balance holds with respect to π(θ, u) since π(θ, u) q(θ θ) q(u θ ) α(θ, u; θ, u ) = q(u θ) q(u θ ) [ˆπ(θ u)q(θ θ) ˆπ(θ u )q(θ θ )], which is invariant to (θ, u) (θ, u ). 2

3 One-diensional representation The estiator of the likelihood can be rewritten as ˆp(y θ, U) = W p(y θ), iplictly defining W := ˆp(y θ, U) p(y θ) with E [W ] = because the estiator is unbiased. The acceptance probability is therefore α(θ, w; θ, w ) = π(θ )q(θ θ )w π(θ)q(θ θ)w, where w and w are the ultiplicative noises in the estiates of the likelihood at the current and proposed θ values. W arises fro soe (hypothetical) proposal distribution q(w θ ) := q(u θ )du. u :ˆp(y θ,u )=wp(y θ) Of course w, q(w θ) or π(θ) are unknown. However, this representation provides intuition into the behaviour of pseudo-arginal MH and is used in theoretical analyses of the algorith. Firstly we realise that the pseudo-arginal algorith can be viewed as a Markov chain on (θ, w). The extended target is in fact π(θ, w) := π(θ)w q(w θ), () and, at stationarity, the conditional density of W θ is w q(w θ); this is a density as E q [W ] =. Ordering pseudo-arginal algoriths Since kw is a concave function of W and W k is a concave function of W, we ay apply Jensen s inequality twice to find (Andrieu and Vihola, 205): E w q(w θ), q(w θ ) [α(θ, W ; θ, W )] = dwdw w q(w θ) q(w θ ) α(θ, W ; θ, W ) = [ ( )]] π(θ E q(w θ) [E q(w )q(θ θ ) θ ) W π(θ)q(θ θ) W π(θ )q(θ θ ) π(θ)q(θ θ). Therefore the acceptance rate of a pseudo-arginal MH algorith is never greater than that of the ideal MH algorith. In fact, this ordering extends to the spectral gap and to the variance of the estiator of E π [f(θ)] for any f L 2 0(π). 3

4 Andrieu and Vihola (205) generalise these results to pairs of pseudo-arginal algoriths: whenever one algorith can be viewed as a noisy version of another then the noisier one is always less efficient. In particular, a PMMH algorith that uses an average of two or ore unbiased estiators is always ore efficient than an algorith which uses just one of the estiators. Tuning when ˆp is obtained using a particle filter The ultiplicative noise in the log-posterior, W, can, in general, have any distribution provided it is non-negative and E [W ] =. However, when ˆp(y θ, U) is obtained via a particle filter (or SMC) then in the liit as the nuber of data points, T and with the nuber of particles = t/β, for soe β > 0 then, subject to ixing conditions (Bérard et al., 204) the noise in a new proposal satisfies: log W N ( 2 ) σ2, σ 2, for soe σ 2 > 0 which, typically, depends on the paraeters, θ, well as the data generating process. We will provide a heuristic for this result, but first let us note soe consequences. Suppose that σ does not depend on θ. For convenience, set V := log W and V := log W. Thus V N( σ 2 /2, σ 2 ) and iediately fro () and the line beneath, the conditional (and arginal) density of V is exp[v] [ σ 2π exp ] 2σ (v + 2 σ2 /2) 2 = [ σ 2π exp ] 2σ (v 2 σ2 /2) 2, so that V N(σ 2 /2, σ 2 ), or ( ) log W N 2 σ2, σ 2 and log W log W N ( σ 2, 2σ 2). Thus, the ratio W /W in the pseudo-arginal acceptance probability has a lognoral distribution. This is the starting point for several papers (Pitt et al., 202; Sherlock et al., 205; Doucet et al., 205; Neeth et al., 206) that provide advice on tuning PMMH algoriths when using a particle filter. All recoend choosing to give soe approxiately optial ˆσ 2 value, with the recoended ˆσ 2 soewhere between 0.8 and 3.3. More realistically, σ(θ) varies slowly with θ so if q(θ θ) is a local ove, σ 2 (θ ) σ 2 (θ) and the following result holds approxiately. 4

5 Sketch proof of the Gaussian liit For siplicity, suppose that the data, Y :T := (Y,..., Y t ) are iid. Conditional on the tth data point, y t, we generate independent auxiliary variables, U t,i, (i =,..., ). Our estiator of the likelihood is ˆp(y :T θ, U) = T ˆp (y t θ, U t,i ) = T ˆp (y t θ)w t,i = p(y :T θ) T W t,i, where ˆp (y θ, u) is the unbiased estiator of the likelihood of a single observation, p (y θ), given the auxiliary variable u, and W t,i := ˆp (y t θ, U t,i )/p(y t θ). Applying a second-order Taylor expansion, log ˆp(y :T θ, U) log p(y :T θ) is { [ ]} T log + W t,i [ ] [ T W t,i 2 W t,i ]. 2 The W t,i are independent; set τt 2 := Var(W t,i ) <, and denote τ 2 = E [τt 2 ], where expectation is over the distribution of Y t. For siplicity, we ignore the detail that T = [β] rather than T = β. The first ter in the expansion is [ ] T W t,i = β β A t N ( 0, βτ 2), β by the SLLN, where A t := (W t,i ) N(0, τ 2 t ) by the CLT. Siilarly, by the SLLN we obtain [ ] 2 T W t,i = β β a.s. B t βτ 2, β [ 2 where B t := (W t,i )] are independent with finite eans of τ 2 these two liits leads to the required result. t. Cobining References Andrieu, C. and Roberts, G. O. (2009). The pseudo-arginal approach for efficient Monte Carlo coputations. Ann. Statist., 37(2): Andrieu, C. and Vihola, M. (205). Convergence properties of pseudo-arginal arkov chain onte carlo algoriths. Ann. Appl. Probab., 25(2):

6 Andrieu, C. and Vihola, M. (205). Establishing soe order aongst exact approxiations of MCMCs. ArXiv e-prints. Beauont, M. A. (2003). Estiation of population growth or decline in genetically onitored populations. Genetics, 64: Bérard, J., Moral, P. D., and Doucet, A. (204). A lognoral central liit theore for particle approxiations of noralizing constants. Electron. J. Probab., 9:no. 93, 28. Doucet, A., Pitt, M., Deligiannidis, G., and Kohn, R. (205). Efficient ipleentation of Markov chain Monte Carlo when using an unbiased likelihood estiator. Bioetrika. To appear. Neeth, C., Sherlock, C., and Fearnhead, P. (206). Particle Metropolis-adjusted Langevin algoriths. Bioetrika. Accepted for publication. Pitt, M. K., dos Santos Silva, R., Giordani, P., and Kohn, R. (202). On soe properties of Markov chain Monte Carlo siulation ethods based on the particle filter. Journal of Econoetrics, 7(2):34 5. Sherlock, C., Thiery, A., Roberts, G. O., and Rosenthal, J. S. (205). On the efficiency of pseudo-arginal rando walk Metropolis algoriths. Ann. Stat., 43():