SUPPLEMENT TO PAPER CONVERGENCE OF ADAPTIVE AND INTERACTING MARKOV CHAIN MONTE CARLO ALGORITHMS

Transcription

1 Submitted to the Annals of Statistics SUPPLEMENT TO PAPER CONERGENCE OF ADAPTIE AND INTERACTING MARKO CHAIN MONTE CARLO ALGORITHMS By G Fort,, E Moulines and P Priouret LTCI, CNRS - TELECOM ParisTech, LPMA, Université Pierre et Marie Curie This supplement provides a detailed proof of Lemma 42 and Propositions 31, 43 and 52 of Fort, Moulines and Priouret (2010 It also contains a discussion on the setwise convergence of transition kernels (see Section 1 For completeness and ease of references, we repeat the assumptions and the main notations For : X [1, and, Θ, denote by D (, the -variation of the kernels P and P (1 D (, P (x, P (x, = sup x X (x When 1, we use the simpler notation D(, Consider the following assumption: A1 For any Θ, there exists a probability distribution π such that π P = π A2 (a For any ε > 0, there exists a non-decreasing positive sequence {r ε (n,n 0} such that lim sup n r ε (n/n = 0 and [ ] P r lim sup E ε(n T n n rε(n (X n rε(n, π n rε(n ε (b For any ε > 0, lim n rε(n 1 where D is ined in (1 E [ ] D( n rε(n+j, n rε(n = 0, A3 k=1 k 1 ( L k L k 1 6 D ( k, k 1 (X k < + P-as, where D and L are ined in (1 and (3 A4 (a lim sup n π n ( < +, P-as (b For some α > 1, k=0 (k + 1 α L 2α k Corresponding author 1 P k α (X k < +, P-as

2 2 G FORT ET AL A5 For all Θ, P is phi-irreducible, aperiodic and there exist a function : X [1,+, and for any Θ there exist some constants b <, δ (0,1, λ (0,1 and a probability measure ν on X such that P λ + b, P (x, δ ν ( ½ { c }(x c = 2b (1 λ 1 1 For any ε > 0, x X, Θ, set (2 M ε (x, = inf{n 0, P n (x, π T ε} For any Θ, set { (3 L = C (1 ρ 1 C b δ 1 (1 λ 1} γ, where C and ρ (0,1 are finite constants such that P n (x, π C ρ n (x 1 Setwise convergence of kernels In many situations (see eg (Fort, Moulines and Priouret, 2010, Section 3, we are able to prove that for (a fixed x X and any A X, there exists Ω A such that P(Ω A = 1 and for any ω Ω A, lim n P n(ω(x,a = P (x,a This implies that if B 0 is a countable algebra generating the σ-algebra X, there exists Ω 0 such that P(Ω 0 = 1 and for any ω Ω 0 and A B 0, lim n P n(ω(x,a = P (x,a Therefore, we are faced to the question: does it imply that there exists Ω such that P(Ω = 1 and for any ω Ω and A X, lim n P n(ω(x,a = P (x,a The answer is no, in general, as illustrated by the following counter-example Counter-Example: Set P (x,a = µ(a and P n(ω(x,a = µ n (ω,a where µ is a probability distribution such that for any x X, µ({x} = 0

3 SUPPLEMENT 3 µ n (ω,a = n 1 n k=1 ½ A (X k (ω with {X k,k 1} iid rv ined on (Ω, F, P taking value in (X, X, with distribution µ Then, by the strong law of large numbers, for any A X, lim n µ n (,A = µ(a P-as İf we take a countable family of measurable sets B 0, then we may find a P-full set D Ω, such that for any ω D, and A B 0, lim n µ n (ω,a = µ(a Of course, B 0 can be an algebra, and even an algebra generating X Nevertheless, it is wrong to assume that this condition implies the setwise convergence, ie that lim n µ n (ω,a = µ(a for any ω D and A X To see why this is wrong, choose ω D and set A = n {X n(ω } Then µ n (ω,a = 1 for any n, and µ(a = 0 lim n µ n (ω,a 2 Proof of (Fort, Moulines and Priouret, 2010, Proposition 31 I1 π is a continuous positive density on X and π < + I2 (a P is a phi-irreducible aperiodic Feller transition kernel on (X, X such that πp = π (b There exist τ (0,1/T, λ (0,1 and b < + such that (4 PW λw + b with W(x = (π(x/ π τ (c For any p (0, π, the sets {π p} are 1-small (wrt the transition kernel P Proposition (Proposition 31 in Fort, Moulines and Priouret (2010 Assume I1, I2 There exist λ (0,1, b <, such that, for any Θ, P W(x λw(x + b(w In addition, for any p (0, π, the level sets {π p} are 1-small wrt the transition kernels P (whatever and the minorization constant does not depend upon Proof We prove the drift inequality with a = 1 The proof for a < 1 follows from the Jensen s inequality The proof is adapted from (Atchadé, 2010, Lemma 41 Under I2b, we have P W(x (1 υ λ W(x + (1 υ b + υw(x + υ α(x,y {W(y W(x}(dy

4 4 G FORT ET AL By inition of W and of the acceptance ratio α, α(x,y {W(y W(x} (dy ( { } = W(y 1 πβ (y π β 1 πτ (y (x π τ (dy (x { } W(y πβ (y {y,π(y π(x} π β 1 πτ (y (x π τ (dy (x Ψ(τ/β (W where we have used that, for a > 0, sup z(1 z a Ψ(a = a/(a + 1 (a+1/a, z [0,1] as in (Atchadé, 2010, Lemma 151 Combining the two latter inequalities yield P W(x [(1 υ λ + υ]w(x + υψ(τ/β(w + (1 υb The proof of the smallness condition relies on the inequality P (x,a (1 υp(x,a 3 Proof of (Fort, Moulines and Priouret, 2010, Lemma 42 Lemma (Lemma 42 Fort, Moulines and Priouret (2010 Assume A5 For any Θ, let F : X R + be a measurable function such that sup F < + and ine ˆF = n 0P n {F π (F } For any, Θ, (5 π π L 2 { π ( + L 2 (x } D (,, and (6 P ˆF P ˆF sup F Θ L 2 ( L D (, + π π + L 2 F F where L is given by (3

5 SUPPLEMENT 5 Proof The proof of this Lemma is closely related to (Andrieu and Moulines, 2006, Proposition 3 and its refinement in Andrieu et al (2011 These types of results have a rather long history: Benveniste, Métivier and Priouret (1990 and Glynn and Meyn (1996 and the references therein for early references We first establish (5 For any k 1, we decompose P kf Pk f as follows k 1 P k f P k f = ( P j (P P P k j 1 f π (f Under A5, there exist constants C and ρ (0,1 such that P k(x, π C ρ k (x Therefore, for any k 1 and x X, π π π P k (x, + P k (x, P k (x, + P k (x, π (C ρ k + C ρ k (x k 1 ( + sup P j (P P P k j 1 f π (f (x f 1 The second term on the RHS is upper bounded by C D (, k 1 C D (, C 1 ρ ρ k j 1 P j (x k 1 ρ k j 1 {π ( + C ρ j } (x D (, (π ( + C (x Therefore π π (C ρ k + C ρ k (x + C 1 ρ D (, (π ( + C (x which implies the desired result by taking the limit as k + We then establish (6 Under A5, ˆF exists (see (20 of Fort, Moulines and Priouret (2010 and P ˆF (x P ˆF (x = P n {F π (F } n 1 n 1P n {F π (F }

6 6 G FORT ET AL We first show (7 P ˆF P ˆF = n 1 n 1 ( ( P j π (P P P n j 1 F π (F {P n (F F π (F F } {P n 1 n 1π n F π (F } Following the same lines as in (Andrieu and Moulines, 2006, Proposition 3, we have for any n 1, P n f P n f = = n 1 + n 1 n 1 ( ( P j π (P P { π P n j 1 f π P n j ( ( P j π (P P P n j 1 } f P n j 1 f π (f where we used that π P = π Hence, for any n 1, and n 1 P n F P n F = f π (f + π (f π P n f, ( ( P j π (P P P n j 1 F π (F + π (F π P n F + P n (F F P n {F π (F } P n {F π (F } = n 1 ( ( P j π (P P P n j 1 F π (F {P n F π (F } + {P n F π (F } π {P n F π (F } This yields (7 We consider the first term in (7: by Lemma 23 in Fort, Moulines and Priouret (2010 ( ( P j π (P P P n j 1 F π (F C C ρ n 1 j ρ j sup F D (,

7 thus implying SUPPLEMENT 7 n 1 ( ( P j π (P P P n j 1 F π (F n 1 L 2 L 2 sup F D (, Consider now the second term on the RHS of (7 By Lemma 23 in Fort, Moulines and Priouret (2010, P n (F F (x π (F F C ρ n (x F F thus implying P n (F F (x π (F F L 2 (x F F n 1 Finally, the third term on the RHS of (7 can be bounded by π {P n F π (F } = (π π {P n F π (F } π π C ρ n sup F, so that π {P n F π (F } π π L 2 n sup F 4 Proof of (Fort, Moulines and Priouret, 2010, Proposition 43 Proposition (Proposition 43 Fort, Moulines and Priouret (2010 Let X be a Polish space endowed with its Borel σ-field X Let µ and {µ n,n 1} be probability distributions on (X, X Let {f n,n 0} be an equicontinuous family of functions from X to R Assume (i the sequence {µ n,n 0} converges weakly to µ (ii for any x X, lim n f n (x exists, and there exists α > 1 such that sup n µ n ( f n α + µ( lim n f n < + Then, µ n (f n µ(lim n f n

8 8 G FORT ET AL Proof Set f = lim n f n Under the stated assumptions, f is continuous We can assume without loss of generality that f n,f are non-negative functions Let c,ε be positive constants We decompose µ n (f n µ(f µ n (f n µ(f = µ n (f n c µ(f c+µ n ((f n c½ {fn>c} µ ((f c½ {f>c} Choose c large enough so that sup n µ n (fn α/cα 1 + µ(f½ {f>c} ε Then upon noting that ½ {f>c} (f/c α 1, ( ( µ n (f n c½ {fn>c} µ (f c½ {f>c} µ n(fn α c α 1 + µ(f½ {f>c} ε Consider now the bounded functions {f n c,n 0} and f c We write (8 µ n (f n c µ(f c µ n ( f n c f c + µ n (f c µ(f c The sequence {µ n,n 0} is relatively compact and thus tight since X is Polish ((Billingsley, 1999, Theorem 52 Then, for ε > 0, there exists a compact set K such that sup n µ n (K c ε Then µ n ( f n c f c sup K f n c f c +2cµ n (K c sup f n c f c +2cε K Under the stated assumptions, the family of functions { f n = f n c f c,n 0} are equicontinuous, and lim n fn = 0 By (Royden, 1988, Lemma 39, Chapter 7, the convergence is uniform on compact sets Consider now the second term on the RHS of (8 Since f c is a bounded continuous function and {µ n,n 0} converges weakly to µ, lim n µ n (f c = µ(f c This concludes the proof 5 Proof of (Fort, Moulines and Priouret, 2010, Proposition 52 Let (U,δ be a metric space; recall that for a real-valued function f on U, the Lipschitz semi-norm is ined by Denote the supremum norm f(x f(y f Lip(U,δ = sup x =y,(x,y U 2 δ(x, y f,u = sup f(x Let f BL(U,δ = f Lip(U,δ + f,u Here, BL stands for bounded Lipschitz and BL(U, δ is the set of all bounded real valued Lipschitz functions on (U,δ Recall the following extension Theorem for bounded Lipschitz functions (Dudley, 2002, Proposition 1123 x U

9 SUPPLEMENT 9 Theorem 51 If U and f BL(U,δ then f can be extended to a function h BL(,δ with h = f on and h BL(,δ = f BL(U,δ Recall also that if (U,δ is compact, then the set of continuous functions coincides with the set of continuous bounded functions C b (U,δ Theorem 52 If (, δ is a compact metric space, the space of bounded Lipschitz functions BL(,δ equipped with the supremum norm, is a separable space Proof By (Dudley, 2002, Corollary 1125, the space of bounded continuous function C b (,δ equipped with the supremum norm, is separable; denote by {f i,i N} C b (,δ a dense family of bounded functions in C b (,δ for the supremum norm Since, (Dudley, 2002, Proposition 1124, BL(,δ is dense in C b (,δ for the supremum norm,, for any n N and then any p N, we may choose f n,p BL(,δ such that f n f n,p, 1/p By construction, the countable family of function {f n,p,(n,p N 2 } is dense in BL(,δ equipped with the supremum norm Finally, the set of bounded Lipschitz functions is convergence determining (Dudley, 2002, Theorem 1133 Theorem 53 Let µ and {µ n,n 0} be distributions on a separable metric space (U, δ endowed with its Borel σ-field The following properties are equivalent (a {µ n,n 0} converges weakly to µ (b {µ n (f,n 0} converges to µ(f for any function f BL(U,δ Lemma 54 Let (U,d be a separable metric space There exists a metric δ on U ining the same topology as d, such that the space of bounded Lipschitz functions BL(U,δ equipped with the supremum norm,u is separable Proof By (Dudley, 2002, Theorem 282, there exist a space and a metric δ on such that U, (,δ is a compact metric space and U is dense in for the metric δ Denote by {ψ k,k N} BL(,δ be a countable family of functions which is dense in BL(,δ equipped with the supremum norm; see Theorem 52 For any k N, denote by φ k the restriction of the function ψ k to U Clearly, for any k N, φ k BL(U,δ By Theorem 51, for any f BL(U,δ,

10 10 G FORT ET AL there is a function h BL(,δ with f = h on U and h BL(,δ = f BL(U,δ Since for any k 0, sup x U f(x φ k (x sup h(x ψ k (x x the set {φ k,k 0} is dense in BL(U,δ equipped with the supremum norm,u Proposition (Proposition 52 in Fort, Moulines and Priouret (2010 Let (U,d be a metric space equipped with its Borel σ-field B(U Let (Ω, A, P be a probability space, µ be a distribution on (U, B(U and {K n,n 0} be a family of Markov transition kernels K n : Ω B(U [0,1] Assume that, for any f C b (U,d Ω f = {ω Ω : lim sup K n (ω,f µ(f = 0}, n is a P-full set Then { ω Ω : f C b (U,d } lim sup K n (ω,f µ(f = 0 n, is a P-full set Proof By Lemma 54, there is a metric δ on U ining the same topology as d and a countable family {φ k,k 0} BL(U,δ which is dense in the space BL(U,δ equipped with supremum norm,u For any ε > 0 and any function f BL(U,δ, there exists k 0 such that f φ k ε By Theorem 53, it suffices to show that there exists a P-full set Ω 1 such that for any ω Ω 1 and any function f BL(U,δ, lim sup n µ n (ω,f µ(f = 0 Set Ω 1 = k Ω φ k Since P(Ω φk = 1, then P(Ω 1 = 1 For any ω Ω 1, we have µ n (ω,f µ(f µ n (ω,f µ n (φ k + µ n (ω,φ k µ(φ k + µ(φ k µ(f 2 f φ k + µ n (ω,φ k µ(φ k 2ε + µ n (ω,φ k µ(φ k and this concludes the proof since by inition of Ω 1, lim n µ n (ω,φ k = µ(φ k

11 References SUPPLEMENT 11 Andrieu, C and Moulines, E (2006 On the ergodicity property of some adaptive MCMC algorithms Ann Appl Probab Andrieu, C, Jasra, A, Doucet, A and Del Moral, P (2011 On non-linear Markov chain Monte Carlo Bernoulli Atchadé, Y (2010 A cautionary tale on the efficiency of some adaptive Monte Carlo schemes Ann Appl Probab Benveniste, A, Métivier, M and Priouret, P (1990 Adaptive Algorithms and Stochastic Approximations 22 Springer, Berlin Translated from the French by Stephen S S Wilson Billingsley, P (1999 Convergence of probability measures, Second ed Wiley Series in Probability and Statistics: Probability and Statistics John Wiley & Sons Inc, New York A Wiley-Interscience Publication Dudley, R M (2002 Real Analysis and Probability Cambridge University Press Fort, G, Moulines, E and Priouret, P (2010 Convergence of adaptive MCMC algorithms: ergodicity and law of large numbers Technical Report submitted to Ann Statist Glynn, P W and Meyn, S P (1996 A Liapounov bound for solutions of the Poisson equation Ann Probab Royden, H L (1988 Real analysis, Third ed Macmillan Publishing Company, New York MR (90g:00004 G Fort and E Moulines LTCI, TELECOM ParisTech-CNRS 46 rue Barrault Paris Cédex 13, France, gersendefort@telecom-paristechfr ericmoulines@telecom-paristechfr P Priouret LPMA, Université Pierre et Marie Curie (P6 Boîte courrier PARIS Cedex 05, France, priouret@ccrjussieufr