A regeneration proof of the central limit theorem for uniformly ergodic Markov chains

Transcription

1 A regeneration proof of the central limit theorem for uniformly ergodic Markov chains By AJAY JASRA Department of Mathematics, Imperial College London, SW7 2AZ, London, UK and CHAO YANG Department of Mathematics, University of Toronto, M5S 2E4, Toronto ON, Canada November 0, 2005 Abstract Let X n be a Markov chain on measurable space E, E with unique stationary distribution π. Let h : E R be a measurable function with finite stationary mean πh := E hxπdx. Ibragimov and Linnik 97 proved that if X n is geometrically ergodic, then a central limit theorem CLT holds for h whenever π h 2+δ <, δ > 0. Cogburn 972 proved that if a Markov chain is uniformly ergodic, with πh 2 < then a CLT holds for h. The first result was re-proved in Roberts and Rosenthal 2004 using a regeneration approach; thus removing many of the technicalities of the original proof. This raised an open problem: to provide a proof of the second result using a regeneration approach. In this paper we provide a solution to this problem. Keywords: Markov chains; Central limit theorems Introduction Let X n be a Markov chain with transition kernel P : E E [0, and a unique stationary distribution π. Let h : E R be a real-valued measurable function. We say that h satisfies a Central Limit Theorem or n CLT if there is some σ 2 < such that the normalized sum n 2 n i= [hx i πh converges weakly to a N0, σ 2 distribution, where N0, σ 2 is a Gaussian distribution with zero mean and variance σ 2 we allow that σ 2 = 0, and e.g. Chan and Geyer 994, see also Bradley 985 and Chen 999 σ 2 = πh E hxp n hxπdx n= with P n hx = E hyp n x, dy and P n x, dy the n step transition law for the Markov chain. To further our discussion we provide the following definitions. Denote the class of probability measures on E, E as PE. The total variation distance between μ, ν PE is: μ ν := sup μa νa. A E We will be concerned with geometrically and uniformly ergodic Markov chains: Corresponding author, chaoyang@math.toronto.edu

2 Definition.. A Markov chain with stationary distribution π PE is geometrically ergodic if n N: P n x, π Mxρ n where ρ < and Mx < π almost everywhere. If M = sup x E Mx is finite then the chain is uniformly ergodic. Theorem.2 Cogburn, 972. If a Markov chain with stationary distribution π PE is uniformly ergodic, then a n CLT holds for h whenever πh 2 <. Ibragimov and Linnik 97 proved a CLT for h when the chain is geometrically ergodic and, for some δ > 0, π h 2+δ <. Roberts and Rosenthal 2004 provided a simpler proof using regeneration arguments. In addition, Roberts and Rosenthal 2004 left an open problem: To provide a proof of Theorem.2 originally proved by Cogburn 972 using regeneration. Many of the recent developments of CLTs for Markov chains are related to the evolution of stochastic simulation algorithms such as Markov chain Monte Carlo MCMC e.g. Robert and Rosenthal For example, Roberts and Rosenthal 2004 posed many open problems, including that considered here, for CLTs; see Häggström 2005 for a solution to another open problem. Additionally, Jones 2004 discusses the link between mixing processes and CLTs, with MCMC algorithms a particular consideration. For an up-to-date review of CLTs for Markov chains see: Bradley 985, Chen 999 and Jones The proof of Theorem.2, using regeneration theory, provides an elegant framework for the proof of CLTs for Markov chains. The approach may also be useful for alternative proofs of CLTs for chains with different ergodicity properties; e.g. polynomial ergodicity see Jarner and Roberts The structure of this paper is as follows. In Section 2 we provide some background knowledge about the small sets and the regeneration construction, we also detail some technical results. In Section 3 we use the results of the previous Section to provide a proof of Theorem.2 using regenerations. 2 Small Sets and Regeneration Construction 2. Small Sets We recall the notion of a small set: Definition 2.. A set C E is small or n 0, ɛ, ν-small if there exists an n 0 N, ɛ > 0 and a non-trivial ν PE such that the following minorization condition holds x C: P n 0 x, ɛν. It is known e.g. Meyn and Tweedie 993 that if P is uniformly ergodic, the whole state space E is small. That is we have the following lemma: Lemma 2.. If X n on E, E with stationary distribution π PE is uniformly ergodic, then E is small. 2

3 2.2 Regeneration Construction and Some related Technical Results Now we consider the regeneration construction for the proof. Since E is small we use the split chain construction Nummelin, 984, for any x E, A E P n 0 x, A = ɛrx, A + ɛνa where Rx, A = ɛ [P n 0 x, A ɛνa. That is, for a single chain X n, with probability ɛ we choose X n+n0 ν, while with probability ɛ we choose X n+n0 RX n,, if n 0 >, we fill in the missing values as X n+ using the appropriate Markov kernel and conditionals. We let T, T 2,... be the regeneration times, i.e. the times such that X Ti ν, clearly T i = in 0. Let T 0 = 0 and rn = sup{i 0 : T i n}, using the regeneration time, we can break up the sum n i=0 [hx i πh into sums over tours as follows: where rn n [hx i πh = i=0 Qn = T j= T j+ [hx j πh + We begin our construction, by noting the following result. Lemma 2.. Under the formulation above, we have that: Proof. Let and [hx i πh + Qn n T rn+ [hx j πh. Qn n /2 p 0. 2 T Q + n = [hx j πh + T Q n = [hx j πh Q + 2 n = n T rn + Q 2 n = n T rn + [hx j πh + [hx j πh where [hx j πh + = max{hx j πh, 0} and [hx j πh = max{ [hx j πh, 0}. 3

4 The strategy of the proof is to show that Q ± i n/n/2 p 0 as n. Consider Q + n, sn 0 Q + n = [hx j πh + w.p ɛ ɛ s 3 where s N. If Q + n/n/2 p 0, i.e. P ɛ, Q + n > ɛn/2, i.o. = for all n, which means that PQ + n =, i.o. =, which is impossible from 3. So Q+ i n/n/2 p 0 as n. Similarly Q i n/n/2 p 0 as n. For Q 2 we have Q + 2 n l n j=rn+ [hx j πh + = Q + 2 n, where ln = inf {i 0 : T i n}. We know that Q + 2 n has the same distribution with Q+ 2 n, so Q + i n/n/2 p 0 as n and therefore, Q + 2 n/n/2 p 0 as n. Similarly Q 2 n/n/2 p 0 as n. From the above discussion, we conclude that Qn/n /2 p 0. The above lemma indicates that our objective is to find the asymptotic distribution of rn j= Tj+ [hx i πh. Given the definition of T i, each random variable s j = T j+ [hx i πh has same distribution. However, we know that T j depends on X Tj +,, X Tj, but does not depend on the value of X Tj. That is, we have the following lemma: Lemma 2.2. For any 0 i <, s i and s i+ are not independent, but the two collections of random variables: {s i : 0 i m 2} and {s i : i m} are independent for any m 2.Therefore the random variable sequence {s i } i=0 is a one-dependent stationary stochastic processes. Proof. Clearly s i+ depends on the distribution T i+, thus: P X Ti + dx,, X Ti +m dy X Ti = x, T i+ T i > m = ɛrx, dy P m P x, dx P x m, dy x, dy and P X Ti + dx,, X Ti +m dy X Ti = x, T i+ T i = m = ɛνdy P m x, dy P x, dx P x m, dy. Note s i depends on T i+. Therefore s i and s i+ are not independent. However, for any 0 i m 2 < m j <, since X Ti ν and X Tj depends X Tj +,, X Tj, but is independent of all the {X k : k T j }. Thus, we have the result. To prove Theorem.2 we follow the strategy: T Step : Prove that I = E ν i=0 [hx i πh = 0 Step 2: Prove that J = E νdxe [ T i=0 [hx i πh 2 X 0 = x <. Step 3: Prove that a n CLT holds for a stationary, one-step dependent stochastic process. 3 Proof of Theorem.2 T Lemma 3.. I = E ν i=0 [hx i πh = 0 4

5 Proof. Denote T = τm and H k = k+m i=km [hx i πh, then we have: I = E ν [ H k I{k < τ} Consider the splitting m skeleton chain { ˇX nm } as in section 5.. of Meyn and Tweedie 993, we know that ˇα = X is an accessible atom. Then we can apply Theorem 0.0. of Meyn and Tweedie 993 to this splitting chain. That is: ˇτ ˇα πb = ˇπB 0 B = ˇπdwE w [ I{ ˇX km ˇB} ˇα = ɛ πdwe w [ X k= ˇτ ˇα k= I{ ˇX km ˇB} Let ˇτˇα = min{n : ˇXnm ˇα}. Since for any w ˇα, ˇP m w, ν, we have ˇτˇα = τ. Following Theorem 5..3 in Meyn and Tweedie 993, we also have P kn 0 x, B = ˇP kn 0 x, ˇB for any B E. Therefore we have: τ πb = ɛe ν [ I{X km B} = ɛe ν [ I{X km B}I{τ > k} So we have: k= k= I = E ν [E H k I{k < τ} X km = = E ν [E H k I{k < τ} X km E ν [E H k X km I{k < τ} The last equation follows since random variables I{τ > k} and X km are independent. In addition, given τ > k and X km, the distribution of H k is equal to H 0 given X 0 ; therefore I = E ν [E H 0 X 0 I{k < τ} = E π [E H 0 X 0 = E π H 0 = 0. Lemma 3.2. We have: [ T 2 J = E ν [hx i πh <. 4 i=0 5

6 Proof. [ τ J = E ν k+m i=km [ 2 E ν I{k < τ} H k 2 [hx i πh [ = E ν H k 2 I{k < τ} + 2 H k = E ν [ = E ν [ = E ν [ H k 2 + 2H k j=i+ E H k H k E H k H k j=k+ H j I{j < τ} I{k < τ} j=k+ j=k+ H j I{j < τ} {k < τ} H j I{j < τ}i{k < τ} X km, I{k < τ} H j I{j < τ} X km I{k < τ}. In the last equation, we have used the fact that random variables I{τ > k} and X km are independent. Since E H i H i j= H j {j < τ} X im = x = E H H 0 define fx = E H H 0 j= H j {j < τ} X 0 = x then we have: J E ν [ fx 0 I{k < τ} k= j= [ = E ν [fx 0 I{0 < τ} + E ν fx 0 I{k < τ} [ E ν [fx 0 + E ν fx 0 E ν [I{k < τ} The last inequality is follows since:. fx 0 I{k < τ} fx 0 ; 2. When k, I{τ > k} is independent with X 0 k= H j {j < τ} X 0 = x Note E ν [I{k < τ = P ν k < τ ɛ k 6

7 and πdy = E P n 0 x, dyπdx ɛνdy therefore we have J ɛ E ν[fx 0 ɛ 2 E π [fx 0 and m E π [fx 0 E π [ hx i πh 2 i=0 From the above arguments we conclude that J <. Finally, we prove Theorem.2: mπh 2 πh 2 < Proof of Theorem.2. Following Lemma 2., we can obtain: n i=0 lim [hx i πh n n /2 = lim n rn j= Tj+ [hx i πh n /2. 5 Define h i = hx i πh, s j = T j+ + h i and η j = s jm+ + +s j+m for an integer m 2. Following Lemma 2.2 we know that two collections of random variables: {s i : 0 j m 2} and {s i : i m} are independent for any m 2; thus n n s j = [n/m η j + [n/m s mj + n n n j= n m[n/m It should be noted that if j i > n 0, then X i and X j are independent, η j are i.i.d random variables and s mj are i.i.d. so we have: [n/m η j d N0, σ2 m n m [n/m s mj d N0, σ2 s n m s j where σ 2 m = m Es 2 + 2m 2Es s 2 and σ 2 s = E[s 2, letting m, we have σ2 m m Es 2 + 2Es s 2 and m σ 2 s 0, so the CLT holds. Let [ n σ 2 2 = lim n n E [hx i πh i= 7

8 then σ 2 = [ n 2 lim n n E [hx i πh i= = [ rn lim n n E s j 2 = lim n n E j= [rns 2 + 2rn 2s s 2 r By the elementary renewal theorem e.g. Feller 968, lim nn n = ET 2 T. Since P[T 2 T = n 0 s = ε ε s, ET 2 T = s= [n 0sε ε s = n 0 ε <. Therefore if we denote σ 2 = E[s 2 + 2s s 2, then As a result, we conclude that lim n as n. rn j= Acknowledgement σ 2 = n 0 ε E[s2 + 2s s 2 = n 0 ε σ2 6 Tj+ [hx i πh n /2 = lim n d n0 ε rn j= = N0, σ 2 /2 N0, σ 2 Tj+ [hx i πh rn /2 r/2 n n /2 Both authors would like to thank Jeffrey Rosenthal for his assistance in writing this paper. The first author was supported by an Engineering and Physical Sciences Research Council Studentship and would like to thank Dave Stephens and Chris Holmes for their advice relating to this paper. REFERENCES Bradley, R. C On the central limit question under absolute regularity. Ann. Prob., 3, Chan, K. S. and Geyer, C. J Discussion of Markov chains for exploring posterior distributions. Ann. Statist., 22, Chen, X Limit theorems for functionals of ergodic Markov chains with general state space. Mem. Amer. Math. Soc., 39. Cogburn, R The central limit theorem for Markov processes. In Le Cam, L. E., Neyman, J. and Scott, E. L. Eds. Proc. Sixth Ann. Berkley Symp. Math. Statist. and Prob., 2,

9 Feller, W An Introduction to Probability Theory and its Applications. 3rd ed, Wiley, Chichester. Häggström, O On the central limit theorem for geometrically ergodic Markov chains. Prob. Theory and Rel. Fields, 32, Ibragimov, I. A. and Linnik, Y. V. 97. Independent and stationary sequences of random variables. Wolter-Noordhoff, Groiningen. Jarner, S. F. and Roberts G. O Polynomial convergence rates of Markov chains. Ann. Appl. Prob., 2, Jones, G. L On the Markov chain central limit theorem. Prob. Surveys,, Meyn, S. P. and Tweedie, R. L Markov chains and stochastic stability. Springer, New York. Nummelin, E General irreducible Markov chains and non-negative operators. Cambridge University Press, Cambridge. Roberts, G. O. and Rosenthal, J. S General state space Markov chains and MCMC algorithms. Prob. Surveys,,