ESTIMATION OF INTEGRALS WITH RESPECT TO INFINITE MEASURES USING REGENERATIVE SEQUENCES

Transcription

1 Applie Probability Trust (9 October 2014) ESTIMATION OF INTEGRALS WITH RESPECT TO INFINITE MEASURES USING REGENERATIVE SEQUENCES KRISHNA B. ATHREYA, Iowa State University VIVEKANANDA ROY, Iowa State University Abstract Let f be an integrable function on an infinite measure space (S, S,π). We show that if a regenerative sequence {X n} n 0 with canonical measure π coul be generate then a consistent estimator of λ fπ can be prouce. We S further show that uner appropriate secon moment conitions, a confience interval for λ can also be erive. This is illustrate with estimating countable sums an integrals with respect to absolutely continuous measures on R using simple symmetric ranom walk (SSRW) on Z. Keywors: Markov chains, Monte Carlo, improper target, ranom walk, regenerative sequence Mathematics Subject Classification: Primary 65C05 Seconary 60F05 1. Introuction Let (S, S, π) be a measure space. Let f : S R be S measurable, an S f π <. The goal is to estimate λ S fπ. If π is a probability measure, that is, π(s) = 1, a well-known statistical tool is to estimate λ by sample averages f n n j=1 f(ξ j)/n base on ii observations, {ξ j } n j=1, from π. This ii Monte Carlo metho is a funamental notion in statistics an has mae the subject very useful in many areas of science. A refinement of this result is via the central limit theorem (CLT) from which it follows that if S f2 π <, then an asymptotic (1 α) level Postal aress: Department of Mathematics an Statistics, Iowa State University, Ames, Iowa, 50011, USA. Postal aress: Department of Statistics, Iowa State University, Ames, Iowa, 50011, USA. aress: vroy@iastate.eu 1

2 2 K. B. Athreya an V. Roy confience interval for λ can be obtaine as I n ( f n z α σ n / n, f n + z α σ n / n), where σ 2 n = n j=1 f2 (ξ j )/n f 2 n an z α is such that P( Z > z α ) = α where Z is a N(0, 1) ranom variable. Here P(λ I n ) 1 α as n. On the other han, if it is ifficult to sample irectly from π then the above classical (ii) Monte Carlo cannot be use to estimate λ. In the pioneering work of [13] the target istribution π was the so calle Gibbs measure on the configuration space (a finite but a large set) in statistical mechanics an was ifficult to generate ii sample from. In this paper, they constructe a Markov chain {X n } n 0 that was appropriately irreucible an ha π as its stationary istribution. Then they use a law of large numbers for such chains, that asserts that if {X n } n 0 is a suitably irreucible Markov chain an has a probability measure π as its invariant istribution, then for any initial istribution of X 0, the time average n j=1 f(x j)/n converges almost surely to the space average λ = S fπ as n ([14] Theorem ). So n j=1 f(x j)/n provies a consistent estimator of λ. In the late 80 s an early 90 s a number of statisticians became aware of the work of [13] an aapte it to solve some statistical problems. Thus, a new statistical metho (for estimating integrals with respect to probability istributions) known as the Markov chain Monte Carlo (MCMC) metho was born. An since then the subject has exploe in theory an applications see e.g [17]. Here also, if S f2 π < then uner certain conitions on mixing rates of the chain {X n } n 0, a CLT is available for the time average estimator n j=1 f(x j)/n an from which a confience interval estimate for λ can be prouce. Recently, [2] have shown that the stanar Monte Carlo (both ii Monte Carlo an Markov chain Monte Carlo) methos are not applicable for estimating λ in the case of improper targets, that is, when π(s) =. In particular, they showe that the usual time average estimator, n i=1 f(x i)/n, base on a recurrent Markov chain {X n } n 0 with invariant measure π (with π(s) = ) converges to 0 with probability one an hence is inappropriate. They provie consistent estimators of λ base on regenerative sequences of ranom variables whose canonical measure is π. A sequence of ranom variables is regenerative if it probabilistically restarts itself at ranom times an thus can be broken up into ii pieces. Below is the formal efinition of regenerative sequences.

3 Estimation of integrals using regenerative sequences 3 Definition 1. Let (Ω, F, P) be a probability space an (S, S) be a measurable space. A sequence of ranom variables {X n } n 0 efine on (Ω, F, P) with values in (S, S) is calle regenerative if there exists a sequence of (ranom) times 0 < T 1 < T 2 <... such that the excursions {X n : T j n < T j+1, τ j } j 1 are ii where τ j = T j+1 T j for j = 1, 2,..., that is, P(τ j = k j, X Tj+q A q,j, 0 q < k j, j = 1,...,r) r = P(τ 1 = k j, X T1+q A q,j, 0 q < k j ), j=1 for all k 1,..., k r N, the set of positive integers, A q,j S, 0 q < k j, j = 1,...,r, an r 1 an these are inepenent of the initial excursion {X j : 0 j < T 1 }. The ranom times {T n } n 1 are calle regeneration times. The stanar example of a regenerative sequence is a Markov chain that is suitably irreucible an recurrent. A regenerative sequence nee not have the Markov property. In particular, it nee not be a Markov chain (see [2] for examples). Let ( T2 1 ) π(a) E I A (X j ) j=t 1 for A S. (1) The measure π is calle the canonical (or, occupation) measure for regenerative sequence {X n } n 0 with regeneration times {T n } n 0. Let N n = k if T k n < T k+1, k, n = 1, 2,.... That is, N n enotes the number of regenerations by time n. [2] showe that the following estimator ˆλ n, calle the regeneration estimator for estimating λ S fπ (assuming f π < ) is inee consistent. That is, S ˆλ n n j=0 f(x j) N n λ. (2) Thus, given a (proper or improper) measure π, if we can fin a regenerative sequence with π as its canonical measure, then λ fπ can be estimate by the above S regeneration estimator (2). It may be note that this regenerative sequence Monte Carlo (RSMC) works whether π(s) is infinite or finite. If π(s) < then the strong law of large numbers implies that N n, the number of regenerations by time n, grows at the rate n/π(s) (since E(T 2 T 1 ) = π(s)) as n goes to. Thus, when π(s) is finite one has at least three choices of Monte Carlo methos of estimation, namely

4 4 K. B. Athreya an V. Roy the ii Monte Carlo (ii MC), Markov chain Monte Carlo (MCMC), an regenerative sequence Monte Carlo (RSMC). This last Monte Carlo metho, that is, RSMC has a natural universality property, namely, it works whether one knows the target π is finite or infinite measure. The regenerative property of positive recurrent Markov chains has been use in the MCMC literature for calculating stanar errors of MCMC base estimates for integrals with respect to a probability istribution (see for example [15], [10], [17] chapter 12, [1] Section IV.4). Regenerative methos of analyzing simulation base output also have a long history in the operations research literature (see for example [6], [9]). But in these methos, the excursion time τ 1 is assume to have finite secon moment, which oes not hol when the target istribution is improper. In fact, E(τ 1 ) = E(T 2 T 1 ) = π(s) = when π is improper. On the other han, the RSMC metho oes not require the existence of even the first moment of τ 1. [2] evelope algorithms base mainly on ranom walks for estimating λ when S is countable as well as S = R for any 1. This leas to the very important question of how to construct a confience interval for λ base on ˆλ n. An approximate istribution of (ˆλ n λ) can be use for estimating Monte Carlo error of the regeneration estimator ˆλ n. In this paper, we are able to obtain an asymptotic confience interval for λ uner the assumption of finite secon moments of T 2 1 i=t 1 f(x i ) (this is not the same as requiring E(T 2 T 1 ) 2 < ) an a regularly varying tail of the istribution of the regeneration time τ 1. We make use of a eep result ue to [12] for obtaining the limiting istribution of (suitably normalize) (ˆλ n λ). We then apply our general results to the algorithms base on the simple symmetric ranom walk (SSRW) on Z presente in [2] for the case when S is countable as well as S = R for some 1. We provie simple conitions on f uner which a confience interval base on ˆλ n is available in both cases, that is, when S is countable or S = R an π is absolutely continuous an [2] s algorithms base on the SSRW is use for estimating λ. 2. Main Results Theorem 1. Let {X n } n 0 be a regenerative sequence as in Definition 1. Let π be its canonical measure as efine in (1). Let f : S R be S measurable.

5 Estimation of integrals using regenerative sequences 5 1. Assume ( T2 1 ) ( T2 1 ) 2 E f(x j ) = f π < an E f(x j ) <. (3) j=t S 1 j=t 1 Let U i = T i+1 1 j=t i f(x j ), i = 1, 2, 3,..., an 0 < σ 2 EU1 2 λ 2 <. Let for k = 1, 2,.... Then, k j=1 Y k = (U j λ) σ, (4) k (a) as k, Y k N(0, 1). (5) (b) Let Y k (t), t 0 be the linear interpolation of Y k on [0, ), that is, Y k (t) Y [kt] + (kt [kt]) (U [kt]+1 λ) kσ. Then as k, {Y k (t) : t 0} {B(t) : t 0} is the stanar Brownian motion. {B(t) : t 0}, in C[0, ), where 2. Assume P(τ 1 > x) x α L(x) as x, (6) where 0 < α < 1 an L( ) is slowly varying, i.e., 0 < c <, L(cx)/L(x) 1 as x. Then (a) π(s) = E(T 2 T 1 ) = (b) Let N n = k if T k n < T k+1, k 1, n 1. Then N n n α V α as n, (7) /L(n) where P(V α > 0) = 1, an for 0 < x <, P(V α x) = P(Ṽα x 1/α ) with Ṽα being a positive ranom variable with a stable istribution with inex α such that E(exp( sṽα)) = exp( s α Γ(1 α)), 0 s < where Γ(p) = 0 x p 1 exp( x)x, is the Gamma function, 0 < p <.

6 6 K. B. Athreya an V. Roy 3. Assume that an (6) hols. Then ( T 2 1 ) 2 E f(x j ) <. (8) j=t 1 (a) (ˆλ n λ) N n σ N(0, 1) as n. (9) (b) (ˆλ n λ) n α /L(n) σ Q as n, (10) where Q B(V α )/V α, {B(t) : t 0} is the stanar Brownian motion an V α is as in (7) an inepenent of {B(t) : t 0}. (c) Let Then N n σn 2 = Ui 2 /N n ˆλ 2 n. (11) i=1 i. ii. where Q is as in (10). (ˆλ n λ) N n σ n (ˆλ n λ) n α /L(n) σ n N(0, 1) as n. (12) Q as n, (13) Using the results in Theorem 1 one can construct asymptotic confience intervals for λ base on the regenerative sequence {X i } n i=0 as iscusse below in Corollary 1. Corollary 1. Fix 0 < p < 1, an let z p, q p be such that P( Z > z p ) = p an P(Q > q p ) = p, where Z N(0, 1) an Q is as in (10). Let I n1 (ˆλ n z p σ n / N n, ˆλ n + z p σ n / N n ), an I n2 (ˆλ n q p/2 σ n / n α /L(n), ˆλ n q 1 p/2 σ n / n α /L(n)). Let l(i n1 ) = 2z p σ n / N n, an l(i n2 ) = (q p/2 q 1 p/2 )σ n / n α /L(n) be the lengths of the intervals I n1 an I n2 respectively. Then 1. P(λ I ni ) 1 p as n for i = 1, n α /L(n)l(I n1 ) 2z p σ/ V α, where V α is as in (10).

7 Estimation of integrals using regenerative sequences 7 3. n α /L(n)l(I n2 ) (q p/2 q 1 p/2 )σ. Below we consier a special case of Theorem 1 in the case when S is countable. [2] provie an algorithm (Algorithm I in their paper) base on the SSRW on Z for consistently estimating countable sums. We provie a simple sufficient conition for the secon moment hypothesis (8) in this case so that we can obtain confience interval as well. Since S is countable, without loss of generality, we can take S = Z in this case. Theorem 2. Let {X n } n 0 be a simple symmetric ranom walk (SSRW) on Z starting at X 0 = 0. That is, X n+1 = X n + δ n+1, n 0 where {δ n } n 1 are ii with istribution P(δ 1 = +1) = 1/2 = P(δ 1 = 1) an inepenent of X 0. Let N n = n j=0 I(X j = 0) be the number of visits to zero by {X j } n j=0. Assume that π i π(i) 0 for all i. Let f : Z R be such that j Z f(j) π(j) <. Then 1. ˆλ n n j=0 f(x j)π(x j ) N n λ f(i)π i as n. (14) i Z 2. Assume in aition, j Z f(j) π(j) j <. Then (a) E( T 1 1 j=0 f(x j) π(x j )) 2 <, where T 1 =min {n : n 1, X n = 0}, (b) (c) Nn (ˆλ n λ) where σ 2 E( T 1 1 j=0 f(x j)π(x j )) 2 λ 2, σ (ˆλ n λ)n 1/4 σ N(0, 1), as n, (15) B(V 1/2), as n, (16) V 1/2 where {B(t) : t 0} is the stanar Brownian motion, V 1/2 is a ranom variable inepenent of {B(t) : t 0}, an V 1/2 has the same istribution as π/2 Z, Z N(0, 1). () Also the analogues of (12), (13) an Corollary 1 hol.

8 8 K. B. Athreya an V. Roy Lastly we consier the algorithm presente in [2] (Algorithm III in their paper) that is base on the SSRW on Z an a ranomization tool to estimate integrals with respect to an absolutely continuous measure π on any R, <. Let f : R R an π be an absolutely continuous measure on R with Raon-Nikoym erivative p( ). Assume that R f(x) p(x)x <. The following theorem provies a consistent estimator as well as an interval estimator of λ f(x)p(x)x. R Theorem 3. Let {X n } n 0 be a SSRW on Z with X 0 = 0. Let {U ij : i = 0, 1,...;j = 1, 2,..., } be a sequence of ii Uniform ( 1/2, 1/2) ranom variables an inepenent of {X n } n 0. Assume κ : Z Z be 1 1, onto. Define W n := κ(x n ) + U n, n = 0, 1,,... where U n = (U n1, U n2,...,u n ). Note that the sequence {W n } n 0 is regenerative with regeneration times {T n } n 0 being the returns of SSRW {X n } n 0 to zero. Then 1. ˆλ n n j=0 f(w j)p(w j ) N n λ f(x)p(x)x, R (17) where N n = n j=0 I(X j = 0) is the number of visits to zero by {X j } n j=0. 2. Let g : Z R + be efine as g(r) ( E( f(κ(r) + U) p(κ(r) + U)) 2 = ( f(κ(r)+u) p(κ(r)+u)) 2 u [ 1/2,1/2] (18) where U = (U 1, U 2,..., U ) with U i s, i = 1, 2,...,, are ii Uniform ( 1/2, 1/2) ranom variables, an [ 1/2, 1/2] is the -imensional rectangle with each sie being [ 1/2, 1/2]. Assume, r Z g(r) r <. Then (a) E( T 1 1 j=0 f(w j) π(w j )) 2 <, where T 1 =min {n : n 1, X n = 0}, ) 1/2, (b) Nn (ˆλ n λ) where σ 2 E( T 1 1 j=0 f(w j)π(w j )) 2 λ 2, σ N(0, 1), as n, (19) (c) (ˆλ n λ)n 1/4 where {B(t) : t 0}, an V 1/2 are as in (16). σ B(V 1/2), as n, (20) V 1/2

9 Estimation of integrals using regenerative sequences 9 () Also the analogues of (12), (13) an Corollary 1 hol. Remark 1. A sufficient conition for r Z g(r) r < in Theorem 3 is as follows. Let h(r) = sup f(κ(r) + u) p(κ(r) + u). u [ 1/2,1/2] From (18) it follows that g(r) h(r) for all r Z an so a sufficient conition for E( T 1 1 j=0 f(w j) π(w j )) 2 < is r Z h(r) r <. The proofs of Theorems 1-3 are given in the Section 4. The proof of Corollary 1 follows from the proof of Theorem 1 an Slutsky s theorem an hence is omitte. 3. Examples In this section we emonstrate the use of the results in Section 2 with some examples. We first consier estimating λ = m=1 1/m2. [2] use the SSRW chain mentione in Theorem 2 to estimate λ, that is, they use ˆλ n efine in (14) to consistently estimate λ. Note that, in this case f(j) = 1/j 2 if j 1, f(j) = 0 otherwise, an π(j) = 1 for all j Z. Since j Z f(j)π(j) j = j 1 j (1+1/2) <, we can use Theorem 2 to provie a confience interval for λ base on ˆλ n. In particular, an asymptotic 95% confience interval for λ is given by (ˆλ n ± 1.96σ n / N n ), where σn 2 is efine in (11). The left panel in Figure 1 shows the point as well as 95% interval estimates for 6 values (log 10 n = 3, 4,...,8, where log 10 enotes logarithm base 10). The point an 95% interval estimates for n = 10 8 are 1.636, an (1.580, 1.693) respectively. Note that, the true value λ is π 2 /6 = The time to run the SSRW chain for 10 8 steps using R ([16]) in an ol Intel Q GHz machine running Winows 7 is about 3 secons. The next example was originally consiere in [5]. Let f(x, y) = exp( xy) 0 < x, y <. (21) Let f X Y (x y) := f(x, y) = y exp( xy); 0 < x, y <. R + f(x, y)x Thus for each y, the conitional ensity of X given Y = y is an exponential ensity. Consier the Gibbs sampler {(X n, Y n )} n 0 that uses the two conitional ensities

10 10 K. B. Athreya an V. Roy λ^n λ^n no. of iterations (log10 scale) no. of iterations (log10 scale) Figure 1: Point an 95% interval estimates of m=1 1/m2 (left panel) an f X(0.5) (right panel). f X Y ( y) an f Y X ( x) alternately. [5] foun that the usual estimator n j=0 f X Y (ˇx Y j )/n for the marginal ensity f X (ˇx) = R + f(ˇx, y)y = R + f X Y (ˇx y)f Y (y)y = 1/ˇx breaks own. [2] show that the Gibbs sampler {(X n, Y n )} n 0 is regenerative with improper invariant measure whose ensity with respect to the Lebesgue measure is f(x, y) as efine in (21). Thus their Theorem 3 implies that n j=0 f X Y (ˇx Y j )/n converges to zero with probability 1. [2] use ˆλ n efine in (17) for consistently estimating f X (ˇx). In this example, using Remark 1, we have h(r) = 0 for all r 1, h(0) = 1, an h(r) = exp( ˇx(r 1/2)) for all r 1. Since r Z g(r) r r 1 exp( ˇx(r 1/2)) r <, from Theorem 3, we obtain a confience interval for f X (ˇx) base on ˆλ n. The right panel in Figure 1 shows the (point an 95% interval) estimates of f X (2) = 1/2 for the same six n values mentione in the previous example. The estimates for n = 10 8 are an (0.490, 0.505) respectively. 4. Proofs of results We begin with a short lemma that is use in the proof of Theorem 1. Lemma 1. Let {ξ i } i 1 be ii ranom variables with E ξ 1 <. Then ξ n /n 0.

11 Estimation of integrals using regenerative sequences 11 Proof. Since E ξ 1 <, for all ɛ > 0, n=1 P( ξ 1 > ɛn) <. By the Borel- Cantelli lemma, n=1 P( ξ n > ɛn) < implies P( ξ n /n > ɛ i. o.) = 0. This implies that limsup ξ n /n ɛ with probability 1, ɛ > 0. This in turn implies ξ n /n 0. Proof of Theorem 1 Proof. From (1) it follows that λ = E(U 1 ). Since U i s are ii ranom variables with Var (U 1 ) = σ 2, 1(a) an 1(b) follows from the classical central limit theorem an the functional central limit theorem for ii ranom variables (see [4]). From (1) we have π(s) = E(T 2 T 1 ). Since (6) hols an 0 < α < 1, E(T 2 T 1 ) = E(τ 1 ) = implying 2(a). The proof of (7) is given in [8] (see also [2] an [12]). Now we establish (9). Note that (ˆλ n λ) N n σ = T1 1 j=0 f(x Nn j) i=1 + (U i λ) + Nn σ Nn σ n j=t f(x Nn j). (22) Nn σ Now since P(T 1 < ) = 1, P( T 1 1 j=0 f(x j) < ) = 1. Also, N n with probability 1 as n an 0 < σ <. This implies that T 1 1 j=0 f(x j)/ N n σ 0. Next, n TNn+1 1 j=t Nn f(x j ) j=t Nn f(x j ) Nn σ Nn σ η N n Nn σ, where η i T i+1 1 j=t i f(x j ), i = 1, 2,.... Since the conition (8) is in force, we have Eη1 2 <. By Lemma 1, η2 n /n 0. This implies that η n / n 0. Since N n, as n we have η Nn / N n 0. So to establish (9) it suffices to show that Let for 0 t <, Nn i=1 (U i λ) Nn σ N(0, 1). B n (t) [nt] i=1 (U i λ) nσ + (nt [nt]) (U [nt]+1 λ) nσ (23) an A n (t) T [nt] a n where {a n } is such that na α n L(a n) 1. + (nt [nt]) τ [nt] a n, (24) Then, it is known ([4]) by Donsker s invariance principle that {B n ( ) : 0 t < } converges weakly in C[0, ) as n

12 12 K. B. Athreya an V. Roy to stanar Brownian motion B( ). Also it is known ([8] p. 448) that for any 0 < t 1 < t 2 < < t k <, (A n (t 1 ), A n (t 2 ),..., A n (t k )) convergence in istribution as n to (A(t 1 ), A(t 2 ),..., A(t k )) where {A(t) : t 0} is a nonnegative stable process of orer α with A(0) = 0, an E(exp( sa(1))) = exp( s α Γ(1 α)), 0 s <. It has been pointe out by ([12] p. 525) that [18] has shown that this finite imensional convergence of A n ( ) to A( ) implies the convergence in law in the Skoroho space D[0, ). Next, it can be shown that (A n ( ), B n ( )) converges in the sense of finite imensional istributions as n. Since both {A n ( )} n 1 an {B n ( )} n 1 converge weakly in D[0, ) (as pointe out above) both are tight. This implies that the bivariate sequence {A n ( ), B n ( )} n 1 is also tight as processes in D 2 [0, ) D[0, ) D[0, ). Since the finite imensional istributions of (A n ( ), B n ( )) converge as n, this yiels the weak convergence of {A n ( ), B n ( )} n 1 as n in D 2 [0, ). For the limit process (A( ), B( )), one can conclue that the process C( ) = A( ) + B( ) is a Lévy process on [0, ). Now since B( ) has continuous trajectory an A( ) has strictly increasing nonnegative sample paths, it follows by the uniqueness of the Lévy- Itô ecomposition of C( ) that the processes A( ) an B( ) have to be inepenent. This argument is ue to [12]. As note by [18] (see [4]) it is possible to prouce a sequence of processes (Ãn( ), B n ( )) an a process (Ã( ), B( )) all efine in the same probability space such that for each n, (Ãn( ), B n ( )) has the same istribution as (A n ( ), B n ( )) on D 2 [0, ), an (Ã( ), B( )) has the same istribution as (A( ), B( )), an (Ãn( ), B n ( )) converges to (Ã( ), B( )) with probability 1 in D 2 [0, ). More specifically, we can generate on the same probability space sequences {Ũn,i} i 1,n 1 an { T n,i } i 1,n 1 such that for each n, the sequence {Ũn,i, T n,i } i 1 has the same istribution as {U i, T i } i 1 an for each n, the processes Ãn( ) an B n ( ) are efine in terms of the sequence {Ũn,i, T n,i } i 1 an another sequence {Ũi, T i } i 1 also having the same istribution as {U i, T i } i 1 such that (Ã( ), B( )) is efine using {Ũi, T i } i 1. Next, let A 1 n ( ) an A 1 ( ) be the inverses of the nonecreasing nonnegative functions A n ( ), A( ) from [0, ) to [0, ). (For a nonecreasing nonnegative function H on [0, ) we efine the inverse H 1 ( ) by H 1 (y) inf {x : H(x) y}, 0 y <.) It can be shown (see also [11] Theorem A.1) that (Ãn, Ã 1 n, B n ) converges to (Ã, Ã 1, B) with probability 1 in D 3 [0, ). This, in turn, yiels by the continuous mapping theorem

13 Estimation of integrals using regenerative sequences 13 an the fact that P(à 1 (1) > 0) = 1, as n, B n (à 1 n (1)) à 1 n (1) B(à 1 (1)) à 1 (1). (25) Now by inepenence of B an à an the fact that P(à 1 (1) > 0) = 1, the limiting ranom variable on right in (25) is istribute as N(0, 1). Let b n be a sequence such that a bn /n 1 as n where {a n } is as efine earlier satisfying na α n L(a n ) 1. Such a sequence {b n } exists as a n as n. By efinition à 1 n (1) = inf{x : Ãn(x) 1}. Let y < à 1 n (1), then Ãn(y) < 1. This implies T n,[ny] /a n < 1. Let {Ñn,m} m 1 be the sequence of regeneration times associate with {Ũn,i, T n,i } i 1. Then, Tn,[ny] /a n < 1 implies Ñn,a n y < à 1 n (1), we have Ñ [ny] ny 1. So, Ñ n,an /n y 1/n. This being true for all n,an n à 1 n (1) 1/n. (26) Similarly letting y > à 1 n (1), we conclue that Ñn,a n /n y + 1/n. This being true for all y > à 1 n (1), we have Ñn,an n à 1 n (1) + 1/n. (27) From (26) an (27) we have an more specifically à 1 n (1) 1/n Ñn,a n n à 1 n (1) + 1/n, à 1 b n (1) 1/b n Ñb n,a bn b n à 1 b n (1) + 1/b n. (28) Since a bn /n 1 as n, for all ɛ > 0, n(1 ɛ) a bn n(1 + ɛ) for all n large. This implies for all n large Ñb n,n(1 ɛ) Ñb n,a bn Ñb n,n(1+ɛ). This yiels by (28) à 1 b n (1) 1/b n Ñb n,n(1+ɛ) b n = Ñb n,n(1+ɛ) b n(1+ɛ) b n(1+ɛ) b n. (29)

14 14 K. B. Athreya an V. Roy As {b n } is such that a bn /n 1 an na α n L(a n) 1, which implies b n (a bn ) α L(a bn ) 1, that is, b n a α b n /L(a bn ) n α /L(n). So b n(1+ɛ) /b n (1 + ɛ) α L(n)/L(n(1 + ɛ)) (1 + ɛ) α as n for all ɛ > 1. Since à 1 b n (1) à 1 (1) an (29) hols for all ɛ > 0, we may conclue that à 1 (1) liminf Ñ bn,n b n with probability 1. Similarly, à 1 (1) limsup Ñb n,n/b n with probability 1 an hence limñb n,n/b n = à 1 (1) with probability 1. By efinition of (Ãn, B n ), B bn (N n /b n ) Nn /b n has the same istribution as B bn (Ñb n,n/b n ) Ñbn,n/b n. Since Ñb n,n/b n trajectory, à 1 (1), Bbn ( ) B bn (Ñb n,n/b n ) Ñbn,n/b n B( ) in C[0, ), an B( ) has continuous B(à 1 (1)) à 1 (1). From (23) we see that Nn i=1 (U i λ) = B b n (N n /b n ). Nn σ Nn /b n Hence (9) is prove. Next, to prove (10) we see from (22) it suffices to show that Nn i=1 (U i λ) N n σ n α L(n) Q B(V α). V α Applying the above embeing use to prove (9), it is enough to show that B bn (Ñb n,n/b n ) n α Q. Ñ bn,n/b n L(n)b n This follows from the argument use in the proof of (9) an the fact that n α /{L(n)b n } 1 as n. Since by the strong law of large numbers, σ 2 n from Slutsky s theorem, (9) an (10). Proof of Theorem 2 σ 2 as n, (12) an (13) follows

15 Estimation of integrals using regenerative sequences 15 Proof. The SSRW Markov chain {X n } n 0 is null recurrent (see e.g. [14] Section 8.4.3) with the counting measure on Z being the unique (up to multiplicative constant) invariant measure for {X n } n 0. Hence the SSRW Markov chain {X n } n 0 is regenerative with regeneration times T 0 = 0, T r+1 = inf{n : n T r + 1, X n = 0}, r = 0, 1, 2,... an the proof of (14) follows from strong law of large numbers (see also [2]). Let N(j) T 1 1 i=0 I(X i = j) be the number of visits to the state j uring the first excursion {X i } T1 1 i=0 for j Z. Note that X 0 = 0, an N(0) = 1. Without loss of generality, for the rest of the proof we assume that π j = 1 for all j Z. Since T 1 1 j=0 by Minkowski s inequality, we have E ( T 1 1 j=0 f(x j ) = j Z f(j) N(j), ) 2 ( ) 2 ( f(x j ) = E f(j) N(j) f(j) ) 2. E(N(j)) 2 j Z For the SSRW on Z, it has been shown by [3] that for r 0, E(N(r)) = 1 an Var(N(r)) = 4 r 2. So j Z E(N(r)) 2 4 r 1 if r 0 = Var(N(r)) + 1 = 1 if r = 0 Thus j Z f(j) j < implies E( T 1 1 j=0 f(x j) ) 2 <. Since P(T 1 > n) 2/πn 1/2 as n (see e.g. [7] p. 203), from (7) of Theorem 1, we have where Z N(0, 1) (see e.g. [8] p. 173). N n π n 2 Z, Then (15), (16) an Theorem 2 s () follows from (9), (10), an 3(c) of Theorem 1. Proof of Theorem 3 Proof. The proof of (17) is given in [2]. We now show that r Z g(r) r < implies that E( T 1 1 j=0 f(w j) π(w j )) 2 <. Without loss of generality we assume that p(x) 1 for all x R. Since {U ij : i = 0, 1,...;j = 1, 2,...,} s are ii Uniform

16 16 K. B. Athreya an V. Roy ( 1/2, 1/2) an are inepenent of {X n } n 0, we have ( T1 1 2 E f(w j ) ) = E j=0 ( N(r) r Z i=1 f(κ(r) + U i ) ) 2, where N(r) is as efine in the proof of Theorem 2, the number of visits to the state r uring the first excursion {X i } T1 1 i=0 an U i (U i1, U i2,..., U i ), with U ij s are ii Uniform ( 1/2, 1/2). By Minkowski s inequality, we have { E ( N(r) r Z i=1 ) 2 } 1/2 f(κ(r) + U i ) r Z { E [ N(r) i=1 f(κ(r) + U i )] 2 } 1/2. (30) For any fixe r Z, another application of Minkowski s inequality yiels E [ N(r) i=1 ] 2 ( f(κ(r) + U i ) = E E {[ N(r) i=1 ] 2 }) f(κ(r) + U i N(r) ) g(r) 2 E(N(r) 2 ), where g(r) is efine in (18). Hence the rest of the proof follows from (30) an using similar arguments as in the proof of Theorem 2. Acknowlegement The authors thank one reviewer an the eitor for helpful comments an valuable suggestions. References [1] Asmussen, S. an Glynn, P. W. (2007). Stochastic Simulation: Algorithms an Analysis. Springer. [2] Athreya, K. B. an Roy, V. (2012). Monte carlo methos for improper target istributions. Technical report. Iowa State University. [3] Baron, B. an Rukhin, A. L. (1999). Distribution of the number of visits of a ranom walk. Commun. Stat. Stoch. Moels 15, [4] Billingsley, P. (1968). Convergence of Probability Measures 1st. e. John Wiley & Sons, New York.

17 Estimation of integrals using regenerative sequences 17 [5] Casella, G. an George, E. (1992). Explaining the Gibbs sampler. The American Statistician 46, [6] Crane, M. A. an Iglehart, D. (1975). Simulating stable stochastic systems III: Regenerative processes an iscrete event simulations. Operations Research 23, [7] Durrett, R. (2010). Probability: Theory an Examples 4th. e. Cambrige University Press, New York. [8] Feller, W. (1971). An Introuction to Probability Theory an its Applications vol. II. John Wiley & Sons, New York. [9] Glynn, P. W. an Iglehart, D. (1987). A joint central limit theorem for the sample mean an regenerative variance estimator. Annals of Operations Research 8, [10] Hobert, J. P., Jones, G. L., Presnell, B. an Rosenthal, J. S. (2002). On the applicability of regenerative simulation in Markov chain Monte Carlo. Biometrika 89, [11] Karlsen, H. A. an Tjøstheim, D. (2001). Nonparametric estimation in null recurrent time series. The Annals of Statistics 29, [12] Kasahara, Y. (1984). Limit theorems for Lévy processes an Poisson point processes an their applications to Brownian excursions. J. Math. Kyoto Univ. 24, [13] Metropolis, N., Rosenbluth, A., Rosenbluth, M. N., Teller, A. H. an Teller, E. (1953). Equations of state calculations by fast computing machines. Journal of Chemical Physics 21, [14] Meyn, S. P. an Tweeie, R. L. (1993). Markov Chains an Stochastic Stability. Springer Verlag, Lonon. [15] Myklan, P., Tierney, L. an Yu, B. (1995). Regeneration in Markov chain samplers. Journal of the American Statistical Association 90,

18 18 K. B. Athreya an V. Roy [16] R Development Core Team (2011). R: A Language an Environment for Statistical Computing. R Founation for Statistical Computing Vienna, Austria. ISBN [17] Robert, C. an Casella, G. (2004). Monte Carlo Statistical Methos 2n. e. Springer, New York. [18] Skoroho, A. V. (1957). Limit theorems for stochastic processes with inepenent increments. Theory of Probability an Its Applications 2,