Uniform Consistency for Nonparametric Estimators in Null Recurrent Time Series

Transcription

1 Te University of Adelaide Scool of Economics Researc Paper No Uniform Consistency for Nonparametric Estimators in Null Recurrent Time Series Jiti Gao, Degui Li and Dag Tjøsteim

2 Te University of Adelaide, Scool of Economics Working Paper Series no ( ) Uniform Consistency for Nonparametric Estimators in Null Recurrent Time Series Jiti Gao, Degui Li and Dag Tjøsteim Te University of Adelaide and Te University of Bergen Abstract: Tis paper establises several results for uniform convergence of nonparametric kernel density and regression estimates for te case were te time series regressors concerned are nonstationary null recurrent Markov cains. Under suitable conditions, certain rates of convergence are also establised for tese estimates. Our results can be viewed as an extension of some well known uniform consistency results for te stationary time series to te nonstationary time series case. Keywords: β null recurrent Markov cain; nonparametric estimation; rate of convergence, uniform consistency Abbreviated Title: Uniform Consistency for Nonparametric Estimators Jiti Gao is from te Scool of Economics, Te University of Adelaide, Adelaide SA 5005, Australia.

3 2. Introduction As discussed in te literature, uniform consistency for nonparametric kernel density and regression estimators is not only important in estimation teory, but also useful in deriving results in specification testing teory. Existing studies by many autors mainly focus on te case were an observed time series satisfies a type of stationarity. Suc studies include Liero (989), Roussas (990), Liebscer (996), Masry (996), Bosq (998), Fan and Yao (2003), Ould-Saïd and Cai (2005) and oters. Suc existing results basically focus on uniform convergence on fixed compact sets. In a recent paper by Hansen (2008), te autor makes significant progress towards establising uniform convergence on unbounded sets for a general class of nonparametric functionals for te case were te time series data are stationary and strong mixing. By contrast, tere is little work for uniform consistency of nonparametric kernel estimators involving nonstationary time series. Pillips and Park (998) were among te first to study nonparametric estimation in an autoregression model wit integrated regressors and tey developed a local time approac for te establisment of teir asymptotic teory. In te same period, Karlsen and Tjøsteim (998, 200) independently discuss nonparametric kernel estimation in te nonstationary case were te time series regressors are nonstationary null recurrent Markov cains. Te autors establis various asymptotic results. For te recent development of te nonparametric and semiparametric estimation in nonstationary time series, we refer to Karlsen, Myklebust and Tjøsteim (2007), Cen, Gao and Li (2009), Wang and Pillips (2008, 2009) and te references terein. In te field of model specification testing, Gao et al (2009a, 2009b) establis asymptotically consistent tests in bot autoregression and co integration cases. A closely related paper by Cai, Li and Park (2009) considers nonparametric estimation in functional coefficient models wit nonstationarity. Tis paper tus establises strong uniform convergence wit rates for a class of nonparametric kernel density and regression estimators for te case were te time series data involved are nonstationary null recurrent Markov cains. Te uniform convergence results not only strengten existing point wise convergence results given in Karlsen and Tjøsteim (200), but also are natural extensions of some corresponding results in Hansen (2008) for te stationary time series case. Te rest of te paper is organized as follows. Some basic definitions and results for Markov cains are summarized in Section 2. Te main results are stated in

4 3 Section 3. Applications of te main results to density estimation and bot te Nadaraya Watson kernel and te local linear kernel estimation metods are given in Section 4. Te conclusions are given in Section 5. Some additional basic results in Markov teory are contained in Appendix A. All te proofs are given in Appendix B. 2. Some basic results for Markov cains Let {X t, t 0} be a Markov cain wit transition probability P and state space (E, E), and φ be a measure on (E, E). Trougout te paper, {X t } is assumed to be φ irreducible Harris recurrent (see Appendix A for definition). Te class of stocastic processes we are dealing wit in tis paper is te class of β null recurrent Markov cains. DEFINITION. A Markov cain {X t } is β null recurrent if tere exist a small nonnegative function f( ) (see Appendix A for te definition of small function), an initial measure λ, a constant β (0, ) and a slowly varying function L f ( ) suc tat E λ [ t=0 f(x t ) ] Γ( + β) nβ L f (n) as n, (2.) were E λ stands for te expectation wit initial distribution λ and Γ( ) is te usual Gamma function. It is sown in Karlsen and Tjøsteim (200) tat wen tere exist some small measure ν and small function s wit ν(e) = and 0 s(v), v E, suc tat ten {X t } is β null recurrent if and only if were L s = L f π sf below. P α (S α > n) = P s ν, (2.2) Γ( β)n β ( + o()), (2.3) L s (n) and π s is te invariant measure defined in (A.2) of Appendix A We ten introduce a useful decomposition wic is critical in te proofs of uniform convergence for nonparametric estimation in null recurrent time series. Let f be a real function defined in R = (, ). We now decompose te partial sum

5 4 S n (f) = n f(x t ) into a sum of identically distributed random variables wit one t=0 main part and two asymptotically negligible minor parts. Define τ 0 f(x t ), k = 0, t=0 τ k Z k = f(x t ), k, t=τ k + f(x t ), k = (n), t=τ + were te precise definitions of τ k as te recurrence times and as te number of regenerations will be given in Appendix A. Ten S n (f) = Z 0 + Z k + Z (n). (2.4) k= From Nummelin (984) s result, we know tat {Z k, k } is a sequence of independent and identically distributed (i.i.d.) random variables. In te decomposition (2.4) of S n (f), plays a kind of role as te number of observations. It follows from Lemma 3.2 in Karlsen and Tjøsteim (200) tat Z 0 and Z (n) converge to zero almost surely wen tey are divided by. Furtermore, Karlsen and Tjøsteim (200) sow tat if (2.2) olds and f(x) π s (dx) <, ten for an arbitrary initial distribution λ we ave were π s (f) = f(x) π s (dx). S n(f) π s (f) almost surely (a.s.), (2.5) In Section 3 below, we establis uniform convergence results for a general nonparametric quantity for te case were a nonstationary null recurrent time series is involved. 3. Main results Let {e t } be a sequence of i.i.d. random variables and independent of {X t }. Define a general nonparametric quantity of te form Φ n (x) = Xt x L e t, (3.) t=

6 were L( ) is a smoot function satisfying Assumption A2(i) below, is te bandwidt and is te number of regenerations, corresponding to te sample size n in te stationary time series case. To establis strong uniform consistency results for te nonparametric quantity Φ n (x) defined by (3.), we need te following assumptions. Assumption A(i) Te invariant measure π s of te β null recurrent Markov cain {X t } as a uniformly continuous density function p s ( ) on R wit p s (x) <. x R (ii) {e t } is a sequence of i.i.d. random variables and independent of {X t }. Assumption A2(i) L( ) as some compact port C(L) and satisfies a Lipsitz type condition of te form: L(x) L(y) C l x y for all x, y C(L) and some constant C l > 0. (ii) Te bandwidt satisfies for some 0 < ε 0 < β, n ε 0 0 and n β ε 0 as n. (3.2) A(i) corresponds to analogous conditions on te density function in te stationary time series case. Moreover, it can be verified wen {X t } is generated by a random walk model of te form X t = X t + u t, t =, 2,, X 0 = 0, (3.3) were {u t } is a sequence of i.i.d. random variables. Nummelin (984) sows in tis case tat te invariant density function p s (x). A(ii) is imposed to make sure tat te compound process X t, e t )} is still β null recurrent. A2(i) is a quite natural condition (see, for example, Fan and Yao 2003; Hansen 2008) and te condition of te compact port of te kernel function L( ) is imposed for te brevity of our proofs. A2(ii) also imposes some mild conditions on te bandwidt parameter for te null recurrent time series (cf. Karlsen, Myklebust and Tjøsteim 2007) and it corresponds to 0 and n in te stationary time series case. In te stationary case, Hansen (2008) is concerned wit a nonparametric estimate of te form Ψ(x) = n n d t= K X t x Yt, were X t, Y t ) : t } is a (d + ) dimensional vector of random variables. Bot weak and strong uniform convergence results are establised in Teorems 2 and 3 of Hansen (2008). 5

7 6 In Teorems 3. and 3.2 below, we establis strong uniform convergence results for te nonparametric quantity defined by (3.). Teorem 3.. Let A and A2 old. If, in addition, E [ ] +β 2[ ] e ε 0 <, ten Φ n (x) p s (x)µ e µ l = o(), a.s., (3.4) were T n = M 0 n β L s (n), M 0 is any given positive constant, µ l = L(u)du, µ e = E[e ], and [x] x is te integer part of x. Remark 3.. Teorem 3. can be viewed as an extension of a corresponding result in te stationary time series case to te nonstationary null recurrent time series case. Equation (3.4) implies tat tere exists some relationsip between te bandwidt condition and te moment condition on {e t }. As ε 0 decreases (te bandwidt condition becomes weaker), we need iger order moment condition on {e t }. Furtermore, wen µ e = E[e ] = 0, (3.4) reduces to Φ n (x) = o(), a.s.. In Teorem 3.2 below, we furter establis a rate of uniform convergence under a sligtly stronger condition on te moments of {e t }. Teorem 3.2. Suppose tat A. and A.2 old. If, in addition, [ ] [ ] E e 2m 0 4β ( + θ)ε0 + 4 < wit m 0 = + for some 0 < θ <, (3.5) 2( θ)ε 0 ten, Φ n (x) E[Φ n (x)] = o n β θε 0 were ε 0 is defined as in A2(ii). a.s., (3.6) Remark 3.2. Equation (3.6) can be viewed as a result corresponding to an existing result in te stationary time series case (see, for example, Teorem 2 of Hansen 2008). Wen β = 2 and θ 0, te rate of convergence in (3.6) as a limit tat is proportional to n, wic could be compared wit a rate of te ( ) order O previously obtained by several autors in te stationary time log n n series case. Te different rates may be interpreted as follows. In te null recurrent case, te amount of time spent by te time series around any particular point is of

8 order n rater tan n. Tus, te order is rougly proportional to n for te nonstationary case rater tan n for te stationary case. Before we prove Teorems 3. and 3.2 in Appendix B below, we give some useful corollaries and applications of te teorems in Section 4 below for te nonparametric kernel density and regression estimation of nonstationary null recurrent time series Applications in density and regression estimation Te nonparametric quantity defined by (3.) is of a general form. Tus, we can obtain uniform convergence results for various nonparametric kernel estimators, suc as te kernel density estimator, te Nadaraya Watson (NW) estimator and te local linear estimator. Define te kernel density estimator of te invariant density function p s (x) by p n (x) = Xt x K, (4.) t= were K( ) is a probability kernel function. We now ave te following uniform consistency result for p n (x); its proof is given in Appendix B below. Teorem 4.. Suppose tat A and A2(ii) old. Let p s (x) be twice continuously differentiable wit p s(x) C p <. Suppose tat K( ) as some compact x R port C(K) and satisfies te Lipsitz type condition: K(x) K(y) C k x y for all x, y C(K) and some constant C k > 0. In addition, K( ) is a symmetrical probability kernel function. Ten, we ave for n large enoug p n (x) E [ p n (x)] = o n β θε 0 a.s. (4.2) and p n (x) p s (x) = O( 2 ) + o n β θε 0 were θ and ε 0 are te same as defined in Teorem 3.2. a.s., (4.3) Remark 4.. Te above teorem can be viewed as an extension of Teorem 5.3 in Fan and Yao (2003) and Teorem 7 in Hansen (2008) from te stationary time series case to te nonstationary time series case. Karlsen and Tjøsteim (200)

9 8 obtain te point wise consistency of p n (x) in te null recurrent time series case were n ε 0 0 and n β/2 ε 0 for 0 < ε 0 < β 2. Teorem 4. not only weakens teir bandwidt condition but also extends teir point wise convergence to uniform convergence wit possible rates. We now consider a nonlinear nonstationary regression model of te form Y t = m(x t ) + e t, t n, (4.4) were {X t } is a β null recurrent Markov cain, {e t } is a sequence of i.i.d. errors independent of {X t } wit E[e ] = 0, and m( ) is a nonlinear function. Nonlinear nonstationary models ave been studied by several autors. Karlsen, Myklebust and Tjøsteim (2007), and Wang and Pillips (2009) consider estimating te regression function by te NW estimator of te form were w n,t (x) = K np s= Xt x K( Xs x m n (x) = w n,t (x)y t, (4.5) t= ). Tey ten establis asymptotic distributions of m n(x) using different metods. As anoter application of our main results in Section 3, we give a rate of strong uniform convergence of te NW estimator m n (x) in Teorem 4.2 below. Te proof is given in Appendix B below. Teorem 4.2. Assume tat te conditions of Teorem 4. are satisfied. If, in addition, m(x) is twice continuously differentiable, δ 2 nn β θε 0, 2 δ n 0, δ in i 0 for i =, 2, (4.6) were δ n = inf p s (x) and δin = m (i) (x) for i =, 2, and [ ] E e 2m 0 < wit m 0 = ten we ave ( m n (x) m(x) = o δ n n β θε 0 [ 4β ( + θ)ε ( θ)ε 0 ] +, (4.7) ) + o (δ n) + O ( δ 2n 2) a.s.. (4.8)

10 Remark 4.2. (i) Te conditions imposed for te establisment of Teorem 4.2 are reasonable and justifiable. We now sow tat te conditions in (4.6) can be easily verified in te case were {X t } is of an integrated form as in (3.3). In tis case, p s (x) and tus te first two parts of (4.6) reduce to te mild conditions imposed in (3.2). Te last part of (4.6) imposes certain restrictions on te functional form of m( ). Several classes of functional forms of m( ) are included as long as m(x) is of te form m(x) = O ( x +ζ) for some 0 < ζ < wen x is large enoug. Particularly wen m(x) = a + bx, te last part of (4.6) is satisfied trivially. (ii) Teorem 4.2 can be viewed as an extension of Teorem 3.3 in Bosq (998) and Teorem 9 in Hansen (2008) from te stationary time series case to te nonstationary time series case. For te random walk defined by (3.3), it is easy to ceck tat (4.8) olds wit δ n = and β = 2. We finally apply te local linear estimation metod and establis te uniform convergence rate of it. As in Fan and Gijbels (996), te local linear estimator of m(x) is defined by were w n,t (x) = e K x, (X t) np ek x, (X s) s= ) ( Sn,2 (x) ( X t x m n (x) = w n,t (x)y t, t= wit K x, (X t ) = K n ( Xt x ) Sn, (x) ) wit S n,j (x) = 9 ) (, in wic Xt x) Kn = n s= K ( X s x ) ( Xs x) j for K ( X t x j =, 2. Te following teorem can be viewed as an extension of Teorem in Hansen (2008) from te stationary time series case to te nonstationary time series case. Its proof is given in Appendix B below. we ave Teorem 4.3. Assume tat te conditions of Teorem 4.2 are satisfied. Ten, ( m n (x) m(x) = o δ n n β θε 0 ) + O ( δ 2n 2) a.s.. (4.9) Note tat te first order bias term involved in (4.8) is eliminated wen te local linear estimation metod is employed. As a consequence, te class of functional forms for m(x) is enlarged to include te case were m(x) = O ( x 2+ζ) for some 0 < ζ < wen x is large enoug.

11 0 5. Conclusions We ave establised several results for strong uniform consistency wit rates of some commonly used nonparametric estimators for te case were te regressors are nonstationary null recurrent time series. Our main results ave extended some existing uniform consistency results for te stationary time series case. As for te stationary case, te establised results are expected to be useful in establising asymptotic teory in bot nonparametric and semiparametric estimation and testing for nonstationary null recurrent time series. 6. Acknowledgments Te main ideas of tis paper ad been discussed during several visits by te first autor to Norway and te tird autor to Australia since October Te work of te autors was mainly ported financially by two Australian Researc Council Discovery Grants under Grant Numbers: DP and DP Te tird autor would also like to acknowledge port from te Danis Researc Council under Grant Number: REFERENCES Bosq, D. (998) Nonparametric Statistics for Stocastic Processes: Estimation and Prediction, 2nd ed. Lecture Notes in Statistics 0. Springer-Verlag. Cai, Z., Q. Li & J. Park (2009) Functional coefficient models for nonstationary time series data. Journal of Econometrics 48, 0 3. Cen, J., J. Gao & D. Li (2009) Semiparametric regression estimation in null recurrent time series. Available at Fan, J. & I. Gijbels (996) Local Polynomial Modelling and Its Applications. Capman & Hall, London. Fan, J. & Q. Yao (2003) Nonlinear Time Series: Nonparametric and Parametric Metods. Springer, New York. Gao, J. (2007) Nonlinear Time Series: Capman & Hall/CRC, London. Semiparametric and Nonparametric Metods. Gao, J., M. L. King, Z. Lu & D. Tjøsteim (2009a) Specification testing in nonstationary time series autoregression. Fortcoming in te Annals of Statistics.

12 Gao, J., M. L. King, Z. Lu & D. Tjøsteim (2009b) Nonparametric specification testing for nonlinear time series wit nonstationarity. Fortcoming in Econometric Teory. Hansen, B. E. (2008) Uniform convergence rates for kernel estimation wit dependent data. Econometric Teory 24, Karlsen, H. A. & D. Tjøsteim (998) Nonparametric estimation in null recurrent time series. Discussion paper available at Sonderforscungsbereic , Humboldt University. Karlsen, H. A. & D. Tjøsteim (200) Nonparametric estimation in null recurrent time series. Annals of Statistics 29, Karlsen, H. A., T. Mykelbust & D. Tjøsteim (2007) Nonparametric estimation in a nonlinear cointegration type model. Annals of Statistics 35, Liebscer, E. (996) Strong convergence of sums of α mixing random variables wit applications to density estimation. Stocastic Processes and Teir Applications 65, Liero, H. (989) Strong uniform consistency of nonparametric regression function estimates. Probability Teory and Related Fields 82, Masry, E. (996) Multivariate local polynomial regression for time series: uniform strong consistency and rates. Journal of Time Series Analysis 7, Nummelin, E. (984) General Irreducible Markov Cains and Non-negative Operators. Cambridge University Press. Ould-Saïd, E. & Z. Cai (2005) Strong uniform consistency of nonparametric estimation of te censored conditional mode function. Journal of Nonparametric Statistics 7, Pillips, P. C. B. & J. Park (998) Nonstationary density estimation and kernel autoregression. Cowles Foundation Discussion Paper 8. Roussas, G. G. (990) Nonparametric regression estimation under mixing conditions. Stocastic Processes and Teir Applications 36, Wang, Q. Y. & P. C. B. Pillips (2008). Structural nonparametric cointegrating regression. Cowles Foundation Discussion Paper No Fortcoming in Econometrica. Wang, Q. Y. & P. C. B. Pillips (2009) Asymptotic teory for local time density estimation and nonparametric cointegrating regression. Econometric Teory 25, Weeden, R. L. & A. Zygmund (977) Measure and Integral. Dekker.

13 2 Appendix A. Useful results in Markov teory To make tis paper more self contained, we summarize some useful terms and facts in Markov teory in tis appendix. We adopt te same notation as used in Nummelin (984) and Karlsen and Tjøsteim (200). Let {X t, t 0} be a class of Markov cains wit transition probability P and state space (E, E), and φ be a measure on (E, E). Te sequence {X t, t 0} is said to be φ irreducible if eac φ positive set A is communicating wit te wole state space E, i.e. P n (x, A) > 0, for all x E wenever φ(a) > 0. n= Denote te class of nonnegative measurable functions wit φ positive port by E +. For a set A E, we write A E + if A E +, were A stands for te indicator function of te set A. Te cain {X t } is Harris recurrent if for all A E +, x E, P (S A < X 0 = x), S A = min{n, X n A}, or equivalently, if given a neigborood N x of x, x E, wit φ(n x ) > 0, {X t } will return to N x wit probability one. Tis is wat makes asymptotics for our nonparametric estimation possible. Let η be a nonnegative measurable function and λ be a measure. We define te kernel η λ by η λ(x, A) = η(x)λ(a), (x, A) (E, E). If K is a kernel, we define te function Kη, te measure λk and te number λη by Kη(x) = K(x, dy)η(y), λk(a) = λ(dx)k(x, A), λη = λ(dx)η(x). Te convolution of two kernels K and K 2 is defined by K K 2 (v, A) = K (v, dy)k 2 (y, A). A function η E + is said to be a small function if tere exist a measure λ, a positive constant b and an integer m, so tat P m bη λ. And if λ satisfies te above inequality for some η E +, b > 0 and m, ten λ is called a small measure. A set A is small if A is a small function. By Teorem 2. and Proposition 2.6 in Nummelin (984), we know tat for a φ irreducible Markov cain, tere exists a minorization inequality: tere are a small function s, a probability measure ν and an integer m 0 suc tat P m0 s ν.

14 As pointed out by Karlsen and Tjøsteim (200), it causes some tecnical difficulties to ave m 0 > and it is not a severe restriction to assume m 0 =. So in tis appendix, we always assume tat te minorization inequality 3 P s ν (A.) olds wit ν(e) =, 0 s(x), x E. We apply te so called Markov cain splitting metod wen we prove some of our results. In tis metod, an important role is played by te split cain under te minorization inequality (A.). Tis allows for te decomposition of te cain into independent and identically distributed main parts and remaining parts tat are asymptotically negligible. Denote Q(x, A) = ( s(x)) (P (x, A) s(x)ν(a))(s(x) < ) + A (x)(s(x) = ). Ten te transition probability P (x, A) can be decomposed as P (x, A) = ( s(x))q(x, A) + s(x)ν(a). Wen (A.) olds, it can be verified tat Q is a transition probability. As 0 s(x) and ν(e) =, P can be seen as a mixture of te transition probability Q and te small measure ν. Since ν is independent of x, te cain regenerates eac time wen ν is cosen wit probability s(x). For more details, we refer to Nummelin (984). Now we introduce te split cain X t, T t ), t 0}, were te auxiliary cain {T t } only takes te values 0 and. Given X t = x, T t = t t, T t takes te value wit probability s(x) and ten te cain generates. Tus, α = E {} is a proper atom of te split cain. Te distribution of X t, T t ), t 0} is determined by its initial distribution λ, te transition probability P and (s, ν). We use P λ and E λ for te distribution and expectation of te Markov cain wit initial distribution λ. Wen λ = δ x we write P x instead of P δx, wic is te conditional distribution of (T 0, X t, T t ), t }) given X 0 = x. Wen λ = δ α (x, ), i.e., X 0 = x for arbitrary x E and T 0 =, ten we write P α and E α. As sown in Karlsen and Tjøsteim (200), if we let π s = νg s,ν, were G s,ν = (P s ν) n, (A.2) ten π s = π s P, wic implies tat π s is an invariant measure. We ten give some definitions of te stopping times of te Markov cain. Let n=0 and τ = τ α = min{n 0 : T n = } S α = min{n : T n = }. As {X t, T t t 0} is Harris recurrent, P α (S α < ) =. Moreover, define { inf{n 0 : T n = }, k = 0, τ k = inf{n > τ k : T n = }, k, (A.3) (A.4) (A.5)

15 4 and denote te total number of regenerations in te time interval [0, n] by, tat is, { max{k : τ k n}, if τ 0 n, = (A.6) 0, oterwise. Equations (A.3) (A.6) are used in te decomposition (2.4) in Section 2. B. Proofs of te teorems To prove te main results in Sections 3 and 4, we need te following lemma. Lemma B.. Let te conditions of Lemma 5.2 of Karlsen and Tjøsteim (KT) (200) old. If, in addition, x R p s (x) <, ten teir conclusion can be strengtened to obtain a uniform bound of te form: E [ U 2m ( g ) ] d m 2m+ wit d m (x) M, x R were U( g ) is as defined in Lemma 5.2 of KT (200). Proof: In view of te proof of Lemma 5.2 of KT (200), it suffices to sow tat tere is an absolute constant 0 < M < suc tat x R d m (x) M. Te main issue is to deal wit te inequality in te middle of page 404 of KT (200). Note in our case tat te function ξ 0, so tat c 2 is independent of x. Similarly, note tat on page 404 of KT (200), c K li x, c 2 li I Nx were (p. 399, KT) N x = N x () = {y : K x, (y) 0}. According to B 2 (p. 399, KT) N x is a small set (under weak assumptions it can be taken to be compact). Tis means tat N x can be taken as a set of regeneration wit a corresponding minorization inequality as in (3.4) of KT (one can make tis more explicit by using te construction in Example 3. of KT, but tis requires an extra assumption on {X t }). Now let x be fixed. By te definition of G s,ν in (3.6) and (3.8) of KT, we ave G s,ν I Nx = E y ( τ n=0 I N x (X n )). We need to sow tat y ( τ n=0 I N x (X n )) is bounded and independent of x. Consider first te case of y not belonging to N x. Let τ be te first time te cain its N x, and wit no loss of generality assume τ τ. Ten ) ( τ ) ( τ ) E y ( τ n=0 I Nx (X n ) z N x E z n=τ I Nx (X n ) z N x E z n=0 I Nx (X n ) (B.) so tat it suffices to look at te case y N x. Ten te cain regenerates wit probability s(y), were s = s x is as in te minorization inequality (3.4) of KT, and were wit no loss of generality we may take 0 < s(y) <. If te cain regenerates ( τ ) E y I Nx (X n ) = p s (z)dz p s (z)leb(n x ) C, (B.2) n=0 N x z R

16 were Leb(N x ) is Lebesgue measure of N x, and were C is independent of y and x, since Leb(N x ) = Leb(N 0 ) (see p. 399 of KT). Tus, by conditioning on te first step and letting A be te event tat te cain leaves te set N x in te first step, we ave for y N x E y ( τ n=0 I Nx (X n ) ) + ( s(y))p (A c ) z N x E z s(y)c + ( s(y))p (A ) z N x E z ( τ ) I Nx (X n ) n=0 ( τ ) I Nx (X n ) (s(y) + ( s(y))c = C, n=0 were A c is te complement of A and we ave also used equations (B.) and (B.2). Te rest of te proof of identical to tat of Lemma 5.2 of KT (200). Proof of Teorem 3.. Since {e t } is assumed to be i.i.d. and independent of {X t }, X t, e t )} is still β null recurrent by Lemma 3. of Karlsen, Myklebust and Tjøsteim (2007). Let Γ t (x) = L Xt x e t. In te following proof, we write a n b n to mean a n = o(b n ). Define J n (β) = { n β ζ ε 0 n β+ζε0 }, were ζ > 0 is to be cosen later. Observe tat for any 5 given η > 0 = { x T { n x T { n x T ({ n t= t= } Γ t (x) p s (x)µ e µ l > η } Γ t (x) p s (x)µ e µ l > η J n (β) t= } Γ t (x) p s (x)µ e µ l > η Jn(β) c } ) Γ t (x) p s (x)µ e µ l > η J n (β) Jn(β). c t= (B.3) By Lemma 3.4 in Karlsen and Tjøsteim (200), in order to prove (3.4), it suffices to sow tat ) } P Γ t (x) p s (x)µ e µ l > η J n (β), i.o. = 0. (B.4) t= Te set {x : x T n } can be covered by a finite number of subsets {S i } centered at s i wit radius O(n β ζ2ε0 2 ), were ζ 2 > 0 is cosen suc tat ζ 2 > ζ. Letting Q(n) be te number of tese sets, ten Q(n) = O ( T n n +ζ2ε0 β 2).

17 6 Hence, max j Q(n) t= + max j Q(n) x S j Γ t (x) p s (x)µ e µ l Γ t (s j ) p s (s j )µ e µ l { } (Γ t (x) Γ t (s j )) + p s(x) p s (s j ) µ e µ l. t= t= (B.5) Assumption A2(i) implies tat tere is some constant C l > 0 suc tat L Xt x Xt s j s j x L C l C n β ζ2ε0 2 l. (B.6) Since te conditions of Teorem 3. imply E [ e ] <, it ten follows tat for ζ 2 > ζ { } max j Q(n) [Γ t (x) Γ t (s j )] + p s(x) p s (s j ) ( x S j ) t= n n = O β ζ 2 ε 0 2 n + o() = O β ζ 2 ε o() (B.7) = O + o() = o() a.s.. n β ζ 2 ε 0 n β ζ ε 0 In view of (B.5) and (B.7), in order to prove (B.4), it suffices to sow for any η > 0, ) } P max Γ j Q(n) t (s j ) p s (s j )µ e µ l > η J n (β), i.o. = 0. (B.8) t= We ten apply te independence decomposition tecnique as used in (2.4) to sow (B.8). Define τ 0 Γ t (s j ), k = 0, t=0 τ k Z k (s j ) = Γ t (s j ), k, t=τ k + Γ t (s j ), k = (n), t=τ + were τ k, k 0, are defined as in Karlsen and Tjøsteim (200). Ten Γ t (s j ) = Z 0 (s j ) + Z k (s j ) + Z (n) (s j ). t= k= (B.9) From Nummelin (984), we know tat {Z k (s j ), k } is a sequence of i.i.d. random variables for eac fixed j. By arguments similar to tose used in te proof of Teorem 5. in Karlsen and Tjøsteim (200), we ave max Z 0(s j ) = o() and j Q(n) Let ν(s j ) = E [Z k (s j )]. max Z (n)(s j ) = o() a.s. j Q(n) (B.0) (B.)

18 By A2(i), te continuity of L( ) and Bocner s lemma (cf. Weeden and Zygmund 977), we ave max ν(s j) p s (s j )µ e µ l = o(). j Q(n) By (B.0), it suffices to sow P max j Q(n) Z k (s j ) ν(s j ) > η J n (β), k= i.o. = 0. 7 (B.2) (B.3) We prove (B.3) troug using Bernstein s inequality and te truncation metod. Similarly to te proof of Lemma B., we ave [ ] max E Z k (s j ) 2p0 C 2p0+ wit p 0 = [( + β)/ε 0 ], (B.4) j Q(n) were te constant C depends neiter on s j nor on n. Define Z k (s j ) = Z k (s j )I( Z k (s j ) < n β ζ3ε0 ) and Z k (s j ) = Z k (s j ) Z k (s j ), (B.5) were ζ 3 is cosen suc tat 0 < ζ < ζ 3 < and 2β ( ζ ζ2)ε0+2 ( ζ 3)ε 0 coice of (ζ, ζ 2, ζ 3 ) implies teir existence. By standard arguments, we ave + P P P max j Q(n) k= max j Q(n) max j Q(n) k= k= ) } Z k (s j ) ν(s j ) > η J n (β) (Z k (s j ) E [ Z k (s j )) ] ) } > η/2 J n (β) ( Z [ k (s j ) E Zk (s j ))] ) } > η/2 J n (β). < 2p 0. Note tat te (B.6) In view of (B.5) and te coice of (ζ, ζ 2, ζ 3 ) suc tat 2β + ε 0 + ζ 2 ε 0 + ζ ε 0 2p 0 ( ζ 3 )ε 0 <, te Markov inequality implies P max n= j Q(n) ( Z [ k (s j ) E Zk (s j ))] ) } > η/2 J n (β) k= C Q(n)P n= ( Z [ k (s ) E Zk (s ))] ) } > η/2 J n (β) k= C Q(n)n β+ζε0 P { } Z (s ) n β ζ3ε0 n= (B.7) C Q(n)n β+ζε0 2p0 n 2p0(β ζ3ε0) n= C T n n +ζ2ε0+ζε0 2p0(β ζ3ε0) 2p0 n= C n 2β+ ε0+ζ2ε0+ζε0 2p0( ζ3)ε0 L s (n) <. n=

19 8 Meanwile, by Bernstein inequality for i.i.d. random variables we ave P max n= j Q(n) (Z k (s j ) E [ Z k (s j )) ] ) } > η/2 J n (β) k= C [c 2n β+ζ ε 0 ] { l Q(n) P l (Z k (s ) E [ Z k (s )) ] } > η/2 n= l=[c n β ζ ε 0 ] k= C [c 2n β+ζ ε 0 ] Q(n) n= l=[c n β ζ ε 0 ] exp { ln β+ζ 3ε 0 } C Q(n) exp { } c(l) n (ζ3 ζ)ε0 <, n= (B.8) were c > 0, c 2 > 0 and c(l) > 0 are some constants, and [x] x denotes te largest integer part of x. By (B.6) (B.8) and Borel Cantelli Lemma, equation (B.3) is proved. By (B.9), (B.0) and (B.3), equation (B.8) olds. Hence, te proof of Teorem 3. is completed. Proof of Teorem 3.2. Let Γ t (x) be defined as in te proof of Teorem 3. and J n(β) = { } n β ξθε0 n β+ξθε0, were ξ will be cosen later. By (B.3) and Lemma 3.4 in Karlsen and Tjøsteim (200), in order to prove (3.6), it suffices to sow tat for any η > 0, ) } P (Γ t (x) E[Γ t (x)]) > η J n β θε 0 n(β), i.o. = 0. (B.9) t= As in te proof of Teorem 3., te set {x : x T n } can be covered by a finite number of subsets {S i } centered at s i wit radius ( r n = O n (β+θε0 2ξ2θε0 2)/2 3/2), were ξ 2 is cosen suc tat 0 < ξ < ξ 2 < and 6β ( + θ)ε 0 + 2(ξ + ξ 2 )θε ( θ)ε 0 < m 0, in wic m 0 is as defined in te conditions of Teorem 3.2. Letting U(n) be te number of tese sets, ten U(n) = O T n rn. Similarly to te derivation in (B.6), we ave (Γ t (x) E[Γ t (x)]) t= ( max j U(n) Γt (s j ) E[Γ t(s j )]) { t= + max ( Γt j U(n) x S j (x) Γ t (s j ) + E Γt (x) Γ t (s j ) )} (B.20) t= =: Π n, + Π n,2.

20 9 In a derivation similar to (B.7), conditions A(ii) and A2(i) imply ( ) n rn n 2 β+ 2 θε0 ξ2θε0 3 2 Π n,2 = O 2 = O n β ξθε0 2 = o. (B.2) n β θε 0 In view of (B.20) and (B.2), in order to prove (B.9), we need only to consider Π n,. We will apply te independence decomposition tecnique and truncation metod as in te proof of Teorem 3.. Letting Z k (s j ) be defined as above, Π n, = max j<u(n) Z 0(s j ) + k= Z k (s j ) + Z (n) (s j ). (B.22) We first sow tat P max j U(n) k= Z k (s j ) ν(s j ) > η J n(β), i.o. = 0, (B.23) were ν(s j ) is as defined in (B.). Similarly to te proof of Lemma B., we ave [ ] max E Z k (s j ) 2m0 C 2m0+, (B.24) j U(n) were te constant C depends neiter on s j nor on n. Define ( Ẑ k (s j ) = Z k (s j )I Z k (s j ) < n (β θε0)/2 /2) and Z k (s j ) = Z k (s j ) Ẑk(s j ). (B.25) Let η n = η n β θε 0. As in (B.6), we ave + P P P max j U(n) k= max j U(n) max j U(n) k= k= ) } Z k (s j ) ν(s j ) > η n J n(β) (Ẑk(s j ) E [Ẑk (s j ))] ) } > η n /2 J n(β) ) } (Z k (s j ) E [Z k (s j ))] > η n/2 J n(β). (B.26)

21 20 Since (ξ, ξ 2 ) is cosen suc tat 4β (+θ)ε0+2(ξ+ξ2)θε0+4 2( θ)ε 0 < m 0, by (B.25) we ave ) } P max n= j U(n) (Z k (s j ) E [Z k (s j ))] k= > η n/2 J n(β) ) } C U(n)P n= (Z k (s ) E [Z k (s ))] k= > η n/2 J n(β) C U(n)n β+ξθε0 P { Z (s ) n (β θε0)/2 /2} n= (B.27) C U(n)n β+ξθε0 2m0 n m0(β θε0) m0 n= C n= C n= rn n 2β+ξθε0 m0(β θε0) L s (n) m0 n 2β (+θ)ε 0 2 +(ξ +ξ 2)θε 0+ ( θ)ε 0m 0 L s (n) <. Meanwile, by Bernstein inequality we ave P max n= j U(n) (Ẑk(s j ) E [Ẑk (s j ))] ) } > η n /2 J n(β) k= C [c 4n β+ξ θε 0 ] { l U(n) P l (Ẑk(s ) E [Ẑk (s ))] } > η n /2 n= l=[c 3n β ξ θε 0 ] k= C [c 4n β+ξ θε 0 ] U(n) exp { } n ( ξ )θε 0 <, n= l=[c 3n β ξ θε 0 ] (B.28) were c 3 and c 4 are some positive constants. By (B.26) (B.28) and Borel Cantelli Lemma, (B.23) is proved. Furtermore, following te argument in (B.27), we ave ( max Z0 (s j ) + Z (n) (s j ) ) = o a.s. (B.29) j U(n) n β θε 0 Ten, by (B.23) and (B.29), we ave Π n, = o n β θε 0 a.s. (B.30) In view of (B.20), (B.22) and (B.30), equation (B.9) is proved. Proof of Teorem 4.. By taking L(u) = K(u) and using te tecnique of te proof of Teorem 3.2, we can prove (4.2). Equation (4.3) follows from (4.2) and E [ p n (x)] p s (x) = (p s (x + u) p s (x)) K(u)du = p s(x) uk(u)du + O( 2 ) = O( 2 ).

22 2 Proof of Teorem 4.2. By te definition of m n (x), we ave m n (x) = t= w n,t (x)e t + t= w n,t (x)m(x t ). By Teorem 3.2, we can sow tat Xt x K e t = o n β θε 0 t= a.s.. (B.3) Meanwile, by Teorem 4. we ave Xt x K p s (x) = O(2 ) + o n β θε 0 t= a.s. (B.32) By (4.6), (B.3) and (B.32), we ave t= By standard arguments, we ave ( w n,t (x)e t = o δ n n β θε 0 ) a.s. (B.33) = w n,t (x)m(x t ) m(x) t= m (x) t= t= K ( X t x) m(xt ) p n (x) K ( X t x = p n (x) =: Ξ n, (x) + Ξ n,2 (x). ) ( Xt x) m(x) p n(x) = p n (x) + m (x) 2 t= K ( X t x 2 p n (x) t= K ( X t x) (m(xt ) m(x)) ) ( Xt x) 2 p n (x) ( + o()) Since te conditions of Teorem 3. are satisfied wit L(u) = K(u)u, we ave Xt x Xt x K = o() a.s.. (B.34) t= Tus, by te conditions of Teorem 4.2 we obtain Ξ n, (x) = o (δn) a.s.. (B.35) Similarly, te conditions of Teorem 3. are also satisfied wit L(u) = K(u)u 2. Tis implies 2 Xt x Xt x K = o() a.s., (B.36) t=

23 22 wic, along wit te conditions of Teorem 4.2, implies Ξ n,2 (x) = O ( δ2n 2) a.s.. (B.37) Hence, in view of (B.33) (B.35), equation (4.8) in Teorem 4.2 olds. Proof of Teorem 4.3. By te definition of m n (x), we ave m n (x) = w n,t (x)e t + t= w n,t (x)m(x t ). t= Following te te proof of (B.33), we ave t= On te oter and, note tat ( w n,t (x)e t = o δ n n β θε 0 ) a.s. (B.38) w n,t (x)m(x t ) m(x) = t= were p n (x) = t= K x, (X t ) and t= K x, (X t ) (m(x t ) m(x)), p n (x) Xt x K n (m(x t ) m(x)) t= = m (x) (X t x) K Xt x n + m (x + ϑ 2 t(x t x)) t= t= (X t x) 2 Xt x Kn = m (x) (X t x) 2 Xt x Kn ( + o()), a.s., 2 j= were we ave used te fact tat and tat m ( ) is continuous, and 0 < ϑ t < for t =,, n. Finally, using te proof of (B.37), we ave n t= (X t x) K ( Xt x) n = 0 from te local linear metod w n,t (x)m(x t ) m(x) = O ( δ2n 2) a.s.. (B.39) t= By (B.38) and (B.39), te proof of Teorem 4.3 is terefore completed.