Empirical likelihood confidence intervals for the mean of a long-range dependent process

Transcription

1 Empirical likelihood confidence intervals for the mean of a long-range dependent process Daniel J. Nordman Dept. of Mathematics University of Wisconsin - La Crosse La Crosse, WI Philipp Sibbertsen Dept. of Statistics University of Dortmund Dortmund, Germany Soumendra N. Lahiri Dept. of Statistics Iowa State University Ames, IA Abstract: This paper considers blockwise empirical likelihood for real-valued time processes which may exhibit either short- or long-range dependence. Empirical likelihood approaches intended for weakly dependent time series can often break down in the presence of strong dependence. However, a modified blockwise method is proposed for confidence interval estimation of the process mean, which is valid for various dependence structures including long-range dependence. The finite-sample performance of the method is evaluated through a simulation study. Key Words: blocking, confidence interval, empirical likelihood, FARIMA, long-range dependence Contact address: Dan Nordman, University of Wisconsin-La Crosse, 1725 State Street, La Crosse, Wisconsin 54601, USA 1

2 1 Introduction Empirical likelihood (EL), proposed by Owen (1988, 1990), makes possible likelihood-based statistical inference without a specified distribution for the data. For independent data, the method generates a nonparametric likelihood that shares many qualities associated with parametric likelihood, such as limiting chi-square distributions for log-likelihood ratios [cf. Hall and La Scala (1990), Qin and Lawless (1994)]. Owen (2001) describes further properties and applications of EL, where much research has focused on EL with independent observations. The aim of this paper is to extend EL inference to dependent data, which could exhibit either weak or potentially strong forms of dependence. We show that EL inference on the process mean is possible for a large class of stationary time series, which we next detail. Let Z denote the set of integers. Suppose that {Y t }, t Z, is a sequence of stationary random variables with mean µ and integrable spectral density function f(λ), λ π, that satisfies f(λ) C d λ 2d, as λ 0, (1) for some d ( 1/2, 1/2) and positive constant C d > 0 (where denotes that the ratio of two terms equals one in the limit). When 0, we refer to the process {Y t } as weakly or shortrange dependent (SRD). The process {Y t } will be called strongly or long-range dependent (LRD) if d > 0. For d < 0, the process shall be termed anti-persistent. This classification of {Y t } as SRD or LRD depending on the value of d is common [cf. Hosking (1981)], in which long-range dependence (LRD) entails a pole of f at the origin [cf. Beran (1994)]. Formulations of weak dependence based on mixing assumptions often imply the 0 case of short-range dependence (SRD). For weakly dependent time series that satisfy a mixing condition to ensure SRD, Kitamura (1997) has proposed a blockwise EL method. This method applies the Owen (1990) formulation of EL to observational blocks rather than individual observations. As described in Kitamura (1997), blockwise EL involves data blocking techniques used successfully by other nonparametric likelihood methods for handling weak dependence, such as the block bootstrap and subsampling procedures [cf. Carlstein (1986), Künsch (1989), Politis and Romano (1993)]. However, blocking methods developed for SRD can often fail under LRD and require 2

3 careful modification. For example, the moving block bootstrap of Künsch (1989) and Liu and Singh (1992) is known to break down under forms of LRD [c.f. Lahiri (1993)]. The blockbased subsampling methods of Politis and Romano (1993) and Hall and Jing (1996), also developed for weak dependence, must be significantly reformulated for variance estimation with LRD [c.f. Hall, Jing and Lahiri (1998), Nordman and Lahiri (2004)]. Since the blockwise EL of Kitamura (1997) also involves data blocking intended for SRD, one might expect strong dependence to create complications with this version of EL as well. Indeed, our results indicate that this blockwise EL formulation can break down under either LRD or anti-persistence (i.e., d < 0), partially because the usual standardization n(ȳn µ) of a size n sample mean Ȳn might fail to produce a non-degenerate limit. This paper finds that an adapted version of Kitamura s (1997) EL is applicable for confidence intervals with time processes exhibiting very different kinds of dependence structures. We give a modified blockwise EL method for interval estimation of the process mean that is valid for SRD or LRD time series satisfying (1). This EL approach also has an advantage over confidence intervals based on the sample mean Ȳn in that direct variance estimation Var(Ȳn) is not required. In the weak dependence case (e.g., 0), we establish results on blockwise EL for the mean in a manner alternative to Kitamura (1997), which does not involve mixing conditions to quantify the decay rate of the process {Y t } covariances. The rest of the paper is organized as follows. Section 2 details assumptions and other background for describing the behavior of EL under strong and weak forms of dependence. A blockwise EL ratio for the mean and its limiting distribution under LRD are given in Section 3. For processes satisfying (1), a method of setting EL confidence intervals for the process mean is proposed. Section 4 provides a simulation study of these EL confidence intervals. Section 5 contains proofs of the main results. 2 Background 2.1 Time process assumptions We assume that the data (Y 1,..., Y n ) represent a realization from a (strictly) stationary process {Y t }, t Z, with spectral density satisfying (1). There then exists a process {ε j } of uncorrelated random variables with mean E(ε j ) = 0 and variance 0 < E(ε 2 j ) = σ2 < such 3

4 that {Y t } has a moving average representation of the form [cf. Chapter 16.7, Ibragimov and Linnik (1971)]: Y t = µ + j Z b j ε t j, t Z, where the constants {b j } fulfill 0 < j Z b2 j < and f(λ) = σ2 b(λ) 2 /(2π), λ π, with b(λ) = j Z b je jλ 1. We require two fairly mild assumptions for proving the results. Assumptions: (A.1). The strictly stationary process {Y t }, t Z, has a spectral density as in (1). For 0 < d < 1/2, the process autocovariances r(k) = Cov(Y t, Y t+k ), k Z satisfy r(k) C d R d k (1 2d), as k, (2) for some real R d 0; if 0, then f is bounded on every compact subinterval of (0, π]. (A.2). The random variables {ε j } are independent and identically distributed (iid). The process assumptions encompass many types of time series used for describing both SRD and LRD. In particular, our framework includes two popular models for LRD: the fractional Gaussian processes of Mandelbrot and Van Ness (1968) and the fractional autoregressive integrated moving average (FARIMA) models of Adenstedt (1974), Granger and Joyeux (1980) and Hosking (1981). When d > 0, we assume the covariances of the LRD process decay very slowly so that the sum of autocovariances k=1 r(k) diverges, whereas the process covariances sum to zero k Z r(k) = 0 when d < 0. Under Assumption (A.1), the behavior of f in (1) and the autocovariance decay in (2) are closely related when d 0. For d > 0, both (1) and (2) are often used to formulate LRD [cf. Beran (1994)], although the conditions are not necessarily equivalent [Robinson (1995a), p. 1634]. The autocovariance representation (2) implies the limit of f in (1) for the case d < 0 if R 2Γ(1 2d) sin(dπ) [cf. (2.6.V), (2.24.V) of Zygmund (1968)]; additionally for d > 0, if f is of bounded variation on compact subintervals of (0, π], then the pole of f in (1) implies that (2) holds. If the covariances r(k) are decreasing [Bingham et al. (1987), p. 240] or quasimonotonically convergent to zero [Robinson (1995a), p. 1634], then (1) and (2) are equivalent for prescribing LRD that is for d > 0. 4

5 By our condition for SRD under (A.1) (i.e., 0), we assume that a finite, positive limit lim n n 1 n k=1 (n k)r(k) exists but we do not explicitly require the absolute summability of the covariances k=1 r(k) <, which is a slightly stronger assumption that is often used to categorize SRD. The innovation assumption (A.2) could perhaps be weakened to allow strictly stationary innovations that satisfy a mixing condition [cf. Lahiri (2003)]. But to facilitate our exposition and proofs, we do not pursue this further. 2.2 Asymptotic distribution of the sample mean and related issues To help motivate a theory for EL under LRD, we review some differences in the behavior of the sample mean Ȳn = n i=1 Y i/n of SRD and LRD data (Y 1,..., Y n ). Theorem 1 describes how the correct scaling a n for the standardized sample mean (Ȳn µ)/a n to have a limit distribution depends crucially on the dependence parameter d in (1). In the following, denote convergence in distribution by d. Theorem 1 Let {Y t }, t Z be a process which satisfies Assumption (A.1), d ( 1/2, 1/2). (a) As n, n 1 2d Var(Ȳn) V d > 0, where V C d R d /{d(1+2d)} if d 0 and V 0 = 2πC 0 for 0. (b) Let a 2 n = n 1+2d V d. If Assumption (A.2) holds in addition, then (Ȳn µ)/a n n, where Z denotes a standard normal random variable. d Z as Under SRD, Var(Ȳn) decays at the usual O(1/n) rate associated with iid data. However, under LRD or anti-persistence, the decay rate of Var(Ȳn) may be slower or faster than O(1/n), which implies that the usual n scaling of the sample mean n(ȳn µ) may fail to produce a limit distribution. This aspect of LRD complicates inference on the process mean E(Y t ) = µ based on either the sample mean Ȳn or data blocking techniques developed for SRD, such as the block bootstrap, subsampling and even the blockwise EL in Kitamura (1997) [cf. Lahiri (1993), Hall, Jing and Lahiri (1998)]. To recognize potential problems with blockwise EL under LRD, it helps to understand why the method is successful for SRD. For inference on the process mean E(Y t ) = µ, Kitamura (1997) explains that the non-blocking version of EL fails under SRD by neglecting the data dependence structure (i.e., using individual observations) which causes the variance 5

6 Var(Ȳn) of a sample mean to be improperly estimated. However, the blockwise EL of Kitamura (1997) permits correct variance estimation under weak dependence (e.g., 0) by exploiting the following property of SRD processes: Var(Ȳn) Var(Ȳl) l n (3) for time series lengths l, n, l/n 0. Hence, in the inner mechanics of the blockwise EL under SRD, a collection of smaller length l < n data blocks from the available series (Y 1,..., Y n ) can be used to estimate the variance Var(Ȳl) of a size l sample mean and this estimate can be rescaled for inference on the full size n sample mean by (3): Var(Ȳn) (l/n)var(ȳl). Consequently, blockwise EL statistics in Kitamura (1997) require adjustments involving n/l [cf. Kitamura (1997), p. 2089]. Note that, under SRD, subsampling techniques provide similar block-based variance estimates of Var(Ȳn) from (3) [cf. Politis and Romano (1993)]. However, we see from Theorem 1 that (3) may fail outside of the SRD case which implies the blockwise EL of Kitamura (1997) must be modified for LRD. 3 Blockwise empirical likelihood under long-range dependence 3.1 Construction of blockwise empirical likelihood for the mean We give an EL function for the process mean E(Y t ) = µ, following the blockwise formulation given in Kitamura (1997). Let 1 l n be the block length. Under a maximally overlapping (OL) version of blocking, we define blocks B l i,ol = (Y i,..., Y i+l 1 ), 1 i N OL n l + 1. In this case, we use the maximal number of length l blocks among the observed time sequence (Y 1,..., Y n ). We can also consider another common blocking scheme involving nonoverlapping (NOL) blocks given by B l i,nol = Bl 1+l(i 1),OL for 1 i N OL n/l, where n/l represents the smallest integer not exceeding n/l. In the following, let N denote either N OL or N NOL and write the associated blocks B l i,ol or B l i,nol as Bl i, 1 i N. Let M li denote the sample mean of the l variables in B l i, 1 i N (e.g., M li = i+l 1 j=i Y j /l if B l i = Bl i,ol ). To assess the plausibility of a value for the mean parameter, we assign probabilities {p i } N i=1 to each block sample mean {M li } N i=1 to develop a multinomial likelihood function, N i=1 p i. 6

7 The profile blockwise EL function for µ is given by { N } N N L n (µ) = sup p i : p i > 0, p i = 1, p i M li = µ i=1 i=1 i=1 (4) where we maximize the product in (4) over distributions or probabilities {p i } N i=1 on {M li} N i=1 with expectation µ. The function L n (µ) is positive for µ ( min M l,i, max M l,i) and 1 i N 1 i N outside of this interval we may define L n (µ) =. When positive, L n (µ) represents a maximum realized at unique weights p i = N 1 {1+λ µ (M li µ)} 1 where λ µ R is determined by 0 = N i=1 M li µ 1 + λ µ (M li µ) g µ(λ µ ). (5) Owen (1988, 1990) discusses these and further computational aspects of EL. Without the mean µ linear constraint in (4), the product N i=1 p i is maximized when each p i = N 1, corresponding to the empirical distribution of {M li } N i=1. The profile empirical likelihood ratio for the mean µ is then given by R n (µ) = L n(µ) N N = N {1 + λ µ (M li µ)} 1 (6) i=1 and a confidence interval for the mean is determined by those values of µ with relatively high EL, namely, intervals of the form {µ : R n (µ) A}, (7) where A > 0 is chosen to obtain a desired confidence level [cf. Hall and La Scala (1990), Theorem 2.2]. We next discuss the asymptotic distribution of the EL ratio in (6) required for calibrating the confidence intervals in (7). 3.2 Limit distribution of empirical likelihood ratio A key feature of EL with independent data is that it allows a nonparametric casting of Wilks theorem for constructing confidence regions [Wilks (1938)], meaning that the logarithm of the EL ratio has an asymptotic chi-square distribution when evaluated at a true parameter value [cf. Owen (1988, 1990)]. Kitamura (1997) showed a similar result applies to the blockwise EL with weakly dependent mixing processes. 7

8 We now give the asymptotic distribution of the blockwise EL ratio in (6) for SRD or LRD time processes fulfilling (1). As in the EL framework of Kitamura (1997), we require a block correction factor to ensure our EL ratio has a non-degenerate limit. Write B n = N 1 (n/l) 1 2d for the block adjustment term. Theorem 2 Suppose {Y t }, t Z satisfies Assumption (A.1)-(A.2) and l 1 + l/n = o(1), (n/l 2 ) O(1). If E(Y t ) = µ 0 denotes the true process mean, then as n, 2B n log R n (µ 0 ) d χ 2 1 (8) where χ 2 1 denotes a chi-square random variable with 1 degree of freedom. In the case of SRD with 0 in (1), the block adjustment factor B n reduces to the one (n/l)n 1 used by Kitamura (1997) [p. 2089] for weakly dependent processes. However, we find that a blocking adjustment B n that is appropriate for SRD or LRD must generally incorporate the dependence exponent d from (1). We also note one important implication about the block size l in Theorem 2. Under LRD, we found the block condition (n/l 2 ) O(1) to be necessary in deriving the distribution of the blockwise log-el ratio. This represents an additional constraint over the usual l 1 +l/n = o(1) block stipulation under weak dependence (i.e., 0). However, a block size choice of l = C n, C > 0, fulfills all requirements of Theorem 2 without knowledge of the exact value of d in (1). Interestingly, Hall, Jing and Lahiri (1998) proposed similar block lengths l = C n (e.g., C = 1, 3) for a subsampling method under LRD and in a sense the distributional result in Theorem 2 supports this blocking intuition. 3.3 Empirical likelihood confidence intervals for the mean When the dependence structure is unknown, we require a consistent estimator of d in the block adjustment B n without rigid model assumptions and several such estimators ˆd n are available based on the periodogram (see Section 3.4). Provided with a consistent estimator ˆd n of d, we use (8) to set an EL confidence interval for the mean µ of a time process satisfying (1). Namely, an approximate two-sided 100(1 α)% confidence interval for µ, of the form (7), is given by I n (1 α) = { µ R : 2 B } n log R n (µ) χ 2 1;1 α, Bn = N 1 (n/l) 1 2 ˆd n, (9) 8

9 where χ 2 1;1 α represents the 1 α percentile of a χ2 1 random variable, 0 < α < 1. One-sided approximate 100(1 α)% lower/upper confidence bounds for µ correspond to the lower/upper endpoints of the interval I n (1 2 1 α). By choosing a block length l = C n, C > 0, the above EL interval has asymptotically correct coverage for any dependence type d under (1), as detailed in Theorem 3. Let p denote convergence in probability in the following. Theorem 3 Suppose {Y t }, t Z satisfies Assumption (A.1)-(A.2), the estimator ˆd n of d ( 1/2, 1/2) fulfills ˆd n d log n the true process mean, then as n, p 0, and l = C n for some C > 0. If E(Y t ) = µ 0 denotes ( ) P µ 0 I n (1 α) 1 α. EL confidence intervals for the mean µ have an advantage over intervals set immediately with Ȳn under Theorem 1 because direct variance estimation of Var(Ȳn) is not needed. That is, confidence intervals based on a normal approximation of (Ȳn µ)/a n would require at least estimation of the constant C d in (1) in addition to an estimate of the exponent d. However, semi-parametric estimates for C d, which do not require strong assumptions on f, are known to be less accurate (i.e., slower convergence rates) than corresponding estimators for d, as in the case of log-periodogram regression [cf. Robinson (1995b)]. 3.4 Estimation of d We describe two possible frequency domain estimators for the dependence parameter d required in the adjustment B n from (8). Define the periodogram I n (λ) = (2πn) 1 n t=1 Y te λt 1 2, λ [0, π] and let m be a bandwidth parameter such that m, m/n 0 as n. The popular Geweke-Porter-Hudak (GPH) estimator of d is found by a regression of the log-periodogram against the first m Fourier frequencies: ˆd GP H m,n = m j=1 (g j ḡ) log(i n (λ j )) m j=1 (g j ḡ) 2, where ḡ = m j=1 g j/m with g j = 2 log 1 e λ j 1, λ j = 2πj/n, j = 1,..., m [Geweke and Porter-Hudak (1983)]. Consistency of the estimator, with a rate of convergence ( ˆd GP H m,n d) = O p (m 1/2 ), can be obtained without knowledge of the distribution of the data generating process [cf. Robinson (1995b) or Hurvich et al. (1998)]. However, because the representation 9

10 (1) of the spectral density holds only near the origin, a trade-off between the bias and variance of ˆd GP H m,n must be made in choosing the optimal bandwidth m. Whereas Geweke and Porter-Hudak(1983) proposed m = n 1/2, which is still used in many applications, Hurvich et al. (1998) showed that a bandwidth m = O(n 4/5 ) is mean squared error (MSE)-optimal. A second possibility for ˆd, proposed by Künsch (1987), is the so-called Gaussian semiparametric estimate (GSE) or local Whittle estimate ˆd GSE m,n = argmin d [ 1/2+ɛ,1/2 ɛ] log 1 m m j=1 j I n (λ j ) 2d m λ 2d m log(λ j ), involving a small fixed ɛ > 0. Robinsion (1995a) established particularly mild conditions for the consistency of ˆd GSE m,n at a rate m 1/2 in probability. Compared to the GPH estimator that only requires a linear regression, the GSE estimator is more complicated to compute but, under certain conditions, its asymptotic variance Var( ˆd GSE m,n ) 1/(4m) is smaller relative to Var( ˆd GP H m,n ) π 2 /(24m), as n. The MSE-optimal bandwidth for be m = O(n 4/5 ) [cf. Andrews and Sun (2004)]. See Moulines and Soulier (2003) for further theoretical properties of other possible estimators of d. j=1 ˆd GSE m,n is also known to ˆd GP H m,n, ˆd GSE m,n and for 4 A numerical study We conducted a simulation study of the finite-sample coverage accuracy of blockwise EL confidence intervals. The design and results of the study are described in the following. 4.1 Simulation parameters Let {Ỹt}, t Z, represent a FARIMA(0, d, 0) series Ỹ t = j=0 Γ(j + d) Γ(j + 1)Γ(d) ε t j, (10) involving iid (mean zero) innovations {ε t }, t Z and the gamma function Γ( ). To examine the performance of the EL method for several time series fulfilling (1), we considered FARIMA processes Y t = ϕy t 1 + Ỹt + ϑỹt 1, t Z, formed with one of the following ARMA filters, innovation distributions and d values: 10

11 ϕ = ϑ = 0 (); ϕ = 0.3, ϑ = 0.1 (); ϕ = 0.7, ϑ = 0.3 (); {ε t } in (10) has a standard normal or t distribution with 3 degrees of freedom; 0.1, 0, 0.1, 0.25, 0.4. The processes {Y t } above fulfill (1) with various decay rates d based on innovations that may be Gaussian or non-gaussian with heavy tails. Each process also has mean zero E(Y t ) = 0. We obtained size n time stretches (Y 1,..., Y n ) from each FARIMA series above by using a sample Ỹn = (Ỹ1,..., Ỹn) from the process (10) as the innovations in the appropriate ARMA model. Gaussian samples Ỹn were simulated by the circulant embedding method of Wood and Chan (1994) with FARIMA(0, d, 0) covariances [cf. Beran (1994)], whereas non- Gaussian Ỹn were generated by truncating the moving average expression in (10) after the first M = 1000 terms and then using n+m independent t variables to build an approximate truncated series [see Bardet et al (2003), p. 590 for details]. The sample sizes considered were n = 100, 500, For each series type and sample size, we calculated lower and upper approximate 95% one-sided confidence intervals (or bounds) with the blockwise EL method from (9) using both overlapping (OL) or non-overlapping (NOL) blocks of length l = C 1 n 1/2, C 1 {1/2, 1, 2} as well as the two estimators ˆd GP H m,n and ˆd GSE m,n of d from Section 3.4 with bandwidths m = C 2 n 4/5, C 2 {1/4, 1/2, 1}. Coverage probabilities for the EL confidence bounds for the process mean µ = 0 were approximated by an average over 1000 simulation runs for each process and sample size n. 4.2 Coverage accuracy of EL intervals To report the simulation results, we firstly note that the block type had little effect on coverage accuracy of the EL intervals. Under weak dependence, similarities in the finite-sample efficacy of OL and NOL blocks have also been observed with other block-based methods [cf. Hall and Jing (1996)]. Focusing on OL blocks, it suffices to summarize numerical findings only for the upper confidence bounds, because the lower bounds closely matched the upper bounds in performance at each level combination in the study. Figures 1-4 display the coverage accuracy, expressed as a percentage, of upper 95% EL confidence bounds for the mean. These graphics are differentiated by the innovation type (Gaussian/t) and the estimator of d (GPH/GSE) 11

12 used; each figure also involves the same two bandwidths m = n 4/5 /4 or n 4/5 /2. Based on the figures, we make some observations on the effects of the following factors: d estimator: There are few differences in the EL coverage accuracies across GPH and GSE estimators. Perhaps the performance with the GPH estimator is slightly better when the memory parameter d is larger. innovation type: The EL intervals perform slightly better for t-distributed innovations than for Gaussian innovations, which appear to have somewhat lower coverage probabilities for 0.4 in particular. bandwidth m: Performance differences in bandwidths m = n 4/5 /4 or n 4/5 /2 appear only with, which introduces a high positive autoregressive component. For this filter, the EL confidence bounds become conservative when m = n 4/5 /2 and perform better with a shorter bandwidth m = n 4/5 /4. Because a process with a high positive autoregressive coefficient can mimic long memory and can be difficult to distinguish from LRD, a shorter bandwidth may help to refine estimation of the memory exponent d in this situation. The EL coverage accuracies with bandwidths m = n 4/5 and m = n 4/5 /2 were comparable throughout the study. block length l: The blockwise EL confidence intervals tend to become more anticonservative as the block sizes increase. This behavior is most apparent with the smallest sample size n = 100 and also under the strongest forms of LRD 0.4. The smallest block size considered l = n 0.5 /2 appears to yield the most accurate EL intervals. sample size n: With the smallest sample size n = 100, the coverage accuracy of the EL intervals quickly deteriorates as the memory parameter d increases. This type of behavior is well-known in location estimation with strong LRD processes due to the persistence of the series [cf. Beran (1994)], so that a size n = 100 sample may be too small for inference in these cases. However, the coverage probabilities of EL intervals become closer to the nominal level as the sample sizes increase and the accuracy is often quite good even for a sample size of n = 500. The simulation evidence here indicates that a block size l = n 0.5 /2, along with GPH estimator of d with bandwidth m = n 0.8 /4, may be recommended for the coverage accuracy of the 12

13 blockwise EL confidence bounds for the process mean. The performance of the EL method generally depends on the strength of the underlying dependence structure, but improves with increasing sample sizes. It is presently unclear if an optimal block size l exists for computing the blockwise EL intervals (9). Under weak dependence, optimal block lengths (and corresponding estimators) are available for implementing some block-based methods, such as the block bootstrap and subsampling [cf. Hall and Jing (1996), Lahiri (2003), Chapter 7]. However, more research on block lengths is required both with blockwise EL and with strong dependence. 5 Proofs For the proofs, we denote the positive integers as N and use C to denote generic positive constants that do not depend on n. For two sequences {s n }, {t n } of nonzero numbers, we write s n t n to denote lim n s n /t n = Proof of Theorem 1 Theorem 1(b) follows from part (a) because the process {Y t } is linear with iid innovations and Var(n Ȳn) [cf. Davydov (1970), Theorem of Ibragimov and Linnik (1971)]. Theorem 1(a) is next briefly justified. When d > 0, the result is standard and follows from the covariance decay in (2) [c.f. Corollary A2 of Taqqu (1977)]. When 0, we use the nonnegative Fejer kernel K n (λ) = (2πn) 1 n j= n (n j )ejλ 1, λ π, to write π nvar(ȳn) 2πC 0 = 2π K n (λ) [ ] f(λ) C 0 dλ, π since π π K n(λ)dλ = 1. For a fixed 0 < δ π, it holds that f(λ) C 0 is bounded on δ λ π and δ λ π K n(λ)dλ 0 as n [c.f. Brockwell and Davis (1987), Section 2.11] so that lim sup n nvar(ȳn) 2πC 0 2π sup 0< λ δ f(λ) C 0. (11) Letting δ 0, the righthand side above converges to zero by (1), establishing the case 0. When d < 0, we use (2) and k Z r(k) = 0 [which follows from k Z r(k) < and (1) by 13

14 Corollary of Brockwell and Davis (1987)] to deduce that A 1n n k= n r(k) = 2 k=n+1 r(k) C dr d d n2d, A 2n 2n 1 n k=1 kr(k) 2C dr d 1 + 2d n2d, as n by Corollary A2 of Taqqu (1977). Since nvar(ȳn) = A 1n A 2n, Theorem 1(a) now follows for d < Proof of Theorem 2 To prove Theorem 2, we require some preliminary results. For LRD and SRD linear processes with bounded innovations, Proposition 1 implies that overlapping block means at distant lags are asymptotically uncorrelated, based on a condition suggested by Hall, Jing and Lahiri (1998), Theorem 2.4. Proposition 1 Suppose l 1 + l/n = o(1) and {Y t }, t Z, satisfies the conditions of Theorem 1(b) with bounded innovations, i.e., P ( ɛ t C) = 1 for some C > 0. Then, for any integers a, b N {0} and 0 < ɛ < 1, max nɛ i n E [ a a n (M l1,ol µ) a a b n (M li,ol µ) b] E(Z a ) E(Z b ) = o(1), as n, where M li,ol = i+l 1 j=i Y j /l, i 1, and Z is a standard normal variable. Proof of Proposition 1. The case d > 0 is treated in Nordman and Lahiri (2004, Lemma 2) which requires Var(Ȳn)/Var(Ȳl) = (l/n) 1 2 o(1) and n 2 Var(Ȳn). The cases 0 and d < 0 follow with a modification to show (essentially for the a = b = 1 situation) that n,ɛ max nɛ i n Cov(M l1,ol, M li,ol ) /a 2 l = o(1) holds for a fixed 0 < ɛ < 1. When d > 0, we may use (2) and Theorem 1(a) to show ( n,ɛ Cl 1 2d max max r(i + j) = O (l/n) 1 2d) = o(1). nɛ i n j l When 0 and a 2 l = 2πC 0l 1 by Theorem 1(a), we have with the Fejer kernel K l that Cov(M l1,ol, M li,ol ) a 2 l = 1 π K l (λ) e (i 1)λ 1 [ ] f(λ) C 0 dλ, i > l, C 0 π holds since π π K l(λ) e (i 1)λ 1 dλ = 0 for i > l; then, n,ɛ = o(1) follows from the same argument as in (11) as l using e (i 1)λ 1 = 1, λ π, i N. 14

15 Proposition 2 Suppose l 1 + l/n = o(1) and {Y t }, t Z, satisfies the conditions of Theorem 1(b). Let M nµ = N 1 N i=1 (M li µ) and â 2 lµ = N 1 N i=1 (M li µ) 2, where N represents either N OL or N NOL (i.e., either block type). Then, as n, (a) M nµ /a n (b) â 2 lµ /a2 l d Z, a standard normal variable; p 1; (c) P ( min 1 i N M li < µ < max 1 i N M li ) 1. Proof of Proposition 2. We consider only the overlapping block case N = N OL = n l+1. The proof with N = N NOL = n/l is very similar and hence omitted. Noting that the first and last l observations in (Y 1,..., Y n ) contribute less to M nµ, we may write N M nµ = n(ȳn µ) H n where lh n = l j=1 (l j)(y j +Y n j+1 2µ) = l 1 i=1 i j=1 (Y j + Y n j+1 2µ). Using Holder s inequality and n 2 a 2 n = n 1+2d V d, n N, we find E(Hn) 2 4 l 1 i Var l i=1 j=1 Y j ( ) 1 l 1 = O i 2 a 2 i l i=1 ( = O l 1+2d) = o(n 2 a 2 n) so that (na n ) 1 H n p 0 follows. Then applying Slutsky s theorem with Theorem 1(b) and n/n 1, we have the distributional result for M nµ /a n in Proposition 2(a). Let E(ε 2 0 ) = σ2 and denote the indicator function as I{ }. For each c N and t Z, define variables ε t,c = ε t I{ ε t c} E(ε t I{ ε t c}) and Y t,c = µ + j Z b j(σε t j,c /σ c ), where w.l.o.g. σc 2 = E(ε 2 0,c ) > 0. For each n, c, i N, write analogs M li,c = i+l 1 j=i Y j,c /l and â 2 lµ,c = N 1 N i=1 (M li,c µ) 2 with respect to Y 1,c,..., Y n,c. The processes {Y t } and {Y t,c } have identical means µ and covariances because each series involves the same linear filter with iid innovations of mean 0 and variance σ 2. Since E â 2 lµ â2 lµ,c E {(M l1 µ) + (M l1,c µ)}(m l1 M l1,c ) holds, we may apply Holder s inequality to show that for each c N, E â 2 lµ â2 lµ,c 4Var(Ȳl){1 σ 1 c σ 1 E(ε 0 ε 0,c )} 1/2, (12) using Var(Ȳl) = Var(M l1 ) = Var(M l1,c ) and Var(M l1 M l1,c ) = Var(Ȳl) Var(σ 1 ε 0 σ 1 c ε 0,c ) by the iid property of the innovations. For each c N, we find that E(â 2 lµ,c ) = Var(Ȳl) a 2 l and, through applying Proposition 1, that Var(â 2 lµ,c ) = o(a4 l ) follows as in the proof of 15

16 Theorem 2.5 of Hall, Jing and Lahiri (1998). From this and (12), we deduce that â 2 lµ 1 lim sup E 1 lim sup {E â 2 lµ n â2 lµ,c + [ } Var(â 2 lµ,c )] 1/2 + E(â 2 lµ,c ) a 2 l n a 2 l for each c N. Proposition 2(b). a 2 l 4{1 σ 1 c σ 1 E(ε 0 ε 0,c )} 1/2 Because lim c σc 1 E(ε 0 ε 0,c ) = σ, it follows now that â 2 lµ /a2 l p 1 in For x R, let Φ(x) = P (Z x) denote the distribution function of a standard normal Z and define Φ n (x) = N 1 N i=1 {(M li µ)/a l x}. Fix δ > 0. To show Proposition 2(c), it suffices to prove that if δ = δ, then Φ n (δ ) Φ(δ ) as n ; such convergence implies ( ) P min (M i µ) δ < 0 < δ < max (M i µ) 1 1 i N 1 i N p because 0 < Φ(±δ) < 1. We next establish that lim n E Φ n (δ ) Φ(δ ) = 0, for δ = ±δ, which will follow from (13)-(14) below. For x R and c N, define Φ n,c (x) = N 1 N i=1 {(M li,c µ)/a l x}. Since the process {Y t,c } has all moments finite, the convergence in Proposition 1 applies, and (M l1,c µ)/a l converges to a standard normal by Theorem 1(b), we have that lim E Φ n,c (δ ) Φ(δ ) = 0 (13) n holds for any fixed c N by Theorem 2.4 of Hall, Jing and Lahiri (1998) [see the proof of their Theorem 2.4]. Using the standard normal limits of (M l1 µ)/a l and (M l1,c µ)/a l under Theorem 1 and that lim n E(M l1,c M l1 ) 2 /a 2 l c N, we can show = 2{1 σ 1 c σ 1 E(ε 0 ε 0,c )} for each lim c lim sup E Φ n (δ ) Φ n,c (δ ) 2 n lim lim sup c n ( { (Ml1 µ) E I δ } { (Ml1,c µ) I δ }) 2 = 0, (14) a l a l as in the proof of Theorem 1 of Nordman and Lahiri (2004). Proof of Theorem 2. To save space, we consider only for the overlapping version of blockwise EL from Section 3.1 with N = N OL = n l + 1. The proof with non-overlapping blocks is analogous but less involved. 16

17 Applying Lemma 2(c) with µ = µ 0, we find P (R n (µ 0 ) > 0) 1 as n so that, with probability approaching 1 as n, R n (µ 0 ) can be written as in (6) with probabilities p i = N 1 {1 + λ µ0 (M li µ 0 )} 1, 1 i N, where g µ0 (λ µ0 ) = 0 in (5). Let Z nµ0 = max 1 i N M li µ 0. By the finiteness of E Y t 2, the Borel-Cantelli lemma and stationarity may be used to show max 1 i n Y i µ 0 = o p (n 1/2 ) so that Z nµ0 = o p (l 1 n 1/2 ) as n [cf. Lemma 3 of Owen (1990)]. Then, we find that ( ) n a 2 d l a n Z nµ0 = O O p (n 1/2 l Z nµ0 ) = O(1)o p (1) = o p (1), (15) l 2d where n d /l 2 O(1) by assumption. Let M nµ0 = N 1 N i=1 (M li µ 0 ) and â 2 lµ 0 = N 1 N i=1 (M li µ 0 ) 2. Similar to step (2.12) of Owen (1990), it follows from g µ0 (λ µ0 ) = 0 in (5) that using M nµ0 /â 2 lµ 0 = O p (a n /a 2 l ) from Proposition 2 and (15). λ µ0 = O p (a n /a 2 l ), (16) Define θ i = λ µ0 (M li µ 0 ) for 1 i N. The equation 0 = g µ0 (λ µ0 ) can be solved for λ µ0 = ( M nµ0 + I nµ0 )/â 2 lµ 0, where I nµ0 = N 1 N i=1 θ2 i (M li µ 0 )/(1 + θ i ) satisfies I nµ0 λ µ0 2 Z nµ0 â 2 lµ 0 max 1 i N 1 + θ i 1 = o p (a n ), by (15)-(16), Proposition 2(b), and max 1 i N θ i λ µ0 Z nµ0 = o p (1). When λ µ0 Z nµ0 < 1, a Taylor expansion gives log(1 + θ i ) = θ i θ 2 i /2 + ν i for each 1 i N where ν i λ µ0 3 Z nµ0 (M li µ 0 ) 2 (1 λ µ0 Z nµ0 ) 3. (17) Using this expansion of log(1 + θ i ), 1 i N, in (6) along with the expression for λ µ0 B n = N 1 a 2 n a 2 l, we now write and where Q 1n = (a 1 n 2B n log R n (µ 0 ) = Q 1n Q 2n + Q 3n, M nµ0 ) 2 /(a 2 l â 2 lµ 0 ) d χ 2 1 by Proposition 2; Q 2n = (a 1 n I nµ0 ) 2 /(a 2 l â 2 lµ 0 ) = o p (1) applying I nµ0 = o p (a n ); and for Q 3n = 2B n N i=1 ν i, we may bound Q 3n B n N i=1 ν i 2a 2 l a 2 n λ µ0 3 Z nµ0 â 2 lµ 0 (1 λ µ0 Z nµ0 ) 3 = o p (1) by (17). Theorem 2 follows now by applying Slutsky s theorem. Proof of Theorem 3. From Theorem 2 and l = C n in (9) by assumption, it suffices to show B n /B n = (C 2 n) (d ˆd n ) p 1, which follows from (d ˆd n ) log(c 2 p n) 0. 17

18 Acknowledgments This research was partially supported by US National Science Foundation grants DMS and DMS and by the Deutsche Forschungsgemeinschaft (SFB 475). References Adenstedt, R. K. (1974) On large-sample estimation for the mean of a stationary sequence. Ann. Statist. 2, Andrews D. W. K. and Sun, Y. (2004) Adaptive local polynomial Whittle estimation of long-range dependence. Econometrica 72, Bardet, J., Lang, G., Oppenheim, G., Philippe, S. and Taqqu, M. S. (2003) Generators of long-range dependent processes: a survey. In P. Doukhan, G. Oppenheim and M. S. Taqqu (eds.), Theory and Applications of Long-range Dependence, pp Birkhäuser, Boston. Bingham, N. H., Goldie, C. M. and Teugels, J. L. (1987) Regular variation. University Press, Cambridge. Beran, J. (1994) Statistical methods for long memory processes. Chapman & Hall, London. Brockwell, P. J. and Davis, R. A. (1987) Time series: theory and methods. Springer-Verlag, New York. Carlstein, E. (1986) The use of subseries methods for estimating the variance of a general statistic from a stationary time series. Ann. Statist. 14, Davydov, Y. A. (1970) The invariance principle for stationary processes. Theor. Prob. Appl. 15, Geweke, J. and Porter-Hudak, S. (1983) The estimation and application of long-memory time series models. J. Time Series Anal. 4, Granger, C. W. J. and Joyeux, R. (1980) An introduction to long-memory time series models and fractional differencing. J. Time Series Anal. 1, Hall, P. and La Scala, B. (1990) Methodology and algorithms of empirical likelihood. Internat. Statist. Rev. 58, Hall, P. and Jing, B.-Y. (1996) On sample reuse methods for dependent data. J. R. Stat. Soc., Ser. B 58, Hall, P., Jing, B.-Y. and Lahiri, S. N. (1998) On the sampling window method for long-range dependent data. Statist. Sinica 8, Hosking, J. R. M. (1981) Fractional differencing. Biometrika 68,

19 Hurvich, C. M., Deo, R. and Brodsky, J. (1998) The mean square error of Geweke and Porter- Hudak s estimator of the memory parameter of a long memory time series. J. Time Series Anal. 19, Ibragimov, I. A. and Linnik, Y. V. (1971) Independent and stationary sequences of random variables. Wolters-Noordhoff, Groningen. Kitamura, Y. (1997) Empirical likelihood methods with weakly dependent processes. Ann. Statist. 25, Künsch, H. R. (1987) Statistical aspects of self-similar processes. Probability Theory and Applications 1, Künsch, H. R. (1989) The jackknife and bootstrap for general stationary observations. Ann. Statist. 17, Lahiri, S. N. (1993) On the moving block bootstrap under long range dependence. Statist. Probab. Lett. 18, Lahiri, S. N. (2003) Resampling methods for dependent data. Springer, New York. Liu, R. Y. and Singh, K. (1992) Moving blocks jackknife and bootstrap capture weak dependence. In Exploring the Limits of Bootstrap, , R. LePage and L. Billard (editors). John Wiley & Sons, New York. Mandelbrot, B. B. and Van Ness, J. W. (1968) Fractional Brownian motions, fractional noises and applications. SIAM Rev. 10, Moulines, E. and Soulier, P. (2003) Semiparametric spectral estimation for fractional processes. In P. Doukhan, G. Oppenheim and M. S. Taqqu (eds.), Theory and Applications of Long-range Dependence, pp Birkhäuser, Boston. Nordman, D. J. and Lahiri, S. N. (2004) Validity of the sampling window method for long-range dependent linear processes. Preprint. Department of Statistics, University of Dortmund, Germany. Owen, A. B. (1988) Empirical likelihood ratio confidence intervals for a single functional. Biometrika 75, Owen, A. B. (1990) Empirical likelihood confidence regions. Ann. Statist. 18, Owen, A. B. (2001) Empirical likelihood. Chapman & Hall, London. Politis, D. N. and Romano, J. P. (1993) On the sample variance of linear statistics derived from mixing sequences. Stochastic Processes Appl. 45, Robinson, P.M. (1995a) Gaussian semiparametric estimation of long range dependence. Ann. Statist. 23, Robinson, P.M. (1995b) Log-periodogram regression of time series with long range dependence. Ann. Statist. 23,

20 Qin, J. and Lawless, J. (1994) Empirical likelihood and general estimating equations. Ann. Statist. 22, Taqqu, M. S. (1977) Law of the iterated logarithm for sums of non-linear functions of Gaussian variables that exhibit a long range dependence. Z. Wahr. Verw. Gebiete 40, Wilks, S. S. (1938) The large-sample distribution of the likelihood ratio for testing composite hypotheses. Ann. Math. Statist. 9, Wood, A. T. A. and Chan, G. (1994) Simulation of stationary Gaussian processes in [0, 1] d. J. Comput. Graph. Statist. 3, Zygmund, A. (1968) Trigonometric Series Volumes I, II. Cambridge University Press, Cambridge, U.K. 20

21 OL block length n^{0.5}/2; GPH m=n^{0.8}/4 n= 100 n= 500 n= 1000 OL block length n^{0.5}; GPH m=n^{0.8}/4 OL block length 2n^{0.5}; GPH m=n^{0.8}/4 OL block length n^{0.5}/2; GPH m=n^{0.8}/2 n= 100 n= 500 n= 1000 OL block length n^{0.5}; GPH m=n^{0.8}/2 OL block length 2n^{0.5}; GPH m=n^{0.8}/2 Figure 1: Gaussian innovations: Coverage percentages for approximate 95% one-sided upper EL confidence bounds using GPH estimator with bandwidth m = C 2 n 4/5, C 2 {1/4, 1/2}. 21

22 OL block length n^{0.5}/2; GPH m=n^{0.8}/4 n= 100 n= 500 n= 1000 OL block length n^{0.5}; GPH m=n^{0.8}/4 OL block length 2n^{0.5}; GPH m=n^{0.8}/4 OL block length n^{0.5}/2; GPH m=n^{0.8}/2 n= 100 n= 500 n= 1000 OL block length n^{0.5}; GPH m=n^{0.8}/2 OL block length 2n^{0.5}; GPH m=n^{0.8}/2 Figure 2: t-innovations: Coverage percentages for approximate 95% one-sided upper EL confidence bounds using GPH estimator with bandwidth m = C 2 n 4/5, C 2 {1/4, 1/2}. 22

23 OL block length n^{0.5}/2; GSE m=n^{0.8}/4 n= 100 n= 500 n= 1000 OL block length n^{0.5}; GSE m=n^{0.8}/4 OL block length 2n^{0.5}; GSE m=n^{0.8}/4 OL block length n^{0.5}/2; GSE m=n^{0.8}/2 n= 100 n= 500 n= 1000 OL block length n^{0.5}; GSE m=n^{0.8}/2 OL block length 2n^{0.5}; GSE m=n^{0.8}/2 Figure 3: Gaussian innovations: Coverage percentages for approximate 95% one-sided upper EL confidence bounds using GSE estimator with bandwidth m = C 2 n 4/5, C 2 {1/4, 1/2}. 23

24 OL block length n^{0.5}/2; GSE m=n^{0.8}/4 n= 100 n= 500 n= 1000 OL block length n^{0.5}; GSE m=n^{0.8}/4 OL block length 2n^{0.5}; GSE m=n^{0.8}/4 OL block length n^{0.5}/2; GSE m=n^{0.8}/2 n= 100 n= 500 n= 1000 OL block length n^{0.5}; GSE m=n^{0.8}/2 OL block length 2n^{0.5}; GSE m=n^{0.8}/2 Figure 4: t-innovations: Coverage percentages for approximate 95% one-sided upper EL confidence bounds using GSE estimator with bandwidth m = C 2 n 4/5, C 2 {1/4, 1/2}. 24