On the coverage bound problem of empirical likelihood methods for time series

Transcription

1 J. R. Statist. Soc. B (06) On the coverage ound prolem of empirical likelihood methods for time series Xianyang Zhang University of Missouri Columia, USA and Xiaofeng Shao University of Illinois at Urana-Champaign, Champaign, USA [Received January 04. Final revision Feruary 05] Summary. The upper ounds on the coverage proailities of the confidence regions ased on lockwise empirical likelihood and non-standard expansive empirical likelihood methods for time series data are investigated via studying the proaility of violating the convex hull constraint. The large sample ounds are derived on the asis of the pivotal limit of the lockwise empirical log-likelihood ratio otained under fixed asymptotics, which has recently een shown to provide a more accurate approximation to the finite sample distriution than the conventional χ -approximation. Our theoretical and numerical findings suggest that oth the finite sample and the large sample upper ounds for coverage proailities are strictly less than and the lockwise empirical likelihood confidence region can exhiit serious undercoverage when the dimension of moment conditions is moderate or large, the time series dependence is positively strong or the lock size is large relative to the sample size. A similar finite sample coverage prolem occurs for non-standard expansive empirical likelihood.to alleviate the coverage ound prolem, we propose to penalize oth empirical likelihood methods y relaxing the convex hull constraint. Numerical simulations and data illustrations demonstrate the effectiveness of our proposed remedies in terms of delivering confidence sets with more accurate coverage. Some technical details and additional simulation results are included in on-line supplemental material. Keywords: Convex hull constraint; Coverage proaility; Fixed asymptotics; Heteroscedasticity auto-correlation roustness; Moment condition. Introduction Empirical likelihood (EL) (Owen, 988, 990) is a non-parametric methodology for deriving estimates and confidence sets for unknown parameters, which shares some of the desirale properties of parametric likelihood (see DiCiccio et al. (99) and Chen and Cui (006)). Because of its effectiveness and flexiility, it has advanced in many ranches in statistics, such as regression models, time series and censored data; see Owen (00) for a nice treatment of the suject. The EL-ased confidence sets inherit some good features from their parametric likelihood counterparts, ut there is a finite sample upper ound for the coverage of the EL ratio confidence region (see Owen (00), page 09, and Tsao (004)) due to the convex hull constraint, which may limit its applicaility and make it less appealing. For example, the EL confidence region for the mean of a random sample is nested within the convex hull of the data and its coverage level is necessarily smaller than that of the convex hull itself. The upper ound can e much smaller than Address for correspondence: Xianyang Zhang, Department of Statistics, University of Missouri Columia, Columia, MO 65, USA. zhangxiany@missouri.edu 05 Royal Statistical Society /6/78000

2 X. Zhang and X. Shao nominal coverage level α in the small sample and multi-dimensional situations. Following the terminology in Tsao and Wu (03), the finite sample coverage ound prolem is due to the mismatch etween the domain of the EL and the parameter space, so it is also called a mismatch prolem. There have een a few recent proposals to alleviate or resolve the mismatch prolem; see, for example adjusted EL (Chen et al., 008; Emerson and Owen, 009; Liu and Chen, 00; Chen and Huang, 0), penalized EL (Bartolucci, 007; Lahiri and Mukhopadhyay, 0) and the domain expansion approach (Tsao and Wu, 03, 04). However, all these works deal with independent estimation equations, and their direct applicaility to the important time series case is not clear. In this paper, our interest concerns the coverage ound prolems for EL methods tailored to stationary and weakly dependent time series. Although many variants have een proposed to extend EL to the time series setting (see Nordman and Lahiri (04) for a recent review), it seems that no investigation has een conducted regarding the coverage ound prolem, which is expected to exist ut its effect in the time series setting is unknown. We focus on two EL methods: lockwise EL (BEL), proposed y Kitamura (997), and non-standard expansive BEL (EBEL), recently proposed y Nordman et al. (03). BEL applies EL to the lockwise-averaged moment conditions to accommodate the dependence in time series non-parametrically and it has some useful properties of EL, such as Wilks s theorem. In Kitamura (997), the limiting χ -distriution for the empirical log-likelihood ratio (up to a multiplicative constant) was shown under traditional small asymptotics, in which, the fraction of lock size relative to sample size, goes to 0 as the sample size n. Adopting fixed asymptotics (Kiefer and Vogelsang, 005), in which.0, / is held fixed as n, Zhang and Shao (04) derived the pivotal limit of the empirical log-likelihood ratio at the true parameter value and used that as the asis for confidence region construction. The pivotal limit depends on and the simulations show that the fixed--ased confidence set has more accurate coverage than the small counterpart, indicating that the approximation y the fixed pivotal limit is more accurate than the small counterpart (i.e. χ ). Since this paper is related to our previous work in Zhang and Shao (04), it pays to highlight the difference. The focus of this paper is rather different from that of Zhang and Shao (04), and we investigate the coverage upper ound prolem of the lock-ased EL methods for time series. The technique that we use to derive the large sample ound, which depends on, is completely different from that involved in the derivation of the fixed limit of the EL ratio statistic in Zhang and Shao (04). The main contriution of the current paper is (a) to identify the coverage ound prolem for lock-ased EL methods in time series settings and to study the factors (e.g. sample size, lock size, joint distriution of time series and the form of moment conditions) that determine its magnitude (the large sample ound that we derive under fixed asymptotics provides an approximation to its finite sample counterpart and the approximation is accurate for large n and () to propose penalized BEL and EBEL methods as remedies of the coverage ound prolem, and to show their effectiveness through theory and simulations. Let β n denote the proaility that the convex hull of the moment conditions at the true parameter value contains the origin as an interior point and it is a natural upper ound on the coverage proaility of the BEL ratio confidence region (with any finite critical values) regardless of its confidence level. In Tsao (004), a finite sample upper ound was derived for independent estimation equations and the EL method. Tsao s technique is tailored to the independent case and seems not applicale to time series data. The calculation of the finite sample ound in the dependent and BEL case is challenging since it depends on the sample

3 Coverage Bound Prolem of Empirical Likelihood Methods 3 size, lock size, dimension and form of moment conditions as well as the joint distriution of time series. To shed some light on the coverage ound β n, we approximate β n y its large sample counterpart β, where β is shown to e the proaility that the pivotal limit (under fixed asymptotics) equals. We further provide an analytical formula for β as a function of in the case k =, and we derive an upper ound for β in the case k>, where k denotes the dimension of moment conditions. Interestingly, we discover that β = β./ > 0 for any >0 and β can e close to for fixed.0, / if the dimension of moment conditions k is moderately large. Compared with Tsao (004) and Kitamura (997), the large sample ound prolem (i.e. β > 0) is a unique feature that is associated with BEL under fixed asymptotics and it does not occur under the traditional small asymptotic approximation or for independent estimation equations. It is also worth pointing out that the large sample ound is always under small asymptotics regardless of the choice of lock size, and it provides an inaccurate approximation of the finite sample ound and could lead to an overoptimistic and thus misleading inference. In corrooration with our theoretical results, our simulations show that the finite sample coverage ound can deviate sustantially from when (a) the lock size is large relative to the sample size (i.e. is large), () the dimension of moment conditions is moderate or high and (c) the time series dependence is positively strong. In any one of these cases, constructing a confidence set of a conventional nominal level (say, 95% or 99%) is likely to lead to undercoverage. Thus our finding represents a cautionary note on the recent (theoretical) extension of BEL in the high dimensional setting (see Chang et al. (05)), where the dimension of moment conditions can also grow to as the sample size n grows to. EBEL uses a sequence of nested locks with growing sizes so no choice of lock size is involved, and the empirical log-likelihood ratio at the true parameter value converges to a pivotal ut nonstandard limit. Unlike BEL, there is no large sample ound prolem for EBEL as the proaility that the pivotal limit of EBEL equals is 0. However, the finite sample ound can e far elow the nominal level as shown in our simulations and results in a severe undercoverage. To alleviate the finite sample undercoverage prolem that is caused y the convex hull constraint, we propose to penalize BEL and EBEL y dropping the convex hull constraint. Penalized EL (PEL) was first introduced y Bartolucci (007) for the inference of the mean of independent and identically distriuted (IID) data, and our generalization to the time series context requires a non-trivial modification. In particular, we introduce a new normalization matrix that takes the dependence into account and we derive the limit of log-el ratio at the true value under fixed asymptotics. Our numerical results in the on-line supplementary material suggest that fixed asymptotics not only provide etter approximation for the original BEL (see Zhang and Shao (04)) ut also tends to provide a etter finite sample approximation for its penalized counterpart. Our new PEL ratio test statistic can e viewed as an intermediate etween the empirical log-likelihood ratio test statistic and the self-normalized score test statistic (see expression () in Section 4.) with the tuning parameter in the penalization term determining the amount of relaxation of the convex hull constraint. Our numerical results show the effectiveness of the two penalizationased EL methods in terms of delivering more accurate confidence sets for a range of tuning parameters. It is worth noting that the undercoverage prolem that is associated with BEL methods may e alleviated y applying a lock ootstrap approximation, as pointed out y a referee. However, the lock ootstrap does not completely solve the coverage ound prolem. In particular, when the finite sample ound is elow the nominal level, the undercoverage is ound to occur for any finite critical values, including ootstrap ased. By contrast, the finite

4 4 X. Zhang and X. Shao sample ounds for our penalized BEL and EBEL are always, so they are free of the coverage ound prolem; see remark in Section 3. for more discussions. A word on notation: let D[0, ] e the space of functions on [0, ] which are right continuous and have left limits, endowed with the Skorokhod topology (Billingsley, 999). Weak convergence in D[0, ] or more generally in the R q -valued function space D q [0, ] is denoted y, where q N. Convergence in proaility and convergence in distriution are denoted y p and d respectively. Let a e the integer part of a R. The notation N.v, Σ/ is used to denote the multivariate normal distriution with mean v and covariance Σ. Technical details and some simulation results are gathered in the on-line supplementary material. The data sets and R code that are used for this paper can e found at the second author s personal we page Blockwise empirical likelihood and expansive lockwise empirical likelihood Suppose that we are interested in the inference of a p-dimensional parameter vector θ, which is identified y a set of moment conditions. Denote y θ 0 the true parameter of θ which is an interior point of a compact parameter space Θ R p. Let {z t } n t= e n oservations from an R l -valued stationary time series and assume that the moment conditions E[f.z t, θ 0 /] = 0, t =,, :::, n,./ hold, where f.z t, θ/ : R l Θ R k is a map which is differentiale with respect to θ and rank{e[@f.z t, θ 0 /=@θ ]} = p with k p. To deal with time series data, we consider the fully overlapping smoothed moment condition (Kitamura, 997) which is given y f tn.θ/ = t+m f.z j, θ/ m j=t with t =,, :::, n m + and m = n for.0, /. The overlapping data locking scheme aims to preserve the underlying dependence etween neighouring time oservations. Consider the profile empirical log-likelihood function ased on the fully overlapping smoothed moment conditions, { N } N N L n.θ/ = sup log.π t / : π t 0, π t =, π t f tn.θ/ = 0, N := n m + :./ t= t= t= Standard Lagrange multiplier arguments imply that the maximum is attained when π t = N{ + λ f tn.θ/}, N f tn.θ/ t= + λ f tn.θ/ = 0, where λ is the Lagrange multiplier. By duality, the empirical log-likelihood ratio function (up to a multiplicative constant) is given y elr.θ/ = n max N log{ + λ f tn.θ/}, θ Θ:.3/ λ R k t= Under traditional small asymptotics, i.e. n + =.n/ 0asn, and suitale weak dependence assumptions (Kitamura (997); also see theorem of Nordman and Lahiri (04)), it can e shown that

5 Coverage Bound Prolem of Empirical Likelihood Methods 5 elr.θ 0 / d χ k :.4/ As pointed out y Nordman et al. (03), the coverage accuracy of BEL can depend crucially on the lock length m = n and appropriate choices can vary with respect to the joint distriution of the series. To capture the choice of lock length in the asymptotics, Zhang and Shao (04) adopted the fixed approach that was proposed y Kiefer and Vogelsang (005) in the context of heteroscedasticity auto-correlation roust testing and derived the non-standard limit of elr.θ 0 /. To proceed, we make the following assumption which can e verified under suitale moment and weak dependence assumptions on f.z j, θ 0 / (see for example Phillips (987)). Assumption. Assume that Σ nr j= f.z j, θ 0 /= n ΛW k.r/ for r [0, ], where ΛΛ = Ω = Σ j= Γ j with Γ j = E[f.z t+j, θ 0 /f.z t, θ 0 / ] and W k.r/ is a k-dimensional vector of independent standard Brownian motions. Under assumption, Zhang and Shao (04) showed that, when n and is held fixed, elr.θ 0 / d U el,k./ := max log[ + λ {W k.r + / W k.r/}]dr,.5/ λ R k 0 where we define log.x/ = for x 0: The asymptotic distriution U el,k./ is non-standard yet pivotal for a given, and its critical values can e otained via simulation or the ootstrap. Given.0, /, a 00. α/% confidence region for the parameter θ 0 is then given y { CI. α; / = θ Θ : elr.θ/ } u el,k.; α/,.6/ where u el,k.; α/ denotes the 00. α/% quantile of the distriution P{U el,k./=. / x}. It was demonstrated in Zhang and Shao (04) that the confidence region ased on the fixed approximation has more accurate coverage than the traditional counterpart. Our analysis in the next section reveals an interesting coverage upper ound prolem associated with the fixed approach in the BEL framework. This result provides some insight on the use of fixed--ased critical values as suggested in Zhang and Shao (04). It also sheds some light on the finite sample coverage ound prolem that can occur as long as the BEL ratio statistic is used to construct the confidence region. Moreover, we propose a penalized version of the fixed--ased BEL, which improves the finite sample performance of the method in Zhang and Shao (04). To avoid the choice of lock length and also to improve the finite sample coverage, Nordman et al. (03) proposed a new version of BEL which uses a non-standard data locking rule. To descrie their approach, we let f tn.θ/ = ω.t=n/ t f.z j, θ/ n for t =,, :::, n, where ω. / : [0, ] [0, / denotes a non-negative weight function. The lock collection, which constitutes a type of forward scan in the lock susampling language of McElroy and Politis (007), contains a data lock of every possile length for a given sample size n. It is worth noting that this non-standard data locking rule ears some resemlance to recursive estimation in the self-normalization approach of Shao (00). Following Nordman et al. (03), we define the EBEL ratio function as ẽlr.θ/ = n max λ R k j= n log{ + λ f tn.θ/}:.7/ t=

6 6 X. Zhang and X. Shao For the smooth function model, Nordman et al. (03) showed that ẽlr.θ 0 / d U eel,k.ω/ = max log{ + λ ω.r/w k.r/}dr:.8/ λ R k 0 The numerical studies in Nordman et al. (03) indicate that EBEL generally exhiits comparale (or in some cases even etter) coverage accuracy than BEL with χ -approximation and suitale lock size. Though the fixed--ased BEL and EBEL provide an improvement over traditional χ -ased BEL, our study in the next section reveals that oth fixed--ased BEL and EBEL can suffer seriously from the coverage upper ound prolem in finite samples. To the est of our knowledge, this is the first time that the coverage upper ound prolem has een revealed for EL methods in time series. 3. Bounds on the coverage proailities 3.. Large sample ounds In the framework of BEL, asymptotic theory is typically estalished under small asymptotics, where the large sample ound prolem does not occur as the empirical log-likelihood ratio statistic converges to a χ -limit. However, in finite samples, the coverage upper ound β n can deviate significantly from. To shed some light on the finite sample coverage ound, we derive a limiting upper ound on the coverage proailities of the BEL ratio confidence region ased on the fixed limiting distriution given in expression (5). The fixed method that is adopted here reflects the coverage upper ound prolem in the asymptotics, whereas the original BEL under the small asymptotics is somewhat overoptimistic as the corresponding upper ound in the limit is always regardless of what the finite sample ound is. Define D k.r; /=W k.r +/ W k.r/ and A = A = {λ R k : min r.0, / { + λ D k.r; /} 0}: Let t k.r; / = D k.r; / D k.r; / I{ D k.r; / > 0} e the direction of D k.r; / on the (k )-dimensional sphere S k, where denotes the Euclidean norm and I{ } denotes the indicator function. We first present the following lemma regarding the unoundedness of A. Lemma. Define the convex hull H.D k / = {Σ s j= α j D k.r j ; / : s N, α j 0, Σ s j= α j =, r j.0, /}. Then the set A is unounded if and only if the origin is not an interior point of H.D k /. From the proof of lemma (which is given in the on-line supplementary material) and the fact that the components of D k.r; / are linearly independent (with proaility ), we know that {A is unounded} implies that {U el,k./ = }. However, when U el,k./ =, it is easy to see that A cannot e ounded. Therefore we have P.A is unounded/ = P{U el,k./ = }. Let H n.θ 0 ; / = {Σ N t= α t f tn.θ 0 / : α t 0, Σ N t= α t = } and denote y H o n.θ 0; / the interior of H n.θ 0 ; /. By lemma and strong approximation, we have, for large n, P{the origin is not contained in H o n.θ 0; /} P.A is unounded/ = P{U el,k./ = }: It was conjectured in Zhang and Shao (04) that P.A is unounded/>0, which implies that P.U el,k./ = />0. In what follows, we give an affirmative answer to this conjecture and provide an explicit formula for the proaility P.A is unounded/ when k =: For k =, we must have {A is unounded} = {D.r; / 0, r.0, ]} {D.r; / 0, r.0, ]}. Bythe symmetry of a Wiener process, we have P.A is unounded/ = P{D.r; / 0, r.0, ]}: For β > 0, we let φ β. / = φ. = β/= β with φ.x/ = {=. π/} exp. x =/ eing the standard

7 Coverage Bound Prolem of Empirical Likelihood Methods 7 normal density. For two vectors x =.x, x, :::, x L / and y =.y, y, :::, y L / of real numers with L N, define the matrix Q β,l.x, y/=.φ β.x i y j // L i,j=. Let q β,l.x, y/ e the determinant of Q β,l.x, y/. For a vector x =.x, x, :::, x L /, denote y x s :s =.x s, x s +, :::, x s / the suvector of x for s s L: Using similar arguments to those in Shepp (97) (also see Karlin and Mcgregor (959)), we prove the following result. Theorem. If L = = is a positive integer, we have P{D.r; / 0, r.0, ]/ = q,l.x :L, x :.L+/ /dx dx 3 :::dx L+,.9/ 0=x <x <x 3 <:::<x L+ where x =.x, :::, x L+ /.IfL + τ = with L eing a positive integer and 0 < τ <,wehave P{D.r; / 0, r.0, ]} = ::: q ξ,l+.x, y/q ξ,l.x :.L+/, y :L /dy dx dy S :::dx L+ dy L+, 0< ξ = τ= <,.0/ where x =.x, :::, x L+ / with x = 0, y =.y, :::, y L+ / and the integral is over the set S := {.y, x, y, :::, x L+, y L+ / R L+ :0<x <:::<x L+, y <y <:::<y L+ }. Theorem provides an exact formula for the proaility P.A is unounded/ when k =. The proaility can e manually calculated when L is small. In particular, if = (i.e. L = ), we have P.A is unounded/ = {φ. x /φ.x x 3 / φ. x 3 /φ.0/}dx dx 3 0<x <x 3 { 0 } = Φ.0/ φ.0/ Φ.x/dx = 0:869, where Φ. / denotes the distriution function of the standard normal random variale. When = 3 (i.e. L = 3), direct calculation yields that { } P.A is unounded/ = Φ 3.0/ + φ.0/ + φ. x 3 /φ.x 3 x /Φ.x x 3 /dx dx 3 4 0<x <x { 3 φ.x 3 x /Φ. x 3 /dx dx 3 + φ.0/φ.0/ 0<x <x 3 } + φ. x /φ.0/φ.x x 3 /dx dx 3 0<x <x 3 { = 8 + φ.0/ 0 + φ.u/φ.u/du + φ.0/ 4 Φ. x 3 /Φ. } x 3 /dx 3 φ.0/.4π/ 0 = 0:03635: The calculation for larger L is still possile ut is more involved. An alternative way is to approximate the proailities in expression (9) and (0) y using Monte Carlo simulation; see Tale and Fig.. Utilizing the result in theorem, we can derive a (conservative) upper ound on P.A is ounded/ (i.e. β) in the multi-dimensional case. For k>, we let D.j/.r; / e the k

8 8 X. Zhang and X. Shao L EBEL Fig.. Bounds on the coverage proailities for the BEL and EBEL (the data are generated from a multivariate standard normal distriution with n D 5000 and the numer of Monte Carlo replications is 0000):, k D 5I4, k D 0IC, k D 5I, k D 0I}, k D 50 jth element of D k.r; / and V j = {D.j/ k.r; / 0, r.0, ]} {D.j/ k.r; / 0, r.0, ]} with j k: By the independence of the components of D k.r; /, it is easy to derive that P.A is unounded/ P. k j= V j/ = P. k j= Vc j / = Pk.V c / = [ P{D.j/ k.r; / 0, r.0, ]}]k : Therefore, we otain the following result. Proposition. When L = = is a positive integer, we have { } k P{U el,k./ < / q,l.x :L, x :.L+/ /dx dx 3 :::dx L+,./ 0=x <x <x 3 <:::<x L+ where x =.x, :::, x L+ /. When L + τ = with L eing a positive integer and 0 < τ <,we have P{U el,k./ < } { } k ::: q ξ,l+.x, y/q ξ,l.x :.L+/, y :L /dy dx dy :::dx L+ dy L+, S 0 < ξ = τ= <,./ where x =.x, :::, x L+ / with x = 0, y =.y, :::, y L+ / and the integral is over the set S := {.y, x, y, :::, x L+, y L+ / R L+ :0<x <:::<x L+, y <y <:::<y L+ }. When k =, the inequality ecomes equality in expressions () and (). If the (asymptotic) critical value ased on the fixed pivotal limit U el,k./ is used to construct a 00. α/% confidence region, then the following several cases can occur.

9 Coverage Bound Prolem of Empirical Likelihood Methods 9 Tale. Bounds on the coverage proailities for BEL n ρ k Bounds (%) for the following values of L = =: The numer of Monte Carlo replications is for k = (0000 for k = ). For the last row n =,weapproximate the proaility P.A is ounded/ y simulating independent Wiener processes, where the Wiener process is approximated y a normalized partial sum of for k = (0000 for k = ) IID standard normal random variales and the numer of replications is for k = (50000 for k = ). (a) P{U el,k./ < } = β α; then the fixed--ased critical value is. In this case, it is impossile to construct a meaningful confidence region as {θ Θ elr.θ/ } = Θ. In the case k =, the value of β is known ut, in the case k = or higher, only an upper ound for β is provided in proposition. Thus, if the upper ound is no greater than α, then we cannot construct a sensile confidence region ased on fixed critical values.

10 0 X. Zhang and X. Shao () P{U el,k./ < } = β > α; then the fixed--ased critical value is finite. The 00. α/% quantile of the distriution of U el,k./=. / (i.e. u el,k.; α/) isthe 00γ% quantile of the conditional distriution P{U el,k./=. / x U el, k./ < }, where γ =. α/=. β/. In the simulation experiment of Zhang and Shao (04), the 00. α/% quantile of the conditional distriution was used as the critical value. Note that the largest considered in Zhang and Shao (04) was 0:, which corresponds to β 0:9985 = 0:005 when k = and β 0:987 = 0:08 when k =, as seen from Tale. This suggests that the critical values that were used in Zhang and Shao (04) are wrong, ut not y much. (c) In the event that u el,k.; α/ is finite, which occurs in case () aove or when the χ -ased critical values are used, P{θ 0 CI. α; /} P{the origin is contained in H o n.θ 0; /} = β n, which is a finite sample ound. The quantity β n depends on joint distriutions of time series, the form of f, the lock size and the sample size, so it is in general difficult to calculate. We present some numerical results on β n in Section 3. elow. If β n α, then the confidence region is ound to undercover and the amount of undercoverage ecomes severe when β n is further from 0. Proposition shows that, for any fixed.0, /, the ound decays exponentially to zero as the dimension k grows. This result suggests that caution needs to e taken in the recent extension of the BEL to the high dimensional setting (see Chang et al. (05)), where the dimension of moment condition k can grow with respect to sample size n. In Chang et al. (05), small asymptotics were adopted, and no discussion on such a coverage ound issue (either finite sample or large sample) seems provided. It would e interesting to extend the fixed asymptotic approach to BEL in the high dimensional setting and we leave it for future investigation. The large sample ound on the coverage proailities depends crucially on how the smoothed moment conditions are constructed. By lemma of Nordman et al. (03), we know that, for EBEL, the set A ω = {λ R k : min r.0,/ { + λ ω.r/w k.r/} 0} is ounded with proaility, Tale. Bounds on the coverage proailities for EBEL ρ k Bounds (%) for the following values of n: The numer of Monte Carlo replications is The ounds on the coverage proailities for EBEL do not depend on the choice of the weight function ω. /.

11 Coverage Bound Prolem of Empirical Likelihood Methods which implies that P{U eel,k.ω/< } =. Thus, for large samples, no upper ound prolem occurs for EBEL. However, the numerical results in Tale show that the finite sample ounds on the coverage proailities of the EBEL ratio confidence regions can e significantly lower than, which indicates that the convergence of the EBEL ratio statistic ẽlr.θ 0/ to its limit U eel,k.ω/ is in fact slow and sustantial undercoverage can e associated with EBEL-ased confidence regions in any one of the following three cases: (a) the dependence is positively strong; () the sample size n is small; (c) k is moderate, say k 3. Remark. The convex hull constraint is related to the underlying distance measure etween π =.π, :::, π N / and.=n, :::,=N/ in EL. If we consider alternative non-parametric likelihood such as the Euclidean likelihood or, more generally, memers of the Cressie Read power divergence family of discrepancies, then the origin is allowed to e outside the convex hull of the smoothed moment conditions as long as the weights are allowed to e negative. No coverage upper ound prolem occurs for these alternative non-parametric likelihoods ut, since EL has a certain optimality property (Kitamura, 006; Kitamura et al., 03), it is still a worthwhile effort to seek remedies of the coverage ound prolem ased on EL. Remark. An alternative way to calirate the sampling distriution is the lock ootstrap. To illustrate the idea, consider the linear model, i.e. y t = x t θ + u t, where x t and θ are.l /- dimensional vectors (p = l in this case), and the stationary time series {x t } and {u t } are uncorrelated. Define f t.θ/=x t.y t x t θ/ and z t =.x t, y t/ : Assume that n=d n l n, where d n denotes the lock size in the ootstrap and l n is the numer of locks. Let M, :::, M ln e independent and identically distriuted (IID) uniform random variales on {0, :::, n d n } and let z Å.j /d n +i = z Mj +i with j l n and i d n : Let f Å tn. ˆθ/ e the smoothed moment condition ased on the ootstrap sample {z Å i }, where ˆθ is the ordinary least squares estimator ased on the original sample. The naive ootstrap version of elr.θ 0 / is given y elr Å. ˆθ/ = n max N λ R k t= log. + λ [f Å tn. ˆθ/ E Å {f Å tn. ˆθ/}]/, where E Å denotes the expectation conditional on {z i } n i=. Alternatively, ˆθ may e replaced y E Å.Σ n t= xå t xå t / E Å.Σ n t= xå t yå t /. The ootstrap critical value otained from this procedure is expected to provide a etter finite sample approximation compared with the χ -caliration. The intuition is that, esides the time series dependence (which is captured y the locking strategy in the lock ootstrap) and the effect of sample size, the choice of lock size n is also reflected in the ootstrap statistic through the construction of f Å tn. This is essentially the rationale ehind the fixed approach. On the asis of the arguments in Gonçalves and Vogelsang (0), it is expected that the ootstrap test statistic elr Å. ˆθ/ has the same limiting distriution (conditionally on the data) as that of elr.θ 0 / derived under fixed asymptotics. Therefore, the ootstrap caliration indeed has a deep connection with the fixed approach. Investigation along this direction is very interesting and will e pursued in the future. However, a remedy to the coverage ound prolem seems necessary when the (finite sample) ound is less than the nominal level, which could happen in the case of high dimension, small sample size, strong dependence or large. In this case, a naive application of the aove ootstrap method to calirate may not work as there is an intrinsic coverage upper ound for whatever critical values (including ootstrap ased). This motivates us to develop the penalized approach in the next section, which is free of the coverage ound prolem.

12 X. Zhang and X. Shao 3.. Finite sample results on coverage ounds To evaluate the upper ounds on the coverage proailities for BEL and EBEL, we simulate time series from auto-regressive (AR()) models with AR() coefficient ρ = 0:5, 0, 0:, 0:5, 0:8, and IID standard normal errors. The sample size n = 50, 00, 500, 000, 5000,. We approximate the proaility P.A is ounded/ y simulating independent Wiener processes, where the Wiener process is approximated y the normalized partial sum of IID standard normal random variales and the numer of Monte Carlo replications is When k>, we simulate vector AR (VAR()) processes with the coefficient matrix A = ρi k for ρ = 0:5, 0, 0:, 0:5, 0:8, and standard multivariate normal errors. Tale summarizes the upper ounds on the coverage proailities for BEL with = =L for L =, 3, :::, 0, 5, 0, and Tale provides the finite sample upper ounds on the coverage proailities for EBEL. For BEL, it is seen from Tale that the upper ound on the coverage proaility decreases as the lock size increases and the positive dependence strengthens. The ound in the multi-dimensional case is lower than its counterpart in the univariate case, which is consistent with our theoretical finding. It is interesting that negative dependence (corresponding to ρ = 0:5) tends to ring the upper ound higher. In practice, if the dependence is expected to e positively strong, a large lock size is preferale. However, our result indicates that the corresponding upper ound on the coverage proailities will e lower for larger lock size. It is also worth noting that the upper ounds on the coverage proailities generally increase as the sample size grows and the result in proposition provides conservative ounds on β when k =. For EBEL, though its large sample ound is, its finite sample ound can e significantly lower than as seen from Tale. To assess the effect of the dimensionality k further, we present the coverage upper ounds for k = 5, 0, 5, 0, 50 and L =, 3, :::, 0, 30, 40, 50 in Fig., where data are generated from a multivariate standard normal distriution with sample size n = We oserve that (a) as k grows, a smaller (or larger L) is required to deliver meaningful finite sample upper ounds (say, larger than nominal level) and () the coverage upper ound for EBEL can e close to zero for k = 5 or larger. We expect that the ound can grow worse when we increase the positive dependence in the oservations. On the asis of the numerical results for this specific setting, we suggest that special attention is paid to the potential coverage ound prolem for the following cases: (i) the nominal level is close to (such as 99%); (ii) the dimension of moment conditions k is moderate or high; (iii) the (positive) dependence is strong; (iv) is large. 4. Penalized lockwise empirical likelihood and expansive lockwise empirical likelihood The convex hull constraint violation underlying the mismatch is well known in the EL literature (see Owen (990, 00)). Various methods have een proposed to ypass this constraint, such as PEL (Bartolucci, 007; Lahiri and Mukhopadhyay, 0), adjusted EL (Chen et al., 008; Emerson and Owen, 009; Liu and Chen, 00; Chen and Huang, 0) and extended EL (Tsao and Wu, 03, 04). Motivated y the theoretical findings as well as the finite sample results in Section 3., we propose a remedy ased on penalization to circumvent the coverage ound prolem which leads to improved coverage accuracy under fixed asymptotics.

13 Coverage Bound Prolem of Empirical Likelihood Methods Penalized lockwise empirical likelihood To overcome the convex hull constraint violation prolem, Bartolucci (007) dropped the convex hull constraint in the formulation of EL for the mean of a random sample and defined the likelihood y penalizing the unconstrained EL y using the Mahalanois distance. Recently, Lahiri and Mukhopadhyay (0) introduced a modified version of Bartolucci s (007) PEL in the mean case. Under the assumption that the oservations are IID and the components of each oservation are dependent, Lahiri and Mukhopadhyay (0) derived asymptotic distriutions of the PEL ratio statistic in the high dimensional setting. Other variants of PEL where a penalty function is added to the standard EL were considered y Otsu (007) for efficient estimation in semiparametric models and Tang and Leng (00) for consistent parameter estimation and variale selection in linear models. Otsu (007) and Tang and Leng (00) either penalized high dimensional parameters or roughness of unknown non-parametric functions, and their PELs still suffer from the same convex hull constraint violation prolem as standard EL does. In what follows, we shall consider a penalized version of the BEL ratio test statistic in the moment condition models, which allows weak dependence within the moment conditions and may e computed even when the origin does not elong to the convex hull of the smoothed moment conditions. Compared with existing penalization methods in the literature, our method is different in three aspects. First, our method is designed for dependent data where existing methods are applicale only to independent moment conditions. Second, our theoretical result is estalished under fixed asymptotics, which are expected to provide a etter approximation to the finite sample distriution. And we suggest the use of the fixed--ased critical values that capture the choice of tuning parameters (also see the simulations in the on-line supplementary material). Third, our formulation produces a new class of statistic etween the empirical loglikelihood ratio statistic and the self-normalized score statistic which is of interest in its own right. To illustrate the idea, we first consider the case k = p, i.e. the moment condition is exactly identified (see remark 3 for the general overidentified case). Define the simplex N = {π =.π, :::, π N / : π t 0, Σ N t= π t = } and the quadratic distance measure δ n.μ/ := δ Ψn.μ/ = μ Ψ n μ for μ R k, where Ψ n R k k is an invertile normalization matrix. Let μ π.θ/ = Σ N t= π t f tn.θ/ with π =.π, :::, π N / N. We consider penalized BEL (PBEL) as follows: [ N L pel,n.θ/ = max π t exp nτ ] π N δ n{μ π.θ/} :.3/ t= The PBEL ratio test statistic is then defined as elr pel.θ/ = n log{nn L pel,n.θ/} = min π N [ n N t= log.nπ t / + τ ] δ n{μ π.θ/} : Under the constraint that μ = Σ N t= π t f tn.θ/, it is not difficult to derive that π t = N[+ λ {f tn.θ/ μ}], N t= f tn.θ/ μ + λ {f tn.θ/ μ} = 0, y using standard Lagrange multiplier arguments. Denote y H n.θ; / = {Σ N t= π t f tn.θ/ : π N }: Thus we deduce that ( elr pel.θ/ = min μ H n.θ;/ n max N log[ + λ {f tn.θ/ μ}] + τ ) δ n.μ/,.4/ λ R k t= where μ is minimized to alance the empirical log-likelihood ratio and the penalty term.

14 4 X. Zhang and X. Shao Proposition. If the space that is spanned y {f tn.θ/} N t= is of k dimension, we have ( elr pel.θ/ = min μ R k n max N log[ + λ {f tn.θ/ μ}] + τ ) λ R k t= δ n.μ/ :.5/ The condition that the space that is spanned y {f tn.θ/} N t= is of k dimension is fairly mild ecause k is fixed and N grows with n. Note that the minimizer μ Å of equation (5) is necessarily contained in H n.θ; /, which implies that the origin of R k is contained in the convex hull of {f tn.θ/ μ Å } N t=. In addition, since the empirical log-likelihood ratio and the penalty term in equation (5) are oth convex functions of μ, it is not difficult to otain μ Å in practice. Let τ = c Å n with c Å eing a non-negative constant which controls the magnitude of the penalty term, and suppose that Ψ n d.λφ k Λ / as n, where Φ k R k k is a pivotal limit. For example, if we let Q., / : [0, ] R e a positive semidefinite kernel, then one possile choice of the normalization matrix Ψ n is given y Ψ n. ˆθ n / = ( n n t Q n n, j n t= j= ) f.z t, ˆθ n /f.z j, ˆθ n /,.6/ where ˆθ n is a preliminary estimator otained y solving the equation Σ n j= f.z j, θ/=0. In practice, one can choose Q.r, s/ = κ.r s/ with κ. / eing the kernels that are used in the heteroscedasticity and auto-correlation consistent estimation, such as the Bartlett kernel or the quadratic spectral kernel. Under appropriate conditions (see, for example, Kiefer and Vogelsang (005) and Sun (03)), it can e shown that Ψ n. ˆθ n / d Λ Q.r, s/db k.r/db k.s/λ := ΛΦ k Λ,.7/ 0 0 where Φ k = 0 0 Q.r, s/db k.r/db k.s/ with B k.r/ = W k.r/ rw k./. Therefore, under assumption, we have elr pel.θ 0 / U d pel,k./ ( = min μ H./ max λ R k 0 { log [+λ Λ D k.r; / }] μ dr + cå ) μ.λφ k Λ / μ,.8/ where H./ denotes the convex hull of {ΛD k.r; /=:r.0, /}. When μ is outside the convex hull of {ΛD k.r; /= : r.0, /}, the separating hyperplane theorem (see for example section of Rockafellar (970)) implies that max λ R k 0 log[ + λ {ΛD k.r; /= μ}]dr =. Thus we have the simplified expression ( { U pel,k./ = min μ R k max log [ + λ Λ D }] k.r; / μ dr + cå ) λ R k 0 μ.λφ k Λ / μ ( [ = min μ R k max log + λ { }] D k.r; / μ dr + cå ) λ R k 0 μ Φ k μ,.9/ where λ = Λ λ and μ = Λ μ: Note that the limiting distriution U pel,k./ is pivotal and its critical values can e simulated y approximating the Brownian motion with the standardized or normalized partial sum of IID standard normal random variales. As to the pivotal limit U pel,k./, we have the following result. Proposition 3. For.0, / and c Å > 0, P{U pel,k./ < } =. Thus, compared with BEL, PBEL is well defined and does not suffer from the convex hull

15 Coverage Bound Prolem of Empirical Likelihood Methods 5 violation prolem in oth large sample and finite sample cases, though it involves the choice of additional tuning parameters such as c Å and Ψ n. When c Å =,wehaveμ Å = 0 and the PBEL ratio statistic reduces to the BEL ratio statistic. In contrast, and elr pel.θ/ = c Å ( min μ R k c Å n max λ R k max λ R k N t= N t= { log [ + λ f tn.θ/ N ) log[ + λ {f tn.θ/ μ}] + n δ n.μ/,.0/ N t= }] f tn.θ/ = 0: Thus, for small c Å, the minimizer μ Å should e close to Σ N t= f tn.θ/=n. In this case, the penalty term dominates and the PBEL ratio statistic evaluated at the true parameter value ehaves like the self-normalized score statistic which is defined as S n.θ 0 / = nδ n ( N t= f tn.θ 0 / N ) = n ( N t= f tn.θ 0 / N ) Ψ n. ˆθ n / ( N t= ) f tn.θ 0 / :./ N We call S n.θ 0 / the self-normalized score statistics as f tn.θ/ plays the role of the score in likelihoodased inference and the self-normalizer Ψ n. ˆθ n / is an inconsistent estimator of the asymptotic variance matrix Ω in the spirit of the self-normalized approach of Shao (00). Therefore, on the asis of the quadratic distance measure, the penalized BEL ratio statistic can e viewed as a comination of the BEL ratio statistic and the self-normalized score statistic. Remark 3. When the moment condition is overidentified (i.e. k>p), we shall consider the normalization matrix Ψ n = Ψ n. ˆθ n / with ˆθ n eing a preliminary estimator such as the one-step generalized method-of-moments estimator with the weighting matrix W n p W 0, where W 0 is a k k positive definite matrix. To illustrate the idea, define G t.θ/ = n t j, and G 0 = E{G n.θ 0 /}. Let û j = {G n. ˆθ n /W n G n. ˆθ n /} G n. ˆθ n /W n f.z j, ˆθ n /: Consider the normalization matrix Ψ n. ˆθ n / = ( n n t Q n t= j= n, j ) û t û j n : Under suitale conditions (see Kiefer and Vogelsang (005)), it can e deduced that Ψ n. ˆθ n / d Δ 0 0 Q.r, s/db p.r/db p.s/δ, where Δ R p p is an invertile matrix such that ΔΔ =.G 0 W 0G 0 / G 0 W 0ΩW 0 G 0.G 0 W 0G 0 / : In this case, the PBEL ratio test statistic can e defined as ( elr pel.θ/ = min μ R p n max N log[ + λ {g tn.θ/ μ}] + τ ) λ R p μ Ψ n. ˆθ n /μ, t= where g tn.θ/ = {G n. ˆθ n /W n G n. ˆθ n /} G n. ˆθ n /W n f tn.θ/ is the transformed smooth moment condition. Following the arguments aove, it can e shown that elr pel.θ 0 / admits the same pivotal limit,

16 6 X. Zhang and X. Shao elr pel.θ 0 / d U pel,p./ = min μ R p { max λ R p 0 [ log + λ { }] D p.r; / μ dr + cå μ Φ p }: μ./ 4.. Penalized expansive lockwise empirical likelihood As demonstrated in Section 3., EBEL suffers seriously from the convex hull violation prolem in finite samples. To deal with the convex hull condition, we introduce penalized EBEL (PEBEL) which is shown to provide significant finite sample improvement in Section 5. We descrie the idea for exactly identified moment condition models. The results elow can e extended to more general cases following the discussion in remark 3. Recall that f tn.θ/ = ω.t=n/ t f.z j, θ/ n j= for t =,, :::, n. We consider the PEBEL ratio test statistic which is defined as elr peel.θ/ = [ n log{nn L peel,n.θ/} = min ] n log.nπ t / + τδ n { μ π n n π.θ/}, τ = c Å n, t= where n L peel,n.θ/ = max π t exp[ nτδ n { μ π.θ/}],.3/ π n t= and μ π.θ/ = Σ n t= π t f tn.θ/ with π =.π, :::, π n / n. Following similar derivations in the proof of proposition 3, we deduce that ( elr peel.θ/ = min μ H n.θ/ = min μ R k ( n max λ R k n n max λ R k t= n t= ) log[ + λ { f tn.θ/ μ}] + τδ n.μ/ ) log[ + λ { f tn.θ/ μ}] + τδ n.μ/,.4/ where H n.θ/ denotes the convex hull of { f tn.θ/} n t=. Under suitale assumptions (see Nordman et al. (03)), it can e shown that elr peel.θ 0 / d min μ R k ( max λ R k 0 log[ + λ {ω.r/w k.r/ μ}]dr + c Å μ Φ k ) μ :.5/ Note that PEBEL is free of, ut again it requires the choice of a tuning parameter c Å. For large c Å,wehaveμ Å 0 and δ n.μ Å / 0 with μ Å eing the minimizer in equation (4). Thus PEBEL ehaves like EBEL when c Å is large. Following the discussion in Section 4., as c Å grows close to 0, μ Å approaches Σ n t= f tn.θ/=n which satisfies that max λ R k n t= { log [ + λ f tn.θ/ n t= f tn.θ/ n }] = 0: Thus, for small c Å, the ehaviour of the PEBEL ratio statistic evaluated at the true parameter value is closely related to the self-normalized score statistic given y ( n ) ( f S n.θ 0 / = nδ tn.θ 0 / n ) f n = n tn.θ 0 / ( n ) Ψ n t= n t= n. f ˆθ n / tn.θ 0 / :.6/ t= n

17 Coverage Bound Prolem of Empirical Likelihood Methods 7 Remark 4. To resolve the coverage upper ound prolem, we may consider adjusted versions of BEL and EBEL, which retain the formulation of BEL and EBEL ut add one or two pseudooservations to the sample (see Chen et al. (007) and Emerson and Owen (009)). However, a direct extension to the current setting may not work ecause of temporal dependence in moment conditions. A possile strategy is to add a small fraction of artificial data points instead of one or two pseudo-oservations, and to derive the limiting distriutions under fixed asymptotics. This approach also requires the choice of additional tuning parameters such as the fraction of points eing added. The extended EL method (Tsao and Wu, 03, 04) is a nice alternative to the original EL, and it has een shown to enjoy the Bartlett correctaility in the IID data case. Nevertheless, an extension to the time series setting seems very non-trivial. We shall investigate these alternative solutions in future research. 5. Numerical results In this section, we conduct simulation studies to evaluate the finite sample performance of the penalization methods that were proposed in Section 4. We shall focus on the confidence region for the mean of univariate or multivariate time series. In the univariate case, we consider the AR() process z t = ρz t + " t with ρ = 0:5, 0:, 0:5, 0:8, and the moving average MA() process z t =θɛ t +ɛ t with θ = 0:5, 0:, 0:5, 0:95, where {" t } and {ɛ t } are two sequences of IID standard normal errors. In the multi-dimensional case (i.e. k>), we generate multivariate time series with each component eing independent AR() or MA() processes. The sample sizes considered are n = 00 and n = 400. In the on-line supplementary material, we present additional simulation results for time series regression models, where the results are qualitatively similar to those for the mean. 5.. Penalized lockwise empirical likelihood To implement PBEL, we consider the self-normalization matrix Ψ n (Shao, 00), which is defined as Ψ n. ˆθ n / = n n i= j= ( n i j n ).z i z n /.z j z n /,.7/ where ˆθ n = z n = Σ n j= z j=n: The tuning parameter c Å is chosen etween 0.0 and. As pointed out in Section 4., the limiting distriution of PBEL under the fixed asymptotics is pivotal and it can e approximated numerically. Tale in the on-line supplementary material summarizes the simulated critical values for the limiting distriutions of BEL and PBEL. Selected simulation results are presented in Figs and 3. In the univariate case, the performances of PBEL with c Å = and BEL are generally comparale in terms of the coverage proaility and interval width. PBEL with c Å =0:0 delivers more accurate coverage compared with the two alternatives especially when the positive dependence is strong, although the corresponding interval width is slightly wider for relatively small. This finding is presumaly ecause the finite sample ounds for BEL do not deviate much from for k = and not quite large (see Tale ). The simulation results for the MA models are quantitatively similar and thus are not presented here for revity. In the case k =, PBEL tends to provide etter coverage uniformly over compared with BEL (when the dependence is positive). The improvement ecomes more significant as the lock size grows. Also PBEL with c Å = 0:0 delivers the most accurate coverage in most cases. Unreported numerical results show that, for k =, and c Å etween 0:0 and, the performance of PBEL is generally etween the two cases reported here. To assess the effect of dimensionality, we