Size and Shape of Confidence Regions from Extended Empirical Likelihood Tests

Transcription

1 Biometrika (2014),,, pp C 2014 Biometrika Trust Printed in Great Britain Size and Shape of Confidence Regions from Extended Empirical Likelihood Tests BY M. ZHOU Department of Statistics, University of Kentucky, Lexington, Kentucky, , U.S.A. mai@ms.uky.edu 5 AND M.-O. KIM Division of Biostatistics and Epidemiology, Cincinnati Children s Hospital Medical Center, Cincinnati, Ohio , U.S.A. miok.kim@cchmc.org SUMMARY 10 Empirical likelihood is a general non-parametric inference methodology using likelihood principle in a way that is analogous to that used with parametric likelihoods. In a wide range of applications the methodology was shown to provide likelihood ratio statistics that have limiting chi-square distributions and observe a nonparametric version of Wilks theorem. Amongst recent extensions to the analysis of censored data, longitudinal data and semi-parametric regres- 15 sion model, however, the property only remained true in some but not in others. This motivates our discussion of relative optimality of extended empirical likelihood methods. We compare extended empirical likelihood methods and evaluate their relative optimality by comparing the confidence regions provided by inverting the likelihood ratio tests. We show that those extension methods which yield the likelihood ratio statistic observing the Wilks theorem provides the 20 smallest confidence region. Specific examples are provided for the case of censored data analysis and estimating equations involving nuisance parameters. Some key words: Empirical Likelihood, Likelihood Ratio, Wilks Confidence region. 1. INTRODUCTION Empirical likelihood is a general non-parametric inference method using likelihood principle 25 in a way that is analogous to that used with parametric likelihoods. Since its introduction by Owen (1988, 1990), the methodology has been extended to the analysis of censored data (Li, Li & Zhou (2005)), longitudinal data (e.g., Wang, Qian & Carroll (2010)) and semi-parametric regression model (e.g., Shi & Lau (2000)). In a wide range of applications as outlined in Owen (2001) empirical likelihood ratio statistics were shown to have limiting chi-square distributions 30 and observe a nonparametric version of Wilks theorem. This property remains true with some extended methods but not with others, however. This motivates discussion of relative optimality of extended methods. In this paper we consider several extension of empirical likelihood methods that have the likelihood function maximized at the same location, thus yielding equivalent maximum empirical 35 likelihood estimators, but provide different likelihood ratio statistics. We discuss the relative optimality of the methods by comparing the confidence regions provided by inverting the likelihood

2 M. ZHOU AND M.-O. KIM ratio tests. We show that the method that yields the likelihood ratio statistic observing the Wilks theorem provides the smallest confidence region. The remainder of this paper proceeds as follows. Section 2 formally defines a set-up where multiple empirical likelihood methods can be defined that yield equivalent maximum empirical likelihood estimators. The section also provides asymptotic results for the comparison of the corresponding confidence regions. Section 3 describes specific examples for the cases of censored data analysis and estimating equations involving nuisance parameters. Section 4 provides empirical results. Section 5 concludes. All the proofs are deferred to the Appendix. 2. COMPARISON OF EMPIRICAL LIKELIHOOD CONFIDENCE REGIONS 50 Given a random sample Z n = {Z i } n with a common cumulative distribution function F 0 and Z 1 R k, we consider an (extended) empirical likelihood defined for some discrete cumulative distribution function F (denoted by EL(Z n, F )) that gives rise to empirical likelihood ratio test for a finite parameter θ = T (F ) R p with the test statistic 2 log ELR n (θ) = 2[max F F log EL(Z n, F ) max F log EL(Z n, F )], 55 where F is a set of discrete cumulative distribution functions that satisfy the constraint T (F ) = θ. The true value of the parameter is given as θ 0 = T (F 0 ), and the maximum empirical likelihood estimator is given as ˆθ n = T (F n ) by the maximizer F n = argmax F log EL(Z n, F ). We invert the likelihood ratio test and obtain a confidence region C n = { θ 2 log ELR n (θ) c}, where the critical value c is chosen according to the (asymptotic) distribution of 2 log ELR n (θ) under null hypothesis for some desired confidence level. The asymptotic distribution is given by the following results: under some regularity conditions n 1/2 (ˆθ n θ 0 ) U (1) 2 log ELR n (θ 0 ) = n(ˆθ n θ 0 ) V 1 (ˆθ n θ 0 ) + o p (1), where U is a p variate random variable with U N p (0, W ), and W and V are p p (semi- )positive definite matrices. Without loss of generality we consider two methods that provide the same maximum empirical likelihood estimators, ˆθ 1n = ˆθ 2n, but different likelihood ratio statistics ELR 1n (θ) and ELR 2n (θ). We suppose that the asymptotic results in (1) hold with V = V 1 and V = V 2 with W V1 1 = I and W V2 1 I for ELR 1n (θ) and ELR 2n (θ) respectively. By the results of Hjort, McKeague & Keilegom (2009) and Qin & Lawless (1994), the empirical likelihood with V = V 1 admits a chi-square with p degree of freedom as the limiting distribution of 2 log ELR 1n (θ), whereas the empirical likelihood with V = V 2 admits a weighted sum of p independent chi-squares of 1 degree of freedom as the limiting distribution. We consider respectively defined confidence regions with the two methods at the same confidence level as follows: C 1n = { θ 2 log ELR 1n (θ) c 1 }, C 2n = { θ 2 log ELR 2n (θ) c 2 }, 75 where the critical values c 1 and c 2 are appropriately chosen by the limiting distributions. C 1n and C 2n are centered at the same value in the sense that 2 log ELR 1n (ˆθ n ) = 2 log ELR 2n (ˆθ n ) = 0 where ˆθ n denotes the common maximum empirical likelihood estimator. The following theorem shows the asymptotic relative optimality of C 1n (θ).

3 Size and Shape of Confidence Regions from Extended Empirical Likelihood Tests 3 THEOREM 1. Assume regularity conditions under which the results in (1) holds for ELR 1n (θ) and ELR 2n (θ) respectively. Then, for confidence levels 1 αn 1/2 (0, 1), lim V ol(c 1n) < lim V ol(c 2n), n n where V ol(c) for any measurable set C in R p is defined as V ol(c) = R p I [t C] µ(dt) with µ the Lebesgue measure in R p. Theorem 1 implies that the empirical likelihood method that provides the likelihood ratio statistic which is only incompletely self-studentizing is not asymptotically optimal. Remark 1. In several papers (e.g., Wang & Jing (2001), Wang & Li (2002)), it is suggested that 80 one may adjust the empirical likelihood ratio of type 2 log ELR 2n by multiplying a factor, that will turn the asymptotic null distribution into a regular chi square, and may even improve the performance of the resulting confidence regions. It is clear that if the multiplying factor is (asymptotically) a constant and does not depend on θ, then the resulting confidence regions are the same as before, in the sense that the shape, size and orientation of the resulting confidence region is the 85 same as before. To achieve optimality, the multiplying factor must depend on θ and be the value log ELR 1n (θ)/ log ELR 2n (θ) = (ˆθ θ 0 ) V1 1 (ˆθ θ 0 )/(ˆθ θ 0 ) V2 1 (ˆθ θ 0 ) + o p (1). Obviously, computing this factor is equivalent to computing the statistic 2 log ELR 1n, if not more difficult. 3. EXTENDED EMPIRICAL LIKELIHOODS 90 In this section we present examples of extended methods that provide likelihood ratio statistics that do not observe the non-parametric version of the Wilks theorem. In each of these examples the construction of the likelihood does not match the structure of data. The ill-constructed likelihoods provide the likelihood ratio statistics that are only incompletely self-studentizing Analysis of Estimating Equations Involving Nuisance Parameters 95 We suppose that the unknown parameters θ R p is related to the common cumulative distribution function F 0 via a p variate vector of estimating functions involving finite dimensional nuisance parameters ψ R q, m(, θ, ψ), so E{m(Z 1, θ 0, ψ 0 )} = 0 for the true values θ 0 and ψ 0. We also suppose a q variate vector of estimating functions h(, θ, ψ) exists that defines the true parameter values θ 0 and ψ 0 uniquely as a solution to the following estimating equations jointly: 100 E{m(Z 1, θ 0, ψ 0 )} = 0, E{h(Z 1, θ 0, ψ 0 )} = 0. We consider empirical likelihood jointly for θ and ψ as follows: { n EL n (θ, ψ) = max w i m(z i, θ, ψ) = 0, w i h(z i, θ, ψ) = 0, w i } w i = 1, 0 w i, where w i denotes the point mass assigned to the i th observation Z i. Under some regular- 105 ity conditions, this likelihood is maximized by w i = n 1, i = 1,..., n, and the maximum empirical likelihood estimator (ˆθ n, ˆψ n ) is given by the solution to the following equations: n 1 n m(z i, θ, ψ) = 0, n 1 n h(z i, θ, ψ) = 0. Two methods have been proposed for the inference of θ alone. Qin and Lawless (1994) used the profile likelihood EL 1n (θ) = max ψ EL n (θ, ψ). The other or so-called plug-in method 110

4 M. ZHOU AND M.-O. KIM (Hjort, McKeague & Keilegom (2009)) used the following likelihood: { n } EL 2n (θ) = max w i m(z i, θ, ψ) = 0, w i = 1, 0 w i, w i where ψ is a n consistent estimator, so that E{m(Z i, θ 0, ψ)} = 0. Without externally given ψ available, a common choice is the maximum empirical likelihood estimator ˆψ n. With this particular choice, the plug-in method has ˆθ n as the maximum empirical likelihood estimator and max θ EL 2n (θ) = n n 1, same as the profile likelihood method. The likelihood ratio statistics are given respectively: ELR 1n (θ) = max ψ EL n(θ, ψ)/ n n 1, ELR 2n (θ) = EL 2n (θ)/ n n 1. (2) The following theorem shows that results in (1) hold for the profile and the plug-in method. THEOREM 2. Assume conditions under which the below results hold: ( ) ˆθn n 1/2 θ 0 N p (0, Σ), ˆψ n ψ 0 ( ) in distribution, where Σ = {S 12 S11 1 m/ θ m/ ψ S 21} 1 with S 12 = S21 = E, S h/ θ h/ ψ 11 = ( mm E mh ) hm hh, and the expectations taken at θ = θ 0 and ψ = ψ 0. Also assume that for any ψ with ψ ψ 0 = O p (n 1/2 ), n 1 n m(z i, θ 0, ψ)m (Z i, θ 0, ψ) E(mm ). Suppose Σ (jk), j, k = 1, 2, denotes the blocks of Σ that correspond to θ and ψ. Then, [ 1 U N p (0, Σ (11) ), W = V 1 = Σ (11), V 2 = E( m/ θ) {E(mm )} E( m/ θ)] 1. Specifically, 2 log ELR 2n (θ 0 ) p j=1 c jξ j in distribution, where ξ j are independent chisquared distributed random variables with degree of freedom 1 and c 1,..., c p are eigenvalues of V 2 W 1. The weighted χ 2 asymptotic results follow from the ill-construction of the likelihood: the likelihood of EL 2n (θ) takes the same form as that of Owen s likelihood for the case of independent data, whereas m(z i, θ, ˆψ n ), i = 1,..., n, are not independent anymore as they all involve ˆψ n Censored Data Analysis We suppose that {Z i } n = {(T i, δ i )} n where (T i, δ i ) denote right censored observations from lifetime variables {Y i } with a common cumulative distribution function F 0 subject to random censoring as follows: T i = min(y i, C i ) and δ i = I[Y i C i ] for i = 1,..., n. The censoring variable C i are assumed independent of the Y i with a cumulative distribution function G 0. We suppose the true value θ 0 of an unknown parameter θ R p is related to F 0 via estimating equations such that E{g(Y i )} = θ 0, where g = (g 1,..., g p ) are p functions. We examine three empirical likelihood methods that have been proposed for the inference of θ.

5 Size and Shape of Confidence Regions from Extended Empirical Likelihood Tests 5 Censored Likelihood Method: The first method uses the empirical likelihood defined for the censored data. Kaplan & Meier (1958), Owen (2001) and others have defined the empirical likelihood of the censored data {Z i } n for some cumulative distribution function F as n EL(Z n, F ) = { F (T i )} δ i { j:t j >T i F (T j )} 1 δi where F (t) = F (t+) F (t ) is the jump of F at t or a probability mass assigned to the point 135 t. As in the case of uncensored case, the definition assumes a discrete F ( ) with possible jumps only at the observed times. With w i = F (T i ), i = 1,..., n, the likelihood above can be written in term of the jumps and the log likelihood is log EL 1n (F ) = δ i log w i + (1 δ i ) log w j I [Tj >T i ], (3) where I [ ] denotes an indicator function. We note that the likelihood is maximized at the well known Kaplan-Meier estimator (Kaplan & Meier (1958)) with w i = ˆF KM (T i ), i = 1,..., n. Using the empirical likelihood of the censored data in (3) Zhou (2011) proposed the empirical likelihood for θ as EL 1n (θ) = max w i n w δ i i w j I [Tj >T i ] j=1 1 δ i j=1 w i g(t i ) = θ, w i = 1, 0 w i. The maximum empirical likelihood estimator ˆθ n is given by the equation, g(t i ) ˆF KM (T i ) = ˆθ n. (4) The likelihood ratio statistic is given as 2 log ELR 1n (θ) = 2 log{el 1n (θ)/el 1n (ˆθ n )} ac- 140 cordingly. Synthetic data and uncensored empirical likelihood method: Recall the parameter θ is defined as Eg(Y i ) = θ. The second method was motivated by the following lemma, which can be proved easily. LEMMA 1. For any function g( ) such that Eg(Y i ) is well defined, we consider X i = 145 {g(t i )δ i }/{1 G 0 (T i )}, where G 0 ( ) is the cumulative distribution function of censoring variable C i. Then, we have Eg(Y i ) = E(X i ). If G 0 were known, {X i } n or the synthetic data would be completely observed and the empirical likelihood of uncensored data could be considered as follows: { n } EL 2n (θ) = max w i w i X i = θ, w i = 1, 0 w i. w i The first and second methods utilize different likelihoods. EL 1n (θ) is based on the likelihood of censored data {(T i, δ i )} n with w i denoting the point mass of a discrete cumulative function of Y i. EL 2n (θ) is based on the likelihood of {X i } n with w i denoting the point mass of a cu- 150 mulative distribution function of X i, assuming the knowledge of the censoring time distribution G 0.

6 6 M. ZHOU AND M.-O. KIM Usually G 0 (t) or 1 G 0 (t) is not known, and thus X i are not available. An alternative method based on the synthetic data uses an estimator Ĝ(t), often the Kaplan-Meier estimator ĜKM, instead and defines the likelihood as { n } EL 2n(θ) = max w i w i X w i = θ, w i = 1, 0 w i, i where Xi = {g(t i)δ i }/{1 ĜKM(T i )}. The replacement, however, creates a problem that Xi are not independent since all of them involve the same estimator ĜKM(T i ). Since ĜKM(t) is uniformly consistent for G 0 (t), the mean of the (no longer independent identical) sequence {Xi } is still asymptotically same as Eg(Y ), but the variance of the sequence {Xi } is different from the variance of X i. To be precise, under some regularity conditions, we can show that as n, n( X θ 0 ) N(0, σ1 2) and n( X θ 0 ) N(0, σ2 2 ) in distribution with σ1 2 > σ2 2, if g( ) is a scalar function (p = 1). This result similarly holds in a multivariate case (p > 1). We refer to Srinivasan & Zhou (1994) for the calculation and the expression of the asymptotic variances or variance-covariance matrix. Like the plug-in method, the second synthetic data method uses the empirical likelihood method developed for independent identical sequences, but applies it to the sequence {Xi } that is not independent and identical. Interestingly, if we use the Kaplan-Meier estimator Ĝ KM (t) in the above, the empirical likelihood [ EL 2n (θ) is ] maximized when θ takes the value ˆθ n defined in (4), as ˆF KM (T i ) = δ i / n{1 ĜKM(T i )}. Therefore, even though the likelihoods EL 1n (θ) and EL 2n (θ) looks different, they maximize at the same value, and thus the confidence regions obtained by inverting the tests, are centered at the same point ˆθ n. Accordingly the likelihood ratio statistic is given as 2 log ELR 2n(θ) = 2 log{el 2n(θ)/EL 2n(ˆθ n )}. This method is also a plug-in method in the sense that we plug ĜKM(t) for G 0 (t). It differs from the plug-in method described in section 3 1 in that the problem has been switched from original data to synthetic data before plug-in and the parameter that got plugged is of infinite dimensional. Weighted empirical likelihood: This third method was proposed by Ren (2001, 2008). Given fixed weights v i 0, a weighted empirical likelihood is defined as follow: n EL 3n (w 1,..., w n ) = (w i ) v i, w i = 1. It is not hard to show that EL 3n is maximized when w i = v i / j v j. With the particular choice of v i = ˆF KM (T i ) the maximizer of EL 3n is wi = ˆF KM (T i ), the jump of Kaplan-Meier. Thus, with this choice of weights, the confidence regions obtained by inverting the likelihood ratio test defined below are also centered at the same value ˆθ n defined in (4). The weighted empirical likelihood under the null hypothesis can be computed by solving the constrained maximization problem: given v i = ˆF KM (T i ), { n } EL 3n (θ) = max (w i ) v i w i δ i g(t i ) = θ, w i = 1, 0 w i. w i The likelihood ratio statistic is given as 2 log ELR 3n (θ) = 2 log{el 3n (θ)/el 3n (ˆθ n )}. In this likelihood, observations i and j are supposed to be independent, from which the product form i

7 Size and Shape of Confidence Regions from Extended Empirical Likelihood Tests 7 of the empirical likelihood rises. With v i = ˆF KM (T i ), however, observations are implicitly connected via the weights. The following theorem shows that results in (1) hold for all three methods. THEOREM 3. Assume mild regularity conditions under which the Kaplan-Meier integral is consistent and asymptotically normal. Then, 180 n(ˆθn θ 0 ) U, (5) in distribution, where U N(0, Σ). Furthermore, 2 log ELR 1n (θ 0 ) = n(ˆθ n θ 0 ) V 1 1n (ˆθ n θ 0 ) + o p (1), (6) 2 log ELR2n(θ 0 ) = n(ˆθ n θ 0 ) V2n 1 n θ 0 ) + o p (1), (7) 2 log ELR 3n (θ 0 ) = n(ˆθ n θ 0 ) V3n 1 n θ 0 ) + o p (1), (8) where lim n V 1n = Σ, lim n V 2n Σ and lim n V 3n Σ. 185 It follows form Theorem 3 and Hjort, McKeague & Keilegom (2009) that 2 log ELR 1n (θ 0 ) χ 2 p in distribution whereas the Wilk s theorem does not hold for 2 log ELR2n (θ 0) and 2 log ELR 3n (θ 0 ). Theorems 1 and 3 prove that the confidence region provided by inverting the test 2 log ELR 1n (θ 0 ) by the censored empirical likelihood method is asymptotically optimal EMPIRICAL STUDIES We illustrate the relative optimality of confidence regions using two simulation studies regarding the cases of estimating equations and censored data analyses. Confidence regions were obtained by inverting the likelihood ratio tests described above. In each case the extended likelihood methods that yield the likelihood ratio statistics that do not observe the Wilk s theorem 195 are computationally challenging as the asymptotic distributions are empirically estimated. The associated confidence regions are also sub-optimal Simulation Studies 1 We used the following regression model with heteroscedastic errors and applied the profile likelihood and the plug-in method of section 3 1 the inference of two slope parameters: y i = α + β 1 x 1i + β 2 x 2i + e i x 2i, where x 1i N(0, 1), x 2i χ 2 1 and e i N(0, 1). Here the parameters of interest is θ = (β 1, β 2 ) and the intercept is the nuisance parameter (ψ = α). Consider Z i = (Z 1i, Z 2i ) with Z 1i = y i 200 and Z 2i = (x 1i, x 2i ). The estimating functions m(z i, θ, ψ) and h(z i, θ, ψ) are given respectively as m(z i, θ, ψ) = (Z 1i θ Z 2i )Z 2i and h(z i, θ, ψ) = (Z 1i θ Z 2i ). Figure 1 shows empirically estimated the null distributions of the resulting two likelihood ratio statistics i.e., the profile likelihood ratio statistic 2 log ELR 1n (θ 0 ) and the plug-in likelihood ratio statistic 2 log ELR 2n (θ 0 ) in (2), based on 5,000 Monte Carlo samples. The Q-Q plots show that the 205 null distribution of the plug-in likelihood ratio statistic remains deviating from a chi-squared distribution with degree of freedom 2 even in a large sample case. This implies that the plug-in likelihood ratio statistic remains only incompletely studentizing. Figure 2 shows an example of confidence regions. The confidence regions provided by the plug-in empirical likelihood method are larger relative to those by the profile likelihood and 210 shaped differently.

8 8 M. ZHOU AND M.-O. KIM Profile EL: n=100 Plug in EL: n=100 QQ Plot: n=100 Density Density Plug in EL LLR LLR Profile EL Profile EL: n=2000 Plug in EL: n=2000 QQ Plot: n=2000 Density Density Plug in EL LLR LLR Profile EL Fig. 1. Empirically estimated null distributions of the profile and the plug-in likelihood ratio statistic for the case of estimating equations with nuisance parameters n=100 n= True Value MELE Profile EL Plug in EL Fig. 2. Confidence regions drawn for a sample of the case of estimating equations with nuisance parameters 4 2. Simulation Studies 2 We used the following censored data setting: lifetime Y i and censoring times C i are independent identical with Y i exp(1) and C i exp(0.6).

9 Size and Shape of Confidence Regions from Extended Empirical Likelihood Tests % confidence regions EL region, solid line weighted EL region, dash line plug in EL region, dotted line Fig. 3. Confidence regions drawn for a sample data of n = 1, 000 subject to 37% censoring From the above Y i and C i we obtain T i and δ i as described in section 3 2. The two expectations 215 we are going to test is EY i = a 1 ; EI[Y i > 0.5] = a 2 for various values of θ = (a 1, a 2 ). The true value of a 1 = 1 and a 2 = e 0.5. As implied by Theorem 3, the null distribution of the censored empirical likelihood ratio statistic is known, whereas the null distributions of the plug-in and the weighted empirical likelihood ratio statistics 2 log ELR 2n (θ 0) and 2 log ELR 3n (θ 0 ) need to be estimated. We empirically 220 estimated the null distributions based on 10,000 Monte Carlo samples and used the simulated null distribution to set the confidence level. Figure 3 shows three confidence regions for θ = (a 1, a 2 ) drawn for a sample data with the sample size n = 1, 000 subject to 37% censoring. We see that the orientation and shape of the confidence regions by the plug-in and the weighted empirical likelihood methods (dotted/dashed lined confidence region) are very different from the 225 confidence region by the censored empirical likelihood method (solid lined confidence region). Also, the confidence region by the censored empirical likelihood method is the one with the smallest area. 5. CONCLUSION In this paper we consider extensions of empirical likelihood method with some providing 230 likelihood ratio statistics that observe a nonparametric version of Wilks theorem similarly as the Owen s empirical likelihood ratio statistic and others not. Limiting to the extensions that provide the same maximum empirical likelihood estimators, we evaluate relative optimality of the methods by comparing the confidence regions obtained by inverting the likelihood ratio tests.

10 M. ZHOU AND M.-O. KIM We show that the extension method which yield the likelihood ratio statistic observing the Wilks theorem provides the smallest confidence region. ACKNOWLEDGEMENT First author is supported by US NSF Grant DMS Second author is supported by US NSF Grant DMS APPENDIX Proof of Theorem 1. It follows from (1) that C 1n and C 2n are asymptotically equivalent to C 1n = { θ n(ˆθ θ) V 1 1 (ˆθ θ) c 1 }, C 2n = { θ n(ˆθ θ) V 1 2 (ˆθ θ) c 2 }, where n 1/2 (ˆθ θ) N p (u, W ) for θ = θ 0 + n 1/2 u with a p variate constant vector u. By Lemma 2 below, we have lim n V ol(c1n ) < lim n V ol(c2n ) unless V 2 = W LEMMA 2. (i) Consider two types of confidence regions based on an estimator ˆθ that has distribution N(θ 0, I): C 1 = {θ : (ˆθ θ) (ˆθ θ) < c}, C 2 = {θ : (ˆθ θ) B(ˆθ θ) < c }, where B I is a symmetric non-negative definite matrix. When both confidence region have same coverage probability 1 α (0, 1), C 1 is superior to C 2 in the sense that V ol(c 1 ) < V ol(c 2 ). (ii) If ˆθ is normally distributed with N(θ 0, Σ), then the optimal confidence region for the parameter θ 0 is {θ : (ˆθ θ)σ 1 (ˆθ θ) < c}. Remark 2. Obviously if B = σ 2 I, then the two confidence regions are identical. The constant σ 2 will be canceling out when adjust the cut off level c. So the Lemma should be understood as to use any symmetric non-negative definite matrix other than σ 2 I type. Remark 3. If confidence regions are allowed not centered at ˆθ then there may be better regions available (Stein phenomena). We only consider a limited class of confidence regions that are centered at the same ˆθ since they are derived from extended empirical likelihood ratio tests. Proof of Lemma 2. First we notice (ii) is an easy consequence of (i), so we only prove (i). Note that the coverage probability of C 1 is θ 0 x 2 <c f(x)dx, where f(x) denotes the density of ˆθ. Define a distance by d B (θ 0, ˆθ) = (θ 0 ˆθ) B(θ 0 ˆθ). Then similarly the coverage probability of C 2 is d B (θ 0 x)<c f(x)dx. By substitute y = ˆθ θ 0 then y N(0, I), and the two integrals become f(y)dy y 2 <c and f(y)dy. d B (y)<c Denote the regions of the two integrations above as R 1 and R 2. Also, let R 1 R 2 = P. Then the requirement of equal coverage probability imply f(y)dy = f(y)dy and f(y)dy = f(y)dy. (9) R 1 R 2 R 1 \P R 2 \P Now, the density f(y) on R 1 \ P has values at least as large as f(y ) = K for a y with y 2 = c, since the multi-variate normal N(0, I) density is a decreasing function of y 2.

11 Size and Shape of Confidence Regions from Extended Empirical Likelihood Tests 11 On the other hand, the density f(y) on R 2 \ P has value at most K = f(y ). Thus 265 f(y)dy > Kdy = K dy = K V ol(r 1 \ P ) R 1 \P R 1 \P and by (9) the left hand side is also equal to f(y)dy < R 2 \P R 2 \P R 1 \P Kdy = K V ol(r 2 \ P ). By the above two inequalities, we have V ol(r 2 \ P ) > V ol(r 1 \ P ) and thus we also have V ol(r 2 ) > V ol(r 1 ). It is obvious that the inequality is strict unless R 2 coincide with R 1. Proof of Theorem 2. Define Q n = ( n 1 n m(z i, θ 0, ψ 0 ), n 1 n h(z i, θ 0, ψ 0 ) ). From the proof of Theorem 1 of Qin and Lawless (1994) we have 270 n 1/2 (ˆθ n θ 0 ) = ( Σ (11) Σ (12) ) S21 S 1 11 (n1/2 Q n ) + o p (1), (10) n 1/2 ( ˆψ n ψ 0 ) = ( Σ (21) Σ (22) ) S21 S 1 11 (n1/2 Q n ) + o p (1). Also from the proof of Corollary 5 of Qin and Lawless (1994), we have [ 2 log ELR 1n (θ 0 ) = (n 1/2 Q n ) S11 1 { S 12 ΣS 21 S 12(2) S21(2) S11 1 S } ] 1 12(2) S21(2) S11 1 (n1/2 Q n ), where S 21(2) = S12(2) = E { ( m/ ψ), ( h/ ψ) }. After some matrix manipulation we have 275 [ 2 log ELR 1n (θ 0 ) = (n 1/2 Q n ) S11 1 S 12 Σ ( )] { S 21(2) S11 1 S } 1 S 21 S11 1 (n1/2 Q n ) 12(2) ( ) = (n 1/2 Q n ) S11 1 S Σ(11) 12 (Σ Σ (11) ) 1 [Σ (11), Σ (12) ]S 21 S11 1 (n1/2 Q n ) (21) = n(ˆθ n θ 0 ) V 1 1 (ˆθ n θ 0 ), where V 1 = Σ (11). On the other hand, { ( 2 log ELR 2n (θ 0 ) = 2 log 1 + λ m(z i, θ 0, ˆψ ) ( n ) log 1 + ˆλ m(z i, ˆθ n, ˆψ n )) }, 280 where ˆλ and λ are defined by solutions to the equations 0 = n 1 m(z i, ˆθ n, ˆψ n ) 1 + ˆλ m(z i, ˆθ n, ˆψ n ), 0 = n 1 m(z i, θ 0, ˆψ n ) 1 + λ m(z i, θ 0, ˆψ n ). By the standard arguments of empirical likelihood methods, and (10), we have { 2 log ELR 2n (θ 0 ) = λ E(mm )} { λ ˆλ E(mm )} ˆλ + op (1) ( =n(ˆθ n θ 0 ) E m ) { ( 1 E(mm )} E m ) (ˆθ n θ 0 ) + o p (1). 285 θ θ Proof of Theorem 3. When p = 1 the result (5) is given by Akritas (2001). Zhou (2011) show that for p > 1 the equation (5) also holds with the jkth element of Σ defined by df 0 (x) σ jk = [g j (x) ḡ j (x)][g k (x) ḡ k (x)] 1 G 0 (x ), (11)

12 M. ZHOU AND M.-O. KIM where ḡ j (t) denoting the advanced time transformation of g j in Efron and Johnstone (1990). Zhou (2011) also showed that the equation (6) holds with the jkth element of V 1n given by ˆσ jk = [g j (T i ) ḡ j (T i )][g k (T i ) ḡ k (T i )] ˆF KM (T i ) 1 ĜKM(T i ). Since V 1n is just Σ with the unknown F 0 and G 0 replaced by their Kaplan-Meier estimators and the well known fact that the Kaplan-Meier estimators are uniformly consistent, V 1n Σ in probability as n. We refer to Zhou (2011) for details. It follows from Owen (2001, p ) (with Xi in place of X i there) that where 2 log ELR 2n(θ 0 ) = U 2nV 1 2n U 2n + o p (1) U 2n = n[ X θ 0 ] = n(ˆθ n θ 0 ), V 2n = n 1 (Xi θ 0 )(Xi θ 0 ). Since ĜKM(t) is uniformly consistent for G 0 (t), V 2n = n 1 (X i θ 0 )(X i θ 0 ) + o p (1) = V ar(x 1 ) + o p (1). By the results in Zhou (1992) and Srinivasan and Zhou (1994), var(x 1 ) Σ however. This suffices to show (7). As for the equation (8), when p = 1, Ren (2001) showed that the limiting distribution of this test statistic with θ = θ 0 is a scaled chi-square distribution with degree of freedom 1. To obtain a multivariate version (with p > 1), we modify the proof of Ren along the steps of Owen (2001, p ) to incorporates weights, and derive the following: where U n = n[ 2 log ELR 3n (θ) = U n V 1 3n U n + o p (1) g(t i ) ˆF KM (T i ) θ 0 ] = n(ˆθ n θ 0 ) and V 3n is a matrix with the jkth element given by v jk = (g j (T i ) θ 0 )(g k (T i ) θ 0 ) ˆF KM (T i ). An application of the law of large number for the Kaplan-Meier integral gives v jk = (g j (t) θ 0 )(g k (t) θ 0 )d ˆF KM (t) (g j (t) θ 0 )(g k (t) θ 0 )df 0 (t) in probability as n. Since Σ depends on the censoring distribution G 0, while lim V 3n does not, lim n V 3n Σ. REFERENCES AKRITAS, M. (2000). The central limit theorem under censoring. Bernoulli 6, EFRON, B. & JOHNSTONE, I. M. (1990). Fisher s information in terms of the hazard rate. Ann. Statist. 18,

13 Size and Shape of Confidence Regions from Extended Empirical Likelihood Tests 13 HJORT, N. L., MCKEAGUE, I. W. & KEILEGOM, I. V. (2009). Extending the scope of empirical likelihood. Ann. 310 Statist. 37, KAPLAN, E. L. & MEIER, P. (1958). Nonparametric Estimation From Incomplete Observations. J. Amer. Statist. Assoc. 53, LI, G., LI, R. & ZHOU, M. (2005). Empirical likelihood in survival analysis. In Contemporary Multivariate Analysis and Design of Experiments Edited by J. Fan and G. Li. pp The World Scientific Publisher. 315 OWEN, A. (1988). Empirical Likelihood Ratio Confidence Intervals for a Single Functional. Biometrika 75, OWEN, A. (1990). Empirical Likelihood Ratio Confidence Regions. Ann. Statist. 18, OWEN, A. (2001). Empirical Likelihood. New York: CRC Press. QIN, J. & LAWLESS, J. (1994). Empirical Likelihood and General Estimating Equations. Ann. Statist. 22, REN, J. (2001). Weighted empirical likelihood ratio confidence intervals for the mean with censored data. Ann. Inst. 320 Statist. Math. 53, REN, J. (2008). Weighted empirical likelihood in some two-sample semiparametric models with various types of censored data. Ann. Statist. 36, SHI, J. & LAU, T.-S. (2000). Empirical Likelihood for Partially Linear Models. J. Multivariate Anal. 72, SRINIVASAN, C. & ZHOU, M. (1994). Linear regression with censoring. J. Multivariate Anal. 49, TURNBULL, B. W. (1976). The empirical distribution function with arbitrarily grouped, censored and truncated data. J. R. Statist. Soc. B 38, WANG, S., QIAN, L. & CARROLL, R. (2010). Generalized empirical likelihood methods for analyzing longitudinal data. Biometrika 97, WANG, Q. H. & JING, B. Y (2001). Empirical likelihood for a class of functionals of survival distribution with 330 censored data Ann. Inst. Statist. Math. 53, WANG, Q. H. & LI, G. (2002). Empirical likelihood semiparametric regression analysis under random censorship J. Multivariate Anal. 83, ZHOU, M. (1992). Asymptotic normality of the synthetic data regression estimator for censored survival data. Ann. Statist. 20, ZHOU, M. (2011). A Wilks theorem for the censored empirical likelihood of several means. Preprint mai/research/note3.1.pdf [Received June Revised ]