exclusive prepublication prepublication discount 25 FREE reprints Order now!

Size: px

Start display at page:

Download "exclusive prepublication prepublication discount 25 FREE reprints Order now!"

Alexis Hubbard
5 years ago
Views:

1 Dear Contributor Please take advantage of the exclusive prepublication offer to all Dekker authors: Order your article reprints now to receive a special prepublication discount and FREE reprints when you order 0 or more. This offer is only available through the link below. Thank you for being a Dekker Author. Order now!

2 000_LSTA_0_R_00 0 Symmetric Location Estimation/Testing by Empirical Likelihood Kyoungmi Kim and Mai Zhou Author Query AQ Au: Please provide street name & fax. no. if any in CAF.

3 0 Symmetric Location Estimation/Testing by Empirical Likelihood Kyoungmi Kim and Mai Zhou* Department of Statistics, University of Kentucky, Lexington, Kentucky, USA ABSTRACT The problem of estimating the center of a symmetric distribution is well studied and many nonparametric procedures are available. It often serves as the test problem for many nonparametric estimation procedures, and stimulated the development of efficient nonparametric estimation theory. We use this familiar setting to illustrate a novel use of empirical likelihood method for estimation and testing. Empirical likelihood is a general nonparametric inference method, see Owen [Owen, A. (0). Empirical Likelihood. London: Chapman and Hall]. However, for symmetric location problem (and some other problems) empirical likelihood has difficulties. Owen (0) call them challenges for the empirical likelihood. We propose and study a way to use the empirical likelihood with such problems by modifying the parameter space. We illustrate this approach by *Correspondence: Mai Zhou, Department of Statistics, University of Kentucky, Lexington, KY 0-00, USA; mai@ms.uky.edu. DOI: 0.0/STA-000 Copyright # 0 by Marcel Dekker, Inc. 000_LSTA_0_R_00 COMMUNICATIONS IN STATISTICS Theory and Methods Vol., No., pp., (Print); -X (Online) AQ

4 000_LSTA_0_R_00 Kim and Zhou 0 applying it to the symmetric location problem. We show that the usual asymptotic theory of empirical likelihood still holds and the asymptotic efficiency of the so obtained empirical NPMLE of location is studied. Key Words: Many constraints of symmetry; Asymptotic chi-square distribution; Nonparametric information bound. AMS Subject Classification: Primary G0; Secondary G0.. INTRODUCTION Empirical likelihood, see Owen (0), is a general nonparametric method that provides the advantages of a (parametric) likelihood ratio based inference without having to assume a parametric family of distributions. The advantages are more profound for censored=truncated data analysis where the traditional (Wald) approach becomes more complicated due to the difficulty in the estimation of variance. While the procedure discussed in this paper can clearly be used in the case of censored data, we shall focus on the non-censored data setting for clarity. Definition (Owen, 0). For given n i.i.d. observations X ;...; X n with common distribution function F X ðtþ, the nonparametric or empirical likelihood for the distribution function F X ðtþ is LðFÞ ¼ Yn i¼ w i where w i ¼ DF X ðx i Þ is the probability P F ðx ¼ x i Þ. The empirical distribution function, bf n, maximizes LðFÞ among all the distribution functions. However, when we restrict the parameter space to be all the symmetric distributions, Y ¼fF j F is symmetric wrt y; some y R g; the maximization of the above empirical likelihood has problems: the NPMLE does not exist or there are many NPMLE s having the same (empirical) likelihood value. When the true F is continuous, it is easy to see that PðbF n is symmetricþ ¼0 where bf n is the empirical distribution function. ðþ

5 000_LSTA_0_R_00 Symmetric Location Estimation/Testing 0 Example. Suppose X < X < X are three ordered observations from a continuous symmetric distribution F 0 ðtþ, with an unknown location of symmetry, y. Without loss of generality, assume X X ¼ X X. By adding one extra jump point to bf n, we can make bf n into Fn ðtþ which is symmetric. Unfortunately there are more than one candidate of Fn ðtþ s that can have the same empirical likelihood value. If we believe that y is located in the middle of X and X, one more jump at the location X on the right side of X is needed to produce an Fn ðtþ which is symmetric about y with the empirical likelihood value LðFÞ ¼w w w, and w þ w þ w þ w ¼. Otherwise, if we believe that y is located in the middle of X and X, then an extra jump at location X 0 to the left of X is needed so that Fn ðtþ is symmetric about y with the empirical likelihood value LðFÞ ¼w w w with w 0 þ w þ w þ w ¼. It is clear the two different Fn ðtþ s can achieve the same likelihood value. Indeed the local maximum is achieved for the first case at w ¼ w ¼ = and w ¼ w ¼ = and for the second case at w 0 ¼ w ¼ = and w ¼ w ¼ =. Both Fn ðtþ are NPMLE s having the same empirical likelihood value but with a very different y and Fn ðtþ. This implies that NPMLE of y and Fn ðtþ is not unique. For larger samples there are even more candidates, Fn, that are symmetric and can achieve the same maximum empirical likelihood value. There are many other setups that have the same difficulty, including two-sample location-shift problem. See Zhou (0) for a two-sample location problem, and Kim (0) for other cases and more discussions. Roughly speaking, when the knowledge=restriction on the distributions cannot be achieved by adjusting the jump sizes of the empirical distribution function, then there often is difficulty. (Here we have to add an extra jump to the empirical distribution to make it symmetric.) One of the purpose of this paper is to illustrate the use of the envelope empirical likelihood method of Zhou (0) to overcome this difficulty. The idea is to apply the empirical likelihood on a carefully constructed sequence of shrinking parameter spaces Y k, that converge to the Y ¼fall symmetric CDFg. On each of the Y k, the NPMLE uniquely exists and the (regular) empirical likelihood theory works beautifully. This sequence of shrinking parameter spaces is called the envelope parameter space. This approach is quite general. First, it can easily work with censored data. Second, this approach also works for other challenging situations like location-scale problems where the parameter space is not all the possible distributions but is still infinite dimensional and requires the CDF to

6 000_LSTA_0_R_00 Kim and Zhou 0 have jump points other than the observed data points. Finally we point out that the method proposed can easily be generalized to handle higher dimensional data. We, however, will stick to the one sample symmetric location problem with uncensored data in this paper for the clarity of the presentation. The semi-parametric problem of estimating symmetric location has been studied by many people, see many examples in the book by Bickel et al. (). Our approach is closer to the Empirical Process Approach of Hsieh (). The advantage of the method proposed here is the simplicity of the procedure, we have a chi-square null distribution to set the P-value and there is no need to estimate the variance covariance matrix when construct confidence interval=region. In the empirical process approach of Hsieh (and many other adaptive estimation procedures) you need to first estimate a variance covariance matrix of the empirical process involved and then use the estimated matrix to do a weighted least squares to produce the estimator. For doubly censored data, the variance covariance can be difficult to estimate. The advantages of our procedure are of course inherited from the (empirical) likelihood ratio method.. ENVELOPE EMPIRICAL LIKELIHOOD FOR SYMMETRIC DISTRIBUTIONS Suppose X ;...; X n are i.i.d. observations from a symmetric distribution F with an arbitrary location parameter y (i.e., the center of symmetry of F is y). Maximizing the log empirical likelihood, log L ¼ Xn i¼ log w i ¼ Xn i¼ log DFðX i Þ; over all symmetric distributions is not well defined as seen in the example of section one. We enlarge the parameter space to Y. It will be shown later that the NPMLE is well defined on this space. The enlarged parameter space is defined as Y ¼ ff : all distributions satisfy ðþg: ðþ For given t i, i ¼ ; ;...; k for some y; Fðy t i Þ¼ Fðy þ t i Þ: ðþ ðþ

7 000_LSTA_0_R_00 Symmetric Location Estimation/Testing 0 We may rewrite the above as integrations: for some y; Z y ti If we take the functions dfðtþ ¼ Z yþt i dfðtþ; i ¼ ; ;...; k: ðþ g i ðy tþ ¼I ½0y t ti Š ¼ I ½ty ti Š and g i ðy tþ ¼I ½0y tþt i Š ¼ I ½tyþti Š in the above, the integration equations can take the form of Z g i ðy tþdfðtþ ¼ Z gi ðy tþdfðtþ; i ¼ ; ;...; k: ðþ We can in fact use g i and gi that are smooth and define the symmetry similarly. It turns out that maximizing the log empirical likelihood log L defined above among distributions in the parameter space Y is well defined and thus yields both the (envelope) empirical NPMLE, ^y and bfðtþ. The estimate bfðtþ is symmetric at least on k points: t i. We shall only focus on the study of the estimator ^y in this paper. We note that the newly defined parameter space actually is dependent on the choice and number of the functions g i and gi, or the t i points. When the points t i in () becomes dense then the space Y becomes the space of all symmetric distributions. The many choices of the space Y is similar to the choices of bandwidth in the histogram estimation of a density function. However, as sample size grows, we do not have to adaptively chose the t i points like choosing the bandwidth in density estimation. If we use a fixed choice of Y, (not changing with sample size) we still obtain a root n consistent estimator of y and a Wilks theorem in likelihood ratio test. If we do adaptively change the space Y, we may improve efficiency. In the following we only work with a fixed Y.. ENVELOPE EMPIRICAL LIKELIHOOD RATIO TEST Suppose FðÞ is symmetric about y. Consider testing the hypothesis: H 0 : y ¼ y 0 ; vs. H A : y ¼ y 0 : The test statistic we propose is the likelihood ratio statistics n o T ¼ max log L max log L : ðþ Y with y¼y 0 Y with yr

8 000_LSTA_0_R_00 Kim and Zhou 0 We show below that this empirical likelihood ratio test statistics will have an approximate chi-square distribution with one degree of freedom under the null hypothesis. We reject H 0 for larger values of T. Confidence intervals for y can be obtained by inverting the chi-square test. The y value that achieve the maximum in the second term of () will be our (envelope) NPMLE of the location, ^y. Denote the column vectors gðy tþ ¼fg ðy tþ;...; g k ðy tþg T ; g ðy tþ ¼fg ðy tþ;...; g k ðy tþgt ; l ¼fl ;...; l k g T : and Lemma. Suppose X ;...; X n are n i.i.d. observations from a symmetric distribution F with an arbitrary location parameter y 0. Then for any fixed y, the probability w i that maximizes the log likelihood function () satisfying the constraints of () is given by w i ðl; yþ ¼ n nl T ðgðy x i Þ g ðy x i ÞÞ ; where l T gðy x i Þ denotes the inner product P k j¼ l jg j ðy x i Þ. The l value in () is obtained as the solution of the following k equations, respectively, h r ðl; yþ ¼ Xn i¼ ½g r ðy x i Þ g r ðy x iþš n nl T ðgðy x i Þ g ðy x i ÞÞ ¼ 0; ðþ r ¼ ;...; k: ðþ Clearly the so determined l value depends on the y given, so in the subsequent discussions we shall write l as lðyþ. Proof. Standard Lagrange multiplier calculation similar to Owen (0). &. LARGE SAMPLE RESULTS Lemma. Under mild regularity conditions, the solution l of the constraint equation in () under the null hypothesis has the following

9 000_LSTA_0_R_00 Symmetric Location Estimation/Testing 0 asymptotic representations: (i) Let y 0 be the true parameter, and assume h ðl; y 0 Þ l¼0 ð0; y 0 s is an invertible k k matrix, then we have pffiffiffi n lðy0 Þ! D Nð0; SÞ; as n! where the variance covariance matrix is S ¼ lim ½h 0 ð0; y 0 ÞŠ : n! (ii) In addition, p assume that gðþ and g ðþ are smooth and jy y 0 j¼oð= ffiffiffi n Þ,wehave lðyþ ¼lðy 0 Þ h 0 ð0; y 0 Þ Gðy y 0 Þþo p ðjy y 0 jþ where G is a k matrix with its column defined as G ¼ ( ) Xn g 0 ðy 0 x i Þ g 0ðy 0 x i Þ ;...; Xn gk 0 ðy 0 x i Þ gk 0ðy T 0 x i Þ : i¼ n Proof. Use Taylor expansion on h with respect to l in Eq. (). & Remark. h 0 ð0; y 0 Þ¼ The k k r s l¼0 i¼ i¼ " # Xn ½g r gr Š½g s gs Š n is easy to verify to be symmetric and at least non-negative definite. Proper choice of the g, g function will guarantee it to be positive definite. Remark. As n!, the limit of G is easily seen to be E½g 0 ðy 0 XÞ g 0 ðy 0 XÞŠ;...; E½g 0 k ðy 0 XÞ g 0 k ðy 0 XÞŠ : n

10 000_LSTA_0_R_00 Kim and Zhou 0 Notice that we have assumed the smoothness of the g function in the above. However, even if g is an indicator function as in (), we may use Dirac s delta function to obtain that Z E½g 0 ðy 0 XÞŠ ¼ g 0 ðy 0 xþfðxþdx ¼ fðy 0 t Þ and Z E½g 0 ðy 0 XÞŠ ¼ and the limit of G in this case will be g 0 ðy 0 xþfðxþdx ¼ fðy 0 þ t Þ f½fðy 0 t Þþfðy 0 þ t ÞŠ;...; ½fðy 0 t k Þþfðy 0 þ t k ÞŠg: Now the main theorems. Theorem. Under the same conditions as in Lemma, the empirical likelihood ratio statistic T defined in () has asymptotically a chi-square distribution with one degree of freedom: Proof. T! D w ðþ ; as n!: See Appendix. Theorem. Let ^y be the location parameter that maximizes the second term in (). The asymptotic distribution of the (envelope) maximum empirical likelihood estimator, ^y, is given by where Proof. pffiffiffi n ð^y y 0 Þ! D Nð0; s Þ; s lim n! fg T ½h 0 ð0; y 0 ÞŠ Gg : See Appendix.

11 000_LSTA_0_R_00 Symmetric Location Estimation/Testing 0. EFFICIENCY We now take a closer look at the asymptotic variance of the (envelope) NPMLE obtained in Theorem above. By noticing the special structure of the matrix, h 0 ð0; y 0 Þ, we can show (see Kim, 0 for details), that the quadratic form appeared in the variance above, G T ½h 0 ð0; y 0 Š G, can be explicitly computed as G T ½h 0 ð0; y 0 ÞŠ G ¼ X r ðg r G r Þ h r h r where G r is the rth component of the vector G, andh r is the rth diagonal element of the matrix h 0 ð0; y 0 Þ.(G 0 ¼ 0 and h 0 ¼ 0 by convention.) In view of Remark, the numerators in each term of the above summation has limit as n!. Similar calculation is available for the denominator. Putting them together we see that the above summation have a limit X k r¼ ð½fðy 0 t r Þþfðy 0 þ t r ÞŠ ½fðy 0 t r Þþfðy 0 þ t r ÞŠÞ ½Fðy 0 t r Þþ Fðy 0 þ t r ÞŠ ½Fðy 0 t r Þþ Fðy 0 þ t r ÞŠ : ð0þ It is worth pointing out that (under mild regularity conditions on f) (0) is a finite sum approximation (from below) to the integral Z y0 ½f 0 ðtþš fðtþ dt ¼ Z ½f 0 ðtþš fðtþ dt; ðþ which is the information bound of the semiparametric model for estimating y, see Bickel et al. (). This shows that the (envelope) NPMLE obtained in Theorem on the space Y is close to be fully asymptotically efficient, because the asymptotic variance obtained in Theorem equals to the inverse of the finite sum, (0), which in turn is approximately the semiparametric information bound, (). Proof of Theorem. fðlðyþ; yþ ¼ Xn i¼ APPENDIX Define a function of y by log w i ðlðyþ; yþ:

12 000_LSTA_0_R_00 0 Kim and Zhou 0 The log empirical likelihood ratio statistic, T, is then T ¼ ffðlðy 0 Þ; y 0 Þg þ minffðlðyþ; yþg: y By the Taylor expansion on the second term above, we have T ¼ ffðlðy 0 Þ; y 0 Þg þ min y fðlðy 0 Þ; y 0 Þþðy y þ ðy y þ o pðþ ðþ ¼ min y fðy y 0 ÞA þðy y 0 Þ B þ o p ðþg ðsayþ: ðþ Aside from the small term, we immediately get that the minimum value achieved by T is A =B, and the ^y that achieves this minimum satisfy ^y y ¼ A=B. The rest of the proof is just calculating the derivatives and checking that A =B has the correct asymptotic distribution. Notice that the at (l ¼ lðy 0 Þ; y ¼ y 0 ) is zero. And from Lemma (ii), we have the derivative of lðyþ: l 0 ðyþj y¼y0 ¼ ½h 0 ð0; y 0 ÞŠ G: After tedious but straight forward calculations, we have T ¼ A B þ o pðþ ¼ ð pffiffiffi n lðy0 ÞGÞ þ o p ðþ: B=n On the other hand, we can show that as n! P lim B=n ¼ P lim G T ½h 0 Š G where P lim denote the limit in probability. Applying the asymptotic distribution of lðy 0 Þ shown in Lemma (i): pffiffiffi n lðy0 Þ! D Nð0; SÞ; as n!; and the Slutsky theorem, the log empirical likelihood ratio statistic T is seen to have the limit of chi-square with one degree of freedom. &

13 000_LSTA_0_R_00 Symmetric Location Estimation/Testing 0 Proof of Theorem. From the proof of Theorem we already noticed that (aside from a negligible term) the y value that achieves the minimum is ^y y 0 ¼ A B þ o pðþ ¼ nlðy 0ÞG þ o p ðþ B and, therefore, by Lemma pffiffiffi pffiffiffi n lðy0 ÞG n ð^y y 0 Þ¼ þ o p ðþ B=n! D Nð0; ðg T h 0 ð0; y 0 Þ GÞ Þ: REFERENCES Bickel, P., Klaassen, C., Ritov, Y., Wellner, J. (). Efficient and Adaptive Estimation for Semiparametric Models. Baltimore: The Johns Hopkins University Press. Hsieh, F. (). Empirical process approach in two-sample locationscale model with censored data. Ann. Statist. :0. Kim, K. (0). Empirical Likelihood Ratio Method When Additional Information is Known. Ph.D. dissertation, Department of Statistics, University of Kentucky. Owen, A. (0). Empirical Likelihood. London: Chapman and Hall. Zhou, M. (0). Empirical Envelope MLE and LR Test. Technical Report, University of Kentucky, Department of Statistics.

EMPIRICAL ENVELOPE MLE AND LR TESTS. Mai Zhou University of Kentucky

EMPIRICAL ENVELOPE MLE AND LR TESTS Mai Zhou University of Kentucky Summary We study in this paper some nonparametric inference problems where the nonparametric maximum likelihood estimator (NPMLE) are