A central limit theorem for the Hellinger loss of Grenander-type estimators

Transcription

1 Delft University of Technology A central limit theorem for the Hellinger loss of Grenander-type estimators Lopuhaa, Rik; Musta, Eni DOI./stan.53 Publication date 8 Document Version Other version Published in Statistica Neerlandica Citation APA) Lopuhaä, H. P., & Musta, E. 8). A central limit theorem for the Hellinger loss of Grenander-type estimators. Statistica Neerlandica, Important note To cite this publication, please use the final published version if applicable). Please check the document version above. Copyright Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the authors) and/or copyright holders), unless the work is under an open content license such as Creative Commons. Takedown policy Please contact us and provide details if you believe this document breaches copyrights. We will remove access to the work immediately and investigate your claim. This work is downloaded from Delft University of Technology. For technical reasons the number of authors shown on this cover page is limited to a maximum of.

2 Received: December 6 Revised: 6 November 7 Accepted: July 8 DOI:./stan.53 ORIGINAL ARTICLE A central limit theorem for the Hellinger loss of Grenander-type estimators Hendrik P. Lopuhaä Eni Musta Delft Institute of Applied Mathematics, Delft University of Technology, Delft, The Netherlands Correspondence Eni Musta, Delft Institute of Applied Mathematics, Delft University of Technology, 68 XE Delft, The Netherlands. E.Musta@tudelft.nl We consider Grenander-type estimators for a monotone function λ [, ] R +, obtained as the slope of a concave convex) estimate of the primitive of λ. Our main result is a central limit theorem for the Hellinger loss, which applies to estimation of a probability density, a regression function or a failure rate. In the case of density estimation, the limiting variance of the Hellinger loss turns out to be independent of λ. KEYWORDS central limit theorem, Grenander estimator, Hellinger distance, isotonic estimation INTRODUCTION One of the problems in shape-constrained nonparametric statistics is to estimate a real-valued function under monotonicity constraints. Early references for this type of problem can be found in Grenander 956), Brunk 958), and Marshall and Proschan 965), concerning the estimation of a probability density, a regression function, and a failure rate under monotonicity constraints. The asymptotic distribution of these types of estimators was first obtained by Prakasa Rao 969, 97) and reproved by Groeneboom 985), who introduced a more accessible approach based on inverses. The latter approach initiated a stream of research on isotonic estimators, for example, see Groeneboom and Wellner 99), Huang and Zhang 994), Huang and Wellner 995), and Lopuhaä and Nane 3). Typically, the pointwise asymptotic behavior of isotonic estimators is characterized by a cube-root n rate of convergence and a nonnormal limit distribution. The situation is different for global distances. In Groeneboom 985), a central limit theorem was obtained for the L -error of the Grenander estimator of a monotone density Groeneboom, Hooghiemstra, & Lopuhaä, 999), and a similar result was established in Durot ) for the regression context. Extensions to general L p -errors can be found in Kulikov and Lopuhaä 5) and Durot 7), where the latter provides a unified approach that applies to a variety of statistical... This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited. 8 The Authors. Statistica Neerlandica published by John Wiley & Sons Ltd on behalf of VVS. Statistica Neerlandica. 8; 7. wileyonlinelibrary.com/journal/stan

3 LOPUHAÄ AND MUSTA models. For the same general setup, an extremal limit theorem for the supremum distance was obtained in Durot, Kulikov, and Lopuhaä ). Another widely used global measure of departure from the true parameter of interest is the Hellinger distance. It is a convenient metric in maximum likelihood problems, which goes back to the works of LeCam 97, 973), and it has nice connections with Bernstein norms and empirical process theory methods to obtain rates of convergence, due fundamentally to the works of Birgé and Massart 993), Wong and Shen 995), and others; see section 3.4 of van der Vaart and Wellner 996) or chapter 4 in van de Geer ) for a more detailed overview. Consistency in Hellinger distance of shape-constrained maximum likelihood estimators was investigated by Pal, Woodroofe, and Meyer 7), Seregin and Wellner ), and Doss and Wellner 6), whereas rates on Hellinger risk measures were obtained in Seregin and Wellner ), Kim and Samworth 6), and Kim, Guntuboyina, and Samworth 6). In contrast with L p -distances or the supremum distance, there is no distribution theory available for the Hellinger loss of shape-constrained nonparametric estimators. In this paper, we present a first result in this direction, that is, a central limit theorem for the Hellinger loss of Grenander-type estimators for a monotone function λ. This type of isotonic estimator was also considered by Durot 7) and is defined as the left-hand slope of a concave or convex) estimate of the primitive of λ,basedonn observations. We will establish our results under the same general setup of Durot 7), which includes estimation of a probability density, a regression function, or a failure rate under monotonicity constraints. In fact, after approximating the squared Hellinger distance by a weighted L -distance, a central limit theorem can be obtained by mimicking the approach introduced in the work of Durot 7). An interesting feature of our main result is that, in the monotone density model, the variance of the limiting normal distribution for the Hellinger distance does not depend on the underlying density. This phenomena was also encountered for the L -distance in Groeneboom 985) and Groeneboom et al. 999). In Section, we define the setup and approximate the squared Hellinger loss by a weighted L -distance. A central limit theorem for the Hellinger distance is established in Section 3. We end this paper by a short discussion on the consequences for particular statistical models and a simulation study on testing exponentiality against a nonincreasing density. DEFINITIONS AND PREPARATORY RESULTS Consider the problem of estimating a nonincreasing or nondecreasing) function λ [, ] R + on the basis of n observations. Suppose that we have at hand a cadlag step estimator Λ n for t Λt) = λu)du, t [, ]. If λ is nonincreasing, then the Grenander-type estimator λ n for λ is defined as the left-hand slope of the least concave majorant of Λ n,with λ n ) = lim t λ n t). Ifλ is nondecreasing, then the Grenander-type estimator λ n for λ is defined as the left-hand slope of the greatest convex minorant of Λ n,with λ n ) = lim t λ n t). We aim at proving the asymptotic normality of the Hellinger distance between λ n and λ defined by H λ n,λ)= λ n t) λt)) dt). )

4 LOPUHAÄ AND MUSTA 3 We will consider the same general setup as in the work of Durot 7), that is, we will assume the following conditions: A) λ is monotone and differentiable on [, ] with < inf t λ t) sup t λ t) <. A') Let M n =Λ n Λ.ThereexistC > such that, for all x > andt =,, [ ] E sup M n u) M n t)) Cx u [,],x t u x n. ) Durot 7) also considered an additional condition A) in order to obtain bounds on pth moments; see theorem and corollary in Durot 7). However, we only need condition A') for our purposes. A3) λ n ) and λ n ) are stochastically bounded. A4) Let B n be either a Brownian bridge or a Brownian motion. There exists q >, C q >, L [, ] R, and versions of M n =Λ n Λand B n,suchthat P n q sup M nt) n B n Lt) ) > x C q x q t [,] for x, n]. Moreover, L is increasing and twice differentiable on [, ] with sup t L t) < and inf t L t) >. In Durot 7), a variety of statistical models are discussed for which the above assumptions are satisfied, such as estimation of a monotone probability density, a monotone regression function, and a monotone failure rate under right censoring. In Section 4, we briefly discuss the consequence of our main result for these models. We restrict ourselves to the case of a nonincreasing function λ. The case of nondecreasing λ can be treated similarly. Note that, even if this may not be a natural assumption, for example, in the regression setting, we need to assume that λ is positive for the Hellinger distance to be well defined. The reason that one can expect a central limit theorem for the Hellinger distance is the fact that the squared Hellinger distance can be approximated by a weighted squared L -distance. This can be seen as follows: λ n t) λt)) dt = λ n t) λt) ) λ n t)+ λt)) dt λ n t) λt) ) 4λt)) dt. 3) Because L -distances for Grenander-type estimators obey a central limit theorem e.g., Durot, 7; Kulikov & Lopuhaä, 5), similar behavior might be expected for the squared Hellinger distance. An application of the delta method will then do the rest. The next lemma makes the approximation in 3) precise. Lemma. Assume A), A'), A3), and A4). Moreover, suppose that there are C > and s > 3 4 with λ t) λ x) C t x s, for all t, x [, ]. 4) If λ is strictly positive, we have that λ n t) λt)) dt = λ n t) λt) ) 4λt)) dt + o p n 5 6 ).

5 4 LOPUHAÄ AND MUSTA In order to prove Lemma, we need the preparatory lemma below. To this end, we introduce the inverse of λ n, defined by { Û n a) =argmax Λ + n u) au }, for all a R, 5) u [,] where { } Λ + n t) =max Λ n t), lim Λ n u) u t. Note that λ n t) a Û n a) t. 6) Furthermore, let g denotetheinverseofλ. Wethenhavethefollowingresult. Lemma. Under the conditions of Lemma, it holds λ n t) λt) 3 dt = op n 5 6 ). Proof. We follow the line of reasoning in the first step of the proof of theorem in Durot 7) with p = 3. For completeness, we briefly sketch the main steps. We will first show that λ n t) λt) 3 λ) dt = 3 λ) Ûnb) gb) λ gb)) db + o P n 5 6 ). To this end, consider I = λ n t) λt) ) 3 dt, I + = λt) λ n t) ) 3 dt, + where x + = max{x, }. We approximate I by J = { λ n t) λt)+a 3 } dadt. λ) λt)) 3 From the reasoning on page,9 of Durot 7), we deduce that n 3 log n I J λ n t) λt) ) 3 dt + λ + n ) λ) 3 {n 3 Û n λ))>log n}. Because the λ n ) is stochastically bounded and λ) is bounded, together with lemma 4 in Durot 7), the second term is of the order o p n 5/6 ). Furthermore, for the first term, we can choose p [, ) such that the first term on the right-hand side is bounded by n 3 log n λ n ) λ) 3 p λ n t) λt) p dt. As in Durot 7), we get [ ] n 3 log n E λ n t) λt) p dt Kn +p ) 3 log n = o n 5 6), by choosing p 3, ). It follows that I = J + o P n 5/6 ). By a change of variable b = λt) +a /3,wefind λ) Û n b) I = 3b λt)) {gb)<ûn b)} dtdb + o p n 5 6 ). λ) gb)

6 LOPUHAÄ AND MUSTA 5 Then, by a Taylor expansion, A), and 4), there exists a K >, such that b λt)) { gb) t) λ gb)) } Kt gb)) +s, 7) for all b λ),λ)) and t gb), ].Wefind λ) Û n b) I = 3t gb)) λ gb)) λ) {gb)<ûn b)} dtdb + R n + o p n 5 6 ), 8) gb) where λ) Û n b) R n 3Kt gb)) +s {gb)<ûn b)} dtdb λ) gb) λ) 3K 3+s 3 + s λ) Ûnb) gb) db = Op n 3+s) 3 ) = o p n 5 6 ), by using 3) in Durot 7), that is, for every q < 3q ),thereexistsk q > suchthat It follows that I = λ) E [ n 3 Û n a) ga) ) q ] K q, for all a R. 9) λ) Ûn b) gb) ) 3 λ gb)) {gb)<ûn b)} db + o p n 5 6 ). In the same way, one finds λ) I = gb) Ûn b) ) 3 λ gb)) {gb)>ûn b)} db + o p n 5 6 ), λ) and it follows that λ n t) λt) 3 λ) dt = I + I = 3 λ) Ûnb) gb) λ gb)) db + o p n 5 6 ). Now, because λ is bounded, by Markov's inequality, for each ε>, we can write λ) P n 5 6 ) 3 λ) Ûnb) gb) λ gb)) db >ε λ) [ E n Û cεn 6 λ) n b) gb) 3 ] db Kn 6. For the last inequality, we again used 9) with q = 3. It follows that which finishes the proof. λ) λ) 3 Ûnb) gb) λ gb)) db = o P n 5 6 ), ) Proof of Lemma. Similar to 3), we write λ n t) λt)) dt = λ n t) λt) ) 4λt)) dt + R n, where R n = λ n t) λt) ) { λ n t)+ } λt)) 4λt)) dt.

7 6 LOPUHAÄ AND MUSTA Write 4λt) λ n t)+ λt)) = λt) λ n t) λt) λ n t) ) λt) = λt) λ n t) ) λt) + λ n t)+. λt) Because <λ) λt) λ) <, this implies that R n λ n t) λt) ) 4λt) λ n t)+ ) λt) 4λt) λ n t)+ ) dt C λ n t) λt) 3 dt λt) for some positive constant C only depending on λ) and λ). Then, from Lemma, it follows that n 5/6 R n = o P ). 3 MAIN RESULT In order to formulate the central limit theorem for the Hellinger distance, we introduce the process X, defined as { Xa) =argmax Wu) u a) }, a R, ) u R with W being a standard two-sided Brownian motion. This process was introduced and investigated by Groeneboom 985, 989) and plays a key role in the asymptotic behavior of isotonic estimators. The distribution of the random variable X) is the pointwise limiting distribution of several isotonic estimators, and the constant k = cov X), Xa) a ) da ) appears in the limit variance of the L p -error of isotonic estimators e.g., Durot,, 7; Groeneboom, 985; Groeneboom et al., 999; Kulikov & Lopuhaä, 5). We then have the following central limit theorem for the squared Hellinger loss. Theorem. Assume A), A'), A3), A4), and 4). Moreover, suppose that λ is strictly positive. Then, the following holds: { n 6 n 3 λ n t) ) } λt) dt μ N,σ ), where μ = E [ X) ] where k is defined in ). λ t)l t) 3 dt, 3 λt) σ = 3 k λ t)l t) 3 L t) dt, λt)

8 LOPUHAÄ AND MUSTA 7 Proof. According to Lemma, it is sufficient to show that n 6 n 3 I n μ ) N,σ ),with I n = λ n t) λt) ) 4λt)) dt. Again, we follow the same line of reasoning as in the proof of theorem in Durot 7). We briefly sketch the main steps of the proof. We first express I n in terms of the inverse process Û n, defined in 5). To this end, similar to the proof of Lemma, consider Ĩ = λ n t) λt) ) + 4λt)) dt, Ĩ = For the first integral, we can now write Then, if we introduce we obtain J = Ĩ = λ) λt)) 4λt) Ĩ J Û n λ)) { λ n t) λt)+ 4aλt)} dadt. λ) λt)) 4aλt) 4λ) Û n λ)) λt) λ n t) ) + 4λt)) dt. { λ n t) λt)+ 4aλt)} dadt, 3) λ n t) λt) ) + dt. { λ n t) λt)+ 4aλt)} dadt Similar to the reasoning in the proof of Lemma, we conclude that Ĩ = J + o p n 5 6 ). Next, the change of variable b = λt)+ 4aλt) yields J = λ) λ) = λ) λ) + λ) λ) Û n b) gb) Û n b) gb) Û n b) gb) b λt) λt) { Û n b)>gb)} dtdb b λt) b { Û n b)>gb)} dtdb b λt)) bλt) { Û n b)>gb)} dtdb. 4) Let us first consider the second integral on the right-hand side of 4). We then have λ) λ) Û n b) gb) λ) = λ) 6λ) b λt)) bλt) { Û n b)>gb)} dtdb λ) λ) sup x [,] sup x [,] Û n b) gb) b λt)) { Û n b)>gb)} dtdb λ) Û λ n b) x) { Û n b)>gb)} t gb)) dtdb λ) gb) λ) λ x) { Û n b)>gb)}ûn b) gb) ) 3 db = op n 5 6 ), λ)

9 8 LOPUHAÄ AND MUSTA again by using 9) with q = 3. Then, consider the first integral on the right-hand side of 4). Similar to 7), there exists K > suchthat b λt) gb) t)λ gb))) Kt gb))+s, for all b λ),λ)) and t gb), ]. Taking into account that λ gb)) <, similar to 8), it follows that where Ĩ = λ) λ) R n λ) λ) Û n b) gb) Û n b) gb) λ gb)) t gb)) b { Û n b)>gb)} dtdb + R n + o p n 5 6 ), K λ) t gb))+s {gb)<ûn b)} dtdb λ) K +s λ) + s) λ) Ûnb) gb) db = Op n +s) 3 ) = o p n 5 6 ), by using 9) once more, and the fact that s > 3 4. It follows that In the same way, Ĩ = λ) λ) Ĩ = λ) λ) λ gb))ûn b) gb) ) { Û 4b n b)>gb)} db + o p n 5 6 ). λ gb))ûn b) gb) ) { Û 4b n b)<gb)} db + o p n 5 6 ), so that λ) I n = Ĩ + Ĩ = Ûn b) gb) ) λ gb)) db + o P n 5 6 ). λ) 4b We then mimic step in the proof of theorem in Durot 7). Consider the representation B n t) =W n t) ξ n t, where W n is a standard Brownian motion, ξ n = ifb n is a Brownian motion, and ξ n is a standard normal random variable independent of B n if B n is a Brownian bridge. Then, define W t u) =n 6 { W n Lt)+n 3 u ) W n Lt)) }, for t [, ], which has the same distribution as a standard Brownian motion. Now, for t [, ], let dt) =λ t) L t) ) and define { Ṽt) =argmax Wt u) dt)u }. 5) u log n Then, similar to 6) in Durot 7), we will obtain n 3 I n = Ṽt) n 6 ξ n λ t) dt) L t) 4λt) dt + o P n 6 ). 6) To prove 6), by using the approximation Û n a) ga) LÛ n a)) Lga)) L ga))

10 LOPUHAÄ AND MUSTA 9 and a change of variable a ξ = a n / ξ n L ga)), we first obtain n 3 I n = n 3 λ) δ n λ)+δ n LÛ n a ξ )) Lga ξ )) λ ga)) L ga))) 4a da + o p n 6 ), where δ n = n 6 log n. Apart from the factor 4a, the integral on the right-hand side is the same as in the proof of theorem in Durot 7) for p =. This means that we can apply the same series of succeeding approximations for LÛ n a ξ )) Lga ξ )) as in Durot 7), which yields λ) δ n 3 I n = n 3 n Ṽga)) n 6 ξ n λ ga)) λ)+δ n dga)) L ga))) 4a da + o p n 6 ). Finally, because the integrals over [λ),λ)+δ n ] and [λ) δ n,λ)] are of the order o p n /6 ), this yields 6) by a change of variables t = ga). Thenextstepistoshowthatthetermwithξ n can be removed from 6). This can be done exactly as in Durot 7), because the only difference with the corresponding integral in Durot 7) is the factor 4λt), which is bounded and does not influence the argument in Durot 7). We find that n 3 I n = Ṽt) λ t) L t) 4λt) dt + o P n 6 ). Then, define Y n t) = Ṽt) E [ Ṽt) ]) λ t) L t) 4λt). 7) By approximating Ṽt) by { Vt) =argmax Wt u) dt)u }, u R and using that, by Brownian scaling, dt) /3 Vt) has the same distribution as X) see Durot, 7, for details), we have that E [ Ṽt) ] λ t) L t) 4λt) dt = E [ X) ] dt) 4 3 λ t) L t) = μ + o n 6). 4λt) dt + o n 6) It follows that We then first show that n 6 I n μ )=n 6 Var n 6 Y n t)dt + o P ). ) Y n t)dt σ. 8) Once more, following the proof in Durot 7), we have ) v n = Var Y n t)dt = s λ t) λ s) L t) L s) 4λt) 4λs) cov Ṽt), Ṽs) ) dtds.

11 LOPUHAÄ AND MUSTA After the same sort of approximations as in Durot 7), we get min,s+c n ) v n = λ s) 4 s L s) 4λs)) cov V t s), V s s) ) dtds + o n 3), where c n = n 3 log n inf t L t) and where, for all s and t, { V t s) =argmax Wt u) ds)u }. u R Then, use that ds) /3 V t s) has the same distribution as X n 3 ds) 3 Lt) Ls)) ) n 3 ds)lt) Ls)) so that the change of variable a = n /3 ds) /3 Lt) Ls)) in v n leads to n 3 v n λ s) 4 L s) 4λs)) ds) 3 L s) cov Xa), X) ) dads k λ s) 4 3 L s) 7 3 L s) ds = σ, 4λs)) λ s) 3 which proves 8). Finally, asymptotic normality of n 6 Y nt)dt follows by Bernstein's method of big blocks and small blocks in the same way as in step 6 of the proof of theorem in Durot 7). Corollary. Assume A), A' ), A3), A4), and 4) and let H λ n,λ) be the Hellinger distance defined in ). Moreover, suppose that λ is strictly positive. Then, n 6 { n 3 H λ n,λ) μ } N, σ ). μ = μ and σ = σ 8μ, where μ and σ are defined in Theorem. Proof. This follows immediately by applying the delta method with φx) = x to the result in Theorem. 4 EXAMPLES The type of scaling for the Hellinger distance in Corollary is similar to that in the central limit theorem for L p -distances. This could be expected in view of the approximation in terms of a weighted squared L -distance see Lemma ), and the results, for example, in Kulikov and Lopuhaä 5) and Durot 7). Actually, this is not always the case. The phenomenon of observing different speeds of convergence for the Hellinger distance from those for the L and L norms was considered by Birgé 986). In fact, this is related to the existence of a lower bound for the function we are estimating. If the function of interest is bounded from below, which is the case considered in this paper, then the approximation 3) holds; see Birgé 986) for an explanation. When we insert the expressions for μ and σ from Theorem, then we get σ k = 4E [ λ t)l t) 3 L t)λt) dt X) ], λ t)l t) 3 λt) dt

12 LOPUHAÄ AND MUSTA where k is defined in ). This means that, in statistical models where L =Λin condition A4) and, hence, L = λ, the limiting variance σ = k 4E[X) ]) does not depend on λ. One such a model is estimation of the common monotone density λ on [, ] of independent random variables X,, X n.then,λ n is the empirical distribution function of X,, X n,and λ n is Grenander's estimator Grenander, 956). In that case, if inf t λt) >, the conditions of Corollary are satisfied with L = Λ see theorem 6 in Durot, 7), so that the limiting variance of the Hellinger loss for the Grenander estimator does not depend on the underlying density. This behavior was conjectured in Wellner 5) and coincides with that of the limiting variance in the central limit theorem for the L -error for the Grenander estimator, first discovered by Groeneboom 985); see also Durot, 7), Groeneboom et al. 999), and Kulikov and Lopuhaä 5). Another example is when we observe independent identically distributed inhomogeneous Poisson processes N,, N n with common mean function Λ on [, ] with derivative λ,forwhich Λ) <.Then,Λ n is the restriction of N + + N n ) n to [, ]. Also in that case, the conditions of Corollary are satisfied with L =Λsee theorem 4 in Durot, 7), so that the limiting variance of the Hellinger loss for λ n does not depend on the common underlying intensity λ. However, note that, for this model, the L -loss for λ n is asymptotically normal according to theorem in Durot 7) but with limiting variance depending on the value Λ) Λ). Consider the monotone regression model y i,n = λi n)+ε i,n,fori =,, n,wheretheε i,n 's are i.i.d. random variables with mean zero and variance σ >. Let Λ n t) = y i,n, t [, ] n i nt be the empirical distribution function. Then, λ n is a slight modification of) Brunk's 958) estimator. Under appropriate moment conditions on the ε i,n, the conditions of Corollary are satisfied with Lt) =tσ see theorem 5 in Durot, 7). In this case, the limiting variance of the Hellinger loss for λ n depends on both λ and σ,whereasthel -loss for λ n is asymptotically normal according to theorem in Durot 7) but with limiting variance only depending on σ. Suppose we observe a right-censored sample X, Δ ),, X n, Δ n ),wherex i = mint i, Y i ) and Δ i = {Ti Y i },withthet i 's being nonnegative i.i.d. failure times and the Y i 's are i.i.d. censoring times independent of the T i 's. Let F be the distribution function of the T i 's with density f and let G be the distribution function of the Y i 's. The parameter of interest is the monotone failure rate λ = f F) on [, ].Inthiscase,Λ n is the restriction of the Nelson Aalen estimator to [, ]. If we assume A) and inf t λt) >, then, under suitable assumptions on F and G, the conditions of Corollary hold with Lt) = t λu) du, t [, ]; Fu)) Gu)) see theorem 3 in Durot 7). This means that the limiting variance of the Hellinger loss depends on λ, F, andg, whereas the limiting variance of the L -loss depends only on their values at and. In particular, in the case of nonrandom censoring times, L = F ), the limiting variance of the Hellinger loss depends on λ and F, whereas the limiting variance of the L -loss depends only on the value F).

13 LOPUHAÄ AND MUSTA 5 TESTING EXPONENTIALITY AGAINST A NONDECREASING DENSITY In this section, we investigate a possible application of Theorem, that is, testing for an exponential density against a nonincreasing alternative by means of the Hellinger loss. The exponential distribution is one of the most used and well-known distributions. It plays a very important role in reliability, survival analysis, and renewal process theory, when modeling random times until some event. As a result, a lot of attention has been given in the literature to testing for exponentiality against a wide variety of alternatives, by making use of different properties and characterizations of the exponential distribution Alizadeh Noughabi & Arghami, ; Haywood & Khmaladze, 8; Jammalamadaka & Taufer, 3; Meintanis, 7). In this section, we consider a test for exponentiality, assuming that data come from a decreasing density. The test is based on the Hellinger distance between the parametric estimator of the exponential density and the Grenander-type estimator of a general decreasing density. In order to be able to apply the result of Corollary, we first investigate a test whether the data is exponentially distributed with a fixed parameter λ >. Because such a test may not be very interesting from a practical point of view, we also investigate testing exponentiality, leaving the parameter λ> unspecified. 5. Testing a simple null hypothesis of exponentiality Let f λ x) =λe λx {x } be the exponential density with parameter λ >. Assume we have a sample of i.i.d. observations X,, X n from some distribution with density f and for λ > fixed, we want to test H f = f λ against H f is nonincreasing. Under the alternative hypothesis, we can estimate f on an interval [,τ] by the Grenander-type estimator f n from Section. Then, as a test statistic, we take T n = H f n,f λ ), the Hellinger distance on [,τ] between f n and f λ, and at level α, we reject the null hypothesis if T n > c n,α,λ,for some critical value c n,α,λ >. According to Corollary, it follows that T n is asymptotically normally distributed, but the mean and the variance depend on the constant k defined in ). To avoid computation of k,we estimate the mean and the variance of T n empirically. We generate B =, samples from f λ. For each of these samples, we compute the Grenander estimator f n,i and the Hellinger distance T n,i = H f n,i,f λ ),fori =,,, B. Finally, we compute the mean T and the variance s T of the values T n,,, T n,b. For the critical value of the test, we take c n,α,λ = T+q α s T,whereq α is the α)% quantile of the standard normal distribution. Note that, even if in the density model the asymptotic variance is independent of the underlying distribution, the asymptotic mean does depend on λ, that is, the test is not distribution free. Another possibility, instead of the normal approximation, is to take as a critical value c n,α,λ the empirical α)% quantile of the values T n,,, T n,b. To investigate the performance of the test, we generate N =, samples from f λ.for each sample, we compute the value of the test statistic T n = H f n,f λ ) and we reject the null hypothesis if T n > c n,α,λ or if T n > c n,α,λ ). The percentage of rejections gives an approximation of the level of the test. Table shows the results of the simulations for different sample sizes n and two values of λ and α =.,.5,.. Here, we take τ = 5 because the mass of the exponential distribution with parameter one or five outside the interval [, 5] is negligible. We observe that the percentage of rejections is close to the nominal level if we use c n,α,λ as a critical value for the

14 LOPUHAÄ AND MUSTA 3 TABLE Simulated levels of T n using top) c n,α,λ and bottom) c n,α,λ,with α =.,.5,., under the null hypothesis varying the sample size n and the parameter λ λ = λ = 5 α =. α =.5 α =. α =. α =.5 α =. n = n = n = FIGURE Simulated powers using solid) c n,α,λ and dashed) c n,α,λ,withα =.5, of T n and the power of dotted) the likelihood ratio test for λ =, ν =.4,.45,,, and n = test, but it is a bit higher if we use c n,α,λ. This is due to the fact that, for small sample sizes, the normal approximation of Corollary is not very precise. Moreover, to investigate the power, we generate a sample from the Weibull distribution with shape parameter ν and scale parameter λ. Recall that Weibull,λ ) corresponds to the exponential distribution with parameter λ and that a Weibull distribution with ν<has a decreasing density. We compute the Hellinger distance T n = H f n,f λ ) and we reject the null hypothesis if T n > c n,α,λ or if T n > c n,α,λ ). After repeating the procedure N =, times, we compute the percentage of times that we reject the null hypothesis, which gives an approximation of the power of the test. The results of the simulations, done with n =, λ =, α =.5, and alternatives for which ν varies between.4 and by steps of.5, are shown in Figure. As a benchmark, we compute the power of the likelihood ratio LR) test statistic for each ν. As expected, our test is less powerful with respect to the LR test, which is designed to test against a particular alternative. However, as the sample size increases, the performance improves significantly and the difference of the results when using c n,α,λ or c n,α,λ becomes smaller.

15 4 LOPUHAÄ AND MUSTA TABLE Simulated levels of R n under the null hypothesis varying the sample size n and the parameter λ λ n = n = n = n =, Testing a composite null hypothesis of exponentiality Assume we have a sample of i.i.d. observations X,, X n from some distribution with density f and we want to test H f = f λ, for some λ> against H f is nonincreasing. Under the null hypothesis, we can construct a parametric estimator of the density that is given by f λ n,where λ n = n n i= X i is the maximum likelihood estimator of λ. On the other hand, under the alternative hypothesis, we can estimate f on aninterval [,τ] by the Grenander-type estimator f n from Section. Then, as a test statistic, we take R n = H f n,f λ n ), the Hellinger distance on [,τ] between the two estimators, and at level α, we reject the null hypothesis if R n > d n,α for some critical value d n,α >. Because the limit distribution of the test statistic is not known, we use a bootstrap procedure to calibrate the test. We generate B =, bootstrap samples of size n from f λ n, and for each of them, we compute the estimators f λ f n,i, and the test statistic n,i R = Hf n,i λ f n,i, n,i ),fori =,,, B. Then, we determine the αth upper percentile d n,α of the values R n,,, R n,b. Finally, we reject the null hypothesis if R n > d n,α. To investigate the level of the test, for α =.5 and λ>fixed, we start with a sample from an exponential distribution with parameter λ and repeat the above procedure N =, times. We count the number of times we reject the null hypothesis, that is, the number of times the value of the test statistics exceeds the corresponding 5th upper percentile. Dividing this number by N gives an approximation of the level. Table shows the results of the simulations for different sample sizes n and different values of λ. The rejection probabilities are close to.5 for all the values of λ, which shows that the test performs well in the different scenarios slightly and strongly decreasing densities). To investigate the power, for α =.5 and fixed < ν < andλ >, we now start with a sample from a Weibull distribution with shape parameter ν and scale parameter λ and compute the value R n = Hf λ n, f n ). In order to calibrate the test, we treat this sample as if it were an exponential sample and estimate λ by λ n = n n i= X i. Next, we generate B =, bootstrap samples of size n from the exponential density with parameter λ n. For each bootstrap sample, we compute the test statistic R = Hf n,i λ f n,i, ),fori =,,, B, and we determine n,i the 5th upper percentile d n,.5 of the values R n,,, R. Finally, we reject the null hypothesis n,b if R n > d n,.5. After repeating the above procedure N =, times, each time starting with a Weibull sample, we compute the percentage of times that we reject the null hypothesis, which gives an approximation of the power of the test. We compare the Hellinger distance test to some of the tests from Alizadeh Noughabi and Arghami ), which are designed to test exponentiality against all the possible alternatives, that is, not only against decreasing densities. These tests are all distribution free, which means that their critical values can be computed independently of λ. Then, for each of the Weibull samples generated before, we count the percentage of times that the tests

16 LOPUHAÄ AND MUSTA a) b) FIGURE Simulated powers of black solid) the Hellinger distance test and some other competitor tests, that is, blue) T, green) T, yellow) ω n, brown) S n,red)ep n,purple)kl mn, and orange) CO n, and the power of black dotted) the likelihood ratio test for left) n =, λ =,.4 ν and right) β 8. a) Weibull. b) Beta T, T,ω n, S n, EP n, KL mn, and CO n see Alizadeh Noughabi & Arghami,, for a precise definition) reject the null hypothesis. Finally, we also compare the power of our test with the LR test for each ν. The results of the simulations, done with n =, λ =, and alternatives for which ν varies between.4 and, are shown in the left panel in Figure. Actually, we also investigated the power for different choices of λ, and we observed similar behavior as for λ =. The figure shows that the test based on the Hellinger distance performs worse than the other tests. In this case, the test of Cox and Oakes CO n has greater power. However, Alizadeh Noughabi and Arghami ) concluded that none of the tests is uniformly most powerful with respect to the others. We repeated the experiment taking, instead of the Weibull distribution, the beta distribution with parameters α = and β 8 as alternative. Note that it has a nonincreasing density on [, ] proportional to x) β, and the extreme case β = corresponds to the uniform distribution. Results are shown in the right panel in Figure. We observe that, for small values of β, the Hellinger distance test again behaves worse than the others, and in this case, R n and EP n have greater power. However, for larger β, the Hellinger distance test outperforms all the others. ORCID Eni Musta REFERENCES Alizadeh Noughabi, H., & Arghami, N. R. ). Testing exponentiality based on characterizations of the exponential distribution. Journal of Statistical Computation and Simulation, 8), Birgé, L. 986). On estimating a density using Hellinger distance and some other strange facts. Probability Theory and Related Fields, 7), 7 9.

17 6 LOPUHAÄ AND MUSTA Birgé, L., & Massart, P. 993). Rates of convergence for minimum contrast estimators. Probability Theory and Related Fields, 97 ), 3 5. Brunk, H. D. 958). On the estimation of parameters restricted by inequalities. Annals of Mathematical Statistics, 9, Doss, C. R., & Wellner, J. A. 6). Global rates of convergence of the MLEs of logconcave and s-concave densities. The Annals of Statistics, 443), Durot, C. ). Sharp asymptotics for isotonic regression. Probability Theory and Related Fields, ), 4. Durot, C. 7). On the L p -error of monotonicity constrained estimators. The Annals of Statistics, 353), 8 4. Durot, C., Kulikov, V. N., & Lopuhaä, H. P. ). The limit distribution of the L -error of Grenander-type estimators. The Annals of Statistics, 43), van de Geer, S. A. ). Applications of empirical process theory. Vol. 6. Cambridge Series in Statistical and Probabilistic Mathematics pp. xii 86). Cambridge, UK: Cambridge University Press. Grenander, U. 956). On the theory of mortality measurement II. Skand Aktuarietidskr, 39, Groeneboom, P. 985). Estimating a monotone density. In Berkeley Conference in Honor of Jerzy Neyman and Jack Kiefer, Vol. II. The Wadsworth & Brooks/Cole Statistics/Probability Series pp ). Groeneboom, P. 989). Brownian motion with a parabolic drift and airy functions. Probability Theory Related Fields, 8), Groeneboom, P., Hooghiemstra, G., & Lopuhaä, H. P. 999). Asymptotic normality of the L error of the Grenander estimator. The Annals of Statistics, 74), Groeneboom, P., & Wellner, J. A. 99). Information bounds and nonparametric maximum likelihood estimation. Vol. 9). Basel, Switzerland: Birkhäuser Verlag. DMV Seminar. Haywood, J., & Khmaladze, E. 8). On distribution-free goodness-of-fit testing of exponentiality. Journal of Economics, 43), 5 8. Huang, J., & Wellner, J. A. 995). Estimation of a monotone density or monotone hazard under random censoring. Scandinavian Journal of Statistics Theory and Applications, ), Huang, Y., & Zhang, C. H. 994). Estimating a monotone density from censored observations. The Annals of Statistics, 3), Jammalamadaka, S. R., & Taufer, E. 3). Testing exponentiality by comparing the empirical distribution function of the normalized spacings with that of the original data. Journal of Nonparametric Statistics, 56), Kim, A. K. H., Guntuboyina, A., & Samworth, R. J. 6). Adaptation in log-concave density estimation. arxiv: Kim, A. K. H., & Samworth, R. J. 6). Global rates of convergence in log-concave density estimation. The Annals of Statistics, 446), Kulikov, V. N., & Lopuhaä, H. P. 5). Asymptotic normality of the L k -error of the Grenander estimator. The Annals of Statistics, 335), LeCam, L. 97). On the weak convergence of probability measures. Annals of Mathematical Statistics, 4, LeCam, L. 973). Convergence of estimates under dimensionality restrictions. The Annals of Statistics,, Lopuhaä, H. P., & Nane, G. F. 3). Shape constrained non-parametric estimators of the baseline distribution in Cox proportional hazards model. Scandinavian Journal of Statistics Theory and Applications, 43), Marshall, A. W., & Proschan, F. 965). Maximum likelihood estimation for distributions with monotone failure rate. Annals of Mathematical Statistics, 36, Meintanis, S. G. 7). Test for exponentiality against Weibull and gamma decreasing hazard rate alternatives. Kybernetika Prague), 433), Pal, J. K., Woodroofe, M., & Meyer, M. 7). Estimating a Polya frequency function. Complex datasets and inverse problems. IMS Lecture Notes Monograph Series, 54, Prakasa Rao, B. L. S. 969). Estimation of a unimodal density. Sankhyā Statistics) The Indian Journal of Statistics Series A, 3, Prakasa Rao, B. L. S. 97). Estimation for distributions with monotone failure rate. Annals of Mathematical Statistics, 4, Seregin, A., & Wellner, J. A. ). Nonparametric estimation of multivariate convextransformed densities. The Annals of Statistics, 386), With supplementary material available online.

18 LOPUHAÄ AND MUSTA 7 van der Vaart, A. W., & Wellner, J. A. 996). Weak convergence and empirical processes: With applications to statistics. Springer Series in Statistics, pp. xvi 58. New York, NY: Springer-Verlag. Wellner, J. A. 5). Musings about shape constrained estimation and inference: Some problems. Paper presented at the workshop of Shape Constrained Inference: Open Problems and New Directions, Lorentz Centre, Leiden, The Netherlands. Wong, W. H., & Shen, X. 995). Probability inequalities for likelihood ratios and convergence rates of sieve MLEs. The Annals of Statistics, 3), How to cite this article: Lopuhaä HP, Musta E. A central limit theorem for the Hellinger loss of Grenander-type estimators. Statistica Neerlandica. 8; 7.