arxiv: v2 [math.st] 20 Nov 2016

Transcription

1 A Lynden-Bell integral estimator for the tail inde of right-truncated data with a random threshold Nawel Haouas Abdelhakim Necir Djamel Meraghni Brahim Brahimi Laboratory of Applied Mathematics Mohamed Khider University Biskra Algeria Abstract arxiv:6.0547v2 [math.st] 20 Nov 206 By means of a Lynden-Bell integral with deterministic threshold Worms and Worms [A Lynden-Bell integral estimator for etremes of randomly truncated data. Statist. Probab. Lett. 206; 09: 06-7] recently introduced an asymptotically normal estimator of the tail inde for randomly right-truncated Pareto-type data. In this contet we consider the random threshold case to derive a Hill-type estimator and establish its consistency and asymptotic normality. A simulation study is carried out to evaluate the finite sample behavior of the proposed estimator. Keywords: Etreme value inde; Heavy-tails; Lynden-Bell estimator; Random truncation. AMS 200 Subject Classification: 60F7 62G30 62G32 62P05.. Introduction Let X i Y i ) i N be a sample of size N from a couple XY) of independent random variables rv s) defined over some probability space Ω A P) with continuous marginal distribution functions df s) F and G respectively. Suppose that X is truncated to the right by Y in the sense that X i is only observed when X i Y i. We assume that both survival functions F := F and G := G are regularly varying at infinity with respective negative indices /γ and /γ 2. That is for any > 0 Fz) lim z Fz) = /γ Gz) and lim z Gz) = /γ 2..) It is well known that in etreme value analysis weak approimations are achieved in the second-order framework see e.g. de Haan and Ferreira 2006 page 48). Thus it seems Corresponding author: necirabdelhakim@yahoo.fr addresses: nawel.haouas@yahoo.com N. Haouas) djmeraghni@yahoo.com D. Meraghni) brah.brahim@gmail.com B. Brahimi)

2 2 quite natural to suppose that F and G satisfy the second-order condition of regular variation which we epress in terms of the tail quantile functions pertaining to both df s. That is we assume that for > 0 we have U F t)/u F t) γ lim t A F t) = γ τ τ.2) and U G t)/u G t) γ 2 lim = γ τ 2 2.3) t A G t) τ 2 where τ τ 2 < 0 are the second-order parameters and A F A G are functions tending to zero and not changing signs near infinity with regularly varying absolute values at infinity with indices τ τ 2 respectively. For any df K the function U K t) := K /t) t > stands for the tail quantile function with K u) := inf{v : Kv) u} 0 < u < denoting the generalized inverse of K. FromLemma 3 in Hua and Joe20) thesecond-order conditions.2) and.3) imply that there eist constants d d 2 > 0 such that F) = d y /γ l ) and G) = d 2 y /γ 2 l 2 ) > 0.4) where lim l i ) = and l i is regularly varying at infinity with tail inde τ i γ i = 2. This ccondition is fullfilled by many commonly used models such as Burr Fréchet Generalized Pareto absolute Student log-gamma distributions to name but a few. Also known as heavy-tailed Pareto-type or Pareto-like distributions these models take a prominent role in etreme value theory and have important practical applications as they are used rather systematically in certain branches of non-life insurance as well as in finance telecommunications hydrology etc... see e.g. Resnick 2006). Let us now denote X i Y i ) i =...n to be the observed data as copies of a couple of rv s XY) corresponding to the truncated sample X i Y i ) i =...N where n = n N is a sequence of discrete rv s which in virtue of the weak law of large numbers satisfies n N /N P p := PX Y) as N. We denote the joint df of X and Y by 0 Hy) := PX Y y) = PX miny)y y X Y) which is equal to y p Fminz))dGz). The marginal distributions of the rv s X and Y respectively de- noted by F and G are given by F ) = p Gz)dFz) and Gy) = p y Fz)dGz). 0 0 Since F and G are heavy-tailed then their right endpoints are infinite and thus they are equal. Hence from Woodroofe 985) we may write dfy)/fy) = df y)/cy) where Cz) := PX z Y). Differentiating the previous equation leads to the following crucial equation C) df) = F) df ) whose solution is defined by F) =

3 ep { df z)/cz)}. This leads to Woodroofe s nonparametric estimator Woodroofe 985) of df F given by F W) n ) := i:x i > { ep nc n X i ) which is derived only by replacing df s F and C by their respective empirical counterparts F n ) := n n X i ) and C n ) := n n X i Y i ). There eists a more i= popular estimator for F known as Lynden-Bell nonparametric maimum likelihood estimator Lynden-Bell 97) defined by i= } F n LB) ) := ) nc i:x i > n X i ) which will be considered in this paper to derive a new estimator for the tail inde of df F. Note that the tail of df F simultaneously depends on G and F while that of G only relies on G. By using Proposition B..0 in de Haan and Ferreira 2006) to the regularly varying functions F and G we show that both F and G are regularly varying at infinity as well with respective indices /γ := γ +γ 2 )/γ γ 2 ) and /γ 2. In view of the definition of γ Gardes and Stupfler 205) derived a consistent estimator for the etreme value inde γ whose asymptotic normality is established in Benchaira et al. 205) under the tail dependence and the second-order conditions of regular variation. Recently by considering a Lynden-Bell integration with a deterministic threshold t n > 0 Worms and Worms 206) proposed another asymptotically normal estimator for γ as follows: γ LB) t n ) := nf LB) n t n ) n i= X i > t n ) FLB) n X i ) log X i. C n X i ) t n Likewise Benchaira et al.206a) considered a Woodroofe integrationwith a random threshold) to propose a new estimator for the tail inde γ given by γ W) := nf W) n X n k:n ) k i= F W) n X n i+:n ) C n X n i+:n ) log X n i+:n X n k:n where given n = m = m N Z :m... Z m:m denote the order statistics pertaining to a sample Z...Z m and k = k n is a random) sequence of integers such that given n = m < k m < m k m and k m /m 0 as N. The consistency and asymptotic normality of γ W) are established in Benchaira et al. 206a) through a weak approimation 3

4 4 to Woodroofe s tail process D n W) ) := k ) F W) n X n k:n ) /γ > 0. F W) n X n k:n ) More precisely the authors showed that under.2) and.3) with γ < γ 2 there eist a function A 0 t) A F t) := A F /FUF t)) ) t and a standard Wiener process {Ws); s 0}definedontheprobabilityspaceΩAP)suchthatfor0 < ǫ < /2 γ/γ 2 and 0 > 0 sup 0 /2 ǫ)/γ /γ 2 DW) n ) Γ;W) /γ τ /γ γ τ ka0 n/k) = o P).5) as N provided that given n = m k m A 0 m/k m ) = O) where {Γ;W); > 0} is a Gaussian process defined by Γ;W) := + γ { /γ /γ W /γ) W) } γ γ γ +γ 2 /γ 0 s γ/γ 2 { /γ W /γ s ) Ws) } ds. In view of the previous weak approimation the authors also proved that if given n = m km A F m/k m) λ then ) k γ γ ) D λ N σ 2 as N where σ 2 := τ γ 2 +γ /γ 2 ) +γ /γ 2 ) 2) / γ /γ 2 ) 3. Recently Benchaira et al. 206b) followed this approachtointroduce akernel estimator toγ which improves thebiasof γ W).Inthispaper we are interested in Worm s estimator γ LB) t n ) but with a threshold t n that is assumed to be random and equal to X n k:n. This makes the estimator more convenient for numerical implementation than the one with a deterministic threshold. In other words we will deal with the following tail inde estimator: Note that F LB) n γ LB) := we have C n )df LB) n 2009) then nf LB) n X n k:n ) k i= F n LB) X n i+:n ) log X n i+:n. C n X n i+:n ) X n k:n ) = and write F LB) n X n k:n ) = X n k:n df n LB) y). On the other hand ) = F n LB) )df n ) see e.g. Strzalkowska-Kominiak and Stute F LB) n X n k:n ) = X n k:n F LB) n ) C n ) df n) = n This allows us to rewrite the new estimator into k γ LB) := a i) n log X n i+:n X n k:n i= k i= F LB) n X n i+:n ) C n X n i+:n ).

5 where a i) n FLB) n X n i+:n ) := C n X n i+:n ) / k i= F LB) n X n i+:n ) C n X n i+:n ). It is worth mentioning that for complete data we have n N and F n F n C n it follows that a i) n k i =...k and consequently both γ LB) and γ W) reduce to the classical Hill estimator Hill 975). The consistency and asymptotic normality of γ LB) will be achieved through a weak approimation of the corresponding tail Lynden-Bell process that we define by D n LB) ) := ) F LB) n X n k:n ) k /γ > 0. F LB) n X n k:n ) The rest of the paper is organized as follows. In Section 2 we provide our main results whose proofsarepostponedtosection4. Thefinitesamplebehavioroftheproposedestimator γ LB) is checked by simulation in Section 3 where a comparison with the one recently introduced by Benchaira et al. 206a) is made as well. 2. Main results We basically have three main results. The first one that we give in Theorem 2. consists in an asymptotic relation between the above mentioned estimators of the distribution tail namely F W) n and F LB) n. This in turn is instrumental to the Gaussian approimation of the tail Lynden-Bell process D n LB) ) stated in Theorem 2.2. Finally in Theorem 2.3 we deduce the asymptotic behavior of the tail inde estimator γ LB). Theorem 2.. Assume that both F and G satisfy the second-order conditions.2) and.3) respectively with γ < γ 2. Let k = k n be a random sequence of integers such that given n = m k m and k m /m 0 as N then for any 0 > 0 we have F W) sup /γ n X n k:n ) F LB) n X n k:n ) = O P k/n) ) γ /γ. 0 FX n k:n ) Theorem 2.2. Assume that the assumptions of Theorem 2. hold and given n = m k +γ /2γ) m /m 0 2.6) and k m A 0 m/k m ) = O) as N. Then for any 0 > 0 and 0 < ǫ < /2 γ/γ 2 we have sup 0 /2 ǫ)/γ /γ 2 DLB) n ) Γ;W) /γ τ /γ γ τ ka0 n/k) = o P). 5

6 6 Theorem 2.3. Assume that.) holds with γ < γ 2 and let k = k n be a random sequence of integers such that given n = m k m and k m /m 0 as N then γ LB) Assume further that the assumptions of Theorem 2.2 hold then ) k γ LB) ka0 n/k) γ = γw) τ γ + γ +γ 2 0 γ 2 γ γlogs)s γ/γ 2 Ws)ds+o P ). If in addition we suppose that given n = m k m A F m/k m) λ < then ) ) k γ LB) D λ γ N σ 2 as N. τ 3. Simulation study P γ. In this section we illustrate the finite sample behavior of γ LB) and at the same time we compare it with γ W). To this end we consider two sets of truncated and truncation data both drawn from Burr s model: F) = + /δ) δ/γ G) = + /δ ) δ/γ 2 0 where δγ γ 2 > 0. The corresponding percentage of observed data is equal to p = γ 2 /γ +γ 2 ). We fi δ = /4 and choose the values 0.6 and 0.8 for γ and 55% 70% and 90%forp.Foreachcoupleγ p)wesolve theequationp = γ 2 /γ +γ 2 )togetthepertaining γ 2 -value. We vary the common size N of both samples X...X N ) and Y...Y N ) then for each size we generate 000 independent replicates. Our overall results are taken as the empirical means of the results obtained through all repetitions. To determine the optimal number of top statistics used in the computation of the tail inde estimate values we use the algorithm of Reiss and Thomas 2007) page 37. Our illustration and comparison are made with respect to the estimators absolute biases abs bias) and the roots of their mean squared errors rmse). We summarize the simulation results in Tables and 3.3 for γ = 0.6 and in Tables and 3.6 for γ = 0.8. After the inspection of all the tables two conclusions can be drawn regardless of the situation. First the estimation accuracy of both estimators decreases when the truncation percentage increases and this was quite epected. Second we notice that the newly proposed estimator γ LB) and γ W) behave equally well. 4. Proofs 4.. Proof Theorem 2.. For 0 we have { F n W) X n k:n ) = ep X n k:n } df n y). C n y)

7 7 γ = 0.6; p = 0.55 γ LB) γ W) N n abs bias rmse k abs bias rmse k Table 3.. Estimation results of Lynden-Bell based leftt pannel) and Woodroofe based right pannel) estimators of the shape parameter γ = 0.6 of Burr s model through 000 right-truncated samples with 45%-truncation rate. γ = 0.6; p = 0.7 γ LB) γ W) N n abs bias rmse k abs bias rmse k Table 3.2. Estimation results of Lynden-Bell based left pannel) and Woodroofe based right pannel) estimators of the shape parameter γ = 0.6 of Burr s model through 000 right-truncated samples with 30%-truncation rate.

8 8 γ = 0.6; p = 0.9 γ LB) γ W) N n abs bias rmse k abs bias rmse k Table 3.3. Estimation results of Lynden-Bell based left pannel) and Woodroofe based right pannel) estimators of the shape parameter γ = 0.6 of Burr s model through 000 right-truncated samples with 0%-truncation rate. γ = 0.8; p = 0.55 γ LB) γ W) N n abs bias rmse k abs bias rmse k Table 3.4. Estimation results of Lynden-Bell based left pannel) and Woodroofe based right pannel) estimators of the shape parameter γ = 0.8 of Burr s model through 000 right-truncated samples with 45%-truncation rate.

9 9 γ = 0.8; p = 0.7 γ LB) γ W) N n abs bias rmse k abs bias rmse k Table 3.5. Estimation results of Lynden-Bell based left pannel) and Woodroofe based right pannel) estimators of the shape parameter γ = 0.8 of Burr s model through 000 right-truncated samples with 30%-truncation rate. γ = 0.8; p = 0.9 γ LB) γ W) N n abs bias rmse k abs bias rmse k Table 3.6. Estimation results of Lynden-Bell based left pannel) and Woodroofe based right pannel) estimators of the shape parameter γ = 0.8 of Burr s model through 000 right-truncated samples with 0%-truncation rate.

10 0 We show that the latter eponent is negligible in probability uniformly over 0. Indeed note that both F n y)/f y) and Cy)/C n y) are stochastically bounded from above on y < X n:n see e.g. Shorack and Wellner 986 page 45 and Strzalkowska-Kominiak and Stute 2009 respectively) it follows that X n k:n By a change of variables we have X n k:n df y) Cy) = F X n k:n) CX n k:n ) df n y) C n y) = O df y) P) X n k:n Cy). 4.7) CX n k:n ) CX n k:n t) df X ) n k:nt). 4.8) F X n k:n ) Recall that X n k:n P and that F is regularly varying at infinity with inde /γ. On the other hand fromassertion i) of Lemma A.2 in Benchaira et al. 206a) we deduce that /C is also regularly varying at infinity with inde /γ 2. Thus we may apply Potters inequalities see e.g. Proposition B..0 in de Haan and Ferreira 2006) to both F and /C to write: for all large N any t 0 and any sufficiently small δν > 0 with large probability F X n k:n t) F X n k:n ) t /γ < δt /γ±ν and CX n k:n ) CX n k:n t) t/γ 2 < δt/γ 2±ν 4.9) where t ±a := mat a t a ). These two inequalities may be rewritten into F X n k:n t) F X n k:n ) = t /γ +o P t ±ν )) and CX n k:n) CX n k:n t) = t/γ 2 +o P t ±ν )) uniformly on t 0. This leads to CX n k:n ) CX n k:n t) df X n k:nt) F X n k:n ) = γ γ /γ +o P ±ν )). 4.0) Inviewof.4)Benchaira et al.206a)showed inlemmaathatf y) = +o))c y /γ and Gy) = +o))c 2 y /γ 2 as y for some constants c c 2 > 0. In other words U F s) = +o))c s) γ ass andcy) = +o))c 2 y /γ 2 asy.ontheother hand from Lemma A4 in Benchaira et al. 206a) we have X n k:n = +o P ))U F n/k) it follows that X n k:n = +o P ))c γ k/n) γ. Note that γ/γ 2 = γ/γ hence F X n k:n ) CX n k:n ) = +o P))c γ/γ 2 c 2 k/n) γ/γ. 4.) Plugging results 4.0) and 4.) in equation 4.8) yields X n k:n df y) Cy) = k/n)γ/γ c γ/γ 2 c 2 γ /γ +o P ±ν )). 4.2)

11 By combining equations 4.7) and 4.2) we obtain X n k:n df n y) C n y) = O P)k/n) γ/γ /γ +o P ±ν )) 4.3) which obviously tends to zero in probability uniformly on 0 ). We may now apply Taylor s epansion e t = +t+ot 2 ) as t 0 to get { } df n y) df n y) ep = C n y) C n y) +O P X n k:n In other words we have F W) n X n k:n ) = X n k:n X n k:n X n k:n ) 2 df n y) N. C n y) df n y) C n y) +R n) N 4.4) ) where R n ) := O P k/n) 2γ/γ 2/γ +o P ±ν )). Net we show that F LB) df n y) n X n k:n ) = X n k:n C n y) +R n2) N. 4.5) Observe that by taking the logarithme then its eponential in the definition of F LB) n we have F n LB) X n k:n ) = ep { n i= ) } X i:n > X n k:n )log nc n X i:n ) { which may be rewritten into ep n ) } log df n X n k:n y). To get nc n X n k:n y) approimation 4.5) it suffices to apply successively in the previous quantity Taylor s epansions e t = +t+ot 2 ) and log t) = t+ot 2 ) as t 0) with similar arguments as above we omit further details). Combining 4.4) and 4.5) and setting R n ) := R n ) R n2 ) yield ) F W) n X n k:n ) F LB) n X n k:n ) = R n ) N. 4.6) On the other hand by once again using Taylor s epansion we write FX n k:n ) = X n k:n df y) Cy) + R n ) N. From equation 4.2) we infer that FX n k:n ) = c 2 c γ/γ k/n) γ/γ +o P )) which implies in view of 4.6) that /γ FLB) n X n k:n ) F W) n X n k:n ) FX n k:n ) = O P k/n) γ/γ ) /γ ±ν.

12 2 Observe now that for a sufficiently small ν > 0 we have /γ ±ν = O P ) uniformly on 0 > 0 as sought Proof Theorem 2.2. In a similar way to what is done with D W) n ) in the proof of Theorem 2. in Benchaira et al. 206a) we decompose k /2 D n LB) ) into the sum of N n3 ) := N n ) := /γ FLB) n X n k:n ) FX n k:n ) FX n k:n ) N n2 ) := FX n k:n) F LB) n X n k:n ) FX n k:n ) F LB) n X n k:n ) FX n k:n ) FX n k:n ) F LB) n X n k:n ) /γ ) F LB) and N n4 ) := FX n k:n )/FX n k:n ) /γ. If we let n X n k:n ) FX n k:n ) FX n k:n ) M n ) := /γ FW) n X n k:n ) FX n k:n ) FX n k:n ) then byapplyingtheorem2.wehave /γ N n ) = /γ M n )+ /γ O P k/n) γ/γ uniformly on 0. By assumption we have k +γ /2γ) /n P 0 which is equivalent to kk/n) γ/γ P 0 as N therefore ) /γ knn ) = /γ kmn )+o P /γ ). 4.7) In view of this representation we show that both D W) n ) and D n LB) ) are weakly) approimated in the probability space ΩAP) by the same Gaussian process Γ;W) given in.5). Indeed for a sufficiently small ǫ > 0 and 0 < η < /2 Benchaira et al. 206a) see the beginning of the proof of Theorem 2. therein) showed that /γ kmn ) = Φ)+o ) P η)/γ±ǫ { γ whereφ) := /γ W /γ) γ + 0 γ γ +γ t γ/γ 2 W /γ t ) } dt.thenbyusingrepresentation 4.7) we get /γ knn ) = Φ)+o ) P 2 ) /γ +op η)/γ±ǫ. In particular for = we have ) F LB) k n X n k:n ) = kn n ) = Φ)+o P ) 4.8) FX n k:n )

13 leading to F LB) n X n k:n )/FX n k:n ) P as N. By applying Potters inequalities to F as it was done for F in 4.)) together with the previous limit we obtain FX n k:n ) F LB) n X n k:n ) = +O P ±ǫ )) /γ. 4.9) By combining 4.8) and4.9) we get /γ knn2 ) = Φ)+o P ±ǫ ). For the third term N n3 ) we use similar arguments to show that /γ knn3 ) = o P /γ ±ǫ ) +o P /γ + η)/γ±ǫ ). Observe that /γ η 0 )/γ o P /γ ±ǫ ) and /γ η 0 )/γ o P /γ + η)/γ±ǫ ) respectively equal o P η 0 )/γ±ǫ ) and o P η η 0 )/γ±ǫ ) for γ/γ 2 < η 0 < η < /2 and that both the last two quantities are equal to o P ) for any small ǫ > 0 and 0 > 0. Finally by following the same steps at the end of the proof of Theorem 2. in Benchaira et al. 206a) we get 3 Consequently we have { /γ η 0 )/γ knn4 ) = /γ τ /γ γ τ ka0 n/k)+o p /γ + η)/γ±ǫ ). D LB) n ) Γ;W) /γ τ /γ } ka0 n/k) = o P ) γ τ uniformly over 0. Recall that /γ = /γ /γ 2 then letting η 0 := /2 ξ yields 0 < ξ < /2 γ/γ 2 and achieves the proof Proof of Theorem 2.3. The proof is similar mutatis mutandis as that of Corollary 3. in Benchaira et al. 206a). Therefore we omit the details. Concluding note On the basis of Lynden-Bell integration we introduced a new estimator for the tail inde of right-truncated heavy-tailed data by considering a random threshold. This estimator may be an alternative to that based on Woodroofe integration recently proposed by Benchaira et al. 206a). Indeed the simulation results show that there is an equivalence between the asymptotic behaviors of both estimators with respect to biases and rmse s. However from a theoritical point of view the asymptotic normality of the former requires an additional condition on the sample fraction k of upper order statistics namely 2.6) which is stonger than the usual assumption in the contet of etremes k/n 0).

14 4 References Benchaira S. Meraghni D. Necir A On the asymptotic normality of the etreme value inde for right-truncated data. Statist. Probab. Lett Benchaira S. Meraghni D. Necir A. 206a. Tail product-limit process for truncated data with application to etreme value inde estimation. Etremes Benchaira S. Meraghni D. Necir A. 206b. Kernel estimation of the tail inde of a right-truncated Pareto-type distribution. Statist. Probab. Lett Gardes L. Stupfler G Estimating etreme quantiles under random truncation. TEST de Haan L. Ferreira A Etreme Value Theory: An Introduction. Springer. Hill B.M A simple general approach to inference about the tail of a distribution. Ann. Statist Hua L. Joe H. 20. Second order regular variation and conditional tail epectation of multiple risks. Insurance Math. Econom Lynden-Bell D. 97. A method of allowing for known observational selection in small samples applied to 3CR quasars. Monthly Notices Roy. Astronom. Soc Reiss R.D. Thomas M Statistical Analysis of Etreme Values with Applications to Insurance Finance Hydrology and Other Fields 3rd ed. Birkhäuser. Resnick S Heavy-Tail Phenomena: Probabilistic and Statistical Modeling. Springer. Shorack G.R. Wellner J. A Empirical processes with applications to statistics. New York: John Wiley & Sons. Strzalkowska-Kominiak E. Stute W Martingale representations of the Lynden-Bell estimator with applications. Statist. Probab. Lett Woodroofe M Estimating a distribution function with truncated data. Ann. Statist Worms J. Worms R A Lynden-Bell integral estimator for etremes of randomly truncated data. Statist. Probab. Lett