Extreme value statistics for truncated Pareto-type distributions

Transcription

1 arxiv: v3 [math.s] 23 Dec 204 Extreme value statistics for truncated Pareto-type distributions Beirlant J. a, Fraga Alves, M.I. b, Gomes, M.I. b, Meerschaert, M.M. c, a Department of Mathematics and Leuven Statistics Research Center, KU Leuven b Department of Statistics and Operations Research, University of Lisbon c Department of Statistics and Probability, Michigan State University December 24, 204 Abstract Recently attention has been drawn to practical problems with the use of unbounded Pareto distributions, for instance when there are natural upper bounds that truncate the probability tail. Aban, Meerschaert and Panorsa 2006 derived the maximum lielihood estimator for the Pareto tail index of a truncated Pareto distribution with a right truncation point. he Hill 975 estimator is then obtained by letting. he problem of extreme value estimation under right truncation was also introduced in Nuyts 200 who proposed a similar estimator for the tail index and considered trimming of the number of extreme order statistics. Given that in practice one does not always now whether the distribution is truncated or not, we discuss estimators for the Pareto index and extreme quantiles both under truncated and non-truncated Pareto-type distributions. We also propose a truncated Pareto QQ-plot in order to help deciding between a truncated and a non-truncated case. In this way we extend the classical extreme value methodology adding the truncated Pareto-type model with truncation point as the sample size n. Finally we present some practical examples, asymptotics and simulation results. Corresponding author: Jan Beirlant, KU Leuven, Dept of Mathematics and LStat, Celestijnenlaan 200B, 300 Heverlee, Belgium; jan.beirlant@wis.uleuven.be

2 Introduction Considering positive data, the Pareto distribution is a simple and very popular model with power law probability tail. Using the notation from Aban et al. 2006, the right tail function PW > w = τ α w α for w τ > 0 and α > 0 is considered as the standard example in the max domain of attraction of the Fréchet distribution. For instance, losses in property and casualty insurance often have a heavy right tail behaviour maing it appropriate for including large events in applications such as excess-of-loss pricing and enterprise ris management ERM. here might be some practical problems with the use of the Pareto distribution and its generalization to the Pareto-type model, because some probability mass can still be assigned to loss amounts that are unreasonable large or even physically impossible. In ERM this leads to the concept of maximum probable loss. For other applications of natural truncation, such as probable maximum precipitation, see Aban et al hese authors considered the upper-truncated Pareto distribution with right tail function RF for 0 < τ x, where τ <. PX > x = τ α x α α τ/ α 2 Aban et al derived the conditional maximum lielihood estimator MLE based on the + 0 < n largest order statistics representing only the portion of the tail where the truncated Pareto approximation holds. hey showed that, with X,n... X n,n X n +,n... X n,n denoting the order statistics of an independent and identically distributed i.i.d. sample of size n from X, the MLE s are ˆ A = X n,n, ˆτ A = /ˆα A X n,n n n Xn,n /X n,n ˆα A /ˆαA while ˆα A solves the equation log X n j+,n log X n,n = + X n,n/x n,n ˆαA logx n,n /X n,n. ˆα A X n,n /X ˆα n,n A j= 2 3

3 his estimator /ˆα A can be considered as an extension of Hill s 975 estimator H,n = j= log X n j+,n log X n,n to the case of a truncated Pareto distribution with <, while H,n was introduced as an estimator of /α when =. Independently, Nuyts 200 considered an adaptation of the Hill 975 estimator through the estimation of Elog W W [L, R] = R L logwfwdw R L fwdw 4 for some positive numbers 0 < L < R, taing W to be the strict Pareto in. hen, and 4 lead to Elog W W [L, R] = α + L α log L R α log R L α R α. 5 Denoting by Qp := inf{x : F x p} the upper quantile function, consider L = Q /n and R = Q r/n in 5, the /n and r/n r < < n upper quantiles which are estimated by X n +,n and X n r+,n respectively. he estimator of /α in Nuyts 200 is then obtained from solving r j=r logx n j+,n = α + X α n +,n log X n +,n Xn r+,n α log X n r+,n X α n +,n X α n r+,n with r = r +. After some algebra, r logx n j+,n logx n,n j=r = α + X n +,n/x n r+,n α logx n,n /X n r+,n X n +,n /X n r+,n α + logx n +,n/x n,n X n +,n /X n r+,n α. he last term on the right hand side is of smaller order than the other terms as can be shown by asymptotic arguments as developed in the Appendix. Hence one is led to delete the last term, and then, in case r =, this equation is only a minor adaptation of 3. We conclude that the estimators of Nuyts 3

4 200 and Aban et al are basically the same. Deleting the last term in the above expression and considering the trimming procedure from Nuyts 200 we consider the estimator ˆα r,,n defined from r logx n j+,n /X n,n j=r = /ˆα r,,n + X n,n/x n r+,n ˆα r,,n logxn,n /X n r+,n X n,n /X n r+,n ˆα r,,n for r < < n. In what follows we use the notation H r,,n = log X n j+,n log X n,n r and j=r R r,,n = 6 X n,n X n r+,n. 7 Furthermore note that H,n = H,,n. Hence the estimator ˆα r,,n is defined as the solution of the equation corresponding to 6: H r,,n = α + Rα r,,n log R r,,n. 8 Rr,,n α he solution of 8 can be approximated using Newton-Raphson iteration on the equation f := H r,,n α α Rα r,,n log R r,,n = 0 Rr,,n α to get ˆα l+ r,,n = ˆα l r,,n + H r,,n ˆα l ˆα l r,,n R ˆα l r,,n r,,n r,,n 2 R ˆα l r,,n r,,n 4 log R r,,n R ˆαl r,,n r,,n log2 R r,,n R ˆαl r,,n r,,n 2, l = 0,,... 9

5 where for instance Hill s trimmed estimator can serve as an initial approximation: ˆα 0 = /H r,,n. We will study the behaviour of ˆα r,,n in the case of both Pareto-type and truncated Pareto-type distributions. We will mae use of the RF F x = P[X > x] and of the tail quantile function U defined by Ux = Q /x x >. Pareto-type distributions are defined by F x = x α l F x, α > 0, 0 where l F is a slowly varying function at infinity, i.e. lim t l F tx/l F t = for every x > 0. In extreme value statistics the parameter ξ := /α is referred to as the extreme value index EVI. he EVI ξ is the shape parameter in the generalized extreme value distribution G ξ x = exp + ξx /ξ, for + ξx > 0. his class of distributions is the set of the unique non-degenerate limit distributions of a sequence of maximum values, linearly normalized. In case ξ > 0 the class of distributions for which the maxima are attracted to G ξ corresponds to the Pareto-type distributions in 0. We define the truncated version of a Pareto-type RF by F x = C x α l F x α l F. he constant C = τ α l F τ α l F is specified by the condition F τ = 0, where τ > 0 is the lower bound of the range: x τ,. Below in Proposition a we derive that the corresponding upper quantile function Q p can be written as Q p = + p /α ζ /p,, 2 D where D = C l α with l = l /α F, and ζ /p, as and p 0, assuming that the quantity p/d remains bounded away from as p 0 and. Note that in case of the simple truncated Pareto distribution in 2 Q p = C /α D + p /α = + p /α, 3 D 5

6 with C = τ α α and D = C α. An expansion of + p/d /α for p/d 0 yields Q p = p + o. 4 α D Using for instance 2. in Beirlant et al. 2004, it follows that the truncated Pareto distribution belongs to the Weibull domain of attraction for maxima with EVI ξ = when p/d 0. he quantity D equals the odds ratio PX > /PX of the truncated probability mass under the untruncated Pareto-type distribution in 0. If the underlying X is truncated at a quantile level = Q π, then D = π/ π. Hence asymptotic conditions on p/d, as p 0 and, amount to conditions on the relative behaviour between the odds ratio π/ π and p p/ p p 0, i.e. between the truncated probability and the tail probability p of interest. For instance, if p is negligible compared to D, or p/d 0, then the truncation is significant when estimating the quantile Q p. As suggested in Clar 203 α could also be taen to be zero or negative. For instance formally setting α = in 2, one obtains the tail of a uniform type distribution. Finite tail distributions following 4 with negative value of α show a fast rate of convergence to when p 0 and is a big number, due to the presence of D in 4. In the applications we have in mind here, convergence to is slow and hence a positive value of α is appropriate. Moreover we allow the truncation point to be large, expressed by. In an asymptotic setting this means that we consider a sequence of models indexed by the truncation point. his approach appears to be new and allows to bridge tail models with ξ = and Pareto models in the sense that the Pareto-type distributions are considered as limits of truncated Pareto-type distributions. In the truncation case we improve upon the well-established extreme quantile and endpoint estimation methods. Moreover in developing statistical methods we consider the cases / 0, respectively / κ > 0 finite, and /, corresponding to whether the truncated probability mass α l F is large, intermediate or small with respect to the proportion of data used in the extreme value estimation. he final case with / can then be considered adjacent to the untruncated Pareto-type models with ξ > 0 and appear 6

7 in practice when is so high corresponding to the data that no truncation effect is visible from the data. In the next section we provide estimators for and for extreme quantiles. We also consider the problem of deciding between a ParetoPa-type case in 0, and a truncated ParetoPa-type case in. o this end we construct a Pa QQ-plot. In section 3 we discuss the asymptotic properties of ˆα r,,n and the extreme quantile estimators under 0 and. We also consider the effect of the trimming parameter r. In asymptotic settings we will consider, n with an intermediate sequence, i.e. /n 0. Concerning the trimming parameter r we assume r/ λ [0,. Finally we conclude with simulation results and practical examples. 2 Statistical methods for truncated Paretotype distributions From 3 it is clear that the estimation of D is an intermediate step in important estimation problems following the estimation of α, namely of extreme quantiles and of the endpoint. From 3 Q α n Q r = n + r = + r + n D r n. 5 + D Motivated by 5 and estimating Q /n/q r/n by R r,,n in 7, we propose ˆD := ˆD,r,,n = Rˆα r,,n r,,n λ r, 6 n Rˆα r, r,,n as an estimation method for D in case of truncated and non-truncated Patype distributions, where λ r, = r/ + is a continuity correction of r/ in the numerator of 6. In practice we will mae use of the admissible estimator { } ˆD 0 := max ˆD, 0. In case D > 0, in order to construct estimators of and extreme quantiles 7

8 q p = Q p, as in 5 we find that α Q p Q = + + p n D = D + n D + p. 7 hen taing logarithms on both side of 7 and estimating Q /n by X n,n we find an estimator ˆq p := ˆq p,r,,n of q p : log ˆq p,r,,n = log X n,n + ˆα r,,n log Note that ˆq p can also be rewritten as log ˆq p,r,,n = log X n,n + ˆα r,,n log ˆq p,r,,n = X n,n np /ˆαr,,n ˆD + n ˆD + p. 8 + n ˆD + p, 9 ˆD + n ˆD + ˆD p /ˆαr,,n. 20 An estimator ˆ r,,n of follows from letting p 0 in the above expressions for ˆq p,r,,n : { log ˆ r,,n = max log X n,n + log + } ˆα r,,n n ˆD, log X n,n. 2 he maximum of log X n,n and the value following from 9 by taing p 0, is taen in order for this endpoint estimator to be admissible. It now follows that in case ˆD > 0 ˆq p,r,,n = ˆ r,,n + pˆd /ˆαr,,n, which is consistent with 3. Equation 20 for ˆq p,r,,n constitutes an adaptation to the Pa case of the Weissman 978 estimator ˆq W p,,n = X n,n np H,,n 22 8

9 which is valid under 0. Expression 20 is more adapted to the case /. Version 9 can be lined to the cases where / is bounded away from. In practice we always use version 8 which can be applied in all cases. Note that such alternative expressions do not exist for the estimation of the endpoint as in case D = 0 no finite endpoint exists. Based on a chosen value ˆD,r,,n for particular, we propose the Pa QQ-plot to verify the validity of 2: log X n j+,n, log ˆD,r,,n + j/n, j =,..., n. 23 Note that when = or D = 0 the Pa QQ-plot agrees with the classical Pareto QQ-plot log X n j+,n, logj/n, j =,..., n. 24 Under 2 an ultimately linear pattern should be observed to the right of some anchor point, i.e. at the points with indices j =,..., for some < < n. From this, we propose to choose the value of in practice as the value that maximizes the correlation between log X n j+,n and log ˆD,r,,n + j/n for j =,..., and > 0. his choice can be improved in future wor since the covariance structure of the deviations of the points on the Pa QQ-plot from the reference line are neither independent nor identically distributed. his issue was addressed for the Pa QQ-plot in Beirlant et al. 996 and Aban and Meerschaert 2004 and should be considered in the truncated case too. 3 Asymptotic distributions of the estimators In this section we derive the large sample distribution of ˆα r,,n and ˆq p,r,,n defined in 8 and 8 for Pa-type distributions in case / is bounded away from, or when /. he case of Pa-type distributions in 0 can then be considered as a limit case when /. he proofs are deferred to the Appendix. We first develop more precise expressions for the upper quantile function 9

10 Q p for Pa-type distributions assuming that F is continuous. o this end set l α x = l F x and let l denote the de Bruyn conjugate of l satisfying lxl xlx as x. Solving the equation y /α = xlx by x = y /α l y /α see for instance Proposition 2.5 and page 80 in Beirlant et al., 2004, we find that the tail quantile function U y = Q y corresponding to F is given by U y = C y + l α /α l /C y + l α /α = C /α y /α + D l C /α y + D /α. In order to derive asymptotic results for our estimators we have to consider the different cases for the balance between /y and D as y,. hese are specified in the following Proposition. In the asymptotic results we will apply this with y = n/. Proposition. a If yd is bounded away from 0 as y,, then U y = + /α {l l l } D y where l l l and l l [+ D y ] /α. l l b If yd 0 as y,, then l U y = yc α l yc /α α + D y yc where l α [+D y] /α l yc α. l l [ + D y ] /α l l, 25 yc α [ + D y] /α l yc α, 26 Note that in case yd 0 as y, the tail quantile function U y is asymptotically equivalent to the Pa-type tail quantile function yc α l yc α which is the model corresponding to the case =. Hence in case b we have that ξ = /α > 0, compared to the case where 0

11 yd is bounded away from 0 in which case ξ =. In order to derive the asymptotic results for the estimators we will mae use of a second order slow variation condition on l specifying the rate of convergence of l tx/l x to as x, which is used typically in all asymptotic results in extreme value methods see for instance heorem in de Haan and Ferreira, 2006: lim x b x log l tx l x = h ρ t 27 with ρ < 0, h ρ t = t ρ /ρ, and b regularly varying with index ρ, i.e. b tx/b x t ρ as x for every t > 0. hroughout we will also assume that as r, for some λ [0, λ r, λ = O. 28 Condition 28 guarantees that we can interchange r/ and λ in the asymptotic results and proofs. Furthermore W represents a standard Wiener process. heorem. Let 27 and 28 hold and let n, = n, /n 0,. hen a if / 0 and / 3/2 0, nd 2 /ˆα r,,n /α = 3/2 α λ N 2 λ + b l α ρ α +o 2 p, with N λ = W + W λ 2 λ λ W udu N 0, λ/2;

12 b if / κ 0,, /ˆα r,,n /α = δ κ,λ α W ud log + κu λ λ + W { + κλ } + κ λ κ λ log + κλ W λ { + κ } + κ λ κ λ log + κλ +b l β κ,λ + o p, δ κ,λ with asymptotic variance σ 2 κ,λ /α2 and β κ,λ = A κ,λ B κ,λ c κ,λ, A κ,λ = λ λ h ρ [ + κu] /α du h ρ [ + κ] /α, B κ,λ = h ρ [ + κ] /α h ρ [ + κλ] /α, c κ,λ = + κλ + κλ + κ + log λκ λ 2 κ 2 σκ,λ 2 =, λδ κ,λ δ κ,λ = + κλ + κ λ 2 κ 2 log 2 + κλ + κ + κ ; + κλ c if / and D = on/ +ρ /α σ 2 λ /ˆα r,,n /α = α N 2 λ + b C n/ /α βλ + o p, with N 2 W u W λ = du + λ u λ + λ λ N 0, σ 2 λ,, W λ log λ λ + log λ λ 2

13 and βλ = σ 2 λ = λ log2 λ λ 2 ρ λ α ρ ρ λ ρ + λ logλ λ h ρ /λ λ +, α λ λ log2 λ λ. 2 Here β0 = α ρ /α and σ 2 0 =. Remar. heorem a entails that in case / 0, should grow with n to infinity as n η where 0 < η < /3 in order to obtain a reasonable estimation rate. his means in practice that in case of a Pa-type distribution the number of extremes should be taen large. Also the presence of D in the standard deviation guarantees even faster convergence for large values of. Moreover there is a bias of order b l which is only negligible if is a reasonably large value and ρ is sufficiently large. Remar 2. Robustness under Pa-type models has received quite some attention in the literature see for instance Hubert et al., 203, and the references therein while the classical estimators such as Hill s 975 estimator are nown to be highly non robust against outliers. he estimator ˆα r,,n provides a way to robustify the Hill estimator H,,n using a trimming procedure with r >. rimming of course maes the estimator more robust against outliers, but decreases the efficiency of the estimator. his is illustrated in Figure 8 of Appendix 3, plotting the functions σ 2 λ and βλ for λ [0, /4]. he robustness properties of the estimation procedures presented here will be studied elsewhere. Remar 3. In case / and λ = 0 the asymptotic result for ˆα r,,n is identical to that of the Hill estimator H,,n as given for instance in Beirlant et al. 2004, section 4.2. o see this notice that the main slowly varying component of the tail quantile function U equals l U x := C /α l C x /α. Based on 27 we find that for every t > 0 and x log l Utx l U x b C x /α h ρ t /α = b U xh ρ /αt, 3

14 where b U x = b C x /α /α is regularly varying with index ρ /α. Hence the asymptotic bias in heorem c when λ = 0 equals b U n// ρ /α which is the form found in literature for the bias of the Hill estimator. Concerning asymptotic results for the extreme quantile estimator ˆq p,r,,n we confine ourselves to the cases / 0 and / due to the complexity of the intermediate case. A similar result when / κ 0, can readily be obtained using similar techniques as in the cases presented here. In the case / we confine ourselves to the case r =. In fact, even light trimming entails inferior behaviour in mean squared error sense for the quantile estimator in case of no truncation as it will be shown in the simulations. heorem 2. Let 27 and 28 hold and let n, = n, /n 0, and p = p n such that np n = o. a Let / 0 and / 3/2 0. hen log ˆq p,r,,n log q p = O 2 + o b l + o p. b Let r =, np n, lognp n = o, D /p n log/np n 0, and 0. hen log ˆq p,r,,n log q p { } = log /np n α N 2 λ + b C n/ /α β0 + o p E + o p α np n where E is a standard exponential random variable. Remar 4. In case / 0 both the asymptotic bias and the stochastic part of ˆq p are of smaller order than the asymptotic bias of the estimator /ˆα r,,n. his is also confirmed by the simulation results in section 4 where the plots of the quantile estimators are found to be quite horizontal as a function of, compared to other estimators found in extreme value analysis. In case /, note that the quantile estimator ˆq p is only consistent if np n 0, this is for quantiles q p situated maximally up to the border 4

15 of the sample q n, using for instance a sequence of the type p n = log τ /n for some τ > 0. he extra factor + n ˆD / + ˆD /ˆαr,,n p in 20 compared to the Weissman estimator ˆq p,,n W induces this restriction. he first term in the expansion of log ˆq p in heorem 2b is indeed the asymptotic expansion of ˆq p,,n W as given for instance in Beirlant et al. 2004, section 4.6. If 0 then the expansion of the Weissman estimator is np n log/np n dominant, while if be retained. np n log/np n the second term in the expansion is to 4 Practical examples and simulations For a first illustration we use the data set containing fatalities due to large earthquaes as published by the U.S. Geological Survey on usgs.gov/earthquaes/world/, which were also used in Clar 203. It contains the estimated number of deaths for the 24 events between 900 and 20 with at least 000 deaths. In Figure top left the Pa QQ-plot or log-log plot in 24 is given. A curvature is appearing at the largest observations which indicates that the unbounded Pareto pattern could be violated in this example. On this plot the extrapolations using a Pareto distribution linear pattern and a truncated Pareto model using the truncated Pareto model 2 are plotted based on the largest 2 data points as it was proposed in Clar 203. In Figure middle the estimates ˆα,,n and ˆD,, are plotted against =,..., n. Here we have chosen = 00 as a typical value where both plots are horizontal in. he Pa QQ-plot in 23 is given in Figure top right, using the above mentioned value = 00. Finally in Figure bottom the estimates of the extreme quantile q 0.0 using 8 and the endpoint using 2 are presented as a function of. hey are contrasted with the values obtained by the classical method of moment estimates as introduced in Deers et al. 989 illustrating the slow convergence of the classical extreme value methods in the Pa-type model we 5

16 study here. For any real EVI, the classical moment ξ-estimator is defined by ˆξ n, MOM := M n, + ˆξ n,, ξ n, := [ 2 M n, 2 /M 2 n,], 29 with M j n, := i=0 lnj X n i,n /X n,n, j =, 2, which constitutes a consistent estimator for ξ R. he Hill estimator is M n, = H,,n. he MOM-estimators for high quantiles and right endpoint, based on the MOM moment estimator ˆξ n,, are defined by see de Haan and Ferreira, 2006, 4.3.2, for details. and ˆq p MOM := X n,n + X n,n M n, ˆξ n, ˆ MOM := max ˆξMOM n, np ˆξ n, MOM ˆ M, X n,n, ˆ M := X n,n X n,nm n, ˆξ n, ˆξ MOM n, Notice that in 3 ˆ MOM corresponds to the admissible version of the moment endpoint estimator ˆ M, since the latter can return values below the sample maximum. If we focus on Figure bottom right it is clear that ˆ MOM does not add any extra information compared with the sample maximum, for the range of thresholds 5, contrasting with the behaviour of the proposed ˆ,,n. Concerning the high quantile estimation, the chosen value p = 0.0 is directly related with the modest sample size here of n = 24. Similar to the endpoint estimation, for this data set the new quantile estimates ˆq 0.0,,,n also reveals a stable pattern on, in Figure bottom left. Overall, on the basis of Figure we can conclude that the Pa-type model with a truncation point around 400,000 deaths offers a convincing fit, and leads to a useful estimator for extreme quantiles. Another example where the Pa-type model is fitting well to the tail is found with the distribution of seismic moments of shallow earthquaes at depth less than 70 m, between 977 and 2000, which can be found in Pisareno and Sornette he tails of these distributions were also 6

17 Figure : Earthquae fatalities data set. op: Pa QQ-plot left with extrapolations anchored at logx n 2,n based on a non-truncated Pa model in dotted line and a truncated Pareto model in 2 full line as proposed in Clar 203; Pa QQ-plot right for the earthquae fatalities data set using r = and = 00. Middle: plots of Pareto index ˆα,,n left and odds ratio ˆD,,,n estimates =,..., 24 right maring the values at = 00. Bottom: quantile estimates ˆq 0.0,,,n left and endpoint estimates ˆ,,n right contrasted with the method of moments quantile and endpoint estimators, in 30 and 3, respectively. considered in Section 6.3 in Beirlant et al both for subduction and mid ocean ridge zones. Here we concentrate on the subduction zone data. In Beirlant et al. 2004, page 200, the use of = 57 is suggested in order to obtain a proper fit to the upper tail of the underlying distribution. he tail 7

18 fit is revisited here in Figure 2 top left using truncated and non-truncated Pareto models. For this data set, in Figure 2 top right, the Pa QQ-plot Figure 2: Seismic moments data set. op: Pa QQ-plot left with extrapolations based on a non-truncated Pareto model in dotted line and a truncated Pareto model in 2 full line; Pa QQ-plot right using r = and the top = 398 data. Middle: plots of Pareto index ˆα,,n left and odds ratio ˆD,,,n right estimates. Bottom: quantile estimates ˆq ,,,n left and endpoint estimates ˆ,,n right contrasted with the method of moments quantile and endpoint estimators, in 30 and 3, respectively. in 23, associated with the validity of 2, has been built on the chosen value = 398, which maximizes the correlation between log X n j+,n and log ˆD + j/n, j =,,, for 0 < < n. For the Pareto index, odds 8

19 ratio, high quantile and right endpoint estimation in Figure 2 we get similar conclusions to the ones of the earthquae fatalities data set. he finite sample behaviour of the proposed estimators ˆα r,,n based on 8 and 9, ˆq p,r,,n from 8, and ˆ r,,n from 20 has been studied through an extensive Monte Carlo simulation procedure with 000 runs, both for truncated and non-truncated Pareto-type distributions. Here we will only present results concerning Pareto and Burr distributions, with truncated and non-truncated versions:. Non-truncated models a Paretoα, α =, 2 b Burrα, ρ, α =, 2, ρ = 2. runcated models F x = x α, x >, α > 0, 32 F x = + x ρα /ρ, x > 0, ρ < 0, α > a runcated-paretoα,, α = 2 and a high quantile from the corresponding Pareto model 32 F x = x α, < x <, α > α Here we use = 3.623, respectively =.442, the 90 percentile, respectively the median, of the corresponding non-truncated Pareto model. b runcated-burrα, ρ,, α = 2, ρ = and a high quantile from the corresponding Burr model in 33 F x = + x ρα /ρ, 0 < x <, ρ < 0, α > ρα /ρ Here we use = 3, being the 90 percentile of the corresponding non-truncated Burr distribution. Note that in case of 35 l y = + y αρ + o/αρ when y and ρ = αρ. 9

20 For a particular data set from an unnown but apparently heavy-tailed distribution, the practitioner does not now if the distribution comes from a truncated or a non-truncated Pareto-type distribution and hence we have to study the behaviour of the proposed estimators under both cases, and compare them with the existing extreme value estimators. Our simulation results will illustrate that applying the new estimators of the Pareto index, and extreme quantiles with p = /n, based on a Pa-type model, are appropriate in both cases. As mentioned before, Pa-type distributions belong to the Weibull domain of attraction for maxima with EVI ξ = so that the moment estimator in 29 almost surely converges to -. Also, for these models, /H,,n does not constitute a consistent estimator either for α or for ξ, since in case ξ < 0 the Hill estimator H,,n almost surely tends to zero when /n 0 as, n. Only when = we have that ˆα r,,n and /H,,n estimate the same value /ξ. When estimating an extreme quantile the estimator in 30 based on the moment estimator is designed both for truncated and non-truncated cases and is to be compared with the estimation procedure defined in 20. he same holds for endpoint estimators in 2 and 3 in case of truncated models. Finally ˆq p,r,,n and the Weissman 978 extreme quantile estimator ˆq W p,,n from 22 are competitors in case of non-truncated Pa-type distributions only. In Figures 6-7, the trimmed-hill, not corrected refers to H r,,n, which coincides with the Hill estimator for r =. he α-estimator ˆα is the solution of 8, approximated using the Newton-Raphson iteration as in 9, with an initial value ˆα 0 = /H r,,n. In Figures 3-5 we present the relative performance of the estimators with r = and r = 0 for the truncated Pareto model in 34 and the truncated Burr model 35, while in Figures 6-7 we consider the corresponding nontruncated models in 32 and 33. Observe that ˆα r,,n appears to be not too sensitive for small changes of r. It appears that when the model is Pa-type, whether truncated or not, the estimators proposed here are performing well. In Figure 3 the EVI moment estimator systematically overestimates the true value of ξ = for this upper tail truncated distribution with odds ratio D = /9. Only for a lower truncation point, here with D =, the situation naturally improves for the MOM-estimators Figure 4 top. his confirms 20

21 Figure 3: Paα = 2, = 3.623: Estimation of α using the Newton-Raphson procedure with initial value ˆα 0 = /H r,,n, r =, 0 top. Estimation of the high quantile q 0.00 middle and right endpoint bottom using ˆq 0.00,r,,n defined in 8 and ˆ r,,n as in 2, r =, 0. that the methods proposed here are especially useful for Pa-type models with a truncation point equal to a high quantile of the corresponding nontruncated Pa-type distribution. In case of the truncated Burr distribution the behaviour of the Pareto index estimator ˆα r,,n Figure 5 top is underestimating α, not uncommon in extreme value analysis; see for instance Figure 7 for the Hill estimator. 2

22 On the other hand, for these Pa-type models, the convergence of the new quantile and endpoint estimators seems to be attained at low thresholds or high with high accuracy, contrasting with higher thresholds or low for MOM class estimators. With quantile estimation in Figures 3-5 an erratic behaviour appears for some smaller or larger values of which becomes more apparent when corresponds to lower quantiles of the underlying Pareto distribution. his is a consequence of the use of = max{ ˆD, 0} rather than ˆD in practice. If we assume that is finite then using simply ˆD 0 rather than ˆD produces much smoother performance in extreme quantile estimation. On the other hand in case of non-truncated models the use of 0 ˆD instead of ˆD, leads to extreme quantile estimates that are quite sensitive with respect to the value of D. While the stable parts in the plots of 0 quantile estimates are readily apparent anyway, we here use ˆD in 8. ˆD 0 In case of non-truncated Pa-type models see Figures 6 and 7 concerning high quantile estimation, we also consider the Weissman 978 estimator ˆq p W defined in 22 besides the newly proposed estimator ˆq p,r,,n and the MOMestimator ˆq p MOM, defined in 8 and 30 respectively. aing into account that H,,n and ˆq p W are designed for this particular situation, we can conclude that the newly proposed estimators perform reasonably well at p = /n if MOM we compare with the classical extreme value estimators H,,n and ˆξ n,. For instance in case of the Burr distribution in Figure 7 it appears that our quantile estimator is slightly worse than the moment estimator but better than the Weissman 978 estimator. In the strict Pareto case in Figure 6 left on average ˆq p,,,n underestimates Q0.999, but this larger bias is balanced by a lower variance, which results in an MSE competitive with ˆq p MOM. Finally note that trimming has a serious negative influence on the estimation of extreme quantiles here and hence should be avoided with non-truncated Pa-type distributions. 22

23 Figure 4: Paα = 2, =.442: Estimation of α using the Newton-Raphson procedure with initial value ˆα 0 = /H r,,n, r =, 0 top. Estimation of the high quantile q 0.00 middle and right endpoint bottom using ˆq 0.00,r,,n defined in 8 and ˆ r,,n as in 2, r =, 0. 23

24 Figure 5: runcated-burrα = 2, ρ =, = 3: Estimation of α top, high quantile q 0.00 middel and right endpoint bottom using ˆα r,,n, ˆq 0.00,r,,n defined in 8 and ˆ r,,n as in 2, for r =, 0. 24

25 Figure 6: Paα = 2: Estimation of α using the Newton-Raphson procedure with initial value ˆα 0 = /H r,,n top, r =, 0. Estimation of high quantile q 0.00 using ˆq 0.00,r,,n defined in 8 bottom, r =, 0. Figure 7: Burrα = 2; ρ = : Estimation of α using the Newton-Raphson procedure with initial value ˆα 0 = /H r,,n top, r =, 0. Estimation of high quantile q 0.00 using ˆq 0.00,r,,n defined in 8 bottom, r =, 0. 25

26 5 Conclusion We have extended the wor on estimating the Pareto index α under truncation from Aban et al and Nuyts 200 from Pareto to regularly varying tails. he main proposals and findings are he new estimator of the Pareto index α is effective whether the underlying distribution is truncated or not, thus unifying previous approaches. Although based on a truncated model, our estimator is competitive with the existing benchmar methods even when the underlying distribution is unbounded. Our method leads to new quantile and endpoint estimators which are especially effective in the case of moderate truncation / 0. In case the data come from an non-truncated Pa-type distribution, which is the case when the Pa QQ-plot 24 is linear in the right tail, the extreme quantile estimator 8 should not be used for extrapolation far out of range of the available observations as discussed in Remar 4. Nuyts 200 has proposed to trim the most extreme data points. he robustness will then be enhanced, but our results indicate that the MSE is worse: only slightly for Pa-type models, but rather severely in case of unbounded regularly varying tails, especially in case of quantile estimation. A new Pa QQ-plot is constructed that can assist in verifying the validity of the Pa-type model. Several possible areas for new research appear from this wor. For instance lining truncation with all domains of attraction for maxima, especially in case of the Gumbel domain of attraction with ξ = 0. Also bringing in covariate information in the model appears of importance. For instance modelling large earthquaes using geographical information is a problem of interest. Finally the robustness properties of the estimators proposed here should be studied further. Appendix. Derivation of Proposition Set l α x = l F x and let l denote the de Bruyn conjugate of l. Solving the equation y /α = xlx by x = y /α l y /α see for instance Proposition 2.5 and 26

27 page 80 in Beirlant et al., 2004, we find that the tail quantile function U y = Q y corresponding to the RF F is given by U y = C y + l α /α l /C y + l α /α. 36 First consider the case where yd is bounded away from 0 as y,. hen apply [ C y + l α] ] /α /α = l [ + D y twice so that U y = l + /α [ l l + ] /α, D y D y while multiplying by l l on both the top and bottom leads to U y = + /α l l [ + {l l D y ] /α l } D y l. l As we have l l l by the definition of the de Bruyn conjugate. Assuming that for some constants 0 < m < M < we have m < D y < M as y,, then it follows from the uniform convergence theorem for regularly varying functions Seneta, 976 that [ ] /α l l + D y l. 37 l he limit in 37 clearly also holds when yd tends to. Alternatively, when yd 0 as y,, use [ C y + l α] /α = yc /α [ + yd ] /α twice to rewrite 36 as U y = yc /α [ + yd ] /α l yc /α [ + yd ] /α. Multiplying by l yc /α on both the top and bottom leads to 26. Finally it follows that l yc α [ + D y] /α /l yc α as yc α and [ + yd ] /α. Appendix 2. Outline of proof of heorems and 2 Proof of heorem he mean value theorem implies that /ˆα r,,n /α = f/α/f / α where α = α r,,n is between α and ˆα r,,n, with f α = H r,,n 27

28 /α Rα r,,n Rr,,n α log R r,,n. hen we obtain that the limit distribution of /ˆα r,,n is found from the asymptotic distribution of α 2 R α r,,n log2 R r,,n R α r,,n 2 H r,,n /α Rα r,,n log R r,,n R α r,,n Hence the asymptotic behaviour of H r,,n and log R r,,n constitute essential building blocs in the derivation of the asymptotics for ˆα r,,n. We consider these in the following Propositions. For this we mae use of the result see de Haan and Ferreira, 2006, that for some standard Wiener process W with EW sw t = mins, t we have uniformly over all j =,...,, as, n, /n 0 n U j,n j W j p Proposition 2. Let 27 and 28 hold and let n, = n, /n 0,. hen a if / is bounded away from,. H r,,n = + / λ r, log α λ r, / + α W / + / λ r, where +b l A r,,n, + o p, + + λ r, λ r, W ud log + u + o p A r,,n, = λ r, λ r, h ρ [ + u] /α du h ρ [ + ] /α, 28

29 b if /, H r,,n = + λ r, log λ r, α λ r, + α W λ r, W u λ r, u du + o p +b C α n/ α h ρ u /α du + o p λ r, λ r, + + log λ r, + o p, α λ r, where the first two terms in this expansion are the limits for / of the first two lines in the expansion in case a. Proof Let j r = j r + and let U,n U 2,n... U n,n denote the order statistics from an i.i.d. sample of size n from the uniform 0, distribution. hen using summation by parts and the fact that X n j+,n = d Q U j,n j =,..., n H r,,n = j r log X n j+,n log X n j,n r j=r = j r log Q U j+,n log Q U j,n r j=r = + λ r, + j=r j + + λ r, Uj+,n U j,n d log Q w. Using 38 H r,,n can now be approximated as, n by the integral { } u+w u/ +/ u λ r, λ r, λ r, u+w u/ d log Q n w du. Using the mean value theorem on the inner integral between u+w u/ and u+ W u/ +/, followed by an integration by parts, we obtain the approximation u λ r, d log Q u + W u λ r, λ r, n = log U n / + W + λ r, 29 λ r, log U n / u + W u du. 39

30 First, let / be bounded away from. hen using Proposition a the approximation 39 of H r,,n equals α log + log l l α λ r, + λ r, + W [ + λ r, log λ r, log l + l + W ] /α u + W u du [ + u + W u ] /α du. Next, add and subtract log l l from the second and fourth line respectively, and use the approximations log + u + W u = log + u + W u with 0 < u and u between u and u + W u/, and l l [ + u + W u ] /α log l l = b l h ρ [ + u] /α + o p. Finally, using partial integration we have λ r, λ r, log + u du = + log + + λ r, from which one obtains the stated result in a. + u, λ r, log + λ r, log + + λ r, λ r, Secondly, consider /. hen using Proposition b we obtain using 30

31 similar steps as in the preceeding case that approximation 39 of H r,,n equals α log + W + α log + α λ r, α λ r, λ r, log + λ r, his is now approximated by + λ r, log λ r, α λ r, + α λ r, log + log λ r, λ r, +b C α n/ α λ r, + W u + W u du + α u + W u du log l C n/u α [ + /u] /α du. λ r, l C α n/ α [ + /] /α + W α λ r, λ r, h ρ W u λ r, u u /α + nd /u + / he result then follows using the approximation + u /α nd /u /α du λ r, + / λ r, h ρ λ r, λ r, h ρ u /α du. du /α du. In a similar way we obtain an asymptotic result for log R r,,n. Proposition 3. Let 27 and 28 hold and let n, = n, /n 0,. hen 3

32 a if / is bounded away from, log R r,,n = + α log λ r, + where W /nd + / W λ r, / + λ r, / α +b l B r,,n, + o, + o p B r,,n, = h ρ [ + / ] /α h ρ [ + / λ r, ] /α, b if /, log R r,,n = α log λ r, α b C /α W W λ r, + o p λ r, n/ /α h ρ λ /α r, + o + α /λ r, + o, where the first two terms in this expansion are the limits for / of the first two lines in the expansion in case a. Proof of heorem cont d. First we derive the consistency of ˆα r,,n under the conditions of heorem, so that then α p α. Aban et al. 2006, see A.4 showed that ft := t + Rt r,,n log R r,,n H Rr,,n t r,,n is a decreasing function in t 0,. Moreover lim t ft = Hr,,n < 0 and lim t 0 ft = log Rr,,n /2 H r,,n. Showing that asymptotically under the conditions of the theorem log R r,,n /2 H r,,n > 0 using Propositions 2 and 3 in both cases a and b, we have then that there is a unique solution to the equation ft = 0. Note with Propositions 2 and 3 that for the true value α we have fα = o p, since H r,,n and α + Rα r,,n log R r,,n R r,,n α asymptotically are equal, namely to α + λ log + in case a, and λ + λ nd α + λ log λ λ in case b. So the true value α asymptotically is a solution from which the consistency follows. Now using Propositions 2 and 3 we obtain that α 2 R α r,,n log2 R r,,n R α r,,n 2 = δ r,,n, + o p, 40 32

33 where δ r,,n, = + λ r, + λ r, 2 2 log 2 + λ r, +. Next, consider gh r,,n, log R r,,n = H r,,n /α Rα r,,n log R r,,n R α r,,n with e αy gx, y = x α y e αy. he aylor approximation of gh r,,n, log R r,,n around the asymptotic expectated value E H r,,n and E log R r,,n yields ge H r,,n, E log R r,,n + H r,,n E H r,,n + log R r,,n E log R r,,n g y E H r,,n, E log R r,,n, 4 with, based on Proposition 3, where g y E H r,,n, E log R r,,n = c r,,n, + o, 42 c r,,n, = + λ r, λ r, λ r, log λ r, +. From Propositions 2 and 3, 4, 42, and 40 we find that the stochastic part in the development of ˆα r,,n α is given by δ r,,n, α W ud log λ r, λ r, { + W λ r, W λ r, λ r, { + λ r, + λ r, log + λ r, log u + + λ r, + + λ r, } } Developing for / 0, respectively taing limits for / κ and / leads to the stated asymptotic variances in the different cases a, b, respectively c. From 4, 42, and the asymptotic bias expressions in Propositions 2 and 3 33.

34 one finds the asymptotic bias expressions of ˆα r,,n. For instance in case / is bounded away from we find that ge H r,,n, E log R r,,n = b l A r,,n, B r,,n, c r,,n, + o. 43 Condition D = on/ + ρ α in heorem b entails that the bias term due to the factor + D y /α in Proposition b is negligible with respect to the bias term due to the last factor based on l in Proposition 2b. Proof of heorem 2. First consider the case / 0. hen observe that p/d = o/. Also, after some algebra, starting from 9, using 27, log X n,n = d log Q U +,n and 38, we find log ˆq p,r,,n log q p = log + W α + p + D + p log D + + ˆα r,,n α nd log + p D + log + n ˆD ˆα r,,n + p log ˆD + + p D + + p D =: + log l l [ + U +,n D ] /α l l [ + p D ] /α 4 i= i r,,n. Using the mean value theorem we obtain that r,,n = W / + o p. α Next, using the result from heorem a, 2 r,,n = 2 α λ 2 N λ + o p +b l α ρ /α 2 + o p. 34

35 Furthermore using 27 and the fact that nu +,n / as, n, /n 0 and that p/d = o/ [ 4 r,,n = b l h ρ [ + ] /α h ρ [ + p ] ] /α + o p D = α b l + o p. Remains the evaluation of 3 r,,n. o this end note that log + n ˆD = log λ r, Rˆα r,,n r,,n λ r, We hence need to develop an asymptotic expansion for ˆα r,,n log R r,,n. heorem a, Proposition 3 and developing log + λ r, / + / 0 leads to ˆα r,,n log R r,,n α nd 2α 3/2 λ 2 N λ b l α ρ λ + O 2 b λ l α α +O p } λ { + b l ρ + O 2 α + 2 N λ λ + o p,. Using for where the last step follows from the assumption /2 = o/. From this we obtain that R α r,,n r,,n λ r, λ + O 2 ρ /α λb l + 2 N λ λ + o p, 35

36 from which log λ r, log λ r, Rˆα r,,n r,,n + O + O 2 b l ρ α + 2 λ 2 N λ + o p 2 + ρ α b l 2 λ 2 N λ + o p and log + n ˆD log + O 2 + ρ α b l 2 λ 2 N λ + o p. Furthermore is asymptotically equivalent to p/d + p/d log + pˆd log + p D D ˆD D = o ˆD and hence asymptotically negligible with respect to log + Using heorem a then leads to 3 n ˆD r,,n = O 2 + ρ α 2 b l + o 2 α λ 2 N λ + o p. log +. Combining the developments for i r,,n, i =,..., 4 leads to the stated result. Next, in the case /, starting from expression 20 and Proposition 36

37 b, we obtain log ˆq p,,,n log q p = α {log/u +,n logn/} + ˆα,,n α log/np + ˆα,,n α log + n ˆD,,n =: + α log + α { log + n ˆD + ˆD p + + ˆD,,n log p + + D p log + D U +,n + log l C /U +,n /α [ + D U +,n ] /α 6 i= Y i,n. l C /α p /α [ + D p ] /α } Using 27 and the fact that nu +,n / as, n, /n 0, one obtains Y 6,n ρ b C n/ /α. his can be derived using Lemma in de Haan and Ferreira 2006 replacing Ut by t /α l C t /α, at by α t/α l C t /α, ρ by ρ, At by b C t /α, and x = xt by /np /α. Next using the mean value theorem so that using 38 Y 5,n n α U +,n n nd + Y,n + Y 5,n W + α. Moreover using heorem c and / 0 Y 2,n + Y 3,n log np + D /p σ 2 0 α N b C n/ /α β0. 37

38 Remains Y 4,n. First remar that log + / log + D /p / D /p, which is o p log/np n / by assumption. Next, since α log + n ˆD + ˆD = α log np Rˆα,,n np,,n / p an asymptotic expansion of Rˆα,,n,,n is to be developed. Since R,,n = Q U +,n /Q U,n, with U j,n j =,..., n denoting the order statistics of an i.i.d. uniform 0, sample of size n as before, we have R,,n α = U,n l C /α U /α +,n [ + D U α +,n ] /α + D U,n U +,n l C /α U /α,n [ + D U,n ] /α + D U. +,n Now U,n /U +,n = d E /E E + where E,..., E + are i.i.d. standard exponentially distributed, so that U,n /U +,n d E if. Using similar arguments as in the proof of part a, we obtain that as, n, /n 0 and 0 l C /α U /α +,n [ + D U +,n ] /α l C /α U /α,n [ + D U,n ] /α + b C n/ /α h ρ α + ρ b C n/ /α and so that + D U,n + D U +,n + +, R α,,n = E + o + α ρ b C n/ /α + o. Moreover from heorem c we have ˆα,,n α = N 0 + b C n/ /α αβ0 + o p. Hence Rˆα,,n ˆα,,n /α,,n = R,,n α E + o = + α ρ b C n/ /α 38 ˆα,,n /α

39 and E /ˆα,,n/α = E E Finally we obtain that exp { } N 0 + b C n/ /α αβ0 log E { N 0 log E b C n/ /α log E αβ0 Rˆα,,n,,n = E + o + O p log / + Olog b C n/ /α. heorem 2b now follows from combining the different developments of the Y i,n i =,..., 6. }. Appendix 3. he effect of trimming on bias and variance of ˆα r,,n rimming the estimator ˆα r,,n decreases its efficiency with respect to the case r =. his is illustrated in Figure 8, plotting the functions σ 2 λ and βλ for λ [0, /4], from heorem c. References [] I.B. Aban and M.M. Meerschaert Generalized least squares estimators for the thicness of heavy tails. Journal of Statistical Planning and Inference 9, [2] I.B. Aban, M.M. Meerschaert, and A.K. Panorsa Parameter Estimation for the runcated Pareto Distribution, Journal of the American Statistical Association: heory and Methods 0473, [3] Beirlant, J., Vyncier, P. and eugels, J ail index estimation, Pareto quantile plots and regression diagnostics. J. Amer. Statist. Assoc., 9, [4] Beirlant, J., Goegebeur, Y., eugels, J. and Segers, J., Statistics of Extremes: heory and Applications, Wiley, UK. 39

40 Figure 8: Functions σ 2 λ and βλ for λ [0, /4]. [5] Clar, D.R A note on the upper-truncated Pareto distribution. In Proc. of the Enterprise Ris Management Symposium, April 22-24, Chicago IL. [6] Deers, A., Einmahl, J. and de Haan, L A moment estimator for the index of an extreme-value distribution. Ann. Statist., 7, [7] de Haan, L. and Ferreira, A., Extreme Value heory: an Introduction, Springer Science and Business Media, LLC, New Yor. [8] Hill, B.M A simple general approach about the tail of a distribution. Annals of Statistics 3, [9] Hubert, M., Diercx, G. and Vanpaemel, D Detecting influential data points for the Hill estimator in Pareto-type distributions. Computational and Statistical Data Analysis 65, [0] Nuyts, J Inference about the tail of a distribution: improvement on the Hill estimator. International Journal of Mathematics and Mathematical Sciences, Article ID

41 [] Seneta, E., 976. Regularly Varying Functions. Lecture Notes in Math. 508, Springer-Verlag, Berlin. [2] Weissman, I Estimation of parameters and large quantiles based on the largest observations. Journal of the American Statistical Association 73,