Goodness-of-fit testing and Pareto-tail estimation

Transcription

1 Goodness-of-fit testing and Pareto-tail estimation Yuri Goegebeur Department of Statistics, University of Southern Denmar, Jan Beirlant University Center for Statistics, K.U.Leuven, Tertius de Wet Department of Statistics and Actuarial Science, University of Stellenbosch,

2 Pareto-type distribution A distribution function F X is said to be of Pareto-type if for some γ > 0 1 F X (x) = x 1 γl F (x), x > 0, with l F a slowly varying function at infinity : l F (λx) l F (x) 1 as x, λ > 0. γ: extreme value index Notation: F X D(G γ ) (Max-domain of attraction of GEV distribution with index γ)

3 Define the tail quantile function U as ( U(x) = Q 1 1 ), x { = inf y : F X (y) 1 1 }, x > 1. x For Pareto-type distributions U(x) = x γ l U (x) x > 1 with l U again a slowly varying function at infinity.

4 X 1,...,X n i.i.d. sample, X 1,n... X n,n order statistics. Strict Pareto distribution so Pareto-quantile plot 1 F X (x) = x 1 γ, x > 1, log U(x) = γ log x, x > 1, resulting in the QQ-plot coordinates: ( log n + 1 ), log X n j+1,n, j = 1,...,n. j In case of good fit: linear with slope approximating γ. Exponential quantile plot of log-transformed data!

5 Pareto-type distribution log U(x) = γ log x + log l U (x) ( = log x γ + log l ) U(x) log x γ log x x. In case of a good fit by a Pareto-type model the Pareto quantile plot will be linear but only in the largest observations. Again the slope of the linear part is given by γ.

6 Pareto quantile plot - diamond data Value of 1914 diamonds obtained from a imberlite deposit log(value) Standard Exponential Quantile

7 Pareto quantile plot - Zaventem wind speed data Daily maximal wind speeds in Zaventem, Belgium log(wind Speed) Standard Exponential Quantile

8 A goodness-of-fit test for the exponential distribution X 1,...,X n i.i.d. Exp(λ) (exponential distribution with mean 1/λ). X 1,n... X n,n ascending order statistics. Jacson statistic (Jacson, 1967) T n = n j=1 t j,nx j,n n j=1 X j (1) where t j,n = λe(x j,n ) j 1 = n i + 1, i=1 j = 1,...,n. Correlation lie statistic based on exponential quantile plot.

9 Theorem 1. (Jacson, 1967) Assume X 1,...,X n i.i.d. variables, then for n Exp(λ) random n(tn 2) L N(0, 1).

10 A goodness-of-fit test for Pareto-type behavior X 1,...,X n i.i.d. according to some F X, with F X D(G γ ) Modified Jacson statistic T = 1 j=1 C j+1,z j 1 j=1 Z j where Z j = j(log X n j+1,n log X n j,n ) and C j+1, = 1 log j , j = 1,...,. Note: denominator is the Hill estimator (Hill, 1975) for γ, which is consistent for, n, /n 0.

11 Limiting distribution of T as,n, /n 0? Assumption 1. (R l ) There exists a real constant ρ < 0 and a rate function b satisfying b(x) 0 as x, such that for all λ 1, as x, l(λx) l(x) 1 1 b(x)λρ. ρ Theorem 2. (Beirlant, de Wet and Goegebeur, 2006) Assume X 1,...,X n i.i.d. random variables according to distribution function F X, where F X D(G γ ) for some γ > 0, l U satisfies R l, then as, n, /n 0 and b(n/) c, (T 2) L N ( ) cρ γ(1 ρ) 2,1.

12 Simulation: p5, p50 and p95 as a function of Pa(1) Burr - gamma=1, rho= Burr - gamma=1, rho= GPD - gamma=-0.5, sigma=

13 Bias-corrected Jacson statistic Approximate representation for log-spacings of order statistics (Beirlant, Diercx, Goegebeur and Matthys, 1999) or, equivalently, ( ) ρ D j Z j γ + b(n/) + ε j j = 1,...,, + 1 Z j b(n/) ( ) ρ j D γ + εj, j = 1,...,. + 1 This then motivates the following bias-corrected Jacson statistic T (ˆρ) = ( 1 j=1 C j+1, Z j ˆb ( ) ) ˆρ LS, (ˆρ) j +1 ˆγ LS, (ˆρ)

14 Simulation: p5, p50 and p95 as a function of Pa(1) Burr - gamma=1, rho= Burr - gamma=1, rho= GPD - gamma=-0.5, sigma=

15 Bias-corrected Jacson statistic: limiting distribution Theorem 3. (Beirlant, de Wet and Goegebeur, 2006) Assume X 1,...,X n i.i.d. random variables according to distribution function F X, where F X D(G γ ) for some γ > 0, l U satisfies R l and ˆρ is a consistent estimator for ρ, then as, n, /n 0 and b(n/) c, ( ( ) ( ) ) 2 L ρ T (ˆρ) 2 N 0,. 1 ρ Note: Whatever c, the normal limit is centered at 0, Compared to T, T (ˆρ) has a smaller asymptotic variance.

16 Diamond data T

17 Diamond data log(value) Standard Exponential Quantile

18 Windspeed data T

19 Generalization: ernel goodness-of-fit statistic Exponential case: goodness-of-fit statistics are quite often a ratio of two estimators for the exponential scale parameter. Pareto-type case: use log-transformed data and apply to the largest observations. General ernel statistic 1 H,n K j=1 ( ) j Z j, + 1 with K a ernel function satisfying 1 0 K(u)du = 0.

20 Theorem 4. (Goegebeur, Beirlant and de Wet, 2006) Consider X 1,...,X n i.i.d. random variables according to distribution function F X, where F X D(G γ ) for some γ > 0. Assume l U satisfies R l and let K(t) = 1 t t u(v)dv for some function 0 u satisfying j/ (j 1)/ u(t)dt f( j +1 ) for some positive continuous function f defined on (0,1) such that 1 0 log+ (1/w)f(w)dw <, 1 0 K(u) 2+δ dw < for some δ > 0 and 1 ( ) j=1 K j +1 0 as. Then as, n, /n 0 and b(n/) c, 1 H,n K j=1 ( ) ( j L c Z j N + 1 γ 1 0 K(u)u ρ du, 1 0 ) K 2 (u)du.

21 Some specific members Jacson statistic K J (u) = 1 log u. Bias-corrected Jacson statistic K BCJ (u) = 1 log u + 2ρ 1 ρ ( u ρ 1 ). 1 ρ

22 Lewis statistic T L = 1 j=1 j +1 Z j H,n. Proposition 1. Assume X 1,...,X n i.i.d. random variables according to distribution function F X, where F X D(G γ ) for some γ > 0 and l U satisfying R l. Then as, n, /n 0 and b(n/) c, (T L 0.5) = 1 H,n ( ) ( j L cρ K L Z j N + 1 2γ(1 ρ)(2 ρ), 1 ). 12 j=1 Note K L (u) = u 0.5.

23 Bias-corrected Lewis statistic T BCL (ˆρ) = ( 1 j j=1 +1 2) ( 1 Z j ˆb ( ) ) ˆρ LS, (ˆρ) j +1 ˆγ LS, (ˆρ) Kernel: K BCL (u; ρ) = u (1 ρ)(1 2ρ) 2ρ(2 ρ) ( u ρ 1 ). 1 ρ Proposition 2. Assume X 1,...,X n i.i.d. random variables according to distribution function F X, where F X D(G γ ) for some γ > 0 and l U satisfying R l with ρ 1. Then as, n, /n 0 and b(n/) c, 1 ˆγ LS, (ρ) j=1 ( ) ( j K BCL + 1 ;ρ L Z j N 0, 1 12 ( ) ) ρ. 2 ρ

24 Conclusion Goodness-of-fit testing for Pareto-type models Analogy with exponential distribution Need for bias-correction Lin ernel G.O.F. statistic - selection of for H,n Lin ernel G.O.F. statistic - estimation of ρ Goodness-of-fit testing for case γ = 0 and/or γ < 0?