Empirical likelihood-based methods for the difference of two trimmed means

Transcription

1 Empirical likelihood-based methods for the difference of two trimmed means Latvijas Universitate

2 Contents 1 Introduction 2 Trimmed mean 3 Empirical likelihood 4 Empirical likelihood for the trimmed mean 5 Simulation study 6 Conclusions and further work

3 Introduction Introduction Owen (1988) - empirical likelhood (EL) method for the mean and certain M-estimators Valeinis (2007, 2010, 2011) - general method for EL in two-sample case ROC curves, P-P and Q-Q plots differences of two sample means, quantiles, location-scale models R package EL (in collaboration with Cers) based on smooth estimating equations for EL method Valeinis, Velina, Luta (ICORS 2011); Velina (MSc Thesis 2012) - a robust EL method for the difference of two smooth Huber estimators Qin and Tsao (2002) - EL method for the trimmed mean Goal - to establish (a robust) EL method for the difference of two trimmed means

4 Trimmed mean Trimmed mean Aim: a robust estimator for location parameter Classical statistics constructed to be optimal at some model (e.g., at normal distribution) even small deviations from the model can badly distort the statistical inferencene (large variance, large bias). Robust statistics derive methods that produce reliable parameter estimates, tests and confidence intervals even if the model holds approximately (outliers in data, gross errors) methods lose some efficiency at the model

5 Trimmed mean Trimmed mean Let X 1, X 2,..., X n i.i.d., X 1 F 0, and X (1), X (2),..., X (n) be ordered statistics. Definition µ αβ = 1 ξ1 β xdf 0, 1 α β ξ α where 0 < α < 1/2, 0 < β < 1/2 are trimming proportions, and ξ p := F 1 0 (p) for any 0 p 1. µ αβ is the mean of the truncated distribution: 0, t < ξ α F F T (t) = 0 (t) α 1 β α, ξ α t ξ 1 β (1) 1, t > ξ 1 β

6 Trimmed mean The sample trimmed mean Definition X α,β = 1 m s X (i), where 0 < α < 1/2, 0 < β < 1/2 are trimming proportions, r = [nα] + 1, s = n [nβ], m = n [nα] [nβ]. i=r The assymptotic value of X αβ is the trimmed mean µ αβ. if trimming proportion too small, VAR( X α,β ) can be drastically inflated by outliers or sampling from heavy-tailed distributons, if trimming proportion too big, standard error might be too large under the normal distribution. common optimal choice: set α = β =: γ = 0.1.

7 Trimmed mean Determining the trimming empirically Adaptive trimmed means: choose γ according to some criterion, e.g., standard error: Estimate standard error of X γ,γ for, e.g., γ = 0, 0.1 and 0.2, Chosse the γ corresponding to the estimate with the smallest standard error. Other possible criteria: some measure of skewness or heavy-tailedness.

8 Trimmed mean Estimating the standar error of the trimmed mean A common mistake: Estimate the standard error of the sample mean, s/ n, where s 2 = 1 n 1 n (X i X ) 2, Estimate the standard error of the trimmed mean as s/ m, where m = n [nα] [nβ]. i=1

9 Trimmed mean Estimating the standar error of the trimmed mean A common mistake: Estimate the standard error of the sample mean, s/ n, where s 2 = 1 n 1 n (X i X ) 2, Estimate the standard error of the trimmed mean as s/ m, where m = n [nα] [nβ]. WRONG! i=1

10 Trimmed mean Estimating the standar error of the trimmed mean (approximately) Winsorized random sample X (r), X i X (r), W i = X i, X (r) < X i < X (s), X (s), Xi X (s). Winsorized mean X w = 1 n n i=1 W i, sample Winsorized variance s 2 w = 1 n 1 n i=1 (W i W i ) 2, standard error of the trimmed mean s w (1 α β) n.

11 Trimmed mean Example - trimming and windsoring Table 1. Self-Awareness data (Dana, 1990) Table 2. Winsorized Self-Awareness data, γ = Table 3. Estimators for Self-Awareness data γ Trimmed Winsorized Winsorized stdev s.e. of the Trimmed

12 Empirical likelihood Empirical (nonparametric) likelihood Definition Let X 1, X 2,..., X n i.i.d with unknown F 0. For distribution F the nonparametric empirical likelihood function is L(F ) = n (F (X i ) F (X i )) = i=1 where p i = P(X = X i ) and n i=1 p i = 1. n p i, i=1 L(F) is maximized by ECDF F n with p i = 1/n, Idea: to express the parameter of interest θ as a functional from F, i.e., θ = θ(f ).

13 Empirical likelihood EL in general one sample case Estimating function w(x, θ), θ Θ R p with E F0 {w(x, θ)} = 0, Profile empirical likelihood ratio n n n R(µ) = sup{ np i : p i 0, p i = 1, p i w(x i, θ) = 0}. i=1 i=1 i=1 for the mean θ 0 = µ 0, take w(x i, θ) = X i µ. Theorem (Qin and Lawless, 1994) Under some (smoothness) conditions on w(x, θ), 2 log R(µ 0 ) d χ 2 1.

14 Empirical likelihood for the trimmed mean EL method for the trimmed mean (Qin and Tsao, 2002) EL method is established for independent observations, while trimmed sample consists of dependent observations. Define the EL ratio for the trimmed sample. Let weights p i = 0 for i < r and i > s, p i 0 for r i s and s i=r p i = 1. EL estimating equation: W (i) = X (i) µ αβ for r i s. Empirical likelihood ratio s s s R(µ αβ ) = sup{ (mp i ) : p i 0, p i = 1, p i W (i) = 0}. i=r i=r i=r

15 Empirical likelihood for the trimmed mean EL method for the trimmed mean (Qin and Tsao, 2002) Note: a fact from ordered statistics ( n 1 s m i=r (X ) d (i) µ αβ ) N(0, τ 2 αβ ), 1 m s i=r (X (i) µ αβ ) 2 p σ 2 αβ. Theorem (Qin and Tsao, 2002) Let µ 0 αβ be the true value of µ αβ. Then a [ 2 log R(µ 0 αβ )] d χ 2 1, where a = σ 2 αβ /((1 α β)τ 2 αβ ).

16 Empirical likelihood for the trimmed mean For a known distribution F 0, σαβ 2 and τ αβ 2 are given in closed form σαβ 2 = 1 ξ1 β x 2 df 0 (x) µ 2 αβ (1 α β), (2) ξ α τ 2 αβ = 1 (1 α β) 2 ((1 α β)σ2 αβ + β(1 β)(ξ 1 β µ αβ ) 2 2αβ(ξ α µ αβ )(ξ 1 β µ αβ ) + α(1 α)(ξ α µ αβ ) 2 ). (3) Alternatively, estimate ˆσ 2 αβ and ˆτ 2 αβ from the data.

17 Empirical likelihood for the trimmed mean Scaling constants - example Table 1. Values of the scaling constant a, for N(0, 1) Trimming level σ 2 τ 2 a 20% % % % % % Qin and Tsao (2002): empirical likelihood-based ratio interval is more accurate than the normal approximation based interval in situations where the distribution of interest is contaminated on one side.

18 Empirical likelihood for the trimmed mean Empirical likelihood in general two sample case X 1,... X n1, Y 1,... Y n2 i.i.d. with unknown F 1, F 2. is univariate parameter of interest and θ 0 is a nuisance parameter associated with F 1. Profile EL ratio E F1 w 1 (X, θ 0, ) = 0, E F2 w 2 (Y, θ 0, ) = 0, n 1 n 2 R(, θ) = sup (n 1 p i ) (n 2 q j ), θ,p,q i=1 j=1 where p i, q j 0, n 1 i=1 p i = 1, n 2 i=1 q j = 1 and n 1 i=1 p i w i (X i, θ, ) = 0, n 2 j=1 q j w 2 (Y j, θ, ) = 0.

19 Empirical likelihood for the trimmed mean To find p i, q j, it is necessary to solve the equation system n 1 i=1 n 1 i=1 n 2 j=1 w 1 (X i, θ, ) 1 + λ 1 (θ)w 1 (X i, θ, ) = 0, w 2 (Y j, θ, ) 1 + λ 2 (θ)w 2 (Y j, θ, ) = 0, n2 λ 1 (θ)α 1 (X i, θ, ) 1 + λ 1 (θ)w 1 (X i, θ, ) + j=1 λ 2 (θ)α 2 (Y j, θ, ) 1 + λ 2 (θ)w 2 (Y j, θ, ) = 0. α 1 and α 2 are the derivatives of w 1 and w 2 with respect to θ.

20 Empirical likelihood for the trimmed mean Theorem (Valeinis, 2010, 2011) Under some conditions (in particular, differentiability) on w 1 (X, θ, ), w 2 (Y, θ, ), 2 log R( 0, ˆθ) d χ 2 1.

21 Empirical likelihood for the trimmed mean Empirical likelihood for the difference of trimmed means Parameter of interest = µ 2 α 1 β 1 µ 1 α 2 β 2, µ 1 α 1 β 1 and µ 2 α 2 β 2 are the assymptotic values of trimmed means of samples X and Y respectively, w 1 (X, µ 1 α 1 β 1, ) = X µ 1 α 1 β 1, w 2 (Y, µ 1 α 1 β 1, ) = Y + µ 1 α 1 β 1. EL ratio for trimmed means s 1 s 2 R(, µ 1 αβ ) = sup { (m 1 p i ) (m 2 p i )}, p i,q j, i=r 1 i=r 2 where p i, q j 0, s 1 i=r 1 p i = 1, s 2 i=r 2 q j = 1 and s 1 i=r 1 p i w i (X i, θ, ) = 0, s 2 j=r 2 q j w 2 (Y j, θ, ) = 0.

22 Empirical likelihood for the trimmed mean Theorem (Velina, Valeinis, 2012) where a ( 2 log R( 0, ˆµ 1 αβ )) d χ 2 1, a = 1 m 2 σ1 2 + m 1σ2 2 (1 α 1 β 1 )(1 α 2 β 2 ) (n 2 τ1 2 + n 1τ2 2). σ1 2 := σ2 1α 1 β 1, τ1 2 := τ 1 2 α 1 β 1 are associated with sample X, σ2 2 := σ2 2α 2 β 2, τ2 2 := τ 2 2 α 2 β 2 are associated with sample Y, and defined as in (2) and (3).

23 Simulation study Simulation setting Test the empirical coverage accuracy of the EL-based confidence interval of the difference of two trimed means 1 Models Two samples without contamination: X N(0, 1), Y N(0, 1) Add contamination to one of the samples, X N(0, 1), Y (1 ɛ)n(0, 1) + ɛn(20, 0.01), Add contamination to both samples, X (1 ɛ)n(0, 1) + N(20, 0.01), Y (1 ɛ)n(0, 1) + ɛn(20, 0.01), For simplicity, generate samples with equal size n 1 = n 2 set equal trimming proportions α 1 = β 1 = α 2 = β 2 2 Methods tested two-sample t-test Bootstrap-t (cf. R. Wilcox, R function yuenboot()) EL for trimmed means, trimming = 10%, EL for trimmed means, trimming = 20%.

24 Simulation study Table 2. 95% empirical coverage accuracy of the confidence interval X N(0, 1) and Y N(0, 1) t-test Bootstr20% Bootstr10% EL.Trim.20% EL.Trim.10% n1=n2 acc len acc len acc len acc len acc len

25 Simulation study Table 3. 95% empirical coverage accuracy of the confidence interval X (1 ɛ)n(0, 1) + ɛn(20, 0.01) and Y (1 ɛ)n(0, 1) + ɛn(20, 0.01), ɛ = 0.05 t-test Bootstr20% Bootstr10% EL.Trim.20% EL.Trim.10% n1=n2 acc len acc len acc len acc len acc len

26 Simulation study Table 4. 95% empirical coverage accuracy of the confidence interval X N(0, 1) and Y (1 ɛ)n(0, 1) + ɛn(20, 0.01), ɛ = 0.05 t-test Bootstr20% Bootstr10% EL.Trim.20% EL.Trim.10% n1=n2 acc len acc len acc len acc len acc len

27 Simulation study Limiting and simulated distributions Figure 1. Simulated and limiting 2log(R(, µ 1 1β 1 )) distribution at X N(0, 1), Y (1 ɛ)n(0, 1) + ɛn(20, 0.01). (a) trimming = 20%, (b) trimming = 10%. Density Density el.est el.est

28 Conclusions and further work Conclusions and further work Conclusions 1 Newly established EL method for the difference of trimmed means works reasonably well in simulation setting with no contamination 2 Method has significant advantages over two sample t-test and works similarly to bootstrap method in situations with contaminated data 3 It is important to choose the correct trimming level Further work 1 Evaluate the performance of the method within real data examples and more simulation examples 2 Include the procedure for the difference of trimmed means in the available R package EL

29 Conclusions and further work References [1] Owen, A.B. (1988). Empirical likelihood ratio confidence intervals for a single functional, Biometrika, 75, [2] Qin, G. and Tsao, M. (2002) Empirical likelihood confidence interval for the trimmed mean, Communications in statistics, Theory and Methods, 31 (12), p [3] Valeinis, J., Cers, E., and Cielens, J. (2010). Two-sample problems in statistical data modelling, Mathematical Modelling and Analysis, 15, [4] Wilcox, R.R. (2005). Introduction to Robust Estimation and Hypothesis Testing, Academic Press; 2nd edition.