Tail bound inequalities and empirical likelihood for the mean

Transcription

1 Tail bound inequalities and empirical likelihood for the mean Sandra Vucane 1 1 University of Latvia, Riga 29 th of September, 2011 Sandra Vucane (LU) Tail bound inequalities and EL for the mean / 21

2 Motivation Dealing with confidence intervals for mean for small samples sizes from highly skewed, heavy tailed probability distributions Rosenblum and van der Laan (2009) Claimed normal-based methods (z test, t test) and nonparametric bootstrap give poor coverage Provided alternative method using exponential type inequalities Idea: Compare the method proposed by Rosenblum and van der Laan with the empirical likelihood method. Sandra Vucane (LU) Tail bound inequalities and EL for the mean / 21

3 Bernstein s inequality X 1,..., X n independent, bounded r.v. P( X i EX i W ) = 1 v n D(X i) Bernstein s inequality (1927) For all x > 0 ( 1 n P n (X i EX i ) > x/ ) n ( 2 exp nx 2 ) 2(v + W nx/3) Alternatives: Bennett s inequality (1962), Hoeffding s inequality (1963), Berry-Esseen inequality (1941, 1942) Sandra Vucane (LU) Tail bound inequalities and EL for the mean / 21

4 Example - standard confidence intervals perform poorly Construction of random variable X is as follows: X = A + Y, A and Y are independent random variables, Y P ois(λ) P(A = 0) = 1 δ, P(A = t) = δ, where δ (0, 1) EX = EA + EY = tδ + λ, DX = DA + DY = t 2 δ(1 δ) + λ.!!!for simulations we choose δ = 0.01, t = 50, λ = 1. Sandra Vucane (LU) Tail bound inequalities and EL for the mean / 21

5 Example - standard confidence intervals perform poorly Sandra Vucane (LU) Tail bound inequalities and EL for the mean / 21

6 Example - standard confidence intervals perform poorly Table: Coverage accuracy for the mean of X from Example, α = 0.05 t B perc B norm B basic B stud Bernst n = n = n = n = n = n = Sandra Vucane (LU) Tail bound inequalities and EL for the mean / 21

7 Empirical likelihood method (EL) Empirical likelihood method was introduced by Art B. Owen in The idea - model data using distributions placing point masses on the data. L(F ) = n P (X = X i ) = n p i, n p i = 1.!!! Empirical likelihood method is the only nonparametric method that admits Bartlett adjustment. Sandra Vucane (LU) Tail bound inequalities and EL for the mean / 21

8 Empirical likelihood X 1,..., X n iid with EX i = µ 0 R. g(x, µ) such that E {g(x, µ)} = 0 (g(x i, µ) = X i µ); Empirical likelihood for µ : n L(µ) = P (X = X i ) = n L(µ) is maximized subject to constraints: p i p i 0, i p i = 1, i p i g(x i, µ) = 0. Sandra Vucane (LU) Tail bound inequalities and EL for the mean / 21

9 Empirical likelihood Empirical likelihood ratio statistic for µ R(µ) = L(µ) n L(ˆµ) = {1 + λ(µ)g(x i, µ)} 1, Theorem (Owen, 1988) X 1,..., X n i.i.d. with µ 0 <. Then W (µ 0 ) = 2 log R(µ 0 ) d χ 2 1. Sandra Vucane (LU) Tail bound inequalities and EL for the mean / 21

10 Bartlett correction Bartlett correction Simple correction of W (µ 0 ) with its mean E{W (µ 0 )} reduces coverage error from O(n 1 ) to O(n 2 ). Edgeworth series Approximate a probability distribution in terms of its cumulants. X 1,..., X n i.i.d. with mean θ 0 and finite variance σ 2. S n = n 1/2 (ˆθ θ 0 )/σ, where ˆθ = X. Edgeworth expansion for S n P(S n x) = Φ(x) + n 1/2 p 1 (x)φ(x) + n 1 p 2 (x)φ(x) Sandra Vucane (LU) Tail bound inequalities and EL for the mean / 21

11 Example - standard confidence intervals perform poorly t B perc B norm B basic B stud Bernst n = n = n = n = n = n = Sandra Vucane (LU) Tail bound inequalities and EL for the mean / 21

12 Example - standard confidence intervals perform poorly t B perc B norm B basic B stud Bernst n = n = n = n = n = n = t EL EL B EL F EL F B Bernst n = n = n = n = n = n = Sandra Vucane (LU) Tail bound inequalities and EL for the mean / 21

13 Example: Lognormal distribution LogN(µ, σ 2 ) σ 2 = 0.5 σ 2 = 1 Sandra Vucane (LU) Tail bound inequalities and EL for the mean / 21

14 Example: Lognormal distribution LogN(µ, σ 2 ) σ 2 = 2 σ 2 t EL EL B EL F EL F B B perc B norm B basic B stud Bernst n = n = Sandra Vucane (LU) Tail bound inequalities and EL for the mean / 21

15 Summary for independent case Bernstein s method can be recommended for very sparse data usually with big variance Coverage accuracy for larger samples grow to 1, that makes the comparison more difficult For any distribution we can find such parameter W and v values that the coverage accuracy of Bernstein s method is 95%, but these parameter values usually don t satisfy assumptions Sandra Vucane (LU) Tail bound inequalities and EL for the mean / 21

16 Dependent case 1 Empirical likelihood Kitamura (1997) - empirical likelihood for weakly dependent processes Kitamura (1997) - Bartlett correction for blockwise empirical likelihood of the smooth function Nordman et al. (2007) - empirical likelihood for the mean of long-range dependent processes 2 Exponential type inequalities Bosq (1996) Dedecker, J., Doukhan, P., Lang, G., León R., J. R., Louhichi, S. and Prieur, C. (2007) Work in progress: comparison for weakly dependent processes Sandra Vucane (LU) Tail bound inequalities and EL for the mean / 21

17 Empirical likelihood for weakly dependent processes {X t } R d valued stationary stochastic process that satisfy strong mixing condition α X (k) 0 as k ; Parameter of interest θ 0 Θ R p, r p; Estimating equation f : R d Θ R r and E {f(x t, θ 0 )} = 0; M length of block, L block number Q = [(N M)/L] + 1 and A N = QM/N; T i (θ) = φ M (B i, θ), where B i is the ith block of observations and mapping φ M : R rm Θ R r has following form φ M (B i, θ) = M f(x (i 1)L+n, Θ)/M Sandra Vucane (LU) Tail bound inequalities and EL for the mean / 21

18 Empirical likelihood for weakly dependent processes Empirical likelihood for θ : L(θ) = Q L(θ) is maximized subject to constraints: p i p i > 0, i p i = 1, i p i T i (θ) = 0. Blockwise empirical log-likelihood ratio for θ Q 2 log (L(θ)/L F ) = 2 log (1 + γ N (θ) T i (θ)). Theorem (Kitamura, 1997) Q 1 LR 0 = 2A N log (1 + γ N (θ) T i (θ)) d χ 2 r p. Sandra Vucane (LU) Tail bound inequalities and EL for the mean / 21

19 Simulations AR(1) with X t = 0.3X t 1 + ε t N/M Sandra Vucane (LU) Tail bound inequalities and EL for the mean / 21

20 Exponential type inequalities Bosq (1996) Let (X t, t Z) be a zero-mean real-valued process such that sup 1 t n X t b. Then for each integer q [1, n 2 ] and each ε > 0 ( ) ( P( S n > nε) 4 exp ε2 q 8b b ) 1/2 ([ ]) n qα. ε 2q Doukhan (1995) Let (X t, t N) be a zero-mean real-valued process such that σ 2 R +, 1 n, m N : m (X n X n+m ) 2 σ 2 and t N : X t M. Then for each ε > 0, θ = ε2 4 and q ( n 1+θ (1 ε)x 2 ) P( S n x) 4 exp 2(nσ n β( qθ 1) q. + qmx/3)!!!problem - estimation of mixing coefficients McDonald et al (2011)-method for estimating beta-mixing coefficients Sandra Vucane (LU) Tail bound inequalities and EL for the mean / 21

21 Exponential type inequalities Bennett-type inequality Let (X i ) i>0 such that X i, and M i = σ(x k, 1 k i). Let S k = k (X i E(X i )) and S n = max 1 k n S k. Let q be some positive integer, v q some nonnegative number such that [n/q] v q X q[n/q] X n X (i 1)q X iq 2 2 and h(x) = (1 + x) log (1 + x) x. Then for λ > 0, ( P( S n 3λ) 4 exp v ( )) q λqm (qm) 2 h + n λ τ 1,q(q + 1).!!!Problem - estimation of mixing coefficient v q Sandra Vucane (LU) Tail bound inequalities and EL for the mean / 21

22 Thank you for your attention! Sandra Vucane (LU) Tail bound inequalities and EL for the mean / 21