Minimax estimators of the coverage probability of the impermissible error for a location family

Transcription

1 Minimax estimators of the coverage probability of the impermissible error for a location family by Miguel A. Arcones Binghamton University arcones@math.binghamton.edu Talk based on: Arcones, M. A. (2008). Minimax estimators of the coverage probability of the impermissible error for a location family. Statistics & Decisions. 28, Available at 1

2 1 Confidence regions with a constrained Lebesgue measure for a location family. Let {X j } j=1 be a sequence of Rd valued i.i.d.r.v. s with a p.d.f. belonging to the family {f( θ) : θ R d }. Fix L > 0. We would like to find a translation equivariant confidence region C HL,L (X 1,..., X n ) such that: 1. For each x := (x 1,..., x n ), R m I(y C HL,L ( x)) dy L. (1) 2. For any confidence region C(X 1,..., X n ) satisfying (1), inf P θ{θ C HL,L ( X)} inf P θ{θ C( X)} θ R d θ R d 2

3 If C( X) is a translation equivariant confidence region, then C( X) = X n + C(Z 1,..., Z n 1, 0) = X n + C ( Z). We would like to find C ( Z) maximizing inf θ R d P θ {θ X n + C ( Z)} = P 0 { X n C ( Z)} (2) = I( x n C ( z)) n 1 j=1 f(z j + x n ) f(x n ) dx n d z = I(y C ( z)) n 1 j=1 f(z j y) f( y) dy d z, subject that for each z, I(y x n +C ( z)) dy = I(y C ( z)) dy L. (3) R m R m Conditioning on z, we get that the optimal region is C HL( z) = {y R d : λ HL ( z) where λ HL ( z) is such that It is easy to see that n 1 j=1 λ HL ( z) n 1 j=1 f(z j y) f( y)) f(z j y) f( y))}. dy = L. C HL,L (x 1,..., x n ) = x n + C HL ( z) = {θ R d : λ HL ( z) n j=1 f(x j θ)}. 3

4 Theorem 1. Suppose that: (i) For each 0 < M <, inf{f(x) : x M} > 0. (ii) lim x f(x) = 0. (iii) sup x R d f(x) <. Let C(X 1,..., X n ) be a confidence region for θ such that for each x 1,..., x n, R d I(y C(x 1,..., x n )) dy L. Then, inf θ R d P θ {θ C HL,L ( X)} inf θ R d P θ {θ C( X)}. 4

5 2 Impermissible error Let T n := T n ( X) be an estimator of θ. The error of the estimator T n is T n θ. Suppose that we select a number b, b > 0, such that any error of estimation bigger than b is impermissible. The maximum coverage probability of the impermissible error of the estimator T n is sup P θ { T n θ > b}. (4) θ Θ We would like to obtain an estimator T HL,b minimizing (4) for a location family. Finding T HL,b is equivalent to find a confidence interval of length 2b with coverage probability as large as possible. If d = 1, and C HL,2b ( X) is an interval, we take T n,hl,b ( X) as the middle point of C HL,2b ( X). By Theorem 1, for each estimator T n, sup P θ { T n θ > b} sup P θ { T n,hl,b θ > b}. θ R θ R 5

6 3 Estimation via the m.l.e. An mle ˆθ n of θ, over a parametric family {f(x, θ) : θ Θ} is a r.v. such that n n 1 log f(x j, ˆθ n ) = sup n 1 θ Θ j=1 n log f(x j, θ). (5) Suppose that there exists a r.v. ˆθn = ˆθ n (X 1,..., X n ) satisfying (5) when f(x, θ) = f(x θ). We may assume that ˆθ n is equivariant. This implies that ˆθ n θ is a pivotal quantity. Suppose that θ ˆθ n has a distribution absolutely continuous with respect to the Lebesgue measure, when θ obtains. Let h n ( ) be the pdf of θ ˆθ n when θ obtains. Let L > 0. Given C B(R d ), a confidence region for θ based on θ ˆθ n is determined by {θ R d : θ ˆθ n C} = {θ R d : θ ˆθ n + C}. j=1 6

7 Between all sets C such that I(θ ˆθ n + C) dθ = a set C which maximizes is P θ {θ ˆθ n C} = C mle,l = {x R d : λ mle,l h n (x)} where λ mle,l is such that {x R d :λ mle,l h n (x)} C C 1 dθ L, (6) h n (θ) dθ (7) 1 dx = L. Hence, using the mle, a confidence region satisfying (6) and maximizing (7) is ˆθ n + C mle,l = {θ R d : λ mle,l h n (θ ˆθ n )}. Notice h n (θ x), x R, is the p.d.f. of ˆθ n when θ obtains. 7

8 The obtained confidence regions maximize the coverage probability over all the regions with expected Lebesgue measure less or equal than L: Theorem 2. Let C(ˆθ n ) be a confidence region for θ such that Then, [ ] sup E θ I(y C(ˆθ n )) dy θ R d R d L. (8) inf P θ{θ C(ˆθ n )} inf P θ{θ ˆθ n + C mle,l }. (9) θ R d θ R d The previous theorem implies that if C(ˆθ n ) is a confidence region for θ such that for each x R d, then, (9) holds. R d I(y C(x)) dy L, (10) 8

9 Suppose that d = 1 and we would like to find the estimator T (ˆθ n ) (based on the m.l.e.) minimizing: sup P θ { T (ˆθ n ) θ > b}. (11) θ R If C mle,2b is an interval, then the middle point T n,mle,b of ˆθ n + C mle,2b minimizes (11) between all the estimators based on the m.l.e. 9

10 4 Asymptotic results. Let {X j } j=1 {f( θ) : θ R}. be a sequence of i.i.d.r.v. s from the family Theorem 3. Suppose that: (i) For each x R, f(x) > 0. (ii) log f( ) is a strictly concave function (iii) For each t R, there exists λ t > 0 such that Then, where E 0 [exp(λ t log f(x t) )] <. lim n n 1 log P θ {θ C HL,L (X 1,..., X n )} = S(L), Besides, S(L) = inf λ R log R (f(x L))λ (f(x)) 1 λ dx. (a) S is increasing on [0, ) and S is decreasing in (, 0]. (b) S is a continuous function. (c) lim t S(t) =. 10

11 Theorem 4. Suppose that: (i) For each x R, f(x) > 0. (ii) log f is a strictly concave function. (iii) R f (x) dx = 0. (iv) E[(f(X)) 2 (f (X)) 2 ] <. Then, the m.l.e. ˆθ n is well defined and θ ˆθ n, when θ obtains, has pdf ( n ) h n (t) := E 0 [I j=1 (f(x j + t)) 1 f (X j + t) > 0 ] n j=1 (f(x j)) 1 f (X j ), t R. Besides, h n is nonincreasing in [0, ) and nondecreasing in (, 0]. 11

12 Theorem 5. Suppose that: (i) For each x R, f(x) > 0. (ii) log f is a concave function. (iii) E 0 [(f(x)) 1 f (X)] = 0. (iv) E 0 [(f(x)) 2 (f (X)) 2 ] <. (v) ˆθ n θ satisfies the LDP with speed n and continuous rate function R(t) := inf λ R log E 0[exp(λ(f(X t)) 1 f (X t))], t R. Then, lim n n 1 log P θ {θ ˆθ n C mle,l (ˆθ n )} = S mle (L) where S mle (L) := inf{u 0 : x:r(x) u 1 dx L}. Besides, (i) For each t R, R(t) <. (ii) R(0) = 0 (iii) R is increasing in [0, ) and decreasing in (, 0]. (iv) R is continuous in R. (v) There exists a t 0 (0, L) such that R(t) = R(t 0 L) = S mle (L). 12

13 Example 1. If f(x) = (2π) 1/2 σ 1 exp( 2 1 σ 2 x 2 ), x R, (12) where σ > 0, then, it is easy to see that: (i) T HL,b = T mle,b = ˆθ n = X := n 1 n j=1 X j. (ii) C HL,2b (X 1,..., X n ) = C mle,2b (ˆθ n ) = [ X b, X + b]. (iii) S mle (2b) = S(2b) = R(b) = 2 1 σ 2 b 2. P (iv) For each θ R, T θ HL,b θ. 13

14 Example 2. If f(x) = (Γ(α)) 1 γ α α exp (αγx αe γx ) (13) where α > 0 and γ 0, then, it is possible to see that: (i) ˆθ ( n = γ 1 log n 1 ) n j=1 eγx j. (ii) C HL,2b (X 1,..., X n ) = C mle,2b (ˆθ n ) = [ˆθ n + γ 1 log ( 2 1 γ 1 b 1 (e 2γb 1) ) 2b, ˆθ n + γ 1 log ( 2 1 γ 1 b 1 (e 2γb 1) ) ]. (iii) T HL,b = T mle,b = ˆθ n +γ 1 log ( 2 1 γ 1 b 1 (e γb e γb ) ). (iv) For each b > 0, S(2b) = S mle (2b) = R(b) = α log(2 1 γ 1 b 1 (e 2 γ b 1)) α(e 2 γ b 1) 1 (e 2 γ b 1 2 γ b). (v) For each b > 0 and each θ R, P T θ HL,b θ + γ 1 log(2 1 γ 1 b 1 (e γb e γb )) θ. By Theorem 2 in Ferguson [1] the location families in examples 1 and 2 are the only one dimensional location families, which are exponential families. 14

15 Example 3. If f(x) = c exp( a 1 exp(τ 1 x) a 2 exp( τ 2 x)), (14) where a 1, a 2, τ 1, τ 2 > 0 and c makes f a p.d.f. Then, (i) ( n j=1 eτ 1X j, n j=1 e τ 2X j ) is a minimal sufficient stat. (ii) The family {f( θ) : θ R} is a curved exponential family. (iii) The m.l.e. ˆθ n of θ is (τ 1 + τ 2 ) 1 log (iv) where a 1 2 τ 1 2 a 1τ 1 n j=1 e τ 2X j 1 n j=1 e τ 1X j C HL,2b (X 1,..., X n ) = C mle,2b (ˆθ n ) = ˆθ n + [t 0 2b, t 0 ], (v) t 0 = (τ 1 + τ 2 ) 1 log ( τ 1 1 τ 2(e 2bτ 1 1)(1 e 2bτ 2 ) 1). T HL,b = T mle,b = ˆθ n +(τ 1 + τ 2 ) 1 log ( τ 1 1 τ 2(e bτ 1 e bτ 1)(e bτ 2 e bτ 2) 1). 15.

16 Example 4. If f(x) = 2 1 exp( x ), x R, (15) then, ˆθ n is not uniquely defined. is concave, but not strictly concave. The function log f( ) Let X (1),..., X (n) be the order statistics. X 1,..., X n are all different wit probability one. Assume that X 1,..., X n are all different. If n is odd, ˆθ n = X (2 1 (n+1)). If n is even, then ˆθ n = X (2 1 n) and ˆθ n = X (2 1 n+1) are both m.l.e. s. It is easy to see that whatever choice for the m.l.e. is made theorems 1 5 apply giving that (i) C HL,2b (X 1,..., X n ) C mle,2b (ˆθ n ). (ii) For each b > 0, S(2b) = b log(1 + b). (iii) For each b > 0, S mle (2b) = b 2 1 log ( 2e b 1 ). (iii) For each b > 0, S mle (2b) < S(2b). P (iv) For each θ R, T θ HL,b θ. 16

17 Theorem 6. Suppose that: (i) For each x R, f(x) > 0. (ii) log f is a concave function. (iii) f is even. Then, (i) For each b > 0, R(b) S(2b). (ii) Assume that f has a third derivative and that f is not a p.d.f. from the families of p.d.f. s in examples 1 and 3, then for some b 0, R(b) < S(2b). References [1] Ferguson, T. S. (1962). Location and scale parameters in exponential families of distributions. Ann. Mathemat. Statist