Statistics 581, Problem Set 8 Solutions Wellner; 11/22/2018

Transcription

1 Statistics 581, Problem Set 8 Solutions Wellner; 11// (a) Show that if θ n = cn 1/ and T n is the Hodges super-efficient estimator discussed in class, then the sequence n(t n θ n )} is uniformly square-integrable. (b) Let R n (θ) ne θ (T n θ) where T n is the Hodges superefficient estimator as in Example (so T n = δ n of Example.5, Lehmann and Casella pages ). Show that R n (n 1/4 ) as n. Solution: (a) First recall that (with δ n = T n ) since n(x θ) d = Z N(0, 1) we can write n(tn θ) = n(x n 1 [ Xn >n 1/4 ] + ax n1 [ Xn n 1/4 ] θ) d = Z1 [ Z+θ n >n 1/4 ] + [az + nθ(a 1)]1 [ Z+θ n n 1/4 ] = Z + [(a 1)Z + (a 1) nθ]1 [ Z+θ n n 1/4 ] = Z (1 a)[z + nθ]1 [ Z+θ n n 1/4 ]. Thus (as we showed in class) when θ n = cn 1/ we have n(tn θ n ) d = Z1 [ Z+c >n 1/4 ] + [az + c(a 1)]1 [ Z+c n 1/4 ] = Z + [(a 1)Z + (a 1)c]1 [ Z+c n 1/4 ] = Z (1 a)[z + c]1 [ Z+c n ]. 1/4 (1) Thus where Thus Y n n(t n θ n ) } d = Z (1 a)[z + c]1 [ Z+c n 1/4 ] ( Z + (1 a) (Z + c) ) Y E(Y ) = ( E(Z ) + (1 a) E(Z + c) ) <. } lim sup EY n 1 [Yn λ]} EY 1 [Y λ] } 0 n as λ. Hence Y n } is uniformly integrable; that is, n(t n θ n )} is uniformly square-integrable. (b) (a ) Note that the identity (1) in (a) above holds. Thus b n (θ) = E θ (T n ) θ = n 1/ EZ (1 a)e[z + } nθ]1 [ Z+θ n n 1/4 ] = 1 a E[Z + nθ]1 n [ Z+θ n n 1/4 ] = 1 a n n 1/4 n 1/4 xφ(x nθ)dx 1

2 since Z + θ n N(θ n, 1). (b ) Differentiating the result in (a ) gives b n(θ) = 1 a n n 1/4 n 1/4 xφ (x nθ)( n) dx n 1/4 = (1 a) x(x nθ)φ(x nθ) dx n 1/4 0 if θ 0 since φ (x) = xφ(x) by the dominated convergence theorem since x(x nθ)φ(x nθ)1 [ n 1/4,n 1/4 ](x) 0 for each fixed x and is dominated by the integrable function 4e 1 φ(x)/( θ 1) (for n (3/ θ ) 4 ). Details of this domination: For x n 1/4 it follows that while x x nθ n 1/4 n 1/4 nθ n 1/ + n 3/4 θ n 3/4 ( θ 1) φ(x nθ) = φ(x) exp( nθx nθ /) φ(x) exp( θ n 3/4 nθ /) = φ(x) exp( θ n 3/4 (1 n 1/4 θ /)) φ(x) exp( 1 θ n3/4 ) if 1 n 1/4 θ / < 1/ or, equivalently, when n > (3/ θ ) 4. Combining these two bounds yields x x nθ φ(x nθ) φ(x)n 3/4 ( θ 1) exp( θ n 3/4 /) n = φ(x) 3/4 exp( θ n 3/4 /) if θ < 1 n 3/4 θ exp( θ n 3/4 /) if θ 1 (4/ θ )(n = φ(x) 3/4 θ /) exp( θ n 3/4 /) if θ < 1 4(n 3/4 θ /) exp( θ n 3/4 /) if θ 1 4e 1 θ 1 φ(x). (b), Second (more elegant) solution: from the lecture notes, 3.3 (3), it follows that R n (θ) = E[n(T n θ) ] = nv ar[t n ] + nb n (θ) a + nb n (θ). Using the formula for b n (θ) from part (a) above, it follows that it is enough to show that n 1/4 xφ(x n 1/4 )dx n. 1/4 But we have, with Z N(0, 1) (and hence E Z < ), n 1/4 xφ(x n 1/4 0 )dx = n (y + n 1/4 )φ(y)dy 1/4 n 1/4

3 0 0 n1/4 φ(y)dy yφ(y)dy n 1/4 n 1/4 n 1/4 (Φ(0) Φ( n 1/4 )) E Z.. (Super-efficiency at two parameter values) Suppose that X 1,..., X n are i.i.d. N(θ, 1) where θ R) Let a, b [0, 1) and define the estimator T n as follows: X n if X n 1 > n 1/4 and X n + 1 > n 1/4, T n = ax n + (1 a) if X n 1 n 1/4, bx n + (1 b)( 1) if X n + 1 n 1/4. (a) Find the limiting distribution of n(t n θ) when: (i) θ 1 and θ 1; (ii) θ = 1; (iii) θ = 1. (b) Find the limiting distribution of n(t n θ n ) when: (i) θ n = 1 + cn 1/ ; (ii) θ n = 1 + cn 1/. (c) Could we have super-efficiency at a countable collection of parameter values? Solution: (a) Note that n(x n θ) d = Z N(0, 1) for all θ R and n N. Thus we find that n(tn θ) = n(x n θ)1 [ Xn 1 >n 1/4 ] 1 [ X n+1 >n 1/4 ] where + n(ax n + (1 a) θ)1 [ Xn 1 n 1/4 ] + n(ax n (1 b) θ)1 [ Xn+1 n 1/4 ] d = Z 1 [ n Xn θ+θ 1 >n 1/4 ] 1 [ n X n θ+θ+1 >n 1/4 ] d + a n(x n θ) + n(aθ θ + (1 a)) } 1 [ n(xn θ)+ n(θ 1) n 1/4 ] + b n(x n θ) + n(bθ θ (1 b)) } 1 [ n(xn θ)+ n(θ+1) n 1/4 ] Z if θ 1, θ 1, az if θ = 1, bz if θ = 1, N(0, V (θ)) V (θ) = 1 1,1} (θ) + a 1 1} (θ) + b 1 1} (θ). (b) If θ = θ n = 1 + cn 1/, d n(tn θ n ) = Z1 [ Z+c >n 1/4 ] + (az + c(a 1))1 [ Z+c n 1/4 ] + o p (1) d az + c(a 1) N(c(a 1), a ). In the same way, if θ = θ n = 1 + cn 1/, we find that n(tn θ n ) d bz + c(b 1) N(c(b 1), b ). (c) A similar construction works to yield superefficiency at all θ Z = 0, ±1, ±,...}. 3

4 3. Suppose that X 1,..., X n are i.i.d. with distribution function F having a continuous density function f. Let F n be the empirical distribution function of the X i s, suppose that b n is a sequence of positive numbers, and let ˆf n (x) = F n(x + b n ) F n (x b n ) b n. (a) Compute E ˆf n (x)} and V ar( ˆf n (x)). (b) Show that E ˆf n (x) f(x) if b n 0. (c) Show that V ar( ˆf n (x)) 0 if b n 0 and nb n. (d) Use some appropriate central limit theorem to show that (perhaps under some suitable further conditions that you might need to specify) nbn ( ˆf n (x) E ˆf n (x)) d N(0, f(x)). Hint: Write ˆf n (x) in terms of some Bernoulli random variables and identify p = p n. Solution: (a) First note that nb n = n(f n (x+b n ) F n (x b n )) is a Binomial(n, p n ) random variable with p n = F (x + b n ) F (x b n ). Hence if b n 0 E f n (x) = F (x + b n) F (x b n ) = p n b n nb n = 1 F (x + bn ) F (x) b n + F (x) F (x b n) b n } 1 f(x) + f(x)} = f(x). (b) Furthermore V ar( f n (x)) = np n(1 p n ) (nb n ) 1 p n = (1 p n ) nb n b n 0 f(x) 1 = 0 if nb n and b n 0. (c) Since nb n fn (x) = n i=1 X ni where X ni Bernoulli(p n ), it follows that σni = p n (1 p n ) so that σn = Var( n i=1 X ni) = np n (1 p n ), and γ n n γ ni = i=1 n E X ni µ ni 3 i=1 = np n (1 p n )(1 p n ) + p n} np n (1 p n ) so that γ n /σ 3 npn (1 p n ) = nbn (p n /b n )(1 p n ) 0 4

5 if b n 0 and nb n. Thus, by the Liapunov CLT, nb n ( f n (x) E f(x)) npn (1 p n ) N(0, 1) if b n 0 and nb n. Thus nbn ( f n (x) E f n (x)) = nb n( f n (x) E f n (x)) npn (1 p n ) N(0, 1) f(x) = N(0, f(x)). np n (1 p n ) nb n 4. Suppose that (T Z) Weibull(λ 1 e γz, β), and Z G η on R with density g η with respect to some dominating measure µ. Thus the conditional cumulative hazard function Λ(t z) is given by and hence (Recall that λ(t) = f(t)/(1 F (t)) and Λ(t) t 0 Λ γ,λ,β (t z) = (λe γz t) β = λ β e βγz t β λ(s)ds = λ γ,λ,β (t z) = λ β e βγz βt β 1. t 0 (1 F (s)) 1 df (s) = log(1 F (t)) if F is continuous.) Thus it makes sense to re-parametrize by defining θ 1 βγ (this is the parameter of interest since it reflects the effect of the covariate Z), θ λ β, and θ 3 β. This yields You may assume that λ θ (t z) = θ 3 θ exp(θ 1 z)t θ 3 1 a(z) ( / η) log g η (z) exists and Ea (Z)} <. Thus Z is a covariate or predictor variable, θ 1 is a regression parameter which affects the intensity of the (conditionally) Weibull variable T, and θ = (θ 1, θ, θ 3, θ 4 ) where θ 4 η. (a) Derive the joint density p θ (t, z) of (T, Z) for the re-parametrized model. (b) Find the information matrix for θ. What does the structure of this matrix say about the effect of η = θ 4 being known or unknown about the estimation of θ 1, θ, θ 3? (c) Find the information and information bound for θ 1 if the parameters θ and θ 3 are known. (d) What is the information bound for θ 1 if just θ 3 is known to be equal to 1? (e) Find the efficient score function and the efficient influence function for estimation of θ 1 when θ 3 is known. (f) Find the information I 11 (,3) and information bound for θ 1 if the parameters θ and θ 3 are unknown. (Here both θ and θ 3 are in the second block.) (g) Find the efficient score function and the efficient influence function for estimation 5

6 of θ 1 when θ and θ 3 are unknown. (h) Specialize the calculations in (d) - (g) to the case when Z Bernoulli(θ 4 ) and compare the information bounds. Solution: (a) Integrating λ θ (t z) with respect to t gives Λ θ (t z) = θ exp(θ 1 z)t θ 3, and hence the conditional survival function 1 F θ (t z) is given by 1 F θ (t z) = exp( Λ θ (t z)) = exp( θ exp(θ 1 z)t θ 3 ). () It follows that and hence that f θ (t z) = θ θ 3 e θ 1z t θ 3 1 exp( θ e θ 1z t θ 3 ), p θ (y, z) = f θ (y z)g η (z) = θ θ 3 e θ 1z t θ 3 1 exp( θ e θ 1z t θ 3 )g η (z) = = θ θ 3 e θ 1z t θ 3 1 exp( θ e θ 1z t θ 3 )g θ4 (z). (b) We first calculate the scores for θ. Note that the random variable W θ exp(θ 1 Z)Y θ 3 has, conditionally on Z, a standard Exponential(1) distribution: by (). We calculate l(θ Y, Z) = log p θ (Y, Z) P θ (W > w Z) = P θ (θ exp(θ 1 Z)Y θ 3 > w Z) = e w = log θ + log θ 3 + θ 1 Z + (θ 3 1) log Y θ e θ1z Y θ 3 + log g θ4 (Z), l 1 (Y, Z) = Z Zθ e θ1z Y θ 3 = Z(1 W ), l (Y, Z) = 1 θ θ e θ 1Z Y θ 3 θ = 1 θ (1 W ), l 3 (Y, Z) = 1 θ 3 + log Y θ e θ 1Z Y θ 3 log Y = 1 + log Y 1 θ e θ1z Y θ 3 } θ 3 = log θ } e θ1z Y θ 3 θ 3 θ e 1 W } θ 1Z = 1 θ log W log(θ e θ 1Z )}1 W } } = 1 θ 3 [1 (W 1) log W ] + (W 1) log(θ e θ 1Z ) } l 4 (Y, Z) = a(z) = a(z, η). Moreover, l13 (Y, Z) = Zθ e θ1z Y θ 3 log Y = Z 1 ( ) θ e θ1z Y θ θ 3 e θ1z Y θ 3 log θ 3 θ e θ 1Z = Z θ 3 W log W log(θ e θ 1Z )} 6

7 = z θ 3 W log W log(θ ) θ 1 Z} l3 (Y, Z) = e θ 1Z Y θ 3 log Y = 1 θ θ 3 θ e θ 1Z Y θ 3 log = 1 θ θ 3 W log W log(θ e θ 1Z )} = 1 θ θ 3 W log W log(θ ) θ 1 Z}, l33 (Y, Z) = 1 θ W [log W log(θ e θ 1Z ] }. ( ) θ e θ1z Y θ 3 θ e θ 1Z Thus we calculate easily: I 11 (θ) = E θ ( l 1 (Y, Z) ) = E θ E[Z (1 W ) Z]} = EZ E[(1 W ) Z]} = E(Z ), I (θ) = E θ ( l (Y, Z) ) = E θ E[θ (1 W ) Z]} = θ, I 33 (θ) = θ3 1 + E[W (log W ) ] E(W log W )log θ + θ 1 E(Z)} + E(log θ + θ 1 Z) } } = θ3 1 + B Alog θ + θ 1 E(Z)} + E(log θ + θ 1 Z) } } I 1 (θ) = E θ ( l 1 (Y, Z) l (Y, Z)) = E θ E[Zθ 1 (1 W ) Z]} = θ 1 E(Z), I 13 (θ) = E θ l 13 (Y, Z)} = θ3 1 E(Z)[A log θ ] θ 1 E(Z )}, I 3 (θ) = E θ l 3 (Y, Z)} = (θ θ 3 ) 1 A log θ θ 1 E(Z)} where A EW log W } = 0 (w log w) exp( w)dw = 1 γ, B EW (log W ) } = π /6 + (1 γ) 1. Note that since l 4 (y, z) = a(z) is just a function of Z, it follows easily that for j = 1,, 3 we also have I j4 (θ) = E θ l j (Y, Z) l 4 (Y, Z)} = Eg j (W, Z)a(Z)} = EE[g j (W, Z)a(Z) Z]} = Ea(Z)E[g j (W, Z) Z]} = Ea(Z) 0} = 0, Because of this orthogonality, the information bounds for (θ 1, θ, θ 3 ) are the same when θ 4 = η is unknown as when it is known. (c) If θ and θ 3 are known, then the information bound for estimation of θ 1 is just I11 1 (θ) = 1/E(Z ). It follows that the information matrix for θ is of the following form: E(Z ) θ 1 E(Z) θ3 1 C 0 I(θ) = θ 1 E(Z) θ (θ θ 3 ) 1 D 0 θ3 1 C (θ θ 3 ) 1 D θ3 E Ea (Z) 7

8 where C = E(Z)(A log θ ) θ 1 E(Z ) D = A log θ θ 1 E(Z) E = 1 + B A(log θ + θ 1 E(Z)) + E(log θ + θ 1 Z). (d) If θ 3 = 1 is known, then the information bound for θ 1 is I 1 11 where I 11 (θ) = I 11 I 1 I 1 I 1 = E(Z ) (E(Z)/θ ) θ = E(Z ) (EZ) = V ar(z). Thus I 1 11 = 1/V ar(z). (e) When θ 3 is known, the efficient score function and the efficient influence function for estimation of θ 1 are given by and l 1(Y, Z) = l 1 I 1 I 1 l = Z(1 W ) θ 1 E(Z)θ 1 (1 W ) θ = Z(1 W ) E(Z)(1 W ) = (Z E(Z))(1 W ), l1 (Y, Z) = I11 l 1 1(Y, Z) 1 = (Z E(Z))(1 W ). V ar(z) (f) When both the parameters θ and θ 3 are unknown, the information I 11 (,3) is given by where I 1 (,3) I 11 where the second block contains both θ, θ 3 = I 11 I 1 I 1 I 1 (3) I 1 = (θ 1 E(Z), θ3 1 C), ( ) θ I 1 = E θ θ 3 D 1 θ θ 3 D θ3 E D. Thus the second term in (3 ) is [E(Z)] E E(Z)CD + C } /(E D ). (4) Now the denominator is E D = 1 + B A(log θ + θ 1 E(Z)) + E(log θ + θ 1 Z) (A log θ θ 1 E(Z)) = 1 + B A(log θ + θ 1 E(Z)) + E(log θ + θ 1 Z) [A A(log θ + θ 1 E(Z)) + (log θ + θ 1 E(Z)) = 1 + B A + V ar[log θ + θ 1 Z] = π /6 + θ 1V ar(z), 8

9 and, upon noting that C E(Z)D = E(Z)(A log θ ) θ 1 E(Z ) E(Z)(A log θ ) θ 1 [E(Z)] } = θ 1 V ar(z), it follows that the numerator of (4) is C E(Z)CD + [E(Z)] E = C E(Z)CD + [E(Z)] D + [E(Z)] (E D ) = (C E(Z)D) + [E(Z)] π /6 + θ 1V ar(z)} = θ 1[V ar(z)] + [E(Z)] π /6 + θ 1V ar(z)}. It follows that the information for θ 1 when θ and θ 3 are unknown is equal to I 11 (,3) = E(Z ) [E(Z)] π /6 + θ1v ar(z)} π /6 + θ1v ar(z) = π /6 π /6 + θ1v ar(z) V ar(z) V ar(z) E(Z ) with equality in the first inequality if and only if θ 1 = 0. Note that the information decreases as θ 1 increases, and it converges to π /(6θ1) as V ar(z). (g) When θ and θ 3 are unknown the efficient score function for θ 1 is, with the second block containing both θ and θ 3, l 1 = l 1 I 1 I 1 l = l 1 (θ (E(Z)E CD), θ 3 (C DE(Z)) l /(E D ) = E(Z)E CD Z(1 W ) (1 W ) E D = θ 1 V ar(z) π /6 + θ 1V ar(z) + Z E(Z)E CD + log(θ e θ1z ) π /6 + θ1v ar(z) + [1 (W 1) log W ] + (W 1) log(θ e θ 1Z ) } } (1 W ) θ1v ar(z) 1 (W 1) log W }. π /6 + θ1v ar(z) (h) When Z Bernoulli(η), then I 11 = E(Z ) = η = θ 4, I 11 = V ar(z) = η(1 η) = θ 4 (1 θ 4 ), π /6 I 11 (,3) = π /6 + θ1v ar(z) V ar(z) = π /6 η(1 η). π /6 + θ1η(1 η) The corresponding information bounds are given by the reciprocals of these quantities. See the following figures for comparisons of the information and information bounds. 9

10 Figure 1: Plots of I 11, I 11, and I 11 (,3) as a function of η = θ 4, and for θ 1 =.5, 1.0, Figure : Plots of I 1 11, I 1 11, and I 1 11 (,3) as a function of η = θ 4, and for θ 1 =.5, 1.0,