Theoretical Statistics. Lecture 23.

Size: px

Start display at page:

Download "Theoretical Statistics. Lecture 23."

Merilyn Skinner
6 years ago
Views:

1 Theoretical Statistics. Lecture 23. Peter Bartlett 1. Recall: QMD and local asymptotic normality. [vdv7] 2. Convergence of experiments, maximum likelihood. 3. Relative efficiency of tests. [vdv14] 1

2 Local asymptotic normality We ve seen that, for a QMD model P θ, the log likelihood ratio, log dpn θ 0 +h/ n dp n θ 0 (X i ), is asymptotically normal. This is useful for: 1. Comparing null θ 0 and shrinking alternative θ 0 +h/ n with a likelihood ratio test. 2. Understanding the local behavior of a statistict n. If we assume that θ is fixed, and we understand T n s asymptotics under P θ, we can use the asymptotics of the log likelihood ratio to understand the asymptotics oft n in a local neighborhood ofθ. The appropriate local scale is typically1/ n. 2

3 Recall: QMD and local asymptotic normality Theorem: If Θ is an open subset of R k, and P θ is QMD at θ Θ, then 1. P θ lθ = I θ = P θ lθ l T θ exists. 3. For every h n satisfying nh n h, log n i=1 p θ+hn p θ (X i ) = 1 n n θ N i=1 h T lθ (X i ) 1 2 ht I θ h+o Pθ (1) ( 1 ) 2 ht I θ h,h T I θ h. 3

4 Recall: Quadratic mean differentiability Definition: The root density θ p θ (for θ R k ) is differentiable in quadratic mean at θ if there exists a vector-valued measurable function l θ : X R k such that, forh 0, ( pθ+h p θ 1 ) 2 2 ht lθ pθ dµ = o( h 2 ). 4

5 Recall: Asymptotically linear statistics Suppose the model {P θ : θ Θ} is QMD, and a statistic T n satisfies n(tn µ θ ) = 1 n n i=1 ψ θ (X i )+o Pθ (1), where P θ ψ θ = 0 andp θ ψ θ ψθ T = Σ. Then for nh n h, ( ) n(tn µ θ ),log dpn θ+h n θ 0 dpθ n N, Σ 1 2 ht I θ h τ T τ, h T I θ h where τ = P θ ψ θ h T lθ. So n(t n µ θ ) θ+h n N ( ) P θ ψ θ h T lθ,σ. 5

6 Asymptotically linear statistics That is, we know that under θ, n(tn µ θ ) θ N(0,Σ). And we can use the asymptotics of the log likelihood ratio to determine the asymptotics of this statistic under the shrinking alternative θ +h/ n: n(tn µ θ ) θ+h/ n N ( ) P θ ψ θ h T lθ,σ. 6

7 Asymptotically linear statistics: Example Location families: Suppose that p θ (x) = f(x θ), where f is positive, continuously differentiable, and satisfies µ = xf(x)dx = 0, σ 2 = x 2 f(x)dx <, I θ = ( f (x) f(x) ) 2 f(x)dx <. This family is QMD. 7

8 Asymptotically linear statistics: Example 1. Consider the t-statistic for the null hypothesisθ = 0, T n = 1 n n i=1 X i S n ntn = 1 n n i=1 X i σ +o P 0 (1). Thus,T n is an asymptotically linear statistic, with ψ θ (x) = x σ, l θ (x) = f (x θ) f(x θ). 8

9 Asymptotically linear statistics: Example Hence, forh n satisfying nh n h, ( ) h ntn n N P 0 ψ 0 h l 0,P 0 ψ0 2, P 0 ψ 0 h l 0 = P 0 X σ f (X) f(x) h = h σ xf (x)dx = h σ f(x)dx = h σ. P 0 ψ 2 0 = 1 σ 2P 0X 2 = 1. ntn h n N ( h σ,1 ). 9

10 Asymptotically linear statistics: Example 2. Suppose that P 0 (X > 0) = 1/2 and consider the sign statistic for the null hypothesis θ = 0, s n = 1 n ( 1[X i > 0] 1 ). n 2 i=1 Thus,s n is an asymptotically linear statistic, with ψ θ (x) = 1[x > 0] P θ (X > 0), l θ (x) = f (x θ) f(x θ). Hence, forh n satisfying nh n h, ( ) h nsn n N P 0 ψ 0 h l 0,P 0 ψ0 2 10

11 Asymptotically linear statistics: Example ( P 0 ψ 0 h l 0 = P 0 1[X > 0] 1 ) f (X) 2 f(x) h ) = h = h 2 ( 1[x > 0] 1 2 ( 0 f (x)dx P 0 ψ0 2 = 1 4. ( h nsn n N hf(0), 1 ). 4 0 f (x)dx ) f (x)dx = hf(0). 11

12 Convergence of local statistical experiments ( Theorem: If Pθ : θ Θ R k) is QMD at θ with nonsingular Fisher information I θ, T n are statistics in the local experiments ( Pθ+h/ n : h R k) h, and for everyhthere is a lawl h s.t.t n Lh. Then there is a randomized statistic T in the experiment ( N ( h,i 1 ) ) θ : h R k such that for each h,t n h T. The proof uses the Le Cam lemmas (change of measure via the asymptotically normal log-likelihood ratio) 12

13 Convergence of local statistical experiments For the local statistical experiment, ( P n θ+h/ n : h Rk), think ofθ as a particular parameter value, and θ +h/ n as a nearby value. We are interested in the asymptotic behavior of statistics when the parameter is near the value θ. Motivation: IfT n defines a test, then the power P h (T n > c) depends on the law of T n, so we can study its asymptotics via statistics in a normal experiment. IfT n is an estimator, then we can study the asymptotics of the expected squared error E h (T n h) 2 via statistics in a normal experiment. 13

14 Maximum likelihood Consider the maximum likelihood estimator T n = ĥn for the local experiment ( P n θ+h/ n : h Rk). (Notice that ĥn = n(ˆθ n θ).) Typically, the matching asymptotic statistic in the limit experiment is the maximum likelihood estimator T = X N ( h,i 1 ) θ. So we expect the asymptotic distribution of n(ˆθn θ) to be N ( 0,I 1 ) θ under θ. Note that the previous theorem does not imply that this particular statistic in the limit experiment (the maximum likelihood estimator) is the weak limit of the T n. This needs some additional conditions. 14

15 Maximum likelihood Theorem: Suppose 1. (P θ : θ Θ) is QMD at θ with nonsingular Fisher information I θ, 2. for every x,θ logp θ (x) is Lipschitz, and 3. the maximum likelihood estimator ˆθ n is consistent. Then n(ˆθn θ) θ N(0,I 1 θ ). 15

16 Relative efficiency of tests Example: Suppose X 1,...,X n P θ, where 1. P θ has density f(x θ) onr, 2. f is symmetric about zero (so the mean=median of P θ isθ), 3. f has a unique median (f(0) 0), 4. f has a finite variance. We wish to test H 0 : θ = 0 versus H 1 : θ > 0. 16

17 Relative efficiency of tests Example: Candidate tests: 1. Sign test: S n = 1 n 1[X i > 0]. n 2. t-test: T n = 1 n Which is better? n i=1 i=1 X i S n. 17

18 Relative efficiency of tests: sign test Thus, S n = 1 n n 1[X i > 0]. i=1 n σ(θ) (S n µ(θ)) N(0,1), where 2 n µ(θ) = 1 F( θ), σ 2 (θ) = (1 F( θ))f( θ). ( S n 1 ) 0 N(0,1). 2 Reject H 0 if2 n(s n 1/2) > z α. 18

19 Relative efficiency of tests: sign test Definition: The power function of a test that rejects the null hypothesis when the statistict n falls in the critical region K n is π n (θ) = P θ (T n K n ). For the sign test, ( ) π n (θ) = P θ n(sn µ(0)) > σ(0)z αn ( n = P θ σ(θ) (S n µ(θ)) > σ(0)z ) α n + n(µ(0) µ(θ)) σ(θ) ( σ(0)zαn + ) n(µ(0) µ(θ)) = 1 Φ +o(1). σ(θ) 19

20 Relative efficiency of tests: sign test Forθ = 0, we have π n (0) = 1 Φ(z αn ) = α n. Forθ > 0,µ(0) µ(θ) = F( θ) F(0) < 0. Provided α n 0 sufficiently slowly, ( σ(0)zαn + ) n(µ(0) µ(θ)) π n (θ) = 1 Φ σ(θ) 0 if θ = 0, 1 if θ > 0. +o(1) So the limiting power function is perfect. This is typical: any reasonable test can distinguish a fixed alternative, given unlimited data. 20

21 Relative efficiency of tests So how do we compare tests? We need to make the problem of discriminating between the null and the alternative more difficult asn increases. It is natural to consider a shrinking alternative, that converges to the null. Recall our example: We wish to test H 0 : θ = 0 versus H 1 : θ n > 0, with θ n 0. 21

22 Relative efficiency of tests For the sign test, ( σ(0)zα + ) n(µ(0) µ(θ n )) π n (θ n ) = 1 Φ σ(θ n ) +o(1). The level of the test converges: What about the power? π n (0) = 1 Φ(z α )+o(1) α. It depends on the asymptotics of n(µ(0) µ(θ n )). SinceF is differentiable at 0, n(µ(0) µ(θn )) = n(f( θ n ) F(0)) = nθ n f(0)+o( nθ n ). 22

23 Relative efficiency of tests If θ n θ faster than 1/ n, n(µ(0) µ(θ n )) 0, soπ n (θ n ) α. The test fails: these alternatives are too hard. Forθ n θ slower than 1/ n, n(µ(0) µ(θ n )), so π n (θ n ) 1. These slowly shrinking alternatives are too easy. Consider an intermediate rate: nθn h. 23

Theoretical Statistics. Lecture 22.

Theoretical Statistics. Lecture 22. Peter Bartlett 1. Recall: Asymptotic testing. 2. Quadratic mean differentiability. 3. Local asymptotic normality. [vdv7] 1 Recall: Asymptotic testing Consider the asymptotics