Hypothesis Testing. Robert L. Wolpert Department of Statistical Science Duke University, Durham, NC, USA

Transcription

1 Hypothesis Testing Robert L. Wolpert Department of Statistical Science Duke University, Durham, NC, USA An Example Mardia et al. (979, p. ) reprint data from Frets (9) giving the length and breadth (in millimeters) of the heads of the first and second son in a sample of n = 5 families, from a study of heredity in humans. If we assume a multivariate normal model then the following statistics are sufficient: x = 85.7 x = x = 5. x 3 = x 4 = 49.4 n S = , 44. the sample mean ˆµ = x = n Xα and the sample covariance ˆΣ = n S where S := (X α x)(x α x). If we model {X α } iid No(µ,Σ) for α 5, the log likelihood function for µ and Λ := Σ is l(µ,λ) = n log Λ/π tr ΛS n ( x µ) Λ( x µ) In this section we ll consider only the length measurements of the two sons, X and X 3. We will test each of the null hypotheses H 0 :µ = 80 H 0 :µ 3 = 80 H 3 0 :µ = µ 3 = 80 against the omnibus alternative first for known Λ, then for unknown. For now we ll follow the sampling-theory paradigm and find P-values for these

2 hypotheses on the basis of the n = 5 observations of the p = -dimensional data [x,x 3 ], with summary statistics ] [ ] [ x = x = x 3 = S = Likelihood Ratio Tests Each of our hypotheses will be of the form H j : θ Θ j for some set Θ j Θ of possible parameters θ governing the distribution of the observables through their joint pdf f(x θ). The traditional sampling-theory approach to testing a hypothesis H 0 of this form against an alternative H is to construct the likelihood ratio against the Null B(x) := sup θ Θ f(x θ) sup θ Θ0 f(x θ) or, equivalently, twice its logarithm, the deviance where δ(x) = [l (x) l 0 (x)] l j (x) = log sup θ Θ j f(x θ) for j = 0,, and reject H 0 for sufficiently large values of B(x) (or of δ(x)) say, for δ(x) c. The significance level of the test is the maximum rejection probability P[l(X) c θ] if the hypothesis is true (i.e. for θ Θ 0 ), while the P-value is P(x) = sup θ Θ0 P[δ(X) δ(x) θ] for the observed data value x, the probability of observing B(x) (or δ(x)) at least this large if H 0 is true. Under suitable regularity conditions (asymptotic normality and a bit more), if Θ 0 Θ R q with dim(θ 0 ) = r < q, the asymptotic distribution of δ(x) for large sample-size n is δ(x) χ q r.. One-dimensional Hypotheses, known Λ First consider only the first son s head width, X, and hypothesis H 0 that its mean is µ = 80. If we are given the precision say, σ = /00

3 then the maximum log likelihoods under H 0 : µ = 80 and its alternative H : µ R are log f(x ˆθ j ) where ˆθ j is the MLE under the restriction θ Θ j, l 0 = n log(λ/π) ΛS n ( x 80) Λ( x 80) = n l = n and hence 0.0 log π 0.0 log π δ = [l l 0] 0.0S 5 ( ) 0.0( ) 0.0S = nλ( x 80) = = Since Θ 0 is r = 0-dimensional and Θ is q = -dimensional, δ(x) has approximately a χ distribution under the null hypothesis and so the P-value would be approximately P[χ > 8.796] = Φ( ) = , so the hypothesis would be rejected at level α = 0.0. The critical values of δ(x) for rejecting at levels α = 0.0 and α = 0.05 would be.58 = and.96 = 3.84, respectively. Similarly, the hypothesis H 0 : µ 3 = 80 would have δ(x) = [l l 0] = nλ( x 3 80) = = , leading to P-value P(x) = Φ ( ) = , so H0 rejected at level α = cannot be.. Composite Hypothesis H 3 0 How can we test the p = -dimensional hypothesis H0 3 : µ = µ 3 = 80? Simply noting that one of the two one-dimensional hypotheses was rejected at level α = 0.0 is not enough to reject H0 3 at that level because of the multiple comparisons issue the probability of rejecting at least one of k hypotheses at level α may have probability greater than α if H 0 is true. By subadditivity it can t have probability more than k α, though, so the naïve Bonferroni multiple-comparison correction is valid reject H0 3 at level α if either H0 or H 0 can be rejected at level α/. Somewhat better are any of:. Since x and x 3 are independent, the probability of rejecting either at level γ is [ ( γ) ] if H0 3 is true, which will be no more than 3

4 α if we take γ = α; thus we can reject at levels α = 0.0 or α = 0.05 if either individual hypothesis may be rejected at level γ = α = or 0.053, respectively (slightly higher than Bonferroni).. Under H 3 0, each of z i := nλ( x i 80) has a standard normal No(0,) distribution, hence so too does (z + z )/ ; a valid test of H 3 0 could be based on P-value Φ( (z + z )/ ). For these data z =.86 and z =.9, and hence z = (z + z )/ = would lead to P(x) = and rejection of H With z j as above, under H 3 0 the test statistic Y = (z ) + (z 3 ) has a χ distribution, leading to P(x) = exp( Y/) = e = , and rejection again... LLR for Composite Hypothesis H 3 0 A more principled approach is to compute the log likelihood ratio for the r = 0-dimensional hypothesis H0 3 and its q = -dimensional alternative: l 0 = n log Λ/π tr ΛS n ( x µ 0)Λ( x µ 0 ) = n [ 0.0 ] [ ][ ] log π tr 5 [ ] [ ][ ] π l = n [ 0.0 ] [ ][ ] log π tr 0 π and hence δ(x) = 0.5( ) =.866, leading (as in 3. above) to P(x) = exp(.866/) = Confidence Ellipses The same calculations lead to confidence ellipses of the form C α (x) = {µ : n( x µ) Λ( x µ) c α } 4

5 for c α chosen such so that P[δ(x) > c α H 0 ] = α; in this problem c α = log α, so for example the 95% ellipse is C 0.95 = {µ : 5[(µ 85.7) /00 + (µ 83.84) /00] 5.99} = {µ : (µ 85.7) + (µ 83.84) 3.966}, the circle of radius centered at [ x, x 3 ]..3 Unknown Precision Now consider the same problem with Λ unknown. Lemma. If D S + p and n > 0 then the function f(g) = n log G tr G D of G S + p attains its maximum value at G = nd, and there takes the value np log n n log D np. Proof. Let D = EE and set H := E G E; then G = EH E, so and G = E H E = D / H, tr G D = tr G EE = tr E G E = tr H, so we can rewrite f(g) = g(h) with g(h) = n log D + n log H tr H. Now write H = TT with T lower-triangular; then the maximum of g(h) = n log D + n log T tr TT p = n log D + log t i=(n ii t ii ) i>j occurs at t ii = n and t ij = 0, i j, or H = ni. Then G = n EE = n D. t ij 5

6 As functions of Σ = Λ, twice the log likelihood l(µ,λ) is of the form considered in Lemma() under both H 0 and H ; thus l(µ,λ) = np log π + n log Λ tr Λ[S + n( x µ)( x µ) ] l 0 = sup l(µ 0,Λ) = l ( µ 0,n(S + n dd ) ) where d := ( x µ) Λ P + = np log π n log n S + dd np l = sup l(µ,λ) = l ( x,ns ) µ R,Λ P + = np log π n log n S np and hence the deviance is δ(x) = [ l ( x,ns ) l ( µ 0,n(S + n dd ) )] = n log S + n dd n log S, a monotone increasing function δ(x) = n log R of R = S + n ( x µ)( x µ) S = + n( x µ) S ( x µ) = + n n T, where T := ν( x µ) S ( x µ) with ν := n has Hotelling s Tp (ν) distribution, while n p p ( x µ) S ( x µ) has Snedecker s. For these data, F p n p F = n p p(n ) T = 3 = [ ] [ ][ ] leading to an exact P-value of P(x) = Pr[F3 > 3.947] = , with rejection at α = 0.05 but not at α = 0.0. The deviance here was δ(x) = n log ( + n ( x µ) S ( x µ) = , leading to an approximate P-value of P(x) exp( /) = 0.05, 6

7 which would lead to the same conclusions. Confidence ellipses are again available; for example, since P[F3 > 3.4] = 0.05 and (/3) 3.4 = , a 95% confidence set can be constructed as { C 0.95 (x) = µ : ( x µ) S ( x µ) p c } α n p { [ ] [ ][ ] } µ µ 85.7 = µ : µ µ where c α = 0.34 is the appropriate critical value of the F p n p distribution. This also leads to simultaneous 95% confidence intervals for all possible linear combinations α µ + α µ (for example, for [µ µ ] and [ µ +µ ]). References Frets, G. P. (9), Heredity of head form in man, Genetica, 3, Mardia, K. V., Kent, J. T., and Bibby, J. M. (979), Multivariate Analysis, New York, NY: Academic Press. 7