Chapter 9: Hypothesis Testing Sections

Chapter 9: Hypothesis Testing Sections 9.1 Problems of Testing Hypotheses 9.2 Testing Simple Hypotheses 9.3 Uniformly Most Powerful Tests Skip: 9.4 Two-Sided Alternatives 9.6 Comparing the Means of Two Normal Distributions 9.7 The F Distributions 9.8 Bayes Test Procedures 9.9 Foundational Issues Hypothesis Testing 1 / 25

9.3 Uniformly Most Powerful Tests Uniformly Most Powerful (UMP) Tests H 0 : θ Ω 0 vs H 1 : θ Ω 1 A test δ is a uniformly most powerful test at level α 0 if for any other level α 0 test δ π(θ δ) π(θ δ ) for all θ Ω 1 I.o.w: It has the lowest probability of type II error of any test, uniformly for all θ Ω 1. First control the probability of type I error by setting the level (size) of the test low, then control the probability of type II error. If π(θ δ ) is high for all θ Ω 1, the test is often called powerful Hypothesis Testing 2 / 25

UMP Tests Chapter 9 9.3 Uniformly Most Powerful Tests Example 1: Simple hypotheses. H 0 : θ = θ 0 vs. H 1 : θ = θ 1 LRT is UMP by Nayman-Pearson lemma (Theorem 9.2.2) Example 2: One-sided hypotheses: H 0 : θ θ 0 vs. H 1 : θ > θ 0 In a large class of problems (the distribution has a monotone likelihood ratio ), we can show that reject H 0 if T t" is a UMP for some T (Ch 9.3) Example 3: Two-sided hypotheses: H 0 : θ = θ 0 vs. H 1 : θ θ 0 UMP tests do not exist (Page 565) Hypothesis Testing 3 / 25

9.3 Uniformly Most Powerful Tests Chapter 9: Hypothesis Testing Sections 9.1 Problems of Testing Hypotheses Skip: 9.2 Testing Simple Hypotheses 9.3 Uniformly Most Powerful Tests Skip: 9.4 Two-Sided Alternatives 9.6 Comparing the Means of Two Normal Distributions 9.7 The F Distributions 9.8 Bayes Test Procedures 9.9 Foundational Issues Hypothesis Testing 4 / 25

The t-test The t-test is a test for hypotheses concerning the mean parameter in the normal distribution when the variance is also unknown. The test is based on the t distribution The setup for the next few slides: Let X 1,..., X n be i.i.d. N(µ, σ 2 ) and consider the hypotheses H 0 : µ µ 0 vs. H 1 : µ > µ 0 (1) The parameter space here is < µ < and σ 2 > 0, i.e. Ω = (, ) (0, ) And Ω 0 = (, µ 0 ] (0, ) and Ω 1 = (µ 0, ) (0, ) Hypothesis Testing 5 / 25

The one-sided t-test The t test: a likelihood ratio test (see p. 583-585 in the book) Let U = ( n(x n µ 0 ) σ where σ 1 = n 1 If µ = µ 0 then U has the t n 1 distribution Tests based on U are called t tests ) 1/2 n (X i X n ) 2 i=1 Hypothesis Testing 6 / 25

The one-sided t-test Chapter 9 Let Tm 1 be the quantile function of the t m distribution The test δ that rejects H 0 in (1) if U T 1 n 1 (1 α 0) has size α 0 (Theorem 9.5.1) To calculate the p-value: Theorem 9.5.2: p-values for t Tests Let u be the observed value of U. The p-value for the hypothesis in (1) is 1 T n 1 (u). Hypothesis Testing 7 / 25

Example Example: Acid Concentration in Cheese (Example 8.5.4) Have a random sample of n = 10 lactic acid measurements from cheese, assumed to be from a normal distribution with unknown mean and variance. Observed: x n = 1.379 and σ = 0.3277 Perform the level α 0 = 0.05 t-test of the hypotheses Compute the p-value H 0 : µ 1.2 vs H 1 : µ > 1.2 Hypothesis Testing 8 / 25

The complete power function Need the power function to decide the sample size n The power function π(µ, σ 2 δ) is a non-central t m distributions Def: Non-central t m distributions Let W N(ψ, 1) and Y χ 2 m be independent. The distribution of X = W (Y /m) 1/2 is called the non-central t distribution with m degrees of freedom and non-centrality parameter ψ Hypothesis Testing 9 / 25

Non-central t m distribution Hypothesis Testing 10 / 25

The complete power function For the one-sided t-test Theorem 9.5.3 U has the non-central t n 1 distribution with non-centrality parameter ψ = n(µ µ 0 )/σ. The power function of the t-test that rejects H 0 in (1) if U T 1 n 1 (1 α 0) = c 1 is π(µ, σ 2 δ) = 1 T n 1 (c 1 ψ) Hypothesis Testing 11 / 25

Power function for the one-sided t-test Example: n = 10, µ 0 = 5, α 0 = 0.05 Note that the power function is a function of both σ 2 and µ Hypothesis Testing 12 / 25

The other one-sided t-test Now consider the hypothesis H 0 : µ µ 0 vs. H 1 : µ < µ 0 (2) The test δ that rejects H 0 if U T 1 n 1 (α 0) has size α 0 (Corollary 9.5.1) Theorem 9.5.2: p-values for t Tests Let u be the observed value of U. The p-value for the hypothesis in (2) is T n 1 (u). Theorem 9.5.3 U has the non-central t n 1 distribution with non-centrality parameter ψ = n(µ µ 0 )/σ. The power function of the t-test that rejects H 0 in (2) if U T 1 n 1 (α 0) = c 2 is π(µ, σ 2 δ) = T n 1 (c 2 ψ) Hypothesis Testing 13 / 25

Two-sided t-test Consider now the test with a two-sided alternative hypothesis: H 0 : µ = µ 0 vs. H 1 : µ µ 0 (3) Size α 0 test δ: rejects H 0 iff U T 1 n 1 (1 α 0/2) = c If u is the observed value of U then the p-value is 2(1 T n 1 ( u )) The power function is π(µ, σ 2 δ) = T n 1 ( c ψ) + 1 T n 1 (c ψ) Hypothesis Testing 14 / 25

Notes on one sample t tests Paired t tests are conducted in the same way For large n, the distribution of the test statistic under H 0 is close to the standard normal, i.e., the corresponding test is close to a Z test Hypothesis Testing 15 / 25

The two-sample t-test Chapter 9 9.6 Comparing the Means of Two Normal Distributions Comparing the means of two populations X 1,..., X m i.i.d. N(µ 1, σ 2 ) and Y 1,..., Y n i.i.d. N(µ 2, σ 2 ) The variance is the same for both samples, but unknown We are interested in testing one of these hypotheses: a) H 0 : µ 1 µ 2 vs. H 1 : µ 1 > µ 2 b) H 0 : µ 1 µ 2 vs. H 1 : µ 1 < µ 2 c) H 0 : µ 1 = µ 2 vs. H 1 : µ 1 µ 2 Hypothesis Testing 16 / 25

Two-sample t statistic Chapter 9 9.6 Comparing the Means of Two Normal Distributions Let X m = 1 m X i and Y n = 1 m n i=1 n i=1 m SX 2 = n (X i X m ) 2 and SY 2 = (Y i Y n ) 2 i=1 Y i i=1 m + n 2 (X m Y n ) U = ( 1 m + 1 ) 1/2 ( ) n S 2 X + SY 2 1/2 Theorem 9.6.1: If µ 1 = µ 2 then U t m+n 2 Theorem 9.6.4: For any µ 1 and µ 2, U has the non-central t m+n 2 distribution with non-centrality parameter µ 1 µ 2 ψ = σ (1/m + 1/n) 1/2 Hypothesis Testing 17 / 25

Two-sample t test summary Proofs similar to the regular t-test 9.6 Comparing the Means of Two Normal Distributions a) H 0 : µ 1 µ 2 vs. H 1 : µ 1 > µ 2 Level α 0 test: Reject H 0 iff U T 1 m+n 2 (1 α 0) p-value: 1 T m+n 2 (u) b) H 0 : µ 1 µ 2 vs. H 1 : µ 1 < µ 2 Level α 0 test: Reject H 0 iff U T 1 m+n 2 (α 0) p-value: T m+n 2 (u) c) H 0 : µ 1 = µ 2 vs. H 1 : µ 1 µ 2 Level α 0 test: Reject H 0 iff U T 1 m+n 2 (1 α 0/2) p-value: 2(1 T m+n 2 ( u )) Hypothesis Testing 18 / 25

9.6 Comparing the Means of Two Normal Distributions Power function is now a function of 3 parameters: π(µ 1, µ 2, σ 2 δ) The two-sample t-test is a likelihood ratio test (see p. 592) Important difference: Paired t test vs. two sample t test Two-sample t test with unequal variances Proposed test-statistics do not have known distribution, but approximations have been obtained Approach 1: The Welch statistic V = X m Y n ( S 2 X m(m 1) + S2 Y n(n 1) ) 1/2 can be approximated by a t distribution Approach 2: The distribution of the likelihood ratio statistic can be approximated by the χ 2 1 distribution if the sample size is large enough Hypothesis Testing 19 / 25

F-distributions Chapter 9 9.7 The F Distributions In light of the previous slide, it would be nice to have a test of whether the variances in the two normal populations are equal need the F m,n distributions Def: F m,n -distributions Let Y χ 2 m and W χ 2 n be independent. The distribution of X = Y /m W /n = ny mw is called the F distribution with m and n degrees of freedom The pdf of the F m,n distribution is f (x) = Γ ((m + n)/2) mm/2 n n/2 Γ(m/2)Γ(n/2) x m/2 1 (mx + n) (m+n)/2 x > 0 Hypothesis Testing 20 / 25

F-distributions Chapter 9 9.7 The F Distributions Hypothesis Testing 21 / 25

9.7 The F Distributions Properties of the F-distributions The 0.95 and 0.975 quantiles of the F m,n distribution is tabulated in the back of the book for a few combinations of m and n Theorem 9.7.2: Two properties (i) If X F m,n then 1/X F n,m (ii) If Y t n then Y 2 F 1,n Hypothesis Testing 22 / 25

9.7 The F Distributions Comparing the variances of two normals Comparing the variances of two populations X 1,..., X m i.i.d. N(µ 1, σ1 2) and Y 1,..., Y n i.i.d. N(µ 2, σ2 2) All four parameters unknown Consider the hypotheses: (I) H 0 : σ 2 1 σ2 2 vs. H 1 : σ 2 1 > σ2 2 and the test that rejects H 0 if V c, where This test is called an F-test σ 2 2 σ 2 1 V F m 1,n 1 If σ 2 1 = σ2 2 then V F m 1,n 1 V = S2 X /(m 1) SY 2 /(n 1) Hypothesis Testing 23 / 25

The F test Chapter 9 9.7 The F Distributions Let G m,n (x) be the cdf of the F m,n distribution Theorem 9.7.4 Let δ be the test that rejects H 0 in H 0 : σ 2 1 σ2 2 vs. H 1 : σ 2 1 > σ2 2 if V c = G 1 m 1,n 1 (1 α 0). Then δ is of size α 0 and ( ) π(µ 1, µ 2, σ1 2, σ2 2 δ) = 1 G σ 2 2 m 1,n 1 c and σ1 2 p-value = 1 G m 1,n 1 (v), where v is the observed value of V Hypothesis Testing 24 / 25

9.7 The F Distributions The F test two sided alternative Two sided alternative: H 0 : σ 2 1 = σ2 2 vs. H 1 : σ 2 1 σ2 2 Equal-tailed two-sided F test Let δ be the F-test that rejects H 0 when V c 1 = G 1 m 1,n 1 (α 0/2) or V c 2 = G 1 m 1,n 1 (1 α 0/2). Then δ is a level α 0 test and the p-value is 2 min{1 G m 1,n 1 (v), G m 1,n 1 (v)}. Hypothesis Testing 25 / 25