Week 12 Hypothesis Testing, Part II Comparing Two Populations

Transcription

1 Week 12 Hypothesis Testing, Part II Week 12 Hypothesis Testing, Part II

2 Week 12 Objectives 1 The principle of Analysis of Variance is introduced and used to derive the F-test for testing the model utility hypothesis in the SLR model. 2 The notion of precision in hypothesis testing is discussed. 1 The concepts of type I error, type II error, and power of a test procedure are discussed. 2 Formulas for the calculation of the power, when testing for a mean or a proportion, are given. 3 Sample size calculation in the same contexts is also presented. 3 Procedures for comparing two means or two proportions are introduced. Week 12 Hypothesis Testing, Part II

3 1 2 3 Comparing Two Means Comparing Two Proportions Week 12 Hypothesis Testing, Part II

4 The ANOVA F test is an alternative (but equivalent) way of conducting the model utility test. ANOVA (which stands for ANalysis Of VAriance) is a very common and generally applicable approach to hypothesis testing. As the name suggests, this approach decomposes (i.e. analyses) the total variability of Y, aka total sum of squares (SS, for short), which is defined to be Total SS = (Y i Y ) 2, into the Regression SS and the Error SS: Total SS = Regression SS + Error SS. Week 12 Hypothesis Testing, Part II

5 The Error SS is the same thing as the SSE (sum of squared errors) we saw in Week 10. It expresses the variability due to the intrinsic error. Regression SS (SSR for short) represents the part of the total variability explained by the regression model The larger the ratio R 2 = (Regression SS), (Total SS) the better the predictive power of the regression model. R 2 equals the square of Pearson s correlation coefficient between X and Y, is called the coefficient of determination. The term Mean Sum of Squares refers to the sum of squares divided by the corresponding degrees of freedom. Week 12 Hypothesis Testing, Part II

6 The F test statistic for testing the model utility test is F = (Regression SS)/1 (Error SS)/(n 2) = (MSR) (MSE) If the null hypothesis is true, i.e. if β 1 = 0, then either exactly (if the errors have a normal distribution), or approximately (without normality but n 30) F F 1,n 2, where F ν1,ν 2 denotes the F-distribution with ν 1 and ν 2 degrees of freedom. H 0 : β 1 = 0 is rejected at level α if F > F 1,n 2,α. Week 12 Hypothesis Testing, Part II

7 The sums of squares, the mean squares and the F statistic are summarized in the so-called ANOVA table. Source df SS MS F Regression 1 SSR MSR= SSR 1 Error n 2 SSE MSE= SSE n 2 Total n 1 SST F = MSR MSE Week 12 Hypothesis Testing, Part II

8 Example Measurements from the river Ijse (tributary to the river Dijle, Belgium) on temperature ( o C) and dissolved oxygen (mg/l) taken from March 1991 to December 1997, can be found in OxygenTempData.txt. Can temperature (X) can be used for predicting the amount of dissolved oxygen (Y ). The data yield MSE = and MSR = Solution: A scatterplot of the data suggests that the assumption of linearity of the regression line of Y on X, and the assumption of homoscedasticity, are, at least approximately, satisfied but a Q-Q plot of the residuals suggests that the normality assumption may not be realistic; see Example 8.3-3, pp for the plots. Since the sample size is n = 59, the T and F tests can still be applied. Week 12 Hypothesis Testing, Part II

9 Solution continued: The degrees of freedom for MSR, MSE and MST are 1, n 2 = 57, and n 1 = 58, respectively. Thus we have the following ANOVA table: Source df SS MS F Regression Error Total The R command qf(.99, 1, 57) yields F 1,57,0.01 = 7.1. Since the F statistic is > 7.1, H 0 : β 1 = 0 is rejected in favor of H a : β 1 0, meaning that X can be used for predicting Y. Week 12 Hypothesis Testing, Part II

10 Type I and Type II Errors Rejecting H 0 when it is true is called a type I error. Not rejecting H 0 when it is false is called a type II error. Outcome H 0 of test H a H 0 Correct decision Type I Truth H a Type II Correct decision Week 12 Hypothesis Testing, Part II

11 By construction, the (maximum) probability of committing type I error equals the level of significance α: α = P(type I error) = P θ0 (Rejecting H 0 : θ = θ 0 ) The probability of type II error is denoted by β and depends on the true value of θ. If θ a denotes an alternative value, β(θ a ) = P θa (type II error) = P θa (Not rejecting H 0 : θ = θ 0 ) The probability of rejecting H 0 when it is false is called the power of the test procedure: Power at θ a = 1 β(θ a ) Week 12 Hypothesis Testing, Part II

12 We want small probability for committing type I error, and plenty of power (small probability for type II error). However, we cannot decrease the probability of both types of errors (while keeping the sample size fixed). This is because the events involved are complementary: Shrinking the RR (thereby decreasing α), expands the complement of it (thereby increasing the probability of type II error). This is demonstrated in the following example. Week 12 Hypothesis Testing, Part II

13 Example Using X Bin(20, p), it is desired to test H 0 : p 0.9 vs H a : p < 0.9. For each of the the two RR find α, and β(p a = 0.8). Rule 1: X 16, Rule 2: X 17, Solution: For rejection rule 1 we have α = P(X 16 p = 0.9, n = 20) = β(0.8) = P(X > 16 p = 0.8, n = 20) = = Week 12 Hypothesis Testing, Part II

14 Example (Continued) For rejection rule 2 we have α = P(X 17 p = 0.9, n = 20) = 0.323, β(0.8) = P(X > 17 p = 0.8, n = 20) = = Week 12 Hypothesis Testing, Part II

15 Precision in hypothesis testing is quantified by the power of the test at a particular value of interest of the parameter in the alternative hypothesis region. In the previous example, if detecting the alternative of p = 0.8 is of interest, the procedure that rejects H 0 when X 16 has precision about = If a higher level of precision is desired, then the increase in power should not be accomplished at the expense of higher probability of type I error. In the previous example the precision increase from 0.59 to almost 0.8 was accomplished at the expense of the type I error probability which increased from about 0.13 to about Week 12 Hypothesis Testing, Part II

16 The power of a test procedure increases with the sample size. Power n = 10 n = True value of the mean Figure: Power functions for H 0 : µ 0 vs H a : µ > 0 Week 12 Hypothesis Testing, Part II

17 As the figure of power functions suggests, the power also increases as the parameter value moves farther away from the null hypothesis. Consider again the rule X 16 for rejecting H 0 : p 0.9, where X Bin(20, p). Find the power of this test at p0.6. Solution: power(0.6) = P(X 16 p = 0.6, n = 20) = We will want to achieve a desired level of precision through sample size selection. Formulas for sample size determination follow from those for the probability of type II error, which we give first. Week 12 Hypothesis Testing, Part II

18 Power Calculations for Z tests Example The change of the tire design is justifiable only if the average lifetime of tires with the new design exceeds 20,000 miles. The lifetimes are normally distributed with σ = 1, 500. n = 16 new tires will be used for the decision. What is the probability of rejecting H 0 : µ = 20, 000, at α = 0.01, when µ = 21, 000? Solution: Here, H a : µ > 20, 000. Since σ is known, we will use ) Z H0 = (X µ 0 /(σ/ n) N(0, 1), so that the appropriate rejection region is Z H0 > z α. Thus, Week 12 Hypothesis Testing, Part II

19 β(21, 000) = P(type II error µ = 21, 000) ( ) X µ0 = P σ/ n < z α µ = 21, 000 ( ) σ = P X < µ 0 + z α n µ = 21, 000 ( µ0 21, 000 = Φ σ/ + z α ). n With µ 0 = 20, 000, σ = 1, 500, and α = 0.01, so that z α = 2.33, we obtain β(21, 000) = Φ( 0.34) = , so the power at µ a = 21, 000 is = Week 12 Hypothesis Testing, Part II

20 Formulas for β(µ a ) The formulas for the probability of type II error, when testing H 0 : µ = µ 0 against the different alternative hypotheses, are: H a β(µ ( a ) for a Level ) α test µ µ > µ 0 Φ 0 µ a σ/ n + z α ( ) µ µ < µ 0 1 Φ 0 µ a σ/ n z α ( ) ( ) µ µ µ 0 Φ 0 µ a σ/ n + z µ α/2 Φ 0 µ a σ/ n z α/2 Week 12 Hypothesis Testing, Part II

21 Formulas for β(p a ) The formulas for the probability of type II error, when testing H 0 : p = p 0 against the different alternative hypotheses, are: H a p > p 0 p < p 0 p p 0 β(p ( a ) for a Level α test ) p Φ 0 p a+z α p0 (1 p 0 )/n pa(1 pa)/n ( ) p 1 Φ 0 p a z α p0 (1 p 0 )/n pa(1 pa)/n ( Φ ) ( p 0 p a+z α p0 (1 p 0 )/n Φ pa(1 pa)/n ) p 0 p a z α p0 (1 p 0 )/n pa(1 pa)/n Week 12 Hypothesis Testing, Part II

22 Formulas for n when testing for µ The formulas for the sample size needed to achieve a prescribed level, β, of probability of type II error, when testing H 0 : µ = µ 0 against the different alternative hypotheses, at level α, are H a µ > µ 0 n = µ < µ 0 n = µ µ 0 n = Sample Size Needed for β(µ a ) to Equal β [ ] 2 [ µ 0 µ a ] 2 σ(z α+z β ) µ 0 µ a σ(z α+z β ) [ ] 2 σ(z α/2 +z β ) µ 0 µ a Week 12 Hypothesis Testing, Part II

23 Example Consider testing H 0 : µ = 20, 000 assuming that the tire life times are normally distributed with σ = 1, 500, and find the sample size needed to achieve β(21, 000) = 0.1. Solution. The alternative hypothesis here is H a : µ > 20, 000. According to the formula n = [ ] 1, 500(zα + z 0.1 ) 2 = µ 0 21, 000 = 29.32, which is rounded up (as usual) to n = 30. [ ] 1500( ) 2 = (5.42) 2 20, , 000 Week 12 Hypothesis Testing, Part II

24 Formulas for n when testing for p The formulas for the sample size needed to achieve a prescribed level, β, of probability of type II error, when testing H 0 : p = p 0 against the different alternative hypotheses, at level α, are H a Sample Size Needed for β(p a ) to Equal β [ ] 2 z p > p 0 n = α p0 (1 p 0 )+z β pa(1 pa) p 0 p a [ ] 2 z p < p 0 n = α p0 (1 p 0 )+z β pa(1 pa) p 0 p a [ ] 2 z p p 0 n = α/2 p0 (1 p 0 )+z β pa(1 pa) p 0 p a Week 12 Hypothesis Testing, Part II

25 Example It is thought that more than 70% of all faults in transmission lines are caused by lightning. To gain evidence in support of this contention a random sample of n faults from a large data base will be examined. What should n be to assure β(0.8) = 0.1 when testing at α = 0.01? Solution: Here, H a : p > 0.7. According to the formula, n = [ ] z (1 0.7) + z (1 0.8) 2 = Thus, use n = 250. Week 12 Hypothesis Testing, Part II

26 Comparing Two Means Comparing Two Proportions The comparisons will be based on a simple r.s. from each of the two populations: X 11,..., X 1n1 from population 1, and X 21..., X 2n2 from population 2. In the Bernoulli case only p i (or T i = n i p i ) is typically given. With the exception of the section on paired data, we will make the further assumption that the two samples are independent. The sample mean and sample variances are denoted by X i = 1 n i X ini, Si 2 = 1 n i ( ) 2 X ij X i, i = 1, 2. n i n i 1 j=1 j=1 Week 12 Hypothesis Testing, Part II

27 Comparing Two Means Comparing Two Proportions Comparing Two Means Comparing Two Proportions Week 12 Hypothesis Testing, Part II

28 The two cases Outline Comparing Two Means Comparing Two Proportions We will present T CIs and hypothesis tests for the difference µ 1 µ 2 of two population means. The form of the CIs and tests statistics depends on whether or not the two population variances are assumed to be equal. The reason for this is because the two cases, i.e., σ1 2 = σ2 2 and σ1 2 σ2 2 lead to different distributional results. The distributional results are given first. Week 12 Hypothesis Testing, Part II

29 Comparing Two Means Comparing Two Proportions Result 1: σ 2 1 = σ2 2 If σ 2 1 = σ2 2, under normality and for any n 1, n 2, X 1 X 2 (µ 1 µ 2 ) ( ) T n1 +n 2 2, n1 n2 S 2 p where S 2 p = (n 1 1)S (n 2 1)S 2 2 n 1 + n 2 2 is the pooled estimator of the common variance. Without normality, the above is approximately true if n 1, n Week 12 Hypothesis Testing, Part II

30 Comparing Two Means Comparing Two Proportions Result 2: No assumption about σ 2 1 = σ2 2 Under normality, and without assuming that σ 2 1 = σ2 2, where ν = X 1 X 2 (µ 1 µ 2 ) S 2 1 n 1 + S2 2 n 2 ( S 2 1 n 1 + S2 2 n 2 ) 2 (S1 2/n 1) 2 n (S2 2 /n 2) 2 n 2 1 T ν, is the Smith-Satterthwaite degrees of freedom. (Note: [x] means x rounded down.) Without normality the above approximation holds if n 1, n Week 12 Hypothesis Testing, Part II

31 Comparing Two Means Comparing Two Proportions T CIs for µ 1 µ 2 Let σ1 2 = σ2 2. Under normality a (1 α)100% CI is: ( 1 X 1 X 2 ± t α/2,n1 +n 2 2 Sp ). n 1 n 2 Under normality an approximate (1 α)100% CI is: X 1 X 2 ± t α/2,ν S 2 1 n 1 + S2 2 n 2, where ν is the Smith-Satterthwaite DF. Without normality both hold approximately if n 1, n Week 12 Hypothesis Testing, Part II

32 Comparing Two Means Comparing Two Proportions A rule of thumb, for deciding if σ 2 1 σ2 2, i.e. that σ2 1, σ2 2 are approximately equal, is: max{s 2 1, S2 2 } min{s 2 1, S2 2 } < 5 if n 1, n if n 1, n if n 1, n 2 30 Rule of thumb for checking if σ 2 1 σ2 2 Week 12 Hypothesis Testing, Part II

33 Comparing Two Means Comparing Two Proportions T Tests for µ 1 µ 2 We will test H 0 : µ 1 µ 2 = 0 vs one of the following H a µ 1 µ 2 > 0, or µ 1 µ 2 < 0, or µ 1 µ 2 0. The test statistic is T H0 = X 1 X 2 0, where If σ 2 1 = σ2 2, σ X 1 X 2 = S 2 p σ X 1 X 2 ( 1 n n 2 Without assuming σ 2 1 = σ2 2, σ X 1 X 2 = ). S 2 1 n 1 + S2 2 n 2. Week 12 Hypothesis Testing, Part II

34 Comparing Two Means Comparing Two Proportions The rejection regions are: where H a RR at level α µ 1 µ 2 > 0 T H0 > tα µ 1 µ 2 < 0 T H0 < tα µ 1 µ 2 0 T H0 > tα/2 If σ 2 1 = σ2 2, t α = t α,n1 +n 2 2, t α/2 = t α/2,n 1 +n 2 2. Without assuming σ 2 1 = σ2 2, t α = t α,ν, t α/2 = t α/2,ν, where ν denotes the Smith-Satterthwaite DF. Without normality these test procedures require n 1, n Week 12 Hypothesis Testing, Part II

35 Comparing Two Means Comparing Two Proportions The p-values are: 1 G ν (T H0 ) for µ 1 µ 2 > 0 P-value = G ν (T H0 ) for µ 1 µ 2 < 0 2 [ 1 G ν ( T H0 ) ] for µ 1 µ 2 0 where ν = n 1 + n 2 2 if σ 2 1 = σ2 2, and ν = ν, the Smith-Satterthwaite DF, if σ1 2 = σ2 2 is not assumed. Week 12 Hypothesis Testing, Part II

36 Comparing Two Means Comparing Two Proportions Example A simple r.s. of n 1 = 32 specimens of cold-rolled steel give average strength X 1 = 29.8 ksi, and S 1 = 4. A simple r.s. of n 2 = 35 specimens of two-sided galvanized steel give X 2 = 34.7 ksi, and S 2 = Construct a 99% CI for µ 1 µ 2. Solution. Assuming σ1 2 = σ2 2, we calculate the pooled variance Sp 2 = and use t 0.005,65 = This gives a 99% CI of ( 8.54, 1.26). [NOTE: By the rule of thumb σ 1 σ 2.] Without the assumption of σ1 2 = σ2 2, we find the Smith-Satt. DF to be ν = 56.12, and use t 0.005,56 = This gives a 99% CI of ( 8.48, 1.32). Week 12 Hypothesis Testing, Part II

37 Comparing Two Means Comparing Two Proportions Example (Cold-rolled and two-sided galvanized steel) Are the strengths of the two types of steel different? Test at α = 0.01 and compute the p-value. Solution. Since zero is not included in the 99% CIs, H 0 : µ 1 = µ 2 is rejected at α = 0.01 with both test statistics. As illustration, without the assumption of σ1 2 = σ2 2, the TS is ) S 2 T H0 = (X 1 X 2 / 1 n 1 + S2 2 n 2 = Since T H0 = 3.65 > t 0.005,56 = 2.666, H 0 is rejected. The p-value is 2 [ 1 G 56 (3.65) ] = Week 12 Hypothesis Testing, Part II

38 Comparing Two Means Comparing Two Proportions Comparing Two Means Comparing Two Proportions Week 12 Hypothesis Testing, Part II

39 Comparing Two Means Comparing Two Proportions Z CIs for p 1 p 2 When n i p i 8, n i (1 p i ) 8 for both i = 1, 2, ) p 1 p 2 N (p 1 p 2, σ 2 p1 p2, where σ 2 p 1 p 2 = p 1 (1 p 1 ) n 1 + p 2 (1 p 2 ) n 2. Thus the (approximate) (1 α)100% CI for p 1 p 2 is p 1 (1 p 1 ) p 1 p 2 ± z α/2 + p 2 (1 p 2 ). n 1 n 2 Week 12 Hypothesis Testing, Part II

40 Comparing Two Means Comparing Two Proportions Z Tests for p 1 p 2 We will test H 0 : p 1 p 2 = 0 vs H a : p 1 p 2 > 0, or p 1 p 2 < 0, or p 1 p 2 0. The test statistic for testing this null hypothesis is Z H0 = p 1 p 2 0, where σ p1 p 2 = σ p1 p 2 p 1 (1 p 1 ) n 1 + p 2 (1 p 2 ) n 2. Week 12 Hypothesis Testing, Part II

41 Comparing Two Means Comparing Two Proportions For testing H 0 : p 1 = p 2 (so 0 = 0), it is customary to use: ( 1 σ p1 p 2 = p(1 p) + 1 ). n 1 n 2 where p = n 1 p 1 + n 2 p 2 n 1 + n 2. Thus the TS for testing H 0 : p 1 = p 2 is: Z H0 = p 1 p 2 σ p1 p 2. Week 12 Hypothesis Testing, Part II

42 Rejection Rules and p-values Comparing Two Means Comparing Two Proportions The rules for rejecting H 0 in favor of H a are: H a RR at level α p 1 p 2 > 0 Z H0 > z α p 1 p 2 < 0 Z H0 < z α p 1 p 2 0 Z H0 > z α/2 The p-value is computed the same way as for the one-sample Z -test: 1 Φ(Z H0 ) for upper-tailed test P-value = Φ(Z H0 ) for lower-tailed test 2 [ 1 Φ( Z H0 ) ] for two-tailed test Week 12 Hypothesis Testing, Part II

43 Comparing Two Means Comparing Two Proportions Example Tractors are assembled in assembly lines L 1 and L 2. A simple r.s. of 200 tractors from L 1 yield 16 tractor requiring adjustment. A simple r.s. of 400 tractors from L 2 yield 14 requiring extensive adjustments. Do a 99% CI for p 1 p 2, the difference of the two proportions. Solution. Here p 1 = 16/200 = 0.08, p 2 = 14/400 = 0.035, and z α/2 = z.005 = Thus the 99% CI is ± (.08)(.92) (.05)(.95) 400 = ( 0.012, 0.102). Week 12 Hypothesis Testing, Part II

44 Comparing Two Means Comparing Two Proportions Example (Assembly lines L 1 and L 2 continued) It is suspected that a higher proportion of tractors from L 1 require adjustments. Formulate a hypothesis testing problem, test at α =.01, and calculate the p-value. Solution. We want to test H 0 : p 1 = p 2 vs H a : p 1 > p 2. The pooled estimate of the common (under H 0 ) p is p = ( )/600 = The TS is Z H0 = (.05)(.95)(1/ /400) = Since 2.38 > z.01 = 2.33, H 0 is rejected. The p-value is p-value = 1 Φ(2.38) = = Week 12 Hypothesis Testing, Part II