6.4 Type I and Type II Errors

Size: px

Start display at page:

Download "6.4 Type I and Type II Errors"

Alexandra Lauren Burke
6 years ago
Views:

1 6.4 Type I and Type II Errors Ulrich Hoensch Friday, March 22, 2013

2 Null and Alternative Hypothesis Neyman-Pearson Approach to Statistical Inference: A statistical test (also known as a hypothesis test) revolves around two hypotheses, a null hypothesis H 0 and an alternative hypothesis H a. A decision of a statistical test is either accept H 0 and reject H a, or reject H 0 and accept H a. A hypothesis is said to be a simple hypothesis if it uniquely specifies the distribution from which the sample is taken. Otherwise, it is called a composite hypothesis.

3 We want to investigate whether a certain coin is fair. Let p be the (unknown) probability of heads. We choose the null hypothesis to express that the coin is fair: H 0 : p = 0.5. There are at least three choices for the alternative hypothesis. H a : p 0.5; this means the coin is not fair (two-sided alternative). H a : p > 0.5; the coin is biased in favor of heads (one-sided alternative). H a : p < 0.5; the coin is biased in favor of tails (one-sided alternative). Note that in this example H 0 is a simple hypothesis, and all choices for H a are composite. We may also choose a simple alternative such as H a : p = 0.8.

4 Type I and Type II Error A type I error occurs if H 0 is rejected when in fact H 0 is true. The probability of a type I error is denoted by α. Thus, α = P(reject H 0 H 0 is true). A type II error occurs if H 0 is accepted when in fact H a is true. The probability of a type II error is denoted by β. Thus, Remarks. β = P(accept H 0 H a is true). Ideally, we want both errors to be small; generally, however, decreasing α is accompanied with increasing β, and v.v. To compute α, we need a simple null hypothesis; to compute β, we need a simple alternative hypothesis.

5 In the coin-tossing example, choose H 0 : p = 0.5. Suppose we decide to reject H 0 if in a sample of 10 coin tosses, heads comes up 8 or more times. We know that for a fair coin, the number X of heads in 10 independent trials has the B(10, 0.5) distribution. Thus, α = P(X 8 H 0 is true) = 1 binomcdf(10,0.5,7) There is about a 5% chance of concluding that the die is biased when it is in fact fair. If we choose the simple alternative H a : p = 0.8, then under H a, X B(10, 0.8). We accept H 0 if X 7. Thus, β = P(X 7 H a is true) = binomcdf(10,0.8,7) There is about a 32% chance of concluding that the die is fair if it is in fact biased with proportion of heads being 80%.

6 We have a normal population with known variance σ 2 = 9, and the population mean is unknown. Suppose we want to test the null hypothesis H 0 : µ = 15 against the alternative hypothesis H a : µ = A random sample of size n = 36 is collected from the population, and the sample mean x is recorded. Which decision should we make if we want the test to have α = 5%? We know that if H 0 is true, X N(15, 9/36) = N(15, 1/4). We reject H 0 if a sample mean greater than c is observed: P(X > c H 0 is true) = α = 0.05, or equivalently, P((X 15)/ 1/4 > (c 15)/ 1/4) = Now, (X 15)/ 1/4 N(0, 1), and the 95th percentile of a standard normal distribution is z = This means (c 15)/ 1/4 = 1.645, or c = We reject H 0 if x >

7 What is the value of β if H a : µ = 16.5? If µ = 16.5, X N(16.5, 9/36) = N(16.5, 1/4). The probability of a type II error is β = P(X < H a is true) = normalcdf( 1E99, , 16.5, 1/4) = Accept H 0 Reject H 0 Β 8.8 Α 5 13

8 Statistical Power and Effect Size The power of a statistical test is one minus to probability of a type II error: power = 1 β. Thus, a test should ideally have small α and large power. The effect size of a statistical test with null hypothesis H 0 : θ = θ 0 and alternative hypothesis H a : θ = θ 1 is the absolute difference between the two hypothesized parameters: effect size = θ 1 θ 0. In the example above, the power of the test is 1 β = = 91.2%, and the effect size is = 1.5.

9 A survey is to be conducted about student satisfaction with a college s wireless internet service. One of the questions on the survey is: Overall, are you satisfied with the speed and dependability of RMC s wireless internet service? College administrators think that improvements to the wireless infrastructure are needed if only half of the students answer yes to this question, and they are satisfied with the college s wireless if 70% of students answer yes. Thus, we should conduct a one-sample z-test for the proportion of students who answered yes using the hypotheses H 0 : p = 0.5, H a : p = 0.7. What sample size is required so that the statistical test will have α = 0.05 and β = 0.10, and what is the critical value for ˆp?

10 Under H 0 : p = 0.5, the sample proportion has the (approximate) distribution ˆp N(0.5, (0.5)(0.5)/n), or: ˆp 0.5 (0.5)(0.5)/n N(0, 1). The cutoff for the top 5% of a standard normal distribution is invnorm(0.95, 0, 1) = 1.645, so we have ˆp 0.5 (0.5)(0.5)/n = (1)

11 Under H a : p = 0.7, the sample proportion has the (approximate) distribution ˆp N(0.7, (0.7)(0.3)/n), or: ˆp 0.7 (0.7)(0.3)/n N(0, 1). The cutoff for the lower 10% of a standard normal distribution is invnorm(0.10, 0, 1) = 1.282, so we have ˆp 0.7 (0.7)(0.3)/n = (2)

12 Solving equations (1) and (2) gives ˆp = and n = Thus, if the cutoff for the sample proportion is ˆp = and the sample size is n = 50, the test will have a significance level of α = 5% and a power of 90%. Accept H 0 Reject H 0 Β 10 Α 5 0.0

13 Homework Problems for Section 6.4 (Points) p : (use H a : µ > 10, 2), (3), (3), (2). Homework problems are due at the beginning of the class on Wednesday, April 3, 2013.

Lecture Testing Hypotheses: The Neyman-Pearson Paradigm

Lecture Testing Hypotheses: The Neyman-Pearson Paradigm Math 408 - Mathematical Statistics Lecture 29-30. Testing Hypotheses: The Neyman-Pearson Paradigm April 12-15, 2013 Konstantin Zuev (USC) Math 408, Lecture 29-30 April 12-15, 2013 1 / 12 Agenda Example: