With our knowledge of interval estimation, we can consider hypothesis tests

Transcription

1 Chapter 10 Hypothesis Testing 10.1 Testing Hypotheses With our knowledge of interval estimation, we can consider hypothesis tests An Example of an Hypothesis Test: Statisticians at Employment and Immigration Canada believe that the average duration of unemployment in Alberta is less than 6weeks. They want to test: H 0 : μ 6 H 1 : μ>6. NOTE: 1. H 0 is called the null hypothesis. Itistypicallywhatweareinterestedin. Itis a maintained hypothesis that is held to be true unless sufficient evidence to the contrary is obtained. 2. H 1 is called the alternative hypothesis. Itisahypothesisagainstwhichthenull hypothesis is tested and which will be held to be true if the null is held false. 1

2 2 CHAPTER 10. HYPOTHESIS TESTING The Null Hypothesis, H 0 States the assumption (numerical) to be tested Example: The average number of TV sets in U.S. Homes is equal to three ( H 0 :μ = 3 ) Is always about a population parameter, not about a sample statistic H 0 :μ = 3 H 0 : X = 3 Statistics for Busi ness and Economi cs, 6e 2007 Pearson Education, Inc. Chap 10-4 Figure 10.1:

3 10.2. AN TWO SIDED HYPOTHESIS TEST EXAMPLE An Two Sided Hypothesis Test Example A firm s sales records show that customers spend on average $5.50 per month on their product. They wish to know whether this has changed using a significance level of.01 this year. They survey 30 customers and find that the mean expenditure is $5.10 with a sample standard deviation of $.90. We can follow the steps outlined earlier: 1. Formulate Null H 0 : μ =5.50, null hypothesis H A : μ 6= 5.50, alternative hypothesis 2. Level of Significance of test say α = Calculate test statistic as t cal = X μ 0 s/ n = / 30 = Critical Region Rejection rule reject H 0 if t cal >t α 2,n 1 The critical value is t.01/2,29 =2.756 ( n =30) We do not reject since t cal <t.01/2,29 = Interpretation and Notes 1. We say the null hypothesis that the population mean is equal to 5.5 is not rejected at the 1% level of significance. This is to make it clear that it might be if we were to choose a higher level of significance (Try α =.10). Notice we never accept a null hypothesis. 2. Theideaisthatdifferences between X and μ 0 are not significant, since it could arise from sampling variability under the null. Even if H 0 were true we would expect to see some samples with X > If we reject H 0 it implies the difference between X and μ 0 is too large to be attributed to ordinary sampling variability.

4 4 CHAPTER 10. HYPOTHESIS TESTING 4. The whole trick in hypothesis testing is to figure out the correct statistic to construct (one with a known sample distribution) and then to find a rejection region. Drawing a picture often helps one avoid mistakes with rejection regions. 5. We know if the X i are normal then we can use the t-distribution in situations where we are estimating the variance. 6. On the other hand, if X i are not normally distributed we can appeal to the central limit theorem so that X is approximately normally distributed as n gets large, so that we can still use the t-distribution. Also as we have seen for n>30 the t- distribution is close to the normal and often critical values are calculated directly from the normal tables Confidence Intervals and Hypothesis Tests A final relation between hypothesis tests and confidence intervals can be stated. If one calculates a 95% confidence interval for μ and finds that the value μ 0 is contained in the interval, then we know that the null hypothesis (H 0 : μ = μ 0 ) would not be rejected at the 5% level of significance. Similarly if the confidence interval did not contain the value μ 0,thenthenull hypothesis is rejected. Once we have calculated the confidence interval, we have in fact obtained all the possible null hypotheses that would be retained at the chosen significance level forthisparticularsample. The two-sided confidence interval is: X ± t.01/2,29 s n The calculation gives: (4.65,5.56) which contains 5.5, the null hypothesis Definitions and Terms 1. Test Statistic: A test statistic is a random variable whose value determines whether we reject or do not reject the null hypothesis. 2. Decision or Rejection Rule: A decision rule specifies the set of values for the test statistic for which the null hypothesis H 0 will be rejected and the set of values for which H 0 will not be rejected.

5 10.7. SUMMARY OF CONCEPTS OF A HYPOTHESIS TEST 5 3. Critical Region or Rejection Region: The critical region of a test consists of all the values of the test statistic for which H 0 will be rejected. 4. Non-Rejection Region: The non-rejection region of a test consists of all the values of the test statistic for which H 0 will not be rejected. 5. Critical Values: Critical values of a test statistic separate the critical region from the non-rejection region. 6. Level of Significance (Usually denoted as α): The level of significance of a test is the probability that the test statistic lies in the critical region or rejection region when H 0 is true 7. Two-Sided Alternative: An alternative hypothesis involving all possible values of a population parameter other than the value specified by a simple null hypothesis. 8. One-Sided Alternative: An alternative hypothesis involving all possible values of a population parameter on either one side or the other of (that is, either greater than or less than) the value specified by a simple null hypothesis Summary of Concepts of a Hypothesis Test Judgments in the form of the hypothesis testing involve an a priori assumption about the value of an unknown parameter. If the sample information provides evidence against the null hypothesis we reject it, otherwise we do not reject it. The evidence from a sample is summarized in the form of a test statistic which is used at arriving at a verdict concerning the hypothesis Steps in Conducting an Hypothesis Test 1. Formulate the null and alternative hypotheses (H 0,H 1 ). 2. Choose the level of significance α and hence define the critical value (i.e. divide the region into rejection and non-rejection regions. 3. Calculate the test statistic using sample information. 4. If the calculated statistic falls within the rejection region, reject the null hypothesis; if it is in the non-rejection region, do not reject the null hypothesis.

6 6 CHAPTER 10. HYPOTHESIS TESTING 10.9 An Generic Example for Two-Sided Alternative: [Transparency 9.4] 1. Formulate Null and Alternative Hypotheses H 0 : μ = μ 0 H 1 : μ 6= μ Choose level of Significance α (say 5% level) 3. Calculate test statistic (with data) t cal = X μ 0 s n We are testing on the basis of sample information whether μ = μ 0 (where μ 0 is aspecified (known) number). In this case the test statistic is a t-test. Ifthenull hypothesisistrue(μ = μ 0 ) then this statistic is distributed as a t n Critical Region for decision rule: Reject H 0 if t cal >t α/2,n 1 Do not Reject H 0 if: t α/2,n 1 <t cal <t α/2,n 1 Notice carefully that there are really two critical values for two-sided tests: ±t α/2,n 1

7 ONE-SIDED ALTERNATIVES One-Sided Alternatives The hypothesis tests above was for a two-sided alternative H 1 : μ 6= μ 0. Suppose in the previous example it was thought that sales probably had fallen from 5.5 (everyone was confident that they could rule out a rise in sales) We might wish to incorporate this belief right into the hypothesis test This is accomplished by a one-sided alternative: Redo example for this 1. Formulate the Hypothesis (for some specified value of μ 0 ) H 0 : μ 5.5 H 1 : μ<5.5. Notice that the alternative is narrowed in the direction where we think sales are (in the event that the null is false) 2. Level of significance is still α 3. The test statistic is unchanged 4. Decision rule is beased on the critical value t α,n 1 (not t α 2,n 1 )sothatwereject thenullhypothesisif t cal < t α,n 1 Otherwise we retain or do not reject H 0. The calculated value is unchanged at but t α,n 1 = 2.462, which meansthatwebarelyretainthenullhypothesisfortheone-siedalternative Notes on One-Sided Alternatives This is an example of a one-sided test, since the alternative hypothesis includes either the less than < or the greater than > condition. H 0 : μ μ 0 H 1 : μ<μ 0.

8 8 CHAPTER 10. HYPOTHESIS TESTING We could change the nature of the critical value (and hence the rejection and non-rejection region) by changing the hypothesis test to: H 0 : μ μ 0 H 1 : μ>μ 0. [Transparency 9.3] In this case we would calculate the same test statistic as above but the rejection rule would be t cal >t α,n 1 The inequality for H 1 is a useful memory aid to decide whether you want to use the positive critical value ( >) or the negative critical value (<) Of course the two-sied alternative you use ± critical value ( 6=) Reason for a One-Sided Alternative We note that since t α, n 1 <t α 2,n 1, that for the same calculated value it is possible, to retain the null hypothesis for the two-sided alternative while rejecting for the one-sided alternative Whether we want to reject the null or not depends on whether it is true or not We never know whether the null hypothesis is true (afterall why would we test it if we knew) TypeI&TypeIIErrors It is very easy to lose sight of the fact that we DO NOT KNOW whether the null is true or not (if we did why do we need to do any test). There are 2 kinds of errors we can make: 1. Type I Error: Rejecting H 0 when H 0 is true 2. Type II Error: Not Rejecting H 0 when H 0 is false

9 PROBABILITY OF TYPE I AND II ERRORS 9 Decision/State of W orld H 0 is true H 0 is false Do not Reject Reject H 0 Correct Decision (1 α) Type I Error (α) Type II Error (β) Correct Decision (called power) (1 β) Probability of Type I and II Errors α = P (Type I Error) =P (we reject H 0 H 0 is true) = P (Test statistic lies in the rejection region H 0 is true). β = P (TypeII Error) =P (we do not reject H 0 H 0 is false). Power =1 β Power measures the probability of correctly rejecting H 0 when H 0 is false Example of Probability of Type I and II 1. What is the P (Type I Error)? Answer α which for the above example α = What is the P(Type II Error) and Power? To answer this question we must consider values for μ that are in H 1 and Calculate the probabilities. of retaining H 0 under various values in the alternative ThenullcanbefalseinMANYwaysunderthealternative: Power Calculation Let us calculate the probability of retaining H 0 : μ =5.5 when the true μ A =5.1 (which also happens to be the sample mean, but other values could also be chosen). Suppose Truth: μ A =5.1 Test Null at: μ =5.5 What is our Decision Rule?: Rejection rule: reject H 0 if t > 2.756

10 10 CHAPTER 10. HYPOTHESIS TESTING Calculating P(Type II Error) and Power 1. We assume that the variance is unchanged under H 0 and H 1 and use the estimate s n. 2. We want to calculate what are the critical values in terms of X. 3. We know that we retain H 0 : μ = 5.5 whenever our calculated t-statistic t cal < P { < X μ 0 s < 2.756} =.99 n which after some manipulation can be written P {μ s < X < μ n s } =.99 n Substituting μ 0 =5.5 and the estimated standard deviation gives: P { < X < } = This leads to 99% critical values in terms of the sample the sample mean X: (5.047, 5.953) (10.1) This gives all the values for the sample mean that would not be rejected for H 0 : μ =5.5 at the α =.01 level of significance. Note that our sample mean X =5.1 is in the interval and hence we did not reject the null hypothesis We want to find out the probability of being the interval (5.047, 5.953) for various values in the alternative we start with μ A = Calculate the probability of a Type II Error β = P {Type II Error} = P {we do not reject H 0 H 0 false} = P {Test Statistic lies in non rejection region H 0 is false] = P {5.047 < X < μ A =5.1} = P { < X μ A s n = P [.3225 < Z < } = P {(Z <.3225} =.626 < ] 30 Power =1 β =1.626 =.374

11 NOTES ON POWER Notes on Power The probability of a Type II error when testing H 0 : μ =5.5 when the true value of μ =5.1 Note the interval (5.047, 5.953) is not the same as the confidence interval The confidence interval for the population mean μ X ± t.01/2,29 s n which gives (4.65, 5.56) We can repeat this calculation for all possible alternatives under H 1 : μ A 6=5.5. For example let us do another calculation on the other side of the null, say μ A =5.7 β = P (5.047 < X < μ A =5.7) Ã P.9 < X! μ A s <.9 30 n 30 = P ( <Z<1.594) = F Z (1.594) =.9441 Now we can calcultate Probability of Type II Error and Power for a variety of values under H A Power Calculations for Testing H 0 : μ =5.5 Value under H A Probability of Type II Error= β Power =1 β μ A = μ A = μ A = μ A = μ A = μ A = μ A = μ A = μ A = Power Curve: Plot of power on y axis and value of μ A on x -axis

12 12 CHAPTER 10. HYPOTHESIS TESTING Notes on Type I and II Error We can make a Type I error only when we reject H 0 and a Type II error only when we do not. We want both α and β to be small. While we would like both Type I and Type II Errors to be as small as possible, there is in fact a trade-off. Suppose the null hypothesis is that those charged with crimes are innocent. Then a legal test which never convicts the innocent (has α =0)wouldfreemany who are guilty (large β). Lowering α will result in a wider non- rejection region which makes it more likely that a false null hypothesis will be retained. To see this redo the above exercise with α =.005 and.05. Since our interest is usually centered on the null hypothesis α is usually chosen to be small; 10 percent or less. The null and alternative are not treated symmetrically; rejecting the null does not imply that the alternative is true. Alternativeisnotundertest. Do not say we accept the alternative. We have seen that the closeristhetruevalueofμ in H A to the value under H 0 the larger is the probability of Type II error and hence the lower is power Prob- or p-values [Transparency 9.10 and 9.11] It is arbitrary that the rejection/non-rejection of a test depends on the choice of α. An alternative way to report one s results is to quote the test statistic with a p-value, orprob-value. This allows the user to choose a particular α and make their own decision using the reported p value A p-value is simply the probability that a test statistic is as large (in absolute values) as that calculated under the null hypothesis. It is simply the area in the statistic s density beyond the point actually observed.

13 TESTING PROPORTIONS [TRANSPARENCY 9.5] Example of a p value In our example of the expenditure on customer sales our test statistic was This hasap-valueof: p = P (t < 2.43) =.011 For two-sided alternatives you will see authors report the p-value as =.022 (reflecting that both large positives and negatives of the statistic are possible). Only 2% chance of observing a mean of 5.1, if the null hypothesis H 0 : μ =5.5 against H 1 : μ 6= 5.5 is true. p values can be found from tables in the textbook or from the tables built into computer packages like STATA Interpretation and Use of P-Values If the p value is less than a chosen level of the test α one rejects the null hypothesis at the α level. p value < α Reject H 0 (10.2) p value α Do not Reject H 0 (10.3) In the above example p value =0.022 Therefore for an α =.05 we would reject H 0 but α =.01 we would retain H 0 One can also report p-values and let readers decide on their own significance levels. We now have all the tools necessary to do any hypothesis test. In the rest of this chapter we will consider other applications Testing Proportions [Transparency 9.5] We can test hypotheses about the number of successes in n trials, or about the proportion of successes.

14 14 CHAPTER 10. HYPOTHESIS TESTING In the binomial distribution we know the standard deviation of X and p under the null which we can use together with the standard normal tables (in fact better approximation results can often be obtained by using t-distributions) for one- or two-sided tests. H 0 : π = π 0 H 1 : π 6= π 0 Form test statistic (either a Z statistic or t depending on the degrees of freedom): t = p π 0 p π0 (1 π 0 )/n Example of Test for Population Proportion Let us return to the mini survey conducted by Employment and Immigration Canada. They survey n =9unemployed persons. We have seen how they tested an hypothesis about the mean duration of unemployment. Now suppose they want to learn the proportion of searchers who receive a job offer within the first six weeks of unemployment. Of the 9 people surveyed, 2 receive such offers. Suppose that: H 0 : π.5 Let α =.05. H A : π<.5 Then the rejection rule is: reject H 0 if t< (t.05,8 =1.860). The test statistic is: t cal = p.5(.5)/9 = The null hypothesis is not rejected at the 5 percent level. You can see in this example that the null is not rejected, even though there seems tobealargegapbetweentheagency shypothesisandthesampleproportion. The null is not easily rejected because the sample is so small, so that sampling variability is large.

15 TESTS OF DIFFERENCES OF POPULATION MEANS 15 The p-value for this problem P(t < 1.667) =.067, again showing that the hypothesis would not be rejected at the 5 percent level Tests of Differences of Population Means Supposethatwehavetwosampleswithn 1 observations, a mean X 1,andsample standard deviation s 1 in the first, and n 2 observations, a mean X 2,andsample standard deviation s 2 in the second. Data with this property are most likely to arise from experiments in which two treatments are applied. The testing problem is: which can be written as: H 0 : μ 1 = μ 2 H 0 : μ 1 μ 2 =0 A general hypothesis test of differences is: H 0 : μ 1 μ 2 = d 0, where d 0 is the hypothesized difference.(usually d 0 =0) Testing Differences when Variances are the same Weassumethatthetwopopulationshavethesamevariance(seeChapters7and 8), σ 1 = σ 2 = σ,

16 16 CHAPTER 10. HYPOTHESIS TESTING The population standard deviation of the difference (assuming independence) is A pooled estimate of σ 2 is s r σ σ2 2 1 = σ + 1. n 1 n 2 n 1 n 2 s 2 pooled = P n1 (X i1 X 1 ) 2 + P n 2 (X i2 X 2 ) 2 n 1 + n 2 2 = (n 1 1)s 2 1 +(n 2 1)s 2 2, n 1 + n 2 2 Then we multiply the root of this by r 1 n n 2. Taking the square root gives us the estimated sample standard deviation: s X 1 X 2 = r s 2 pooled [ 1 n n 2 ] The test statistic is: t cal = X 1 X 2 (μ 1 μ 2 ) s X1 X 2 As before we have three different rejection rules (all use the same test statistic) depending on the alternative: H 0 : μ 1 = μ 2 H 1 : μ 1 6= μ 2 Rejection Rule: t cal >t α/2,n1 +n2 2 then reject H 0 H 0 : μ 1 μ 2 H 1 : μ 1 >μ 2

17 TESTING DIFFERENCES WHEN VARIANCES ARE THE SAME 17 Rejection Rule: t cal >t α,n1 +n2 2 then reject H 0 H 0 : μ 1 μ 2 H 1 : μ 1 <μ 2 Rejection Rule: t cal < t α,n1 +n2 2 then reject H Example of Testing Differences in Population Means A market research firm wishes to know if the mean number of hours of TV watching per week is the same for teenage boys as for teenage girls. The following data were obtained: Boys: n 1 =20, X 1 =24.5, s 2 1 =64 Girls: n 2 =12, X 2 =28.7, s 2 2 =71 Carryoutahypothesistestthatboysandgirlswatchthesamenumberofhoursof TV at the 5% level of significance. H 0 : μ 1 = μ 2 H 1 : μ 1 6= μ 2 Why 2 sided? DO not have any reason to believe girls and boys are different since t cal = q = 1.41 [ ] s 2 pooled = (n 1 1)s 2 1 +(n 2 1)s 2 2 n 1 + n 2 2 = (19)64 + (11)71 30 = As t = 1.41 lies in the non-rejection region (t.05/2,30 = ±2.042) wedonotreject thehypothesisatthe5% level of significancethatboysandgirlswatchthesame number of TV hours. Note that P (t < 1.41) =.084 and therefore the p-value = =.168.

18 18 CHAPTER 10. HYPOTHESIS TESTING Testing Differences when Variances are Different Tests are conducted exactly the same way as before except the formula for s X1 X 2 and the degrees of freedom are different. s X 1 X 2 = s s 2 1 n 1 + s2 2 n 2 and the degrees of freedom is a crazy formula that can be found in the book: v = h s 2 X n X + s2 Y n Y i 2 s 2 X s2 n X + Y n Y (n X 1 ) (n Y 1 ) For convenience I have always used ν = n 1 + n 2 2 and hoped I was not too far off. Redo the above TV example by not assuming the variances are the same. Is there any difference to your conclusions? Testing Differences of Population Proportions Recall that the variance of a sample proportion p j is estimated by p j (1 p j )/n j. Then if we have two independent samples, for hypotheses about the difference between the two population proportions (and hypotheses are always about populations) H 0 : π 1 π 2 = d 0 the test statistic is: H A : π 1 π 2 6= d 0, Z = (p 1 p 2 ) d 0 pp1 (1 p 1 )/n 1 + p 2 (1 p 2 )/n 2.

19 TESTING DIFFERENCES OF POPULATION PROPORTIONS 19 Often the null hypothesis will be that the two proportions are equal: H 0 : π 1 = π 2. In the equality of proportion case the formula simplifies to: Z = p 1 p 2 p ppool (1 p pool )(1/n 1 +1/n 2 ), where p pool = X 1 + X 2 n 1 + n Example of Testing Differences in Population Proportions In a sample of 400 products produced by Machine 1, 23 were defective and in a sample of 400 products produced by Machine 2, 17 were defective. Test: against H 0 : π 1 π 2 =0 using a 5% level of significance. H 1 : π 1 π 2 6=0 Answer Fromthequestionweknow p 1 = p 2 = The pooled estimate p pool is: Therefore p pool = X 1 + X 2 n 1 + n 2 = = Z cal = p = (.05)(.95)(1/ /400)

20 20 CHAPTER 10. HYPOTHESIS TESTING Since Z cal is in the non-rejection region (-1.96,1.96) we do not reject H 0 at the 5% level of significance. Note that since P (Z >.8111) =.21 that the p-value=.21 2 = Testing the Hypothesis μ 1 = μ 2 with Paired Data In testing whether the two means were different form 0 or d 0 we assumed that the two samples were independent. Hence the estimated variance of the difference was the sum of the two variances. We could do the testing under the assumption that the variances were the same (pooled variance) or different. On occasion, we may have paired data. This is data that is grouped or paired so that the variation in responses between the members of any pair are less than the variation between members of different pairs. We can improve the efficiency (lower the variance) of the experiment by randomizing the two treatments over the two members of each pair. We restrict the randomization so that the treatment is given to one member of each pair. and obtain a separate estimate of the difference between the treatment effects for each pair. The variation among the pairs is not included in our estimate of the variance. Hence if this variation is large relative to the variation within pairs, the variance from paired tests will be smaller than that from a completely randomized (independent) sampling experiment. This motivates the use of twins in some experiments. In economics we seldom (ever?) have paired data and so we will not puirsue this matter In hospital, clinical and drug testing setting, this is often the casewher there is paired data