Tests of Hypotheses Based o a Sigle Sample Devore Chapter Eight MATH-252-01: Probability ad Statistics II Sprig 2018 Cotets 1 Hypothesis Tests illustrated with z-tests 1 1.1 Overview of Hypothesis Testig.......... 1 1.2 Example: z tests for populatio mea....... 2 1.3 P -value....................... 2 1.3.1 Case whe H a : µ > µ 0........... 3 1.3.2 Case whe H a : µ < µ 0........... 3 1.3.3 Case whe H a : µ µ 0........... 3 1.4 Type I ad Type II Errors............. 3 1.5 Specifics of z test.................. 4 1.6 False Dismissal Probability............. 4 1.7 Sample Size Determiatio............. 5 2 t-tests ukow variace 5 2.1 Large-Sample Approximatio........... 6 3 Hypothesis Tests ad Cofidece Itervals 6 4 Tests Cocerig Proportio 7 4.1 Large Sample Tests................. 7 Copyright 2018, Joh T. Whela, ad all that 4.1.1 False Dismissal Probability......... 7 4.2 Small Sample Tests................. 8 Thursday 1 February 2018 Note that the structure of Chapter 8 has chaged somewhat betwee the eighth ad ith editios of Devore, due mostly to a pedagogical choice to make P -values more cetral to the discussio. As this is a reasoable choice, we ll be makig the same shift i lecture. For a alterative treatmet which itroduces P - values later, see the previous otes at http: // ccrg. rit. edu/ ~ whela/ courses/ 2016_ 3fa_ MATH_ 252/ otes08. pdf 1 Hypothesis Tests illustrated with z-tests 1.1 Overview of Hypothesis Testig We ow cosider aother area of statistical iferece kow as hypothesis testig. The usual formulatio starts with a ull hypothesis H 0 ad a alterative hypothesis H a, which produce 1
differet probabilistic predictios about the outcome of a experimet, ad the, based o the observed data, decides betwee two alteratives: 1. Reject H 0 2. Do t reject H 0 The full scope of hypothesis testig is quite geeral, but for this itroductio, we ll make some simplifyig assumptios: 1. The data {X i } are a sample of size from a probability distributio with pdf fx; θ or pmf px; θ, if it s a discrete distributio. 2. The ull hypothesis H 0 specifies a sigle value for the parameter θ = θ 0. This is kow as a poit hypothesis because it gives a sigle value of θ completely specifies the distributio. 3. The alterative hypothesis H a specifies some rage of values for θ which are icosistet with θ 0, typically oe of the followig: a θ θ 0 b θ > θ 0 c θ < θ 0 Ay of these is a composite hypothesis because it correspods to a set of θ values, ad therefore to a family of distributios. 4. The test is defied by costructig a statistic Y = ux 1,..., X ad rejectig H 0 or ot accordig to the value of Y. 1.2 Example: z tests for populatio mea For example, cosider a sample of size draw from a ormal distributio with ukow mea µ ad kow variace σ 2. Previously, we used the fact that the sample mea X is a Nµ, σ 2 radom variable to defie a pivot variable X µ σ/ which was kow to be stadard ormal. Now, we ll evaluate a ull hypothesis H 0 which says that µ = µ 0 where µ 0 is some specified value which may be zero, but eed ot be. There are three choices of alterative hypothesis H a : µ µ 0, µ > µ 0, ad µ < µ 0. I each case the test statistic will be Z = X µ 0 σ/ 1.1 If H 0 is true, this is agai stadard ormal [N0, 1]. If H a is true, Z will still be a ormal radom variable with a variace of oe, but its mea will be µ µ 0 σ/. Depedig o the choice of alterative hypothesis, EZ might be kow to be positive, kow to be egative, or simply o-zero. The way we coduct the test is to covert the actual data {x i } ito a actual value z = x µ 0 σ/ where x is the actual observed sample mea. Sice we kow that this test statistic Z is stadard ormal if the ull hypothesis H 0 is true, we should begi to doubt H 0 if the observed z is too far from zero. Depedig o the form of the alterative hypothesis H a, we could reject H 0 if z were too high, or too low, or perhaps either. 1.3 P -value The decisio to reject the ull hypothesis H 0 or ot is based o whether the data seem cosistet with H 0. We quatify that with somethig called the P value. This is the probability that data geerated accordig to the ull hypothesis would geerate a test statistic value at least as extreme as the actual value observed. 2
1.3.1 Case whe H a : µ > µ 0 For cocreteess, suppose the alterative hypothesis H a is that µ > µ 0. The H a tells us the test statistic Z is a ormal radom variable with a variace of oe ad a positive mea. Thus a sufficietly high positive value of z will look icosistet with the ull hypothesis H 0 ad cosistet with the alterative hypothesis H a. A very egative value of z will look cosistet with either hypothesis, but more like H 0 tha H a. We defie the P value as the chace of fidig a test statistic value of z or higher if H 0 is true: P = P Z z H 0 is true = 1 Φz = Φ z 1.2 This is the upper tail probability Table A.3 of Devore. For istace, if z = 1, P 0.8413, but if z = 3, P 0.0013. The lower the P value, the more icosistet the data are with H 0. 1.3.2 Case whe H a : µ < µ 0 If the alterative hypothesis is µ < µ 0, the it s low values of Z which are cosidered aomolous i a way cosistet with the alterative hypothesis, so the P value correspodig to a z score is the probability, uder the ull hypothesis, of fidig a lower or more egative value: P = P Z z H 0 is true = Φz 1.3 1.3.3 Case whe H a : µ µ 0 Now we ca cosider either high or low z scores to be aomolous, so the P value is the probability of fidig Z farther from zero tha z, assumig the ull hypothesis. So if z = 2.5, this is the probability that Z 2.5 or Z 2.5. If z = 1.2, it is the probability that Z 1.2 or Z 1.2. This ca be summarized as P = P Z z H 0 is true = P Z z H 0 is true + P Z z H 0 is true = Φ z + 1 Φ z = Φ z + Φ z 1.4 Type I ad Type II Errors = 2Φ z 1.4 To get back to the cocept of a test as a way to decide betwee rejectig H 0 ad ot rejectig it, this statistic-based method actually defies a whole family of tests. For ay value of α betwee 0 ad 1, we ca defie a test which rejects H 0 if P α. Recall that lower P values are worse ews for H 0. The value α has may ames, but a commo oe is sigificace. Note that this ame ca be a bit cofusig, because a test with a sigificace of α =.01 will be more striget require more evidece to reject H 0 tha oe with a sigificace of α =.05. Because of the radom ature of the experimet, there will be some probability that a give test will reject H 0. Eve if the ull hypothesis H 0 is true, we will geerally have a ozero probability of rejectig it. Likewise, eve if the alterative hypothesis H a is true, the probability that the data will lead us to reject H 0 will still geerally be less tha oe. A perfect test would have us ever reject H 0 if it s true, ad always reject H 0 if H a is true, but i most situatios there is o perfect test. A give test thus has some probability of makig a error. If H 0 is true ad we reject it. this is called a Type I Error, also kow as a false alarm. We have claimed to see a effect which was ot there. If H a is true, but we do ot reject H 0, this is called a Type II Error, also kow as a false dismissal. We have failed to fid a effect which is there. The probability of each 3
of these errors happeig has to be uderstood as a coditioal probability sice it assumes oe hypothesis or the other is true. The probability of a type I error, or the false alarm probability, is P reject H 0 H 0 is true = P P α H 0 is true 1.5 Sice the defiitio of the P value implies that, if H 0 is true, it will be a uiform radom variable betwee 0 ad 1 e.g., we should fid P.2 twety percet of the time, the false alarm probability is actually just the sigificace α: P reject H 0 H 0 is true = P P α H 0 is true = α 1.6 The probability of a type II error, or the false dismissal probability, is β = 1 P reject H 0 H a is true 1.7 Actually, sice we re takig the alterative hypothesis H a to be a composite hypothesis, this depeds o the actual value of θ: βθ = 1 P reject H 0 parameter value θ 1.8 We d geerally like to have α ad β as small as possible. I practice, oe usually decides what false alarm probability α oe ca afford, ad the desigs a test which miimizes βθ for ay θ give that costrait. A related quatity is the power of the test, which is the probability of rejectig H 0 if H a is true. It is writte γθ ad equal to 1 βθ. 1.5 Specifics of z test Returig to the case where we have a sample of size draw from a ormal distributio or populatio with ukow mea µ ad kow variace σ 2, ad ull hypothesis H 0 : µ = µ 0, we cosider the threshold o Z = Xbar µ 0 σ/ correspodig to a desired false alarm probability α. As oted, this correspods i each case to a test which rejects H 0 if P α. Sice P = 1 Φz = P Z z H 0 is true 1.9 ad 1 Φz α = α by defiitio, the threshold of α o P is a threshold of z α o z. Similar calculatios i the other two cases tell us: 1. If H a is µ > µ 0, we reject H 0 if X µ 0 σ/ > z α. This is called a upper-tailed test. 2. If H a is µ < µ 0, we reject H 0 if X µ 0 σ/ < z α. This is called a lower-tailed test. 3. If H a is µ µ 0, we reject H 0 if either X µ 0 σ/ < z α/2 or X µ 0 σ/ > z α/2. This is called a two-tailed test. Practice Problems 8.1, 8.7, 8.9, 8.13, 8.19 Tuesday 6 February 2018 1.6 False Dismissal Probability To get the false dismissal probability βµ, or equivaletly the power γµ = 1 βµ, we eed to cosider the probability of the sample ladig i the rejectio regio for a give µ = µ cosisted with the alterative hypothesis H a. I the case of a ormal distributio with kow σ, the test statistic Z = X µ 0 σ/ will still be ormally distributed, but ow, sice X µ, σ 2 /, the mea of Z will be µ µ 0 σ/. The variace will still be 1. Thus, 4
if H a is µ > µ 0 X βµ µ0 = P σ/ z α µ = µ = Φ z α µ µ 0 σ/ 1.10 while if H a is µ < µ 0 X βµ µ0 = P σ/ z α µ = µ = 1 Φ z α µ µ 0 σ/ = Φ z α µ 0 µ σ/ 1.11 Note that Φz α = 1 α, ad i each case the argumet is less tha z α, so βµ < 1 α, which meas γµ > α. This makes sese, sice you d expect the test to be more likely to reject H 0 if H a is true tha if H 0 is true. For a two-tailed test, the calculatio of the false dismissal probability is also straightforward: βµ = P z α/2 X µ 0 = Φ z α/2 µ µ 0 σ/ σ/ z α/2 µ = µ Φ z α/2 µ µ 0 σ/ 1.7 Sample Size Determiatio 1.12 We ca tur the false dismissal probability expressios for oetailed tests aroud, ad ask what sample size will allow us to produce a test with a specified false alarm probability α ad false dismissal probability β for a omial populatio mea µ. We use the fact that β = 1 Φz β = Φ z β, 1.13 which meas that z β = { zα µ µ 0 σ/ upper tailed z α µ 0 µ σ/ lower tailed 1.14 I either case if we solve for we get the miimum sample size 2 σzα + z β = 1.15 µ µ 0 2 t-tests ukow variace Now suppose we have a sample of size draw from a ormal distributio, but both the mea ad the variace is ukow. If we wat to assess the ull hypothesis H 0 : µ = µ 0, we ca t costruct the usual test statistic X µ 0 σ/ because the stadard devatio σ is ukow. As i the case of cofidece itervals, we use the sample variace as a estimate. The resultig test statistic T = X µ 0 S2 / 2.1 is Studet-t-distributed with 1 degrees of freedom. Agai, this is a cosequece of Studet s Theorem. We the reject the ull hypothesis if the statistic T is too far from zero i the appropriate directio. Specifically, if we wat to defie tests at a fixed sigificace α, they go as follows: 1. If H a is µ > µ 0, we reject H 0 if T > t α; 1. 2. If H a is µ < µ 0, we reject H 0 if T < t α; 1. 3. If H a is µ µ 0, we reject H 0 if either T < t α/2; 1 or T > t α/2; 1. To describe the outcome i terms of the P -value, we eed to use the cumulative distributio fuctio of the t-distributio, 5
F T t; 1 or equivaletly the tail probability area to the right of t 1 F T t; 1 = F T t; 1. The, defiig t = x µ 0 s/, 1. If H a is µ > µ 0, P = F T t; 1 = f t T u; 1 du 2. If H a is µ < µ 0, P = F T t; 1 = t f T u; 1 du 3. If H a is µ µ 0, P = 2F T t ; 1 = f t T u; 1 du Devore has the t tail probability t f T u; ν du 2.2 for various umbers of degrees of freedom tabulated. Note that the false dismissal probability βµ or the power γµ = 1 βµ is ot so easy to calculate for a t-test. This is because T = X µ 0 does t have a simple probability distribu- S 2 / tio whe µ µ 0. 2.1 Large-Sample Approximatio As with cofidece itervals, we ca take advatage of the fact that, as the umber of degrees of freedom becomes large, the Studet-t distributio approaches the stadard ormal distributio. So if > 40 or so, we ca assume the test statistic X µ 0 S 2 / is approximately stadard ormal i fact, it s commo to call it Z rather tha T i this case ad 1. If H a is µ > µ 0, the P 1 Φ x µ 0 s/ ad we reject H 0 if X µ 0 S > z α. 2 / 2. If H a is µ < µ 0, the P Φ x µ 0 s/ ad we reject H 0 if X µ 0 S < z α. 2 / 3. If H a is µ µ 0, the P 2Φ x µ 0 s/ ad we reject H 0 if either X µ 0 S < z 2 α/2 or X µ 0 / S > z α/2. 2 / Thaks to the cetral limit theorem, we do t eve eed to kow that the uderlyig distributio is ormal i the large-sample case. As log as it has a fiite variace, the test statistic based o X ad S 2 will be approximately stadard ormal. 3 Hypothesis Tests ad Cofidece Itervals You may have oticed that the procedures ivolved i coductig a hypothesis test at specified sigificace α ad costructig a cofidece iterval of specified cofidece level 1 α are similar. Let s examie that more closely, usig the example of a sample of size draw from a ormal distributio with ukow µ ad σ. The two-sided cofidece iterval will the be from x t α/2, 1 s2 / to x + t α/2, 1 s2 / 3.1 O the other, had the rules of a two-tailed hypothesis test say Reject H 0 or equivaletly if x µ 0 s2 / t α/2, 1 or x µ 0 s2 / t α/2, 1 3.2 Reject H 0 uless t α/2, 1 < x µ 0 s2 / < t α/2, 1 3.3 A little bit of algebra shows this is equivalet to Reject H 0 uless x t α/2, 1 s2 / < µ 0 < x+t α/2, 1 s2 / 3.4 6
I.e., you reject the ull hypothesis if the specified value µ 0 lies outside the correspodig cofidece iterval. Practice Problems 8.31, 8.37, 8.39, 8.55 Thursday 8 February 2018 4 Tests Cocerig Proportio Now we tur oce agai to the case of a biomial-type experimet, e.g., samplig members from a large populatio where some fractio or proportio p of the members have some desired trait, or doig idepedet trials with a probability p for success o each trial. As usual, the laguage about sample ad radom variables is a little differet. We could cosider this to be a sample of size from a Beroulli distributio Bi1, p, or a sigle biomial radom variable X Bi, p. I ay evet, it s more coveiet to work with the estimator ˆp = X/, which has mea Eˆp = p = p 4.1 ad variace V ˆp = p1 p 2 = p1 p 4.2 We ca cosider two regimes whe testig a ull hypothesis H 0 which states p = p 0 : if p 0 10 ad 1 p 0 10, we ca treat the distributio of ˆp as approximately ormal with the mea ad variace give above, which meas we ca use the testig procedures already defied. If ot, we eed to use the biomial cumulative distributio fuctio to defie ad evaluate the tests. 4.1 Large Sample Tests Supposig the ormal approximatio to be valid, the test statistic appropriate whe the ull hypothesis H 0 is p = p 0 will be Z = ˆp p 0 p0 1 p 0 / 4.3 sice ˆp is approximately Np 0, p 0 1 p 0 /, Z will be approximately stadard ormal, This meas ˆp p 0 P p0 1 p 0 / > z α µ = µ 0 α 4.4a ˆp p 0 P p0 1 p 0 / < z α µ = µ 0 α 4.4b ad [ ] [ ˆp p 0 P p0 1 p 0 / < z ˆp p 0 α/2 α/2] p0 1 p 0 / > z µ = µ 0 That makes the large-sample tests 1. If H a is p > p 0, we reject H 0 if 2. If H a is p < p 0, we reject H 0 if 3. If H a is p p 0, we reject H 0 if either or ˆp p 0 > z p0 1 p 0 α/2. / 4.1.1 False Dismissal Probability ˆp p 0 > z p0 1 p 0 α. / α ˆp p 0 < z p0 1 p 0 α. / 4.4c ˆp p 0 < z α/2 p0 1 p 0 / Estimatig βp for these tests as a fuctio of the actual proportio p is a little differet tha i the case of a populatio 7
mea, sice ow the variace depeds o the parameter p as well, i.e., if p = p, we kow Eˆp = p ad V ˆp = p 1 p /, so p p 0 EZ = 4.5 p0 1 p 0 / ad V Z = p 1 p / p 0 1 p 0 / = p 1 p p 0 1 p 0 4.6 So for example if H a is p > p 0, we have false dismissal probability ad similarly for lower-tailed ad two-tailed test. I geeral, we wo t be able to produce a test with exactly the desired false alarm probability, but we ca pick oe which is close. Practice Problems 8.35, 8.39, 8.43, 8.45 βp = P Z z α p = p z α p p 0 / p 0 1 p 0 / = Φ [p 1 p ]/[p 0 1 p 0 ] z α p0 1 p 0 / p p 0 = Φ p 1 p / 4.7 4.2 Small Sample Tests If the sample size is too small or p 0 is to close to zero or oe to use the ormal trick, we basically have to costruct the test usig the biomial cdf x x Bx;, p = bx;, p = p x 1 p x 4.8 x y=0 I practice we wo t actually evaluate the sum; we ll look it up i a table or ask a statistical software package to do it for us. A test which rejects H 0 whe X c, i.e., ˆp c/, appropriate for alterative hypothesis H a : p > p 0, will have a false alarm probability of α = P X c p=p 0 = 1 P X c 1 p=p 0 = 1 Bc 1;, p 0 4.9 y=0 8