10 Hypothesis testing

Transcription

1 10 Hypothess testng 10.1 Introducton In ths chapter we wll study hypothess testng for a populaton parameter. There are other type of hypothess testng n statstcs. We wll always have two hypotheses: the null hypothess, H 0, and the alternatve hypothess, H a. Dependng on the data, we wll ether reject the null hypothess and accept the alternatve hypothess or not reject the null hypothess. We start wth an example. Example: In an electon between two canddates t takes 50% or more of the votes to wn. McSally s one of the canddates. We thnk she s gong to lose and want to test ths hypothess. Let p be the fracton of the voters that wll vote for McSally. Our hypotheses are H 0 : p = 0.5 (1) H a : p < 0.5 (2) Note that we take the null hypothess to be that p = 0.5. We take a poll wth 15 people and sees how many say they wll vote for her. How do we decde between the two hypotheses? Example: A drug company has a drug, call t A, for lowerng blood pressure. They have just developed a new drug, B, that they thnk s better. They want to test t to decde f they should qut sellng drug A and startng sellng drug B. Let µ A be the average amount a patent s blood pressure s lowered by drug A, and let µ B be the average amount a patent s blood pressure s lowered by drug B. Our hypotheses are H 0 : µ A = µ B (3) H a : µ A > µ B (4) Agan, note that the null hypothess nvolves an equalty. Hypothess testng s lke a court tral (Wkpeda). The null hypothess s that the defendant s not gulty. The alternatve s that he or she s gulty. The evdence (data) must reach a certan level (beyond a reasonable doubt) for us to reject the null hypothess n favor of the alternatve. 1

2 10.2 Elements of a statstcal test Our hypothess test nvolves the followng elements: 1. Null hypothess 2. Alternatve hypothess 3. A test statstc 4. Rejecton regon We wll always take the null hypothess to be of the form H 0 : θ = θ 0 (5) where θ 0 s a known number. The alternatve hypothess can be of three forms. The two sded alternatve s There are two possble one-sded alternatves: or H a : θ θ 0 (6) H a : θ > θ 0 (7) H a : θ < θ 0 (8) The test statstc s (lke all statstcs) a functon of the random sample. The rejecton regon s the set of values of the test statstc for whch we reject the null hypothess and so conlude the alternatve hypothess holds. If the test statstc does not fall n the rejecton regon, we do not reject the null hypothess. However, we do not accept the null hypothess. We just conclude that our data does not support the concluson that the null hypothess s false. Example: Return to the electon example. We have already stated the hypotheses. We poll n people and let Y n be the number of them that say they wll vote for McSally. The test statstc s Y n. If Y n s small enough we should reject H 0 and accept H a. So the rejecton regon should be of the form Y n k. What should k be? Example: Return to the drug example. We take a bunch of patents, randomly dvde them nto two groups and gve one group drug A and the other 2

3 group drug B. We let Y A be the average reducton n the blood pressure n group A, Y A the average reducton n the blood pressure n group B. Our test statstc s Y A Y B. If t s sgnfcantly bgger than 0 we should reject H 0 and accept H a. So the rejecton regon should be of the form Y A Y B k. If H 0 s true, then there s stll some probablty that Y A Y B wll be postve. So we should not take the rejecton regon to be just Y A Y B > 0. Obvously k should be postve, but how large should t be? There s always some chance that the random sample we get s atypcal and so the concluson we draw based on t s wrong. There are two possble types of errors. Defnton 1. If H 0 s true and we reject t, ths s called a type I error. We let α be the probablty of a type I error. α s called the level of the test. If H a s true and we accept H 0, ths s called a type II error. We let β be the probablty of a type II error. Note that f H 0 s true, then we know the value of θ. So we can compute the probablty the test statstc falls n the rejecton regon,.e., we can compute α. It wll just be a number. Of course t depends on the rejecton regon. But f we know that H a s true, then we know somethng about θ, but we don t know the actual value of θ. So when we compute β the probablty wll depend on θ. So β s a functon of θ. Example: Return to the electon example. Suppose we sample 15 people and we take the rejecton regon to be Y 2. What s α? α = P (Y 2 p = 0.5) = 2 y=0 ( ) 15 (0.5) y (1 0.5) 15 y = (9) y The value of β depends on p. Suppose p = 0.3. Then we have 15 ( ) 15 β = P (Y > 2 p = 0.3) = (0.3) y (1 0.3) 15 y = (10) y y=3 Ths s not good. We are almost certan to make a type II error even f p s 0.3. To make β smaller we need to make the rejecton regon bgger. If we do ths, then α wll ncrease. So there s a trade-off between α and β. A small rejecton regon makes α smaller but β larger. A larger rejecton 3

4 regon makes α larger but β smaller. To do better overall we need to make the sample sze larger. More on ths later. Whch s worse - a type I or type II error? That depends very much on the partcular problem. In the electon example, concludng she wll wn when she wll not s comparable to concludng she wll lose when she wll n fact wn. But consder ths example. When a drug company starts tesng a new drug they may start wth tests just to see f the drug has harmful sde effects. To be extreme, suppose they want to test f the drug s actually fatal. Let p be the probablty that the drug klls a patent. Take H 0 : p = 0 (11) H a : p > 0 (12) Suppose H a s true and we mstakenly accept H 0 (a type II error) and conclude the drug safe. Ths s really bad. The company wll kll people. On the other hand, f H 0 s true and we mstakenly reject H 0 (a type I error), then we wll mstakenly conclude the drug s dangerous when t s n fact safe. So we wll probably abandon the drug and the company may lose all the money t mght have made from the new drug Common large sample tests Revew: one populaton mean, one populaton proporton, dfference of two populaton means, dfference of two populaton proportons. Suppose our hypothess nvolves a populaton mean µ. (So θ s µ.) We have a pont estmator for µ, namely the sample mean Y. We could use Y as the test statstc. The mean of Y s µ and ts varance s σ 2 /n. If the sample sze s large, then the CLT says that Y µ σ2 /n (13) s approxmately a standard normal. Note that ths nvolves the unknown parameter µ, so ths s not a vald statstc. Now suppose our hypotheses are H 0 : µ = µ 0 (14) H a : µ > µ 0 (15) 4

5 where µ 0 s known. We defne Z = Y µ 0 σ2 /n (16) Note that we used µ 0, not µ. So Z s a vald statstc. It does not depend on the unknown µ. If the null hypothess s true, then the dstrbuton of Z s approxmately standard normal. We should reject H 0 f Y s sgnfcantly larger than µ 0,.e., f Z s sgnfcantly larger than 0. We can fnally quantfy what sgnfcantly larger should mean snce we know the dstrbuton of Z. If H 0 s true the values of Z are usually between 2 and 2 and so a reasonble choce for the rejecton regon would be to reject H 0 f Z > 2. Note that f the null hypothess s false, then Z does not have a standard normal dstrbuton snce the mean µ s not µ 0. The rejecton regon s of the form Z > z c. The probablty of a type I error s the probablty that Z > z c when the null hypothess s true. Ths s just P (Z > z c ). So f we have a desred value of α, ths determnes the cutoff z c. It should just be z α where P (Z > z α ) = α. Note that the rejecton regon Z > z α s the same as Y > µ 0 + σ n z α (17) Example: An assembly lne makes wdgets. They clam that the number of defectve wdgets per day s on average 15. We suspect the number s hgher than ths. We randomly pck 36 days and see how many defectve wdgets were made each of those 36 days. The sample mean s 17.0 and the sample varance s 9.0. Test the companes clam wth sgnfcance level α = Let µ be the average number of defectve wdgets per day. We take our hypotheses to be Our test statstc s H 0 : µ = 15 (18) H a : µ > 15 (19) Z = Y µ 0 σ/ n (20) 5

6 Wth a sgnfcance level of 0.05, z 0.05 = So our rejecton regon s Z > We have µ 0 = 15 and n = 36. We don t know σ 2 so we approxmate t by the sample varance. So Z = Y 15 9/ 36 (21) In our test we got Y = 17. So Z works out to 4. The s n the rejecton regon, so we reject H 0 and conclude the company s understatng the number of defectves. End of lecture on Thurs, 3/22 Example: Comparng vsual reacton tmes of men vs. women. (Reference: Int J Appl Basc Med Res May-Aug; 5(2): ) Suppose we want to test f the reacton tmes of males and females are dfferent. We wll use a sgnfcance level of α = Subjects watch a screen and when a red dot appears they have to ht the space bar. The study had 60 men and 60 women. The unts are mllseconds. Males : mean , stan dev Females: mean , stand dev We let µ m and µ f be the average vsual reacton tme for males and for females H 0 : µ m = µ f (22) H a : µ m µ f (23) The estmator for µ m µ f s Y m Y f. If the null hypotheses s true, then the mean of ths estmator s 0. Its varance s σ 2 m/n m + σ 2 f /n f. So we take our test statstc to be Z = Y m Y f σm/n 2 m + σf 2/n f (24) Note that we now have a two-sded alternatve. So we should reject H 0 f we get an unusually large value of Z or an unusually small (negatve) value. So the rejecton regon should be of the form Z > z c. So gven a desred level α, we want to choose z c so that P ( Z z c ) = α. So z c s z α/2, whch 6

7 s So we reject H 0 and conclude that the reacton tmes are dfferent f Z > 1.96 or Z < For our data Z = (13.04)2 /60 + (19.92) 2 /60 = 5.14 (25) whch s well nsde the rejecton regon. So we conclude there s a dfference n reacton tmes. Note that snce we do not know the exact values of σm 2 and σf 2, we had to approxmate them wth the correspondng sample varances n the above. Suppose we are dong hypotheses testng for two populaton proportons. So our test statstc s Z = ˆp A ˆp B ˆpA (1 ˆp A )/n A + ˆp B (1 ˆp B )/n B (26) Note that we are estmatng p A by ˆp A and p B by ˆp B n the denomnator. If the null hypothess s of the form H 0 : p A = p B, then when H 0 s true the two populatons have the same p. So t s better to use a pooled estmator for ths common parameter p. Z = ˆp A ˆp B ˆp(1 ˆp)/nA + ˆp(1 ˆp)/n B (27) = ˆp A ˆp B ˆp(1 ˆp)(1/nA + 1/n B ) (28) wth ˆp = ˆp An A + ˆp B n b n A + n B (29) Example: Drug Gemfbrozl lower bad cholestorol and so hopefully reduces heart attacks. 5 year experment. Some patents get the drug, some get a placebo. The control group has 2030 subjects and 84 had a heart attack durng the 5 year perod. The group takng the drug has 2051 subjects and 56 had a heart attack durng the 5 year perod. Test at the 5% sgnfcance level f the drug reduces heart attacks. Let 1 be control, 2 the drug group. H 0 : p 1 = p 2 (30) H a : p 1 > p 2 (31) 7

8 We are dong a one-sded alternatve and z α = 1.645, so we wll reject H 0 f Z > Poolng So ˆp = = (32) Z = ˆp A ˆp B ˆp(1 ˆp)(1/nA + 1/n B ) = 2.47 (33) So we reject H 0. The data supports the concluson that the drug works. Summary: For these four scenaros - one populaton mean, dfference between two populaton means, one populaton proporton, dfference between two populaton proportons- we do the followng for a level α test. The null hypothess s H 0 : θ = θ 0. The test statstc s Z = ˆθ θ 0 σˆθ (34) Upper one-sded alternatve (H a : θ > θ 0 ). Reject f Z > z α. Lower one-sded alternatve (H a : θ < θ 0 ). Reject f Z < z α. Two-sded alternatve (H a : θ θ 0 ). Reject f Z > z α/2,.e., Z > z α/2 or Z < z α/2. Whch alternatve hypothess? There are three possble forms of the hypothess. Whch one should be used? The answer depends on the problem/experment. However, t should not depend on the data. You should decde what H a s before you see the data. If takng the null hypothess to be H 0 : µ µ 0 would lead to the same conclusons as H 0 : µ = µ 0, then the alternatve should be µ > µ 0. If takng the null hypothess to be H 0 : µ µ 0 would lead to the same conclusons as H 0 : µ = µ 0, then the alternatve should be µ < µ 0. Consder the example of an assembly lne makng wdgets, some of whch are defectve. The factory says the number of defectve wdgets per day s on average 15. We thnk they are dong a worse job than ths and the number s actually hgher. The null hypothess s H 0 : µ = 15. If we reject H 0 then we 8

9 should conclude that µ > 15. If we had taken H 0 to be µ 15, then when we reject H 0 we would stll conclude that µ > 15. So the alternatve should be H a : µ > 15. Now consder the same assembly lne makng wdgets, but now suppose we thnk they are dong a better job than what they clam,.e., the averge number of defectves s less than 15. Null hypothess s stll µ = 15. Now rejectng the null hypothess should mean we conclude the average number of defectves s less than 15. If we have taken H 0 : µ 15, then rejectng the null hypothess would stll lead to the concluson that µ < 15. So we would test aganst H a : µ < 15. Fnally, we could thnk that they just made ths number up or that they don t know how to do a proper experment to estmate ths number. So we mght want to test aganst the alternatve µ 15. Consder the blood pressure drug example. In that example we only cared f the new drug was better,.e., lowered blood pressure more than the old drug. If t lowered t less, our decson would be the same as f t lowered t the same amount,.e., stop development of the new drug. But now suppose we are not tryng to fnd a better drug, we are just dong research to understand how the exstng drug works. Drug B s a modfcaton of the old drug A whch may or may not change t effcacy. So we would test aganst the alternatve H a : µ A µ B. To llustrate why you should decde what H a s before you see the data consder the followng example for the blood pressure medcatons. Suppose we use the test statstc Z = Y A Y B σ Y A Y B (35) We want to take α = 5%. We are testng f the drugs have dfferent effcaces, so we take H 0 : µ A = µ B and H a : µ A µ B. Ths s a two taled test, so we reject H 0 f Z > z α/2 = Now suppose our data gve Z = Then we do not reject H 0. Suppose nstead that we looked at our data before decdng on H a. We mght be tempted to say that t looks lke f there s a dfference then t s drug A that s better. So we take H a : µ A > µ B. Then we would reject f Z > z α = Snce Z = 1.83, we reject H 0 and conclude (possbly ncorrectly) that drug A s better. 9

10 10.4 Calculatng probablty of type II error and fndng the sample sze for Z tests What s the probablty of a type II error? Z does not have the standard normal dstrbuton now, but Y s stll normal f the sample sze s large. So we can stll compute β. It depends on µ, so we wrte t as β(µ). We start wth an example. Example: Return to the defectve wdget example. We have a sample of sze 36. Our test statstc s Z = Y µ 0 σ/ n = Y 15 1/2 (36) Wth a sgnfcance level of 0.05, z 0.05 = So our rejecton regon s Z > We are nterested n type II errors now, so we want to consder what happens when H a s true. When ths happens Z s not standard normal. So we express the rejecton regon n terms of Y Z > s equvalent to Y > /2 = So we accept H 0 when Y Snce µ = 16, If µ = 17, If µ = 18, β(16) = P (Y ) = P ( Y ) (37) 1/2 1/2 = P (Z 0.355) = (38) = β(17) = P (Y ) = P ( Y ) (39) 1/2 1/2 = P (Z 2.355) = (40) = β(18) = P (Y ) = P ( Y ) (41) 1/2 1/2 = P (Z 4.355) = (42) End of lecture on Tues, 3/27 10

11 Consder how the dstrbuton of Z changes Pcture of sldng normal. Recall that we can decrease β by changng the rejecton regon. If we enlarge the rejecton regon then β wll decrease. But α wll then ncrease. Suppose we want to keep α at Then we can decrease β by ncreasng the sample sze. Example: We contnue wth the wdget example. We use our exstng data to estmate σ. So σ S = 3. Suppose we want to fnd the sample sze that wll make β(16) = Now consder a sample of sze n. The rejecton regon s Z > and Z = Y 15 σ/ n (43) So the rejecton regon s Y σ n. Now β(16) = P (Y σ ) = P ( Y 16???) (44) n n We can fnd a general formula for the sample sze when we are gve a desred α and β(µ). We consder the case where the alternatve hypothess s µ > µ 0. So σ / β(µ) = P µ (Y µ 0 + σ n z α ) (45) = P µ ( Y µ σ/ n µ 0 µ + σ n z α σ/ ) (46) n = P µ ( Y µ σ/ n µ 0 µ σ/ n + z α) (47) We have put a subscrpt µ on P to remnd ourselves that t depends on µ snce the alternatve hypothess s true when we consder a type II error. If H a s true, then we do not know µ but t s greater than µ 0. So µ 0 µ/σ/ n s negatve. Wth the sample sze fxed, as µ ncreases, ths quantty get more negatve and β(µ) decreases. The larger sample sze s, the faster t decreases. The formula for the sample sze s n = (z α + z β ) 2 σ 2 (µ a µ 0 ) 2 (48) 11

12 10.5 Hypothess testng vs. confdence ntervals Not many notes here. The punchlne here s that n a two-sded test wth sgnfcance level α, we reject the null hypothess f and only f the true value of θ s outsde the confdence nterval wth sgnfcance 1 α p-values Suppose we are dong a large sample test whch uses a test statstc Z that s approxmately standard normal. We are dong a two sded alternatve and α = We reject f Z Now compare two dfferent outcomes of the experment. In one outcome our data gves Z = 4.2. In the other t gves Z = 2.1. In both cases we reject the null hypotheses. But ths does not fully reflect what out data tell us. In the frst scenaro the value of Z s well nsde the rejecton regon whch the second t s close to the boundary. We could convey more nformaton by actually reportng the value of Z that we got. However, we wll eventually look at test wth statstcs that have other dstrbutons. So we would lke a way to report the result that does not nvolved the dstrbuton of the test statstc. That s what p-values do. Defnton 2. For a gven set of date, the p-value s the smallest value of the level α whch would lead to us rejectng the null hypthosess. Another way to say ths s that f we get a value z 0 for the test statstc, then p s the probablty of a value of the test statstc that would gve even stronger evdence to reject H 0 than Z = z 0. We spell ths out for the three possble H a. Suppose that our data gves Z = z 0. If we have an upper-taled test (H a : θ > θ 0 ) and the rejecton regon s Z k, then p = P (Z z 0 H 0 s true) (49) If we have a lower-taled test (H a : θ < θ 0 ) and the rejecton regon s Z k, then p = P (Z z 0 H 0 s true) (50) If we have a two-taled test (H a : θ θ 0 ) and the rejecton regon s Z k, then p = P ( Z z 0 H 0 s true) (51) 12

13 Example: Suppose we are testng H 0 : µ = 22 (52) H a : µ < 22 (53) and we get a Z of Then p = P (Z 1.53) = So we would reject H 0 f α = 10%, but we would accept H 0 f α = 5%. What f we got Z = Then p = P (Z 0.53) = Ths s a huge p value. We would not reject H 0 for any reasonable α. Note that a postve value of Z means the sample mean was actually larger than 22, so ths certanly does support acceptng the alternatve that µ < 22. Example: The assembly lne makes wdgets. We were dong a one-sded test: H 0 : µ = 15 (54) H a : µ > 15 (55) For our sample of 36 days we found 17.0 and the sample varance s 9.0. So the test statstc was Z = 4. So p = P (Z 4) = Example: Comparng vsual reacton tmes of men vs. women. We were testng f ther average reactons tmes were dfferent. The study had 60 men and 60 women. Males : mean , stan dev Females: mean , stand dev Our test statstc was H 0 : µ m = µ f (56) H a : µ m µ f (57) Z = Y m Y f = 5.14 (58) σm/n 2 m + σf 2/n f We are dong a two-sded H a, so p = P ( Z 5.14) = 0.. Example: Drug Gemfbrozl to reduce heart attack rsk. Let 1 be control, 2 the drug group. H 0 : p 1 = p 2 (59) H a : p 1 > p 2 (60) 13

14 Z = ˆp 1 ˆp 2 ˆp(1 ˆp)(1/n A + 1/n B ) (61) We reject H 0 f Z s large. For our data we got Z = So p = P (Z 2.475) = End of lecture on Thurs, 3/ comments 10.8 Small sample testng Suppose we want to test a hypothess concernng the mean of a populaton. As before Y s a natural statstc to look at. If the sample sze s not large, then t need not be normal. Furthermore, the approxmaton of replacng σ by S s not justfed. In ths sectno we assume that the populaton s normal, so Y s normal. But the replacement of σ by S s stll not justfed. Recall that Y µ s/ n (62) has a t-dstrbuton wth n 1 degrees of freedom. As before we consder tests where the null hypothess s H 0 : µ = µ 0 wth µ 0 known and the alternatve s one of H a : µ < µ 0, H a : µ > µ 0, H a : µ µ 0. We take the test statstc to be T = Y µ 0 s/ n (63) Note that we have µ 0 here, not the unknown µ. So f the null hypothess s true than T has a t-dstrbuton, but f t s not true t does not. Example: (from the book) a new gunpowder manufacturer clams the muzzle velocty for t s 3000 ft/sec. We want to test the clam that s t s ths hgh wth α = 2.5%. We test 8 shells and fnd an average velocty of 2959 ft/sec wth a standard devaton of 39.1 ft/sec. H 0 : µ = 3000 (64) H a : µ < 3000 (65) 14

15 R says that qt(0.025, 7) = So we should reject the null f T < For the test statstc we fnd T = Y 3000 s/ n = T = / 8 = (66) So we reject the null and conclude that the manufacturer s wrong. The average muzzle velocty s less than The p-value s P (T 2.966) = pt( 2.966, 7) = Now suppose we have two populatons wth means µ 1 and µ 2. We want to test a hypothess nvolvng µ 1 µ 2. For large samples, we could assume Y 1 Y 2 was normal and we could replace σ 1 by S 1 and σ 2 by S 2. We now consder small samples, but add the assumpton that the populatons are normal and they have the same varance σ 2. In ths case we estmate ths common varance by the pooled estmator S 2 p = (n 1 1)S (n 2 1)S 2 2 n 1 + n 2 2 (67) In ths case Y 1 Y 2 (µ 1 µ 2 ) (68) 1 S p n n 2 has a t dstrbuton wth n 1 + n 2 2 d.f. We assume the null hypthess s H 0 : µ 1 = µ 2. Then we take our test statstc to be T = Y 1 Y 2 0 (69) 1 S p n n 2 Example: Does addng a calcum supplement lower your blood pressure? Take 21 subjects. 10 take the supplement (group 1) and 11 take a placebo (group 2) for 12 weeks. We measure ther BP before and after the 12 weeks and fnd the decrease n BP. We test at the α = 10% level. There are = 19 d.f. And R says qt(0.1, 19) = So we should reject H 0 f T > For group 1 the average decrease was wth S = For group 2 the average decrease was wth S = We fnd SE = and 15

16 T = So we reject H 0 and conclude the supplement does help lower BP. The p-value s P (T > 1.604) = So f we had tested at the α = 5% level we would not have rejected H 0. End of lecture on Tues, 4/ Tests nvolvng the varance We now consder tests nvolvng the populaton varance. We start wth a sngle populaton wth varance σ 2 and consder testng H 0 : σ = σ 0 aganst one of the alternatves H a : σ 2 > σ 2 0, H a : σ 2 < σ 2 0, H a : σ 2 σ 2 0 (70) The natural statstc to look at s S 2. normal. Recall that n ths case, We assume that the populaton s (n 1)S 2 σ 2 (71) has a χ 2 dstrbuton wth n 1 df. We defne our test statstc to be χ 2 = (n 1)S2 σ 2 0 (72) Note that we use the null hypothess value n ths defnton. So f the null hypothess s true, then χ 2 wll have a χ 2 dstrbuton. Note that the χ 2 dstrbuton s not symmetrc. So n a two taled test our rejecton regon s not symmetrc about σ 0. Let χ 2 α be the number such that P (χ 2 χ 2 α) = α. The rejecton regon should be H a : σ 2 > σ 2 0 (73) RR : χ 2 > χ 2 α (74) H a : σ 2 < σ 2 0 (75) RR : χ 2 < χ 2 1 α (76) H a : σ 2 σ 2 0 (77) RR : χ 2 < χ 2 1 α/2 or χ 2 > χ 2 α/2 (78) 16

17 Example: A company produces ppes. It s mportant that the lengths be very nearly the same,.e., the varance n the lengths s small. They clam that the standard devaton of the length s at most 1.2 cm. In a sample of 25 ppes we fnd a sample standard devaton of 1.5 cm. Test the company s clam at the 5% sgnfcance level. H 0 : σ = 1.2 (79) H a : σ > 1.2 (80) R says that qchsq(0.95, 24) = So we wll reject H 0 f χ 2 > For our data the value of the test statstc s χ 2 (n 1)S2 = = 24(1.5)2 = 37.5 (81) σ0 2 (1/2) 2 So we reject H 0. The data provdes evdence that the company s clam s not correct. The p-value s p = 1 pchsq(37.5, 24) = Example: A manufacturer of hard hats test them by applyng a large force to the top of the helmet and seeng how much force s transmtted to the head. They clam that at most 800 lbs of force s transmtted on average and the standard devaton s 40 lbs. We want to test f the value of 40 for the standard devaton s correct. We wll use α = 5%. The test statstc s H 0 : σ = 40 (82) H a : σ 40 (83) χ 2 = (n 1)S2 σ 2 0 (84) The rejecton regon s χ 2 < or χ 2 > For our data χ 2 = , so we do not reject H 0. Snce ths s a two-taled test, the p-value s p = 2P (χ ) = = (85) Now suppose we have two normal populatons and we want to test f they have the same varance. So the null hypothess s H 0 : σ 2 1 = σ 2 2. The three possble alternatve hypotheses are H a : σ 2 1 > σ 2 2, H a : σ 2 1 < σ 2 2, H a : σ 2 1 σ 2 2 (86) We revew the def of the F-dstrbuton. 17

18 Defnton 3. F-dstrbuton Let W 1 and W 2 be ndependent RV s wth χ 2 dstrbutons wth ν 1 and ν 2 degrees of freedom. Defne F = W 1/ν 1 W 2 /ν 2 (87) Then the dstrbuton of F s called the F-dstrbuton wth ν 1 degrees of freedom and ν 2 denomnator degrees of freedom. numerator If we have two normal populatons wth varances σ1 2 and σ2, 2 and we take random samples from each one wth szes n 1 and n 2 and sample varances S1 2 and S2, 2 then we know that (n 1)S 2 /σ 2 have χ 2 dstrbutons. So the followng has an F-dstrbuton: S 2 1/σ 2 1 S 2 2/σ 2 2 (88) If the null hypothess s true, then ths smplfes to S 2 1/S 2 2. So we defne our test statstc to be F = S2 1 S 2 2 (89) Under the null hypothess the dstrbuton of F s the F dstrbuton wth n 1 1 numerator degrees of freedom and n 2 1 denomnator degrees of freedom. Let F α be the number such that P (F F α ) = α. Note that t depends on n 1 and n 2. For the three possble alternatve hypotheses our rejecton regon (RR) s H a : σ 2 1 > σ 2 2 (90) RR : F > F α (91) H a : σ 2 1 < σ 2 2 (92) RR : F < F 1 α (93) H a : σ 2 1 σ 2 2 (94) RR : F < F 1 α/2 or F > F α/2 (95) We can compute values of F α usng R. The order of arguments for numerator df, then denomnator df. For example, qf(0.95, n, m) wll gve F 0.05 for n numerator df and m numerator df. 18

19 Example: A psychologst was nterested n explorng whether or not male and female college students have dfferent drvng behavors. The partcular statstcal queston she framed was as follows: Is the mean fastest speed drven by male college students dfferent than the mean fastest speed drven by female college students? The psychologst conducted a survey of a random n = 34 male college students and a random m = 29 female college students. We take populaton 1 to be the the female populaton and populaton 2 to be the male populaton. The data s Y 1 = 90.9, S 1 1 = 12.2 (96) Y 2 = 105.5, S 1 2 = 20.1 (97) (98) We want to test at the α = 5% level f the varances of the two populatons are the same. We are dong a two taled test R tells us that qf(0.025, 28, 33) = , qf(0.975, 28, 33) = So we wll reject H 0 f F > or F < For our data F = (12.2) 2 /(20.1) 2 = So we reject H 0 and conclude the varances are not equal. To fnd the p-value, remember ths s a two taled test. So p = 2P (F < 0.368) = = (99) If F has an F dstrbuton, then 1/F wll also have an F dstrbuton but wth the number of degrees of freedom swtched. So n our example, nstead of computng qf(0.975, 28, 33) = , we could have used 1/qf(0.025, 33, 28) = Ths was a bg deal when we had to use tables, not a bt deal now Power of a test and the Neyman-Pearson Lemma Consder a test nvolvng a paramter θ and suppose the null hypothess s H 0 : θ = θ 0 and the alternatve s the two sded H a : θ θ 0. The power of a test s closely related to the probabty of a type two error. Recall that a type two error s the probablty of acceptng H 0 when t s not true. Ths probablty depends on the actual value of the parameter θ. So we have been denotng t by β(θ) = P (accept H 0 θ) (100) 19

20 where θ s not θ 0. The power s just 1 β(θ): Defnton 4. power(θ) = P (reject H 0 θ) (101) The power when θ = θ 0 s the probablty we reject H 0 when t s n fact true. Ths s just α, the probablty of a type I error. So the power at θ = θ 0 s α. Typcally the power wll be a contnuous functon of θ. So t wll stll be close to α when θ s close to θ 0. Typcally t wll approach 1 as θ moves away from θ 0 Pcture of typcal power functon, H a : θ a θ 0 Now suppose we are testng wth a one-sded hypothess. Consder frst the alternatve H a : θ a > θ 0. When θ = θ 0 the power wll agan be α. As θ ncreases from θ 0, the probablty we reject H 0 ncreases and so the power ncreases, approachng 1 as θ gets farther away from θ 0. On the other sde, as θ decreases from θ 0 the probablty we reject H 0 wll be even smaller than α. So the graph of the power functon looks lke : Pcture of typcal power functon, H a : θ a > θ 0 If the alternatve s H a : θ a < θ 0, the graph looks lke Pcture of typcal power functon, H a : θ a < θ 0 Next we defne smple and composte hypotheses. Suppose the populaton pdf has just one unknown parameter θ. Under the null hypothess H 0 : θ = θ 0, the populaton dstrbuton s completely determned. However, under the alternatve hypothess t s not. The null hypothess s an example of a smple hypothess, the alternatve s an example of a composte hypothess. Defnton 5. A hypothess s a smple hypothess f t completely specfes the dstrbuton of the populaton. Otherwse t s called a composte hypothess. Untl now our alternatve hypothess has always been of the form θ θ 0, θ < θ 0, or θ > θ 0. Now we wll also consder alternatve hypotheses of the form H a : θ = θ a where θ a s not equal to θ 0 and s known. In ths case the alternatve hypothess s smple. 20

21 Lemma 1. (the Neyman-Pearson lemma) Suppose we want to test the null hypothess H 0 : θ = θ 0 versus the alternatve H a : θ = θ a usng a random sample Y 1, Y 2,, Y n from a populaton whch depends on a parameter θ. Gven a value of α the test that maxmzes the power at θ a s the test wth rejecton regon L(y 1,, y n θ 0 ) L(y 1,, y n θ a ) < k (102) where the constant k s chosen so that the probablty of a type I error s α. Such a test s called the most powerful α-level test for H 0 vs. H a. The theorem does not say anythng when the alternatve hypothess s composte. But n some stutatons t can. Suppose the alternatve s H a : θ > θ 0 and suppose that the when we fnd the rejecton regon n the theorem, t does not depend on the value of θ a. It only depends on α. Then the test s the most powerful test for the composte alternatve hypothess H a : θ > θ 0. We say that the test s the unformly most powerful test for H 0 : θ = θ 0 vs H a : θ > θ a. Example: Suppose the populaton s normal wth unknown mean µ but known varance σ 2. So the lkelhood functon s L(y 1,, y n µ) = (2πσ 2 ) n/2 exp( (y µ) 2 /(2σ 2 )) (103) So L(y 1,, y n µ 0 ) L(y 1,, y n µ a ) = exp[ So the rejecton regon s (y µ 0 ) 2 /(2σ 2 ) + (y µ a ) 2 /(2σ 2 )] (104) (y µ 0 ) 2 /(2σ 2 ) + (y µ a ) 2 /(2σ 2 ) < ln k (105) whch can be rewrtten as y (µ 0 µ a ) < n 2 (µ2 0 µ 2 a) + σ 2 ln k (106) If µ a < µ 0, ths s equvalent to Y < c where the constant c depends on k, µ 0, µ a, n and σ 2. If µ a > µ 0, ths s equvalent to Y > c. The constant k 21

22 s determned by the requrement that that the probablty of a type I error should be α. So we mght as well forget about k and just solve for c. It s determned by P (Y > c θ = θ 0 ) = α (107) As we have seen before ths gves c = µ 0 + z α σ/ n. It does not depend on the value of µ a. So the rejecton regon does not depend on the value of µ a. So n ths case the test s the unformly most powerful test. Example: Suppose we want to test a populaton proporton. So the populaton pdf s just f(y p) = p y (1 p) 1 y (108) where y can only be 0 or 1. So the lkelhood functon s a bnomal dstrbuton: [ ] L(y 1,, y n p) = p y (1 p) 1 y p = (1 p) n y (109) 1 p where all the sums on are from 1 to n. So L(y 1,, y n p 0 ) L(y 1,, y n p a ) = So the rejecton regon s [ 1 p0 1 p a ] n [ p0 (1 p a ) p a (1 p 0 ) ] y (110) [ ] p0 (1 p a ) y < k (111) p a (1 p 0 ) where k depends on k and p 0, p a. Takng the logarthm, ths s equvalent to ( [ ] p0 (1 p a ) y ) ln < ln k (112) p a (1 p 0 ) A lttle algebra shows p 0 > p a f and only f p 0 (1 p a ) p a (1 p 0 ) > 1 (113) So we see that f p 0 > p a then the rejecton regon s of the form Y < c. The value of k s determned by α, but as n the prevous example we mght as 22

23 well forget k and just fnd c by the requrement that P (Y < c) = α. As before ths leads to c = p 0 z α p0 (1 p 0 )/n. If p 0 < p a then the rejecton regon s of the form Y > c, and c = p 0 + z α p0 (1 p 0 )/n. In both cases we fnd that the rejecton regon does not depend on the value of p a. So the test s the unformly most powerful test. Suffcent statstc: Suppose there s a suffcent statstc U for θ. So by the factorzaton theorem L(y 1,, y n θ) = g(u, θ)h(y 1,, y n ) (114) Snce h does not depend on θ, ths gves L(y 1,, y n θ 0 ) L(y 1,, y n θ a ) = g(u, θ 0) g(u, θ a ) (115) So when there s a suffcent statstc, the rejecton regon for the test from the Neyman-Pearson lemma depends on the random sample only through the suffcent statstc. End of lecture on Tues, 4/17 Proof of Neyman-Pearson lemma We need a lttle notaton. Let d(y 1,, y n ) be the functon whch s 1 f the test n the Neyman-Pearson lemma says we should reject H 0 and s 0 f t does not. So { 1 f L(y1,, y d(y 1,, y n ) = n θ 0 ) < kl(y 1,, y n θ a ) (116) 0 f L(y 1,, y n θ 0 ) kl(y 1,, y n θ a ) Suppose we have another test wth the same α. Let d (y 1,, y n ) be the functon whch s 1 f ths test says we should reject H 0 and s 0 f t does not. The power of the Neyman-Pearson test at θ = θ a s P (d(y 1,, y n ) = 1 θ = θ a ) = d(y 1,, y n )L(y 1,, y n θ a )dy (117) where the ntergral s over R n and dy s shorthand for dy 1 dy n. The power of the other test s P (d (y 1,, y n ) = 1 θ = θ a ) = d (y 1,, y n )L(y 1,, y n θ a )dy (118) 23

24 We clam that [d(y 1,, y n ) d (y 1,, y n )][kl(y 1,, y n θ a ) L(y 1,, y n θ 0 )] 0 Note that both d and d only take on the values 0 and 1. So f d > d we must have d = 1. So n ths case kl(y 1,, y n θ 0 ) L(y 1,, y n θ 0 ) > 0. Ths verfes the clam n the case that d > d. If d < d we must have d = 0. So n ths case kl(y 1,, y n θ 0 ) L(y 1,, y n θ 0 ) < 0. Ths verfes the clam n the other case. So the clam s proved. Now ntegrate the clam over R n. Note that [d(y 1,, y n ) d (y 1,, y n )]L(y 1,, y n θ 0 )]dy = α α = 0 (119) So we get k d(y 1,, y n )L(y 1,, y n θ a ) dy k d (y 1,, y n )L(y 1,, y n θ a )dy By equatons 117 and 118 ths says that the power of the test from the Neyman-Person lemma s at least as large as the power from the other test. Ths completes the proof. Example: Populaton has Posson dstrbuton wth parameter λ. So f(y λ) = e λ λ y, y = 0, 1, 2, (120) y! We want to test H 0 : λ = λ 0 vs. H a : λ = λ a. We wll do the case of λ a > λ 0. L(y 1,, y n λ) = e nλ λ y y! (121) Snce the parameter s λ we wll denote the test statstc by Λ n ths example. Λ = e n(λ 0 λ a) ( ) λ0 y λ a (122) The rejecton regon s then gven by Λ < k, whch we rewrte as ( ) λ0 y λ a < k (123) 24

25 where k s... Next we take the log and note that snce λ a > λ 0, ln(λ 0 /λ a ) < 0. So our rejecton regon can be wrtten smply as y > c. As always, c s chosen to make the probablty of a type I error be α. If the sample sze s large, than Y s approxmately normal. When H 0 s true, Y has mean λ and varance λ/n. So standardzng, P (Y > c λ = λ 0 ) = P (Z c λ 0 λ/n ) (124) So (c λ 0 )/ λ/n = z α. So our RR becomes y λ 0 + z α λ0 n (125) Note that ths rejecton regon does not depend on λ a except for the assumpton that λ a > λ 0. So f the alternatve hypothess s H a : λ a > λ 0, then ths rejecton regon gve a unformly most powerful test. If λ a < λ 0 we fnd that the RR s of the form y λ 0 z α λ0 n (126) Ths does not depend on λ a, other than the fact that λ a < λ 0, so f the alternatve hypothess s H a : λ a < λ 0, then we get a unformly most powerful test. If we want to test wth a two sded alternatve H a : λ a λ 0, then there wll not be a unformly most powerful test. For two sded tests there usually do not ext unformly most powerful tests Lkelhood rato tests In ths secton we use the lkelhood rato to develop a very general test for hypotheses. We allow any number of parameters θ 1,, θ n. We denote them by Θ. So Θ takes values n R n. Let Ω 0 and Ω a be subsets of R 2. The hypothess are H 0 : Θ Ω 0, (127) H a : Θ Ω a (128) The only constrant on Ω 0 and Ω a s that they be dsjont. These wll be composte hypotheses unless Ω 0 or Ω a just conssts of a sngle pont. We 25

26 let Ω = Ω 0 Ω a. To keep the notaton smple, we wll denote the lkehood functon L(y 1,, y n Θ) by just L(Θ). Defnton 6. The lkelhood rato test for level α s defned as follows. The test statstc s λ = max Θ Ω 0 L(Θ) max Θ Ω L(Θ) (129) The rejecton regon s of the form λ < k where the constant k s chosen so that max P (accept H 0 Θ) = α (130) Θ Ω 0 Example: Consder a normal populaton wth varance 1 and unknown mean µ. So f(y µ) = 1 2π exp( (y µ) 2 /2) (131) We want to test H 0 : µ = µ 0, (132) H a : µ > µ 0 (133) So Ω 0 just conssts of the sngle pont µ 0 and Ω a s (µ 0, ). And we have Ω = [µ 0, ). The lkelhood functon s L(µ) = (2π) n/2 exp( 1 (y µ) 2 ) (134) 2 Fndng the maxmmum of L(µ) over Ω 0 s trval. It s just L(µ 0 ). Fndng the maxmmum of L(µ) over Ω takes a lttle calculus. As we often do, the algebra s a bt smpler f we look at ln L(µ): So ln L(µ) = n 2 ln(2π) 1 (y µ) 2 (135) 2 d dµ ln L(µ) = (y µ) = n(y µ) (136) 26

27 So there s one crtcal pont at µ = y. Note however that ths value of µ can be outsde Ω. So the max occurs at µ = y f y µ 0, and at µ = µ 0 f y < µ 0. Note that snce the alternatve s µ > µ 0, f we get a sample wth y < µ 0, any reasonble test would not reject the null hypothess. If y µ 0, then λ = L(µ 0) L(y) = exp( 1 2 (y µ 0 ) (y y) 2 ) (137) 2 = exp(nµ 0 y 1 2 ny2 1 2 nµ2 0 = exp( 1 2 n(y µ 0) 2 ) (138) So the rejecton regon λ < k s equvalent to y µ 0 > c for some constant c. Remember that we are dong the case of y µ 0. So ths s equvalent to y µ 0 + c. The constant c s determne by requrng the probablty of a type I error to be α. Example (contnued): We contnue the example above but now consder a composte null hypothess: H 0 : µ µ 0, (139) H a : µ > µ 0 (140) We need to fnd the max of L(µ) over µ µ 0. There s one crtcal pont at µ = y. So f y < µ 0 the max s L(y) and f y µ 0 the max s L(µ 0 ). Frst consder what happens f y < µ 0. Then the max n the numerator s at µ = y and the max n the denomnator s at the same value. So the lkelhood rato wll be 1 whch wll be n the rejecton regon. So from now on we just look at the case that y µ 0. So the max n the numerator s L(µ 0 ). Now the computaton goes just as n the prevous example. End of lecture on Thurs, 4/19 Example (book): Normal wth σ 2 and µ both unknown. So H 0 : µ = µ 0, (141) H a : µ > µ 0 (142) f(y µ, σ) = 1 σ µ)2 exp( (y ) (143) 2π 2σ 2 27

28 L(y 1,, y n ) = (2πσ 2 ) n/2 exp( (y µ) 2 2σ 2 ) (144) Frst we fnd the max over Ω 0. Ths means µ s fxed to µ 0 but σ 2 can be any postve number. So we need to maxmze L as functon of σ 2. Some calculus shows the max occurs at Thus ˆσ 0 2 = 1 (y µ 0 ) 2 (145) n max Ω 0 L(µ, σ 2 ) (146) = (2π ˆσ 2 0) n/2 exp( (y µ 0 ) 2 2 ˆσ ) = [2π] n/2 ( ˆσ 0) 2 n/2 e n/2 (147) 0 2 Next we need to maxmze L over Ω. So µ µ 0 and σ 2 can be any postve number. The max over σ 2 goes as before. It occurs at ˆσ 2 = 1 (y ˆµ) 2 (148) n Now maxmze over µ µ 0. Takng dervatve wrt µ of ln L, we fnd one crtcal pont at µ = y. But as before ths may be outsde of [µ 0, ). When t s outsde, the max occurs at µ = µ 0. So we fnd that max s at ˆµ where ˆµ = y f y µ 0 and ˆµ = µ 0 f y < µ 0. Thus we fnd So the lkelhood rato s max Ω L(µ, σ) = (2π) n/2 ( ˆσ 2 ) n/2 e n/2 (149) λ = max Θ Ω 0 L(Θ) max Θ Ω L(Θ) (150) = ( ˆσ 0) 2 n/2 ( ˆσ (151) 2 ) { n/2 [ ] = (y y) 2 n/2 (y µ 0 f y µ ) 2 0 (152) 1 f y < µ 0 28

29 The rejecton regon s λ < k. We wll always have k < 1, so the second case n the above does not matter. So we can rewrte λ < k as (y y) 2 (y µ 0 ) 2 < k (153) where k = k 2/n. Recall that s 2 = 1 n 1 (y y) 2 (154) And (y µ 0 ) 2 = (y y + y µ 0 ) 2 = (y y) 2 + n(y µ 0 ) 2 (155) After some algebra we fnd that we can wrte the rejecton regon as y µ 0 s/ n c (156) Then we fnd c to make the probablty of a type I error be α. In order to carry out a lkelhood rato test we need to be able to fnd k. In our examples so far, the test statstc λ was relatvely smple and we could do ths explctly. Ths need not be the case as the followng example shows. Example: Two plants manufacture wdgets. We look at the number of defects they make each day. We assume the dstrbuton of the number of defects follows a Posson dstrbuton wth parameter θ 1 for plant 1 and θ 2 for plant 2. We want to test whether the two defect rates are equal wth a sgnfcance level of α = 1%. We randomly choose 100 days for each of the plants and observe how many defects occur each of those days. For plant 1 we fnd a total of 2072 defects from the 100 days. For plant 2 we fnd a total of 2265 defects from the 100 days. Note that we have two populatons here. We use x 1,, x 100 to denote the random sample from populaton 1 and y 1,, y 100 the random sample from populaton 2. To keep the notaton under control we wll denote these 100-tuples by just x and y. The lkelhood functon s L(x, y θ 1, θ 2 ) = 1 k θ x 1 e nθ 1 θ 29 y 2 e nθ 2 (157)

30 where k = x! y! (158) The hypotheses are H 0 : θ 1 = θ 2, (159) H a : θ 1 θ 2 (160) So Ω 0 = {(θ, θ) : θ > 0} (161) Ω a = {(θ 1, θ 2 ) : θ 1, θ 2 > 0, θ 1 θ 2 } (162) To maxmze the lkelhood over Ω 0, we need to compute The max occurs at max θ ˆθ = (163) 1 k θ x + y e 2nθ (164) x + y 2n To keep the notaton under control, let x = 1 x, n (165) y = 1 y (166) n So and ˆθ = x + y 2 max L = 1 Ω 0 k ˆθ nx+ny e 2nˆθ (167) (168) To maxmze the lkelhood over Ω, we need to compute 1 max θ 1,θ 2 k (θ 1) nx e nθ 1 (θ 2 ) ny e nθ 2 (169) 30

31 The max occurs at ˆθ 1 = x, ˆθ2 = y, (170) and Note that nˆθ 1 + nθ 2 = 2nˆθ. So max Ω L = 1 k (ˆθ 1 ) nx e nˆθ 1 (ˆθ) ny e nˆθ 2 (171) λ = max Ω 0 L max Ω L = nx+ny (ˆθ) The rejecton regon s λ < k where k s chosen to make (ˆθ 1 ) nx (ˆθ 2 ) ny (172) max Ω 0 P (λ < k) = α (173) However, λ s complcated and we have no hope of computng ts dstrbuton explctly. So we cannot fnd k explctly. The followng theorem says that for large samples the dstrbuton of λ s approxmately related to the χ 2 dstrbuton. Theorem 1. Let r 0 be the number of free parameters n Ω 0, r the number of free parameters n Ω. Suppose that r > r 0. Under certan regularty condtons, the dstrbuton of 2 ln λ s approxmately a χ 2 dstrbuton wth r r 0 degrees of freedom f the sample sze s large. In the lkelhood rato test we reject the null hypothess f λ < k. Ths s equvalent to 2 ln λ > 2 ln k. So the rejecton regon wth sgnfcance level α wll be 2 ln λ > χ 2 α. Example contnued In our example r = 2 and r 0 = 1. For plant 1 we had a total of 2072 defects, for plant 2 a total of 2265 defects. So x = , y = , θ = (174) whch yelds 2 ln(λ) = We have χ = qchsq(0.99, 1) = So we reject the null hypothess and conclude the defect rates are dfferent for the two factores. 31

32 End of lecture on Tues, 4/24 Example: Ths s one of the problems on the last homework set. We just start t. You wll fnsh t for the homework. Ths s problem n the book. There are four poltcal wards n a cty and we want to compare the fracton of voters favorng canddate A n each of the wards. We randomly poll 200 voters n each ward. In ward 1 we fnd 76 favor A, n ward 2 53 favor A, n ward 3 we fnd 59 favor A, and n ward 4 we fnd 48 favor A. We want to test f the percentages favorng A n the four wards are all the same wth a sgnfcance level of 5%. Let x 1,, x 200 be the sample from ward 1. Each x s 0 f the th voter does not favor A, 1 f the voter does favor A. We denote x 1,, x 200 by just x. We let y, z, w be the random samples from wards 2,3,4. The lkelhood s L(x, y, z, w p 1, p 2, p 3, p 4 ) = p nx 1 (1 p 1 ) n(1 x) (175) p ny 2 (1 p 2 ) n(1 y) p nz 3 (1 p 3 ) n(1 z) p nw 4 (1 p 4 ) n(1 w) (176) where x = 1 x, n y = 1 y, n z = 1 z, n w = 1 w (177) n 32