Chapter 22: What is a Test of Significance?

Transcription

1 Chapter 22: What is a Test of Sigificace? Thought Questio Assume that the statemet If it s Saturday, the it s the weeked is true. followig statemets will also be true? Which of the If it s the weeked, the it s Saturday. If it s ot the weeked, the it s ot Saturday. Assume that the statemet If A is true, the B is true is true. statemets will also be true? Which of the followig If B is true, the A is true. If B is false, the A is false. 1

2 Reasoig of Statistical Tests of Sigificace Goal: to determie how cosistet observed sample data is with a claimed value of the populatio proportio p. Termiology Sigificace test: asks if sample data gives good evidece agaist a claim. Example: Coffee People of taste are supposed to prefer fresh-brewed coffee to the istat variety. But perhaps may coffee drikers just eed their caffeie fix. A skeptic claims that coffee drikers ca t tell the differece. Let s do a experimet to test this claim. Each of 50 subjects tastes two umarked cups of coffee ad says which he or she prefers. Oe cup i each pair cotais istat coffee; the other, fresh-brewed coffee. The statistic that records the result of our experimet is the proportio ˆp of the sample who say they like the fresh-brewed coffee better. We fid that 36 of our 50 subjects choose the fresh coffee. Calculate ˆp. ˆp = 36/50 = 0.72 How strog is the evidece from the sample that the majority of the populatio prefer fresh coffee? Claim: The skeptic claims that coffee drikers ca t tell fresh from istat, so that oly half will choose fresh-brewed coffee, i.e., p = 0.5. Assume that the claim p = 0.5 is true. What is the samplig distributio of the sample proportio ˆx if we repeat the experimet may times? The samplig distributio of ˆp is Normal with mea = p = 0.5 ad stadard deviatio = p(1 p) (0.5)(0.5) =

3 How commo is a ˆp value of 0.72 or larger if p = 0.5? Very uusual. How commo is a ˆp value of 0.56 or larger if p = 0.5? Quite commo. What is the probability that a sample gives a ˆp value of 0.72 or larger? What is the probability that a sample gives a ˆp value of 0.56 or larger? 0.20 Does a ˆp value of 0.56 provide strog evidece agaist the claim that p = 0.5? No - this outcome or more extreme will happe 20% of the time if p = 0.5, which is ot that ulikely. Does a ˆp value of 0.72 provide strog evidece agaist the claim that p = 0.5? Yes - this outcome or more extreme will happe 0.1% of the time if p = 0.5, which is very ulikely. 3

4 Hypotheses ad P -values Structure of the Hypotheses The claim ca take two forms. The proportio has ot chaged from a previously determied value p 0. The proportio has chaged from a previously determied value p 0. Null Hypothesis H 0 The claim beig tested i a test of sigificace. The test is desiged to assess the stregth of the evidece agaist H 0. Usually H 0 is a statemet of o effect or o differece. Alterative Hypothesis H a The ame of the statemet suspected to be true istead of H 0. The Statistical Evidece About the Hypothesis: P -value The probability, assumig that H 0 is true, that the sample outcome would be as extreme or more extreme tha the actually observed outcome. 4

5 Steps to Testig Sigificace 1. State the ull ad alterative hypotheses, H 0 ad H a. 2. Idetify the samplig distributio of ˆp whe H 0 is true. 3. Calculate the value of ˆp o which the test will be based. 4. Fid the P -value for the observed value of ˆp. 5. Iterpret what the P -value idicates about additioal samples. 6. Draw a coclusio based o the P -value. 5

6 Example: Cois I 400 flips, a coi lads heads 228 times. Is the coi a fair coi? Hypotheses: H 0 : p = 0.5 H a : p 0.5 Distributio of ˆp whe H 0 is true: The samplig distributio is ormal with mea = p = 0.50 ad stadard deviatio p(1 p) (0.5)(0.5) σ = = = Values of ˆp Observed value of ˆp: ˆp = 228/400 = 0.57 P -value: stadard value = ( )/0.025 = 2.8 The P -value is = Iterpretatio: The probability, assumig that p = 0.5, of observig a sample proportio at least as large as 0.57 is Coclusio: This value of ˆp provides strog evidece agaist the claim that the coi is fair. 6

7 Reasoig of Statistical Tests of Sigificace Basic Statistical Belief The sample is ot uusual but is commo. Two Types of Statistical Evidece Large P -value idicates the statistic (ˆp) or worse occurs frequetly; i.e., if H 0 is true, the observed sample would be a commo sample; is cosistet with the basic statistical belief; does ot provide evidece agaist H 0 ; coclude that there is ot sufficiet statistical evidece to reject H 0 ; i.e., do ot reject H 0. Small P -value idicates the statistic (ˆp) or worse rarely occurs i a radom sample; i.e., if H 0 is true, the sample would be a rare sample (If A is true, the B is true); does ot agree with the basic statistical belief: Two possible scearios (1) H 0 is true ad the sample is a uusual oe; (2) The sample is ot uusual (B is false). The H 0 is probably false (A is false). provides evidece agaist H 0 ; coclude there is sufficiet statistical evidece to reject H 0 ; i.e., the evidece idicates H a is true. 7

8 Example: Poll A politicia believes he ca vote ay way he wats o a cotroversial bill without sufferig ay fallout provided he has the support of 40% (or more) of his electorate. Suppose a radom sample of 225 costituets reveals support from oly 81 voters. Is this sigificat evidece to idicate that the politicia should be cocered about political fallout from this vote? H 0 : p = 0.40 H a : p < 0.40 Distributio of ˆp whe H 0 is true: The samplig distributio is ormal with mea = p = 0.40 ad stadard deviatio p(1 p) (0.4)(0.4) σ = = = Values of ˆp Observed value of ˆp: ˆp = 81/225 = 0.36 P -value: stadard value = ( )/ = 1.22 The P -value = Iterpretatio: The probability, assumig that p = 0.4, of observig a sample proportio o larger tha 0.36 is Coclusio: This value of ˆp does ot provide strog evidece agaist the claim that 40% of the electorate support the politicia. The politicia should be ot be too cocered about political fallout from this vote 8

9 Example: DVDs A maufacturer of DVDs believes that, uder adequate quality cotrol, o more tha 6% of the DVDs should be retured as faulty. For a radom sample of 250 sales of these DVDs, it was foud that 22 were retured as faulty. Determie the P -value for a test of the hypothesis that the percetage of all DVDs retured as faulty is at most 6% ad state what this idicates about the percetage of faulty discs uder this productio process. H 0 : p = 0.06 H a : p > 0.06 Distributio of ˆp whe H 0 is true: The samplig distributio is ormal with mea = p = 0.06 ad stadard deviatio p(1 p) (0.06)(0.94) σ = = = Observed value of ˆp: ˆp = 22/250 = P -value: stadard value = ( )/ The P -value = = Iterpretatio: The probability, assumig that p = 0.06, of observig a sample proportio at least as large as is Coclusio: This value of ˆp provides strog evidece agaist the claim that o more tha 6% of the DVDs are faulty. 9