Oe-Sample Test for Proportio Approximated Oe-Sample Z Test for Proportio CF Jeff Li, MD., PhD. November 1, 2005 c Jeff Li, MD., PhD. c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 1 I DM-TKR Data, there are 5 ifective patiets of total 78 patiets, the sample proportio is 6.41% = 5/78. The ifectiive probability i U.S. is about 1%. Do our sample differ from U.S. populatio? 1. π be ifective probability i populatio 2. Radom sample of observatios 3. X i be radom variables for each idividual, i = 1,..., 1, ifectio with probability π, X i = 0, o ifectio with probability 1 π. 1 4. Y = i=1 X i has Biomial distributio, π There is quite a variety of hypotheses about the DM populatio ifective probability π. c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 2 c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 3 Hypothesis Test Statistics H 0 : π = π 0 = 0.01, H A : π π 0. The observable sample proportio ˆπ = Y = i=1 X i, 2 The sample distributio of the sample proportio ˆπ π1 π ˆπ N π, 3 The obsereved sample test statistic uder H 0 ˆπ π Z = 0 N0, 1 4 π0 1 π 0 / approximate distributio It is called approximated Z test sice it use the Z statistic. c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 4 c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 5
Testig Hypothesis: Z Value Method The test statistic Z depeds upo 1. The sample proportio ˆπ 2. The hypothesized target geeral populatio proportio π 3. The populatio stadard deviatio, π1 π. If the ull hypothesis H 0 is true, the the hypothesized populatio proportio π 0 = 0.01 is equal to the populatio proportio, π. Testig Hypothesis: Z Value Method 1. Prescirbe Type I Error α 2. Z 1 α/2 be the correspodig percetile from N0, 1 such tat PZ < Z α =α 3. Uder H 0 : π = π 0, the observed test statistic z = ˆπ π 0 π0 1 π 0 /. 5 c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 6 c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 7 Testig Hypothesis: Z Value Method 1. For two-sided alterative test, H A : π π 0 2. Reject the H 0 whe z > Z 1 α/2. Critical Value ad Critical Regio Methods Give the sigificat level α P Z > Z 1 α/2 =α ˆπ π P > Z1 α/2 = α π1 π/ π1 π π1 π P ˆπ < π Z 1 α/2 or ˆπ > π + Z 1 α/2 c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 8 c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 9 Critical Value ad Critical Regio Methods Uder H 0 : π = π 0,wechoosethetwocritical values for the two-sided Ztestare π0 1 π c α,1 = π 0 Z 0 1 α/2 6 ad c α,2 = π 0 + Z 1 α/2 π0 1 π 0. 7 We will reject the H 0 basedothecritical regio whe ˆπ = y = i=1 x i ˆπ < c α,1 = π 0 Z 1 α/2 π0 1 π 0. 8 or ˆπ > c α,2 = π 0 + Z 1 α/2 π0 1 π 0. 9 Cofidece Iterval Method The two-sided 1 α 100% cofidece iterval of the populatio proportio π based o the sample statistic ˆπ, ad the two-sided alterative hypothesis H A : π π 0,is P[ Z < Z 1 α/2 ]=1 α [ ˆπ π ] P < Z1 α/2 = 1 α π1 π/ π1 π ] P[ ˆπ π < Z1 α/2 = 1 α [ π1 π P π > ˆπ Z 1 α/2 π1 π ] ad π < ˆπ + Z 1 α/2 = 1 α. c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 10 c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 11
Cofidece Iterval Method The two-sided 1 α 100% cofidece iterval of the populatio proportio π based o the sample statistic ˆπ, is ˆπ1 ˆπ ˆπ1 ˆπ ˆπ Z 1 α/2, ˆπ + Z 1 α/2. 10 We will reject the two-sided test whe the two-sided 1 α 100% cofidece iterval of the populatio does ot cotai the hypothesized populatio proportio π 0 uder H 0. Cofidece Iterval Method For H 0 : π = π 0 versus H A : π π 0, we will reject the H 0 whe ˆπ1 ˆπ π 0 < ˆπ Z 1 α/2, 11 or π 0 > ˆπ + Z 1 α/2 ˆπ1 ˆπ. 12 That is whe the hypothesized proportio π 0 is below the lower or above the upper cofidet limit, we will reject H 0. c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 12 c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 13 p-value Method 1. We have collected the data ad the observed sample statistic is ˆπ. 2. Cosider the two-sided hypothesis H 0 : π = π 0 versus H A : π π 0. 3. The observed two-sided Z test sample statistic is z = ˆπ π 0 π 0 1 π 0 /. 13 4. The p-value is defied as The p-value is the probability of obtaiig a result as/or more extreme tha you did by chace aloe assumig the ull hypothesis H 0 is true. p-value Method The p-value for two-sided test is calculated as P Y > ȳ π = π 0 = P ˆπ π 0 > x π 0 π = π 0 ˆπ π = P 0 π0 1 π 0 / x π > 0 π0 1 π 0 / π = π 0 = P Z > z π = π 0 = 2[1 PZ z π = π 0 ] = 2[1 Φ z ], We will reject the two-sided ull hypothesis H 0 whe p-value, 2[1 Φ z ], is less tha the sigificat level α. c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 14 c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 15 I DM-TKR Data, there are 5 ifective patiets of total 78 patiets, the sample proportio is 6.41% = 5/78. The ifectiive probability i U.S. is about 1%. Do our sample differ from U.S. populatio? 1. We wish to test the ull hypothesis ad alterative hypothesis are H 0 : π = π 0 = 0.01 versus H A : π π 0. 2. We have collected the data. 3. The observed sample proportio ˆπ, test statistic is 6.4%. 4. Let the sigificat level α = 0.05, adz 1 α/2 = 1.960. c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 16 c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 17
For two-sided test, the critical value ad critical regio for ˆπ is π0 1 π π 0 Z 0 1 α/2 = 0.01 0.99 0.01 1.960 78 = 0.01 + 1.960 0.01127 = 0.0121 ad π0 1 π π 0 + Z 0 0.01 0.99 1 α/2 = 0.01 + 1.960 78 = 0.01 + 1.960 0.01127 = 0.0322. We decide to reject the ull hypothesis H 0 if π1 π ˆπ < 0.0121 = π 0 Z 1 α/2 or ˆπ > 0.0322 = π 0 + Z 1 α/2 π1 π. Now the observed sample proportio ˆπ = 6.41% > 0.0322, so we reject the ull hypothesis. Critical values, c α,1, c α,2,are 0.0121, 0.0322. c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 18 c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 19 1. The observed sample test statistic, z, is calculated as z = ˆπ π 0 π0 1 π 0 / 0.0641 0.01 = = 4.80. 0.01 0.09/ 78 2. The observed sample test statistic, z, is 4.80 which is greater tha the Z critical value, Z 1 α/2 = 1.960. 3. So we reject the ull hypothesis H 0. 1. The two-sided 1 α 100% cofidece iterval for DM populatio proportio π based o the sample statistic, ˆπ, cabecalculatedas ˆπ1 ˆπ ˆπ1 ˆπ ˆπ Z 1 α/2, ˆπ + Z 1 α/2 = 0.00975, 0.1185. 2. The 1 α 100% cofidece iterval for DM populatio proportio π is 0.00975, 0.1185. 3. This iterval does ot cotai π 0 = 0.01. 4. So we reject the ull hypothesis H 0. c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 20 c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 21 1. The p-value based o the observed sample test statistic, z = 4.80, ca be calculated as 2[1 P ˆπ > x =2[1 Φ z ] = Φ4.80 < 0.0001. 2. The p-value, < 0.0001, is less tha the sigificat level α = 0.05. 3. So we reject the ull hypothesis. : R > y<-5; <-78 # assig y ad i biomial > alpha<-0.05 # assig sigificat level alpha > pihat<-y/ # sample proportio > pihat [1] 0.06410256 > qihat<-1-pihat > se0<-sqrtpi0*1-pi0/ # s.e. uder H0 > se1<-sqrtpihat*qihat/ # s.e. Uder HA > Z1alpha<-qorm1-alpha/2 # Z_{1-alpha/2} quatile > ztest<-pihat-pi0/se0 # sample Z test statistic > ztest [1] 4.802281 c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 22 c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 23
: R > crit1<-pi0-z1alpha*se0 # critical vale c_{alpha,1} > crit2<-pi0+z1alpha*se0 # critical value c_{alpha,2} > crit1 [1] -0.01208098 > crit2 [1] 0.03208098 > Z.CI.L<-pihat-Z1alpha*se1 # C.I. Lower > Z.CI.U<-pihat+Z1alpha*se1 # C.I. Upper > Z.CI.L [1] 0.009745922 > Z.CI.U [1] 0.1184592 of Oe-sample Z Test for Proportio 1. The first thig is to decided the possible π value uder H A,sice differet π value uder H A will have differet power. 2. Suppose i the DM-TKR example, we have π 0 = 0.01 ad π = π A = 0.005 or π A = 0.05 uder H A. 3. What is power of the two-sided test? power = Preject H 0 H A is true 4. is the probability of makig the correct decisio whe the ull hypothesis is ot true. Specially, power = 1 β = Preject H 0 : π = π 0 H A is true. c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 24 c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 25 If H A : π = π A < π 0 is true, the we will reject H 0 whe π0 1 π ˆπ < c α = π 0 Z 0 1 α/2 14 or π0 1 π ˆπ π A π 0 Z 1 α/2 0 πa 1 π A / < π A πa 1 π A /. 15 If H A : π = π A < π 0 is true, power = 1 β = P ˆπ < c α π = π A 16 π0 1 π 0 = P ˆπ < π 0 Z 1 α/2 π = π A 17 = P Z = = P Z < π0 1 π 0 ˆπ π A πa 1 π A / < π 0 Z 1 α/2 π A πa 1 π A / π = π A π0 1 π 0 Z 1 α/2 + π 0 π A π = π A πa 1 π A π0 1 π 0 / π 0 1 π 0 = Φ Z 1 α/2 + π 0 π A. 18 πa 1 π A π0 1 π 0 / c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 26 c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 27 If H A : π = π A > π 0 is true, the we will reject H 0 whe π0 1 π ˆπ > c α = π 0 + Z 0 1 α/2, 19 or π0 1 π ˆπ π A π 0 + Z 1 α/2 0 πa 1 π A / > π A πa 1 π A /. 20 If H A : π = π A > π 0 is true, power = 1 β = P ˆπ > c α π = π A 21 π0 1 π 0 = P ˆπ > π 0 + Z 1 α/2 π = π A 22 = P Z = = P Z > π0 1 π 0 ˆπ π A πa 1 π A / > π 0 + Z 1 α/2 π A πa 1 π A / π = π A π0 1 π 0 + Z 1 α/2 + π 0 π A π = π A πa 1 π A π0 1 π 0 / = π 0 1 π 0 1 Φ + Z 1 α/2 + π 0 π A πa 1 π A π0 1 π 0 / = π 0 1 π 0 Φ Z 1 α/2 + π 0 π A. πa 1 π A π0 1 π 0 / 23 c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 28 c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 29
So the power of the two-sided test H 0 : π = π 0 versus H A : π π 0 for the specific alterative π = π A, where the uderlyig distributio is approximately ormal ad the populatio variace σ 2 is estimated as π A 1 π A, is give exactly by [ π0 1 π power = Φ 0 Z 1 α/2 + π 0 π A πa 1 π A π0 1 π 0 / ]. 24 1. The power formula has othig to do with observed sample statistic z, however, the power depeds o π A ad its variace π A 1 π A. 2. If we cosider H A : π = ˆπ, that is, we calculated the power after the study, this is sometime called post-hoc power. It is ot recommed by may statisticias. 3. For oe-sided test, we use Z 1 α istead of Z 1 α/2. c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 30 c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 31 1. Suppose H A : π = π A, for example, π A = 0.05. Wehavepower π Φ 0 1 π 0 π0 π A = 0.76612. πa 1 π A π0 1 π 0 / 2. If we assume H A : π A = 0.09, the the power is π Φ 0 1 π 0 Z 1 α/2 + π 0 π A = 0.9630. πa 1 π A π0 1 π 0 / The power depeds o the variace of π A, so the directio of power of the approximate Z test for proportio is ot as clear as ormal distributio. Note: the power formula has othig to do with observed sample statistic z. If we cosider H A : π = ˆπ, that is, we calculated the power after the study, this is sometime called post-hoc power. Itisot recommed by may statisticias. c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 32 c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 33 : R > power.prop.two.sided.z.test<-fuctiopi0,pia,alpha, # t { power<-sqrtpi0*1-pi0/pia*1-pia *-qorm1-alpha/2 +abspi0-pia*sqrt/sqrtpi0*1-pi0 power<-pormpower cat"power = ",power,"\" } > power.prop.two.sided.z.test0.01,0.05,0.05,78 power = 0.7661206 > power.prop.two.sided.z.test0.01,0.09,0.05,78 power = 0.9630652 Exact Small-Sample Iferece Exact Test for Proportio c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 34 c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 35
Exact Test for Proportio 1. The approximate Z test procedure to test the hypothesis H 0 : π = π 0 versus H A : π π 0 depeds o the assumptios is oly true if π 0 1 π 0 5. 2. With moder computatioal power, it is ot ecessary to rely o large-sample approximatio for the distributio of statistics such as ˆπ. 3. Tests ad cofidece itervals ca use the biomial distributio directly rather tha its ormal approximatio. Such ifereces occur aturally for small samples, but apply for ay. Exact Test for Proportio 1. We will base our test o exact biomial probabilities. 2. Let Y Bi, π 0 uder H 0. 3. Let ˆπ = y/, be the observed sample proportio. 4. The computatio of the p-value depeds o whether ˆπ π 0 or ˆπ π 0. c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 36 c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 37 1. If ˆπ π 0,the p-value of the Exact Test for Proportio p value/2 = P y successes i trials H 0 25 = y π k 0 k 1 π 0 k k=0 26 2. If ˆπ π 0,the p value/2 = P y successes i trials H 0 27 = π k 0 k 1 π 0 k 28 k=y 1. If ˆπ π 0,the p-value of the Exact Test for Proportio p value = 2 PY y 29 = [ y ] mi 2 π k k 0 1 π 0 k, 1 k=0 30 2. If ˆπ π 0,the p value = 2 PY y 31 [ ] = mi 2 π k k 0 1 π 0 k, 1 32 k=y c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 38 c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 39 p-value of the Exact Test for Proportio We illustrate by testig H 0 : π = 0.5 agaist H A : π 0.5 for the survey results, y = 0, with = 25. We oted that the score statistic equals z = 5.0. Theexactp-value for this statistic, based o the ull Bi25, 0.5 distributio, is P z 5.0 =PY = 0 or Y = 25 =0.5 25 + 0.5 25 = 0.00000006. C.I. of the Exact Test for Proportio 1. The exact 1001 α% cofidece itervals cosists of all π for which p-values exceed α i exact biomial tests. 2. The best kow iterval Clopper ad Perso, 1934 uses the tail method for formig cofidece itervals. it requires each oe-sided p-value to exceed α/2. c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 40 c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 41
C.I. of the Exact Test for Proportio The lower ad upper edpoits are the solutios i π 0 to the equatios π k 0 k 1 π 0 k = α/2 k=y y ad π k 0 k 1 π 0 k = α/2, 33 k=0 except that the lower boud is 0 whe y = 0 ad the upper boud is 1 whe y =. C.I. of the Exact Test for Proportio 1. The Clopper ad Perso cofidece iterval equals [ 1 + ] 1 [ ] 1 y + 1 y + 1 < π < 1 +, 34 y + 1 F 2y+1,2 y,1 α/2 yf 2y,2 y+1,α/2 where F a,b,c deotes the c quatile form the F distributio with degrees of freedom a ad b. 2. Example, Whe y = 0 with = 25, theclopper-pearso 95% cofidece iterval for π is 0, 0.137. c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 42 c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 43 : Exact Test The exact 1 α 100% cofidet iterval is 0.0211, 0.1433. The exact two-sided test p-value is 0.001152 > α = 0.05 SAS: 0.0023. We reject the ull hypothesis H 0 based o the exact cofidece iterval ad p-value. : Exact Test with R biom.test > biom.testx=5,=78,p=0.01,alterative = c"two.sided", cof.level = 0.95 Exact biomial test data: 5 ad 78 umber of successes=5, umber of trials=78, p-value=0.001152 alterative hypothesis: true probability of success is ot equal to 0.01 95 percet cofidece iterval: 0.02113972 0.14328760 sample estimates: probability of success 0.06410256 c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 44 c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 45 : Exact C.I. ad Asymptotic C.I. with R > libraryhmisc > helpbicof > bicofx=5,=78,alpha=0.05,method="all" PoitEst Lower Upper Exact 0.06410256 0.021139720 0.1432876 Wilso 0.06410256 0.027689315 0.1414360 Asymptotic 0.06410256 0.009745922 0.1184592 : R > helpprop.test > prop.testx=5,=78,p=0.01,alterative = c"two.sided", correct=f,cof.level = 0.95 1-sample proportios test without cotiuity correctio data: 5 out of 78, ull probability 0.01 X-squared = 23.0619, df = 1, p-value = 1.569e-06 alterative hypothesis: true p is ot equal to 0.01 95 percet cofidece iterval: 0.02768932 0.14143595 sample estimates: p 0.06410256 c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 46 c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 47
: Exact Test with R Warig message: Chi-squared approximatio may be icorrect i: prop.testx=5, =78, p=0.01,alterative=c"two.sided", : SAS title "FREQ: Oe-sample Z test for proportio with 95% C.I." ; proc freq data=dmtkaew order=data ; exact biomial ; tables ifect / bi p=0.01 ; ru; c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 48 c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 49 : SAS title "Categroical Data: Graphics of Oe-sample"; proc gchart data=dmtkaew ; vbar ifect / discrete ; hbar ifect / discrete ; pie ifect / discrete ; pie ifect / discrete explode=1 slice=arrow percet=iside ; ru; : SAS The FREQ Procedure Cumulative Cumulative ifect Frequecy Percet Frequecy Percet ----------------------------------------------------------- 1 5 6.41 5 6.41 0 73 93.59 78 100.00 c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 50 c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 51 : SAS Biomial Proportio for ifect = 1 ----------------------------------- Proportio P 0.0641 ASE 0.0277 95% Lower Cof Limit 0.0097 95% Upper Cof Limit 0.1185 : SAS Exact Cof Limits 95% Lower Cof Limit 0.0211 95% Upper Cof Limit 0.1433 c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 52 c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 53
: SAS Test of H0: Proportio = 0.01 ASE uder H0 0.0113 Z 4.8023 Oe-sided Pr > Z <.0001 Two-sided Pr > Z <.0001 : SAS Exact Test Oe-sided Pr >= P 0.0012 Two-sided = 2 * Oe-sided 0.0023 Sample Size = 78 c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 54 c Jeff Li, MD., PhD. Oe Sample Test for Proportio, 55