NCSS Statistical Software

Transcription

1 Chater 54 Two Proortios Itroductio This rogram comutes both asymtotic ad eact cofidece itervals ad hyothesis tests for the differece, ratio, ad odds ratio of two roortios. Comarig Two Proortios I recet decades, a umber of otatio systems have bee used to reset the results of a study for comarig two roortios. For the uroses of the techical details of this chater, we will use the followig otatio: Evet No-Evet Total Grou Grou Totals m m NN I this table, the label Evet is used, but might istead be Success, Attribute of Iterest, Positive Resose, Disease, Yes, or somethig else. The biomial roortios P ad P are estimated from the data usig the formulae ad Three commo comariso arameters of two roortios are the roortio differece, roortio (risk ratio, ad the odds ratio: Parameter Notatio Differece δ P P Risk Ratio φ P / P Odds Ratio ψ P / P / ( P ( P Although these three arameters are (o-liear fuctios of each other, the choice of which is to be used should ot be take lightly. The associated tests ad cofidece itervals of each of these arameters ca vary widely i ower ad coverage robability. 54-

2 Two Proortios Differece The roortio (risk differece δ P P is erhas the most direct method of comariso betwee the two evet robabilities. This arameter is easy to iterret ad commuicate. It gives the absolute imact of the treatmet. However, there are subtle difficulties that ca arise with its iterretatio. Oe iterretatio difficulty occurs whe the evet of iterest is rare. If a differece of 0.00 were reorted for a evet with a baselie robability of 0.40, we would robably dismiss this as beig of little imortace. That is, there usually is little iterest i a treatmet that decreases the robability from to However, if the baselie robably of a disease was 0.00 ad 0.00 was the decrease i the disease robability, this would rereset a reductio of 50%. Thus we see that iterretatio deeds o the baselie robability of the evet. A similar situatio occurs whe the amout of ossible differece is cosidered. Cosider two evets, oe with a baselie evet rate of 0.40 ad the other with a rate of 0.0. What is the maimum decrease that ca occur? Obviously, the first evet rate ca be decreased by a absolute amout of 0.40 while the secod ca oly be decreased by a maimum of 0.0. So, although creatig the simle differece is a useful method of comariso, care must be take that it fits the situatio. Ratio The roortio (risk ratio φ / gives the relative chage i risk i a treatmet grou (grou comared to a cotrol grou (grou. This arameter is also direct ad easy to iterret. To comare this with the differece, cosider a treatmet that reduces the risk of disease from to Which sigle umber is most elighteig, the fact that the absolute risk of disease has bee decreased by , or the fact that risk of disease i the treatmet grou is oly 55.8% of that i the cotrol grou? I may cases, the ercetage (00 risk ratio commuicates the imact of the treatmet better tha the absolute chage. Perhas the biggest drawback of this arameter is that it caot be calculated i oe of the most commo eerimetal desigs: the case-cotrol study. Aother drawback, whe comared to the odds ratio, is that the odds ratio occurs aturally i the likelihood equatios ad as a arameter i logistic regressio, while the roortio ratio does ot. Odds Ratio Chaces are usually commuicated as log-term roortios or robabilities. I bettig, chaces are ofte give as odds. For eamle, the odds of a horse wiig a race might be set at 0-to- or 3-to-. How do you traslate from odds to robability? A odds of 3-to- meas that the evet will occur three out of five times. That is, a odds of 3-to- (.5 traslates to a robability of wiig of The odds of a evet are calculated by dividig the evet risk by the o-evet risk. Thus, i our case of two oulatios, the odds are O P ad P P P For eamle, if P is 0.60, the odds are 0.60/ I some cases, rather tha reresetig the odds as a decimal amout, it is re-scaled ito whole umbers. Thus, istead of sayig the odds are.5-to-, we may equivaletly say they are 3-to-. O 54-

3 Two Proortios I this cotet, the comariso of roortios may be doe by comarig the odds through the ratio of the odds. The odds ratio of two evets is ψ O O P P P P Util oe is accustomed to workig with odds, the odds ratio is usually more difficult to iterret tha the roortio (risk ratio, but it is still the arameter of choice for may researchers. Reasos for this iclude the fact that the odds ratio ca be accurately estimated from case-cotrol studies, while the risk ratio caot. Also, the odds ratio is the basis of logistic regressio (used to study the ifluece of risk factors. Furthermore, the odds ratio is the atural arameter i the coditioal likelihood of the two-grou, biomial-resose desig. Fially, whe the baselie evet-rates are rare, the odds ratio rovides a close aroimatio to the risk ratio sice, i this case, P P, so that P P P ψ φ P P P Oe beefit of the log of the odds ratio is its desirable statistical roerties, such as its cotiuous rage from egative ifiity to ositive ifiity. Cofidece Itervals Both large samle ad eact cofidece itervals may be comuted for the differece, the ratio, ad the odds ratio. Cofidece Itervals for the Differece Several methods are available for comutig a cofidece iterval of the differece betwee two roortios δ P P. Newcombe (998 coducted a comarative evaluatio of eleve cofidece iterval methods. He recommeded that the modified Wilso score method be used istead of the Pearso Chi-Square or the Yate s Corrected Chi-Square. Beal (987 foud that the Score methods erformed very well. The lower L ad uer U limits of these itervals are comuted as follows. Note that, uless otherwise stated, α / is the aroriate ercetile from the stadard ormal distributio. Cells with Zero Couts Etreme cases i which some cells are ero require secial aroaches with some of the tests give below. We have foud that a simle solutio that works well is to chage the eros to a small ositive umber such as 0.0. This roduces the same results as other techiques of which we are aware. C.I. for Differece: Wald Z with Cotiuity Correctio For details, see Newcombe (998, age

4 Two Proortios 54-4 ( ( L ( ( U C.I. for Differece: Wald Z For details, see Newcombe (998, age 875. ( ( L ( ( U C.I. for Differece: Wilso s Score as modified by Newcombe For details, see Newcombe (998, age 876 B L ˆ C U ˆ where ( ( u u l l B ( ( l l u u C ad l ad u are the roots of ( 0 P P P ad l ad u are the roots of ( 0 P P P C.I. for Differece: Miettie-Nurmie Score Miettie ad Nurmie (985 roosed a test statistic for testig whether the odds ratio is equal to a secified valueψ 0. Because the aroach they used with the differece ad ratio does ot easily eted to the odds ratio, they used a score statistic aroach for the odds ratio. The regular MLE s are ad. The costraied MLE s are ad, These estimates are costraied so that ψ ψ 0. A correctio factor of N/(N- is alied to make the variace estimate less biased. The sigificace level of the test statistic is based o the asymtotic ormality of the score statistic. The formula for comutig the test statistic is

5 where ψ 0 ( ψ 0 MNO Two Proortios ( ( q q N N q N q N B B 4AC A ( ψ A N 0 B N ψ N M ( ψ C M 0 0 Miettie ad Nurmie (985 roosed ivertig their score test to fid the cofidece iterval. The lower limit is foud by solvig ad the uer limit is the solutio of MND α / MND α / C.I. for Differece: Farrigto-Maig Score Farrigto ad Maig (990 roosed a test statistic for testig whether the differece is equal to a secified valueδ 0. The regular MLE s ad are used i the umerator of the score statistic while MLE s ad costraied so that δ0 are used i the deomiator. The sigificace level of the test statistic is based o the asymtotic ormality of the score statistic. The formula for comutig the test is δ0 FMD q q where the estimates ad are comuted as i the corresodig test of Miettie ad Nurmie (985 give above. Farrigto ad Maig (990 roosed ivertig their score test to fid the cofidece iterval. The lower limit is foud by solvig ad the uer limit is the solutio of FMD α / FMD α / 54-5

6 Two Proortios C.I. for Differece: Gart-Nam Score Gart ad Nam (990 age 638 roosed a modificatio to the Farrigto ad Maig (988 differece test that corrected for skewess. Let FM ( δ stad for the Farrigto ad Maig differece test statistic described above. The skewess corrected test statistic GN is the aroriate solutio to the quadratic equatio γ δ γ 0 where 3/ V q q γ 6 ( ( GND ( GND FMD( ( δ ( q ( q Gart ad Nam (988 roosed ivertig their score test to fid the cofidece iterval. The lower limit is foud by solvig ad the uer limit is the solutio of GND α / GND α / C.I. for Differece: Che s Quasi-Eact Method Che (00 roosed a quasi-eact method for geeratig cofidece itervals. This method roduces itervals that are close to ucoditioal eact itervals that are available i secialied software like StatXact, but do ot require as much time to comute. Che s method iverts a hyothesis test based o Farrigto ad Maig s method. That is, the cofidece iterval is foud by fidig those values at which the hyothesis test that the differece is a give, o-ero value become sigificat. However, istead of searchig for the maimum sigificace level of all ossible values of the uisace arameter as the eact tests do, Che roosed usig the sigificace level at the costraied maimum likelihood estimate of as give by Farrigto ad Maig. This simlificatio results i a huge reductio i comutatio with oly a mior reductio i accuracy. Also, it allows much larger samle sies to be aalyed. Note o Eact Methods A word of cautio should be raised about the hrase eact tests or eact cofidece itervals. May users assume that methods that are based o eact methods are always better tha other, o-eact methods. After all, eact souds better tha aroimate. However, tests ad cofidece itervals based o eact methods are ot ecessarily better. I fact, some romiet statisticias are of the oiio that they are actually worse (see Agresti ad Coull (998 for oe eamle. Eact simly meas that they are based o eact distributioal calculatios. They may be, however, coservative i terms of their coverage robabilities (the robability that the cofidece iterval icludes the true value. That is, they are wider tha they eed to be because they are based o worst case scearios. Cofidece Itervals for the Ratio C.I. for Ratio: Miettie-Nurmie Score Miettie ad Nurmie (985 roosed a test statistic for testig whether the ratio is equal to a secified value φ 0. The regular MLE s ad are used i the umerator of the score statistic while MLE s ad costraied so that / φ0 are used i the deomiator. A correctio factor of N/(N- is alied to make the variace estimate less biased. The sigificace level of the test statistic is based o the asymtotic ormality of the score statistic. 54-6

7 Here is the formula for comutig the test where φ 0 MNR Two Proortios / φ0 q q N φ0 N B B 4AC A A Nφ 0 [ φ0 φ0 ] B N N C M Miettie ad Nurmie (985 roosed ivertig their score test to fid the cofidece iterval. The lower limit is foud by solvig ad the uer limit is the solutio of MNR α / MNR α / C.I. for Ratio: Farrigto-Maig Score Farrigto ad Maig (990 roosed a test statistic for testig whether the ratio is equal to a secified value φ 0. The regular MLE s ad are used i the umerator of the score statistic while MLE s ad costraied so that / φ0 are used i the deomiator. A correctio factor of N/(N- is alied to icrease the variace estimate. The sigificace level of the test statistic is based o the asymtotic ormality of the score statistic. Here is the formula for comutig the test FMR / φ0 q q φ0 where the estimates ad are comuted as i the corresodig test of Miettie ad Nurmie (985 give above. Farrigto ad Maig (990 roosed ivertig their score test to fid the cofidece iterval. The lower limit is foud by solvig ad the uer limit is the solutio of FMR α / FMR α / 54-7

8 Two Proortios C.I. for Ratio: Gart-Nam Score Gart ad Nam (988 age 39 roosed a modificatio to the Farrigto ad Maig (988 ratio test that φ stad for the Farrigto ad Maig ratio test statistic described above. The FM corrected for skewess. Let ( skewess corrected test statistic GN is the aroriate solutio to the quadratic equatio where ϕ u / 6 3 q q q q u ( q ( q ( ϕ φ ϕ 0 ( GNR ( GNR FMR( Gart ad Nam (988 roosed ivertig their score test to fid the cofidece iterval. The lower limit is foud by solvig ad the uer limit is the solutio of GNR α / GNR α / C.I. for Ratio: Kat Logarithm This was oe of the first methods roosed for comutig cofidece itervals for risk ratios. For details, see Gart ad Nam (988, age 34. L ˆ φ e q q U ˆ φ e q q where ˆ φ C.I. for Ratio: Walters Logarithm / For details, see Gart ad Nam (988, age 34. where ˆ φ e l l L φ e ( u ( u U φ e 54-8

9 uˆ q q V φ φ q q q ( q ( ( q ( q q µ 3/ 3 v v q q Two Proortios C.I. for Ratio: Che s Quasi-Eact Method Che (00 roosed a quasi-eact method for geeratig cofidece itervals. This method roduces itervals that are close to ucoditioal eact itervals that are available i secialied software like StatXact, but do ot require as much time to comute. Che s method iverts a hyothesis test based o Farrigto ad Maig s method. That is, the cofidece iterval is foud by fidig those values at which the hyothesis test that the differece is a give, o-ero value become sigificat. However, istead of searchig for the maimum sigificace level of all ossible values of the uisace arameter as the eact tests do, Che roosed usig the sigificace level at the costraied maimum likelihood estimate of as give by Farrigto ad Maig. This simlificatio results i a huge reductio i comutatio with oly a mior reductio i accuracy. Also, it allows much larger samle sies to be aalyed. Cofidece Itervals for the Odds Ratio The odds ratio is a commoly used measure of treatmet effect whe comarig two biomial roortios. It is the ratio of the odds of the evet i grou oe divided by the odds of the evet i grou two. The results give below are foud i Fleiss (98. Symbolically, the odds ratio is defied as P ψ P P P C.I. for Odds Ratio: Simle Techique The simle estimate of the odds ratio uses the formula ψ ˆ 54-9

10 Two Proortios The stadard error of this estimator is estimated by se ( ψ ˆ ψˆ Problems occur if ay oe of the quatities,,, or are ero. To correct this roblem, may authors recommed addig oe-half to each cell cout so that a ero caot occur. Now, the formulas become ad ψ ˆ ( 0.5( 0.5 ( 0.5( 0.5 se ( ψ ˆ ψˆ The distributio of these direct estimates of the odds ratio do ot coverge to ormality as fast as does their logarithm, so the logarithm of the odds ratio is used to form cofidece itervals. The formula for the stadard error of the log odds ratio is ad ( ψ L l se ( L A 00( α % cofidece iterval for the log odds ratio is formed usig the stadard ormal distributio as follows ( α ( ( ψ e lower L se L / ( α ψ e uer L se L / C.I. for Odds Ratio: Iterated Method of Fleiss Fleiss (98 resets a imrove cofidece iterval for the odds ratio. This method forms the cofidece iterval as all those value of the odds ratio which would ot be rejected by a chi-square hyothesis test. Fleiss gives the followig details about how to costruct this cofidece iterval. To comute the lower limit, do the followig.. For a trial value of ψ, comute the quatities X, Y, W, F, U, ad V usig the formulas X ψ Y F ( m ( m X 4mψ ψ ( W A B C D ( A W α / U B C A D 54-0

11 V Two Proortios [( A U W ( ] T A where A X Y ( ψ B m A C A D f A ψ T Y N ψ ( ψ Y [ X ( m m ( ] Fially, use the udatig equatio below to calculate a ew value for the odds ratio usig the udatig equatio ψ ( ( ψ V k k F. Cotiue iteratig util the value of F is arbitrarily close to ero. The uer limit is foud by substitutig for i the formulas for F ad V. Cofidece limits for the relative risk ca be calculated usig the eected couts A, B, C, ad D from the last iteratio of the above rocedure. The lower limit of the relative risk φ lower φ uer C.I. for Odds Ratio: Matel-Haesel The commo estimate of the logarithm of the odds ratio is used to create this estimator. That is l A B A B lower lower uer uer ( ψ ˆ l The stadard error of this estimator is estimated usig the Robis, Breslow, Greelad (986 estimator which erforms well i most situatios. The stadard error is give by where AA BB CC DD A AD BC B se( l( ψ C CD D 54-

12 Two Proortios The cofidece limits are calculated as ( ( α / ( ( ( ( α / ( ( ψ e l ψ l ψ lower se ψ e l ψ l ψ uer se C.I. for Odds Ratio: Miettie-Nurmie Score Miettie ad Nurmie (985 roosed a test statistic for testig whether the odds ratio is equal to a secified valueψ 0. Because the aroach they used with the differece ad ratio does ot easily eted to the odds ratio, they used a score statistic aroach for the odds ratio. The regular MLE s are ad. The costraied MLE s are ad, These estimates are costraied so that ψ ψ 0. A correctio factor of N/(N- is alied to make the variace estimate less biased. The sigificace level of the test statistic is based o the asymtotic ormality of the score statistic. The formula for comutig the test statistic is ( ( q q MNO N N q N q N where ψ 0 ( ψ 0 B B 4AC A ( ψ A N 0 B N ψ N M ( ψ 0 0 C M Miettie ad Nurmie (985 roosed ivertig their score test to fid the cofidece iterval. The lower limit is foud by solvig ad the uer limit is the solutio of MNO α / MNO α / 54-

13 Two Proortios C.I. for Odds Ratio: Farrigto-Maig Score Farrigto ad Maig (990 idicate that the Miettie ad Nurmie statistic may be modified by removig the factor N/(N-. The formula for comutig this test statistic is ( ( q q FMO N q N q where the estimates ad are comuted as i the corresodig test of Miettie ad Nurmie (985 give above. Farrigto ad Maig (990 roosed ivertig their score test to fid the cofidece iterval. The lower limit is foud by solvig ad the uer limit is the solutio of C.I. for Odds Ratio: Coditioal Eact FMO α / FMO α / The coditioal eact cofidece iterval of the odds ratio is calculated usig the ocetral hyergeometric distributio as give i Sahai ad Khurshid (995. That is, a 00( α % cofidece iterval is foud by searchig for ψ L ad ψ U such that k m k k m k k k ( ψ L k k k ( ψ L k k α where ( 0 k ma, m ad k mi (, m k m k k m k k ( ψu k k k k ( ψu k k α 54-3

14 Two Proortios Hyothesis Tests A wide variety of statistical tests are available for testig hyotheses about two roortios. Some tests are based o the differece i roortios, others are based o the ratio of roortios, ad still others are based o the odds ratio. Some tests are coditioal, while others are ucoditioal. Some tests are said to be large samle, while others are said to be eact. I this sectio, some of these distictios are elaied. Tyes of Hyothesis Tests Hyothesis tests cocerig two roortios ca be searated ito three categories: large samle, coditioal eact, ad ucoditioal eact. Large Samle Tests Large samle (or asymtotic tests are based o the cetral limit theorem (CLT which states that for large samles, the distributio of may of these test statistics aroach the ormal distributio. Hece, sigificace levels ca be comuted usig the ormal distributio which has bee etesively tabulated ad ca ow be easily comuted. A difficult determiatio whe decidig whether to use a large samle test is whether or ot the samle is large eough for the CLT to roerly take effect. Eact Tests i Geeral Because of the iaccuracy of alyig a large samle rocedure to a small samle study, aother class of tests has bee devised called eact tests. The sigificace levels of these tests are calculated from their eact distributio, usually by cosiderig either the biomial or the hyergeometric distributio. No aeal is made to the CLT. Because these tests are comutatioally itesive, they have icreased i oularity with icrease of the comutatioal abilities of comuters. Eve with the availability of moder comuters, aroimate large samle techiques caot be abadoed comletely i favor of eact tests, due to the assumtios required by the eact tests. The distributio of the roortios i a -by- table ivolves two arameters: ad δ i the case of the differece ad ad / φ i the case of the ratio. The hyothesis oly ivolves oe arameter, the differece or the ratio. The other arameter,, is called a uisace arameter because it is ot art of the hyothesis of iterest. That is, the hyothesis that δ 0 or φ does ot ivolve. I order to test hyotheses about the arameter of iterest, the uisace arameter must be elimiated. This may be accomlished either by coditioal methods or ucoditioal methods. Coditioal Eact Test The uisace arameter ca be elimiated by coditioig o a sufficiet statistic. Fisher s eact test is a eamle of this. The coditioig occurs by cosiderig oly those tables i which the row ad colum totals remai the same as for the data. This removes the uisace arameter from the distributio formula. This has draw criticism because most eerimetal desigs do ot fi both the row ad colum totals. Others have argued that sice the sigificace level is reserved ucoditioally, the test is valid. Ucoditioal Eact Test The ucoditioal eact test aroach is to remove the uisace arameter by comutig the sigificace level at all ossible values of the uisace arameter ad choosig the largest (worst case. That is, fid the value of which gives the maimum sigificace level (least sigificat for the hyothesis test. That is, these tests fid a uer boud for the sigificace level. 54-4

15 Two Proortios The roblem with the ucoditioal aroach is that the uer boud may occur at a value of that is far from the true value. For eamle, suose the true value of is 0.7 where the sigificace level is However, suose the maimum sigificace level of 0.3 occurs at Hece, ear the actual value of the uisace value, the results are statistically sigificat, but the results of the eact test are ot! Of course, i a articular study, we do ot kow the true value of the uisace arameter. The message is that although these tests are called eact tests, they are ot! They are aroimate tests comuted usig eact distributios. Hece, oe caot say broadly that eact tests are always better tha the large-samle test couterarts. Hyothesis Test Techical Details The sectios that follow give formulaic details of the hyothesis tests associated with this rocedure. Notatio for Hyothesis Test Statistics The followig otatio is used i the formulas for the test statistics. Evet No-Evet Total Samle Proortio Grou Grou Total m m NN mm NN Hyotheses for Iequality Tests of Proortio Differece Oe should determie i advace the directio of the ull ad alterative hyothesis of the test. Two-Sided HH 0 : PP PP vvvv. HH aa : PP PP Oe-Sided (Lower HH 0 : PP PP vvvv. HH aa : PP < PP Oe-Sided (Uer HH 0 : PP PP vvvv. HH aa : PP > PP Large-Samle (Asymtotic Iequality Tests of Proortio Differece The traditioal aroach was to use the Pearso chi-square test for large samles, the Yates chi-square for itermediate samle sies, ad the Fisher Eact test for small samles. Recetly, some author s have begu questioig this solutio. For eamle, based o eact eumeratio, Uto (98 ad D Agostio (988 cautio that the Fisher Eact test ad Yates test should ever be used. Wald Z-Test (Oe- ad Two-Sided The statistic for the Wald -test is comuted as follows ( 54-5

16 Two Proortios Wald Z-Test with Cotiuity Correctio (Oe- ad Two-Sided With the cotiuity correctio, the statistic becomes ( ssssssss( Chi-Square Test of Differece (Two-Sided Oly or Ideedece This hyothesis test takes its lace i history as oe of the first statistical hyothesis tests to be roosed. It was first roosed by Karl Pearso i 900. The two-sided test is comuted as χχ NN( mm mm where this statistic is comared to a chi-square distributio with oe degree of freedom. Chi-Square Test with Cotiuity Correctio of Differece (Two-Sided Oly or Ideedece With the cotiuity correctio, the chi-square test statistic becomes χχ NN( NN mm mm which also is comared to a chi-square distributio with oe degree of freedom. Coditioal Matel Haesel Test of Differece (Oe- ad Two-Sided The coditioal Matel Haesel test, see Lachi (000 age 40, is based o the ide frequecy,, from the table. The formula for the -statistic is where ( E V ( c m N m m N ( N E( ( V Likelihood Ratio Test of Differece (Two-Sided Oly or Ideedece I 935, Wilks showed that the followig quatity has a chi-square distributio with oe degree of freedom. This test is reseted, amog other laces, i Uto (98. The eressio for the statistic ca be reduced to LLLL llll NN NN mm mm mm mm c 54-6

17 Two Proortios Small-Samle (Eact Iequality Tests of Proortio Differece Fisher s Eact Test of Differece (Oe- ad Two-Sided or Ideedece Fisher s Eact test cosists of eumeratig all -by- tables that have the same margial frequecies as the observed table ad the summig the robability of the observed table ad all those that have robability less tha or equal to the observed table. The robability of a idividual table is derived from the hyergeometric robability distributio, where Pr (,,,!! mm! mm! N!!!!! Geeral Form of the Other Eact Tests i NCSS All of the eact tests follow the same atter. We will reset the geeral rocedure here, ad the give the secifics for each test. Secify the Null ad Alterative Hyotheses The first ste is to select a method to comare the roortios ad determie if the test is to be oe-, or two-, sided. These may be writte i geeral as H 0 h j, ( P P 0 : θ H h j, ( P P 0 : θ where (for two-sided tests could be relaced with < or > for a oe-sided test ad the ide j is defied as ( P P P. ( P, P P P P / ( ( P P, P P /( P h, P h h / 3 Secify the Referece Set The et ste is to secify the referece set of ossible tables to comare the observed table agaist. Two referece sets are usually cosidered. Defie Ω as the comlete set of tables that are ossible by selectig observatios from oe grou ad observatios from aother grou. Defie Γ as the subset from Ω for which m. Tests usig Ω are ucoditioal tests while tests usig Γ are coditioal tests. Secify the Test Statistic The et ste is to select the test statistic. I most cases, the score statistic is used which has the geeral form DD( h jj(, θθ 0 VV hjj (θθ 0 ad ( where reresets a table with elemets,,, umerator with the costrait that the ull hyothesis is true. V h j θ 0 is the estimated variace of the score 54-7

18 Two Proortios Select the Probability Distributio The robability distributio a ucoditioal test based o the score statistic is f ( ( (, The robability distributio of a coditioal test based o the score statistic is f ψ ( ψ ψ Γ Calculate the Sigificace Level The sigificace level (rejectio robability is foud by summig the robabilities of all tables that for which the comuted test statistic is at least as favorable to the alterative hyothesis as is the observed table. This may be writte as where ( D( y D( I, is a idicator fuctio. ( y f ( I ( D( y, D( Maimie the Sigificace Level The fial ste is to fid the maimum value (suremum of the sigificace level over all ossible values of the uisace arameter. This may be writte as, su 0 < I ( f ( su <, ( D( y, D( Note that the choice of either or as the uisace arameter is arbitrary. Fisher, Pearso, ad Likelihood Ratio Coditioal Eact Test of the Differece 0 Here, there are three coditioal eact tests for testig whether the differece is ero. The most famous of these uses Fisher s statistic, but similar tests are also available usig Pearso s statistic ad the likelihood ratio statistic. Null Hyothesis: P P 0 Hyothesis Tyes: Referece Set: Both oe-sided ad two-sided Fisher s Test Statistic: D( l f ( Γ Pearso s Test Statistic: D ( D ( l(. N 3/ m m 5 ( m / N ij m i j i j l i j / N L.R. Test Statistic: ( ij i j mi j / N Two-Sided Test: I ( D( y, D( D( D( y ij 54-8

19 Two Proortios Lower Oe-Sided Test: I ( D( y, D( D( D( y Uer Oe-Sided Test: I ( D( y, D( D( D( y Barard s Ucoditioal Eact Test of the Differece 0 Barard (947 roosed a ucoditioal eact test for the differece betwee two roortios. It is iterestig that two years later he retracted his article. However, the test has bee adoted i site of his retractio. Here are the details of this test: P Null Hyothesis: 0 Hyothesis Tyes: P Both oe-sided ad two-sided Referece Set: Ω. Test Statistic: ( D where ( Two-Sided Test: I ( D( y, D( D( D( y Lower Oe-Sided Test: I ( D( y, D( D( D( y Uer Oe-Sided Test: I ( D( y, D( D( D( y y y Hyotheses for Iequality Tests of Proortio Ratio Oe should determie i advace the directio of the ull ad alterative hyothesis of the test. Two-Sided HH 0 : PP /PP vvvv. HH aa : PP /PP Oe-Sided (Lower HH 0 : PP /PP vvss. HH aa : PP /PP < Oe-Sided (Uer HH 0 : PP /PP vvvv. HH aa : PP /PP > Large-Samle (Asymtotic Iequality Tests of Proortio Ratio Wald Z-Test (Oe- ad Two-Sided The Wald -test for the ratio give is idetical to the -test for the differece, which may ot ecessarily be a recommeded rocedure for ratios. The statistic for the -test is comuted as follows ( Small-Samle (Eact Iequality Tests of Proortio Ratio Barard s Eact Test of the Ratio (Oe- ad Two-Sided Barard s eact test for the ratio is idetical to that for the differece. 54-9

20 Two Proortios Hyotheses for Iequality Tests of Proortio Odds Ratio Oe should determie i advace the directio of the ull ad alterative hyothesis of the test. Two-Sided HH 0 : OO /OO vvvv. HH aa : OO /OO Oe-Sided (Lower HH 0 : OO /OO vvvv. HH aa : OO /OO < Oe-Sided (Uer HH 0 : OO /OO vvvv. HH aa : OO /OO > Large-Samle (Asymtotic Iequality Tests of Proortio Odds Ratio Log Odds Ratio Test (Oe- ad Two-Sided The statistic for the log odds ratio -test is comuted as follows log /( /( Matel-Haesel Test (Oe- ad Two-Sided The statistic for the Matel-Haesel -test is comuted as follows where log /( /( AAAA AAAA BBBB CC CCCC BBBB DD AA BB CC DD Small-Samle (Eact Iequality Test of Proortio Odds Ratio Eact Test of the Odds Ratio (Oe- ad Two-Sided The eact test here follows the rocedure described i Sahai ad Khurshid (995 begiig o age 37. Geeral Form of the Other Eact Tests i NCSS All of the eact tests follow the same atter. We will reset the geeral rocedure here, ad the give the secifics for each test. 54-0

21 Two Proortios Secify the Null ad Alterative Hyotheses The first ste is to select a method to comare the roortios ad determie if the test is to be oe-, or two-, sided. These may be writte i geeral as H 0 h j, ( P P 0 : θ H h j, ( P P 0 : θ where (for two-sided tests could be relaced with < or > for a oe-sided test ad the ide j is defied as ( P P P. ( P, P P P P / ( ( P P, P P /( P h, P h h / 3 Secify the Referece Set The et ste is to secify the referece set of ossible tables to comare the observed table agaist. Two referece sets are usually cosidered. Defie Ω as the comlete set of tables that are ossible by selectig observatios from oe grou ad observatios from aother grou. Defie Γ as the subset from Ω for which m. Tests usig Ω are ucoditioal tests while tests usig Γ are coditioal tests. Secify the Test Statistic The et ste is to select the test statistic. I most cases, the score statistic is used which has the geeral form DD( h jj(, θθ 0 where reresets a table with elemets,,, umerator with the costrait that the ull hyothesis is true. VV hjj (θθ 0 ad ( Select the Probability Distributio The robability distributio a ucoditioal test based o the score statistic is f ( ( (, The robability distributio of a coditioal test based o the score statistic is f ψ ( ψ ψ Γ V h j θ 0 is the estimated variace of the score 54-

22 Two Proortios Calculate the Sigificace Level The sigificace level (rejectio robability is foud by summig the robabilities of all tables that for which the comuted test statistic is at least as favorable to the alterative hyothesis as is the observed table. This may be writte as where ( D( y D( I, is a idicator fuctio. ( y f ( I ( D( y, D( Maimie the Sigificace Level The fial ste is to fid the maimum value (suremum of the sigificace level over all ossible values of the uisace arameter. This may be writte as, su 0 < I ( f ( su <, ( D( y, D( Note that the choice of either or as the uisace arameter is arbitrary. Data Structure This rocedure ca summarie data from a database or summaried cout values ca be etered directly ito the rocedure ael i oe of two ways: grou samle sies ad grou successes, or grou successes ad grou o-successes. Procedure Otios This sectio describes the otios available i this rocedure. Data Tab The data values ca be etered directly o the ael as cout totals or tabulated from colums of a database. Tye of Data Iut Choose from amog three ossible ways of eterig the data. Summary Table of Couts: Eter Row Totals ad First Colum I this sceario, the grou samle sie is etered followed by the umber of evets for each grou. Grou Samle Sie Evet The label Evet is used here, but might istead be Success, Attribute of Iterest, Positive Resose, Disease, or somethig else, deedig o the sceario. 54-

23 Two Proortios Summary Table of Couts: Eter the Idividual Cells For this selectio, each of the four resose couts is etered directly. Grou Evet No-Evet The labels Evet ad No-Evet are used here. Alteratives might istead be Success ad Failure, Attribute ad No Attribute, Positive ad Negative, Yes ad No, Disease ad No Disease, or 0, or somethig else, deedig o the sceario. Tabulate Couts from Database: Select Two Categorical Variables Use this otio whe you have raw data that must be tabulated. You will be asked to select two colums o the database, oe cotaiig the grou values (such as ad or Treatmet ad Cotrol ad a secod variable cotaiig the outcome values (such as 0 ad or No ad Yes. The data i these colums will be read ad summaried. Headigs ad Labels (Used for Summary Tables Headig Eter headigs for the grou ad outcome variables. These headigs will be used o the reorts. They should be ket short so the reort ca be formatted correctly. Labels Eter labels for the first ad secod grous ad the first ad secod outcomes. These labels will be used o the reorts. They should be ket short so the reort ca be formatted correctly. Couts (Eter Row Totals ad First Colum of Table Total Couts Eter the couts (samle sies of the two grous. Sice these are couts, they must be a o-egative umbers. Each must be greater tha or equal to the first colum cout to the right. Usually, they will be itegers, but this is ot required. First Colum Couts Eter the evet-couts of the two grous. Sice these are couts, they must be a o-egative umbers. Each must be greater tha or equal to the total cout to the left. Usually, they will be itegers, but this is ot required. Couts (Eter the Idividual Cells Couts Eter the couts i each of the four cells of the -by- table. Sice these are couts, they must be a o-egative umbers. Usually, they will be itegers, but this is ot required. Database Iut Grou Variable(s Secify oe or more categorical variables used to defie the grous. If more tha oe variable is secified, a searate aalysis is erformed for each. 54-3

24 Two Proortios This rocedure aalyes two grous. If the grou variable cotais more tha two uique values, a searate aalysis is created for each air of values. Sortig The values i each variable are sorted alha-umerically. The first value after sortig becomes grou oe ad the et value becomes grou two. If you wat the values to be aalyed i a differet order, secify a custom Value Order for the colum usig the Colum Ifo Table o the Data Widow. Outcome Variable(s Secify oe or more categorical variables used to defie the outcomes. If more tha oe variable is secified, a searate aalysis is erformed for each. This rocedure aalyes two outcomes. If the outcome variable cotais more tha two uique values, a searate aalysis is created for each air of values. Sortig The values i each variable are sorted alha-umerically. The first value after sortig becomes outcome oe ad the et value becomes outcome two. If you wat the values to be aalyed i a differet order, secify a custom Value Order for the colum usig the Colum Ifo Table o the Data Widow. Frequecy Variable Secify a otioal colum cotaiig the umber of observatios (cases rereseted by each row. If this otio is left blak, each row of the dataset is assumed to rereset oe observatio. Break Variables Eter u to five categorical break variables. The values i these variables are used to break the outut u ito searate reorts ad lots. A searate set of reorts is geerated for each uique value (or uique combiatio of values if multile break variables are secified. Zero Cout Adjustmet Add a small adjustmet value for ero couts Whe ero couts are reset, calculatio roblems for some formulas may result. Check this bo to secify how you wish to add a small value either to all cells, or to all cells with ero couts. Addig a small value to cells is cotroversial, but may be ecessary for obtaiig results. Zero Cout Adjustmet Method Zero cell couts cause may calculatio roblems with ratios ad odds ratios. To comesate for this, a small value (called the Zero Adjustmet Value may be added either to all cells or to all cells with ero couts. This otio secifies whether you wat to use the adjustmet ad which tye of adjustmet you wat to use. Zero Cout Adjustmet Value Zero cell couts cause may calculatio roblems. To comesate for this, a small value may be added either to all cells or to all ero cells. The Zero Cout Adjustmet Value is the amout that is added. Addig a small value is cotroversial, but may be ecessary. Some statisticias recommed addig 0.5 while others recommed 0.5. We have foud that addig values as small as may also work well. 54-4

25 Bootstra & Eact Tab Two Proortios Bootstra Cofidece Iterval Otios Bootstra Samles This is the umber of bootstra samles used. A geeral rule of thumb is that you use at least 00 whe stadard errors are your focus or at least 000 whe cofidece itervals are your focus. If comutig time is available, it does ot hurt to do 4000 or We recommed settig this value to at least C.I. Method This otio secifies the method used to calculate the bootstra cofidece itervals. The reflectio method is recommeded. Percetile The cofidece limits are the corresodig ercetiles of the bootstra values. Reflectio The cofidece limits are formed by reflectig the ercetile limits. If X0 is the origial value of the arameter estimate ad XL ad XU are the ercetile cofidece limits, the Reflectio iterval is ( X0 - XU, X0 - XL. Retries If the results from a bootstra samle caot be calculated, the samle is discarded ad a ew samle is draw i its lace. This arameter is the umber of times that a ew samle is draw before the algorithm is termiated. We recommed settig the arameter to at least 50. Percetile Tye The method used to create the ercetiles whe formig bootstra cofidece limits. You ca read more about the various tyes of ercetiles i the Descritive Statistics chater. We suggest you use the Ave X([] otio. Radom Number Seed Use this otio to secify the seed value of the radom umber geerator. Secify a umber betwee ad 3000 to seed (start the radom umber geerator. This seed will be used to start the radom umber geerator, so you will obtai the same results wheever it is used. If you wat to have a radom start, eter the hrase 'RANDOM SEED'. Eact Test ad Eact Cofidece Iterval Otios Maimum N Secify the maimum allowable value of N N N for eact hyothesis tests. Whe N is greater tha this amout, the "eact" results are ot calculated. Because of the (sometimes rohibitively log ruig time eeded for eact calculatios with larger samle sies (N > 50, this otio allows you to set a ca for N for such tests. Fortuately, the results of may of the asymtotic (o-eact tests are very close to the eact test results for larger samle sies. Number of Search Itervals Secify the umber of itervals to be used i the grid searches used i the eact tests ad eact cofidece itervals. Usually, 40 will obtai aswers correct to three laces. For tables with large N, you may wat to reduce this to 0 because of the legthy comutatio time. 54-5

26 Summary Reorts Tab Two Proortios Test Alha ad Cofidece Level Alha for Tests Eter the value of alha to be used for all hyothesis tests i this rocedure. The robability level (-value is comared to alha to determie whether to reject the ull hyothesis. Cofidece Level This is the cofidece level for all cofidece iterval reorts selected. The cofidece level reflects the ercet of the times that the cofidece itervals would cotai the true roortio differece if may samles were take. Tyical cofidece levels are 90%, 95%, ad 99%, with 95% beig the most commo. Data Summary Reorts Use these check boes to secify which summary reorts are desired. Differece Reorts Tab Cofidece Itervals of the Differece (P P Use these check boes to secify which cofidece itervals are desired. Iequality Tests of the Differece (P P Use these check boes to secify which tests are desired. Test Directio Use these dro-dows to secify the directio of the test. For o-iferiority ad sueriority tests, the determiatio of whether higher roortios are better or lower roortios are better imlicitly defies the directio of the test. Ratio Reorts Tab Cofidece Itervals of the Ratio (P/P Use these check boes to secify which cofidece itervals are desired. Iequality Tests of the Ratio (P/P Use these check boes to secify which tests are desired. Test Directio Use these dro-dows to secify the directio of the test. 54-6

27 Odds Ratio Reorts Tab Two Proortios Cofidece Itervals of the Odds Ratio (O/O Use these check boes to secify which cofidece itervals are desired. Iequality Tests of the Odds Ratio (O/O Use these check boes to secify which tests are desired. Test Directio Use these dro-dows to secify the directio of the test. Reort Otios Tab Reort Otios These otios oly aly whe the Tye of Data Iut otio o the Data tab is set to Tabulate Couts from Database. Variable Names This otio lets you select whether to dislay oly variable ames, variable labels, or both. Value Labels This otio lets you select whether to dislay data values, value labels, or both. Use this otio if you wat the outut to automatically attach labels to the values (like Yes, No, etc.. See the sectio o secifyig Value Labels elsewhere i this maual. Reort Decimal Places Couts Percetages These otios secify the umber of decimal laces to be dislayed whe the data of that tye is dislayed o the outut. This is the umber of digits to the right of the decimal lace to dislay for each tye of value. If oe of the Auto otios is used, the edig ero digits are ot show. For eamle, if Auto (U to 7 is chose, is dislayed as 0.05 ad is dislayed as The outut formattig system is ot desiged to accommodate Auto (U to 3, ad if chose, this will likely lead to lies that ru o to a secod lie. This otio is icluded, however, for the rare case whe a very large umber of decimals is desired. Table Formattig These otios oly aly whe Idividual Tables or Combied Tables are selected o the Summary Reorts tab. Colum Justificatio Secify whether data colums i the cotigecy tables will be left or right justified. 54-7

28 Two Proortios Colum Widths Secify how the widths of colums i the cotigecy tables will be determied. The otios are Autosie to Miimum Widths Each data colum is idividually resied to the smallest width required to dislay the data i the colum. This usually results i colums with differet widths. This otio roduces the most comact table ossible, dislayig the most data er age. Autosie to Equal Miimum Width The smallest width of each data colum is calculated ad the all colums are resied to the width of the widest colum. This results i the most comact table ossible where all data colums have the same width. This is the default settig. Custom (User-Secified Secify the widths (i iches of the colums directly istead of havig the software calculate them for you. Custom Widths (Sigle Value or List Eter oe or more values for the widths (i iches of colums i the cotigecy tables. This otio is oly dislayed if Colum Widths is set to Custom (User-Secified. Sigle Value If you eter a sigle value, that value will be used as the width for all data colums i the table. List of Values Eter a list of values searated by saces corresodig to the widths of each colum. The first value is used for the width of the first data colum, the secod for the width of the secod data colum, ad so forth. Etra values will be igored. If you eter fewer values tha the umber of colums, the last value i your list will be used for the remaiig colums. Tye the word Autosie for ay colum to cause the rogram to calculate it's width for you. For eamle, eter Autosie 0.7 to make colum be ich wide, colum be sied by the rogram, ad colum 3 be 0.7 iches wide. Plots Tab The otios o this ael allow you to select ad cotrol the aearace of the lots outut by this rocedure. Select ad Format Plots To dislay a lot for a table statistic, check the corresodig checkbo. The lots to choose from are: Couts Row Percetages Colum Percetages Table Percetages Click the aroriate lot format butto to chage the corresodig lot dislay settigs. Show Break as Title Secify whether to dislay the values of the break variables as the secod title lie o the lots. 54-8

29 Two Proortios Eamle Large-Samle Aalysis of the Differece of Two Proortios This sectio resets a eamle of a stadard, large-samle aalysis of the differece betwee two roortios. I this eamle, 3 of 66 receivig the stadard treatmet resoded ositively ad 995 of 378 receivig the eerimetal treatmet resoded ositively. You may follow alog here by makig the aroriate etries or load the comleted temlate Eamle by clickig o Oe Eamle Temlate from the File meu of the Two Proortios widow. Oe the Two Proortios rocedure. Usig the Aalysis meu or the Procedure Navigator, fid ad select the Two Proortios rocedure. O the meus, select File, the New Temlate. This will fill the rocedure with the default temlate. Secify the Data. Select the Data tab. Set Tye of Data Iut to Summary Table of Couts: Eter Row Totals ad First Colum. I the Grou, Headig bo, eter Treatmet. I the Grou, Label of st Value bo, eter Eerimetal. I the Grou, Label of d Value bo, eter Stadard. I the Outcome, Headig bo, eter Resose. I the Outcome, Label of st Value bo, eter Positive. I the Outcome, Label of d Value bo, eter Negative. I the Eerimetal, Total bo, eter 378. I the Eerimetal, Positive bo, eter 995. I the Stadard, Total bo, eter 66. I the Stadard, Positive bo, eter 3. 3 Secify the Summary Reorts. Select the Summary Reorts tab. Check Couts ad Proortios. Check Proortios Aalysis. 4 Secify the Differece Reorts. Select the Differece Reorts tab. Check Wald Z with Cotiuity Correctio uder Cofidece Itervals of the Differece (P P. Check Wilso Score with Cotiuity Correctio uder Cofidece Itervals of the Differece (P P. Set Test Directio to Two-Sided. Check Wald Z uder Iequality Tests of the Differece (P P. 5 Ru the rocedure. From the Ru meu, select Ru Procedure. Alteratively, click the gree Ru butto. 54-9

30 Two Proortios Couts ad Proortios Sectios Couts ad Proortios Resose Treatmet Positive Negative Total Cout Cout Cout Proortio* Eerimetal Stadard *Proortio Positive / Total Proortios Aalysis Statistic Value Grou Evet Rate ( Grou Evet Rate ( Absolute Risk Differece Number Needed to Treat / Relative Risk Reductio - / 0.05 Relative Risk /.05 Odds Ratio o/o.9 These reorts documet the values that were iut, ad give various summaries of these values. Cofidece Iterval Cofidece Itervals of the Differece (P - P Cofidece Lower 95% Uer 95% Cofidece Iterval Differece C.L. of C.L. of Iterval Name - P - P P - P Width Wald Z c.c Wilso Score c.c This reort rovides two, large samle cofidece itervals of the differece based o formulas show earlier i this chater. I this case, they are early idetical. The Wilso Score with cotiuity correctio has bee show to be oe of the best. The Wald Z (or Simle Z cofidece iterval is ofte show i elemetary statistics books. Iequality Test Two-Sided Tests of the Differece (P - P H0: P P vs. Ha: P P Test Test Reject Statistic Differece Statistic Prob H0 at Name - Value Level α 0.05? Wald Z Yes This reort rovides the Wald Z large-samle test. The -value of the test is the Prob Level

31 Plots Two Proortios These bar charts show the cout ad row ercetages of the data. 54-3