An accurate test for homogeneity of odds ratios based on Cochran s Q-statistic

Transcription

1 Kulnskaya and Dollnger BMC Medcal Research Methodology (2015) 15:49 DOI /s x TECHNICAL ADVANCE Open Access An accurate test for homogenety of odds ratos based on Cochran s Q-statstc Elena Kulnskaya 1* and Mchael B Dollnger 2 Abstract Background: A frequently used statstc for testng homogenety n a meta-analyss of K ndependent studes s Cochran s Q. For a standard test of homogenety the Q statstc s referred to a ch-square dstrbuton wth K 1 degrees of freedom. For the stuaton n whch the effects of the studes are logarthms of odds ratos, the ch-square dstrbuton s much too conservatve for moderate sze studes, although t may be asymptotcally correct as the ndvdual studes become large. Methods: Usng a mxture of theoretcal results and smulatons, we provde formulas to estmate the shape and scale parameters of a gamma dstrbuton to ft the dstrbuton of Q. Results: Smulaton studes show that the gamma dstrbuton s a good approxmaton to the dstrbuton for Q. Conclusons: Use of the gamma dstrbuton nstead of the ch-square dstrbuton for Q should elmnate naccurate nferences n assessng homogenety n a meta-analyss. (A computer program for mplementng ths test s provded.) Ths hypothess test s compettve wth the Breslow-Day test both n accuracy of level and n power. Keywords: Meta-analyss, 2 2 tables, Heterogenety test, Interacton test, Fxed effect model, Random effects model Background The combnaton of the results of several smlar studes has many applcatons n statstcal practce, notably n the meta-analyss of medcal and socal scence studes and also n mult-center medcal trals. An mportant frst step n such a combnaton s to decde whether the several studes are suffcently smlar. Ths decson s often accomplshed va a so-called test of homogenety. The outcomes of the studes may be expressed n a varety of effect measures, such as: sample means; odds ratos, relatve rsks or rsk dfferences arsng from 2 2tables; standardzed mean dfferences of two arms of the studes; and many more. A varety of statstcs for use n tests of homogenety have been proposed; some are specfc to the type of effect measure, and some are applcable to several measures. Ths paper has ts man focus on the test statstc frst ntroduced by Cochran [1] and [2] and ts applcaton to testng homogenety when the effects of nterest are *Correspondence: e.kulnskaya@uea.ac.uk 1 School of Computng Scences, Unversty of East Angla, NR4 7TJ Norwch, UK Full lst of author nformaton s avalable at the end of the artcle odds ratos arsng from experments wth dchotomous outcomes n treatment and control arms. Cochran s Q statstc s defned by Q = ŵ( θ θ w ) 2 where θ s the effect estmator of the th study, θ w = ŵ θ / ŵ s the weghted average of the estmators of the effects, and the weght ŵ s the nverse of the varance estmator of th effect estmator. The use of nverse varance weghts has the appealng feature of weghtng larger and more accurate studes more heavly n the weghted mean θ w and n the statstc Q. Ths statstc was nvestgated for the case that the study effects are normally dstrbuted sample means by Cochran and also by Welch [3] and James [4]. Perhaps the frst applcaton of the Q statstc to testng homogenety of the logarthm of odds ratos s due to Woolf n 1955 [5]. DerSmonan and Lard [6] extended the use of Q for studes wth bnomal outcomes to dfference of proportons as well as to log odds ratos n the context of the random effects model n whch the studes are assumed to be sampled from a hypothetcal populaton of potental studes. However, the use of Q n a test of homogenety s the same whether a random effects or fxed effects model s used Kulnskaya and Dollnger. Open Access Ths artcle s dstrbuted under the terms of the Creatve Commons Attrbuton 4.0 Internatonal Lcense ( whch permts unrestrcted use, dstrbuton, and reproducton n any medum, provded you gve approprate credt to the orgnal author(s) and the source, provde a lnk to the Creatve Commons lcense, and ndcate f changes were made. The Creatve Commons Publc Doman Dedcaton waver ( apples to the data made avalable n ths artcle, unless otherwse stated.

2 Kulnskaya and Dollnger BMC Medcal Research Methodology (2015) 15:49 Page 2 of 19 Under farly general condtons, n the absence of heterogenety, Q wll follow asymptotcally (as the ndvdual studes become large) the ch-square dstrbuton wth K 1 degrees of freedom where K s the number of studes. It s common practce to assume that Q has ths null dstrbuton, regardless of the szes of the ndvdual studes or the effect measure. But ths null dstrbuton s naccurate (except asymptotcally), and ts use causes nferences based on Q to be naccurate. Ths concluson of naccuracy should also apply to nferences based on any statstcs whch are derved from Q, suchasthei 2 statstc (see [7] and [8]). Lttle s known of a theoretcal nature about the null dstrbuton of Q under non-asymptotc condtons. In our prevous work, together wth Bjørkestøl, we have provded mproved approxmatons to the null dstrbuton of Q when the effect measure of nterest s the standardzed mean dfference [9] and the rsk dfference [10]. In ths paper we use a combnaton of theoretcal and smulaton results to estmate the mean and varance of Q when the effects are logarthms of odds ratos. We use these estmated moments to approxmate the null dstrbuton of Q by a gamma dstrbuton and then apply that dstrbuton n a homogenety test based on Q (to be denoted Q γ ) that s substantally more accurate than the use of the ch-square dstrbuton. We also compare the accuracy and power of ths test wth those of other homogenety tests, such as that of Breslow and Day [11]. Brefly, both the accuracy and the power of our test are comparable to those of the Breslow-Day test (see Sectons Accuracy of the level of the homogenety test and Power of the homogenety test). After ntroducng notaton and the man assumptons n Secton Notaton and assumptons, we proceed to our study of the moments of Q for log odds ratos n Secton ThemeanandvaranceofQ and to ther estmaton n Secton Estmatng the moments and dstrbuton of Q LOR. Results of our smulatons of the acheved level and power of the standard Q test, the Breslow-Day test and the proposed mproved test of homogenety based on Q γ are gven n Sectons Accuracy of the level of the homogenety test and Power of the homogenety test. Secton Example: a meta-analyss of Stead et al. (2013) contans an example from the medcal lterature to llustrate our results and to compare them to other tests. Secton Conclusons contans a dscusson and summary of our conclusons. We provde nformaton on the desgn of our smulatons n the Appendx; and more results of the smulatons for varous sample szes, ncludng unbalanced desgns and unequal effects, are contaned n the accompanyng Further Appendces, together wth addtonal nformaton about the dervaton of our procedures. Our R program for calculaton of the Q γ test of homogenety can be downloaded from the Journal webste. Methods Notaton and assumptons We assume that there are K studes each wth two arms, whch we call treatment and control and use the subscrpts T and C. The szes of the arms of the th study are n T and n C ;letn = n T + n C and let q = n C /N.Data n the arms have bnomal dstrbutons wth probabltes p T and p C. The effect of nterest s the logarthm of the odds rato θ = log[ p T /(1 p T )] log[ p C /(1 p C )]. The null hypothess to be tested s the equalty of the odds ratos (or equvalently ther logarthms) across the several studes,.e., θ 1 = =θ K := θ. To estmate θ, we follow Gart, Pettgrew and Thomas [12] who showed that f x successes occur from the bnomal dstrbuton Bn(n; p), then among the estmators of log[ p/(1 p)]gvenbyl a (x) = log[ (x + a)/(n x + a)], the estmator wth a = 1/2 has mnmum asymptotc bas; and ndeed, ths s the only choce of a for whch all terms for the bas n the expanson of L a (x) havng order O(1/n) vansh. Gart et al. [12] also show that Var[ L 1/2 ] = 1 (1 + 2p)2 + np(1 p) 2n 2 p 2 (1 p) 2 + O ( 1/n 3) (1) and suggest the use of the followng unbased estmator of the varance: (x + 1/2) 1 + (n x + 1/2) 1. Accordngly, f x and y are the number of successes n the treatment and control arms of the th study, we estmate θ by θ = L 1/2 (x ) L 1/2 (y ). We estmate the varance of θ by Var[ θ ] = 1 x + 1/2 + 1 n T x + 1/2 + 1 y + 1/2 1 + n C y + 1/2. Aweghtw s assgned to the thstudyasthenverse of the varance of θ,andtheweghtsestmatedbyŵ = Var[ θ ] 1. The weghted average of the log odds rato effects s gven by θ w = ŵ θ / ŵ. Then Cochran s Q statstc s defned as the weghted sum of the squared devatons of the ndvdual effects from the average; that s, Q = (2) K ŵ ( θ θ w ) 2. (3) =1 The standard verson of the Q statstc, denoted Q stand does not add 1/2 to the number of events n both arms when calculatng log-odds unless ths s requred to defne ther varances. The dstrbuton of Q under the null hypothess of equalty of the effects θ depends on the value of the common effect θ, the number of studes K and the sample szes n T and n C. However addtonal nformaton s needed to specfy a unque dstrbuton for Q. For example, the

3 Kulnskaya and Dollnger BMC Medcal Research Methodology (2015) 15:49 Page 3 of 19 common effect θ = 0 (that s, the probabltes for the treatment and control arms are equal), could arse wth all probabltes equal to 1/2 (n both arms of all studes) or wth some of the studes havng probabltes of 1/4 n both arms and others havng probabltes of 1/3 n both arms. To unquely specfy a dstrbuton for Q, we need to ntroduce a nusance parameter ζ for each study. It s convenent to take ζ = log[ p C /(1 p C )]tobethelog odds for the control arm of the th study and to estmate t as descrbed above,.e., ζ = L 1/2 (y ). The mean and varance of Q The Q statstc has long been known to behave asymptotcally, as the sample szes become large, as a ch-square dstrbuted random varable wth mean K 1andvarance, whch s necessarly twce the mean, 2(K 1). However, the choce of effect (e.g., log odds rato, sample mean, standardzed mean dfference) has a substantal mpact on the dstrbuton of Q for small to moderate sample szes, whch n turn affects the use of Q as a statstc for a test of homogenety. For ths secton, we shall use the notaton Q SM for Q when the effect s a normally dstrbuted sample mean and Q LOR when the effect s the logarthm of the odds rato. Assumng that the data from the studes are dstrbuted N(μ, σ 2 ), Welch [3] and James [4] frst studed the moments of Q SM under the null hypothess of homogenety; usng the normalty propertes, they calculated asymptotc expansons for the mean and varance of Q SM, and Welch matched these moments to those of a re-scaled F-dstrbuton to create a homogenety test now known as the Welch test. It s useful, for comparson wth Q LOR, to examne Welch s mean and varance for Q SM.Omttng terms of order 1/n 2 and smaller, Welch found E[ Q SM ] = (K 1) / ( Wσ 2 ) n 1 Var[ Q SM ] = 2(K 1) / ( Wσ 2 ) n 1 where W s the sum of the theoretcal weghts n /σ 2. Notce the followng facts about these moments. 1) They converge to the ch-square moments as the sample szes ncrease. 2) Both moments are larger than the correspondng ch-square moments. We shall call the dfference between the moments of Q and the correspondng chsquare moments: correctons. 3) The varance s more than twce the mean. 4) The moments depend on the nusance parameters σ 2, whch are estmated ndependently of the effects of nterest (the sample means). Based on a combnaton of theoretcal expansons and extensve smulatons, we have determned that, when the effect enterng nto the defnton of Q s the log odds rato, (4) (5) the mean and varance of Q LOR (under the null hypothess of equal odds ratos) have the followng propertes. 1) They each converge to the correspondng ch-square moments of K 1and2(K 1) as the sample szes ncrease. 2) Both moments are less than the correspondng ch-square moments. That s, the correctons are negatve rather than postve as for Q SM.3)Thevarance s not only less than the ch-square varance, t s less than twce the mean. 4) The moments depend on nusance parameters, whch are not ndependent of the effects. The two plots of Fgure 1 show the relaton of the varance of Q LOR to ts mean for a representatve set of smulatons. (See Appendx A for a complete descrpton of the smulatons conducted). The two plots have dentcal data, but the ponts are colored accordng to the value of N n the left plot and accordng to the value of K n the rght plot. The mean and varance of Q LOR have been dvded by K 1 n order to place the data on the same scale. The man message of the rght plot (and a key fndng of our smulatons) s that ths re-scalng s effectve the dfferent values of K (5, 10, 20 and 40) are farly unformly dstrbuted throughout the plot, ndcatng that after ths re-scalng the moments of Q LOR have lttle dependence on the number of studes. In the plots, we see that the mean of Q LOR s less than K 1, that the varance of Q LOR s less than 2(K 1),and that the varance s less than twce the mean. We also see n the left plot that the departure of the mean and varance from the ch-square values of K 1and2(K 1) (that s, the correctons ) are greater for the study sze N = 90 (.e., 45 n each arm) than for the study sze N = 150. It s not evdent from the graphs, but the correctons needed are also greater when the bnomal probabltes p T and p C are more dstant from the central value of 1/2. Estmatng the moments and dstrbuton of Q LOR In ths secton, we outlne a method for estmatng the mean and varance of Q LOR. The method nvolves farly complcated formulas, but n the Appendx we provde more detals and a lnk to a program n R for carryng out the calculatons. Kulnskaya et al. [10] presented a very general expanson for the mean of Q for arbtrary effect measures n terms of the frst four central moments of the effect and nusance parameters as well as the weght functon expressed n terms of these parameters. Necessary formulas for the applcaton of ths expanson to the frst moment of Q LOR can be found n Appendx B.2. The resultng expanson provdes an approxmaton to the mean of Q LOR, whch we wll denote E th [ Q LOR ]where the subscrpt th ndcates that ths expectaton s entrely theoretcal. It depends on the number of studes K, the sample szes of the separate arms of the studes, the estmated values of the nusance parameters ζ, the values of

4 Kulnskaya and Dollnger BMC Medcal Research Methodology (2015) 15:49 Page 4 of 19 a b Fgure 1 Varance vs mean of Q. Ths scatter plot of Var[ Q] /(K 1) vs. E[ Q] /(K 1) contans the results of smulatons of the moments of Q LOR for the 144 confguratons of parameters: K = 5, 10, 20, 40; N = 90, 150, dvded equally nto the two arms; log odds ratos: 0, 0.5, 1, 1.5, 2, 3; and control probabltes: 0.1, 0.2, 0.4. The studes n each smulaton all have the same parameters. The smulatons for each confguraton were replcated 10,000 tmes. The grey reference lne (Var[ Q] = 2E[ Q]) ndcates the relaton that would be expected f Q followed a ch-square dstrbuton. (a): N = 90 black and N = 150 red. (b): K = 5 (black), K = 10 (red), K = 20 (blue) and K = 40 (green). The black curve corresponds to the ftted quadratc equaton Var[ Q LOR ] /(K 1) = E [Q LOR] /(K 1) [ E [Q LOR] /(K 1) ] 2. the estmated weghts and the estmated value of the effect θ under the null hypothess. When we compared E th [ Q LOR ] wth the smulated values for the mean of Q LOR, we found that t does an excellent job of dentfyng the stuatons where correctons are needed to the ch-square moment, but that t over-estmates the sze of the correcton by a constant percentage of slghtly more than 1/3 (R 2 = 97.0%). More precsely, denotng the mean of Q LOR by E[ Q LOR ], we have the relaton (K 1) E[ Q LOR ] = 0.687[ (K 1) E th [ Q LOR ]]. (6) Although ths equaton s based partly on theoretcal calculatons and partly on the results of smulatons (the factor), we note that after decdng on the use of the factor we conducted new smulatons to verfy that t was not just a random consequence of the orgnal smulatons. More detals on our smulatons for ths formula can be found n Appendx B.1. Kulnskaya et al. [10] also deduced a very general theoretcal expanson for the second moment of Q, but when we appled ths expanson to Q LOR and compared t to our smulatons, we found that the expanson s much too naccurate to be of any use. We conjecture that ths

5 Kulnskaya and Dollnger BMC Medcal Research Methodology (2015) 15:49 Page 5 of 19 naccuracy s due to non-unform convergence of the expansons wth respect to both the number of studes K and the values of the bnomal parameters. Accordngly we have chosen to estmate the varance of Q LOR usng a quadratc regresson formula from our smulatons, as seen n Fgure 1, but usng more complete data than shown n those plots. As n the regresson for the mean of Q LOR we ftted a formula for the varance and then checked t aganst addtonal smulatons (See the Appendx B.2 for more detals on our procedures). Our formula for estmatng Var[ Q LOR ]s Var[ Q LOR ] = 4.74(K 1) 12.17E[ Q LOR ] E[ Q LOR ] 2 /(K 1) The quadratc regresson ft, usng 487 of our more than 1400 smulatons, had an R 2 value of 98.5%. In usng ths equaton, we frst need to calculate E[ Q LOR ]usng Equaton 6. Ths quadratc regresson s depcted by the black curve on the plot (b) of Fgure 1. Although we do not have a theoretcal justfcaton for usng a quadratc relaton between the mean and varance of Q, such a functonal relaton between the mean and the varance of Q s often found under varous condtons. For examples, n the asymptotc ch-square dstrbuton of Q, the varance (twce the mean) s a lnear functon of the mean; and n the normally dstrbuted sample mean stuaton of Equatons (4) and (5), a lttle algebra shows that agan the varance s a lnear functon of the mean. Further, n a common one-way random effects model, Bggerstaff and Tweede [13] show that the varance of Q s a quadratc functon of the mean. Our smulatons show that the famly of gamma dstrbutons fts the dstrbuton of Q LOR qute well. By matchng the mean and varance of Q LOR wth the mean and varance of a gamma dstrbuton, we arrve at an approxmaton for the dstrbuton of Q LOR whch can be used to conduct a test of homogenety for the equalty of log odds ratos usng Q LOR as the test statstc. (The shape parameter α of the gamma dstrbuton s estmated by α = E[ Q LOR ] 2 /Var[ Q LOR ], and the scale parameter β s estmated by β = Var[ Q LOR ] /E[ Q LOR ].) The accuracy of ths test statstc and a comparson wth other test statstcs are dscussed n the next secton. Results and dscusson Accuracy of the level of the homogenety test In ths secton we present the results of extensve smulatons desgned to analyze the accuracy of the levels of the test of homogenety of log odds ratos usng the Q statstc together wth the gamma dstrbuton estmated from the data by the methods of Secton Estmatng the moments and dstrbuton of Q LOR. We denote ths test by Q γ. The use of smulatons to determne the accuracy (7) of varous dfferent tests of homogenety of log odds ratos has often been dscussed n the lterature. See, for example, Schmdt et al. [14], Bhaumk et al. [15], Bagher et al. [16], Lu and Chang [17], Gavaghan et al. [18], Res et al. [19], Paul and Donner [20,21], and Jones et al. [22]. Our smulatons ncluded comparsons wth some of the tests proposed by these authors. The comparsons of ours confrmed (as several of the above authors also dscovered) that the Breslow-Day [11] (denoted by BD) s often the best avalable among the prevously consdered tests. The Breslow-Day test for homogenety of odds-ratos s based on the statstc K XBD 2 = (x j X j ( ˆψ)) 2, Var(x j ˆψ) j=1 where x j, X j ( ˆψ) and Var(x j ˆψ) denote the observed number, the expected number and the asymptotc varance of the number of events n the treatment arm of the jth study gven the overall Mantel-Haenszel odds rato ˆψ, respectvely. Its dstrbuton s approxmated by the χ 2 dstrbuton wth K 1 degrees of freedom. We found that usng the Tarone [23] correcton to the Breslow- Day test had such small dfferences from BD that the two were vrtually equvalent. In addton to the BD and Tarone tests, we smulated proposals by Lu and Chang [17] for testng the homogenety of log odds ratos based on the normal approxmaton to the dstrbuton of the z-, square-root and log-transformed Q stand statstc. The logtransformaton was also suggested by Bhaumk et al. [15]. We do not report these results due to our concluson that none were superor to BD. Accordngly, n our comparatve graphs below, we compare our Q γ test wth BD and wth the commonly used test (denoted Q χ 2), whch uses the standard statstc Q stand (calculated wthout addng 1/2 to the numbers of events when calculatng log-odds) together wth the ch-square dstrbuton. Our smulatons for testng the null hypothess of equal odds ratos (all conducted subsequent to the adopton of the regressons of Equatons 6 and 7) are of two types. For the frst type, the parameters of all studes are dentcal; these smulatons nclude the followng parameters: number of studes K = 5, 10, 20 and 40; total study szes N = 90, 150, and 210; proporton of the study sze n the control arm q = 1/3, 1/2, 2/3; null hypothess value of the log odds rato θ = 0, 0.5, 1, 1.5, 2, and 3; and the log odds of the control arm ζ = 2.2 (p C = 0.1), 1.4 (p C = 0.2) and 0.4 (p C = 0.4). The second type of smulaton fxes the null hypothess values of equal log odds rato at θ = 0, 0.5, 1, 1.5, 2, and 3, but the ndvdual studes are qute heterogeneous concernng all other parameters. For example, for a null value of θ = 0.5 and K = 5 studes, one confguraton wth an average study sze of 150 has dfferent sample szes of 96, 108,

6 Kulnskaya and Dollnger BMC Medcal Research Methodology (2015) 15:49 Page 6 of , 120, 312, each dvded equally between the two arms (q = 1/2) and dfferent control arm probabltes p C of 0.15, 0.3, 0.45, 0.6, and 0.75; note that the condton θ = 0.5 when used wth the fve dfferent control arm probabltes then unquely specfes fve probabltes p T for the treatment arms. A complete descrpton of the heterogeneous smulatons can be found n Appendx A. When K = 5, 10 and 20, all smulatons were replcated 10,000 tmes and thus approxmate 95% confdence ntervals for the acheved levels are ±0.004; but when K = 40, the smulatons were replcated only 1,000 tmes, gvng approxmate 95% confdence ntervals for the levels of ± The frst panel of graphs (see Fgure 2) shows the acheved levels, at the nomnal level of 0.05, for the three tests plotted aganst the dfferent null values of θ n the range0to3undertheconfguratonnwhchallk studes have dentcal parameters and the study szes are N = 90 wth the subjects splt equally between the two arms (q = 1/2). The twelve graphs n the panel use K = 5, 10, 20 and 40; and p C = 0.1, 0.2, and 0.4. Note that the acheved levels for both BD and Q γ are almost always n the range 0.04 to 0.06, wth BD slghtly better for many stuatons, but wth Q γ occasonally slghtly better. The test Q χ 2 s almost always nferor; and when p C = 0.1, t s much too conservatve (not rejectng the null hypothess frequently enough); ndeed, when θ = 0, the acheved levels for Q χ 2 are less than In the four rght graphs, when p C = 0.4, we see that all three tests perform well when 0 θ 1.5; these parameters correspond to p T = 0.4, 0.52, 0.64 and We also note that n the farly extreme stuaton when θ = 3andp C = 0.4 (and hence p T = 0.93) the qualty of all the tests worsens, however BD performs best here and Q χ 2 performs very badly. These results for the test Q χ 2 are perhaps more easly understood when expressed n terms of the natural parameters, the bnomal probabltes p C and p T,rather than the log odds rato θ. WeseethatQ χ 2 s extremely conservatve whenever ether bnomal parameter s far from the central values of 0.5, but that ts performance s reasonable when the bnomal parameters are relatvely close to the central values of 0.5. Fgure 2 s representatve of a number of addtonal panels of graphs for equal study szes whch can be found n Appendx B.1, Fgures 9 and 10. There we have ncluded panels of graphs frst for balanced arms wth study szes of 150 and 210. These panels are qute smlar to the one presented n Fgure 2 except that all levels become closer to the nomnal level of 0.05 as the study sze ncreases from 90 to 150 to 210. Ths behavor s consstent wth the Fgure 2 Acheved levels for homogeneous studes, N = 90. Comparson of acheved levels, at the nomnal level of 0.05, for the three tests Q γ (sold lne), BD (dot-dash), and Q χ 2 (dash) plotted aganst the log odds rato θ. Here all studes have the same parameters: 90 subjects n each study wth equal arms of 45 each (N = 90 and q = 1/2).

7 Kulnskaya and Dollnger BMC Medcal Research Methodology (2015) 15:49 Page 7 of 19 known fact that the tests are asymptotcally correct as the study szes tend to. However, we note that even when N = 210, the test Q χ 2 s stll qute conservatve when p C = 0.1. Appendx B.1 contans two addtonal panels of graphs (Fgures 11 and 12) whch are analogous to the panel n Fgure2exceptthatthetwoarmsofeachstudyareunbalanced. In the frst of these, all studes have twce the number of subjects n the treatment arm (q = 1/3) and the second s reversed wth all studes havng twce the number of subjects n the control arm (q = 2/3). The results are smlar to those of Fgure 2 wth the followng modfed conclusons. When q = 1/3 andp c = 0.1, the Q χ 2 test s partcularly conservatve, rejectng the null hypothess less than 1% of the tme, ndependent of the number of studes K. Generally both the BD test and the Q γ tests are reasonably close to nomnal level, but the BD test s mostly (but not always) somewhat better than the Q γ test. When θ = 3, all tests experence a declne n accuracy, wth the BD test mostly superor. Fgure 3 s a typcal example showng the acheved levels for one set of confguratons n whch all the studes are dstnct. Here the studes are of average sze 150. When K = 5, the total study szes are 96, 108, 114, 120, 312; n selectng these szes, we have followed a suggeston of Sánchez-Meca and Marín-Martínez [24] who selected study szes havng the skewness 1.464, whch they consdered typcal for meta-analyses n behavoral and health scences. For a gven θ the fve studes had dfferent values for the control arm and treatment arm probabltes (see Appendx for detals). For K = 10, 20 and 40, the parameters for K = 5 were repeated 2, 4 and 8 tmes respectvely. We see that BD and Q γ are farly close n outcome wth acheved levels almost always between and 0.055, whle the levels for Q χ 2 mostly cluster around Note that the performance of Q χ 2 s somewhat better than seen n Fgure 2 for two reasons. Frst, the study szes are larger (average of 150 rather than all havng sze 90); and second, because the bnomal parameters vary among the dfferent studes, many of them are closer to the central values of 0.5 where we have seen that the performance of the Q χ 2 test mproves. It s worth notng that when we conducted smulatons for the average sample sze of 90 for the same scenaro (so that the sample szes were 36, 48, 54, 60, 252), we dscovered that the Breslow-Day test does not perform well Fgure 3 Acheved levels for heterogeneous studes, N = 150. Comparson of acheved levels, at the nomnal level of 0.05, for the three tests Q γ (sold lne), BD (dot-dash), and Q χ 2 (dash) plotted aganst the log odds rato θ for heterogeneous studes. Here the studes have average sze 150 dvded equally between arms, but the study szes and the bnomal parameters vary for each study. In the left graphs, the smallest control probabltes are pared wth the smallest study szes. In the rght graphs, the smallest control probabltes are pared wth the largest study szes.

8 Kulnskaya and Dollnger BMC Medcal Research Methodology (2015) 15:49 Page 8 of 19 and may even not be defned for large numbers of studes K duetothesparstyofthedata.thssthereasonthat, for comparatve purposes, we use larger sample szes n Fgure3thanusednFgure2. Power of the homogenety test Inthssectonwereportontheresultsfromour(lmted) smulatons of power of the three tests: the Q γ,bd and Q χ 2 tests. Power comparsons are not really approprate when the levels are naccurate and dffer across the tests. Unfortunately t s mpossble to equalze the levels or adjust for the dfferences. Nevertheless we consder power comparsons at a nomnal level of 0.05 to be mportant to nform the practce. We have performed smulatons only for the case of K dentcal studes wth balanced sample szes (q = 1/2). The values for the total study szes N, the number of studes K, control arm probabltes p C and the common log-odds rato θ were dentcal to those used n smulatng the levels for the dentcal studes gven n Secton Accuracy of the level of the homogenety test. For each combnaton of N, K, p C, θ, accordng to the random effects model of meta-analyss, we smulated K wthn-studes log odds ratos θ from the N(θ, τ 2 ) dstrbuton for the values of the heterogenety parameter τ from 0 to 0.9 n the ncrements of 0.1. Gven the values of p C and θ, we next calculated the probabltes n the treatment groups p T and smulated the numbers of the study outcomes from the bnomal dstrbutons Bn(n, p C ) and Bn(n, p T ) for = 1,, K. All smulatons were replcated 1000 tmes. The frst panel of graphs (see Fgure 4) shows the power for the three tests when θ = 0 plotted aganst the dfferent values of heterogenety parameter τ n the range 0 to 0.9 under the confguraton n whch all K studes have dentcal parameters, the study szes are N = 90 wth the subjects splt equally between the two arms (q = 1/2). The twelve graphs n the panel use K = 5, 10, 20 and 40; and p C = 0.1, 0.2, and 0.4. Note that the power for both BD and Q γ are almost always hgher than for Q χ 2, wth the dfference beng especally pronounced for p C = 0.1. The nferorty of Q χ 2 s due to ts conservatveness noted n the Secton Accuracy of the level of the homogenety test. There s no clear-cut wnner between the BD and the Q γ,wthbdslghtlybetter for some stuatons, but slghtly worse for others. In the three rght graphs, when p C = 0.4, we see that all three tests perform equally well. Fgure 4 Power when the log odds rato θ = 0. Comparson of power for the three tests Q γ (sold lne), BD (dot-dash), and Q χ 2 (dash) plotted aganst τ, the square root of the random varance component τ 2. Here all studes have the parameters: 90 subjects n each study wth equal arms of 45 each (N = 90 and q = 1/2) and the log odds rato θ = 0.

9 Kulnskaya and Dollnger BMC Medcal Research Methodology (2015) 15:49 Page 9 of 19 The second panel of graphs (see Fgure 5) shows the power for the three tests when θ = 3. The power of the Q χ 2 test s stll the lowest of the three tests. But here the power of the Q γ test appears to be somewhat hgher then for the BD when p C = 0.1, about the same when p C = 0.2, and notceably lower n the extreme stuaton when p C = 0.4. These dfferences n power between the BD and Q γ tests are both the consequences of the fact that the Q γ test s somewhat lberal for p C = 0.1 and somewhat conservatve for p C = 0.4, as can be seen from Fgure 2. The BD test s the closest to the nomnal level n these crcumstances. Example: a meta-analyss of Stead et al. (2013) Ths secton llustrates the theory of Sectons The mean and varance of Q and Estmatng the moments and dstrbuton of Q LOR and gves an ndcaton of the mprovement n accuracy of the homogenety test. The calculatons can be performed usng our computer program (Addtonal fles 1, 2 and 3). We use the data from the revew by Stead et al. [25] of clncal trals on the use of physcan advce for smokng cessaton. Comparson [25], p.65 consdered the subgroup of nterventons nvolvng only one vst. We use odds rato n our analyss below although relatve rsk was used n the orgnal revew. The frst verson of the revew was publshed n Update 2, publshed n 2004, ncluded 17 studes for ths comparson. Summary data and the results from the standard analyss of these 17 trals are found n Fgure 6, produced by the R package meta [26]. Note that meta does not add 1/2 tothe number of events n calculaton of the log-odds, and therefore calculates the standard statstc Q stand for the test of homogenety. The value of Cochran s Q statstc s The standard ch-square approxmaton wth 16 df yelds the p- value of for the test for homogenety. The estmated mean E th [ Q] of the null dstrbuton of Q s and the corrected mean usng Equaton 6 s E[ Q] = The estmated varance calculated from Equaton 7 s The parameters of the approxmatng gamma dstrbuton are α = 8.90 and β = The p-value usng ths gamma dstrbuton s The Breslow-Day statstc value s and the p-value s 0.051; the Tarone correcton provdes the same values to 4 decmal places. To evaluate the correctness of these p-values, we smulated one mllon values of Q from the fxed null dstrbuton wth each study havng the null value θ w = 1.58 for the odds rato Fgure 5 Power when the log odds rato θ = 3. Comparson of power for the three tests Q γ (sold lne), BD (dot-dash), and Q χ 2 (dash) plotted aganst τ, the square root of the random varance component τ 2. Here all studes have the parameters: 90 subjects n each study wth equal arms of 45 each (N = 90 and q = 1/2) and the log odds rato θ = 3.

10 Kulnskaya and Dollnger BMC Medcal Research Methodology (2015) 15:49 Page 10 of 19 Fgure 6 Forest plot of the meta-analyss by Stead et al. [25]. Forest plot of the meta-analyss by Stead et al. (2013) ncludng 17 pre-2004 studes only, produced by the R package meta [26]. together wth the orgnal ndvdual values for the control parameters p C. The concluson, based on the emprcal results, s that the p-value should be Thus for ths example, the gamma dstrbuton result s closest to that gven by the smulatons and the standard ch-square value s furthest. The most current verson of the revew (Update 4) contans only one more tral by Unrod (2007) for ths comparson. The values are event T = 28, event C = 18, n T = 237, n C = 228. Wth the addton of these data, the test of heterogenety results n Q = , and the p-value of s obtaned by the standard ch-square approxmaton wth 17 df. Our method results n E th [ Q] = 15.14, and the corrected value E[ Q] = 15.72, Var[ Q] = 26.22, wth the gamma dstrbuton parameters α = 9.43 and β = The p-value from the gamma approxmaton s The BD test statstc s and ts p-value s 0.071; the Tarone correcton, once more, results n the same values to 4 decmal places. Another set of one mllon smulatons from the null dstrbuton yelded the emprcal p-value of For the data n these two examples, the gamma approxmaton results n lower and more accurate p-values than the p-values of both the standard ch-square approxmaton and the Breslow-Day test. However, n our more extensve smulatons there were cases n whch the Breslow-Day test was superor. Note that ths example has farly low numbers of events (between 1% and 5% for many studes), whch, as mentoned at the end of Secton Accuracy of the level of the homogenety test, s a stuaton where the Breslow-Day test may struggle. Fgures 7 and 8 provde a comparson whch ndcates the excellence of the ft of our gamma approxmaton to the entre dstrbuton of Q and the poor ft of the ch-square approxmaton. Usng the data of Stead et al. wth 17 studes, we smulated 10,000 values of Q to provde an emprcal dstrbuton of Q. Fgure 7 shows the ft of our estmated gamma dstrbuton (α = 8.90 and β = 1.66). Note that the ft s qute good throughout the entre emprcal dstrbuton. On the other hand, Fgure 8 shows that the emprcal dstrbuton of Q departs substantally from the ch-square dstrbuton wth 16 df, agan throughout the entre dstrbuton. Conclusons Cochran s Q statstc s a popular choce for conductng a homogenety test n meta-analyss and n mult-center trals. However users must be cautous n referrng Q to a ch-square dstrbuton when the study szes are small or moderate. Here we have studed the dstrbuton of Q when the effects of nterest are (the logarthms of) odds ratos between two arms of the ndvdual studes. We have shown that the dstrbuton of Q n these crcumstances does not follow a ch-square dstrbuton, especally f the bnomal probablty n at least one of the two arms s far from the central value of 0.5, say outsde the nterval [ 0.3, 0.7]. Further, the convergence of the dstrbuton of Q to the asymptotcally correct ch-square dstrbuton s relatvely slow as the szes of the studes ncrease. The mean and varance of Q (when the effects are log odds ratos and under the null hypothess of homogenety) are often substantally less than the correspondng ch-square values. We have provded formulas for estmatng these moments and have found that matchng these moments to those of a gamma dstrbuton provdes a good ft to the dstrbuton of Q. The use of ths dstrbuton for Q yelds a reasonably good test of homogenety (denoted Q γ ) whch s compettve wth the well known

11 Kulnskaya and Dollnger BMC Medcal Research Methodology (2015) 15:49 Page 11 of 19 Fgure 7 Qualty of ft of the gamma approxmaton. Qualty of ft of the gamma approxmaton (α = 8.90 and β = 1.66) to the emprcal dstrbuton of Q usng the data of Stead et al. (2013) wth 17 studes, produced by the R package ftdstrplus [30]. Breslow-Day test both n accuracy of level and n power. However, ths Qγ test does not seem to be superor (ether n accuracy of level or n power) to the Breslow-Day test. Accordngly we recommend that the smpler Breslow-Day test be used routnely for testng the homogenety of odds ratos. We note that when the data are very sparse, the Breslow-Day test does not perform well and may even not be defned. We have met ths dffculty n our unequal smulatons descrbed n Secton Accuracy of the level of the homogenety test. The Qγ test s always well defned and s recommended for use n such stuatons. In our study of the moments of Q for log odds ratos, we found that the varance of Q can be well approxmated by a functon of the mean of Q. Thus when fttng a gamma dstrbuton to Q, at least approxmately, the resultng dstrbuton comes from a one parameter sub-famly of the gamma famly of dstrbutons. The ch-square dstrbutons also form a one parameter sub-famly of the gamma famly, but our concluson s that t s the wrong sub-famly to apply to Q. Intutvely, one would expect that a two parameter famly of dstrbutons would be needed because two ndependent bnomal parameters (pt and pc ) for each study enter nto the defnton of Q. Thus t would be of nterest to have a theoretcal explanaton of ths property of Q, but we have been unable to provde ths explanaton. The Q statstc wth ts dstrbuton approxmated by the ch-square dstrbuton s wdely used not only for testng homogenety, but perhaps a more wdespread and more mportant use s ts applcaton to estmate the random varance component τ 2 n a random effects model. Numerous moment-based estmaton technques, such as Fgure 8 Qualty of ft of the ch-square approxmaton. Qualty of ft of the ch-square (16 degrees of freedom) approxmaton to the emprcal dstrbuton of Q usng the data of Stead et al. (2013) wth 17 studes, produced by the R package ftdstrplus [30].

12 Kulnskaya and Dollnger BMC Medcal Research Methodology (2015) 15:49 Page 12 of 19 the very popular DerSmonan-Lard [6,27] and Mandel- Paule [28,29] methods use the frst moment (K 1) and the ch-square percentles appled to the dstrbuton of Q to provde, respectvely, pont and nterval estmaton of τ 2. The latter s acheved through proflng the dstrbuton of Q,.e., nvertng the Q test (see Vechtbauer [27]). From our prevous work wth Bjørkestøl on the homogenety test for standardzed mean dfferences [9] and for the rsk dfferences [10], t s clear that the non-asymptotc dstrbuton of Q strongly depends on the effect of nterest. Ths concluson s confrmed here for Q when the effects are log odds ratos. The use of the correct moments and mproved approxmatons to the dstrbuton of Q for the pont and nterval estmaton of τ 2 for a varety of dfferent effect measures may provde greatly mproved estmators, especally for small values of heterogenety and wll be the subject of our further work. Appendx Appendx A: Informaton about the smulatons All of our smulatons for assessng the accuracy of the levels and the power of varous homogenety tests used K studes wth K = 5, 10, 20 and 40. All smulatons were replcated 10,000 tmes for K = 5, 10 and 20, and (due to tme consderatons) only 1000 tmes for K = 40, unless stated otherwse. The set of smulatons wth all studes havng dentcal parameters were as follows: study sze N = 90, 150 and 210; proporton of each study n the control arm q= 1/2, 1/3 and 2/3; log odds rato (null hypothess) θ = 0, 0.5, 1.0, 1.5, 2.0 and 3.0; and bnomal probabltes n the control arm p C = 0.1, 0.2 and 0.4. It s easer and more ntutve to select values of p C than to select values of the actual nusance parameter ζ = log(p C ) log(1 p C ). For the smulatons usng unequal parameters among the varous studes, the parameter choces can be descrbed as follows. For K = 5, we use three vectors of study szes: < N >=< 36, 48, 54, 60, 252 >; < 96, 108, 114, 120, 312 >; and< 163, 173, 178, 184, 352 >. These three vectors have average study szes 90, 150 and 210 respectvely, whch corresponds to the study szes of the equal smulatons. The null hypothess values of the log odds rato θ are 0, 0.5, 1.0, 1.5, 2 and 3. For each fxed value of θ, we chose fve values of p C wth the goal of keepng p T away from 1.0 (see below for these values). Denote the vector of these values of p C by < P > and the vector of the same values but n reverse order by < P >. From Fgure 9 Acheved levels for homogeneous studes, N = 150. Acheved levels for the three tests Q γ (sold lne), BD (dot-dash), and Q χ 2 (dash) plotted aganst the log odds rato θ. Here all studes have the same parameters: 150 subjects n each study wth equal arms of 75 each (N = 150 and q = 1/2).

13 Kulnskaya and Dollnger BMC Medcal Research Methodology (2015) 15:49 Page 13 of 19 θ and < P >, t s easy to calculate the correspondng values of p T ; although these are not needed here, we nclude the approxmate range of p T for nformaton purposes. θ = 0 < P >=< 0.1, 0.3, 0.5, 0.7, 0.9 > the range of p T s [ 0.1, 0.9] θ = 0.5 < P >=< 0.15, 0.3, 0.45, 0.6, 0.75 > the range of p T s [ 0.22, 0.83] θ = 1.0 < P >=< 0.1, 0.25, 0.4, 0.55, 0.7 > the range of p T s [ 0.23, 0.86] θ = 1.5 < P >=< 0.1, 0.25, 0.4, 0.55, 0.7 > the range of p T s [ 0.33, 0.91] θ = 2 < P >=< 0.1, 0.2, 0.3, 0.4, 0.5 > the range of p T s [ 0.45, 0.88] θ = 3 < P >=< 0.1, 0.17, 0.24, 0.31, 0.38 > the range of p T s [ 0.69, 0.92] For K = 5, we conducted smulatons for each value of θ parng the frst value of < N > wth the frst value of < P >, etc. whch we denote order = 1, and then we par the frst value of < N > wth the frst value of < P >, etc, whch we denote order = 2. By reversng the orders, we frst par the largest study sze wth the largest bnomal probablty and then par the largest study sze wth the smallest bnomal probablty. We used balanced studes for these smulatons (.e., q = 1/2). For K = 10, we repeat these parngs twce, and for K = 20 and K = 40 the vectors of study szes and control arm probabltes are repeated4and8tmesrespectvely. We conducted many addtonal smulatons wth unequal sze studes, some wth all control probabltes equal except for 20% of the studes whch had dfferent control probabltes, and some wth one or more of the studes beng unbalanced (q = 1/3 andq = 2/3). These smulatons dd not add substantal nformaton to our conclusons, so they are not reported here. For the power smulatons we only consdered the case of K studes wth the above dentcal parameters (ncludng the values of the common log odds rato θ) and balanced sample szes (q = 1/2). For each combnaton of N, K, p C, θ, accordng to the random effects model of meta-analyss, we smulated K wthn-studes log odds ratos θ from the N(θ, τ 2 ) dstrbuton for the values of the heterogenety parameter τ from 0 to 0.9 n Fgure 10 Acheved levels for homogeneous studes, N = 210. Acheved levels for the three tests Q γ (sold lne), BD (dot-dash), and Q χ 2 (dash) plotted aganst the log odds rato θ. Here all studes have the same parameters: 210 subjects n each study wth equal arms of 105 each (N = 210 and q = 1/2).

14 Kulnskaya and Dollnger BMC Medcal Research Methodology (2015) 15:49 Page 14 of 19 the ncrements of 0.1. Gven the values of p C and θ,we next calculated the probabltes n the treatment groups p T and smulated the numbers of the study outcomes from the bnomal dstrbutons Bn(n, p C ) and Bn(n, p T ) for = 1,, K. All smulatons were replcated 1000 tmes. Appendx B B.1 Addtonal graphs for accuracy of level and for power The frst two fgures of ths Appendx are smlar to Fgure 2 of the man artcle wth the change beng that the study szes are 150 (nstead of 90) n Fgure 9 and 210 n Fgure 10. These panels are qute smlar to the one presented n Fgure 2 except that all levels become closer to the nomnal level of 0.05 as the study sze ncreases from 90 to 150 to 210. Ths behavor s consstent wth the known fact that the tests are asymptotcally correct as the study szes tend to. However, we note that even when N = 210, the test Q χ 2 s stll qute conservatve when p C = 0.1. Fgures 11 and 12 contan addtonal panels of graphs analogous to that n Fgure 2 of the man artcle wth the excepton that the two arms of each study are unbalanced. In the frst of these, all studes have twce the number of subjects n the treatment arm (q = 1/3) and the second s reversed wth all studes havng twce the number of subjects n the control arm. The results are smlar to those of Fgure 2 wth the followng modfed conclusons. When q = 1/3 andp C = 0.1, the Q χ 2 test s partcularly conservatve, rejectng the null hypothess less than 1% of the tme, ndependent of the number of studes K. Generally both the BD test and the Q γ test are reasonably close to nomnal level, but the BD test s mostly (but not always) somewhat better than the Q γ test. When θ = 3, all tests experence a declne n accuracy, wth the BD test mostly superor. The fnal two fgures n ths appendx are analogous to Fgures 4 and 5 n the man artcle, comparng the power of the three tests Q γ,bdandq χ 2 when the log odds rato s 0 and 3 respectvely. The panels here (Fgures 13 and 14) dffer n that the sample szes have been ncreased from N = 90 to N = 150. Qualtatvely the plots here are qute smlar to those n the man artcle, wth the man dfference, as would be expected, beng that the power when N = 150 s somewhat greater than when N = 90. As before, Q γ and BD have smlar power whle Q χ 2 s most nferor n the two cases: θ = 0andp C = 0.1; and θ = 3 Fgure 11 Acheved levels for homogeneous studes, N = 90, q = 1/3. Acheved levels for the three tests Q γ (sold lne), BD (dot-dash), and Q χ 2 (dash) plotted aganst the log odds rato θ. Here all studes have the same parameters: 90 subjects n each study wth unequal arms wth 60 n the treatment arm (N = 90 and q = 1/3).

15 Kulnskaya and Dollnger BMC Medcal Research Methodology (2015) 15:49 Page 15 of 19 Fgure 12 Acheved levels for homogeneous studes, N = 90, q = 2/3. Acheved levels for the three tests Q γ (sold lne), BD (dot-dash), and Q χ 2 (dash) plotted aganst the log odds rato θ. Here all studes have the same parameters: 90 subjects n each study wth unequal arms wth 30 n the treatment arm (N = 90 and q = 2/3). and p C = 0.4. These two cases share the property that one or both of the bnomal probabltes s far from the central value of 0.5; n the frst case, p C = p T = 0.1 and n the second case, p T = B.2 Informaton about formulas for mean and varance of Q LOR In ths appendx we present addtonal nformaton concernng the data and methods that entered nto Equatons 6 and 7 whch provde formulas for estmatng the mean and varance of Q LOR under the null hypothess of equal odds ratos. The data for Equaton 6 nclude 648 parameter combnatons n whch all K studes had dentcal parameters. The parameters are: K = 5, 10, 20, 40; N = 90, 150, 210; q = 1/3, 1/2, 2/3; p C =0.1, 0.2, 0.4; and θ = 0, 0.5, 1, 1.5, 2, 3. The smulatons for K = 40 were replcated 1,000 tmes, and the other smulatons were replcated 10,000 tmes. For each combnaton of parameters, we calculated an estmate of the mean of Q LOR (to be denoted smply Q n ths secton) usng the theoretcal expanson of Kulnskaya et al. [10]. We denote ths quantty by E th [ Q]. For each parameter combnaton, we also found the mean of Q from the smulatons, whch we denote by Qbar. These two quanttes were then dvded by K 1toplacethe data on a scale common for all K. Ascatterplotwth a ftted lne s found n Fgure 15. Note that the ftted lne (whch has an R 2 value of 97.0%) essentally goes through the pont (1, 1); the mportance of the ftted lne gong through (1,1) s that both estmates agree when there s zero correcton from the re-scaled ch-square moment. Thus we subtracted 1 from both varables n Fgure 15 and ft a regresson through the orgn, yeldng a relaton whch we use to adjust the correctons to the ch-square frst moments K 1 whch are gven by the the expanson E th [ Q]. Ths relaton s found n Equaton 6 of the man paper. (The four outlers n the lower left of Fgure 15 belong to the extreme parameter values θ = 3, N = 90, q = 2/3, p T = 0.93, p C = 0.4 and for the four values of K = 5, 10, 20 and 40; omttng them made very lttle dfference n the regresson, so they were ncluded n the analyss). Smulatons for all of the parameter confguratons that entered nto Equaton 6 of the man paper were redone, and these new smulatons were the ones used n analyzng the accuracy of our test Q γ.

16 Kulnskaya and Dollnger BMC Medcal Research Methodology (2015) 15:49 Page 16 of 19 Fgure 13 Power when the log odds rato θ = 0andN = 150. Power for the three tests Q γ (sold lne), BD (dot-dash), and Q χ 2 (dash) plotted aganst τ, the square root of the random effect varance. Here all studes have the parameters: 150 subjects n each study wth equal arms of 75 each (N = 150 and q = 1/2) and the log odds rato θ = 0. To arrve at the relaton n Equaton 7, we used smulatons for 486 parameter combnatons n whch all K studes have the same parameters: K = 5, 10, 20; N = 90, 150, 210; q = 1/3, 1/2, 2/3; p C = 0.1, 0.2, 0.4; and θ = 0, 0.5, 1, 1.5, 2, 3, each replcated 10,000 tmes. For each parameter combnaton, let Qbar bethemeanofthe 10,000 values of Q and VarQbar bethevaranceofthese 10,000 values of Q, and re-scale these values by dvdng by K 1. Fgure 16 contans a scatter plot of these data together wth a quadratc functon ft. The quadratc ft has an R 2 value of 98.5%. We have used ths regresson n Equaton 7 of the man artcle. We note agan that smulatons for all of the parameter confguratons that entered nto Equaton 7 of the man paper were redone, and these new smulatons were the ones used n analyzng the accuracy of our test Q γ. B.3 The general expanson for the frst moment of Q appled to Q LOR The general expanson for the frst moment of Q (denoted E th [ Q] n Secton Estmatng the moments and dstrbuton of Q LOR ) as found n Kulnskaya et al. [10] s reproduced at the end of ths appendx. In the formulas below, we use the notaton = θ θ and Z = ζ ζ ; also, we express the weght estmators as functons of the parameter estmators ŵ = f ( θ, ζ ).Thetheoretcal weghts under the null hypothess are then w = f (θ, ζ ). For the weghts as defned n Equaton 2 of the man artlcle, some algebra produces the formula for the weght functon [ ] 1 (1 + e θ + ζ ) 2 ŵ = f ( θ, ζ ) = + (1 + e ζ ) 2 (n T + 1)e θ + ζ (n C + 1)e ζ The formulas below requre that the central moments of θ and ζ satsfy the followng order condtons: O(E[ ] ) = 1/n 2, O(E[ 2 ] ) = 1/n, O(E[ 3 ] ) = 1/n 2 and O(E[ 4 ] ) = 1/n2 and smlar condtons for the central moments of ζ. These order condtons for thespecfccaseoftheestmatorsofthelogoddsrato (as defned n Secton Notaton and assumptons ) follow from the work of Gart et al. [12]. However, nstead of usng the approxmatons for the central moments gven by Gart et al., our R-program calculates these exactly. (8)