Replicability and meta-analysis in systematic reviews for medical research

Size: px
Start display at page:

Download "Replicability and meta-analysis in systematic reviews for medical research"

Transcription

1 Tel-Aviv University Raymond and Beverly Sackler Faculty of Exact Sciences Replicability and meta-analysis in systematic reviews for medical research Thesis submitted in partial fulfillment of the requirements for the M.Sc. degree in Applied Statistics in the School of Mathematical Sciences, Tel-Aviv University by Liat Shenhav The research work for this thesis has been carried out at Tel-Aviv University under the supervision of Prof. Yoav Benjamini and Prof. Ruth Heller October 2016

2 Contents 1 Introduction 6 2 Fixed effect model The r-value for replicability The r-value computation Sensitivity analysis for confidence intervals - sensitivity interval Sensitivity interval computation Leave-u-out sensitivity procedure The complexity of the Leave-u-out sensitivity analysis Lower bound on the extent of replicability Accounting for multiplicity Random effects model Point estimates of µ and τ The point estimate of the overall population treatment effect µ Point estimates of the between studies variance τ Tests and confidence intervals for the overall population treatment effect Simulation study Methods and results for meta-analysis Coverage probability Power

3 3 4.2 Methods and results for replicability analysis Replicability type I error Cochrane Collaboration systematic reviews 48 The lack of replicability in systematic reviews Calculating and reporting the r-value: examples Extensions Discussion 61 Appendices 66 A Simulations - coverage probabilities 67 B Simulations - power 69 C Simulations - replicability analysis 76

4 Abbreviations DL DerSimonian and Laird heterogeneity estimation method [DerSimonian and Laird, 1986] PM Paule and Mandel heterogeneity estimation method [Paule and Mandel, 1982] ML Maximum likelihood heterogeneity estimation method [Harville, 1977] REML Restricted maximum likelihood heterogeneity estimation method [Rukhin et al., 2000] SJKH Sidik-Jonkman and Hartung-Knapp heterogeneity estimation method [IntHout et al., 2014] GLMM Generalized linear mixed model 4

5 Abstract In order to assess the effect of a health care intervention, it is useful to look at an ensemble of relevant studies. The Cochrane Collaboration s admirable goal is to provide systematic reviews of all relevant clinical studies, in order to establish whether or not there is a conclusive evidence about a specific intervention. This is done mainly by conducting a meta-analysis: a statistical synthesis of results from a series of systematically collected studies. Health practitioners often interpret a significant meta-analysis summary effect as a statement that the treatment effect is consistent across a series of studies. However, the meta-analysis significance may be driven by an effect in only one of the studies. Indeed, in an analysis of two domains of Cochrane reviews we show that in a non-negligible fraction of reviews, the removal of a single study from the meta-analysis of primary endpoints makes the conclusion non-significant. Therefore, reporting the evidence towards replicability of the effect across studies in addition to the significant metaanalysis summary effect will provide credibility to the interpretation that the effect was replicated across studies. We suggest an objective, easily computed quantity, we term the r-value, that quantifies the extent of this reliance on single studies. We suggest adding the r-value, accompanied by a sensitivity interval, to the main results and to the forest plots of systematic reviews. 5

6 Chapter 1 Introduction In systematic reviews, several studies that examine the same questions are analyzed together. Viewing all the information is extremely valuable for practitioners in the health sciences. A notable example is the Cochrane systematic reviews on the effects of healthcare interventions. The process of preparing and maintaining Cochrane systematic reviews is described in detail in their manual [Higgins et al., 2011]. The reviews attempt to assemble all the evidence that is relevant to a specific healthcare intervention. Deriving conclusions about the overall health benefits or harms from an ensemble of studies can be difficult, since the studies are never exactly the same and there is danger that these differences affect the inference. For example, factors that are particular to the study, such as the specific cohorts in the study that are from specific populations exposed to specific environments, the specific experimental protocol used in the study, the specific care givers in the study, etc., may have an impact on the treatment effect. A desired property of a systematic review is that the effect has been observed in more than one study, i.e., the overall conclusion is not entirely driven by a single study. If a significant meta-analysis finding becomes non-significant by leaving out one of the studies, this is worrisome for two reasons: first, the finding may be too particular to the single study (e.g., the specific age group in the study); second, there is greater danger that the significant meta-analysis finding is due to bias in the single study (e.g., due to improper randomization or blindness). We view this problem as a replicability problem: the conclusion 6

7 7 about the significance of the effect is completely driven by a single study, and thus we cannot rule out the possibility that the effect is particular to the single study, i.e., that the effect was not replicated across studies. A replicability claim is not merely a vague description. A precise computation of the extent of replicability is possible. An objective way to quantify the evidence that the meta-analytic findings do not rely on single studies is as follows. For a meta-analysis of several studies (N studies), the minimal replicability claim is that results have been replicated in at least two studies. This claim can be asserted if the meta-analysis results remain significant after dropping (leaving-out) any single study. We suggest accompanying the review with a quantity we term the r-value [Heller et al., 2014], which quantifies the evidence towards replicability of the effects across studies. The r-value is the largest of these N leave-1-out meta-analysis p-values. Like a p-value, which quantifies the evidence against the null hypothesis of no effect, the r-value quantifies the evidence against no replicability of effects. The smaller the r-value, the greater the evidence that the conclusion about a primary outcome is not driven by a single study. The report of the r-value is valuable for meta-analyses of narrow scope as well as of broad scope. In Chapter 5.6 of the manual [Higgins et al., 2011] the scope of the review question is addressed. If the scope is broad, then a review that produced a single meta-analytic conclusion may be criticized for mixing apples and oranges, particularly when good biologic or sociological evidence suggests that various formulations of an intervention behave very differently or that various definitions of the condition of interest are associated with markedly different effects of the intervention. The advantage of a broad scope is that it can give a comprehensive summary of evidence. The narrow scope is more manageable, but the evidence may be sparse, and findings may not be generalizable to other settings or populations. If the r-value is large (say above 0.05) for a meta-analyses with a narrow scope, this is worrisome since the scope has already been selected, and the large r-value indicates that an even stricter selection that removes one single additional study can change the significant conclusion. If the r-value is large for a meta-analyses with a broad scope, this is worrisome since the reason for the significant finding may be the single orange among the several (null) apples. [Anzures-Carbera and Higgins, 2010] write that a useful sensitivity analysis is one in which the metaanalysis is repeated, each time omitting one of the studies. A plot of the results of these meta-analysis,

8 8 called an exclusion sensitivity plot by [Bax et al, 2006], will reveal any studies that have a particularly large influence on the results of the meta-analysis. In this work, we concur with this view, but recommend the most relevant single number of summary information of such a sensitivity analysis, the r-value, be added to the report of the main results, and to the forest plot, of the meta-analysis. The general outline of this work is as follows. In Chapter 2, we present in detail the replicability formulation, measurement and procedures for the fixed effect model. In Chapter 3 we use similar definitions and procedures for the random effects model. Nevertheless, in the random effects model we also have to account for the variability between the different studies included in the meta-analysis and we therefore examine five different heterogeneity estimation methods and asses their effect on the meta-analysis conclusion in general and specifically on the replicability analysis. In Chapter 4 we show, using simulation study, that random effects meta-analysis can be significant even if the effect is greater than zero in only one study out of N, regardless of the heterogeneity estimation method. In particular we show that the replicability type I error, that is, the probability of rejecting the null hypothesis with a single outlying study, can be as high as 6 times the nominal level (0.05) using the common heterogeneity estimation method (DerSimonian and Laird) and the z-test, and as high as 3 times the nominal level using different heterogeneity estimation methods and the t-test. In other words, our simulation results indicate that the common (and questionable) heterogeneity estimation method may aggravate the replicability problem and we therefore explore the alternatives. In Chapter 5 we examined the extent of the replicability problem in the Cochrane Collaboration systematic reviews. We found that there may be lack of replicability in a large proportion of studies. Specifically, we show that out of the 21 reviews with a significant meta-analysis result on the most important outcomes of interest published on breast cancer, 13 reviews were sensitive to leaving one study out of the meta-analysis. The problem was less profound in the reviews published on influenza, where 2 reviews were sensitive to leaving one study out of the meta-analysis, out of 6 updated reviews with significant primary outcomes. The code used in this work for calculating the r-value and sensitivity interval for both fixed and random effects model meta-analysis can be found in the R-package Meta-analysis Replicability.

9 Chapter 2 Fixed effect model Consider a collection of N studies, the ith of which has estimated effect y i and true effect θ i. A general model is then specified by y i = θ i + ɛ i ; ɛ i N(0, σ 2 i ), for i = 1, 2,..., N (2.1) The ɛ i indicate random deviations from the true effect and are assumed independent with mean zero and variance σi 2. The variance of ɛ i, σi 2, is the sampling variance reflecting within-study variance and the sample size of the study. This implies that the estimated effect y i is normally distributed with mean θ i and variance σ 2 i. y i can be any measure of effect, provided the assumption of normality is (at least approximately) appropriate. Common examples are a log-odds ratio, log-risk ratio or difference in means. In general, the parameter of interest is the overall effect, denoted by µ. The fixed effect model assumes θ i = µ for i = 1,..., N, implying that each study in the meta-analysis has the same underlying effect. Note that even if the θ i are assumed to be the same, the observed effects y i are not identically distributed due to the possibility of differing σ 2 i. The estimator of µ is generally a simple weighted mean of y i, with the optimal weights proportional to w i = 1 var(y i). In practice, the within study variances are not known 9

10 10 so estimated variances s 2 i are used to estimate both µ and var(ˆµ). N i=1 ˆµ = ŵiy i N = N i=1 ŵi 1 var(ˆµ) ˆ = N i=1 ŵi y i s 2 i=1 i = 1/ / N 1 s 2 i=1 i N 1 s 2 i=1 i (2.2) (2.3) A fixed effect meta-analysis p-value tests the intersection null hypothesis that θ i = 0 in all N studies. Therefore, a fixed effect meta-analysis finding is a claim that θ i 0 in at least one study. Note that this claim may be true even when the replicability claim is false: if θ i 0 in exactly one study, and θ i = 0 in all other studies. 2.1 The r-value for replicability For a meta-analysis based on N studies, a replicability claim is a claim that the conclusion remains significant (e.g., rejection of the null hypothesis of no treatment effect) using a meta-analysis of each of the ( N u 1) subsets of N u + 1 studies, where u = 2,..., N is a parameter chosen by the investigator. Specifically, for u = 2, a replicability claim is a claim that the conclusion remains significant using a meta-analysis of each of the N subsets of N 1 studies. The r-value for the single endpoint design is the smallest significance level at which we claim replicability (for a pre-specified u). The smaller the r-value, the stronger the evidence against no replicability of effects. The r-value computation Let p L i 1,...,i m and p R i 1,...,i m be, respectively, the left- and right- p-values (the two possible one-sided tests) from a meta-analysis on the subset (i 1,..., i m ) {1,..., N} of the N studies in the full meta-analysis,

11 11 m < N. Let Π(m) denote the set of all possible subsets of size m. The r-value for replicability analysis, where we claim replicability if the conclusion remains significant using a meta-analysis of each of the ( N u 1) subsets of N u + 1 studies is computed as follows. For leftsided alternative, the r-value is r L = max (i 1,...,i N u+1 ) Π(N u+1) pl i 1,...,i N u+1. For right- sided alternative, the r-value is r R = max (i 1,...,i N u+1 ) Π(N u+1) pr i 1,...,i N u+1. For two-sided alternavies, the r-value is r = 2 min(r L, r R ). 2.2 Sensitivity analysis for confidence intervals - sensitivity interval The sensitivity interval is the union of all the meta-analysis confidence intervals using the ( N u 1) subsets of N - u + 1 studies. The meta-analysis is non-sensitive (at level α and at the desired value u) if and only if the sensitivity interval does not contain the null hypothesis value. Sensitivity interval computation The upper limit of the (1 α) sensitivity interval is the upper limit of the (1 α 2 ) confidence interval from the meta-analysis on (i L 1,..., i L N u+1 ), where (il 1,..., i L N u+1 ) is the subset that achieves the maximum p-value for the left-sided r-value computation. Similarly, the lower limit of the (1 α) sensitivity interval is the lower limit of the (1 α 2 ) confidence interval from the meta-analysis on (ir 1,..., i R N u+1 ), where

12 12 (i R 1,..., i R N u+1 ) is the subset that achieves the maximum p-value for the right-sided r-value computation. The meta-analysis is non-sensitive (at the desired value of u) if and only if the sensitivity interval does not contain the null hypothesis value. This follows from the following argument. To see this, note that r α, if and only if r L α/2 or r R α/2. Since r L α/2 if and only if the upper limit of all the meta-analysis (1 α 2 ) confidence intervals of subsets of size N u + 1 is below the null value, and r R α/2 if and only if the lower limit of all the meta-analysis (1 α 2 ) confidence intervals of subsets of size N u + 1 is above the null value, the result follows. 2.3 Leave-u-out sensitivity procedure For a meta-analysis with N studies and θ1,..., θ effects (two-sided alternative) : 1. Compute meta-analysis of each of the ( N u 1) subsets of N u + 1 studies. 2. Compute the left and right-sided r-value - r L and r R. 3. Compute the two-sided r-value : r = 2 min(r L, r R ). 4. If the r-value 0.05, the replicability is established in at lease u studies. Otherwise, there is a significant effect in at most u 1 studies (for u=2, r-value > α means that the finding is not replicable). The complexity of the Leave-u-out sensitivity analysis Lemma In a meta-analysis, the complexity of ranking N studies and leaving out u studies is O(N u 1 ). Proof We first show for u = 2. Recall that in the fixed effect meta-analysis, the overall effect and its corresponding variance are:

13 13 N i=1 ˆµ = w iy i N N i=1 w = i var(ˆµ) = 1 N i=1 w i y i σ 2 i=1 i = 1/ / N 1 σ 2 i=1 i N 1 σ 2 i=1 i The Z-test statistic for testing the meta-analysis null hypothesis is z = ˆµ N se(ˆµ) = i=1 w iy i 1 N i=1 w 1 = i N i=1 wi N i=1 w iy i N i=1 w i (2.4) Using the Z-test statistic we calculate the corresponding p-values: p R i 1,...,i N 1 = P r ( Z z ) p L i 1,...,i N 1 = P r ( Z z ) In order to find the r-value, we will separately the following two scenarios : (1) the original meta-analysis Z-test statistic value is positive and (2) the original meta-analysis Z-test statistic value is negative. In the first scenario we would like to decrease the Z value in the maximum amount possible by a single study and the second, we would like to increase the Z value in the maximum amount possible by a single study. The difference between the original value of the Z-test statistic (based on N studies) and the new value of the Z-test statistic (based on N 1 studies) for each j; j = 1,..., N is notes as N i=1 w N iy i i=1,i j N w iy i N i=1 w i i=1,i j w i (2.5)

14 14 Arranging expression (2.5) yields N i=1 w N iy i i=1,i j w i N N i=1 w i N i=1 w N iy i i=1,i j N w iy i N = i=1 w i i=1,i j w i ( N i=1,i j w ) ( N iy i i=1,i j w N i = i=1,i j w N iy i N i=1,i j w i i=1 w N i) + wj y j i=1,i j w i N i=1 w N = i i=1,i j w i i=1 w i = = ( N i=1,i j w iy i ) ( 1 N i=1 wi N i=1,i j wi ) + wj y j N i=1 w i By denoting a = N i=1 w i and b = N i=1 w iy i we get that the last expression can be re-written as ( ) ( b wj y j 1 a ) a w j + wj y j b b a a w j + w j y a j a w j = = a a b + ( ) b + w j y a j a w j = = b + w jy j b a a a wj Therefore N i=1 w N iy i i=1,i j N w iy i N = b + w jy j b (2.6) i=1 w i i=1,i j w a a wj i For a meta-analysis with a positive (negative) Z-test statistic we need to find the study that maximizes (minimizes) the non-constant part of expression, over all values j, for j = 1,.., N. k = arg max j=1,...,n { wj y j b } a wj

15 15 k = arg min j=1,...,n { wj y j b } a wj By omitting study k (k ) that maximizes (minimizes) the non-constant part of expression (2.6), we decrease (increase) the Z-test statistic in the maximum amount possible by a single study and increase the corresponding p-value in the maximum amount possible by a single study. Hence, in O(N) we get the maximum p-value over all subsets of N 1 studies, which is exactly the r-value for u = 2. For a general u, expression (2.6) can be written as N i=1 w iy i N i=1 w i j S w iy i = b + j S w i a j S w jy j b a j S w j (2.7) where S is a subset of N u + 1 studies. For a meta-analysis with a positive (negative) Z-test statistic we need to find a subset of S studies that maximize (minimize) the non-constant part of expression (2.7) over all possible combinations of N u + 1 studies, ( N u 1). { j S S = arg max w jy j b } S [N] a j S w j { S j S = arg min w jy j b } S [N] a j S w j To summarize: by omitting u 1 studies that maximize (minimize) the non-constant part of expression

16 16 (2.7), we decrease (increase) the Z-test statistic in the maximum amount possible by u 1 studies and increase the corresponding p-value in the maximum amount possible by u 1 studies. Hence, in O(N u 1 ) we get the maximum p-value over all subsets of N u + 1 studies, which is exactly the r-value for a general u. Algorithm (for general u) 1. If the original meta-analysis Z-test statistic value is positive: rank the N studies, in ascending Order, according to the ratio w j y j b a wj else, rank the N studies, in descending order, according to the ratio w j y j b a wj 2. Leave out the study with the highest rank (N) and compute a meta-analysis using the remaining N 1 studies. The r-value is the p-value of this meta-analysis. 3. If the r-value 0.05, the replicability is established in at lease 2 studies. Otherwise, the finding is not replicable. 4. If u > 2 repeat steps 1-3 additional u 2 times, each repetition update N to be N j, where j = 1, 2,.., u Lower bound on the extent of replicability A meta-analysis is less sensitive than another meta-analysis if a larger number of studies are excluded without reversing the significant conclusions. The bigger the number of studies that can be dropped, the stronger the replicability claim. Testing in order at significance level α, results in a 1-α confidence lower bound on the number of studies with an effect in a fixed-effect meta-analysis (see [Heller, 2011] for proof). Note that although we have a lower bound on the number of studies that show an effect, we

17 17 cannot point out to which studies these are. This is so since the pooling of evidence in the same direction in several studies increases the lower bound, even though each study on its own maybe non-significant. 2.5 Accounting for multiplicity When more than one primary endpoint is examined, the r-value needs to be smaller in order to establish replicability. This is exactly the same logic as with p-values, for which we need to lower the significance threshold when faced with multiplicity of endpoints. Family-wise error rate (FWER) or false discovery rate (FDR) controlling procedures can be applied to the individual r-values in order to account for the multiple primary endpoints, see [Benjamini et al, 2009] for details.

18 Chapter 3 Random effects model A random effects model for meta-analysis specifies that the observed treatment effect from the ith study, y i, is made up of two additive components: the true treatment effect for the study, θ i, and the sampling error ɛ i. That is y i = θ i + ɛ i ; ɛ i N(0, σ 2 i ) for i = 1,..., N (3.1) The variance of ɛ i, σi 2, is the sampling variance reflecting within-study variance and the sample size of the study. σi 2, is usually unknown and is estimated from the data of the i-th study. In addition to the sampling error associated with each study, the random effects model assumes the true treatment effect in each study will be influenced by several factors, including patient characteristics as well as design and execution of the study. The model explicitly accounts for this possible heterogeneity in the true treatment effect and specifies that θ i = µ + δ i, where θ i is the true treatment effect in the i-th study, µ is the overall treatment effect for a population of possible treatment evaluations and δ i = θ i µ is the deviation of the i-th study s effect from the overall effect µ. The variance of δ i, τ 2 0, is the between-studies variance and represents both the degree to which true treatment effects vary across 18

19 19 experiments as well as the degree to which individual studies give biased assessments of treatment effects. That is θ i = µ + δ i ; δ i N(0, τ 2 ) for i = 1,..., N (3.2) With this formulation, the model assumes that the observed treatment effects y 1,..., y N are realizations of independent random variables from a normal distribution with overall effect µ and variances τ 2 + σ1, 2..., τ 2 + σn 2. The variances reflect the two components of variance assigned to each observed effect: a between-studies variance τ 2 which reflects treatment effects heterogeneity and a within study variance σ 2 i which reflects within study sampling variance. Note that the assumption of normally distributed random effects, and particularlly normally distributed between study deviations (δ i ) is not easily verified or justified due to small number of studies. This is especially relevant since [Davey J, et al., 2011] show, using a sample of 22, 453 meta-analyses 1, that the number of studies in a meta-analysis is often relatively small, with a median of 3 studies (Q1 Q3: 26), and only 1% of meta-analyses containing 28 studies or more. The issue of validating the assumption of normality is being addressed by Hardy and Thompson [Hardy et al., 1998], however they consider only relatively large values of studies (N > 20). We do not offer an alternative to the normal distribution assumption but rather a relaxation: we assume that model (3.2) is true, up to u 1 studies (outliers). We should emphasize that even if the assumption of normality is reasonable, since we estimate the parameters it is not straightforward how to take this additional uncertainty into account when inferring on the overall treatment effect. In the medical research field and specifically in the Cochrane Collaboration systematic reviews the common heterogeneity (between study deviations: δ i ) estimation method is the DerSimonian and Laird 1 The sample was taken from the January 2008 issue of the Cochrane Database of Systematic Reviews (with at least two studies in every review).

20 20 [DerSimonian and Laird, 1986] method. This method which assumes normally distributed random effects poses the following limitation: The variation between true study effects (θ i, for i = 1,..., N) is taken into account via the inclusion of the deviations δ i with variance τ 2. It is however only an estimate of this variance which is added into the weights, and the model takes no account of the uncertainty associated with this estimate in both the null distribution and the test statistic. Due to this issue, we examine other heterogeneity estimation methods and null distributions both theoretically and by simulations. Specifically, we use simulations to compare all methods, including the DerSimonian and Laird, in terms of µ coverage probabilities. In addition, since we focus on replicability in meta-analysis, we also compare the different methods in terms of type I error indicating the extant of the replicability problem. Our goal in this examination and comparison of heterogeneity estimation methods is twofold. First is to improve the estimation and inference of the meta-analysis overall effect µ, and secondly is to show that different heterogeneity estimation methods may yield different results in terms of the overall effect replicability. Specifically, our simulations results indicate that the common (and questionable) heterogeneity estimation method may aggravate the replicability problem. Therefore, we explore the alternatives. Note that in the random effects model we use similar definitions and procedures as in the fixed effect model for measuring and testing replicability (r-value, sensitivity interval and leave-u-out sensitivity procedure). 3.1 Point estimates of µ and τ 2 In this Section we present point estimation methods for the overall population treatment effect (µ) and the between studies variance (τ 2 ) in the framework of the random effects meta-analysis. Our main focus is the impact of the different τ 2 estimation methods on the overall population treatment effect point estimation ˆµ and its variance Var(ˆµ).

21 21 The point estimate of the overall population treatment effect µ Given the observed effects, y 1,..., y N, and the sampling variances, σ 1 2,..., σ N 2, the first step in metaanalysis based on a random effects model is to estimate the between studies variance τ 2 and then estimate the overall population treatment effect µ and its standard error. If σ 1 2,..., σ N 2 and τ 2 were known, a weighted estimator of µ would be ˆµ w = N i=1 wiyi N i=1 wi, where w i = 1 1 τ 2 +σ 2 i, and its standard error would be SE(ˆµ w ) = N. In practice, the variances σ 2 1,..., σ 2 N and i=1 wi τ 2 are usually unknown and are estimated from the data. Suppose s 1 2,..., s N 2 and ˆτ 2 are estimates of σ 1 2,..., σ N 2 and τ 2, respectively. By substituting the estimated variances for σ 1 2,..., σ N 2 and τ 2 in ˆµ w we get the following estimate for µ N i=1 ˆµ w = ŵiy i N i=1 ŵi (3.3) where ŵ i = 1 s i2 + ˆτ 2 (3.4) for i = 1,..., N and an approximated standard error for ˆµ w SE ˆµ w = 1 N i=1 ŵi (3.5) Expression (3.5) for SE ˆµ w is a conditional standard error of ˆµ w under the assumption that the s 1 2,..., s N 2 and ˆτ 2 are equal to the true variances σ 1 2,..., σ N 2 and τ 2, respectively. It is difficult to determine an expression for the unconditional (true) standard error of ˆµ w involving the uncertainty that arises from the use of estimates s 1 2,..., s N 2 and ˆτ 2 for σ 1 2,..., σ N 2 and τ 2, respectively. According to [Kacker and Harville, 1984] expression (3.5) is an underestimate of the true standard error of ˆµ w.

22 22 Point estimates of the between studies variance τ 2 In addition to the sampling variance estimates, s 2 1,..., s 2 N, expressions (3.3) and (3.5) require an estimate ˆτ 2 for τ 2. The common heterogeneity estimation method in the the medical research field and specifically in the Cochrane Collaboration systematic reviews is the DerSimonian and Laird [DerSimonian and Laird, 1986] method. paragraph In the following paragraphs, we describe several methods for estimating τ 2 that yields slightly different results for the overall population treatment effect estimated ˆµ w and its standard error SE(ˆµ w ). These methods include the non-iterative estimates proposed by DerSimonian and Laird [DerSimonian and Laird, 1986] and Sidik-Jonkman and Hartung-Knapp [IntHout et al., 2014] and three iterative estimates, two based on maximum likeliood estimation (regular maximum likelihood [Hardy et al., 1996] and the restricted maximum likelihood Rukhin et al. 2000) and the estimate proposed by Paule and Mandel [Paule and Mandel, 1982]. The DerSimonian and Laird estimator will be referred to as DL, Sidik-Jonkman and Hartung-Knapp estimator will be referred to as SJKH, the regular maximum likelihood estimator will be referred to as ML, the restricted maximum likelihood estimator will be referred to as REML, and the estimator proposed by Paule and Mandel will be referred to as PM. General method-of-moments estimate for τ 2 Heterogeneity is commonly tested using a Q-statistic defined by [Cochran, 1954]: N Q w = a i (y i y w ) 2 (3.6) i=1 where y w = N i=1 aiyi N i=1 ai and a 1,..., a N are any positive constants. The null hypothesis of homogeneity is

23 23 τ 2 = 0 against a one-sided alternative. [Kacker, 2004] showed that the expected value of Q w is: ( N E(Q w ) = E a i (y i y w ) 2) N = a i (τ 2 + σi 2 ) i=1 i=1 N i=1 a i(τ 2 + σ 2 i ) N i=1 a i (3.7) which can be expressed as: ( N E(Q w ) = E a i (y i y w ) 2) = τ 2( N a i N i=1 a i i=1 i=1 N i=1 a2 i ) + ( N a i σi 2 i=1 N i=1 a2 i σ2 i N i=1 a i ) (3.8) By equating the expression Q w = N i=1 a i(y i y w ) 2 to its expected value, given by equation (3.8) and solving for τ 2 we get the following general method-of-moments estimator ˆτ 2 for τ 2 : ˆτ 2 = ( N i=1 a N i(y i y w ) 2 i=1 a iσi 2 N ) i=1 a2 i σ2 i N i=1 ai N i=1 a N (3.9) i=1 i a2 i N i=1 ai Since τ 2 0, the estimate ˆτ 2 is set to zero when it s computed value turns out to be negative, see [Kacker, 2004]. DerSimonian and Laird estimate for τ 2 The DL heterogeneity estimator is commonly used in the random effects model meta-analysis, and specifically, this is the estimator being used in the Cochrane Collaboration systematic reviews. Based on the previously introduced general method-of-moments, with a i = w idl = 1, assuming the σi 2 variances σ 2 i are known, the Q statistic according to DL is

24 24 N (y i y w ) 2 Q w = i=1 σ 2 i (3.10) where under the null hypothesis, Q w χ 2 N 1. With w idl = 1 σ 2 i for i = 1,..., N, equation (3.9) yields the DL estimate, ˆτ 2 DL for τ 2 : N ˆτ DL 2 i=1 = w i (y DL i y )2 wdl (N 1) N i=1 w i N i=1 w2 i N DL DL i=1 wi DL (3.11) where y wdl = N i=1 wi DL yi N. i=1 wi DL Since ˆτ DL 2 as computed above can attain negative values a truncated version is considered: ˆτ 2 DL+ = max{0, ˆτ 2 DL} In practice, since the within study variances are unknown, estimates ŵ i = 1 s 2 i are used giving the statistic: N (y i y w ) 2 Qŵ = s 2 i=1 i (3.12) which is approximately χ 2 N 1 under the null hypothesis. With ŵ idl = 1 s 2 i for i = 1,..., N, the DL estimate, ˆτ 2 DL for τ 2 is in practice:

25 25 N ˆτ DL 2 i=1 = ŵi (y DL i yŵdl )2 (N 1) N i=1 ŵi N i=1 ŵ2 i N DL DL i=1 ŵi DL (3.13) where yŵdl = N i=1 ŵi DL yi N. i=1 ŵi DL Substituting ˆτ 2 DL for τ 2 in the weights used in equation (3.3) yields the (commonly used) corresponding DL estimate ˆµˆτDL for µ and its approximate variance as defined by equation (3.5). Note that for known variances, σ1, 2..., σn 2, the untruncated version of the DL estimator is unbiased by construction. By substituting σ 2 1,..., σ 2 N with s2 1,..., s 2 N the (untruncated) estimator is no longer unbiased. DerSimonian and Laird proposed that this estimate can then be incorporated into the random effects weights giving ŵ i (ˆτ 2 DL+) = 1 ˆτ 2 DL+ + s2 i (3.14) An estimator of µ is then given by ˆµˆτDL = N i=1 ŵi(ˆτ 2 DL+ )y i N i=1 ŵi(ˆτ 2 DL+ ) (3.15) With variance estimated by 1 var(ˆµˆτdl ˆ ) = N i=1 ŵi(ˆτ DL+ 2 ) (3.16) Note that this is simply straight substitution of ˆτ 2 DL into the variance of ˆµ τ, derived assuming τ 2 is known.

26 26 Maximum likelihood estimate for τ 2 The random-effects model described above is a special case of the general linear mixed-effects model (GLMM) of the form y = Xβ + Zγ + e (3.17) where y is a (N 1) vector of random variables, X is a (N p) matrix of known constants for the (p 1) fixed effects parameter vector, Z is the (N q) design matrix for the (q 1) random effects parameter vector,γ and e is a (N 1) vector of random error terms. We assume E(γ) = 0, E(e) = 0, and Cov(γ, e) = 0. Define D as the (qxq) covariance matrix of the random effects parameters in γ and R as the (N N) covariance matrix of e. Then V, the (N N) covariance matrix of y, is equal to ZDZ T + R. Assuming normality of the random terms in the model, we obtain y N(Xβ, V ). Denoting the variance components in V by the vector σ 2, we can write the log-likelihood function of β and σ 2 as logl ( β, σ 2 y ) = 1 2 log V 1 2 (y Xβ)T V 1 (y Xβ) (3.18) leaving out the additive constant. For the meta-analytic random-effects model, y consists of the N effect estimates, X is a (N 1) vector composed entirely of 1s, β includes only the grand mean, Z is the (N N) identity matrix, γ is comprised of the δ i values at the population level, and e includes the random error terms, ɛ 1,..., ɛ N. Then V is diagonal with v i = (τ 2 + σ 2 i ) and y N( 1µ, V ). Substituting s 2 1,..., s 2 N for σ2 i,..., σ2 N and treating the sampling variances as known, the log-likelihood function of µ and τ therefore simplifies to logl(µ, τ 2 y) = 1 2 N log(s 2 i + τ 2 ) 1 2 i=1 N i=1 (y i µ) 2 (s 2 i + τ 2 ), µ R, τ 2 0 (3.19)

27 27 Equations (3.20) and (3.21) are obtained, in the usual way, by taking the partial derivatives of the log likelihood specified in equation (3.19), setting to zero and then rearranging in the form which is convenient for the iteration process: ˆµ m = N y i i=1 s 2 i +ˆτ m 2 N 1 i=1 s 2 i +ˆτ m 2 (3.20) ˆτ 2 m = N (y i ˆµ m) 2 s ( ) 2 i i=1 2 s 2 i +ˆτ m 2 N 1 i=1 s 2 i +ˆτ m 2 (3.21) The ML estimators of the two parameters of interest are derived by solving equations (3.20) and (3.21) in an iterative manner, beginning by substituting an initial value of ˆτ 2 m into equation (3.20). Since τ 2 cannot be negative, the ML estimates ˆµ ml and ˆτ ml 2 are then (ˆµ m, ˆτ m) 2 if ˆτ (ˆµ ml, ˆτ ml) 2 m 2 > 0 = (ˆµ F E, 0) if ˆτ m 2 0 (3.22) Where ˆµ F E is the fixed effects estimate of µ. The variance is derived from the covariance matrix of (ˆµ ml, ˆτ ml 2 ) and is given by 1 var(ˆµ ˆ ml ) = N i=1 (s2 i + ˆτ (3.23) ml 2 ) 1

28 28 Under the assumption of asymptotic normality we therefore have ( 1 ) ˆµ ml N µ, N i=1 (s2 i + ˆτ ml 2 ) 1 as N (3.24) Restricted maximum likelihood estimate for τ 2 The ML estimator of τ 2 tends to underestimate the population heterogeneity in finite samples by failing to account for the fact that µ in equation (3.22) is also estimated from the data [Harville, 1977]. The REML estimator compensates for this underestimation by using a linear combination of the y vector, so that the transformed data are free of the fixed effects in β. Specifically, let M be equal to (N 1) linearly independent columns of I X(X T X) 1 X T. Then M T y is independent of β in the sense that M T y N(0, M T XM). In fact, we can take any matrix M of full rank, as long as M T X = 0. Then the log-likelihood function is given by logl ( σ 2 y ) = 1 2 log V 1 2 log XT V 1 X 1 2 (y X β) T V 1 (y X β) (3.25) where β is a ML solution of β for fixed σ 2 [Harville, 1977]. For the meta-analytic random-effects model this simplifies to logl(τ 2 y) = 1 2 N log(s 2 i + τ 2 ) 1 N 2 log 1 (s 2 i + τ 2 ) 1 2 i=1 i=1 N i=1 (y i ˆµ ml ) 2 (s 2 i + τ 2 ), τ 2 0 (3.26) The REML estimator of τ 2 is then given by

29 29 ˆτ 2 reml = N (y i ˆµ ml ) 2 s ( ) 2 i i=1 2 s 2 i +ˆτ m 2 N ( 1 i=1 s 2 i +ˆτ m 2 ) 2 + N i=1 ( 1 1 s 2 i +ˆτ 2 m ) 2 (3.27) and is obtained in the same iterative manner as described for the ML estimator. An estimator of µ is then given by ˆµ reml = N y i i=1 s 2 i +ˆτ reml 2 N 1 i=1 s 2 i +ˆτ reml 2 (3.28) With variance estimated by 1 var(ˆµ ˆ reml ) = N i=1 (s2 i + ˆτ (3.29) ml 2 ) 1 Paule and Mandel estimate for τ 2 We can write a generalization of the Q w statistic used by DL using a i = w ip M = 1 τ 2 +σ 2 i : Q w (τ 2 ) = N i=1 (y i y wp M ) 2 τ 2 + σ 2 i (3.30) where y wp M = N i=1 wi P M yi N. Provided that the variances σ1, 2, σ i=1 wi N 2 and τ 2 are known, equation (3.8) P M reduces to E ( Q w (τ 2 ) ) ( N = E a i (y i y wp M ) 2) = N 1 (3.31) i=1

30 30 This is true even when the underlying distributions are not normal, provided that the necessary moments exist. By equating the statistic Q w (τ 2 ) to its expected value N 1, and then substituting σ 2 1,..., σ 2 N with s 2 1,..., s 2 N, we get the PM estimating equation N F (τ 2 ) = w i (y i y wp M ) 2 (N 1) = 0 (3.32) i=1 For unknown variances, σ 2 1,..., σ 2 N are replaced by their estimates s2 1,..., s 2 N. The solution ˆτ 2 P M of the estimating equation F (τ 2 ) = 0 is the PM estimate for τ 2. The function F (τ 2 ) defined in equation (3.32) is strictly decreasing and convex (or concave up) [Kacker, 2004]. The maximum of F (τ 2 ) occurs at τ 2 = 0 and F (τ 2 ) (N 1) as τ 2. When the value of F (τ 2 ) at τ 2 = 0,i.e., F(0), is positive, then by the intermediate value theorem of calculus a value of τ 2 exist for which F (τ 2 ) = 0. Since F (τ 2 ) is strictly decreasing, such τ 2 is unique. Note that when F(0) is negative, equation (3.32) has no positive solution; in that case the estimate ˆτ 2 P M is set to zero. When F(0) = 0 the solution is τ 2 = 0. The solution ˆτ 2 P M of the estimating equation F (τ 2 ) = 0 can be determined through the following algorithm: Start with τ 2 (previous) = 0 or with a number slightly above Calculate the weights ŵ ip M = 1 τ 2 +s 2 i for i = 1,..., N. 2. Calculate the function F (τ 2 ) : If F (τ 2 ) at τ 2 = 0 is negative, set ˆτ 2 P M = 0 If F (τ 2 (previous)) = 0, set ˆτ 2 P M = τ 2 (previous) If F (τ 2 (previous)) > 0, determine the correction:

31 31 τ 2 = N i=1 ŵi P M (yi yw P M )2 (N 1) N i=1 ŵi P M (yi y w P M ) 2 3. The next iterative value of τ 2 is τ 2 (next) = τ 2 (previous) + τ Repeat steps 2 and 3 until F (τ 2 (previous)) = 0. The final value of τ 2 is ˆτ 2 P M [Paule and Mandel, 1982] proposed that this estimate can then be incorporated into the random effects weights giving ŵ i (ˆτ 2 P M ) = 1 ˆτ 2 P M + s2 i (3.33) An estimator of µ is then given by ˆµˆτP M = N i=1 ŵi(ˆτ 2 P M )y i N i=1 ŵi(ˆτ 2 P M ) (3.34) With variance estimated by 1 var(ˆµˆτp ˆ M ) = N i=1 ŵi(ˆτ P 2 M ) (3.35) Sidik-Jonkman and Hartung-Knapp estimate for τ 2 The SJHK method uses the truncated DL estimator for τ 2, ˆτ DL+ 2, but takes into account the uncertainty associated with this estimate by using a different estimate of the variance of ˆµˆτDL and by using the t- distribution for ˆµˆτ instead of the normal distribution. Specifically, SJHK uses the same weights ŵ i (ˆτ 2 DL+) = 1 ˆτ 2 DL+ + s2 i (3.36)

32 32 and the same estimate of µ, as the ones used by DerSimonian and Liard ˆµˆτSJHK = N i=1 ŵi(ˆτ 2 DL+ )y i N i=1 ŵi(ˆτ 2 DL+ ) (3.37) but with variance estimated by var(ˆµˆτsjhk ˆ ) = N i=1 ŵi(ˆτ DL+ 2 )(y i yŵdl )2 (N 1) k i=1 ŵi(ˆτ DL+ 2 ) (3.38) 3.2 Tests and confidence intervals for the overall population treatment effect As a natural continuation of the previous Section,here we present few possible tests and confidence intervals for the overall population treatment effect (µ) which differ in their heterogeneity (τ 2 ) estimation method and as a consequence, in the distribution of the overall population treatment effect. DerSimonian and Laird estimate for τ 2 As mentioned above, the DL heterogeneity estimator is commonly used in the random effects model meta-analysis, and specifically, this is the estimator being used in the Cochrane Collaboration systematic reviews. In hypothesis testing or obtaining confidence intervals for µ using ˆµˆτDL and var(ˆµˆτdl ˆ ), it is a common practice to maintain the assumption of normality for ˆµˆτDL, despite the use of ˆτ 2 DL and s2 i in place of τ 2 and σ 2 i, respectively. The z-test statistic for testing the meta-analysis null hypothesis that the mean effect µ is zero zˆτdl = ˆµˆτDL var(ˆµˆτdl ˆ ) (3.39)

33 33 Using the z-test statistic we calculate the corresponding p-values: For right sided alternative, the p-value is p Rˆτ DL = P r ( Z zˆτdl ) (3.40) For left sided alternative, the p-value is p Ḽ τ DL = P r ( Z zˆτdl ) (3.41) For two-sided alternative, the p-value is pˆτdl = 2 min(p L, p R ) (3.42) The (1 α) confidence interval for µ ˆµˆτDL ± Z 1 α ˆ 2 var(ˆµˆτdl ) (3.43) The main issue arises from the general application of the random effects model as described above is that the variation between true study effects (θ i, for i = 1,..., N) is taken into account via the inclusion of the deviations δ i with variance τ 2. It is however only an estimate of this variance which is added into the

34 34 weights, and the model takes no account of the uncertainty associated with this estimate. In particular the distribution used for ˆµˆτDL is not altered. As shown in Chapter 4, this results in confidence intervals for µ which are narrower on average than they should be. It is common practice to use a t-distribution to account for the error associated with a variance estimate. The t-test statistic for testing the meta-analysis null hypothesis that the mean effect µ is zero tˆτdl = ˆµˆτDL var(ˆµˆτdl ˆ ) (3.44) Using the t-test statistic we calculate the corresponding p-values: For right sided alternative, the p-value is p Rˆτ DL = P r ( T tˆτdl ) (3.45) For left sided alternative, the p-value is p Ḽ τ DL = P r ( T tˆτdl ) (3.46) For two-sided alternative, the p-value is pˆτdl = 2 min(p L, p R ) (3.47) The (1 α) confidence interval for µ

35 35 ˆµˆτDL ± t 1 α 2,N 1 var(ˆµˆτdl ˆ ) (3.48) Maximum likelihood estimate for τ 2 In the case of the ML estimation, one major advantage lies in the large body of asymptotic theory existing for such estimators. In regular cases a ML estimator from a sample of N independent and identically distributed random variables has a normal distribution as N. It should be noted that the N variables y i, i = 1,..., N from a meta-analysis are independent but not identically distributed, since var(y i ) = σi 2 + τ 2. The standard assumptions still apply however, in meta-analysis with large enough N. Using the asymptotic distribution, it is possible to construct a confidence interval for µ. However, this is only an approximate interval, since the asymptotic variance of ˆµ ml depends on the unknown τ 2. The variance of ˆµ ml is of the same form as that for the DerSimonian and Laird random effects method. In this case var(ˆµ ml ) is estimated using ˆτ ml 2 without any modification to the distribution of ˆµ ml. Like the DerSimonian and Laird method, the greatest source of error in simple likelihood confidence intervals comes from estimating var(ˆµ ml ) by substituting ˆτ ml 2 in for τ 2, and using a normal approximation, despite this estimation. The z-test statistic for testing the meta-analysis null hypothesis that the mean effect µ is zero zˆτml = ˆµ ml var(ˆµml ˆ ) (3.49) Using the z-test statistic we calculate the corresponding p-values: For right sided alternative, the p-value is

36 36 p Rˆτ ml = P r ( Z zˆτml ) (3.50) For left sided alternative, the p-value is p Ḽ τ ml = P r ( Z zˆτml ) (3.51) For two-sided alternative, the p-value is pˆτml = 2 min(p L, p R ) (3.52) The (1 α) confidence interval for µ ˆµ ml ± Z 1 α var(ˆµ ˆ 2 ml ) (3.53) Restricted maximum likelihood estimate for τ 2 As in the maximum likelihood case, using the asymptotic normal distribution, it is possible to construct a confidence interval for µ. However, this is only an approximate interval, since the asymptotic variance of ˆµ ml depends on the unknown τ 2. The variance of ˆµ reml is of the same form as that for the ML and the DL random effects method. In this case var(ˆµ reml ) is estimated using ˆτ 2 reml without any modification to the distribution of ˆµ reml. Like the

37 37 DL method, the greatest source of error in REML confidence intervals comes from estimating var(ˆµ reml ) by substituting ˆτ 2 reml in for τ 2, and using a normal approximation, despite this estimation. The z-test statistic for testing the meta-analysis null hypothesis that the mean effect µ is zero zˆτreml = ˆµ reml var(ˆµreml ˆ ) (3.54) Using the z-test statistic we calculate the corresponding p-values: For right sided alternative, the p-value is p Rˆτ reml = P r ( Z zˆτreml ) (3.55) For left sided alternative, the p-value is p Ḽ τ reml = P r ( Z zˆτreml ) (3.56) For two-sided alternative, the p-value is pˆτreml = 2 min(p L, p R ) (3.57) The (1 α) confidence interval for µ

38 38 ˆµ reml ± Z 1 α var(ˆµ ˆ 2 reml (3.58) Paule and Mandel estimate for τ 2 The z-test statistic for testing the meta-analysis null hypothesis that the mean effect µ is zero zˆτp M = ˆµ P M var(ˆµp ˆ M ) (3.59) Using the z-test statistic we calculate the corresponding p-values: For right sided alternative, the p-value is p Rˆτ P M = P r ( Z zˆτp M ) (3.60) For left sided alternative, the p-value is p Ḽ τ P M = P r ( Z zˆτp M ) (3.61) For two-sided alternative, the p-value is pˆτp M = 2 min(p L, p R ) (3.62)

39 39 Using the Paule and Mandel estimation for τ 2, the (1 α) confidence interval for µ is given by ˆµˆτP M ± Z 1 α ˆ 2 var(ˆµˆτp M ) (3.63) Statistical optimality of the PM : [Rukhin et al., 2000] investigated the properties of the PM estimator under normality. In particular, they show that when the normally assumption holds and a weighted mean is used as an estimate for the parameter µ, then the PM estimate ˆτ P 2 M is the conditionally REML of τ 2. A REML estimate of a variance component is an improvement over the ML because it accounts for the loss in degrees of freedom resulting from the estimation of µ [Harville, 1977]. Rukhin et al. [Rukhin et al., 2000] also show that the estimate of µ determined by using ˆτ 2 P M for τ 2 is an approximate generalized Bayes estimate based on non-informative prior distributions for the statistical parameters µ, τ 2, and σ 2 1,, σ 2 N. Thus under normality, ˆτ 2 P M and ˆµˆτ 2 P M for τ 2 and µ, respectively. are statistically optimal estimates Note: Under the normality assumption, the PM s approach is statistically optimal [Liat: in what sense is it optimal?], but the method itself does not require a normality assumption. Hence, when normality assumptions do not hold, the PM method is more robust for estimating τ 2 than the method of DerSimonian and Liard which is based on large sample assumptions. Sidik-Jonkman and Hartung-Knapp estimate for τ 2 In the SJHK estimation method, we use the t-distribution with N 1 degrees of freedom to derive p-values and confidence intervals, where N is the number of studies in the meta-analysis.

40 40 Therefore the t-test statistic for testing the meta-analysis null hypothesis that the mean effect µ is zero tˆτsjhk = ˆµ SJHK var(ˆµsjhk ˆ ) (3.64) Using the t-test statistic we calculate the corresponding p-values: For right sided alternative, the p-value is p Rˆτ SJHK = P r ( T tˆτsjhk ) (3.65) For left sided alternative, the p-value is p Ḽ τ SJHK = P r ( T tˆτsjhk ) (3.66) For two-sided alternative, the p-value is pˆτsjhk = 2 min(p L, p R ) (3.67) The (1 α) confidence interval for µ ˆµˆτSJHK ± t 1 α 2,N 1 var(ˆµˆτsjhk ˆ ) (3.68)

41 Chapter 4 Simulation study We had two main goals in the simulation study. First, to assess the validity of the inference of the meta-analysis overall effect µ using the different methods described in Section 3.2. Second, to assess the validity and power of the replicability analysis based on the different methods of Section 3.2. Specifically, we would like to identify the meta-analysis method which can be incorporated into the replicability analysis (by performing the meta-analysis on all ( N u 1) subsets) to provide valid replicability analysis inference and good power to identify the replicated findings. In the following simulation study we examined different scenarios using different values for the number of studies in the meta-analysis (N) and the variance between these studies (τ 2 ). These values were chosen using the results from [Davey J, et al., 2011] and real data examples taken from Cochrane Collaboration systematic reviews as presented in Section 5. 41

Effect and shrinkage estimation in meta-analyses of two studies

Effect and shrinkage estimation in meta-analyses of two studies Effect and shrinkage estimation in meta-analyses of two studies Christian Röver Department of Medical Statistics, University Medical Center Göttingen, Göttingen, Germany December 2, 2016 This project has

More information

Biostat Methods STAT 5820/6910 Handout #9a: Intro. to Meta-Analysis Methods

Biostat Methods STAT 5820/6910 Handout #9a: Intro. to Meta-Analysis Methods Biostat Methods STAT 5820/6910 Handout #9a: Intro. to Meta-Analysis Methods Meta-analysis describes statistical approach to systematically combine results from multiple studies [identified follong an exhaustive

More information

Prediction intervals for random-effects meta-analysis: a confidence distribution approach

Prediction intervals for random-effects meta-analysis: a confidence distribution approach Preprint / Submitted Version Prediction intervals for random-effects meta-analysis: a confidence distribution approach Kengo Nagashima 1 *, Hisashi Noma 2, and Toshi A. Furukawa 3 Abstract For the inference

More information

Prediction intervals for random-effects meta-analysis: a confidence distribution approach

Prediction intervals for random-effects meta-analysis: a confidence distribution approach Statistical Methods in Medical Research 2018. In press. DOI: 10.1177/0962280218773520 Prediction intervals for random-effects meta-analysis: a confidence distribution approach Kengo Nagashima 1 *, Hisashi

More information

Heterogeneity issues in the meta-analysis of cluster randomization trials.

Heterogeneity issues in the meta-analysis of cluster randomization trials. Western University Scholarship@Western Electronic Thesis and Dissertation Repository June 2012 Heterogeneity issues in the meta-analysis of cluster randomization trials. Shun Fu Chen The University of

More information

Controlling Bayes Directional False Discovery Rate in Random Effects Model 1

Controlling Bayes Directional False Discovery Rate in Random Effects Model 1 Controlling Bayes Directional False Discovery Rate in Random Effects Model 1 Sanat K. Sarkar a, Tianhui Zhou b a Temple University, Philadelphia, PA 19122, USA b Wyeth Pharmaceuticals, Collegeville, PA

More information

Statistical Applications in Genetics and Molecular Biology

Statistical Applications in Genetics and Molecular Biology Statistical Applications in Genetics and Molecular Biology Volume 5, Issue 1 2006 Article 28 A Two-Step Multiple Comparison Procedure for a Large Number of Tests and Multiple Treatments Hongmei Jiang Rebecca

More information

On Estimating Residual Heterogeneity in Random-Effects Meta-Regression: A Comparative Study

On Estimating Residual Heterogeneity in Random-Effects Meta-Regression: A Comparative Study Journal of Statistical Theory and Applications, Vol. 12, No. 3 September 2013), 253-265 On Estimating Residual Heterogeneity in Random-Effects Meta-Regression: A Comparative Study Thammarat Panityaul 1,

More information

Math 423/533: The Main Theoretical Topics

Math 423/533: The Main Theoretical Topics Math 423/533: The Main Theoretical Topics Notation sample size n, data index i number of predictors, p (p = 2 for simple linear regression) y i : response for individual i x i = (x i1,..., x ip ) (1 p)

More information

Standard RE Model. Critique of the RE Model 2016/08/12

Standard RE Model. Critique of the RE Model 2016/08/12 Standard RE Model Alternative weighting schemes in the random-effects model to reduce the impact of publication bias Wolfgang Viechtbauer Maastricht University The Netherlands http://www.wvbauer.com y

More information

Previous lecture. Single variant association. Use genome-wide SNPs to account for confounding (population substructure)

Previous lecture. Single variant association. Use genome-wide SNPs to account for confounding (population substructure) Previous lecture Single variant association Use genome-wide SNPs to account for confounding (population substructure) Estimation of effect size and winner s curse Meta-Analysis Today s outline P-value

More information

A Very Brief Summary of Statistical Inference, and Examples

A Very Brief Summary of Statistical Inference, and Examples A Very Brief Summary of Statistical Inference, and Examples Trinity Term 2008 Prof. Gesine Reinert 1 Data x = x 1, x 2,..., x n, realisations of random variables X 1, X 2,..., X n with distribution (model)

More information

Restricted Maximum Likelihood in Linear Regression and Linear Mixed-Effects Model

Restricted Maximum Likelihood in Linear Regression and Linear Mixed-Effects Model Restricted Maximum Likelihood in Linear Regression and Linear Mixed-Effects Model Xiuming Zhang zhangxiuming@u.nus.edu A*STAR-NUS Clinical Imaging Research Center October, 015 Summary This report derives

More information

Model Selection Tutorial 2: Problems With Using AIC to Select a Subset of Exposures in a Regression Model

Model Selection Tutorial 2: Problems With Using AIC to Select a Subset of Exposures in a Regression Model Model Selection Tutorial 2: Problems With Using AIC to Select a Subset of Exposures in a Regression Model Centre for Molecular, Environmental, Genetic & Analytic (MEGA) Epidemiology School of Population

More information

TESTING FOR HOMOGENEITY IN COMBINING OF TWO-ARMED TRIALS WITH NORMALLY DISTRIBUTED RESPONSES

TESTING FOR HOMOGENEITY IN COMBINING OF TWO-ARMED TRIALS WITH NORMALLY DISTRIBUTED RESPONSES Sankhyā : The Indian Journal of Statistics 2001, Volume 63, Series B, Pt. 3, pp 298-310 TESTING FOR HOMOGENEITY IN COMBINING OF TWO-ARMED TRIALS WITH NORMALLY DISTRIBUTED RESPONSES By JOACHIM HARTUNG and

More information

Likelihood-Based Methods

Likelihood-Based Methods Likelihood-Based Methods Handbook of Spatial Statistics, Chapter 4 Susheela Singh September 22, 2016 OVERVIEW INTRODUCTION MAXIMUM LIKELIHOOD ESTIMATION (ML) RESTRICTED MAXIMUM LIKELIHOOD ESTIMATION (REML)

More information

EPSE 594: Meta-Analysis: Quantitative Research Synthesis

EPSE 594: Meta-Analysis: Quantitative Research Synthesis EPSE 594: Meta-Analysis: Quantitative Research Synthesis Ed Kroc University of British Columbia ed.kroc@ubc.ca January 24, 2019 Ed Kroc (UBC) EPSE 594 January 24, 2019 1 / 37 Last time Composite effect

More information

A simulation study comparing properties of heterogeneity measures in meta-analyses

A simulation study comparing properties of heterogeneity measures in meta-analyses STATISTICS IN MEDICINE Statist. Med. 2006; 25:4321 4333 Published online 21 September 2006 in Wiley InterScience (www.interscience.wiley.com).2692 A simulation study comparing properties of heterogeneity

More information

Post-Selection Inference

Post-Selection Inference Classical Inference start end start Post-Selection Inference selected end model data inference data selection model data inference Post-Selection Inference Todd Kuffner Washington University in St. Louis

More information

Statistics: Learning models from data

Statistics: Learning models from data DS-GA 1002 Lecture notes 5 October 19, 2015 Statistics: Learning models from data Learning models from data that are assumed to be generated probabilistically from a certain unknown distribution is a crucial

More information

Primer on statistics:

Primer on statistics: Primer on statistics: MLE, Confidence Intervals, and Hypothesis Testing ryan.reece@gmail.com http://rreece.github.io/ Insight Data Science - AI Fellows Workshop Feb 16, 018 Outline 1. Maximum likelihood

More information

Meta-analysis. 21 May Per Kragh Andersen, Biostatistics, Dept. Public Health

Meta-analysis. 21 May Per Kragh Andersen, Biostatistics, Dept. Public Health Meta-analysis 21 May 2014 www.biostat.ku.dk/~pka Per Kragh Andersen, Biostatistics, Dept. Public Health pka@biostat.ku.dk 1 Meta-analysis Background: each single study cannot stand alone. This leads to

More information

An accurate test for homogeneity of odds ratios based on Cochran s Q-statistic

An accurate test for homogeneity of odds ratios based on Cochran s Q-statistic Kulinskaya and Dollinger TECHNICAL ADVANCE An accurate test for homogeneity of odds ratios based on Cochran s Q-statistic Elena Kulinskaya 1* and Michael B Dollinger 2 * Correspondence: e.kulinskaya@uea.ac.uk

More information

Extending causal inferences from a randomized trial to a target population

Extending causal inferences from a randomized trial to a target population Extending causal inferences from a randomized trial to a target population Issa Dahabreh Center for Evidence Synthesis in Health, Brown University issa dahabreh@brown.edu January 16, 2019 Issa Dahabreh

More information

Inference for correlated effect sizes using multiple univariate meta-analyses

Inference for correlated effect sizes using multiple univariate meta-analyses Europe PMC Funders Group Author Manuscript Published in final edited form as: Stat Med. 2016 April 30; 35(9): 1405 1422. doi:10.1002/sim.6789. Inference for correlated effect sizes using multiple univariate

More information

CHL 5225H Advanced Statistical Methods for Clinical Trials: Multiplicity

CHL 5225H Advanced Statistical Methods for Clinical Trials: Multiplicity CHL 5225H Advanced Statistical Methods for Clinical Trials: Multiplicity Prof. Kevin E. Thorpe Dept. of Public Health Sciences University of Toronto Objectives 1. Be able to distinguish among the various

More information

Paule Mandel estimators for network meta analysis with random inconsistency effects

Paule Mandel estimators for network meta analysis with random inconsistency effects Paule Mandel estimators for network meta analysis with random inconsistency effects Jackson, D, Veroniki, AH, Law, M, Tricco, AC and Baker, RD http://dx.doi.org/10.1002/jrsm.1244 Title Authors Type URL

More information

Bootstrap Procedures for Testing Homogeneity Hypotheses

Bootstrap Procedures for Testing Homogeneity Hypotheses Journal of Statistical Theory and Applications Volume 11, Number 2, 2012, pp. 183-195 ISSN 1538-7887 Bootstrap Procedures for Testing Homogeneity Hypotheses Bimal Sinha 1, Arvind Shah 2, Dihua Xu 1, Jianxin

More information

Fall 2017 STAT 532 Homework Peter Hoff. 1. Let P be a probability measure on a collection of sets A.

Fall 2017 STAT 532 Homework Peter Hoff. 1. Let P be a probability measure on a collection of sets A. 1. Let P be a probability measure on a collection of sets A. (a) For each n N, let H n be a set in A such that H n H n+1. Show that P (H n ) monotonically converges to P ( k=1 H k) as n. (b) For each n

More information

Robustness and Distribution Assumptions

Robustness and Distribution Assumptions Chapter 1 Robustness and Distribution Assumptions 1.1 Introduction In statistics, one often works with model assumptions, i.e., one assumes that data follow a certain model. Then one makes use of methodology

More information

Inferences about Parameters of Trivariate Normal Distribution with Missing Data

Inferences about Parameters of Trivariate Normal Distribution with Missing Data Florida International University FIU Digital Commons FIU Electronic Theses and Dissertations University Graduate School 7-5-3 Inferences about Parameters of Trivariate Normal Distribution with Missing

More information

Multivariate Statistical Analysis

Multivariate Statistical Analysis Multivariate Statistical Analysis Fall 2011 C. L. Williams, Ph.D. Lecture 9 for Applied Multivariate Analysis Outline Addressing ourliers 1 Addressing ourliers 2 Outliers in Multivariate samples (1) For

More information

Stat/F&W Ecol/Hort 572 Review Points Ané, Spring 2010

Stat/F&W Ecol/Hort 572 Review Points Ané, Spring 2010 1 Linear models Y = Xβ + ɛ with ɛ N (0, σ 2 e) or Y N (Xβ, σ 2 e) where the model matrix X contains the information on predictors and β includes all coefficients (intercept, slope(s) etc.). 1. Number of

More information

This paper has been submitted for consideration for publication in Biometrics

This paper has been submitted for consideration for publication in Biometrics BIOMETRICS, 1 10 Supplementary material for Control with Pseudo-Gatekeeping Based on a Possibly Data Driven er of the Hypotheses A. Farcomeni Department of Public Health and Infectious Diseases Sapienza

More information

So far our focus has been on estimation of the parameter vector β in the. y = Xβ + u

So far our focus has been on estimation of the parameter vector β in the. y = Xβ + u Interval estimation and hypothesis tests So far our focus has been on estimation of the parameter vector β in the linear model y i = β 1 x 1i + β 2 x 2i +... + β K x Ki + u i = x iβ + u i for i = 1, 2,...,

More information

Statistics 3858 : Maximum Likelihood Estimators

Statistics 3858 : Maximum Likelihood Estimators Statistics 3858 : Maximum Likelihood Estimators 1 Method of Maximum Likelihood In this method we construct the so called likelihood function, that is L(θ) = L(θ; X 1, X 2,..., X n ) = f n (X 1, X 2,...,

More information

PQL Estimation Biases in Generalized Linear Mixed Models

PQL Estimation Biases in Generalized Linear Mixed Models PQL Estimation Biases in Generalized Linear Mixed Models Woncheol Jang Johan Lim March 18, 2006 Abstract The penalized quasi-likelihood (PQL) approach is the most common estimation procedure for the generalized

More information

Maximum-Likelihood Estimation: Basic Ideas

Maximum-Likelihood Estimation: Basic Ideas Sociology 740 John Fox Lecture Notes Maximum-Likelihood Estimation: Basic Ideas Copyright 2014 by John Fox Maximum-Likelihood Estimation: Basic Ideas 1 I The method of maximum likelihood provides estimators

More information

Probability and Statistics Notes

Probability and Statistics Notes Probability and Statistics Notes Chapter Seven Jesse Crawford Department of Mathematics Tarleton State University Spring 2011 (Tarleton State University) Chapter Seven Notes Spring 2011 1 / 42 Outline

More information

A simulation study for comparing testing statistics in response-adaptive randomization

A simulation study for comparing testing statistics in response-adaptive randomization RESEARCH ARTICLE Open Access A simulation study for comparing testing statistics in response-adaptive randomization Xuemin Gu 1, J Jack Lee 2* Abstract Background: Response-adaptive randomizations are

More information

Data Mining Chapter 4: Data Analysis and Uncertainty Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University

Data Mining Chapter 4: Data Analysis and Uncertainty Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University Data Mining Chapter 4: Data Analysis and Uncertainty Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University Why uncertainty? Why should data mining care about uncertainty? We

More information

Practice Problems Section Problems

Practice Problems Section Problems Practice Problems Section 4-4-3 4-4 4-5 4-6 4-7 4-8 4-10 Supplemental Problems 4-1 to 4-9 4-13, 14, 15, 17, 19, 0 4-3, 34, 36, 38 4-47, 49, 5, 54, 55 4-59, 60, 63 4-66, 68, 69, 70, 74 4-79, 81, 84 4-85,

More information

Lawrence D. Brown* and Daniel McCarthy*

Lawrence D. Brown* and Daniel McCarthy* Comments on the paper, An adaptive resampling test for detecting the presence of significant predictors by I. W. McKeague and M. Qian Lawrence D. Brown* and Daniel McCarthy* ABSTRACT: This commentary deals

More information

Econometrics. Week 4. Fall Institute of Economic Studies Faculty of Social Sciences Charles University in Prague

Econometrics. Week 4. Fall Institute of Economic Studies Faculty of Social Sciences Charles University in Prague Econometrics Week 4 Institute of Economic Studies Faculty of Social Sciences Charles University in Prague Fall 2012 1 / 23 Recommended Reading For the today Serial correlation and heteroskedasticity in

More information

A CONDITIONALLY-UNBIASED ESTIMATOR OF POPULATION SIZE BASED ON PLANT-CAPTURE IN CONTINUOUS TIME. I.B.J. Goudie and J. Ashbridge

A CONDITIONALLY-UNBIASED ESTIMATOR OF POPULATION SIZE BASED ON PLANT-CAPTURE IN CONTINUOUS TIME. I.B.J. Goudie and J. Ashbridge This is an electronic version of an article published in Communications in Statistics Theory and Methods, 1532-415X, Volume 29, Issue 11, 2000, Pages 2605-2619. The published article is available online

More information

Meta-analysis of epidemiological dose-response studies

Meta-analysis of epidemiological dose-response studies Meta-analysis of epidemiological dose-response studies Nicola Orsini 2nd Italian Stata Users Group meeting October 10-11, 2005 Institute Environmental Medicine, Karolinska Institutet Rino Bellocco Dept.

More information

Multilevel Models in Matrix Form. Lecture 7 July 27, 2011 Advanced Multivariate Statistical Methods ICPSR Summer Session #2

Multilevel Models in Matrix Form. Lecture 7 July 27, 2011 Advanced Multivariate Statistical Methods ICPSR Summer Session #2 Multilevel Models in Matrix Form Lecture 7 July 27, 2011 Advanced Multivariate Statistical Methods ICPSR Summer Session #2 Today s Lecture Linear models from a matrix perspective An example of how to do

More information

Measurement And Uncertainty

Measurement And Uncertainty Measurement And Uncertainty Based on Guidelines for Evaluating and Expressing the Uncertainty of NIST Measurement Results, NIST Technical Note 1297, 1994 Edition PHYS 407 1 Measurement approximates or

More information

Estimation in Generalized Linear Models with Heterogeneous Random Effects. Woncheol Jang Johan Lim. May 19, 2004

Estimation in Generalized Linear Models with Heterogeneous Random Effects. Woncheol Jang Johan Lim. May 19, 2004 Estimation in Generalized Linear Models with Heterogeneous Random Effects Woncheol Jang Johan Lim May 19, 2004 Abstract The penalized quasi-likelihood (PQL) approach is the most common estimation procedure

More information

A Very Brief Summary of Bayesian Inference, and Examples

A Very Brief Summary of Bayesian Inference, and Examples A Very Brief Summary of Bayesian Inference, and Examples Trinity Term 009 Prof Gesine Reinert Our starting point are data x = x 1, x,, x n, which we view as realisations of random variables X 1, X,, X

More information

Lecturer: Dr. Adote Anum, Dept. of Psychology Contact Information:

Lecturer: Dr. Adote Anum, Dept. of Psychology Contact Information: Lecturer: Dr. Adote Anum, Dept. of Psychology Contact Information: aanum@ug.edu.gh College of Education School of Continuing and Distance Education 2014/2015 2016/2017 Session Overview In this Session

More information

1 Mixed effect models and longitudinal data analysis

1 Mixed effect models and longitudinal data analysis 1 Mixed effect models and longitudinal data analysis Mixed effects models provide a flexible approach to any situation where data have a grouping structure which introduces some kind of correlation between

More information

MFM Practitioner Module: Risk & Asset Allocation. John Dodson. February 18, 2015

MFM Practitioner Module: Risk & Asset Allocation. John Dodson. February 18, 2015 MFM Practitioner Module: Risk & Asset Allocation February 18, 2015 No introduction to portfolio optimization would be complete without acknowledging the significant contribution of the Markowitz mean-variance

More information

Richard D Riley was supported by funding from a multivariate meta-analysis grant from

Richard D Riley was supported by funding from a multivariate meta-analysis grant from Bayesian bivariate meta-analysis of correlated effects: impact of the prior distributions on the between-study correlation, borrowing of strength, and joint inferences Author affiliations Danielle L Burke

More information

Sample size determination for a binary response in a superiority clinical trial using a hybrid classical and Bayesian procedure

Sample size determination for a binary response in a superiority clinical trial using a hybrid classical and Bayesian procedure Ciarleglio and Arendt Trials (2017) 18:83 DOI 10.1186/s13063-017-1791-0 METHODOLOGY Open Access Sample size determination for a binary response in a superiority clinical trial using a hybrid classical

More information

Stat 206: Estimation and testing for a mean vector,

Stat 206: Estimation and testing for a mean vector, Stat 206: Estimation and testing for a mean vector, Part II James Johndrow 2016-12-03 Comparing components of the mean vector In the last part, we talked about testing the hypothesis H 0 : µ 1 = µ 2 where

More information

Lec 3: Model Adequacy Checking

Lec 3: Model Adequacy Checking November 16, 2011 Model validation Model validation is a very important step in the model building procedure. (one of the most overlooked) A high R 2 value does not guarantee that the model fits the data

More information

EXAMINATION: QUANTITATIVE EMPIRICAL METHODS. Yale University. Department of Political Science

EXAMINATION: QUANTITATIVE EMPIRICAL METHODS. Yale University. Department of Political Science EXAMINATION: QUANTITATIVE EMPIRICAL METHODS Yale University Department of Political Science January 2014 You have seven hours (and fifteen minutes) to complete the exam. You can use the points assigned

More information

arxiv: v1 [stat.me] 28 Jul 2017

arxiv: v1 [stat.me] 28 Jul 2017 Sensitivity Analysis for Unmeasured Confounding in Meta-Analyses Maya B. Mathur 1,2 and Tyler J. VanderWeele 1,3 arxiv:1707.09076v1 [stat.me] 28 Jul 2017 1 Department of Biostatistics, Harvard T. H. Chan

More information

ISyE 691 Data mining and analytics

ISyE 691 Data mining and analytics ISyE 691 Data mining and analytics Regression Instructor: Prof. Kaibo Liu Department of Industrial and Systems Engineering UW-Madison Email: kliu8@wisc.edu Office: Room 3017 (Mechanical Engineering Building)

More information

Empirical Bayes Moderation of Asymptotically Linear Parameters

Empirical Bayes Moderation of Asymptotically Linear Parameters Empirical Bayes Moderation of Asymptotically Linear Parameters Nima Hejazi Division of Biostatistics University of California, Berkeley stat.berkeley.edu/~nhejazi nimahejazi.org twitter/@nshejazi github/nhejazi

More information

Bayesian vs frequentist techniques for the analysis of binary outcome data

Bayesian vs frequentist techniques for the analysis of binary outcome data 1 Bayesian vs frequentist techniques for the analysis of binary outcome data By M. Stapleton Abstract We compare Bayesian and frequentist techniques for analysing binary outcome data. Such data are commonly

More information

Basic Statistics. 1. Gross error analyst makes a gross mistake (misread balance or entered wrong value into calculation).

Basic Statistics. 1. Gross error analyst makes a gross mistake (misread balance or entered wrong value into calculation). Basic Statistics There are three types of error: 1. Gross error analyst makes a gross mistake (misread balance or entered wrong value into calculation). 2. Systematic error - always too high or too low

More information

Pairwise rank based likelihood for estimating the relationship between two homogeneous populations and their mixture proportion

Pairwise rank based likelihood for estimating the relationship between two homogeneous populations and their mixture proportion Pairwise rank based likelihood for estimating the relationship between two homogeneous populations and their mixture proportion Glenn Heller and Jing Qin Department of Epidemiology and Biostatistics Memorial

More information

1/24/2008. Review of Statistical Inference. C.1 A Sample of Data. C.2 An Econometric Model. C.4 Estimating the Population Variance and Other Moments

1/24/2008. Review of Statistical Inference. C.1 A Sample of Data. C.2 An Econometric Model. C.4 Estimating the Population Variance and Other Moments /4/008 Review of Statistical Inference Prepared by Vera Tabakova, East Carolina University C. A Sample of Data C. An Econometric Model C.3 Estimating the Mean of a Population C.4 Estimating the Population

More information

Person-Time Data. Incidence. Cumulative Incidence: Example. Cumulative Incidence. Person-Time Data. Person-Time Data

Person-Time Data. Incidence. Cumulative Incidence: Example. Cumulative Incidence. Person-Time Data. Person-Time Data Person-Time Data CF Jeff Lin, MD., PhD. Incidence 1. Cumulative incidence (incidence proportion) 2. Incidence density (incidence rate) December 14, 2005 c Jeff Lin, MD., PhD. c Jeff Lin, MD., PhD. Person-Time

More information

In Defence of Score Intervals for Proportions and their Differences

In Defence of Score Intervals for Proportions and their Differences In Defence of Score Intervals for Proportions and their Differences Robert G. Newcombe a ; Markku M. Nurminen b a Department of Primary Care & Public Health, Cardiff University, Cardiff, United Kingdom

More information

More on Unsupervised Learning

More on Unsupervised Learning More on Unsupervised Learning Two types of problems are to find association rules for occurrences in common in observations (market basket analysis), and finding the groups of values of observational data

More information

MA 575 Linear Models: Cedric E. Ginestet, Boston University Non-parametric Inference, Polynomial Regression Week 9, Lecture 2

MA 575 Linear Models: Cedric E. Ginestet, Boston University Non-parametric Inference, Polynomial Regression Week 9, Lecture 2 MA 575 Linear Models: Cedric E. Ginestet, Boston University Non-parametric Inference, Polynomial Regression Week 9, Lecture 2 1 Bootstrapped Bias and CIs Given a multiple regression model with mean and

More information

EXAMINATIONS OF THE ROYAL STATISTICAL SOCIETY

EXAMINATIONS OF THE ROYAL STATISTICAL SOCIETY EXAMINATIONS OF THE ROYAL STATISTICAL SOCIETY GRADUATE DIPLOMA, 00 MODULE : Statistical Inference Time Allowed: Three Hours Candidates should answer FIVE questions. All questions carry equal marks. The

More information

A matrix-based method of moments for fitting the multivariate random effects model for meta-analysis and meta-regression

A matrix-based method of moments for fitting the multivariate random effects model for meta-analysis and meta-regression Biometrical Journal 55 (2013) 2, 231 245 DOI: 10.1002/bimj.201200152 231 A matrix-based method of moments for fitting the multivariate random effects model for meta-analysis and meta-regression Dan Jackson,1,

More information

Statistical Models with Uncertain Error Parameters (G. Cowan, arxiv: )

Statistical Models with Uncertain Error Parameters (G. Cowan, arxiv: ) Statistical Models with Uncertain Error Parameters (G. Cowan, arxiv:1809.05778) Workshop on Advanced Statistics for Physics Discovery aspd.stat.unipd.it Department of Statistical Sciences, University of

More information

Bias Variance Trade-off

Bias Variance Trade-off Bias Variance Trade-off The mean squared error of an estimator MSE(ˆθ) = E([ˆθ θ] 2 ) Can be re-expressed MSE(ˆθ) = Var(ˆθ) + (B(ˆθ) 2 ) MSE = VAR + BIAS 2 Proof MSE(ˆθ) = E((ˆθ θ) 2 ) = E(([ˆθ E(ˆθ)]

More information

MULTISTAGE AND MIXTURE PARALLEL GATEKEEPING PROCEDURES IN CLINICAL TRIALS

MULTISTAGE AND MIXTURE PARALLEL GATEKEEPING PROCEDURES IN CLINICAL TRIALS Journal of Biopharmaceutical Statistics, 21: 726 747, 2011 Copyright Taylor & Francis Group, LLC ISSN: 1054-3406 print/1520-5711 online DOI: 10.1080/10543406.2011.551333 MULTISTAGE AND MIXTURE PARALLEL

More information

Discrete Mathematics and Probability Theory Spring 2016 Rao and Walrand Note 14

Discrete Mathematics and Probability Theory Spring 2016 Rao and Walrand Note 14 CS 70 Discrete Mathematics and Probability Theory Spring 2016 Rao and Walrand Note 14 Introduction One of the key properties of coin flips is independence: if you flip a fair coin ten times and get ten

More information

Part III. A Decision-Theoretic Approach and Bayesian testing

Part III. A Decision-Theoretic Approach and Bayesian testing Part III A Decision-Theoretic Approach and Bayesian testing 1 Chapter 10 Bayesian Inference as a Decision Problem The decision-theoretic framework starts with the following situation. We would like to

More information

Beta-binomial model for meta-analysis of odds ratios

Beta-binomial model for meta-analysis of odds ratios Research Article Received 9 June 2016, Accepted 3 January 2017 Published online in Wiley Online Library (wileyonlinelibrary.com) DOI: 10.1002/sim.7233 Beta-binomial model for meta-analysis of odds ratios

More information

One-stage dose-response meta-analysis

One-stage dose-response meta-analysis One-stage dose-response meta-analysis Nicola Orsini, Alessio Crippa Biostatistics Team Department of Public Health Sciences Karolinska Institutet http://ki.se/en/phs/biostatistics-team 2017 Nordic and

More information

Empirical Bayes Moderation of Asymptotically Linear Parameters

Empirical Bayes Moderation of Asymptotically Linear Parameters Empirical Bayes Moderation of Asymptotically Linear Parameters Nima Hejazi Division of Biostatistics University of California, Berkeley stat.berkeley.edu/~nhejazi nimahejazi.org twitter/@nshejazi github/nhejazi

More information

Robust Monte Carlo Methods for Sequential Planning and Decision Making

Robust Monte Carlo Methods for Sequential Planning and Decision Making Robust Monte Carlo Methods for Sequential Planning and Decision Making Sue Zheng, Jason Pacheco, & John Fisher Sensing, Learning, & Inference Group Computer Science & Artificial Intelligence Laboratory

More information

Interval estimation. October 3, Basic ideas CLT and CI CI for a population mean CI for a population proportion CI for a Normal mean

Interval estimation. October 3, Basic ideas CLT and CI CI for a population mean CI for a population proportion CI for a Normal mean Interval estimation October 3, 2018 STAT 151 Class 7 Slide 1 Pandemic data Treatment outcome, X, from n = 100 patients in a pandemic: 1 = recovered and 0 = not recovered 1 1 1 0 0 0 1 1 1 0 0 1 0 1 0 0

More information

A new strategy for meta-analysis of continuous covariates in observational studies with IPD. Willi Sauerbrei & Patrick Royston

A new strategy for meta-analysis of continuous covariates in observational studies with IPD. Willi Sauerbrei & Patrick Royston A new strategy for meta-analysis of continuous covariates in observational studies with IPD Willi Sauerbrei & Patrick Royston Overview Motivation Continuous variables functional form Fractional polynomials

More information

Recall the Basics of Hypothesis Testing

Recall the Basics of Hypothesis Testing Recall the Basics of Hypothesis Testing The level of significance α, (size of test) is defined as the probability of X falling in w (rejecting H 0 ) when H 0 is true: P(X w H 0 ) = α. H 0 TRUE H 1 TRUE

More information

The square root rule for adaptive importance sampling

The square root rule for adaptive importance sampling The square root rule for adaptive importance sampling Art B. Owen Stanford University Yi Zhou January 2019 Abstract In adaptive importance sampling, and other contexts, we have unbiased and uncorrelated

More information

Florida State University Libraries

Florida State University Libraries Florida State University Libraries Electronic Theses, Treatises and Dissertations The Graduate School 2011 Individual Patient-Level Data Meta- Analysis: A Comparison of Methods for the Diverse Populations

More information

Regression Clustering

Regression Clustering Regression Clustering In regression clustering, we assume a model of the form y = f g (x, θ g ) + ɛ g for observations y and x in the g th group. Usually, of course, we assume linear models of the form

More information

Wald s theorem and the Asimov data set

Wald s theorem and the Asimov data set Wald s theorem and the Asimov data set Eilam Gross & Ofer Vitells ATLAS statistics forum, Dec. 009 1 Outline We have previously guessed that the median significance of many toy MC experiments could be

More information

Parametric Techniques Lecture 3

Parametric Techniques Lecture 3 Parametric Techniques Lecture 3 Jason Corso SUNY at Buffalo 22 January 2009 J. Corso (SUNY at Buffalo) Parametric Techniques Lecture 3 22 January 2009 1 / 39 Introduction In Lecture 2, we learned how to

More information

STATISTICS 4, S4 (4769) A2

STATISTICS 4, S4 (4769) A2 (4769) A2 Objectives To provide students with the opportunity to explore ideas in more advanced statistics to a greater depth. Assessment Examination (72 marks) 1 hour 30 minutes There are four options

More information

Computationally Efficient Estimation of Multilevel High-Dimensional Latent Variable Models

Computationally Efficient Estimation of Multilevel High-Dimensional Latent Variable Models Computationally Efficient Estimation of Multilevel High-Dimensional Latent Variable Models Tihomir Asparouhov 1, Bengt Muthen 2 Muthen & Muthen 1 UCLA 2 Abstract Multilevel analysis often leads to modeling

More information

Chapter 3 ANALYSIS OF RESPONSE PROFILES

Chapter 3 ANALYSIS OF RESPONSE PROFILES Chapter 3 ANALYSIS OF RESPONSE PROFILES 78 31 Introduction In this chapter we present a method for analysing longitudinal data that imposes minimal structure or restrictions on the mean responses over

More information

Gauge Plots. Gauge Plots JAPANESE BEETLE DATA MAXIMUM LIKELIHOOD FOR SPATIALLY CORRELATED DISCRETE DATA JAPANESE BEETLE DATA

Gauge Plots. Gauge Plots JAPANESE BEETLE DATA MAXIMUM LIKELIHOOD FOR SPATIALLY CORRELATED DISCRETE DATA JAPANESE BEETLE DATA JAPANESE BEETLE DATA 6 MAXIMUM LIKELIHOOD FOR SPATIALLY CORRELATED DISCRETE DATA Gauge Plots TuscaroraLisa Central Madsen Fairways, 996 January 9, 7 Grubs Adult Activity Grub Counts 6 8 Organic Matter

More information

Quantitative Introduction ro Risk and Uncertainty in Business Module 5: Hypothesis Testing

Quantitative Introduction ro Risk and Uncertainty in Business Module 5: Hypothesis Testing Quantitative Introduction ro Risk and Uncertainty in Business Module 5: Hypothesis Testing M. Vidyasagar Cecil & Ida Green Chair The University of Texas at Dallas Email: M.Vidyasagar@utdallas.edu October

More information

Classification. Chapter Introduction. 6.2 The Bayes classifier

Classification. Chapter Introduction. 6.2 The Bayes classifier Chapter 6 Classification 6.1 Introduction Often encountered in applications is the situation where the response variable Y takes values in a finite set of labels. For example, the response Y could encode

More information

Lecture 6 April

Lecture 6 April Stats 300C: Theory of Statistics Spring 2017 Lecture 6 April 14 2017 Prof. Emmanuel Candes Scribe: S. Wager, E. Candes 1 Outline Agenda: From global testing to multiple testing 1. Testing the global null

More information

For more information about how to cite these materials visit

For more information about how to cite these materials visit Author(s): Kerby Shedden, Ph.D., 2010 License: Unless otherwise noted, this material is made available under the terms of the Creative Commons Attribution Share Alike 3.0 License: http://creativecommons.org/licenses/by-sa/3.0/

More information

Nonconvex penalties: Signal-to-noise ratio and algorithms

Nonconvex penalties: Signal-to-noise ratio and algorithms Nonconvex penalties: Signal-to-noise ratio and algorithms Patrick Breheny March 21 Patrick Breheny High-Dimensional Data Analysis (BIOS 7600) 1/22 Introduction In today s lecture, we will return to nonconvex

More information

BAYESIAN ESTIMATION OF LINEAR STATISTICAL MODEL BIAS

BAYESIAN ESTIMATION OF LINEAR STATISTICAL MODEL BIAS BAYESIAN ESTIMATION OF LINEAR STATISTICAL MODEL BIAS Andrew A. Neath 1 and Joseph E. Cavanaugh 1 Department of Mathematics and Statistics, Southern Illinois University, Edwardsville, Illinois 606, USA

More information

On Procedures Controlling the FDR for Testing Hierarchically Ordered Hypotheses

On Procedures Controlling the FDR for Testing Hierarchically Ordered Hypotheses On Procedures Controlling the FDR for Testing Hierarchically Ordered Hypotheses Gavin Lynch Catchpoint Systems, Inc., 228 Park Ave S 28080 New York, NY 10003, U.S.A. Wenge Guo Department of Mathematical

More information

arxiv: v1 [stat.me] 25 Dec 2016

arxiv: v1 [stat.me] 25 Dec 2016 Adaptive multigroup confidence intervals with constant coverage Chaoyu Yu 1 and Peter D. Hoff 2 1 Department of Biostatistics, University of Washington-Seattle arxiv:1612.08287v1 [stat.me] 25 Dec 2016

More information