Statistical considerations in indirect comparisons and network meta-analysis

Transcription

1 Statistical considerations in indirect comparisons and network meta-analysis Said Business School, Oxford, UK March 18-19, 013 Cochrane Comparing Multiple Interventions Methods Group Oxford Training event, March 013 1

2 Handout S8-L Problems introduced by multi-arm trials: full network meta-analysis Georgia Salanti Department of Hygiene and Epidemiology University of Ioannina School of Medicine Cochrane Comparing Multiple Interventions Methods Group Oxford Training event, March 013

3 Fluoride data No. studies Gel (G) Rinse (R) Varnish (V) Toothpaste(T) Placebo (P) Nothing (N) Salanti et al, 009 3

4 Network meta-analysis Simultaneous comparison of multiple treatments in a single analysis y i : the relative effect (SMD here) between the treatments compared in study i with variance v i We define a set of basic parameters and specify the rest as functional parameters y μ δ ε i i i μ μ μ PB PA A,B μ,μ,μ,μ,μ PN PT PR PV PG N,T,R,V,G δ i~n(0,τ ) ε ~N(0,v ) i i estimates s i Assumptions in *every* meta-analysis: y i s are independent 4

5 Fluoride data No. studies Gel (G) Rinse (R) Varnish (V) Toothpaste(T) Placebo (P) Nothing (N) Salanti et al, 009 5

6 Multi-arm studies: a significant problem? Chan et al examined a representative sample of 519 randomized trials (those published in December 000, and listed in PubMed) Number of arms All trials (n = 519) Parallel group trials (n = 381) 379 (73%) 86 (75%) 3 85 (16%) 60 (16%) 4 37 (7%) 5 (7%) > 4 18 (3%) 1 (3%) Approximately a quarter of randomized trials have multiple arms Chan et al, 005 6

7 Multi-arm studies They introduce dependent observations y i are not independent when they refer to the same study From an A, B, P study we need to extract two y i s: AP and BP We need to model them properly using multivariate methods! 7

8 Multivariate methods Simultaneous (joint) analysis of more than one variable Why: because they are correlated! Example: systolic (S) and diastolic (D) blood pressure Univariate fixed effects: y ~N(μ,σ ) i,s S i,s S y ~N(μ,σ ) i,d D i,d D Multivariate fixed: y μ i,s S i,s S y ~ N, y i,d μd cov y σ i,d D yi,s μs σ 0 i,s S ~ N, y i,d μd 0 σ i,d D σ cov 8

9 Multivariate fixed effects: y μ σ cov i,s S i,s S y ~ N, y i,d μd cov y σ i,d D Multivariate methods FE Multivariate fixed effects (equivalent): yi,s μ ε S i,s = + y i,d μ ε D i,d εi,s 0 σ cov S y i,s ~N, ε i,d 0 cov y σ i,d D Within study covariance 9

10 Multivariate methods RE Multivariate random effects: yi,s μs δi,s εi,s = + + y i,d μ δi,d ε D i,d ε cov i,s 0 σi,s S y ~N, ε i,d 0 cov y σ i,d D δ 0 τ cov i,s S δ ~N, δ i,d 0 cov τ δ D Within study covariance Between studies covariance 10

11 Network meta-analysis: Simple example Study No. arms No. effects Data Contrast i=1 T 1 = 1 y 1,1, v 1,1 i= T = 1 y,1, v,1 AC i=3 T 3 = 1 y 3,1, v 3,1 BC Let us choose basic parameters to represent and AC y μ δ ε 1,1 ΑΒ 1,1 1,1 y μ δ ε,1 ΑC,1,1 y μ μ δ ε 3,1 ΑC ΑB 3,1 3,1 ε ~N(0,v ) 1,1 1,1 δ ~N(0,τ ) 1,1 11

12 Network meta-analysis: Simple example Study No. arms No. effects Data Contrast i=1 T 1 = 1 y 1,1, v 1,1 i= T = 1 y,1, v,1 AC i=3 T 3 = 1 y 3,1, v 3,1 BC y1,1 1 0 δ1,1 ε1,1 μ ΑΒ y,1 0 1 δ,1 ε,1 μ ΑC y 3,1 1 1 δ 3,1 ε 3,1 δ1,1 0 ταβ 0 0 δ,1 ~ N 0, 0 τac 0 δ 3, τ BC y1,1 μαβ δ1,1 y= Xμ ε1,1 + δ+ ε y μ δ ε,1 ΑC δ,1 ~Ν(0,diag(τ,1 )) y3,1 μαc μαb ε~ν(0,diag(v δ3,1 ε3,1 )) i,j ε1,1 0 v1,1 0 0 ε,1 ~ N 0, 0 v,1 0 ε 3, v 3,1 1

13 Network meta-analysis: Simple example Study No. arms No. effects Data Contrast i=1 T 1 = 1 y 1,1, v 1,1 i= T = 1 y,1, v,1 AC i=3 T 3 = 1 y 3,1, v 3,1 BC i=4 T 4 =3 y 4,1, v 4,1 y 4,, v 4, y1,1 1 0 δ1,1 ε1,1 y,1 0 1 δ,1 ε,1 μ y 3,1 = δ3,1 + ε3,1 μac y4,1 1 0 δ4,1 ε4,1 y4, 0 1 δ4, ε4, AC These two estimates are correlated 13

14 Multi-arm studies If we model treatment effect estimates, these are correlated if they include the same treatment A B C y 4,1 y 4, In random-effects network meta-analyses, treatment effect parameters may be correlated when they come from the same study A B C δ 4,1 δ 4,1 14

15 Simple example Study No. arms No. effects Data Contrast i=1 T 1 = 1 y 1,1, v 1,1 i= T = 1 y,1, v,1 AC i=3 T 3 = 1 y 3,1, v 3,1 BC i=4 T 4 =3 y 4,1, v 4,1 y 4,, v 4, cov(y 4,1, y 4, ) AC So we introduce their covariance 15

16 Study No. arms No. effects Data Contrast i=1 T 1 = 1 y 1,1, v 1,1 i= T = 1 y,1, v,1 AC i=3 T 3 = 1 y 3,1, v 3,1 BC y i=4 4,1, v 4,1 T 4 =3 y 4,, v 4, AC cov(y 4,1, y 4, ) y1,1 1 0 δ1,1 ε1,1 y,1 0 1 δ,1 ε,1 μ y 3,1 = δ3,1 + ε3,1 μac y4,1 1 0 δ4,1 ε4,1 y4, 0 1 δ4, ε4, Multivariate meta-regression approach The within-study variance-covariance matrix ε1,1 0 v1, ε,1 0 0 v, ε 3,1 ~N 0, 0 0 v3,1 0 0 ε4, v4,1 covy 4,1,y4, ε4, cov y 4,1,y4, v4, 16

17 Study No. arms No. effects Data Contrast i=1 T 1 = 1 y 1,1, v 1,1 i= T = 1 y,1, v,1 AC i=3 T 3 = 1 y 3,1, v 3,1 BC y i=4 4,1, v 4,1 T 4 =3 y 4,, v 4, AC cov(y 4,1, y 4, ) y1,1 1 0 δ1,1 ε1,1 y,1 0 1 δ,1 ε,1 μ y 3,1 = δ3,1 + ε3,1 μac y4,1 1 0 δ4,1 ε4,1 y4, 0 1 δ4, ε4, Multivariate meta-regression approach The between-studιes variance-covariance matrix δ 1,1 0 τ δ,1 0 0 τac δ 3,1 ~N 0, 0 0 τbc 0 0 δ4, τ covδ 4,1,δ4, δ4, cov δ 4,1,δ4, τac 17

18 Network meta-analysis and multivariate approaches We can look at network meta-analysis as either a multivariate meta-regression or a multivariate meta-analysis Multivariate meta-regression (what we had so far): extends the meta-regression approach we saw earlier to allow for multi-arm trials dummy 1, -1 and 0 codes for treatments (with a reference in mind) assumes a common heterogeneity variance Involves the covariances in the random errors and random effects Multivariate meta-analysis: no covariates required requires a common reference arm for every study a problem that is surmountable using data augmentation (see later) 18

19 Study No. arms No. effects Data Contrast i=1 T 1 = 1 y 1,1, v 1,1 i= T = 1 y,1, v,1 AC i=3 T 3 =3 i=4 T 4 =3 y 3,1, v 3,1 y 3,, v 3, cov(y 3,1, y 3, ) y 4,1, v 4,1 y 4,, v 4, cov(y 4,1, y 4, ) Multivariate meta-analysis approach y =μ +δ +ε 1,1 1,1 1,1 AC AC y,1=μ AC +δ,1 +ε,1 y3,1 μ δ3,1 ε 3,1 = + + y3, μac δ3, ε3, y4,1 μ δ4,1 ε 4,1 = + + y4, μac δ4, ε4, δ 3,1 0 ταβ cov(δ 3,1,δ 3,) ~Ν, δ 3, 0 cov(δ 3,1,δ 3,) ταc ε 3,1 0 v3,1 cov y 4,1,y4, ~Ν, ε 3, 0 cov y 4,1,y4, v3,1 19

20 We can write y1,1 1 0μ δ1,1 ε1,1 as = μ AC.. y,1 0 1μ δ,1 ε,1 = μ AC.. y3,1-1 1μ δ3,1 ε3,1 = μ AC.. y4,1 1 0μ δ4,1 ε 4,1 = + + y4, 0 1μAC δ4, ε4, y1,1 1 0 δ1,1 ε1,1 y,1 0 1 δ,1 ε,1 μ y 3,1 = δ3,1 + ε3,1 μac y4,1 1 0 δ4,1 ε4,1 y4, 0 1 δ4, ε4, Multivariate metaanalysis Multivariate metaregression y = Xμ + δ+ ε 0

21 Assumption of consistency Multivariate meta-regression: The consistency equations are explicitly written in the design matrix X Multivariate meta-analysis: The consistency assumption is implicit when we impute the missing arm Transitivity: the missing arm is missing at random 1

22 WITHIN-STUDY CORRELATION WITHIN-STUDY COVARIANCE

23 Estimating within-study covariance (1) The covariance cov y between two estimates from the same study (sharing a common treatment group) is simply the variance of the data in the shared arm E.g. for odds ratios, r f logor log Bi Ai logit p logit fr p Bi Ai Bi Ai varlogor var logit p var logit r f r f r f logor ACi log fr Ci Ai Ci Ai p i Ai Bi Ai Ai Bi Bi var logor ACi rai fai rci fci cov( logor i,logor ) ACi = var(logit ( p Ai )) = r Ai f Ai 3

24 Estimating within-study covariance () The covariance between two estimates from the same study (sharing a common treatment group) is simply the variance of the data in the shared arm e.g. for continuous data, SDAi cov MDi, MDACi n Ai cov SMD, SMD i ACi 1 n Ai 4

25 Estimating within-study covariance (3) We usually need to estimate the within-study covariance ourselves e.g. using simple for loops in Stata (see practical later) Within study covariances are irrelevant when we model the armlevel data directly When we have arm-level data, e.g. number of successes and failures per arm, and we model them using the binomial likelihood But if we are faced with data that can t readily be modelled at the arm level (perhaps due to the study design) then we might be modelling y s and v s Then we need to account for within-study correlation 5

26 BETWEEN-STUDY CORRELATION BETWEEN-STUDY COVARIANCE 6

27 Specifying between-study covariance A particular challenge is the estimation of between-studies variance-covariance matrix The covariance of two random effects is a function of the heterogeneities Within a multi-arm study, we must have true effects satisfying the consistency equations implying that deviations satisfy δ =δ -δ i,bc i,αc i, so τ =var δ = var δ +var δ -cov δ,δ BC i,βc i,ac i, i,ac i, =τ +τ -cov δ,δ AC i,ac i, and therefore i,ac i, cov δ,δ = τ +τ -τ AC BC 7

28 A common assumption about heterogeneity In network meta-analysis we very often assume the same heterogeneity variance for every comparison τ +τ -τ τ cov δ i,ac,δ i,, = = In the running example, this gives the between-studies covariance matrix τ τ τ 0 0 τ τ τ τ 8

29 A common assumption about heterogeneity This assumption makes estimation easier It is also particularly valuable when there are comparisons in the network with only one contributing study in which case the between-study variance can t be estimated without an assumption such as this 9

30 IMPLEMENTATION: STATA 30

31 Multivariate meta-analysis approach Either the multivariate approach or the multivariate metaregression approach can be used We ll use the former We assume all studies include treatment A When some trials don t include arm A, we augment the observed data we create an arm A with a very small amount of data e.g individuals with 10% success What does this do? Enables us to use the estimation method in mvmeta Adding a near-empty arm A to a trial B vs C yields treatment effects B vs A and C vs A with very large standard errors large covariance so that the data still convey the evidence about B vs C 31

32 Heterogeneity variance-covariance matrix This can be structured e.g. setting all heterogeneity variances equal, as discussed earlier or unstructured estimates a heterogeneity variance for each treatment comparison requires at least two studies for each comparison 3

33 Example: smoking data raw study design da na db nb dc nc dd nd 1 ACD BCD AC AC AD BC BD CD CD

34 Smoking data augmenting arm A study design da na db nb dc nc dd nd 1 ACD BCD AC AC AD BC BD CD CD

35 Smoking data handling zero cells study design da na db nb dc nc dd nd 1 ACD BCD AC AC AD BC BD CD CD

36 Smoking data treatment effects study design yb yc yd SBB SBC SBD SCC SCD SDD 1 ACD BCD AC AC AD BC BD CD CD Actually

37 Smoking data consistency model. mvmeta y S Note: using method reml Note: using variables yb yc yd Note: 4 observations on 3 variables Variance-covariance matrix: unstructured initial: log likelihood = rescale: log likelihood = rescale eq: log likelihood = Iteration 0: log likelihood = Iteration 1: log likelihood = (not concave) Iteration : log likelihood =

38 Smoking data consistency model Multivariate meta-analysis Variance-covariance matrix = unstructured Method = reml Number of dimensions = 3 Restricted log likelihood = Number of observations = Coef. Std. Err. z P> z [95% Conf. Interval] Overall_mean yb yc yd Estimated between-studies SDs and correlation matrix: SD yb yc yd yb yc yd Unstructured heterogeneity matrix: estimable because each contrast (, BC ) occurred in >1 trial 38

39 Summary When we have multi-arm studies, we need to move to a multivariate meta-analysis framework Each study may contribute two or more treatment effects Effect estimates are correlated (within the study) not a problem if we model the arms rather than the effect estimates, e.g. using (random-effects) logistic regression mvmeta lets us do network meta-analysis using multivariate methods in the practicals, we ll use a data augmentation procedure, and pick one of the treatments to be a reference 39

40 References White IR. Multivariate random-effects meta-analysis. Stata Journal 009; 9: White IR. Multivariate random-effects meta-regression: updates to mvmeta. Stata Journal 011; 11: Chan AW, Altman DG. Epidemiology and reporting of randomised trials published in PubMed journals. Lancet 005; 365: Dias S, Welton NJ, Sutton AJ, Ades AE. NICE DSU Technical Support Document : A generalised linear modelling framework for pairwise and network meta-analysis of randomised controlled trials (updated August 011). Decision Support Unit, ScHARR, University of Sheffield. Jackson D, Riley R, White IR. Multivariate meta-analysis: potential and promise. Stat Med 011; 30: Lu G, Ades A. Modeling between-trial variance structure in mixed treatment comparisons. Biostatistics. 009; 10: Salanti G, Higgins JPT, Ades AE, Ioannidis JPA. Evaluation of networks of randomized trials. Stat Meth Med Res 008; 17: Mavridis D, Salanti G. A practical introduction to multivariate meta-analysis. Stat Methods Med Res. 01 Feb 16. Gleser LJ, Olkin I. Meta-analysis for x tables with multiple treatment groups. In: Stangl D.K., Berry D.A., eds. Meta-analysis in medicine and health policy. Marcel Dekker, New York, 000. Geeganage C, Bath PMVW. Vasoactive drugs for acute stroke. Cochrane Database of Systematic Reviews 010, Issue 7. 40