Overview. Multiple Treatment Meta Analysis II

Transcription

1 Multple Treatment Meta Analyss II Sofa Das Unversty of Brstol SMG Tranng Course, March 010, Cardff Wth thanks to: Georga Salant, Ncky Welton, Tony Ades, Debb Caldwell, Alex Sutton Overvew Bayesan parwse meta analyss Extenson to multple treatments Consstency assumptons Measures of model ft and model comparson Inconsstency models How many nconsstences? how drect and ndrect evdence combne graphcal/statstcal outputs (p values) Further readng and possble extensons Model Lkelhood: wll depend on type of outcome Normal for log OR, log RR, Rsk dff, mean, mean change from baselne, mean dff, log HR Bnomal for no. events/total Posson for no. events gven person years at rsk Scale for model: wll depend on lkelhood Normal lkelhood, pooled effect on natural scale Bnomal lkelhood, pooled effect on logt scale (logstc regresson) Posson lkelhood, pooled effect on log scale (log lnear model) Arm based summares wll estmate a baselne effect plus a relatve effect E.g. log odds=baselne + relatve effect 3 Computaton Usng Markov Chan Monte Carlo Straghtforward n WnBUGS Some Statstcal knowledge recommended Probably true for all Meta analyses anyway! 4

2 Generc Fxed Effect Model Y ~ Normal, V A quck overvew of BAYESIAN PAIRWISE META ANALYSIS Y s the observed effect n study wth varance V All studes assumed to be estmatng the same underlyng effect sze µ Statstcal Homogenety For a Bayesan analyss, a pror dstrbuton must be specfed for µ, for example on ln(or) scale, ~ Normal 0,10 6 Appendx 1: Choce of Pror for µ Often amount of nformaton n studes would overwhelm any reasonable pror therefore choce not crtcal A pror we would be 9% certan that true value of µs between ( and )* On an odds rato scale that s equvalent to (10 69 to ).e. very vague and essentally flat over the realstc range of nterest Could, of course, nclude nformatve prors *Note: 316 = 10 7 Generc Random Effects Model Y s the observed effect n study wth varance V Across studes Y ~ Normal, V ~ Normal, pror dstrbuton for µ, as before ~ Normal 0,10 Pror dstrbuton for τ: Unform(0,10) or halfnormal Requres care when evdence sparse (Lambert et al, SM 00) 8

3 Some Advantages of Bayesan MA Can cope wth zero cells Incorporates uncertanty n the heterogenety parameter Easly extended to ncorporate covarates Predctve dstrbutons straghtforward Can nclude nformatve pror dstrbutons for eg heterogenety parameter, when evdence sparse Normalty of true effects n a random effects analyss Can be easly relaxed n WnBUGS to eg t dstrbuton An extenson to MULTIPLE TREATMENT META ANALYSIS (MIXED TREATMENT COMPARISONS, NETWORK META ANALYSIS) 9 Assumptons Approprate modellng of data (as before) Lkelhood and lnk functon Comparablty of studes exchangeablty n all aspects other than partcular treatment comparson beng made Equal heterogenety (RE varance) n each comparson not strctly necessary (Lu and Ades, Bostatstcs 009) Bayesan Multple Treatment Meta analyss 1. Four treatments A, B, C, D. Take treatment A as the reference treatment makes no dfference to relatve effects, but ads nterpretaton f eg. placebo s chosen 3. Then the treatment effects (eg. log odds ratos) of B, C, D relatve to A are the basc parameters 4. Gven them prors: µ AB, µ AC, µ AD ~ N(0,10 ) 11 1

4 Functonal parameters n MTC The remanng contrasts are functonal parameters µ BC = µ AC µ AB µ BD = µ AD µ AB CONSISTENCY assumpton µ CD = µ AD µ AC A B C D All comparsons relatve to A Any nformaton on functonal parameters tells us ndrectly about basc parameters There s a degree of redundancy n the network Ether FE or RE model satsfyng these condtons 13 Consstency We assume that the treatment effect µ BC estmated by BC trals, would be the same as the treatment effect estmated by the AC and AB trals f they had ncluded B and C arms Assume that tral arms are mssng at random reason they are mssng s not related to treatment effect 14 Generc random effects Model Y ~ Normal, V ~ Normal, bk Generc random effects Model Y ~ Normal, V ~ Normal, Ak Ab Consstency assumptons 1 So tral of BvsC wll have k=c and b=b For a Bayesan analyss, pror dstrbutons are requred for and all basc parameters µ Aj. Models whch do not assume common heterogenety are avalable (Lu and Ades Bostatstcs, 009) 16

5 Example: Treatment for acute myocardal nfarcton* 8 thrombolytc drugs and surgery 9 treatments, 0 trals Two very large 3 arm trals 16 drect comparsons (out of 36) SK (1) r-pa () 1 Thrombo: Treatment Network ASPAC (9) PTCA (7) 11 SK + t-pa (4) 1 1 TNK (6) Acc t-pa (3) *see eg Das et al SM n press, for detals t-pa () 3 UK (8) Queston: In a network wth 9 dfferent treatments how many basc parameters? FE Model =1,, 0 trals; k=1,,3 arm number Lkelhood r ~ Bnomal(, n ) k k k Lnk functon (scale) logt( k ) ( 1 t 1 ) k t I 1 k 1 Consstency assumptons prors ~Normal(0,10) ~ Normal(0,10 ), j,...,9 treatments 1 j Nusance parameters 19 0

6 Thrombo: log odds ratos (FE model) Treat No of Parwse MA MTC X Y studes Par ˆ XY var MTC ˆ XY var RE Model =1,, 0 trals; k=1,,3 arm number Lkelhood Lnk functon logt( ) k r ~ Bnomal(, n ) k k k k Ik 1 RE dstrbuton k ~ Normal 1 t 1, k t 1 prors Consstency assumptons ~Normal(0,10) 1 j ~ Normal(0,10 ),,...,9 treatments ~ Unf(0,10) j 1 FE or RE model? Is heterogenety always present? For a well defned populaton and decson problem, there may be lttle heterogenety Or ths may be explaned by covarates Is ths ever the case n lumped Cochrane Revews? Outputs from RE model harder to nterpret Problems wth estmaton of varance of RE dstrbuton when data sparse FE model preferable f t can be justfed FE or RE model? Choose between two models To assess model ft calculate resdual devance Compare to number of unconstraned data ponts For model comparson use DIC Penalses a better ft by the effectve number of parameters, pd 3 4

7 Resdual Devance The best ft we can get s where the model predctons equal the observed data Saturated model Resdual devance s the devance for the current model, mnus the devance for a saturated model D (loglkmodel loglk sat) res Calculated at each teraton of MCMC algorthm Summarsed by posteror mean D res If the model s an adequate ft, we expect D res to be roughly equal to the number of unconstraned data ponts Appendx : Calculatng At each teraton, the resdual devance, D res, s calculated as the sum of the devances for each data pont, eg for Bnomal r r n r Dres r log ( n r )log rˆ ˆ n r dev D res = observed no. events rˆ pn = expected no. events from current model dev s the devance resdual Summarsed by the posteror mean D res (over M teratons) 6 Model Comparson Devance Informaton Crtera (DIC) Take devance for current model (= loglk for current model) Penalse by effectve no of parameters DIC D p model Extenson of Akake s Informaton Crteron Trade off between ft and complexty Dfferences of (?3) are mportant D res Can also use posteror mean of resdual devance (dffers only by a constant does not matter for comparsons) D Effectve No. Parameters, p D Fxed Effects Model p D = no. parameters Random Effects Model p D depends on between study varance, For close to 0, θ = µ; 1 parameter (as n fxed effects model) For very large, θ = θ ; one parameter for each study 7 Spegelhalter et al JRSS B,00 8

8 Appendx 3: Calculatng p D At each teraton, calculate D res dev For each data pont, posteror mean of (mean taken over M teratons) dev Calculate posteror mean of ftted values, eg n Bnomal r s the posteror mean of rˆ. Calculate devance at the posteror mean of the ftted values D (replace r wth r n formula for resdual devance) ˆ D Appendx 3 (cont): Calculatng p D The effectve number of parameters p D s calculated as the sum, over all data ponts, of the leverages,.e. the sum of the posteror mean of the resdual devances, mnus the devances at the posteror mean of the ftted values pd leverage D D 9 30, 0 studes ( k 1), from common dstrbuton k How many parameters n ths example? Fxed effects model (τ =0), 0 studes, fxed treatment effects for 8 basc parameters 1k Random effects model (τ >0) 8 parameters Independent effects model (τ ), 0 studes ( k 1), no shrnkage n treatment effects for arms k p D =61.6 Up to 10 parameters 31 Model Random Effects Fxed Effects Fxed v Random Effects Models Resdual Devance* (posteror mean) *Compare to 10 data ponts p D DIC Heterogenety (posteror medan) RE model appears to ft best but no advantage gven more parameters unless beleve heterogenety 3

9 Dagnostc Plots Plot: ndvdual data ponts contrbutons to the DIC (wth sgn gven by dfference between ftted and observed values) aganst leverages (.e. ndvdual data ponts contrbutons to pd) Hghlght poorly fttng or hghly nfluental data ponts: Add parabolas of the form x +y=c These represent contrbutons of c to the DIC Ponts outsde parabola wth c=3, say, are hghlghted leveragemtc Leverage plot for FE MTC Trals, compare treatments 3 and 9 Spegelhalter et al JRSS B,00 33 devance resduals 34 Leverage plot for RE MTC leverage Trals, compare treatments 3 and 9 Checkng for INCONSISTENCY devance resduals 3

10 What about nconsstency? The true treatment effects must be consstent But there may be nconsstences n the EVIDENCE How to check for ths? Queston: How many nconsstences could there be? Treatments A,B,C. Trals or sets of trals AB, AC, BC A B C How many nconsstences? Inconsstences are propertes of loops Inconsstency degrees of freedom (ICDF) s the maxmum number of possble nconsstences* Informally descrbed as the number of ndependent 3 way loops n the evdence structure In ths example the ICDF s seven Count ndependent 3 way loops Dscount any loops formed only by 3 arm trals One such loop (1,3,4), n ths example. Multple testng? SK (1) r-pa () Thrombo: Treatment Network t-pa () ASPAC (9) PTCA (7) UK (8) SK + t-pa (4) TNK (6) Acc t-pa (3) 39 * Lu and Ades, JASA

11 Inconsstency & heterogenety heterogenety n treatment effects s the varaton n treatment effects between trals WITHIN par wse contrasts, eg wthn AB trals nconsstency s varaton n treatment effects BETWEEN par wse contrasts, eg AB, AC results nconsstent wth BC. Both due to mssng covarates: factors that nteract wth the treatment effect but vary between trals To measure heterogenety, must look at trals. To measure nconsstency, can focus on the pooled summares of evdence on par wse contrasts Inconsstency & heterogenety We can have nconsstency when no heterogenety s present (.e. n a FE model) But wll a RE model dsguse true nconsstences? Possbly, depends on the evdence network Not the case n the Thrombolytcs example 41 4 Inconsstency Heterogenety best nterventon nterventons Recall Multple meta-analyses of RCTs Meta-analyss of RCTs RCTs heterogenety relates to varablty of dstrbuton of random treatment effects Inconsstency relates to valdty of consstency assumptons, e across comparsons Man deas for checkng consstency 1. Compare posteror dstrbutons obtaned from drect and ndrect evdence for each comparson. Model ft/comparson problem Ft models wth and wthout consstency assumptons Compare model ft (resdual devance, DIC) 3. A mxture of both 43

12 Comparson of drect and ndrect estmates Method for trangles (Bucher JCE, 1997) Separate Parwse meta analyses on all contrasts Calculate ndrect estmate (usng consstency equatons) Ignores network Evaluaton of concordance wthn closed loops prevous sesson Can be extended to whole networks Das et al. Statstcs n Medcne (n press, 010) Problems when three arm trals ncluded or when random effects models used. Model comparson In a complex treatment network, what s drect evdence for one comparson s ndrect for another and there are multple ways n whch to form an ndrect comparson Better to thnk as model crtcsm Is my consstency model reasonably supported by the evdence? 46 Consstency model 9 treatments, 8 basc parameters Inconsstency model Add 7 parameters (8+7=1 parameters) Compare model ft Inconsstency models* 1, x, y ~ N(0, Inconsstency ) WARNING: Ths model requres careful parametersaton 1,, 1,3, 1,4, ~ N (0,10 ),3 1,3 1, 8,9 1,9 1,8,7 1,7 1,,8 1,8 1, 3,9 1,9 1,3 1,,7 1,, 8 1,3,9 * Lu and Ades, JASA Box plot of nconsstency factors ω (RE model).0 {1,3,9} 1.0 {1,,7} {1,3,8} {1,3,7} {1,,8} {1,,9} {1,3,}

13 Independent mean effects model Consstency model 9 treatments, 8 basc parameters Independent mean effects model 1 parameters (one for each parwse contrast) NOTE: Same number of parameters as nconsstency model! 1,, 1,3, 1,4,, 3,7, 3,8, 3,9 ~ N (0,10 ) No consstency assumptons 1,, 1,3, 1,4,, 1,9 ~ N(0,10 ),3 1,3 1, 8,9 1,9 1,8 49 leverage Leverage plot for RE MTC devance resduals 0 Leverage plot ndep. mean effects Compare resdual devance for each data pont leverage devance resduals 1 Devance ndependent mean effects Devance MTC Trals, compare treatments 3 and 9

14 Node splttng* Leverage plot when node(3,9) splt Splts on each contrast, µ (node, eg. vs3) Studes whch compare and 3 drectly nform drect estmate Rest of data wth arms and 3 removed nform ndrect estmate Relaxes consstency assumpton for one contrast at a tme Compare model ft Check between tral heterogenety parameters resdual devance, DIC statstcs Draw plots of posteror dstrbutons based on drect and ndrect evdence Bayesan p value to check for consstency Computatonally ntensve Needs to be done for every node leverage * Marshall & Spegelhalter, Bayesan Analyss (007) Das et al. Statstcs n Medcne (n press, 010) devance resduals 4 Compare resdual devance for each data pont Compare model ft (RE model) Devance splt node (3,9) Trals, compare treatments 3 and 9 Model Resdual devance* pd DIC Between tral heterogenety (posteror medan) MTC Independent mean effects Inconsstency (ω factors) nconsstency varance = 0.3 Node (3,9) splt * Compare to 10 data ponts Devance MTC 6

15 Compare drect and ndrect evdence Inconsstent! Densty full Full MTC MTM MTC ndrect excl. drect drect Densty drect full Full MTC MTM MTC ndrect excl. drect Densty MTC excl. drect ndrect full Full MTC MTM drect Node (3,9) s splt Drect evdence on (3,9) conflcts wth ndrect evdence Bayesan p value < 0.00 MTM domnated by ndrect evdence from very large trals Only small trals drectly compare treats 3 and 9 Consstent log-odds rato log-odds rato Possbly nconsstent? log-odds rato 7 8 Queston Would you trust the drect head to head trals or the MTM results? Drect log OR = 1.407, varance = Based on two small trals MTM log OR = 0.194, varance = Based on borrowng strength from evdence on all other trals (some very large) And on the assumpton of CONSISTENCY! Why s there nconsstency? We have found evdence of nconsstency n node (3,9) and evdence loop (1,3,9) The two drect trals comparng treat 3 vs 9 have less absolute mortalty n control arm than other studes usng treatment 3 as control Other baselne characterstcs dd not reveal other causes (same tme perod, no apparent dfference n clncal factors) 9 60

16 Consderatons on Inconsstency ALL these methods can ONLY detect nconsstency n a general sense. They cannot say whch evdence s wrong. Inconsstency s a property of evdence loops, not of partcular edges. Identfyng whch edge, or edges, are wrong s a task for clncal epdemology, not statstcs. Need to queston f reasonable to combne trals n MTM a pror When no evdence of nconsstency, we can be reassured that the core MTM assumptons are met Extensons and FURTHER READING 61 Bas adjustment Gven a mechansm for bas e.g. lack of allocaton concealment or blndng Estmate and adjust for bas wthn the network Usng degree of redundancy afforded by consstency assumpton Requres a large network wth multple combnatons of based and unbased evdence Das S, Welton NJ, Marnho V, Salant G, Hggns JPT and Ades AE. Estmaton and adjustment of bas n randomsed evdence usng mxed treatment comparson meta analyss. In press, Journal of the Royal Statstcal Socety Seres A. References Bucher HC, Guyatt GH, Grffth LE and Walter SD. The Results of Drect and Indrect Treatment Comparsons n Meta Analyss of Randomzed Controlled Trals. Journal of Clncal Epdemology 1997; 0: Das S, Welton NJ, Caldwell DM and Ades AE. Checkng consstency n mxed treatment comparson meta analyss. In press, Statstcs n Medcne. Lambert PC, Sutton AJ, Burton P, Abrams KR and Jones D. How vague s vague? Assessment of the use of vague pror dstrbutons for varance components. Statstcs n Medcne 00; 4: Lu G and Ades AE. Assessng evdence consstency n mxed treatment comparsons. Journal of the Amercan Statstcal Assocaton 006; 101: 7 9. Lu G and Ades AE. Modelng between tral varance structure n mxed treatment comparsons. Bostatstcs 009; 10: Marshall EC and Spegelhalter DJ. Identfyng outlers n Bayesan herarchcal models: a smulaton based approach. Bayesan Analyss 007; (): Spegelhalter DJ, Best NG, Carln BP and van der Lnde A. Bayesan measures of model complexty and ft. Journal of the Royal Statstcal Socety (B) 00; 64:

17 For further detals on MTM, ncludng courses 6