Systematic Review & Systematic review

Transcription

1 Systematc Revew & Meta-analyssanalyss Ammarn Thakknstan, Ph.D. Secton for Clncal Epdemology and Bostatstcs Faculty of Medcne, Ramathbod Hosptal Tel: , Fax: e-mal: Systematc revew Revew methodology Poolng methods: meta-analyss Dchotomous outcome Contnuous outcome

2 Systematc revew A revew that has been prepared usng a systematc approach to mnmse bases and random error Ratonale Tool for health care worker Tool for health care worker, researchers, consumers, and also polcy maker who want to keep up wth the evdence that s accumulatng n ther feld

3 Ratonale More objectve apprasal of the evdence than tradtonal narratve revews - Narratve revew: subjectve, selecton bas, lmtaton of sngle or few studes, unhelpful descrptons, e.g., no clear evdence, a weak relatonshp, a strong relatonshp. - Systematc revew: more transparent apprasal, allow reader to replcate, quanttatve concluson. Meta-analyss: Ratonale Estmates treatment effects Leadng to reduces probablty of false negatve results (ncrease power of test) Potentally to a more tmely ntroducton of effectve treatments. 3

4 Ratonale Exploratory analyses: subgroups of patents who are lkely to respond partcularly well to a treatment (or the reverse) may generate promsng new research questons to be addressed n future studes. Systematc revew may demonstrate the lack of adequate evdence and thus dentfy area where further studes are needed Termnology Systematc revew Overvew Meta-analyss Research synthess Summarzng Poolng 4

5 Methodologc revew Good Ratonale Clearly state research queston Objectve Identfy relevant studes/locate studes Explctly descrbe ncluson & excluson crtera of studes Data extracton: study factor, outcome Data analyss Results Dscusson Ratonale Why do we need to perform the revew How were results of prevous ndvdual and revew studes Postve results Negatve results Methodologc ssues Sample sze/power of test Narratve revews? Selectve bas Poolng effect szes? Qualty of the studes 5

6 Good research queston Evdence-base Medcne (EBM) Patent Interventon Comparator Outcome PICO Research queston Is there assocaton between VDR and BMD/osteopoross n women? To compare rates of osteopoross n women wth BB wth bb genotypes Is Rosgltazone hgher rsk of developng myocardal nfarcton compare wth Metformn n dabetc patents? To compare ncdence of MI between dabetc patents receved Rosgltazone and Metformn 6

7 Locate studes. Defnes source of database MEDLINE - 949to present Over 6 mllon references Snce 005, between,000-4,000 completed references are added each day Tuesday through Saturday Cover 500 worldwde journals n 40 languages - Uses medcal subject headng (MeSH) for ndex - Includes bomedcne and health scence journals - Englsh abstracts for 79% on references - 90% are Englsh language artcles - 47% of journals covered are publshed n the US - PubMed avalable free of charge From Defnes source of databases EMBASE - Over mllon records from 974-present - Over mllon records from 974-present - More than 600,000 records added annually - Covers over 4,800 actve peer-revewed journals publshed n 70 countres/ 30 languages - uses EMTREE for ndexng ncludes Englsh abstracts for 80% of references - ncludes Englsh abstracts for 80% of references - daly update, wthn two weeks of recept of the orgnal journal - Produced by Elsever, no free verson avalable 7

8 Defnes source of database The Cochrane Controlled Trals Regster (CCTR) ClncalTrals.gov HUGE NET Revew Reference lsts Hand searchng of key journals Personal communcaton wth expert n the feld Locate studes. Defne the software & verson used for searchng - PubMed - OVID verson.0. - Slver Platter 8

9 Natonal Center for Botechnology Informaton Natonal Center for Botechnology Informaton 9

10 0

11

12 3. Defnes searchng terms Specfy perod of searchng Plan for update searchng Combnatons of search terms Interventon: treatment/study factor Outcome of nterest Comparator* Patent Example VDR& BMD/Osteopoross(J Bone Mner Res. 004;9(3):49-8.). vtamn D receptor or VDR (MeSH). genotype(s) or allele(s) or polymorphsm(s) (MeSH) 3. bone mneral densty or BMD or bone densty (MeSH) 4. low bone mneral densty or low densty (textword) 5. osteopoross (MeSH) 6. fracture (MeSH) 7. and and 3 8. and and 4 9. and and 5 0. and and 6

13 Explctly descrbe ncluson & excluson crtera Defne pror performng extracton of data to mnmze selecton bas Incluson crtera Study desgn randomzed controlled tral, observatonal studes (cohort, case-control, cross-sectonal study) Treatment (exposure), Outcome Subjects Full paper Languages Englsh, French, others Restrctons due to sample sze or other crtera Multple publcatons of the same Multple publcatons of the same studes, choose the recent one or the one has provded more completeness of data 3

14 Excluson Incompleteness of nformaton Contact authors at least two tmes for ncomplete data Not clearly descrbe defnton or how to dagnose the outcome Select studes Two revewers ndependently select studes Go through all retreved abstracts Make decson to select from the abstracts/full artcles Retreved all relevant full artcles Perform searchng every 3 months whle dong a revew 4

15 Thakknstan et al. Meta-analyss of molecular assocaton studes: vtamn D receptor gene polymorphsms and BMD as a case study. J Bone Mner Res 004;9(3):49-8. Any observatonal study (cohort, case-control, and cross-sectonal study), regardless of sample sze, whch determned the dfference n mean BMD accordng to the VDR genotype, or tested t the assocaton between osteopoross or fracture and the VDR genotypes and whch fulflled the followng crtera: VDR & BMD/Osteopoross - The BMD measurements were performed at lumbar spne or hp by Dual Energy X-ray Absorptometry (DEXA) or Dual Photon Absorptometry (DPA) method. - suffcent detals: mean and SD of BMD, and number of subjects for each VDR genotype, and frequences of genotype among case and control group were reported for dchotomous outcomes 5

16 VDR & BMD/Osteopoross Partcpants were adult women who were ether pre- or post menopausal. Study factor & outcome Bsm/Apa/Taq/Fok BMD/Osteopoross Data extracton (DE) At least two revewers Desgn DEF, plot, & revse DEF The artcle Study ID, Author, Year & source of publcaton The study characterstcs Type of studes subjects ethncty, t Adults vs chldren Postmenopause, premenopause study desgn (RCT, CS, CC, CrS) Methods used/crtera for measurng outcomes 6

17 Patents DE (cont.) Demographc and clncal features of study's partcpants that mght affect outcomes mean age, gender, BMI, smokng, underlyng dseases Table of study factors/nterventons t t versus outcomes DE (cont.) Dchotomous outcome Frequences between study factor/nterventon vs outcome Group Dsease Yes No N I Rx (Exp+) a b n a/n Placebo (Exp-) c d n c/n - OR (95% CI), RR (95% CI), HR (95% CI) 7

18 DE (cont.) Contnuous outcome n, mean (95% CI) Group n mean SD A n mean SD B n mean SD Rsk of bas n ndvdual studes Qualty Assessment (QA) Consder nternal & external valdty 8

19 RCT Rsk of bas (cont.) TC Chalmers From Egger M, Smth GD, Altman DG. Systematc revews n health care: Meta-analyss n context. London: BMJ Books, 00. Jadad s qualty assessment scale The Cochrane Collaboraton s tool for assessng rsk of bas 009 Preferred reports of tems for systematc revew and meta-analyss-prisma gudelne Doman Descrpton Revew authors judgement Sequence generaton. Descrbe the method used to generate the allocaton sequence n suffcent detal to allow an assessment of whether h t should produce comparable groups. Was the allocaton sequence adequately generated? Allocaton concealment. Descrbe the method used Was allocaton to conceal the allocaton adequately sequence n suffcent concealed? detal to determne whether nterventon allocatons could have been foreseen n advance of, or durng, enrolment. 9

20 Blndng of partcpants, Descrbe all measures used, Was knowledge of personnel and outcome f any, to blnd study the allocated assessors Assessments should partcpants and personnel be made for each man outcome from knowledge of whch nterventon adequately prevented (or class of outcomes). nterventon a partcpant durng the study? receved. Provde any nformaton relatng to whether the ntended blndng was effectve. Incomplete outcome data Assessments should be made for each man outcome (or class of outcomes). Descrbe the completeness of outcome data for each man outcome, ncludng attrton and exclusons from the analyss. State whether attrton and exclusons were reported, the numbers n each nterventon group (compared wth total randomzed partcpants), reasons for attrton/exclusons where reported, and any renclusons n analyses performed by the revew authors. Were ncomplete outcome data adequately addressed? Selectve outcome reportng. State how the possblty of selectve outcome reportng was examned by the revew authors, and what was found. Are reports of the study free of suggeston of selectve outcome reportng? Other sources of bas. State any mportant concerns Was the study about bas not addressed n apparently free of other the other domans n the tool. If partcular questons/entres were pre-specfed n the revew s protocol, responses should be provded for each queston/entry. Tral methodology Statstcal analyss problems that could put t at a hgh rsk of bas? Premature tral termnaton Post-randomzaton excluson Unbalance baselne characterstcs Adequately descrbe methods of data analyss -use per-protocol analyss, modfed ITT 0

21 Observatonal studes Thakknstan A, D'Este DEsteC, Esman J, Nguyen T, Atta J. Meta-analyss of molecular assocaton studes: vtamn D receptor gene polymorphsms and BMD as a case study. J Bone Mner Res 004;9:49-8. Crtera A. Representatveness of cases - Consecutve/randomly selected from cases populaton wth clearly defned random frame - Consecutve/randomly selected from cases populaton wthout clearly defned random frame or wth extensvely ncluson crtera - Not descrbe method of selecton B. Representatveness of controls - Consecutve/randomly drawn from area (ward/communty) as cases wth the same crtera - Consecutve/randomly drawn from dfferent area as cases - Not descrbe Score 0 0

22 Crtera C. Ascertanment of osteopoross/fracture - Clearly descrbed objectve crtera of dagnoss of osteopoross/osteoporotc fracture wth provng dagnoss, e.g. measure BMD usng DEXA, X-ray for fracture - Dagnoss of osteopoross/osteoporotc fracture by patents hstory - Not descrbe D. Ascertanment of control - Controls were proved that they were not osteopoross/osteoporotc p fracture,.e. measured BMD - Only mentoned that controls were subjects who were not osteopoross/osteoporotc fracture wthout provng - Not descrbe Score 0 0 Crtera E. Ascertanment of VDR examnaton - Genotypng done under blnd condton - Unblnd or not menton F. Data analyss HWE: - Checkng goodness of ft n control group wth approprate statstcs - Checkng goodness of ft of HWE n case & control group, or usng napproprate statstcs - Not mentoned G. Assocaton assessment: - Approprate statstc used wth adjustng for confounders e.g. logstc regresson, or matched case-control desgn - Approprate statstc used wthout adjustng - Inapproprate statstc Score 0 0 0

23 Crtera Score H. Response rate - Response rates for both group are the same or dfferent between groups 5% - Response rates are dfferent between 5% - 0% - Response rates are dfferent 0% or 0 more, or not menton about response rates Total 5 Net work Cochrane collaboraton RCT Dagnostc studes 3

24 4

25 5

26 Meta-analyss 6

27 Meta-analyss: Dchotomous outcome Group Dsease Yes No N Treatment a b n Placebo c d n Assumpton of poolng True effect of treatment/exposure s the same across studes Fxed effect model: Mantel-Haenzel, Peto s, nverse varance method Effects for ndvdual studes are assumed to vary Random effect model: DerSmonan & Lard, Baysan method Treatment/exposure effect: OR, RR, RD 7

28 Steps of data analyss For each study, estmate Effect sze, e.g. OR Varance of ln(or) Weght /varln(or) Estmate pooled effect sze base on fx-effect model Check for heterogenety, f exst, explore possble sources: by graph or meta-regresson Heterogenety test Ho: θ θ θ k θ OR LEVEL OF SIGNIFICANCE AT LEAST 0. Chose approprate poolng method Fxed-effect effect model: MH, IV, Peto Random: DerSmonan-Lad Subgroup analyss Assess publcaton bas Senstvty analyss 8

29 Estmaton of pooled (summary, combned) effect sze No varaton between studes (Homogenety) Fxed effect model Mantel-Haenzel Peto Inverse varance Varatons between studes (Heterogenety) Random effect model Der-Smonan and Lard Mantel-Haenzel ^ ln OR MH k ^ w θ k w ^ θ ln OR w var ad ln( c b bc N ) 9

30 30 ln ˆ k ^ w OR θ Inverse varance ) ln( ln ˆ ^ k p w b c a d OR w θ θ ln var ln var ^ ^ d c b a ) OR ( ) OR ( w Pooled RR θ w k ^ ^ n c n a RR θ w RR k ^ ) / / ln( ln ln n c n a RR RR w varln var ln +

31 Random-effect model DerSmonan and Lard ^ ln OR ^ DL ln OR w * var k w * k ^ θ w * ad ln( ) bc var + τ + + a b c + d Between study varaton (Tau) τ Q w ( k w w ) 3

32 Heterogenety test k Q w (θ ˆ θˆ ) p θˆ lnor (or lnrr,lnhr ) θˆ ln pooledor p v Q ~ χ wth df k - Degree of heterogenety I [Q-(k-)]/Qx00 < 5% low 5% - 75% moderate > 75% hgh 3

33 Study Example: ASA & death Asprn Placebo No. of Pt Death(a) Alve(b) No. of Pt Death(c) Alve(d). UK CDPA GAMS UK Pars AMIS Poolng odds rato of death for ASA versus non-asa Study % ID OR (95% CI) Weght.UK- 0.7 (0.49,.06) 0.5.CDPA 0.68 (0.46,.0) GAMS 0.8 (0.47,.38) UK (0.6,.06) Pars 0.80 (0.55,.5).6 6.AMIS.3 (0.93,.37) 4.54 Overall (I-squared 49.4%, p 0.079) 0.90 (0.80,.0)

34 metan a b c d, fxed or label(namevarauthor) Study OR [95% Conf. Interval] % Weght UK CDPA GAMS UK Pars AMIS I-V pooled OR Heterogenety ch-squared 9.88 (d.f. 5) p I-squared (varaton n OR attrbutable to heterogenety) 49.4% Test of OR : z.59 p 0. Heterogenety test Ho :lnor lnor... lnork H :At least oneor s dfferent a j Heterogenety ch-squared 9.88 (d.f. 5) p I-squared (varaton n OR attrbutable to heterogenety) 49.4% 34

35 Poolng usng random effect model metan a b c d, random or label(namevarauthor) xlab(0.4, 0.6,0.8,,.,.,.3) Study OR [95% Conf. Interval] % Weght UK CDPA GAMS UK Pars AMIS D+L pooled OR Heterogenety ch-squared 9.88 (d.f. 5) p I-squared (varaton n OR attrbutable to heterogenety) 49.4% Estmate of between-study varance Tau-squared Test of OR: z.7 p Poolng odds rato usng random effect model Study % ID OR (95% CI) Weght.UK- 0.7 (0.49,.06) 4.58.CDPA 0.68 (0.46,.0) GAMS 0.8 (0.47,.38) UK (0.6,.06) Pars 0.80 (0.55,.5) AMIS.3 (0.93,.37) 6.6 Overall (I-squared 49.4%, p 0.079) 0.84 (0.70,.0) NOTE: Weghts are from random effects analyss

36 Publcaton bas Postve studes are more lkely to be publshed to be publshed n Eng to be cted by other authors To produce multple publcaton Large studes are more lkely to be publshed even they have negatve results Qualty of study Lower qualty of methodology shows larger effects Bas due to assocaton between treatment effect and study sze Funnel Plot Treatment effect (X ) versus precson (/varance, Y) or SE 36

37 /varance Ln(effect sze) Statstc Tests Assess correlaton between effect szes and ther varances Begg s test Assumptons Small studes would have both a near-zero precson (/v) and a near-zero standardzed d d effect 37

38 Effect sze versus precson effect precson Kendall s rank correlaton t * t t * v t k k t v v * k v v s the weght average effect sze v 38

39 Egger s test Use standard weght lnear regresson by fttng t* to s - Where t* t /(v ) / s - /(v ) / t α + β s * Funnel plot wth pseudo 95% confdence lmts 0. 6.AMIS s.e. of lnor..uk-.cdpa 4.UK- 5.Pars 3.GAMS Odds rato (log scale) 39

40 metabas a b c d, egger graph Note: data nput format tcases tnoncases ccases cnoncases assumed. Note: odds ratos assumed as effect estmate of nterest Note: Peters or Harbord tests generally recommended for bnary data Egger's test for small-study effects: Regress standard normal devate of nterventon effect estmate aganst ts standard error Number of studes 6 Root MSE Std_Eff Coef. Std. Err. t P> t [95% Conf. Interval] slope bas Test of H0: no small-study effects P Intercept (α): Sze of ntercept provdes measure of asymmetry of funnel plot. The larger the devaton from zero, the greater the asymmetry Drecton of the ntercept t provdes nformaton on the form of bas: Postve: ES(estmated from smaller studes) > ES (estmated from larger studes), or small studes over estmated the ES, or negatve/ non-sgnfcant small studes are not ncluded Negatve: ES(estmated from smaller studes) < ES (estmated from larger studes) or small/ postve studes are more lkely to be ncluded β: pooled ES adjusted for publcaton bas 40

41 metabas a b c d f Study ~6, egger graph Note: data nput format tcases tnoncases ccases cnoncases assumed. Note: odds ratos assumed as effect estmate of nterest Note: Peters or Harbord tests generally recommended for bnary data Egger's test for small-study effects: Regress standard normal devate of nterventon effect estmate aganst ts standard error Number of studes 5 Root MSE Std_Eff Coef. Std. Err. t P> t [95% Conf. Interval] slope bas Test of H0: no small-study effects P metan a b c d f Study ~6, label(namevarauthor) or Study OR [95% Conf. Interval] % Weght UK CDPA GAMS UK Pars M-H pooled OR Heterogenety ch-squared 0.63 (d.f. 4) p I-squared (varaton n OR attrbutable to heterogenety) 0.0% Test of OR : z 3.0 p 0.00 Exclude study wth mean age 65 & serum cholesterol 300 mg/dl 4

42 Meta-analyss for contnuous outcome Data layout Study N Mean SD Rx/Exp+ N Mean SD Cont/Exp N Mean SD Methods of poolng Standardsed mean dfference (SMD) Dfferent scale of measurements, e.g. BMD (Hologc, Lunar, Norland), depresson score Unstandardsed mean dfference (USMD) The same scale of measurements 4

43 43 ˆ k k w w d D SMD ) var( sd x x d d w w method)...(cohen's ) ( ) var( ) ( ) ( ) ( + + n d n n n d n n sd n sd n sd x x d ) ( USMD sd sd d x x d ) var( ) ( + n n ) (

44 Heterogenety test Ho: D D,, D k Q Dˆ w k k k w w w var( d (d d ) D) ˆ Example: Specalst versus general nternst and LOS Source Specalst care Routne care N LOS SD N LOS SD. Edndburgh. Orpngton_mld 3. Orpngton_Modere 4. Orpngton_Severe 5. Montreal_Home 6. Montreal-Transfer Upsala Total

45 Example Vtamn D receptor (VDR) gene & Bone mneral densty VDR Bsm polymorphsm BB Bb bb BB s the rsk genotype Domnant mode of nhertance Summarzed mean dference of BMD (BB versus Bbbb) usng random effect model Study ID SMD (95% CI) % Weght Melhus H et al. Kroger H et al. Rggs BL et al. Berg JP et al. Garnero P et al. Jorgensen HL et al. McClure L et al. Vandevyver C et al. Gennar L et al. Hansen TS et al. Gomez C et al. Langdahl BL et al. Marc J et al. Barger-Lux MJ et al. Fleet JC et al. Rggs BL et al. Garnero P et al. Jorgensen HL et al. Alahar KD et al. Wllng M et al. Rubn LA et al. Holmberg-Marttla D et al. Overall (I-squared 35.%, p 0.054) NOTE: Weghts are from random effects analyss -0.8 (-.06, 0.50) (-.54,.36) (-0.64, 0.47) (-0.69, 0.6) 0. (-0.3, 0.46) (04 (-0.4, 007) 0.07) (-0.76, 0.68) 0.00 (-0., 0.) -0.9 (-0.55, -0.03) 0.03 (-0.30, 0.36) -0.3 (-0.65, 0.8) -0.8 (-0.65, 0.30) -0.9 (-.43, -0.40) -.5 (-.06, -0.44) (-.0, 0.05) (-0.80, 0.65) 0.00 (-0.38, 0.38) 0.0 (-0.74, 0.78) 0.8 (-0.46, 0.83) 0.0 (-0.5, 0.8) 0. (-0.09, 0.3) 0.35 (-0.4,.) -0. (-0., 0.00) Heterogenety ch-squared 3.5 (d.f. ) p Test of WMD0 : z.9 p