Distributed analysis in multi-center studies

Transcription

1 Distributed analysis in multi-center studies Sharing of individual-level data across health plans or healthcare delivery systems continues to be challenging due to concerns about loss of patient privacy, unauthorized uses of transferred data, inaccurate analysis or interpretation of data, or contractual or legal restrictions. Although these challenges can be addressed in part by proper governance and appropriate updates to existing regulations, newer privacy-protecting analytic and data-sharing methods offer another potential solution. This presentation will describe the use of privacy-protecting analytic methods that allow robust and flexible statistical analysis using aggregate-level information, without centralized pooling of individual-level datasets across data sources. We will present several comparative safety and effectiveness studies of medical treatments that employ these methods to generate actionable real-world evidence. 1. Toh S, Gagne JJ, Rassen JA, Fireman BH, Kulldorff M, Brown JS. Confounding adjustment in comparative effectiveness research conducted within distributed research networks. Med Care 2013:51(8 Suppl 3):S4-S10 2. Toh S, Hampp C, Reichman ME, Graham DJ, Balakrishnan S, Pucino F, Hamilton J, Lendle S, Iyer A, Rucker M, Pimentel M, Nathwani N, Griffin MR, Brown NJ, Fireman BH. Risk of hospitalized heart failure among new users of saxagliptin, sitagliptin, and other antihyperglycemic drugs: A retrospective cohort study. Ann Intern Med 2016;164(11): (PMC ) 3. Toh S, Reichman ME, Graham DJ, Hampp C, Zhang R, Butler MG, Iyer A, Rucker M, Pimentel M, Hamilton J, Lendle S, Fireman BH; for the Mini-Sentinel AMI-Saxagliptin Surveillance Writing Group. Prospective post-marketing surveillance of acute myocardial infarction in new users of saxagliptin: A population-based study. Diabetes Care 2018;41(1):39-48

2 Distributed analysis in multi center studies Darren Toh, ScD Department of Population Medicine Harvard Medical School & Harvard Pilgrim Health Care Institute Boston, MA November 18, 2018

3 Disclosures Research support Patient Centered Outcomes Research Institute (ME ) Office of the Assistant Secretary for Planning and Evaluation & Food and Drug Administration (HHSF I) National Institutes of Health (U01EB023683) Agency for Healthcare Research and Quality (R01HS026214) Board of Directors, International Society for Pharmacoepidemiology My spouse is an employee of Biogen 2

4 Overview Evolution of multi center studies Analytic methods in multi center studies Select examples Discussion 3

5 Overview Evolution of multi center studies Analytic methods in multi center studies Select examples Discussion 4

6 Multi center studies Many studies are now done in multi center settings 5

7 Why do multi center studies? Larger sample sizes Allow studies of rare treatments or rare outcomes Allow studies in specific subpopulations Allow studies to be done more quickly More diverse populations Allow more generalizable findings Allow assessment of treatment effect heterogeneity 6

8 Multi center studies v1.0 Analysis center 7

9 Multi center studies v1.0 Pooling study specific individual level datasets 8

10 Typical datasets shared in multi center studies v1.0 PatID Exposure Outcome Time Age Sex DM HTN CVD

11 Typical datasets shared in multi center studies v1.0 PatID Exposure Outcome Time Age Sex DM HTN CVD Each row represents an individual

12 Typical datasets shared in multi center studies v1.0 PatID Exposure Outcome Time Age Sex DM HTN CVD Each column represents a covariate

13 Typical datasets shared in multi center studies v1.0 Data Partner 1 PatID Exposure Outcome Time Age Sex DM HTN CVD Data Partner 2 PatID Exposure Outcome Time Age Sex DM HTN CVD Site PatID Exposure Outcome Time Age Sex DM HTN CVD

14 Multi center studies v2.0 Individual data partners Data standardization (common data model) Data accessible to research projects Site 1 Site 2 Site 3 Site 4 Site 1 Site 2 Site 3 Site 4 Research projects Programs written against common data model Data quality improvement feedback loop Adapted from: eb86 400e 8c74 2d42ac57fa4F/VDW.Infographic jpg 13

15 Data standardization Common data model 14

16 Distributed analysis in networks with common data model 1 Analysis Center Secure Network Portal 1. User creates and submits query Review & Run Query Data Partner 1 Review & Return Results Enrollment Demographics Utilization Pharmacy Etc Review & Run Query Data Partner 2 Review & Return Results Enrollment Demographics Utilization Pharmacy Etc 15

17 Distributed analysis in networks with common data model 1 Analysis Center Secure Network Portal 1. User creates and submits query 2. Data partners retrieve query 2 Review & Run Query Data Partner 1 Enrollment Demographics Utilization Pharmacy Etc Review & Return Results Review & Run Query Data Partner 2 Review & Return Results Enrollment Demographics Utilization Pharmacy Etc 16

18 Distributed analysis in networks with common data model 1 Analysis Center Secure Network Portal 1. User creates and submits query 2. Data partners retrieve query Review & Run Query 2 3 Data Partner 1 Enrollment Demographics Utilization Pharmacy Etc Review & Return Results 4 3. Data partners review and run query against their local data 4. Data partners review results Review & Run Query Data Partner 2 Review & Return Results 3 Enrollment Demographics Utilization Pharmacy Etc 4 17

19 Distributed analysis in networks with common data model 1 Analysis Center 6 1. User creates and submits query Secure Network Portal 2. Data partners retrieve query Review & Run Query 2 3 Data Partner 1 Enrollment Demographics Utilization Pharmacy Etc Review & Return Results Data partners review and run query against their local data 4. Data partners review results Review & Run Query Data Partner 2 Review & Return Results 5. Data partners return results via secure network 3 Enrollment Demographics Utilization Pharmacy Etc 4 6. Results are aggregated and reported 18

20 Typical datasets shared in multi center studies v2.0 Data Partner 1 PatID Exposure Outcome Time Age Sex DM HTN CVD Data Partner 2 PatID Exposure Outcome Time Age Sex DM HTN CVD Site PatID Exposure Outcome Time Age Sex DM HTN CVD

21 Concerns about data sharing in multi center studies v1 & v2 Loss of patient privacy Unauthorized uses of data Inaccurate analysis or interpretation of data Disclosures of sensitive institutional or corporate information Contractual restrictions 20

22 Data sharing A balancing act Analytic flexibility Granularity or identifiability of information 21

23 Multi center studies v3.0 Analysis Center 22

24 Multi center studies v3.0 Pooling study specific summary level datasets 23

25 Overview Evolution of multi center studies Analytic methods in multi center studies Select examples Discussion 24

26 Privacy protecting methods for multi center studies v3.0 Summary score based methods Meta analysis of database specific effect estimates Distributed regression 25

27 Summary scores Confounders PS DRS Treatment Outcome PS: Propensity scores DRS: Disease risk scores 26

28 Individual level dataset with individual covariates PatID Exposure Outcome Time Age Sex DM HTN CVD

29 Individual level dataset with summary scores PatID Exposure Outcome Time PS

30 Summary score based method #1 Matching PatID Exposure Outcome Time PS Persons in exposed Persons in unexposed Events in exposed Events in unexposed

31 Summary score based method #1 Matching Data Partner 1 Persons in exposed Persons in unexposed Events in exposed Events in unexposed Data Partner 2 Site Persons in exposed Persons in unexposed Events in exposed Events in unexposed Persons in exposed Persons in unexposed Events in exposed Events in unexposed

32 Summary score based method #2 Stratification PatID Exposure Outcome Time PS PS or DRS stratum Persons in exposed Persons in unexposed Events in exposed Events in unexposed

33 Summary score based method #3 Risk set analysis PatID Exposure Outcome Time PS Event Event time Event exposed Risk set exposed Risk set unexposed

34 Meta analysis of database specific effect estimates PatID Exposure Outcome Time Age Sex DM HTN CVD Hazard ratio Lower 95% CI Upper 95% CI

35 Distributed regression Type Name Intercept E X1 X2 Y ID E X1 X2 Y A A A A A A SSCP Intercept SSCP E SSCP X SSCP X SSCP Y MEAN STD N Variable Parameter estimate Standard error Intercept E X X Analyst inputs individual level dataset into statistical software Statistical software produces intermediate statistics as part of computing process Statistical software produces final results 34

36 Distributed regression Type Name Intercept E X1 X2 Y ID E X1 X2 Y A A A A A A SSCP Intercept SSCP E SSCP X SSCP X SSCP Y MEAN STD N Variable Parameter estimate Standard error Intercept E X X Regular regression shares this Analyst inputs individual level dataset into statistical software Statistical software produces intermediate statistics as part of computing process Statistical software produces final results 35

37 Distributed regression Type Name Intercept E X1 X2 Y ID E X1 X2 Y A A A A A A SSCP Intercept SSCP E SSCP X SSCP X SSCP Y MEAN Distributed regression shares this STD N Variable Parameter estimate Standard error Intercept E X X Analyst inputs individual level dataset into statistical software Statistical software produces intermediate statistics as part of computing process Statistical software produces final results 36

38 Overview Evolution of multi center studies Analytic methods in multi center studies Select examples Discussion 37

39 Example 1 stroenterology/laparoscopic_adjustable_gastric_banding_135,63/ erology/roux en y_gastric_bypass_weight loss_surgery_135,65/ 38

40 Study design 1/1/2005 Start of follow up (discharge date) 12/31/ days Time 21 years at time of bariatric surgery 1 BMI of 35kg/m 2 or greater Continuous enrollment w/ benefits No prior bariatric surgery No prior diagnosis of study outcome Re hospitalization Death Health plan disenrollment 12/31/ days of follow up Index bariatric hospitalization Contributing person times Toh et al, Med Care, 2014;52:

41 Confounders Age Sex Race/ethnicity Diabetes* Baseline BMI* Year of procedure Charlson comorbidity score* Atrial fibrillation* GERD* Hypertension* Sleep Apnea* Asthma* Deep vein thrombosis* Pulmonary embolism* Congestive heart failure* Hyperlipidemia* Coronary artery disease* Oxygen use* Assistive walking device* Smoking status* Blood pressure* Length of stay assoc. with procedure *Identified during the 365 day baseline period prior to the index bariatric hospitalization Toh et al, Med Care, 2014;52:

42 Statistical analysis Propensity score stratification Analysis Pooled patient level data analysis (benchmark) Risk set based analysis PS stratified analysis (by quintile) Meta analysis of site specific effect estimates Toh et al, Med Care, 2014;52:

43 Select baseline patient characteristics Characteristics Adjustable gastric band (n=1,550) Roux en y gastric bypass (n=5,792) N %* N %* Mean age (SD) Age > 65 years Female sex 1, , Race/ethnicity Black or African American White 1, , Hispanic Other Unknown Baseline BMI , , , Toh et al, Med Care, 2014;52:

44 Individual level data analysis, by site Site Adjusted HR 95% CI Site , 1.02 Site , 1.15 Site , 1.04 Site , 1.50 Site , 1.48 Site , 0.75 Site , 1.01 Toh et al, Med Care, 2014;52:

45 Results, by method Method Adjusted HR 95% CI Individual level , 0.84 Risk set , 0.84 PS stratification , 0.83 Meta analysis , 0.84 Toh et al, Med Care, 2014;52:

46 Example 2 Distributed regression Distributed Regression vs. Pooled Patient Level Regression LINEAR Covariates Distributed Regression Pooled Patient Level Differences in Differences in Parameter Estimates Standard Errors Parameter Estimates Standard Errors Parameter Estimates Standard Errors Intercept E E 14 Variable E E 16 Variable E E 15 Variable E E 15 Distributed Regression vs. Pooled Patient Level Regression LOGISTIC Covariates Distributed Regression Pooled Patient Level Differences in Differences in Parameter Estimates Standard Errors Parameter Estimates Standard Errors Parameter Estimates Standard Errors Intercept E E 16 Variable E E 14 Variable E E 16 Variable E E 16 Distributed Regression vs. Pooled Patient Level Regression COX Covariates Distributed Regression Pooled Patient Level Differences in Differences in Parameter Estimates Standard Errors Parameter Estimates Standard Errors Parameter Estimates Standard Errors Variable E E 17 Variable E E 17 Variable E E 17 45

47 Example 3 PCORnet Bariatric Study Use of bariatric surgery has expanded considerably Evidence on the comparative effectiveness and safety of these procedures is limited 46

48 Study design Comparisons Main analysis RYGB vs. SG RYGB vs. AGB AGB vs. SG Aggregate analysis RYGB vs. SG Outcomes Weight change 1, 3, and 5 yrs postsurgery Diabetes remission and relapse Major adverse events Weight change 1 yr post surgery Analysis One model that combines all data Additional data driven approaches to select covariates Site specific PS model Fixed set of covariates 47

49 48

50 49

51 Combining propensity scores with distributed regression Parameter estimate Standard error Variable Pooled individuallevel data analysis Pooled individuallevel data analysis RYGB vs. SG PS stratum 1 Reference Reference PS stratum PS stratum PS stratum PS stratum PS stratum PS stratum PS stratum PS stratum PS stratum

52 Combining propensity scores with distributed regression Parameter estimate Standard error Variable Pooled individuallevel data analysis Distributed regression Pooled individuallevel data analysis Distributed regression RYGB vs. SG PS stratum 1 Reference Reference Reference Reference PS stratum PS stratum PS stratum PS stratum PS stratum PS stratum PS stratum PS stratum PS stratum

53 Example 4: Prospective surveillance of saxagliptin Sentinel_AMI and Anti Diabetic Agents_Protocol_0.pdf 52 52

54

55 SAVOR TIMI 53 Trial 54 54

56 Prospective surveillance of saxagliptin Sentinel_AMI and Anti Diabetic Agents_Protocol_0.pdf 55 55

57 Saxagliptin vs. sitagliptin 56 56

58 Saxagliptin vs. pioglitazone 57 57

59 Saxagliptin vs. sulfonylureas 58 58

60 Saxagliptin vs. long acting insulin 59 59

61 Comparisons with SAVOR TIMI 53 trial Characteristics SAVOR TIMI 53 Trial Mini Sentinel surveillance* Comparator Placebo Select anti hyperglycemics No. saxagliptin users 8,280 82,264 No. comparator users 8, ,045 to 452,969 Interim results from the first 5 sequential analyses were No. AMI in saxagliptin to 171 made available to FDA prior to the publication of SAVOR No. AMI in comparator to 1,085 TIMI 53 findings Length of follow up 2.1 years (median) 4 to 8 months (mean) Statistical analysis Intention to treat As treated Hazard ratio for AMI 0.95 (95% CI: 0.80, 1.12) 0.54 to 1.17 * From end of surveillance analysis that included all patients 60 60

62 Overview Evolution of multi center studies Analytic methods in multi center studies Select examples Discussion 61

63 Analytical flexibility vs. granularity of information Analytic flexibility Individuallevel data with individual covariates Individuallevel data with summary scores Risk set data Summarytable data Intermediate statistics Effectestimate data Privacy protection 62

64 Analytic methods in multi center studies Covariate summarization technique Data sharing approach Covariate adjustment technique Outcome type Individual covariates* Individual level data Matching Continuous What to share? How to share? What can we do? What outcome? Propensity scores Summary table data Stratification Binary Disease risk scores Risk set data Restriction Count Summary scores + individual covariates Effect estimate data Weighting Survival A hybrid of above Intermediate statistics Modeling 63

65 Analytic methods in multi center studies Covariate summarization technique Data sharing approach Covariate adjustment technique Outcome type Individual covariates Individual level data Matching Continuous Propensity scores Summary table data Stratification Binary Disease risk scores Risk set data Restriction Count Summary scores + individual covariates Effect estimate data Weighting Survival A hybrid of above Intermediate statistics Modeling 64

66 Conclusion A suite of analytic methods are available for multi center studies There are often trade offs between analytic flexibility and identifiability of information shared Some newer methods offer excellent analytic flexibility and good privacy protection 65

67 @darrentoh_epi 66