Near/Far Matching. Building a Stronger Instrument in an Observational Study of Perinatal Care for Premature Infants

Transcription

1 Near/Far Matching Building a Stronger Instrument in an Observational Study of Perinatal Care for Premature Infants Joint research: Mike Baiocchi, Dylan Small, Scott Lorch and Paul Rosenbaum

2 What this talk is about We ve developed an IV method that feels a lot like propensity score matching. Under some conditions, we can design studies with stronger instruments.

3 An Encouragement Design

4 References Instrumental Variables: Angrist, Imbens, Rubin: Identifcation of causal effects using instrumental variables (with Discusssion). JASA 91, (1996) Encouragement Design: Holland, P.W.: Causal inference, path analysis, and recursive structural equations models. Sociological Methodology 18, (1988)

5 Note on Terminology Encouragement = Instrumental in selection

6 What s the difference: Strong vs. Weak

7 Example: Weaker IV 1,000

8 Example: Weaker IV 500 1,

9 Example: Weaker IV ,

10 Example: Weaker IV 250 1,

11 Example: Stronger IV 250 1,

12 Example: Stronger IV 250 1,

13 Example: Stronger IV 400 1,

14 Example: Weaker IV 250 1,

15 Key Take-Aways People worry about weak instruments. They are easily biased. They provide large (sometimes HUGE) confidence intervals. If you re not careful, you can get inappropriate confidence intervals. Angrist & Kreuger (1991) Does compulsory school attendance affect schooling and earnings? Near/far matching can create studies with stronger instruments. Near/far matching feels a lot like a randomized study with noncompliance.

16 Neonatal Intensive Care Units

17 Application: Regionalization Hospitals vary in their ability to care for premature infants. The American Academy of Pediatrics recognizes levels: 1, 2, 3A, 3B, 3C, 3D and Regional Centers. Regionalization of care refers to a policy that suggests or requires that high-risk mothers deliver at hospitals with greater levels of capabilities.

18 Application: Regionalization

19 Application: Regionalization L H

20 Application: Regionalization L H

21 Application: Regionalization L H

22 Application: Regionalization L H

23 Application: Regionalization L H

24 Application: Regionalization L H

25 The task at hand Regionalization is complex Focus on estimating the difference in death rates

26 The task at hand Regionalization is complex Focus on estimating the difference in death rates

27

28 Outcome Outcome

29 Outcome Outcome

30 The data Every baby delivered in a 10+ year period California Pennsylvania Missouri Mothers information ICD9 codes Delivery Post-delivery complications Some pre-delivery Some SES information Zip code of residence Birth/death certificates Census information PA and MO have zip code level CA will have block group

31 The data Every baby delivered in a 10+ year period California Pennsylvania Missouri Mothers information ICD9 codes Delivery Post-delivery complications Some pre-delivery Some SES information Zip code of residence Birth/death certificates Census information PA and MO have zip code level CA will have block group Pre-delivery Severity?

32 Summary of Problem Want to quantify effect of level of NICU on rate of death Observational data Selection bias Some selection variables are unobserved

33 Instrument: Excess Travel Time L H

34 Instrument: Excess Travel Time L H Excess Travel Time

35 Instrument: Excess Travel Time L H Excess Travel Time

36 Instrument: Excess Travel Time L H Excess Travel Time

37 Instrument: Excess Travel Time L H McClellan, McNeil & Newhouse; "Does more intensive treatment of acute myocardial infarction reduce mortality? JAMA. 272(11): , September 1994

38 Fewer Pairs at Greater Distances

39 Our method a quick sketch Use the idea of block design / pair matching to control observed variation. Use the idea of instrumental variables/encouragement to control unobserved variation.

40 Our method: 1 st step Summarize discrepancies in subjects covariates We used Mahalanobis distance D M x 1, x 2 = (x 1 x 2 ) S 1 (x 1 x 2 )

41 Our method: 1 st step D x = d 11 d 12 d 13 d 1n d 21 d 22 d 31 d n1 d nn d ij = Mahalanobis distance between preemies i and j

42 Our method: 2 nd step Create a penalty for preemies with similar instrument values (e.g., calipers)

43 Our method: 2 nd step D x = d 11 d 12 d 13 d 1n d 21 d 22 d 31 d n1 d nn

44 Our method: 2 nd step D x = d 11 d 12 d 13 d 1n d 21 d 22 d 31 d n1 d nn

45 Our method: 2 nd step D x = d 11 d 12 d 13 d 1n d 21 d 22 d 31 d n1 d nn C z = c 11 c 12 c 13 c 1n c 21 c 22 c 31 c n1 c nn

46 Instrument: Excess Travel Time H H Selection is potentially biased!

47 Instrument: Excess Travel Time H H Selection is potentially biased!

48 Instrument: Excess Travel Time H H Selection largely due to the instrument!

49 Instrument: Excess Travel Time H H Selection largely due to the instrument!

50 Our method: 2 nd step D x = d 11 d 12 d 13 d 1n d 21 d 22 d 31 d n1 d nn C z = c 11 c 12 c 13 c 1n c 21 c 22 c 31 c n1 c nn

51 Our method: 2 nd step D x = d 11 d 12 d 13 d 1n d 21 d 22 d 31 d n1 d nn C z = c 11 c 12 c 13 c 1n c 21 c 22 c 31 c n1 c nn Diff Covariates + Diff Encouragement = Discrepancy Matrix

52 Our method: 2 nd step D x = d 11 d 12 d 13 d 1n d 21 d 22 d 31 d n1 d nn C z = c 11 c 12 c 13 c 1n c 21 c 22 c 31 c n1 c nn Diff Covariates + Diff Encouragement = Discrepancy Matrix (near) (far) (barrier to being paired)

53 Our method: 2 nd step D x = d 11 d 12 d 13 d 1n d 21 d 22 d 31 d n1 d nn C z = c 11 c 12 c 13 c 1n c 21 c 22 c 31 c n1 c nn D x + C z = D

54 Our method: 3 rd step Something has got to give: As we force separation in the instrument, it will be more difficult to find preemies with similar covariates. Allow some subjects to be removed from the study design by matching to sinks.

55 Our method: 3 rd step Let k=number of sinks. Then augment the matrix like so: D 0 0 D = n n discepancy matrix, after first two steps 0 = n k matrix, with all entries 0 = k k matrix, with entries

56 Two matched comparisons, one stronger and one weaker

57 The two matches Two matches 1) No sinks / no forced separation 2) 50% of babies matched to sinks / 25min separation

58 The two matches: Variables Excess Travel Time (i.e., Encouragement/Instrument) Pregnancy and birth variables Mother variables Mother s health insurance Mother s neighborhood Rare congenital anomalies Year Missing indicators

59 The two matches: Variables Excess Travel Time (i.e., Encouragement/Instrument) Pregnancy and birth variables Mother variables Mother s health insurance Mother s neighborhood Rare congenital anomalies Year Missing indicators In total: 45 covariates

60 The two matches 1. Weak Instrument a) No sinks (99,174 pairs) b) No forced separation 2. Strong Instrument a) 50% of babies matched to sinks (49,587 pairs) b) 25min separation

61 Weaker Instrument No sinks Number of pairs: 99,174 Stronger Instrument 50% of babies matched to sinks Number of pairs: 49,587 Variable Variable Type Encouraged Mean Unencouraged Mean Δ/sd Encouraged Mean Unencouraged Mean Δ/sd Excess travel time to hihg-level NICU (minutes) Birthweight (grams) Gestational age (weeks) Gestational diabetes, 1/0 Prenatal care (month) Singel birth, 1/0 Parity Mother's education (scale) Mother's age White, 1/0 Black, 1/0 Asian, 1/0 Other race, 1/0 Race missing, 1/0 Income ($1,000) Home value ($1,000) Has high school degree (fr) Has college degree (fr) Rent (fr) Below poverty (fr) Magnitude of encouragement Pregnancy and birth Mother Mother's neighborhood (zip code/census) , , , , , , , , , , , ,

62 Weaker Instrument No sinks Number of pairs: 99,174 Stronger Instrument 50% of babies matched to sinks Number of pairs: 49,587 Variable Variable Type Encouraged Mean Unencouraged Mean Δ/sd Encouraged Mean Unencouraged Mean Δ/sd Excess travel time to hihg-level NICU (minutes) Birthweight (grams) Gestational age (weeks) Gestational diabetes, 1/0 Prenatal care (month) Singel birth, 1/0 Parity Mother's education (scale) Mother's age White, 1/0 Black, 1/0 Asian, 1/0 Other race, 1/0 Race missing, 1/0 Income ($1,000) Home value ($1,000) Has high school degree (fr) Has college degree (fr) Rent (fr) Below poverty (fr) Magnitude of encouragement Pregnancy and birth Mother Mother's neighborhood (zip code/census) , , , , , , , , , , , ,

63 Application: Two matches Weaker Instrument No sinks Stronger Instrument 50% of babies matched to sinks Number of pairs: 99,174 Number of pairs: 49,587 Variable Variable Type Encouraged Mean Unencouraged Mean Δ/sd Encouraged Mean Unencouraged Mean Δ/sd Excess travel time to hihg-level NICU (minutes) High-level NICU, 1/0 Dead, 1/0 Magnitude of encouragement Delivery at a high-level NICU(Dij) Infant mortality (Rij)

64 Application to the Study of Perinatal Care

65 Pop death rate: ~1.90% Application: Estimating λ Inference about the effect ratio, λ, under the assumption of random assignment of excess travel time within pairs matched for covariates Weaker Instrument Stronger Instrument 99,174 Pairs of Two Babies 49,587 Pairs of Two Babies Point Estimate 0.92% 0.90% 95% CI (0.36%, 1.48%) (0.57%, 1.23%) Length of 95% CI 1.12% 0.66%

66 Pop death rate: ~1.90% Application: Estimating λ Inference about the effect ratio, λ, under the assumption of random assignment of excess travel time within pairs matched for covariates Weaker Instrument Stronger Instrument 99,174 Pairs of Two Babies 49,587 Pairs of Two Babies Point Estimate 0.92% 0.90% 95% CI (0.36%, 1.48%) (0.57%, 1.23%) Length of 95% CI 1.12% 0.66% Bound, Jaeger & Baker. (1995), Problems With Instrumental Variables Estimation When the Correlation Between the Instruments and the Endogenous Explanatory Variable Is Weak, JASA, 90,

67 General Method: Quantifying Departures from Random Assignment

68 Sensitivity Analysis: Framework Up to this point we have assumed Pr Z = z F, Z = 1/ Ω for each z Ω. Now we will allow Pr Z = z F = π ij. Consider matched pair i, then 1 Γ π ij 1 π ij π ij 1 π ij Γ for all i, j, j with x ij = x ij.

69 Sensitivity Analysis: Numerical Examples Gamma of 1.25 Doubling of the odds of death. Doubling of the odds of treatment. Gamma of 1.08 Doubling of the odds of death Increase of 25% of the odds of treatment

70 Application to the Study of Regionalization of Perinatal Care

71 Pop death rate: ~1.90% Application: Sensitivity Analysis Inference about the effect ratio, λ with sensitivity analysis. Weaker Instrument Stronger Instrument 99,174 Pairs of Two Babies 49,587 Pairs of Two Babies Point Estimate 0.92% 0.90% 95% CI (0.36%, 1.48%) (0.57%, 1.23%) Length of 95% CI 1.12% 0.66% Sensitivity (Γ) 1.07 >1.22

72 Pop death rate: ~1.90% Application: Sensitivity Analysis Inference about the effect ratio, λ with sensitivity analysis. Weaker Instrument Stronger Instrument 99,174 Pairs of Two Babies 49,587 Pairs of Two Babies Point Estimate 0.92% 0.90% 95% CI (0.36%, 1.48%) (0.57%, 1.23%) Length of 95% CI 1.12% 0.66% Sensitivity (Γ) 1.07 >1.22 Small & Rosenbaum (2008), War and Wages: The Strength of Instrumental Variables and Their Sensitivity to Unobserved Biases, JASA, 103,

73 What changes when an instrument is strengthened?

74 What changes? 1. Smaller study looks less like population Fewer black mothers Fewer renters That is, less urban

75 Weaker Instrument No sinks Number of pairs: 99,174 Stronger Instrument 50% of babies matched to sinks Number of pairs: 49,587 Variable Variable Type Encouraged Mean Unencouraged Mean Δ/sd Encouraged Mean Unencouraged Mean Δ/sd Excess travel time to hihg-level NICU (minutes) Birthweight (grams) Gestational age (weeks) Gestational diabetes, 1/0 Prenatal care (month) Singel birth, 1/0 Parity Mother's education (scale) Mother's age White, 1/0 Black, 1/0 Asian, 1/0 Other race, 1/0 Race missing, 1/0 Income ($1,000) Home value ($1,000) Has high school degree (fr) Has college degree (fr) Rent (fr) Below poverty (fr) Magnitude of encouragement Pregnancy and birth Mother Mother's neighborhood (zip code/census) , , , , , , , , , , , ,

76 Weaker Instrument No sinks Number of pairs: 99,174 Stronger Instrument 50% of babies matched to sinks Number of pairs: 49,587 Variable Variable Type Encouraged Mean Unencouraged Mean Δ/sd Encouraged Mean Unencouraged Mean Δ/sd Excess travel time to hihg-level NICU (minutes) Birthweight (grams) Gestational age (weeks) Gestational diabetes, 1/0 Prenatal care (month) Singel birth, 1/0 Parity Mother's education (scale) Mother's age White, 1/0 Black, 1/0 Asian, 1/0 Other race, 1/0 Race missing, 1/0 Income ($1,000) Home value ($1,000) Has high school degree (fr) Has college degree (fr) Rent (fr) Below poverty (fr) Magnitude of encouragement Pregnancy and birth Mother Mother's neighborhood (zip code/census) , , , , , , , , , , , ,

77 What changes? 1. Smaller study looks less like population Fewer black mothers Fewer renters That is, less urban 2. Compliers change Larger study: 14 minutes difference Smaller study: 34 minutes difference

78 Why is it OK to throw out data?

79 Stronger instrument by design In some sense we already know this tradeoff: Larger studies with poor design and low compliance Vs. Smaller studies with good design and high levels of compliance

80 Stronger Instruments by Design

81 Our method Advantages Other researchers worry about weak instruments. Now we do something about it! More compliance leads to better estimates Less bias Tighter confidence intervals Sensitivity analysis Easier to explain Mimics an experimental design Avoids MLE (i.e., parametric assumptions)

82 The End More questions?

83 The End Baiocchi, Small, Lorch & Rosenbaum: Building a Stronger Instrument in an Observational Study of Perinatal Care for Premature Infants JASA. Dec 2010, Vol. 105, No. 492:

84 Pop death rate: ~1.90% Application: Estimating λ Inference about the effect ratio, λ, under the assumption of random assignment of excess travel time within pairs matched for covariates

85 Notation: Treatment Effects, Treatment Assignments

86 Notation Indices i denotes which pair There are I matched pairs, thus i = 1,, I. j denotes which subject within the matched pair Thus j = 1, 2. Covariates Observed: x ij Unobserved: u ij

87 Notation Matching on observed covariates x i1 =x i2 for all pairs i But it may be that u i1 u i2

88 Notation Instrument/Encouragement Z ij = 1, if subject j in the i th pair was encouraged Z ij =0, if subject j in the i th pair was unencouraged Note that, within a matched pair, Z i1 +Z i2 =1 Potential outcomes framework (Neyman 1923; Rubin 1974)

89 Instrument Design Response (R) Instrument (Z) Dose (D) Response (R) Response (R) Dose (D) Response (R)

90 Notation Dose (d Tij,d Cij ) Can t observe: d Tij -d Cij Can observe: D ij =Z ij d Tij +(1-Z ij ) d Cij Response (r Tij,r Cij ) Can t observe: r Tij -r Cij Can observe: R ij =Z ij r Tij +(1-Z ij ) r Cij

91 Notation Let F = {(d Tij, d Cij, r Tij, r Cij, x ij, u ij ), i=1,,i,j=1,2} Let Z = Z 11, Z 12,, Z I2

92 Notation Let Z be the event that Z Ω Let Ω be the set containing the Ω = 2 I z of Z Then, in a randomized experiment: Pr(Z=z F, Z)=1/ Ω for each z Ω

93 Effect Ratios

94 The Effect Ratio λ = I i=1 I i=1 2 j =1 2 j =1 (r Tij r Cij ) (d Tij d Cij ) λ is parameter of the 2I subjects, fixed under F. λ is not directly observable. Under Fisher s sharp null hypothesis, H 0 : r Tij = r Cij, for all i, j, it follows that λ=0.

95 The Effect Ratio I 2 λ = I i=1 I i=1 2 j =1 2 j =1 (r Tij r Cij ) (d Tij d Cij ) i=1 j=1 r Tij r Cij is the effect of encouragement on response. I 2 i=1 j=1 d Tij d Cij is the effect of the encouragement on the dose. λ is the ratio of these effects. If λ = 1/100 then for every 100 discouraged by distance from delivering at a high level NICU there is one additional infant death.

96 Inference about an Effect Ratio in a Randomized Experiment

97 Inference Composite Null Consider H 0 λ : λ = λ0 This is a composite null, because no assumptions on distribution of F. Need to consider the supremum over null hypotheses of the probability of rejection.

98 Models have incredible power Models force you to clarify your thinking. By writing down a model, you take a stand on how the world works. If the model is correct, there are very powerful mathematical machines you can deploy to get precise answers.

99 Models have incredible power If the model is wrong, then the power (in the statistical sense) of your inference is not credible.

100 Inference Test Statistic Consider the following test statistic T λ 0 = 1 I I 2 Z ij (R ij λ 0 D ij ) 2 (1 Z ij )(R ij λ 0 D ij ) = 1 I i=1 I i=1 j =1 V i (λ 0 ) j =1

101 Inference Unobserved to Observed Note that in T λ 0 If Z ij = 1, then R ij λ 0 D ij = r Tij λ 0 d Tij, and If Z ij = 0, then R ij λ 0 D ij = r Cij λ 0 d Cij Thus we can write: 2 2 V i λ 0 = j=1 Z ij r Tij λ 0 d Tij j=1 1 Z ij r Cij λ 0 d Cij For the variance of the test statistic S 2 λ 0 = 1 I 2 V I I 1 i=1 i λ 0 T λ 0

102 Inference t-stat! Then, for each k>0, lim sup I Pr T I λ S I λ k F I, Z I Φ( k) lim sup I Pr T I λ S I λ k F I, Z I Φ k.

103 Visualizing our method

104 Encouragement Our method two criteria for matching Distance in Covariates

105 Encouragement Our method near in covariates Distance in Covariates

106 Encouragement Our method far in covariates Distance in Covariates

107 Encouragement Our method far in covariates Distance in Covariates

108 Encouragement Our method good pair Distance in Covariates

109 Encouragement Our method ok pair Distance in Covariates

110 Encouragement Our method strength of instrument Distance in Covariates

111 Encouragement Our method strength of instrument Weak IV Distance in Covariates

112 Encouragement Our method strength of instrument Medium IV Distance in Covariates

113 Encouragement Our method strength of instrument Strong IV Distance in Covariates

114 Encouragement Our method good pair Strong IV Distance in Covariates

115 Encouragement Our method near in instrument Strong IV Distance in Covariates

116 Encouragement Our method near in instrument Strong IV Distance in Covariates

117 Encouragement Our method near in instrument Strong IV Distance in Covariates

118 Encouragement Our method near in instrument Possible Bias Strong IV Distance in Covariates

119 Encouragement Our method near in instrument Strong IV Distance in Covariates

120 Encouragement Our method far in instrument Strong IV Distance in Covariates

121 Encouragement Our method far in instrument Strong IV Distance in Covariates

122 Encouragement Our method far in instrument Stronger Encouragement Strong IV Distance in Covariates

123 Encouragement Find the experiment Strong IV Distance in Covariates

124 Encouragement Find the experiment Strong IV Distance in Covariates

125 The framework Treatment (T) Encouragement (Z) Treatment (T)

126 The framework Treatment (T) Encouragement (Z) Treatment (T)

127 The framework Response (R) D A Treatment (T) Encouragement (Z) Treatment (T) Response (R) D A

128 The framework Response (R) D A Treatment (T) Encouragement (Z) Treatment (T) Response (R) D A

129 The framework Response (R) D A Encouragement (Z) Treatment (T) Treatment (T) Response (R) Response (R) Response (R) D A D A D A

130 Clarifying who we re talking about Compliance Class Encouragement (Z) Treatment (T) 1 1 Complier Never Taker Always Taker Defier 0 1

131 Clarifying who we re talking about Compliance Class Encouragement (Z) Treatment (T) 1 1 Complier Never Taker Always Taker Defier 0 1

132 Clarifying who we re talking about Compliance Class Encouragement (Z) Treatment (T) 1 1 Complier Never Taker Always Taker Defier 0 1

133 Building intuition for implementation

134 Variable Type High NICU Low NICU sd Δ/sd Mortality Outcome 2.26% 1.25% 13.33% 0.08 Difference in Travel Time Instrument % attending high level NICU Treatment 100.0% 0.0% 49.7% 2.01 Birth weight 2, , Preemie covariates Gestational age GI 0.9% 0.6% 8.7% 0.04 GU 0.9% 0.8% 9.0% 0.01 CNS 0.9% 0.4% 8.3% 0.05 Pulmonary 0.8% 0.7% 8.8% 0.01 % of preemies with type of Cardio 1.4% 0.7% 10.5% 0.06 congenital disorders Skeletal 0.7% 0.9% 9.0% Skin 0.0% 0.0% 0.0% 0.00 Chromosomes 0.4% 0.3% 6.3% 0.02 Other_Anomaly 0.8% 0.1% 7.0% 0.09 Gestational_DiabetesM 4.9% 4.3% 21.0% 0.03 Mother's education Insurance - Fee for service 24.0% 24.5% 42.8% Insurance - HMO 32.3% 27.8% 46.0% 0.10 Insurance - Government 23.5% 24.2% 42.6% Insurance - Other Mother covariates 16.8% 21.4% 39.1% Uninsured 2.2% 1.6% 13.7% 0.04 Prenatal care Single birth (y/n) 79.0% 86.1% 38.3% Parity Mother's age Median income 41, , , Median home value 97, , , % completed high school 79.9% 80.0% 9.7% Census level covariates % completed college 22.2% 19.4% 13.1% 0.21 % renting 31.4% 27.9% 12.8% 0.28 % below poverty line 13.4% 11.8% 9.9% 0.16

135 Pre-matching Variable Type High NICU Low NICU sd Δ/sd Mortality Outcome 2.26% 1.25% 13.33% 0.08 Difference in Travel Time Instrument % attending high level NICU Treatment 100.0% 0.0% 49.7% 2.01 Birth weight 2, , Preemie covariates Gestational age

136 Covariates across the instrument 1st Quartile 2nd Quartile 3rd Quartile 4th Quartile max(δ/sd) Mortality 1.93% 2.08% 1.47% 1.74% 0.05 Difference in Travel Time (3.19) % attending high level NICU 81.1% 69.8% 49.9% 21.6% 1.20 Birth weight 2, , , , Gestational age

137 Post-matching Matched Pairs 49,587 Variable Type Encouraged Mean Unencouraged Mean Mortality Outcome 1.54% 1.94% 12.86% Difference in Travel Time Instrument % attending high level NICU Treatment 68.6% 25.4% 49.7% 0.87 Birth weight 2, , Preemie covariates Gestational age sd Δ/sd

138 Result Point Estimate 95% CI Length of 95% CI Sensitivity (Γ) Weaker Instrument Stronger Instrument 99,174 Pairs 49,587 Pairs of Two Babies of Two Babies 0.92% 0.90% (0.36%, 1.48%) (0.57%, 1.23%) 1.12% 0.66% 1.07 >1.22

139 Final thoughts Near/Far deals with binary outcomes 2SLS can lead to logical absurdities when outcomes are binary Increase the strength of the instrument Stronger instruments lead to more robust results Sensitivity analysis for this method is available This technique has the potential to be a Philosopher s stone

140 Why not use 2SLS? Linear probability models (LPMs) have trouble when your parameter values are up against the edge of parameter space

141 Quality of Care Trouble with time Treatment effect High level NICU Low level NICU Time

142 Quality of Care Trouble with time Treatment effect Treatment effect High level NICU Low level NICU Time

143 Modeling hospital choice

144 Instrument: Probabilities Empirical distributions M

145 Instrument: Probabilities Empirical distributions?

146 Instrument: Probabilities Near Neighbors?

147 Instrument: Probabilities Conditional logistic model/bayesian hierarchical modeling H H M

148 Instrument: Probabilities Conditional logistic model/bayesian hierarchical modeling H H M

149